Mr Phil
I've got rifles in 45 Colt, .44 Spl. and 44-40. Since you have a self-imposed a limit of 50 yds, either calibre will do. It's a question of which do you prefer to carry?
My 44-40 Henry is a bit on the heavy side, off set by the fact that it has a sling. It's rock steady in my hands and 'hangs' like a muzzle loader. My .45 Colt 'Lightning' is a joy to carry but will hangup if not operated in a very positive manner.
The .45 Colt throws a heavier ball, but a well placed 200 gr RNFP is going to do the job just as well. No deer will notice the difference.