I have an ASM '61 Navy that had an arbor pin that was too short. It failed to but against the bottom of the arbor pin hole in the barrel by about 1/8 of an inch. If you have this condition it causes the weapon to shoot high, because when you install the wedge and it pulls the barrel back, it makes it point up instead of straight ahead. This is a pretty common problem in Colt repros and the reason many Colt repros shoot high (not because the sights were regulated for 100 yards).
It is a lot of work to fix, but a properly fitted arbor pin will make the gun shoot POA, tighten the groups, and lessen strain (wear) on the wedge and wedge slot. Guys who shoot Colt pattern repros in NMLRA target pistol competition have known this for years and that's why most of them shoot Remingtons!
With the wedge removed, put the barrel on and hold it up to the light, so you can see the barrel/cylinder gap. Push back on the barrel, then lift up slightly on the end. If the gap closes at the top, your arbor pin is too short.
The fix is to weld extra material to the end of the arbor pin, file or lathe it round, then carefully try the barrel on, filing a little off the lenght of the pin, till it fits squarely against the bottom of the arbor pin hole in the barrel.
It is best to remove the arbor pin from the frame but it can be done with the pin installed.
A Colt pattern pistol with a properly fitted arbor pin (the way real Colts were made) is a wonderful thing and a tack driving C&B pistol (or conversion).