I hate to rain on your parade, but the first Burnside contract deliveries were made in March of 1865. A 21K numbered Burnside would have been made after the war was over. The 5th NY stayed in the army until mid-July of ’65. At the time, there was a lot of consternation about France’s doin’s in Mexico, so the better units were held in preparation for invasion from south of the border. The 5th could have been rearmed just before they got mustered out, in anticipation of service in Texas. This would be about the only circumstance that would explain a Burnside contract Spencer in that unit, especially one with such a high number.
Also, keep in mind that serial numbers in your range overlap between three models. Spencer restarted with #1 on the M-1865s, and Burnside also started with #1. Carbines between #1 and ~11K could be Burnside or Spencer made 1865s. Numbers between ~11K and 25K could be M-1860 Spencer made carbines or M 1865 carbines from either contractor. So, with a hit at SRS from 1865, there is only a 33% chance that it’s your gun. If it’s a pre-1865 hit, the gun recorded has to be a Model 1860 56-56 carbine or rifle.
One caveat. It appears that the first 2500 or so M-1865s made by Spencer were rifles, probably for the State of Mass. I haven’t found the records, but virtually every M-1865 rifle I have seen is numbered below 3000, most with inspection stamps.