What Cholla, Coffin, and pony said. Back in the day when I started reloading ('76), I roll crimped some .357 fodder to much and they were a problem, wouldn't fully seat in the cylinder. Was going by the theory that a round, esp in a magnum caliber, needed a heavy roll crimp. They didn't look that bad. Fortunately I had only crimped around a dozen and checked them in my Colt revolver to see the error of my ways. I assume you are roll crimping your 32 Special reloads. Excessive roll crimp in a round in revolver or rifle wouldn't matter.
I also had a problem with some cast lead .451 bullets I purchased for reloading 45 acp. after careful reloading and shooting several magazine's full, the slide on my 1911 pistol wouldn't fully close. Found the problem to be lube from the bullets building up in the forward area chamber of the 1911 and since a 1911 45 acp headspaces on the rim of the brass, the excess lube was preventing the slide from fully going into battery. Cleaned the rest of the 80 some reloads and no further problem, didn't reload any more of the .451's for my 1911. The lead store bought bullets did have excessive lube on them IMO as the entire bullet was covered, not just lube in the lube grooves. I've never had a problem with other purchased lead bullets for 45 acp, the ones I cast and lube myself, or FMJ 230 grain RN in my 1911's.
One other possibility if your .323 bullets diameter is excessive and I as the others posters think they aren't (.002 "shouldn't" make a problem unless your Winchester chamber is really tight) is to maybe go with a Lee Factory Crimp die. I used to use them on a lot of calibers I reloaded for as they really gave a nice crimp, minor case length differences didn't matter as much as in roll crimping, and it was faster and easier adjusting a Lee FCD than doing so with the regular bullet seater/crimp die. I quit using them, esp for cast bullet loads (which are the majority of my revolver and lever rifle loads) as several of the members/posters on this forum brought out a interesting and factual thing about the Lee's.
When you proceed to crimp a round with the Lee FCD (factory crimp die) after you seat your bullet, it not only crimps the bullet, but there is a carbide sizer ring that resizes the case as you work your press handle. This sizer ring also will resize what ever lead bullet you've seated into the brass case and if you've resized your cast bullet or have bought them a specific diameter that works best in your rifle/revolver, if that diameter is more than what the Lee FCD is, the bullet will be swaged to the diameter that the carbide ring is set at. Made sense to me, so I quit using them and went back to my two stage process of seating the bullet in one step and then roll crimping on another using the seater/crimper die in whatever die set for the caliber I'm reloading for.
It's been pointed out by some shooters/reloaders that the carbide resizer ring in a Lee FCD can be knocked out and then the die used for crimping. I haven't got around to doing this yet. Flatlander, buying a Lee FCD may work for ya as they aren't very pricey if going back to your reloading procedure (crimping) doesn't work for ya. Buy a small 100 round box of .321 bullets and see if they solve your problem. You could possibly resize your .323 bullets, but that would be an expense unless you get into casting that would maybe be for a one time run of bullets that you may never use the sizer again (as Coffinmaker said). A RCBS or Lyman sizer die would nessisitate buying a sizer press, more expense than your remaining 400 bullets are worth unless you get into casting. Lee does make some sort of die that I believe fit's in a standard press for sizing bullets for bullet sizing, but then again will you ever need it again. If you're friends with a caster/sizer and they have a .321 sizer die they may do it for ya. If you attend Gunshows, you may be able to sell the remainder of your bullets there, recouping your cost or near so. Might be just as cost effective rather than buying a die to resize. I'd first go back as has been posted and reload some, readjusting the roll crimp and see how that goes. WOW, I was only gonna submitt a couple of sentences and got windy. Good luck in your endeavour. CC