Howdy!!
Without this becoming a Dillon lovefest, I think Dillon is the way to go. If someone is leery about starting out with a "full-auto" progressive, then Dillon also makes a splendid Turret press, the AT 500. (Edited late 2006 - AT500 no longer available.) Lyman makes a great turret press as well as Lee. I'm not sure, but I think (I'd bet on it) RCBS Makes one, too. The big advantage of the Dillon is that the AT 500 will convert to a full progressive, the RL550B. However, Dillon while not TOO bad on the pocketbook initially, can get expensive as you add calibers.
No one has mentioned single stage presses, yet. I started out helping my best buddy on his RCBS Rockchucker Jr. That was about 30 years ago. He still has it and that is his only press - no matter HOW hard I've tried to get him to make the Dillon plunge! After loading on his for half-a-dozen years I went into the Army, and when I re-enlisted the first time, I bought myself a couple re-up presents: a 6" Electroless Nickle Python and a Bonanza loading press. That was in 1982. I still have and USE both. The Bonanza is pretty much the Cadillac of single-stage presses.
A good friend bought a Lee set that, except for the powder measure, has worked well for him. I personally don't much care for Lee presses, but I have to admit they're a bargain! I DO like Lee dies and much of their associated equipment, and have a number of tools and dies with Lee marked on them.
I have little experience with Lyman equipment, except for their turret press, but it's built like a tank. Also, I had a Lyman 55 powder measure for years and it worked like a champ. It did very "repeatable" settings. (If you don't know what repeatable is, it is the ability of any piece of adjustable equipment to go back to the same setting, time and time again: this is VERY good.)
All in all, it takes about a thousand rounds to make up your equipment cost vs. savings. (This is figured using my Dillon RL550B.) It's probably cheaper for some of the less expensive equipment. If you buy in bulk, say 1000-2000 primers at once; 500-1000 bullets at a time; a 4 lb. or 8 lb container of your chosen powder, altho there are no 4 or 8 lb. containers of Black Powder that I know of - you can still save by buying it in quantity; and buy 1000 Win. or other cases at once will save money. Yes, the initial outlay can be daunting, but you'll save BIG in the end.
Now all this has to be taken with a grain of salt: if you LIKE to shoot, and tend to shoot a lot, your cost SAVINGS can be eaten up by the fact that you now can buy, load, and shoot MORE, so perhaps you're not really saving any money, but you ARE shooting and practicing more, and hopefully having MORE FUN !!!