.22LR is a good starter, and still a lot of fun once you get decent at shooting. You can get a decent .22 rifle for well under $200, and it will last for decades if you take care of it. Plus the ammunition is dirt-cheap, unless you get target ammo, which I wouldn't recommend starting out.

Shooting .22's at the range costs me about $10 for a session. If I get out something bigger, my ammunition costs shoot up to at least $50.