If the key does not end up in the correct 'run' position the alternator light will stay on. You might try working the key back and forth a little and make sure the contacts are being made.

I second the above advice. Autozone, Advance, almost anyone that sells batteries will test yours for you.

A quick battery test is to try to start the car with the headlights on. If the headlights dim way down, the battery is weak.

Never hurts to clean the battery terminals. It is amazing how many odd conditions that can be created by dirty battery terminals. Check other connections as well as you can.

If the brushes in your alternator are reaching their wear limit they may not always make contact. You might try tapping the back of your alternator. Be careful not to hit the terminals or anything that will break.