If your vacuum pump is bad your engine will not shut off using the key, your power brakes won't work, door locks won't work properly, the trans will shift pretty hard, and usually there will be quite a bit of oil in your air cleaner housing (and subsequently on the ground beneath the passenger side of your car.)

I would suggest looking in your air cleaner housing. if there's a lot of oil it's there because the diaphram in the vacuum pump is torn, causing oil from the crankcase to be forced through the positive pressure dicharge line that discharges through the lid of the air filter housing.

rebuild kits are available for about $20-35 dollars. i recently rebuilt mine, and within 2 weeks it failed again. It is very possible that it failed because I didn't install it properly. I've ordered a new kit, and will see if I can find torque specs for all of the bolts. If it works I'll let you know.