Welcome Burnzy! You're off to a good start: using the search function AND owning a Mityvac already? Your future here is bright....
If your shutoff valve is in fact bad, it will start sending engine oil back through the vac line and into your ignition switch. Then it will drip on your leg. Yes, I do know this to be a fact....
With your car running (the car that should be in your sig.
), hook up your Mityvac to the shutoff on the ip and pump it a few times. If your car shuts right off, that usually means the shutoff valve is OK. Since you had 23hg of vac going to it and it didn't work well, then I imagine your car won't shutoff that quickly with the Mityvac.
A good rule of thumb when it won't shut off is to unplug and plug up all vac lines except brown shutoff lines and see how it goes.
Again, welcome to our humble website, and you won't be a noob for long at this rate.....