Your oil pressure gauge is part of an electric circuit. With the key off there is no electrical power in that circuit. A good gauge should always read zero I would think. In your description the movement may be sticking for example. Does the gauge normalise if you tap the dash?

At least it really does not sound like you have low oil pressure issues in the engine.