I have seen this before. For some reason, the shut-off valve
develops a small internal leak. Sometimes if you twist the valve completely shut and then re-open it, it will either stop the leak or make it worse. You have a 50/50 chance.
For saftey sake, I have always replaced these when they start leaking.