First the vacuum has to be a-OK before adjusting the modulator.
The vacuum control has to main components: the switchover valve on the valve cover and the vacuum control valve on the IP. From what I read you are not measuring the vacuum correctly.
Measure the vacuum at the red spot:
The switchover valve switches between full pump vacuum (blue dot) at idle and controlled vacuum from the VCV (green dot) when the accelerator pedal is pressed.