You can't really set the modulator without a proper pressure gauge hooked up to the service port on the transmission. The modulator adjustment isn't really that sensitive. I've found that a full turn will only result in a small pressure change. Unfortunately, I don't remember the specifics. It's been too long.
It's really easy to make a pressure gauge. I grabbed an 8mm banjo bolt from the salvage yard, about 8 feet of plastic tubing from the Home Depot, and a 100psi gauge off Ebay. Total cost under 15 bucks.
There used to be an excellent write-up out on the web on how to adjust the vacuum source to the modulator. A google or a search here should find it. If i remember right, it was written by Steve Brotherton.
Found it:
http://www.continentalimports.com/ser_ic20242.html