Play in the steering is aggrevated tremendously if the steering is stiff. By this I mean that if the alignment is right the car should tend to follow the road, the steering is just for direction. IF the steering gear is keeping the wheels from following the road (stiff - this is hard to feel) then one is constantly chasing the road.
The numerous cars I mentioned fixing all had these syptoms which sound just like yours. The play that is adjustable at the top of the pitman shaft is the play between the reciprocating ball carrier and the pitman internal lever. This is a ball and socket. When you adjust the allen counterclockwise one is pulling the socket up to the ball. Once they are together further adjustment causes the box to have enough stiffness that it can keep the wheels slightly turned or anywhere in between. In this state the car seems to wander. The remaining play at this point is in the worm-ball-carrier and can't be removed (assuming no other play in tierods etc.).
I would try readjusting the box till you are sure there is no interference. I do this by reaching down with my right hand and grab the steering coupling right at the input to the gearbox. With your left hand screw the adjustment clockwise till your right hand can feel the the changes your left hand makes (slightly twist the coupling with the right hand back and forth as the adjusment is made).
When I make an adjustment on the car I always feel the play in this method and stop before the play is all gone to avoid the tight situation. This is due to the fact that the stiff box can not be felt except through the idnetification of symptoms, as in your case.
Bosch Master, ASE Master, L1
33 years MB technician