Probably not the vacuum shut-down valve. To be sure, just pull the vac tube to it before trying to start next time. The tube runs beside the VCV on top of the injection pump.

I agree it is most likely in the starter motor, perhaps the solenoid that is integral with it. But, could also be a voltage supply issue to the starter. Remove and clean the battery terminals. Some older U.S. V-8 engines had issues where the starter was over-heated by the exhaust, so insure all your heat shields are in place. In 1984, I recall that is just one bolted under the exhaust rear and a little one to protect the battery. 1985 CA engines have many more.

The starter isn't trivial to remove. I would insure it is getting >8 V right at the starter before condemning it. That is measured from the case (blk lead) to the big stud on the starter (red lead). You could have a voltage drop from block to frame, so don't put the blk lead on BAT- and assume that is a valid measurement.

__________________
1984 & 1985 CA 300D's
1964 & 65 Mopar's - Valiant, Dart, Newport
1996 & 2002 Chrysler minivans