Sounds like the Starter or the solenoid is going bad if your battery has juice and you're just getting a click.
You could try tapping the starter with a hammer next time it happens, and then see if it cranks. There is a small 3-terminal block on the passenger side fender in front of the battery. If you short the small gauge wire with one of the large ones, it will fire the starter without having to use the key switch- handy for troubleshooting.

__________________
'83 300D Turrrbo 295K miles