I agree...engine is nearly bulletproof...it's the ancillary components that give you headaches!
The radiator will crack at the inlet neck at around 100K.
You already know about the head gasket problem. It's a problem for BOTH the M103 & M104, just the failure symptoms are different. Oil leaking at the front is the timing cover seal, at the rear, it's the head gasket.
Water pump, alternator, and other parts depend on usage and conditions, but they will go belly up eventually.
Loud clanking sound at idle? Connecting rod? No. More like a worn bushing on the belt tensioner shock.
Loud grinding sound at startup? Bearings going bad on the air pump. The bearings are (of course) not a readily-available size, so total replacement is necessary.
The OVP relay is the source of all sorts of weird idle and starting problems. If yours is original, be prepared.