On a W126 and I assume on all others the fuel pump relay lets the pumps start in position II, just one notch before the start turn. Should you then not turn the key and start the car, the pump should stop after about a second, as there is no need to pump anymore. I guess it is part of the safety concept that lets the pump stop if you have an accident and don't want the pump to spill gasoline in a dangerous situation. There is a sensor that feeds the turning of the flywheel or crankshaft to the ignition system. No turning of engine, no fuel pumping.
You can override this relay in case of malfunction by connecting I believe pin 30 and pin 87 (all on a W126, yours might differ). Pump then gets electricity from battery bypassing the relay and will pump.
Again on a 126 ('87) the check valve is on the exit side of the pump to prevent backflow of fuel and the pump I bought came with new checkvalve installed. I didn't know either and have now 2 spare check valves.
Hope this answers your questions