You have a fuel leak or defective hand pump. The old style hand pumps will go bad and leak air into the fuel. So will a bad fuel line -- the black fabric covering can hide deteriorated rubber.
On the W123 the fuel will siphon back into the tank rather than leak out -- if you can, park the car on a steep hill, engine down. This will cause the fuel to leak OUT rather than back into the tank, allowing you to locate the problem.
On a warmt start, the glowplugs don't operate until you run the starter, and if you have fuel deliver problems you can have staring problems.
I just replaced the glowplugs in the Volvo for this reason -- bad cold starts, but simply had to crank forever to get it going after it had been sitting for a while. New glowplugs fixed that, but I think you may have a fuel problem.
Wouldn't hurt the check the glowplugs, though!
1972 220D ?? miles
1988 300E 200,012
1987 300D Turbo killed 9/25/07, 275,000 miles
1985 Volvo 740 GLE Turobodiesel 218,000
1972 280 SE 4.5 165, 000 - It runs!