How old is the battery? Is it "winter" where you live? The next time you are allowed to let it sit for the 3 or 4 days, before you start it, check the voltage (or if you have a hydrometer, the specific gravity) of the battery. It should be close to 12.3-12.6volts (1.20-1.26 spec gravity), anything less may be hindering full current flow to your fuel pump. I believe the Powertrain control module richens the mixture when it's cold (at start until it warms up) by extending the pulse duration of your fuel injector driver. Which may hide the inadequacy of the fuel volume/pressure. Once it reaches operating temperature and in closed loop, then evidence of POSSIBLE low current is exposed. At least, until the Alternator has re-charged the battery to a decent output. Before digging into the fuel system, I'd conduct this little investigation. Nothing that will set you back if it proves the battery's OK.