My '91 SE has done exactly that for 15 years. Tried everything to fix it over the years, except replacing the whole fuel distributor. Mine is definitely due to insufficient fuel during hot start after heat soaking awhile. When engine is cool, or when it only sits less than 15 minutes when hot, it starts and idles perfectly. After heat soaking it starts on the second try, and idles rough for a few seconds, then smoothes right out.
If anyone has this fix, I'd definitely appreciate it; I've posted a hundred times over the years each time I tried something new, but still no fix.
Bottom line, you may have to live with it.

DG