I have had that happen too. I leave it in gear, and the rpm fall off by themselves in a short while. I figure it's the Electronic Diesel System that pulled the rack out a bit when stone cold, and forgot to put it back. It's intermittent, like yours, too.

I sometimes lose the idle control completely. Tach drops to zero, idle drops to mechanical low idle around 500 rpm. Next time engine speed exceeds 1,000 rpm the tach comes back and the idle is OK. This comes and goes too, in warmer weather. Haven't chased that gremlin down yet.

Best Regards,
Jim