You'll have problems at higher revs if something's wrong with the catalytic converter/s. A bad cat won't cause problems only at idle.
My uneducated guess is that something's wrong with your warm-up regulator or a temperature sensor. I wouldn't replace those items on a hunch. Keep doing the research to figure out what's really wrong. Search the archives. I'm sure somehas has asked about the same symptoms.
FWIW, my engine is really rough for a couple of seconds after I start the engine. Then it's almost butter smooth until it gets to operating temperature. Then the idle is rough enough to be annoying but not rough enough for me to take it to the shop.
81 300SD (on ebay)