Grinding sound coming from starter motor?

[DieselHead]

I have a 1983 300D-T with 122,500 miles on it and now, when I start the car, I hear this awful groaning/grinding sound right right after the engines starts up. So I'll turn the key to start, the engine will crank once (it usually starts with only one or two cranks) and as soon as I release the key, I'll hear this pretty loud groaning sound. It sounds like the starter motor isn't disengaging quickly enough.
So, assuming it is the starter motor (although you guys can correct me if you don't think it is), how hard is that sucker to replace? The car starts perfectly. In fact, the only thing really wrong is the sound, otherwise, it's fine. I want to fix it to get rid of the sound though because it's awfully embarassing. I'm a pretty handy DIYer but I'd like to know what I'm getting myself into. Thanks a lot!

Alex