Randy:
In looking at the Mercedes shop manual for the 617 engine, I think your analysis may be correct. In one of the pictures it shows 'shading' between the damper and the piece through which the bolts attach the damper to the balancer. That 'shading' could be a glue or 'rubbery' substance of some kind. If that is where the problem lies, and the center bolt and allen bolts are tight, I don't think there is any damage to the crank yet.
I would say you have two choices at this point: Remove the pulley and vibration damper (I'm pretty sure this only requires the removal of the six allen bolts and not the more difficult central bolt) and replacing the damper with a new one or a used one from a junkyard. The other option, which may be possible if you can align the vibration damper up with its original position is to get a lot of 5 minute epoxy and glue it back on if you can clean it up enough. I don't think I'd recommend this second alternative unless the first was simply impossible.
I would still be reluctant to drive it in its current condition because the vibration caused by the loose damper may loosen the balancer and screw up the balancer in its attachment to the crank.

__________________
1977 300d 70k--sold 08
1985 300TD 185k+
1984 307d 126k--sold 8/03
1985 409d 65k--sold 06
1984 300SD 315k--daughter's car
1979 300SD 122k--sold 2/11
1999 Fuso FG Expedition Camper
1993 GMC Sierra 6.5 TD 4x4
1982 Bluebird Wanderlodge CAT 3208--Sold 2/13