High speeds, weird noise, big problems?

[1983 300TD]

Hi guys.

I drive a 250 mile stretch pretty often on I-75. On my past two trips, after an extended duration of 75-80 mph speeds, I start to here a knock of some sort from the rear of the car (I think?). It isn't high pitched, no grinding or rubbing noises, just like a knock or clunking. I don't think it's engine or tranny related, I'm guessing it's possibly the drivetrain/differential. Weird thing is, once I exit the interstate and slow the car down, the noise goes away. But once I get back on, after 40-50 miles, it comes back. Any ideas? Possibly something loose or that should be replaced?
Thanks.
Damon Fite ---1983 300TD Wagon---300,000 miles+!!!