I was able to replace the idler arm bushing on my '88 260 without removing the exhaust and was still able to get the new bolt in the right way. Once the bottom nut was off I pried out the lower plastic bushing and loosened the upper bushing allowing the bolt to skew past the exhaust. "Installation is the reverse of removal" as the Haynes Manual would say.

I'm not sure if the exhaust on the 300e is different from the 260 in which case my method may not work.