I left the bracket in place and used the guide rod itself to help push out the rubber mount by turning the rod all the way out to the end of the threads, then used a drift to pop it out.
Just make sure you remove the cap first.