You can get the old rivet out without a drill press if you use a dremel tool and grind the flange completely away. Then the rivet easily gets pushed out.

As others have mentioned, you MUST take the new plastic piece apart before installing it and peening over the soft steel. Otherwise, you WILL be ordering another plastic piece. I found that out the hard way.