It's not fun, but I've had good luck removing silicone from painted surfaces (like when I removed the old body kit from one of the w201's and it had been glued on with silicone!) just using the friction from my fingers. See if rubbing your fingers on the edge with some pressure doesn't take most of it off.
You can also try a blunt plastic scraper...I would try to knock off any sharp edges on the scraper with some 200 grit sandpaper first, though. Sometimes they have casting ridges that are sharp/strong enough to scratch paint.
Good luck...I never realized how much prior owners/mechanics had ruined my cars until I started doing all of my own repair work...
