I AM a chemist and my experience isn't paint prep work but sealing of lab equipment used for sulfide corrosion testing. We use silicone lube on reuseable components that are broken down after testing, cleaned and reused. Since we wanted the components as clean as possible, we researched solvent methods of removing silicone from the parts for reuse. There aren't any. We had to wipe off as much as we could. Similar to what you described, we followed up with Scotchbrite and abrasive detergent. It's the mechanical action of Scotchbrite (in your case, a nylon brush) that removed the silicone. The detergent simply helped carry the silicone away.
95 E320 Cabriolet, 131K