Spray the area where the driver inserts into the cam with some penetrant and move it in and out (the 3mm you have already) to allow the varnish to soften up.
You're in a better position than I have been a few times, sometimes they are seized in there pretty good.
BE CAREFUL, I have broken a few in case like this.
Small seepage of oil past the radial seal is where the varnish comes from.
The rear plastic plate is free to move, it's what helps seal the distributor cap. If it's badly chipped now, you should probably get another one.