Any mechanical scraping will be really noticeable. Inside the cam shaft cover the only one that I've read about is when the timing chain has ground its way into the cover. This however seems most likely to happen when some one has decided not to use a cover seal and has used RTV sealant instead - the clearance is somewhat different - stoopid thing to do.

Whilst I'm droning on about cover seals - I would also replace your cover seal whilst you are at it - this will stop any weird leaks down the side of the engine that might be concerning you!