I don't know what is this "paper residue" that you are gettin' when you are vacuuming. It seems the bottom of my mats are made out of a durable foam-like material. Nevertheless, you should probably go to the dealership or wherever you bought the mats from and let them check it out. They shouldn't be fallin' apart so quickly.
------------------
1995 Mercedes-Benz E420 (W124 - Black Pearl/Black)