Tranny Pan Gasket

I just changed my transmission fluid for the first time and everything went okay, but I got a little worried when I saw residue from the old gasket on the tranny body. Logically, I knew I had to clean it off for a good fit of the new gasket but I was concerned about the residue getting into the works. The residue was real flaky and was a little difficult to control. Apparently, none got inside the tranny because it's working find. Have any of you had a problems with clearing the old gasket residue off the surface and how have you handled it?