Does the compressor cut out? If so, it could be the switch and you could have a bad clutch bearing. That's pretty serious, but not as serious as the other alternative.

If the compressor does not cut out, you're hearing it when you slow down because 1: your car isn't making as much noise and 2: your head pressure has risen dramatically, and the compressor is working harder.