I removed an air pump that went out on my Volvo and put over 120,000 more miles on it without any ill consequences. If the pump goes out on my 300e I will do the same.
I believe the air pump on your M103 is driven by the serpentine belt. If you want to completely remove the pump you may have to put a smaller serpentine belt in. I have an '92 300e with an air pump and a '88 260e without one. A belt for a '88 260e from Fastlane it should fit your engine once the air pump is off. The only other thing you will need to do is plug the hoses real good.