The air pump has a test done on it by the ECU , where it gets energized to see if the o2 sensor reads a lean condition..
You have unplugged the pump, but the Pump SOV is also energized at the same time [ from the same power source] , opening the control valve between the pump and the engine.
BC you say it happens when the pump would come on, I would look for a vac leak at this main hose [ around the control valve], as the valve is still energized even though the pump is not...
I would also see if the check valve is gummed up and stuck open,, this would open the exhaust to pump plumbing when the SOV gets a signal....