The noise is present because the USB charger creates a "noisy" power supply to the phone. Noisy power supply to the phone makes a noisy signal out of the DAC feeding the amplifier. You hear the noisy "sound floor" because the amplifier amplifies everything that comes into it, whether it be noise or actual audio signal. The noise is still present on your headphones, but since the audio signal is not being amplified a 2nd time, you don't really hear it since the DAC output is being loaded by the impedance of the headphones.

There is such a thing as a ground loop isolator, it's essentially just an isolation transformer. You can try something like that to see if it helps, but if the issue is a noisy signal from the charger, the issue will still be present. A new/better phone charger is also *MUCH* cheaper than an isolator. The same noise problem was present on PC's and laptops in the early 2000s due to crappy filtering on the switching power supplies they had onboard, so it's not an unknown/uncommon phenomenon. An actual ground loop would typically cause "hum" (think open mic cord) or alternator whine through the sound system.