Yes indeed there is lots of foam around the heater core. A new heater core comes with all the foam strips necessary to seal it up and keep it from rattling.

I would get some weather stripping from the home center and seal it and pad it with that.

Good luck,