Absolutely not. EGR has almost no impact on performance or fuel economy. The manifold may not have to be cleaned out every 150K miles, but that's not a big deal either. If you manifold is dirty, it's that much less that you're forcing people to breathe. EGR does reduce nitrogen oxides significantly, so keeping it in place is the responsible thing to do.

HOWEVER, removing it is illegal, an carries a $3500 civil penalty. While there may be no inspections at the moment, the more idiots there are removing pollution equipment, the sooner it that will end. If you for some reason think it's inconvenient to have EGR, think about how much less convenient it would be to be scrounging around for expensive parts to get past inspection. So keep your EGR, it's really no problem.