I think you should go by your ear.

If the car really isn't any louder, or at least doesnt bother you or other drivers, I see no need to add a rear muffler.

And your mechanic is right -- less exhaust restriction is almost always a good thing.

I say leave it.