I tried to improve things by putting some nice Polk Audio speakers in my rear deck, and I don't think I gained a thing. Kinda wish I'd left it alone. The factory speakers aren't the greatest, but considering how small they are, aftermarket one don't seem to be able to improve things much. The real improvement came when I added the subwoofer in the trunk to fill in the bass. Much more better now!
I was going to replace the front ones too, but decided not to after being disappointed with the rear swap. Now I'm sure i won't.