From what you've described (and thank you for all of the details, too many such posts are far too sketchy), it indeed needs to be replaced. It's just plug and play, but as you allude to coolant, when is the last time it was flushed and refilled with the proper mix of spec coolant and distilled water? If you don't remember, then this is as good a time as any. If you flush with tap water you should pull the block drains as well, but given the cost of distilled water at discount places (target, wallyworld) that's not a bad option either.

Good luck.