I hear ya. You know the evils lurking in your car, but not in one you look at / buy. Many of us tend to be a bit self-critical. Knowing the issues with my cars, I usually assume, incorrectly, other cars are better.

My vote is keep the car and drive it until the wheels fall off. (Assuming no safety concerns with the rust.)