Sounds like the PO (tried to) fit non-standard rims. Your tie-rod end may be damaged from scrubbing the wheel (if nothing else the resulting frictional heat would be bad), but replacing it won't fix the rubbing. The end of the tie rod is in a fixed position relative to the wheel.

Thicker spacers (and new lug bolts) or correct wheels is the actual fix.

It is *possible* that you have something bent that wasn't repaired in the accident, so it may be worth having a body shop look at it. But your reference to the wheels being changed (and that they already have a spacer) suggests otherwise.

Worn ball joints will give you play in the vertical plane (top to bottom, 12-6 o'clock), not side-to-side.

I'd probably start at a body shop, tell them what's going on, same as you did in this thread, and if they find nothing out-of position, then you know the next step.

Good luck.