The only way you will find your problem is by applying the 4 P's, Push, Pull, Pry and Prod on everything suspension related under the car. Don't overlook the members in the rear suspension. A worn dogbone in the rear suspension will effect handling, probably more than most things that can wear out in the front.

Hope this helps,