I agree that the issue is probably drag from the brakes.
The looseness you mention doesn't sound right. You may want to check the following thread to see if any of it may help you:
Steering vibration comes and goes
Don't buy any bearings stamped "Made in China"!