I go through this as well and regularly I get the customary " You speak English well " comment often. I have been here since I was 14 and generally if you are not white, they assume you are a FOB immigrant and English is out of your realm. They are also not aware that in many countries, English has been taught as long or longer than it has been instructed in the US. What annoys me most is when I and my wife are in hick towns, the stares get to your nerve big time, probably they see an Indian/Mexican married to an Oriental and find it pretty amusing
