One thing you should be aware of; lower fan speeds will normally yield lower exit port temperatures. The lower flow speed allows the air to "dwell" longer as it passes through the cooling coils thus maximizing transfer of the cooling effect. The higher speed seems to cool faster on the basis of volume flow but a thermometer at the exit port will show cooler air at lower fan speed.

That's not to say you don't have a problem but part of the difference you see is normal.