The intercooler's function is to cool the compressed "boost" air before it enters the intake. All fluids (air is a fluid) heat-up when compressed and cooler air is denser (more oxygen per unit of volume) than warmer air so the theory is if you can lower the temperature of the pressurized air, back to near ambient temps it will produce more power when mixed with the fuel. Cooling the intake air probably also help lower exhaust gas temps too.
As others have said, if you disconnect the intercooler (and don't connect the boost pipe directly to the intake) then you will get no boost and be able to draw unfiltered air into the intake...both bad things.
If, however, you were able to bypass the intercooler and connect the turbo's boost pipe directly to the intake all you would have is a car with boost that wasn't cooled and would not be as powerful and efficient as it is with the intercooler. You might also cause it to run hotter than it was designed to but since most of us don't have a way to monitor EGT I don't know how we would know if we did any damage or not.
Here is an article that explains it with numbers:
http://autospeed.com/cms/A_109789/printArticle.html