You need to pressure-check the cooling system, wherever the coolant comes out is where the leak is.

Keep the thermostat in, taking it out will cause overheating in some situations.

You said water, but you really mean a 50/50 mix of water and coolant right? Straight water is not acceptable for use in the cooling system.