Boost isn't free, it causes backpressure in the exhaust that makes the pistons work harder to push exhaust out of the cylinders.

The 617 is good for 88hp without the turbo and it only takes around 20hp to push the car down the road at a constant 55mph. The lower the boost the less work the engine is doing to make that 20hp.