The answer lies in the covenants in the deed.

If the city has a right of way over that line (and I'm sure that it does), the deed will tell all.

He builds on it.........they need to service the line.........they tear down his building.