Welcome to the wonderful world of entropy! Here comes a thermodynamic lesson whether you wanted it or not.
Of the overall power generated by the combustion of the compressed fuel/air mixture in the combustion chamber of your car's engine, only about 25%-30% of it actually goes to the flywheel, driving your car's transmission/drive train/wheels. As you have already noticed, a large portion goes out the tailpipe as heated gases, a large portion is transferred to the engine coolant which then transfers it to the environment, and a large portion goes via radiant and convection heat transfer to heating up the air and the areas around the engine compartment. Unfortunately these are thermodynamic facts of life, and you can't do much better than that 25%-30% efficiency without taking extreme measures. Diesels are a little more efficient than spark-engines because their combustion temperature is higher.
If you hadn't guessed, yes I'm one of those engineer types.
Actually, your engine's exhaust temperature may indeed be too high, which may be a reason for concern, but if you're talking about the normal temperature of the gases leaving the tailpipe after running the car for a period of time, I would fully expect them to be too hot to hold you hand in.