The original wheel is 17x8.5 and the tires are Michelin 245/45R17. Can I replace these with an AMG 18x8 wheel and 245/40R18 tires and receive better handling in addition to better looks?

first thing to do is compare the weight of the two wheel/tire combinations.
Tire manufacturer sites may be able to give you the weight of each of their tires.

if the weight of the 18 inch wheels are heavier - 1lb/wheel is a lot - then ride or braking or handling can suffer depending on how fast you drive. The 18" wheels/tires could very well weigh less.

It's not like your using the larger wheels for fitting a larger brake.

If your one goal where to gain high speed through a smooth killer turn then the lower profile could help, though if you exceed traction the break away characteristics could suffer.

If your driving broken roads then your suspension will benefit by less weight and thicker sidewall.

Then you factor in slip angles and steering reaction and depending on the tire
you could either detect the difference or not notice a thing.

Bottom line, eighteen inch are nice and if you drive hard and use good tires it could be a lot of fun. depends on the model car.