I've seen this happen when the aux fan doesn't work on hot days. It is the aux fan's job to cool the AC radiator when no air is moving across the coolant. This is the same as the radiator fan for the engine coolant.
That could be your case if this only happens when it is hot. Check the fuses and confirm that the fan comes on. It should only take a few minutes at idle with the AC on for the fan to engage.
'87 300TD 304kmi (RIP)
'95 Toyota Camry Wagon 125kmi