Update:
This was the old system of holding belt tension:
Seems the bolts weren't enough to keep the belt tight. It even loosened after driving about 120 miles without the compressor engaged
Jim, AKA engawork looked at it and suggested bolts to hold tension here on the bottom:
He actually suggested welding lugs on the side of the bracket with through bolts but I drilled/tapped a plate and welded it on. I need to replace the bolts with shorter ones and install lock nuts. But I think this'll solve my loose belt problem. Before it was near impossible to hold tension on the belt AND tighten the adjustment bolt...
I'm going on a 400 mile trip tomorrow and I'll test it out with the compressor off.
Wish me luck!