I've had a bad rebuilt before, it happens.

That should have been done 4 starters ago! It is unlikely 4 starters in a row give intermittent starts.

I disconnected and cleaned all of the ground connections and still no crank with the O'reilly's starter. I installed the old starter that would occasionally hang up on the flywheel and it cranked first try.

The failures of the previous starters were all different, so I don't think that there is an ignition issue.

Bench test the O'reilly starter. Is is dead? I have a suspicion it is not. Since cleaning the connections where they are bolted on does not seem to solve these long standing intermittent problems, you have to go further. I think I have mentioned this a few months ago in your other posts: You need to cut open the heat shrink and inspect the crimps of all the power and ground leads in the starter circuit. Wire brush and use plenty of flux and solder all the crimps with a 100 W iron with a fat tip. Or buy all new cables. Don't know if they are still available from the dealer.

I got my money back from O'Reilly's for that Ultima starter.

I took a long drive up in the hills and got the engine nice and hot after installing the spare starter I have that occasionally hangs up on the flywheel. After parking the car and waiting a minute, I turned the key and it fired right up. I'd like to put a new solenoid on the other starter that was failing when hot. Can anyone recommend a brand? Are there other parts of the starter that fail after getting heat soaked?

Bosch for the win!
 

Bosch is absolutely the way to go if you're going to replace the solenoid. I had a cheap chinese crap solenoid give me fits for weeks in my 240D. It'll cost you (I think mine was $55, had to special order it), but it put an end to the nonsense. For that matter, the genuine Bosch rebuilt starter I bought through the site here for my W201 was absolutely worth the money - spins noticeably faster than whatever was in there before, never hesitates on the solenoid. I'll never buy a cheap local rebuild again.

Kurt