Sounds to me like you have a bad connecvtion at the temp sensor or a failing sensor..the relay would not click if it was stuck in the ON pospsiton

Yoy can tell which relay by unplugging it when the fan is on. Sould be the green one