Yes, you should replace the cover. It keeps battery fumes and rainwater from the connections.
The relay just plugs in, no click, but it should be relatively hard to get out once shoved in. If you put the new one in and it doesn't fix the problem, verify that the fuse is hot (battery voltage) with a test light or voltmeter. It of course won't work without being fully plugged in.
BTW if you still have the original OVP the new one will be slightly different with an extra pin. Don't worry about it.
Bosch Master, ASE Master, L1
33 years MB technician