Having lived in cold climates I would guess it is moisture inside the door. Questions:
-how do you know it isn't the switch?
-does it happen more often, say first thing in the morning than middle of the day?
-have your tried rotating the fuses (sometimes they look good but are actually bad)
'03 E320 Wagon-Sold
'95 E320 Wagon-Went to Ex
'93 190E 2.6-Wrecked
'91 300E-Went to Ex
'65 911 Coupe (#302580)