First check the blower fuse located in an auxiliary fuse holder ahead of the brake fluid reservoir. It is a 30 amp strip fuse that can crack and open intermittently. Touch it in the center and see if a crack opens up.
Also check the lower current fuse(s) in the fuse box that feed the climate control system. The ones with the silver coloured element can develop corrosion at the contact points. Spin them for a temporary fix but replacing with the copper/brass coloured ones is best.
Next time it acts up, wiggle the ignition switch. There is a terminal coming off the switch that powers the blower. It can become intermittent at the worst possible time. Mine acted up at -30 degrees C.
The push button assembly is also a possibility. Solder joint cracks develop that can cause a wide variety of problems.