"after I flung the switch up and down real fast for a minute I would see a spark in the switch and the fan would come on."

You gave the switch WHIPLASH .....

Now really, you need to take the switch out and clean the contacts or replace it.... and make sure it is getting good electricity to it...and from it to the fan.... check your vehicle grounding straps... bat to engine, engine to body, etc...