Chances are that the culprit is the switch. The same motor is used to move the seat up or down...........you just swap the polarity of the motor to get it to reverse direction (DC motor). So, if the seat moves in one direction, it should move in the reverse direction. The switchs get gummy over time, and the gum will insulate the swiitch contacts causing the switch to fail.
You can buy a new switch (about $85 for the driver's side), or you can remove the switch and pretty much flood the switch with electrical contact cleaner. DO NOT use any oil product such as WD-40 since it will leave a film that is just as bad as the gum you are trying to remove. Get a can of electrical contact cleaner.........or TV tuner cleaner (from Radio Shack) and spray it in all the openings. At that time, cycle the switch 20 to 30 times in each direction of travel. The contact cleaner will flash off and should leave no residue. Reinstall the switch and give it a try before you put all the trim pieces back in place. If this doesn't do it, replace the switch. It's an eay repair.