The brakes should never apply themselves. Are you sure you just aren't stuck in a low gear and the engine RPM is slowing the car down? If so, it's entirely possible that you get an abrupt shift when you hit the gas again. Check for vacuum leaks, barring that sounds like time to have the transmission checked out.