Yep, it does matter which ones go back where. But look on the bright side, you have a 50/50 chance of getting it right.
If the bearings are in reasonable shape, I'd chance it and put them in again. Drive a little, the jack each side side up and check for play/noise and adjust as req'd.