Do you have an aftermarket alarm system? I had one in a former car where the alarm brain would sometimes suck the battery dry! Yanked the unit out and never had a problem since.
Check your battery cells (if yours is not the sealed type) to make sure each cell has electrolyte in them. Add distilled water until the water level covers the plates.
Check the alternator for loose connections. I believe there are two...a larger primary lead, and a smaller possibly ground wire. That may be why you see voltage drop, then come back up, as the car vibrations cause the charging lead to be connected and disconnected at times.
It could be as you stated, the voltage regulator.
The regulator on some models is external and can be replaced separately (and WAY less expensive than a full alternator replacement). Otherwise, if you have the internal regulator, you may have to get the whole thing replaced or reconditioned at an alternator shop.
To me really, it sounds like the alternator is on its way out. Problem could be intermittent, and later, occur more frequently until...total failure!
I once had an alternator checked out to be in good health at a local service station one week before I ventured out on a road trip. By the end of the trip, it completely died!
2009 ML350 (84K) - Family vehicle
2001 CLK430 Cabriolet (71K) - Wife's car
2005 BMW 645CI (124K) - My daily driver
2012 Mustang V6 (60K) - Daughter's car