Willrev,

A battery should have about 12.5 +/- volts - without the engine and no load. With the car running and the alt working you should be able to measure 13.5 or perhaps a bit higher across the battery.

I would try that with no load first, then add headlights and other loads. If it drops again under load the alt is not keeping up for some reason.

Chuck