This could be one of those problems that until it gets a little worse you may not solve.
I do not know if the panel has a common ground?
Do you notice a volt drop at idle indicating an alternator problem-- this should not happen. But then again the battery will take up some of this problem.
I have had a bad alternator that caused a marginal battery to fail in the end -- Even A new battery can not take it for long