When is the last time the calipers were serviced? Do you keep up with the necessary brake bleeding? I get the impression your pads are fairly new.
Your problem could be from a number of things....sticky caliper piston, dry or corroded slides on the calipers, brake fluid leaking onto the pads or shoes. A history of when and how your brakes were last serviced, and a good inspection may help determine what the problem is.
95 E320 Cabriolet, 131K