There are two attachments on this panel: one high and one low. The lower one is a straight in, plastic, ribbed stud into a pressed socket in the quarter panel. Use a flat plasic wedge (or plastic spatula) from the bottom and pry outwards. The upper attach. (your loose one) is a clip that "hooks" down into a slot. This lifts up (or could be slides down). You'll feel the direction once you pry the lower stud loose.
The Golden Rule
1984 300SD (bought new, sold it in 1988, bought it back 13 yrs. later)