if you look at that link that I posted you can see how the mounting works. There is a hefty spring pushing two peices of metal out: the assembly looks like this (minus the periods):

.......
.....=========
............|
........MIRROR

That is pushed up into a metal bracket. So you need pull down and to one side hard enough to compress the spring a bit and pull the mirror out of the bracket.

__________________
Daily Driver: 02 E430 4MATIC
In the family: '03 E500 // '04 ML500 // 64 220SE