You're doing nothing wrong.
Try holding the button down a little longer. This has happened to me, too, from time to time. I was either not holding the button down long enough, or I needed to turn off the ignition and turn it back on to get it to work.
2001 E430, Bourdeaux Red, Oyster interior.
1973 280SE 4.5, 170,000 miles. 568 Signal Red, Black MB Tex. "The Red Baron".