I've done it twice
First time I removed the glove box and the speaker with its grille. Very difficult to access levers this way. The second time I removed the duct assembly and blower from below the dash and this made accessing the entire control assembly much easier. As I recall, guiding the control cables into their stays in the levers was the only tricky part upon reassembly.