There may actually be some form of "sub harness" in the console. From the '81 wiring diagram with the unlighted switches it describes the C107 connector (underlined numbers in the pdf) as a 6 pin connector and the C108 connector as a 2 pin. This would provide the hot and ground wires for the switches (4 + 2) and lockout connector (2). So the other side of C107 may have connected to the backs of the switches. In the later versions C107 is described as 8 pin.

It's possible it goes something like this:

Fuse box/chassis ground >>>> C107 >>>> backs of switches

and

C108 >>>> backs of switches.

If you can find where C107 and C108 plug in maybe you can get a plug-n-play install.