(for now at least)

1) Pull each GP and test on the battery. Its critical at this point to know that they are all functional

2) Before you reconnect the plug wires, put a 12 test light in-line between the plug wire and ground. It should light up for 30-45 sec in the pre-glow position.

3) Verify that the wiring Crossed wires. Easy enough to check, just make sure that the same colored wire is running from pin #1 to plug #1. This idea is based on a bad plug (wired to #1) in a different cylinder. (admittedly kind of stupid, but I've seen stranger things)

If the first two tests are positive, then I'd say that yes you have a relay that is functionally good, but for some reason won't send a signal to the dash light. Personally, I'd ignore it until I could find an inexpensive (salvage) replacement. The dash light is unimportant really, you'll know if the GP's aren't working.

BTW, make sure your block heater is in good working order before your first Chicago winter.