The drive from NJ upward is terrible because it goes through some very unpleasant freeways around NYC, and CT isn't more pleasant either. It'll probably take like 4 hours to negotiate that traffic one way... If it was in NY, NJ, PA, DE, or even MD, I'd come... Too bad the NJ experience wasn't good to you guys - sounds like you didn't ask the right person to find a good place to meet in NJ!!
The weekend of June 24th is out for me, too.