It is against my user agreement to allow my router to be used by the neighbors.

However, even with their blessing, I would have to be an idiot to leave it open. You do not know what someone is downloading through your connection, and won't find out until the boys in blue carry off all of your computer equipment. Sure, you're not guilty of anything, but you are still without your computers for a few months.

I don't know which is worse, between the stupidity of someone who doesn't lock their equipment down, or the unethical behavior of those who look for such stupidity.

You tell who's using your equipment by looking at the logs. Logs are often configurable. You may be able to log to a remote (inside) machine, for ease of access.