That may work, but only if the jerk remembers the numbers. For strong encryption, this means remembering two fairly large primes.

I don't know about you, but I can't remember a phone number that I only dial once. Ten minutes later and it's about gone.

The decryption game is often one of factoring a composite number into its (two) prime constituents. That is a hard problem, where you essentially need to try all primes and actually do the division, until you get one that yields no remainder.