There is indeed an allen screw inside the door.You don't need to remove the door panel. You only need to remove the weather strip at door lock cylinder height. This will reveal a little hole where the screw that holds the lock cylinder is located.