Yes replace the primer pump.

Before you do however, check the engine mount. If it has been leaking for some time, you may need to replace that as well . . . (from experience - see thread "When bad mounts go worse"