Grind, file, or cut the brass sleeve off the shaft. A pair of wire cutters (dykes) works well for this purpose. Next best thing is a coarse file.