Polish the shaft of the button and the inner opening it slides through until it shines like a mirror.
With magnets, it will slide like butter.
Keep an eye on the rice pin. Brass is soft. If it wobbles, wrap it with Teflon tape or similar, or just replace it.
These are a few tips and
mods off the top of my head.
Edit: for performance, copper contacts....or upgrade to a copper nemesis.