Oh I didn't process your original post correctly. You are actually getting aluminum rubbing off on your fingers? I have not experienced that at all. I just get a greying where my fingers hit on the mod itself, but not on my fingers. I took the finish on mine up to 1500 I think, but it has since been rattling around in a cup holder, dropped, and maybe used to open a bottle or two (I cannot confirm or deny the latter).
If I was getting oxidation on my hands, I would throw it on a buffer and then put a single coat of rattle-can clear coat on it. That would protect the finish and prevent your fingers from turning grey. If you could get it anodized, it would also achieve the same effect.