Short answer. Yes, always. Every mech ive bought has machile oils of some kind and/or residues left behind by the machining process. Plus, brass oxidizes, quickly. Noalox or other lubricants help to protect againat that.
Clean em. Besides they will look far better after, as well as perform their best. Brasso is your friend... and yes it cleans stainless steel just as well.