If you're asking about testing voltage red lead on top of battery black on bottom, set on a 6v or 9v range/scale.
If your asking about how to determine if they need to replaced ( end of lifespan) you can't really using a multimeter.
Replace them if they won't charge to 4.2v or drop when resting a few minutes to hours, or have a noticeably shorter run time than when new.
They have an average lifespan of 300 - 500 recharge cycles, many things can influence that to be lower, heat, cold, improper storage, age, over charging / discharging, charging rate, amp demand to name a few.