Hello Zod,
You can calculate voltage under a load if you calculate for cell resistance, which may be easier to do while the cell is on the charger if you can tap into the charging circuit (may be easier to do than tapping into the connection to the atty). You can calculate for cell resistance via simple algebra if you know the cell voltage, charging voltage, and charging current. The charging voltage differential divided by X = charge rate. X = cell resistance. Voltage under a load is the voltage remaining after calculating for the voltage drop from the cell resistance. The open circuit voltage divided by the The cell resistance divided by the total circuit resistance represents the proportion of the voltage that will be "lost" as a result of cell resistance.
18650s and [I think] your Riva should be reading closer to 4.2V fresh off the charger. You might want to check that against a different DMM to be sure you are getting good readings or check again right after a charge to be sure.
Eric