Thanks to
cigatron,
SlickWilly,
dwcraig1, and
awsum140 about the advice about gluing magnets (hopefully I didn't forget anybody). Unfortunately I started before all of the help. But now I know next time. I used superglue twice and failed. I only let it set for an hour or two. I tried again a third time (
three time's a charm) and let it set for 10 hours and all of the magnets held this time.
Although after having one set on my desk for a few hours, I heard a solid click from one of them. I thought oh no, a magnet released. I checked and sure enough a magnetic let go. But wait, that isn't one I glued! It was one on the other side that released. Glued that one with superglue and let it cure for 10 hours. That was over 12 hours ago and everything is holding for now. But if it happens again, I know what to do thanks to you guys.