Yup, file until one just barely falls in, you can always use another magnet to pull it back out. Then I put a little gel super glue inside the hole along the edge and push the magnet back in. After doing all 4 and allowing them dry, I put the lid on and put the magnets in the lid with it in place. Then I dump some gel on top of them and let dry. I then fill the remaining space with black acrylic paint and let dry to hide everything. I'll see if I can snap a pic after work.
I should also mention that I reverse the polarity of the magnets from top to bottom. I find the door always fits a little better in one orientation over the other. This way it's a non point.
Tapa-shutup