Volumetric Lightmap Holes Appear Every Fifth Build

Volumetric lightmap holes appear every fifth build. 

Found in 4.19 CL# 4033788, 4.20 CL# 4302132, and 4.21 CL# 4348893

1. Open the Editor
2. Create an empty level
3. Enable volumetric lightmaps
4. Set detail cell size to 50
5. Go to Show->Visualise->Volumetric Lightmaps
6. Add some cubes and a static light
7. Bake 5-10 times

Results:
The volumetric lightmap will have areas missing on the fifth build.

Expected:

No gaps would be in the lightmap.