I've had that happen randomly, sometimes when a spy has died in a settlement, the fog of war decends. Yet if I move a unit out, it's raised. Move it back into the settlement, fow down again. Weird.