Cryoelectron microscopy structures of human CNNM4 in distinct conformational states reveal how an unexpected dimer-of-asymmetric-dimers architecture mediates Mg2+/Na+ exchange and identify an intracellular acidic patch that may function in cellular magnesium sensing.