We show how the manifold $T\ast SU(2,2)$ arises as a symplectic reduction from eight copies of the twistor space. Some of the constraints in the twistor space correspond to an octahedral configuration of twelve complex light rays in the Minkowski space. We discuss a mechanism to break the conformal symmetry down to the twistorial parametrisation of $T\ast SL(2,C)$ used in loop quantum gravity.