The OFDM chirp signal is suitable for MIMO radar applications due to its large time-bandwidth product, constant time-domain, and almost constant frequency-domain modulus. Particularly, by introducing the time-frequency structure of the non-linear frequency modulation (NLFM) signal into the design of an OFDM chirp waveform, a new OFDM-NLFM waveform with low peak auto-correlation sidelobe ratio (PASR) and peak cross-correlation ratio (PCCR) is obtained. IN-OFDM is the OFDM-NLFM waveform set currently with the lowest PASR and PCCR. Here we construct the optimization model of the OFDM-NLFM waveform set with the objective function being the maximum of the PASR and PCCR. Further, this paper proposes an OFDM-NLFM waveform set design algorithm inspired by alternating optimization. We implement the proposed algorithm by the alternate execution of two sub-algorithms. First, we keep both the sub-chirp sequence code matrix and sub-chirp rate plus and minus (PM) code matrix unchanged and use the particle swarm optimization (PSO) algorithm to obtain the optimal parameters of the NLFM signal's time-frequency structure (NLFM parameters). Next, we keep current optimal NLFM parameters unchanged, and optimize the sub-chirp sequence code matrix and sub-chirp rate PM code matrix using the block coordinate descent (BCD) algorithm. The above two sub-algorithms are alternately executed until the objective function converges to the optimal solution. The results show that the PASR and PCCR of the obtained OFDM-NLFM waveform set are about 5 dB lower than that of the IN-OFDM.