Abstract: In real electromagnetic environments, the emergence of unknown modulation types poses significant challenges to signal recognition. To address this issue, an open-set modulation recognition method based on multi-modal semantic reconstruction was proposed. The method combined the I/Q and the time-frequency modalities to deeply explore the semantic features of signals across time, space, and frequency dimensions. A cross-correlation based alignment approach was designed to ensure feature consistency between modalities. Based on this, identification of unknown classes was achieved through semantic reconstruction, which enabled the identification of unknown classes and exhibited stronger robustness to noise and parameter variations. Experimental results demonstrate that when there are four unknown modulation types, the average AUROC and OSCR metrics on simulated dataset reach 95.24 and 96.25, respectively, fully validating the effectiveness and robustness of the method.