Open-Vocabulary mmWave Radar Air-Writing Recognition via Cross-Modal Physical Synthesis

Abstract

Millimeter-wave (mmWave) radar provides a promising modality for contactless air-writing interaction, yet the high cost of collecting real radar data makes it difficult to move beyond closed-vocabulary recognition and generalize to unseen words. This paper presents mmScribe++, a cross-modal trajectory-to-radar synthesis and domain-adaptive recognition framework that addresses this limitation. It leverages publicly available online handwriting trajectories as kinematic priors and synthesizes micro-Doppler spectrograms using a lightweight position-driven FMCW radar simulation engine that models plausible slow-time phase evolution and recognition-relevant Doppler broadening. A staged domain-adaptive training strategy first learns character-level sequence priors from mixed synthetic and real data and then adapts the radar feature extractor, encoder, and CTC branch to the real domain while keeping the attention decoder fixed. Statistical language models are optionally applied for inference-time spelling correction. Compared with real-data-only training, synthetic vocabulary expansion and domain adaptation reduce the OOV CER from 51.03% to 39.35% and improve the OOV WAR from 4.02% to 21.80%. Language-model post-processing further increases the OOV WAR to 29.93%, with an OOV CER of 40.69%.