Multimodal fusion via ship trajectory understanding for cognitive maritime intelligence: A case study of the Fujian sea
Abstract
The maritime traffic in coastal areas is characterized by high complexity, and certain vessels frequently alter their static Automatic Identification System (AIS) data to evade regulatory oversight. This practice leads to inaccuracies and omissions in vessel tracking, making it essential to classify vessel types through a multidimensional analysis integrating diverse information sources and advanced artificial intelligence techniques. However, the direct application of deep learning models to maritime vessel trajectory data is hindered by high dimensionality, sparsity, and inherent noise, which impede semantic and spatial information extraction and often result in suboptimal classification. To overcome these limitations, this study proposes a grid-based maritime trajectory classification model that leverages cross-modal fusion. By integrating joint text-image bimodal modeling with a novel dynamic alignment mechanism, the proposed approach enables real-time guidance of local features through semantic information. Specifically, AIS trajectories are discretized into grid-based text sequences and trajectory images. Textual features are extracted using a Bidirectional Long Short-Term Memory network, while visual representations are processed by a Dilated Residual Network. A dynamic alignment module then adaptively modulates image features with semantic information from the text encoder, enhancing deep cross-modal interactions. Experimental validation on real-world AIS datasets of Fujian sea demonstrates that the proposed model outperforms seven unimodal baselines and five cross-modal models, consistently achieving a 2% improvement in both classification accuracy and F1-score among cross-modal approaches. Systematic ablation studies further confirm the effectiveness of the feature fusion mechanism. In summary, this research advances maritime domain awareness by enabling more accurate and robust vessel trajectory classification. The proposed framework offers significant potential for monitoring illegal vessel activities, contributing to improved maritime security and operational oversight for shipping management.