main.pdf (4.11 MB)
Download file

Masked Token Enabled Pre-training: A Task-Agnostic Approach for Understanding Complex Traffic Flow

Download (4.11 MB)
posted on 2022-02-10, 05:09 authored by Lu HouLu Hou, Yunxin Geng, Lingyi Han, Haojun Yang, Kan Zheng, Xianbing Wang

The conventional deep learning model performs well for traffic flow analysis by training with a large number of labeled data using a one-model-for-one-task approach, leading to huge computational complexity in dynamic intelligent transportation system (ITS) applications. To overcome this limitation, this paper propose a Token-based Self-Supervised Network (TSSN), which can learn TF features in a task-agnostic way, and provide a well bootstrapped pre-training model for a variety of tasks. TSSN tokenizes TF data into segments, each of which is named as a token and comprised of numerous consecutive points. Masked Token Prediction (MTP), a pretext task, is designed to understand the TF correlations by forecasting tokens that are randomly masked. MTP enables TSSN to capture the high-level intrinsic semantics of TF, and provide general-purpose token embeddings. Therefore, TSSN can be more generalized while keeping high performance. As a result, by replacing the final fully-connected layer with a set of untrained new layers and fine-tuning with small-scale task- specific data, TSSN can be deployed for a variety of downstream tasks. The simulation results demonstrate that the TSSN can improve overall performance on various downstream tasks when compared to state-of-the-art models.


Email Address of Submitting Author

ORCID of Submitting Author


Submitting Author's Institution

Beijing University of Posts and Telecommunications

Submitting Author's Country

  • China

Usage metrics