loading page

BIDIRECTIONAL MOTION LEARNING USING TRANSFORMER BASED SIAMESE NETWORK FOR VIDEO ANOMALY DETECTION
  • Nazia Aslam ,
  • Krishnan Srinivasan Narayanan ,
  • Maheshkumar H Kolekar
Nazia Aslam
Indian Institute of Technology Patna

Corresponding Author:[email protected]

Author Profile
Krishnan Srinivasan Narayanan
Author Profile
Maheshkumar H Kolekar
Author Profile

Abstract

Video anomaly detection aims to identify unusual activity in videos. Recently, reconstruction and future frame prediction-based approaches have been frequently used to detect anomalies. However, due to the high generalization capability of deep neural networks, the reconstruction-based algorithms recreate the abnormal pixels with the normal ones. Therefore, we have developed a novel framework based on future prediction, capable of learning the video data's bidirectional motion with the help of a transformer-based siamese network. During the forward pass, the siamese network predicts the future sequence and learns the forward motion. In contrast, the backward movement is realized by predicting the inverse sequence. The model has trained on normal video data adversarially. In addition to adversarial loss, inverse loss and soft-DTW loss have been incorporated to make the model more objective. Extensive experiments have been conducted on the benchmark datasets to validate the efficacy of model.