The proposed work develops a Long Short Term Memory (LSTM) model for multi class classification of driving maneuver from sensor fusion time series dataset. The work also analyzes the significance of sensor fusion data change rule and utilized the idea with deep learning time series multi class classification of driving maneuver. We also proposed some hypotheses which are proven by the experimental results. The proposed model provides Train Accuracy: 99.98, Test Accuracy: 97.2021, Precision: 0.974848, Recall: 0.960154 and F1 score: 0.967028. The Mean Per Class Error (MPCE) is 0.01386. The significant rules can accelerate the feature extraction process of driving data. Moreover, it helps in automatic labeling of unlabeled dataset. Our future approach is to develop a tool for generating categorical label for unlabeled dataset. Besides, we have plan to optimize the proposed classifier using grid search.