loading page

A Comparison of Neural Network-Based Intrusion Detection Against Signature-Based Detection in IoT Networks
  • Max Schrötter,
  • Andreas Niemann,
  • Bettina Schnor
Max Schrötter
Institute of Computational Science, University of Potsdam

Corresponding Author:[email protected]

Author Profile
Andreas Niemann
Institute of Computational Science, University of Potsdam
Bettina Schnor
Institute of Computational Science, University of Potsdam


Over the last few years, a plethora of papers presenting machine learning-based approaches for intrusion detection has been published. However, the majority of those papers does not compare their results with a proper baseline of a signature-based intrusion detection system. Thus violating good machine learning practices. In order to evaluate the pros and cons of the machine learning-based approach, we replicated a research study which use a deep neural network model for intrusion detection. The results of our replicated research study expose several systematic problems with the used datasets and evaluation methods. In our experiments, a signature-based intrusion detection system with a minimal setup was able to outperform the tested model even under small traffic changes. While testing the replicated neural network on a new dataset recorded in the same environment with the same attacks using the same tools showed that the accuracy of the neural network dropped to 54%. Furthermore, the often claimed advantage of being able to detect zero-day attacks could not be seen in our experiments.
01 Feb 2024Submitted to TechRxiv
09 Feb 2024Published in TechRxiv