loading page

Toward Improved Reliability of Deep Learning Based Systems Through Online Relabeling of Potential Adversarial Attacks
  • +4
  • Shawqi Al-Maliki ,
  • Faissal El Bouanani ,
  • Kashif Ahmad ,
  • Mohamed Abdallah ,
  • Dinh Hoang ,
  • Dusit Niyato ,
  • Ala Al-Fuqaha
Shawqi Al-Maliki
Hamad Bin Khalifa University, Hamad Bin Khalifa University

Corresponding Author:[email protected]

Author Profile
Faissal El Bouanani
Author Profile
Kashif Ahmad
Author Profile
Mohamed Abdallah
Author Profile
Dinh Hoang
Author Profile
Dusit Niyato
Author Profile
Ala Al-Fuqaha
Author Profile


Deep Neural Networks (DDNs) have shown vulnerability to well-designed adversarial examples. Researchers in industry and academia have proposed many adversarial example defense techniques. However, none can provide complete robustness. The cutting-edge defense techniques offer partial reliability. Thus, complementing them with another layer of protection is a must, especially for mission-critical applications. This paper proposes a novel Online Selection and Relabeling Algorithm (OSRA) that opportunistically utilizes a limited number of crowdsourced workers to maximize the ML system’s robustness. OSRA strives to use crowdsourced workers effectively by selecting the most suspicious inputs and moving them to the crowdsourced workers to be validated and corrected. As a result, the impact of adversarial examples gets reduced, and accordingly, the ML system becomes more robust. We also proposed a heuristic threshold selection method that contributes to enhancing the prediction system’s reliability. We empirically validated our proposed algorithm and found that it can efficiently and optimally utilize the allocated budget for crowdsourcing. It is also effectively integrated with a state-of-the-art black-box defense technique, resulting in a more robust system. Simulation results show that OSRA can outperform a random selection algorithm by 60% and achieve comparable performance to an optimal offline selection benchmark. They also show that OSRA’s performance has a positive correlation with system robustness.
2023Published in IEEE Transactions on Reliability on pages 1-16. 10.1109/TR.2023.3298685