loading page

Loss Mask is all Your Need
  • +1
  • HuaShun Li ,
  • Mengshi Zhu ,
  • JingZhao Li ,
  • Meng Zhu
HuaShun Li
Anhui University of Science and Technology, Anhui University of Science and Technology, Anhui University of Science and Technology

Corresponding Author:[email protected]

Author Profile
Mengshi Zhu
Author Profile
JingZhao Li
Author Profile

Abstract

In this paper, we propose an idea to improve various types of loss functions. It is different from the current idea of balancing the errors by increasing the number of input samples in each batch. We directly mask the top values of the error ranking to zero. During the error back-propagation, this means that the samples corresponding to that loss will not affect the parameter update of the network. In other words, even if a small number of samples are artificially mislabeled, it will not theoretically have much impact on the performance of the network. Instead, deliberately discarding anomalous losses will help smooth the training of the network. We conduct experiments on several regression and classification tasks, and the results show that the proposed method in this paper can effectively improve the expected performance of the network.