Label Wise Significance Cross Entropy

In the process of machine learning, models are essentially defined by a
group of parameters in multiple layers. The parameters are learnt in a
process of optimization of the loss function which measures the
differences between model output and expected outputs (labels).
Normally, the expected class digit is expected to be 1 and the other
digits are 0s. All the digits are contributing to the loss function in
the cross-entropy loss function. If the number of classes is high, the
total number of 0 digits is much larger than the number of 1 digit, and
the total loss of 0 digits would be higher than the total loss of 1
digit loss. The direct result is the loss does not conform with the
object of the optimization. This paper introduced Label Wise
Significance Cross Entropy (LWSCE) as the loss function in model
training for effectively allocating the weighted loss function on the
classes bearing significant differences between model output and
expected output and ignoring the minor differences of 0 digits if the
differences are not significant enough. This method was experimented on
CIFAR10 and CIFAR100 dataset training. The experiments of cross-entropy,
cross-entropy with label smoothing and LWSCE were carried out in this
paper. The result shows LWACE outperforms better than the other loss
functions in CIFAR100 dataset.