Orthogonal sparse convolution module for deep neural networks acceleration

This paper introduces a method to generate a new efficient sparse convolutional module as a substitute for regular convolution layer.In contrast to previous group convolution modules, this module firstly expand the input features to avoid information loss.By a orthogonalization pruning algorithm, only the linearly independent convolution kernels are conserved.This novel orthogonalization pruning algorithm can significantly reduce the redundant information contained in the features by eliminating the kernels' relevance.The reconfigured and fine-tuned module has a competitive accuracy with less computation resources cost comparing to regular convolution.This approach could be implemented in any modern CNN model.

The source codes are available at \url{https://github.com/wangjincheng722/OrthSCB