It may be time to improve the neuron of artificial neural network

Artificial neural networks (ANNs) have won numerous contests in pattern recognition, machine learning, or artificial intelligence in recent years. The neuron of ANNs was designed by the stereotypical knowledge of biological neurons 70 years ago. Artificial Neuron is expressed as f(wx+b) or f(WX). This design does not consider dendrites' information processing capacity. However, some recent studies show that biological dendrites participate in the pre-calculation of input data. Concretely, biological dendrites play a role in extracting the interaction information among inputs (features). Therefore, it may be time to improve the neuron of ANNs. According to our previous studies (DD), this paper adds the dendrites' function to artificial Neuron. The dendrite function can be expressed as W^i,i-1A^i-1 ○ A^{0|1|2|...|i-1} . The generalized new neuron can be expressed as f(W(W^i,i-1A^i-1 ○ A^{0|1|2|...|i-1})).The simplified new neuron be expressed as f(∑(WA ○ X)). After improving the neuron, there are so many networks to try. This paper shows some basic architecture for reference in the future.

Interesting things: (1) The computational complexity of dendrite modules (W^i,i-1A^i-1 ○ A^i-1) connected in series is far lower than Horner's method. Will this speed up the calculation of basic functions in computers? (2) The range of sight of animals has a gradient, but the convolution layer does not have this characteristic. This paper proposes receptive fields with a gradient. (3) The networks using Gang neurons can delete traditional networks' Fully-connected Layer. In other words, the Fully-connected Layers' parameters are assigned to a single neuron, which reduces the parameters of a network for the same mapping capacity.

One important thing: ResDD can replace the current all ANNs' Neurons (ResDD modules+One Linear module)! ResDD has controllable precision for better generalization capability!

Gang neuron code is available at https://github.com/liugang1234567/Gang-neuron.