loading page

Improving Image Recognition Accuracy in Neural Networks Using Fractional Natural Gradient Descent
  • Ruslan Abdulkadirov
Ruslan Abdulkadirov
North-Caucasus Center for Mathematical Research

Corresponding Author:[email protected]

Author Profile

Abstract

This paper proposes a modified natural gradient descent, containing fractional derivatives of Riemann-Liouville, Caputo and Grunwald-Letnikov types. Such approach belongs to information-geometric optimization methods, which take into account not only directions of gradients or momentum parameters, but the convexity (curvature) of minimizing function. This technique, comparing with second order optimization algorithms, lets to increase the rate of convergence. With fractional order derivatives it is possible to adjust the descent toward the neighborhood of the global minimum. In experiments, we demonstrated the increasing accuracy of image recognition on MNIST and CIFAR10, using the proposed optimization algorithm.