A non-convex optimization framework for large-scale low-rank matrix
factorization
Abstract
Low-rank matrix factorization problems such as non negative matrix
factorization (NMF) can be categorized as a clustering or dimension
reduction technique. The latter denotes techniques designed to find
representations of some high dimensional dataset in a lower dimensional
manifold without a significant loss of information. If such a
representation exists, the features ought to contain the most relevant
features of the dataset. Many linear dimensionality reduction techniques
can be formulated as a matrix factorization. In this paper, we combine
the conjugate gradient (CG) method with the Barzilai and Borwein (BB)
gradient method, and propose a BB scaling CG method for NMF problems.
The new method does not require to compute and store matrices associated
with Hessian of the objective functions. Moreover, adopting a suitable
BB step size along with a proper nonmonotone strategy which comes by the
size convex parameter $\eta_k$, results in a new
algorithm that can significantly improve the CPU time, efficiency, the
number of function evaluation. Convergence result is established and
numerical comparisons of methods on both synthetic and real-world
datasets show that the proposed method is efficient in comparison with
existing methods and demonstrate the superiority of our algorithms.