Cascaded Compressed Sensing Networks: A Reversible Architecture for Layerwise Learning

Recently, the method that learns networks layer by layer has attracted increasing interest for its ease of analysis. For the method, the main challenge lies in deriving an optimization target for each layer by inversely propagating the global target of the network. The target propagation is an ill-posed problem, due to involving the inversion of nonlinear activations from low-dimensional to high-dimensional spaces. To address the problem, the existing solution is to learn an auxiliary network to specially propagate the target. However, the network lacks stability, and moreover, it leads to higher complexity for network learning. In the letter, we show that target propagation could be achieved by modeling the network's each layer with compressed sensing, without the need of auxiliary networks. Experiments show that the proposed method could achieve better performance than the auxiliary network-based method.