TechRxiv
SPL_20211019.pdf (227.13 kB)
Download file

Cascaded Compressed Sensing Networks: A Reversible Architecture for Layerwise Learning

Download (227.13 kB)
preprint
posted on 25.10.2021, 21:23 authored by weizhi luweizhi lu, Mingrui Chen, kai guo, Weiyu Li
Recently, the method that learns networks layer by layer has attracted increasing interest for its ease of analysis. For the method, the main challenge lies in deriving an optimization target for each layer by inversely propagating the global target of the network. The target propagation is an ill-posed problem, due to involving the inversion of nonlinear activations from low-dimensional to high-dimensional spaces. To address the problem, the existing solution is to learn an auxiliary network to specially propagate the target. However, the network lacks stability, and moreover, it leads to higher complexity for network learning. In the letter, we show that target propagation could be achieved by modeling the network's each layer with compressed sensing, without the need of auxiliary networks. Experiments show that the proposed method could achieve better performance than the auxiliary network-based method.

History

Email Address of Submitting Author

wzlu@sdu.edu.cn

Submitting Author's Institution

Shandong University

Submitting Author's Country

China

Usage metrics

Licence

Exports