TechRxiv
MRM-Net Towards Deep Multi-Resolution Mutual Learning_IEEE.pdf (7.69 MB)
Download file

MRM-Net: Towards Deep Multi-Resolution Mutual Learning for Progressive Image Inpainting

Download (7.69 MB)
preprint
posted on 21.04.2022, 14:17 by Huan ZhangHuan Zhang, Zhao ZhangZhao Zhang, Haijun Zhang, Yi YangYi Yang, Shuicheng Yan, Meng Wang

Deep image inpainting methods have improved the inpainting performance greatly due to the powerful representation ability of deep learning. However, current deep inpainting models still tend to produce unreasonable structures and blurry textures due to the ill-posed properties of the task, i.e., image inpainting is still a challenging topic. In this paper, we therefore propose a novel deep multi-resolution mutual learning (DMRML) strategy for progressive inpainting, which can fully explore the information from various resolutions. Specifically, we design a new image inpainting network, termed multi-resolution mutual network (MRM-Net), which takes the damaged images of various resolutions as input, then excavates and exploits the correlation among different resolutions to guide the image inpainting process. Technically, the setting of MRM-Net designs two new modules called multi-resolution information interaction (MRII) and adaptive content enhancement (ACE). MRII aims at discovering the correlation of multiple resolutions and exchanging information, and ACE focuses on enhancing the contents using the interacted features. We also present an memory preservation mechanism (MPM) to prevent from the information loss with the increasing layers. Extensive experiments on Paris Street View, Places2 and CelebA-HQ datasets demonstrate that our proposed MRM-Net can effectively recover the textures and structures, and performs favorably against other state-of-the-art methods. 

Funding

This work is partially supported by National Natural Science Foundation of China (62072151, 61732007, 61932009 and 62020106007), and Anhui Provincial Natural Science Fund for Distinguished Young Scholars (2008085J30).

History

Email Address of Submitting Author

cszzhang@gmail.com

Submitting Author's Institution

Hefei University of Technology

Submitting Author's Country

China