Please cite this work as Yanyan Wei, Zhao Zhang, Huan Zheng, Richang Hong, Yi Yang and Meng Wang, “SGINet: Toward Sufficient Interaction Between Single Image Deraining and Semantic Segmentation,” In: Proceedings of the 30th ACM International Conference on Multimedia (ACM MM), Lisbon, Portugal, June 2022.

Deep learning (DL) models have significantly improved the performance of text classification and text regression tasks. However, DL models are often strikingly vulnerable to adversarial attacks. Many researchers have aimed to develop adversarial attacks against DL models in realistic black-box settings (i.e., assumes no model knowledge is accessible to attackers) that typically operate with a two-phase framework: (1) sensitivity estimation through gradient-based or deletion-based methods to evaluate the sensitivity of each token to the prediction of the target model, and (2) perturbation execution to craft adversarial examples based on the estimated token sensitivity. However, gradient-based and deletion-based methods used to estimate sensitivity often face issues of capturing the directionality of tokens and overlapping token sensitivities, respectively. In this study, we propose a novel eXplanation-based method for Adversarial Text Attacks (XATA) that leverages additive feature attribution explainable methods, namely LIME or SHAP, to measure the sensitivity of input tokens when crafting black-box adversarial attacks on DL models performing text classification or text regression. We evaluated XATA’s attack performance on DL models executing text classification on three datasets (IMDB Movie Review, Yelp Reviews-Polarity, and Amazon Reviews-Polarity) and DL models conducting text regression on three datasets (My Personality, Drug Review, and CommonLit Readability). The proposed XATA outperformed the existing gradient-based and deletion-based adversarial attack baselines in both tasks. These findings indicate that the ever-growing research focused on improving the explainability of DL models with additive feature attribution explainable methods can provide attackers with weapons to launch targeted adversarial attacks.

Unsupervised blind motion deblurring is still a challenging topic due to the inherent ill-posed properties, and lacking of paired data and accurate quality assessment method. Besides, virtually all the current studies suffer from large chromatic aberration between the latent and original images, which will directly cause the loss of image details. However, how to model and quantify the chromatic aberration appropriately are difficult issues urgent to be solved. In this paper, we propose a general unsupervised color retention network termed CRNet for blind motion deblurring, which can be easily extended to other tasks suffering from chromatic aberration. New concepts of blur offset estimation and adaptive blur correction are introduced, so that more detailed information can be retained to improve the deblurring task. Specifically, CRNet firstly learns a mapping from the blurry image to motion offset, rather than directly from the blurry image to latent image as previous work. With obtained motion offset, an adaptive blur correction operation is then performed on the original blurry image to obtain the latent image, thereby retaining the color information of image to the greatest extent. A new pyramid global blur feature perception module is also designed to further retain the color information and extract more blur information. To assess the color retention ability for image deblurring, we present a new chromatic aberration quantization metric termed Color-Sensitive Error (CSE) in line with human perception, which can be applied to both the cases with/without paired data. Extensive experiments demonstrated the effectiveness of our CRNet for the color retention in unsupervised deblurring.

Single Image Deraining task aims at recovering the rain-free background from an image degraded by rain streaks and rain accumulation. For the powerful fitting ability of deep neural networks and massive training data, data-driven deep SID methods obtained significant improvement over traditional ones. Current SID methods usually focus on improving the deraining performance by proposing different kinds of deraining networks, while neglecting the interpretation of the solving process. As a result, the generalization ability may still be limited in real-world scenarios, and the deraining results also cannot effectively improve the performance of subsequent high-level tasks (e.g., object detection). To explore these issues, we in this paper re-examine the three important factors (i.e., data, rain model and network architecture) for the SID problem, and specifically analyze them by proposing new and more reasonable criteria (i.e., general vs. specific,synthetical vs. mathematical, black-box vs. white-box). We also study the relationship of the three factors from a new perspective of data, and reveal two different solving paradigms (explicit vs. implicit) for the SID task. We further discuss the current mainstream data-driven SID methods from five aspects, i.e., training strategy, network pipeline, domain knowledge, data preprocessing, and objective function, and some useful conclusions are summarized by statistics. Besides, we profoundly studied one of the three factors, i.e., data, and measured the performance of current methods on different datasets through extensive experiments to reveal the effectiveness of SID data. Finally, with the comprehensive review and in-depth analysis, we draw some valuable conclusions and suggestions for future research. Please cite this work as: Zhao Zhang, Yanyan Wei, Haijun Zhang, Yi Yang, Shuicheng Yan and Meng Wang, “Data-Driven Single Image Deraining: A Comprehensive Review and New Perspectives,” Pattern Recognition (PR), May 2023.  

A document by Zhao Zhang . Click on the document to view its contents.