Differentiable Image Data Augmentation and Its Applications: A Survey

Data augmentation is an effective method to improve model robustness and generalization. Conventional routines of data augmentation pipeline are used as a preprocessing modules for neural networks with predefined heuristics and restricted differentiability. Some recent works indicated that the differentiable data augmentation (DDA) could effectively contribute to the training of neural networks and the augmentation policy searching strategies. This survey provides a comprehensive and structured overview of the advances in DDA, with a focus on fundamental elements including differentiable operations, operation relaxations, and gradient estimations. We then categorize existing DDA works accordingly. Afterwards, the survey investigates the utilization of DDA in selected of practical applications, specifically neural augmentation and differentiable augmentation search. Finally, we discuss current challenges of DDA and some promising future research directions.