loading page

FrequentNet: A frequency-based neural network architecture with joint temporal and frequency domains
  • +2
  • hong zhang ,
  • Jiaxu Wan ,
  • yifan yang ,
  • Ziqi He ,
  • ding yuan
hong zhang
Author Profile
Jiaxu Wan
Author Profile
yifan yang
Author Profile
ding yuan
Author Profile

Abstract

The frequency domain plays a crucial role in image processing. However, modern neural networks, such as Convolution Neural Networks and Transformers, only operate in the temporal domain, resulting in a contradiction concerning information aggregation. In contrast, the frequency domain has distinct advantages to solving the contradiction. In this paper, we introduce a frequency-based neural network architecture with joint temporal and frequency domains named as FrequentNet. We analyze the challenges in frequency-based neural networks associated with combining temporal and frequency domain information. Moreover, we find that the absence of frequency-domain downsampling methods and complex computations also affect the frequency models’ performance. To tackle the abovementioned problems, we introduce a residual connection that separates the temporal and frequency domains to resolve information aliasing. Furthermore, we devise a frequency domain down-sampling method based on the mapping. Finally, we use Discrete Cosine Transform as the frequency domain transformation operator to avoid the need for complex computations.  Comprehensive experiments demonstrate that our approach surpasses existing frequency-based backbones in diverse fields, including image classification, object detection, and semantic segmentation, whose superiority stems from the frequency domain’s robust and efficient information aggregation capability.