Abstract
Advances in the design of multi-spectral cameras have
led to great interests in a wide range of applications, from
astronomy to autonomous driving. However, such cameras
inherently suffer from a trade-off between the spatial and
spectral resolution. In this paper, we propose to address
this limitation by introducing a novel method to carry out
super-resolution on raw mosaic images, multi-spectral or
RGB Bayer, captured by modern real-time single-shot mo-
saic sensors. To this end, we design a deep super-resolution
architecture that benefits from a sequential feature pyramid
along the depth of the network. This, in fact, is achieved
by utilizing a convolutional LSTM (ConvLSTM) to learn the
inter-dependencies between features at different receptive
fields. Additionally, by investigating the effect of different
attention mechanisms in our framework, we show that a
ConvLSTM inspired module is able to provide superior at-
tention in our context. Our extensive experiments and anal-
yses evidence that our approach yields significant super-
resolution quality, outperforming current state-of-the-art
mosaic super-resolution methods on both Bayer and multi-
spectral images. Additionally, to the best of our knowledge,
our method is the first specialized method to super-resolve
mosaic images, whether it be multi-spectral or Bayer.
