loading page

Image Prediction Using Coordinated Hyperspectral and RGB Video of Dynamic Natural Water Scenes
  • +3
  • Charles Bachmann ,
  • Nayma Binte Nur ,
  • Kimberly Union ,
  • Christopher Lapszynski ,
  • Dylan Shiltz
Rochester Institute of Technology

Corresponding Author:[email protected]

Author Profile
Charles Bachmann
Author Profile
Nayma Binte Nur
Author Profile
Kimberly Union
Author Profile
Christopher Lapszynski
Author Profile
Dylan Shiltz
Author Profile


A bimodal video imaging platform combining RGB and 371-band hyperspectral imaging systems was used to collect time-series data of the Lake Ontario shoreline at Hamlin Beach State Park in Rochester, New York, USA. We predicted the hyperspectral image frames of dynamic natural water scenes at previous and later points in time using a paired relationship between the time-series hyperspectral imagery and RGB video. The time-series hyperspectral image data was collected using our Headwall Hyperspec micro-HE line-scanning imaging spectrometer integrated into a General Dynamics pan-tilt unit. RGB video data was collected with a low-cost consumer GoPro Hero 8 Black. We detail our data collection methods and characterize the predictions using distributions of absolute and normalized residuals in reflectance spaces. Within visible wavelengths, 95% of the scene is predicted to within 2% absolute reflectance. The normalized error percentage of these residuals translates to approximately 30% of signal level for water spectra. In the near-infrared regime, the normalized error percentage of the residuals sharply increases to approximately 90% for 95% of the scene due to lack of band information from the RGB video imagery of our shallow water scene.