On Image-to-image Translation from Sentinel-1 SAR to Indices derived by Sentinel-2

Abstract

Since images from Sentinel-2 are based on optical information, some useful geospatial indices in a wide range of applications, such as Normalized Difference Vegetation Index (NDVI) and Normalized Difference Water Index (NDWI), can be extracted. Particularly, the NDVI is one of the commonly used indicators to estimate live vegetation and plays a key role in precision agriculture. Despite this practical usefulness, optical measurements suffer from its susceptibility to clouds and atmospheric conditions. To complement this disadvantage, some indices using Synthetic Aperture Radar (SAR) images to assess vegetation, such as Radar Vegetation Index (RVI), have been proposed. However, as the relationship between the NDVI and SAR can be complex and dynamic due to the changing crop growth stages, establishing robust and adaptive predictive models from SAR is still a challenging task. The framework of Generative Adversarial Networks (GANs) brought a remarkable breakthrough in image analysis, and applications of GANs to satellite imageries including super-resolution and cross-sensor transfer have been actively studied in the remote sensing field. While several conditional GAN-based models to generate optical bands from SAR data are developed and assessed in terms of accuracy of the predicted NDVI values, to our knowledge, whether and how the specific structures of the networks and data preprocessing such as feeding the existing radar-based vegetation indices to the models affects the predictive performance has not been sufficiently investigated. In this presentation, we apply a GAN-based image-to-image translation method which is called pix2pix to SAR-based vegetation assessment with several architectural complexities and data preprocessing. The networks consume VV and VH polarizations of Sentinel-1 SAR data and some derivatives including RVIs, and estimate NDVI values calculated from Sentinel-2 data in crop regions. Through the experiments, we assess the effects of the network architecture and supplemental input information.