Generative adversarial networks for hyperspectral image spatial super-resolution

doi:10.19682/j.cnki.1005-8885.2020.0032

中国邮电高校学报(英文) ›› 2020, Vol. 27 ›› Issue (4): 8-16.doi: 10.19682/j.cnki.1005-8885.2020.0032

• Artificial Intelligence • 上一篇下一篇

Generative adversarial networks for hyperspectral image spatial super-resolution

蒋伊琳邵然唐三强

哈尔滨工程大学

收稿日期:2020-02-28 修回日期:2020-08-13 出版日期:2020-08-31 发布日期:2020-08-31
通讯作者: 邵然 E-mail:shaoran@hrbeu.edu.cn

Generative adversarial networks for hyperspectral image spatial super-resolution

Jiang Yilin, Shao Ran, Tang Sanqiang

Received:2020-02-28 Revised:2020-08-13 Online:2020-08-31 Published:2020-08-31
Contact: Shao Ran E-mail:shaoran@hrbeu.edu.cn

摘要/Abstract

摘要：

It is becoming increasingly easier to obtain more abundant supplies for hyperspectral images ( HSIs). Despite this, achieving high resolution is still critical. In this paper, a method named hyperspectral images super-resolution generative adversarial network ( HSI-RGAN ) is proposed to enhance the spatial resolution of HSI without decreasing its spectral resolution. Different from existing methods with the same purpose, which are based on convolutional neural networks ( CNNs) and driven by a pixel-level loss function, the new generative adversarial network (GAN) has a redesigned framework and a targeted loss function. Specifically, the discriminator uses the structure of the relativistic discriminator, which provides feedback on how much the generated HSI looks like the ground truth. The generator achieves more authentic details and textures by removing the place of the pooling layer and the batch normalization layer and presenting smaller filter size and two-step upsampling layers. Furthermore, the loss function is improved to specially take spectral distinctions into account to avoid artifacts and minimize potential spectral distortion, which may be introduced by neural networks. Furthermore, pre-training with the visual geometry group (VGG) network helps the entire model to initialize more easily. Benefiting from these changes, the proposed method obtains significant advantages compared to the original GAN. Experimental results also reveal that the proposed method performs better than several state-of-the-art methods.

关键词:

HSI-SRGAN, hyperspectral image, super-resolution

Abstract: It is becoming increasingly easier to obtain more abundant supplies for hyperspectral images ( HSIs). Despite this, achieving high resolution is still critical. In this paper, a method named hyperspectral images super-resolution generative adversarial network ( HSI-RGAN ) is proposed to enhance the spatial resolution of HSI without decreasing its spectral resolution. Different from existing methods with the same purpose, which are based on convolutional neural networks ( CNNs) and driven by a pixel-level loss function, the new generative adversarial network (GAN) has a redesigned framework and a targeted loss function. Specifically, the discriminator uses the structure of the relativistic discriminator, which provides feedback on how much the generated HSI looks like the ground truth. The generator achieves more authentic details and textures by removing the place of the pooling layer and the batch normalization layer and presenting smaller filter size and two-step upsampling layers. Furthermore, the loss function is improved to specially take spectral distinctions into account to avoid artifacts and minimize potential spectral distortion, which may be introduced by neural networks. Furthermore, pre-training with the visual geometry group (VGG) network helps the entire model to initialize more easily. Benefiting from these changes, the proposed method obtains significant advantages compared to the original GAN. Experimental results also reveal that the proposed method performs better than several state-of-the-art methods.

Key words: HSI-SRGAN, hyperspectral image, super-resolution

中图分类号:

TP 751

Jiang Yilin, Shao Ran, Tang Sanqiang. Generative adversarial networks for hyperspectral image spatial super-resolution[J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(4): 8-16.

参考文献

[1] Wang L, Zhao C. Hyperspectral image processing. New York, NY, USA: Wiley, 2015.

[2] Bioucas-Dias J M, Plaza A, Camps-Valls G, et al. Hyperspectral remote sensing data analysis and future challenges. IEEE Geoscience and Remote Sensing Magazine, 2013, 1(2): 6-36.

[3] Tsai R Y, Huang T S. Multi-frame image restoration and registration. In: Huang T S (ed). Advances in Computer Vision and Image Processing. Greenwich, CT, USA: JAI Press, 1984: 317-339.

[4] Yuen P W, Hunt B R. Constraining information for improvements to MAP image super-resolution. Image Reconstruction and Restoration: Proceedings of the International Symposium on Optics, Imaging, and Instrumentation (SPIE'94), 1994, Sept 30, San Diego, CA, USA. SPIE 2302. Bellingham, WA, USA: Society of Photo-Optical Instrumentation Engineers (SPIE), 1994: 144-155.

[5] Ozkan M K, Tekalp A M, Sezan M I. POCS-based restoration of space-varying blurred images. IEEE Transactions on Image Processing, 1994, 3(4): 450-454.

[6] Yuan H, Yan F X, Chen X M, et al. Compressive hyperspectral imaging and super-resolution. Proceedings of the IEEE 3rd International Conference on Image, Vision and Computing (ICIVC'18), 2018, Jun 27-29, Chongqing, China. Piscataway, NJ, USA: IEEE, 2018: 618-623.

[7] Zhao C H, Liu W. Compressive sensing theory and its application in imaging technology. CAAI Transactions on Intelligent Systems, 2012, 7(1): 25-36.

[8] Yang J C, Wright J, Huang T S, et al. Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873.

[9] Dong C, Loy C C, He K M, et al. Learning a deep convolutional network for image super-resolution. Computer Vision: Proceedings of the 13th European Conference on Computer Vision (ECCV'14): Part Ⅳ, 2014, Sept 6-12, Zurich, Switzerland. LNCS 8692. Berlin, Germany: Springer, 2014: 184-199.

[10] Hu J, Li Y S, Xie W Y. Hyperspectral image super-resolution by spectral difference learning and spatial error correction. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10): 1825-1829.

[11] Li Y S, Hu J, Zhao X, et al. Hyperspectral image super-resolution using deep convolutional neural network. Neurocomputing, 2017, 266: 29-41.

[12] Jia J R, Ji L Y, Zhao Y C, et al. Hyperspectral image super-resolution with spectral–spatial network. International Journal of Remote Sensing, 2018, 39(22): 7806-7829.

[13] Wang C, Liu Y, Bai X, et al. Deep residual convolutional neural network for hyperspectral image super-resolution. Image and Graphics: Proceedings of the 9th International Conference on Image and Graphics (ICIG'17): Part Ⅲ, 2017, Sept 13-15, Shanghai, China. LNCS 10668. Berlin, Germany: Springer, 2017: 370-380.

[14] Ledig C, Theis L, Huszar F, et al. Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'17), 2017 Jul 21-26, Honolulu, HI, USA. Piscataway, NJ, USA: IEEE, 2017: 105-114.

[15] Wang X T, Yu K, Wu S X, et al. ESRGAN: Enhanced super-resolution generative adversarial networks. Computer Vision: Proceedings of the 15th European Conference on Computer Vision (ECCV'18), 2018, Sept 8-4, Munich, Germany. LNCS 11214. Berlin, Germany: Springer, 2018: 63-79.

[16] Jolicoeur-Martineau A. The relativistic discriminator: A key element missing from standard GAN. Proceedings of the 7th International Conference on Learning Representations (ICLR'19), 2019, May 6-9, New Orleans, LA, USA. 2019.

[17] Song D H, Xu C, Jia X, et al. Efficient residual dense block search for image super-resolution. Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI'20), 2020, Feb 7-12, New York, NY, USA. Palo Alto, CA, USA: AAAI Press, 2020.

[18] Single hyperspectral image super-resolution with grouped deep recursive residual network. Liyong8490/HSI-SR-GDRRN. Lansing, MI, USA: Jackson National Life Insurance Company, 2019.

[19] Zhang Y L, Li K P, Li K, et al. Image super-resolution using very deep residual channel attention networks. Computer Vision: Proceedings of the 15th European Conference on Computer Vision (ECCV'18), 2018, Sept 8-4, Munich, Germany. LNCS 11214. Berlin, Germany: Springer, 2018: 294-310.

[20] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations (ICLR'15), 2015, May 7-9, San Diego, CA, USA. 2015.

[21] Arad B, Ben-Shahar O. Sparse recovery of hyperspectral signal from natural RGB images. Computer Vision: Proceedings of the 14th European Conference on Computer Vision (ECCV'16), 2016, Oct 8-16, Amsterdam, Netherlands. LNCS 9911. Berlin, Germany: Springer, 2016: 19-34.

[22] Kruse F A, Lefkoff A B, Boardman J W, et al. The spectral image processing system (SIPS)--Interactive visualization and analysis of imaging spectrometer data. Remote Sensing of Environment, 1993, 44(2/3): 145–163.

[23] Yasuma F, Mitsunaga T, Iso D, et al. Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum. IEEE Transactions on Image Processing, 2010, 19(9): 2241–2253.

Generative adversarial networks for hyperspectral image spatial super-resolution

Generative adversarial networks for hyperspectral image spatial super-resolution

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价