Generative adversarial networks for hyperspectral image spatial super-resolution

doi:10.19682/j.cnki.1005-8885.2020.0032

The Journal of China Universities of Posts and Telecommunications ›› 2020, Vol. 27 ›› Issue (4): 8-16.doi: 10.19682/j.cnki.1005-8885.2020.0032

• Artificial intelligence • Previous Articles Next Articles

Generative adversarial networks for hyperspectral image spatial super-resolution

Jiang Yilin, Shao Ran, Tang Sanqiang

Received:2020-02-28 Revised:2020-08-13 Online:2020-08-31 Published:2020-08-31
Contact: Shao Ran E-mail:shaoran@hrbeu.edu.cn

Abstract

Abstract: It is becoming increasingly easier to obtain more abundant supplies for hyperspectral images ( HSIs). Despite this, achieving high resolution is still critical. In this paper, a method named hyperspectral images super-resolution generative adversarial network ( HSI-RGAN ) is proposed to enhance the spatial resolution of HSI without decreasing its spectral resolution. Different from existing methods with the same purpose, which are based on convolutional neural networks ( CNNs) and driven by a pixel-level loss function, the new generative adversarial network (GAN) has a redesigned framework and a targeted loss function. Specifically, the discriminator uses the structure of the relativistic discriminator, which provides feedback on how much the generated HSI looks like the ground truth. The generator achieves more authentic details and textures by removing the place of the pooling layer and the batch normalization layer and presenting smaller filter size and two-step upsampling layers. Furthermore, the loss function is improved to specially take spectral distinctions into account to avoid artifacts and minimize potential spectral distortion, which may be introduced by neural networks. Furthermore, pre-training with the visual geometry group (VGG) network helps the entire model to initialize more easily. Benefiting from these changes, the proposed method obtains significant advantages compared to the original GAN. Experimental results also reveal that the proposed method performs better than several state-of-the-art methods.

Key words: HSI-SRGAN, hyperspectral image, super-resolution

CLC Number:

TP 751

Jiang Yilin, Shao Ran, Tang Sanqiang. Generative adversarial networks for hyperspectral image spatial super-resolution[J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(4): 8-16.

References

[1] Wang L, Zhao C. Hyperspectral image processing. New York, NY, USA: Wiley, 2015.

[2] Bioucas-Dias J M, Plaza A, Camps-Valls G, et al. Hyperspectral remote sensing data analysis and future challenges. IEEE Geoscience and Remote Sensing Magazine, 2013, 1(2): 6-36.

[3] Tsai R Y, Huang T S. Multi-frame image restoration and registration. In: Huang T S (ed). Advances in Computer Vision and Image Processing. Greenwich, CT, USA: JAI Press, 1984: 317-339.

[4] Yuen P W, Hunt B R. Constraining information for improvements to MAP image super-resolution. Image Reconstruction and Restoration: Proceedings of the International Symposium on Optics, Imaging, and Instrumentation (SPIE'94), 1994, Sept 30, San Diego, CA, USA. SPIE 2302. Bellingham, WA, USA: Society of Photo-Optical Instrumentation Engineers (SPIE), 1994: 144-155.

[5] Ozkan M K, Tekalp A M, Sezan M I. POCS-based restoration of space-varying blurred images. IEEE Transactions on Image Processing, 1994, 3(4): 450-454.

[6] Yuan H, Yan F X, Chen X M, et al. Compressive hyperspectral imaging and super-resolution. Proceedings of the IEEE 3rd International Conference on Image, Vision and Computing (ICIVC'18), 2018, Jun 27-29, Chongqing, China. Piscataway, NJ, USA: IEEE, 2018: 618-623.

[7] Zhao C H, Liu W. Compressive sensing theory and its application in imaging technology. CAAI Transactions on Intelligent Systems, 2012, 7(1): 25-36.

[8] Yang J C, Wright J, Huang T S, et al. Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873.

[9] Dong C, Loy C C, He K M, et al. Learning a deep convolutional network for image super-resolution. Computer Vision: Proceedings of the 13th European Conference on Computer Vision (ECCV'14): Part Ⅳ, 2014, Sept 6-12, Zurich, Switzerland. LNCS 8692. Berlin, Germany: Springer, 2014: 184-199.

[10] Hu J, Li Y S, Xie W Y. Hyperspectral image super-resolution by spectral difference learning and spatial error correction. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10): 1825-1829.

[11] Li Y S, Hu J, Zhao X, et al. Hyperspectral image super-resolution using deep convolutional neural network. Neurocomputing, 2017, 266: 29-41.

[12] Jia J R, Ji L Y, Zhao Y C, et al. Hyperspectral image super-resolution with spectral–spatial network. International Journal of Remote Sensing, 2018, 39(22): 7806-7829.

[13] Wang C, Liu Y, Bai X, et al. Deep residual convolutional neural network for hyperspectral image super-resolution. Image and Graphics: Proceedings of the 9th International Conference on Image and Graphics (ICIG'17): Part Ⅲ, 2017, Sept 13-15, Shanghai, China. LNCS 10668. Berlin, Germany: Springer, 2017: 370-380.

[14] Ledig C, Theis L, Huszar F, et al. Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'17), 2017 Jul 21-26, Honolulu, HI, USA. Piscataway, NJ, USA: IEEE, 2017: 105-114.

[15] Wang X T, Yu K, Wu S X, et al. ESRGAN: Enhanced super-resolution generative adversarial networks. Computer Vision: Proceedings of the 15th European Conference on Computer Vision (ECCV'18), 2018, Sept 8-4, Munich, Germany. LNCS 11214. Berlin, Germany: Springer, 2018: 63-79.

[16] Jolicoeur-Martineau A. The relativistic discriminator: A key element missing from standard GAN. Proceedings of the 7th International Conference on Learning Representations (ICLR'19), 2019, May 6-9, New Orleans, LA, USA. 2019.

[17] Song D H, Xu C, Jia X, et al. Efficient residual dense block search for image super-resolution. Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI'20), 2020, Feb 7-12, New York, NY, USA. Palo Alto, CA, USA: AAAI Press, 2020.

[18] Single hyperspectral image super-resolution with grouped deep recursive residual network. Liyong8490/HSI-SR-GDRRN. Lansing, MI, USA: Jackson National Life Insurance Company, 2019.

[19] Zhang Y L, Li K P, Li K, et al. Image super-resolution using very deep residual channel attention networks. Computer Vision: Proceedings of the 15th European Conference on Computer Vision (ECCV'18), 2018, Sept 8-4, Munich, Germany. LNCS 11214. Berlin, Germany: Springer, 2018: 294-310.

[20] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations (ICLR'15), 2015, May 7-9, San Diego, CA, USA. 2015.

[21] Arad B, Ben-Shahar O. Sparse recovery of hyperspectral signal from natural RGB images. Computer Vision: Proceedings of the 14th European Conference on Computer Vision (ECCV'16), 2016, Oct 8-16, Amsterdam, Netherlands. LNCS 9911. Berlin, Germany: Springer, 2016: 19-34.

[22] Kruse F A, Lefkoff A B, Boardman J W, et al. The spectral image processing system (SIPS)--Interactive visualization and analysis of imaging spectrometer data. Remote Sensing of Environment, 1993, 44(2/3): 145–163.

[23] Yasuma F, Mitsunaga T, Iso D, et al. Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum. IEEE Transactions on Image Processing, 2010, 19(9): 2241–2253.

Metrics

Comments

Copyright © 2020 The Journal of China Universities of Posts and Telecommunications
　 Adress: P.O. Box 231,Beijing University of Posts and Telecommunications,10 Xi Tucheng Road,Beijing 100876,P.R.China　Post Code: 100081
Tel：86-010-62282493　Fax： 86-010-62283461　E-mail: jchupt@bupt.edu.cn
Support by: Beijing Magtech Co.Ltd

Generative adversarial networks for hyperspectral image spatial super-resolution

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 0

Recommended Articles

Metrics

Comments