Back in time: digital restoration techniques for the millennium Dunhuang murals

doi:10.19682/j.cnki.1005-8885.2022.0012

The Journal of China Universities of Posts and Telecommunications ›› 2022, Vol. 29 ›› Issue (2): 13-23.doi: 10.19682/j.cnki.1005-8885.2022.0012

Special Issue: 文化计算专题

Previous Articles Next Articles

Back in time: digital restoration techniques for the millennium Dunhuang murals

Received:2022-01-29 Revised:2022-03-15 Online:2022-04-26 Published:2022-04-26
Contact: Chuang ZHANG E-mail:zhangchuang@bupt.edu.cn
Supported by:
This work was supported by the Ministry of Education-China Mobile Communications (MCM20190701)

Abstract

Abstract:

In the long history of more than 1 500 years, Dunhuang murals suffered from various deteriorations causing irreversible damage such as falling off, fading, and so on. However, the existing Dunhuang mural restoration methods are time-consuming and not feasible to facilitate cultural issemination and permanent inheritance. Inspired by cultural computing using artificial intelligence, gated-convolution-based dehaze net (GD-Net) was proposed for Dunhuang mural refurbishment and comprehensive protection. First, a neural network with gated convolution was applied to restore the falling off areas of the mural to ensure the integrity of the mural content. Second, a dehaze network was applied to enhance image quality to cope with the fading of the mural. Besides, a Dunhuang mural dataset was presented to meet the needs of deep learning approach, containing 1 180 images from the Cave 290 and Cave 112 of the Mogao Grottoes. The experimental results demonstrate the effectiveness and superiority of GD-Net.

Key words: image inpainting|image enhancement|mural restoration

References

[1] YU T X, ZHANG S J, LIN C, et al. Dunhuang Grottoes painting dataset and benchmark. arXiv Preprint, arXiv: 1907. 04589, 2019.

[2] TELEA A. An image inpainting technique based on the fast arching method. Journal of Graphics Tools, 2004, 9(1): 25 -36.

[3] CRIMINISI A, PÉREZ P, TOYAMA K. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing, 2004, 13(9): 1200 -1212.

[4] BARNES C, SHECHTMAN E, FINKELSTEIN A, et al. PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics, 2009, 28 (3): Article 24.

[5] ZHAO L, MO Q H, LIN S H, et al. UCTGAN: diverse image inpainting based on unsupervised cross-space translation.

Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020, Jun 13 -19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 5741 - 5750.

[6] LIU H Y, WAN Z Y, HUANG W, et al. PD-GAN: probabilistic diverse GAN for image inpainting. Proceedings of the 2021 IEEE/ CVF Conference on Computer Vision and Pattern Recognition ( CVPR’21 ), 2021, Jun 20 -25, Nashville, TN, USA. Piscataway, NJ, USA: IEEE, 2021: 9371 -9381.

[7] NAZERI K, NG E, JOSEPH T, et al. EdgeConnect: generative image inpainting with adversarial edge learning. arXiv Preprint, arXiv: 1901. 00212, 2019.

[8] YU J H, LIN Z, YANG J M, et al. Free-form image inpainting with gated convolution. Proceedings of the 2019 IEEE/ CVF International Conference on Computer Vision (ICCV’19), 2019, Oct 27 - Nov 2, Seoul, Republic of Korea. Piscataway, NJ, USA: IEEE, 2019: 4471 -4480.

[9] XIAO B, XU Y Q, TANG H, et al. Histogram learning in image contrast enhancement. Proceedings of the 2019 IEEE/ CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’19), 2019, Jun 16 -17, Long Beach, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 1880 -1889.

[10] LIU R S, MA L, ZHANG J A, et al. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. Proceedings of the 2021 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’21), 2021, Jun 20 - 25, Nashville, TN, USA. Piscataway, NJ, USA: IEEE, 2021: 10561 -10570.

[11] ZHU R, WANG L J. Improved wavelet transform algorithm for single image dehazing. Optik, 2014, 125(13): 3064 -3066.

[12] XU K, YANG X, YIN B C, et al. Learning to restore low-light images via decomposition-and-enhancement. Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition ( CVPR’20), 2020, Jun 13 - 19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 2281 -2290.

[13] ATOUM Y, YE M, REN L, et al. Color-wise attention network for low-light image enhancement. Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition Workshops ( CVPRW’20), 2020, Jun 14 - 19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 506 -507.

[14] YU T X, LIN C, ZHANG S J, et al. End-to-end partial convolutions neural networks for Dunhuang Grottoes wall-painting restoration. Proceedings of the 2019 IEEE/ CVF International Conference on Computer Vision Workshop (ICCVW’19), 2019, Oct 27 - 28,Seoul, Republic of Korea. Piscataway, NJ, USA: IEEE, 2019: 1447 -1455.

[15] WANG H, LI Q Q, ZOU Q. Inpainting of dunhuang murals by sparsely modeling the texture similarity and structure continuity. Journal on Computing and Cultural Heritage, 2019, 12 (3 ): Article 17.

[16] PATHAK D, KRÄHENBÜHL P, DONAHUE J, et al. Context encoders: feature learning by inpainting. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16 ), 2016, Jun 27 - 30, Las Vegas, NV, USA. Piscataway, NJ, USA: IEEE, 2016: 2536 -2544.

[17] LIU G L, REDA F A, SHIH K J, et al. Image inpainting for irregular holes using partial convolutions. Proceedings of the 15th European Conference on Computer Vision ( ECCV’18), 2018, Sept 8 - 14, Munich, Germany. Berlin, Germany: Springer, 2018: 85 -105.

[18] YI Z L, TANG Q, AZIZI S, et al. Contextual residual aggregation for ultra high-resolution image inpainting. Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition ( CVPR’20), 2020, Jun 13 - 19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 7508 -7517.

[19] WANG T F, OUYANG H, CHEN Q F. Image inpainting with external-internal learning and monochromic bottleneck.

Proceedings of the 2021 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’21), 2021, Jun 20 - 25, Nashville, TN, USA. Piscataway, NJ, USA: IEEE, 2021: 5120 -5129.

[20] MOU C, ZHANG J, WU Z Y. Dynamic attentive graph learning for image restoration. Proceedings of the 2021 IEEE/ CVF International Conference on Computer Vision (ICCV’21), 2021, Oct 10 - 17, Montreal, Canada. Piscataway, NJ, USA: IEEE, 2021: 4328 -4337.

[21] LI J Y, WANG N, ZHANG L F, et al. Recurrent feature reasoning for image inpainting. Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition ( CVPR’20 ), 2020, Jun 13 - 19, Seattle, WA, USA.

Piscataway, NJ, USA: IEEE, 2020: 7757 -7765.

[22] LAND E H, MCCANN J J. Lightness and Retinex theory. Journal of the Optical Society of America, 1971, 61(1): 1 -11.

[23] CAI W T, LIU Y X, LI M C, et al. A self-adaptive homomorphic filter method for removing thin cloud. Proceedings of the 19th International Conference on Geoinformatics, 2011, Jun 24 - 26, Shanghai, China. Piscataway, NJ, USA: IEEE, 2011: 1 -4.

[24] STARCK J L, MURTAGH F, CANDÉS E J, et al. Gray and color image contrast enhancement by the curvelet transform. IEEE Transactions on Image Processing, 2003, 12(6): 706 -717.

[25] BYCHKOVSKY V, PARIS S, CHAN E, et al. Learning photographic global tonal adjustment with a database of input/

output image pairs. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11), 2011, Jun 20 - 25, Colorado Springs, CO, USA. Piscataway, NJ, USA: IEEE, 2011: 97 -104.

[26] YANG W H, WANG S Q, FANG Y M, et al. From fidelity to perceptual quality: a semi-supervised approach for low-light image enhancement. Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020, Jun 13 - 19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 3063 -3072.

[27] LIANG L M, ZHARKOV I, AMJADI F, et al. Guidance network with staged learning for image enhancement. Proceedings of the 2021 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’21), 2021, Jun 20 - 25, Nashville, TN, USA. Piscataway, NJ, USA: IEEE, 2021: 836 -845.

[28] SCHECHNER Y Y, NARASIMHAN S G, NAYAR S K. Polarization-based vision through haze. Applied Optics, 2003,

42(3): 511 -525.

[29] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE

Transactions on Image Processing, 2004, 13(4): 600 -612.

[30] PARK J, LEE J Y, YOO D, et al. Distort-and-recover: color enhancement using deep reinforcement learning. Proceedings of the 2018 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’18), 2018, Jun 18 -23, Salt Lake City, UT, USA. Piscataway, NJ, USA: IEEE, 2018: 5928 -5936.

Metrics

Comments

Copyright © 2020 The Journal of China Universities of Posts and Telecommunications
　 Adress: P.O. Box 231,Beijing University of Posts and Telecommunications,10 Xi Tucheng Road,Beijing 100876,P.R.China　Post Code: 100081
Tel：86-010-62282493　Fax： 86-010-62283461　E-mail: jchupt@bupt.edu.cn
Support by: Beijing Magtech Co.Ltd

Back in time: digital restoration techniques for the millennium Dunhuang murals

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 4

Recommended Articles

Metrics

Comments

[1]	. Multi-scale fusion residual encoder-decoder approach for low illumination image enhancement [J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(2): 63-72.
[2]	DUAN Lian. Low-light image enhancement algorithm using a residual network with semantic information [J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(2): 52-62.
[3]	. Infrared image enhancement algorithm based on adaptive weighted guided filter [J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(2): 73-84.
[4]	Wu Xiaochu, Tang Guijin, Liu Xiaohua, Cui Ziguan, Luo Suhuai. Low-light color image enhancement based on NSST [J]. The Journal of China Universities of Posts and Telecommunications, 2019, 26(5): 41-48.