Back in time: digital restoration techniques for the millennium Dunhuang murals

doi:10.19682/j.cnki.1005-8885.2022.0012

中国邮电高校学报(英文) ›› 2022, Vol. 29 ›› Issue (2): 13-23.doi: 10.19682/j.cnki.1005-8885.2022.0012

所属专题：文化计算专题

• Special Topic: Cultural Computing • 上一篇下一篇

Back in time: digital restoration techniques for the millennium Dunhuang murals

陶艺峰,宋宇程,徐梦秋,张闯,吴铭,白苏乐

北京邮电大学

收稿日期:2022-01-29 修回日期:2022-03-15 出版日期:2022-04-26 发布日期:2022-04-26
通讯作者: 张闯 E-mail:zhangchuang@bupt.edu.cn
基金资助:
This work was supported by the Ministry of Education-China Mobile Communications (MCM20190701)

Back in time: digital restoration techniques for the millennium Dunhuang murals

Received:2022-01-29 Revised:2022-03-15 Online:2022-04-26 Published:2022-04-26
Contact: Chuang ZHANG E-mail:zhangchuang@bupt.edu.cn
Supported by:
This work was supported by the Ministry of Education-China Mobile Communications (MCM20190701)

摘要/Abstract

摘要：

In the long history of more than 1 500 years, Dunhuang murals suffered from various deteriorations causing irreversible damage such as falling off, fading, and so on. However, the existing Dunhuang mural restoration methods are time-consuming and not feasible to facilitate cultural issemination and permanent inheritance. Inspired by cultural computing using artificial intelligence, gated-convolution-based dehaze net (GD-Net) was proposed for Dunhuang mural refurbishment and comprehensive protection. First, a neural network with gated convolution was applied to restore the falling off areas of the mural to ensure the integrity of the mural content. Second, a dehaze network was applied to enhance image quality to cope with the fading of the mural. Besides, a Dunhuang mural dataset was presented to meet the needs of deep learning approach, containing 1 180 images from the Cave 290 and Cave 112 of the Mogao Grottoes. The experimental results demonstrate the effectiveness and superiority of GD-Net.

关键词: image inpainting, image enhancement, mural restoration

Abstract:

Key words: image inpainting|image enhancement|mural restoration

参考文献

[1] YU T X, ZHANG S J, LIN C, et al. Dunhuang Grottoes painting dataset and benchmark. arXiv Preprint, arXiv: 1907. 04589, 2019.

[2] TELEA A. An image inpainting technique based on the fast arching method. Journal of Graphics Tools, 2004, 9(1): 25 -36.

[3] CRIMINISI A, PÉREZ P, TOYAMA K. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing, 2004, 13(9): 1200 -1212.

[4] BARNES C, SHECHTMAN E, FINKELSTEIN A, et al. PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics, 2009, 28 (3): Article 24.

[5] ZHAO L, MO Q H, LIN S H, et al. UCTGAN: diverse image inpainting based on unsupervised cross-space translation.

Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020, Jun 13 -19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 5741 - 5750.

[6] LIU H Y, WAN Z Y, HUANG W, et al. PD-GAN: probabilistic diverse GAN for image inpainting. Proceedings of the 2021 IEEE/ CVF Conference on Computer Vision and Pattern Recognition ( CVPR’21 ), 2021, Jun 20 -25, Nashville, TN, USA. Piscataway, NJ, USA: IEEE, 2021: 9371 -9381.

[7] NAZERI K, NG E, JOSEPH T, et al. EdgeConnect: generative image inpainting with adversarial edge learning. arXiv Preprint, arXiv: 1901. 00212, 2019.

[8] YU J H, LIN Z, YANG J M, et al. Free-form image inpainting with gated convolution. Proceedings of the 2019 IEEE/ CVF International Conference on Computer Vision (ICCV’19), 2019, Oct 27 - Nov 2, Seoul, Republic of Korea. Piscataway, NJ, USA: IEEE, 2019: 4471 -4480.

[9] XIAO B, XU Y Q, TANG H, et al. Histogram learning in image contrast enhancement. Proceedings of the 2019 IEEE/ CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’19), 2019, Jun 16 -17, Long Beach, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 1880 -1889.

[10] LIU R S, MA L, ZHANG J A, et al. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. Proceedings of the 2021 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’21), 2021, Jun 20 - 25, Nashville, TN, USA. Piscataway, NJ, USA: IEEE, 2021: 10561 -10570.

[11] ZHU R, WANG L J. Improved wavelet transform algorithm for single image dehazing. Optik, 2014, 125(13): 3064 -3066.

[12] XU K, YANG X, YIN B C, et al. Learning to restore low-light images via decomposition-and-enhancement. Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition ( CVPR’20), 2020, Jun 13 - 19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 2281 -2290.

[13] ATOUM Y, YE M, REN L, et al. Color-wise attention network for low-light image enhancement. Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition Workshops ( CVPRW’20), 2020, Jun 14 - 19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 506 -507.

[14] YU T X, LIN C, ZHANG S J, et al. End-to-end partial convolutions neural networks for Dunhuang Grottoes wall-painting restoration. Proceedings of the 2019 IEEE/ CVF International Conference on Computer Vision Workshop (ICCVW’19), 2019, Oct 27 - 28,Seoul, Republic of Korea. Piscataway, NJ, USA: IEEE, 2019: 1447 -1455.

[15] WANG H, LI Q Q, ZOU Q. Inpainting of dunhuang murals by sparsely modeling the texture similarity and structure continuity. Journal on Computing and Cultural Heritage, 2019, 12 (3 ): Article 17.

[16] PATHAK D, KRÄHENBÜHL P, DONAHUE J, et al. Context encoders: feature learning by inpainting. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16 ), 2016, Jun 27 - 30, Las Vegas, NV, USA. Piscataway, NJ, USA: IEEE, 2016: 2536 -2544.

[17] LIU G L, REDA F A, SHIH K J, et al. Image inpainting for irregular holes using partial convolutions. Proceedings of the 15th European Conference on Computer Vision ( ECCV’18), 2018, Sept 8 - 14, Munich, Germany. Berlin, Germany: Springer, 2018: 85 -105.

[18] YI Z L, TANG Q, AZIZI S, et al. Contextual residual aggregation for ultra high-resolution image inpainting. Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition ( CVPR’20), 2020, Jun 13 - 19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 7508 -7517.

[19] WANG T F, OUYANG H, CHEN Q F. Image inpainting with external-internal learning and monochromic bottleneck.

Proceedings of the 2021 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’21), 2021, Jun 20 - 25, Nashville, TN, USA. Piscataway, NJ, USA: IEEE, 2021: 5120 -5129.

[20] MOU C, ZHANG J, WU Z Y. Dynamic attentive graph learning for image restoration. Proceedings of the 2021 IEEE/ CVF International Conference on Computer Vision (ICCV’21), 2021, Oct 10 - 17, Montreal, Canada. Piscataway, NJ, USA: IEEE, 2021: 4328 -4337.

[21] LI J Y, WANG N, ZHANG L F, et al. Recurrent feature reasoning for image inpainting. Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition ( CVPR’20 ), 2020, Jun 13 - 19, Seattle, WA, USA.

Piscataway, NJ, USA: IEEE, 2020: 7757 -7765.

[22] LAND E H, MCCANN J J. Lightness and Retinex theory. Journal of the Optical Society of America, 1971, 61(1): 1 -11.

[23] CAI W T, LIU Y X, LI M C, et al. A self-adaptive homomorphic filter method for removing thin cloud. Proceedings of the 19th International Conference on Geoinformatics, 2011, Jun 24 - 26, Shanghai, China. Piscataway, NJ, USA: IEEE, 2011: 1 -4.

[24] STARCK J L, MURTAGH F, CANDÉS E J, et al. Gray and color image contrast enhancement by the curvelet transform. IEEE Transactions on Image Processing, 2003, 12(6): 706 -717.

[25] BYCHKOVSKY V, PARIS S, CHAN E, et al. Learning photographic global tonal adjustment with a database of input/

output image pairs. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11), 2011, Jun 20 - 25, Colorado Springs, CO, USA. Piscataway, NJ, USA: IEEE, 2011: 97 -104.

[26] YANG W H, WANG S Q, FANG Y M, et al. From fidelity to perceptual quality: a semi-supervised approach for low-light image enhancement. Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’20), 2020, Jun 13 - 19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 3063 -3072.

[27] LIANG L M, ZHARKOV I, AMJADI F, et al. Guidance network with staged learning for image enhancement. Proceedings of the 2021 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’21), 2021, Jun 20 - 25, Nashville, TN, USA. Piscataway, NJ, USA: IEEE, 2021: 836 -845.

[28] SCHECHNER Y Y, NARASIMHAN S G, NAYAR S K. Polarization-based vision through haze. Applied Optics, 2003,

42(3): 511 -525.

[29] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE

Transactions on Image Processing, 2004, 13(4): 600 -612.

[30] PARK J, LEE J Y, YOO D, et al. Distort-and-recover: color enhancement using deep reinforcement learning. Proceedings of the 2018 IEEE/ CVF Conference on Computer Vision and Pattern Recognition (CVPR’18), 2018, Jun 18 -23, Salt Lake City, UT, USA. Piscataway, NJ, USA: IEEE, 2018: 5928 -5936.

Back in time: digital restoration techniques for the millennium Dunhuang murals

Back in time: digital restoration techniques for the millennium Dunhuang murals

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价