3D reconstruction algorithm for movable cultural relics based on salient region optimization

doi:10.19682/j.cnki.1005-8885.2023.0012

中国邮电高校学报(英文) ›› 2023, Vol. 30 ›› Issue (5): 11-31.doi: 10.19682/j.cnki.1005-8885.2023.0012

所属专题： Special Topic on Digital Human

• Special Topic : Digital Human • 上一篇下一篇

3D reconstruction algorithm for movable cultural relics based on salient region optimization

王文灏¹,赵海英²

1. 北京信息科技大学
2. 北京邮电大学

收稿日期:2023-07-22 修回日期:2023-09-12 出版日期:2023-10-31 发布日期:2023-10-30
通讯作者: 赵海英 E-mail:hxg510487306@qq.com
基金资助:
the National Key Research and Development Project (2021YFF0901700).

3D reconstruction algorithm for movable cultural relics based on salient region optimization

1. School of Information and Communication Engineering, Beijing Information Science and Technology University, Beijing 100192, China 2. School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2023-07-22 Revised:2023-09-12 Online:2023-10-31 Published:2023-10-30
Supported by:
the National Key Research and Development Project (2021YFF0901700).

摘要/Abstract

摘要：

How to protect cultural retics is of great significance to the transmission and dissemination of history and culture. Digital 3-dimensional (3D) modeling of cultural relics is an effective way to preserve them. The efficiency and complexity of cultural relic model reconstruction algorithms are significant challenges due to redundant data. To tackle the above issue, a 3D reconstruction algorithm, named COLMAP + LSH, was proposed for movable cultural relics based on salient region optimization. COLMAP + LSH algorithm introduces saliency region detection and locality-sensetive Hashing (LSH) to achieve efficient, accurate, and robust digital 3D modeling of cultural relics. Specifically, 400 cultural model data were collected through offline and online collection. COLMAP + LSH algorithm detects the salient region interactively and reduces the number of images in the salient region by feature diffusion. Additionally, COLMAP + LSH algorithm utilizes LSH to calculate the image selection scores and employs the image selection scores to reduce the redundant image. The experiments on the self-constructed cultural relics dataset show that COLMAP + LSH algorithm can efficiently achieve image feature diffusion and ensure the quality of artifact reconstruction while selecting most of the redundant image data.

关键词: 文物数字化|三维重建|显著区域检测|局部敏感哈希。

Abstract:

Key words:

digitization of cultural relics, 3-dimensional (3D) reconstruction, saliency region detection, locality-sensetive Hashing (LSH)

Wang Wenhao , Zhao Haiying. 3D reconstruction algorithm for movable cultural relics based on salient region optimization[J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(5): 11-31.

参考文献

[1] PAN J, HAN Y Q, WANG C, et al. Analysis of microbial community and biodeterioration of maritime cultural relics (ironware, porcelain, axes, hull wood) from the Nanhai No. 1 shipwreck. Annals of Microbiology, 2023, 73(1): Article 1/1 -12.

[2] WANG Y C, CHEN C L, DENG Y Y. Museum-authorization of digital rights: a sustainable and traceable cultural relics exhibition

mechanism. Sustainability, 2021, 13(4): Article 2046/1 -25.

[3] TANG H, GENG G H, ZHOU M Q. Application of digital processing in relic image restoration design. Sensing and Imaging, 2020, 21: Article 6/1 -10.

[4] WANG Z W, ZHUANG Q D, MA Y, et al. A study on the path of narrative renewal of traditional villages: a case of Shawan Ancient

Town, Guangdong, China. International Journal of Environmental Research and Public Health, 2022, 20(1): Article 372/1 -22.

[5] PENG J Y,YU K, WANG J, et al. Mining painted cultural relic patterns based on principal component images selection and image

fusion of hyperspectral images. Journal of Cultural Heritage, 2019, 36: 32 -39.

[6] XU C, GONG A, LIANG L, et al. Vulnerability assessment method for immovable cultural relics based on artificial neural networks: an example of a heavy rainfall event in Henan Province. International Journal of Disaster Risk Science, 2023, 14(1): 41 -51.

[7] YANG S, HOU M L, SHAKER A, et al. Modeling and processing of smart point clouds of cultural relics with complex geometries. ISPRS International Journal of Geo-Information, 2021, 10(9): Article 617/1 -18.

[8] YANG X X, ZHU S S. Application of 3D laser scanner in digitization of movable cultural relics. Proceedings of the 2021 IEEE International Conference on Power Electronics, Computer Applications ( ICPECA'21 ), 2021, Jan 22 - 24, Shenyang, China. Piscataway, NJ, USA: IEEE, 2021: 550 -553.

[9] TONG Y D, CAI Y Z, NEVIN A, et al. Digital technology virtual restoration of the colours and textures of polychrome Bodhidharma statue from the Lingyan Temple, Shandong, China. Heritage Science, 2023, 11(1): 1 -17.

[10] AICARDI I, CHIABRANDO F, LINGUA A M, et al. Recent trends in cultural heritage 3D survey: the photogrammetric computer vision approach. Journal of Cultural Heritage, 2018, 32: 257 -266.

[11] MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: representing scenes as neural radiance fields for view synthesis.

Communications of the ACM, 2021, 65(1): 99 -106.

[12] WANG Y H, DAN X Z, LI J R, et al. Multi-perspective digital image correlation method using a single color camera. Science China: Technological Sciences, 2018, 61: 61 -67.

[13] LIANG H Y, LIU M, HUI M, et al. 3D reconstruction of typical entities based on multi-perspective images. Proceedings of the SPIE, Vol 12319: Optical Metrology and Inspection for Industrial Applications IX, 2022: 338 -345.

[14] JIANG T, GAN X E, LIANG Z, et al. AIDM: artificial intelligent for digital museum autonomous system with mixed reality and software-driven data collection and analysis. Automated Software Engineering, 2022, 29(1): Article 22/1 -22.

[15] PAN Z Q, YUAN F, LEI J J, et al. VCRNet: visual compensation restoration network for no-reference image quality

assessment. IEEE Transactions on Image Processing, 2022, 31: 1613 -1627.

[16] LIU J M, SUN Y, ELDENIZ C, et al. RARE: image reconstruction using deep priors learned without ground truth. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(6): 1088 -1099.

[17] LEHNERT C, TSAI D, ERIKSSON A, et al. 3D move to see: multi-perspective visual servoing towards the next best view within unstructured and occluded environments. Proceedings of the 2019 IEEE/ RSJ International Conference on Intelligent Robots and Systems ( IROS'19 ), 2019, Nov 3 - 8, Macau, China. Piscataway, NJ, USA: IEEE, 2019: 3890 -3897.

[18] BORJI A, CHENG M M, JIANG H Z, et al. Salient object detection: a benchmark. IEEE Transactions on Image Processing, 2015, 24(12): 5706 -5722.

[19] LI C Y, YUAN Y C, CAI W D, et al. Robust saliency detection via regularized random walks ranking. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'15), 2015, Jun 7 -12, Boston, MA, USA. Piscataway, NJ, USA: IEEE, 2015: 2710 -2717.

[20] SHI J P, YAN Q, XU L, et al. Hierarchical image saliency detection on extended CSSD. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(4): 717 -729.

[21] ZHOU L, YANG Z H, YUAN Q, et al. Salient region detection via integrating diffusion-based compactness and local contrast. IEEE Transactions on Image Processing, 2015, 24(11): 3308 -3320.

[22] YAO Q, LU H C, XU Y Q, et al. Saliency detection via cellular automata. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'15), 2015, Jun 7 - 12, Boston, MA, USA. Piscataway, NJ, USA: IEEE, 2015: 110 -

119.

[23] KIM J H, HAN D Y, TAI Y W, et al. Salient region detection via high-dimensional color transform and local spatial support. IEEE Transactions on Image Processing, 2016, 25(1): 9 -23.

[24] PENG H W, LI B, LING H B, et al. Salient object detection via structured matrix decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 818 -832.

[25] LEI J J, WANG B R, FANG Y M, et al. A universal framework for salient object detection. IEEE Transactions on Multimedia,

2016, 18(9): 1783 -1795.

[26] WANG Z L, XIANG D, HOU S H, et al. Background-driven salient object detection. IEEE Transactions on Multimedia, 2017,

19(4): 750 -762.

[27] DENG C, YANG X, NIE F P, et al. Saliency detection via a multiple self-weighted graph-based manifold ranking. IEEE Transactions on Multimedia, 2020, 22(4): 885 -896.

[28] CHEN T S, LIN L, LIU L B, et al. DISC: deep image saliency computing via progressive representation learning. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(6): 1135 -1149.

[29] HE S F, LAU R W H, LIU W X, et al. SuperCNN: a superpixelwise convolutional neural network for salient object detection. International Journal of Computer Vision, 2015, 115(3): 330 -344.

[30] LEE G, TAI Y W, KIM J. Deep saliency with encoded low level distance map and high level features. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'16 ), 2016, Jun 27 - 30, Las Vegas, NV, USA. Piscataway, NJ, USA: IEEE, 2016: 660 -668.

[31] LI G B, YU Y Z. Deep contrast learning for salient object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'16), 2016, Jun 27 - 30, Las Vegas, NV, USA. Piscataway, NJ, USA: IEEE, 2016: 478 -487.

[32] LIU N, HAN J W. DHSNet: deep hierarchical saliency network for salient object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'16 ), 2016, Jun 27 - 30, Las Vegas, NV, USA. Piscataway, NJ, USA: IEEE, 2016: 678 -686 .

[33] ZHANG J, DAI Y C, PORIKLI F. Deep salient object detection by integrating multi-level cues. Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV'17), 2017, Mar 24 - 31, Santa Rosa, CA, USA. Piscataway, NJ, USA: IEEE, 2017: 1 -10.

[34] HOU Q B, CHENG M M, HU X W, et al. Deeply supervised salient object detection with short connections. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41 (4): 815 -828.

[35] ZHANG P P, DONG W, LU H C, et al. Learning uncertain convolutional features for accurate saliency detection. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV'17), 2017, Oct 22 -29, Venice, Italy. Piscataway, NJ, USA: IEEE, 2017: 212 -221.

[36] ZHANG J, LI B, DAI Y C, et al. Integrated deep and shallow networks for salient object detection. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP'17), 2017, Sep 17 - 20, Beijing, China. Piscataway, NJ, USA: IEEE, 2017: 1537 -1541.

[37] WANG L J, LU H C, WANG Y F, et al. Learning to detect salient objects with image-level supervision. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition ( CVPR'17), 2017, Jul 21 - 26, Honolulu, HI, USA. Piscataway, NJ, USA: IEEE, 2017: 3796 -3805.

[38] ZHANG J, ZHANG T, DAI Y C, et al. Deep unsupervised saliency detection: a multiple noisy labeling perspective. Proceedings of the 2018 IEEE/ CVF Conference on Computer Vision and Pattern Recognition, 2018, Jun 18 - 23, Salt Lake City, UT, USA. Piscataway, NJ, USA: IEEE, 2018: 9029 - 9038.

[39] XIE Y X, SUN J H. A graph-based top-down visual attention model for lockwire detection via multiscale top-hat transformation. Expert Systems with Applications, 2023, 214: Article 119218/1 -14.

[40] SUN C Y, YANG Y Q, GUO H X, et al. Semi-supervised 3D shape segmentation with multilevel consistency and part substitution. Computational Visual Media, 2023, 9(2): 229 -247.

[41] YANG X, XIA D, KIN T, et al. A two-step surface-based 3D deep learning pipeline for segmentation of intracranial aneurysms. Computational Visual Media, 2023, 9(1): 57 -69.

[42] CHENG M M, MITRA N J, HUANG X, et al. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 569 -582.

[43] LIU T, YUAN Z J, SUN J, et al. Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(2): 353 -367.

[44] BOYKOV Y Y, JOLLY M P. Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. Proceedings of the 8th IEEE International Conference on Computer Vision ( ICCV'01 ): Vol 2, 2001, Jul 7 - 14, Vancouver, Canada. Piscataway, NJ, USA: IEEE, 2001: 741: 105 -112.

[45] BENTLEY J L. Multidimensional binary search trees used for associative searching. Communications of the ACM, 1975, 18(9): 509 -517.

[46] NISTER D, STEWENIUS H. Scalable recognition with a vocabulary tree. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), 2006, Jun 17 - 22, New York, NY, USA. Piscataway, NJ, USA: IEEE, 2006: Article 264.

[47] SVARM L, SIMAYIJIANG Z, ENQVIST O, et al. Point track creation in unordered image collections using Gomory-Hu trees. Proceedings of the 21st International Conference on Pattern Recognition (ICPR'12), 2012, Nov 11 - 15, Tsukuba, Japan. Piscataway, NJ, USA: IEEE, 2012: 2116 -2119.

[48] JEGOU H, DOUZE M, SCHMID C. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(1): 117 -128.

[49] BAO Y, LIN P, LI Y, et al. Parallel structure from motion for sparse point cloud generation in large-scale scenes. Sensors (Basel), 2021, 21(11): Article 3939/1 -21.

[50] CHENG J, LENG C, WU J X, et al. Fast and accurate image matching with cascade Hashing for 3D reconstruction. Proceedings

of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014, Jun 23 - 28, Columbus, OH, USA.Piscataway, NJ, USA: IEEE, 2014: 1 -8.

[51] WAECHTER M, MOEHRLE N, GOESELE M. Let there be color! Large-scale texturing of 3D reconstructions. Proceedings of the 13th European Conference on Computer Vision (ECCV'14): Part V, 2014, Sep 6 - 12, Zurich, Switzerland. LNIP 8693. Berlin, Germany: Springer, 2014: 836 -850.

[52] SALVI J, ARMANGUE X, BATLLE J. A comparative review of camera calibrating methods with accuracy evaluation. Pattern Recognition , 2002, 35(7): 1617 -1635.

[53] WENG J, COHEN PHERNIOU M. Camera calibration with distortion models and accuracy evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(10): 965 -980.

[54] GUIDI G, BERALDIN J A, ATZENI C. High-accuracy 3D modeling of cultural heritage: the digitizing of donatello's “maddalena”. IEEE Transactions on Image Processing, 2004, 13(3): 370 -380.

[55] YAO Y, LUO Z X, LI S W, et al. BlendedMVS: a large-scale dataset for generalized multi-view stereo networks. Proceedings of the 2020 IEEE/ CVF Conference on Computer Vision and Pattern Recognition ( CVPR'20), 2020, Jun 13 - 19, Seattle, WA, USA. Piscataway, NJ, USA: IEEE, 2020: 1790 -1799.

[56] SCHONBERGER J L, FRAHM J M. Structure-from-motion revisited. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR-16), 2016, Jun 27 - 30, Las Vegas, NV, USA. Piscataway, NJ, USA: IEEE, 2016: 4104 -4113.

[57] ITTI L, KOCH C, NIEBUR E. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern

Analysis and Machine Intelligence, 1998, 20(11): 1254 -1259.

[58] WANG Y F, GONG B H, WEI Y, et al. Video-based vehicle re-identification via channel decomposition saliency region network. Applied Intelligence, 2022, 52(11): 12609 -12629.

[59] ROTHER C, KOLMOGOROV V, BLAKE A. “ GrabCut ": Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics, 2004, 23(3): 309 -314.

[60] MUJA M, LOWE D G. Fast approximate nearest neighbors with automatic algorithm configuration. Proceedings of the 4th International Conference on Computer Vision Theory and Applications (VISAPP'09): Vol 1, 2009, Feb 5 - 8, Lisboa, Portugal. Setubal, Portugal: Institute for Systems and Technologies of Information, Control and Communication (INSTICC) Press, 2009: 331 -340.

[61] LOWE D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60: 91 -110.

3D reconstruction algorithm for movable cultural relics based on salient region optimization

3D reconstruction algorithm for movable cultural relics based on salient region optimization

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价