Shared experiences across multiple devices: a new digital interactive experience method for cultural heritage based on mixed reality

doi:10.19682/j.cnki.1005-8885.2022.0011

中国邮电高校学报(英文) ›› 2022, Vol. 29 ›› Issue (2): 1-12.doi: 10.19682/j.cnki.1005-8885.2022.0011

所属专题：文化计算专题

• Special Topic: Cultural Computing • 下一篇

Shared experiences across multiple devices: a new digital interactive experience method for cultural heritage based on mixed reality

鲁志超,赵海英

北京邮电大学

收稿日期:2022-01-19 修回日期:2022-03-15 出版日期:2022-04-26 发布日期:2022-04-26
通讯作者: 赵海英 E-mail:zhaohaiying@bupt.edu.cn
基金资助:
This work was supported by the National Key Research and Development Program of China (2020YFF0305304)

Shared experiences across multiple devices: a new digital interactive experience method for cultural heritage based on mixed reality

Lu Zhichao, Zhao Haiying

1. Beijing University of Posts and Telecommunications

Received:2022-01-19 Revised:2022-03-15 Online:2022-04-26 Published:2022-04-26
Contact: Hai-Ying ZHAO E-mail:zhaohaiying@bupt.edu.cn
Supported by:
This work was supported by the National Key Research and Development Program of China (2020YFF0305304)

摘要/Abstract

摘要：

In order to study the role of the new technological concept of shared experiences in the digital interactive experience of cultural heritage and apply it to the digital interactive experience of cultural heritage to solve the current problems in this field, starting from the mixed reality (MR) technology that the shared experiences rely on, proper software and hardware platforms were investigated and selected, a universal shared experiences solution was designed, and an experimental project based on the proposed solution was made to verify its feasibility. In the end, a proven and workable shared experiences solution was obtained. This solution included a proposed MR spatial alignment method, and it integrated the existing MR content production process and standard network synchronization functions. Furthermore, it is concluded that the introduction and reasonable use of new technologies can help the development of the digital interactive experience of cultural heritage. The shared experiences solution for the digital interactive experience of cultural heritage balances investment issues in the exhibition, display effect, and user experience. It can speed up the promotion of cultural heritage and bring the vitality of MR technology to relevant projects.

关键词: mixed reality (MR), augmented reality (AR), shared experiences, cultural heritage, digital interactive experience

Abstract:

Key words: mixed reality (MR), augmented reality (AR), shared experiences, cultural heritage, digital interactive experience

Lu Zhichao, Zhao Haiying. Shared experiences across multiple devices: a new digital interactive experience method for cultural heritage based on mixed reality[J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(2): 1-12.

参考文献

[1] BECKER H S. Library of congress digital library effort. Communications of the ACM, 1995, 38(4): 66.

[2] ZHOU M Q, GENG G H, WU Z K. Digitization of cultural heritage. Digital Preservation Technology for Cultural Heritage. Beijing, China: Higher Education Press, 2012: 69 -99.

[3] ZHAO Z H. Digital protection method of intangible cultural heritage based on augmented reality technology. Proceedings of the 2017 International Conference on Robots & Intelligent System (ICRIS’17), 2017, Oct 15 - 16, Huai’An, China. Piscataway, NJ, USA: IEEE, 2017: 135 -138.

[4] PEPE M, COSTANTINO D, ALFIO V S, et al. Scan to BIM for the digital management and representation in 3D GIS environment of cultural heritage site. Journal of Cultural Heritage, 2021, 50: 115 -125.

[5] MIYATA K, TAKIGUCHI T, NAKAGUCHI T et al. An application of projection imaging systems for museum exhibitions. Proceedings of SPIE 6807, Color Imaging XIII: Processing,Hardcopy, and Applications. 2008, DOI: 10. 1117/12. 765785.

[6] BASBALLE D A, HALSKOV K. Projections on museum exhibits: engaging visitors in the museum setting. Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction (OZCHI’10), 2010, Nov 22 -26, Brisbane, Australia. New York, NY, USA:

ACM, 2010: 80 -87.

[7] HU Q W, YU D B, WANG S H, et al. Hybrid three-dimensional representation based on panoramic images and three-dimensional models for a virtual museum: data collection, model, and visualization. Information Visualization, 2016, 16 (2): 126 -138.

[8] WOJCIECHOWSKI R, WALCZAK K, WHITE M, et al. Building virtual and augmented reality museum exhibitions. Proceedings of the 9th International Conference on 3D Web Technology (Web3D’04), 2004, Apr 5 - 8, Monterey, CA, USA. New York, NY, USA: ACM, 2004: 135 -144.

[9] GIANGRECO I, SAUTER L, PARIAN M A, et al. VIRTUE: a virtual reality museum experience. Proceedings of the 24th International Conference on Intelligent User Interfaces: Companion (IUI’19), 2019, Mar 16 -20, Marina del Ray, CA, USA. New York, NY, USA: ACM, 2019: 119 -120.

[10] HAJIRASOULI A, BANIHASHEMI S, KUMARASURIYAR A, et al. Virtual reality-based digitisation for endangered heritage sites: theoretical framework and application. Journal of Cultural Heritage, 2021, 49: 140 -151.

[11] LAW E L C. Augmenting the experience of a museum visit with a geo-located AR App for an associated archaeological site. VERMEEREN A, CALVI L, SABIESCU A ( eds). Museum Experience Design. Berlin, Germany: Springer, 2018: 205 -224.

[12] PARK D, NAM T J, SHI C K. Designing an immersive tour experience system for cultural tour sites. Proceedings of the 2006 Conference on Human Factors in Computing Systems (CHI’06), 2006, Apr 22 - 27, Montréal, Canada. New York, NY, USA: ACM, 2006: 1193 -1198.

[13] CRUZ-NEIRA C, SANDIN D J, DEFANTI T A, et al. The CAVE: audio visual experience automatic virtual environment. Communications of the ACM, 1992, 35(6): 65 -72.

[14] CRUZ-NEIRA C, SANDIN D J, DEFANTI T A. Surround-screen projection-based virtual reality: the design and implementation of the CAVE. Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’93), 1993, Aug 2 - 6, Anaheim, CA, USA. New York, NY, USA: ACM, 1993: 135 -142.

[15] HETTINGER L J, RICCIO G E. Visually induced motion sickness in virtual environments. Presence: Teleoperators and Virtual Environments, 1992, 1(3): 306 -310.

[16] OHYAMA S, NISHIIKE S, WATANABE H, et al. Autonomic responses during motion sickness induced by virtual reality. Auris Nasus Larynx, 2007, 34(3): 303 -306.

[17] STYLIANI S, FOTIS L, KOSTAS K, et al. Virtual museums, a survey and some issues for consideration. Journal of Cultural Heritage, 2009, 10(4): 520 -528.

[18] CARROZZINO M, BERGAMASCO M. Beyond virtual museums: experiencing immersive virtual reality in real museums. Journal of Cultural Heritage, 2010, 11(4): 452 -458.

[19] HUANG Y T, LIU Y, WANG Y T. AR-view: an augmented reality device for digital reconstruction of Yuangmingyuan. Proceedings of the 2009 IEEE International Symposium on Mixed and Augmented Reality: Arts, Media and Humanities, 2009, Oct 19 - 22, Orlando, FL, USA. Piscataway, NJ, USA: IEEE,

2009: 3 -7.

[20] SEO B K, KIM K, PARK J, et al. A tracking framework for augmented reality tours on cultural heritage sites. Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and its Applications in Industry (VRCAI’10), 2010, Dec 12 - 13, Seoul, Republic of Korea. New York, NY, USA: ACM, 2010: 169 -174.

[21] KESHAVARZ B. Exploring behavioral methods to reduce visually induced motion sickness in virtual environments. Proceedings of the 8th International Conference on Virtual, Augmented and Mixed Reality ( VAMR’16), 2016, Jul 17 - 22, Toronto, Canada. LNISA 9740. Berlin, Germany: Springer, 2016: 147 -155.

[22] TAYEH R, BADEMOSI F, ISSA R R A. BIM-GIS integration in HoloLens. Proceedings of the 18th International Conference on Computing in Civil and Building Engineering ( ICCCBE’20 ), 2020, Aug 18 - 20, São Paulo, Brazil. Lecture Notes in Civil Engineering ( LNCE) 98. Berlin, Germany: Springer, 2021:

1187 -1199.

[23] HAN J, KANG H J, KIM M, et al. Mapping the intellectual structure of research on surgery with mixed reality: bibliometric network analysis ( 2000—2019 ). Journal of Biomedical Informatics, 2020, 109: Article 103516.

[24] ZHANG H, ZHANG X S, HUANG J. A demonstration system for the generation and interaction of battlefield situation based on HoloLens. Proceedings of the 2nd International Conference on Information Systems and Computer Aided Education (ICISCAE’19), 2019, Sept 28 -30, Dalian, China. Piscataway, NJ, USA: IEEE, 2019: 293 -296.

[25] WANG G J, LU Z C, ZHANG Y F, et al. Application of mixed reality technology in education with the case of a Huangmei Opera cultural education system. Proceedings of the IEEE 2nd International Conference on Computer Science and Educational Informatization (CSEI’20), 2020, Jun 12 -14, Xinxiang, China. Piscataway, NJ, USA: IEEE, 2020: 301 -305.

[26] WANG G J, QIAN Y, ZHANG Y F, et al. Design and implementation of children’s games based on mixed reality. Proceedings of the 2019 IEEE International Conference on Computer Science and Educational Informatization ( CSEI’19), 2019, Aug 16 - 19, Kunming, China. Piscataway, NJ, USA: IEEE, 2019: 176 -180.

[27] ASGARY A, BONADONNA C, FRISCHKNECHT C. Simulation and visualization of volcanic phenomena using Microsoft HoloLens: case of Vulcano Island (Italy). IEEE Transactions on Engineering Management, 2020, 67(3): 545 -553.

[28] CAUDELL T P, MIZELL D W. Augmented reality: an application of heads-up display technology to manual manufacturing processes. Proceedings of the 25th Hawaii International Conference on System Sciences, 1992, Jan 7 - 10, Kauai, HI, USA. Piscataway, NJ, USA: IEEE, 1992: 659 -669.

[29] MILGRAM P, TAKEMURA H, UTSUMI A, et al. Augmented reality: a class of displays on the reality-virtuality continuum. Das H ( ed ). Proceedings of SPIE 2351: Telemanipulator and Telepresence Technologies. Bellingham, WA, USA: Society of Photo-Optical Instrumentation Engineers, 1995: 282 -292.

[30] MILGRAM P, KISHINO F. A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems, 1994, 77(12): 1321 -1329.

[31] OCCHIPINTI E. Holomuseum: a prototype of interactive exhibition with mixed reality glasses HoloLens. Master Thesis. Torino, Italy: Politecnico di Torino, 2017.

[32] SPEICHER M, HALL B D, NEBELING M. What is mixed reality? Proceedings of the 2019 ACM CHI Conference on Human Factors in Computing Systems ( CHI’19), 2019, May 4 - 9, Glasgow, UK. New York, NY, USA: ACM, 2019: Paper 537.

[33] ONG S, SIDDARAJU V K. Shared experiences. Beginning Windows Mixed Reality Programming: for HoloLens and Mixed Reality Headsets. Berkeley, CA, USA: Apress, 2021: 189 -218.

[34] VASSILAKIS C, KOTIS K, SPILIOTOPOULOS D, et al. A semantic mixed reality framework for shared cultural experiences ecosystems. Big Data and Cognitive Computing, 2020, 4 (2): Article 6.

[35] MITSUNO D, UEDA K, HIROTA Y, et al. Effective application of mixed reality device HoloLens. Plastic and Reconstructive Surgery, 2019, 143(2): 647 -651.

[36] ZHANG Z Y. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330 -1334.

[37] LIU Y, DONG H W, ZHANG L Y, et al. Technical evaluation of HoloLens for multimedia: a first look. IEEE MultiMedia, 2018, 25(4): 8 -18.

[38] TANG B H, CAO S X. A review of VSLAM technology applied in augmented reality. IOP Conference Series: Materials Science and Engineering, 2020, 782: 042014.

[39] HEARN D D, BAKER M P, CARITHERS W. Computer graphics with OpenGL. 4th ed. Upper Saddle River, NJ, USA: Pearson Education Limited, 2011: 273 -301.

[40] CARMIGNIANI J, FURHT B, ANISETTI M, et al. Augmented reality technologies, systems and applications. Multimedia Tools and Applications, 2011, 51(1): 341 -377.

[41] HAMMADY R, MA M H. Designing spatial UI as a solution of the narrow FOV of Microsoft HoloLens: prototype of virtual museum guide. Dieck M C T, Jung T ( eds). Augmented Reality and Virtual Realit. Berlin, Germany: Springer, 2019: 217 -231.

[42] MACLNTYRE B, SMITH T F. Thoughts on the future of WebXR and the immersive web. Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct’18), 2018, Oct 16 - 20, Munich, Germany. Piscataway, NJ, USA: IEEE, 2018: 338 -342.

Shared experiences across multiple devices: a new digital interactive experience method for cultural heritage based on mixed reality

Shared experiences across multiple devices: a new digital interactive experience method for cultural heritage based on mixed reality

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价