JAILCD2026-015: Mixed Reality Display and Interaction Modalities in for Heritage Interpretation in Outdoor Architectural Spaces

Abstract

Visitors typically experience heritage sites passively by viewing, reading, or listening. The remaining structures convey only limited stories, while deeper historical details are often hidden in text or narrative forms that demand excessive focus or feel too abstract to grasp. This makes many heritage sites outdated as learning environments, leaving visitors less engaged and often unsatisfied. Mixed Reality offers a promising improvement by enabling immersive engagement, interactive learning, and enjoyment. However, the design trade-offs between display modalities (optical see-through versus video pass-through) and interaction modalities (natural gestures versus controller-based input) remain underexplored in outdoor heritage contexts where immersion can make or break the experience. This paper synthesizes empirical studies on peripheral vision, presence, usability, cognitive load, safety and cybersickness to examine how display and interaction modalities shape spatial immersion, embodied interaction, and learning engagement, highlighting their respective strengths and limitations. Results indicate that video pass-through improves clarity, resolution, and color accuracy but increases cognitive load and cybersickness, whereas optical see-through maintains environmental awareness and reduces perceptual strain. Similarly, natural gesture interaction enhances spatial presence, embodiment, and emotional engagement, while controllers provide precision and haptic feedback but introduce ergonomic constraints and require layout learning. Therefore, the optimal configuration for outdoor heritage learning combines an optical see-through display with natural hand interaction, balancing authenticity, usability, and interpretive depth. A feasibility test confirmed that the selected device (Microsoft HoloLens 2) functioned effectively on-site as recommended. Possible benefits include improved learning, positive post-visit behavior, and cultural appreciation.