科技藝術書報討論

AIMS FELLOWS 洪寶惜 111005520

A DESIGN FRAMEWORK FOR SMART GLASS AUGMENTED REALITY EXPERIENCES IN HERITAGE SITES

Link： https://dl.acm.org/doi/10.1145/3490393

ACM Journals > Journal on Computing and Cultural Heritage 

A visitor with the Hololens on looking at the virtual minuet dancing shoes. Image shows the south side of the Great Chamber. Combined footage of real space and AR view.

A visitor with the Hololens on. Image shows the north and west side of the Great Chamber

1. INTRODUCTION

AR/VR技術應用於博物館和文化遺產場所，提供豐富且互動性強的參觀體驗，可能是未來文化遺產展示和解說的重要趨勢。這篇論文主要強調幾個重點方向：

• Transdisciplinary approach： A transdisciplinary approach is important in cases where knowledge production and problem solving are quite complex and involve many traditional disciplines and discipline-specific knowledge 。 Interaction designers, historians, AR developers, museum educators, and visitor experience experts, worked together 。
• Storytelling medium： It presents one design approach that focuses on AR as a storytelling medium and views the design through a dramaturgical lens to help create affective experiences 。
• Design case of Sutton House Stories：It is documented in this article as a starting point for researchers and practitioners who wish to assess it and develop it further. (URL Link )

​

1. INTRODUCTION

內文主要分為以下幾個部分：

• Introduces the connections between affect and learning
• Introduces Sutton House Stories and describes its evaluation and feedback discussing also related projects from the literature
• Introduces the framework by discussing the feedback and clustering the insights into the framework’s design pillars
• Concludes with some thoughts for the future

有別於之前傳統AR設計中的技術限制、內容複雜性和人因素等問題，作者希望能為未來文化遺產展示和解說提供新的可能性。

2. BACKGROUND

本段落介紹了在文化遺產場所進行情感敘事(Affective storytelling)學習的概念。

• Learning in heritage sites is experiential, enactive, and situated。
• provoking the audience to learn about issues related to the site, construct and discover personal meaning, and forge personal connections with things, places, people, and concepts, rather than simply communicating information through displayed artefacts。
• Authentic stories help with the design of the visitor’s trajectory in the site and help communicate its cultural significance in a seamless manner that allows and promotes visitor imagination. The affective factor in the creation of the story around a place fuels experiential learning, which has always been important to heritage sites.

智能眼鏡擴增實境體驗可以支持多感官的互動學習，特別是當使用者互動設計為情感敘事時。全身參與和情感互動創造了一個積極的學習空間，有助於知識的保留和學習。

3 .SUTTON HOUSE STORIES

• Sutton House Stories is a four-minute long smart glass AR experience using Hololens 1, designed and developed in 2019 for Sutton House, a Tudor house in the London Borough of Hackney (UK).
• The experience was designed during two full-day workshops that gathered the house’s historians, educators, visitor experience experts, the AR developers, and the author as an interaction designer.
• 使用者戴上頭戴式顯示器，繞行宅邸中最大的房間，即大廳，並由三個聲音引導，分別代表了四個世紀曾居住在宅邸中的三個關鍵人物，專業的配音演員被聘用來以獨白的形式演繹劇本，彷彿正在與觀眾對話。觀眾在聽取解說的同時，會看到和聽到與解說相關的疊加內容，觀眾還可以進行互動，例如，有一個場景中，一雙舞鞋會進行一個古老的喬治時期流行舞蹈。觀眾可以自由地在舞蹈鞋周圍走動，觀看它們的舞蹈，甚至嘗試模仿舞步，彷彿與舞蹈鞋一同跳舞。另也能看到當時戶外的自然景觀。
• focusing more on creating opportunities for play (which can broadly be contrasted to games as a non-goal-oriented activity) than a game, such as the minuet dance, and designing in components that suspended further the disbelief to create a “humane” experience.

​

​

​

3. SUTTON HOUSE STORIES

The watercoloured virtual view out of the window. Combined footage of real space and AR view

3.1 EVALUATION

本研究對參訪者的學習效果進行了評估，與一般參訪房屋的方式進行對比。

• The qualitative evaluation included observing the participants during the experience, focus groups, and a tailored Personal Meaning Mapping (PMM) exercise pre- and post-experience.
• In total, 36 participants took part in the evaluation in five groups and as six individuals.
• Sutton House staff: three who had participated in the initial workshops and two who had not.
• All participants were adults, and six of them were in the age group 55–70, with two of them having mobility issues.
• The evaluation took place over five consecutive days.
• Situated learning is a complex process that can not often be measured with a short-term evaluation.
• 本次評估試圖通過監測參與者的情感參與度作為學習的衡量指標，結合他們在PMM和焦點小組中對體驗的直接反思。

​

3.1.1 PERSONAL MEANING MAPPING

（PMM）是由Adams等人於2013年開發的一種研究方法，旨在捕捉博物館參觀者的多面學習體驗，關注其個人解釋和個體背景如何幫助他們浮現。該方法基於學習是一個相對性和建構性的過程，取決於個人、社會文化和物理背景因素。為了形成對學習過程的整體理解，研究人員在評估學習體驗時需要考慮這些因素。PMM將參觀者的議題置於中心，在PMM活動中，參與者會獲得一張空白紙，在中間位置有一個提示詞、名字、短語或圖像，然後被要求寫下任何與這個中心概念相關的想法，發展成一個思維導圖。完成後，參與者會被採訪，關於他們所寫的內容進行詳細說明。採訪者會用不同顏色的筆墨修改思維導圖，添加擴展的回應。在參觀結束時，參與者會被要求重新審視他們的地圖，並使用不同顏色的筆墨進行任何更改。然後，他們會接受關於他們的變更的採訪。這使得採訪者能觀察到參觀前後的知識變化。PMM的分析過程涵蓋了個體內部和個體之間的比較。

由於PMM關注理解參觀者的學習體驗並重視參觀者獨特的背景，因此被認為是評估Sutton House Stories項目的理想工具。評估的目標是觀察參觀者在Sutton House Stories項目中的學習體驗，包括他們對展示內容的解釋、興趣、觀點和情感反應。PMM方法的適應包括在提示詞中使用與Sutton House Stories相關的概念，並根據項目的研究問題調整採訪和分析的內容。研究人員使用PMM方法進行了對參觀者的採訪和思維導圖分析，並關注了參觀前後的知識變化、觀點轉變和情感體驗。通過PMM的應用，研究人員能夠深入了解參觀者在Sutton House Stories項目中的個人學習體驗，並獲得對項目成效和效果的豐富洞察。

* Visits were not shorter than 15 minutes and no longer than 30 minutes.

3.2 RESULTS�3.2.1 SITUATED LEARNING.

• “alive” and “came to life.” 90% of participants used this word to describe how they viewed the house after the AR experience, whereas a big percentage mentioned that they connected deeper with the place and the people that were part of its history, often using the phrase “emotional connection” to describe this.
• There was strong evidence of situated learning, as 85% of participants wrote and discussed the new historic knowledge they gained through the AR experience.
• On their post-experience maps they mentioned that they learned about the person who built the house, how the house was used across the centuries, and how the surroundings looked back then. As the AR experience happened at the end of their visit, this evidence indicates that they would normally leave the house without this information, even though it was given to them through labels scattered around the house.
• All staff members who were part of the design workshops had a profound emotional response to the AR experience despite already knowing in depth its history
• All participants favoured and spent most time at two parts of the experience: the minuet dance and the window overlooking the countryside. Analysis showed that the reason behinds this is that both went considerably beyond conveying historical facts.

3.2 RESULTS�3.2.1 SITUATED LEARNING.

綜合來說，AR眼鏡的PMM結果顯示了情境化學習在參與者中產生了積極的影響，包括對房子和歷史產生更深的情感連結、獲得新的歷史知識和提高記憶和知識保留能力。這些結果強調了AR技術在教育和學習領域的潛在應用，特別是在歷史和文化領域。AR體驗可以通過情感投入和互動性來提升學習體驗，激發學習者的興趣和好奇心，幫助他們更深入地理解和記憶學習內容。這種情境化學習的方法可以使學習變得更加生動有趣，並且有助於學習者與學習內容建立情感連結，從而提高學習效果和長期記憶。

然而，也有一些參與者提到了一些潛在的挑戰和限制。例如，一些參與者對AR技術的穩定性和操作性提出了一些疑慮，包括眼鏡的舒適性、使用者界面的友好性和AR應用的流暢度。此外，一些參與者也提到了AR體驗的時間限制和對硬體設備的依賴性，可能會限制AR在學習場景中的應用。這些挑戰需要在未來的AR教育應用中得到解決，以確保AR技術能夠充分發揮其潛在的學習增強效果。

總的來說，情境化學習在AR技術中的應用對於提升學習體驗和學習效果具有潛在價值。AR體驗可以通過情感連結、提供新的歷史知識和促進記憶和知識保留能力來豐富學習內容，使學習者更深入地理解和記憶所學內容。然而，還需解決一些潛在的挑戰，以充分發揮AR技術在教育領域的應用價值。未來，AR技術在情境化學習中的應用有望在教育和學習中發揮更大的作用。

3.2 RESULTS�3.2.1 SITUATED LEARNING.

• 特別提到了壁爐，其中有一個虛擬的火焰和伴隨的聲音，Sir Ralph Sadlier坐在書桌前寫字，書桌上散落著信件，附近有一支燃燒的蠟燭，還有一幅實時在真實的畫架上畫的女性藝術家的虛擬畫像。這些火焰和信件等元素都被設計成不立即可見，而是等待參與者發現。大多數參與者表示，他們希望能夠與虛擬物體互動，例如拉上窗簾或感覺自己觸摸到它們。故事的某些部分提供了進行物理互動的區域，一些參與者希望跟隨圓舞曲的舞步，有些人想要抓住並閱讀Ralph爵士的信件，而真實窗外的虛擬自然景觀創造了一種引人入勝的氛圍，使人停下來靜靜思考。一些參與者提到，這種AR體驗以一種非常個人和寧靜的方式讓他們享受，無需與導遊互動或主動聆聽，而是可以按照他們自己想要的方式和節奏參與增強現實空間。
• 這些討論點強調了創建一種身臨其境且能動的體驗的重要性和獨特性，有人喜歡透過視覺、有人喜歡透過聽覺感受，這是構建意義的基本要素，在AR體驗中，虛擬和現實之間的界限變得模糊，使參與者感覺彷彿置身於一個全新的世界中。他們可以在虛擬空間中自由運動，觀察虛擬物體和角色，並與之互動。這種身臨其境的感覺讓參與者對虛擬世界充滿了好奇和探索的慾望，並且激發了他們對科技和創新的興趣。

​

​

3.2.2 USABILITY

• 由於可用性並非PMM過程的重點，因此地圖上並未反映出可用性問題，而是在焦點小組討論中進行了討論。
• 評估的一部分涉及不同年齡組使用設備的情況。參與者中沒有人抱怨眼鏡的重量，實際上，每個人都對體驗表示滿意，並對體驗結束相對較早感到失望。
• 主要的缺點是使用者需要進行的手勢操作，即“捏”手勢，以在不同章節之間切換。對於一些參與者來說，在相機可以看到的位置形成這種手勢是困難的，並且這一情況與年齡無關。過去的研究使用物理對象來解決這個問題，但這種方法可能會干擾使用者體驗。
• 研究人員評估了點擊次數對使用者體驗的影響，結果顯示使用者對點擊次數感到滿意，並認為這是一種有效的導航工具。一些參與者希望能夠返回之前的章節並重新播放，並提到在空間中可見的點擊提示對於選擇要聽的故事也可能很有用。

4 . TOWARDS A FRAMEWORK FOR SMART GLASS AR HERITAGE EXPERIENCES

雖然過去20年來移動AR項目眾多，但在設計AR體驗的框架方面，尤其是在文化遺產領域，僅有少數嘗試。大都從產品/服務特性、體驗周圍的時間期限和應用的特定背景開始考慮。作者保持了以使用者為中心的設計方法，不將焦點放在可用性上，基於三個設計支柱來建立AR文化遺產體驗，如下圖。

4.1 INTERPRETATION

Interpretation在選擇要講述的故事和塑造敘事內容方面扮演著關鍵角色。

4.1.1 Learning Objectives 。每個文化遺產遺址都有自己的戰略優先項目，包括他們希望向觀眾傳達的信息和學習目標。這是創建體驗和設定背景的第一步。例如，在Sutton House中，面臨的挑戰是如何在一個歷史悠久的房子中呈現四個世紀的豐富歷史，這個房子多次更換角色，幾乎沒有物品和家具。需要注意的是，優先項目可能會在跨學科的創作過程中發生變化。

4.1.2 Time Navigation 。在這個過程中，敘事的時間軸也被定義。線性的時間軸對於按照順序來追隨所有章節和序列的參觀者是有用的。然而，將故事拆分成章節對於選擇性的導覽也是有用的。在未來廣泛採用AR智能眼鏡的情況下，擁有章節對於希望選擇體驗哪個時代/事件/人物故事或類似內容的參觀者可能是理想的。

4.1.3 Archival Material. 。在創建全息內容的過程中，有機會使用通常被隱藏和未使用的實體檔案作為地方的活躍檔案。博物館教育工作者和公共歷史學家的參與對於故事的發展、以情感為背景講述故事的方式、從檔案中獲取素材以及向不同的受眾傳達信息是至關重要的。他們與AR設計師和開發人員的密切合作對於檔案作為情感背景下的故事元素浮現是至關重要的。

4.2 AFFECTIVE STORYTELLING�4.2.1 HISTORICAL EMPATHY

Affective Storytelling concerns the content, the kind of stories, and the sequence in which they are told, if any, the characters, the narration, the virtual material, and the way it is weaved into the physical space, and the interactions between visitors, space, content, and AR media.

4.2.1 Historical Empathy - Historical empathy involves the learner’s understanding and context-ualisation of how historical characters thought, felt, made decisions, acted, and faced consequences within a specific historical and social context through cognitive and affective engagement with them.

Tools to assist with this process have been explored in related literature, such as the importance of spatial arrangement of objects in an exhibition or heritage site and the design of the visitor’s walk through them, , mnemonic bridging and the use of physical objects, practices of imitation, and replication, and balancing multiple-perspective narration.

4.2.2 NARRATION

• One-person narration creates a more emotional engagement, whereas multiple perspectives support a more cognitive approach [Stradling 2003]
• In Sutton House Stories, the first-person voice-acted narration proved to be fundamental to the successful animation of the space, while the fact that the narration was firmly connected to the AR material made it an appropriate synergy for increasing participant empathy with the characters.
• The script plays a very important role in creating empathy, and much attention must be given in the selection of words to edit out affective or suggestive language that may interfere with the visitors’ own response to the unfolding stories [Bedford 2001].
• In addition, the virtual material that was not connected to the stories but provided a virtual animated backdrop helped make the virtual world more meaningful and strengthen the immersion.
• In Sutton House Stories, we chose pivotal historic periods with different atmosphere to portray but also created a unifying narrative to tie them together that was present but not explicitly told until the end. In this way, visitors realised the different “lives” of the house and how it has been kept“alive” so far by people and communities caring forit and using it for different purposes through the years.

​

4.2.3 VISITOR’S ROLE AND ON-/OFF-BOARDING

• The design must consider the visitor’s smooth transition across the different roles from visitor to viewer, interactor, explorer, and experiment with ways to integrate it with the narrative.
• Whether they are given a role or not, particularly important is how the visitors are introduced to the experience and how they leave it (AN.d), called the user “on- and off-boarding,” an area that has not been researched much in AR heritage design.
• the viewers must be given enough time to get accustomed to the new technology, and this part ideally should be integrated into the narrative.
• In addition, the transition from the physical to the hybrid world and back must be designed to be smooth and meaningful.

4.2.4 AGENCY

​

• Attention should be given to the different levels of agency that viewers will have on how the story develops and the multiple ways they can interact with the content.
• This is a rich area where the role of the creative writer and (digital) storyteller will be fundamental within the transdisciplinary collaboration.
• In Sutton House Stories, we focused on playful interaction that was integrated at points throughout the experience, such as trying out the minuet dance and going closer over Ralph Sadlier’s shoulder to read his letters.
• should be careful consideration of the balance between designed level of agency and situated, embodied learning.
• suggest looking more at playful interactions within the embodied experience rather than focussing on creating a game.

​

​

​

​

​

4.2.5 EMBODIED COGNITION

• Within the playful interaction discourse, discussions during the focus groups revealed that a closed loop between action and perception can strengthen the visitor’s embodied understanding of the hybrid physical-virtual space and further add to the immersion of the viewer.
• Such moments happen, for example, when the viewer goes closer to a 3D character and the character turns towards them to talk.
• one participant mentioned how it would be nice if there were some physical props that would give you the visual information but only when you touch them.
• Such interactions were not designed in Sutton House Stories but were suggested by participants as areas for further development, especially since new devices will offer additional and better gestural interaction. This area is certainly a promising emerging research playground.

​

​

​

​

4.3 TECHNOLOGY CONSIDERATIONS�4.3.1 NAVIGATION MECHANISMS.

科技考量（Technology Considerations）是指擴增實境（AR）設備的限制和機會。正如在設計過程中所看到的，AR眼鏡具有特定的挑戰，這些挑戰會影響其他兩個領域的設計決策。

• Navigation mechanisms should include different options for the several types of visitors.
• Wearing AR glasses changes the body’s kinesthetic perception, so it is crucial to consider how the viewer will move around with it.
• For example, a couple of participants mentioned that, because of the excitement of using a new technology, they took some time to get used to the visuals and did not pay much attention to the narration of the first chapter.

因每個參觀者的背景不同，對AR畫面的期許也不同，比如常玩遊戲的會希望箭頭的形狀不同且更小的視覺指示器可以用來指示附近的其他興趣點，而不僅僅是順序中的下一個。

​

​

4.3.2 CHOREOGRAPHY.

Choreography involves the design of a coherent path between chapters, in the case these are played from start to end and finding creative solutions for when viewers take time to explore instead of following what unfolds in the hybrid space.

• the script and sound as important factors in creating the viewer choreography in the space.
• issues, and the best way to address it currently is for the designers to acknowledge it and take it into consideration in creating the sequences. The viewers should also be made aware of it during on-boarding. For example , Movement in space has several challenges when the user is in a small space together with many other visitors.

需要考慮章節之間的平衡和節奏，以確保觀眾在整個體驗中保持興趣和投入。太長或太短的情節可能會使觀眾感到厭倦或錯過關鍵內容。因此，設計者需要謹慎地選擇情節的時長和節奏，以確保觀眾可以在適當的時間內獲得完整的體驗。

最後，設計師還應該考慮觀眾的多樣性和個體差異，例如年齡、文化背景、興趣和偏好。觀眾對於虛擬體驗的反應和需求可能因人而異，因此在設計過程中應該考慮到不同觀眾的需求，以確保體驗對於多樣性的觀眾都具有吸引力和價值。

​

4.3.3 TC.C SENSORIAL INPUT

Sensorial Input. Finally, the opportunities for sensorial interaction that the technology affords should be used as tools to enhance affective narrative. These include sound, gesture, and modalities that were not utilised in Sutton House Stories, such as gaze and eye tracking. The combination of these tools with tools for creating historical empathy, agency , and utilising embodied cognition open new avenues for creating embodiment, presence, and meaning.

5 FURTHER RESEARCH

• This article presented a prototype framework for designing affective smart glass AR experiences for heritage sites using AR smart glasses.
• The framework’s design pillars can be used as design starting points in future applications. The pillars are not hierarchical, and stakeholders can decide how much weight each design consideration will have in the overall composition, depending on the project’s production goals, parameters, and limitations.
• The proposed framework is only a milestone in the process of identifying, understanding, and addressing design problems in the creation of AR learning experiences in heritage sites.
• Therefore, this framework is given here as a live, expandable, and adjustable base to be further developed as its design pillars change and readjust.

作者介紹

Mariza Dima

• Brunel University London
• The University of Edinburgh

聯想與心得

• 戶外的世界文化遺產更需要 !!!

感謝您的聆聽

​

​