ABSTRAKT🦋
Project Brief
In collaboration with Zeros2Heroes and their cross-cultural team that includes partners from Germany and Israel, Team Flatland, a multi-disciplinary team of students from the Centre for Digital Media (CDM). worked on artifact and interaction design of a polished vertical slice of a fully-featured immersive VR experience of Berlin’s Holocaust Memorial.
The goal of this project is to build a polished vertical slice of a fully featured immersive VR experience of the Holocaust-Mahnmal, or Berlin’s Holocaust Memorial (Memorial). The focus of the CDM team is on artifact and interaction designs. Through artifacts that are grounded in real people and authentic history, the project outcome will help the target users empathize with Holocaust survivors.
Berlin’s Holocaust Memorial VR experience
Project by Team Flatland, Master Students from Centre for Digital Media:
Chalita Termpaiboon (Me) - UX designer
Junyi Sun - UX Designer
Wing Tsui - Project Manager
Kennedy Kan - 3D Artist
Peter Luong - Developer
YiZhen Zhou - Developer
My Contributions: Researches, VR Interaction Design, User Experience Design, User Interface Design.
Duration: 3 months
Tools: Figma, Adobe Suite, Miro, Unity
Date: May-August 2022
🤔
"How might we engage middle and high school students in North America emotionally to encourage them to understand the Holocaust history and its significance today?"
Problem Setting:
- Many youths today have a lack of Holocaust education and awareness: two thirds of American youth do not know that Auschwitz was a concentration camp, and 22% claimed to have never heard of the Holocaust.
- To encourage them to understand the Holocaust history and its significance today, Flatland develop digital representation of the Memorial in a Virtual Reality space, augmented with unique experiences within VR.
- Through environmental storytelling, artifacts and interactions in this experience,
This project aims to help the younger generation who is familiar with technology build empathy and counter the rise of Neo-Nazism and carryon the legacy of Holocaust survivors. Our goal is trying to build empathy while educate young people about WWII
Solution: what did we create ?
To engage middle and high school students in North America emotionally and encourage them to understand the Holocaust history and its significance today, we designed a VR experience of the Berlin’s Holocaust Memorial with interactive artifacts of the Holocaust's victims and understand their experiences. In this prototype, we focus on Hannah Gofrit’s experience. Our team reviewed the holograms with her sharing her experience during World War II(WWII). There are two types of artifacts in this experience:
Personal Artifacts
Personal artifacts are related to Hannah’s personal experience during WWII. Each personal artifact is placed in a scene that builds up the tension and get the user immersed step by step. It starts with isolation and alienation, then to rejection and frustration.
Historical Artifacts
historical artifacts are iconic items that are related to WWII history. They are placed along the way to the centre of the Memorial but interacting with them is optional. They are either placed on the stelae or on a wall shelf. In the current prototype, WWII posters, radio and postcard are the historical artifacts.
Why 2 types of artifacts?
To finish this VR experience, the interactions with personal artifacts are compulsory, but interacting with historical artifacts are optional. Every interactive object is highlighted with an icon showing how the user can interact with it.
The reason why I decided to give them optional interactions because I still want to give a freedom and sense of exploration to the audience. Regarding to the user research, I found out that target group also responded very well with game like elements and they don't like to be forced or following the rules all the time. Having historical artifacts available as an optional "side quest" would increase the engagement from the audience.
Personal Artifact (1) - Book of mountain
This scene shows a huge stack of books on the ground with three interactive children’s books that have no words inside.
Thoughts behind the idea for this artifact:
It was inspired by Hannah’s personal experience. When she was at the school age, she did not get a chance to enter the classroom on her first day of class and receive formal education. She could only learn from her father.
Our goal is to create the feeling of isolation and confusion as Hannah’s experience. At first, we came up with a lot of ideas based on her experience. The idea was originally was a part of the personal scene 3 (the classroom loop) since it shared the same setting. We decided to separate this idea, as stakeholders suggested, to enhance the storytelling effectiveness and build up emotions for the overall experience. As the first artifact of this experience, this scene will buildup user’s emotions as it arouses user’s curiosity and evokes their emotions.
The idea of mountain of books was inspired from the iconic Holocaust history - the Mountain of Shoes. With the huge number of books stacking up, the scene will leave a strong impression to users, and create an illusion that users are interacting from a child’s point of view. The design also gave users the feeling of out of reach since they cannot read nor get any information from books even if there is a large stack of book in front of them. This amplifies the emotions our team would like to convey through the scene and meets our goal of engaging students in North America emotionally.
Personal Artifact (2) - Children's Drawing
This scene shows Hannah’s desk with three drawings showing the animation of Hannah playing with other children happily.
Thoughts behind the idea for this artifact:
I have created this scene to resonate with one of Hannah’s childhood memories during World War II, when she was hiding for 2 years. During that time, she was staying with another Catholic family and lived on the 6th floor of the building. There were some rules that she needed to follow for safety reasons: be quiet, no shoes were allowed because they lived on the 6th floor and people might hear the foot steps and never went near to the windows.
She often heard children playing in the playground and as a young child, she always wanted to go out and joined them. She could only imagine herself as a butterfly to fly down and played with them. I tried to portray emotion of isolations and exclusion from her experience by using children’s doodle with crayons to reflect what children in WWII used for drawing in that time. One of the children, Hannah, is also disappearing as the user interacts with represent the idea of how Hannah’s childhood was taking away from her.
Personal Artifact (3) - Classroom loop
This scene shows two identical corridors outside of a classroom, which create an illusion that the user gets back to the start after they open the classroom door.
Thoughts behind the idea for this artifact:
As mentioned in personal artifacts (1), this idea also came from Hannah’s experience when she could not go to school and receive formal education. I came up with a lot of different ideas to achieve the following three goals for this scene: effectively portray confusion, fear and anxiety, building up for the tableau moment, and maintaining a smooth user experience in VR environment.
After considering all factors, the current idea of having a loop in the corridor outside of a classroom was deemed to be the most effective in creating the targeted emotions since users are ‘stuck’ in the loop for a certain amount of time in a relatively dark and narrow environment. This idea was inspired by a horror game called Silent Hill P.T., which requires players to spend a certain time in the same environment before moving on to the next point. We achieve an eerie and odd feeling with a glimpse of hope by letting users hear students and teachers chatting sound when they are approaching each door in the loop but never letting them go into the classroom. The disappointment followed by the loss of hope after each interaction creates a strong emotional impact on users.
Besides, this idea was approved and recommended by the technical lead of the larger project team since it has lower technical risks when combining with procedural generated maze, which will be created by another project team.
Historical Artifact (1) - Posters
The user can interact with posters during WWII in the below way:
1. The user notices the visual cues –posters are highlighted with an icon.
2. The user peels off the colorful poster on top and reveals the Nazi poster underneath. During this interaction, the user hears the sound effect of paper peeling.
3. The user does the same interaction to another set of posters.
Thoughts behind the idea for this artifact:
Poster idea was inspired by propaganda posters and war art during WWII. We put the vibrant, cheerful advertisement poster son top of the dark, frightful war posters to convey the idea of how the war changed people’s life. Like the poster arrangement, everything seems to be normal and lively at first. Then, when the war began, symbolize by the first layer of poster being ripped off, people’s everyday life was disrupted, and bright days were gone. What remained were what are shown on propaganda war posters–dark and terrifying moments.
Historical Artifact (2) - Radio
The user can interact with an old radio in the below way:
1. The user hears static radio sounds when they step into the hit radius of the radio
2. The user sees an icon and the highlight around the radio
3. The user moves the knob to tune the radio to listen to music
4. Nazi's propaganda starts to play after 5 seconds, replacing the music no matter how many times the user tries.
5. After annoying the user with Nazi propaganda for 10 seconds, the radio gets turned off automatically and disappears with the moth effect
Thoughts behind the idea for this artifact:
Radio was one of the common communication ways and widely used during WWII. I wanted to depict how people got effected from the war inevitably no matter how hard they tried to escape from it. Therefore, after hearing lively music for a few seconds, users have no choice but to listen to Hitler’s speech with no ways to turn off the radio until the speech ends.
Historical Artifact (3) - Postcard and Post box
The user can interact with a postcard in the below way:
1. The user notices the visual cue around the pen
2. The user picks up the pen and notices the highlight around the postcard
3. Using the grab button, the user writes the postcard saying they need some food.
4. Once the user finishes writing, censor stamps appear over the content of the postcard, leaving only some of the content saying they do not need food.
5. The user notices the highlight around the post box.
6. When the user tries to send the postcard, the postcard disappears with moth effect.
Thoughts behind the idea for this artifact:
This artifact is inspired by concentration camp postcards. After researching on how people used to communicate to each other during the time and what was happening in the camp from the survivors, I found out that Jewish people who lived in concentration camps were only allowed to write about positive things on postcards to the outside world. Since they could not express their emotions nor ask for help, I came up with the idea of having parts of the context censored to change the meaning of postcard, which symbolize how Jewish lost their freedom of expression during WWII.
Transition Sequence - Cabinet
To ensure the art style of the artifacts in this prototype is consistent, our team also created a cabinet and its basic interactions for the transition sequence. Similar to the personal artifacts, 5x3 stelae are removed so that the scene can spawn in the Memorial. The user can get in and out of the cabinet and they can hear the sound cue of a female voice (mother) calling “Hannah, here here” leading them to enter the scene.
Thoughts behind the idea for this artifact:
The transition sequence is designed to create a smooth transition from the walking experience to the tableau moment. In Hannah’s experience, she often had to hide in the cabinet for a long time while someone was coming over to visit the family, who provided a space for Hannah and her mother to hide.This is one of the key moments in Hannah’s life as she often used her imagination to escape from reality and stay positive in that time. It suits perfectly as the transition to tableau moment, which users are prompted to hide in the cabinet, creating the most intense in this VR experience.Also, it helps transiting the emotion of the experience from fear, anxiety and isolation to hope and optimism.
Let's talk about process!
Research
After we got the brief from client, I started off with doing research about the real site that we will be working on. Since we are based in Canada and none of us has been there before, we have to rely a lot on secondary research, talked to people who went to the memorial before, and analyzing walk through videos.
The Memorial to the Murdered Jews of Europe is in Berlin, Germany. It was designed by architect Peter Eisenman throughout 2004 and 2005. As stated by Eisenman, the monument is arranged in grid pattern. In total, there are 2,711 concrete pillars covered in the 13,000 square metres site
As the visitor walks into the middle part of memorial, the stone pillars get taller, and the pathway is also change from being flat to more rippled. All the visitors can have different experiences with the memorial. People often describe that they can feel the sense of disorientation, lost, loneliness, and isolation, which are relatable to experience of Jewish people in World War II.
The team also visited the Holocaust Education Centre in Vancouver to gain more information about the artifacts and 3D virtual experience that we will high likely work on. Also to better understand the timeline of the Holocaust and explore different ways to present artifacts in an innovative way. The team took a couple of references that represent the camp and Jewish community during WWII. From the visit, we learned a lot more about Jewish people experience through Hahn’s family collections. We also got inspired by the ideas of telling story through a person or a family because it gave out an individualized touch which can be more relatable.
Here is team Flatland on a field trip ! (Missing Kennedy)
Study about the target group - Generation Z
While doing research, based on these findings, we further discussed and investigated how we generate emotional interactions with user groups through the design and optimization of interactive behaviors in our project. And on this basis, we discuss the concept of possible interaction modes. I believe the overall experience should be bright and hopeful at the end. Emotionally, I want the user to be curious and expectant at the beginning, lost and confused when interacting with the artifact and then moved by the end of the experience so that they can have empathy and feel more connected to the history.
In terms of the overall interactive narrative, I hope that by creating these emotional interactions, the experience will leave a strong impression on our target group after completing the interaction. Instead of forcibly pushing these ideas into their heads, I found that building and creating an environment for young people to rethink about history and making them empathize with these historical facts is the best way to achieve our educational goal. I want them to be invested in the experience but also contained in the serious tone since it was a sensitive topic.
Unfortunately due to the ethical issue in researching minors, we cannot interview the target group directly so we have to rely on secondary research heavily.
Brain storming ideas
User research
Empathy map
User Persona
In order to better design the interactive experience of the project, I conducted detailed research on the target audience of the project at the beginning of the project. Then, I formed user persona based on the results of our target group research.
In order to better design the interactive experience of the project, I conducted detailed research on the target audience of the project at the beginning of the project. Then, I formed user persona based on the results of our target group research.
User Persona
Summary of target audience
Environmental and VR storytelling
To have a better understanding of the market, the team also researched on environmental and VR storytelling. Here are some of the findings:
To have a better understanding of the market, the team also researched on environmental and VR storytelling. Here are some of the findings:
VR Interactions:
After I did some research on VR experience and environment storytelling, I started to work on what could be our potential interaction for this project. I also decided to use simple interactions that can be complete by using only one button to make the experience simple and avoid steep learning curve. I are also decided to generalize with the target to be more universal in terms of interactions so that not just only target group can use, other people can also have an easy interactions in the experience too.
Universal Interactions:
•Touch
•Push and Pull
•Read
•Pick up the objects
•Drag and rotate
•Drag and drop
•Hand gesture
•Holding object
VR Interactions research
Designing the interactions
As confirmed with the client in the beginning phrase, all interactions should happen in the Memorial without adding any cutscenes. I learned from the research that having cut scenes will break the immersion of the experience. Cutscenes are also risky in VR as it can make the audience feels disorient if their viewports were constantly switching between cameras angel, causing confusion and possibility of motion sickness. I decided use sound and visual cue to direct the players attention towards the artifacts.
The first aspect I discussed and ideate was artifact placement. The UX team first listed out all possible ways of placing artifacts in the Memorial, and later categorize them into ways for personal scenes and ways for a single historical artifact based on the amount of space needed for each method.
Using research results from the previous phrase, the UX team categorized possible historical artifacts into groups based on their size, shape and materials. I believed artifacts in the same category would share similar interactions. This approach allowed the client to have more freedom to make changes on artifacts without severely affecting our development process as long as changes were made within the same category. I also suggested which categories of artifacts worked better as personal or historical artifacts.
Artifact placement ideas brainstorming
Categories of possible artifacts
Me and another UX designer performed a VR interaction test using a low-fi prototype made by papers and whiteboard. We mimicked the narrow pathways of the Memorial using whiteboards to test whether the possible interactions we brainstormed earlier could be performed in the environment. We verified all our proposed interactions through this test.
User Journey
After identifying the basic direction of the project, we brainstormed possible interactions. We ultimately chose to utilize both personal and historical scenes for the environmental narrative storytelling. We determined the basic interaction logic by distinguishing different scenarios in which the interaction occurs.
In personal scenes, I built a narrative environment in a relatively open space. Not only does the narrative environment include the main artifact that the user needs to interact with, but also includes a richer environment, sounds and other elements to enhance the narrative. Users complete the narrative by exploring and interacting with the whole environment. In historical artifact scenes, I build the artifact on the user's way to personal scenes. The user can choose whether to interact with these historical scenarios and artifacts.
After finalizing artifacts and interactions, I designed a user journey map. In addition to user interactions, I also designed the cue system of the project. The system is intended to better guide users through the overall storyline and to complete interactions. The system consists of two parts: visual cues and sound cues. The visual system includes highlighting and interactive prompt icons. In the above user journey map, I also included expected user emotions and feelings as well as user behaviors.
As shown in user journey, personal artifact (1) is placed at the edge of the Memorial, personal artifact (2) is placed nearer to the centre of the Memorial, and personal artifact (3) is placed the nearest to the middle of the Memorial. Therefore, the scene with personal artifact (1) is the brightest and the one with personal artifact (3) is the darkest.
To spawn personal scenes, some stelae are removed to clear an area for the artifact. Depending on the artifact design, each scene removed a different number of stelae. 5x3 stelae, 3x3 stelae and 6x10 stelae are removed for personal artifact (1), (2) and (3) respectively
Visual Cue System
When designing a cue system, there are many design details for the user experience. Idesigned the moment when visual cues and sound cues appear and disappear. These moments are all triggered by user behaviors and change as the user's behavior is implemented. For example, I envisioned how we could advance the overall interaction progress and push the user to the next interaction point if the user spends too much time in a scene, which could be implemented when the project is further developed.
User Journey & Visual Cue
Artifact dropping locations
User Interface Design
To ensure the user understand how to navigate in the virtual space and which object is interactive, I add a simple tutorial guide at the beginning of the experience and mark all the interactive objects with visual cues. Controls The user can use the joystick on their left controller to control the walking direction and the one on the right to change camera angle. To interact with objects, the user can use the grab button on the side.
This experience is for middle and high school students, depending on their background, they might have different levels of familiarity with VR. Therefore, this VR experience is designed in a way that all interactions can be done using the same button. At the beginning of the VR experience, a tutorial guide is shown to the user explaining the controls of this experience and allowing them to try interacting with both controllers. This ensures less-experienced users understand they sometimes need to use both controllers to interact with objects, which helps users to complete the interactions with personal artifact (1)
All interactive objects in this VR experience are highlighted in yellow and have an icon showing how might the user interact with it.
Tutorial Guide
Icons and Visual Cues
User Testings
To get better feedback, we designed and conducted two user tests. They tested the project from the aspects of interaction mechanism, logic, ease of use, narrative, feelings, and performance. Before the user test, our team discussed the execution of each test and conducts multiple internal tests to ensure we would collect enough effective data during the user test session. I documented the entire process and feedback of the test in detail. Then, I organized the feedback we received to further optimize the experience.
Junyi, my UX partner is preparing the prototype for user testing
User tester reacted to the scene
User Test #1
The objectives of this round of user test are as follows:
•How the interaction experience of historical artifacts is perceived.
•Whether there is a problem with the interaction logic of historical artifacts.
•Whether the current VFX works as expected.
•Whether the sound and highlighting in the cue system works, or if it breaks the immersion.
•The performance of users with different VR proficiencies and experiences to infer the difficulty of each artifact interaction.
After user testing #1, here are the progress we made:
• Included personal artifact (1) and (2) with 3D assets in personal scene (2), which could be test for fidelity
• Updated the highlight effects to give clearer indication of interactive artifacts
• Added icons for visual cues
• Added sound cues for personal artifact(1)
• Rearranged the artifact placements
User Testing #2
We conducted the second user testing with MDM students. During the usability testing, six users tested our prototypes.
Objective
• How artifacts and interactions are perceived.
• Explore potential problems with the interaction logic.
• Whether the user's feeling after completing the personal scene is as expected.
• Whether personal scenes’ interactions and environment meet our narrative requirements.
• Determine the effectiveness of the modified cue system, which includes new UI and particle effects.
• Whether users can tell which artifact is interactive and how to complete the interaction.
• Whether the overall interaction flow is smooth and reasonable, including the connection and narrative coherence between the personal scene and the historical scene.
• Observe the performance of users with different VR proficiencies and experiences to infer the difficulty of each artifact interaction.
• Test the performance of the overall VR device after adding personal scenes and visual effect.
[[cut down objectives, put results UT#2 here ]]
summarize to actionable items - how can you improve it
Lesson learned as UX Designer
• Empathy workshop with Juliana, one of the director, in the beginning inspired the whole team to rethink the way of our thinking process, help us empathize with users, and structure the experience we try to create.
• Having one-on-one meetings with stakeholders was very helpful since feedback and answers to our questions can be received instantly.
• While waiting for the confirmation of content, it was helpful for the team to start with secondary research on the Holocaust history first. The research results I did it at the beginning helped us to speed up the process after we got confirmation.
• Reaching out to other stakeholders and subject matter experts for help and suggestions can speed up the process !
• To avoid conflict, communications within the team is the key!
• Hybrid working mode works very well for our team since we have 2 days focusing on our tasks and 2 on site days for UX brainstorming workshops or bugs fixing for the development team.
• Having stakeholders joining our meetings in person is very important. We also receive feedback on spot, and it is easier to communicate and exchange ideas.
What if I have more time...?
• Let stakeholders try it on VR - Throughout the development process, stakeholders did not have chance to try out the prototype in VR headset due to the shortage of time. We only recorded our progress in a VR headset and presented the video during weekly meetings. It might be beneficial if we arranged testing sessions for stakeholders in between so that they could experience the prototype in headsets.
• More user testing and refinement - We did not have a chance to conduct user testing on personal artifact (3) since approval on the design was only received on week 10. The team had to rely on internal reviews and feedback from UX team to make decisions.
• Integrate other art style into the experience - such as surrealism, and try to integrate them with the experience so we can make the experience more unique and memorable.
