Project Overview
While working as a user experience researcher at Immerse, the product and Unity game design team finished building the direct-to-consumer (D2C) VR Sign-Up flow and free trial demo experience to be launched on the Meta game store. This project followed the users’ journey while attempting to go through the sign-up flow, create an account, and accomplish the free trial demo experience. Our primary goal was to conduct generative research to gather a deeper understanding of our users and how they might want to interact with the Immerse application.
User participants
team members
hours of video recorded
App Updates
User Premise
You came across an Immerse ad on Instagram, clicked on it and were directed to the website, and got sold on Immerse VR platform. You downloaded the Immerse app because you are interested in learning a language and you are about to go through the 7-day trial experience.
Goals & Objectives
Value Proposition
We wanted to gather insights into how our users might derive value from the Immerse app.
Pain Points
We wanted to uncover critical pain points of the users and identify how we might resolve these constraints to streamline the account creation process before launching on the Meta app store.
Problems & Expectations
We wanted determine what problems users might be having, what expectations are not being met, and how we the product can be iterated.
Validate Affordances
We wanted to determine to uncover what expectations are not being met and how the product can be iterated.
Research Process
The Immerse sign-on flow and free trial demo was in the final stages of development on the product cycle. As such, we wanted to examine how many users were able to successfully or unsuccessfully create an account, attempt the free trial experience or skip the free trial, interact or fail to interact with the 3D objects, and successfully reach the end of the free trial demo.
1) Methodology
We conducted user interviews with a moderated usability interview protocol to conduct generative research.
4) Facilitating Interviews
I conducted and recorded all of the interviews via Zoom while simultaneously using the Oculus Quest 2 headset for users to experience Immerse application. Generative questions were asked throughout the entire session and we encouraged back-and-forth conversations to determine specific pain points.
2) Recruitment
We selected 20 users from our internal recruitment pool of participants and chose a range of users from beginner to novice VR users spanning from 18 to 50+ years of age.
4) Data Analysis
After rewatching the video recordings, documenting the notes, and cleaning the data, I processed all of the participant’s data into a spreadsheet. This allowed me to conduct a thematic analysis on each of the questions asked by grouping similar responses together and assigning child codes, parent codes, and themes to triangulate my findings and identify trends.
3) Interview script development
We then created the interview questions to generate a co-constructed semi-scripted interview protocol.
5) Ux Report & Presentation
Lastly, after I organized this data into a hierarchy, I sorted it by themes and created a UX report and slide deck to disseminate the qualitative findings to my cross-functional partners.
Results
My task was to explain why users were able to successfully complete the sign-up flow and trial experience and further explicate the tensions users faced during the unsuccessful attempts.
Sign-up Flow
In this space, users are presented with the value points of the Immerse app and a login screen to create an account by entering their email and password.
Key Insights #1
Value Affordances Alignment
We found that 75% of users resonated with the last two benefits of the Immerse affordances and value cards at the very beginning [grow to fluency & real-life skills].
Approximately 15% of users resonated with the [approach learning differently], while only 10% of users resonated with [gain an international community].
“These two [far right two] because this is probably what I want to get out of this is whenever you spend time on something there has to be a purpose or goal and like personally for me.
“I like I like that, where it’s like active real life scenes instead of like a classroom. That was interesting period and then.”
“I feel like these points reference the promise on what I will benefit from and I hope to get real-life experiences without traveling.”
“I want something more than practicing on Duolingo and I think VR can help be grow in my language development.”
In summary, we found that the majority of users want to grow in their Spanish language fluency and experience real-life interactions through the affordances of virtual reality. We saw that users value the situated immersion that VR can provide and hope to see how well the Immerse application can deliver its promises.
Buggy virtual Keyboard
During the interviews, we found that the keyboard appeared to be a major issue. Specifically, the shift key to access symbols, missing underscore key, and toggle button to switch back to QWERTY were missing. We also found that the password instructions and requirements were vague. The password had a minimum character requirement, but that wasn’t communicated on-screen or after a failed attempt causing frustrations to many users.
“Felt like a standard account setup process.”
“Oh wait i’m confused, why not, why is it not letting me make my password”
“The “@” button was not working, by the way. I have to click you to gmail or Yahoo.”
“Where’s the underscore, is there no underscore, i don’t see the underscore, that’s a squiggly line, seems to be no underscore.”
As a result, we uncovered some major flaws in the virtual QWERTY keyboard that restricted users from accurately inputting their emails and passwords. This finding was presented to our UNITY game designers as well as engineers in order to fix this bug in the keyboard that halted the sign up process.
Free Trial Demo Walkthrough
In this space, users undergo a tutorial orientation to help acclimate learners to the VR language learning space, learn the interactive mechanics, and explore the virtual world.
Key Insights #2
Game mechanics & Navigability
All users had difficulties understanding the movement options. For some, reading the instructions was very hard to understand. After making their selection and being tasked to move, they required additional assistance.
“Select your movement style, you can always change it movement style later in your settings menu, you can either teleport per slide, which i’m not sure what that means.”
“I’m not sure what I’m doing right now, because it was too quick, so I’m just pushing something forward and I can’t remember now what the other button does and I don’t know if I can go back and figure that out.”
“Not sure about a fully understand the use case of movement styles before understanding when I will be using. Is this like how I moved through a space?”
Based on user responses, we needed to provide additional support on how to move and navigate within the virtual environment. The learning curve for all users was quite high, even more so for first-time users. As a result, we have developed an introduction video of what “teleport” and “slide” actually look like. We also devised and deployed a plan to create an FAQ toggle for users to refer back to at anytime.
Interactive Guidance
All users had some level of trouble learning the VR mechanics with the controllers on how to grab, move, pick up, and select. The instructions written on the cards alone may not be sufficient for users’ understanding of how to perform the action. The instruction cards are often misinterpreted or missed altogether. While the visual gifs were an improvement to help the user, it still was difficult for users to map the on-screen instructions with the physical controls.
“So my question would be then how to customize this, because i can select them here..maybe clothing, hairstyle”
“I’m getting confused as when to use the index finger button and the middle finger button. I think the two buttons makes sense once i’ve grabbed something, but once i’m inside something, I’m not sure which to use.”
“I guess right now i’m just thinking about more how to like use this stuff in the backpack for like the lessons and stuff. I probably want to see [a video] or something and then try it out on my own.”
“I’m looking at now because you told me, too, I think I would have ignored it and then just walk right past and into not room.”
We discovered many tensions from users when attempting to interact with the various objects in the free trial. More specifically, while the instruction cards were shown, the placement of the cards, lack of interactive guidance, counterclockwise directionality, lack of audio, and dynamic movement may have contributed to users’ confusion. As a result, we developed new interactive guided instruction cards with voiceover audio, glowing arrows to guide the user through the scene, and videos demonstrating each of the 3D interactions.
Reflection
By conducting this project from beginning to end, I learned that users’ interpretations of interfaces, instructions, and interactions may differ drastically from the intended use case. When I presented these findings to my cross-functional stakeholders, I felt like I was an advocate for the participants to convey information that I knew would have a direct impact on what features should be added. By doing so, our team knew which areas to target based on data-driven recommendations, culminating in new and updated features being shipped to ultimately improve and deliver a quality experience to our users.
A challenge I faced while conducting this study was being able to include all of the user’s voices in the report. All of the users provided so many rich suggestions to help improve the product; however, given the time constraints for the app launch, I was not able to include all of the recommendations in my report. In order to select which suggestions to include, I looked for overlapping trends in the data and evaluated the frequency of suggested features mentioned to prioritize what was actually feasible in the timeline. In this way, I was able to rigorously consider what immediate actions needed to be made and which recommendations can be queued up.