Project Overview
This rapid user research pilot invited users and teachers to explore a new feature the Immerse app was launching, the teacher lesson planning hub. The teacher lesson planning hub was a dashboard that allowed teachers to author changes to VR lesson “on-the-fly.” This afforded teachers the ability to change their lessons in real time and customize their lessons to fit the classroom context.
We conducted a usability test deploying a functional prototype and asked users to fill out a survey immediately after using the lesson planning hub. Our goal was to explore how users might practically use the newly designed lesson planning hub and how we might improve the planning experience for teachers.
User participants
Cohorts
live testing sessions
Hours of video recorded
team members
User Premise
You are a VR instructor and will be creating a lesson with the newly developed lesson planning hub in the Immerse app. There are a variety of pre-built template elements to choose from, or you may decide to create your lesson plan entirely on your own. Try to craft a lesson plan in the next 20 minutes.
During this time, the research team will be collecting data to contextualize your experience with the Immerse platform. We will ask a series of questions to describe your feelings, emotions, affordances, and constraints with scales and open-ended questions. We value your honest opinions, feedback, and journey to help improve the Immerse learning experience.
Goals & Objectives
Intended Use
We wanted to explore users’s intentions with the planning hub and what feature they might actually use.
Improvement
We wanted to identify what elements might be missing so that we can begin to incorporate new features.
Validation
We wanted to validate that the product features, interactions, and the experience was meeting our users’ expectations.
Live Edits
We wanted to determine if making live edits during a lesson was important for teachers while teaching.
Research Process
I led the team in crafting the survey design, questions, and build out on Qualtrics. After users completed the usability session, the survey followed immediately after to capture users thoughts, feelings, and reactions. The affordance of this method allowed direct reflections to tasks users just completed. I analyzed this data by conducting descriptives and frequency analysis. We also analyzed users’ open-ended survey responses to better understand user insights on the planning hub.
1) Methodology
We conducted unmoderated usability testing while users tried out the planning hub followed by a survey with Likert and open-ended questions.
4) Facilitating Testing Sessions
The usability testing sessions were conducted online via Zoom. We had 10 users per cohort and instructed users how to access the prototype. Then, users followed the prompts while the research team observed and documented notes.
2) Recruitment
We selected 113 users from our internal recruitment pool of participants and chose a range of users from beginner to novice VR users spanning from 18 to 50+ years of age.
4) Data Analysis
I analyzed this data by conducting descriptives and frequency analysis. We rewatched the recordings and analyzed the open-ended questions to determine users’ thoughts and reactions on the planning hub. Finally, we triangulated our the quant and qual findings to suggest practical recommendations for product iteration.
3) Survey design
I led the team in survey design and development. I drafted all of the survey instruments and open-ended questions which were reviewed by the UXR lead. I then used Qualtrics to build out the survey to collect the data for this study.
5) Ux Report & Presentation
Lastly, after organizing the data, I created a UX report and slide deck to disseminate the mixed-method findings to my cross-functional partners.
Results
My task was to evaluate what aspects of the teacher planning hub was useful and to what features were usable for teachers to user in their lesson planning. By corroborating our quantitative and qualitative findings, we were able to evaluate the prototype and provide key insights that validated the planning hub designs and uncovered areas for improvement.
Key Insights #1
Lesson Plan Utility (Quant)
Results indicate that 34.62% of users review detailed plans or guides, provided by their institution and make a short list of lesson highlights to reference during a class.
Results indicate that 30.77% of users write detailed lesson plans and that during class, they refer to an annotated list of highlights rather than looking at the full lesson plan.
26.92% of users generally do not write lesson plans, but they do make short lists of lesson notes prior to teaching to provide structure.
Lastly, we see that 7.69% of users reference detailed lesson plans and generally refer to detailed plans when teaching.
In summary, we found few teachers reference the entirety of their detailed lesson plans and that a majority of teachers make use of an abbreviated guide with lesson highlights. A majority of teachers either use highlights provided by their institution or create their own highlights. Others create lesson notes to provide structure to their lessons. These findings shed light on how teachers currently utilize lesson plans in their teaching. Results also let us know that teachers use both templates and their own authored content to guide their lesson flow.
LESSON PLAN UTILITY (Qual)
After analyzing the open-ended questions and focus group data, we coded each of the user responses and categorized each code into overarching themes. After sorting our data, we took a deductive approach when looking at the qualitative results to explain potential reasonings for our quantitative findings. Below is an excerpt of the user commentaries.
| What teachers said about references lesson plans during class. |
|
|
|
From these commentaries, we see that teachers need lesson plans, regardless of who is creating the plan. While most teachers do not write their own lesson plans, those that do include useful details relevant to ensuring their lessons run smoothly. Finally, we also see that teachers rarely refer to detailed plans during a lesson, but do use their notes to stay on task.
Key Insights #2
Lesson Plan Appearance
The data revealed that 96.15% of users prefer to have the lesson plan appear on the right side of the screen where the students are not obscured and the window could be minimized.
| To better understand these results, we looked at what users were saying about their preference for lesson plan view: |
|
|
|
Based on user responses, teachers prefer to have a clear, unobstructed view of the classroom at all times when students are present. However, when planning prior to the lesson, teachers want to have full control over how they author their lessons, with a suite of options and tools to support students’ learning.
Key Insights #3
Lesson Plan Access
More than half the respondents (46%) indicated they do not need access to full lesson plans during a lesson.
A majority of respondents (77%) felt access to lesson plan notes is necessary during a lesson.
96 out of 113 teachers (85%) felt being able to edit content in an activity during a lesson was important.
Most teachers (81%) felt it was important to be able to add content for learners during a lesson.
Overall, teachers felt it was important, or very important, to be able to make changes to content during a lesson. Additionally, teachers felt strongly that it was important, or very important, to be able to add content during a lesson.
Reflection
Having served as the UX researcher on this project, I had the unique opportunity to observe our users pilot a new product that was still in its development stage and identify ways in which research might help to move this feature toward launch. By conducting a usability study followed by an online survey, we were able to capture users’ thoughts, immediate reactions, and understandings as to how they would value a lesson plan hub, why or why they might not use it, and what features were most important to them.
One of my favorite aspects of this project was gleaning users’ value propositions in order to inform the design of the product. Specifically, by understanding what users expect to see in a planning hub, what features they might want to see included next time, and what aspects don’t serve any use. we are able to advocate for design iterations that are user-centric and tailored to our users’ needs.
This project was quite rewarding for me personally as it was the first project I took on in my UX research role, leading our team from beginning to end. It was also rewarding for me to be a part of the research process while the product was still in its early phases to make suggestions that were grounded in research. This experience has taught me the importance of conducting user research at alls stages of the product cycle and how working cross-functionally can lead to an impact on the business.