Usability Evaluation
Discover usability issues with a social-networking app prior to product launch
📌 PROJECT SCOPE
Client: POP, a local startup
Timeframe: 12 weeks
My Role: UX Researcher
Team: Mia Eltiste, Metinee Ding, Ayu Larasati, Janani Ravikumar
Methods: Usability Testing, Prototyping, Heuristic Evaluation, Competitive Analysis, Subject-Matter-Expert Interview
Tools: Figma, Miro, Airtable, Zoom
Project Overview
🚀 CLIENT KICKOFF
The POP app facilitates friendships based on shared interests for University students. POP’s long-term goal was to improve the mental health of university students by providing meaningful social connections.
🔎 OBJECTIVES
Understand the pain points of students and their social network at college
Assess the usability and user experience of the POP app
Prioritize recommendations so POP can address users’ deepest pain points
✏️ NOTES
Student project conducted in grad school.
At the time of our assessment, POP had a working prototype in Figma where we conducted our heuristic evaluation as a team. Shortly after, POP released their app on TestFlight and has since launched at the University of Texas at Austin.
Methodology
Heuristic Evaluation
Competitive Analysis
User Interviews
Affinity Diagram
Data Analysis
📋 Heuristic Evaluation
The founder already designed out all their screens and had a working prototype in Figma. Our team started with a Heuristic Evaluation so we could hand off some things to work on while the rest of the project progressed.
We evaluated two different “sections” of POP’s app:
The onboarding process that every user will go through one time to create their account
The matchmaking and chatting process that users will go through repeatedly as they interact with the app
Our team went through the prototype together and took notes using Airtable so we could tag notes by the possible 10 heuristics violated and mark the level of severity.
One other team member and I documented all the findings into one slide deck. We included the relevant screen, what heuristic(s) it violated, the severity of the issue, and our team’s recommended changes.
Heuristic evaluation findings
The most heuristic violations were:
Visibility of system status (29 issues)
Consistency and standards (18 issues)
User control and freedom (18 issues).
🏅 Competitive Analysis
Our team looked at typical matchmaking apps like Bumble. At the time of the study, they were the only dating app offering platonic connections. We compared POP, Bumble, Meetup, Friender, Nextdoor, and Meet my dog.
Competitive analysis findings
The competitive analysis showed POP had a unique proposition but a very similar idea was previously tried and saw limited success. We recommended research on the desirability of the app and assessing product-market fit should be major priorities.
👥 User Interviews
Methodology
Screener - We selected a diverse group of students that varied in their age (college freshmen-seniors), their residency (in & out - of state, international students), and transfer students.
Remote Moderation - This project was started in January 2020 and so our team had to move our interviews to be conducted online using Zoom.
The Process
We chose six tasks for users to do during the usability test
Since we were a team of four, we all covered four sessions, two moderating and two sessions of note-taking
We created an interview script, a deck for users to reference tasks and likert scale choices, and a codebook for the notetaker
🗂️ Affinity Diagram
The Process
We grouped things by which task/screen it was associated with. I made sure each sticky referenced the task and participant id before importing them to miro
Within each task, we grouped things by their impact: positive, negative, and neutral.
After discovering themes across multiple interviews, we worked together to synthesize the data and agreed on insights. From these insights, we created our first suggestions that would later become more polished recommendations for the POP team.
📊 Data Analysis
The Findings
POP was a fairly usable app since it was pretty similar to matchmaking apps. Since POP’s target user was young university students, most were pretty advanced tech users.
Users struggled most with new features that were different from other matchmaking app conventions.
Usability and user interface issues were minor inconveniences for users. Users struggled with the purpose of the app.
💭 Reflection
Industry Takeaways
POP was a fairly usable app so our recommendations to improve the UI/UX felt nonessential. The problems we discovered during our user interviews were rooted in problems with the desirability of the app and not understanding users’ real pain points.
This project demonstrated the limitations of evaluative research when generative research was not done to assess the initial product-market fit.
Personal Takeaways
Coming from a startup background, I recognized some problems with POP’s app that are traditionally outside of a UX researcher’s scope, like appropriate revenue streams.