.png)
Project Summary
Project Type
UX Research
Project Deliverables
Duration
February 1st through February 17th 2021
Paper Prototype and Documentation
Team Members
Jordan Cooley, Madelyn Ray, Tyler Nielson
My Role
For this project, I led the brainstorming section, research, and the prototype development process: sketching, iteration, and final prototype.
Project Details
My teammates and I have had prior precedent experiences with the task of having to rate a service or an experience within an app. With the different subjective insights that we shared, a simple question ignited: What is Lyft's current rating system evaluating?
To answer this question, we conducted secondary and primary research to understand the pros and cons of Lyft's rating system with other rating systems and proposed a final paper prototype based on our findings.
Design Process
Phase 1
Conducting Research
Phase 2
Synthesizing the Data
Phase 3
Delivering the Data
Research Question
What is Lyft's current rating system evaluating?
Secondary Research
We started the project by conducting quantitative and qualitative research on the evaluation of Lyft's current rating system. I assumed that the current rating system is a two-way rating system for the driver and the passenger to rate themselves. I was delighted to discover how right I was through the use of the app.
Below are our findings on the current issue and the issue within Lyft's current rating system.
Ride Experience
The five star rating doesn't help riders properly rate their experiences
The Users
Riders don't understand how 4 stars and below could hurt the drivers overall rating score
The issue within Lyft's current rating system
With Great Power, Comes Great Responsibility
“Ratings are a strange amount of powers given to the users that can put the job of the driver in jeopardy."
Riders have a different understanding for the amount of stars given to the drivers
“Riders are not aware of the fact that if a driver continues to earn a rating less than five-stars, they are at risk of losing their job. "
Riders are benefitting more than the drivers
“Lyft's cut of the driver commission is significantly high which means the driver has to depend on the tips, the distance covered and occassional bonuses. ”
Understanding different rating systems
The five-star system
This rating system successfully achieves in finding low-quality products or services when users have a bad experience.
Binary Systems
This is most effective with videos that are on streaming platforms like Youtube or Twitch
Slider System
Frequently used within a survey for research purposes or a persons participation in a group to cover a series of scenarios
Each rating system provides users with a level of "granularity spectrum" which can either be on the higher end of the spectrum or the lower end of the spectrum. The five-star system and the binary rating system fall within the middle end of the spectrum which allows users to spend less time rating a service or a video with less effort required, and not a whole ton of options provided.
However, these two rating systems have their challenges. Lyft, for instance, has the following challenges that we like to focus on:
1
Riders are in disarray about how a rating lower than five stars hurts drivers.
2
Providing drivers with more ways to make more revenue
3
A humanized rating system in which drivers can improve in a category
Interviews
After conducting secondary research, I wanted to obtain a general sense of the experience a user may face daily by using Lyft, the factors they might consider when rating their driver, etc
Before my team and I initiated our primary research, I wrote down some goals we would like to figure out and confirm our findings about the rating system with riders.
Questions asked in the interview include:
-
What do you consider when rating your driver?
-
Authentic realism: These are people striving to earn an income to provide for their families.
-
-
Have you ever come across a bad driver scenario?
-
What would you like to change about the rating system?
-
Do you feel a sense of constraint from the current rating system? i.e. being able to speak your mind
-
Do you feel a sense of constraint from the current rating system? i.e being able to speak your mind based on your experience
User stories
Jake
-
Frequent flyer to New York, and other Metropolitan cities
-
22 years old
-
Uses Lyft/other ride-sharing apps for family travels
-
Conducted in-person
-
He believes that not much thought is put into meticulously rating drivers.
-
He wishes that the rating system has more specific questions that target the experience of the ride (friendliness, safety, energy, etc.)
-
"Pretty generous" in her ratings doesn’t go below 3 stars if it wasn’t a great experience; “rates 5 for excellent drivers and 4 for okay drivers”.
Tyrone
-
Active user during family vacations. Personally use is limited to three times a week
-
28 years old
-
Conducted via a phone call
-
He always gives the drivers 5 stars unless something major happens like a life-threatening event from overspeeding and braking too hard.
-
He mentioned that he has never forgotten to rate his driver before because once you reopen the app, the rating for the driver pops up again.
Jackson
-
Active user during family vacations. Personally use is limited to three times a week
-
28 years old
-
Conducted via a phone call
-
Jackson talked about his frustration of not being able to provide the feedback he wanted.
-
Apps like Uber and Lyft, I feel like they don’t provide a deeper rating ability past 5 stars…and sometimes, I feel like there needs to be more
-
He has had a mix of good and bad experiences, and some in between, but he feels he has not been able to correctly give his feedback on his drivers, especially those who are in-between
Key Takeaways
01
Users seem to have a surface understanding of how to rate their experiences since they consider factors such as safety, friendliness, hospitality, etc
03
Users need a rating system that covers the depth of their experience that is being provided by the driver
02
Users are still not able to realize that if a driver continues to receive a rating lower than 5 stars, the driver is out of a job.
04
Users need a rating system that allows them to rate a journey in the middle
Affinity Diagram
After researching extensively about Lyft's rating system, my team and I categorized every piece of information into pros, cons, and where the rating system could be seen. This activity allowed us to unravel the true meaning of the rating systems in their proper scope and the features that users dislike about them.
Click to enlarge
Click to enlarge
The current rating system for Lyft has failed the drivers in two areas: to provide context on how their rating system should be used, especially for the users who evaluate these drivers on how the ride went, and rewarding these drivers who don’t earn much from short trips; Lyft has also failed the riders who need a middle ground when their experiences were in between good and bad.
Problem Statement
Project Goal
The goal for the team is to replace the current five-star rating system that is hurting the drivers to be seen as people striving to help their families or themselves by the riders who rate them and Lyft who pays them. We must consider the information that our research has provided in order to create a well-rationalized rating system.
Our final deliverables will accomplish this by:
-
A rating system that has a generic meaning shared across both users
-
A reward system to help out the drivers
-
A rating system that measures the riders’ experience of the journey created by the driver into three categories: Safety, Cleanliness, and Friendliness
Early Sketches
Listed below are our thoughts on what we like our sketches to depict:
-
understanding thoroughly who is being rated: the ride or driver,
-
how we could incentivize the drivers to help out these drivers
-
how could also prevent the drivers from being cut off when their rating is below a 4.7 overall rating score
This is tentatively the direction we're going to pursue:
-
two or more categories that the riders can rate instead of the driver
-
the driver's personality is important for not only safety but to know a little about your driver
This rating system meets the need of users that weren't satisfied with their experience
Up to now our focus has been merely helping the riders understanding the rating system. The image above is an idea that I thought about to incentive drivers to make more money.
Critique and Refined Sketches
The purpose of the group critique is for our classmates to pour scorn on our project ideas and shed light on the areas that are proving to be an obstacle or obstacles that is preventing us from moving forward to the next stage. My teammates and I were facing designer-block so this critique session was extremely beneficial for us.
We faced the issue of creating an entirely new rating system that will be easy to use and convenient without both users creating two different interpretations of the new system that will provide obscurity of what is actually being rated. A classmate suggested that we should combine the best aspects of sketches 1 & 2 and then later connect the prototype to the reward system.
Approach
-
From Sketch 1, we kept the categories, adding a “cleanliness” category
-
These category ideas came from our interviews, and we determined the top 3 categories by importance.
-
-
The more talkative/less talkative question in the optional section is based on the introvert/extrovert button, but we found that using the phrases “more talkative” and “less talkative” evokes no bias while “introvert” and “extrovert” have negative and positive connotations respectively;
-
Our intention for this question is for riders with preferences to be able to choose a talking level they want (inspired by interviews), and not judge the driver for being introverted or extroverted, which is why we changed the wording.
-
Result
Emojis are an easily understandable form of communication for the riders to use to rate the experiences that the driver is providing them by using Lyft's services
Testing
Now that our new rating system has been established, my teammates and I decided to conduct usability testing.
Before we started, we brainstormed goals and tasks that we wanted to achieve at the end of our evaluation for the features of our application that might plausibly need changing. Some of these goals include:
In order for us to have useful insights at the end of the session, we decided to create some real-life scenarios to see how users would interact with our new rating system.
Takeaways
-
Our takeaways from usability testing were that our system had a “nice layout, and it’s very clear.”
-
The pie charts were a favorite aspect as they “provide a clear explanation of what [drivers are] lacking and excelling in.”
-
It was clear to users that riders and drivers can choose areas to improve on simply by looking at which categories have the reddest and yellow in their charts.
-
The biggest issue users had with our rating system was that the faces weren’t expressive enough, as one user wanted to “add angry eyebrows to the red face to get the point across.
-
-
-
Another user was “confused about what the optional box was asking [him] to do,” and suggested that we add questions for more clarification (such as “Would you say your driver. . . Wears Mask or Doesn’t Wear Mask?).
-
All users prioritized the categories in this order from most important to least important: Safety, cleanliness, and friendliness; this confirms that we have the right categories in the right order.
-
Solution
New rating system
Based on this feedback from Usability testing and Interviews:
-
they made changes to the design, including adding eyebrows to an angry face
-
Using a 3-face rating system that combines the benefits of a 5-star and binary rating system.
-
We also made changes to the optional box display to make it more clear and more user-friendly.
The goal of these changes was to create an interface that is easy to use and understand for users.
Process of Choosing A Driver
-
This screen was added to allow users to easily see and compare the ratings of available drivers, including their majority rating for each category, before choosing a driver for a ride.
-
Users can also view a report of a driver's pie charts and optional box results by clicking the "Driver Profile" button. This feature is useful for users who want a quick overview of a driver's ratings or who want to prioritize certain ratings when choosing a driver.
Driver's Profile
-
The driver profile screen has been redesigned to be in a vertical format, which makes it easier for drivers to compare their ratings across categories and for users to see optional box rating results for drivers who have more specific preferences.
-
The new format also makes the driver profile feel more like a "stats" screen in a videogame, which is intended to make the ratings feel more lighthearted for drivers and users.
Reward System
A reward system sketch was created to encourage more ratings and reward drivers for using Lyft more.
-
The sketch includes "hotspots" in a city where users and drivers can earn points by starting or ending rides, and displays the number of points needed to win a prize.
-
The prizes also apply to drivers, and the system is based on Lyft's current reward system, which gives users Delta Skymiles and Hilton Honors points.
Conclusion
Reflection
At the beginning of the project, I wasn't sure exactly what the team and I should focus on until we came across the score range that the drivers must maintain in to remain driving for Lyft.
Moreover, I am happy with our final deliverables with tackled the challenges that surfaced from conducting secondary research about what is Lyft's current rating system rating. However, I wish we also considered having an overall rating score, the process of being connected with a driver, and testing our new rating system with drivers who are the primary focus of this project.