VOGUE APP
Runway Genius
Exclusive to the Vogue app, the Runway Genius quiz is released daily. The game was originally part of the wider app relaunch, but drove high retention rates, and with small improvements continued to engage users. The next step was to make improvements to encourage daily play and competition, ahead of the paywall release on app.
Role
Senior Product Designer
Key collaborators
Vogue Editorial
Product Manager
Engineering
Content Design
Initial state of play
The quiz was a simple slideshow of images, where the user has to guess the designer. A 60 second timer to get through 20 questions.
The quiz was part of the much wider app relaunch, and had shown high engagement, but had potential to be improved.
Quick wins
01 Removing the timer. The game was reviewed as very difficult by users. To test a new approach, the timer was removed and replaced by the user’s score. Completion rate dropped significantly (as was to be expected: everyone previously “completed” the quiz, even if they didn’t reach question 20 in the timeframe), but time spent increased, meaning users spent more time trying to get through the entire quiz.
02 Adding question count. Helping give users a sense of progression through the quiz.
03 Allowing users to play past quizzes. Originally, users could only play today’s quiz. Allowing users to view past quizzes, once they had completed the current quiz, allowed them to stay in ‘game’ mode and stay engaged, increasing time spent.
Creating competition
While the quiz allowed users to share their scores via a share card, it soon became clear users wanted to see how they stacked up against other players. The business wanted to build on the sense of community behind the Vogue app, giving users an exclusive insight to Vogue, alongside the editors.
Working closely with content design, the different messages for users finishing the quiz were updated, depending on their score, meaning they wouldn’t see the same message twice if they played multiple quizzes in one day.
Heightening community also required re-thinking the scoring and streak system. How might we…
• Ensure users are kept engaged without making things too complicated
• Scatter these “moments” through the experience to show the user their achievements?
Here’s some quick interaction and information experiments:
Goals and next steps
01 Increase engagement ahead of the paywall release on the app
02 Identify key areas that could present future opportunities for subscriber value
How might we strengthen retention and loyalty to cultivate subscribers on the app?
Initial insights
Though the quiz was still in it’s infancy at this point, we had a lot of data to look through and understand potential areas of growth. Product and I used this data, alongside workshop feedback, to form the basis of the roadmap and key initiatives to explore further.
01 Completion rates are declining as the quiz is considered 'hard’
03 The Top 100 leaderboard is beneficial for only a small group of users
02 Quizzes, past and present, are hard to find within the app
04 There is potential for this to be a learning opportunity, rather than just a test
Competitor research
The NYT Games app, Duolingo and other quiz apps all provided inspiration and benchmarking for our quiz.
Product and editorial workshops
Working closely with product, we defined key themes with potential for growth and improvement within the Runway Genius world. From editorial workflow, user difficulty modes and community, these themes were deliberately broad to allow the wider product and design teams to brainstorm as many HMWs and ideas as possible to takeaway and refine ahead of editorial workshops.
I crafted multiple explorations ahead of running workshops with key editorial stakeholders within Vogue, which formed the basis of discussion.
Monthly releases
We wanted to release (and learn) fast. Monthly app submissions meant we could break down the priorities from the workshops, focus on one or two changes at a time and keep an eye on the data.
01 Beginner mode
Hypothesis
Providing different difficulty levels will allow more users to enjoy and complete the quiz
Solution
Potential answers are reduced to 2 possibilities in Beginner mode. Users can select their mode on the splash screen, and gain 0.5 points for each correct answer. This meant users couldn’t “cheat” their way up the leaderboard, or share inflated scores with friends.
02 Custom leaderboards
Hypothesis
Creating custom leaderboards would encourage competitive play amongst friends, and allow users to feel like they are affecting their position, and increase ongoing participation
Solution
One of the more complex monthly iterations, which required back and forth with engineering to reduce complexity, this update included showing all users their ranking (rather than just the top 100). Taking inspiration from apps like Strava, which show users where they rank in relation to people just above and below them, as well as the top scorers.
Bi-weekly working sessions with product and engineering were critical here to work through the many layered flows here: onboarding all users, if friends are signed up to the app, if they’re not, if they want to create multiple leaderboards or remove friends, to deliver a seamless journey into Runway Genius and the Vogue app.
03 Quiz discovery
Hypothesis
Making quiz discovery easier will encourage repeat play.
Solution
Creating a specific landing page for quiz content gave all aspects of the improved quiz, and allowed editorial to have more control over the quizzes promoted.
While the archive page already existed, as more and more quizzes were released daily, they became harder to find. Reusing filters from Image Search within the app, filters for quiz type (theme, model) were implemented and tagging was introduced into the editorial workflow.
04 Learning moments
Hypothesis
Educating users on Runway designers will engage and retain those users who find the quiz more difficult
Solution
Increasing visibility of the editors behind the quizzes, and inserting notes in certain questions which educate the user on the designer, context of the image, or editor’s thoughts.
