Problem:

At Coursera, the existing homepage had not been revamped in many years, during which time siloed growth and marketing teams had tacked on new features according to their needs at the time. This eroded the original vision and story of Coursera. Attempts to update the homepage were started and scrapped again and again and teams could not align on a single story for the page.

In addition, user research showed new learners did not understand what the company did or what types of programs you could learn on the platform. In fact, many learners were surprised and delighted by the depth and breadth of what was offered once it was explained - a huge gap to attract different learner segments. Several UX hotspots were also identified.

Opportunity

Despite these issues, the homepage was the first interaction with Coursera for a majority of visitors as well as the single page with the attributing to the highest number of new learners as well as driving a majority of the Coursera for Business new leads. Improvements to the homepage could thus have a huge impact on all business lines.

As Growth leader Brian Balfour at Reforge has said, if your users forms “a great first impression you have tons of opportunity to leverage that into long-term retention and habits.”

Approach

We tackled the homepage in three main phases: alignment, functional update, story (qualitative) update.

Phase 1 was the alignment phase. Given the past problems coming together as a single team, it was imperative cross-functional stakeholders were aligned - product, data science, engineering, design, user research, marketing, partnerships, sales, and more. Having leads in marketing, design, user research, and product focus on areas such as competitive research, past test results and research, as well as design principles of current and past homepages allowed us focus the conversation on key aspects of the page. Creating a time box for critical discussions - such as the goals and success metrics - ensured that we came to a conclusion at the end of the discussion. Another win was pulling in stakeholders across all impacted business lines (which was most of the company) to ensure we weren’t missing critical insight and had buy-in on timelines and goals.

Phase 2 was the functional testing phase. Here we focused on pieces of the homepage that directly led to a user taking an action. This included the CTAs, footer links, and hero navigation. By focusing on the areas that drove action, we could isolate parts of the page and run smaller tests in order to understand the impact of the features. We ended up with many learnings on types of photos to use, language and more.

Phase 3 was the story (qualitative) update. While driving registration or discovery was critical to our business success, we had a hypothesis that telling the story of what Coursera is and what it could do for new learners was key. By keeping this “story” test separate from our CTA test we could add a numerical value to what this story represented to the business. After two rounds of user research and many rounds of internal feedback, came up with three different variants which added or subtracted elements of the story in order to see what was most effective. We were surprised and delighted to find out that the original and longest story was also most effective.

Results

Our updates improved annual registrations numbers by over 4% (over 12% relative), with the greatest impact coming from updates to the mobile CTA and adding the qualitative story.

Learnings

With a homepage that impacted the entire business, we received an incredible amount of feedback from key stakeholders and executives. By implementing Asana’s “Do, Try, Consider” feedback framework we were able to easily digest the feedback on what was something we MUST do vs. something we should think over and discard as we saw fit.