At the end of 2021, the Karoo 2 cycling computer was coming up on it's second anniversary. Our company decided it was time to transition away from the incremental improvement of the product and towards the development of the next generation device. I was tasked to spearhead the process for product discovery of the new product. Up to that point, our team had spent most of our time iterating in short feature-specific epics. We hadn't gone through developing an entirely new SKU in years. So, I went back to proven methods of product development. Over the course of 6 months, we landed on the product requirements of the Karoo 24– the most advanced and intuitive cycling computer to date.

I decided early on to follow a processes inspired by traditional Design Thinking methods popularized by IDEO and the D.School. I modified the process to allow us to follow principals of Jobs-to-be-Done Theory. One of the gaps between product and upper management's understanding of the process had always been trusting that feature sets would reduce risk finding product market fit. Many times I've seen teams using JTBD theory stop short on validating their deliverables with data. I made it my goal to prove the relative market impact of the features we were pursuing. In order to do so, we decided to lean heavily into both qualitative and quantitative measures validating user problems.

To start, we had to define the question we were trying to answer:

What will Karoo24, our next flagship head unit, be when it launches?

To answer that, we had to start by asking: What will Karoo24 be in three years time? At that point, we only knew what needed to succeed with Karoo24 (and what did not):

Karoo24 will be a compelling upgrade from K2.
Karoo24 will be a compelling alternative to Wahoo, Garmin, and other head units
Karoo24 is NOT a compelling product for non-head unit users to get on board

Research Plan Structure

Using a design thinking approach based on research, we hoped to identify most pressing problems that we were well-positioned to solve. We sought out focused interviews as well as a large quantitative dataset to understand all aspects of our riders. We began with the interview sessions to explore all potential problem areas before refining our focus and developing a targeted survey.

Phase 1– Problem Discovery
Moderated Interviews
Annotation & Analysis

Phase 2– Problem Scoping
Quantitative Surveys
Analysis

Phase 3– Feature Discovery
Ideation Workshop

Phase 4– Feature Scoping
Feature Packaging
Internal Release Materials

1 – Problem Discovery

We started by seeking out the riders we expected to see using our product based on personas. Our goal was to uncover the “jobs” these riders are trying to accomplish as cyclists.

Phase 1 Deliverable: Functional job statements and the underlying steps to accomplish that job. This may be unique to each persona or common themes across multiple personas. This answers the question: What jobs are these riders solving for with regards to cycling?

Personas

Before starting our research, we had identified three distinct personas that, by satisfying their needs, would build an attractive product for a broader demographic. While personas are helpful in understanding our target market, they should not be viewed as a definitive or conclusive definition. Rather, they should be seen as a starting point for further product research and exploration.

Interviews

Together, we conducted ten 1.5-hour calls with riders who fell somewhere within the range of our personas. Although we had some jumping-off points, the purpose of the calls was to keep the discussion open-ended. Our goal was to discover new insights and hear perspectives that we hadn't previously considered. As such, we aimed to identify problem areas that had not been discussed during our typical feature-focused research calls.

Annotation & Analysis

With our Zoom recordings in hand, we went about reviewing, tagging, and synthesizing insights. Using this data, we created a board of "job statements" that succinctly describe the tasks and goals our users may seek to accomplish by using our product.

With each job statement, we can map the individual steps required to achieve that job. This helps us write outcome statements, which are customer-defined performance metrics tied to the job-to-be-done. Outcome statements make value creation measurable, controllable and predictable. By the end of this analysis, we recognized 50 jobs spanning the rider experience. This netted us 645 unique outcomes that any one of our riders may face. This is not exhaustive, but we found that it was sufficient to uncover the vast majority of scenarios.

2 – Problem Scoping

After building our list of outcomes our riders use to measure job success, we wanted to figure out where the biggest opportunities lie. The hope was to understand which outcomes were ripe for solutions, and which weren’t valuable enough to work on.

Phase 2 Deliverable: A statistically significant dataset ranking the opportunity for each outcome. This answers the question: What outcomes provide the biggest opportunities to solve?

Survey

To create the survey, we initially narrowed down the top 100 outcomes that our team believed would produce the most significant results in line with our strategic plan. Since the job steps had been written as outcome statements, their consistent format made the survey building straightforward. For each outcome statement we ask 2 questions:

1. How important is that outcome to the rider?
2. How satisfied are they with their current solution?

We enlisted the assistance of Maggie Kay and James Peterson at SRAM to distribute the survey to a diverse group of riders that included owners of Karoo, Garmin, and Wahoo products, as well as SRAM owners and riders of other drivetrain brands. The survey yielded 7,142 submissions. The emails had a 65% open rate (10% higher than SRAM road survey, 45% higher than industry), and 22% click-through (10% higher than SRAM road survey, 20% higher than industry).

Analysis

One of the important metrics calculated from the survey is what we call the opportunity score. Put simply, the opportunity score is the difference in how important an outcome is from how satisfied riders are with their current solution. The bigger the difference, the bigger the opportunity.

Opportunity score can say a lot on it’s own, but it’s not the entire story. So, we created a matrix visualization with importance on the x-axis, and satisfaction on the y-axis. By filtering the data based on the head units that riders use, we can gain valuable insights into our strengths and areas for improvement.We can also group together similar outcomes to understand problem spaces.

By the end of analysis, we had built a powerful dashboard in Google Data Studio that enabled us to analyze the survey data and determine which outcomes and problem areas should be prioritized.

3 – Feature Discovery

Before we could start defining the scope of features for Karoo24, we needed to generate ideas for features. To do this, we assembled a cross-disciplinary team at Hammerhead and organized an ideation workshop using the outcomes.

Phase 3 Deliverable: Concept feature sets that may be practical to build, solving for important problems that users face. This answers the question: Which opportunities are we best positioned to act on?

Workshop

During the workshop, we focused on several high-priority jobs that we had identified, along with their corresponding steps and outcomes. With these as our starting point, we held a series of focused individual and team sessions to brainstorm novel, yet practical solutions that had significant product potential. We welcomed developers, software designers, industrial designers, and key leadership to ideate together. We also invited James Peterson & Matt Schweiker to get input from SRAM stakeholders.

For each “Job Block”, we spent 2 hours following a structured ideation process. We allocated two hours for each "Job Block" and followed a structured ideation process. We selected jobs to ideate against based on two conditions: (1) if one or more particular outcomes related to the job had a very high opportunity score; or (2) if a number of outcomes related to the job had high enough opportunity scores that, together, they were compelling to address. This answered: What does it look like to help our riders perform each job?

With 6 job ideation blocks, we finished up the workshop with 90 unique and impactful ways we could help riders.

4 – Feature Scoping

This stage involved bundling together features into packages that addressed multiple pain points in a sensible way. Our goal was to address at least 60% of the pain points identified in our research, differentiate our product from competitors, and ensure the features were feasible to implement within the product's lifespan.

Phase 4 Deliverable: Answer the question… What is Karoo 24?

Filtering Workshop Outputs

The first step was to identify the outcomes that were most compelling to solve, then select the solutions that we believed were most likely to address those outcomes effectively. Next, we looked at each individual solution, and tried to poke holes. We identified and dropped concepts that failed for technical or practical reasons. We also included an "inspiration section" from our notes highlighting products and services that effectively address the job being done. Finally, we considered our product ecosystem to determine the optimal touch points to incorporate these feature sets. Understanding this is critical to forming any kind of roadmap.

Inputting Survey for Prioritization

We re-categorized features and compared them to the survey, identifying the most important to Highlight, Improve, and Overhaul based on what mattered most to non-Karoo owners. We analyzed the survey results and looked at each outcome individually, comparing how satisfied and important it was to both Karoo and non-Karoo users. Based on this, we identified the strong and weak points of our current product, opportunities for innovation, and potential pitfalls.

• Major Strengths: 15% or higher ∆ satisfaction with Karoo
• Minor Strengths: 10-15% ∆ satisfaction with Karoo
• Major Weaknesses: 15% or higher ∆ satisfaction with a different bike computer
• Minor Weaknesses: 10-15% higher ∆ satisfaction with a different computer
• Innovation Opportunities: No one doing it well, but it’s important to everyone
• Deal Breakers: More important to competitors’ customers, and they are currently much more satisfied than our customers at it. These were things we absolutely had to put effort into for K24
• Potential Pitfalls: Competitors’ customers care about it more, but our customers are satisfied right now.
• Thumbs Down: Non-compelling opportunities. Do not pursue.

SRAM Integration Opportunities

Our generative research didn’t yield any outcomes directly related to our acquiring company, SRAM, and their software platform, AXS. The nature of the job outcome exercise means that generated statements are solution-agnostic. So, we took the two lists of feature suggestions from the AXS Team, stepped through it idea by idea, and determined if they mapped to any goals. We gave a strong YES to about half of the ideas, and only a single clear NO. We had deep doubts about the remaining concepts, so we clarified by figuring out what it would take to overcome them.

We also came up with a few heuristics & guiding principles to navigate our product decisions regarding AXS:

• I. We do not compete with the AXS app
• II. We will solve problems common to connected devices
• III. AXS can and should be comprehensive to AXS products
• IV. We should push users to the AXS app when they need to go there (e.g. Deep-link between AXS & Hammerhead using their respective native platforms, iOS universal linking)

Navigation & SDK Opportunities

We wanted to separately and explicitly consider navigation given our history and reputation asa leading GPS device. We also wanted to tackle the question of SDK head-on, as Karoo 2 had launched without a clear plan for how this would evolve, and we wanted to not repeat that mistake:

• I. We will get users out the door and riding quicker, safer, and with fewer stops
• II. We will lean into on-the-fly navigation, helping in unforeseen changes to the plan
• III. SDK is in for the vision of the product, but not at launch (To bring this to market, we must set up a focused program with dedicated resources)

Putting it all together

We rearranged the structure of our data by prioritizing Property first, followed by Investment Required, and Feature Category. This was based on the combined output of our Ideation Workshops and Survey, which was initially organized primarily by investment requirements. It’s at this step that we added the appropriate Navigation, SDK, and SRAM opportunities identified outside of the workshop/survey process.

For the first time, we had a picture of the problems we’d aim to solve with Karoo24.

Roadmapping

After figuring out the feature set for the lifespan of Karoo24, we had to figure out how to get there. Only through roadmapping could we iterate and understand what Karoo24 would be at launch. Once roadmapping had been finished, we could present the product vision with actionable next steps to the greater Hammerhead and SRAM organizations. We’ve answered:

What is Karoo24, our next flagship head unit, when it launches?