(2022-09-27) Cohen The Iterative-Hypothesis Customer Development Method

Jason Cohen: The Iterative-Hypothesis customer development method. The goal is to uncover the truth, not to sell

If you don’t come away knowing something new and actionable at the end of the interview, you’ve wasted your time and theirs.

your mindset should be: “What does this person know, that invalidates something I thought was true?”

The Process

1. Goals: What you’re trying to learn

Decide on your list of goals first, as they will drive the content of your interviews.

2. Hypotheses: What you currently think the answers are

The first reason to do this comes from the literature on the science of predictions. Specifically, people are much better and more objective at seeking the truth when they’re forced to record their predictions—for example as formal “bets”—and observe how reality confirms or clashes with those bets

The second reason to write down hypotheses is that they will lead us to great questions for the interview.

Here’s an example list of hypotheses I had about WordPress hosting in 2009 before I started WP Engine

I found that half of these hypotheses were wrong. The ones that were correct still needed to be tuned in detail.

3. Questions: What you ask during the interview

write one question per hypothesis. Sometimes one question can cover a few hypotheses, if they’re closely related.

Questions must be open-ended.

On “security:” Do you ever think about website security? If so, how do you think about that? Do you do anything about it today?

4. Iterate the hypotheses

take notes in the spreadsheet, in a new column, next to each question (which in turn is next to each hypothesis)

If you hear anything surprising, ask follow-up questions. Surprise means you’re learning

Also create new hypotheses (and associated questions) based on new learnings

5. Stop when it’s boring

A typical mistake is to do three interviews, and then stop because “I’m not learning anything.” With so little input, you might not be genuinely seeking to learn

Here are more tips.

Emergent segmentation (market segment)

You might notice that customers are segmented.

In this case, it’s useful to make a note of the segments you think exist. First, write hypotheses and questions that you believe are the determining characteristics of the segments. You’ll ask these at the top of the call. Then, keep separate spreadsheets of hypotheses, one per segment

Discuss price

In early interviews for WP Engine, near the end of the call I would float a price of $50/mo for a service that made their website faster, more scalable, more secure, and came with genuinely good customer service. The responses were immediate, emotional, and vehement. One group was shocked—shocked—that the price tag would be so high;

Another group said they would only buy the service if it were much more expensive, because otherwise they know it couldn’t possibly fulfill its promises

I think quoting a specific price is an acceptable breach of protocol. Putting a specific price in front of people elicits a strong, visceral response

You can also tie pricing questions into your other questions, to test whether the person really values that topic.

This worked in practice—most bloggers said security is important to them, but they wouldn’t pay extra to have more of it.

Expect contradictions

You’re going to get all sorts of contradictory signals.

Some of your hypotheses will end up reflecting the variation.

Knowing what patterns don’t exist prevents you from making false assumptions.

Real patterns will stand out from that noise; that’s your fundamental truth, that you can build products and strategies around. There might not be much of it. All the more reason to highlight it.

Everyone “knows” what everyone else wants, except they don’t

starts talking on behalf of other people.

when you see all the crazy, different things people think, you realize that it takes tons of interviews to uncover even a modicum of truth. The person you’re interviewing hasn’t done that, so they don’t know the truth.

So be polite, calmly tell them that’s great insight, reinterpret “everyone else wants X” to either mean “I want X” or just throw out the comment completely, and redirect the conversation back to themselves and their own specific situation.