Writing a good hypothesis

Every experiment should start with a hypothesis: a clear statement of what you will change, what you expect to happen and why. A good hypothesis keeps the test focused and makes the result meaningful whichever way it goes.

A simple template

Because we observed [evidence],
we believe that [change]
for [audience]
will cause [effect on metric].
We will know this is true when [primary metric]
changes by at least [minimum effect] over [duration].

An example

Because session recordings show many visitors scrolling
past the plan comparison without choosing, and 6 of 8
interviewees did not notice the annual discount,
we believe that showing the annual saving directly on the
plan buttons
for new visitors on the pricing page
will increase trial sign ups.
We will know this is true when the trial sign up rate
increases by at least 10 percent relative (4.0 to 4.4
percent) over three weeks.

Why each part matters

• Evidence shows the idea is grounded in something real. Tests based on evidence succeed more often than random ideas.
• The change must be specific enough that anyone could build it the same way.
• The audience defines who is included. New visitors and returning customers may react very differently.
• The metric and minimum effect define success before you see data, which protects you from moving the goalposts.

Link to a decision

Add one more line: what you will do with each outcome. "If the variant wins, we roll it out and test the same idea in onboarding emails. If there is no difference, we keep the current page and investigate the plan comparison layout instead." If no outcome would change what you do, the test is not worth running.

Avoid hypothesis soup

A common mistake is changing many things at once: a new headline, new colours, new images and a new form. If it wins, you do not know why, and you cannot apply the lesson elsewhere. Sometimes a bold redesign test is the right choice, but do it knowingly and accept that you learn less about individual elements.

Directional thinking

Be clear about what you expect, but also consider how the change might hurt. Showing an annual saving prominently might increase trials but shift people towards annual plans that some later regret, increasing refunds. Thinking this through leads directly to guardrail metrics, the topic of the next lesson.

Write a hypothesis for the shorter sign up form idea using the template.