GrowthPlaybook
A growth experiment needs a decision at the end.
A simple experiment brief that connects a hypothesis, a meaningful measure, and a decision—so every test leaves you with something useful.
It’s easy to finish an experiment with a chart and no decision. A button changed. A campaign ran. Some numbers moved. The team has an update, but nobody is sure what to do next.
A useful experiment starts by describing the decision it is supposed to inform. That doesn’t guarantee a clean answer. It does make the work easier to interpret.
Put the uncertainty in a sentence
Start with something you don’t know, rather than a tactic you want to try.
“We should change the headline” is an activity. “New visitors may not understand which team this product is for” is an uncertainty. You can now ask what evidence would support or weaken that explanation.
A hypothesis connects a proposed change to a reason:
If we make the intended audience explicit in the headline, more relevant visitors may take the next step because they can recognize that the product is for them.
The word may matters. You are writing an expectation to test, not a result you already know.
Choose one primary measure
Pick a measure that relates to the decision. For a headline test, that could be a relevant signup or an intentful visit to the next page, depending on your funnel and available traffic.
Then add a small number of guardrails: measures that might reveal an unwanted consequence. More signups aren’t necessarily better if almost none of the new users match the audience or activate the product.
Define the event precisely. “Engaged” is ambiguous. “Created a workspace and invited a teammate within seven days” is more concrete, if that behavior is meaningful for your product.
Avoid selecting the winning metric after you see the results. It makes it too easy to turn any movement into a success story.
Write the decision rules before the test
Describe the plausible outcomes and what each would mean for your next step:
- The primary measure improves and guardrails hold: consider keeping the change, subject to the strength of the evidence.
- The primary measure improves but quality drops: investigate the tradeoff before expanding the change.
- There is no clear difference: don’t claim equivalence; the test may simply be inconclusive.
- The result contradicts the hypothesis: revise the explanation, rather than just trying another color or headline.
If you plan a randomized experiment, choose a sample-size and analysis approach appropriate to the effect you need to detect. Avoid repeatedly checking the results and stopping at the first attractive number unless your statistical method supports that process.
If traffic is too low for a useful quantitative test, say so. A set of comprehension interviews or a limited qualitative trial can still reduce uncertainty. Just make a narrower claim about what it shows.
Keep a one-page brief
Use this structure before launching:
| Field | What to write |
|---|---|
| Decision | The choice this experiment will help us make |
| Audience | Who is eligible, and who is excluded |
| Hypothesis | The expected behavior change and its proposed reason |
| Change | Exactly what differs from the current experience |
| Primary measure | The event, denominator, and measurement window |
| Guardrails | The outcomes that must not deteriorate |
| Method | How assignment, duration, and analysis will work |
| Decision rules | What would lead us to keep, revise, or stop |
The brief is deliberately small. It should expose gaps in thinking without becoming a project in its own right.
Separate observation from explanation
After the experiment, write two short sections.
First, describe what you observed: the sample, the measured outcomes, the uncertainty, and any implementation problems. Then describe your interpretation: why you think it happened and what competing explanations remain.
This distinction helps when a team revisits a result later. “Trial starts increased in this sample” is an observation. “The audience finally understood our value” is an explanation that may need additional evidence.
Leave the next person a better starting point
Finish with the decision you made and one follow-up question. Keep the brief and result together, including inconclusive and unsuccessful tests.
A useful experiment log isn’t a trophy cabinet. It’s a record of what the team can stop guessing about—and what still deserves a closer look.
A field note from Switchcase. Clear thinking for your next move.
Follow the field notes with RSS