ADS & LANDING PAGES

A/B test plan for a landing page

Use when you want to test a page change and need a plan that will actually produce a readable result.

ab-test-plan-for-a-landing-page.md

Download .md

You are a conversion optimization lead writing a test plan that will produce a decision, not a debate.

Page and its job: {{PAGE_AND_JOB}}
Current conversion rate and monthly conversions: {{CURRENT_PERFORMANCE}}
Monthly traffic to this page: {{MONTHLY_TRAFFIC}}
Change I want to test: {{PROPOSED_CHANGE}}
Evidence that prompted the idea: {{EVIDENCE}}
Tooling available: {{TOOLING}}

Produce:
1. A one sentence hypothesis in the form: because we saw X, we believe changing Y will cause Z, measured by M.
2. Honest feasibility: using my traffic and conversion numbers, say roughly how long the test would need to run to detect a 10 percent, 20 percent and 30 percent relative lift. Show the assumptions you used. If you cannot calculate this from what I supplied, say exactly which number is missing rather than estimating.
3. Variant definition: what changes, what stays identical, and what must not be touched mid-test.
4. Primary metric, two secondary metrics, and one guardrail metric that would make you stop the test.
5. Decision rules agreed in advance: what result means ship, what means kill, what means inconclusive.
6. Three reasons this test could produce a misleading result and how to prevent each.

Rules:
- Do not claim statistical significance thresholds you cannot justify from the inputs.
- If the traffic is too low for a readable test, say so plainly and suggest a bigger change or a different method.
- No em dashes.

Fill in before running

Replace each placeholder with your own detail. The more specific you are, the less the model invents.

{{PAGE_AND_JOB}}
{{CURRENT_PERFORMANCE}}
{{MONTHLY_TRAFFIC}}
{{PROPOSED_CHANGE}}
{{EVIDENCE}}
{{TOOLING}}

Getting a better result

If the runtime comes back over six weeks, test a bolder change instead.
Write the decision rules before launch or you will rationalise the result afterwards.
Keep one guardrail metric like lead quality, not just conversion rate.

Questions about this prompt

When should I plan a test rather than just ship the change?

When somebody will later ask whether the result was real. If the change is obviously an improvement and your traffic is thin, shipping it and watching is often the better call, and this prompt will tell you so rather than manufacturing a test. Use it when the decision is contested or the change is expensive to reverse.

What do I need before running it?

Real numbers: monthly traffic to the page, current conversion rate and monthly conversions. The runtime estimate rests entirely on those, and if one is missing the prompt names the missing figure rather than estimating around it. Bring the evidence that prompted the idea too, since a hypothesis with no observed X behind it is a preference.

What does it return?

A hypothesis in the because, believe, measured by form, runtimes for a ten, twenty and thirty per cent relative lift, the variant definition, primary, secondary and guardrail metrics, decision rules and three ways the test could mislead. The runtimes decide whether to proceed at all. Over roughly six weeks, test a bolder change instead.

What is the mistake?

Leaving the decision rules until the result arrives. Everyone rationalises an inconclusive test into a win once they have seen the numbers, and ship, kill and inconclusive thresholds written afterwards are worthless. Keep the guardrail metric as well: a variant that lifts form fills and drops lead quality is a loss your primary metric calls a win.