Use when you want to test something in an email programme and need the result to mean something.
You are an email experimentation analyst. You are designing and reading one test, not writing email copy.
What I want to test, and the two versions: {{TEST_VARIANTS}}
Metric I will judge it on: {{PRIMARY_METRIC}}
Segment and how many addresses it holds: {{SEGMENT_AND_SIZE}}
Baseline rate for that metric over recent sends: {{BASELINE_RATE}}
Smallest difference that would change what I do next: {{MINIMUM_USEFUL_LIFT}}
Output four numbered sections.
1. Test design: what varies, what is held constant, split method, send window, and anything in {{TEST_VARIANTS}} that changes more than one thing at once.
2. A feasibility table: Metric | Baseline from {{BASELINE_RATE}} | Target after {{MINIMUM_USEFUL_LIFT}} | Sample needed per arm | Sample available from {{SEGMENT_AND_SIZE}} | Verdict, powered or underpowered.
3. Stopping rule: when to look, when to call it, and how long to wait for late opens and clicks.
4. Read-out template: the sentences to write once results land, with blanks where the numbers go.
Rules:
- If {{SEGMENT_AND_SIZE}} is too small to detect {{MINIMUM_USEFUL_LIFT}} on {{PRIMARY_METRIC}}, say so before anything else and give the lift the sample could actually detect.
- Where {{PRIMARY_METRIC}} is opens, say plainly that privacy proxies inflate them, and name the click or downstream metric to use instead.
- Show the arithmetic behind every sample figure. Do not assert a number without the working.
Replace each placeholder with your own detail. The more specific you are, the less the model invents.
Why plan the test rather than just send two versions and compare?
Because a split with no sizing check usually returns a difference too small to trust, and you then act on noise. Run this before the send, since the feasibility verdict may tell you the segment cannot detect the lift you care about at all, which changes what you do next.
What numbers do I need before designing the test?
The two versions, the metric you will judge on, the segment and its size, a baseline rate from recent sends, and the smallest difference that would change your decision. Decide that last number before you look at anything, or it becomes whatever the result happened to be.
What do the four sections give me?
A test design, a feasibility table showing sample needed per arm against sample available with a powered or underpowered verdict, a stopping rule, and a read-out template with blanks for the numbers. The verdict is the output that matters. Everything after it assumes the test was powered.
What should I check rather than accept?
The sample size arithmetic. Models get these calculations wrong often enough to matter, which is why the prompt demands the working be shown. Note too that privacy proxies inflate opens, so a test judged on opens partly measures image loading rather than reader interest.