Cadastre-se grátis

Design an A/B test results workbook

Autor: AILesson9 min de configuraçãoTestado com:ChatGPTRevisado em: 2026-08-28

Resposta rápida

Structure assignment, exposure, outcomes, exclusions, uncertainty, and decision rules without overstating a test. Forneça: Experiment design, Data schema and counts, Metrics and analysis rules. Resultado esperado: An auditable experiment workbook with metric definitions, formulas, balance checks, and decision notes.

1

Adicione seu contexto

Seu texto permanece neste navegador. O AILesson Prompts não o envia para um modelo ou servidor.

2

Seu prompt

Campos não preenchidos permanecem visíveis como marcadores de posição, para que você ainda possa copiar e editar o prompt

Design an Excel workbook for the supplied A/B test without turning association into a stronger causal claim than the design supports.

Hypothesis, variants, randomization, eligibility, dates, stopping, exclusions, and owner:
[design]

Fields, grain, IDs, assignment, exposure, outcomes, timestamps, missingness, duplicates, counts, and samples:
[data]

Metrics, denominators, population, confidence method, practical threshold, segments, multiplicity, and review limits:
[analysis]

Separate experiment plan from observed results. Define one row per randomization unit or a reproducible aggregation from event data. Preserve assigned group even when exposure fails for the primary intention-to-treat view; label any per-protocol analysis as secondary. Specify formulas for denominator, rate or continuous outcome, absolute and relative difference, uncertainty, and guardrails. Add sample-ratio, eligibility, duplicate-ID, missing-outcome, timing, contamination, and pre-experiment balance checks. Do not choose a winner from p-value alone, peek without the planned stopping rule, infer user-level effects from event rows, or claim causality when assignment was not randomized. Return: sheet architecture; data dictionary; metric registry; formulas using supplied cells/tables; quality checks; results table; segment policy; decision template; limitations; and tests that recalculate a small known example.
Experimente no Playground
Privado por padrãoA montagem do prompt acontece localmente no seu navegador. Evite colocar informações confidenciais em qualquer serviço de IA, a menos que sua organização permita.

Da entrada ao resultado

Um exemplo prático

Veja como um contexto concreto transforma esta receita em um resultado útil

Entrada real

Experiment design
Randomized email subject test for eligible newsletter subscribers. Control A is current subject, B is shorter subject. Randomization unit subscriber ID, 1:1 assignment on 3 August 2026 before send. One email per eligible subscriber. Primary hypothesis: B increases unique click-through within 72 hours. Run once to the full eligible list; no early stopping. Exclude only prelisted internal/test IDs and hard bounces under the approved metric rule. Marketing lead owns decision.
Data schema and counts
Raw event export can contain many rows per subscriber. Fields SubscriberID, AssignedVariant, Eligible, Sent, HardBounce, UniqueClick72h, Unsubscribe72h, Revenue72h, AssignmentTime, SendTime. Aggregate to one row per SubscriberID. Expected assigned counts A=5,012, B=4,988. Hard bounces A=102, B=98. Unique clicks A=421, B=455. Unsubscribes A=23, B=31. Revenue is skewed and missing for 14 rows because tracking failed. Duplicate SubscriberID in assignment is invalid. Exposure crossover can be detected if sent subject differs from assignment.
Metrics and analysis rules
Primary unique CTR denominator is sent minus hard bounce, by assigned variant; record that this differs from pure all-assigned ITT and provide all-assigned sensitivity. Primary test two-sided 95% confidence interval for difference in proportions. Practical threshold +0.5 percentage points absolute. Guardrail unsubscribe rate denominator sent, non-inferiority margin +0.2 percentage points. Revenue secondary descriptive only. Segments exploratory with n and no winner claims; no multiplicity adjustment planned.

Exemplo de saída

Workbook: Plan (locked hypothesis, population, stopping and thresholds), UnitData (one row per SubscriberID), MetricRegistry, QualityChecks, Results, and DecisionLog. UnitData must be aggregated from events with explicit MAX rules for binary outcomes and a duplicate-assignment stop; never count clicks as independent rows.

Primary denominators are A 5,012−102=4,910 and B 4,988−98=4,890. CTRs are A 421/4,910=8.574% and B 455/4,890=9.305%; absolute difference B−A=0.731 percentage points and relative lift about 8.53%. Excel rate formulas can use =SUMIFS(UnitData[UniqueClick72h],UnitData[AssignedVariant],A2)/SUMIFS(UnitData[PrimaryDenominatorFlag],UnitData[AssignedVariant],A2). Use a preapproved two-proportion confidence calculation or reviewed analysis tool; do not improvise the statistical method after seeing results. The point estimate exceeds the +0.5 pp practical threshold, but the decision also requires the interval and guardrail.

Guardrail: A unsubscribe 23/5,012=0.459%; B 31/4,988=0.622%; difference +0.163 pp, below but close to the +0.2 pp non-inferiority margin. Report its interval before calling the guardrail met. QualityChecks: assigned split 50.12%/49.88%; compare with planned allocation using the approved sample-ratio check; zero duplicate assignments; all assignments predate send; list crossover, ineligible, unsent, missing tracking, and exclusions by group. All-assigned sensitivity CTRs are 8.400% versus 9.122%. Revenue remains descriptive with n/missingness and median or quantiles, not a causal success claim. Decision template: proceed, hold, or reject, citing primary interval, practical threshold, guardrail interval, quality issues, and any divergence between denominator definitions.

Por que isso funciona

  1. 1

    Keeping the randomization unit and denominator explicit prevents event volume from masquerading as independent sample size.

  2. 2

    Decision thresholds and guardrails keep statistical uncertainty connected to practical consequences.

Verifique o resultado

  • Does each analysis row represent the randomization unit, with assignment retained regardless of exposure?

  • Are primary metric, denominator, uncertainty method, stopping rule, and practical threshold defined before reading results?

  • Are imbalance, missingness, contamination, multiplicity, and guardrails visible beside the result?

Use com confiança

Perguntas frequentes

Respostas práticas sobre quando usar esta receita, o que fornecer e onde a revisão humana ainda é importante

What should I prepare before using “Design an A/B test results workbook”?

For “Design an A/B test results workbook,” prepare Experiment design, Data schema and counts, and Metrics and analysis rules. Replace placeholders only with information you can verify. If a detail is unknown, preserve that uncertainty explicitly instead of asking the model to infer it.

When is the “Design an A/B test results workbook” result not ready to use?

The result is not ready if it does not yet deliver the stated outcome—An auditable experiment workbook with metric definitions, formulas, balance checks, and decision notes—from the supplied evidence, or if it relies on unresolved assumptions, missing approvals, or invented details. Use the checks as release gates: revise the source inputs or assign a named, authorized reviewer instead of polishing an unsupported output.

Which AI tools have recorded tests for “Design an A/B test results workbook”?

The published test record for “Design an A/B test results workbook” lists ChatGPT as of 2026-08-28. This confirms recorded runs, not guaranteed compatibility or identical results in later product versions. For another tool or version, keep every constraint visible and repeat the result checks before use.

Mais maneiras de explorar

Onde esta receita se encaixa

Prompts de textoAnálise de dadosExcel e planilhas

Mantenha o trabalho em andamento

AILesson · Cursos recomendados

Seu próximo passo: use a IA na prática

Passe de entender a IA para fazer as coisas acontecerem. Pratique como criar prompts, revisar e melhorar resultados por meio de lições interativas para o trabalho e o dia a dia.

Explore todos os cursos
Capa de IA para iniciantes

15 lições interativas

IA para iniciantes

Inicie uma conversa útil com IA, melhore a resposta, confira o que importa e saiba quando o próximo passo é seu.

Explore este curso
Capa de Fundamentos do ChatGPT

10 lições interativas

Fundamentos do ChatGPT

Aprenda a usar o ChatGPT por meio de três fluxos de trabalho práticos: conclua uma primeira conversa, trabalhe a partir de um arquivo e use Projects para levar um plano verificado a uma nova conversa.

Explore este curso
Capa de Como a IA generativa funciona

7 lições interativas

Como a IA generativa funciona

Entenda como chats comuns de IA aprendem padrões da linguagem, usam mensagens atuais, criam respostas e incorporam fontes e ferramentas.

Explore este curso