Update nexus: fix conflicts and sync local changes
This commit is contained in:
Shen Wei
@@ -1,105 +1,105 @@
|
||||
---
|
||||
title: "A/B Testing Framework"
|
||||
type: concept
|
||||
tags: ["optimization", "statistics", "ppc", "creative"]
|
||||
---
|
||||
|
||||
## Definition
|
||||
|
||||
A/B Testing Framework（A/B 测试框架）是创意优化的标准方法论，通过对照实验验证假设，区分真实效果提升与随机波动，以数据驱动决策。
|
||||
|
||||
## Core Principles
|
||||
|
||||
1. **假设驱动（Hypothesis-Driven）**：每个测试始于明确假设
|
||||
2. **控制变量（Single Variable）**：每次只改变一个变量
|
||||
3. **统计显著性（Statistical Significance）**：基于置信区间判断结果可靠性
|
||||
4. **可重复性（Reproducibility）**：测试结果可推广至更大规模
|
||||
|
||||
## Test Design
|
||||
|
||||
### Hypothesis Template
|
||||
|
||||
> "If [change], then [expected outcome], because [reason]."
|
||||
|
||||
示例：
|
||||
> "If we add urgency language to headlines, then CTR will increase by 10%, because scarcity drives action."
|
||||
|
||||
### Sample Size Calculation
|
||||
|
||||
| 转化率 | 最小样本（每变体） | 预估测试周期 |
|
||||
|--------|-------------------|-------------|
|
||||
| 5% | 2,500 | 7-14 天 |
|
||||
| 2% | 6,500 | 14-28 天 |
|
||||
| 1% | 13,000 | 28-56 天 |
|
||||
|
||||
**公式**（简化版）：
|
||||
```
|
||||
n = 16 × σ² / δ²
|
||||
其中：σ = 标准差，δ = 最小可检测差异
|
||||
```
|
||||
|
||||
### Statistical Significance
|
||||
|
||||
| 置信度 | Z-score | 可靠性 |
|
||||
|--------|---------|--------|
|
||||
| 90% | 1.645 | 初步参考 |
|
||||
| 95% | 1.96 | 标准基准 ✅ |
|
||||
| 99% | 2.576 | 高确定性 |
|
||||
|
||||
## Testing Workflow
|
||||
|
||||
```
|
||||
1. Define Hypothesis → 2. Design Test → 3. Launch → 4. Monitor → 5. Analyze → 6. Scale/Iterate
|
||||
```
|
||||
|
||||
### Step 1: Define Hypothesis
|
||||
- 明确要测试的变量（Headline A vs Headline B）
|
||||
- 设定预期提升目标
|
||||
- 确定主要指标（CTR/CVR/CPA）
|
||||
|
||||
### Step 2: Design Test
|
||||
- 流量分配（50/50 或 80/20）
|
||||
- 测试持续时间（2-4 周）
|
||||
- 样本量计算
|
||||
|
||||
### Step 3: Launch
|
||||
- 确保变体间随机分配
|
||||
- 记录测试开始时间
|
||||
- 不在测试期间修改其他变量
|
||||
|
||||
### Step 4: Monitor
|
||||
- 每日检查基本数据
|
||||
- 避免提前终止（除非严重错误）
|
||||
- 监控外部因素（季节性/节假日）
|
||||
|
||||
### Step 5: Analyze
|
||||
- 计算统计显著性
|
||||
- 分析次级指标（CVR/CPA）
|
||||
- 撰写结论报告
|
||||
|
||||
### Step 6: Scale/Iterate
|
||||
- 胜出方案规模化推广
|
||||
- 败出方案归档（积累学习）
|
||||
- 从败出中提取新假设
|
||||
|
||||
## Common Test Types
|
||||
|
||||
| 类型 | 测试内容 | 适用场景 |
|
||||
|------|---------|---------|
|
||||
| Headline Test | 不同标题变体 | RSA 优化 |
|
||||
| CTA Test | 不同行动号召 | 转化率优化 |
|
||||
| Image Test | 不同图片/颜色 | Display/Social |
|
||||
| Landing Page Test | 不同落地页 | 转化路径优化 |
|
||||
| Audience Test | 不同受众 | 定向策略优化 |
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- **统计显著性**：95%+ 置信度
|
||||
- **测试周期**：2-4 周
|
||||
- **最小样本**：每变体至少 1000+ 转化
|
||||
- **Winner Criteria**：显著优于控制组（10%+）
|
||||
|
||||
## Sources
|
||||
|
||||
- [[paid-media-creative-strategist]]
|
||||
- [[paid-media-ppc-strategist]]
|
||||
---
|
||||
title: "A/B Testing Framework"
|
||||
type: concept
|
||||
tags: ["optimization", "statistics", "ppc", "creative"]
|
||||
---
|
||||
|
||||
## Definition
|
||||
|
||||
A/B Testing Framework（A/B 测试框架）是创意优化的标准方法论，通过对照实验验证假设，区分真实效果提升与随机波动，以数据驱动决策。
|
||||
|
||||
## Core Principles
|
||||
|
||||
1. **假设驱动（Hypothesis-Driven）**：每个测试始于明确假设
|
||||
2. **控制变量（Single Variable）**：每次只改变一个变量
|
||||
3. **统计显著性（Statistical Significance）**：基于置信区间判断结果可靠性
|
||||
4. **可重复性（Reproducibility）**：测试结果可推广至更大规模
|
||||
|
||||
## Test Design
|
||||
|
||||
### Hypothesis Template
|
||||
|
||||
> "If [change], then [expected outcome], because [reason]."
|
||||
|
||||
示例：
|
||||
> "If we add urgency language to headlines, then CTR will increase by 10%, because scarcity drives action."
|
||||
|
||||
### Sample Size Calculation
|
||||
|
||||
| 转化率 | 最小样本（每变体） | 预估测试周期 |
|
||||
|--------|-------------------|-------------|
|
||||
| 5% | 2,500 | 7-14 天 |
|
||||
| 2% | 6,500 | 14-28 天 |
|
||||
| 1% | 13,000 | 28-56 天 |
|
||||
|
||||
**公式**（简化版）：
|
||||
```
|
||||
n = 16 × σ² / δ²
|
||||
其中：σ = 标准差，δ = 最小可检测差异
|
||||
```
|
||||
|
||||
### Statistical Significance
|
||||
|
||||
| 置信度 | Z-score | 可靠性 |
|
||||
|--------|---------|--------|
|
||||
| 90% | 1.645 | 初步参考 |
|
||||
| 95% | 1.96 | 标准基准 ✅ |
|
||||
| 99% | 2.576 | 高确定性 |
|
||||
|
||||
## Testing Workflow
|
||||
|
||||
```
|
||||
1. Define Hypothesis → 2. Design Test → 3. Launch → 4. Monitor → 5. Analyze → 6. Scale/Iterate
|
||||
```
|
||||
|
||||
### Step 1: Define Hypothesis
|
||||
- 明确要测试的变量（Headline A vs Headline B）
|
||||
- 设定预期提升目标
|
||||
- 确定主要指标（CTR/CVR/CPA）
|
||||
|
||||
### Step 2: Design Test
|
||||
- 流量分配（50/50 或 80/20）
|
||||
- 测试持续时间（2-4 周）
|
||||
- 样本量计算
|
||||
|
||||
### Step 3: Launch
|
||||
- 确保变体间随机分配
|
||||
- 记录测试开始时间
|
||||
- 不在测试期间修改其他变量
|
||||
|
||||
### Step 4: Monitor
|
||||
- 每日检查基本数据
|
||||
- 避免提前终止（除非严重错误）
|
||||
- 监控外部因素（季节性/节假日）
|
||||
|
||||
### Step 5: Analyze
|
||||
- 计算统计显著性
|
||||
- 分析次级指标（CVR/CPA）
|
||||
- 撰写结论报告
|
||||
|
||||
### Step 6: Scale/Iterate
|
||||
- 胜出方案规模化推广
|
||||
- 败出方案归档（积累学习）
|
||||
- 从败出中提取新假设
|
||||
|
||||
## Common Test Types
|
||||
|
||||
| 类型 | 测试内容 | 适用场景 |
|
||||
|------|---------|---------|
|
||||
| Headline Test | 不同标题变体 | RSA 优化 |
|
||||
| CTA Test | 不同行动号召 | 转化率优化 |
|
||||
| Image Test | 不同图片/颜色 | Display/Social |
|
||||
| Landing Page Test | 不同落地页 | 转化路径优化 |
|
||||
| Audience Test | 不同受众 | 定向策略优化 |
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- **统计显著性**：95%+ 置信度
|
||||
- **测试周期**：2-4 周
|
||||
- **最小样本**：每变体至少 1000+ 转化
|
||||
- **Winner Criteria**：显著优于控制组（10%+）
|
||||
|
||||
## Sources
|
||||
|
||||
- [[paid-media-creative-strategist]]
|
||||
- [[paid-media-ppc-strategist]]
|
||||
|
||||
Reference in New Issue
Block a user