The Burstiness Engine
The results
Prompting moves rhythm.
Where it lands depends on the model.
Explore five prompting modes against the same human reference. The important differences are in the direction, size and uncertainty of each gap.
Explore the benchmark
Change the prompt. Watch the pattern change.
The aim problem
One prompt. Nine landings.
Without an explicit rhythm instruction, the baseline gaps already differ.
- DeepSeek V4 Pro+0.03595% bootstrap interval +0.011 to +0.053. Control +0.035.
- Gemini 3.1 Pro+0.12895% bootstrap interval +0.102 to +0.140. Control +0.128.
- Gemma 3 27B+0.06995% bootstrap interval +0.046 to +0.099. Control +0.069.
- GLM-5.2+0.12695% bootstrap interval +0.102 to +0.150. Control +0.126.
- Grok 4.3+0.02795% bootstrap interval +0.005 to +0.044. Control +0.027.
- Grok 4.5-0.00995% bootstrap interval -0.036 to +0.003. Control -0.009.
- Kimi K2.6+0.09595% bootstrap interval +0.070 to +0.120. Control +0.095.
- Claude Opus 4.8+0.11995% bootstrap interval +0.091 to +0.141. Control +0.119.
- Qwen3.6-27B+0.09795% bootstrap interval +0.071 to +0.116. Control +0.097.
The same instruction narrows some gaps, widens one and produces overshoot.
- DeepSeek V4 Pro+0.00095% bootstrap interval -0.012 to +0.026. Control +0.035.
- Gemini 3.1 Pro+0.07195% bootstrap interval +0.050 to +0.090. Control +0.128.
- Gemma 3 27B+0.04995% bootstrap interval +0.031 to +0.067. Control +0.069.
- GLM-5.2+0.06195% bootstrap interval +0.047 to +0.082. Control +0.126.
- Grok 4.3+0.05095% bootstrap interval +0.030 to +0.068. Control +0.027.
- Grok 4.5-0.02595% bootstrap interval -0.045 to -0.006. Control -0.009.
- Kimi K2.6-0.00695% bootstrap interval -0.034 to +0.020. Control +0.095.
- Claude Opus 4.8-0.03795% bootstrap interval -0.060 to -0.013. Control +0.119.
- Qwen3.6-27B+0.02895% bootstrap interval +0.013 to +0.046. Control +0.097.
With examples, every family remains significantly flatter than its human baseline.
- DeepSeek V4 Pro+0.02995% bootstrap interval +0.009 to +0.049. Control +0.035.
- Gemini 3.1 Pro+0.16995% bootstrap interval +0.148 to +0.183. Control +0.128.
- Gemma 3 27B+0.06395% bootstrap interval +0.047 to +0.100. Control +0.069.
- GLM-5.2+0.14895% bootstrap interval +0.123 to +0.169. Control +0.126.
- Grok 4.3+0.10895% bootstrap interval +0.085 to +0.133. Control +0.027.
- Grok 4.5+0.07795% bootstrap interval +0.058 to +0.112. Control -0.009.
- Kimi K2.6+0.12395% bootstrap interval +0.112 to +0.152. Control +0.095.
- Claude Opus 4.8+0.03995% bootstrap interval +0.011 to +0.058. Control +0.119.
- Qwen3.6-27B+0.11995% bootstrap interval +0.094 to +0.138. Control +0.097.
Planning produces a different pattern of gaps. Near zero alone does not establish equivalence.
- DeepSeek V4 Pro+0.01395% bootstrap interval -0.010 to +0.040. Control +0.035.
- Gemini 3.1 Pro+0.07095% bootstrap interval +0.049 to +0.088. Control +0.128.
- Gemma 3 27B+0.06195% bootstrap interval +0.046 to +0.081. Control +0.069.
- GLM-5.2+0.08095% bootstrap interval +0.062 to +0.099. Control +0.126.
- Grok 4.3+0.06195% bootstrap interval +0.038 to +0.085. Control +0.027.
- Grok 4.5-0.00995% bootstrap interval -0.023 to +0.019. Control -0.009.
- Kimi K2.6+0.02695% bootstrap interval +0.007 to +0.054. Control +0.095.
- Claude Opus 4.8-0.00095% bootstrap interval -0.022 to +0.022. Control +0.119.
- Qwen3.6-27B+0.04595% bootstrap interval +0.023 to +0.065. Control +0.097.
Self-refinement narrows several gaps and pushes others beyond the human reference.
- DeepSeek V4 Pro-0.01895% bootstrap interval -0.032 to +0.003. Control +0.035.
- Gemini 3.1 Pro+0.03995% bootstrap interval +0.016 to +0.053. Control +0.128.
- Gemma 3 27B+0.02995% bootstrap interval +0.009 to +0.048. Control +0.069.
- GLM-5.2+0.01195% bootstrap interval -0.010 to +0.038. Control +0.126.
- Grok 4.3+0.01995% bootstrap interval -0.003 to +0.042. Control +0.027.
- Grok 4.5-0.05995% bootstrap interval -0.085 to -0.024. Control -0.009.
- Kimi K2.6-0.07095% bootstrap interval -0.091 to -0.035. Control +0.095.
- Claude Opus 4.8-0.06595% bootstrap interval -0.088 to -0.045. Control +0.119.
- Qwen3.6-27B+0.03995% bootstrap interval +0.018 to +0.063. Control +0.097.
Selected mode Control 95% CI
Zero is the human reference. ΔBₙ = human − model. Each row is a paired comparison, not a ranking.
Exact estimates and intervals
| Family | Gap | 95% bootstrap CI | Control |
|---|---|---|---|
| DeepSeek V4 Pro | +0.035 | +0.011 to +0.053 | +0.035 |
| Gemini 3.1 Pro | +0.128 | +0.102 to +0.140 | +0.128 |
| Gemma 3 27B | +0.069 | +0.046 to +0.099 | +0.069 |
| GLM-5.2 | +0.126 | +0.102 to +0.150 | +0.126 |
| Grok 4.3 | +0.027 | +0.005 to +0.044 | +0.027 |
| Grok 4.5 | -0.009 | -0.036 to +0.003 | -0.009 |
| Kimi K2.6 | +0.095 | +0.070 to +0.120 | +0.095 |
| Claude Opus 4.8 | +0.119 | +0.091 to +0.141 | +0.119 |
| Qwen3.6-27B | +0.097 | +0.071 to +0.116 | +0.097 |
| Family | Gap | 95% bootstrap CI | Control |
|---|---|---|---|
| DeepSeek V4 Pro | +0.000 | -0.012 to +0.026 | +0.035 |
| Gemini 3.1 Pro | +0.071 | +0.050 to +0.090 | +0.128 |
| Gemma 3 27B | +0.049 | +0.031 to +0.067 | +0.069 |
| GLM-5.2 | +0.061 | +0.047 to +0.082 | +0.126 |
| Grok 4.3 | +0.050 | +0.030 to +0.068 | +0.027 |
| Grok 4.5 | -0.025 | -0.045 to -0.006 | -0.009 |
| Kimi K2.6 | -0.006 | -0.034 to +0.020 | +0.095 |
| Claude Opus 4.8 | -0.037 | -0.060 to -0.013 | +0.119 |
| Qwen3.6-27B | +0.028 | +0.013 to +0.046 | +0.097 |
| Family | Gap | 95% bootstrap CI | Control |
|---|---|---|---|
| DeepSeek V4 Pro | +0.029 | +0.009 to +0.049 | +0.035 |
| Gemini 3.1 Pro | +0.169 | +0.148 to +0.183 | +0.128 |
| Gemma 3 27B | +0.063 | +0.047 to +0.100 | +0.069 |
| GLM-5.2 | +0.148 | +0.123 to +0.169 | +0.126 |
| Grok 4.3 | +0.108 | +0.085 to +0.133 | +0.027 |
| Grok 4.5 | +0.077 | +0.058 to +0.112 | -0.009 |
| Kimi K2.6 | +0.123 | +0.112 to +0.152 | +0.095 |
| Claude Opus 4.8 | +0.039 | +0.011 to +0.058 | +0.119 |
| Qwen3.6-27B | +0.119 | +0.094 to +0.138 | +0.097 |
| Family | Gap | 95% bootstrap CI | Control |
|---|---|---|---|
| DeepSeek V4 Pro | +0.013 | -0.010 to +0.040 | +0.035 |
| Gemini 3.1 Pro | +0.070 | +0.049 to +0.088 | +0.128 |
| Gemma 3 27B | +0.061 | +0.046 to +0.081 | +0.069 |
| GLM-5.2 | +0.080 | +0.062 to +0.099 | +0.126 |
| Grok 4.3 | +0.061 | +0.038 to +0.085 | +0.027 |
| Grok 4.5 | -0.009 | -0.023 to +0.019 | -0.009 |
| Kimi K2.6 | +0.026 | +0.007 to +0.054 | +0.095 |
| Claude Opus 4.8 | -0.000 | -0.022 to +0.022 | +0.119 |
| Qwen3.6-27B | +0.045 | +0.023 to +0.065 | +0.097 |
| Family | Gap | 95% bootstrap CI | Control |
|---|---|---|---|
| DeepSeek V4 Pro | -0.018 | -0.032 to +0.003 | +0.035 |
| Gemini 3.1 Pro | +0.039 | +0.016 to +0.053 | +0.128 |
| Gemma 3 27B | +0.029 | +0.009 to +0.048 | +0.069 |
| GLM-5.2 | +0.011 | -0.010 to +0.038 | +0.126 |
| Grok 4.3 | +0.019 | -0.003 to +0.042 | +0.027 |
| Grok 4.5 | -0.059 | -0.085 to -0.024 | -0.009 |
| Kimi K2.6 | -0.070 | -0.091 to -0.035 | +0.095 |
| Claude Opus 4.8 | -0.065 | -0.088 to -0.045 | +0.119 |
| Qwen3.6-27B | +0.039 | +0.018 to +0.063 | +0.097 |
How to read the figure
Right of zero: generated prose is flatter than the matched human articles. Left of zero: it is more dispersed.
Filled dots show median paired gaps for the selected mode; lines show their 95% bootstrap confidence intervals. Hollow dots keep the control condition visible.
A small gap is not proof of equivalence. DeepSeek instruction meets a 0.03 equivalence margin on Bₙ; Kimi meets 0.05. Those are bounded statistical claims, not statements about overall writing quality.
Rows stay in a fixed order. Provider decoding differs, so each family should be interpreted against its own paired human baseline. Movement between dots is descriptive, not a directly tested mode contrast.
See the full table ↓Design
Compare each family with the same human baseline.
200 English Wikinews articles from 2016–2021 provide headline, topic, era and register matching. Each family has five modes and three fixed seeds: 3,000 scheduled cells per family, 27,000 across the benchmark. These are planned cells, not a claim of 27,000 completed outputs.
Generated text is normalized uniformly before scoring. Seeds are averaged within headline; paired Wilcoxon tests use the headline as the independent unit. Positive Δ = human − model means the model is flatter on Bₙ; negative Δ means it is more dispersed.
Provider decoding is not matched. These are nine separate paired comparisons, not a model leaderboard. Differences between modes describe estimates and are not directly tested mode contrasts.
Table 1
Sentence-length dispersion, mode by mode.
Cells show median Δ, with matched rank-biserial in parentheses. Stars show nominal two-sided significance: * p < .05, ** p < .01. All starred Bₙ cells survive Benjamini–Hochberg correction within family. Grok 4.5 instruction does not survive the stricter Holm correction.
| Model family | Control | Instruction | Few-shot | Plan | Self-refine |
|---|---|---|---|---|---|
| DeepSeek V4 Pro200 headline pairs per mode | +0.035**(+0.34) | +0.000(+0.05) | +0.029**(+0.24) | +0.013(+0.15) | -0.018(-0.10) |
| Gemini 3.1 Pro200 headline pairs per mode | +0.128**(+0.85) | +0.071**(+0.56) | +0.169**(+0.94) | +0.070**(+0.58) | +0.039**(+0.27) |
| Gemma 3 27B200 headline pairs per mode | +0.069**(+0.58) | +0.049**(+0.44) | +0.063**(+0.65) | +0.061**(+0.57) | +0.029**(+0.30) |
| GLM-5.2197–200 headline pairs per mode | +0.126**(+0.85) | +0.061**(+0.57) | +0.148**(+0.94) | +0.080**(+0.65) | +0.011(+0.13) |
| Grok 4.3200 headline pairs per mode | +0.027**(+0.27) | +0.050**(+0.42) | +0.108**(+0.76) | +0.061**(+0.56) | +0.019*(+0.19) |
| Grok 4.5200 headline pairs per mode | -0.009(-0.14) | -0.025*(-0.20) | +0.077**(+0.69) | -0.009(-0.02) | -0.059**(-0.41) |
| Kimi K2.6198–199 headline pairs per mode | +0.095**(+0.69) | -0.006(-0.00) | +0.123**(+0.83) | +0.026**(+0.24) | -0.070**(-0.57) |
| Claude Opus 4.8200 headline pairs per mode | +0.119**(+0.80) | -0.037**(-0.25) | +0.039**(+0.34) | -0.000(+0.01) | -0.065**(-0.50) |
| Qwen3.6-27B200 headline pairs per mode | +0.097**(+0.67) | +0.028**(+0.34) | +0.119**(+0.80) | +0.045**(+0.41) | +0.039**(+0.26) |
Missing outputs are excluded before seed averaging. Qwen has four missing outputs; Grok 4.5 has two empty self-refine outputs; Kimi has 25 explicit moderation refusals; GLM has 36 silent empty responses. Sample sizes therefore vary for some family-by-mode comparisons.
Source: Table 1 and Appendix B. Download the scored table data (JSON).
Across all five modes
Most comparisons are still flatter.
Across 45 family-by-mode comparisons on Bₙ. No significant difference does not establish equivalence.
DeepSeek instruction meets the 0.03 equivalence margin on Bₙ; Kimi meets the 0.05 margin. Their bootstrap confidence intervals lie within those bounds. This is not a claim of complete human-like rhythm across all dimensions.
Opus and Grok 4.5 overshoot under instruction. Opus survives both correction methods; Grok 4.5 is exploratory under Holm. Few-shot is significantly flatter in all nine families, flattest by rank-biserial in seven, and largest by median gap in six.
Across the 180 tests on four dimensions, 124 are nominally significant, 120 survive Benjamini–Hochberg, and 103 survive Holm. Corrections are within family, not across all families.
Normalization
Formatting can change the apparent result.
On the audited Qwen leg, the instruction gap changes from +0.005 (p = .10) on raw output to +0.028 (p = .00003) after normalization. The uninstructed gap changes from +0.043 to +0.097. Raw non-significance was not evidence of equivalence.
The normalization removes echoed headline blocks and unwraps markdown while preserving prose. The human baseline is unchanged. The word-loss audit reports median 0.0%, mean 1.7% and maximum 7.4%.
Companion dimensions
Dispersion does not describe everything.
Gini is a robustness check. Segmentation entropy is a separate diversity axis, and lag-1 memory M measures order. Surprisal is outside the construct and is not scored in this benchmark. The stars below are nominal; the JSON includes the complete per-cell correction results.
DeepSeek V4 Pro
Of 20 within-family tests: 8 nominal, 8 Benjamini–Hochberg, 6 Holm.
| Mode | Bₙ | Gini | Seg. entropy | Memory M |
|---|---|---|---|---|
| Control | +0.035**(+0.34)n = 200 | +0.018**(+0.35)n = 200 | -0.050(-0.12)n = 200 | -0.025(-0.07)n = 200 |
| Instruction | +0.000(+0.05)n = 200 | -0.005(-0.02)n = 200 | -0.145**(-0.47)n = 200 | +0.006(+0.07)n = 200 |
| Few-shot | +0.029**(+0.24)n = 200 | +0.014*(+0.20)n = 200 | -0.068**(-0.26)n = 200 | +0.012(+0.01)n = 200 |
| Plan | +0.013(+0.15)n = 200 | +0.003(+0.08)n = 200 | -0.117**(-0.36)n = 200 | -0.003(+0.03)n = 200 |
| Self-refine | -0.018(-0.10)n = 200 | -0.010(-0.11)n = 200 | -0.139**(-0.49)n = 200 | +0.027(+0.14)n = 200 |
Gemini 3.1 Pro
Of 20 within-family tests: 16 nominal, 16 Benjamini–Hochberg, 13 Holm.
| Mode | Bₙ | Gini | Seg. entropy | Memory M |
|---|---|---|---|---|
| Control | +0.128**(+0.85)n = 200 | +0.056**(+0.81)n = 200 | -0.062*(-0.19)n = 200 | +0.043*(+0.18)n = 200 |
| Instruction | +0.071**(+0.56)n = 200 | +0.026**(+0.47)n = 200 | -0.053(-0.16)n = 200 | +0.109**(+0.38)n = 200 |
| Few-shot | +0.169**(+0.94)n = 200 | +0.080**(+0.93)n = 200 | +0.117**(+0.39)n = 200 | +0.051**(+0.24)n = 200 |
| Plan | +0.070**(+0.58)n = 200 | +0.028**(+0.48)n = 200 | -0.026(-0.06)n = 200 | +0.114**(+0.47)n = 200 |
| Self-refine | +0.039**(+0.27)n = 200 | +0.013**(+0.21)n = 200 | -0.051(-0.12)n = 200 | +0.047(+0.12)n = 200 |
Gemma 3 27B
Of 20 within-family tests: 16 nominal, 16 Benjamini–Hochberg, 16 Holm.
| Mode | Bₙ | Gini | Seg. entropy | Memory M |
|---|---|---|---|---|
| Control | +0.069**(+0.58)n = 200 | +0.037**(+0.58)n = 200 | +0.063**(+0.25)n = 200 | +0.023(+0.11)n = 200 |
| Instruction | +0.049**(+0.44)n = 200 | +0.025**(+0.41)n = 200 | +0.086**(+0.36)n = 200 | +0.040(+0.10)n = 200 |
| Few-shot | +0.063**(+0.65)n = 200 | +0.033**(+0.63)n = 200 | +0.145**(+0.48)n = 200 | +0.005(+0.02)n = 200 |
| Plan | +0.061**(+0.57)n = 200 | +0.033**(+0.55)n = 200 | +0.129**(+0.45)n = 200 | +0.003(+0.03)n = 200 |
| Self-refine | +0.029**(+0.30)n = 200 | +0.015**(+0.29)n = 200 | +0.094**(+0.32)n = 200 | +0.048**(+0.21)n = 200 |
GLM-5.2
Of 20 within-family tests: 16 nominal, 16 Benjamini–Hochberg, 14 Holm.
| Mode | Bₙ | Gini | Seg. entropy | Memory M |
|---|---|---|---|---|
| Control | +0.126**(+0.85)n = 198 | +0.058**(+0.82)n = 198 | +0.068**(+0.25)n = 198 | +0.053*(+0.19)n = 198 |
| Instruction | +0.061**(+0.57)n = 198 | +0.029**(+0.51)n = 198 | +0.064**(+0.26)n = 198 | +0.076**(+0.24)n = 198 |
| Few-shot | +0.148**(+0.94)n = 197 | +0.068**(+0.92)n = 197 | +0.221**(+0.74)n = 197 | +0.022(+0.07)n = 197 |
| Plan | +0.080**(+0.65)n = 197 | +0.034**(+0.59)n = 197 | +0.091**(+0.32)n = 197 | +0.034*(+0.20)n = 197 |
| Self-refine | +0.011(+0.13)n = 200 | +0.002(+0.09)n = 200 | +0.050**(+0.33)n = 200 | -0.019(-0.02)n = 200 |
Grok 4.3
Of 20 within-family tests: 13 nominal, 13 Benjamini–Hochberg, 9 Holm.
| Mode | Bₙ | Gini | Seg. entropy | Memory M |
|---|---|---|---|---|
| Control | +0.027**(+0.27)n = 200 | +0.010**(+0.23)n = 200 | -0.046(-0.16)n = 200 | -0.004(+0.01)n = 200 |
| Instruction | +0.050**(+0.42)n = 200 | +0.026**(+0.39)n = 200 | +0.062*(+0.19)n = 200 | +0.028(+0.10)n = 200 |
| Few-shot | +0.108**(+0.76)n = 200 | +0.052**(+0.74)n = 200 | +0.180**(+0.56)n = 200 | +0.001(-0.05)n = 200 |
| Plan | +0.061**(+0.56)n = 200 | +0.026**(+0.50)n = 200 | +0.136**(+0.35)n = 200 | +0.023(+0.03)n = 200 |
| Self-refine | +0.019*(+0.19)n = 200 | +0.009(+0.14)n = 200 | +0.055**(+0.21)n = 200 | +0.028(+0.14)n = 200 |
Grok 4.5
Of 20 within-family tests: 14 nominal, 13 Benjamini–Hochberg, 9 Holm.
| Mode | Bₙ | Gini | Seg. entropy | Memory M |
|---|---|---|---|---|
| Control | -0.009(-0.14)n = 200 | -0.011*(-0.17)n = 200 | -0.249**(-0.74)n = 200 | -0.023(-0.07)n = 200 |
| Instruction | -0.025*(-0.20)n = 200 | -0.014*(-0.20)n = 200 | -0.099**(-0.40)n = 200 | +0.048**(+0.22)n = 200 |
| Few-shot | +0.077**(+0.69)n = 200 | +0.039**(+0.68)n = 200 | +0.089**(+0.28)n = 200 | -0.002(-0.05)n = 200 |
| Plan | -0.009(-0.02)n = 200 | -0.005(-0.05)n = 200 | -0.062**(-0.22)n = 200 | +0.021(+0.06)n = 200 |
| Self-refine | -0.059**(-0.41)n = 200 | -0.033**(-0.44)n = 200 | -0.133**(-0.44)n = 200 | +0.071**(+0.31)n = 200 |
Kimi K2.6
Of 20 within-family tests: 12 nominal, 10 Benjamini–Hochberg, 9 Holm.
| Mode | Bₙ | Gini | Seg. entropy | Memory M |
|---|---|---|---|---|
| Control | +0.095**(+0.69)n = 199 | +0.043**(+0.63)n = 199 | -0.048*(-0.16)n = 199 | -0.025(-0.09)n = 199 |
| Instruction | -0.006(-0.00)n = 199 | -0.014(-0.14)n = 199 | -0.024(-0.11)n = 199 | -0.014(+0.08)n = 199 |
| Few-shot | +0.123**(+0.83)n = 199 | +0.063**(+0.79)n = 199 | +0.152**(+0.55)n = 199 | -0.073**(-0.26)n = 199 |
| Plan | +0.026**(+0.24)n = 199 | +0.006(+0.09)n = 199 | +0.032(+0.05)n = 199 | +0.055*(+0.17)n = 199 |
| Self-refine | -0.070**(-0.57)n = 198 | -0.040**(-0.58)n = 198 | +0.061(+0.16)n = 198 | -0.052**(-0.23)n = 198 |
Claude Opus 4.8
Of 20 within-family tests: 16 nominal, 16 Benjamini–Hochberg, 15 Holm.
| Mode | Bₙ | Gini | Seg. entropy | Memory M |
|---|---|---|---|---|
| Control | +0.119**(+0.80)n = 200 | +0.054**(+0.77)n = 200 | +0.116**(+0.39)n = 200 | -0.069**(-0.24)n = 200 |
| Instruction | -0.037**(-0.25)n = 200 | -0.031**(-0.36)n = 200 | +0.032(+0.09)n = 200 | -0.117**(-0.38)n = 200 |
| Few-shot | +0.039**(+0.34)n = 200 | +0.012**(+0.26)n = 200 | +0.253**(+0.75)n = 200 | -0.045**(-0.25)n = 200 |
| Plan | -0.000(+0.01)n = 200 | -0.004(-0.10)n = 200 | +0.108**(+0.34)n = 200 | -0.059**(-0.25)n = 200 |
| Self-refine | -0.065**(-0.50)n = 200 | -0.043**(-0.59)n = 200 | -0.005(+0.01)n = 200 | +0.039*(+0.18)n = 200 |
Qwen3.6-27B
Of 20 within-family tests: 13 nominal, 12 Benjamini–Hochberg, 12 Holm.
| Mode | Bₙ | Gini | Seg. entropy | Memory M |
|---|---|---|---|---|
| Control | +0.097**(+0.67)n = 200 | +0.046**(+0.67)n = 200 | -0.001(+0.10)n = 200 | -0.007(-0.05)n = 200 |
| Instruction | +0.028**(+0.34)n = 200 | +0.010**(+0.28)n = 200 | -0.018(-0.04)n = 200 | -0.052*(-0.16)n = 200 |
| Few-shot | +0.119**(+0.80)n = 200 | +0.060**(+0.79)n = 200 | +0.127**(+0.47)n = 200 | -0.071**(-0.26)n = 200 |
| Plan | +0.045**(+0.41)n = 200 | +0.017**(+0.35)n = 200 | +0.057(+0.16)n = 200 | -0.028(-0.07)n = 200 |
| Self-refine | +0.039**(+0.26)n = 200 | +0.021**(+0.24)n = 200 | +0.029(+0.10)n = 200 | +0.028(+0.09)n = 200 |
Limits
A descriptive result in one register.
The study covers English news, fixed prompt wording and contemporary instruction-tuned models. A three-paraphrase DeepSeek probe changes the instruction estimate by about 0.03. The limited contamination probe finds little verbatim overlap in three control legs; it does not prove contamination-free data.
Bₙ is blind to sentence order and distribution shape. Memory observes only lag 1. The reliability floor is ten sentences. The Goodhart guard and coherence gate are proposed screens for future selection or control; neither filters these benchmark generations.