The Burstiness Engine
Dispersion
Bₙ measures relative variation in sentence length. Gini provides a robustness check.
The research
Before you can aim at rhythm,
you have to define it.
The study turns an intuition about prose into a measurement problem, then tests whether familiar prompting techniques can control the result.
01 / Define the target
Variation relative to length.
Scaling the same sentence-length pattern to longer sentences increases raw variance even when relative variation is unchanged. A useful measure of burstiness should separate that scale effect from the mix of short and long sentences.
We apply the established Goh–Barabási burstiness coefficient, with the Kim–Jo finite-sample correction, to sentence lengths in words. The resulting Bₙ is scale-free: multiplying every sentence length by the same factor leaves the score unchanged.
Diversity
Segmentation entropy describes the diversity of punctuation and segmentation symbols.
Order
Lag-1 memory captures association between neighboring sentence lengths.
These answer different questions. Bₙ is blind to sentence order and distribution shape; no single score describes the whole rhythm of a text. Surprisal remains outside the core construct.
02 / Audit the instrument
The formatting was part of the signal.
Models often echo a headline or wrap prose in markdown. Those additions can change where sentences begin and end, inflating a score intended to describe the writing.
The normalization audit exposed the consequence. On Qwen, the raw instruction gap was +0.005, with p = .10. Removing formatting artifacts widened it to +0.028, with p = .00003. The uninstructed gap grew from +0.043 to +0.097.
We therefore normalize generated text uniformly before scoring, while leaving the frozen human baseline unchanged. This removes echoed headline blocks and unwraps markdown rather than rewriting the prose. The word-loss audit reports a median of 0.0%, a mean of 1.7% and a maximum of 7.4%.
The methodological lesson
A non-significant raw score can be a measurement artifact. It is not evidence that a model matches human dispersion.
03 / Run the comparison
Hold the writing task steady. Change the prompt.
The benchmark pairs nine model families with 200 English Wikinews articles from 2016–2021. Each family writes from the same headlines under five prompting modes, with three seeds per prompt.
- ControlGenerate the article without an explicit burstiness instruction.
- InstructionAsk for a natural human journalistic voice.
- Few-shotProvide examples of the desired writing.
- PlanOutline the article before writing.
- Self-refineDraft, critique rhythm and pacing, then rewrite.
After normalization, seeds are averaged within each headline. Paired tests use the headline as the independent unit. Empty or refused outputs are excluded according to the scoring protocol; paired sample sizes are recorded for every comparison.
Provider decoding is not matched, so these are nine separate comparisons against human articles. They are not a cross-model leaderboard.
See what the experiment found04 / Follow the evidence
The question became one of aim.
A universal ceiling would mean prompting can never reach human dispersion. The nine-family results do not support that account: two instruction-mode results meet bounded equivalence criteria, while others overshoot.
The sharper result is inconsistency. The same instruction helps four families approach the human reference without reaching it, moves one further away, and leaves two beyond it. Few-shot prompting is the flattest mode by rank-biserial in seven of nine families.
This motivates a distinction between moving a measurable property and reliably targeting it. The paper establishes the measurement and descriptive benchmark. It does not demonstrate an in-generation controller or a human-perception benefit.
Read further
The paper, the evidence and the research record.
Research history and earlier iterations
A supporting record of how the scope and manuscript developed. Some GitHub sources require collaborator access.
Original research release
The Burstiness Engine website v1.0.0 presented an early research direction. It remains a historical snapshot and does not describe the accepted paper.
Source recordPaper 1 becomes the measurement and prompting benchmark
The research scope moved from a controller-centered proposal to defining burstiness and measuring the limits of prompt-level control. Controller work became a separate research track.
Source recordCoauthor review fixes the paper's scope
Vittoria's review retained both Grok generations and kept the current paper focused. Additional prompt-paraphrase experiments were treated as robustness work rather than a new central contribution.
Source recordNine-family benchmark consolidated
The benchmark expanded to nine model families, five prompting modes and three seeds on 200 matched headlines. Scored artifacts support the 45 family-by-mode comparisons in the published paper.
Source recordNine-family aim-problem manuscript merged
The coauthored nine-family rewrite replaced the earlier prompt-ceiling framing. The final argument distinguishes undershooting, bounded equivalence and overshooting under the same instruction.
Source recordSubmission preparation and coauthor rewrite
A coauthor-revised manuscript superseded the repository's earlier TeX prose. The reproducibility harness added printed-PDF number verification, anonymity checks, and submission preparation tools.
Source recordFinal submitted source snapshot
The anonymous artifact repository linked by the paper records the assembled LP4FM manuscript, figures, scored results and dataset mirror. Its source reproduces the text and checked results of the October 1 PDF. This snapshot date is not a submission receipt.
Source recordAccepted as a poster
The LP4FM decision notified the authors that submission 114 was accepted as a poster.
Source recordPaper published on OpenReview
The public record lists Vittoria Lanzo and Dico Angelo, the final title, LP4FM Poster status and the manuscript under CC BY 4.0. This is the current reference for the website and reproducibility package.
Source recordCamera-ready data handoff completed
Recovered 3,000 original Opus generations and completed new DeepSeek runs with 600 paraphrase and 3,000 baseline outputs. Verified raw data is private and requires collaborator access. The new DeepSeek data uses a later model snapshot; one off-topic baseline response is retained. The accepted PDF and scores remain unchanged while sentence-segmentation correction and rescoring are pending.
Source record