Why Creative Writing Alignment Needs Expert Decision Data
This article was originally written in Chinese. The English version was produced with AI assistance and reviewed against the Chinese original.
A Debate Over "Good Writing"
How difficult is creative writing for large language models? A small incident involving the CEO of OpenAI may give us an intuitive sense of the problem.
In March 2025, Sam Altman wrote on X that OpenAI had trained a new model that was good at creative writing. He said that, for the first time, something written by AI had truly "touched" him. In his view, the text accurately captured the feel of metafiction. He also shared the short story that the model had written from a prompt. Altman's original post
An X user wrote in the comments: "I would encourage you to read some actual writing before declaring this good writing." The comment received over 3,000 likes. When The New Yorker later looked back at the episode, it observed that seemingly no one was as moved by the text as Altman himself. The New Yorker's review
This is not a disagreement over aesthetic taste. The fact is that creative writing produced by current LLMs has not yet reached the level of accomplished writers.
Stories written by LLMs are longer, more coherent, smoother, and more precise in their word choice than before. But they still perform terribly when it comes to compelling characters, emotional impact, and story tension.
LLMs mainly do well on properties of finished texts that are easy to measure at scale. What they do poorly often depends on creative judgments that are difficult for non-experts to express accurately, hard to annotate at scale, and difficult to infer from finished texts. What distinguishes accomplished writers from ordinary people and mediocre writers is precisely these implicit judgments that are hard to explain directly.
To enable LLMs to make these creative judgments like human experts, an important step is to obtain samples of human expert judgment about creative writing.
Unfortunately, very few such records exist. In fields like law, finance, and biomedicine, professional judgment leaves behind extensive records in the form of cases, court rulings, research reports, and clinical guidelines. Creative writing appears to have a vast amount of commentary on finished works, but this is not the kind of "creative writing judgment sample" we are talking about.
Here, I need to clarify one point. The vast majority of popular book reviews are "texts of judgment," but they are closer to "consumer experience reports." They mainly show what readers saw or experienced in a work. A small number of professional reviews do go deeply into structure and technique, but they usually do not leave a record of which options creators compared when facing specific problems, why they chose A over B, or how their judgments formed through counterexamples and revisions. There is no shortage of evaluations of finished works. What is truly scarce is a record of the creative decisions themselves.
The irony is that LLMs cannot even write good story reviews, despite being trained on large amounts of professional criticism written by humans. To readers with some literary judgment, book reviews written by current LLMs still remain in the realm of "performing criticism."
If we want LLMs not only to evaluate what has already been written, but also to improve at the creative stage itself, they need to learn from expert judgment samples about creative decisions. This creates a practical dilemma: LLMs need this kind of data, but very little of it exists in the real world.
Before we answer this question, let us first look at current practice.
When Creative Decisions Are Invisible, Training Can Only Start with Results
Since creative decision samples are so scarce, what signals has the industry actually used over the past few years to train and improve creative writing?
The answer has been to start with visible and quantifiable results: let models learn from finished texts, constrain generation with outlines, structures, and formats, and then use human preferences or reward models to select the better one from several generated versions.
These methods have made substantial progress. Models write longer, more coherent text, and follow explicit instructions better. Tools like Dramatron have brought models into the co-creation workflow of professional screenwriters; recent research like RLMR has also begun to optimize both subjective writing quality and objective constraints such as word count and format.
However, these signals still mainly revolve around observable results and preset intermediate structures. They can constrain, check, score, or select generated content, but they do not automatically record what problems professional creators saw, which options they compared, or why they changed their judgment.
From "Which Is Better" to "Why Write It This Way"
What exactly is the gap between result-side signals and creative decision data?
Our public CQA preview dataset contains a question: why do some character dilemmas stay in readers' memories for years, while some seemingly intense conflicts quickly lose their persuasiveness?
During a discussion, two human experts mentioned a poorly designed police character. Although he could have advanced the investigation through legal and professional means, he instead used unnecessary illegal methods and created obstacles for himself that did not originally exist. One expert first pointed out that the pressure in this story did not come from the situation. It was created by the character himself.
Another participant immediately condensed this judgment into a more direct statement:
"The character creates the trouble himself, and then goes on to solve it."
But the judgment did not stop at the superficial conclusion that "characters should not act foolishly." The next comparison revealed the real condition: a police officer in another story simultaneously carries three roles—father, investigator, and public servant. Rules, family responsibilities, and professional duties conflict with one another. He has tried his best to make things turn out well for everyone, yet he still cannot escape unscathed. The conversation then converged on a more precise principle:
A conflict stays with readers not because the situation becomes dramatic, but because the character is truly forced into a position where no choice can satisfy all sides. Only when the character must pay a price regardless of what they choose does the conflict feel like it belongs to that character, rather than being an obstacle temporarily created by the author.
This principle is valuable, but CQA aims to provide more than just conclusions. It also records how the conclusion was formed: first identifying a failed design, asking where the pressure actually comes from, then using a successful contrasting example to refine the initial judgment, and finally distinguishing between "a character actively creating trouble" and "a character squeezed by multiple responsibilities."
If this material were converted into preference data that kept only the final choice, it could tell the model that readers prefer the second police officer's story. This kind of expert judgment data goes further. It reveals why the second story is better, under what conditions the judgment holds, and why the creative decisions in the other story fail. This is the distance between "which is better" and "why write it this way."
CQA: Turning Expert Creative Judgment into Learnable Data
CQA (Creative Quality Alignment) is our proposed framework for creative writing alignment. Its first area of work is to draw out the basis of expert judgment from forms of creative experience that professional creators handle largely by intuition and seldom put into words. We do this through continued questioning, and then organize the process into checkable, learnable data. Each data item centers on a specific creative question. It presents both how expert judgment develops and the creative principles that the judgments eventually converge on.
The current public samples use a format that combines a focused question, a dual-expert dialogue, and a final answer. This allows readers to examine the complete process through which the judgment develops. But expert dialogue does not naturally map to only one training format. Its comparisons, corrections of misunderstandings, causal analyses, and abstractions of principles can be preserved in a unified source data structure, then exported as different training and evaluation views for different research goals.
This is not merely a data concept. We have collected 104 Chinese-language records of expert knowledge about creative writing. Four of them have undergone in-depth cleaning and have been organized into a public preview. On this basis, we further propose that this kind of expert process signal may lead to better creative writing capabilities in models.
However, we cannot yet claim that this capability has been demonstrated. Whether training on this kind of data can produce reproducible improvements in character, conflict, plot threads, pacing, originality, and narrative tension still needs to be answered through training experiments, reasonable baselines, and professional evaluation.
We have released 4 complete preview samples in CQA: Creative Quality Alignment for Creative Writing Expert CoT (Preview) on Hugging Face. They come from the 104 Chinese-language records of expert knowledge about creative writing that we have collected so far. If you are researching creative writing post-training, expert process data, or creative writing evaluation, we welcome you to examine these public samples and get in touch with us. For further research and applications, we offer additional cleaning and licensing arrangements for the unreleased materials, as well as custom data production for specific model goals.