CreativityNeuro: Steering LLM Weights to Unlock Divergent Thinking and Beat Mode Collapse

Every engineer who has built a generative AI product knows the frustration: ask an LLM to brainstorm ten ideas and you get ten variations of the same idea. Ask it to write a creative story and the prose feels pleasantly competent - and exactly like every other story it has written. This phenomenon, known as mode collapse, isn't just an annoyance. It's a structural problem baked into how modern language models are trained. A new paper - CreativityNeuro: Steering Language Model Weights to Improve Divergent Thinking and Reduce Mode Collapse (arXiv:2607.01433) - proposes a direct intervention: modify the model weights themselves to push outputs toward genuine creative diversity.

The Mode Collapse Problem

Mode collapse in LLMs happens when a model's probability distribution over outputs becomes sharply peaked around high-frequency, "safe" responses. RLHF and instruction tuning, while enormously valuable for alignment and helpfulness, tend to reinforce this: human raters consistently reward fluent, conventional outputs, so the model learns to converge on a narrow band of acceptable responses.

The problem shows up starkly on divergent thinking benchmarks. Tests like the Alternate Uses Task (AUT) - "name as many uses for a brick as possible" - measure a model's ability to generate semantically distant ideas. Research on divergent creativity in both LLMs and humans found that LLMs can surpass average human performance on the Divergent Association Task and approach human creative writing abilities, though they fall short of the typical performance of highly creative humans 1. Notably, even the top-performing LLMs are still largely surpassed by highly creative individuals - a ceiling current models consistently fail to break through 1.

And critically, even when individual outputs score high on creativity, the population of outputs from the same model tends toward homogeneity - a distribution-level failure that per-output evaluations miss entirely. The "Artificial Hivemind" paper (arXiv:2510.22954, presented at NeurIPS 2025) documents this at scale: studying 70+ open and closed-source LLMs across 26,000 real-world open-ended queries, it reveals both intra-model repetition, where a single model repeatedly generates similar outputs, and inter-model homogeneity, where different models independently converge on similar ideas 2.

What CreativityNeuro Does

Rather than addressing creativity through prompting tricks or sampling heuristics - like temperature scaling (making the output distribution flatter to increase randomness) or min-p sampling (a technique that filters out tokens below a dynamic probability floor, preserving coherence at high temperatures) 3 - CreativityNeuro intervenes at the weight level. According to the paper's abstract, the approach steers language model weights to improve divergent thinking and reduce mode collapse, modifying the model's default generative behavior toward exploration rather than requiring adjustments at inference time 4.

The neuroscience framing underlying the paper's approach is supported by recent alignment research. A 2026 study (arXiv:2604.03480) explores brain-LLM alignment during creative thinking using fMRI data from 170 participants performing the AUT, measuring alignment to brain responses via Representational Similarity Analysis (RSA) and specifically targeting the creativity-related default mode and frontoparietal networks 5. These are the distributed, creativity-related brain networks that activate during divergent thinking - distinct from the networks engaged in passive language processing. The hypothesis underlying CreativityNeuro is that analogous "circuits" in transformer models govern whether the model explores or converges, and that these can be targeted directly through weight modification.

Note on the primary source: The specific technical details of the CreativityNeuro methodology - the exact circuit-identification procedure, the weight-editing algorithm, and benchmark results - should be read directly from arXiv:2607.01433. This article is based on the paper's publicly available abstract and framing; claims about internal mechanisms await confirmation from a full reading of the paper.

Why This Matters for Alignment, Too

Mode collapse is usually framed as a creativity problem, but it has a shadow side that's squarely an alignment issue. A model that consistently converges on the same narrow outputs is, among other things, easier to manipulate via adversarial prompts that exploit known "grooves" in its output distribution. Diversity in generation is a proxy for robustness.

There's also the collective diversity concern. Work on ex ante evaluation of AI-induced idea diversity collapse (arXiv:2605.06540) argues that creative AI systems are typically evaluated only at the level of individual utility - yet creative outputs are consumed in populations, where an idea loses value when many others produce similar ones 3. This creates an evaluation blind spot: AI can improve individual outputs while simultaneously increasing population-level crowding 3. A technical fix like CreativityNeuro, if it holds up in broader evaluation, could directly address this homogenization at the model level rather than leaving it for post-deployment mitigation.

The Evaluation Landscape

Evaluating LLM creativity is notoriously slippery. The field has coalesced around a few psychometric proxies:

• AUT (Alternate Uses Task): measures semantic distance between generated uses for an object
• DAT (Divergent Association Task): asks for semantically unrelated words; distance is the score
• TTCT (Torrance Tests of Creative Thinking): assesses fluency, originality, flexibility, and elaboration

But static, single-turn benchmarks have a blind spot of their own. The MUTATE benchmark (arXiv:2605.28465) was introduced specifically to address this: it evaluates agentic divergent thinking at two levels - path-level, where an agent discovers multiple alternative routes to the same goal, and action-level, where individual actions require non-typical, mechanism-shifting object uses 6. Crucially, unlike success-only evaluations, MUTATE scores both completed paths and off-path attempts, capturing divergent reasoning that conventional benchmarks discard 6. Experiments with frontier LLMs using MUTATE reveal a structural blind spot: when exposed to immediate convergence pressure, models fall into "action fixation," failing to improve action-level divergence 6.

This distributional framing is the right methodological choice for work like CreativityNeuro: a model that writes one brilliant creative sentence and nine repetitions of it should not score the same as a model that writes ten distinct brilliant sentences. The broader homogeneity problem has also been studied at the population level - the Artificial Hivemind research documents that individual models and even model ensembles converge on highly similar outputs for open-ended queries with vast plausible answer spaces 2.

Open Questions

A few engineering questions will determine how useful this approach is in practice:

Specificity vs. generality. Weight modifications that boost creativity on brainstorming tasks could degrade performance on tasks where convergence is actually desirable - code generation, factual Q&A, structured data extraction. The paper's results will need to show that the intervention is controllable and doesn't produce regressions across the board.

Composability with fine-tuning. If downstream users fine-tune a CreativityNeuro-modified base model, do the gains survive? RLHF and supervised fine-tuning are powerful forces that may simply re-converge the distribution.

Scale transfer. Weight-level interventions that work on smaller models don't always transfer to larger ones. The circuits driving convergence in a 7B model may be organized very differently than in a 70B model.

Why It Matters

The field has spent enormous effort on making LLMs more accurate - better at retrieving facts, following instructions, and avoiding harmful outputs. Comparatively little work has attacked the structural reasons they are creatively monotonous. CreativityNeuro represents a promising direction: rather than fighting mode collapse at sampling time, it targets the root cause in the model's own parameters.

For engineers building products where creative diversity is a feature - writing tools, ideation platforms, game content generation, drug discovery - this is worth watching closely. If the weight-editing approach proves stable and transferable, it could become a standard post-training step alongside safety tuning and instruction following.