Özer Cem KELAHMETKELAHMET
Articles · Trends

DaisyRose-27B and self-rewriting weights, two new ways to improve an open model after training

This week brought two very different ways to make a 27B open model better after pretraining. RadixArk ran 18 days of asynchronous agentic reinforcement learning to build DaisyRose-27B, while an arXiv paper showed a model editing its own weights with forward passes only.

At a glance
Reading time
17 min
Sections
9
Code examples
1
Tables
2
Sources
7
Published
21 Nov 2026
In 30 seconds
What changed
RadixArk releases DaisyRose-27B
Why it matters
For teams that post-train their own models, DaisyRose is a public recipe showing that a relatively small, carefully reviewed task set can move a 27B model on long agent work.
What to do
Test DaisyRose on your own agent tasks

A pretrained model has read a huge amount of text, but it still has to learn to follow instructions, use tools and stay on task for hours. That later stage is post-training, and this week brought two very different examples of it, both on 27-billion-parameter open models.

The first, RadixArk's DaisyRose-27B, is industrial: about 1,200 reviewed agent tasks, 144 H200 GPUs, hundreds of thousands of sandboxes and an asynchronous reinforcement learning loop. The second is a research paper in which a model edits its own weights directly, with no gradients and no training loop.

One method spends tens of thousands of GPU-hours; the other, its authors say, ran its best sessions in a few hours on one GPU. Comparing them shows what post-training changes inside a model and which claims still need independent checks.

What happened

RadixArk releases DaisyRose-27B (18 November 2026)

RadixArk published DaisyRose-27B, its first model, on the Hugging Face blog on 18 November 2026. It starts from Qwen3.8-27B, keeps its architecture and 256K-token context window, and per the model card was trained with reinforcement learning only, with no supervised fine-tuning or distillation.

The training ran on RadixArk's open-source Miles framework. Of the 144 H200 GPUs, 128 ran the model on agent tasks (the rollouts) and 16 ran the trainer, and the two sides ran fully asynchronously. The roughly 1,200 tasks were chosen from thousands of candidates built with Eigent AI. Each targets a mistake the base model makes. A single attempt could last up to 1.5 hours, with more than 100 tool calls and more than 256K output tokens. Sessions up to 1M tokens were folded into several trajectories by harness summarization.

The sandboxes ran on a service built on AgentENV. RadixArk reports about 944,000 sandboxes between mid-September and mid-November, up to 1,566 running at once, and a cost about 80% lower than popular cloud sandbox providers. It also reports 37% model FLOPs utilization (MFU) on its internal training backend versus 16% for an open-source Megatron-based trainer, and about 16,800 tokens per second end to end versus about 8,300, roughly twice the throughput.

The weights are released under Apache 2.0 in BF16, FP8 and NVFP4 versions (the card notes NVFP4 needs a Blackwell GPU), and RadixArk recommends serving with SGLang plus its DSpark speculative decoding draft model. All benchmark numbers below are company-reported:

Table 1. RadixArk releases DaisyRose-27B (18 November 2026)
Benchmark Qwen3.8-27B (base) DaisyRose-27B
Terminal Bench 4.1 5.6 12.0
Terminal Bench 2.1 73.0 75.5
DeepSWE v1.1 42.2 46.2
SWE Bench Pro v2 91.3 94.7
HLE (text only) 33.9 34.3
IFBench 79.5 82.4
GPQA Diamond 89.2 90.7

One detail needs checking. The blog introduction says the run took "18 days on 144 H200 GPUs", while a later section says it "trained for weeks on 144 H200s, 63,000 GPU-hours in total". The model card says 18 days and about 63,000 GPU-hours. We checked: 144 GPUs for 18 days is 62,208 GPU-hours, and 63,000 GPU-hours on 144 GPUs is about 18.2 days. So the figures agree; "weeks" is a loose word for about two and a half weeks. The sandbox figures, by contrast, cover mid-September to mid-November (nine weeks), longer than the 18-day run itself.

A paper shows a model rewriting its own weights (16 and 18 November 2026)

On 16 November 2026, Jiajun Fan and ten co-authors posted arXiv paper 2611.21885, "Language Models Can Rewrite Their Own Weights", with a second version on 18 November. In this self-rewriting method, the model gets its own weights, a diagnosis of its failures on practice problems, and a sandboxed Python session where it can read and replace weight tensors, generate answers with original or edited weights. Gradients and optimizers are blocked. The session has a fixed budget of 20 forward passes and 200 code runs.

The experiments use the Qwen3.5 family at 2B, 4B, 9B and 27B. Practice problems come from DeepScaleR math data, with overlaps with benchmarks such as MATH-500 and AIME removed, and 1,000 problems are held out and never shown to a session. In one protocol, the 27B model's edit from session 8 scored 6.6 points higher than the unedited model on 490 held-out problems (eight answers each). Across its first eight sessions, the model withdrew its edit five times and submitted three times, and the accuracy of the model in use only went up. The authors also report a smaller, gain of 1.1 points, fixed in advance, (57.2% to 58.3%) in their first protocol.

In a repair test, the team corrupted eight rows of one MLP matrix with large noise, dropping the copy to 0% on held-out math. Working from text problems only, the model found a fix that also restored image skills: across six image benchmarks, the repaired copy reached 83% of the base model's accuracy. Only the 27B model managed this; the 2B model made no edit, and the 4B and 9B models damaged themselves. This is one group's result on one model family and has not been replicated.

How asynchronous RL for agents works

Reinforcement learning for language models has two jobs. First, the model attempts tasks and the results are scored (the rollout). Second, a trainer uses those scores to update the weights so that good attempts become more likely. Methods such as GRPO compare several attempts at one task and favour the better ones.

In a synchronous loop, these jobs take turns. The AReaL paper describes the core problem plainly: generation "must wait until the longest output in the batch is completed" before the update can start. With agent tasks, one attempt may finish in two minutes and another run 90 minutes, and every GPU that finished early sits idle.

Think of a kitchen that cannot start the next order until every table has finished eating: the slowest table sets the pace.

An asynchronous system splits the roles. In DaisyRose's setup, 128 GPUs only generate rollouts and 16 GPUs only train. Rollout workers keep starting attempts; whenever enough finish, the trainer takes a step and pushes new weights back. Neither side waits. The Miles README describes this design: decoupled workers, configurable on-policy or off-policy schedules, and fast weight updates to the SGLang engines.

The price is staleness. An attempt that started 40 minutes ago was produced by older weights than the ones the trainer now holds. Learning from it is "off-policy". A little staleness is fine; too much makes updates noisy or unstable, so systems cap how old a sample may be and correct for the gap. AReaL, for example, balances rollout and training workers to bound staleness and uses a staleness-aware variant of PPO, and reports up to 2.77 times faster training than synchronous systems on the same GPUs.

The second cost driver is the sandbox. Every attempt needs an isolated computer where the agent can run commands and break things safely, which is why sandboxing now matters as much as GPUs for agentic RL. AgentENV, the base of RadixArk's service, runs Firecracker microVMs that, according to its README, boot or resume from snapshots in under 50 milliseconds and release CPU and memory while idle.

What makes a good RL task

The model only learns from the reward signal, so the tasks are the curriculum. RadixArk's description lines up with four properties that most agentic RL teams now look for.

Verifiable outcome. A program, not a person, must be able to say whether the attempt worked: tests pass, a file has the right contents, a service responds correctly.

Targeted at real failures. RadixArk ran the base model on every candidate task and kept tasks aimed at mistakes it actually makes. Tasks the model already solves teach nothing, like a tutor drilling a student on sums they already know.

Tuned solve rate. If the model never succeeds, every attempt scores zero and there is nothing to compare. If it always succeeds, there is also nothing to learn; GRPO-style methods need a mix of wins and losses. RadixArk says it tuned difficulty until the solve rate fell in "a useful learning range", without giving the exact range.

Non-gameable checks. Reward hacking is when the model finds a way to pass the check without doing the work, for example by editing the test file instead of fixing the bug. RadixArk says tasks that could be passed without solving the intended problem, or that had a design flaw, were fixed or removed. This review is slow work, which helps explain why only about 1,200 tasks survived.

The earlier article on RL on expert-built tasks with Nemotron covers a related idea at smaller scale; DaisyRose's twist is that every task is long and multi-step.

Long-horizon trajectories and context folding

A 1.5-hour session with over 100 tool calls produces a huge transcript. RadixArk reports sessions of up to 1M tokens, which is four times the model's 256K context window.

The fix is what RadixArk calls harness summarization. When the session grows too long, the harness asks the model to summarize what has happened so far, then continues the work from that summary in a fresh context. For training, one long session becomes several shorter trajectories. This is a form of context engineering: deciding what the model needs to see at each step.

It is like a nursing shift change: the incoming nurse reads a handover note, not every chart entry, and if the note omits the allergy, the next shift goes wrong. The model must learn to write good handover notes for its later self.

Long sessions also make credit assignment harder: a single pass-or-fail at the end of 1M tokens must be shared across hundreds of steps and several folded segments. RadixArk does not describe how it assigns reward across segments.

How gradient-free self-editing differs from fine-tuning and LoRA

Ordinary fine-tuning, including supervised fine-tuning and RL, works through gradients. The training code measures how wrong the model was and computes, for every one of billions of weights, which small change would reduce that error. Then it nudges all of them a little, over and over. The site's post-training simulation covering SFT and DPO shows this loop step by step.

LoRA is a cheaper variant. It freezes the original weights and learns a small add-on: two thin matrices whose product is added to a big weight matrix. It still uses gradients, but only for the add-on.

Self-rewriting skips gradients entirely. The edit is a rank-one change, written in the paper as W ← W − α·uuᵀW. In plain terms, the model picks a direction u in its hidden state and removes a small fraction α of what one weight matrix writes along that direction. In one example, α = 0.05 on two output matrices of layer 8 changed them by 0.6% and 0.1% of their size. The biggest gain the authors report came from a different kind of edit: raising a time-step bias in every linear-attention layer by 0.5, which reduced answers that ran out of tokens.

How does the model choose such an edit without gradients? Like a scientist with a small lab budget, it reads its failure report, inspects hidden states, makes an edit, generates answers with old and new weights, compares them, and submits or withdraws the edit. The whole search uses forward passes, the same operation as serving the model, and the final edit adds nothing to the cost of later answers.

If fine-tuning is months of orchestra rehearsal and LoRA is a small backing section added on top, self-rewriting is the conductor deciding the brass is too loud and turning one dial.

That precision is also the limit: a few rank-one edits can remove a bad habit or repair damage, but are unlikely to teach a new skill such as using a new tool.

Table 2. How gradient-free self-editing differs from fine-tuning and LoRA
Aspect RL post-training (DaisyRose) Supervised fine-tuning Self-rewriting (arXiv 2611.21885)
Learning signal Pass or fail on verifiable tasks Example answers to imitate Graded answers on practice problems, judged by the model itself
Compute About 63,000 H200 GPU-hours, plus sandboxes Moderate; scales with dataset size Forward passes only; best sessions a few hours on one GPU (author claim)
What changes All weights, through gradients All weights, or a LoRA add-on A few targeted matrices or biases, via rank-one or bias edits
Main strength Teaches new multi-step behaviours Fast, stable, easy to control Cheap, interpretable, no training stack
Main risks Reward hacking, staleness instability, high cost Copies errors in the data, can reduce diversity Overfitting to practice set, unreplicated, only worked at 27B

Evaluation caveats

The site's evaluation simulation and evals guide explain the general traps; here are the specific ones.

Saturated benchmarks hide progress. The base model already scores 91.3 on SWE Bench Pro v2 and 89.2 on GPQA Diamond. Near the ceiling, remaining points may include broken or ambiguous items, so a 3-point gain can mean less than it looks. The jump that tells you more is Terminal Bench 4.1, from 5.6 to 12.0, because that test is far from saturated and matches the long agent tasks DaisyRose trained on.

Small gains may be noise. HLE text-only moved from 33.9 to 34.3. RadixArk does not report run-to-run variance, and it reports each DaisyRose score as the higher of its BF16 and FP8 runs, which slightly favours the new model. A 0.4-point change on its own should not be treated as a real improvement.

Different harnesses give different numbers. The model card itself notes that comparison numbers for other models come from different sources and harnesses.

Self-reported and unreplicated. All DaisyRose results come from RadixArk. The self-rewriting results come from one group, one model family and mostly math practice data. The paper uses held-out problems and paired tests with fixed seeds, but reports no direct comparison against fine-tuning, LoRA or prompting baselines in the parts we could read.

What it means in practice

For teams that post-train their own models, DaisyRose is a public recipe showing that a relatively small, carefully reviewed task set can move a 27B model on long agent work. The expensive parts include task design, anti-hacking review and a sandbox fleet running over a thousand environments at once.

The efficiency claims, if they hold up, matter more than any single benchmark. Going from 16% to 37% MFU means the same run needs far fewer GPU-hours; RadixArk puts it at 57% less GPU time. An 80% cut in sandbox cost would put agentic RL within reach of smaller groups.

For users, the card says the model runs on a single GPU, and FP8 cuts the weights from about 52 GB to about 29 GB.

Self-rewriting is not a production tool yet. Its interest is conceptual: small, auditable fixes without a training pipeline. Its failure below 27B suggests the skill needs a large model, and the authors say every self-made model should be evaluated for safety and confirmed by a person before use.

What to do now

  1. Test DaisyRose on your own agent tasks. Run it beside Qwen3.8-27B in your real harness with the same tools and time limits, rather than trusting the published table.
  2. Use the recommended serving setup first. Start with SGLang, the DSpark draft model and the stated sampling settings (temperature 1.0, top_p 0.95, top_k 20), then tune.
  3. Pick the precision that fits your hardware. Use BF16 or FP8 unless you have a Blackwell GPU, which the card says NVFP4 requires.
  4. Audit your RL tasks for gameability. Before training, try to pass each task without solving it: edit tests, hard-code outputs, skip steps. Fix or drop any task that falls.
  5. Measure the base model's solve rate per task. Drop tasks it always solves or never solves, so every task gives a useful mix of wins and losses.
  6. Measure GPU idle time before buying more GPUs. If your RL loop is synchronous and tasks vary in length, an asynchronous split may raise throughput more than extra hardware, as long as you cap staleness.
  7. Plan context handling for long sessions. If attempts exceed the context window, design and test the summarization step.
  8. Report variance with every benchmark claim. Run several seeds and show the spread.
  9. Treat self-rewriting as a research lead. If you experiment with it, keep held-out sets the session never sees and require human sign-off on every kept edit, as the authors advise.

Risks and open questions

Will the gains hold outside RadixArk's harness? All scores are company-reported.

How much of the gain is benchmark overlap? The 1,200 tasks were built to fix the base model's failures in machine learning, software engineering and science. It is not stated how they were checked against the evaluation sets.

Is asynchronous RL stable at longer horizons? RadixArk does not publish staleness limits or reward curves.

Does self-rewriting generalize beyond math and one model family? The paper tested Qwen3.5 only, mostly on math practice data.

Could a self-editing model optimize for the wrong thing? If a model chooses and judges its own edits, a weak test can approve a harmful change. The paper criticizes its 9B model for testing only on problems it already failed.

Watch next: independent evaluations of DaisyRose-27B on Terminal Bench and other agent benchmarks, any release of RadixArk's internal training backend or task set, replications of self-rewriting on other model families and non-math tasks, and direct comparisons of self-rewriting against LoRA fine-tuning at equal compute.

Frequently asked questions

Is DaisyRose-27B a new model or a fine-tune?

It is a post-trained version of Qwen3.8-27B. It keeps the base architecture and context window and changes the weights through reinforcement learning only, with no supervised fine-tuning or distillation, according to RadixArk.

Why does asynchronous RL make training faster?

In a synchronous loop, every rollout GPU waits for the slowest attempt, and trainer and generators take turns. Asynchronous training lets both run at once, at the cost of some training data coming from slightly older weights.

Did RadixArk's run take 18 days or several weeks?

Both statements appear in the blog, but they are consistent. 144 GPUs running for 18 days equals about 62,000 GPU-hours, close to the stated 63,000, so "weeks" means roughly two and a half weeks.

How can a model change its weights without gradients?

In the self-rewriting paper, the model writes code that reads its weight matrices and applies small, targeted edits, such as removing a fraction of what a matrix writes along one direction. It tests each edit by generating answers with old and new weights and keeps only edits that help. All of this uses forward passes, the same operation as normal inference.

Should I use self-rewriting instead of LoRA?

Not yet. The method has been shown by one group, on one model family, and it only worked at 27B parameters. LoRA remains the reliable choice; self-rewriting is a research direction for small, targeted fixes.

Summary

DaisyRose-27B and self-rewriting show two ends of post-training. RadixArk spent about 63,000 H200 GPU-hours on asynchronous agentic RL over roughly 1,200 reviewed, failure-targeted tasks, and reports gains on every benchmark it lists, most clearly on the unsaturated Terminal Bench 4.1. The self-rewriting paper lets a 27B Qwen3.5 model edit its own weights with forward passes alone, reporting a 6.6-point gain on held-out math and repair of a damaged copy, but only at the largest size and without replication. Both should be judged on independent evaluations, not announcement numbers.

Sources

Cite this article

Kelahmet, Ö. C. (2026, November 21). DaisyRose-27B and self-rewriting weights, two new ways to improve an open model after training. Özer Cem KELAHMET. https://www.ozercemkelahmet.com/articles/daisyrose-rl-self-rewriting-weights

BibTeX
@online{kelahmet2026daisyrose,
  author = {Kelahmet, {\"O}zer Cem},
  title  = {{DaisyRose-27B and self-rewriting weights, two new ways to improve an open model after training}},
  year   = {2026},
  month  = {nov},
  url    = {https://www.ozercemkelahmet.com/articles/daisyrose-rl-self-rewriting-weights}
}
Version historyfirst version
  1. Published

Factual corrections are listed here with what changed. Wording and formatting edits are not listed.

Last updated Nov 21, 2026
Type to search.
    ↑↓ navigate↵ open