An AI that makes images or videos usually receives a score for just one finished frame. But getting to that single frame, the model peels away blurry noise a little at a time over dozens of steps. How do you fix those dozens of steps using one score assigned only at the very end? Let me state the answer from DiffusionART, the subject of this post, up front. From the score differences across several finished outputs, it extracts a single "direction that raises the score," and every intermediate step of generation shares that direction. The name ART says it too — Aggregation-First Reward Tilting: first it aggregates the reward into one direction (aggregation-first), then it tilts the generation distribution that way (tilting).
Why this one line matters becomes clear when you see that earlier methods slipped on exactly this point.
Diffusion models and flow matching models (both belong to the family of generative models that start from noise and build data up bit by bit) are originally trained with one goal only: "imitate the given data distribution." Whether it "looks good," whether "the motion is natural," whether "people prefer it" is nowhere in that goal. So a separate stage is bolted on that touches up the model once more using a reward that evaluates the finished output. What runs here is online reinforcement learning. Instead of pre-collected data, it evaluates and incorporates the outputs the currently-training generator just produced, on the spot. The advantage is that you don't need to know how the score is computed — you only need the score that comes out. It doesn't care whether that's a preference button a person pressed or a number produced by an external evaluation model.
The tricky part is that this score is given exactly once, for the finished output. Making one frame takes dozens of noise-peeling steps, and you have to decide how much of that single score to hand to each of those many steps. This is credit assignment — gauging how much each step contributed to the good result and fixing it by that much. The two existing approaches handled this allocation in opposite ways, and each one's weakness came straight from that.
The approach that follows the generation process and fixes it (representative: Flow-GRPO) touches up the very steps the generator actually walked through. The problem is that a step gets graded by the whole finished output's score even though it only partly influenced the result. How much to fix which step is imprecise, so learning is slow.
The approach that re-adds noise to the finished output and learns from it (representative: DiffusionNFT) re-applies noise to the clean output, then makes the high-reward side look more alike and the low-reward side drift away. With no per-step probability computation, it's efficient but learns unstably. So stabilizers get attached, and among them EMA (a device that ties the model to slowly trail a past model) has to have its strength re-tuned by hand every time the task changes. One side is slow; the other side is high-maintenance.
DiffusionART sets its direction from the finished output (the efficiency of the second approach) but applies that direction to the steps actually traversed (the strength of the first approach).
It works in four moves. Generate several outputs from the same prompt → score each with the reward → estimate the common direction the outputs shift toward as the score rises, and build a single "correction direction" → apply that direction to the intermediate steps of generation. The third move is the heart of it, so let's work through an example.
Say you made three candidates from the same prompt. The model doesn't handle these three as the pictures you see. Internally it represents each as a single value made of many numbers — a point in a space, so to speak. The three candidates are three different points in that space. A "direction" refers to which way to push this point.
Now you score the three points. If higher-scoring points lean one way, that leaning side is the direction that raises the score. Here it does not just pick the single highest-scoring candidate and copy it wholesale. It lays the positional differences among the three points alongside their score differences, and distills the common direction by weighting pairs with larger score gaps more heavily. It amounts to a score-weighted average, so it tilts toward reflecting more strongly the component that lifted several candidates' scores together, rather than a quirky detail that happened to show up in just one candidate. That said, this is a computation that estimates such a common component — it is no guarantee that each candidate's idiosyncratic details cleanly cancel out and only the common component remains. (In the appendix's terms, it's a direction estimating "the covariance between reward and output.")
One thing to nail down clearly. This direction does not necessarily line up with image-quality attributes a human can name. There's no guarantee it translates into words like "raise the brightness" or "get the finger count right." It is, at most, an internal quantity pointing the way the score goes up within that score space.
The direction estimated from the finished output is one. But the intermediate steps of generation are each in a different state. The early phase, thick with noise, and the late phase, nearly cleared, hold completely different pictures. Is it fine to push the same single direction into different steps?
DiffusionART is designed so that within one prompt, every step shares a single direction. But this does not mean "each step's direction is originally the same." The paper explicitly states that this shared direction can diverge from each state's (step's) conditional direction, and that the error can accumulate across steps (§4.1 / Appendix B.3). In other words, sharing the direction is not a guaranteed identity but an approximation — applying a single value estimated from the finished output to all steps — and the more the per-step conditional directions scatter, the larger that error grows. If anything, the paper explains that this design of sharing the direction across states works to reduce the phenomenon in NFT where per-state guidance signals canceled each other out and the restoring force came to dominate the update (§4.2).
What it gives up is also clear. It discards the fineness of splitting the correction amount separately per step. But since that splitting was imprecise to begin with and shook learning, the price of giving up fineness is less shaking. The loss and the gain come from the same spot.
DiffusionART's stabilization does two different jobs. One is to avoid producing an unnecessary correction signal in the first place; the other is to pull the model back when it has moved too far.
First, it eliminates the unnecessary correction signal. The approach of re-adding noise to the finished output drags along a force unrelated to reward. Within the computation that re-applies noise, "the velocity the model put out" and "the ideal velocity" diverge (here velocity is not processing time but a value indicating how much and in which direction the generation state changes in one step). This divergence pulls the model off-course regardless of reward. NFT attached more machinery to suppress this force after the fact. Instead of suppressing it, DiffusionART never creates it to begin with. Because it composes the direction from the output and reward alone and doesn't touch that re-noising computation, there is no place for a divergence to arise.
Second, it reverses excessive movement. The real role EMA played in NFT was not a "slowly moving past distribution" but a restoring force that pulls back by as much as the model overshot — this is the authors' interpretation. Rather than tying up a past model separately, DiffusionART splits one batch of data into two non-overlapping chunks. It first moves the model once with the first chunk. When it updates with the second chunk, it anchors the direction computation on "the model before it moved," not "the model that just moved." This setup creates the restoring force. Since the second update takes the pre-movement state as its reference, the farther the model has strayed from that reference, the stronger the pull-back, and the less it moved, the weaker it is. The restoration scales with distance traveled, so unlike EMA there's no dial to set the strength by hand. This is exactly where per-task re-tuning disappears.
(This second function comes with a supporting "masking" that blocks only updates moving away from the reference while keeping updates that move closer. The authors draw a line that this is a supporting device, not a core contribution. The "tug-of-war" structure omitted from the first function and the details of masking are left in "Going Deeper.")
For when you have only "which of the two is better" instead of a score, there's DiffusionART-Pairwise. It pairs outputs and labels win/loss (ties included), and that label stands in for the score. The judgment is that for images and video, where there's no crisp right answer, pitting two against each other is a more natural form of feedback than forcing a score onto them.
The experiments start with text-to-image (T2I) and widen to text-to-video (T2V) and image-to-video (I2V). In T2I, only a small part of the model is trained with SD3.5-M; in T2V, the full parameters of the representative model HunyuanVideo-13B are fine-tuned. The T2V metrics are three — whether it matches the requested sentence (text alignment, TA), whether the frame looks good (visual quality, VQ), whether the motion is natural (motion quality, MQ). These scores are assigned by a reward model called VideoReward. For a fair comparison, the competing method DiffusionNFT was given exactly the per-task hand-tuned settings from its original paper so it could show its maximum performance.
Before reading the numbers, let's line up three units. A "step" of generation is the number of times noise is peeled to draw one frame (different from the "step" below), while a "step" of training is the number of updates that fix the model once. "GPU-hours" is the amount of running one GPU for one hour, measuring the compute cost spent to reach the same score.
How fast does it learn (speed). In the main HunyuanVideo-13B setting, reaching visual quality VQ 5.5 took DiffusionART 24 steps and 63.4 GPU-hours, whereas the most carefully tuned DiffusionNFT (EMA) took 273 steps and 1,401.4 GPU-hours. That's 11.4× fewer training steps and 22.1× fewer GPU-hours. These numbers are a cost comparison "up to the single point of VQ 5.5," and do not by themselves speak to which is superior in final quality.
Does it collapse mid-learning (stability). The "single-update" variant with restoration (the two-update scheme) removed learned fast but then steadily worsened; adding restoration stabilized it over the long run. Masking behaved the same — the T2V experiment with it removed saw performance drop out around step 200 and then recover, while with it in place performance climbed steadily. It's a contrast showing the two devices each guard against a different collapse.
Does it carry over to another model without changing settings (transfer). Moving to the larger FLUX.1-dev (12B), NFT carried the settings tuned for SD3.5-M straight over, and its reward swung wildly as a result. DiffusionART learned fastest with the same settings. This result points not to "higher absolute performance" but to "it doesn't collapse even without touching the settings."
The result closest to practice is the pairwise experiment with only a "choice." Under the 400-step, VQ-only condition, cost and score split like this.
| Pairwise preference experiment (400 steps, VQ-only) | GPU-hours | VQ quality |
|---|---|---|
| Using the score directly | 891.1h | 13.637 |
| Preference comparison (3 candidate-count settings) | 260.8h / 465.6h / 883.3h | 12.235 / 10.883 / 12.092 |
The three settings vary the candidate count per prompt; reducing the candidate count brings GPU-hours down to 260.8h. Even cutting cost this much, the VQ score doesn't open up a wide gap from the 13.637 of using the score directly — it stays in the 12 range. That said, "whether a score gap this size is a small loss to the eye" can't be settled by numbers alone. VQ is a scale assigned by a reward model, and judging perceived quality needs output samples or acceptance criteria of their own. What the table does make clear — preference comparison yields VQ close to using the score directly while being insensitive to candidate count, so when the GPU budget is tight there's room to try cutting the candidate count first.
The efficiency multiples should be read with care. The size of a multiple is sensitive to how well the comparison baseline was tuned, and the stretches where the multiple widens most are when the opposing method collapses or when settings transferred to a task they don't fit. So two claims must be read apart. "Without tuning, the competing method is unstable" (a settings-transfer gain) and "it beats even a sufficiently tuned competing method" (an absolute-performance edge) mean entirely different things in practice. If the former is true, it's valuable as a "lower-maintenance recipe"; only if the latter is also true does it become "just a better method." The authors point to the phenomenon in NFT where per-state guidance signals cancel and the restoring force dominates the update, explain that DiffusionART reduces that cancellation by sharing the direction across states, and draw a line that masking is a supporting device. If you're weighing adoption, what remains is the verification of comparing against a properly tuned baseline for your own task.
ONTLE has its AI PMs take over clients' content production and hand over bundles of several drafts each week. Let's follow one scene to the end. The AI PM uploads this week's client card news as draft A and draft B, and the client picks "A is better." This choice is not a numeric score but a comparison that picks one of two. The input format DiffusionART-Pairwise handles is exactly this.
So what ONTLE can do right now is not model training but designing records so this choice can be used later. Three records suffice — what was requested (the prompt), which candidates went up, and which one was chosen. From these three alone the win/loss label is built. "Why it was chosen" is not mixed in here; it's attached separately as information for quality analysis. This decision — to capture feedback as pairwise choices rather than scores — can be done today.
Moving on to training is the next story, and three conditions have to clear. Whether enough choice records have accumulated. Whether we can touch the weights of the generation model the client uses, or whether it's an external API and thus walled off. Whether the fine-tuning and inference GPU costs fit within the dispatch unit price. Until all three turn to "yes," today's one task is to gather the choice logs neatly.
The following is a summary for readers who already know the transitions and the KL concept in diffusion/flow matching, and covariance.
These are different experiments, so don't read them together as a single achievement. On HunyuanVideo with VQ:MQ mixed equally, the GPU-hours to reach the same average reward were 12.02× and 11.72× fewer than DiffusionNFT and Flow-GRPO-Fast respectively, and average training reward was +28.8% and +53.0% over the two baselines. In the main setting with TA:VQ:MQ balanced, during training it reached VQ 6 in 29 steps and MQ 3 in 46 steps (the three baselines did not reach these within the recorded range), and final evaluation scores were VQ 8.970 and MQ 4.224, which is VQ +31.4% and MQ +57.4% over the strongest baseline.
The paper specifies a GitHub repository (github.com/vanzll/DiffusionART) and a project page (vanzll.github.io/diffusionart-project). Some implementation details and pseudocode are included in the main text and appendix.
Both are alike in using group-wise rewards. That said, in a language model the tokens are the output, so credit assignment is relatively simple. The hard part in this paper lies in how to reflect a score across the dozens of denoising steps that make a single image or video, and DiffusionART approaches it by giving each step a shared direction instead of a scalar score.
The paper targets rectified flow, which uses linear interpolation between data and noise, but states that this shares its core principle with diffusion. The authors use "diffusion" as an umbrella term, so it reads as having room to apply to the same linear-interpolation family.
Not necessarily. The T2V experiments in the main text used scores assigned by a reward model like VideoReward for training and evaluation. If human preference is available, an additional option is that it can be fed into the Pairwise variant as win/loss labels.