Vidu S2 is a video generation model that lets users step into a flowing video in real time to talk to it and edit it. The interaction itself—intervening live while generation is still in progress—was already opened up in the predecessor Vidu S1, and S2 pushes two directions further on top of that. One is a conversational character whose expressions and movements shift on the fly in response to the other party (Vidu S2-Avatar); the other is real-time editing that swaps the clothing and background of a video stream that is currently being broadcast, right where it is (Vidu S2-Editing).
Most video generation AI works on a "made-to-order" model. You submit a prompt as your order form, and a long wait later a single finished video comes out whole. Quality has improved, but it's hard to jump in mid-production and say "have them wave here" or "change the outfit."
This approach is especially frustrating in two situations.
The Vidu series has aimed squarely at this point. The predecessor Vidu S1 already opened up real-time interaction, letting you break in at any moment during generation via voice instructions, and the crux of Vidu S2 is that it substantially raises real-time 720p quality, dynamic references, instruction following, and stream editing on top of that.
One analogy for understanding Vidu S2 is the shift from "recorded broadcast" to "live broadcast." A recording can only be touched after editing is finished, but a live broadcast has to be intervened in while it's going out. The paper approaches this along two tracks.
The Avatar model creates digital characters that move in real time. There are three improvements.
| Capability | In plain terms |
|---|---|
| Real-time 720p generation | No waiting—video comes out on the spot at fairly sharp quality |
| Dynamic references | The "reference information" the character draws on can be swapped out at any point mid-stream |
| Enhanced instruction following | It understands and carries out requests like "do a dance" better |
Here "dynamic references" is the point especially worth watching. Where conventional approaches locked in a single reference at the start and rode it to the end, Vidu S2-Avatar can keep swapping references in during generation. In the paper's wording, they can be "updated at any moment," and this property becomes the foundation for real-time interaction.
But this is exactly where an interesting tension arises. A streaming video lives or dies by visual continuity—the natural flow from one frame to the next. Meanwhile, "swapping the reference at any moment" means pushing new input into the middle of that flow. If, the instant you change the reference, the person suddenly jumps to look like someone else or the motion breaks, the advantage of real time collapses. In other words, the problem this model actually has to solve the hard way is not "being able to change the reference," but keeping continuity so that before and after still connect even when the reference changes. It amounts to reconciling two mutually opposing demands—changeability and continuity—within a single stream.
The Editing model edits an already-flowing video stream in real time. Four editing types are specified.
Because it changes the background and clothing at the very moment the broadcast is going out—rather than in post—it carries the same continuity problem the Avatar faces, now on the editing side. The rule is: change it, but don't let it break.
On top of this, the paper states that it explored the feasibility of real-time "spatial video" generation on both the Avatar and Editing sides. Spatial video usually means video with a sense of depth that looks immersive on VR and 3D displays. That said, since the phrasing is "explored feasibility," it's more accurate to read this as a direction for expansion than as a finished feature.
The achievement is clear but restrained. The key sentence is as follows.
"Experiments show that Vidu S2 outperforms all baselines."
And an online demo that anyone can try for themselves is available (https://vidu.com/vidu-stream). Real-time, interactive qualities are the kind of thing that's hard to convey in words alone, so the very fact that they provided a playable demo alongside the paper is a meaningful signal.
Meanwhile, the abstract contains no concrete figures (exact latency, frame rate, benchmark scores, magnitude of improvement). So the facts we can confirm are roughly these.
To sum up the significance: Vidu S2 is an attempt to push further the Vidu series' trajectory of moving video generation from "a tool for cranking out finished products" to "an interface you operate together with, in real time." It suggests that video generation can become a building block of interactive services, beyond being a content-creation tool.
This "break in while it's flowing" structure isn't a video-only problem. ONTLE is a SaaS that dispatches an AI project manager to each customer project, and the situation the AI PM faces several times a day has exactly this same shape. Let's follow it through a hypothetical scene.
Mid-meeting on a Wednesday afternoon, the customer says: "I think we need to pull the deadline in by two weeks." What happens to the AI PM at this moment is structurally similar to Vidu S2 swapping a reference in the middle of a stream.
The design problem shared with Vidu lies in step 2. Both systems are streaming structures where new input breaks into an in-progress state, rather than being made all at once and done, and so both solve the same assignment: "change it, but keep it connected to what came before." The video must not have frames that jump; the PM must not have already-made promises and completed work that jump. The act of swapping a reference is the deadline-change input, and maintaining continuity is the judgment of saving the existing progress.
That said, there are clear points where the analogy breaks down. Vidu's continuity is the visual naturalness of frames at the second-to-second level, and the model matches it automatically on the inside. A misaligned frame can just be re-rendered, so reversing is cheap too. The PM's "continuity," by contrast, is days of human labor and contracted promises—once-abandoned work doesn't get re-rendered, and a badly joined plan remains not as visual awkwardness but as damaged trust. Above all, while the model takes responsibility for continuity in video, a PM's change of plan must absolutely pass through an explicit human confirmation, as in step 4. Real time doesn't mean automatic—here the intuition from the video model is useful, but must not be transplanted as is.
There's a lot the abstract alone can't confirm.
In short, the list of directions and capabilities is clear, but the answer to "how well, and at what cost" has to be verified directly in the demo and the full paper.
Real-time interaction—breaking in at any moment during generation via voice instructions—was already supported from Vidu S1. The key differences in S2 are that it raises real-time 720p quality, dynamic references that let you swap the reference mid-stream, enhanced instruction following, and real-time editing (Editing) that fixes a flowing stream right where it is.
Harder than changing the reference information mid-stream is keeping visual continuity so that the frames before and after don't break once it's changed. Changeability and continuity are mutually opposing demands, so the real task is reconciling the two within a single stream.
Four types are specified—style rendering, clothing replacement, character replacement, and background replacement—and they're applied in real time to an already-flowing video stream.
Yes. A playable online demo (https://vidu.com/vidu-stream) is available. Real-time, interactive qualities are more accurately confirmed by hands-on experience than by text.