Hand a computer a photo and ask it to cleanly cut out "the glass the person on the left is holding." It does two things in sequence. First it finds where that glass is, then it cuts along the boundary.
The cutting part has two stages of its own. Segmentation marks the whole region the glass occupies in the photo. Everything over here is glass, everything from there on is background. Matting goes a step further. It assigns, pixel by pixel along the boundary, how transparent each one is. The background showing through the glass is half glass, half background. The result that records this "half" as a number between 0 and 1 is the alpha matte. For a strand of hair or the transparent rim of a glass to composite cleanly, segmentation's "region" isn't enough — you need matting's "transparency."
Until now, locating and cutting out have usually been split across separate models. WeVisOne-VL, released by Tencent's WeChat Vision team, is a VLM (vision-language model — an AI that looks at an image and answers in words) that does both inside a single model. And it does one more thing. When the thing being searched for isn't in the photo, instead of forcibly pointing somewhere, it answers "not here." That is the thread this article follows to the end: a design that makes the model's answers — especially the "not here" answer — trustworthy to the user.
The model handles six tasks. It's easiest to see them in two groups.
"We unified six tasks into one model" is easy to misread. It does not mean every output is stamped out in the same shape. The target and coordinates are specified by the language model as tokens (the basic units a model reads and emits, like fragments of text), while the pixel region is drawn by a dedicated decoder that takes that information. What is unified is not the shape of the output but the path. Whatever the task, it takes instruction the same way, holds the target in a single representation, and from there extracts whichever form is needed — point, box, mask, or alpha matte. If the target isn't in the photo, that slot is filled with "none."
When you need a pixel mask, there's a common approach. The VLM produces up to a box, and that box is handed to a separate large segmentation model like SAM, which returns a mask. Stitching two models together this way comes at a cost.
One cost is weight. Stacking on another model makes the whole thing that much slower and heavier.
The other is nastier. There is no guarantee that the target the VLM pointed at — "that glass" — is the same thing the segmentation model actually cut out. The VLM may have meant the left glass while the segmentation model traces the outline of the water cup beside it. The two have different eyes, and all they exchange in the middle is a single box. When the target named in words and the target cut out in pixels diverge, then no matter how refined the result, you've cleanly cut out the wrong object.
WeVisOne-VL draws the signal it passes to segmentation not from an external model, but from the representation the VLM itself has just produced. Because the one pointing and the one cutting out use the same representation, there's less room for the two to point at different targets. That is the reason for folding the two models into one.
The flow reads as a single line.
Point at the glass → represent that target with a box and one <OBJ> token → predict the region and transparency from that representation.
The language model decides "what, and where," and emits a box. Right after, it produces one more special token, <OBJ>. The internal state of this token serves as a compressed representation of "the very target just pointed at." A lightweight pixel decoder then takes this representation, the image features, and the box just produced, and draws the mask or alpha matte.
The role of the box here is worth noting. To the pixel decoder, the box is an instruction: "focus your attention around here." So if the pointing stage puts the box in the wrong place, the decoder traces a refined outline in the wrong spot. You get a mask that looks plausible but has the wrong target. However good the back end is, this error is not recoverable. That is why, in this design, how accurately you locate sets the ceiling on region quality. If the pointing wobbles, everything beneath it wobbles.
A language model emits tokens one at a time, in order. If you write a coordinate as an ordinary number — like "this cat is at 0.37 across, 0.52 down" — a single number gets split into several tokens and the output grows longer. WeVisOne-VL pre-loads its vocabulary with dedicated tokens from <0> through <1000> to represent coordinates as ratios between 0 and 1. Each coordinate ends in a single token, so the output is shorter and processing cheaper.[^1]
The task determines the output format. Need a box? Four coordinates. Need a point? One pair. If the target isn't there, the coordinate slot is filled with None. "None" is neither an error nor an empty response — it's one of the outputs the model formally produces. The refusal training discussed later aims squarely at this None.
<OBJ> TokenSegmentation and matting have to cover the entire image pixel by pixel, so they don't end in a handful of tokens. The <OBJ> token is the bridge between the two. The decoder confirms "which target" from this token, then traces the outline with reference to the image features and the box.
What this box solves — without an external segmentation model The pixel decoder, including its refinement module, is light at roughly 9.6M parameters. Without bolting on a separate large segmentation model like SAM, it produces both box-level sparse predictions and pixel-level dense predictions from one model. The key is that the decoder is designed to begin its "find this target" query from the
<OBJ>representation.[^2]
Training climbs from the easy to the hard. It first establishes the basics — following instructions and emitting format-compliant output — then uses high-resolution data to firm up not missing small objects and crowded scenes, and finally adds the pixel tasks to broaden from locating to carving out regions. On top of that, it layers reinforcement learning.
What this box solves — folding hallucination into the training objective In the reinforcement learning stage, the model picks the samples it finds most confusing and raises locating accuracy while suppressing the habits of inventing things that aren't there or repeating the same prediction. In other words, reducing hallucination lives inside the training objective, not as a filter bolted on afterward.[^3]
Generality on this scale only comes atop a vast amount of data. That's why nearly half the paper goes to the data engine (WDE). WDE pulls in public data like GRIT, CC3M, LAION, and SA-1B to stock over 100M images and around 3.6 billion bounding-box annotations.
The way it makes annotations reveals the model's character. Dedicated detectors and VLMs have opposite dispositions. Detectors (YOLO, DETR families) churn out dense candidates in crowded scenes but are weak on never-before-seen concepts; VLMs recognize concepts well but miss things in busy scenes. The engine divides these two by role. Expert models generate candidates in bulk,[^4] and Qwen3-VL verifies, one by one, "does this region match this description." It separates the side that produces many from the side that filters them out. When moving from box to mask and again to matte, it carries over the original target description unchanged, so it points at the same instance from start to finish.
What the engine labored over most is the data that teaches "not here." To make the model say an absent thing is absent, you have to deliberately show it traps. These are called hard negatives. But making traps is tricky. Swapping "red car" for "blue car" doesn't mean there's no blue car in that photo, and a detector finding nothing doesn't mean the target is really absent. So it re-checks each candidate trap against the whole photo, and if the real target turns up, it pulls the trap out and returns it to the pile of normal cases. For a case that passes review as "genuinely absent" where the model nonetheless insists on pointing at something, that case is taken as a hard negative.
To match this, it also includes real targets that are easy to miss (hard positives). Teach only refusal hard, and the model learns the lazy trick of "if unsure, say it's not here." You have to include both as a pair so that refusal doesn't harden into an excuse. Training "refuse when absent" and "capture when present" as one bundle is this engine's counterweight.[^5]
The paper evaluates on more than 30 benchmarks, and the comparisons are against generalist models that do a range of tasks and specialist systems tuned to one. This article doesn't cover the per-task scores. Instead, it points out what you have to look at to evaluate this design.
A single line of average score won't read this model properly. The design chases two goals that pull against each other at once. The harder you set refusal, the greater the risk of missing real targets; the harder you try not to miss, the greater the risk of calling an absent thing present. So the first result to look at is a number that shows refusal and recall (how exhaustively it finds targets that are actually present) together. Making just one high is easy. Whether it holds both at once is what makes or breaks this design.
Which side to tighten is decided by the costs on the ground. On an industrial inspection line, falsely reporting a sound part as defective raises re-inspection costs (false alarm), while missing a real defect ships the defect as-is (miss). Where false alarms are expensive, push refusal hard; where misses are expensive, push recall. To find this point, you have to count false alarms and misses separately on a held-out sample whose answers you know.
The second thing to look at is the profit and loss of integration. Folding several models into one makes operations lighter, but there may be points where it falls short of a task-specialized model. This trade-off has to be measured per task so it isn't hidden behind the average score.
A benchmark is a number on a well-curated test set. It does not guarantee the distribution shift, latency, and cost of an actual deployment environment. That's a limitation common to any model.
The authors took product-level defects — hallucination, misreading intent, and missing inconspicuous targets — as their starting point. It's more accurate to read these defects as "what they worked hard to reduce" than as gone. Follow the design and you can see the conditions under which the balance breaks.
The data engine references itself. WDE makes labels by verifying and filtering the predictions of existing models. It mitigates this with sample checks and separating "model error vs. annotation error," but as long as a large share of labels is filled with pseudo-labels, there's no guarantee it doesn't also inherit the blind spots those expert models share. The very criterion for refusal may stand on that blind spot.
The balance point moves by domain. The refuse-versus-capture balance set by training won't stay in the same spot on every site. Where lighting is poor, where the target differs from the training distribution, or where the cost structure is unusual — which way it tilts doesn't show up in the average score. That's why the previous section's "count false alarms and misses separately on a held-out sample" is needed.
The 2B/4B size. The backbone is Qwen3-VL, and the model comes in two sizes, 2B and 4B parameters. The pixel decoder is light, but the backbone, being a VLM, is by no means small. And parameter count isn't speed. Actual latency and throughput vary greatly with input resolution, output length, and batching. How it fares against a specialized detector in edge or real-time settings has to be measured separately as operational numbers, apart from any "efficiency" claim.
A customer uploads dozens of QA screenshots and hands them to ONTLE's AI PM. "Pull out only the screens where the payment button is misoverlapped." There are two common failures. Under the pressure to "find the overlaps," it invents plausible coordinates even on sound screens (false report), or it lumps ambiguous real defects as "not sure" and drops them entirely (miss). For a PM, both are fatal.
ONTLE is a service that takes over the project management a human PM used to do. What to borrow from WeVisOne-VL is not the pixel decoder structure, but the method of putting "what isn't there is reported as not there" into the product's structure rather than into a single prompt line ("state facts only"). Below is a design proposal, not a feature we offer now, and a vision model's training method doesn't transfer wholesale to a PM agent. Read it as a sketch for the next time the same problem is solved.
The core is splitting the model's None into three branches. A model looking at one screen can render three verdicts, and each demands a different next action from the customer.
To this, borrow two more of WeVisOne-VL's habits. One, evaluate refusal (hard negatives) and capture (hard positives) as a pair, to see whether refusal has hardened into a lazy excuse. Two, measure reports split into false reports and misses. People often add a "look once more before reporting" re-review, but if the same model looks again with the same eyes, it waves the same blind spot through as the first time. For re-review to mean anything, it has to filter with a criterion different from the first judgment — for instance, comparison against a sample whose answers are known.
When the customer can believe that a report of "no overlapped screen found" means not-looked-for but genuinely-absent, that report finally becomes useful.
The HTML version of the paper is marked CC BY 4.0, and the project page (tencent.github.io/WeVisOne-VL), code (Tencent/WeVisOne-VL on GitHub), and weights (the tencent/wevisone-vl collection on HuggingFace) are public. For the usage license of the model weights themselves, check that HuggingFace page.
When you only need the region — marking where what is, or erasing a specific object — a segmentation mask is enough. On the other hand, when you have to composite a target with a semi-transparent boundary like hair, glass, or smoke naturally onto a different background (a clean cut-out), you need an alpha matte that carries transparency values. Matting is more precise but heavier.
The front end is shared. For any task, it's the same up to the stage of pointing at a target and emitting coordinates as dedicated tokens. However, tasks that don't need a pixel region (detection, counting, OCR, pointing) skip the `
[^1]: The approach of using dedicated tokens for ratio coordinates is borrowed from Rex-Omni and LocateAnything.
[^2]: The pixel decoder follows a Deformable DETR design (six deformable blocks), but initializes its query slots directly from the <OBJ> representation rather than from learned fixed values. Intermediate ViT features are tidied up by an adapter, and matting uses one more lightweight refinement module for fine boundaries and semi-transparency.
[^3]: The curriculum layers mixed-task reinforcement learning on top of three-stage supervised fine-tuning (SFT). The RL is on-policy (learning from answers the model produces itself), picking uncertain samples first and then applying GRPO (Group Relative Policy Optimization).
[^4]: Candidate generation uses SAM3, an in-house DINO-family detector, Rex-Omni, and others. Which images to gather more of is steered by a knowledge graph starting from Wikipedia and WordNet. Box-to-mask conversion uses SAM2.
[^5]: Matting labels are made by drawing alpha candidates with BiRefNet and MatAnyone2, filtering them by several criteria, then re-annotating the whole corpus with a dedicated matting annotator (a frozen SAM2.1-Base+ with an alpha-refinement head attached, inspired by SAM2Matting). Each training round classifies errors by type, distinguishing "model prediction error" from "annotation error itself," and fixes the label only when it's an annotation defect. The official benchmark is isolated from the training data.