IntBMoE takes a long-standing dilemma in Mixture-of-Experts (MoE) — the standard recipe for scaling large AI models efficiently — and tackles it head-on. Classic MoE designs can't independently control three things: how many experts contribute to a given input, how much computation actually runs, and how many parameters have to be held in memory during that run. IntBMoE decouples these three, letting every expert contribute knowledge while keeping compute and memory low. The technique is already deployed in the generative recommendation system of AMap, China's mapping service, serving hundreds of millions of users within a 60ms latency budget, and it posted a relative 2.4% lift in UVCTR (click-through rate per user) in an online A/B test.
Think of MoE as a brigade of chefs. When a customer (an input token) walks in, the system decides which of the specialist chefs (experts) to call on. The catch is that running this kitchen carries three distinct costs.
The paper draws a sharp line between them.
Ideally, you'd want "every chef contributing knowledge (participation maxed), only a few hands actually moving (execution minimized), and few new sets stacked in the pantry (materialization minimized too)." But existing approaches couldn't set these three independently.
| Existing approach | Participation | Execution (compute) | Materialization |
|---|---|---|---|
| Sparse routing | Low (shrinks) | Low | Low |
| Dense output-mixing | Full | Grows with the number of experts | - |
| Parameter-merging | - | Stays at one expert | Grows with the number of routing decisions |
In other words, pin down one and another springs a leak. This three-way dilemma is exactly what the paper targets.
The heart of IntBMoE (Integrating Block-Level Conditioning) is to pair "dense expert composition" with "sparse block execution" so all three axes come apart at once. Let's keep the kitchen metaphor going.
1) Keep a fixed menu of blocks (a codebook). IntBMoE doesn't chop existing experts into little pieces. Instead, a learned, compact codebook (a fixed menu of blocks) defines the set of "blocks" available, with each codebook slot mapping to one block. Each block isn't a fragment trapped in a single layer; it's a multi-layer structure spanning several internal layers, and at each layer it is synthesized from that layer's full base of experts (see the next point). The key is that the number of blocks is fixed by the codebook, not by the input. No matter how varied the customers are, the number of composed blocks you have to build and hold during execution is capped by the codebook size. That bounds materialization (the composition cost).
2) Assemble one expert on the fly at every layer. At each internal layer, a lightweight hypernetwork (a small network that produces the parameters of another network) merges all the expert bases in that layer's pool into a single composed expert. Because this composed expert draws on the entire pool, participation is full. Every chef's knowledge is dissolved into one freshly assembled chef, in effect.
3) The router actually computes only a few blocks. Participation is full, but the router sends each token to only a few blocks. So execution (compute) stays sparse. All the knowledge is reflected, but only a handful of pieces actually get worked on.
Thanks to this combination of three mechanisms, participation (full), execution (sparse), and materialization (bounded) are each set without tangling with one another.
On top of this sits one more piece: Dual-Path Residual Gating (DPRG). It combines two independently constructed paths via multiplicative gating (one signal opens and closes the other like a gate and multiplies into it). Rather than simply blending two differently assembled dishes, it weaves them so that one modulates the other — a device for boosting expressiveness.
Here's what the abstract reports. I'm carrying over only the items for which concrete numbers are given.
What stands out here is that the method didn't stop at paper benchmarks — it moved real metrics in a large-scale production service under a strict latency constraint (60ms). MoE research often hits a wall at the question "accuracy went up, but what about deployment cost?" IntBMoE, by deliberately binding execution and materialization low in its design, answers that question with a live service. The code is public too (github.com/AMAP-ML/DreamX-Rec).
Judging from the abstract alone, I'll be honest that the following are still open.
In short, the "design philosophy and production results" are clear, but the "quantitative contribution of each mechanism and the breadth of generalization" are the parts you fill in by reading the body.
ONTLE aims to be an "AI project manager, dispatched" product that fills the labor gap for small businesses and solo founders worn down by repetitive work. So what matters to us is a design that nails both "smartness" and "operating cost" at once. IntBMoE's problem framing speaks directly to that.
First, "decoupling participation, execution, and materialization" works as a metaphor for product architecture too. Agent systems face a similar three-way dilemma, where the product-side analog of materialization is storage: with multiple specialist agents (or tools), you want to "reflect all of their knowledge (participation)," but "calling all of them on every request is slow and expensive (execution)," and "keeping every specialist model loaded at all times blows up memory and cost (storage)." IntBMoE hints that you shouldn't force these three to move together — control each with a separate mechanism. That gives us grounds to design our routing (request → run only the right few tools) and our knowledge integration (reflect the full context) as separate layers.
Second, there's the discipline of "real deployment within a 60ms budget." A live service like ONTLE is judged not by demo accuracy but by effective metrics within latency and cost (clicks, completions, return visits). IntBMoE's approach of bounding execution and materialization at the design stage — which holds once blocks are pre-synthesized and cached under a fixed configuration — lines up with our principle in RAG and agent pipelines: "raise quality while fixing response time and infrastructure cost inside a budget."
Third, the generalization to sequential recommendation is a reference point for the personalization and recommendation-style features ONTLE handles (what to process or suggest next). That said, as the limitations above note, the right call is to adopt it only after small-scale validation on our own domain data. ONTLE's way is to keep the order right — borrow the design principles of a good paper and validate with real measurements, rather than "adopt it on the spot."
Mixture-of-Experts is a structure that, instead of one giant neural network, keeps several smaller expert networks and selects only an appropriate subset for each input. It scales model capacity while staying efficient — you don't compute everything every time — which is why it's widely used in large models.
In classic MoE these three were bound together: raising participation grew the compute, while cutting compute either shrank participation (sparse routing) or swelled the synthesized parameters during execution (parameter-merging). IntBMoE pairs dense composition (reflecting all experts), sparse execution (computing only a few blocks), and a codebook that bounds materialization (the number of composed blocks) to tune each of the three axes independently.
According to the abstract, it's fully deployed in AMap's generative recommendation system, serving hundreds of millions of users within a 60ms latency budget, and it posted a relative 2.4% lift in UVCTR (click-through rate per user) in an online A/B test. It also reports consistent improvements in image classification and language modeling (no concrete figures in the abstract).
Yes, the abstract names a public repository: github.com/AMAP-ML/DreamX-Rec. Check the arXiv/HuggingFace links in the references below as well.