Pulling each individual sound back out of a recording where several are mixed together is called audio source separation. It shows up anywhere you have to "untangle a mix" — restoring film soundtracks, remixing music, captioning meeting transcripts.
Before you can feed audio to a separator, you first chop the frequency axis into small pieces. There are hundreds to thousands of those pieces, so feeding them in whole is expensive. That's why today's models merge the pieces into a handful of groups (bands) before feeding them in. One group is the unit of information the model processes at a time — in other words, one token.
The catch is that the boundaries for those groups are fixed in advance by a human. A stretch carrying a bass line and a stretch with a violin solo have their information concentrated in different places, yet the partitions stay identical no matter which sound comes in. This paper keeps the number of slots the same and only moves the partitions to fit the input. It's called Dynamic Band Splitting (DBS).
On DnR v2, a benchmark of film-style mixed audio, a BSRNN using the same 128 bands but with fixed boundaries scored an average SI-SDR of 7.69 dB, while the DBS version that moved the boundaries scored 9.07 dB (SI-SDR measures how close the separated sound is to the ground truth — higher is better). The token count is unchanged. In exchange, as we'll see later, this setting costs more compute.
But contrary to the banner slogan that the model decides "where to cut," the thing that propped up the score the most wasn't the boundaries themselves. We'll get there.
This kind of fix sits between two camps. One nails the band boundaries down at design time and shares them across all inputs (BSRNN, BS-RoFormer, Mel-Band RoFormer, TIGER, and others). The other trains the entire frequency front end from scratch so it can change freely. DBS picks the middle. It leaves the front-end architecture and the original frequency grid intact, and only moves the boundaries between adjacent bands according to the input. Why the middle, specifically? The results answer that.
Convert a sound with the STFT (short-time Fourier transform, which spreads sound out onto a frequency-time grid) and the frequency axis stretches out into hundreds to thousands of bins (the smallest frequency slots). Band splitting groups these bins into a predetermined layout, compressing them down to tens or hundreds of bands. Tokens drop and compute gets lighter, but the grouping spots are always fixed.
Fixed boundaries leak in two places. They either lump components of different character into one wide slot, or they waste fine-grained slots on a stretch with little content. Add more slots and the resolution goes up, but the separator's cost goes up with it. So the thing to fix is clear: hold the number of slots fixed, and only change where that resolution gets concentrated, per input.
DBS moves in three steps.
First, it picks boundaries. Scanning the frequency axis, it gives a high score to spots where "the texture of the sound changes sharply here," and takes a set number from the top. The boundary positions and band widths vary per input, but the count doesn't change.
Next is grouping. It compresses each band — all of different widths — down to one token per band. No matter the width, the tokens have to come out the same size for the separator to take them.
Last is reconstruction, and this is the tricky part. Since a whole wide band got squeezed into a single token, you can't recover the original resolution for free when you reverse it. Values jump like a staircase near the boundaries, and detail lumped into one slot is lost. So DBS uses the boundary probabilities to smooth out the transitions, then mixes neighboring frequency information back in to unfold it onto the original grid.
There's a safeguard so that no stretch ends up receiving zero boundaries. It first divides the frequency axis into four large blocks and allots each block a minimum budget. Boundaries only move within a block. A quiet band doesn't get thrown away wholesale.
That leaves training. The decision to "pick only the top-scoring ones" cuts the gradient at the moment of selection, so normally the final error can't trace back up to the boundary-picking step. DBS actually makes a hard cut to select boundaries at inference time, but during training it approximates the process so the selection scores can be nudged. Thanks to that, however wrong the separation was gets fed back all the way to the boundary predictor.
The real and imaginary parts of each block are processed with a convolution along the time direction, then three magnitude statistics are extracted per frequency bin (time average, peak-to-average ratio, and standard-deviation-to-average ratio). A BiLSTM turns each bin into a context vector (a summary vector that also reflects its surroundings), and a score is assigned to each boundary candidate between two adjacent bins. The score has two axes — a feature term (comparing the learned representations of neighboring bins) and a magnitude term (how much the acoustic statistics of neighboring bins change, which they call the acoustic transition prior). Selection is Hard Top-K: a set number of top boundaries is chosen per block, and the first bin is always kept as a boundary. Bands of varying width are grouped with content-based weighted pooling, sharing the weights across time frames so the tokens come out with the same dimension regardless of width. Training runs the boundary predictor alongside the separation loss via a straight-through estimator (forward pass uses the hard selection, backward pass passes through a soft approximation), and adds a regularization term to hit a per-block target compression ratio.
The reconstruction stage goes through Probability-Guided Smoothing (using the boundary probabilities to smooth out boundary transitions), unfolds the tokens back onto the original STFT grid, and then mixes neighboring-bin features with frequency refinement. From there, a frequency-shared head decodes a complex mask (real and imaginary) and an inverse STFT reconstructs the waveform. Easily confused with smoothing by name, the confidence path is a separate mechanism — it leaves the forward-pass values alone and, on the backward pass, routes extra gradient to the boundary probabilities to help the boundary predictor learn.
Three benchmarks of different character were used. DnR v2 is film-style mixed audio with three sources — music, speech, and sound effects; MUSDB18-HQ is music separation into four stems — vocals, drums, bass, and other (the remaining instruments); EchoSet is two-speaker speech laced with noise and reverb. The analysis bandwidth was deliberately split too. The first two are wideband at 0–22.05 kHz; EchoSet is 0–8 kHz. The design is meant to check whether adaptive allocation works even when the bandwidth differs.
The token budget was fixed per dataset. DnR v2 and MUSDB18-HQ group 1,025 STFT bins into 128 tokens; EchoSet groups 321 into 67 tokens. The split is computed once per inference window and used identically for encoding and reconstruction.
The metrics were matched to each dataset's character. DnR v2 uses per-source SI-SDR and its average, MUSDB18-HQ uses per-stem SDR, and EchoSet uses SDRi and SI-SDRi. That said, these metrics only measure how close the waveform is to the ground truth — not perceived audio quality or latency. So the parameter count and MACs per second (multiply-accumulate operations) are reported separately to surface the cost alongside.
Read in three layers, and it becomes clear how far each number actually speaks.
On DnR v2, DBS-BSRNN scored 9.07 dB, and the fixed BSRNN with the same 128 bands scored 7.69 dB. That +1.38 dB gap is what the whole DBS setup added under the same conditions. The abstract's "29.0% relative improvement" uses a different baseline. It's 9.07 dB against the original BSRNN's reported 7.03 dB, so the two figures shouldn't be mixed. TIGER also went from 8.74 on the fixed version to 10.13 on the DBS version, a +1.39, with all three sources improving.
MUSDB18-HQ (a single model separating all four stems at once, no external data; four-stem average SDR)
| Backbone | Fixed | DBS |
|---|---|---|
| BSRNN | 5.09 | 5.43 |
| BS-RoFormer | 4.50 | 6.34 (+1.84) |
| MelRoFormer | 7.25 | 8.07 |
EchoSet (improvement over the reported reference model)
| Model | SDRi | SI-SDRi |
|---|---|---|
| DBS-BSRNN | +0.46 | +0.43 |
| DBS-TIGER | +0.33 | +0.31 |
MUSDB18-HQ is a setting where one model separates all four stems at once without external data. The conditions differ from music-separation SOTA that uses a dedicated model per stem, so these aren't numbers to line up directly. What we're looking at here is only "does DBS beat the fixed version within the same framework" — and it won regardless of backbone.
Uniform, trained by splitting the boundaries mechanically and evenly, scored 8.00 dB. With the same architecture and schedule, DBS is +1.07 higher. That's the share from "learning to place boundaries to fit the content" (fix them to Mel bands instead and it drops further, to 5.87). Keep the learned weights as-is and, at inference time, swap only the boundaries back to the original BSRNN layout, and it drops from 9.07 to 8.62 — the loss from reverting the boundary positions alone, i.e. 0.45 dB. The boundaries really did move per input. All 64 DnR v2 test mixtures produced different splits, and even neighboring windows within the same recording differed. Conversely, repeating the same input always produced the same split. It adapts to the input, yet the result is deterministic.
Remove the components one at a time and the center of gravity shows itself. Removing the frequency refinement on the reconstruction side dropped it from 9.07 to 3.92 (−5.15). Removing the acoustic transition prior from the boundary scoring was −4.98; cutting blocks from 4 to 2 to 1 went 9.07 → 8.32 → 6.30; using only the statistics without learned features was −2.14; and removing the confidence path was −1.54.
Here's where the title and the results diverge. Picking its own boundaries is this paper's banner, but the loss from reverting only the boundaries at inference (0.45) is far smaller than the loss from removing the single reconstruction stage (5.15). Performance is made jointly by where you group the information and how you unfold it again. Even if you pick the boundaries well, a flimsy unfolding stage bends the score badly.
That said, it's hard to rank the components directly by how far the score drops. Remove one component and another partly takes over its role, or a later stage breaks and can't preserve the earlier stage's gains either. The drop is "how much the whole thing holds up without that component," not "that component's pure contribution on its own."
How the compute changes when DBS is bolted on depends on the dataset and backbone. On DnR v2, DBS-BSRNN (128) has 4.7M parameters — far smaller than the fixed BSRNN's 35.5M — but its MACs per second are 124.0, higher than the fixed version's 76.6. That's the price of predicting boundaries every time and passing through an extra reconstruction stage. Conversely, on MUSDB18-HQ, BSRNN goes 21.8 → 20.8 and BS-RoFormer goes 292.7 → 290.5 G/s, actually dropping, and EchoSet's BSRNN drops sharply from 98.7 → 38.1 G/s. Fewer parameters doesn't always mean lighter, and adding DBS doesn't always mean heavier — the compute depends on the dataset and backbone.
The authors draw their own lines. The number of bands is fixed, and there's a single split per inference window. Even if a sound changes partway through a window, the boundaries stay put for the whole window, so changes shorter than a window can't be tracked. A preliminary variant that also learned the number tended to use up the maximum budget allowed per block. Making the number flexible, they argue, would require a separate constraint on complexity.
The "why the middle, specifically" we set aside earlier gets its answer here, too. The freest option — training the whole front end (a learnable filterbank) — collapsed to −11.72 dB under the same setting. More degrees of freedom buy more expressive power, but that much harder to jointly satisfy reconstruction and an ordered band structure. The compromise of keeping the structure and moving only the boundaries is the choice that sidesteps this difficulty.
ONTLE is a service that attaches an AI project manager to your work. You can't lift DBS wholesale, but one principle is worth borrowing: within a fixed review budget, guarantee a minimum amount of inspection everywhere, and concentrate the leftover share on wherever change is greatest.
Picture a scene. A client sends a message: "move the deadline up by three days." The amount an AI PM can look at deeply in a day (compute, context) is fixed. The default is to run a minimum inspection on every project (like the block budget), but concentrate extra review on this project whose deadline just changed. It looks more deeply at schedule reshuffling, conflicts with upstream tasks, and the owner's available bandwidth. Meanwhile, the other projects flowing along as usual get only a light summary.
The signal to borrow is "how much has the state changed from normal" — things like how often task statuses change, or deadline changes. Whether this pays off gets checked with three numbers. Compared to inspecting everywhere equally: did we miss fewer things (miss rate), what's the review cost of that allocation, and did responses get slower? Just as DBS cut parameters yet raised MACs in some settings, the very act of "deciding where to concentrate" can itself be a cost. If that cost grows larger than the savings, you'd be better off not doing it at all. Application starts here, as a problem to study.
That's the design goal. DBS is built to preserve the number and dimension of the tokens, so it slots right into the input position where an existing backbone used to take tokens. The paper bolts it onto four — BSRNN, TIGER, BS-RoFormer, and MelRoFormer — and all beat their fixed versions. The one condition: the boundary predictor has to be trained together with the separator from the start to be effective.
Because properly restoring a sound needs not only the magnitude at each frequency but also phase (the timing of the waveform). You have to estimate the real and imaginary parts together for the phase to survive.
The paper doesn't separately pin down the reason for per-backbone improvement gaps. The most you can read into it is that the adaptive gain was large at a spot where the fixed version's score (4.50) had been lower than the other backbones.
The paper's abstract publishes addresses for the code and audio demos (https://cslikai.cn/Dynamic-Band-Splitting). The body is available at the arXiv and HuggingFace links below.