The limbs work fine. It's the common sense that's missing.
That's robotics in a sentence these days. Reaching an arm, gripping an object, moving down a hallway—these "motor basics" are learned ahead of time from vast amounts of data (pre-training), and robots do them reasonably well. The trouble starts afterward. Looking at the scene in front of it often isn't enough to settle "so what should I actually do right now?"
There's a reason this problem of "reading the situation" has moved to the center of AI. After large language models (LLMs) succeeded in the world of text, the same question carried over into the physical world. That knack for taking on a new task from a few lines of example—can we bring it back in a robot that has a body?
A recent survey organizes the research tackling exactly this "common sense" problem by where you place the 'channel' through which context flows into action. These are studies in in-context learning (ICL—a way of changing behavior not by retraining the model but by feeding the examples given at that moment in as input), which change a robot's behavior through demonstration and interaction alone, leaving the neural network of an already-deployed robot untouched. The four families are: feeding context directly into the policy input, carrying it over through the geometric relations of objects, routing it through an internal model that predicts the future, and breaking it into procedural blocks to execute. This piece walks through the paper's core skeleton—this taxonomy and its evaluation methodology. For a more detailed comparison of experimental results, I recommend reading the original.
The paper's example is intuitive. A robot packing parts knows how to grip every part. But
The scene is identical, yet the required action differs. No matter how strong the motor skills, this gap doesn't fill itself. The paper calls it the "information gap."
The conventional fix was fine-tuning (readjusting the network's weights with new data). But you have to re-run training for every new task, and the knowledge put in that way can be overwritten or forgotten by later training. ICL's premise is the opposite. New evidence goes in not as something to "learn" but as input "at the moment of deciding." In human terms, instead of memorizing the whole manual, you flip to the page you need and keep it open beside you as you work.
Described in words they sound alike, so let me carry the single "part-packing" task above all the way through, comparing how each of the four methods takes it on. Looking at the same scene: what information it takes as input, how it changes the action, and where it stumbles.
First, context-conditioned policies. This feeds context—demonstrations, interaction—directly into the "input" of the rule (the policy) that looks at a situation and picks the next move. The form of the context fed in is not just one thing: it could be the demonstration video itself, a representation distilled into object keypoints or a relation graph, a retrieval result that pulls in an action that worked in the past, or the history of interaction so far. Whatever the form, it packs the parts in order "the way the context it just received says to." So it's strong at "do as you saw." The downside: when a new part not in the received context, or an arrangement it's never seen, pops up, it wobbles for lack of anything to go on.
Second, geometric demonstration transfer. Watching the same demonstration, this side extracts not the image but the "3D coordinate relations" between the parts and the box. It pulls out relations like "so that this part's notch faces the box's left wall." Since it holds only the relation, it can reapply even when the shape differs a bit, so it applies the same rule to parts of a never-before-seen form. That said, context that doesn't reduce to a coordinate relation—like "it's fragile, so put it on top"—is easy to miss.
Third, world-model-based control. It first runs it in its head—"if I put this part in first, will the next part's spot get blocked?"—and then moves. Its strength is being able to look ahead before the order gets tangled. The trade-off: if this predictive model is inaccurate, the success happens only in imagination.
Fourth, skill-/agent-based execution. It splits the task into procedural blocks—"grip → move → insert"—and carries them out. The steps are as clear as a Lego assembly manual. Conversely, if a new part not in the manual slips in, it can't move on to the next slot.
To sum up, the four methods fill the same gap at different points—policy input, geometric relation, predictive model, skill procedure. So the question converges to one: does that point truly hold the needed information without letting it leak?
This piece isn't an experimental paper contesting a performance table; it's a survey (a literature review that gathers, organizes, and compares many studies). So numbers like "went up X percent" aren't within the scope referenced. What it delivers instead is a way of thinking.
Running across all four methods are two shared mechanisms the survey keeps returning to—'correspondence' and 'memory.' These aren't the yardsticks the evaluation uses; they're the common machinery each family relies on, in its own place.
On top of these two mechanisms, two core contributions of the paper emerge.
First, the insight of 'relation learning' that makes context usable. Just as you continue a conversation by looking at what came before, a robot too sets its next action on the basis of prior demonstration and interaction. The paper points out that what's decisive in robots is exactly this link between "the earlier teaching" and "the later action it changes." The four methods above were, in the end, differences in where you place this link.
Second, the discipline of splitting evaluation into three. What the evaluation actually distinguishes is not 'correspondence' versus 'memory' but three separate questions: ① does it respond to context (context-dependency), ② does it hold up even when the object and environment physically change (physical transfer), ③ does accumulated experience help the next attempt (the gain from built-up experience). Lump these together and, when a robot pulls off a new task, you can't tell apart whether it was thanks to its originally broad motor skills or thanks to understanding the teaching given on the spot. And then you can't judge whether ICL really works.
Here comes the paper's cleverest point. You can achieve the goal and still break the rule. "Placed the object at the correct destination" may be a result that violated the requirement "it must be moved while gripping the handle." So evaluation has to track not just the intended effect but also the specified order and contact, across the whole execution. An evaluation of "as long as the result is right" is exactly what misses this point.
The picture this survey draws still has many blanks. The four methods seen above are the stage of "adapting through post-deployment interaction, or inferring the requirement from teaching." Above that, the paper sets a farther goal: 'self-improvement,' where experience makes the next round of learning better, and then a stage where that learning is shared across many robots. The paper is clear that these two are not already-achieved results but a "research objective"—a direction to aim at.
Robots exchanging knowledge is particularly thorny, because one robot's motion can't be copied verbatim onto another body. So whether that exchange really lifts the learning of the collective as a whole is still an open question.
What's worth noting is that the frontier of ICL research right now is not about declaring "which approach is most superior." Rather, it's moving toward first asking "what kind of information does this task's gap demand?" and then choosing the matching channel, or combining several channels. The four families aren't competing right answers so much as a toolbox you reach into depending on the kind of gap.
The insight that robot evaluation has to track the 'specified process (order, contact)' rather than the final result becomes, if anything, more important in the world of bodiless software agents.
Say an AI finishes a monthly report. Looking only at the output, it's fine. But if that report went out—externally—skipping the approval step, that's not a success but an incident. It's exactly the same as a robot moving a cup without gripping the handle. So the verification metric for work automation shouldn't be "is it done" but "was it done in the specified order and constraints," tracked across the whole execution.
This is precisely the point ONTLE holds onto in designing its AI project manager. When a client says "anything going out externally must get the person-in-charge's approval first," ONTLE doesn't re-bake that instruction into the model; it takes it as input moment to moment and acts on it. And it judges success not by the 'result' that the report went out, but by whether it passed the 'specified order' of approval. That's why this survey on the robots' gap is just as valid for a bodiless agent.
It's a way of getting a robot to do a new task by leaving its neural network untouched and sticking a 'cheat sheet'—like a demonstration video—beside it. It's the same as flipping open just the page you need and keeping it beside you as you work, instead of memorizing the whole manual.
Fine-tuning re-teaches the robot with new data, 'putting' the knowledge into its head (which is why later training can overwrite and erase it). ICL leaves the head untouched and instead lets it 'pull that knowledge up and look at it beside itself at the moment of need.'