A success can contain regression.
Uniform SFT can clone pauses, detours, and unstable corrections hidden inside a successful episode.
2026
Graph-Reweighted Behavior Cloning with Human Interventions for Precise Robotic Manipulation
Find the action chunks that actually move the task forward — even when the data includes detours, failures, and human recoveries.
Demonstrations
Interventions
Failed rollouts
INTERACTIVE TRAJECTORIESProgress, under
Progress, under
the microscope.
A successful trajectory is rarely a straight line. Open the collection, follow four real-world rollouts from three synchronized cameras, and unfold the details of every setback.
Loading the trajectory observatory…
Reading the signal. Graph progress is a graph-derived estimate, not human-annotated ground truth. The default curve uses a centered 15-frame moving average; select Raw trace to inspect the original values. Dip values follow the selected curve. Transition weights use their own 0–1.5 scale. Pale orange marks recorded human-intervention frames; the rose-tinted dip windows follow the supplied event annotations. Captions describe visible motion and possible interpretations, rather than verified causes of the progress change.
Project video
GR-BC in the real world.
A two-minute view of precise manipulation, backbone generalization, iterative training, and robustness evaluations.
Full project video
02:21 · sound available
Why it matters
Mixed-quality experience becomes useful supervision, without learning another model.
- 91%
- average final success across the four tasks in the iterative X-VLA study
- 4
- real-world long-horizon precise manipulation tasks
- 0
- reward models, action critics, progress labels, or VLM annotations
Abstract
Judge actions by their consequence, not the trajectory label.
Long-horizon manipulation data is rarely clean. Even successful demonstrations can contain hesitations, overshoots, failed attempts, and imperfect recoveries. Intervention-assisted rollouts mix autonomous mistakes with delayed expert corrections, while failed episodes can still contain short segments that reconnect the system to a viable path.
GR-BC builds a task-conditioned goal-reachability graph over this mixed-quality data. Reverse graph search estimates the minimal remaining steps to success inside the dataset support, and each physically executed action chunk is weighted by how much it reduces that state potential. New rollouts are appended to the graph and reused in the next offline training round.
The challenge
Trajectory outcome is too coarse for chunk-level credit.
Whether an action is worth imitating depends on what it causes from the current state — not whether the overall episode eventually succeeded or failed.
A failure can contain recovery.
Discarding the whole rollout wastes valid, task-advancing chunks that reconnect to successful regions.
Approach
One graph. One reverse search. Better supervision.
GR-BC leaves the VLA architecture and supervised objective intact. The graph only decides how strongly each observed action chunk should contribute to training.
Build the reachability graph
Multi-view MCR embeddings retrieve candidate neighbors. Mutual-view agreement, a gripper-state hard filter, and locally normalized similarity edges keep connections conservative.
Search backward from success
Reverse Dijkstra search to a virtual success sink yields the dataset-supported remaining-step potential Rrec(s), then interpolation makes it dense.
Reweight executed chunks
Net progress Δt rewards forward consequence; regression depth filters unstable chunks. Unreachable states receive zero training weight.
Real-world tasks
Precision, contact, and recovery over long horizons.
Four tasks cover object search, visual alignment, contact-rich execution, and recovery from external disturbances or previous mistakes.
Assembly Model
Graphics Card
Screw Tightening
Memory Stick
Progress evidence
A progress signal that reacts to local mistakes.
GRP follows task completion while dropping at detours, reorientation, and slip-outs — the local failures that a monotonic trajectory score can miss.
- 0.97
- success–failure progress gap ΔSF
- 0.92
- localized deviation detection ρdev
- 30/30
- successful episodes classified
- 30/30
- failed episodes classified
| Method | ΔSF ↑ | ρdev ↑ | Failure data | Progress labels |
|---|---|---|---|---|
| ReWiND | 0.003 | 0.00 | — | — |
| SARM | 0.24 | 0.48 | — | Required |
| Robo-Dopamine | 0.29 | 0.59 | — | Required |
| RoboMeter | 0.159 | 0.69 | Used | Required |
| GRP (ours) | 0.97 | 0.92 | Used | None |
Policy improvement
91% average final success.
In the iterative X-VLA evaluation, GR-BC reaches the highest final success rate on all four tasks and improves more consistently than DAgger, AWR, and reward-aware BC.
Explore the final round
Success rate by task
Graph ablation Why the conservative graph matters
98.3%failed-rollout node reachability
25.0%zero-weight chunk ratio
0.42pre-intervention / normal weight
0.0%gripper-mismatched edges
Cross-trajectory edges recover goal-reachable states from failed rollouts. The gripper-state constraint removes incompatible visual shortcuts while preserving useful recovery connections.
Limitations
The signal is only as reliable as the graph.
Incorrect visual neighbors can create shortcut paths; conservative filters can also miss valid recovery bridges. Multi-view agreement and gripper consistency reduce common errors, but they do not certify physical reachability.
GR-BC is intentionally dataset-supported and has limited generalization to states far outside the collected experience. Stronger reachability tests and broader embodiments remain future work.
Citation
Cite this work.
Use the BibTeX below to cite this work.
@inproceedings{li2026grbc,
title = {Graph-Reweighted Behavior Cloning with Human Interventions
for Precise Robotic Manipulation},
author = {Li, Xiaoman and Yang, Zhiqiang and Hu, Jiayu and Kong, Jipeng and
Zhang, Yang and Zhang, Jingwei and Yan, Fangzheng and
Shen, Chunhua and Bai, Chenjia},
booktitle = {Conference on Robot Learning},
year = {2026}
}