🤖 RESETTLE
Robotic Recovery through Disagreement-Triggered Retrieval and Efficient Corrective Control
Yuxin Chen · Senqiao Yang · Zixuan Wang · Jinhui Ye · Changsheng Lu
Pengguang Chen · Shu Liu · Zhuotao Tian · Jiaya Jia
✨ Overview · 🎬 Demo · 🧩 Method · 🚀 Getting Started · 📚 Docs · 📝 Citation
RESETTLE_demo.mp4
RESETTLE is a model-agnostic recovery framework for frozen robot policies. It detects persistent disagreement between independently sampled action proposals, retrieves a same-task demonstration reference, and executes one corrective action before returning control to the base policy. See the paper for the full method and experiments.
- 🔍 Detect emerging deviations. Monitor two base-policy action proposals under identical conditioning and intervene when calibrated disagreement persists across decisions.
- 🗃️ Retrieve a useful reference. Use an adapted V-JEPA encoder to find a same-task demonstration and guide correction with its image and state.
- ⚡ Correct and resume. Combine a state-servo prior with a guarded visual residual, without online trajectory optimization or additional vision-language reasoning.
Note
This release covers LIBERO: joint model training, expert goal-bank encoding, an HTTP recovery service, and a guarded executor example for single-tick (K=1) recovery.
Detect → Retrieve → Correct. Two independently sampled base-policy action chunks under identical conditioning are compared at successive decisions. Persistent, calibrated disagreement triggers same-task demonstration retrieval. The retrieved image and state guide a state-servo prior and visual correction for one native action, after which the base policy resumes. This repository covers the LIBERO training, expert bank, and recovery server shown on the right.
Paper motivation: QwenPI on LIBERO-Plus. The plotted training-data P95 and heavier failure tails motivate a two-consecutive-decision trigger. This is a paper result, not a threshold estimated by this package. The two-seed example samples the recovery head twice at one fixed observation to inspect residual variance; those samples must not replace the paper's base-policy proposals or temporal trigger.
RESETTLE/
├── app/ # Training, goal-bank, and recovery-server entry points
├── src/ # Shared models, data transforms, and training utilities
│ └── recovery/ # Recovery models, LIBERO helpers, and guarded executor
├── scripts/ # Train / build-bank / serve launchers
├── examples/ # Two-seed recovery-head sampling example
├── configs/ # LIBERO training and recovery configuration
├── docs/ # HTTP API, runtime contract, and integration guides
├── tests/ # CPU regression and entry-point checks
└── assets/ # Method figures and the original demo video
| Start here | Entry point | Implementation |
|---|---|---|
| Train the recovery model | scripts/train.sh |
app/vjepa_intent_action/ |
| Encode expert goals | scripts/build_bank.sh |
app/build_bank.py |
| Launch recovery inference | scripts/serve.sh |
app/recovery_server.py |
| Integrate guarded execution | Executor guide | src/recovery/guarded_executor.py |
Run the following commands from the repository root. The shell launchers also locate the repository correctly when invoked by absolute path from another directory; use absolute paths for data, checkpoints, and outputs.
Use Python 3.12 and a CUDA-capable PyTorch environment, then install the dependencies:
python -m pip install -r requirements.txtViT-G needs substantial GPU memory. The entry points use the following GPU counts:
| Training (default) | Expert-bank encoding | Recovery inference |
|---|---|---|
| 8 GPUs | 4 GPUs | 1 GPU |
Supply a compatible V-JEPA2-AC ViT-G initial checkpoint and the four LIBERO LeRobot expert suites beneath one directory:
/path/to/libero/
├── libero_object_no_noops_1.0.0_lerobot/
├── libero_goal_no_noops_1.0.0_lerobot/
├── libero_spatial_no_noops_1.0.0_lerobot/
└── libero_10_no_noops_1.0.0_lerobot/
Each suite must contain meta/, data/, and videos/. Models and datasets must be supplied separately; the package does not download them.
export PYTHON=/path/to/python
export LIBERO_DATA_ROOT=/path/to/libero
export VJEPA_PRETRAIN=/path/to/vjepa2-ac-vitg.pt
export RESETTLE_OUTPUT=/path/to/new_output_directory
export NUM_GPUS=8
bash scripts/train.shTraining writes latest.pt and params-pretrain.yaml under RESETTLE_OUTPUT; start in an empty output directory. The context encoder, predictor, and residual head are trained jointly.
⚙️ Training configuration details
configs/libero.yaml specifies four suites, 8 frames at 4 FPS, 256px, a goal at frame 4, K=1, guarded_native_steps=5, the Gaussian residual head, and joint context-encoder/predictor training with dynamics, native-action, gate, and saturation losses. scripts/train.sh fills a temporary YAML with the environment's absolute data/weight/output paths. This is not policy-only fine-tuning.
Use the trained latest.pt for both the bank and the server. The four-GPU entry assigns one suite per GPU, writes shards, merges them, and requires 40 task IDs.
export RESETTLE_CHECKPOINT="$RESETTLE_OUTPUT/latest.pt"
export RESETTLE_BANK=/path/to/new_bank_directory
export BANK_BATCH_SIZE=16
bash scripts/build_bank.shThe output contains success_id_bank.npz (features [N,D], states [N,7]), success_id_bank.jsonl, and success_id_bank_summary.json. Expert video paths in JSONL remain external and must still exist when the server retrieves a goal.
bash scripts/serve.sh --device cuda:0 --host 127.0.0.1 --port 18082The launcher uses RESETTLE_CHECKPOINT and RESETTLE_BANK from the previous step.
| Endpoint | Purpose |
|---|---|
GET /health |
Check checkpoint/bank metadata and the verified native calibration. |
POST /plan-recovery-state-servo-residual |
Retrieve a reference and return a proposed state-servo plus visual-residual correction. |
See the HTTP API and runtime contract for request/response fields, state conventions, and checkpoint compatibility.
The server returns a proposed action. The guarded executor converts it to a LIBERO native command, applies translation/rotation guards, and preserves the baseline gripper action. Follow the integration guide to supply the measured current state, schedule interventions externally, and execute at most one tick before re-observation. The example computes an action; it has no robot or simulator connection.
| Guide | What it covers |
|---|---|
| Recovery service | HTTP payloads, checkpoint/bank contracts, calibration, and inference limits. |
| Guarded executor | Single-tick native conversion, motion guards, and caller responsibilities. |
| Two-seed sampling | Reproducible residual-head sampling and its distinction from the paper's trigger. |
With the project dependencies installed, run the CPU regression suite from the repository root:
python -m unittest discover -s tests -vThese checks cover bank merging, recovery sampling, guarded-action conversion, and entry-point wiring. They do not run training or simulator episodes.
If you find RESETTLE useful in your research, please cite our paper:
@article{chen2026resettle,
title={RESETTLE: Robotic Recovery through Disagreement-Triggered Retrieval and Efficient Corrective Control},
author={Chen, Yuxin and Yang, Senqiao and Wang, Zixuan and Ye, Jinhui and Lu, Changsheng and Chen, Pengguang and Liu, Shu and Tian, Zhuotao and Jia, Jiaya},
journal={arXiv preprint arXiv:2610.12185},
year={2026}
}
