RoboCasa365Leaderboard

Benchmarking generalist robot policies on the multi-task learning benchmark

Leaderboard

Entries coming soon!

1 How to submit

Agentic entries wrap an unchanged base policy with added components such as a planner, memory, retries, or tool use. To submit, open a pull request that adds a JSON file to submissions/agentic/ in the leaderboard repository, following the template in its README. For entries that train or finetune the policy itself, see the Standard leaderboard.

Private information

Everything in the pull request is public, so never put API keys or other credentials in it. After opening the PR, email robocasa@googlegroups.com with a link to your PR, your API key for any closed-API planner (with enough quota for one full eval), and confirmation that closed-source access was granted.

2 Submission requirements

Novelty. Either a new model or a novel agentic harness framework. Using an existing model with an existing coding agent for basic steering does not qualify.
Base policy. Name, HF repo and commit, and license. No weight edits.
Scope. Policy observations only (no privileged simulator state). No ground-truth success signal during an episode. Cross-episode memory must be disclosed.
Reproducibility. Public repo, pinned dependencies, and one-prompt setup. For closed-API planners: provider, model ID, settings, and a test key with enough quota for one full eval. For closed-source code or checkpoints, grant access on GitHub to Abhiram824 and on Hugging Face to Abhi03.
Verification. An arXiv or preprint link. We re-run the full eval, and our numbers are the ones posted.