Did the robot finish?

RoboJev is an open vision-language model that reads a robot attempt and predicts whether the task was completed.

Try a video Get the weights

P(task completed)0.99Task complete

Task Insert the colourful ring into the central wooden peg.

Recorded model result · ten front-camera frames · task held out from training. View receipt.

95.2%Balanced accuracy · 268 SO-101 runs

0.26 sMedian answer time · one H100

$0.19Per 1,000 answers at the measured H100 rate.
See inputs, costs and all results.

Watch the launch

A robot attempt, the frames that go in, and the answer that comes out.

One task,
ten frames.

Give RoboJev the task and ten frames from the whole attempt, plus the final view from any other cameras. It returns a calibrated probability that the task was completed.

The two ring attempts below were held out from training. With all three views, the released checkpoint scores them 0.995 and 0.022.

Try an attempt yourself

Ten frames from each ring attempt, plus final overhead and wrist views, feed into RoboJev and return completion probabilities. — The images shown are the actual model inputs. These are per-attempt probabilities, rather than benchmark success rates.

Compare the answers

The original glass charts, with the recorded data behind every number.

SO-101 balanced accuracy against measured cost per 1,000 answers. RoboJev: 95.2% and $0.19.

Scoreboard comparing RoboJev with frontier language models and decision APIs.

Median response times: RoboJev 0.26 seconds, frontier APIs roughly 5 to 7 seconds.

The input protocols differ. Released RoboJev reads ten frames plus available final views. The recorded frontier-model runs used four front-camera frames. The explorer identifies every protocol.

The comparisons have limits. The SO-101 gap versus Astra includes a tie. The ten-frame base JEV scores 0.900 unseen-lab AUROC versus RoboJev’s 0.890. RoboJev does not lead every metric.

Open the full comparison: all 29 model/protocol entries

Loading recorded results…

Inspect individual answers and compare any two models

Billions of
robot hours

A million robots recording eight hours a day would produce 2.92 billion hours of video a year.

That is a scale scenario, not a shipment forecast. The chart sets out what different fleet sizes would mean for human viewing and automated scoring.

Illustrative robot fleet scenarios, annual video hours, people required to watch and H100s required to score.

Estimated cost to grade 68,555 sampled LeRobot episodes, totalling about 501 hours of video. — The catalogue contains 68,555 sampled episodes from 7,641 public datasets. Grading costs are estimates; this release has not regraded the entire catalogue. Explore the catalogue.

First page of the RoboJev paper with its accuracy and cost chart.

Read the research

Training, input protocols, held-out tasks, uncertainty, and where the model gets things wrong.

RoboJev starts from JEV-27B-VL and is fine-tuned with LoRA on 52,598 robot episodes. The weights, evaluation data, LeRobot reward plugin and video overlay code are open.

Read the paper

Put it to the test.

Try a video Download the model