Artificer by Nua: Frontier robotics task evaluation in 4B parameters
We’re happy to introduce Artificer, our first robotics decision model for evaluating robot task progress and success.
The problem
Robot systems need feedback during training, evaluation, and deployment. An arm might grasp the right object, but place it in the wrong bin. A robot might have only completed part of its task before it stopped. Assessing these outcomes from video often requires manual review or brittle, handcrafted objectives. Though frontier models do well in evaluation, they are expensive and slow.
Artificer is a 4B parameter model that estimates task completion and progress, at the frontier level, beating out Opus 5.5 and GPT-6.1 Sol and GPT-6 Astra. During deployment, these estimates could provide feedback to a robot controller deciding whether to continue an attempt, retry, or move to the next task.
Evaluation
We evaluated Artificer on RoboReward by comparing task-completion scores with reference scores and weighting each dataset’s average error equally. We adapted ArmNet268 for success detection from recorded video, using a fixed threshold and balanced accuracy.
Interested? Check us out at usenua.com!
Know a robotics team working on task evaluation or success detection? We’d appreciate an introduction!
For the robotics founders: we’d love to test Artificer on your task videos!
We’d love to chat more over email: sunny@trynua.dev.