AIBuildAI autonomously post-trains a robot world model to near state-of-the-art video prediction in 29 hours
Given a short task brief and a robot manipulation dataset, AIBuildAI post-trained NVIDIA’s open Cosmos world model for action-conditioned video prediction, entirely without human intervention. Twenty-nine hours later, on the official RT-1 trajectory-to-video benchmark, its model reached 25.56 PSNR and 0.845 SSIM, up from the base model’s 17.75 PSNR and 0.716 SSIM.
Under the benchmark’s official short-trajectory protocol, that SSIM score is the highest reported to date (as of August 2026), above every published method including the current record configuration. The PSNR score ranks fourth among published results, within 0.5 dB of the best published trained model. The full solution is open source: the training code, the evaluation harness, and the final model weights.
Along the way, AIBuildAI settled its data discipline before its training strategy: it carved a fixed held-out validation set from the training episodes and scored every decision of the run against them. The benchmark’s test set stayed outside the run, untouched. From there, the agent managed the eight-GPU node entirely on its own. It ran four fine-tuning strategies in parallel, retired the weakest and handed its GPUs to the leader, recovered from a worker crash, and independently implemented and validated a state-of-the-art inference technique from a recent paper. No human touched the run between launch and delivery.
The task: predicting what a robot will see
A world model for robotics answers a simple question: given what the camera sees now and the actions the robot is about to take, what will the camera see next? Models that answer it well can serve as learned simulators for policy evaluation, planning, and data generation.
The standard public benchmark for this task is built on RT-1, a large real-robot manipulation dataset collected by Brohan et al. at Google [1], using the evaluation protocol introduced with IRASim, a fine-grained robot world model by Zhu et al. published at ICCV 2025 [2]. The test set contains 4,799 short video clips. For each clip, the model receives the first frame and the 15 robot actions that follow, and must generate the next 15 frames. Predictions are scored against ground truth with PSNR (pixel accuracy) and SSIM (structural similarity).
The published leaderboard is competitive. IRASim’s strongest variant, Frame-Ada, has held the best trained-model result (26.05 PSNR) since 2024. The current overall record, 26.44 PSNR, comes from FreeAction, a set of training-free inference enhancements introduced by Kim et al. [3] and applied on top of Frame-Ada.
The starting point: an open world model, out of its domain
AIBuildAI started from the action-conditioned variant of Cosmos-Predict2.5-2B, NVIDIA’s open two-billion-parameter video world model [4]. NVIDIA post-trained this variant on a different robot platform and camera setup, so on RT-1 it starts far from the published pack: in the run’s first tuning round, validation scores sat in the mid-teens of PSNR (Figure 2), roughly 10 dB below the published leaders. Closing that gap is a domain adaptation problem: new robot embodiment, new action-space scaling, new scene statistics.
The input to AIBuildAI was the RT-1 training data, a short task brief describing the prediction objective and deliverables, and an eight-GPU (A100 40GB) node. Everything else, including the choice of training method, was left to the agent.
How the agent ran the project
Figure 1: AIBuildAI’s autonomous loop for this run. A manager agent designs and dispatches parallel fine-tuning strategies, evaluates them on held-out episodes, and reallocates compute as evidence accumulates.
AIBuildAI runs as a small organization of agents (Figure 1). A manager agent reads the task, commissions candidate designs, and dispatches coding-and-training workers in parallel. Each worker owns its own code, checkpoints, and a share of the GPUs. After every round, workers report validation scores on held-out episodes they never train on, and the manager decides what happens next: tune further, revise a design, or reallocate resources.
For this task the manager fielded four fine-tuning strategies spanning a capacity spectrum:
- Full fine-tune: update all 2B parameters, sharded across GPUs with FSDP.
- Partial unfreeze: merge a low-rank adapter into the trunk, then unfreeze attention and MLP layers.
- High-rank adapter: keep the trunk frozen, train a large adapter with rollout-aware noise conditioning.
- LoRA continuation: a conservative low-rank baseline as a safety net.
Two moments during the run stand out as genuinely managerial behavior. When one worker crashed on a context overflow, its line of work was respawned and recovered fully one round later. And when the partial-unfreeze strategy failed twice in a row, first overrunning its round budget and then failing to report results, the manager retired it and handed its two GPUs to the full fine-tune strategy, which from then on trained with six GPUs instead of two. Both decisions were made and executed without any human in the loop.
What the agent discovered
Figure 2: Validation PSNR of the four strategies across the agent’s tuning iterations, measured on held-out episodes that no strategy trains on. The full fine-tune pulls ahead once the manager concentrates GPUs on it; the partial-unfreeze strategy is retired after its second failure.
The learning curves in Figure 2 tell most of the story. All four strategies improve sharply in early iterations as they adapt the model to RT-1’s action scaling and visual statistics. The full fine-tune then separates from the pack, climbing steadily for six straight iterations, from 13.7 to 23.4 validation PSNR, while the lighter-weight strategies plateau in the 20 to 21.5 range.
The most interesting move came at the end, and it involved no training at all. Scanning the literature around the benchmark it was chasing, the agent identified the technique behind the published record: FreeAction’s training-free noise truncation [3], which reshapes the initial diffusion noise at inference time as a function of the commanded actions. The agent implemented the technique from the paper, ran an A/B comparison on its own held-out validation protocol, confirmed the gain, and only then adopted it for the final model. Its delivered configuration is the fully fine-tuned network plus this inference-time enhancement, the same recipe class as the current record holder, discovered and validated autonomously.
Results on the official benchmark
Figure 3: The RT-1 trajectory-to-video leaderboard under the official short-trajectory protocol. AIBuildAI’s model (blue) posts the highest SSIM of any published result and ranks fourth on PSNR, within 0.5 dB of the best published trained model.
After the run, the delivered model was evaluated on the full official protocol: all 4,799 test clips, scored with the benchmark’s released evaluation code. Figure 3 places the result on the published leaderboard.
On SSIM, which measures how well the predicted frames preserve image structure, AIBuildAI’s model scores 0.845, above every published method, including the record configuration (0.840). On PSNR it scores 25.56, above the latent video diffusion model LVDM (25.04) by He et al. [5] and IRASim Video-Ada (25.45) [2], and 0.49 dB below IRASim Frame-Ada (26.05), the strongest published trained model. The overall record (26.44) remains with FreeAction’s enhanced Frame-Ada.
Why this run matters
The result itself matters. The way it was reached matters more. The run showed three capabilities:
- Independent world-model optimization at expert level. The agent did not follow a human-authored recipe. It designed four competing strategies, tested them against each other, and converged on a full fine-tune of all 2B parameters combined with an inference-time enhancement. Every step, from first design to final validation, was driven by scores on held-out episodes and is auditable end to end.
- End-to-end project management. The agent stayed in control of a 29-hour run with real money on the meter: a portfolio of parallel training jobs, compute reallocation as evidence accumulated, recovery from a worker crash, and budget-aware stopping.
- Literature-informed engineering. The agent went beyond hyperparameter tuning. It identified the specific published technique behind the current record, implemented it from the paper, and verified the gain empirically before shipping it.
World models are becoming the substrate of robot learning, and adapting them to new platforms is exactly the kind of well-posed, compute-heavy, iteration-driven work that autonomous ML agents are built for. This run suggests the frontier of that work is no longer out of their reach.
What we open-sourced
Everything needed to verify or reuse this result is public:
- Code. The training script, run configuration, inference script, and the official-protocol evaluation harness, at github.com/aibuildai/AI-Build-AI.
- Weights. The final model, at huggingface.co/AIBUILDAI-Inc/robot-world-model.
- Sample predictions. Five generated clips on official test inputs, for a quick qualitative look.
The released model is a derivative of Cosmos-Predict2.5-2B and ships under the NVIDIA Open Model License. Built on NVIDIA Cosmos.
References
[1] A. Brohan et al. RT-1: Robotics Transformer for Real-World Control at Scale. Robotics: Science and Systems (RSS), 2023. arXiv:2212.06817
[2] F. Zhu et al. IRASim: A Fine-Grained World Model for Robot Manipulation. ICCV, 2025. arXiv:2406.14540
[3] S. Kim, S. Lee, and M. Cho. FreeAction: Training-Free Techniques for Enhanced Fidelity of Trajectory-to-Video Generation. CoRL 2025 Workshop. arXiv:2509.24241
[4] NVIDIA. World Simulation with Video Foundation Models for Physical AI. 2025. arXiv:2511.00062
[5] Y. He et al. Latent Video Diffusion Models for High-Fidelity Long Video Generation. 2022. arXiv:2211.13221