Why we build simulation from real facilities

This post lays out the reasoning behind Trekion in one place. Why robot data is structurally expensive, why the real world does not work as a test bench, why most simulation fails to close the gap, and what we think it actually takes to build a simulation you can make deployment decisions on. Where there are public numbers, I cite them.

1. Robot data has a cost structure problem

Language models had an unusual advantage: their training data existed before anyone tried to train on it. Text, code, and images had been accumulating on the internet for thirty years, and the labs that built the big models mostly had to collect and clean it.

There is no equivalent stockpile of physical interaction. Nobody has been recording joint torques, gripper contacts, and camera streams from millions of hours of real manipulation and navigation, in a consistent format, for decades. So every hour of robot experience has to be produced on purpose. That means a robot occupying real space, a person supervising it, wear on hardware, and access to a facility that has better things to do than host experiments. If you buy the data instead of collecting it, the invoice reflects all of the same costs plus a margin.

This is not a temporary situation that better sensors or cheaper robots will fix. Producing physical experience consumes physical resources. The cost per hour can come down; it does not go to zero. Any plan for scaling physical AI has to answer the question of where the experience comes from, and "we will collect it on real hardware" is not an answer that scales.

2. Driving already went through this

Autonomous driving hit this wall a decade ago and the industry's answer is well documented. Waymo has written publicly about accumulating on the order of twenty billion miles in simulation, against tens of millions of miles on real roads. The real miles matter, but they are the confirmation step. The iteration happens in simulation, where a rare scenario can be replayed exactly, a near-miss can be staged safely, and a software change can be tested against the full scenario library overnight.

It is worth being precise about why this matters for robotics. Driving was, in an important sense, the easier problem for simulation: one vehicle type, one task, and road networks that are heavily standardised. Industrial robotics has hundreds of embodiments, thousands of distinct tasks, and facilities that differ building by building. Less shared structure means more scenarios to cover, and more scenarios to cover means the simulation-to-reality ratio has to be higher, not lower. Whatever multiple driving needed, robotics needs more of it.

The cost asymmetry shows up clearly in evaluation. Genesis AI has published figures from their own work putting a thorough real-hardware evaluation pass at hundreds of hours: one robot, one operator, running continuously, and several such passes if you want to compare checkpoints. The same pass in simulation finishes in well under an hour and can run many times in parallel. Teams that cannot afford the hardware hours simply test less, and the failures they did not find surface after deployment, where they are most expensive to fix.

3. Why the obvious answer usually fails

So the argument for simulation is straightforward. The problem is that most simulation is not good enough to act on, and the ways it fails are specific.

The rendering is wrong. A perception model does not see the world; it sees what a camera sees. Real warehouses have polished concrete that reflects overhead lights, shrink wrap that produces glare, aisles that swing between bright and dark, and cameras with motion blur and sensor noise. Most simulators render something closer to a video game. A vision stack trained or tested against those pixels is being measured against inputs it will never encounter on the floor.

The physics is approximate. Contact is where simulation is hardest. The friction between a worn pallet and a forklift gripper, the way a load shifts when a robot brakes, what happens when a wheel crosses a floor joint. Get these wrong and behaviour that works in simulation falls apart on contact with real objects. This is the classic sim-to-real gap, and it is a physics problem before it is a learning problem.

The site is missing. This is the failure mode people underestimate. Even a simulator with good rendering and good physics is, by default, simulating nobody's actual building. Stock asset libraries do not contain your racking layout, your SKUs, your label formats, your lighting, or the way your floor has worn. A fleet does not operate in the average warehouse. It operates in one specific warehouse, and the conditions that break it are usually specific to that building.

A simulation that fails any of these three does something worse than nothing: it returns an answer that looks rigorous and is wrong. Teams have learned this the hard way, which is why the default posture toward simulation results in robotics is scepticism.

4. The numbers that changed our minds

The clearest public data on this comes from work by SoftServe and NVIDIA with Toyota Material Handling Europe, building perception for autonomous forklifts. They evaluated the same task with three different data recipes, against photographs from a real working warehouse.

A model trained only in a conventional simulator, with stock assets and stock lighting, reached 49.4% recall. In practice that means roughly half of what it needed to detect, it missed. Replacing the stock scenes with world-model-generated synthetic data raised the result to 84.7% recall at 89.6% precision. That is a large jump, and it came from variety and visual realism. But the scenes still belonged to no particular building.

The step that closed the gap was grounding the synthetic data in the specific site: its floor textures, its label formats, its lighting conditions. With that, precision reached 99.5%. That is the difference between a system you monitor closely and one you can rely on.

The way to read these numbers is as a decomposition of where accuracy comes from. Generic simulation contributes the task structure. Large-scale synthetic data contributes visual diversity. And the final stretch, the part that decides whether the system is deployable, comes from the site itself. Each layer is necessary. Only the last one is scarce, because it cannot be downloaded or generated from a prior. Someone has to capture it.

5. What building a trustworthy world involves

This is what our pipeline does, so I will describe it concretely rather than in the abstract.

Capture. We record the working facility: video, photos, and scans covering layout, racking, surfaces, and lighting. The floor keeps operating while this happens. Turnaround from capture to a usable scene is measured in hours.

Reconstruction. Neural reconstruction, splatting and related methods, turns the capture into a scene that holds up from a camera's point of view. The part that took us real engineering effort is that the same scene also has to work for the physics solver: collision geometry, articulation, and metric scale, underneath the photoreal surface. Most pipelines give you one or the other. A scene has to be both at once to be worth testing in, because the perception stack and the control stack are being exercised in the same rollout. We run these scenes in Isaac Sim in real time, and inspecting the camera view and the collision view side by side is the fastest way to see what "both at once" means; there is an example on our home page.

Physics-ready assets. Objects in the scene carry measured properties, not engine defaults: mass, friction pairs against the surfaces they actually meet, articulation where it exists, and collision meshes that reflect real geometry. A standard EUR pallet in our library knows how much it weighs, how it slides on concrete, and how it sits in a gripper, because those numbers came from a real pallet, not from a materials table.

Structured variation. One faithful scene is a start. What testing needs is coverage, so the scene is expanded into thousands of parameterised variants: aisle widths, traffic density, lighting conditions, load states, floor friction. The point is not randomness for its own sake. Each suite sweeps a condition that is known to break fleets, and every run is seeded, so a failure can be reproduced exactly.

6. Evaluation is the honest test

Every simulation vendor, including us, will tell you their world is accurate. The claim cannot be verified from a demo. The only meaningful test is predictive: run a policy in the simulator, run it on the floor, and compare. If the policy that passes in simulation passes in reality, and the one that fails, fails in the same way for the same reason, then the simulation is measuring something real. If not, it is a rendering project.

This is why evaluation is the centre of what we sell rather than an add-on. An evaluation engagement forces our worlds to make falsifiable predictions against metrics the customer already runs their floor on: task completion, error rates, safety events, throughput. The report does not say "the policy scored 0.84". It decomposes each task into subgoals, grades them individually, clusters the failures by cause, and lets you replay any episode step by step. That format exists because of how the results get used: as the evidence base for a go-live decision, by people who will be accountable for the deployment.

It also keeps us honest. Every evaluation produces matched simulation-versus-reality evidence. When they disagree, the discrepancy is a defect in our world, and fixing it is the roadmap. Over enough runs, the accuracy of the simulation stops being our claim and becomes the customer's record.

7. The loop is the product

The pieces above would each be useful alone. What makes them infrastructure rather than a services engagement is the loop between them. A failure on the floor becomes a scenario in the suite. A success confirms the physics. Field data sharpens the assets. Each policy checkpoint gets benchmarked on identical suites and seeds, so progress and regressions are visible version over version, and a regression gets caught in simulation before it reaches hardware.

The consequence is that the worlds compound. Every capture, run, and deployment leaves the simulation more accurate than it found it, across every customer that uses it. An in-house simulation effort, rebuilt separately inside every robotics company, does not get this property. A shared world layer does.

8. Where we are starting

We are starting with warehouse robotics: AMR fleets, navigation, congestion, mixed human and robot traffic, layouts that change quarterly. The problems are facility-scale rather than benchtop-scale, which suits a pipeline built around reconstructing entire buildings. Manufacturing and data centre robotics are next.

If you are shipping robots into real facilities, or operating facilities that robots are coming to, the pipeline described above runs today and we are onboarding early teams. The request-access form on the home page reaches us directly.