There is a video that circulates periodically in robotics circles — a quadruped robot stepping off a treadmill, walking across a cluttered lab floor, climbing a pile of foam blocks, and then descending stairs it has never seen before. The robot moves with a fluency that looks almost casual. What makes it remarkable is not the hardware. The hardware is good, but not extraordinary. What makes it remarkable is that the robot learned almost none of that in the physical world.
It learned it in a computer. Millions of simulated steps, thousands of simulated falls, hundreds of simulated terrain variations — all in software, at speeds no physical training could match, at a cost no real-world test environment could replicate. Then, at some point, the team pressed a metaphorical button and transferred everything the robot had learned into the physical machine. And it worked.
That process — training a robot in simulation and deploying the learned behaviour on real hardware — is called sim-to-real transfer. It has become one of the central techniques in modern robotics, and understanding it is essential for understanding why robots are getting better so fast.
Why Simulate at All?
The obvious answer is speed. Training a robot to walk by having it physically attempt to walk, fall, get up and try again is extraordinarily slow. A physical robot falls at physical speed. It needs to be reset. It accumulates wear. A simulation can run at 10,000 times real speed, run in thousands of parallel instances simultaneously, and reset instantly after every failure without any mechanical consequence.
But speed is only part of the answer. The deeper reason is coverage. A physical training environment, however sophisticated, is a finite set of scenarios. A simulation can generate near-infinite variation: floors of every texture, obstacles of every shape, lighting conditions across every spectrum, unexpected shoves from any direction, joint failures mid-stride. The real world is varied; a good simulation can be more varied still.
"In simulation, you can have your robot fall ten thousand times in a weekend. In the real world, the tenth fall might break something you can't afford to replace." — common aphorism in robotics labs
There is also the matter of data labelling. In reinforcement learning — the technique used to train most modern robot locomotion — the robot learns by receiving rewards for good behaviour and penalties for bad. In simulation, the reward signal can be computed instantly and perfectly: the simulator knows exactly where every joint is, what force was applied, whether the foot slipped, how far the robot moved. In the physical world, extracting that information requires instrumentation that is expensive, slow and imperfect.
The Reality Gap: The Problem Simulation Can't Fully Solve
If simulation were a perfect model of reality, this would all be straightforward. Train in simulation, deploy in the real world, done. The problem is that simulation is not a perfect model. It is an approximation, and the gap between the approximation and reality — called the reality gap or sim-to-real gap — has historically been the central challenge of the field.
Consider what a physics simulation has to model to accurately predict robot behaviour: the exact stiffness of every joint, the precise friction coefficient between foot and floor, the compliance of materials under load, the latency between a control signal and actual motor movement, the thermal behaviour of actuators under sustained use, the effects of air resistance at different speeds. No simulator gets all of this right. Some get most of it right for most cases. None gets all of it right for all cases.
The consequence is that a policy — the set of rules the robot has learned for converting sensor inputs into motor commands — that works perfectly in simulation may fail, partially or completely, when transferred to physical hardware. The robot learned to exploit properties of the simulated world that don't exist in the real one.
A policy in robotics and reinforcement learning is a function that maps observations (joint positions, velocities, contact forces, camera inputs) to actions (motor torques, target joint angles). Training a policy means finding the function that maximises cumulative reward over time. The challenge of sim-to-real transfer is making this function robust enough to work under conditions the simulator did not accurately model.
Domain Randomization: Teaching Robots to Ignore What They Can't Trust
The most widely used solution to the reality gap is called domain randomization. The insight behind it is almost counter-intuitive: instead of trying to make your simulator more accurate, you make it deliberately inaccurate — but in a controlled, varied way.
In practice, this means that during training, the simulator randomly varies the parameters that are hardest to model accurately. Floor friction might be 0.3 in one episode and 0.9 in the next. Joint stiffness might vary by ±20%. Control latency might be 10ms or 40ms. Motor strength might be degraded by a random percentage. The robot is never trained on any single, fixed simulation of reality — it is trained on a vast distribution of possible realities.
The effect is that the robot learns policies that are robust to parameter uncertainty. If the robot has learned to walk reliably on floors with friction values ranging from 0.2 to 1.0, it is very likely to walk reliably on whatever friction value the actual floor presents. The real world becomes just one more sample from the distribution the policy was trained on.
OpenAI demonstrated this elegantly with their Dactyl hand in 2019: a robotic hand trained entirely in simulation that could manipulate a physical Rubik's Cube, including under adversarial conditions like having its fingers tied together or wearing a rubber glove. The domain randomization had been so aggressive that the physical world's quirks barely registered.
The Major Simulators: A Practical Overview
Not all simulation environments are created equal. The choice of simulator significantly affects what kinds of robots can be trained, at what speed, and with what level of physical fidelity. Here are the platforms doing the most serious work in the field today:
| Simulator | Developed by | Primary strength | Typical use |
|---|---|---|---|
| Isaac Sim | NVIDIA | GPU-accelerated, photorealistic rendering, tight Isaac Lab integration | Humanoid locomotion, manipulation, large-scale RL training |
| MuJoCo | DeepMind (via Emo Todorov) | Highly accurate contact physics, fast CPU simulation | Research benchmarks, dexterous manipulation, locomotion baselines |
| Genesis | Genesis team (open source) | Extreme parallelism, 43M steps/sec on a single GPU, multi-physics | Large-scale policy search, data generation, embodied AI research |
| PyBullet / Pybullet Gym | Erwin Coumans / Google | Lightweight, easy Python API, large community | Research prototyping, learning environments, curriculum development |
| Gazebo / Ignition | Open Robotics | ROS integration, sensor simulation, modularity | System-level testing, perception pipelines, mobile robots |
| IsaacGym (deprecated) | NVIDIA | First widely-used GPU physics simulator for RL | Superseded by Isaac Lab / Isaac Sim but still in research use |
The most significant development in recent years has been the shift to GPU-accelerated simulation. NVIDIA Isaac Lab, built on top of Isaac Sim, can run thousands of parallel robot instances on a single GPU — training a locomotion policy that would have taken weeks on traditional CPU simulation in a matter of hours. Genesis, released as open source in 2024, took this further: benchmarks showed it running at over 43 million simulation steps per second, more than 40 times faster than existing GPU simulators for certain tasks.
Speed at that scale changes the nature of what is possible. When you can explore millions of random scenarios in an afternoon, the barrier to training robust policies drops dramatically. Companies that previously needed months of physical testing to harden a locomotion controller can now stress-test it against arbitrary conditions in simulation before the robot has taken a single physical step.
How a Modern Locomotion Policy Is Actually Trained
Let me walk through a concrete example to make this tangible. Suppose you want to train a humanoid robot to walk over uneven terrain. Here is roughly what that process looks like today, using a standard Reinforcement Learning from simulation pipeline:
1. Environment design
You define the simulated world: a flat floor that can be gradually replaced by randomly generated terrain (bumps, steps, slopes). You define the robot model — a precise description of its joints, links, mass distribution and actuators, usually imported from a URDF or MJCF file provided by the manufacturer.
2. Reward function design
You specify what "good behaviour" means numerically. A typical locomotion reward might include: positive reward for forward velocity toward a target, penalty for deviating from an upright posture, penalty for excessive energy use, penalty for foot contact with the floor in unexpected patterns, large negative reward for falling. The reward function is the most important design choice in the entire process and often requires significant iteration.
3. Policy architecture
The policy is typically a neural network — often a relatively simple multilayer perceptron (MLP) or, increasingly, a transformer. Its inputs are the robot's proprioceptive state: joint positions, joint velocities, body orientation, linear and angular velocity, and sometimes foot contact signals. Its outputs are target joint positions or torques. The network has no camera input at this stage — it navigates entirely by feel, like a person walking with eyes closed.
4. Training with domain randomization
The simulator runs thousands of parallel instances of the robot, each with slightly different randomized parameters: friction, mass, motor strength, terrain shape. The RL algorithm — typically Proximal Policy Optimization (PPO) or a variant — collects data from all instances, computes gradients, and updates the policy network millions of times over hours or days.
5. Sim-to-real deployment
The trained network weights are copied to the physical robot's onboard computer. The robot runs the same inference loop: read joint sensors, pass through the network, output motor commands, repeat at 50–200 Hz. If the domain randomization was well-designed, the policy generalises. If it wasn't, the robot does something unexpected — often something entertaining to watch and expensive to diagnose.
The Remaining Limits: What Simulation Still Can't Do Well
Simulation has transformed robot training, but it has not solved it. Several categories of problem remain stubbornly difficult:
Contact-rich manipulation. Simulating the physics of grasping — the way fingers deform a soft object, how friction behaves at the microscopic scale, the compliance of materials under varying load — is significantly harder than simulating locomotion. Locomotion involves relatively simple contact patterns (foot on floor). Manipulation involves complex, multi-point contact with objects of varying geometry and compliance. State-of-the-art simulators handle this poorly enough that many manipulation researchers still rely primarily on real-world data collection.
Long-horizon tasks. Policies trained with RL excel at short, well-defined tasks: walk forward, pick up the cube, insert the peg. They struggle with tasks requiring extended sequential reasoning: clean the kitchen, assemble the flat-pack furniture, diagnose and repair a mechanical fault. Simulation can generate data for these tasks, but the reward function design and the exploration challenge become vastly more difficult.
Perceptual grounding. Most sim-to-real locomotion policies run on proprioception alone and ignore vision. Adding vision — making the robot use its cameras to navigate around obstacles, identify objects, understand its environment — introduces a second reality gap: the gap between simulated rendering and real camera images. Photorealistic rendering has improved dramatically with tools like NVIDIA Isaac Sim, but bridging the full perceptual gap remains an open research problem.
Where This Is Going
The direction is clear enough: simulation is becoming the primary training environment for robot intelligence, with physical testing increasingly reserved for validation rather than discovery. The speed advantage is too large, the cost advantage too compelling, and the tooling improving too quickly for any serious robotics team to ignore.
What changes this picture, and what several major labs are actively pursuing, is learning from human video as a complement to simulation. Large models trained on YouTube footage of humans performing tasks — cooking, folding laundry, assembling objects — can provide a prior that makes sim-to-real transfer easier, because the robot's neural network arrives at simulation training with some structural understanding of how physical tasks are organised. This is the approach underlying systems like RT-2, π0, and several internal projects at the major humanoid companies.
The combination of high-speed simulation, domain randomization, and large-scale video pre-training is arguably the most promising path to general-purpose robot intelligence currently in active development. Whether it arrives in two years or ten depends on a dozen technical problems that are still genuinely unsolved.
But the direction of travel is unmistakable. Every major robot that impresses you in a demo video — the ones that walk, climb, recover from shoves, manipulate objects with apparent ease — has almost certainly spent the majority of its learning life inside a computer. The real world is where robots are deployed. The virtual world is increasingly where they are born.