Sim-to-real: what actually transfers

Locomotion transfers well because it is dominated by rigid-body dynamics a simulator models correctly. Manipulation transfers badly because it is dominated by contact and friction, which no simulator models correctly.

There is a reason quadruped locomotion policies are routinely trained entirely in simulation and deployed to hardware with no real-world fine-tuning, while manipulation policies mostly are not. It is not that locomotion is easier. It is that locomotion lives in the part of physics that simulators get right.

The sim-to-real gap is not one gap. It is a gap in dynamics, a gap in perception, and a gap in the task distribution — and they need entirely different treatments.

Three gaps, three treatments#

The dynamics gap. Your simulator's contact model is an approximation, its friction coefficients are guesses, and its actuator model probably assumes a torque source where you have a position-controlled servo with a gearbox, backlash and an internal PID loop you do not control. For free-flight and rigid-body motion, the error is small. For anything involving sustained contact, sliding, or deformation, it is large and structured.

The perception gap. Rendered images are not camera images. The differences that matter are rarely the ones that look obvious: sensor noise, rolling shutter, motion blur, auto-exposure behaviour, and lens distortion do more damage than imperfect materials. A policy that consumes depth is exposed to a different and generally worse gap, because simulated depth is clean and real depth has holes, flying pixels, and specular dropouts.

The task gap. Your simulated object set is thirty meshes; the real world has an unbounded number of objects with mass distributions you did not model. This gap is not closed by better physics — only by broader assets.

Domain randomization, and its limits#

The standard tool: randomize the parameters you are uncertain about so the policy must be robust to all of them. It works, and it has a specific failure mode worth naming.

RANDOMIZE = {
    # dynamics — the parameters you genuinely cannot measure well
    "friction":        ("loguniform", 0.4, 1.6),
    "restitution":     ("uniform",    0.0, 0.3),
    "link_mass_scale": ("uniform",    0.85, 1.15),
    "motor_kp_scale":  ("loguniform", 0.7, 1.4),
    "joint_damping":   ("loguniform", 0.5, 2.0),
    "action_delay_ms": ("uniform",    0.0, 40.0),   # the one people forget

    # perception
    "light_dir":       ("sphere",),
    "light_intensity": ("uniform",    0.3, 2.0),
    "camera_pos_mm":   ("normal",     0.0, 8.0),    # extrinsics are never exact
    "texture":         ("categorical", "…3k textures"),
    "jpeg_quality":    ("uniform",    55, 95),
}

Two entries in that list do disproportionate work.

Action delay is the most under-randomized parameter in the field. Real systems have latency — network, driver, control loop, actuator response — and a policy trained at zero delay learns a control law that is unstable at 30 ms of it. Adding a randomized delay buffer is a two-line change that fixes a class of oscillation bugs people otherwise chase in the controller.

Camera extrinsics. Your measured camera pose is wrong by several millimetres and a fraction of a degree, and it changes when someone bumps the rig. Randomizing it forces the policy to use relative visual relationships rather than a memorized mapping from pixels to workspace coordinates.

System identification is the higher-yield move#

Domain randomization asks the policy to be robust to a wide range of possible robots. System identification narrows the range by measuring the actual one. Doing the second first makes the first much cheaper.

The practical version is not elaborate:

1. Command a set of trajectories on the real robot — steps, chirps, and a few task-like motions — and record commanded versus achieved joint states.
2. Replay the identical commands in simulation.
3. Optimize the simulator's actuator and friction parameters (CMA-ES is fine; the parameter space is small and the objective is noisy) to minimize the trajectory mismatch.
4. Randomize around the fitted values, with a width set by the fit's residual rather than by intuition.

This routinely turns a ±100% randomization range into a ±15% one, and the policy trained in the narrow range is meaningfully better because it does not have to hedge against robots that do not exist.

The same logic applies to the camera: calibrate intrinsics and extrinsics properly, then randomize by the calibration's uncertainty. A 2 mm randomization around a measured pose is worth far more than a 50 mm randomization around a guess.

Residual policies: let the simulator be wrong#

A different framing that we like for manipulation. Rather than demanding a simulated policy that works unmodified on hardware, train the bulk of the behaviour in simulation and learn a small correction on the real robot.

# base_policy: trained in sim, frozen. residual: small, trained on real data.
def act(obs):
    a_base = base_policy(obs)                       # the plan
    a_res  = residual(obs, a_base)                  # the correction
    return a_base + ALPHA * torch.tanh(a_res)       # bounded, so it can't run away

The tanh bound is load-bearing: it guarantees the residual can adjust but never override, which keeps the safety properties of the base policy intact and makes the real-world training run far less alarming. In our experience the residual converges on tens of episodes rather than thousands, because it is only learning the difference between two dynamics models, which is a much smaller function than the task.

What to simulate and what not to bother with#

| Phenomenon | Simulator fidelity | Recommendation | | --- | --- | --- | | Rigid-body kinematics and inertia | Excellent | Trust it | | Free-space collision | Very good | Trust it | | Actuator dynamics | Poor by default | Identify it, then trust it | | Coulomb friction, stiction | Approximate | Randomize wide | | Deformable objects (cloth, cable) | Poor and slow | Collect real data | | Granular media | Poor | Collect real data | | Suction and adhesion | Usually absent | Model as a constraint, verify on hardware | | Camera image formation | Good with effort | Randomize aggressively | | Depth sensor artefacts | Rarely modelled | Inject holes and noise explicitly |

The honest summary: if the task is dominated by geometry and rigid-body motion, simulation is close to free performance. If it is dominated by contact mechanics or deformation, simulation is a way to pretrain a prior, not a way to avoid collecting real data.

Evaluating transfer without fooling yourself#

Simulation success rate is a poor predictor of real success rate, and it is poor in a direction that flatters you. Some practices that keep the estimate honest:

• Hold out simulation conditions. Train on one set of randomization seeds, object meshes, and lighting; evaluate on another. A policy that cannot transfer from sim to sim will not transfer to reality.
• Fix the real-world protocol before you tune. Same initial-state distribution, same object set, same number of rollouts, written down. Twenty rollouts on hardware puts roughly a ±20-point confidence interval around a 50% success rate; if you are comparing two policies at 60% and 70%, you have measured nothing.
• Report the transfer ratio, not just both numbers. Real success divided by sim success, tracked over time, tells you whether your simulator is getting more predictive. That is the number that indicates whether the sim investment is paying off.

The pragmatic stack#

For most manipulation teams we would advocate: identify the system, simulate the free-space and approach phases, collect real demonstrations for the contact phases, pretrain on the mixture, and learn a bounded residual on hardware. Simulation is used for what it is good at — volume, resets, and safe exploration of the parts of the task where physics is well modelled — and is not asked to do the part where it is wrong.

This is less satisfying than "trained entirely in simulation, deployed zero-shot", and it ships considerably more often.