EXPO Hyperparameter Tuning Guide
This guide applies to the .
Hyperparameter tuning
Edit action scale
The most important hyperparameter of EXPO is the edit action scale, which controls how much the edit policy can deviate from the base policy distribution to maximize value. Tasks where improvement is more like small refinement will benefit from a small edit scale. Tasks where improvement requires more exploration, for example if the prior isn’t as good, will require larger action scales.
Good starting values are:
| Regime | When to use | Values to try |
|---|---|---|
| Small action scale | Improvement is small refinement | 0.01, 0.05, 0.1, 0.15 |
| Large action scale | Improvement requires more exploration | 0.4, 0.5, 0.6, 0.7 |
Number of action samples (N)
Performance is generally insensitive to the number of action samples N. A default of N = 8 works well across tasks, but increasing the dimension of the actions, i.e. with action chunking, can require a larger number of action samples such as 16, 32, or 64. Real-Time EXPO-FT benefits from a larger number of action samples to account for reactivity.
Prior quality
EXPO benefits from having a strong prior. The prior can take many forms — a pretrained policy, offline data, human interventions, or a combination of these — as long as it contains information on behaviors.
There’s no definitive metric for how good a prior is, especially since it may take different forms (e.g., offline data vs. a pretrained policy), but it can be proxied by asking:
If I combine all of this information into a policy, i.e. by training a policy on the offline data, how good is the resulting success rate?
Starting with a prior of ~50% success or above is generally helpful for learning.
Troubleshooting: the robot is jittery
There should be no noticeable jitter during RL training or policy execution. If you’re encountering jitter, check these possible causes:
Controller settings. Controller parameter modifications can introduce jitter. The default DROID controller is used for the EXPO-FT experiments, which does not apply action smoothing or interpolation but does enforce velocity limits. It’s perfectly fine to use a different controller, provided it can execute without jitter.
Inference latency. Model inference time exceeding the control timestep will prevent the system from sustaining VLA inference. The default in EXPO-FT is 10 Hz with 2 H200s. Inference can be made faster with more compute. Real-Time EXPO-FT also handles this.
Incorrect action normalization or scaling. Normalization not done correctly, or mismatched normalization stats between training and inference, can cause jitter.
