Real-Time EXPO-FT: Reinforcement Learning for Real-Time Vision-Language-Action Policies

Real robots do not pause for inference. The scale that makes a pretrained vision-language-action (VLA) model a strong behavior prior also inflates its inference latency — and the physical world keeps moving while the robot computes. The observation an action was inferred from is rarely the observation at the moment it executes, a distribution shift that affects reliability exactly where it matters most. Methods like real-time chunking keep control smooth under this latency, but give the policy no mechanism to move beyond its training distribution. We propose Real-Time EXPO-FT to close that gap: a reinforcement learning framework for fine-tuning real-time VLA policies, so the reliability gains of RL and the reactivity of real-time execution come together.

• Slow action candidate generation. Building on EXPO-FT, we decouple action generation into two stages. First, a large VLA base policy proposes action chunks, which is slow because of the scale of the policy.
• Fast and reactive action edits. By the time the base actions are generated, the environment has already transitioned to a new state. The edit policy transforms the actions using the observation available at the time of execution — so the policy stays maximally reactive.
• Q-value chunk selection. The Q-function is used to select the best action candidate on-the-fly, after the VLA proposes candidate chunks and the edit policy edits the actions.

BibTeX

@misc{dong2026reinforcement,
      title={Reinforcement Learning for Real-Time Vision-Language-Action Policies}, 
      author={Perry Dong and Kuo-Han Hung and Dorsa Sadigh and Chelsea Finn},
      year={2026},
      eprint={2609.18207},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2609.18207}, 
},
}