TreasureX Researchcs.LGQF3: A Faster Way to Teach Robots New Tricks
Chung Min Kim, Brent Yi, David McAllister, Hongsuk Choi, Himanshu Gaurav Singh, Jinkun Cao, Ken Goldberg, Pieter Abbeel, Carmelo Sferrazza, Angjoo Kanazawa · October 6, 2026
Training robots to move like humans usually takes enormous trial and error. A new method called QF3, from researchers at UC Berkeley and collaborators, speeds this up dramatically.
Most modern robot policies are "flow policies" — models that learn smooth motion patterns from human demonstrations. Reinforcement learning can improve them, but existing flow-based RL methods are slow because they require fresh data from the robot at every update. QF3 instead learns off-policy, reusing past experience stored in a replay buffer, and combines flow matching with a critic's action gradient. The trick: it only applies that gradient to action dimensions that stay close to the action actually tried, keeping updates trustworthy.
The result is the first off-policy flow RL method to train humanoid walking policies from scratch and deploy them on real hardware without extra tuning. It also achieves a 10x wall-clock speedup over FPO++, a recent on-policy competitor, and can fine-tune pretrained manipulation policies.
Why it matters economically: faster, cheaper robot training lowers the cost of deploying general-purpose robots in warehouses, factories, and homes. Less real-world data collection means less wear on hardware and shorter development cycles — a meaningful step toward scalable robotics.
Source: arXiv paper 2610.08789 — https://arxiv.org/abs/2610.08789
Educational content only, generated with AI assistance from public research abstracts. Not investment advice. Thank you to arXiv for use of its open access interoperability.