Insight about how 'nature makes no leaps' promises better robot predictions
Machine learning researchers have developed a technique that allows world models to make better predictions about how objects will behave.
Key points
- "When an attempt fails because of something the robot could not see, such as an unexpected load, a current or a weak motor, the counterfactual world model lets it estimate which action would have worked in that same situation," he explained.
- These models could provide better direction to robots if they were able to make inferences about possible actions without being trained for all the possible outcomes.
- In other words, the counterfactual is what actually happened, plus the difference between the model's predictions for the alternative action and for the action taken.
"Our counterfactual world model does all three steps with one subtraction." Balestriero cited an example from the paper: a boat crossing a river with a hidden current and how the JEPA world model and counterfactual world model predicted the boat's movements.
Additional details
- Yet for the authors, the saying explains how they have been able to devise a method for zero-shot physics inference.
- Over a short time, the effect of the action and the effect of the current each grow in proportion to the duration, while their interaction grows only with its square, so over short steps they simply add up, like forces.
- "In our simulated tasks, choosing the action this way reaches the goal in 35 to 69 percent of events, against 14 to 23 percent for a planner that knows the true simulator but ignores the observed outcome.
- Balestriero said that the counterfactual world model experiments were done using off-the-shelf encoders (DINOv2 and MAE) without any fine-tuning and only a small predictor trained on top.
- World models are used to train physical AI systems like robots how to navigate the real world.
How this summary was made: extractive summary — built from the sources' own reporting. It is drawn only from the reporting of the sources listed below — read the originals for full detail. How we summarize →
Advertisement
Every article behind this summary — 1 article from 1 outlet
Get stories like this in your inbox.
Subscribe — it's freeOne free email at 6am — your topics, deduplicated, five minutes. No account, no paywall.