TreasureX Researchcs.AINew AI Model Turns Video Into Replayable 3D Motion for Any Skeleton
Noam Rotstein, Yoad Tewel, Chen Tessler, Haggai Maron, Gal Chechik, Or Litany, Yoni Kasten, Yuval Atzmon · October 20, 2026
Video-generation models can now produce strikingly lifelike movement, but that motion stays trapped in pixels. A new model from NVIDIA researchers, described in a paper on arXiv, tackles this gap by generating video and 3D motion together for any supplied skeleton, or "rig" — not just human bodies. The trick is architectural: instead of forcing a giant video model to also learn 3D movement, the team keeps a separate motion model that supports arbitrary skeletons and links it to a pretrained video generator through a small set of cross-attention layers. This lets the motion model learn from motion-only data, which is cheaper to collect and augment than paired video. The system also learns where each joint appears in the frames, allowing it to recover the camera position and place the motion back into the 3D scene. To train across many skeletons, the team used a generative video editor to insert animated 3D assets into fitting scenes, yielding nearly 200,000 training pairs alongside human and robot footage. The model animates diverse rigs from text more faithfully than video-first pipelines, responds to the surrounding scene, and supports keyframe control. Practical uses include animation, game and film production, robotics simulation, and synthetic training data. If such tools mature, they could lower content-production costs and speed up workflows at studios and robotics firms — though the paper is research-stage, with no product or revenue attached.
Source: arXiv, https://arxiv.org/abs/2610.28632
Educational content only, generated with AI assistance from public research abstracts. Not investment advice. Thank you to arXiv for use of its open access interoperability.