V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning · HackerPapers discussions

V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning — mentioned by D-Machine

Mon, 02 Mar 2026 21:14:43 GMT

> Now LLMs may not be a model for how we do it but they are certainly going to bring back structuralist and "wordcel" positions because they do seem to show, somehow, that "language is all you need" to accomplish whatever it is LLMs accomplish. People will try to bring back these obviously false models of cognition, but, so far, the dismal performance of LLMs on e.g. SpatialBench [1], and, almost certainly ARC-AGI-3, or e.g. the kind of data and effort required to get something like V-JEPA-2 [2], will be strong counter-examples to this. And, yeah, obviously animal cognition, esp. smart…

Paper and discussion history. A mention is not an endorsement.

V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning — mentioned by D-Machine

Tue, 10 Feb 2026 01:29:58 GMT

> e.g. achieving in a non-simulated environment complex tasks or simple tasks in novel environments I think one could probably argue "yes", to "simple tasks in novel environments". This stuff is super new though. Note the "Planning" and "Robot Manipulation" parts of V-JEPA 2: https://arxiv.org/pdf/2506.09985: > Planning: We demonstrate that V-JEPA 2-AC, obtained by post-training V-JEPA 2 with only 62 hours of unlabeled robot manipulation data from the popular Droid dataset, can be deployed in new environments to solve prehensile manipulation tasks using planning with given subgoals. Without…

Paper and discussion history. A mention is not an endorsement.

V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning — mentioned by D-Machine

Mon, 09 Feb 2026 22:59:49 GMT

> Modeling something as an action is not "having a world model". It literally is, this is definitional. See e.g. how these terms are used in e.g. the V-JEPA-2 paper (https://arxiv.org/pdf/2506.09985). EDIT: Maybe you are unaware of what the term means and how it is used, it does not mean "a model of all of reality", i.e. we don't have a single world model, but many world models that are used in different contexts. > A model is a consistently existing thing, but humans don't construct consistently existing models because it'd be a waste of time. You don't need to know what's in your trash in…

Paper and discussion history. A mention is not an endorsement.

V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning — mentioned by D-Machine

Mon, 09 Feb 2026 18:07:15 GMT

AI ≠ LLM. It would be more precise to say that complex world modeling is not done with LLMs, or that LLMs only supplement those world models. Robotics models are AI, calling them LLMs is incorrect (though they may use them internally in places). The middle "L" in LLM refers to natural language. Calling everything language and words is not some gotcha, and sensor data is nothing like natural language. There are multiple streams / channels, where language is single-stream; sensor data is continuous and must not be tokenized; there are not long-term dependencies within and across streams in the…

Paper and discussion history. A mention is not an endorsement.

V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning — mentioned by YeGoblynQueenne

Mon, 30 Jun 2025 17:18:12 GMT

It's not a scholarly article but a blog post but you're still right to be frustrated at the very bad writing. I do get the jargon, despite myself, so I can translate: the authors of the blog post claim that machine learning for autonomous robotics is "solved" thanks to an instance of V-JEPA 2 trained on all videos on youtube. It isn't, of course, and the authors themselves point out the severe limitations of the otherwise promising approach (championed by Yan LeCun) when they say, in a notably more subdued manner: >> the model is basically a diva about camera positioning. move the camera 10…

Paper and discussion history. A mention is not an endorsement.