WTF is a World Model? [D]

[u/neutrino_boy]

Edit: Just to clear up some confusion, I know “what a world model is”; I suppose I’m more interested in the specific questions I asked. I kind of just wrote this on a whim, had some typos, and my asking “WTF is a world model” was a bit tongue-in-cheek. I wasn’t so much looking for simple descriptions, but hoping for a more nuanced discussion about the differences between so-called “world models”, traditional simulators, digital twins, etc and what really counts.

I'm trying to understand what a world model is. I understand it has its roots in cognitive science and reinforcement learning. I understand, at least at the moment, what most people are building, which they call world models, are fancy video generation models. But what actually counts? Does a simulator count as a world model? Some "world models" are described as simulators, or rather, a simulator is described as one type of world model. But is a simulator, like, let's say a physics engine, a world model? There are some video game world models, or computer use world models. Would a hardware/video game emulator count as a world model? And can a digital twin also be a world model with some additional features?

I've seen a definition that says a world model should "operate on learned representations, not exclusively hand-crafted physics, i.e. a physical referent is optional." Which is fair enough, but then would a physics accelerator that uses an ML count as a world model? Like some ML fluid simulator is that a fluid world model?

Are world models just a rebrand of simulation, or is there really a fundamental difference? Should the definition be limited to models that aim to generally model all of the real world? So that would exclude video game world models and also models of specific interactions.

[u/Salt-Bodybuilder-518]

There are several types of world models. Fei fei li developed a taxonomy tag might be helpful here:
https://drfeifei.substack.com/p/a-functional-taxonomy-of-world-models

[u/Relevant-Car9958 → u/Salt-Bodybuilder-518]

The taxonomy's a decent starting point but honestly the whole term's become so overloaded it barely means anything anymore. Every lab slaps "world model" on their project and half the time it's just a diffusion model that generates shaky video clips.

I think the key difference from a plain simulator is the learned representations part, a physics engine with hard-coded equations isn't really building an internal understanding the way something trained on raw sensory data does. Though the line gets blurry when you stick an ML module inside a traditional simulator stack.

At this point I just assume when someone says world model they mean a generative thing that spits out predictions from latent space, not something you'd actually want controlling a robot arm.

[u/neutrino_boy → u/Relevant-Car9958]

So is it really the fact that they’re top-down learned representations? Is there actually no explicit definition of anything required for the simulation?

In a way, formulas are learned relationships. Newton’s equations, for example, are approximations inferred from empirical evidence; they’re not exact descriptions of reality.

What I find interesting is that you describe hard-coded equations as evidence that a system doesn’t really “understand.” But if you go back 300 years, someone who knew F=ma would probably have been considered to understand motion better than someone who simply had an implicit, subconscious intuition for how objects fall. The explicit formulation would have been seen as the deeper understanding.

Yet now, when we machines can operate using explicit equations but lack the kind of implicit, intuitive representations that humans develop through experience, say that they’re missing “understanding.”

What I find particularly interesting is how this shows up with LLMs doing math. An LLM can learn statistical, somewhat roundabout ways of arriving at the answer to a simple arithmetic problem, rather than simply following the explicit rules of arithmetic. That’s why it can sometimes get even very simple calculations wrong. In that sense, it has learned a representation of the relationship rather than actually executing the underlying procedure. We do not view this as understanding mathematics!

I don't have any opinions on this. Just an observation.

[u/DeMatzen → u/neutrino_boy]

I would go even further and say, that a physics-based model IS the (problem-) internal understanding. It uses as little variables as possible and all if them carry an interpretable meaning. F=ma is, if you went into a model-size-exactness-diagram it is the base-line any model has to match. Any ML-model can at best get this value. Thinking an ML-model has an intuition is right in the sense, that it gets the overall trend, but there is bo understanding behind it. Of course physics-based models have their boundaries (I hate dealing with friction) as well, and some things like language don't have a physics-based interpretation.

[u/DarePale → u/neutrino_boy]

Though F=ma is better than intuition it came from the human mind as well. Just that Newton used reasoning and experimentation instead of intuition to explain his observations. He arrived at those results by himself rather than someone handing it to him and him blindly trusting that equation.

[u/neutrino_boy → u/DarePale]

But I don't think it wouldn't be unreasonable to assume a robot could learn F=ma simply by dropping lots of objects in todays world and moving them around. Science in general has shifted more towards data driven discovery. Discoveries before relied a lot on intuition but I don't think they need to. I take your point though LLMs are essentially just handed these definitions.

[u/DarePale → u/neutrino_boy]

Relativity was theorised by Einstein and wasn't experimentally confirmed until much much later. You can't claim that we would never have to do this again in today's world. Creating a world model that is rooted in a hypothesis and then verifying if the predictions of a world model goes in hand with the real world observations is very much inevitable once we upgrade the standard model of physics (GR and Quantum Mechanics with a newer theory that will explain the observations that defy our current physics models such as dark energy and so on).

[u/neutrino_boy → u/DarePale]

But that is why general relativity is an exceptional theory. Most modern physics isn't thinking for 40 years.

I never claimed that we will never again have to do this but progress of physics itself along with technology has resulted in a paradigm shift.

There is only so much you can discover by just theorising and that becomes more difficult when the things are even harder to observe and have intuitions about.

I doubt we will see a single human come up with a theory as extraordinary as GR in physics again. Its just like Michael Jackson, I doubt we see a musician as influential as him again. These type of things can only be done once. Unless society rewinds and we loose knowledge and or technology the way things are done has changed.

Even know theoretical work in mathematics is beginning to be swallowed by LLMs and that has started to seep into theoretical physics. That could mark another paradigm shift.

[u/robobub → u/neutrino_boy]

It's going to learn a lot more of a complicated equation that overfits it's observations.

Sure, it takes into account things like air resistance, drag, etc but it also will conflate things like measurement error, bias in the drag coefficient of objects, etc., without the ability to disentangle them.

Maybe with more data (tm) it will, of course

[u/neutrino_boy → u/robobub]

Surely this just depends on the data. If you train it on reasonably heavy rigid objects then drag will not be a major factor. Also drag/air resistance is still f=ma just on different scales.

Maybe you want the model to be able to learn everything without you telling it what to do but I think realistic ML always has some level of engineering with the data and architecture with some human driven goals.

[u/robobub → u/neutrino_boy]

Surely this just depends on the data.

Well yes, that's exactly what I said.

It's easier to get balanced unbiased data in something that we understand a lot about and is small and controlled. Extend it to larger parts of the world and you have an explosion in confounding factors.

realistic ML always has some level of engineering with the data and architecture with some human driven goals.

Yes, we've known that for a long time and it is still far from building a good understanding of the world or any larger piece of it

[u/neutrino_boy → u/robobub]

Okay apologies if I misunderstood.

[u/neutrino_boy → u/Salt-Bodybuilder-518]

I have read her taxonomy blog post, but it doesn't fully answer my questions, and I have seen many differing opinions. Many will disagree that the "renderers" are not world models. I actually alluded to this same blog post when I mentioned "simulators". I think she is heavily biased because she is working on a company that seems to have mostly produced "renderers" ( World Labs' own RTFM is mentioned in the post).

She literally just describes renderers as a video generation model? "A video model that turns a text prompt into a cinematic drone shot is a renderer". Why do they count as world models and not LLMs or Image generation models?

Some definitions say they should learn state transition functions, for example. Renderers would not fit that definition. Some definitions also say a world model should "support prediction and planning through imagined future states," which a video generation model alone cannot do, so it would also not fit that definition.

Lastly, the "planner" she described seems more like a thing that could use a model than an actual world model itself? And you could say a similar thing about a renderer: a world model could be used in tandem with a renderer or function as a renderer too, but I would think a renderer itself isn't a world model.

[u/QuinnFazigu → u/neutrino_boy]

Can the text prompt be the description of the "set" that you can then animate? For example, a prose blueprint of a house for a sitcom, maybe also the characters and pets and such. I can see that being an useful and intuitive "world model".

[u/rand3289 → u/Salt-Bodybuilder-518]

Thank you for the link!

POMDP leads to turn-based models where "a state change" and agent actions take turns... and there lays the problem! Turn-based models are not good enough for the real world!
I don't get it how noone sees that.

I just made a post about it a couple of days ago:
https://www.reddit.com/r/agi/s/crpJ5jTbDO

[u/Swolnerman → u/Salt-Bodybuilder-518]

Great read!

[u/OctopusGrime]

Not sure what all these comments are about, It comes directly from RL.

A world model is a learned transition model P(s_t+1 | s_t, a_t) and comes from neuroscience e.g. babies spend the first year of their life learning to predict the motion of objects, expected sensations.

Having a world model enables model based RL i.e. planning via search.

[u/LordVoldemort-_- → u/OctopusGrime]

How is this different from RL at all? This seems very similar to a MDP, why the rebranding and hype?

[u/brainExploded99 → u/LordVoldemort-_-]

It has higher sample efficiency and it's more practical for partially observable MDP.

[u/Disastrous_Room_927 → u/brainExploded99]

Any useful links to read about sample efficiency here

[u/dsmirza → u/LordVoldemort-_-]

This "world" is not different from RL. Many RL approaches do not build these world models and are hence called "model-free". Others are model based and they explicitly learn these transition probabilities.

Consider REINFORCE/PPO/SAC. They only try to learn the policy not the transition dynamics. The transition dynamics (P(s{t+1} | s{t}, a_{t}) ) are part of the MDP, yes, but not learnt.

I would not say they are 'ignored' as the policy is learnt from rollouts in the world so the world information becomes implicitly part of the policy. (e.g. don't do this action because this on average leads to low return). But you can say 'ignored' if you understand what you mean by 'ignored'.

[u/OctopusGrime → u/LordVoldemort-_-]

I’m saying it’s literally taken from RL. Only with recent investment they can justify collecting enough data and training big enough models

[u/Lumpy_Ad2192]

I would argue that the original definition of a world model is still the most accurate, but not one that I think any lab has actually achieved in a meaningful way.

Originally early AI, scientist and engineers, realize that the limitation of language models is it even with reasoning they don’t really understand “the world“. That means they don’t really understand how things interact or how things relate in a physical sense.

To be fair humans also struggle with some aspects of this. We have a kinesthetic sense which has evolved with a specific understanding of how objects work in a gravity field, but unlike birds and insects, humans have no real innate understanding of certain quantum principles.

The gap therefore is that we both want models which can contain an understanding of the world that matches ours, and therefore is much more useful to fields where simulation at various levels is critical, but we also want to lead the crown work to build models with superhuman understanding of the universe, such as a native understanding of how quantum and physical realities, interact, and a framework where they could theoretically build into understandings of human network information and other kinds of relationships that require more processing power than humans have.

You could make an argument that models which have trained in various simulators are a good first step, which I think is what most of the world model labs are doing right now. Certainly the labs that are working with robotics that have feedback loops with reality are seeing better results, but most of the successful ones are starting with neural networks and abandoning Transformers entirely.

All of this is to say that I think that “world models“ are almost a philosophical stake in the ground. We will know that an AI has a good world model if it shows an understanding of physics that is equal or greater to humans without additional fine-tuning, and ideally having self trained in some kind of simulated physical environment.

But from a scientific standpoint, that doesn’t give us a meaningful hypothesis that we can test against. Also, it’s not clear that any of our current approaches have the capacity to scale up there, even if they’re doing pretty good with some of the simulations.

So generally, I agree with you that a lot of what’s being talked about is largely marketing to try to get dollars for various approaches which, while some show meaningful progress, none are clearly the correct one that with additional funding and processing power will actually achieve something that meets that very high bar.

[u/BelialSirchade → u/Lumpy_Ad2192]

I mean, doesn't it mean that physics engine is a world model? I don't know, I have tried to do some search into it but, it never made sense to me why it would be "smarter" than LLM in non-physical domains, it still is necessary for robotics but, why would a world model reason better in high level fields like math or physics for example?

of course I think it's the next big thing, as robotics is the next big thing, but for reasoning and abstract thinking, I think an LLM approach will still work better I guess, I don't know what Yann is seeing that I'm not.

[u/nonotan → u/BelialSirchade]

why would a world model reason better in high level fields like math or physics for example?

The idea is that current LLMs are pretty much an extreme manifestation of a "Chinese room", with nothing but superficial statistical inferences between high-level concepts that are not grounded on actually understanding what these concepts are fundamentally describing.

With a notion of what the world under such high-level concepts actually looks like, you've got a whole additional layer to check ideas against, and with which to fine-tune your weights without depending on astounding amounts of human-made data (with us acting as, effectively, the intermediary that does understand the world and crystalizes it into those high-level concepts) that we've pretty much already maxed out today.

It's like saying, "why would modeling temporal coherence improve the quality of video generation beyond reducing some minor artifacts?" -- well, real video doesn't generally have non-physical discontinuities, so to the extent that it does anything, it points at an underlying inaccuracy in the generator.

Of course, whether the things people are introducing under the label of "world model" actually achieve any of these aims in practice, whether the whole concept will work out in practice and outperform existing approaches (plenty of logical-sounding ideas end up not really working out for one reason or another), etc. are separate questions. I'm just explaining why it makes some sense a priori that they would help.

[u/QuantumPhantun → u/Lumpy_Ad2192]

Which quantun principles do insects and birds have an innate understanding of?

[u/AccomplishedLeg1508]

No universal definition exists

[u/evanthebouncy]

I think a pithy description maybe

"A simulator of the environment where you can ask what-if questions"

[u/neutrino_boy → u/evanthebouncy]

Learned representations important to you or not? Implicit / explicit simulator?

[u/evanthebouncy → u/neutrino_boy]

in my lab we work with programmatic world models, so symbolic + explicit simulation. but that's a preference, learned representaiton is also great, just not what we do aha

[u/neutrino_boy → u/evanthebouncy]

Okay interesting. Would I be able to look at some of your work or do you not want to doxx yourself?

[u/evanthebouncy → u/neutrino_boy]

oh i can doxx myself as I work in the public domain. the problem is we don't really have any concrete output yet as we've only recently have gotten into it.

I'll just refer you to this work: https://arxiv.org/abs/2402.12275

[u/Every-Cat-2611]

The way I understand it is a persistent state. If you turn the lights off in your home, you can likely still navigate without bumping into every wall. You’re not “remembering” where everything is, you have a model of your house in your mind. If you stand in a place you’re familiar with, you “know”, not “remember” what’s behind you.

That said it’s now just become a term that companies throw around to get you to buy their product. And the actual definition has gotten a little wishy washy, I doubt anyone actually considers what I said to be the “official” definition.

[u/shifty_lifty_doodah]

Predicts what will happen given observed state and possible actions.

[u/Disastrous_Room_927 → u/shifty_lifty_doodah]

So it’s an MDP?

[u/alexbaas3 → u/Disastrous_Room_927]

Most ‘worlds’ aka environments can be formalised into MDP or POMDPs, a gridworld MDP is of course way easier to understand as environment, but let’s say you try to model the real world as the environment, good luck with formalising that, even as POMDP.

I would say the world model definition according to the comment doesn’t include the reward transition function R at all given the MDP tuple <S,A,T,R>. It just states it will be able to predict future state s’ given the current state s and some possible action a -> in this case the world model thus represents the transition function T in a MDP. Also, world models often represent states S as some latent state distribution z which you can sample from, it’s generative and stochastic, whereas MDPs are deterministic in that case, pretty sure you could model the latent z in as a POMDP but anyways.

There are many examples of world model papers that do include some reward function R, not sure if they formalise it into a PODMP though. AND in robotics learning I think the comment’s definition is closer to the truth. In generative modelling/RL it might be closer to a representation model formalised into a POMDP, often mimicking some complex environment (like the real world, minecraft, etc.)

There are many definitions flying around now, and everyone just inserts world model into their papers because marketing, it’s the new buzz word for ML researchers.

[u/shifty_lifty_doodah → u/Disastrous_Room_927]

Well, maybe.

To me, a model is a predictor.

And a world model predicts what happens in a world.

We can get into different mathematical details in the definition, but this is really the core of what it is, by definition.

[u/relevantmeemayhere → u/shifty_lifty_doodah]

this includes basically every state space model....ever

i guess cs rediscovered financial statistics in the 90s

[u/Antique_Most7958 → u/shifty_lifty_doodah]

A simulator that is learned from data, aka an ML surrogate

[u/ProfMasterBait]

ML researchers love fluff and grand names

[u/Howling_deer]

Investing hogwash. No one knows lol

[u/neutrino_boy → u/Howling_deer]

Loool. Mood.

[u/NeighborhoodFatCat → u/Howling_deer]

This. Similar question can be said about "digital twin", "precision medicine", "smart sensing", and "federated learning"

[u/costafilh0]

A model of the world. 

[u/jdude_]

A world models is an abstract state that represent the world, given actions and time it evolves, usually by using a neural network. The theory behind the main use cases is that when we (humans) make long term plans we use our own version of a world model. It should allow abstractions that will be very useful in RL, abstract resentations and predictive models.

I don't think it needs to operate nesscerly on abstract representations, it's just nesscary for our use cases. Physics simulations and game engines sort of answer the definition, maybe the only issue is that they are very constrained abstracted proxy of a world, and not as useful as what neural networks allow.

[u/atdlss]

People have very different definitions, but in a broader sense: any model (3D, video, etc.) that takes the current and/or previous states + an action as input and predicts the next state.

[u/gannu1991]

Depends what you need the model to do, that's the cleanest way to cut through the definitional mess. A video generator that predicts the next frame is not a world model in the useful sense unless it's action conditioned, meaning you can feed it a hypothetical action and it predicts the resulting state. That's the actual dividing line people care about: can you query it with "what if I do X" and get a plausible next state back. A physics engine does this by construction with hand coded rules. A learned world model does it from data. Both count under that test. A pure video generator that just continues a clip without taking an action as input doesn't, no matter how good the pixels look. Digital twins count if they're predictive and queryable, not just a visualization layer.

[u/Adventurous_Glass494]

I would recommend "The Book of Why" by Judea Pearl.

[u/HelpfulKaleidoscope]

It comes from a RL, let's say you have an agent in a system with a task. A world model is something that says : if I do this action and given tue state of the system, how the system will end up? You predict the world.

[u/neutrino_boy → u/HelpfulKaleidoscope]

Okay I realised here confusion is coming from. In my post I meant to say it has its roots in Cog Science and RL. I already know this but the typo I guess has confused people. I was wondering why people kept saying this but I did not proof read the post.

[u/Hyperhelium]

I wished engineers stopped naming things after terms they don't really understand. That only creates more confusion.

[u/progenitor414]

I would use a functional definition: a world model is an internal predictive model an agent can roll forward under candidate actions. A physics engine can serve as one if it sits inside the planning loop, while a video generator may not if it cannot preserve state or answer action-conditioned counterfactuals. Learned versus hand-coded describes construction, not function.

[u/catsRfriends]

It's not cognitive science or reinforcement learning or w/e. Those are implementation details. Reinforcement learning is the way for the model to learn something based on the reward. In this framework the model makes guesses and then you assess the state of the world. If the state is scored highly then the model learns what it did was good so it tends to perform more in that fashion. Otherwise, less.

The idea is that there is a world with latent factors to be learned. It just so happens that the ones that are interesting are also hard to model correctly/well with point data samples. If nothing else, the combinatorial explosion means the volume of such data required makes the problem into a computationally intractable one.

[u/neutrino_boy → u/catsRfriends]

I understand this. What I was saying was that I understand before generative AI the definition / idea of a world model had its roots in RL / Cognitive Science.

[u/catsRfriends → u/neutrino_boy]

It doesn't appear so, otherwise what is it you're asking? One of the premises of machine learning is that you let the algorithm figure it out when the problem is messy and hard to describe neatly but data and labels are plentiful. So in the sense of machine learning no, the simulator isn't a world model if humans had to code it in detail. Whereas a model that infers 2D platformer physics like in the Google Genie paper is closer to a world model.

[u/neutrino_boy → u/catsRfriends]

There are lots of sentences in my post that end in question marks. Those are the things that I am asking. The second sentence did not have question mark concluding it. Not sure what's confusing about that.

[u/yannbouteiller → u/neutrino_boy]

It is because model-based RL has been learning world models forever. In a general sense these are simulators, either learnt or not, either explicit or latent, that predict a next state from a current state and an action.

[u/relevantmeemayhere → u/yannbouteiller]

the problem is that, if you choose this definition, then there are a bunch of old penalized likelihood models that are now world models.

RL is penalized regression. We use words like 'learning' and 'encoding'; but this terminology makes it sound more sophisticated than it is in many, many cases. This is why people dunk on ml and ai researchers a lot of the time

[u/samas69420]

a world model predicts how the environment's state will change when something happens, in some cases you can also use a simulator to do that but the difference is that a simulator is built while a world model is learned directly from data

[u/relevantmeemayhere → u/samas69420]

this definition then includes almost every statistical learning model, ever.

[u/samas69420 → u/relevantmeemayhere]

indeed they are statistical learning models, whatever you use to learn and predict how the environment will respond for example to a action and what state it will transition to can be considered a world model to my understanding

[u/neutrino_boy → u/samas69420]

If a model tries to build a simulator. So it reconstructs objects and puts them in their own 3d world and learned rules overtime for how these objects respond to forces basically building its own simulator, could be with explicitly defined forces or neural nets or a hybrid, would that count as a world model to you? For example maybe its graph neural networks on the meshes.

I guess what I am trying to get at. Does it matter if the geometry is explicitly defined or is it just the laws of motion and interactions.

[u/Random-Number-1144]

Exactly. WTF is a world model.

It's absolutely rebranding of old ideas.

World model is basically computationalists' (wrong) ideas of how the brain controls and functions. It chnages over time. But the implicit assumption has always been cartesian dualism.

Check out my blog on why it won't achieve anything groundbreaking.

[u/Flyward_Aerospace]

The taxonomy arguments never land for me because they sort by what the model is built out of instead of by what actually breaks. The test I would use is whether anything closes the loop on it. If a human is watching the output, per frame realism is basically the entire spec. If the model's own prediction becomes the input for the next step and something acts on it, per frame realism stops mattering almost completely and the only thing you care about is what the error does over a few hundred steps. Video models that look incredible for five seconds fall apart on exactly that, which is why sim people care so much about whether the thing conserves anything, since a conserved quantity is what puts a bound on drift. Open loop vs closed loop splits the field a lot more cleanly than arguing about whether a renderer counts imo.

[u/sweetjale]

Dreamer, Dreamer 2, and Dreamerv3 are the simplest versions of a world model

[u/brainExploded99 → u/sweetjale]

Why not v4? v4 cuts down on moving parts from v3. Trying reproduce it currently, but the paper lacks details and no code.

I'm also convinced some parts of the paper were either implemented somewhat differently from what was said or are straight up incorrect.

[u/sweetjale → u/brainExploded99]

i see. haven't read v4 yet.

[u/brainExploded99 → u/sweetjale]

You should check it out, maybe I missed something.

Example of paper seeming to be incorrect: It seems like RMSNorm of the dynamics losses was just a bad idea, multiple independent reproductions attempts turned it off because converging with it enabled seems to be difficult.