Blame as Moat

When you type into the ChatGPT textbox seeking answers to a query, you’re taking responsibility for what you do with the information you receive. Even when you don’t realise that’s what you’re doing, that’s what you’re doing

Much ink has been spilled in the Great Hall of debate over whether we, like LLMs, are just sophisticated stochastic parrots, but I think one way we differ from LLMs is we have to take responsibility for the information we put out in the world. The shopping lists we write, the tweets we polish, the emails we send out, the blog posts, and the gossip we share…they have our stamp of authority and provenance, to whatever arbitrary degree.

This is an important point, and I’m going to try to convince you why that is the case. Only those who live in the world can change the state of the world, and only those who can change the world can be blamed for the state of the world.

To participate in the world, you must have a strong world model and be able to cross-check your output against that model. LLMs don’t have this (they have a poor man’s world model, brought about by admittedly clever hacks in the world of REST calls and RAG mechanisms, but they’re painfully inadequate, at least today).

Because LLMs cannot be trusted to emit information that is congruent with the state of the world, there must always be a human-in-the-middle to take the blame. This point might be opaque to you, but when you’re chatting with an LLM, you’re ‘checking’ the LLM (which is why it’s disastrous to use LLMs for domains where you’re dumber than it is). When OpenAi wrapper companies create yet another AI product, they count on their users to ‘check’ that the AI hasn’t veered off course, but they also pore frantically through their logs to catch aberrations. In some cases, reading logs (or at least implementing anomaly alarms) can be a full-time job.

Because someone has to be responsible. The buck has to stop with somebody. A human has to be blamed.

Why this is good news — for now

My bold argument is that as long as LLMs don’t have as strong a world model as humans, they cannot replace humans for meaningful work, however smart they appear.

The higher the stakes of the domain, the more blame is required, the more a human is required to be the fall guy.

The world’s transactions run on the invisible oil of blame: we cross the road vaguely hoping if we’re run over the man behind the wheel will be blamed, we elect government officials to blame them, and we shake the hands of our new employers, ostensibly aware our ‘job description’ is really a list of things we’ve agreed to be blamed for in exchange for money.

That is not changing any time soon.