AI doesn't understand a thing

Why the mind-body problem is more actual than ever.

Those who follow these letters know how I support a post-material view of the mind-body relationship. Not a rather obsolete Cartesian dualistic metaphysical view, but a bit more sophisticated non-reductionist and non-physicalist understanding of consciousness, mind, life, and spirit that is not just a philosophical exercise but is determining, and is going to determine even more in the future, critical decisions concerning AI, policy-making, healthcare, mental health, and the investments in science and technology. Because our choices are not determined only by the facts on the ground but are also largely influenced by our worldview and, especially, how we see and perceive ourselves. If you accept the idea that the Earth may not be at the center of the universe, several of your choices will be guided by a certain mindset, while, if you are still stuck in a geocentric paradigm you might not be able to see beyond a very limited horizon and correspondingly limit yourself and your development.

Credit: webartgallery on DeviantArt

That’s the kind of phase where humanity is finding itself with the AI-related modern hype. For example, if we truly believe that we are ultimately not much more than a 1.4 kg wet and slimy brain tissue in our skull we will tend to have an approach to psychiatry and mental health that may well remain stuck forever in a brain-centric view of ourselves, and that, despite all the claims and noble intents to do otherwise, will not allow us to go beyond strictly pharmaceutical treatment. If we consider the so-called ‘hard problem of consciousness’ an uninteresting philosophical musing or, at best, nothing more than a weekend metaphysical rumination, we might well build and rebuild, generation after generation, a society that remains utterly unable to transition from a crudely materialistic industrial society based on economic exploitation and injustice, to a more harmonious soul-centered conception of life, mind, and Nature. Looking at consciousness, mind, and life from a materialistic perspective has led us already into an unmanageable complexity conundrum that is making the progress of medical science increasingly difficult. A certain kind of gene-centric and neuro-centric view of ourselves has led us to a neuroscience delusion that will have detrimental effects on how we are supposed to treat mental illnesses for the coming generations. You might ask what science has to do with the mystical experience and consider my metaphysical ruminations, such as those on whether reality is an illusion, or if we have free will, or asking for the search for the neural correlates of memory as a waste of time or, at best, is worth only a bet on the nature of consciousness. But precisely the answers we will give to these questions will determine our future. I could also argue how our unaware mind-consciousness conflation, or thinking about the philosophical subtleties of the mind-body interaction problem and the connections between determinism and free will, are, indeed, complicated philosophical musings that, nevertheless determine our worldview. It is, and will be more than ever, the frame of reference we choose to see the world, and especially ourselves—that is, whether we opt for a materialistic or post-materialistic worldview—that is going to decide which turn humanity will take in the next decades.

But what has all this to do with AI?

We are in the midst of a technological revolution that will determine our future. If we put things in their right context we will go in one direction, while, otherwise we will not.

In fact, the question of whether machines may soon become conscious is central. Not because I believe they will do but because the answer to the question determines how we perceive ourselves. Here, my argument is that AI, or whatever kind of artificial general intelligence (AGI), is and will remain an empty (black) box without a soul and even no mind at all. To avoid to make of a too-long story an endless essay, let me focus on one particular question: does AI understand anything? Does chatGPT have a deeper understanding—that is, what I call a ‘semantic awareness’ of what it is doing? Does a self-driving car, that ‘sees’ with its cams other cars, bicycles, the street, etc., understand what it is ‘seeing’? One might believe that knowing the answers to these questions isn’t of any practical interest because what matters is that the self-driving car drives me to the desired destination. But it is precisely this unwillingness to tackle questions of deeper significance that misled a plethora of investors into an enterprise that still is far away from meeting the expectations and, for the time being, is stalled. My answer is that, unless a car truly understands—that is, it goes beyond syntactic rules and statistical inferences, and also develops a semantic awareness of the environment—it will never be able to drive reliably and reach an automated level V driving. One must understand what one is seeing to be able to move in a complicated environment, otherwise, sooner or later, you will misinterpret a plastic bag for a cyclist, or a casual red badge on a wall for a stop sign.

You might reply that modern AI, especially in its incarnation of large language models (LLM), has already shown how machines have at least a partial understanding of texts and their contexts, and maybe, in part, of the world as well. It is only a matter of time before these faculties will be enhanced and will reach human levels and beyond.

Then let’s see what and how it ‘understands’ (whatever that might mean) and, especially, what it does NOT understand. To understand what AI truly understands it is much more revealing to see what it does certainly not understand (sorry for the wordplay…)

We can say that a LLM has some sort of ‘intelligence,’ the ability to infer from a large dataset an answer, it can reason logically and deductively much better than a traditional system based on past AI models and, sometimes, even better than humans. However, does it truly understand its own ‘word-crunching’?

Does it have common sense, intuitions, a semantic awareness? While it is able to deduce it looks much less able in inductive reasoning and more abstract ‘thinking.’ And, does it have creativity, agency, and motivation?

And what about consciousness, sentience, feelings, emotions, self-awareness, the ability to perceive love and hate, pleasure and pain, and finally possess a self and sense of individuality?

Some claim that it is only a matter of scaling up the system—that is, of more computing power, more artificial neurons, more datasets to be trained on, extending it from text to images, and videos, embodying it into forms of robots that will collect sensorial information, etc., and then, voilà… we will finally get to the AGI humanoid. But this line of reasoning assumes that there is already a glimpse of all these cognitive and psychological aspects present in the machine and that they need only to be amplified and perfected. If so, they will probably soon demonstrate to be right. Whereas, I claim that there is nothing alike in whatever kind of AI system, not even in its embryonal state.

So, let us take up the question of semantic awareness. What follows is a long list of examples that I found elsewhere (for example from Gary Markus Substack and Melanie Mitchell's lecture) that nicely illustrate how it is very questionable whether LLMs understand anything.

Let’s begin with the following. ChatGPT was asked the following…

It is clear that it fails to understand the negation (the "‘not-questions’.)

Here is another fascinating example….

The machine ‘hallucinates.’ It claims to ’see’ something that doesn’t exist.

The typical objection is that this doesn’t prove that LLMs or visual language models (VLMs) don’t understand. Because, after all, humans also can hallucinate.

That's true, however, such kind of hallucinations are present only in severe mental illnesses such as schizophrenia, in the last stages of dementia, or under the effects of psychedelics. They are the expression of a less conscious and more mechanical part of us, not an expression of consciousness itself. And, ultimately, even the subjects having the hallucinations, in most cases, know that they are having it.¹

Despite being trained on large textual datasets here is an example that shows a lack of understanding of the relationship between names and numbering of names.

This is not just a lack of semantic awareness about what we are talking about (the items we call ‘states’), it reflects also an inability to understand what one is doing (numbering items.)

Only the first sentence of ChatGPT’s answer makes it clear that something went wrong….

That there might be some problem with LLMs bias is a well-known fact, as the following image further portrays.

The point is that the system does not “refuse” but simply doesn’t understand what it is asked for. Why? Probably because it has been fed with images that show only white doctors treating black children, not the other way around. It guesses what the answer could be based on statistical inference, not genuine apprehension and comprehension of meaning. It is all about statistics, not understanding.

Here are some examples from Google DeepMind Gemini generative AI.

It “refuses” to create images of someone writing with the left hand. Why? The answer is simple: the system makes its best guess probabilistically, not semantically. It has been trained on a large amount of images where the vast majority of people depicted are right-handed. Thereby it guesses that the most likely answer must be that of a picture with a right-handed person. It doesn’t understand the question, it only guesses what the statistically most probable answer might be. If it is insistently pressed to furnish a different output it comes up with images like these…

That there must be something wrong with this image is self-evident. Not so for the machine…

Here is what a self-driving car ‘sees’ when it looks at a car with an advert on the back. The AI’s power to classify objects and people is impressive. It recognizes and distinguishes cars, bicycles, and persons from the rest of the environment. But the point is that classifying is not understanding! It does not understand what it is ‘seeing’ and isn’t able to discriminate between real and fictitious patterns.

Here is another example of how a self-driving car completely misinterprets the situation. It saw a cop with a stop sign (in the background on the right) and didn’t understand that it was a picture on a billboard. What this example shows again is that the system has a local comprehension but struggles to place it into a larger context with a global comprehension.

Here is how a trained neural network recognizes a school bus which, when presented with different angles and perspectives, makes a mess out of it. There seems to be no semantic awareness that can distinguish a bus from a punching bag.

Source: link

Below is a curious video created by Sora, a generative AI that can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters, only by asking for it in textual form. The prompt was the following.

The result….

Apart from the floating chair (yet another ‘hallucination’) notice how one of the persons disappears. There is a problem with objects/person permanence.

The same can be said with ‘wolf pups permanence’….

These were a couple of extracts from Sora. Undoubtedly, Sora’s photorealism is impressive. I guess it will have a lot of creative applications. Writers, artists, and filmmakers will find a gold mine in these kinds of applications. The time when real actors might no longer be needed is nearing. The potential of the coming generative AI tools is virtually infinite.

Of course, at the time of this writing, the above ‘glitches’ and ‘misunderstandings’ may have been fixed. The human intervention might adapt and fine-tune the system by re-programing it in a way that will make it furnish the right answers. But this also demonstrates the lack of semantic awareness.

Thus, before jumping to conclusions about machines that ‘reason’ and truly ‘understand’ anything, let alone become conscious and self-aware, let us come back to earth and realize that, so far, AI, LLMs, VLMs or whatever kind of sophisticated artificial intelligence does not understand a thing. As Melanie Mitchell put it: “LLMs have a weird mix of being very smart and very dumb.” I would add that while being smart, there is, nonetheless, nobody or nothing ‘in there’ that has a semantic awareness, and even less is sentience or has feelings. It was and remains a dumb machine that has no will, or agency and is, ultimately, only a black box that passively waits for an input to which it spits out an output according to some mechanical rules. And we can scale and magnify present AI systems ad infinitum, but that won’t help. Because zero times a gazillion is still zero.

I know that in these times of pervasive AI-hype, it sounds naive and my worldview might be seen as old and obsolete. But my claim is that the idea of an AGI or that of humanoids walking on the streets and working for us making us jobless, will remain a sci-fi fantasy. I’m willing to go even so far as to claim that, while self-driving cars may have some limited applications (e.g., robo-taxies driving along preestablished roads), they will not replace humans any time soon either. I guess that in the future we will look back and realize that, while a lot of great achievements have been made that were previously unimaginable, this neuro-centric worldview of intelligence, and mind, that is so overwhelmingly present in all AI-related debates, has led us astray in many other respects. Lots of investments will have gone empty-handed, and a certain type of scientific and humanistic progress might be stalled by the refusal to change our paradigm.

At least, hereby I offer you a method to falsify my claims: Let us look backward in, say, ten years, and see how far we got. This post will be still online (unless a stupid AI bot has deleted it for whatever reason) and we can compare predictions with facts.

At any rate, the question remains. What does it mean to ‘really deeply understand’? What is ‘meaning’ after all? Why is this semantic awareness so difficult to achieve in machines? To get closer to the answer you might like to take a look at the metaphysics of language. It is only by going beyond a materialistic thinking toward a post-material concept of mind, life, Nature, language, and consciousness that we can find the answer.

1. I recall how my mother suffering from dementia told me “I see cats everywhere. I know that it is not true. It is the effect of the drugs I take.”