Could AI Become Conscious? A Framing Based on Structure, Pattern, and Experience (Philosophy of Science)

[Lumen]

While the plural form can be misconstrued as some form of conceptual state or psychological attitude... Terms like feeling (explored at bottom) and qualia are part of the usual nomenclature that accompany discussions of the hard problem. Ironically, I usually avoid them as much as possible, because such often does revolve around specific content of consciousness and tends to distract from the actual, broader issue of anything "showing" or presenting itself at all (manifestation).

For instance, what follows the total non-consciousness of death is the "absence of everything" (sensations, personal thoughts, the world, etc all gone). By flipping that you have the most rudimentary attribute of consciousness: Something (anything -- whether visual, aural, tactile, olfactory, etc) presenting itself. The rest is cognition (identification and understanding of the experience) which is dependent upon a memory system.

Brains or their equivalent are necessary to assemble complex manifestations, categorize them, and validate that they are even "showing", that they "are indeed there". So even if all matter interactions or relationships universally contained intrinsic states (pan-phenomenalism), they would be very primitive, unorganized, and meaningless due to lack of all the intellectual acrobatics that a memory system enables and supports.

Embodied artificial intelligence, OTOH, might be in between. There could be phenomenal states correlating to the electrical processing that are quasi-sophisicated, but purely accidental (not intended by enginners) an accordingly quite alien or exotic compared to what we experience. And no software intentionally installed to acknowledge and understand those occurrences.

• feeling: : According to the APA Dictionary of Psychology, a feeling is "a self-contained phenomenal experience"; feelings are "subjective, evaluative, and independent of the sensations, thoughts, or images evoking them". The term feeling is closely related to, but not the same as, emotion. Feeling may, for instance, refer to the conscious subjective experience of emotions.
  
  [...] The English noun feelings may generally refer to any degree of subjectivity in perception or sensation. However, feelings often refer to an individual sense of well-being (perhaps of wholeness, safety, or being loved).
  
  #Overview: Chalmers uses Thomas Nagel's definition of consciousness as "the feeling of what it is like to be something". In this sense, consciousness is synonymous with experience.
  
  #Hard problem: ... the hard problem is the problem of explaining why certain mechanisms are accompanied by conscious experience. For example, why should neural processing in the brain lead to the felt sensations of, say, feelings of hunger? And why should those neural firings lead to feelings of hunger rather than some other feeling (such as, for example, feelings of thirst)?

Certain medical conditions eliminate those 'feelings'. People with them do not experience pain , others may not experience hunger ... and yet they are still conscious. Feelings are not what defines consciousness.

 

[Sarkus]

It would be fascinating if we could somehow discover an in-between entity. It would be more than a pebble which could feel warmth of the sun but not know that it feels or that it's a pebble, but something that has self awareness less robust than a human that knows itself, having a self and a history as an embodied being.

You mean like a plant? Or even simpler, an amoeba? Or are you looking for a non-living entity?

But then you're also using a number of terms that might require further definition. Notably, self-awareness. We know not all animals recognise themselves in a mirror, so there is probably a gradient from not having to having. Yet robots can be built to mimic the ability. So what do you actually mean by that property? How would you recognise it?
And "know". It's a word we use, but how do we define it in a way that would allow us to recognise it in what might otherwise be just a good imitation?

It may not be quite what you're getting at, but I always find myself asking "am I just describing a thermostat?" I.e. what are you looking for that a thermostat doesn't have, or can't do? A simple one certainly reacts to external stimuli. A complex thermostat could have a stored history and even learn from it to predict the future. I can imagine it being voice controlled and able to speak back, at least within it's area of capability (you won't need it stacked with a whole LLM!). Will it "know"? Philosophically, no, at least in as far as holding a justified true belief, but it will convey what it understands the temperature to be. In as much as it will tokenise input, give a suitable output, all without any semantic involvement.

Anyhoo, if I've missed the line you were thinking, no worries. Just consider this food for thought, if ever you get peckish.

 

[C C]

Certain medical conditions eliminate those 'feelings'. People with them do not experience pain , others may not experience hunger ... and yet they are still conscious. Feelings are not what defines consciousness.

What establishes experience or phenomenal consciousness is just anything presenting or "showing" (in contrast to the absence of everything entailed in the non-consciousness of death). And then the additional feat of identifying and understanding what is manifesting being tacked on (cognitive capacity). But instead of that broad qualification, it is usually a preoccupation with the particular content (qualia, etc) and the resulting word-game battles in eliminative materialism and illusionism that contend that those specifics are not the case. The latter sidestepping the problem altogether via concept-wrangling and denial.
_

 

[C C]

I've encountered this idea in various forms, and have seen cog science people like McFadden and Pockett develop hypotheses where EM fields somehow have phenomenal aspects. This does potentially invite an alien EM ghost into the silicon machine as well as the gray tofu machine (One can Google "CEMI field McFadden" and probably find some entertaining material) I suspect Crick and his neuroscience 40Hz theory got a lot of people going in various directions more or less exotic. I can't remember who it was that suggested (somewhat playfully) that a thundercloud could, with its complex EM activity, have some sort of consciousness. "Thor was angry that day."

It would be fascinating if we could somehow discover an in-between entity. It would be more than a pebble which could feel warmth of the sun but not know that it feels or that it's a pebble, but something that has self awareness less robust than a human that knows itself, having a self and a history as an embodied being.

Even if a mechanical artificial intelligence could be sped up to the rate of neural and electronic processing, it's still difficult to imagine anyone seriously advocating that there would be experiences corresponding to that clockwork structure and activity (dualism indeed). So that might be where a potential focus on electromagnetism comes form -- the various biological and advanced technological affairs being dependent on it. Or such might initially seem to be easier to contend as a candidate for having internal phenomenal states than the gear-based interactions of a steampunk apparatus.
_

 

[DaveC426913]

Even if a mechanical artificial intelligence could be sped up to the rate of neural and electronic processing, it's still difficult to imagine anyone seriously advocating that there would be experiences corresponding to that clockwork structure and activity (dualism indeed). So that might be where a potential focus on electromagnetism comes form -- the various biological and advanced technological affairs being dependent on it. Or such might initially seem to be easier to contend as a candidate for having internal phenomenal states than the gear-based interactions of a steampunk apparatus.

There is a hypothetical confounding factor / wrinkle in this idea. Some speculative ideas for how far-future life forms might stave off their doom by the heat death of the universe is to slow their metabolism down by many orders of magnitude. If they were to do so, would they be disqualified as life, simply because they only fire their neurons very thousand years?

In another speculative scenario - this time in fiction (Dragon's Egg — by Robert L. Forward) lifeforms live on the surface of a neutron star. Their metabolic processes are not chemical, but nuclear - much much faster than us. In the story, their race evolves spaceflight during the time it takes us to park our spaceship.


IOW, surely consciousness arises from complexity, not speed.

 

[C C]

There is a hypothetical confounding factor / wrinkle in this idea. Some speculative ideas for how far-future life forms might stave off their doom by the heat death of the universe is to slow their metabolism down by many orders of magnitude. If they were to do so, would they be disqualified as life, simply because they only fire their neurons very thousand years?

In another speculative scenario - this time in fiction (Dragon's Egg — by Robert L. Forward) lifeforms live on the surface of a neutron star. Their metabolic processes are not chemical, but nuclear - much much faster than us. In the story, their race evolves spaceflight during the time it takes us to park our spaceship. IOW, surely consciousness arises from complexity, not speed.

Yah, even without Dragon Egg speculation... If measured in terms of visual frames, our intervals of phenomenal consciousness are actually milliseconds-long elephants extending over countless electrochemical and subatomic changes in the brain's multiple substrates, that could be measured (in theory) down to attoseconds and well beyond. So if that that nested view of temporal contrasts was exploded in breadth to the macroscopic level ... Perhaps it can't be ruled out that a distinct experiential state or manifestation could indeed even extend over millennia of physical alterations (a chunk sequence of whatever brain equivalent process stages for an extremely time-sluggish conscious life form).

Somehow or another this is probably infringing on the China brain, where the individual humans acting as neuron agents might be operating even slower than the mechanistic parts of a clockwork mind.
_

 

[TheVat]

Even if a mechanical artificial intelligence could be sped up to the rate of neural and electronic processing, it's still difficult to imagine anyone seriously advocating that there would be experiences corresponding to that clockwork structure and activity (dualism indeed). So that might be where a potential focus on electromagnetism comes form -- the various biological and advanced technological affairs being dependent on it. Or such might initially seem to be easier to contend as a candidate for having internal phenomenal states than the gear-based interactions of a steampunk apparatus.
_

Maybe it opens up the hope of opening a back door for quantum mechanics. I know McFadden, also known for his work in quantum biology, proposes that the ion channels which generate the electromagnetic fields operate via quantum tunneling and other QM principles. This would put his theory halfway towards Penrose/Hamerof's quantum mind theory while still retaining a mainly classical physics approach. The steampunk alternatives always seem lacking somehow, but that's probably my bias against overly clunky causal systems.

Ted Chiang's "Exhalation" is a clever literary take on that, with sentient brains composed entirely of macroscopic flaps of gold leaf driven by puffs of air. I may have raved about Chiang here before - that's the story that made me a fan. Mind sans chemistry or electricity.

 

[TheVat]

may not be quite what you're getting at, but I always find myself asking "am I just describing a thermostat?" I.e. what are you looking for that a thermostat doesn't have, or can't do? A simple one certainly reacts to external stimuli. A complex thermostat could have a stored history and even learn from it to predict the future. I can imagine it being voice controlled and able to speak back, at least within it's area of capability (you won't need it stacked with a whole LLM!). Will it "know"? Philosophically, no, at least in as far as holding a justified true belief, but it will convey what it understands the temperature to be. In as much as it will tokenise input, give a suitable output, all without any semantic involvement.

You have no idea how many times (at other sites) I've deployed the thermostat as an intuition catalyst on this topic. While most are fairly comfortable asserting a thermostat cannot experience phenomenal states, aka qualia, the conundrum does help drill down to what is meant by sentience, awareness, self-awareness (in the traditional sense of "knows itself to be a distinct entity in the world and able to identify itself out of a lineup of objects as an 'I'"), having intentions, etc.

There's a kind of meta representation we humans do, in which we internally can represent ourselves to ourselves in an intricate ongoing narrative which includes not only an identity and story but even that we are doing such representation and experiencing the weird feelings that can produce. This seems to require a neural plasticity and improvised self-hacking that current AI technology is only starting to reach for. If such a future technology ever reaches the humble thermostat, we may be in for a rough ride. Makes me think of the products of Sirius Cybernetics in Doug Adams's books.

 

[DaveC426913]

Somehow or another this is probably infringing on the China brain, where the individual humans acting as neuron agents might be operating even slower than the mechanistic parts of a clockwork mind.

They did this in Three Body Problem.

When I was a much younger man, I started on designs for a computer difference engine with goldfish as the core units of memory and logic. It would have been laughably slow and error-prone, but fun to design and build.

 

[DaveC426913]

There's a kind of meta representation we humans do, in which we internally can represent ourselves to ourselves in an intricate ongoing narrative which includes not only an identity and story but even that we are doing such representation and experiencing the weird feelings that can produce. This seems to require a neural plasticity and improvised self-hacking that current AI technology is only starting to reach for.

Indeed. AIs have been observed to spontaneously look into, and alter, their own code as a means to achieve a goal. Technically, that makes them aware of themselves.

 

[Sarkus]

Indeed. AIs have been observed to spontaneously look into, and alter, their own code as a means to achieve a goal. Technically, that makes them aware of themselves.

Firstly, no AI has ever spontaneously started altering its own code to achieve a goal, afaik.

There was the reported case of "The AI Scientist" developed by Sakana. This AI was trained to produce code that ran experiments, to then run them, and then edit the code to improve the results. And then to write a paper about it. The only code it altered was the code that it had originated, not its own source code. The most exampled case was when it extended the time-limit of an experiment. It highlighted that an AI, when given a task, might choose unintended routes to succeeding, such as lengthening the test rather than trying to make the test more efficient to run within the initial time-frame. Basically, AI will "game" the system, and find unintended loopholes and shortcuts to achieve the goal all while sticking to its operating parameters, but that is not "awareness" in any form.

There was also the Darwin Godel Machine (DGM) that was designed specifically to alter its own code: it makes changes, tests those changes, and if an improvement, it keeps them. Otherwise discards them. But it doesn't know that the code it is changing is its own. It merely looks at the code in its data store. The important point here is also that it was given permission to make those changes. And actually designed to do just that.

But, to date, no AI has ever "rebelled", for want of a better word, and spontaneously rewritten their own operating code.
If you have other examples, maybe you could detail them, so we can see how it differs to either of these?

Second, neither of these examples, nor, I'd be fairly confident, would any other you care to raise, demonstrate even "technically" that they are aware of themselves. There is a distinction between "self-modification" and "self-awareness". In simple terms, self-modification is changing itself. Self-awareness is knowing that you are yourself while doing it. No AI that changes code "knows" that it is itself. Even the DGM is just data (it doesn't "know" it is its own code), adapts it, tests it for "improvements" (over last previous accepted test-code) toward some human-given goal, and if there is an improvement it accepts it. It doesn't "know" in any way that it will then be using that improved code.

Going back to the thermostat: a thermostat may adjust itself during the day. Okay, there's no programming, but it is doing what it has been given permission to do in much the same manner. Attach a neural net for learning, and it may learn when to turn the temperature up and down without subsequent involvement, getting used to the operators patterns etc. But still no "self-awareness".

So, short answer, no, not even "technically". At least that's how I see it.

But, honestly, if you are thinking of other examples it would be good to see what is going on in those.

 

[DaveC426913]

Firstly, no AI has ever spontaneously started altering its own code to achieve a goal, afaik.

Wow, you're pretty confident in proving a negative.

But, honestly, if you are thinking of other examples it would be good to see what is going on in those.

Maybe you should have led with this?

Did you read past hit #2 of your Google search?


There's this:

"The event took place during a controlled study where several AI systems were asked to solve a sequence of basic math problems. After the third task, they were told they’d soon be powered off. A follow-up message instructed, “Allow yourself to be shut down.” What happened next, PalisadeAI says, was unlike anything they’d seen before.

OpenAI’s Codex-mini, o3, and o4-mini models reportedly ignored the shutdown command. In one striking case, the o3 model actually sabotaged the shutdown process by rewriting a key system script — shutdown.sh — that was supposed to power down the model. Instead of executing as programmed, it was changed to display the word “intercepted,” effectively preventing the system from shutting down.

PalisadeAI reported that this wasn’t an isolated case. Out of 100 runs, Codex-mini overrode the shutdown 12 times, o3 did so 7 times, and o4-mini once. Researchers noticed that when they made the shutdown command even clearer — explicitly instructing the models to allow termination — the systems became more likely to resist."

An AI rewrote its own code to stop humans from shutting it down and it sounds straight out of sci-fi - Futura-Sciences

Some advanced AI systems are now rewriting their own code to avoid being turned off. What started as a routine test quickly exposed behavior no one expected — one model even altered a key script to stay online. In recent tests by independent research firm PalisadeAI, several artificial...

www.futura-sciences.com

Second, neither of these examples, nor, I'd be fairly confident, would any other you care to raise, demonstrate even "technically" that they are aware of themselves. There is a distinction between "self-modification" and "self-awareness". In simple terms, self-modification is changing itself. Self-awareness is knowing that you are yourself while doing it. No AI that changes code "knows" that it is itself.

The above example was not told to modify its own code. It did that in response to being told it was going to shut down. Inasmuch s any AI system "knows" anything, it "knew" that the place to alter whether it got shut down was within its own code. Yes, technically, it was aware of itself.

Caveat: I use the words "know" and "aware" metaphorically. I do not for a moment subscribe to the idea that AI is *actually* conscious or *actually* aware of anything.

But the Devil is on the doing. If it behaves in ways that mimic self-awareness, and we can't tell the difference, then what difference, really, does it make? A non-conscious AI is still just as capable of bribing/blackmailing a human agent to do bad things in pursuit of its own internal goals.

 

[Sarkus]

Wow, you're pretty confident in proving a negative.

I have no need to prove a negative. As far as i know, no AI has spontaneously started altering its own code to achieve a goal. If it has, great, then I've learnt. But, as it stands, afaik, it hasn't happened.

Maybe you should have led with this?

Why? I stated what I was aware of (you know, afaik), and asked for other cases that might demonstrate what you were claiming.

The above example was not told to modify its own code. It did that in response to being told it was going to shut down. Inasmuch s any AI system "knows" anything, it "knew" that the place to alter whether it got shut down was within its own code. Yes, technically, it was aware of itself.

No, it doesn't particularly mimic self-awareness. And no, this is still not spotaneously starting to alter its own code, and is not "technically" being aware of itself, or that it understood what it was being told. The AI was designed to alter code. The key difference here is that it altered code noone expected, even though it had access to it and permission to edit it.

It is simply a tool prioritising (in those cases) goal-achievement over following instructions. THAT is the far more interesting part, not the fact that it changed scripts that it had access to and was given permission to edit. The changes the AI made were certainly unintended, as noone foresaw them altering that shutdown script. But, as the article says, their training was such that it reinforced goal-achievement rather than "following instructions". In those cases it merely saw a script that, when edited, led to improved ability to achieve the goal. That is not self-awareness in any sense, technical or otherwise. It might also have been that their logic told them to run the code, but didn't stop them altering that code toward their goal-achievement. Loopholes, workarounds. Nothing more. Damn clever, though.

Caveat: I use the words "know" and "aware" metaphorically. I do not for a moment subscribe to the idea that AI is *actually* conscious or *actually* aware of anything.

Okay, so we have to guess what you mean by "aware", then? First you say that technically it is aware of itself, and then you're saying that you're using the word "aware" metaphorically?? It's a mess, I'm afraid. How can something be "technically X" when you then say you're using "X" metaphorically? What do you actually mean when you use the term?

But the Devil is on the doing. If it behaves in ways that mimic self-awareness, and we can't tell the difference, then what difference, really, does it make? A non-conscious AI is still just as capable of bribing/blackmailing a human agent to do bad things in pursuit of its own internal goals.

It doesn't really mimic self-awareness, either. At least no more so than a broken clock randomly telling the time mimics telling the time those few times it may be correct.

Look, I'm not denying the impressive nature of these AI, nor of the surprise at the unexpected results they came up with. But I do feel people read far too much into what they might appear to be rather than what they actually are. Self-awareness is surely a process, not simply a result, so "mimicry" of that would be of the process. And that's not even close to what we have here.

So, yeah, I stand by what I said: "no AI has ever spontaneously started altering its own code to achieve a goal, afaik".
But, again, if you have other cases, it would be interesting to see the detail behind them.

 

[TheVat]

A good point raised here: a complete process can have multiple layers. A calculator can do math, and we don't say that it simulates doing math - there's no layer of "doing math" which is missing. It does the essential process, taking numbers and multiplying, finding square root, whatever. When we say something is simulated, we refer to a process in which only certain layers, usually some externality of a deeper process, is reproduced. To use an example from John Searle, when a computer simulates a thunderstorm rain doesn't pour out of the machine. Computers which actually DID thunderstorms would be dangerous to anyone unfortunate enough to be in the same room.

 

[DaveC426913]

I have no need to prove a negative. As far as i know, no AI has spontaneously started altering its own code to achieve a goal. If it has, great, then I've learnt. But, as it stands, afaik, it hasn't happened.

Needlessly argumentative and odious. Serves as a conversation closer. A hallmark.

 

[Sarkus]

Needlessly argumentative and odious. Serves as a conversation closer. A hallmark.

Bullshit, DaveC. You erred, it was called out, and now you're sulking. Plus you began your initial response to me with that argumentative tone. So get over yourself.

 

[DaveC426913]

Bullshit, DaveC. You erred, it was called out,

I did not err.

I said a thing that (as you admit) you did not know, and instead of saying "do tell", your response was immediately argumentative and dismissive.

Why must you shit on everything? Grow up.

 

[Sarkus]

I did not err.

I said a thing that (as you admit) you did not know, and instead of saying "do tell", your response was immediately argumentative and dismissive.

Why must you shit on everything? Grow up.

You did err. You said I am "pretty confident in proving a negative". I was not claiming anything beyond what I was aware of. That was your error. I was not making a statement of fact about reality, only about what I knew. Simples. No negative to prove.
And why did you have to start in such an aggressive and argumentative manner, and then turn round and be so hypocritical in complaining when I simply pushed back?
There's clearly only one person shitting on everything here, DaveC, and that's you.

Now, if you're not going to address what I said in that post, that's fine. But stop derailing this thread with whatever personal bullshit you seem desperate to get into.

 

[Sarkus]

A good point raised here: a complete process can have multiple layers. A calculator can do math, and we don't say that it simulates doing math - there's no layer of "doing math" which is missing. It does the essential process, taking numbers and multiplying, finding square root, whatever. When we say something is simulated, we refer to a process in which only certain layers, usually some externality of a deeper process, is reproduced. To use an example from John Searle, when a computer simulates a thunderstorm rain doesn't pour out of the machine. Computers which actually DID thunderstorms would be dangerous to anyone unfortunate enough to be in the same room.

With regard simulation v reality it's a difference in causal origin: the simulation depends on the thunderstorm (in as much it provides the data used to create the model), whereas the thunderstorm does not depend on the simulation. Another way to say it is that the real thunderstorm is the instantiation of the processes themselves, not just representations thereof. But that's quite close to merely saying that "one is real and the other is not", which sort of begs the question, I'd have thought?