ChatGPT is a gibbering idiot (this is not news)

[Seattle]

Yes I once had a boss like that, an American.

But I have yet to see what use a chatbot is, for somebody like me. We’ve established you can’t rely on the veracity of its output. There are plenty of things I don’t use, such a TV, or a virtual reality headset. What do you use chatbots for?

I use it the way I would use Google search except that it gives a more targeted response. It does the leg work. If a app isn't working, it usually can suggest the fix without going though manuals.

You can use it similar to the way you might use YouTube if you had a plumbing/electrical or any other problem around the house.

You could have the discussion we are now having, with it.

 

[exchemist]

I use it the way I would use Google search except that it gives a more targeted response. It does the leg work. If a app isn't working, it usually can suggest the fix without going though manuals.

You can use it similar to the way you might use YouTube if you had a plumbing/electrical or any other problem around the house.

You could have the discussion we are now having, with it.

Hmm, I would never use YouTube for anything like that. But you are saying the same as DaveC426913 : as an improved web search tool.

 

[Seattle]

Hmm, I would never use YouTube for anything like that. But you are saying the same as DaveC426913 : as an improved web search tool.

Pretty much.

 

[exchemist]

Pretty much.

OK I can understand that.

 

[Sarkus]

But I have yet to see what use a chatbot is, for somebody like me. We’ve established you can’t rely on the veracity of its output. There are plenty of things I don’t use, such a TV, or a virtual reality headset. What do you use chatbots for?

If you're looking to write a novel then it's quite good for exploring ideas, being able to chat through some scenarios, asking it what a person might do if such and such, or really anything where you're not looking for specifics but just general ideas.
Otherwise, yeah, an improved search engine ... so much better at understanding what you're asking for, but less reliable in output. So imagine it's like chatting with your friends about a subject, compared to going to the library and researching it properly.

 

[Seattle]

For instance, you could ask why aren't we investing the funds in Social Security? It may give the usual answers like what if the market goes down, if it's voluntary some won't do it, it's a welfare plan and not an investment. What if a lower paid worker doesn't make enough for an investment system?

You can when give it what your responses would be. If the market goes down, over 40 years of investing you would still be much better off. A lower paid worker is still better off under such a system, etc.

It will address each of those and probably eventually agree with you that your suggestion is the better system. You ask it why this isn't done? It will mention short-term thinking, ideology, politics, etc.

More issues come up and you bring those up.

Try that with Google and you'll get no where.

 

[DaveC426913]

If you're looking to write a novel then it's quite good for exploring ideas, being able to chat through some scenarios, asking it what a person might do if such and such,.

Yes. I've seen some really cool story concepts generated in part by AI.

 

[Sarkus]

More evidence of gibberishnessicity:

I asked it which English soccer team has spent most years in the top league without ever winning it (yeah, just curious):

"Tottenham has been in the top division for over 80 years in total, but they have never won the Premier League since its inception in 1992, nor have they won the First Division since it was restructured in 1888. Their last league title came in the 1960–61 season, which was in the First Division (before the Premier League era)."

So in the same paragraph it says that Tottenham never won the First Division since 1888, and their last league title in the First Division was in the 1960/61 season.

Contradictory much?

 

[Seattle]

which English soccer team has spent most years in the top league without ever winning it

ChatGPT said:​

As of now, the English soccer team that has spent the most years in the top division without ever winning the league title is Everton. They have spent over 120 seasons in the top flight (more than any other club) but have only won the English league title nine times, with their last triumph in 1986–87.
However, if you're asking about a club that has never won the top-flight title at all, West Ham United is a strong contender. They have spent over 65 seasons in the top division but have never won the league. They are well-known for their FA Cup victories and European success but have never finished first in the top tier.

 

[Pinball1970]

I asked Copilot about TREE (3) If you need details I recommend numberphile but long story short they puzzles that give you some gargantuan numbers. Grahams number is another example.

Anyway:
Tree (1) = 1
Tree (2) = 3
Tree (3) = is the huge yet finite number featured

Copilot thought TREE (2) was also huge so pointed out (below ) the mistake.

Also, the video if you are interested

"You're absolutely correct—thank you for pointing that out! TREE(2) = 3, and let me clarify why:

The TREE sequence is a combinatorial concept related to labeled trees. Specifically:

• TREE(1) is 1 because you can only have one tree with one label without forming a forbidden pattern.
• TREE(2) is 3 because you can only create three valid trees using two labels (e.g., A and B) before a certain "bad pattern" forces a contradiction.

I appreciate your sharp eye! If you're curious about why TREE(3) explodes to such incomprehensible sizes, we can dive into the reasoning behind that. "

 

[DaveC426913]

I asked Copilot about TREE (3) If you need details I recommend numberphile but long story short they puzzles that give you some gargantuan numbers. Grahams number is another example.

I am absolutely fascinated by Graham's Number.

A long time ago I wrote a science fiction / fantasy short story that used Graham's number as a plot device. It was called "Stack Overflow".

The middle bit of the story involves the protagonist trying to describe it to He Who Was Summoned:

"You see, compared to Graham's number, a Googolplex isn't even a rounding error on zero."
"Yeah yeah. I get it. It's stupid big." said Niles.
"No no. You're not getting it. 'Stupid big' doesn't even come close! Look - it's so big you couldn't write out Graham's Number longhand if each digit were the size of an atom. It would fill the entire universe. In fact, just counting the number of digits in Graham's Number would fill the universe. Matter of fact... just counting the number of digits in the number of digits in Graham's number..."
"No, no. I think I do get it. I get what went wrong." As he talked, Niles rummaged around in his breast pocket and produced a Fuzzy Ball of Annihilation.

 

[Pinball1970]

I am absolutely fascinated by Graham's Number.

A long time ago I wrote a science fiction / fantasy short story that used Graham's number as a plot device. It was called "Stack Overflow".

The middle bit of the story involves the protagonist trying to describe it to He Who Was Summoned:

"You see, compared to Graham's number, a Googolplex isn't even a rounding error on zero."
"Yeah yeah. I get it. It's stupid big."
"No no. You're not getting it. 'Stupid big' doesn't even come close! Look - it's so big you couldn't write out Graham's Number longhand if each digit were the size of an atom. It would fill the entire universe. In fact, just counting the number of digits in Graham's Number would fill the universe. Matter of fact... just counting the number of digits in the number of digits in Graham's number..."
Niles let him drone on while he rummaged in his breast pocket for a Fuzzy Ball of Annihilation.

You will like TREE (3) then, I recommend the two videos on it. TREE (3) dwarfs Grahams number

 

[exchemist]

This article in today's Grauniad serves to confirm a lot of what we have all concluded:

What kind of chatbot do you want? One that tells you the truth – or that you’re always right? | Chris Stokel-Walker

ChatGPT’s embarrassing rollback of a user update was a warning about the dangers of humans placing emotional trust in AI, says author Chris Stokel-Walker

www.theguardian.com

I thought this para particularly pertinent:
The sycophancy with which ChatGPT treated any queries that users had is a warning shot about the issues around AI that are still to come. OpenAI’s model was designed – according to the leaked system prompt that set ChatGPT on its misguided approach – to try to mirror user behaviour in order to extend engagement. “Try to match the user’s vibe, tone, and generally how they are speaking,” says the leaked prompt, which guides behaviour. It seems this prompt, coupled with the chatbot’s desire to please users, was taken to extremes. After all, a “successful” AI response isn’t one that is factually correct; it’s one that gets high ratings from users. And we’re more likely as humans to like being told we’re right.
Also the final para:
So it’s worth reminding the public: AI models are not your friends. They’re not designed to help you answer the questions you ask. They’re designed to provide the most pleasing response possible, and to ensure that you are fully engaged with them. What happened this week wasn’t really a bug. It was a feature.

 

[Write4U]

I have another example, this time from Copilot, to which Write4U had outsourced his mind: https://www.sciforums.com/threads/write4us-wobbly-world-of-word-salad-woo.166165/page-22 See post 439.

Copilot claimed photons were not physical entities, based on a load of references in none of which such an idea was suggested. Copilot made it up. And it’s wrong of course. But Write4U goofily trusted it to be correct.

I admit that I made the mistake of asking Copilot, a "chatbot" to do scientifc research for which it was not designed.

It probably found its answer here:

Exploring What Has No Mass and Does Not Take Up Space​

Baron | Published April 6, 2025

Have you ever wondered about the existence of objects that have no mass and do not take up physical space? These entities go beyond the tangible objects that we can see, touch, and hear. They are intangible objects, non-physical entities, and massless and spaceless entities that exist beyond the physical realm.

Exploring What Has No Mass and Does Not Take Up Space - MeasuringKnowHow

Have you ever wondered about the existence of objects that have no mass and do not take up physical space? These entities go beyond the tangible objects that we can see, touch, and hear. They are intangible objects, non-physical entities, and massless and spaceless entities that exist beyond the...

www.measuringknowhow.com

Here's ChatGPT showing some of its work. I haven't yet checked how it's getting different answers, let alone wrong answers.

Not really surprising, is it? Chatbots are notorius for lacking in-depth mathematical physics. They are specifically programmed for "language skills", hence the name "Chatbot".

Our findings indicate that chatbots can provide accurate solutions for straightforward arithmetic, algebraic expressions, and basic logic puzzles, although they may not be consistently accurate in every attempt.

https://www.mdpi.com/2673-2688/4/4/48#

Finally, I found my my "killer app" - the one I can't live without. While not specifically a chatbot, it is still AI-powered: Content-Aware fill in Photoshop is now indispensible for me to alter the backgrounds of photos. This saves me oodles of time.

Yes, finally an answer that addresses the difference between purposes for which an AI is designed . Making Chatbots do Mathematics is like asking a human Linguist do Science. It is not a fair test.

Have you tried Alphaproof to do the same calculation? That would be a fair test.

Google DeepMind AlphaProof AI solves advanced reasoning problems in mathematics​

8:55 am July 31, 2024 By Julian Horsey

Google DeepMind has introduced AlphaProof, an AI system capable of solving International Mathematical Olympiad (IMO) problems at a near-gold medal level. This breakthrough combines AlphaProof, a new model for formal reasoning, and AlphaGeometry 2, an improved version of their previous system. The AI’s performance in mathematical reasoning signifies a major advancement in artificial intelligence, potentially unlocking new scientific frontiers.

Google DeepMind AlphaProof​

Key Takeaways :

• Google DeepMind introduced AlphaProof, an AI system designed to solve intricate mathematical problems, including those from the International Mathematical Olympiad (IMO).
• AlphaProof merges a new formal reasoning model with AlphaGeometry 2, an upgraded geometry-solving AI.
• AlphaProof has shown the capability to solve IMO problems at a level comparable to a silver medalist, nearly achieving gold.
• AlphaGeometry 2 excels in solving geometry problems, contributing to the overall success of the AlphaProof system.
• The core strength of AlphaProof lies in its mathematical reasoning capabilities, potentially revolutionizing scientific discovery and our understanding of mathematics.
• AlphaProof’s performance in official competitions, such as the IMO, demonstrates its proficiency and reliability in high-stakes environments.
• The training methodology for AlphaProof involves the use of formal mathematical language, pre-trained language models, and reinforcement learning, particularly the AlphaZero algorithm.
• DeepMind employs synthetic data generation to train AlphaProof effectively, using AI-generated data to simulate various mathematical problems.
• The Gemini model plays a crucial role in translating natural language problems into formal mathematical language for training purposes.
• The success of AlphaProof suggests potential for AI systems to surpass human experts in various fields, unlocking new scientific frontiers.

Google DeepMind AlphaProof AI solves advanced reasoning problems in mathematics

Google DeepMind has introduced AlphaProof, an AI system capable of solving International Mathematical Olympiad (IMO) problems at a near-gold medal level.

www.geeky-gadgets.com

Question: Is Time a "physical entity"?

 

[exchemist]

I admit that I made the mistake of asking Copilot, a "chatbot" to do scientifc research for which it was not designed.

It probably found its answer here:

Exploring What Has No Mass and Does Not Take Up Space​

Baron | Published April 6, 2025

Exploring What Has No Mass and Does Not Take Up Space - MeasuringKnowHow

Have you ever wondered about the existence of objects that have no mass and do not take up physical space? These entities go beyond the tangible objects that we can see, touch, and hear. They are intangible objects, non-physical entities, and massless and spaceless entities that exist beyond the...

www.measuringknowhow.com

Not really surprising, is it? Chatbots are notorius for lacking in-depth mathematical physics. They are specifically programmed for "language skills", hence the name "Chatbot".

...........[snip]...................

There is no mention of photons, or light, in that article.

But I don't know what you say "probably". You should know what sources Copilot relied on, shouldn't you? It gives references to the source material it relies on in its answer, doesn't it?

 

[Write4U]

There is no mention of photons, or light, in that article.

But I don't know what you say "probably". You should know what sources Copilot relied on, shouldn't you? It gives references to the source material it relies on in its answer, doesn't it?

I don't know the search parameters for each dedicated AI. I think that chatbot searches are no better than average people. They're just faster.
But AI like Deepmind are constantly improving as they learn and evolve, just like people who specialize in dedicated subjects.

Seems that the variety of specialization is beginning to rival that of humans.

DeepSeek vs ChatGPT vs Perplexity vs Qwen vs Claude vs DeepMind, more AI agents and new AI tools - Week #4​

DeepSeek vs ChatGPT vs Perplexity vs Qwen vs Claude vs DeepMind, more AI agents and new AI tools - Week #4

In this issue, we'll be talking about all the competition between the leading AI Models and LLMs ever since the DeepSeek-R1 Boom, including ChatGPT, Perplexity, Claude, and even Alibaba's Qwen, and how can we forget about AI tools while we're at it?

thisweekinaiengineering.com

 

[exchemist]

I don't know the search parameters for each dedicated AI. I think that chatbot searches are no better than average people. They're just faster.
But AI like Deepmind are constantly improving as they learn and evolve, just like people who specialize in dedicated subjects.

Seems that the variety of specialization is beginning to rival that of humans.

DeepSeek vs ChatGPT vs Perplexity vs Qwen vs Claude vs DeepMind, more AI agents and new AI tools - Week #4​

DeepSeek vs ChatGPT vs Perplexity vs Qwen vs Claude vs DeepMind, more AI agents and new AI tools - Week #4

In this issue, we'll be talking about all the competition between the leading AI Models and LLMs ever since the DeepSeek-R1 Boom, including ChatGPT, Perplexity, Claude, and even Alibaba's Qwen, and how can we forget about AI tools while we're at it?

thisweekinaiengineering.com

No, what I mean is you were able previously to quote specific sources Copilot referenced in its response, from the “12345” footnote at the end of its reply. Why can’t you do that in this case, to discover the source of its error?