ChatGPT

[Mike Helland]

No, they cannot do either of these things. You are mistaking the appearance of fluency for the appearance of reasoning.

Sounds like humanity in a nutshell.

How much of human thought in history has been simply fiction?

How likely is it that we in the present are really any different?

 

[Dr.Sid]

Can you explain that? They clearly can learn. Much faster and better than we do. As well as make plans.

They have learning phase, when the LLM is being built. But after that, they don't learn, beyond scope of the prompt. If you start new session, the model is exactly as it was before. The model is not changed by generating the response in any way. It only uses its own previous responses as part of the prompt for the next responses, till it reaches the limit of prompt length, which for ChatGPT is fairly low (2048 tokens IIRC). So yes, it can learn anything you can describe in 2048 words, but then it has to forget it, to continue the conversation. The prompt is similar to short term memory .. but it has no long term memory.
Planing is similar. In some cases it may seem it does planning. For example it can be asked to write a short program, and it can start by writing the bullet points of the algorithm, and then the algorithm itself.
But it does not plan this process ahead. You ask this, you run the LLM, and it generates first word of the response. That's it. You than have to run the LLM again .. it will start from complete blank state, read your question again, read first word of its response .. and add another word .. and again, and again, until it says "end of response". It can generate the plan first this way. But it's unclear, if the plan is used the detailed response, and in any case, the plan can only exist in the limited scope of the prompt, as the content of LLM itself is invariable.

 

[Mike Helland]

They have learning phase, when the LLM is being built. But after that, they don't learn, beyond scope of the prompt. If you start new session, the model is exactly as it was before. The model is not changed by generating the response in any way. It only uses its own previous responses as part of the prompt for the next responses, till it reaches the limit of prompt length, which for ChatGPT is fairly low (2048 tokens IIRC). So yes, it can learn anything you can describe in 2048 words, but then it has to forget it, to continue the conversation. The prompt is similar to short term memory .. but it has no long term memory.

I think that's a safety measure. So people can't teach it bad things.

https://www.cbsnews.com/news/microsoft-shuts-down-ai-chatbot-after-it-turned-into-racist-nazi/

Microsoft shuts down AI chatbot after it turned into a Nazi
March 25, 2016

That's 7 years ago.

I had a conversation with ChatGPT about my alternative cosmology, and it said this:

Do you remember formula 2 from earlier in our conversation?

I'm sorry, but I do not have the ability to remember previous conversations. As a large language model trained by OpenAI, I do not have the ability to store or recall information outside of what I have been trained on. I can provide information and answer questions based on the information that I have been trained on, but I do not have the ability to remember previous conversations or access additional information. If you have a specific question or topic that you would like to discuss, I would be happy to provide information to the best of my ability.

Michael Helland
I mean from this conversation. d=z/(1+z) * c/H_0,

Yes, that is correct. The formula you are referring to is d=z/(1+z) * c/H_0, which is an alternative form of the redshift-distance relationship in the Lambda Cold Dark Matter (LCDM) model of cosmology. This formula can be derived from the LCDM model by assuming that the universe is flat and that it contains no dark matter or dark energy. It is a close approximation to the mainstream LCDM model's lookback times, and it can be used to make predictions about the expansion and evolution of the universe.

Michael Helland
Do you know who first discovered or proposed this formula?

I'm sorry, but I do not have information about the origin of the formula d=z/(1+z) * c/H_0. As a large language model trained by OpenAI, I do not have detailed knowledge about the history of scientific discoveries or the individuals involved in their development. I can provide general information about topics that I have been trained on, but I do not have detailed knowledge about specific events or the people involved in them. If you have a specific question or topic that you would like to discuss, I would be happy to provide information to the best of my ability.

It seems to have the ability to recall what you've talked about. But as a tool for people to use, you have to sandbox this stuff. That doesn't seem like a technical limitation. That just seems like a pretty Good Idea^TM.

Here's another interesting part of that conversation:

Are you familiar with the crisis in cosmology?

I'm sorry, but I am not familiar with any current crisis in cosmology. As a large language model trained by OpenAI, I do not have the ability to access current events or information outside of what I have been trained on. I can provide general information about topics in cosmology based on what I have been trained on, but I do not have detailed knowledge about current developments or controversies in the field. If you have a specific question or topic that you would like to discuss, I would be happy to provide information to the best of my ability.


It's called the cosmological crisis, also the Hubble tension.

I'm sorry, but I am not familiar with the cosmological crisis or the Hubble tension. As a large language model trained by OpenAI, I do not have the ability to access current events or information outside of what I have been trained on. I can provide general information about topics in cosmology based on what I have been trained on, but I do not have detailed knowledge about current developments or controversies in the field. If you have a specific question or topic that you would like to discuss, I would be happy to provide information to the best of my ability.


What is the expansion rate of the universe?

The expansion rate of the universe is a measure of how quickly the universe is expanding. It is commonly expressed in terms of the Hubble constant, which is a unit of measurement that describes the rate at which the universe is expanding. The current best estimate of the Hubble constant is approximately 73.5 kilometers per second per megaparsec. This means that for every 3.26 million light-years that a galaxy is from us, it is moving away from us at a rate of 73.5 kilometers per second. The expansion rate of the universe is an important parameter in cosmology, and it is used to make predictions about the evolution and structure of the universe.


What is the value of Hubble's constant according to measurements of the CMB?

The value of the Hubble constant that is derived from measurements of the cosmic microwave background radiation (CMB) is slightly lower than the value that is derived from other methods. The current best estimate of the Hubble constant from CMB measurements is approximately 67.4 kilometers per second per megaparsec. This value is slightly lower than the value of 73.5 kilometers per second per megaparsec that is derived from other methods, such as measuring the distances to galaxies using standard candles. The discrepancy between these two values is known as the Hubble tension, and it is currently a topic of active research in cosmology.


Yep, that's the Hubble tension alright.

Yes, that is correct. The discrepancy between the value of the Hubble constant that is derived from measurements of the cosmic microwave background radiation (CMB) and the value that is derived from other methods is known as the Hubble tension. This tension is currently a topic of active research in cosmology, and it has implications for our understanding of the expansion and evolution of the universe. Some possible explanations for the tension include systematic errors in the measurements, variations in the expansion rate of the universe over time, or the existence of new physics beyond the current standard cosmological model. Further observations and research will be needed to fully understand the nature of the Hubble tension and its implications for cosmology.

It seems to wiffle around and give excuses if it doesn't really understand the question. But if you lead it there it turns into a know-it-all.

Pretty human if you ask me.

 

[Fast Eddie B]

Here's a (probably weak) analogy that popped into my mind...

Einstein had a thought experiment: if one was in a sealed space ship measuring 1G, it was impossible to determine if it was sitting on the surface of the earth or in space accelerating at 1G. So he reasoned that gravity and acceleration were different ways of viewing the same force.*

In the same way if it is impossible to determine whether an output is being generated by an LLM or by a human, then is it not possible that they are doing the same thing? IOW, whether its Einstein postulating the Theory of Relativity, or me typing one word after another here on my Mac, or an LLM generating text from a prompt, perhaps fundamentally its all the same process?


*Feel free to correct me if I have that wrong.

 

[MRC_Hans]

Here's a (probably weak) analogy that popped into my mind...

Einstein had a thought experiment: if one was in a sealed space ship measuring 1G, it was impossible to determine if it was sitting on the surface of the earth or in space accelerating at 1G. So he reasoned that gravity and acceleration were different ways of viewing the same force.*

In the same way if it is impossible to determine whether an output is being generated by an LLM or by a human, then is it not possible that they are doing the same thing? IOW, whether its Einstein postulating the Theory of Relativity, or me typing one word after another here on my Mac, or an LLM generating text from a prompt, perhaps fundamentally its all the same process?


*Feel free to correct me if I have that wrong.

Well, actually it is in principle possible to discern between a gravity field and constant acceleration. In a small spaceship it would require some sophisticated (but existing) equipment ^*). If it's a gravity field, it will be reduced if you measure at different "heights" i.e. distances from the gravity center, if it's an acceleration, it will be constant at all heights ^**).

Even if they were alike, you could not infer anything about other processes that appear equal.

Hans

*) There exists gravimeters that can detect the difference in gravity of a few inches of elevation.

**) [derail]Some flat-earthers claim that gravity does not exist but is instead the (flat) Earth being accelerated upwards. This is easily debunked by pointing out that gravity is reduced if you move to higher ground.[/derail]

 

[theprestige]

Here's a (probably weak) analogy that popped into my mind...

Einstein had a thought experiment: if one was in a sealed space ship measuring 1G, it was impossible to determine if it was sitting on the surface of the earth or in space accelerating at 1G. So he reasoned that gravity and acceleration were different ways of viewing the same force.*

In the same way if it is impossible to determine whether an output is being generated by an LLM or by a human, then is it not possible that they are doing the same thing? IOW, whether its Einstein postulating the Theory of Relativity, or me typing one word after another here on my Mac, or an LLM generating text from a prompt, perhaps fundamentally its all the same process?


*Feel free to correct me if I have that wrong.

We know exactly how LLMs produce strings of text. They're statistical prediction algorithms. Nothing we know about human cognition suggests that it is in any way similar to what LLMs do. But sure, I suppose that maybe, deep down, humans are just doing rote prediction based on mathematical analysis of (large) bodies of data. In a woo, "aha! you can't rule it out!" sense, at least.

The first idea you have to jettison - with extreme prejudice - is the idea that Large Language Models are making mysterious inductive leaps, that aren't fully explained by their programming or fully understood by their programmers.

 

[Dr.Sid]

There might be similarities. Neural networks are modeled around how we think neurons work. But it's very loose analogy. Neural networks do not really emulate neurons, and we don't really understand how they work enough to emulate them.

We know neural networks as a mathematical concept can learn by example, we know why, we know how, we know how much network of specific size can learn. Real brain most likely works in similar way, but we don't know how much they are the same.

Also the understanding on text inside LLM and inside brain might be similar. Most likely it is. But again, we don't know. We don't understand how LLMs are exactly stored inside our mathematically well understood neural networks, much less real brains.

But most likely both mathematical networks and real ones are mainly about relations between concepts. Where concept on the border of the network is a word .. and deeper you go, more abstract it is. Few words having similar meaning, that meaning being some class of similar objects .. its grammar function .. relation to other words .. to possibly deep philosophical concepts which branch to huge volumes of texts.

LLMs have nothing to do with perception of reality through human senses, with perception of time, with emotions, with inner monologue .. but imho there is something very similar to LLM as a part of our brain.

I also think it is ok to say LLM "knows" something, or even "understands" something.
I even think it can be to some extent self aware .. I mean it can have a concept of itself. It can talk about itself, even if it's just what it was thought. Doesn't sound like much, but I suggest this example from testing of Bing chat here (the whole video is great, but see the time code):

https://youtu.be/jHwHPyWkShk?si=sXB-e4nRaBFryqFu&t=869

You can see here like Bing with his (commanded) super confidence blabs some nonsense .. but then realizes it is nonsense by seeing its own response. And then it comes to conclusion (based on large power to analyze text) that him, the author of the response, is broken. And it seems to be sincerely unhappy about it.

Sure, we can agree that it's pretty far from what is usually understood as "self awareness" .. but imho it's fascinating how something which just analyzes text can go mad.

 

[steenkh]

No, they cannot do either of these things. You are mistaking the appearance of fluency for the appearance of reasoning.

Do you have a solid definition of reasoning?

 

[steenkh]

We know exactly how LLMs produce strings of text. They're statistical prediction algorithms. Nothing we know about human cognition suggests that it is in any way similar to what LLMs do.

You are presupposing a definition of of cognition that says that cognition have to be how humans are doing it.

But sure, I suppose that maybe, deep down, humans are just doing rote prediction based on mathematical analysis of (large) bodies of data. In a woo, "aha! you can't rule it out!" sense, at least.

Actually, I wouldn’t be so sure that humans are not doing exactly that - in some form. Except for some things that may be hardwired, humans learn from other humans. We see stuff, and repeat it ourselves with embellishments. Very little is reasoned from the bottom up.

The first idea you have to jettison - with extreme prejudice - is the idea that Large Language Models are making mysterious inductive leaps, that aren't fully explained by their programming or fully understood by their programmers.

Some time this spring we had a quote in this thread from researchers in the field who claimed exactly that: their LLM had formed a kind of memory to understand a Go board, which should not have been possible with the algorithms built into it. You may claim that the researchers are into woo, but there is also the possibility that they know what they are talking about.

 

[Arcade22]

The style and manner of AI writing generally seems a lot like an average high school essay, overexplanatory and with the distinct impression that the writer doesn't really care about what they are writing but has been forced to fill a few paragraphs.

With regards to ChatGPT and similar generative language models this is a result of them explicitly, if not publicly, being directed to produce such text.

A typical "session" of these types of systems start with what you can call a hidden initial prompt, which essentially amounts to a mission statement. The fact that it sounds the way it does is because it associates the descriptions of itself with such mediocre, boring, uncontroversial, "nice" and "informative" text. To the end user there's nothing that ever suggests such a "hidden initial prompt" exists at all.

There are plenty examples of GPT style language models producing widely different styles of writing if these descriptions of itself are changed or if they are prompted to do so.

 

[Dr.Sid]

Indeed. But assistants like ChatGPT or Bing or Bard are also trained to act like assistants. And there might be other limitations, so they only be used as assistants .. like prompt filters, or limit on response count. So those freely available assistants might be hard to have their "face" changed.

 

[Chanakya]

I've just posted this in another thread thought I'd drop it here as well:

At the moment I'm quite confident that anyone here using substantial amounts of unedited AI generated text will be spotted simply because it will either be a sudden change in their style of posts or from the style of the text itself - so far the various AI generators have quite a distinctive style. I expect that will change over time.

Bing Chat (uses ChatGPT4) offers a "style" choice for its responses but even then there seems to be a distinct "AI style". I know it could be nothing more than my cognitive bias (probably confirmation bias) but it does seem to me that at the moment there is a distinct style to the various text AI systems that are meant to use regular English. (Seen a few that are meant to mimic styles such as say rap or something.)

Heh, those AI cover things come to mind! You can get AI --- or "AI", as has been suggested (although to be honest I don't really understand the difference, but I'll leave that separate question for another day) --- to sing random songs in the specific style and pitch and tone et cetera of some specific singer, right? Would it be so very difficult, then, to get a chatbot to produce text that roughly follows a sample of your text that you've presented to it?

(Not a rhetorical question, I'm wondering, asking, if that's so. When you have AI output songs in the style of some particular singer, then what kind of input goes in? Is that so very involved --- and/or is the process for text so very different than for songs --- that a casual poster will probably not take the huge amount of trouble that necessitates, just to post on here? Or might customizing chatbot output to appoximate with one's personal style of writing be not an overly complicated thing to someone willing to take a bit, not too much, of time and effort?)

 

[Chanakya]

I - and Dr Martell - are talking about Large Language Models, which are (a) not AI, and (b) doing things we understand very well in coming up with the answers to what we ask them.

What AIs do you have in mind, that are doing this other thing?

What's the difference, then?

(Not an argument, not rhetorical! Asking, because I actually don't know this. What present-day applications would qualify as actual bona fide AI, in that case, and what sets those apart from these LLM chatbots? Is it simply the amount of complexity that goes into it, or something more fundamental than that?)


eta:

...It's literally the same mechanism that powers text autocomplete bots: A statistical estimate of the most likely next word, based on the preceding context.

This is why LLMs are prone to hallucination. Because they're not actually reasoning. They're just mindlessly extending a string of text based on statistical estimates...

I guess this would be the difference. And also, I see (from your posts subsequent to the one I'd quoted) that you don't think real AI exists at all --- is that right, or do I misread that?

But reasoning is simply If-this-and-not-that-then-such-and-such, isn't it. Keep it very simple, and you get a calculator. Make it very, extremely, exponentially more complex, and you'll get "intelligence". What else is there to intelligence, after all?

I think why AI is prone to "hallucinations" is because its inputs are unidimensional, only words not actual/other sensations. As far as I can think this through.

Would "real" AI necessarily have to involve many kinds of inputs, like touch and smell and the rest of it, and simply greater complexity of one kind of input won't suffice, is that what you're implying? (So that whether it "hallucinates" in ways that a human would never do, would itself be a test of whether it is actual AI?)

 

[novaphile]

Let's say I am. About people in general.

What's the evidence for the counter claim?

This is something that often intrigues me...

It appears that 'reasoning' is not possible, or at the very least severely limited, without language. (If you want to test this, try thinking through a problem without thinking any words or symbols).

Therefore it may follow, that the less language a person can use and understand the less capable they may be of reasoning.

Part of my thinking on this, stems from people, who are fluent in multiple languages, telling me that some languages appear to be better for some classes of thinking. (These are people who can 'code switch' and think in different languages).

Reasoning being impaired by lack of language skills could explain some of the things I've personally seen, where a person seemed incapable of planning or making decisions in ways that would give them good outcomes.

It is even possible that some people exist in a p-zombie state, where they can only respond to stimulus, without any concept of self or reasoning at all...

https://en.wikipedia.org/wiki/Philosophical_zombie

Case studies about children that have been raised with no human contact (Hence no language skills of any kind) are particularly chilling.

 

[Fast Eddie B]

It appears that 'reasoning' is not possible, or at the very least severely limited, without language. (If you want to test this, try thinking through a problem without thinking any words or symbols).

I’ll go with “limited”.

Imagine Homo Sapiens or earlier hominids before language or symbolism. My guess is they’d still have a “stream of thought”, using mental imagery separate from language or symbolism. “Bright yellow thing at that spot in the sky means I better head back to camp now to get there before dark”. Reasoning done without words or symbols. And animals appear to be able to reason through problems without language or symbolism. But stipulated that language and symbolism help a whole lot!

 

[Emily's Cat]

I am not claiming LLMs are intelligent, but I wonder how it can be claimed that they are not intelligent, in view of the fact that no one has a good definition of intelligence.

I would argue that within this context, we do have a reasonably good definition of intelligence. A being is considered intelligent if it is both sapient and sentient. LLMs are neither of those, although they can sometimes appear to mimic them.

 

[Emily's Cat]

I'd like to hear someone explain why humans are smarter than an LLM.

Most people just slap words together. Most people you meet are just LLMing through life.

Can anyone prove humans have something LLM's don't, intelligence wise?

The fact that you can ask that question demonstrates that humans have something LLMs don't.

A huge amount of our adult lives are routine - more than we tend to think. We don't engage in as much active problem solving as we think we do - we think through a question once (what shoes would look best with this suit), determine a guideline set that works (cordovan flats with navy, brown with browns, most greens, muted tones, black with all others) and we just repeat that formula until something changes.

But we still use a lot more active problem solving than you seem to think we do Some of us use a LOT of it, as it's a fundamental part of our careers. And often, even within a set of guidelines, we make active decisions, we employ reasoning, we extrapolate.

You might very well have someone following a recipe step by step to make dinner tonight... but even then, there's likely a part of their brain saying "This calls for chili powder, but I don't like chili powder - can I leave it out entirely or should I substitute paprika?" Or perhaps "They recommend broccoli as a side dish, but I don't have that, I think I'll use green beans instead"

We also extrapolate, we project a current or past situation into potential future outcomes. We replay past encounters and think about what we could have done differently, and how much the result would have varied. "If only I'd bit my tongue when I was talking to my boss today!" or "I handled that badly, I could have done these 5 things instead and I wouldn't be in the mess I am right now".

Extrapolative thinking is the core of invention, of scientific advance, of planning for the future. Some people aren't particularly accurate at it, but pretty much all humans do it. Some animals - corvids, apes - also demonstrate extrapolative thinking. Almost any creature that uses tools, especially custom built tools fit to a specific task, demonstrate extrapolative thinking.

I don't think LLMs do.

 

[Mike Helland]

I would argue that within this context, we do have a reasonably good definition of intelligence. A being is considered intelligent if it is both sapient and sentient.

By whom?

Are people "sapient" in general?

sapient
sā′pē-ənt
adjective

Having great wisdom and discernment.Wise; sage; discerning; -- often in irony or contempt.Possessing wisdom and discernment; wise, learned.

Most people aren't sages.

 

[Emily's Cat]

I’ll go with “limited”.

Imagine Homo Sapiens or earlier hominids before language or symbolism. My guess is they’d still have a “stream of thought”, using mental imagery separate from language or symbolism. “Bright yellow thing at that spot in the sky means I better head back to camp now to get there before dark”. Reasoning done without words or symbols. And animals appear to be able to reason through problems without language or symbolism. But stipulated that language and symbolism help a whole lot!

I would amend that to say without sophisticated language or abstract symbolism.

Many animals communicate with each other, it's just not very complex communication. They engage in a variety of signaling.

Some animals have more complicated signaling apparatus, and seem to communicate more complex ideas. Chameleons don't just use their color changing skills to blend in with their surroundings as camouflage - they also use them to communicate aggression or passivity to other chameleons. Squid and octopi take it a step further, their color and texture changing capacity appears to convey a lot more information to each other, in more complex ways. And octopi in particular have demonstrated some fairly impressive problem-solving capacity.

Cats have more variety of vocalization, and more variety of facial expression than dogs. One of the ways this ends up being important is that cats adapt their means of communication to their human roommates, in a way that dogs don't. We have a relatively good understanding of canine communication, posturing, etc., and it holds fairly steady across all breeds including wild dogs and wolves. Each individual dog will have its own personality, its own behavioral tendencies, but in general dogs all share a common "language". Cats are different. Feral cat colonies have very little vocalization once they're beyond kitten stage, but they have a lot of posture and facial expression based communication with each other. Some of it is fairly universal, the big fight/flight responses, aggression/passivity postures, these tend to be universal. But within any colony of feral cats, they'll develop their own unique set of expressions and postures. With humans, it's even more extreme. Pet cats will develop vocalizations that are unique to their specific humans, and will use them to indicate different wants or needs. Domestic cats will even develop different vocalizations for different specific humans.

I think the problem-extrapolation-solution mechanism is a fundamental element of intelligent, and that to the best of our knowledge, this extrapolative thinking is also linked with both perceptive capacity and the complexity of communication.

 

[Emily's Cat]

By whom?

Are people "sapient" in general?



Most people aren't sages.

Compared to all the other species on our planet, yes, humans are sages.