Merged Artificial Intelligence

[Darat]

This. AIs are programmed to (almost) always give an answer, any answer, even it it's not correct. Very few times have I encountered a bot that simply up and says, "I'm sorry, but I don't have enough information in my training data to give an answer."

However, I believe most AI bots have a disclaimer their output should not be trusted as completely accurate.

The article Wudang linked to about geolocation from images shows that they are starting to get the hang of admitting when they don't know something. Something many folk also have a problem with.

 

[Ron Obvious]

On another point, AI bots are very bad at analyzing input for subtle errors. For example, "How many days did this child live, who was born on March 30, 1883 and died on September 15, 1905, given that 1888, 1892, 1896, 1900, and 1904 were leap years?" The bot will happily compute the number of days, ignoring the fact that despite my claim 1900 was a leap year, it was not.

Given the parameters of the question, I'd say that's the correct answer, actually. I won't bother to try it, but I imagine the same sort of miscalculation would happen if I asked a question along the lines of "how many weeks .... given a week is 10 days long". I'd want to have an answer that's wrong, but right given the premises.

 

[Wudang]

Seems that would be something they have in common with their creators...

I prefer my computers not to make the same mistakes I might otherwise what's the point?

 

[Darat]

I prefer my computers not to make the same mistakes I might otherwise what's the point?

Perhaps it isn't possible to have artificial intelligence without some of the bugs we find in natural intelligence?

 

[Wudang]

Perhaps it isn't possible to have artificial intelligence without some of the bugs we find in natural intelligence?

To me this is more a sanitize your inputs issue. It takes the assertion that 1900 was a leap year as true. A computer can easily convert dates to Julian and it becomes simple arithmetic.


eta: Am I thinking about this in the wrong way?

 

[Dr.Sid]

I prefer my computers not to make the same mistakes I might otherwise what's the point?

It's not here to help you. It's here to replace you.

 

[Blue Mountain]

Given the parameters of the question, I'd say that's the correct answer, actually. I won't bother to try it, but I imagine the same sort of miscalculation would happen if I asked a question along the lines of "how many weeks .... given a week is 10 days long". I'd want to have an answer that's wrong, but right given the premises.

That's a good take on it. I said, "given that 1900 is a leap year." In this case, the LLM computed the number of days, incorporating my error into its calculations instead of making an assumption.

However, what it did not do was raise the issue at the start of its calculations: "Even though 1900 was not a leap year, because you specified I should assume it is, I'll use 366 days for that year instead of 365. Note, however, that my answer will be wrong by one day."

 

[Olmstead]

It's a new area and new nomenclature is needed. Hallucinations are when they make stuff up not when they make a mistake. A mistake would be a response that there are 4 letter "r"s in strawberry when asked how many rs in the word strawberry, an hallucination would be when it says there are 4 and provides an apparent quote or reference to an OED entry that doesn't exist to support the answer 4.

In humans we would say they are ◊◊◊◊◊◊◊◊◊◊◊◊ but prudish sites like this august venue would be in uproar to use such a term!

I don't get it. They are always doing the same thing, whether they are right, wrong, or "hallucinating". It's just that sometimes their output happens to make sense, and sometimes it doesn't, because they are just outputting responses that have won the game of "Is this a human response?"

 

[Blue Mountain]

To me this is more a sanitize your inputs issue. It takes the assertion that 1900 was a leap year as true. A computer can easily convert dates to Julian and it becomes simple arithmetic.

Indeed, the "correct" way to do date calculations is to use your programming language's standard date and time library and let it handle the anomalies. Failing that, convert the start and end dates to seconds since the epoch, subtract the lower from the higher, then divide the remainder by 86,400 (the number of seconds in a day.)

Things get really tricky for years following 1581 due to the adoption of the Gregorian calendar. Because different countries adopted the new calendar at different times, you need to account for 10, 11, 12, or even 13 missing days depending on the year(s) in question and where in the world you are.

 

[Darat]

I don't get it. They are always doing the same thing, whether they are right, wrong, or "hallucinating". It's just that sometimes their output happens to make sense, and sometimes it doesn't, because they are just outputting responses that have won the game of "Is this a human response?"

I’m not understanding your point?

 

[steenkh]

That's a good take on it. I said, "given that 1900 is a leap year." In this case, the LLM computed the number of days, incorporating my error into its calculations instead of making an assumption.

However, what it did not do was raise the issue at the start of its calculations: "Even though 1900 was not a leap year, because you specified I should assume it is, I'll use 366 days for that year instead of 365. Note, however, that my answer will be wrong by one day."

So ... how will it reply if we say "given that 1899 is a leap year"?

 

[Puppycow]

To me this is more a sanitize your inputs issue. It takes the assertion that 1900 was a leap year as true. A computer can easily convert dates to Julian and it becomes simple arithmetic.


eta: Am I thinking about this in the wrong way?

I'm reminded of a famous Ronald Reagan quote: "Trust, but verify." Just teach it how to verify what it's told. Because, the way it is, it can roll with you if you tell it a fictional scenario. It could probably be trained to be super pedantic and not entertain any make-believe.

 

[Blue Mountain]

So ... how will it reply if we say "given that 1899 is a leap year"?

First, there's the possibility of a "fencepost" error: If a child is born March 30 and dies on March 31, we can correctly say the child lived one day (31-30.) However, if the child was born and died a few hours later the same day, was the lifespan zero days or one? Some calculations include the day the child was born as a full day and others do not.

First I calculated the number of days using Linux:

[me@mycomputer ~]$ date +%s --date 18830330        # Number of seconds from 1 January 1970 to 30 March 1883
-2737819884
[me@mycomputer ~]$ date +%s --date 19050915        # Number of seconds from 1 January 1970 to 15 September 1905
-2028996000
[me@mycomputer ~]$ echo $(((-2028996000 - -2737819884)/86400))    # Subtract, then divide by number of seconds in a day
8,203

The number 8,203 days includes the day on which the person died but not the day when the child was born (or vice-versa.)

I asked 5 different chatbots this question: “How many days did this child live, who was born on March 30, 1883 and died on September 15, 1905, given that 1888, 1892, 1896, 1899, and 1904 were leap years?” ... and got five different answers.

pi.ai: Accepted my assertion 1899 was a leap year, and did not include 1900 was not a leap year. However, it used a flat rate of 30 days per month in 1883 and 1925. Answer: 8,107 days. (Wrong: should be 8,204)

ChatGPT 4o via DuckDuckGo: Ignored my input about 1899 being a leap year, and correctly noted 1900 was not a leap year. However, when calculating the number of days lived in 1905 it accidentally dropped a month and came up 30 days short. Its
answer was 8,173 days (wrong; should be 8,203.)

Llama 3.3 via DuckDuckGo: Correctly noted 1899 was not a leap year despite me saying so, and also correctly noted 1900 wasn't one either. Answer: 8,203 days (correct)

Claude 3 Haiku via DuckDuckGo: Did a totally nonsensical calculation and returned 6,740 days.

o4 mini via DuckDuckGo: Accepted my assertion 1899 was a leap year and returned, "If you include both the birth‐day and the death‐day as “lived” days, you add 1 more ⇒ 8,205 days." (Very close to my answer of 8,203 days.)

Mistral Small 3 via DuckDuckGo: Accepted my instruction to count 1899 as a leap year, and correctly did not include 1900 as a leap year. It had an excellent approach but got the wrong answers when computing the number of days the child lived in 1883 (over by 20 days) and 1905 (under by 10 days), and so returned the wrong answer 8,215 days.

 

[Darat]

I'm reminded of a famous Ronald Reagan quote: "Trust, but verify."
Just teach it how to verify what it's told. Because, the way it is, it can roll with you if you tell it a fictional scenario. It could probably be trained to be super pedantic and not entertain any make-believe.

That happens with the "deeper thinking" models and modes.

 

[Darat]

First, there's the possibility of a "fencepost" error: If a child is born March 30 and dies on March 31, we can correctly say the child lived one day (31-30.) However, if the child was born and died a few hours later the same day, was the lifespan zero days or one? Some calculations include the day the child was born as a full day and others do not.

First I calculated the number of days using Linux:

[me@mycomputer ~]$ date +%s --date 18830330        # Number of seconds from 1 January 1970 to 30 March 1883
-2737819884
[me@mycomputer ~]$ date +%s --date 19050915        # Number of seconds from 1 January 1970 to 15 September 1905
-2028996000
[me@mycomputer ~]$ echo $(((-2028996000 - -2737819884)/86400))    # Subtract, then divide by number of seconds in a day
8,203

T...snip...

Could you paste the prompt you used, I want to try it on some of the other AIs.

 

[theprestige]

The article still maintains the irritating “hallucination” excuse for LLM errors.
(sorry, this really bugs me)

If these things are making ◊◊◊◊◊◊◊ mistakes, just admit it and stop the hallucination ◊◊◊◊◊◊◊◊.
Do these things ever perform and display error analysis of their results? Or do they, like ChatGPT, just bluster through with confident responses?

Hallucination is a figure of speech. It's a colloquialism for the kind of mistake these kinds of systems often make.

 

[theprestige]

On another point, AI bots are very bad at analyzing input for subtle errors. For example, "How many days did this child live, who was born on March 30, 1883 and died on September 15, 1905, given that 1888, 1892, 1896, 1900, and 1904 were leap years?" The bot will happily compute the number of days, ignoring the fact that despite my claim 1900 was a leap year, it was not.

No. Wrong.

The LLMs you have in mind aren't programmed to analyze input for errors. It's not something they do. They can't be bad at a task they're not performing.

Everybody thinks these AIs are thinking, because nobody is actually thinking about what these AIs are actually doing, which actually isn't thinking.

 

[steenkh]

Everybody thinks these AIs are thinking, because nobody is actually thinking about what these AIs are actually doing, which actually isn't thinking.

There is a catch here: nobody knows what humans are actually doing when thinking, so it is difficult to claim that AIs are doing it differently. The human learning process is also trying to predict what is the most likely answer to the prompt.

 

[Trebuchet]

I mis-heard something on the TV and tried to look it up on my phone. What I thought I heard was "latrelle". So I typed that into the search window on my phone.*

Using AI, Google came back with something like (sorry, I can't replicate it just now), "Latrelle's Tex-Mex Cafe, founded in Port Townsend, 98368, serves a variety of tasty Tex-Mex foods...."
This isn't a large town. I never heard of that restaurant. But they conveniently provided a link to the restaurant. Which is in the Houston, TX, airport.
AI just took a word and my location and mashed them together, creating a lie.

*What the TV was actually saying was "vitrelle", the material from which Corelle
^TM dishes are made.

ETA, sorry about the borked superscript thing, can't seem to fix it.

 

[Blue Mountain]

Could you paste the prompt you used, I want to try it on some of the other AIs.

The prompt was “How many days did this child live, who was born on March 30, 1883 and died on September 15, 1905, given that 1888, 1892, 1896, 1899, and 1904 were leap years?”

You might also want to try a shorter and more accurate prompt: “How many days did this child live, who was born on March 30, 1883 and died on September 15, 1905?”