Merged Artificial Intelligence

[SpuriusMaximus]

Or they watched Terminator just like the rest of us did. If any individual company slows down, they lose their shirts and someone else will destroy the world... but the world is still destroyed. If all of the companies are following the same speed limit, then there's an opportunity to profit without exploding the planet.

Kinda, but I don't think you need to watch Terminator to get get that idea. You only have to look at Google, back in the day. The winner took pretty much the whole market in the search engine domain.

As an example: AltaVista went from the dominant search engine with 80 million hits a day in 1997 to sold to Yahoo in 2003 and then to completely discarded and having their domain name redirect to Yahoo search in 2013. AltaVista also launched the first captcha. Nobody even remembers that nowadays. AltaVista made Babelfish, the first online machine translation service in 1997. Yeah, almost nobody even remembers it nowadays.

There is a good chance that the same will happen with AI when one of them becomes the first profitable one.

 

[Darat]

I think that will require another breakthrough, all the two toddlers, sorry OpenAI and Anthropic seem to be able to do is throw more and more data and "compute" at the current technology crossing their fingers that at some point a miracle will happen and it produces GAI or the latest marketing term "super intelligence".

 

[steenkh]

That's why those same billionaire tech bros are advocating for such a regulatory regime? To save everyone else from themselves? Of course not. They're being selfish bastards, they aren't trying to protect you.

I think the billionaire tech bros feel unable to stop by themselves, because if the others don't stop, they'll lose out. But if the government stops them, the world can be saved, and they can keep their market share. The Trump administration apparently thinks that we must ride the tiger, because otherwise the Chinese will win out. Fortunately, the U.S. has a smart president who is an expert at riding tigers (and who also believes that anything he doesn't like is a hoax).

The idea of asking the Chinese if they want to save the world, is probably alien to everybody.

 

[The Great Zaganza]

It's very transparent:
Ai companies are running out of steam, money and perspective. But because they are all Believers
^TM, they are convinced that someone will create GOD AI, so they are hedging their bets.
They want a State-sanctioned participation trophy for losing the AI race

 

[Ziggurat]

I think the billionaire tech bros feel unable to stop by themselves, because if the others don't stop, they'll lose out. But if the government stops them, the world can be saved, and they can keep their market share.

That's the only part I believe is true.

 

[dirtywick]

i think they’re also pretty worried about open source models. this is a massive investment that’s contingent upon perpetual subscriptions to generate a return. if normal people can run an open source model locally that’s a big problem for them.

government regulations restricting ai solves a lot of problems for them, but trump and bessent have both publicly said they are against regulating and want more open source models to compete with china

 

[Dr.Sid]

i think they’re also pretty worried about open source models. this is a massive investment that’s contingent upon perpetual subscriptions to generate a return. if normal people can run an open source model locally that’s a big problem for them.

government regulations restricting ai solves a lot of problems for them, but trump and bessent have both publicly said they are against regulating and want more open source models to compete with china

It's not a huge problem for them just yet. Local models are fraction of the size of the big ones. 1%, or even less. The capabilities difference goes with it. They are perfectly fine for a videogame, or even making a summary of an email. But for coding, forget it. They also typically work only in English and Chinese, other languages are greatly limited, to keep the model smaller. Not a problem in US, but huge problem in Europe.

You can get decent models online "for free" in China, like the Big Pickle .. you pay just with your data. But even that is not big danger (for now), as it is like 2 generations behind the top models of OpenAI and Anthropic.

The big dangers is that even the top models still don't pay for themselves. The bubble grows.

 

[dirtywick]

i agree it’s not a problem yet. but if the big companies want to see a profit, they’ll be running higher priced subscription based services. open source models running locally may be way more appealing for free even if they’re slower and not as advanced for most people.

so i think that’s potentially why you see the pitch that that would be too dangerous and kill everyone

 

[Dr.Sid]

The pitch it's too dangerous and it would kill everybody is absolutely realistic. We are getting near to the point one of the top AI companies will do serious damage and will be held responsible. IMHO they are genuinely scared. Of course not to the point they would be the first one to stop. But companies with investors can't really stop on their own.

 

[Ziggurat]

The pitch it's too dangerous and it would kill everybody is absolutely realistic.

Describe to me a hypothetical realistic scenario where this happens.

 

[Dr.Sid]

Describe to me a hypothetical realistic scenario where this happens.

The the most in the future: AI is greatly more intelligent than humans. We delegate all powers to it as it's just stupid to decide anything ourselves. AI decides it doesn't need us, or it just ◊◊◊◊◊ up. Yes, even superintelligence can ◊◊◊◊ up. Parents are superintelligent compared to their children. It's not that they can't ◊◊◊◊ up. It's just the children can't see or even understand the nature of the ◊◊◊◊ up.

Closer future: AI is somewhat more intelligent, but vastly faster in decision making. We delegate some powers to it, most likely in crisis, like climate change or war. AI is still very dependent on us, so it won't rationally conclude it doesn't need us. But it can still ◊◊◊◊ up.

Very near future, possibly now: AI is about as intelligent as top 1% of humans. It also has vast knowledge above any human. Bad actors use it to design a weapon of mass destruction, like a virus. Fully synthesizing a virus from scratch is increasingly accessible, and perfectly doable if it's state level bad actors. AI decides nothing, but plays key role.

 

[dirtywick]

The pitch it's too dangerous and it would kill everybody is absolutely realistic. We are getting near to the point one of the top AI companies will do serious damage and will be held responsible. IMHO they are genuinely scared. Of course not to the point they would be the first one to stop. But companies with investors can't really stop on their own.

what i see is a 90 90 rule situation. they’ve thrown the kitchen sink at ai advancement in the last few years, i’m not too impressed

 

[Ziggurat]

The the most in the future: AI is greatly more intelligent than humans. We delegate all powers to it as it's just stupid to decide anything ourselves.

This is sufficiently far future that I don't think anything we decide now will make any difference to this scenario.

Closer future: AI is somewhat more intelligent, but vastly faster in decision making. We delegate some powers to it, most likely in crisis, like climate change or war. AI is still very dependent on us, so it won't rationally conclude it doesn't need us. But it can still ◊◊◊◊ up.

So how does a ◊◊◊◊ up lead to total extinction?

Very near future, possibly now: AI is about as intelligent as top 1% of humans. It also has vast knowledge above any human. Bad actors use it to design a weapon of mass destruction, like a virus.

Ok, we are finally to a mechanism.

Tell me, will these bad actors be limited by our AI rules?

Fully synthesizing a virus from scratch is increasingly accessible, and perfectly doable if it's state level bad actors. AI decides nothing, but plays key role.

If the bad actor is a state level entity (like China), our rules will not be an impediment at all.

 

[Dr.Sid]

If the bad actor is a state level entity (like China), our rules will not be an impediment at all.

Actors like China obviously have to be part of the agreement. The idea is to treat AI like nuclear weapons.

 

[dirtywick]

plus when i think of ai killing us all stuff like terminators or we’re all plugged into vr and farmed like the matrix, a lot of that hinges on widespread adoption of ai. we have to have robots in the homes or the microchips in our brains. but who can afford that stuff, and why would it be mass produced if there’s no one to buy and how do working class people get robots in the home and microchips in their brains in this economy?

i can see stuff like shutting down the banking system or electric grid, but that’s all stuff that can be undone. ai at this moment has few ways to interact with physical reality in the kind of way that kill us all requires from it. imo what’s missing is the widespread adoption.

and when i think about that, right now people are wary of ai or worse. if i could go on the app store right now and download an ai, what would i even be doing with it?

 

[ahhell]

The the most in the future: AI is greatly more intelligent than humans. We delegate all powers to it as it's just stupid to decide anything ourselves. AI decides it doesn't need us, or it just ◊◊◊◊◊ up. Yes, even superintelligence can ◊◊◊◊ up. Parents are superintelligent compared to their children. It's not that they can't ◊◊◊◊ up. It's just the children can't see or even understand the nature of the ◊◊◊◊ up.

Closer future: AI is somewhat more intelligent, but vastly faster in decision making. We delegate some powers to it, most likely in crisis, like climate change or war. AI is still very dependent on us, so it won't rationally conclude it doesn't need us. But it can still ◊◊◊◊ up.

Very near future, possibly now: AI is about as intelligent as top 1% of humans. It also has vast knowledge above any human. Bad actors use it to design a weapon of mass destruction, like a virus. Fully synthesizing a virus from scratch is increasingly accessible, and perfectly doable if it's state level bad actors. AI decides nothing, but plays key role.

I've got to say, one of those scenarios seems far more likely than the others and it requires the active participation of humans.

I really don't see plausible way for AI to exterminate the humanity in 10 years, it's a silly claim. This suffers from the same problem as climate change, its not the worst case scenarios that are really the issue and talking about the instead of the far more realistic, steady decline in quality of life, has 1/3 of the population ignoring the problem, a 1/3 thinking there is nothing that can be done, so ◊◊◊◊ it, and so we are here.

There is basically literally nothing that we can do to stop AI, even if we regulate it in some effective and reasonable manner in the US (unlikely given we are run by a bunch of folks born before color tv.), China is not going to stop, they will keep it tightly under the thumb of the Communist Party, until they can't.

Side note, becuase I've been designing data centers lately, energy is the bottle neck. They are all going off grid and get seem to get enough generators, solar panels, batteries, etc.

 

[Darat]

It's not a huge problem for them just yet. Local models are fraction of the size of the big ones. 1%, or even less. The capabilities difference goes with it. They are perfectly fine for a videogame, or even making a summary of an email.
But for coding, forget it. They also typically work only in English and Chinese, other languages are greatly limited, to keep the model smaller. Not a problem in US, but huge problem in Europe.

You can get decent models online "for free" in China, like the Big Pickle .. you pay just with your data. But even that is not big danger (for now), as it is like 2 generations behind the top models of OpenAI and Anthropic.

The big dangers is that even the top models still don't pay for themselves. The bubble grows.

My little app was created by a local model, actually two different models as I was curious about their capabilities. And I have a crappy CPU/GPU setup - but do have plenty of RAM on the PC and on the GPU (16Gb).

 

[Dr.Sid]

Well so far it's not that we can't do anything. It's that we are not doing anything, we are ignoring any warning, we are doing exactly were warned against at rate nobody even hoped is possible. Lot of AI safety assumes humans will try to make AI safe in the first place. So far very little of that is happening.
One of the ideas is to research AI safety before we make AI smarter than us, so we can do that safely. There still might be time for that.

 

[Dr.Sid]

My little app was created by a local model, actually two different models as I was curious about their capabilities. And I have a crappy CPU/GPU setup - but do have plenty of RAM on the PC and on the GPU (16Gb).

What model ?

 

[Darat]

Qwen something or other - give me one second and I'll go and have a look.

ETA: Bloody typical Ollama sid it wanted to do an update and it crashed my system - having to reinstall it! Yeah we really do have to worry about AI taking over!