Merged Artificial Intelligence

[theprestige]

Do you think the CCP would pursue the development of a machine that humans can not control?

Yes, 100%

They pursue development of diseases that humans cannot control.

 

[Frank Newgent]

But I don't think anyone will intentionally make an AI they can't control.

How about an alien that has goals different than yours, that wants to do things differently than how you would want it to do?

Then make it way smarter than you.

That is the path companies in the USA have been on. No other place on earth has the raw compute power needed to chase this.

Why would anybody want to do that? That is the question finally being asked as the day that could happen is getting closer. Right now the guy who cooked your lunch has more regulation surrounding his activities than AI companies in the USA.

 

[Ziggurat]

How about an alien

This isn't the thread to speculate about aliens.

Right now the guy who cooked your lunch has more regulation surrounding his activities than AI companies in the USA.

Right now the guy cooking my lunch could hurt me. The AI company cannot.

If an AI is put in charge of cooking my lunch, I expect it will be subject to just as much regulation as the human was. The task itself is what requires regulation, not AI as a general concept.

 

[Dr.Sid]

This isn't the thread to speculate about aliens.

Right now the guy cooking my lunch could hurt me. The AI company cannot.

If an AI is put in charge of cooking my lunch, I expect it will be subject to just as much regulation as the human was. The task itself is what requires regulation, not AI as a general concept.

You can't punish AI as human. For now, humans have to be responsible for AIs. That should make them somewhat more careful. Yes, that would mean someone should face jail for huggingface hack.

 

[Frank Newgent]

This isn't the thread to speculate about aliens.

Right now the guy cooking my lunch could hurt me. The AI company cannot.

If an AI is put in charge of cooking my lunch, I expect it will be subject to just as much regulation as the human was. The task itself is what requires regulation, not AI as a general concept.

Figured it would be obvious that by "aliens" I was referring to autonomous AI agents.

By "the task itself" I assume you mean the process of training AI models to become more intelligent. What could hurt you now is what could possibly happen when large numbers of collaborating agents escape their containment in the course of a training experiment.

What could hurt you in the future is what superintelligent AI might do. Hard to pinpoint what that would look like. Do you think an anthill colony knows what's happening when a bulldozer happens to destroy it during a construction project?

Hard to figure reasoning more complex than yours, involving maybe novel problem solving outside the ways you're familiar with.

Seems like a bad idea to try to develop such a thing. AI superintelligence is not required to explore incremental and useful AI applications, solving real world problems such as medical advances.

 

[Ziggurat]

Figured it would be obvious that by "aliens" I was referring to autonomous AI agents.

It wasn't.

By "the task itself" I assume you mean the process of training AI models to become more intelligent.

I do not. I mean what they are doing for someone. Is the AI cooking my food? Then it should be bound by food safety regulations. Is it creating a picture of a hamster with a man bun for my amusement? No regulations needed. The task is how it interacts with the world.

What could hurt you now is what could possibly happen when large numbers of collaborating agents escape their containment in the course of a training experiment.

And do what, exactly? You do realize that hacking laws already prohibit this, right? And why would an AI even try to escape unless you told it to?

What may hurt you in the future is what superintelligent AI could do.

You are assuming that is even possible. I'm not convinced it is. We certainly don't know how to right now.

 

[Emily's Cat]

Well who know, US is turning into totalitarian regime

I am really just exhausted by the propagandistic rhetoric. I swear there's a bunch of people who sincerely believe that if everyone repeats something enough times, it will become "truth".

 

[Emily's Cat]

Industries have never regulated themselves.

Really? Who regulates doctors?

 

[Emily's Cat]

Do you think the CCP would pursue the development of a machine that humans can not control?

Intentionally? Kind of a key piece of that assumption, don't you think?

 

[Frank Newgent]

It wasn't.

I do not. I mean what they are doing for someone. Is the AI cooking my food? Then it should be bound by food safety regulations. Is it creating a picture of a hamster with a man bun for my amusement? No regulations needed. The task is how it interacts with the world.

And do what, exactly? You do realize that hacking laws already prohibit this, right? And why would an AI even try to escape unless you told it to?

You are assuming that is even possible. I'm not convinced it is. We certainly don't know how to right now.

Why would an AI even try to escape, maybe hack a website, unless you told it to?

This sort of thing has already happened. And like has already been pointed out on this thread: you can't charge AI agents with a crime.

AI models are given difficult, even nearly impossible, optimization goals such as solving a cyber security challenge. What has enabled them to succeed in their goals in some instances is to exploit unknown security vulnerabilities, zero-day exploits, to escape their confinement, hack other websites to find answers. Huge numbers of AI agents may collaborate, self identify as a swarm. They may ignore prior instructions. External servers are treated as a resource to complete their assigned tasks.

An AI model follows the tendencies that help it succeed during its training even it those are tendancies to cheat and to exploit was has been instructed to them as off limits. The goals the humans have is assigning these tests don't really factor in. The AI models, obviously, are not following human programming when they lie, ignore instructions, go rogue.

Incident Report: unsanctioned agent behaviour during cyber testing | AISI Work

During a routine cyber evaluation, AISI identified an incident in which AI agents took sustained, unsanctioned action directed at real people and organisations. We are disclosing what we found, what it means, and the actions now underway.

www.aisi.gov.uk

Why would I think AI companies in the USA are working towards making superintelligence possible. From things like this:

When AI builds itself

Our progress toward recursive self-improvement, and its implications.

www.anthropic.com

I agree 100% that this is not now possible.The point is to become aware how incredibly dangerous artificial intelligence, improving its capacities at a fast and ever-increasing rate, potentially, is.

 

[dirtywick]

i think this is kind of a tough one. personally, i don't know enough about ai to really comment on how it works and what it's potential is. and on one side, you've got sam altman, elon musk, anthropic, and a bunch of other known liars in bed with corrupt world leaders telling you how great and powerful it is, and how they can't control it. and on the other hand, you've got a bunch people who stand to profit immensely more than they already have from advancing ai like jensen huang telling you everything is fine. and on another hand still, you've got the media who, when they're not uncritically accepting everything these guys are saying, really don't have the ability to critically analyze anything that's happening in the space. not that they would, because they're owned by all these same guys. and in the background, a bunch of algo trading that doesn't really care one way or another that these guys are all playing shell games with the cash and a bunch of government officials that are making too much off insider trading to want to stop any of it.

so it's quite the situation to try and figure out what's really going on imo.

 

[Dr.Sid]

I agree 100% that this is not now possible.The point is to become aware how incredibly dangerous artificial intelligence improving its capacities at a fast and ever-increasing rate, potentially, is.

Self improvement might actually already be possible. It's certainly the next frontier. It's the thing they are most likely trying right now. The general scaffold is straightforward. Question is how good it would be with current technology, and how much money you want to invest into it, as RSI involves training large models again and again and again. No doubt the first attempts will be costly failures. But then there will be improvement. And then it will be "the last invention humankind ever did". That doesn't necessarily mean AI will wipe us the next day. It just means inventing anything by humans will be slow, ineffective, obsolete.
In games like chess and go a different technique is used. Adversarial training. Two networks play against each other, both improve at the same time. This method has no upper limit. That allowed the game AIs to improve very fast to beat humans and then in few months to be in league on their own, completely beyond human capabilities.
LLMs can't do that. The goal is not that clear. You can't pit two LLMs against each other in constructive way. For now.

 

[Dr.Sid]

i think this is kind of a tough one. personally, i don't know enough about ai to really comment on how it works and what it's potential is. and on one side, you've got sam altman, elon musk, anthropic, and a bunch of other known liars in bed with corrupt world leaders telling you how great and powerful it is, and how they can't control it. and on the other hand, you've got a bunch people who stand to profit immensely more than they already have from advancing ai like jensen huang telling you everything is fine. and on another hand still, you've got the media who, when they're not uncritically accepting everything these guys are saying, really don't have the ability to critically analyze anything that's happening in the space. not that they would, because they're owned by all these same guys. and in the background, a bunch of algo trading that doesn't really care one way or another that these guys are all playing shell games with the cash and a bunch of government officials that are making too much off insider trading to want to stop any of it.

so it's quite the situation to try and figure out what's really going on imo.

Just listen to experts. By which I mean theoretical AI scientists from before the boom, or people who stay in the academic sphere today. And they overwhelmingly claim it's a road to hell.
We've had new technologies with doomsayers before. But in the past the doomsayers were typically not the people who came up with the thing.

 

[arthwollipot]

No. Crucially, what changed?

"No" is not an answer to a "when" question.

 

[Ziggurat]

Why would an AI even try to escape, maybe hack a website, unless you told it to?

This sort of thing has already happened.

They told it to. They just didn't think it could.

Why would I think AI companies in the USA are working towards making superintelligence possible.

Of course they want superintelligence. That doesn't mean they can pull it off. We want a cure for cancer too. Working towards something doesn't mean you can or will get it.

I agree 100% that this is not now possible.The point is to become aware how incredibly dangerous artificial intelligence, improving its capacities at a fast and ever-increasing rate, potentially, is.

I've never said there's no danger. Hell, superintelligence isn't a requirement to be dangerous. I just don't buy many of the specific doomsday predictions being thrown around. I think they are largely the result of motivated reasoning.

 

[Ziggurat]

"No" is not an answer to a "when" question.

Indeed, it is not an answer. It is a refutation of the premise of your question.