Merged Artificial Intelligence

[Dr.Sid]

Indeed. If you link a video, at least one sentence of what it is about should be mandatory. And if the video title itself is completely vague a clickbaity .. the one sentence better be good.

 

[arthwollipot]

Indeed. If you link a video, at least one sentence of what it is about should be mandatory. And if the video title itself is completely vague a clickbaity .. the one sentence better be good.

To be fair, @Puppycow did provide one sentence:

It's about how top people in astrophysics are using AI to do science.

But that one sentence was the wrong sentence to pique my interest. It doesn't actually explain anything about the video.

A video that is one hour and fifteen minutes long is pretty much by definition a waste of my time unless I have a reason to invest in it.

Anyway, let's get back to talking about AI, because this is more or less irrelevant.

 

[arthwollipot]

Anyway, let's get back to talking about AI, because this is more or less irrelevant.

This has popped up on social media a couple of times. It is an AI bot, farming for engagement. The post doesn't exist any more - I checked both the subreddit and the user account.

 

[JayUtah]

This has popped up on social media a couple of times. It is an AI bot, farming for engagement. The post doesn't exist any more - I checked both the subreddit and the user account.

This.

Trusting an AI implementation for some purpose means trusting who built it and who trained it and according to what. It means trusting the system prompt. It means trusting the transformation algorithm.

Grok obviously massages Elon Musk's ego. Cheap AIs may (as you point out) be minimal, badly built system farming for engagement for some reason—either to improve their own standing or as a gateway to some other form of engagement, such as scamming you out of your life's savings.

For people whose work is subject to close scrutiny and must meet safety and completeness standards, AI is just not that much of a help unless it has been specifically, reliably, and testably trained on appropriate data. When the plane crashes into the mountain, it won't be okay to say, "But the AI said it would work."

 

[Puppycow]

Indeed. If you link a video, at least one sentence of what it is about should be mandatory. And if the video title itself is completely vague a clickbaity .. the one sentence better be good.

I feel that it's a waste of my time now, even before starting it.


Okay, good for you. That's not a reason for me to watch it.

Fine, I'm not going to write an essay for you about why you should watch it. If you don't want to watch it, then just don't watch it. The loss is entirely yours, not mine. I already wrote two sentences about what's in it, including one in the original post.

 

[Darat]

Creativity and critical thinking are very different things. LLMs fail utterly at the latter.

Seriously just try some out for yourself. AI's getting crammed into everydamnthing. Every browser and search engine has its own and is desperate for someone to pay for it use it.

FTFY!

 

[dirtywick]

https://www.seattletimes.com/business/openais-biggest-challenge-is-turning-its-ai-into-a-cash-machine/

openai faces a lot of challenges in generating revenue, really seems like they'll have no choice but to move to advertising.

https://www.ibtimes.co.uk/openai-faces-14-billion-deficit-infrastructure-bills-spark-severe-bankruptcy-risk-1772518

it's also operating at a loss and eating through it's reserves trying to build out the infrastructure to support itself, despite probably optimistic projecting a huge surge in revenue


so yeah, they all need someone to pay for it.

 

[Wudang]

A good use of AI in science? I follow @[email protected] and here's an article about using AI

OpenAIRE MONITOR provides SciLifeLab and other research organisations with tailored dashboards that track and analyse research outputs, sharing practices, and impact. In parallel, the OpenAIRE CONNECT service equips SciLifeLab with advanced full-text mining capabilities to automatically identify and link datasets and software within publications, improve metadata quality, and support compliance with national and institutional Open Science policies. Both are built upon the OpenAIRE Graph, a global open database linking research outputs such as publications, data, software, and projects.

For more info on OpenAIRE

OpenAIRE Graph - Home

graph.openaire.eu

Open Access publications are processed by AI algorithms and extract information on authors citations of datasets, software, bioentities, methods, diseases, etc. providing context for discovery and assessment

It looks more old school data mining on the face of it and the data model seems more a weighted graph rather than multidimensional vectors but still a cute small baby in a big bath.

 

[Darat]

I'm afraid I'm going to need a whole lot more than that before deciding to commit to watching a video that's over an hour long.

Well you could use an AI to see if it is interesting for you... but the main purpose of this is to see if one of free AIs can do what Chanakya is asking about:

Prompt to Co-pilot: please provide short summary of this video:

Here’s a clear, concise summary of the video “We Need To Talk About AI…” from the Cool Worlds Podcast, based on the retrieved content .

---

Short Summary
The host of the Cool Worlds Podcast shares a personal, reflective solo episode after visiting the Institute for Advanced Study at Princeton. A particular meeting there left him deeply unsettled about the rapid progress of artificial intelligence. He discusses:

• How quickly AI capabilities are advancing, far faster than many scientists expected.
• Why this acceleration raises profound concerns for science, society, and the future of human agency.
• The emotional impact of seeing cutting‑edge AI research up close — enough to compel him to record this special episode.
• A call for open, honest conversation about the risks, responsibilities, and unknowns surrounding advanced AI systems.

The tone is thoughtful, urgent, and introspective rather than sensational.

---

If you’d like, I can also produce:

• A bullet‑point breakdown of the key arguments
• A longer thematic summary
• A critical analysis of his concerns about AI
• A comparison with other public AI warnings

Just tell me what direction you want to explore.

Prompt to Co-pilot: Please provide a critical analysis of his concerns about AI
(Popping the response in a spoiler tag as it is rather long, removed some white spacing.)

Here’s a thoughtful, critical analysis of the concerns raised in the video — not just repeating them, but examining their strengths, weaknesses, assumptions, and blind spots. I’ll treat his argument with respect while also interrogating it, the way a good academic peer review would.

---

Critical Analysis of His Concerns About AI
1. His central worry: AI progress is accelerating faster than expected
He argues that AI capabilities are advancing at a pace that even experts didn’t anticipate, and that this acceleration creates systemic risk.

Strengths

• Historically accurate: Many breakthroughs (GPT‑4, AlphaFold, diffusion models) arrived earlier than predicted by surveys of AI researchers.
• Valid structural concern: When a technology improves exponentially, governance and social adaptation often lag behind.
• Emotionally honest: His unease reflects a growing sentiment among scientists who are not normally alarmist.

Weaknesses / Blind Spots

• Acceleration ≠ catastrophe: Rapid progress alone doesn’t imply existential risk; it could also imply rapid capability to mitigate harms.
• Selective sampling: Visiting a cutting‑edge research institute naturally exposes one to the most extreme frontier work, which can distort perception of the broader field.
• Lack of countervailing evidence: He doesn’t engage with the fact that many AI systems plateau, fail in real‑world settings, or require enormous human scaffolding.

---

2. His emotional reaction as evidence
He frames his personal discomfort as a signal that something is deeply wrong.

Strengths

• Honesty matters: Scientists rarely admit emotional reactions, and doing so can highlight genuine uncertainty.
• Useful heuristic: When highly informed people feel uneasy, it’s worth paying attention.

Weaknesses

• Emotion is not an argument: Fear can be a starting point for inquiry, but not a conclusion.
• Anthropomorphising risk: Feeling “unsettled” by a demo can lead to overestimating agency or autonomy in current systems.

---

3. His implicit assumption: AI is approaching generality
He hints that the systems he saw were closer to general intelligence than the public realises.

Strengths

• Some evidence supports this: Frontier models increasingly show cross‑domain competence.
• Reasonable to flag: If generality is emerging, society should know.

Weaknesses

• No concrete examples: Without specifics, it’s hard to evaluate whether the systems were genuinely general or simply impressive.
• Generalisation ≠ general intelligence: Models can appear broad while still being brittle, narrow, or dependent on human‑curated training.

---

4. His call for open conversation
He argues that society needs to talk more seriously about AI risks.

Strengths

• Absolutely correct: Public discourse is lagging behind technological reality.
• Democratising the conversation: Scientists speaking plainly helps bridge the gap between research and society.

Weaknesses

• Conversation without specificity can fuel anxiety: Raising alarms without concrete examples risks creating a fog of fear rather than clarity.
• Missing the governance dimension: He doesn’t articulate what kinds of oversight, regulation, or institutional structures he believes are necessary.

---

5. His framing of AI as a potential existential threat
He gestures toward the idea that AI could fundamentally alter or undermine human agency.

Strengths

• Legitimate philosophical question: AI does challenge long‑held assumptions about creativity, labour, and cognition.
• Consistent with some expert views: A minority of AI researchers do consider existential risk plausible.

Weaknesses

• Risk conflation: He blends near‑term risks (misinformation, labour disruption) with long‑term speculative risks (superintelligence), which can muddy the analysis.
• Lack of proportionality: He doesn’t weigh AI risks against other global risks (climate, biosecurity, geopolitics), which could contextualise the threat.

---

Overall Assessment
His concerns are sincere, thoughtful, and grounded in real trends, but they are also shaped by proximity to cutting‑edge research, which can amplify the sense of urgency. The strongest part of his argument is the call for open, honest discussion about AI’s trajectory. The weakest part is the lack of specificity: without concrete examples, the audience is left to fill in the gaps with imagination, which can lead to either undue panic or dismissiveness.

He is right that AI deserves serious scrutiny.
He is less convincing when implying that catastrophe is imminent.

---

If you want, I can also provide:

• A point‑by‑point rebuttal
• A sympathetic defence of his position
• A comparison with other AI risk thinkers (Bostrom, Hinton, Yudkowsky, Bengio)
• A version of this analysis written in a more academic or more conversational tone

Just tell me which direction you want to explore next.

 

[Puppycow]

Th

Well you could use an AI to see if it is interesting for you... but the main purpose of this is to see if one of free AIs can do what Chanakya is asking about:

Prompt to Co-pilot: please provide short summary of this video:



Prompt to Co-pilot: Please provide a critical analysis of his concerns about AI
(Popping the response in a spoiler tag as it is rather long, removed some white spacing.)

Here’s a thoughtful, critical analysis of the concerns raised in the video — not just repeating them, but examining their strengths, weaknesses, assumptions, and blind spots. I’ll treat his argument with respect while also interrogating it, the way a good academic peer review would.

---

Critical Analysis of His Concerns About AI
1. His central worry: AI progress is accelerating faster than expected
He argues that AI capabilities are advancing at a pace that even experts didn’t anticipate, and that this acceleration creates systemic risk.

Strengths

• Historically accurate: Many breakthroughs (GPT‑4, AlphaFold, diffusion models) arrived earlier than predicted by surveys of AI researchers.
• Valid structural concern: When a technology improves exponentially, governance and social adaptation often lag behind.
• Emotionally honest: His unease reflects a growing sentiment among scientists who are not normally alarmist.

Weaknesses / Blind Spots

• Acceleration ≠ catastrophe: Rapid progress alone doesn’t imply existential risk; it could also imply rapid capability to mitigate harms.
• Selective sampling: Visiting a cutting‑edge research institute naturally exposes one to the most extreme frontier work, which can distort perception of the broader field.
• Lack of countervailing evidence: He doesn’t engage with the fact that many AI systems plateau, fail in real‑world settings, or require enormous human scaffolding.

---

2. His emotional reaction as evidence
He frames his personal discomfort as a signal that something is deeply wrong.

Strengths

• Honesty matters: Scientists rarely admit emotional reactions, and doing so can highlight genuine uncertainty.
• Useful heuristic: When highly informed people feel uneasy, it’s worth paying attention.

Weaknesses

• Emotion is not an argument: Fear can be a starting point for inquiry, but not a conclusion.
• Anthropomorphising risk: Feeling “unsettled” by a demo can lead to overestimating agency or autonomy in current systems.

---

3. His implicit assumption: AI is approaching generality
He hints that the systems he saw were closer to general intelligence than the public realises.

Strengths

• Some evidence supports this: Frontier models increasingly show cross‑domain competence.
• Reasonable to flag: If generality is emerging, society should know.

Weaknesses

• No concrete examples: Without specifics, it’s hard to evaluate whether the systems were genuinely general or simply impressive.
• Generalisation ≠ general intelligence: Models can appear broad while still being brittle, narrow, or dependent on human‑curated training.

---

4. His call for open conversation
He argues that society needs to talk more seriously about AI risks.

Strengths

• Absolutely correct: Public discourse is lagging behind technological reality.
• Democratising the conversation: Scientists speaking plainly helps bridge the gap between research and society.

Weaknesses

• Conversation without specificity can fuel anxiety: Raising alarms without concrete examples risks creating a fog of fear rather than clarity.
• Missing the governance dimension: He doesn’t articulate what kinds of oversight, regulation, or institutional structures he believes are necessary.

---

5. His framing of AI as a potential existential threat
He gestures toward the idea that AI could fundamentally alter or undermine human agency.

Strengths

• Legitimate philosophical question: AI does challenge long‑held assumptions about creativity, labour, and cognition.
• Consistent with some expert views: A minority of AI researchers do consider existential risk plausible.

Weaknesses

• Risk conflation: He blends near‑term risks (misinformation, labour disruption) with long‑term speculative risks (superintelligence), which can muddy the analysis.
• Lack of proportionality: He doesn’t weigh AI risks against other global risks (climate, biosecurity, geopolitics), which could contextualise the threat.

---

Overall Assessment
His concerns are sincere, thoughtful, and grounded in real trends, but they are also shaped by proximity to cutting‑edge research, which can amplify the sense of urgency. The strongest part of his argument is the call for open, honest discussion about AI’s trajectory. The weakest part is the lack of specificity: without concrete examples, the audience is left to fill in the gaps with imagination, which can lead to either undue panic or dismissiveness.

He is right that AI deserves serious scrutiny.
He is less convincing when implying that catastrophe is imminent.

---

If you want, I can also provide:

• A point‑by‑point rebuttal
• A sympathetic defence of his position
• A comparison with other AI risk thinkers (Bostrom, Hinton, Yudkowsky, Bengio)
• A version of this analysis written in a more academic or more conversational tone

Just tell me which direction you want to explore next.

that’s a very bland summary of the video that leaves out a lot and feels very homogenized. Almost as if it’s not reacting to just the video but also other similar videos. I’ll post my own thoughts and takeaways from the video later. I’ve been working all day and I’m not home yet but I’ve been thinking about it ever since I watched it.

 

[Darat]

that’s a very bland summary of the video that leaves out a lot and feels very homogenized. Almost as if it’s not reacting to just the video but also other similar videos. I’ll post my own thoughts and takeaways from the video later. I’ve been working all day and I’m not home yet but I’ve been thinking about it ever since I watched it.

I agree that is a bland and rather superficial summary of the video, but I wanted to use one of the free AIs so everyone could play along.

I had a free trial of ChatGPT which finishes on 19/2 and I think that does provide a much better summary, with the same prompt:

(Had to do some formatting editing with EOL and whitespace)
In this video from the Cool Worlds Podcast, David Kipping shares insights from a recent high-level meeting at the Institute for Advanced Study (IAS) concerning the impact of artificial intelligence on the future of science.

Key Themes and Takeaways:​

• AI Supremacy in Technical Skills: Kipping notes a consensus among elite scientists that AI has achieved "complete coding supremacy" [05:14] and is rapidly approaching superior ability in analytical reasoning and complex mathematics [07:02].
  
  
• The Inevitability of Adoption: Many top-tier researchers are surrendering control of their digital lives to agentic AI [10:16], feeling that the productivity boost is so "outsized" that privacy and even some ethical concerns are being sidelined to remain competitive [13:40].
  
  
• Risk of Skill Atrophy: There is a deep worry that as scientists defer to AI for problem-solving, fundamental skills in coding and derivation will atrophy, similar to how human navigation skills have declined with the use of GPS [15:09].
  
  
• Economic and Training Disruptions: AI's ability to perform the work of junior researchers faster and cheaper [38:32] poses a threat to the traditional model of graduate student training and could lead to a future with fewer entry-level scientific positions [34:07].
  
  
• Democratization vs. Overload: While AI could "democratize" science by allowing anyone to conduct high-level research [01:05:31], it also risks creating a "tsunami" of papers that no human can realistically ingest or verify [01:07:58].
  
  
• The Human "Why" of Science: Kipping concludes by questioning the value of a world where discoveries are made by super-intelligence but are incomprehensible to humans. He argues that science is inherently a human endeavor driven by the joy of personal discovery [01:09:05].

Watch the full video here: https://www.youtube.com/watch?v=PctlBxRh0p4

Its critique of the video:

A critical analysis of David Kipping’s video reveals a profound tension between technological pragmatism and scientific idealism. The video serves as a "dispatch from the front lines" of elite academia, and a critique of its arguments highlights several logical conflicts and shifts in the scientific zeitgeist.

1. The Paradox of Democratization vs. Oligarchy​

Kipping presents two conflicting visions of the future:

• Democratization: He suggests that a $20/month subscription allows anyone to produce research comparable to a PhD.
• Oligarchic Disruption: Conversely, he warns of "price jacking" where AI companies could eventually charge thousands per month, effectively pricing out everyone except elite, "wealthy" institutions like Harvard or Princeton.
• Critique: These two outcomes cannot easily coexist. If the "democratization" relies on corporate tools, it is a fragile democratization. The speaker identifies but does not resolve the risk that science could shift from a meritocracy of "brains" to a meritocracy of "budgetary access" to compute.

2. The Devaluation of "Intellectual Labor"​

The video reports a startling concession: elite scientists are willing to trade privacy and ethics for productivity.

• The "I Don't Care" Attitude: Kipping notes that senior faculty are surrendering full "super-user control" of their digital lives to agentic AI, dismissing ethical concerns about climate change or job displacement because the "advantage is too great".
• Critique: This represents a significant shift in scientific integrity. If the "thought leaders" of science prioritize speed over the ethical implications of their tools, it suggests a move toward "industrialized discovery" where the process of science (which includes ethics and skepticism) is sacrificed for the output.

3. Quantitative Ambiguity of "90%"​

The video relies heavily on subjective estimations of AI capability.

• The Claim: A senior faculty member estimated AI could do "90% of the things he can do".
• The Gap: Kipping immediately admits this number is arbitrary, stating it could be "60% or 99%".
• Critique: This highlights a lack of empirical metrics for what "scientific work" actually is. By reducing complex intellectual labor to a percentage, the speakers risk oversimplifying the "final 10%"—the creative, non-linear insights that historically define scientific breakthroughs—as something that will inevitably follow the same trajectory as coding.

4. The Loss of the "Human Narrative"​

Kipping’s most poignant critique of the current trend is the potential loss of comprehensibility.

• The "Black Box" Problem: He fears a world where AI delivers "fusion power" or "new physics," but the underlying mechanics are incomprehensible to humans.
• Critique: This challenges the very definition of science. Historically, science is the quest for human understanding. If AI provides the results without providing the explanation, science ceases to be an intellectual endeavor and becomes a form of "high-tech magic".

5. Contradiction in Collaboration​

• Argument: Kipping reflects on the "social interaction" and "collaborative nature" of science.
• AI Shift: He then suggests that collaborations might shrink to "single authors" because it is easier to prompt an AI than to navigate the "time investment" of a human partner.
• Critique: If AI reduces the need for human-to-human interaction, it may destroy the "check and balance" system of peer-to-peer skepticism. The speaker notes that he "still writes most of his code himself" to maintain quality, yet he predicts a future where others will not, creating a potential "reproducibility crisis" if human oversight becomes an "annoyance".

Summary​

The video effectively documents a "historic moment" of surrender. The critique of the IAS meeting is that it reflects a "race to the bottom" where the fear of being uncompetitive is forcing the world’s smartest people to abandon the traditional, human-centric methods of discovery that made them "elite" in the first place.

Associated Video: https://www.youtube.com/watch?v=PctlBxRh0p4

 

[Beelzebuddy]

No, only in more complicated terms.

I can give it a try. I'll use your cookie analogy.

Imagine every written cookie recipe in the world, scrunched up into a little ball. The very middle of the ball represents the centroid of the cookie embedding, the platonic concept of the cookie: the Ur-Cookie. The Primal Cookie. The Cookiest. It's the point of reference an LLM starts from if you simply ask it for a cookie recipe. Its only point of reference. The output process takes that point and knocks it away a bit, so you don't always get The Cookie but something different each time. The parameter that controls that process is called "temperature," referencing a different metaphor. By varying the temperature you can get anything from the most boring cookie imaginable to purple monkey dishwasher. And everything in between. But that's your only control.

(There are other sliders and such, but they're similarly focused on the mathematical distribution of the response because that's all the LLM is doing)

Point is, you get a mix of cookie and not-cookie that you can vary how you please. Too much cookie and you'll only make slop, sloshing around eternally in a safe and familiar embedding. The creative take you need to reinvent the cookie exists, but by necessity it exists out there beyond the cookie concept. Where there are also a lot of terrible ideas. Using raw onion instead of flour. Mixing in some ground glass, for a spiky taste your guests will remember for years to come. The LLM doesn't know, and isn't capable of caring what is in that not-cookie space you are sampling. That's where the "tee hee, look at these awful recipes" stories come from. The creativity is there, critical appraisal is not. That's just what's at the probabilistic sample you requested. What you do with it is your own business.

 

[Puppycow]

I agree that is a bland and rather superficial summary of the video, but I wanted to use one of the free AIs so everyone could play along.

I had a free trial of ChatGPT which finishes on 19/2 and I think that does provide a much better summary, with the same prompt:



It's critique of the video:

A critical analysis of David Kipping’s video reveals a profound tension between technological pragmatism and scientific idealism. The video serves as a "dispatch from the front lines" of elite academia, and a critique of its arguments highlights several logical conflicts and shifts in the scientific zeitgeist.

1. The Paradox of Democratization vs. Oligarchy​

Kipping presents two conflicting visions of the future:

• Democratization: He suggests that a $20/month subscription allows anyone to produce research comparable to a PhD.
• Oligarchic Disruption: Conversely, he warns of "price jacking" where AI companies could eventually charge thousands per month, effectively pricing out everyone except elite, "wealthy" institutions like Harvard or Princeton.
• Critique: These two outcomes cannot easily coexist. If the "democratization" relies on corporate tools, it is a fragile democratization. The speaker identifies but does not resolve the risk that science could shift from a meritocracy of "brains" to a meritocracy of "budgetary access" to compute.

2. The Devaluation of "Intellectual Labor"​

The video reports a startling concession: elite scientists are willing to trade privacy and ethics for productivity.

• The "I Don't Care" Attitude: Kipping notes that senior faculty are surrendering full "super-user control" of their digital lives to agentic AI, dismissing ethical concerns about climate change or job displacement because the "advantage is too great".
• Critique: This represents a significant shift in scientific integrity. If the "thought leaders" of science prioritize speed over the ethical implications of their tools, it suggests a move toward "industrialized discovery" where the process of science (which includes ethics and skepticism) is sacrificed for the output.

3. Quantitative Ambiguity of "90%"​

The video relies heavily on subjective estimations of AI capability.

• The Claim: A senior faculty member estimated AI could do "90% of the things he can do".
• The Gap: Kipping immediately admits this number is arbitrary, stating it could be "60% or 99%".
• Critique: This highlights a lack of empirical metrics for what "scientific work" actually is. By reducing complex intellectual labor to a percentage, the speakers risk oversimplifying the "final 10%"—the creative, non-linear insights that historically define scientific breakthroughs—as something that will inevitably follow the same trajectory as coding.

4. The Loss of the "Human Narrative"​

Kipping’s most poignant critique of the current trend is the potential loss of comprehensibility.

• The "Black Box" Problem: He fears a world where AI delivers "fusion power" or "new physics," but the underlying mechanics are incomprehensible to humans.
• Critique: This challenges the very definition of science. Historically, science is the quest for human understanding. If AI provides the results without providing the explanation, science ceases to be an intellectual endeavor and becomes a form of "high-tech magic".

5. Contradiction in Collaboration​

• Argument: Kipping reflects on the "social interaction" and "collaborative nature" of science.
• AI Shift: He then suggests that collaborations might shrink to "single authors" because it is easier to prompt an AI than to navigate the "time investment" of a human partner.
• Critique: If AI reduces the need for human-to-human interaction, it may destroy the "check and balance" system of peer-to-peer skepticism. The speaker notes that he "still writes most of his code himself" to maintain quality, yet he predicts a future where others will not, creating a potential "reproducibility crisis" if human oversight becomes an "annoyance".

Summary​

The video effectively documents a "historic moment" of surrender. The critique of the IAS meeting is that it reflects a "race to the bottom" where the fear of being uncompetitive is forcing the world’s smartest people to abandon the traditional, human-centric methods of discovery that made them "elite" in the first place.

Associated Video: https://www.youtube.com/watch?v=PctlBxRh0p4

These summaries seem much more accurate than the earlier ones. I certainly wouldn't be able to match the latter ones myself, which is mainly the reason why I don't like to attempt to "summarize" videos myself when my time is limited. It's because I don't think I could really do it justice.

 

[Puppycow]

This is professor David Kipping from Columbia talking about a meeting he attended at the Institute for Advanced Studies at Princeton:


I'm just 10 minutes into it myself, but it seems very interesting.

Here's a few of my own takeaways from the video after processing it.

1. It made me seriously think about the idea of a "singularity" again. I'm sure that everyone has already heard this and probably dismissed it as I too have in the past. If it does happen (and I mean IF, not when), it may be hard for most people to actually discern it happening. I don't think it has already happened, but maybe we are starting to see signs of it coming. If an AI singularity is like a waterfall, maybe we are just entering the rapids that lead to the waterfall.

2. In case you're thinking about obvious red flags, David Kipping is not a random crazy person shouting on a street corner. He's a tenured professor at a major university. ("But Puppycow, so is Avi Loeb!") Yes, I know. Kipping is still in good standing last I heard.

3. Returning to the notion of a "singularity". I still doubt there would actually be a single identifiable moment, without the benefit of hindsight certainly. Kipping himself is unsure about what this may mean eventually.

4. If robots take our jobs, it seems likely that they come for white collars first, not blue. (I realized this myself long ago.)

 

[Andy_Ross]

Imagining the near future when scammer bots are sending AI dick pics to AI dead people's accounts.

Meta has patented AI that can run a dead person's account, continuing to post and chat on their behalf

It can message and video call by replicating a user's online behavior using their past data

Meta patents AI that takes over a dead person’s account to keep posting and chatting - Dexerto

Meta has been granted a patent outlining an AI system capable of simulating a user’s activity on social media to post after their death.

www.dexerto.com

 

[Dr.Sid]

Weird kink but ok

 

[JayUtah]

The LLM doesn't know, and isn't capable of caring what is in that not-cookie space you are sampling. That's where the "tee hee, look at these awful recipes" stories come from. The creativity is there, critical appraisal is not. That's just what's at the probabilistic sample you requested. What you do with it is your own business.

This is pretty good. What makes embeddings useful in many LLMs is the ability to associate data with metadata. In the recipe space, this means encoding the ingredients and method alongside such things as feedback and ratings. Metadata is analogous to use-case information for technical solutions. This is important because innovation is often realizing that what doesn't work for the typically-represented use cases works best for less-represented use cases.

 

[Chanakya]

No, only in more complicated terms.


LLMs work by encoding semantic content as tensors. Not all semantic content is explicit enough to be separately representable along its own tensor dimension, or some recognizable combination of them. This results in embeddings that embody the underlying assumptions without actually expressing them in the embedding by something that is exposed to computation. You can certainly generate contrary content, but the transformer layers of the LLM will reject it according to probabilistic rules.

You're right, that's not simpler! Didn't understand that either I'm afraid.

Imagine you have a training data for a baking AI that's composed of millions of baked-goods recipes. You have a variety of ingredients, methods, and baking regimes. An AI can synthesize a number of variation on the cookie from this, all of which have various probabilistic landscapes in the overall problem space. They're all credible (although actual cooks note that AI-generated recipes tend to do stupid things).

But if some human baker devises an unprecedented method whereby the dough is actually frozen instead of baked, and the desired chemical reactions occur, then such a thing would simply not be visible in any way in the training data. The training data embeds such notions as chemical composition of the dough, baking temperature, baking time, oven humidity, and so forth. And this can be further enriched with metadata on which combinations have produced good outcomes. But what is not embedded in the data is the decision to use an endothermic process at all. That's simply assumed in all the training data and therefore not represented as a differentiable concept. Hence it will be assumed in all the solutions.

None of that really has anything to do with the higher-level philosophical discussion of such things as Roku's Basilisk.

Well that's like where I was at myself. My impression was that AI isn't, so far, capable of actual critical thinking on its own steam --- as opposed to simply parodying, no matter how engagingly and persuasively, what it's seen and read already. That it isn't capable of critiquing, for instance, Roko's Basilisk off of its own steam, as opposed to simply piecing together what everyone else has had to say about that construct, or about similar constructs.

And likewise, I'd been under the impression that AI isn't, so far, capable of creativity beyond simply collating and combining from and essentially parodying its training material, even if it can do this engagingly.

But then the discussion here seemed to indicate otherwise. To my surprise. And, given that my own understanding of AI isn't technically grounded, I was aware that I could well be mistaken, and thought that maybe AI has already reached to the point where my reservations were no longer valid.

...And here you are now, essentially saying I was right after all, as far as I can make out! (Unless I've misunderstood you? I haven't, have I?)

----------

I'd very much like to get to the bottom of this. This is important, I think. If a post or two more here doesn't settle this one way or the other, then I think I'll start a separate thread focused on this single focused issue. (Not on Roko per se, Roko per se is irrelevant, I mean focused on whether AI is already capable of actual critical thinking and of actual creative ideation, that are not directly derivative of the training material it has absorbed.)

 

[Chanakya]

Creativity and critical thinking are very different things. LLMs fail utterly at the latter...

Sure. I agree they’re different things. I was under the impression that AI so far is able to do neither of these two different things. ...It is being said here that AI is now able to do both. Which is what I’m trying to explore, to see if I should change my view on this, already.

...LLMs fail utterly at the latter.

Seriously just try some out for yourself. AI's getting crammed into everydamnthing. Every browser and search engine has its own and is desperate for someone to use it.

Me, I agree with you, that LLMs fail at critical thinking.

As far as trying it out, well like everyone else I have done that, cursorily and nothing fancy, many times. And these days most critiques are pretty much excellent tbh ---- for instance its critique of Roko, that I checked out quickly just now. But I was under the impression that that’s the result of AI getting better at simply reproducing and parodying what it’s already seen. So that its apparent critical thinking is no more than a parody of others’ critical thinking; and will fail any time it sees something original, that it has not encountered before.

But then, again, in my recent exchanges here I’ve found it suggested that AI is able to critical-think its way to conclusions it’s never ever come across. And again, that opinion of others here, that is different than mine, is what I’m trying to explore, to see if my views are out of date already and need to be changed.

 

[JayUtah]

Sure. I agree they’re different things. I was under the impression that AI so far is able to do neither of these two different things.

An LLM-based generative AI utterly fails at critical thinking because evaluating concepts according to some criterion extrinsic to its training data is just not something the algorithm is even aimed at doing. However, if critical analysis of a concept appears as part of the training data, then a generative AI can synthesize what looks like a critical analysis if prompted to do so. But it won't be an independent analysis of the data. It will just be the critical-analysis part of whatever data it was fed, transformed as appropriate to the prompt.

In the cookie analogy, if a particular set of recipes whose embeddings align a certain way also includes data from reviewers that say those cookies taste terrible, then a prompt asking for an evaluation of matching recipes can certainly report that. And that's often useful information. But the computer algorithm is obviously not tasting the cookies and giving you its impression. The AI isn't responding critically. It's collating and synthesizing what others who contributed to its training might have said if they had been asked.

As to whether a generative AI can be creative, I think that's still mired in the details of what actually constitutes creativity.