Concerns About AI

[Nyarlathotep]

How do you scale up the part of the system that checks the output of an AI?

[UKAtheist]

That sounds like a question for the experts (not sure I even understand what you’re asking), perhaps Hinton or Yudkowsky?

UK Atheist

[mordant]

Here’s a pretty good summary of the relationship between AI doomerism and (a)religion.

First They Built a Secular Apocalypse Belief System. Now They Want Religious...

TL;DR:

AI doomerism borrows religious structure while denying religious content. It offers ex-believers a replacement meaning system with cosmic stakes. Now, it reconnects with organized religion in seeking broader moral authority.

But this is an old playbook: a small group of believers claims it alone understands the coming apocalypse, then asks everyone else to hand it centralized control.

As a religious person myself, I reject the AI doomers’ power play. You should, too.

[Sheldon]

@mordant

I guess we are going to use the term AI whether applicable or not. This was done by LLMs.

Not my field, but I had understood LLM’s to be a form of AI, but not all AI to be LLM’s? Like all thumbs are fingers, but not all fingers are thumbs. Is it the case that the terms are insufficiently accurate rather than inappropriate?

[mordant]

@Sheldon

Not my field, but I had understood LLM’s to be a form of AI, but not all AI to be LLM’s? Like all thumbs are fingers, but not all fingers are thumbs. Is it the case that the terms are insufficiently accurate rather than inappropriate?

LLMs are the technology. There is no other basis or form for what popular usage, egged on by the industry, is coming to call AI. I suppose I will just have to bow to it, ultimately. I will admit, AI is a better marketing term. And marketing terms aren’t exactly known for accuracy.

[Sheldon]

@mordant

LLMs are the technology. There is no other basis or form for what popular usage, egged on by the industry, is coming to call AI. I suppose I will just have to bow to it, ultimately. I will admit, AI is a better marketing term. And marketing terms aren’t exactly known for accuracy.

Language always plays catchup to new ideas i suppose. The term the BIg Bang was originally intended as a derogation of that idea, and of course is far from an accurate description. Scientists often use terms meant as metaphor, and others then make erroneous assumptions based on those terms, like fine tuning for example.

I’m happy to admit this is one of many topics where I am woefully out of my depth though. I just thought on first reading that the story seemed an interesting addition to the thread.

[mordant]

@Nyarlathotep

How do you scale up the part of the system that checks the output of an AI?

Since the output of AIs need to be checked due to frequent hallucinations, this is an excellent question. And “the system” for checking output is mostly humans.

At the other end of the process is scaling the things that increasingly we’re realizing really MUST happen before involving an LLM to begin with, which is, broadly, architectural choices and standards, coding conventions, choice of libraries to use or not use, etc.

I find this amusing for two reasons: it’s exposing that the industry has largely coded by “making it up” / iterating as it goes, rather than planning things out at all; and, it’s returning us to what amounts to waterfall design methods that have been disparaged since the advent of agile coding. The theory is that specifications, needs, and the the business itself change so fast that there’s literally no point in a strict design → build sequence. And yet. Since LLMs have no true comprehension or ability to get an intuitive grasp on the big picture, to have a “sense” of anything to guide them, you have to define the big picture FOR them.

So I think that we are just moving bottlenecks around rather than actually eliminating them on a net basis.

[mordant]

Here is perhaps the best essay I’ve seen on:

• Why LLMs are not remotely conscious
• What it would take to credibly assert that they are conscious, or even close to being conscious.

This is from the Atlantic but appears not to be paywalled. If anyone who wishes to read it can’t, there’s a fair-use quote below, but the article really makes a better case.

No, Artificial Intelligence Is Not Conscious

So what context would cause me to seriously consider the possibility that engineers created a computer program that is conscious and an intentional user of language? Let me outline one potential sequence of steps. The first requirement is that the computer program has a body (either physical or virtual) and sense organs; there are many reasons for this, but for the purposes of this discussion, the most relevant one is the fact that without a body, a computer program could have no desires or emotions, and I believe desires and emotions are necessary for consciousness. Then I’d want to see an embodied agent that could navigate its environment in order to survive as well as, say, a lizard can (and as a point of comparison, certain iguanas can live for decades in the wild). Next, I would want to see an embodied agent with the same capacity to deal with novel situations as a mouse. After that, I’d want to see agents whose social dynamics are as complex as those of wolves, and then agents with the toolmaking abilities of chimpanzees. At that point, I would want to see people successfully teaching such embodied agents how to communicate their desires, perhaps by using a button board or some other nonlinguistic modality, the way that people have taught chimpanzees and domesticated dogs. The agents’ communication abilities would have to withstand all the scrutiny that animal-communication researchers have had to defend their work against. If engineers build an embodied agent that meets these criteria, they will have accomplished something incredible, but it leaves us near the orbit of Pluto, metaphorically speaking; we would still be light-years away from building an entity capable of learning how to express its thoughts in complete grammatical sentences.

[cynical1]

@mordant

Here’s a pretty good summary of the relationship between AI doomerism and (a)religion.

Interesting article.

It certainly points to the major roadblock in the way of AI becoming SKYNET. Humans live through a trifecta of fear, attachment and crisis. This creates and environment of political compromises, tribal loyalties, and emotional irrationality.

AI\AGI does\will not suffer from this dilemma. They exist through pure machine logic and goal\target optimization.

Introduce that into a world of bureaucratic bitching, corruption, cost cutting and infrastructure inconsistencies makes any attempt by an AGI to weaponize itself compromised and fragmented within a human system.

An AI, by design, faces steep climb in attempting to control a species that is perpetually willing to break its own world over a grievance or identity attachment.

Ironically, this consistent ability to make sub-optimal, logically “stupid” decisions based on spite, tribal loyalty, affection, or fear is likely our greatest defense against a Skynet scenario.

[Calilasseia]

I’ve already covered pertinent worries about the glorified Elizabots here.