I think there are far more immediate concerns than potential future AGI.
For example, this article describes academics having an aneurism because a publisher will ban them for a year if their paper contains an AI-hallucinated non-fact.
This is evidece that LLMs are already making academics dumber / sloppier. What is it doing to the rest of us?
I don’t believe this claim of course, and have never seen anything approaching objective evidence for it. Though we could also apply Occam’s razor here, and many simpler explanations could be suggested, that require no unevidenced assumptions about objective reality, anything from poorly designed or even biased studies to outright chicanery for example.
This has been the case in every “study” I’ve see presented as evidence for this claim. Nor is it hard to imagine what the global scientific and theological reaction would be if any such evidence had been reliably demonstrated.
Why won’t an AGI be influenced by these things? I’m not saying AI has these things right now or even that we possess the technology to create them but these kinds of assertions seem predicated on humans (or at least organic life) being somehow special, somehow capable of something that machines are not. IMO the brain, while hundreds (thousands?) of times more complex that any computer system in existence today, is still “just” infrastructure, the thing our intelligence runs on… IOW (again, my opinion), intelligence, sentience, emotion and, yes, identity are all products of massive computing power. In essence, I just don’t buy the idea that machines will never develop these attributes.
Clearly I can’t say you’re not right but it’s a helluva’n assertion because it ignores the possibility that AI (AGI) might one day develop into more than it currently is and that appears to be what some of these leading workers in the field (the Yudkowsky’s and Hinton’s) scientists seem concerned about. TBH, I’m not even sure it matters as even the appearance of intelligence might be enough to bring about our downfall.
Humans have only existed for about three hundred thousand years, been civilised for, what, two thousand years (and then seemingly only some of them)? And the last hundred years of that we seem to have spent closer to “midnight” on the doomsday clock. I realise there’s more than one species there but dinosaurs existed for well over a hundred million years. I mean, we might think we’re successful but, in objective terms, are we really?
That might well be true now but the general thrust of the above strikes me as an argument for humans (or organic life) being in some way special. How can you, I or anyone else know what might happen in ten, fifty, or a hundred years?
Ultimately, I think there is a need for caution and not the apparent headlong dash to create the biggest, bestest (sic) and smartest AI that ever existed which seems to be the way we’re headed right now. To my mind it’s foolish.
UK Atheist
Intelligence and motivation are not the same thing. We can develop intelligent autonomous machines. They will not have immediate motivation due to this intelligence. The “motivation”, or the perception of motivation, will spring from the human programming or “prime directive” motivation.
Currently, the human motivation is to turn a profit, or at bare minimum, break even until they can be profitable. It is the language and structure of the human request to the AGI tool that is genuinely the unknown in this equation.
EXAMPLE: The AGI is given the task of increasing beer sales at a hockey game. The AGI analyzes the problem and determines the best way to increase beer sales is to lock to doors on the stadium and not open them until all the beer is sold.
EXAMPLE 2: The medical AGI is tasked with improving cardiac patient outcomes. It determines that certain symptoms indicate a low prognosis for success…so it refuses hospital admittance to the most questionably ill patients to shift the metrics as requested.
As I said, the machine is no where near as scary as the man behind it.
We will eventually be as successful as the other 99.9% of the 5 billion species who have existed on this planet. We’re just a lot more arrogant and dismissive in ignoring.
Humans seem to have first civilized around 6500 years ago in Mesopotamia. The real problems started about 3000 years ago when we developed a written language.
So, to your point, as a species, we have only been an potentially active threat to ourselves for around 1% of our existence.
You are not alone in your concerns. But we have built systems and devices that can initiate an extinction event for almost 100 years.
AGI becoming Skynet faces one tremendous hurdle in replacing us…the Second Law of Thermodynamics. The panic of a 21st century Armageddon is going to need a 20th century power grid to do it with.
I’m not making those arguments.
Smarter people than me say that AGI cannot arise from LLM technology. That isn’t to say it couldn’t or wont arise from some future tech. In fact I would bank on that happening eventually because I don’t think organic life is special regarding cognition (unique currently, but not special). I think machine cognition would be qualitatively different from organic cognition since it’s not embodied in the extensive ways organic cognition is, but that doesn’t mean it wouldn’t be able to happen.
All that said I don’t see it as an imminent threat. I think it’s never too soon to think about the risks and ethical questions that AGI would produce, and try to lay the groundwork for handling those issues. But IMO the climate crisis and authoritarianism marching across the face of the earth once again are more immediate concerns.
Depending on how one defines the start of civilization, one can argue that it goes back even further, to the foundation of the first proto-cities, like Çatalhöyük in modern-day Turkey, around 9500 years ago, or the neolithic settlement in Ayn Ghazal in modern-day Jordan, estimated to have been founded 10300 years ago. But if one include not only city dwellings, but also social stratification, ruling hierarchies, tax collection, specialisation in work tasks, etc., then sure, ancient Mesopotamia was probably the first civilization.
Yeah, people are scary but I don’t think that invalidates my basic point that machines, if advanced enough, represent an existential threat.
On the plus side writing allowed us to pass on our better ideas effectively, on the minus side writing allowed us to pass on our poorer ideas effectively! Can’t win really!
Can’t argue overmuch about the 1% bit.
Unless they (humans or computers) come up with a better way of using power, maybe solar collectors in deep space or directly sapping energy from the sun (a technological version of Weir’s astrophage perhaps). We’re talking about future developments here.
Smarter people than either of us believe AI is a threat already, Geoffrey Hinton for example.
They are and the tech bros developing AI are taking that very seriously indeed… oh wait, it’s the other thing.
UK Atheist
I think the real threat of AI is made up fear generating stories. We already have part of the human species fear mongering fictitious tales of child molesters in the basement of a pizza parlor that has no basement. Imagine what a good AI could make up that would be believable by those morons.
So you think that Geoffrey Hinton (the “Godfather of AI”) and Eliezer Yudkowsky who, as I said at the start, is prominent in the field of artificial intelligence (AI & AGI) and author of, “If Anyone Builds It, Everyone Dies” where he advances the view that the predictable outcome of building a machine intelligence smarter than us is that everyone on Earth is dead, are just making shit up?
UK Atheist
No, I just don’t think they are incapable of being mistaken.
There IS no “AI” at present so Hinton is not the Godfather of it. He is the inventor of neural networks which are the basis of LLMs which cannot produce AGI, at least not by themselves. I do not minimize his impressive contributions to the field; I just do not think that makes him any better than the next human at making detailed predictions.
Where I can give credence to what he’s saying is that should we ever produce actual self-aware, creative, intuitive artificial minds, we should be very careful about it. Fortunately for us, we haven’t done that yet.
ETA: If you want some insight into the confused hubris of how technical minds tend to think about the world, and themselves, I cannot recommend enough that you read Cyberselfish: A Critical Romp through the Terribly LIbertarian Culture of High Tech by Paulina Borsook. It’s a 26 year old book but holds up really well. Of course Hinton is less a member of the Digerati than a scientist, but I doubt he has escaped the influence of the tech world he is helping to create. He invents stuff, they bestow titles on him, one hand washes the other. People start to think a little too much of the smell of their own shit. Hilarity ensues.
I’ll just say: I don’t trust their opinions on the matter. I don’t think we are any closer to AGI then we were 10 years ago, maybe even 40 years ago.
Just read this on the BBC site, it seems apropos to this thread:
Artificial intelligence has been used to develop a “fundamentally new” type of vaccine that could protect against large swathes of viruses and prevent pandemics, say researchers.
The team at the University of Cambridge say it is the first time a vaccine’s key component has been designed entirely by AI and then trialled in people.
The vaccine was engineered to work on all coronaviruses which would include all Covid variants as well as viruses that currently infect animals yet have the potential to start the next pandemic.
The work is still in the early stages, but the team is already developing separate vaccines that could tackle flu and Ebola.
Prof Saul Faust, who performed some of the trials at the University of Southampton, said the AI design “definitely has potential” and was “really exciting”.
He told the BBC: “What’s really interesting is the technology is an awful lot better at designing vaccines for potential pandemics when viruses are changing.”
In the mid 50’s Minsky intimated “artificial intelligence” could be a reality within a generation. During the 70’s and 80’s Stanford and CMU developed systems around VAX …again, promising AI was only around the corner… In the late 80’s Kurzweil predicted a “human level AI” should arrive by 2029.
Looking at Altman and Amodei claim AGI is only a few years away…it seems they are the new goal post locators for the industry.
It strikes me that the bottlenecks have always been the same. The physical framework of the hardware throttles the speed required to process at a “human” level. Scaling the technology into larger and larger entities runs you straight into the primary road block. The insane and nearly irresponsible sums of money required to scale up.
It strikes me that current LLM and promised AGI is just another example of “fake it until you make it”. No one over the age of 30 will be alive to see it.
The fact that they can’t instantiate a single AGI despite building spoon-bending giant data centers that suck the power grid dry tells you that the approach is wrong. Here I am, a meat machine, running on tiny amounts of electricity, running circles around their non-existent AGI.
[sigh] I guess we are going to use the term AI whether applicable or not. This was done by LLMs. And medical research of this kind, as well as reading radiology imaging, are fortunately a couple of things that they are actually quite good and useful at.
No one is suggesting that we should stop using LLMs for anything they are actually helpful with, or that those uses shouldn’t be prioritized at the expense of making various forms of enshittifying slop.
There’s a reason people don’t want data centers near them, and that people don’t want LLM girlfriends or to have Microsoft shoving AI into every corner of Windows whether anyone asked for it or not. Or to have the vast sums of money being made go to a few rich fucks and no one else.
Since this is an AI thread, I asked Google’s AI for a breakdown of power requirements for a human brain versus a fully functioning AGI, if ever attained:
Human Brain: 12w-20w
Projected AGI: 50,000,000 – 2,700,000,000 Watts (50MW – 2.7GW)
To paraphrase Carville, “It’s the architecture, stupid”
Marvin Minsky?
I recall him saying the following (not exactly, this was decades ago) about computers. Not AI specifically. Ok, so that not specifically on topic. But I offer the recollection in case it’s useful.
There’s going to come a day quite soon when a guy will walk into a room where there are two computers, Charlie and Fred. The man walks over to Fred, taps his fingers on Fred’s top and says, “Hey, how are you guys doin’?”
In the time it took the man to do these things Charlie and Fred will have exchanged more information than the human race has ever written down.
The very same. (20,20,20)
Just so you know, that was aimed at CapriMark1, not you… just sayin’
I disagree on the first part, LLM or not, it’s still AI, just not AGI and Hinton is regarded as said “Godfather” as a mark of respect for his work and expertise (which you subsequently acknowledge).
Which is pretty much what my original post was alluding to.
True although I’m not convinced it really matters.
Said experts disagree and, as it happens, so do I.
UK Atheist