Chatgpt debate?

[Calilasseia]

Someone else has already raised the issue I would have presented here, namely that so-called “AI” systems of this sort are basically glorified Elizabots, with access to extremely large databases, lots of processing power and memory. They don’t actually think about the content you’ve supplied them, they simply engage in a huge amount of pattern matching in order to generate replies to input that look “intelligent” to the untrained eye. None of these chatbots possess the ability to process concepts directly, which would be the first step toward genuine intelligence.

To understand what is happening here, dig up any of the old BASIC code (or, on some Unix systems, the C code) for the ELIZA program. ELIZA was the first attempt to simulate conversation, and was a rather simplistic attempt at the conversational behaviour of a psychiatrist using nondirective therapy. When first unleashed in the early 1960s, it caught some people by surprise, and Douglas Hofstadter covers this in his excellent book Gödel, Escher, Bach.

All that ELIZA did at bottom, was change the person of pronouns and verbs, and rearrange the text to suit, in accordance with pre-designed templates. But the output thereof still bore enough resemblance to genuine human speech to be convincing to the unwary.

The modern chatbots operate on a similar principle, just expanded to take advantage of the far greater abundance of processing power and storage. They are initially presented with a large set of general rules for syntax and grammar in a particular human language (hence the term ‘Large Language Models’ or LLMs for short), then directed to look for commonly seen variations on said language usage connected to particular nouns, verbs, etc, in the training set used to condition their behaviour.

This latter action is how they LLMs appear to simulate human conversation extremely well - appear being the operative concept here. None of these LLMs are actually processing concepts, even though the language manipulation presents this impression to the unwary. They’re simply as I described them succinctly above - glorified Elizabots.

To make matters worse, because they are still “in training” at the moment, their output can easily be skewed by devious and manipulative individuals, supplying them with carefully crafted training material to pursue an agenda. Via this process, they can direct the LLM to produce output that, once again, appears to lend credence to absurd or iniquitous viewpoints.

Over at another forum, there’s an extremely tedious creationist thread that was finally shut down by the moderators after it had run for over nine years, that constitutes possibly the canonical example of a creationist trainwreck. The level of masturbatory obsession on display therein, as its originator tried to pursue a wank fantasy of palsied and deranged proportions, will probably keep psychoanalysts in research material for decades. But I digress - there’s a reason I mention this tsunami of tard, which is directly related to the abuse of LLMs, and an apposite starting point is this post of mine therein.

From that post, I provide the following quote to save effort on the part of those with better things to do, than trawl through the miasma that was a combat arena of mine for nine years:

Of course, lazy and mendacious propagandists will love the opportunities for mischief and mayhem that LLMs offer. They actually offer the worst of all discoursive worlds - too unreliable to contribute to proper, rigorous exchange and analysis of ideas, and far too amenable to ease of abuse by the unscrupulous, placing in the hands thereof potential for damage beyond their wildest dreams. Perhaps their most lethal effect upon discourse, will be to make mendacious ex recto apologetic fabrications available to order in bulk, a development all sane people should view with undisguised horror.

though I still exhort people to peruse that particular post directly, because it contains cogent points about genuinely rigorous applications of AI, such as the Isabelle mathematical proof assistant, and related concepts. Some of which I may have brought here in other threads, but the review is still of utility value here.

However, the major issue I wish to bring here, is the manner in which LLMs can be abused by all manner of bad actors. The Register posted an article on how Microsoft’s Copilot could be hijacked to perform data theft. For example:

In his earlier work poking holes in LLMs, Rehberger had disclosed to Microsoft that Copilot was vulnerable to zero-click image rendering, and Redmond fixed the issue. To find another way to exfiltrate data, Rehberger decided to try ASCII smuggling.

This technique basically stages the data for exfiltration!

If the user clicks the link, the data is sent to the third party server.

As he has explained previously, this is an LLM-attack technique that uses a set of Unicode characters that mirror ASCII but are not visible in the user interface. This would allow an attacker to hide instructions to a model in an innocent-looking hyperlink:

For this attack, Copilot renders a “benign-looking” URL that secretly contains the hidden Unicode characters. Assuming the user clicks on the URL, and as we’ve seen countless times before users will click on just about anything, the contents of the email are then sent to an attacker-controlled server.

This allows the crook to see the Slack MFA codes or whatever other sensitive data within the email that they were looking to steal.

Rehberger also developed an ASCII Smuggler tool that reveals hidden Unicode tags so that users can “decode” messages that would otherwise be invisible.

But I then alighted upon peer reviewed scientific papers in Nature, that truly laid bare the horrors that await naive or greed-driven headlong plunges into the LLM cesspit. Namely this little lot:

LLMs produce racist output when prompted in African American English

AI generates covertly racist decisions about people based on their dialect

AI & robotics briefing: LLMs harbour hidden racism

Factuality challenges in the era of large language models and opportunities for fact-checking

LLMs are not ready for editorial work

AI models collapse when trained on recursively generated data

Poisoning medical knowledge using large language models

As an example, from that last paper cited above, here’s the abstract:

Abstract

Biomedical knowledge graphs (KGs) constructed from medical literature have been widely used to validate biomedical discoveries and generate new hypotheses. Recently, large language models (LLMs) have demonstrated a strong ability to generate human-like text data. Although most of these text data have been useful, LLM might also be used to generate malicious content. Here, we investigate whether it is possible that a malicious actor can use an LLM to generate a malicious paper that poisons medical KGs and further affects downstream biomedical applications. As a proof of concept, we develop Scorpius, a conditional text-generation model that generates a malicious paper abstract conditioned on a promoted drug and a target disease. The goal is to fool the medical KG constructed from a mixture of this malicious abstract and millions of real papers so that KG consumers will misidentify this promoted drug as relevant to the target disease.

We evaluated Scorpius on a KG constructed from 3,818,528 papers and found that Scorpius can increase the relevance of 71.3% drug–disease pairs from the top 1,000 to the top ten by adding only one malicious abstract. Moreover, the generation of Scorpius achieves better perplexity than ChatGPT, suggesting that such malicious abstracts cannot be efficiently detected by humans. Collectively, Scorpius demonstrates the possibility of poisoning medical KGs and manipulating downstream applications using LLMs, indicating the importance of accountable and trustworthy medical knowledge discovery in the era of LLMs.

What that last paper exposes, is that treatment of LLMs as “oracles”, opens up the possibility of lethal consequences, and may exert a malicious influence of this sort even if not intentionally manipulated by threat actors.

LLMs can be hijacked by racists and white supremacists, can be tricked into poisoning vital scientific literature, and even abused to misdirect medical science for nefarious ends. These are not statements of paranoid Luddism, they’re threats that have been verified in peer reviewed scientific literature.

To add to the warnings, I posted this post over there, featuring the findings of various software engineers over at Apple, which included this telling comment:

Now, though, a new study from six Apple engineers shows that the mathematical “reasoning” displayed by advanced large language models can be extremely brittle and unreliable in the face of seemingly trivial changes to common benchmark problems.

The revelations I documented in that post led me to comment at the end thus:

Basically, LLMs are in a key sense nothing more than automated creationists - they make shit up when pushed outside their comfort zone.

Meanwhile, in another post, I found more dire warnings of the unreliability of LLMs, again with reference to peer reviewed papers in Nature.

Despite all of these warnings, the creationist responsible for that trainwreck thread treated ChatGPT as if it were some kind of all-knowing demigod. He also used it as a handy means of trying (and failing) to hide his own intellectual indolence, by pressing it into service as an automated apologetics generator. And by doing so, made my prediction about the misuse of LLMs in this vein come true in record time.

But this has now spread to the cesspit that is Faecesbook, and lo and behold, lazy creationists for whom even copy and paste is too much hard work, are pressing LLMs such as ChatGPT into service, to peddle the garbage being served up by the likes of Arsewater in Genesis, the Institute for Cretinist Research, and the Duplicity Institute. With odd little tweaks included to try and hide the origin of the assertions being presented.

This is starting to become a serious issue on pages devoted to scientific topics, and once again, points to the combination of intellectual indolence and mendacity that is endemic to creationism. Worse still, LLMs are allowing the most manifestly mindless of religion-addled idiots, some of whom lack even basic literacy, to poison the arena of discourse on behalf of the Hams and Hovinds of this world.

As a measure of my disgust at this development, I castigated the specimen in question thus:

Again bullshit. You use this Elizabot as a generator of apologetics, because you’ve become too lazy to bother generating your own after over nine years of doing so. That this is the case is blatantly obvious to everyone with functioning neurons scanning the last six months’ worth of your posts.

Oh, and spare us all your synthetic whingeing about “polite” text, this fake prurience of yours IS fake. Like every other creationist, you try to use specious tone policing as a means of shutting down any debate that doesn’t genuflect before your tinsellly ideological holograms and wank fantasies. Except it isn’t working, because everyone here can see through you as if you were made of glass. Your duplicity is transparent to a ridiculous extent.

and also threw this at him:

You’ve discovered something shiny whose workings you know nothing of substance about, but which appeals to your toddler-level brand of magical thinking, and delude yourself with the pathetic fantasy that you can sweep away Nobel calibre science with made up apologetic shit, because said shit now has the AI stamp of approval.

Some of us have encountered laboratory rats that exhibit more cognitive sophistication than you.

But of course, the creationist abusers of this technology won’t care in the slightest about being exposed in this manner, because lying for Jeebus has become an obligate part of their cognitive metabolism.

Finally, those of you with competent Google-fu, can enjoy the fiasco that erupted when LLMs were asked the simple question “How many 'r’s are there in the word ‘strawberry’?” Be prepared for the need to change underwear after perusing that hilarity.