AI Information Pollution
What are the consequences of content on the web increasingly being generated by generative AI?
Algorithmic mediation of our language
Consider the following two quotes on the algorithmic mediation of natural language (Kaplan-Linguistic Capitalism and Algorithmic Mediation):
Can we anticipate the global linguistic effects of the commodification of words and general algorithmic mediation of textual expression? Should we observe a general tendency toward more regular and less idiomatic forms of linguistic expression, more suitable to for purchase? This might be a reasonable hypothesis if we consider the effect of autocompletion algorithms in isolation and the growing tendency to use them as mediators in our textual expression. Unfortunately, these text-transforming algorithms are often based on statistical models of the texts encountered on the Internet, and these texts are no longer "pure" natural language resources, but have themselves already been altered by various algorithmic mediations. (Kaplan 2014, 61)
As Google bases its normalizing autocompletion etc. on texts of the internet, there is a risk when more and more content on the web is created by machines (secondary texts) rather than humans (primary):
Distinguishing primary from secondary resources is not a trivial issue. In certain cases, human readers can tell the difference based on a feeling of oddity induced by texts produced by algorithms. These oddities are due to specific lexical or syntactic biases of these algorithms that often produce syntactically correct sentences—that no natural speaker would ever write. Unfortunately, it is much more difficult for an algorithm to detect such distinction automatically. As the proportion of these secondary resources compared to primary resource increases, computational linguistic statistical models may include some of the vocabulary and expression forms originating from these algorithms. As a consequence, these expressions may be suggested to us as statistically plausible forms by autocompletion algorithms. If we follow this hypothesis, natural languages could progressively evolve to seamlessly integrate the linguistic biases of algorithms and the economical constraints of the global linguistic economy. (Kaplan 2014, 61)
The Kaplan article is written in 2014. Now, in 2023, generative AI has become available for use by the general public. Kaplan mentions that human readers can sometimes recognize computer generated texts by their oddity, but by now I'd claim the opposite: that often we can only recognize a text is computer generated if it's written too well or too grammatically correct. What is still human all too human are mistakes and idiosyncrasies.
Misinformation campaigns
In 2014 it was already a non-trivial problem to distinguish synthetic from human texts, by now it's even harder. Generative AI can generative texts, imitate the voices of specific individuals, generate convincing images and videos. This, combined with the general availability of generative AI, generated unprecedented possibilities for spreading misinformation.
- Promoting crypto coins
- Political influencing, smear campaigns
Mass-produced garbage content infiltrating
Consciously generated misinformation vs. AI-generated "non-information" (not sure if this term is appropriate, but I'm trying to distinguish this from information that designed to mislead people, although the effect may ultimately be similar):
- AI-Generated Books of Nonsense Are All Over Amazon's Bestseller Lists
- Deepfake specialist Nina Schick predicts 90% of all internet content will be produced by computers in about three years.
- Spotify Will Begin A Takedown Of AI-Generated Songs With Fake Streams
From here:
Unfortunately, that output can seem so plausible that without a clear indication of its synthetic origins, it becomes a noxious and insidious pollutant of our information ecosystem. Not only do we risk mistaking synthetic text for reliable information, but also that noninformation reflects and amplifies the biases encoded in its training data—in this case, every kind of bigotry exhibited on the Internet. Moreover the synthetic text sounds authoritative despite its lack of citations back to real sources. The longer this synthetic text spill continues, the worse off we are, because it gets harder to find trustworthy sources and harder to trust them when we do. (Emily & Hanna, 2013)
Nice quote from here:
Because large language models generate text probabilistically, based on the universe of existing content, mediocrity is built into the package.
Self-consuming synthetic training data
Kaplan explores the idea that computer generated texts will influence our natural language expressions over time.
An extension to this hypothesis is that the training data of large language models will increasingly include synthetic language generated by other large language models. What is the effect? Will this mean that LLMs will become worse over time, because small mistakes will be exacerbated?
- Self-Consuming Generative Models Go MAD
- The curse of recursion: training on generated data makes models forget (Model collapse)
Re-evaluation of human texts
If indeed our information system will be heavily polluted, then it could be the case that data sets with a provably high percentage of human-generated data will increase in value. In other words, while now we see a devaluation of human texts (AI can do it faster and often good enough for most use cases), we may see a re-evaluation once the quality of LLMs will deteriorate.