Analogies for AI
Analogies help explain (or sell) new concepts by relating them to something familiar based on a perceived similarity. This is incredibly helpful when trying to understand complex topics like AI, which can be abstract and difficult to grasp. The risk, however, is that analogies can oversimplify or misrepresent the concept they are trying to explain. By focusing on similarity, we may obscure aspects in which the two concepts differ.
In this note we explore various analogies used for AI, but some disclaimers are in order from the get-go. Firstly, the term "AI" is used to refer to a wide range of technologies, so when people talk about "the" AI, this is inherently ambiguous. By extension, the analogies used for explaining AI tend to highlight one specific aspect or subfield of AI and obfusciate others. This is why I want to scope this note a bit and focus on generative AI, and more specifically on large language models (LLMs). Secondly, to make things meta, the term "AI" is itself an analogy with human intelligence. By calling it "artificial" intelligence, we relate it to "real" human intelligence. Consider for example the following definitions of AI:
- "(...) the science of making machines do things that would require intelligence if done by men" (Marvin Minsky)
- "The art of creating machines that perform functions that require intelligence when performed by people." (Kurzweil, 1990)
So in order to explain what's so "intelligent" about AI, these definitions refer to what we consider to be intelligent behavior in humans. A pervasive computational model in the field of machine learning is the so-called artificial "neural network", where digital "neurons" are connected in layers and trained in such a way that they can perform certain tasks that we consider to be intelligent, such as recognizing objects in images. Each of these computational neurons take inputs from other neurons and "fire" based on an activation function in order to influence other neurons, loosely inspired by how biological neurons work in the human brain. Even though some forms of neural networks, such as convolutional neural networks, are inspired by the visual cortex in the human brain, they are not a direct representation of how the brain works. Even though the term "(artificial) neural network" suggests a resemblance to the human brain and human intelligence, modern practicioners of this form of machine learning learn little to nothing about how human intelligence works. A lot of modern talk about artificial general intelligence (AGI) has no sensitivity whatsoever to the limits of these AI analogies and instead leverage them to promote the idea that we are creating AI that equals or surpasses human intelligence. This is smart, of course, because as we said earlier, analogies can help sell products to the masses!
Funnily enough, these analogies between artificial and human intelligence that are pushed by tech enthousiasts and big players in the AI-industry, are often met with skepticism and dare I say "counter analogies" by cognitive scientists that study human intelligence, for example by calling large language models "stochastic parrot" or "autocomplete on steroids". In other words, the field of AI can be construed as a battefield of analogies!
AI as a robot
Science fiction imagines AI as a robot that is logical, rational, calculating, not creative, and devoid of "illogical" emotions. This analogy is still present to some degree in science fiction books and films, although modern films rather flirt with the uncanny tension between a logical AI interacting on an emotional level with humans (e.g. Ex Machina, Her; both movies I haven't seen by the way). Interestingly enough, this analogy is at odds with the advent of generative AI, whose strength is rather being creative and being able give surprising answers. Because large language models predict the next word in a sentence based on the previous words, they are able to generate new sentences that are not necessarily logical or rational, even though a lot of effort is put into trying to make these models reason more logically.
As Ethan Mollach explains in his blog post Blinded by Analogies:
These models only predict text outputs from prompts, and as such, they aren’t logical or accurate at all. They lie constantly and they produce hallucinatory wrong answers. Yet, in talking to many first-time users, they often find LLMs frustrating because their analogies are to robots and computers. They usually ask for factual information first (such as their own biography) and quickly find that ChatGPT confidently makes up false information. What good is a robot that isn’t accurate or logical?
AI as a stochastic parrot
The most well-known metaphor that counters the perceived "real" intelligence of particularly large language models is the "stochastic parrot", coined by Emily M. Bender in the paper On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 (2021). The core of the metaphor is that, like parrots, large language models (LLMs) can mimic human language but without really understanding its meaning or context. In parrots this is of course quite an impressive feat, but we don't claim that parrots have human level intelligence. In the case of large language models the parroting has been scaled up tremendously to the point where the mimicing is so impressive that we don't recognize it as parroting anymore. But, so goes the argument, if you consider how large language models are essentially trained to predict the next word based on the surrounding context, they are not really "thinking" or "reasoning", but rather are an "autocomplete on steroids". Because LLMs parrot the way people actually write and speak, they fool us into thinking they work like people do.
The analogy of the stochastic parrot downplays the experience that we can often get useful and surprising idea from talking to LLMs. But then again, a lot of human creativity is after all also based on recombining existing ideas and concepts in new ways. It's mainly the human ability to generate truly new and paradigm-shifting ideas that still seem to be out of reach for LLMs (but to be fair, this quality is rare and mysterious in humans too).
Ethan Mollach: "Creative recombination is both a use case where errors and weirdness are expected, and also one where the blurry, compressed version of the web stored by LLMs can lead to unexpected outcomes."
AI as search engine
When I'm programming I personally don't Google and search the top StackOverflow results anymore, but instead I ask LLMs for help. I'm not the only one, see for example Alphabet Shares Take 120$ Billion Blow as Search Warning Blare. In other words, we use LLMs as a search engine nowadays. The analogy of LLMs as a search engine seems obvious, because you ask the LLM a question and it returns information. That's exactly what you expect of a search engine, right? However, LLMs like the GPT models behind ChatGPT are generative AI and not search engines, meaning they don't return actual information from the internet (except from rare cases of "memorization" where the output corresponds near 1-to-1 with some input), rather they generate a new answer based what's known from the training data. The "generative" in "generative AI" is a pharmakon: it is it's core strength, but also its core weakness when it comes to producing factually correct answers. When undesired, we call them "hallucinations". When we use AI as a search engine, we need to become smart about recognizing when the AI is hallucinating and when it is actually producing useful information, but this is actually quite difficult to do. What makes fact checking LLM outputs hard is that they are quite often sort of right, that is, the untruth is often intermingled with the truth. LLMs are designed to sound plausible, so they can often produce plausible sounding nonsense. You can of course ask an LLM nicely to produce source references, but these references themselves may be hallucinated too, or be incomplete, irrelevant, or not actually the foundation for the produced answer. And even if the source is correct, it's still hard to asses whether essential information is omitted. It's hard to fact check what's not there if you're not already an expert on a topic.
A lot of effort goes into making LLMs speak the truth, but the core of the matter is that the stochastic nature of LLMs means that they are not designed for respecting the laws of reasoning and truth.
The LLMs themselves 1) do not know the difference between truth and untruth, 2) don't have a strict reasoning process to guarantee the correctness of their inferences and by extension 3) they cannot check the correctness of their own claims.
With a lot of effort, as it currently stands LLMs can at best be mostly correct most of the time. Techniques such as Retrieval Augmented Generation (RAG) are used to combine LLM with retrieval systems, which allows the LLM to access external knowledge bases and databases to produce more accurate and relevant answers. Because the knowledge needs to be retrieved from a database, this is a bit more similar to enterprise search, but still requires work to keep the database up to date and to prevent hallucinations. Tool calling is a way to leverage existing APIs to include dynamic data in the LLM's output. Because one of these tools includes doing an actual search on the web, this actually starts closing the gap between LLMs and search engines.
Analytical Engine
Ethan Mollach proposes the "Analytic Engine" in analogy to Charles Babbage's Analytical Engine:
Taking inspiration from the pioneering inventor Charles Babbage and his creation of the first analog computers, I have been using a new analogy, referring to modern-day AIs like Bing as Analytic Engines. While imperfect, this analogy holds some truth. Like Babbage's analog computers, AIs such as Bing have the ability to bring together an immense amount of information and perform calculations that were previously impossible. However, they can also be challenging to use and understand, often shrouded in mystery and confusion. It is a flawed analogy, but so are all analogies.
This analogy is mainly flawed because LLMs cannot do proper inference to generalize beyond their training data. LLMs are not systematic in their thought. Consider for example the Reversal Curse, which shows that LLMs trained on "A is B" do not automatically generalize to the reverse direction "B is A". A typical failure was "Who is Tom Cruise's mother?" (A is B) vs "Who is Mary Lee Pfeiffer's son?" (B is A). GPT-4 performed 79% correct on the first question, but only 33% on the second question, indicating that the reverse relation is not learned. This is a case where "A is B" is well represented in the training data, but "B is A" is not. If you understand the world and understand the mother-son relationship, you can easily answer the question "Who is Mary Lee Pfeiffer's son?" based on the knowledge implied in the first question. The reversal curse is alleviated a bit if the memorized fact is included in the prompt.
Gary Marcus summarizes this point:
To begin with, this kind of failure actually goes back to my own 2001 book Algebraic Mind, which focused extensively on the failure of earlier multilayer neural networks to freely generalize universal relationships, and which gave principled reasons to anticipate such failures from these architectures. None of what I raised then has really been adequately addressed in the intervening decades. The core problem, as I pointed out then, is that in many real world problems, you can never fully cover the space of possible examples, and in a broad class of heavily-data-driven systems like LLMs that lack explicit variables and operations over variables, you are out of luck when you try to extrapolate beyond that space of training examples. Was true then, still true now.
Neural networks "still remain pointillistic masses of blurry memory, never as systematic as reasoning machines ought to be".
Another great example is the simple addition of two big integers. I often explain to people that LLMs cannot really do math, but this is surprising to people because LLMs can produce correct results! People see a few correct results and think that the LLM can do math. Gary Marcus: "In math, when one make a conjecture, a simple counterexample suffices. If I say all odd numbers are prime, 1, 3, 5, and 7 may count in my favor, but at 9 the game is over."
AI as Copilot
The "AI as a copilot" analogy is used to describe how AI can assist humans in their tasks, enhancing productivity and decision-making. An AI Copilot is an AI-powered virtual assistant that helps you navigate the complexities of your work. It's particularly being marketed by Microsoft, which offers various Copilots for different tasks. ChatGPT can be seen as an all-purpose standalone copilot, but typically a dedicated AI copilot is integrated in other apps and has access to your data in these apps. Because it's integrated in the apps you use and has knowledge of your specific work context, it typically has a more specific use case than a generic (GenAI-powered) chat bot.
Copilots can help draft emails, summarize meetings, write code, generate reports, and so on. For example you could have a Copilot in PowerPoint that helps you create presentations. The promise: do your work more efficiently.
Perhaps by now this is a cliché: you are not replaced by AI, but you are replaced by someone using AI.
The Copilot analogy assumes that the humans remain the main pilot and AI the copilot.
Dangers:
- Data privacy, don't trust AI with data you wouldn't share with an external advisor as well
- If you enter sensitive data, it can be processed by a third party;
- and it may enter the training data of the AI, which means it can be used in the future by the AI to answer questions from other users that shouldn't have access to the information.
- This problem may already occur in your internal company Copilot if let's an employee asks a question and gets an answer that draws on confidential information from let's say the CEO.
- Automation bias en overreliance on AI
- skill erosion (e.g. AI used to write code, do people forget how to write code)
- biased or poorly informed decisions
- Unreliable output.
- Agent 2 Agent cooperation: how do you keep meaningful human control?
- Especially because we typically do not really understand how the AI works.
- and we often have unaligned expectations of what it can(not) do.
- AI is a very valuable tool, but you can only fact check it if you are an expert yourself as well.
- E.g. people say "coding is dead".
- But after a while you'll need even better engineers to separate the wheat from the chaff in the tremendous amounts of AI-generated content.
As AI agents increasingly "join the workforce", re-skilling may be necessary.
Agent framework.
But who's really the co-pilot? Beware of automation bias.
In het bijzonder omdat we gaan leunen op AI, die zoals we hebben gezien, fouten maakt.
belang van opleiding.
AI and work.
Sources:
- https://www.oneusefulthing.org/p/blinded-by-analogies (by Ethan Mollach)
- https://garymarcus.substack.com/p/elegant-and-powerful-new-result-that