GenAI trends 2025
Predictions, trends, insights to use in presentations.
GenAI != AI
GenAI stack:
- Data management
- Infrastructure
- Models
- AI engineering tools
- Applications
Embedding AI (in tools / in devices)
In future years, a vast majority of applications will have AI embedded in them. We see this happening today with integrated Copilots and vendors suddenly offering integrated GenAI solutions (e.g. Snowflake, Miro). Easy way to quickly adopt GenAI in your workflows, with the downside of having less customizablity and possibly limited control over the privacy of enterprise data.
Past few years we've seen a trend of "bigger is better" in AI: bigger models, more data, more compute to train them, gives better results. An emerging trend is to also develop smaller more efficient LLMs models that are good enough for most tasks. Benefits: less compute to train, less energy consumption, less inference latency, less consumption costs for the end user. If this trend continues, LLMs may run on edge at some point. This trend may be reinforced by sustainability concerns as well.
From unstructured data to structured output.
"Unstructured data is the new structured data".
"Structured output" is becoming a standard feature of LLMs.
70-90% of all enterprise data is unstructured (emails, recordings, documents). Using GenAI it will become increasingly possible to extract insights from this data and combine it with structured data from more centralized data lakes.
LLMs are able to transform unstructured data into structured data. Newer features of LLMs include JSON schema validation, where you can even be sure that the extracted data is compliant with your data models so you can rely on it in your applications.
Example:
json_schema = {
"title": "joke",
"description": "Joke to tell user.",
"type": "object",
"properties": {
"setup": {
"type": "string",
"description": "The setup of the joke",
},
"punchline": {
"type": "string",
"description": "The punchline to the joke",
},
"rating": {
"type": "integer",
"description": "How funny the joke is, from 1 to 10",
"default": None,
},
},
"required": ["setup", "punchline"],
}
Now dear LLM, tell me a joke about cats:
{'setup': 'Why was the cat sitting on the computer?',
'punchline': 'Because it wanted to keep an eye on the mouse!',
'rating': 7}
Multimodal AI
Before: one model for speech-to-text, one model for text, one model for vision.
Now & future: a single multimodal model that can handle all of these modalities in the same model architecture.
Having this in a single model is powerful because it can leverage the relationships between the different modalities and give "holistic" outputs.
Normally a developer would have to use multiple models and stitch them together. For example, to ask a question about audio, you would first have to transcribe the audio to text with a speecht-to-text model and then ask the question about the text (with a text-text language model). In your applications this gives increased latency due to multiple round-trip requests, but typically also decreased performance because the mistakes in each of the disconnected models add up.
For example, instead of transcribing a podcast (with mistakes) and then piping the (imperfect) transcription into an LLM, you can now directly ask questions about the podcast. Particularly when the podcast is long and the answer can be found in a very specific part of the podcast, then the intermediate step of having the whole transcript outputs a lot of noise that the subsequent LLM has to filter out. Multimodal GenAI models are instead becoming quite good at finding the "audio in the haystack" for answering your question directly by paying attention only to the most relevant parts of the audio.
Another example: when asking a question about an image, let's say a picture of a product, the model can take into account the accompanying product description when formulating the answer.
Emerging audio use cases that become increasingly feasible with larger context windows:
- Real-time transcription and translation of spoken language.
- Q&A on informative audio content, such as podcasts.
- Meeting transcription + summarization
- Voice assistants
This opens up possibility of intuitive interfaces where information of different modalities are seamlessly integrated. Multimodal models could also improve the accessiblity of traditional apps, by approaching operations that are traditionally done with text or images with your voice.
The coming five years we'll see similar improvements for interacting with video as well.
Increasing memory
The coming years, such efforts are made possible by the increasing memory limits of LLMs.
GPT 3.5 Turbo instruct started with 4096 tokens context. Latest iterations of GPT4o have 128.000 tokens context, o1/o3 200.000 tokens. Pushing the frontier is Google's Gemini 1.5 accepting 1 million tokens and Gemini 1.5 Pro even 2 million tokens.
Quote from the docs, about what a million tokens look like in practice:
In practice, 1 million tokens would look like:
- 50,000 lines of code (with the standard 80 characters per line)
- All the text messages you have sent in the last 5 years
- 8 average length English novels
- Transcripts of over 200 average length podcast episodes
This unlocks whole new use cases:
- Summarize larger corpuses of text without relying on intermediate summarization steps where you lose source information.
- Q&A without requiring management of a database for RAG.
- LLMs for agentic workflows: an agent requires a stateful memory to keep track of their past and current tasks, relevant information about the world, and their own goals. Having a much larger context window accelerates these use cases.
It also allows "many-shot in-context learning", e.g. you can give a full manual or grammar book in the context of the model so that it can for example learn to translate one language to another without further training of the model. One-shot or few-shot learning entails you give an LLM a few examples of a task so it can learn to perform the task on new examples, without the need to fine tune the model. With a very large window, you can give a very large amount of examples in the model's context to facilitate the in-context learning of more complex tasks without having to fine-tune the model.
Large context in combination with multi-modal models also gives more options for processing of longer video and audio.
Agentic AI
From apps to agents.
LLMs have caused a revival of interest in agentic AI, where an AI agent is responsible for a task or a set of tasks in a given environment.
LLMs have proven to be very decent fuzzy reasoning engines to translate human questions or commands into relevant actions, especially since LLMs started having the "tool calling" mechanism built in.
We have:
- The user interface: the way the user interacts with the agent.
- The agent: the application that coordinates the interaction between the user and the AI model.
- The LLM as the "reasoning engine"
- The knowledge base which represents relevant information about the environment
- The action space i.e. the "tools": the set of actions the agent can take in the environment.
Bold claim from Satya Nadella december 2024 on the BG2 podcast, CEO Microsoft: "SaaS is dead" (SaaS: software as a service, hosted applications in the cloud such as the Microsoft Office 365 suite).
His idea: business applications are built on CRUD databases with embedded business logic. With the advent of LLM-powered AI agents, his vision is that the business logic will move to an AI layer (into Copilots, basically) such that business apps in this new paradigm are thin UI layers around AI agents.
Instead of manipulating database A with application A, and database B with application B, his vision is instead that the end user interacts with an AI that is smart enough to know which data in which database to manipulate to achieve the desired outcome.
The claim "SaaS is dead" is a bit dramatic of course. There will be a stronger focus on making sure systems are ready for AI integration and don't kid yourself: the AI layer is still software as a service, but the difference is that the AI layer is now the core of the application and the business logic is moved to the AI layer.
But this new paradigm opens a whole new can of worms in terms of data security, reliability, and compliance, if indeed we start delegating so much responsibility to LLMs that, as of yet, are definitely still hallucinating and still quite easy to manipulate. Would you trust an AI agent to write a SQL query based on your question and manipulate a database?
Agents can be implemented with varying degrees of autonomy, of course. I would not trust an agent to manipulate a database directly, but I would trust an agent to suggest a SQL query that I can review and check myself.
WIP:
Multimodal + realtime + tools: listen to live customer support conversation, interpret conversation against background knowledge about the person calling, suggest live responses to the support agent.
In a way: personalized and broader task automation.
Relation with RPA.
Single agent vs multi-agent
WIP: "Why it matters: By distributing tasks across multiple agents, organizations can reduce costs and improve scalability. Agentic designs support flexibility, making it easier to adapt AI solutions to different needs while ensuring robust performance. From cybersecurity to testing and reporting, agentic systems improve the effectiveness of GenAI in production environments."
Stronger reasoning
... why is it important?
Can help with assessing the reliability of AI models, and can help with debugging and understanding the model's decisions.
BYOAI and data safety
BYOAI: Bring your own AI (but preferably not external tools).
Increasing focus on safety & security of AI models. Precisely because the current trend is that your tooling uses embedded GenAI, you'll have a lot of different and opaque models running throughout your organisation. To manage data security, a best practice will be to impose AI security measures at a centralized level, e.g. at the level of an "AI gateway" that all (Gen)AI model offerings have to pass through. This may not be feasible for smaller organisations.
As enterprises start to rely more and more on (cloud-based) GenAI solutions, often from third parties, they also introduce new risks in terms of reliability and data security that are not managed by traditional security measures.
If you do not manage these risks by consistently monitoring the performance and compliance of your model offerings, you're bound to experience project failures and unwanted/unintended outcomes, possibly leading to financial or legal issues, reputation loss, suboptimal busines decisions, or social harm. Now that AI is becoming more and more integrated in business processes, the risks of AI failures are becoming more and more business risks and thus the coming years this will be a stronger focus point for organizations that aim for higher AI adoption.
EU AI act live and will take shape.
Domain-general vs domain-specific
The main strength of LLMs is that they have become very good in tasks across many domains (they are "general purpose" models), so good even that some parties claim LLMs are the route to AGI. Many enterprises however do not require LLMs to be good at everything, but rather want them to be very good in their specific domain/industry or even in a specific task or business function.
My take: talk of AGI (or even of superintelligence) is used to feed the hype and attract investors, but what enterprises actually want is a decent AI assistent that supports a particular business function without too much hallucination in that domain. The short future is not a single super intelligent AI agent for the whole organisation, but a multitude of domain-specific AI agents that are very good at their specific task.
Already now we see that to really get the most value out of LLMs, you need to tailor them to your specific context. Still: the beauty is that this can often already be done quite well with general purpose LLMs + inserting domain knowledge using prompt engineering or using methods like RAGS of tool calling.
This does not necessarily mean that the development effort of large parties moves away from training general purpose models. I think there will be a parallel undercurrent that focuses on smaller more domain specific models, with the added benefits of: less hallucination, lower compute, costs, and greener computing.
Example for NS: generative models that generate correct train models for commercials.
Synthetic data
In some highly regulated domains or domains where data is scarce or too expensive, GenAI can be used to generate synthetic data to be used in simulations. Another use is to create new training data.
Example: currently we use LLMs to evaluate the performance of other LLMs. This mechanism can also be used during the training procedure of LLMs.
Example: label images with multi-modal LLMs to quickly and cheaply generate custom training datasets. Normally data labelling for machine learning is very labour intensive and expensive.
Prediction: The coming few years we'll experience whether synthetic data will cause a deterioration in the quality of AI models. Genuine non-AI assisted content is scarce and will become an ever more valuable data commodity if it turns out synthetic data is not good enough.
Sustainability
Sustainability is a big theme in the coming years.
Almost everyone acknowledges that AI has a big environmental impact, but in practice I don't see coherent strategies for the sustainable use of AI.
It's quite hard to precisely measure the environmental impact of AI models. Risk of lot's of green washing and marketing. How are for example the costs of training and inference distributed ("amortized") over all users and a time scale?
Big data centers are pollutors, but the big parties have the resources to invest in specialized hardware that is incredibly efficient specifically for AI training.
Diversifying suppliers: not only for security reasons, but also to stimulate competition and innovation in the field of sustainable AI.
Move towards edge operations: running AI on edge devices is more energy efficient than running it in the cloud.
AI ready data & API first
Data-driven organizations can adapt GenAI more quickly.
Offering standardized data as a service via APIs accelerates the building of AI agents (e.g. using tool calling).
Example from practice: MS Copilots are "safe" assuming all your enterprise data is labelled correctly. This is rarely true in practice, so this is likely a blocker for taking advantage of Copilot. The risk: asking a question and getting an answer that is based on a confidential email from a colleage.
Upside: the desire for AI adoption is a motivator to get the basics in order!
TODO
https://www.gartner.com/en/articles/ai-ready-data
TODO
(Leo maakte een piramide voor de toekomst)
Sources:
- https://www.gartner.com/en/articles/3-bold-and-actionable-predictions-for-the-future-of-genai
- https://www.gartner.com/en/articles/what-it-takes-to-make-ai-safe-and-effective
- https://www.valtech.com/nl-nl/actueel/genai-10-trends-to-watch-in-2025/
- https://ai.google.dev/gemini-api/docs/long-context