AI agent
An AI agent breaks a goal into steps on its own, uses tools such as databases or inboxes to carry them out, and hands over to a person once its remit ends.
An AI agent is a software system that breaks a goal into working steps on its own, uses tools such as databases, APIs or inboxes to carry them out, and returns a result. A chatbot answers a question. An agent completes a task and hands over to a person once its remit ends.
How does an AI agent work?
An agent has four parts. The language model supplies the language understanding and the judgement. The tools give it access to your systems, a CRM, a ticketing system or a price list. The memory holds what has already happened in the case being worked on. The rules define what it may do without asking.
In operation this runs as a loop. The agent receives a goal, plans the next step, calls a tool, checks the result and decides whether to continue or hand off. That loop is what separates an agent from conventional automation: the goal is fixed in advance, the route to it is not.
For this to hold up day to day, an agent needs a clean boundary of responsibility. It has to recognise the cases it may not decide alone and pass them on. How that gets organised is covered under human-in-the-loop.
Why do AI agents matter for companies?
What separates an agent from the ChatGPT experiments of recent years is access. A chat window does not know your data and cannot trigger anything. An agent is connected to your systems and works where the work actually arises.
Bitkom reports that 36% of German companies with 50 to 500 employees now use AI, up from 20% the year before. The jump comes less from better models than from connecting them to systems that already exist.
The value rarely appears where a whole role is meant to be replaced. It appears in the cases that come up many times a day and touch the same two or three systems every time: sorting incoming requests, preparing quotes, checking invoices against orders, moving documents from an inbox into the right place. Work like that can be measured, and so can an agent that takes it over.
Where agents fail in practice
The model is almost never the problem. The MIT "GenAI Divide" study from 2025 found that around 95% of GenAI pilots show no measurable effect on results, and the reasons repeat.
Often the agent has no clean access to the systems where the work really happens. Or nobody defined what a good result looks like, so there is no way to judge afterwards whether it delivered one. Or the handover to a person is missing, and the first edge case leaves the whole thing stuck.
The route into production therefore runs through one narrowly scoped process with a clear definition of success. A broad platform meant to do everything fails more reliably than an agent that does one thing properly. There is more on this in Agentic software in the Mittelstand.
Agent or chatbot?
The two terms get used interchangeably, vendors included. A chatbot is an interface: a person asks, the system answers, and the exchange is over. An agent is a workflow. It often starts with no chat at all, triggered by an incoming email, runs several steps and leaves a result behind in a system.
An agent can have a chat window. It does not need one.
Related terms
AI pilot
An AI pilot tests on one real process whether an AI application holds up in daily operation. It ends in a decision, not a demo.
Large Language Model (LLM)
A large language model is trained on vast amounts of text to predict the next fragment. Answers, summaries and translations all emerge from that single capability.
Retrieval Augmented Generation (RAG)
RAG connects a language model to a search over your own documents. The model answers from the retrieved passages instead of relying on what it learned in training.