Classic software changes only when somebody changes it. An AI agent changes when anything around it changes, and that happens constantly. The model provider deprecates the version the agent was built on. The ERP gets an update and renames a field. A colleague in sales updates the price list the agent loads as context. An attacker finds a new phrasing that makes the agent forget its instructions. None of that is a bug in the agent, and all of it is a maintenance case.
This guide is the second part after operating AI agents. That one is about running the thing day to day: approvals, logs, reviews. This one is about the work that keeps an agent functioning over months and years, and about the contract that governs it. If you have yet to have an agent built, start under have a custom AI agent built.
Why an agent needs maintenance even when nobody touches it
Four sources of change act on every agent in production, and none of them is in your hands.
Models get deprecated and changed. Providers introduce new versions and take old ones off the network after a notice period; OpenAI's deprecations list is one example of how regularly this happens. Every deprecation is a forced migration, and a new model never behaves exactly like the old one. It phrases things differently, follows formatting instructions differently, judges edge cases differently. An agent sitting on a deprecated version has an expiry date.
Upstream systems change formats. The agent reads from CRM, ERP, ticketing and the inbox. An update renames a field, a supplier changes the layout of their order confirmations, an API adds a required field. The agent does not get an error, it gets different data, and it processes it with the same confidence as before.
Knowledge sources go stale. An agent that answers from documents is only as current as those documents. A price list from last quarter, a product description with the old feature set, a template signed by someone who has left the company: the agent finds all of it and uses it. How retrieval works technically is under retrieval-augmented generation; that it needs upkeep is in no manual and in every incident report.
Attack patterns change. Prompt injection, meaning instructions hidden in foreign text that the agent reads as its own, is the most cited weakness in the OWASP Top 10 for LLM applications. The patterns keep developing, and what the agent fended off at go-live says little about next year.
The four kinds of maintenance
Software engineering has distinguished four categories of maintenance for decades, and all four apply to agents, each with its own face.
- Corrective: fixing a fault the agent showed in production. For agents that almost always means a new case in the eval suite, an adjustment to prompt, context or rules, a run of the suite, a rollout.
- Adaptive: the environment changed and the agent follows. A model change, a renamed field in the ERP, a new API version, a new requirement from data protection.
- Perfective: the agent gets better or cheaper without anything having been broken. A smaller model for classification, tighter context, an action type that goes autonomous once the error rates allow it.
- Preventive: work against future failures. Checking deprecation lists, auditing permissions, extending the eval suite with the month's corrections, checking knowledge sources for currency.
A maintenance contract that covers only the first category covers the smallest part of the work. Most of the time goes into adaptive and preventive, meaning changes that no fault triggered.
Catching regressions: the eval suite as a regression test
A regression is a case the agent used to handle and suddenly does not. In deterministic software it shows up in the test suite. In an agent it shows up only if a test contains that case, and that is the eval suite: a few hundred historical cases with known-good results, sorted by error class, with thresholds per class.
Three rules turn the suite into a regression test. It runs on every change — to prompt, model, context, rules or upstream system — and no change ships until it passes. It grows with the corrections from production: every case a person corrected in approval mode is a candidate, and the month's cases get adopted in the review. And it measures per error class, not just on average. A model that scores better overall but worse on complaints with a stated deadline is a regression for a support agent, even if the headline number rises.
For jobs whose result cannot be checked mechanically, such as the tone of a reply, a second model acts as a judge. It is only as good as its calibration: its verdicts get reconciled against human verdicts regularly, and where they diverge the judge gets adjusted, not the person overruled. The engineering discipline behind it is in shipping AI that survives production.
A model migration in five steps
The model change is the most frequent and the most dangerous maintenance case, because it changes everything at once. This is how it runs here:
- Candidate against the eval suite. The new model runs the whole suite with an unchanged prompt. The result per error class shows where it is better and where it is worse than the old one.
- Adjust prompt and context. New models follow formatting instructions differently and often need different instructions for the same edge cases. Adjust until the suite reaches at least the old level in every class.
- Measure cost and latency. A model that answers better but takes twice as long or blows the monthly budget is not a candidate for classification, and perhaps is one for drafting replies.
- Shadow run. The new model runs in parallel on real cases for a few days without its results being used. The divergences from the old model get reviewed.
- Roll out with a rollback. Switch over, with the old combination of model, prompt and configuration one configuration change away. In the first week after the change, the sample going to human review goes up.
Anyone who has done these five steps by hand once understands why the eval suite is the most valuable artefact in the whole system. Without it a model change is a blind flight; with it, a routine of one to two days.
Maintaining context
An agent's context — the documents, datasets, templates and rules it loads — is part of the software and gets versioned like software. Every change to a price list, a template or a knowledge base the agent uses is a change to the agent, and it triggers the same eval run as a change to the prompt. In practice: the agent reads context from a repository or a source with a version, not from a shared drive where anyone can change anything. Sources get an expiry date, after which they surface in the review. And the question "which version of the price list did the agent use yesterday?" is answered in a minute, because it is in the log of every run.
Security maintenance
Three routines belong in every maintenance plan. The eval suite carries its own class of prompt-injection cases, meaning prepared emails and documents the agent must not act on, and that class grows with the known patterns. The agent's permissions get audited quarterly: does it still need every access it has, and has its user in the identity system accumulated rights nobody meant to grant? And the list of sub-processors, meaning the model providers and services data passes through, gets updated on every change and communicated to the data protection officer, as Art. 28 GDPR requires.
What a maintenance contract has to contain
A maintenance contract for an AI agent is a software maintenance contract with four additions: evals, models, context, and the split of roles under the EU AI Act. The ten points that have to be in it:
- Scope of service across all four kinds of maintenance, explicitly including adaptive and preventive work.
- Eval obligation: no change to prompt, model, context or rules ships without the eval suite passing, and the suite is extended monthly with the corrections from production.
- Model migrations are included, with shadow run and rollback, whether the provider forces them or the deployer wants them.
- Monitoring and alerts with defined thresholds for error rates, cost and outages.
- Response times per severity, with named people on both sides. An agent sending wrong commitments to customers is a different case from an agent that has not classified anything for an hour.
- Cost report monthly, with tokens and calls per agent, next to the agreed metric.
- Change budget: a fixed allowance for perfective changes per month, so that "small improvements" do not turn into individual quotes.
- Split of roles: the deployer holds identity, permissions, data and approvals; the vendor answers for code, evals, migrations and regressions. The AI Act classification is an annex.
- Documentation: data flows, sub-processors with regions, the agent's permissions, eval reports — all current and readable by the deployer.
- Exit and handover: code, prompts, evals, context and documentation live in the deployer's repository; on termination there is a handover meeting with defined contents, not a migration project.
If you are missing these ten points in a proposal, you do not have a maintenance offer. You have a hotline.
What maintenance costs
At twigbit, operating and maintaining an agent starts at €2,400 a month. That covers the four kinds of maintenance, the eval obligation, model migrations, monitoring, the cost report and the monthly review. What moves the price up is the number of integrations, the volume, and how much control you need — that is, how many action types stay approval-bound and how large the sample is. The alternative, taking operations in-house, costs one to two engineering days a month and more in months with a model change; it is provided for in the contract as a handover, should you choose it later.
Checklist for procurement
- Does the proposal include adaptive and preventive maintenance, or only fault fixing?
- Is there an eval suite, who owns it, and does it run before every change?
- Are model migrations included, with shadow run and rollback?
- Who holds identity, permissions, data and approvals? Is the AI Act classification an annex to the contract?
- Are there response times per severity with named people?
- Does a cost report arrive monthly, next to the metric?
- Does the code live in your own repository, and is the exit settled as a handover meeting?
- How is the sub-processor list maintained and communicated?
- Does the eval suite get extended with corrections from production, and who decides that?
- Is there a change budget, or is every improvement quoted separately?