Large Language Model (LLM)

A large language model, or LLM, is a model trained on very large amounts of text whose only real skill is predicting the next fragment of text. Answers, summaries, translations and code all emerge from that one skill, because each of them can be framed as a continuation of text.

How does an LLM work?

Text is split into tokens, roughly word fragments. For each position the model calculates which token most probably comes next, picks one, and continues. The answer takes shape word by word.

What the model can do sits in its weights, and those come from training. That explains two properties you run into constantly. It knows nothing that happened after its training cut-off. And it has no separate store of facts, which is why it phrases a hallucination as fluently as a correct answer.

How much text can be considered per request is bounded by the context window. Everything the model should know about your case has to fit in there.

What an LLM is good at, and what it is not

It is good at language: rephrasing, summarising, pulling structured fields out of unstructured text, moving between languages, holding a tone. Those are the tasks where it reliably saves time.

It is weak at anything requiring precision rather than plausibility. Arithmetic, exact quotation from memory, counting. Sub-tasks like those should be handed to tools rather than to the model, which is exactly what integrations such as MCP exist for.

The practical response to that boundary is simple: supply knowledge rather than expect it. A model answering from documents you provide is working in a RAG setup, and those answers can be verified.

What matters when choosing one

Not the leaderboards. For most mid-market applications the available models are good enough, and the difference between a prototype that convinces and one that does not comes down almost every time to the data integration.

Three other questions matter more. Where the computation happens, meaning which data leaves the building, which is also the choice between cloud and on-premise AI. What typical requests cost in production rather than in testing. And how much work it would be to switch models later.

How the costs add up

Billing is per token, in both directions: everything going in and everything coming out. Output is considerably more expensive than input.

That produces a quirk which breaks a lot of forecasts. A conversation's history is resent in full on every request so the model has the context. A long conversation therefore pays for its own beginning over and over. In an agentic workflow with twenty steps that is the largest cost item, not the answer itself.

Two levers work reliably. Trim instead of append: anything the next step does not need belongs out of the context window. And separate the tasks: a small, cheap model triages, and the large one only answers the cases that need it.

A dependable estimate needs only simple arithmetic. Take the typical request, count input and output tokens, multiply by the expected number of cases per month. That figure regularly lands an order of magnitude away from what a round of casual testing suggests.