"AI agent" is being used to sell everything from a chatbot with a new name to fully automated back offices. If you run a business and someone offers to build you one, it helps to know what is actually inside the box. There is no magic in it. An agent is a handful of ordinary parts wired together in a particular way, and once you can name the parts you can ask much better questions about any agent someone wants to sell you.
The model does the thinking, but only the thinking
At the centre of every agent is a large language model: the same kind of thing behind ChatGPT or Claude. Its job is to read what is in front of it, work out what should happen next, and say so in words.
That is all it does. On its own a model cannot send an email, look at your calendar or change a record in your CRM. It can only produce text. A chatbot is a model with a text box on the front. An agent is a model with the other five parts below wrapped around it, so that the text it produces turns into actions.
Which model you use matters less than people think. The model is the easiest part to swap out later, and the differences between the good ones narrow every few months. The parts around it are where the real work is.
The instructions are the job description
Every agent starts from a written brief, usually called a system prompt. It says who the agent is working for, what it is there to do, what it is not allowed to do, and how it should behave when it is unsure.
This is the part most people skim past, and it is the part that decides whether the agent is useful. A vague brief ("help customers") gets vague behaviour. A specific one ("you handle first replies to quote requests for a plumbing firm in Leeds; never quote a price; if the job sounds like an emergency, say so and hand it to a person") gets an agent that does one thing well.
If you are having an agent built, ask to read its instructions. They should make sense to you as a description of the job. If they do not, the agent will not do the job you have in mind.
The tools are what it can actually do
Tools are the bit that separates an agent from a chatbot. A tool is a function the agent is allowed to call: look up a customer, read the last ten emails in a thread, check availability in the diary, create a quote, send a message. Each one has a name, a description of what it does and what information it needs, and the model chooses which to use and when.
The rule we work to is that the agent should have exactly the tools its job needs and no more. An agent whose job is to chase unpaid invoices needs to read the ledger and send an email. It does not need the ability to delete anything, change prices or refund. Keeping the tool list short does two things: the agent makes fewer wrong choices, and the worst thing it can do on a bad day is limited.
Memory is what it knows beyond this conversation
There are two kinds, and they get confused with each other.
Short-term memory is the conversation so far plus whatever the agent has looked up during it. It is large but it is temporary, and it is the reason an agent can follow a thread of "actually, make that Thursday instead" without being told the whole story again.
Long-term memory is anything the agent can go and fetch: your price list, your past jobs, the last conversation with this customer, your house rules for how to reply. This usually lives in an ordinary database or document store, and the agent reads from it when it needs to. It is not stored "inside" the model, and it does not need to be. The model already knows how to write English; what it needs from you is the facts about your business, kept somewhere it can look them up.
The loop is what makes it an agent
Put the model, the instructions, the tools and the memory together and you still do not have an agent until you add a loop. It goes roughly like this: read the situation, decide on the next step, take it, look at what came back, decide again. Keep going until the job is done or something says stop.
That loop is the whole difference between answering a question and getting something done. A chatbot replies once. An agent chasing a quote can look up the customer, notice the quote went out nine days ago, draft a follow-up in your usual tone, check that nobody on your side has already spoken to them this week, and then either send it or put it in front of you for a nod.
The loop is also where things go wrong. An agent that cannot tell when it is done will keep going; one that is too eager to declare victory stops halfway. Building a good stopping rule is unglamorous and it is where a lot of the real engineering time goes.
The guardrails are what let you sleep
The last part is everything around the agent that keeps it in bounds: which actions need a person to approve them first, how many times it can try something before giving up, what it is allowed to spend, a log of every step it took and why, and a plain off switch.
For most of the businesses we work with, the sensible starting point is an agent that prepares work and a person who approves it. The agent drafts the reply; you press send. The agent proposes the diary change; you confirm it. Once you have watched it get a hundred of those right, you widen what it can do on its own. That order is not timidity. It is how you find out where the brief was wrong before it costs you a customer.
What to ask when someone offers you one
You now know enough to ask the six questions that matter:
- Which model, and how easily can it be swapped?
- Can we read the instructions, and do they describe the job as we would?
- What tools does it have, and is that list as short as it can be?
- Where does it get facts about our business from, and who keeps them up to date?
- How does it know when it is finished?
- What does it need our say-so for, and what does it log?
If the answers are clear, you are probably looking at something useful. If they are not, you are probably looking at a chatbot with a new name.