From Prompt to Token: AI Agent Building Blocks

From Prompt to Token: What B2B Decision-Makers Really Need to Know About AI Agents

Seven concepts. One decision. No vendor pitch.

Anyone exploring AI agents for their organization quickly encounters a set of terms that feel simultaneously technical and overhyped: Prompt, Skill, RAG, Memory, API, MCP, Token. None of them exist for their own sake — but each one represents a foundational design decision that determines whether an agent actually works in practice.

This article is not for developers. It is for decision-makers who want to understand what they are actually deciding — before the first budget is approved.

What an AI Agent Actually Is — and What It Is Not

An AI agent is not a search engine with a better interface. It is a system that independently understands a task, plans a response, and executes it — often across multiple steps, with access to data and systems. The critical difference from a chatbot: an agent acts, it does not just answer.

What it does, how well it does it, and which systems it interacts with — that is determined by seven architectural building blocks.

The running example throughout this article: A mechanical engineering company deploys a service agent that autonomously handles incoming customer inquiries about spare parts, delivery status, and technical issues.

Building Block 1 — Prompt (System Prompt)

The agent's job description.

Before an agent does anything, it receives a foundational instruction: Who am I? What am I allowed to do? What am I not allowed to do? How should I communicate? This instruction is called the System Prompt — and it is the invisible foundation of every agent.

In the example: The System Prompt specifies that the service agent only handles spare parts and delivery status inquiries, never engages in price negotiations, always escalates technical safety questions to a human — and responds in English even if the customer writes in another language.

The decision-maker question: Who writes this prompt — and who maintains it? A poorly formulated System Prompt is the most common reason agents behave incorrectly. The model is rarely the problem. The instruction behind it usually is. Prompts are not a one-time setup — they are a living document that evolves with the process.

Building Block 2 — Skill (or: Action)

What the agent can do.

A skill is a defined capability of the agent: "Check delivery status", "Identify spare part number", "Trigger escalation". Skills are not magic — they are structured instructions that define what the agent does in which situation.

In the example: The service agent has three skills — check delivery status, suggest the appropriate spare part, and notify a service technician for unresolved cases. Everything outside these three skills is outside its scope of action.

The critical insight: Skills must be defined in terms of process logic, not technical logic. Organizations that do not know in advance which steps in their service process are automatable cannot build a meaningful skill — regardless of the vendor.

Building Block 3 — RAG (Retrieval-Augmented Generation)

What the agent draws on when it responds.

RAG describes the principle that an agent looks something up before it responds — much like an employee who does not answer from memory but quickly checks the knowledge base first. The AI model itself is not trained on company data. Instead, relevant documents, articles, or database entries are retrieved at the moment of a query and passed to the model as context.

The knowledge base does not need to be a complex database. Structured text documents — process descriptions, technical manuals, FAQs — are often sufficient as a starting point. What matters is not the format but the quality: current, consistent, and findable.

In the example: A customer asks for the correct gasket ring for Machine Type X, built in 2019. The agent searches the spare parts catalog, service history, and technical manuals in real time — and responds based on those sources, not on general knowledge.

The direct consequence: An agent without a clean knowledge base produces inaccurate or fabricated answers — not because the model is poor, but because it has no reliable sources to draw from. The real project work is often data maintenance, not agent implementation.

Building Block 4 — Memory

What the agent remembers.

RAG and Memory sound similar — but they are fundamentally different. RAG accesses documents that humans created and that do not change through conversations. Memory, on the other hand, is what the agent builds through interaction itself: customer history, preferences, prior inquiries.

The analogy: RAG is the manual. Memory is experiential knowledge.

There are two levels to distinguish. Short-term memory covers the context of the current conversation — what was said in this session. Think of it like RAM: available while the conversation runs, gone when it ends. Long-term memory stores information across conversations — like a hard drive: it persists, but must be explicitly written and retrieved.

In the example: Short-term memory ensures the customer does not have to repeat in message 3 which machine they mentioned in message 1. Long-term memory remembers that this customer prefers communication in English — even though that preference is not written anywhere in a document.

Important for decision-makers: Memory is not stored inside the model — the model itself does not change through conversations. The information is stored in an external database that is retrieved automatically at the start of the next conversation. The most elegant solution for B2B is direct CRM integration: what the agent learns lands in the customer record — no parallel system, full control, and existing governance structures apply.

Since memory contains personal data — conversation history, preferences, behavioral patterns — it is not a technical detail but a data protection issue. GDPR compliance means: defined storage locations, clear retention periods, and transparency toward the customer about what is being stored.

Building Block 5 — API Access

Where the agent can intervene.

An agent that only answers is a good chatbot. An agent that acts needs access to systems — via APIs (Application Programming Interfaces), standardized connections between software systems.

In the example: Via API access, the service agent checks delivery status directly in the ERP system, creates a service case in the CRM when needed — and automatically triggers a priority alert in the field service system in the event of a critical machine shutdown.

The decision-maker question is not "Do we have APIs?" — it is "Which systems are we opening to the agent, with what level of access, and who controls that?" API access is always a governance question.

Building Block 6 — MCP (Model Context Protocol)

How the agent connects.

MCP is an open standard that describes how AI agents communicate with external tools and systems — comparable to USB: a unified connector system that prevents every connection from having to be individually programmed. A system that supports MCP can be connected to any agent that speaks the same standard — without proprietary one-off solutions.

In the example: Instead of building a custom interface for every system — ERP, CRM, spare parts catalog — the agent uses MCP-compliant connections that, once configured, are reusable. This reduces integration effort and dependency on individual vendors.

MCP is a young but growing standard. Which existing systems already support it is worth checking concretely during the evaluation process — it directly affects integration effort and long-term flexibility.

Building Block 7 — Training Your Own Model (Fine-Tuning)

When to adapt the model itself — and when not to.

Fine-tuning means further training an existing AI model on your own data, so that it performs certain tasks more precisely or in a more domain-specific way. This sounds appealing — and is rarely necessary in practice.

Most enterprise requirements can be addressed with clean data integration via RAG and well-defined skills. Fine-tuning becomes relevant when highly specific technical vocabulary exists that no public model covers, when outputs require a strictly uniform format, or when regulatory requirements demand it.

The rule of thumb: organizations with frequently changing information need RAG. Those who want to fundamentally and permanently change the model's behavior need fine-tuning. For most B2B projects, fine-tuning is the most expensive path with the least additional benefit compared to a well-configured RAG system.

What Does It Actually Cost? — Token as the Unit of Consumption

AI agents are not licensed as flat-rate software — they are billed by consumption. The unit is called a token: roughly a word fragment, approximately three to four characters. A typical sentence corresponds to around 15–20 tokens.

In the example: A typical service inquiry — customer question, catalog lookup, response generation — consumes between 500 and 2,000 tokens depending on complexity. At current market prices of roughly €1–3 per million tokens, a single inquiry costs fractions of a cent. The calculation becomes significant at volume and complexity.

Three cost drivers decision-makers should know:

• Context length: The more documents the agent searches per inquiry, the more tokens it consumes — even if the customer's question is short.
• Multi-step processes: An agent that executes multiple skills in sequence multiplies token consumption.
• Model choice: More powerful models cost more per token — but not every process requires the most capable model. Simple classification tasks can be handled by less expensive models.

The strategic question is not "What does an agent cost?" — it is "Which processes are economically worthwhile, and which are not?" An agent handling hundreds of repetitive standard inquiries daily pays for itself quickly. An agent that searches through many documents for each inquiry and is rarely used may not.

The Real Project Work Begins Before the Agent

What all seven building blocks have in common: they all require that processes are clearly defined, data is well-maintained, and system access is governed. An AI agent accelerates good processes — it does not repair broken ones.

The most common misallocation in agent projects is not the wrong technology choice. It is the assumption that the agent will complete the homework that was left undone beforehand: unstructured data, unclear responsibilities, unresolved system access.

What Decision-Makers Should Assess Now

• Process: Which workflows are genuinely automatable — and which require human judgment?
• Prompt: Who defines the agent's behavioral framework — and who keeps it current?
• Data: Are the relevant knowledge sources structured, current, and findable?
• Memory: Where is conversation data stored — and is that GDPR-compliant?
• Systems: Which systems should the agent access — and who owns the access rights?
• Cost: Which processes make economic sense at the expected volume?
• Governance: Who monitors the agent — and according to what criteria?

Those who have answered these questions can make a well-founded technology decision. Those who skip them are buying infrastructure for a process that does not yet exist.