--- title: "From Prompt to Token: AI Agent Building Blocks" author: "Daniel Gorld" author_role: "Consulting Director, cbs CX — The cbs Group Salesforce Consultancy" author_url: "https://cx-waves.com/about" publisher: "CX-Waves" canonical_url: "https://cx-waves.com/nodes/ai-agent-building-blocks" date_published: 2026-06-01 date_modified: 2026-09-20 language: en --- # From Prompt to Token: AI Agent Building Blocks Source: Daniel Gorld, CX-Waves — https://cx-waves.com/nodes/ai-agent-building-blocks (published 2026-06-01, updated 2026-09-20) ## What are the core technical concepts a B2B decision-maker needs to understand before implementing AI agents? * **Prompt (System Prompt):** This defines the AI agent's "job description," outlining its purpose, allowed actions, prohibited behaviors, and communication style. It's crucial for establishing the agent's operating parameters and requires ongoing maintenance. * **Skill (or Action):** Skills are the specific capabilities an agent possesses, such as checking delivery status or identifying part numbers. These must be defined based on clear process logic rather than technical implementation details. * **RAG (Retrieval-Augmented Generation):** This mechanism allows an agent to retrieve relevant information from internal knowledge bases before generating a response. It ensures accuracy by providing the model with current, company-specific data, making data quality paramount. * **Memory:** Memory allows an agent to retain information from current and past interactions. Short-term memory maintains conversation context, while long-term memory stores persistent data like customer preferences, often requiring integration with systems like CRM and adherence to data protection regulations. * **API Access:** APIs enable an agent to interact directly with internal systems, such as ERP or CRM, allowing it to perform actions beyond just answering questions. This capability necessitates careful governance of system access and privileges. * **MCP (Model Context Protocol):** MCP is an open standard that facilitates seamless communication between AI agents and external tools, similar to USB for hardware. This standard reduces integration effort and enhances flexibility by providing a reusable connection framework. * **Tokens:** Tokens are the unit of consumption for AI agents, representing fragments of words used in processing. Understanding token consumption is essential for cost estimation, as pricing scales with context length, multi-step processes, and the choice of AI model. ## What an AI Agent Actually Is — and What It Is Not An AI agent functions as a system capable of independently understanding a task, formulating a plan, and executing actions, often involving multiple steps and interactions with various data sources and systems. Unlike a chatbot that primarily provides answers, an AI agent actively performs tasks. The effectiveness and scope of an agent's actions are determined by seven foundational architectural components. For example, a mechanical engineering company might deploy a service agent to autonomously manage customer inquiries regarding spare parts, delivery, and technical issues. ## Building Block 1 — Prompt (System Prompt) The System Prompt serves as the agent's fundamental instruction set, akin to a job description. It dictates the agent's identity, permissible actions, restrictions, and communication style. This internal instruction is critical for guiding the agent's behavior and is a foundational design element. For instance, a service agent's System Prompt might specify that it handles only spare parts and delivery inquiries, prohibits price negotiations, mandates escalation of technical safety questions, and requires all responses to be in English regardless of input language. A poorly defined or outdated System Prompt is a common cause of agent malfunction; thus, its ongoing maintenance and clear formulation are more critical than the underlying AI model's capabilities. ## Building Block 2 — Skill (or: Action) A skill represents a defined capability an agent can execute. These are structured instructions that dictate specific actions an agent can perform under given circumstances. Examples include checking delivery status, identifying a spare part number, or initiating an escalation process. In the service agent example, its skills might be limited to checking delivery status, suggesting appropriate spare parts, and notifying a service technician for unresolved issues, with any other request falling outside its scope. Crucially, skills should be defined based on existing process logic within the organization rather than purely technical specifications; a clear understanding of automatable process steps is a prerequisite for effective skill development. ## Building Block 3 — RAG (Retrieval-Augmented Generation) RAG is a principle where an agent retrieves relevant information from a knowledge base before formulating a response. This means the AI model does not rely on its internal training data for specific company information. Instead, it queries documents, articles, or database entries pertinent to a user's question and incorporates this retrieved information into its generated answer. The knowledge base can comprise various structured text documents like process descriptions, technical manuals, or FAQs. The quality of this information—its currency, consistency, and retrievability—is paramount. For instance, if a customer asks for a specific gasket ring for a machine, the agent uses RAG to search spare parts catalogs and technical manuals in real-time to provide an accurate response. An agent lacks a high-quality knowledge base will produce inaccurate or fabricated information, highlighting that effective RAG implementation often involves significant data maintenance rather than just agent deployment. ## Building Block 4 — Memory Memory allows an AI agent to retain information from interactions, distinguishing it from RAG, which accesses static documents. RAG is like consulting a manual, while memory is akin to experiential knowledge. Agent memory exists in two forms: short-term and long-term. Short-term memory covers the context of the current conversation, ensuring continuity within a session. Long-term memory stores information across multiple conversations, such as customer history or preferences, analogous to a persistent database. For example, short-term memory prevents customers from repeating details within a conversation, while long-term memory can recall a customer’s preferred language for future interactions. This memory is stored externally and retrieved by the agent rather than being integrated into the AI model itself. Integrating long-term memory directly with CRM systems offers advantages in control and compliance. Since memory can contain personal data, its implementation must comply with data protection regulations such as GDPR, requiring defined storage locations, retention periods, and transparency. ## Building Block 5 — API Access For an AI agent to perform actions beyond merely answering questions, it requires API (Application Programming Interface) access to various internal systems. APIs are standardized connections that allow different software systems to communicate and exchange data. In a service scenario, API access enables the agent to check delivery statuses in an ERP system, create service cases in a CRM, or trigger priority alerts in a field service system. The critical consideration for decision-makers is not merely the existence of APIs but rather which systems will be opened to the agent, the specific level of access granted, and who maintains governance over these access permissions. API access inherently involves questions of security and operational control. ## Building Block 6 — MCP (Model Context Protocol) MCP, or Model Context Protocol, is an open standard designed to standardize how AI agents interact with external tools and systems. It functions similarly to USB, providing a unified connector system that eliminates the need for bespoke programming for each connection. Systems supporting MCP can seamlessly integrate with any agent that adheres to the same standard. For example, instead of custom integrations for ERP, CRM, and spare parts catalogs, an agent can use MCP-compliant connections that are reusable once configured. This standardization significantly reduces integration effort and lessens proprietary vendor lock-in. While still developing, the growth of MCP support across existing systems directly impacts the long-term flexibility and integration costs of AI agent projects. ## Building Block 7 — Training Your Own Model (Fine-Tuning) Fine-tuning involves further training an existing AI model using proprietary data to enhance its performance for specific tasks or domain-specific needs. While seemingly attractive, fine-tuning is often unnecessary for most enterprise applications. The majority of business requirements can be met through effective data integration via RAG and precisely defined skills. Fine-tuning becomes pertinent only when dealing with highly specialized terminology not covered by general models, when strictly uniform output formats are required, or due to demanding regulatory mandates. The general rule is that organizations with frequently updated information benefit most from RAG, whereas fine-tuning is reserved for fundamental and permanent changes to the model's behavior. For most B2B initiatives, fine-tuning represents a higher cost with potentially marginal benefits compared to a well-configured RAG system. ## What Does It Actually Cost? — Token as the Unit of Consumption AI agents are typically not licensed through flat fees but are billed based on consumption, measured in units called tokens. A token roughly corresponds to a word fragment, about three to four characters long, meaning a typical sentence might consume around 15–20 tokens. For example, a standard service inquiry could consume between 500 and 2,000 tokens, depending on its complexity. At current market prices, which are fractions of a cent per token, a single inquiry costs barely anything. However, these costs become significant when scaled to high volumes and complex interactions. Three primary factors influence token consumption and, consequently, the cost of using AI agents: - **Context length:** The more documents an agent must process and consider per inquiry, the higher its token consumption, even for short user questions. - **Multi-step processes:** Agents executing multiple skills or steps in sequence will accrue tokens for each step, increasing overall consumption. - **Model choice:** More sophisticated AI models typically have a higher per-token cost. However, not all tasks necessitate the most advanced models; less complex classification tasks can often be handled by more cost-effective options. The strategic question is not merely the direct cost of an agent but identifying which business processes yield an economic return from automation at anticipated usage volumes. An agent handling numerous repetitive inquiries daily demonstrates swift ROI, whereas one that frequently searches extensive documents but is rarely utilized may not be economically viable. ## The Real Project Work Begins Before the Agent The effectiveness of all seven AI agent building blocks hinges on prerequisite conditions: clearly defined processes, meticulously maintained data, and well-governed system access. An AI agent serves to amplify efficient processes, but it cannot rectify existing operational deficiencies. A common pitfall in agent projects is the mistaken belief that the agent will resolve pre-existing organizational issues, such as unstructured data, ambiguous responsibilities, or unresolved system access problems. Successfully implementing AI agents requires prior foundational work in process optimization and data management. ## What Decision-Makers Should Assess Now Before committing to an AI agent project, decision-makers should thoroughly evaluate several key areas to ensure a robust and effective implementation. Firstly, they must identify which workflows are genuinely automatable and which intrinsically demand human judgment. Secondly, defining who is responsible for creating and continually updating the agent's behavioral framework, or Prompt, is critical for consistent performance. Thirdly, an assessment of the organization's data sources is necessary to ensure they are structured, current, and easily retrievable for RAG. Fourthly, decision-makers need to establish where conversation data (Memory) will be stored and confirm its compliance with data protection regulations like GDPR. Fifthly, it is crucial to determine which internal systems the agent will access and establish clear ownership and governance over those access rights. Sixthly, a comprehensive cost analysis is needed to identify which processes will be economically viable given expected usage volumes. Finally, a robust governance framework must be put in place to monitor the agent's performance and behavior according to defined criteria. Addressing these questions upfront lays the groundwork for sound technological decisions and prevents investment in infrastructure for processes that are not yet ready.