AI integration without replacing existing systems

The first serious AI proposal a growing business receives is usually a replacement. A new platform with AI built in, migrating off the CRM, the helpdesk, or the line-of-business application that currently runs the operation. It is presented as modernisation, and the reasoning is that the existing systems were not designed for this.

The business impact of accepting that framing is large and largely avoidable. A replacement carries a migration, a retraining programme, a period during which nobody trusts the numbers, and the loss of years of accumulated configuration that encodes how the business actually works. All of that is spent before any AI capability is delivered, and none of it is spent on the thing that was supposed to create the value.

The engineering reality is that AI does not need to own your data to work with it. It is a component that reads from the systems you already run and writes back through the same interfaces every other integration uses. The systems of record stay where they are. What changes is that something new is now reading them.

Where AI fits inside an existing business

It is worth being concrete about what is actually being added, because the marketing around this makes it sound like a new kind of system. It is not. In architectural terms, an AI model is a service you call over an API that takes text or images and returns text. It holds no state between calls. It has no access to anything you do not give it in the request.

That last point is the one that resolves most of the anxiety about replacement. A model cannot read your CRM. It reads what your integration retrieves from your CRM and includes in the request. Your CRM remains the system of record, your access controls still apply, and the AI component sits alongside the other things that call your APIs.

What actually changes when you add it

One thing, and it changes a great deal. Conventional integration assumes determinism: given the same inputs, a function returns the same output, so it can be tested with assertions and trusted thereafter. A language model does not offer that guarantee. Ask it the same question twice and the answers may differ in wording, in structure, and occasionally in substance.

Almost every practical consequence follows from that. It is why AI features need evaluation against a set of representative cases rather than unit tests with fixed expected outputs. It is why retrying a failed call does not reliably fix anything, unlike a network timeout. It is why irreversible actions need a human in the path. And it is why the output has to be validated before it is written anywhere, in the same way you would validate data arriving from an untrusted third party.

The four integration patterns

Almost every AI integration in a business context is one of four shapes, or a combination of them. Naming them makes the build-or-buy conversation much shorter, because most requirements map cleanly onto one.

The ordering is deliberate. Each row costs more to build and own than the one above it, so the honest sequence is to check whether the capability already exists in a system you pay for, then whether an orchestration platform can do it, and only then to write code. This mirrors the progression described in the custom CRM article, and it holds for the same reason: the cheapest capability is the one somebody else maintains.

The fourth row is not an alternative to the others but a modifier on them. Any of the first three can be built with an approval gate in front of the action, and where the consequence of being wrong is material, it should be.

Connecting a model to your systems

Under the API orchestration pattern, the code that retrieves context and applies results is ordinary integration work: authenticated calls to systems you already run, subject to the same practices as any other integration. The Model Context Protocol has emerged as a standard for this connection layer. MCP describes itself as an open-source standard for connecting AI applications to external systems, and it is supported across a range of AI clients and development tools, which means a connector built once can be reused rather than rebuilt per vendor.

Whether you adopt it or write direct API calls, the architectural point is unchanged: the model asks your systems for information through an interface you control, and you decide what that interface exposes. A connector that can read customer records but not delete them is a configuration choice, and it is the right kind of choice to make explicitly rather than by default.

Retrieval is the part that determines quality

Most disappointing AI deployments in a business context are not model failures. The model answered correctly given what it was told; it was told the wrong things, or not enough of them.

Retrieval-augmented generation, usually shortened to RAG, is the standard approach to this: rather than expecting the model to know anything about your business, you retrieve the relevant documents or records at question time and include them in the request. The model's job becomes reading and summarising material you supplied, which is a task it does well, rather than recalling facts it was never given, which is a task it does badly and confidently.

The consequence is that the quality ceiling is set by retrieval, not by the model. If the answer to a customer's question lives in a document nobody has updated in three years, or in a folder the search index does not cover, no model will produce a correct answer. This is the same data quality problem described in the migration article, arriving in a different context: an AI assistant is an unusually effective way of discovering that your internal documentation is out of date, because it will cheerfully quote the stale version.

Choosing where the model runs

This decision is usually framed as a capability comparison, and that framing is a trap. Model rankings change frequently, and an architecture that depends on which model performs best in a given quarter is an architecture that needs revisiting every quarter. The durable decision is about where inference happens and what leaves your estate, because that is constrained by regulation and contracts rather than by benchmarks.

Self-hosting is chosen more often than it is justified, usually on the reasoning that it must be safer. It moves the security burden onto you rather than removing it, and it adds an infrastructure commitment that has to be capacity-planned and kept current. It is the right answer when data genuinely cannot leave the premises and the wrong answer when it was chosen because hosted felt uncomfortable.

Read the data terms before the benchmarks

The question businesses most want answered is whether their data trains the vendor's model, and the major providers publish explicit answers. OpenAI states that data sent to the OpenAI API is not used to train or improve OpenAI models unless the customer explicitly opts in, and documents that abuse monitoring logs are retained for up to 30 days unless longer retention is legally required, with a zero data retention option available to approved customers.

Microsoft publishes an equivalent commitment for models sold through its Foundry service, stating that prompts and completions are not available to other customers or to the model providers, are not used to improve their models or services, and are not used to train generative AI foundation models without the customer's permission. It further documents that the models are stateless, that no prompts or completions are stored in the model, and that prompts and responses are processed within the customer-specified geography for standard deployments.

These are the commitments that belong in a risk assessment, and they are published rather than negotiated. They also differ between a vendor's consumer product and its business API, which is the distinction most internal debates miss: policies that apply to a free chat interface are frequently not the policies that apply to the paid API a business would actually build against.

When AI should not be used

This section is short and it is the most useful part of the article. Five situations come up repeatedly where the correct engineering answer is that this is not an AI problem.

• The rules are stateable. If the decision can be written as conditions, write it as conditions. A rules engine is cheaper, faster, auditable, and correct every time. Using a language model for arithmetic or eligibility criteria is a category error.
• There is exactly one correct answer and being wrong is expensive. Tax calculations, regulatory thresholds, payment amounts. Determinism is the requirement, and it is precisely what a model does not offer.
• You cannot describe what a good output looks like. If nobody can specify the standard, nobody can evaluate the system, and an unevaluable system cannot be improved or trusted.
• Verification costs more than doing the work. If a person must read every generated output closely enough to catch subtle errors, they have done the task twice. The saving is negative.
• The underlying data is wrong. AI applied to poor data produces confident, well-written, incorrect answers, and does so faster than the manual process it replaced.

The recurring pattern in all five is that the failure mode is fluency. A model that is wrong does not look wrong. Conventional software fails visibly, which is a property that is easy to undervalue until you work with something that fails invisibly.

Data privacy and governance

Two frameworks are worth working against, and both are public.

The NIST AI Risk Management Framework, published as NIST AI 100-1 in January 2023, exists to help organisations manage risks to individuals, organisations and society associated with artificial intelligence, and to incorporate trustworthiness considerations into the design, development, use and evaluation of AI products and systems. It is organised around four functions: Govern, Map, Measure and Manage. For a business rather than a model developer, Govern and Map are where the useful work is: deciding who is accountable for AI use, and mapping which processes it touches and what could go wrong in each.

The OWASP Top 10 for Large Language Model Applications, in its 2025 edition, is the security counterpart and is more directly actionable. Four of its entries describe risks that appear in ordinary business integrations rather than exotic ones.

The eighth is the one most often missed during scoping. Building a searchable index over internal documents creates a second copy of that content outside the system that enforced permissions on it. If the original folder was restricted to finance and the index is not, the restriction has been removed without anyone deciding to remove it.

Alongside these, the governance practices for the surrounding automation — versioning, ownership, secrets handling, alerting — are the same ones any orchestration platform needs, and are covered in the workflow automation article rather than repeated here.

Prompts are not architecture

A prompt is configuration. The system around it is architecture, and the two are routinely confused because the prompt is the part that is visible and the part that appears to produce the result.

The practical symptom is a business-critical capability whose behaviour is defined by a paragraph of text that lives in a workflow step, has never been version controlled, was last edited by somebody who has since changed roles, and has no test covering it. When the output degrades after a change, there is no record of what the previous version said. That is not a prompt problem; it is the absence of the engineering that should surround it.

1. 1

   Version the prompt with the code that calls it

   Prompts belong in the repository, reviewed like any other change. If a prompt lives only in a platform's interface, it is undocumented production logic.

2. 2

   Keep an evaluation set

   Twenty to fifty representative cases with an agreed view of what a good answer looks like. Run them after any change to the prompt, the model, or the retrieval layer. This replaces unit testing for the non-deterministic part of the system.

3. 3

   Constrain the output shape

   Where the result feeds another system, require a defined structure and validate it before use. A response that fails validation is an error to handle, not data to write.

4. 4

   Log inputs and outputs

   When a user reports a wrong answer, the only way to diagnose it is to see exactly what was retrieved and what was returned. Without that record the investigation cannot start.

5. 5

   Pin the model version where the platform allows it

   Providers update models. An integration that silently follows the latest version will change behaviour without a deployment, which is indistinguishable from a regression until somebody checks.

The second item is the one that separates a working deployment from a demonstration. Without an evaluation set there is no way to answer whether a change improved things, and improvement becomes a matter of opinion held by whoever last looked at the output.

Building around CRM, ERP, helpdesk and communications

The general pattern is the same in each case: retrieve from the system of record, generate, validate, and either present to a person or write back through the system's own API. What differs is where the value is and how much autonomy is appropriate.

The right-hand column is the part worth arguing about internally. The instinct is to reduce human involvement as quickly as possible, and the more durable approach is to earn autonomy with measurement: start with proposals, record how often they are accepted unchanged, and expand autonomy where that number justifies it. A capability that has been reviewed for a month and accepted almost every time is a candidate for automation. One that has not been measured is a guess.

None of this requires replacing any of these systems. The CRM continues to be the customer record, the accounts package continues to own the ledger, and the AI component reads and proposes through interfaces that already exist.

A representative scenario

A composite of situations we see repeatedly; no client detail is included.

A services company is quoted for a platform migration to gain AI-assisted support. The stated requirement is that agents should get suggested answers drawn from the company's own documentation and past tickets, and the proposal is to move the helpdesk to a product that offers this natively.

The review found the capability did not require the migration. The existing helpdesk exposed an API for both tickets and responses, so the work was retrieval over the documentation and resolved-ticket history, a model call, and a suggestion presented in the agent's existing interface. No system of record moved.

Two findings shaped the build. First, a substantial share of the documentation was outdated, so the initial phase became a content review rather than an engineering task — without it the assistant would have suggested wrong answers with complete confidence. Second, the index initially included internal notes not visible to all agents, which was caught during design and corrected by applying the same permissions at retrieval that the helpdesk applied to the source records.

It went live as suggestions requiring the agent to accept, edit or discard, with acceptance rates recorded. That measurement is what will justify any later increase in autonomy, and it exists because it was designed in rather than added after somebody asked how well it was working.

Implementation checklist

• The capability has been checked against systems you already pay for, before anything is built.
• The process is one where a wrong answer is detectable and recoverable.
• The information needed to answer correctly exists and is current, and someone has verified that against real questions.
• Retrieval applies the requesting user's permissions, and any index inherits the access model of its source.
• Data handling terms have been read for the specific service being used, not the vendor's consumer product.
• Irreversible, financial or customer-facing actions require human approval.
• The integration holds the narrowest permissions the task needs, and cannot write where it only needs to read.
• Model output is validated against an expected structure before it reaches another system.
• An evaluation set exists and is run when the prompt, model or retrieval layer changes.
• Prompts are version controlled, and inputs and outputs are logged well enough to investigate a complaint.
• A named person owns the capability, and acceptance or correction rates are recorded from day one.

Frequently asked questions

Conclusion

The businesses getting durable value from AI are not the ones that rebuilt their stack around it. They identified a process where being occasionally wrong was survivable, checked that the information needed to be right actually existed, connected a model to the systems already holding that information, and put a person in front of anything irreversible.

That is an unglamorous description of what is usually presented as transformation, and it is the version that survives contact with a real operation. The systems you already run encode how the business works. AI is a capable new component to add to them, and treating it as a component rather than a replacement is what keeps the cost of being wrong proportionate to the benefit of being right.