AI automation projects are rarely derailed by choosing the wrong model. Far more often, they fail because the company lacks a clearly defined problem, realistic expectations, an end-to-end view of the process, an understanding of actual operations, or high-quality data.

For example, a company purchases an integration platform, connects a language model, and creates its first automated workflow within a few weeks. Technically, everything works. Yet employees do not trust the outputs, the process still requires additional manual work, and no one can determine whether the investment has actually paid off.

AI automation is not just a model or a set of connected applications. It is an operational system where processes, data, decision-making rules, human responsibility, and technical infrastructure come together. A weakness in any of these areas will eventually affect the outcome.

1. The Company Chooses a Solution Before Defining the Problem

“We want to implement Make.”

“We need Pipedrive.”

“Build us an AI agent.”

These requirements may sound specific, but they describe a means rather than an objective.

The first step is to determine exactly what needs to change within the process. This could involve reducing the time required to process an enquiry, decreasing the number of manual data transfers, finding information more quickly, or limiting errors when data is passed between systems.

Only then can you assess whether the problem should be solved with a simple rule, a conventional integration, a process adjustment, or AI.

Without a baseline and a target metric, it is also impossible to evaluate the project’s impact. The team may be able to demonstrate a functioning automation, but it cannot prove that it saves time, reduces costs, or improves quality.

Before selecting a specific tool, answer four questions:

• What exactly is not working today, or what consumes an unreasonable amount of work?

• Who is affected by the problem, and how often does it occur?

• How will we measure the current state and the expected improvement?

• Is AI genuinely necessary for this step, or would conventional automation be sufficient?

A sales team, for example, does not need “AI for emails.” It needs to reduce the time between receiving an enquiry and assigning it to the correct salesperson.

Success can be measured by the median assignment time, the percentage of correctly classified enquiries, and the number of cases that require human correction. A requirement defined in this way makes it possible to compare different solution options.

2. AI Is Expected to Provide Human Judgement and 100% Accuracy

Traditional automation follows predefined rules. Generative AI creates outputs based on a probabilistic model.

It may therefore respond differently even to similar inputs. Its answer may be inaccurate, incomplete, or unsuitable for a specific context.

This does not mean AI cannot be used reliably. It means that reliability must be designed at the level of the entire system.

Selecting a model once and considering the project complete is not enough. The system must be tested, monitored, and gradually adjusted based on real-world results.

The operational design should define:

• what level of accuracy is acceptable for the given purpose;

• which errors are merely inconvenient and which could create financial, legal, or security risks;

• when the system should proceed automatically and when the result should be passed to a person;

• how quality will be tested before launch and monitored during operation;

• how an incorrect action can be stopped, reversed, or corrected.

Human oversight does not mean that a person must approve every step forever. A human-in-the-loop approach should reflect the level of risk associated with the specific activity.

An automated summary of an internal meeting note may require a different level of control than sending a quotation, changing customer data, or making a decision that affects a specific individual.

The objective is not to achieve maximum autonomy at any cost. The objective is to design a system that knows what it can do independently and when it should stop.

3. Individual Tasks Are Automated Instead of the Entire Process

An isolated task is relatively easy to automate. A system can transcribe an attachment, create a CRM record, generate a response, or send a notification.

However, business value usually comes from the entire process, from the initial trigger to a verified output.

When individual workflows are created without a shared design, they may compete with one another, process the same input repeatedly, or create different versions of the same data.

As soon as a CRM field, document structure, or data-transfer method changes, several subtly interconnected automations may fail at the same time.

Before implementation, the following should therefore be mapped:

• What triggers the entire process?

• Where does the data come from?

• What decisions are made during the process?

• Which systems and roles are involved?

• What exceptions and retry scenarios exist?

• What constitutes a verified output, and who is responsible for it?

The technical design must also address details that may initially appear to be purely operational. These include preventing duplicate processing, maintaining an audit trail, managing permissions, handling external service outages, dealing with API limits, and defining recovery procedures.

Without these safeguards, manual work does not disappear. It is simply moved from routine operations to error handling, incident management, and unexpected exceptions.

4. The Idealised Process Is Automated Instead of the Real One

Process documentation often describes how work is supposed to happen.

In reality, however, processes contain exceptions. Missing information is retrieved from emails, some clients use an old template, certain requests are approved by a different person, and urgent cases bypass the standard procedure.

When only the “clean” version of a process is automated, the pilot may appear successful. After deployment, however, the system begins to fail on common variations.

The automation then speeds up the part of the work that was described correctly, while unclear inputs and exceptions begin to accumulate with employees.

The correct first step is therefore not to draw the ideal future workflow. You must first observe several real examples of the process and speak to the people who actually perform the work.

Ask questions such as:

• Where do you most frequently have to wait for information?

• What do you regularly need to correct manually?

• How do you recognise an exception?

• Which decisions are not included in the documentation?

• What do you do when one of the systems does not respond?

• Which parts of the process do people bypass, and why?

Even a brief observation of real operations can reveal a significant difference between the official procedure and the work that actually takes place.

For a more complex process, however, multiple roles must be involved, different types of cases must be reviewed, and seasonal or less frequent situations must also be considered. The automation design should not be based solely on one person’s experience or a single example scenario.

Only then does it make sense to simplify the process, remove unnecessary steps, and decide which parts should be automated.

Implementing AI will not repair a broken process. It will simply cause the process to produce errors more quickly and at a greater scale.

5. The System Relies on Poor-Quality or Unmanaged Data

AI may be configured correctly and still produce poor results if it works with incomplete, outdated, duplicated, or contradictory data.

Data quality is not only an input issue. The entire “AI sandwich” must be controlled: the sources before processing and the outputs before a subsequent action is performed.

Before implementation, verify:

• which system is the primary source for each data point;

• who is responsible for its accuracy and maintenance;

• whether the system has permission to use the data for the intended purpose;

• how duplicates, missing values, and conflicts are handled;

• whether it is possible to trace the input used, the version of the rules or prompt, and the resulting action;

• how an incorrect output will be detected before it affects another part of the process.

For a knowledge assistant, an important metric may be the percentage of answers supported by an approved source. For enquiry classification, it may be accuracy within each category and the number of cases that must be taken over by a person.

A single general metric such as “the AI works” is not enough.

Data quality is also not a one-off task completed before launch. Sources change, documents become outdated, input structures evolve, and user requirements may gradually shift. It is therefore necessary to monitor not only the system’s technical availability, but also the quality of its outputs over time.

How Should You Prepare Your First AI Automation Pilot?

A successful pilot does not need to automate an entire business process. It must, however, test a specific assumption and demonstrate whether the solution delivers measurable value at an acceptable level of risk.

1. Select a Narrow but Complete Use Case

Choose a process with a sufficient volume of repetitive work, available data, and a clearly defined owner.

The pilot should include a real input and a verifiable output. It should not merely be an impressive demonstration built around a few manually selected examples.

2. Measure the Current State

Record the time required, error rate, volume of manual intervention, waiting time, and other costs.

Without a baseline, it will not be possible to distinguish a genuine improvement from a subjective impression that the new system is faster or more modern.

3. Prepare Representative Test Cases

The test set should include standard inputs, edge cases, and known exceptions.

Metrics should be selected according to the system’s purpose and risk level. An internal assistant used to retrieve documents should be tested differently from a system that automatically communicates with customers or changes data in internal applications.

4. Design Oversight, Escalation, and Rollback Procedures

Determine in advance when the system should proceed automatically, when it should request additional information, and when a person should take over the case.

For actions that modify data or communicate externally, implement an audit trail and a way to correct mistakes.

5. Evaluate Both the Business and Operational Results

Do not assess the pilot solely on the basis of the model’s accuracy.

Include the time saved by the system, but also the work required for oversight, exception management, operation, and maintenance. Only the overall impact will show whether the solution should be expanded.

Does Every Automation Need Artificial Intelligence?

No. If a decision can be described precisely using stable rules, traditional automation is usually cheaper, more predictable, and easier to test.

AI is more suitable for unstructured inputs, classification, extracting meaning, generating text, or situations in which fixed rules are insufficient.

In some cases, the best solution is a combination of both approaches. AI may, for example, evaluate the content of a document, while the subsequent steps are carried out according to clearly defined rules.

When Does an AI Output Need Human Review?

The level of oversight should correspond to the potential impact of an error.

For low-risk and easily reversible steps, random checks and continuous monitoring may be sufficient. For financial, legal, HR, security-related, or otherwise significant decisions, prior human approval or an additional independent verification mechanism is appropriate.

How Can You Tell Whether a Pilot Is Ready to Scale?

A pilot is ready to scale when it:

• achieves the predefined results on representative data;

• handles both standard and known non-standard cases;

• has a clearly defined owner;

• includes monitoring and escalation procedures;

• allows its decisions and actions to be traced retrospectively;

• remains economically viable after operating costs and human oversight are included.

Process First, Automation Second

Good AI automation does not begin with selecting a model or drawing a workflow. It begins with a measurable problem and an honest understanding of how the work actually happens.

Only when you understand the process, its exceptions, risks, data, and human responsibilities can you correctly determine where AI can add value and where it would merely introduce another layer of complexity.

When preparing your first pilot or trying to understand why an existing automation is not delivering the expected results, begin with an audit of the process, data, and operational risks. This will often reveal the most expensive mistakes before they are written into the code.