The most important automation decision is often made before anyone compares tools. It is the decision to define the problem precisely enough that technology can be judged against an operating need—not the other way around.
The tool is rarely the first constraint
When an organisation begins with a capability—an agent, a model, a workflow platform—it quietly accepts the vendor’s framing of the problem. The resulting programme may be technically competent and still fail commercially or operationally. It can make the wrong work faster, hide ambiguity behind a polished interface, or shift judgement to people who were never given the authority or context to exercise it.
Strategy starts by making choices explicit. What decision are we trying to improve? Who experiences the current failure? What would be observably different if the intervention worked? Which constraints are real, and which are habits? What must remain under human control?
Automation should enter an operating model as a tested intervention, not as a substitute for one.
1. Define the decision, not the feature
“Automate review” is not a decision. “Help a qualified reviewer identify unsupported claims while preserving their final authority” is closer. The second formulation names the user, the object of work, the intended contribution and a boundary.
A useful decision statement should include five elements:
- Actor: who is accountable for the work?
- Object: what case, claim, document or transaction is being considered?
- Decision: what choice or action must follow?
- Evidence: what information makes that choice defensible?
- Boundary: what must the system never decide or infer?
If these cannot be stated, tool selection is premature. The ambiguity will not disappear in implementation; it will be encoded into prompts, queues, permissions and exceptions.
2. Map constraints before optimising flow
Constraints are part of the product definition. In regulated or high-consequence settings they include legal obligations, quality controls, privacy, data provenance, auditability, segregation of duties and the practical capacity of people to review exceptions. In any business they also include economics, service levels, integration cost and the cost of being confidently wrong.
The NIST AI Risk Management Framework organises work around Govern, Map, Measure and Manage, with governance operating across the lifecycle. Its framing is useful well beyond formal AI programmes: understand context and risk, define how performance will be measured, and retain an explicit decision about whether deployment should proceed. The framework also notes that risk work is continuous rather than a one-time gate.
3. Assign decision rights and failure ownership
“Human in the loop” is not an operating model. It does not say which human, at what point, with what information, under what workload, or with which authority to stop the process. A nominal reviewer can become a rubber stamp when the system is fast, persuasive and difficult to challenge.
The OECD AI Principles connect human oversight with accountability and traceability across the system lifecycle. The practical implication is straightforward: every automated contribution needs a named owner for the decision, a route for escalation, and enough trace evidence to reconstruct what informed the output.
Before building, write down who:
- owns the business outcome;
- approves use of the system and its data;
- reviews routine outputs and exceptions;
- can pause or withdraw the system;
- investigates harm, error or drift;
- accepts the residual risk.
4. Design the evidence before the experiment
A prototype that produces an impressive answer proves very little. A useful experiment begins with cases, expected behaviours and failure criteria defined before the run. It compares the intervention with a baseline and records false positives, false negatives, uncertainty, override behaviour and the time required for competent review.
Measures should connect to the intended decision. If the goal is better issue detection, response fluency is irrelevant unless it improves recall without creating an unacceptable volume of unsupported flags. If the goal is faster handling, time saved is not enough if reviewers lose the ability to understand or challenge the result.
Define success, failure and stop conditions together. A programme that cannot produce evidence against itself is advocacy, not evaluation.
5. Choose the smallest reversible intervention
Large automation programmes bundle many assumptions: that the problem is stable, the data is adequate, the workflow is understood, adoption will follow and the economics will survive integration. A smaller intervention can isolate those assumptions.
Start with decision support before autonomous action; synthetic or low-risk data before sensitive data; a narrow workflow before an end-to-end platform; and a manual fallback before dependency becomes irreversible. Reversibility is not timidity. It is a way to buy better evidence at lower cost.
A pre-automation brief
What specific choice or action should improve?
For whom, compared with which baseline?
Which obligations and failure costs shape the design?
Who decides, reviews, escalates and stops?
What would confirm or disconfirm the hypothesis?
What data, action or judgement stays out of scope?
What changes when strategy comes first
The technology conversation becomes more concrete. Teams can compare build, buy and non-technical alternatives against the same need. Governance becomes part of the workflow rather than a document added before launch. Evaluation focuses on decision quality and operating consequences. And stopping becomes a valid outcome when the evidence does not justify scale.
This does not make automation slower. It removes the expensive speed of building the wrong system.
Sources and further reading
- NIST, Artificial Intelligence Risk Management Framework 1.0 — voluntary framework for governing, mapping, measuring and managing AI risk.
- NIST AI RMF Playbook — suggested actions aligned with the framework’s four functions.
- OECD AI Principle: human-centred values and fairness — human agency and oversight safeguards.
- OECD AI Principle: accountability — lifecycle accountability, traceability and systematic risk management.