NESO and the boundary between AI advice and action

TL;DR

NESO’s Dynamic Reserve Setting tool comes with three strong numbers: 380 MW, 1,000 MW and about £30 million a year. None of them gives the tool permission to act. A model’s quality and its permission to trigger action are two separate decisions, and regulated firms need to make both on the record.

• Quality asks whether a recommendation is good enough to support a decision.
• Permission asks who may authorise the action, under which conditions and within which limits.
• Every model-driven decision needs four separate records: recommendation, approval, execution and verification.
• Authority expands in stages, with a fresh approval at each one.
• Fixed-income desks and bank credit functions already run this split. The model just has to be connected to it.

NESO’s reserve-setting model saved 1,000 MW in a trial. That number still doesn’t let it set reserve on its own.

Every model recommendation needs two checks. Quality is how well it supports a decision. Permission is who may authorise the action, under which conditions, and within which limits.

If a strong pilot result has ever been read as sign-off in your steering committee, you already know what happens when the two get mixed up. The NESO example is useful because it puts hard numbers right next to recommendations for assurance, approval authorities, risk thresholds and phased deployment. Regulated firms need an operating design that handles both.

Figure 1. Quality and permission are separate axes.

What do the three NESO figures actually show?

The independent review The Grid We Need Now, led by Lucy Yu and published by the UK Government in September 2026, says NESO is testing probabilistic machine learning for Dynamic Reserve Setting. NESO aims to roll it out by early 2027. The trial close-down report came out in July 2024.

The review reports three figures. They describe three different things.

• During testing, the tool recommended 380 MW less upward reserve overall than the existing approach. This describes a difference between recommendations.
• A December 2023 trial achieved a 1,000 MW reserve saving. This describes a trial result.
• The approach is expected to save about £30 million per year in reserve costs. This describes an expected annual financial saving.

Keep each description attached to its number. A recommended reduction, an achieved trial saving and an expected annual cost saving mean different things, so they can’t be added up or swapped for each other.

Figure 2. Three figures, three meanings.

The annual figure is still an expectation. And none of the three gives the model permission to trigger action.

The review also recommends independent assurance of risk models, explicit approval authorities, defined risk thresholds and phased rollout before national deployment. Those recommendations cover risk-based operation and AI-enabled autonomy in general. They don’t single out the reserve-setting tool, so nothing in the review grants it permission to act.

That points to one design rule: assess the recommendation, then allocate the authority, as two separate steps.

The operating design below is my analysis. It makes no claim about NESO’s current approval arrangements.

Key point: The 380 MW, 1,000 MW and £30 million figures each support a different claim, and none of them grants the tool authority to act.

How to define the decision before allocating authority

Start with the decision the organisation actually needs to make. Name the action that could follow and the consequences that need control.

For reserve setting, that means separating the recommendation about reserve from the decision to act on it. Then work out what authority the action needs.

Map the current process before you design an automated one. Write down who approves the action today, what evidence they ask for and which conditions force escalation.

Carry those responsibilities into the new design on purpose. Any change in who owns or approves the action should need its own approval.

This stops an implementation choice from quietly rewriting the operating model. Wiring a recommendation into an execution path is a grant of authority, so treat it as one.

Define three separate permissions:

1. Produce a recommendation. Specify what the model may assess and propose.
2. Approve an action. Identify the person or authorised process that may accept the proposal.
3. Execute the action. State what the execution mechanism may do after approval.

Organisations can split these permissions in different ways. What matters is that every split is written down and can be reviewed.

Figure 3. Recommend, approve, execute.

What goes in a decision record?

Attach a short decision record to every proposed grant of authority. It should be specific enough that a reviewer can tell whether an action sits inside or outside the permitted scope.

It needs six fields.

1. Purpose. State the decision the system supports and the objective it serves. Define the actions outside that purpose.
2. Authority. Name the accountable role, the approval route and any delegated execution rights. Include who can suspend those rights.
3. Evidence boundary. Identify the evidence supporting the recommendation and the conditions under which that evidence applies.
4. Risk. Define the limits the organisation accepts and the conditions that require escalation.
5. Reversibility. Describe what can be reversed, who can order reversal and which consequences would remain.
6. Verification path. Specify the evidence needed to confirm that the authorised action occurred and that the resulting state meets the acceptance criteria.

Figure 4. The decision record, with illustrative entries.

Write these fields before you expand permission. A better recommendation should land in a decision structure that already has a clear owner.

In the NESO case, the evidence boundary keeps the testing comparison, the trial saving and the annual expectation apart. Each one supports a different statement.

A reviewer should be able to trace any approval back to the evidence that actually supports it. That’s the practical payoff: fewer assumptions sitting between a model output and an action with real consequences.

Key point: Six fields (purpose, authority, evidence boundary, risk, reversibility and verification path) turn a vague permission into one a reviewer can check.

How should decisions be routed by consequence and reversibility?

Match approval effort to how consequential, novel and reversible the action is.

Consequential, irreversible or novel actions go to explicit human approval. Name the responsible role in advance, along with the evidence that role must see.

Limited, reversible actions inside an approved scope can be candidates for delegated execution. Keep that permission narrow, with conditions the system checks before it acts.

A routing rule could split decisions into three cases.

• Inside the approved boundary. Allow the authorised path only when the evidence, action and risk conditions all satisfy the agreed requirements.
• Outside the boundary or uncertain. Hold execution and route the recommendation to the designated reviewer.
• Evidence missing or a limit breached. Prevent execution and use the approved fallback.

Figure 5. Three tests, three destinations.

Spell out the fallback. It might mean getting more evidence, going back to a manual decision or suspending the action.

Teams get a rule they can actually use. Routine decisions have a set path, and exceptions reach someone with authority to resolve them.

Look hard at reversibility. Some reversals don’t restore the prior state, and whatever’s left over belongs in the approval decision.

Key point: Reversible, in-scope actions can run on delegated rules. Everything else goes to a named human with the evidence in hand.

Why should assurance be separate from approval?

The review pairs independent assurance of risk models with explicit approval authorities. They belong together because they answer different questions.

Build both into the operating model.

Assurance has one job: check whether the risk model and its evidence justify the intended use. Set that scope before anyone looks at results.

Approval has a different job: decide whether to permit the action within the organisation’s accepted limits.

The approver should get the assurance findings, their scope and any open conditions. Accountability for the permission stays with the approver.

Keep assurance independent. The team that built the model or proposed the deployment shouldn’t set the scope of its own review. Findings go straight to the approver, without a project sponsor summarising them first.

Set risk thresholds before a recommendation is assessed for execution. Include the conditions where the evidence stops being enough for the authorised use.

Don’t pick limits after the fact to fit a good-looking result. In the NESO case, an expected annual saving is a financial consideration. It doesn’t tell you what risk threshold is acceptable.

Make escalation something people can act on. Every threshold needs an owner, a required response and a rule for execution while the issue stays open.

Key point: Assurance checks whether the model is fit for the use. Approval decides whether to act, and a clean validation report doesn’t make that decision for anyone.

How should permission expand during rollout?

The review recommends phased rollout before national deployment. Turn that into stages, each with explicit permissions and exit criteria.

Start with recommendation-only use where it fits. Judge the outputs against an agreed evaluation method, while execution authority stays inside the existing approval process.

Any move to limited execution needs a fresh approval of scope. State which actions become allowed, what conditions constrain them and what evidence decides whether the stage worked.

Before expanding again, review performance, exceptions, assurance findings and verification results against criteria set in advance.

Continuing a trial, widening its scope and granting broader authority are three separate decisions. Each one changes a different part of the operating model.

Set suspension and rollback conditions at the same time as the criteria for progress. Decide who can invoke them before the stage starts.

Rollout then becomes a sequence of governed decisions. Reviewers also get a clear basis for stopping expansion while the evidence is still incomplete.

Here’s how that looks in four stages:

1. Shadow mode. The model produces recommendations that nobody acts on. Compare them with the decisions made through the existing process and record where they diverge.
2. Advisory use. Recommendations reach the accountable decision-maker as one input. The human approval route stays unchanged, and every accepted or rejected recommendation is logged with its reason.
3. Narrow delegated execution. A defined class of limited, reversible actions may proceed without case-by-case approval, subject to checks the system performs before acting.
4. Broader authority. Scope widens only after a separate approval, based on the evidence collected in the previous stages and a fresh assurance review.

Figure 6. Four stages, each with its own grant of permission.

Key point: Each rollout stage carries its own grant of permission, its own exit criteria and a named person who can stop it.

How do you verify the result before closing a decision?

Approval and execution still need one last step: verification.

Specify the evidence that confirms the authorised action happened. Then define the checks that show whether the resulting state meets the acceptance criteria.

Keep four records apart: recommendation, approval, execution and verified outcome. Together they show what the system proposed, what the organisation permitted, what happened and what the checks found.

Figure 7. The four-record chain.

A completion message is a claim. Back it with fresh observations that fit the action before the decision is marked complete.

If verification fails, send the case to the named owner and apply the agreed response. Record the open condition so it stays visible in later reviews.

Decide which changes trigger reassessment. A change to the model, the evidence, the permitted actions or the operating conditions should prompt a check on whether the existing authority still applies.

A reviewer can then look at the whole decision without assuming approval proves execution, or that execution proves an acceptable outcome.

Key point: A decision closes on observed evidence of the resulting state. The system’s own completion report doesn’t count as that evidence.

How does permission drift happen?

Authority boundaries wear down through small changes that each look reasonable on their own. Name those routes in advance so reviewers can check for them.

• Integration convenience. An engineer connects a recommendation feed to an execution API to remove a manual step. The connection is a grant of authority and needs the same approval as any other.
• Evidence reuse. A trial result is cited to support a decision in conditions the trial never tested. Tie each piece of evidence to the conditions it covers.
• Threshold drift. Limits widen after a run of good results. Treat any change to a threshold as a new approval decision, with its own record.
• Assurance read as approval. A clean validation report circulates as permission to act. Keep the two sign-offs in separate fields of the decision record.
• Verification by message. A system reports completion and the case closes. Require independent observation of the resulting state.

Each route has a matching control, an approval requirement tied to a specific type of change. The decision record gives reviewers a fixed reference to check against.

Key point: Integration shortcuts, reused evidence, widened thresholds, validation read as sign-off and trust in completion messages are the five usual ways authority expands without anyone approving it.

A parallel in investment operations

The same separation applies on a fixed-income desk. Consider an asset manager running portfolios on an order management system like Charles River, with a risk model that recommends duration adjustments when interest rate exposure moves away from the benchmark.

The model proposes shortening a portfolio’s duration by 0.4 years through sales of long-dated gilts and purchases of shorter maturities. The quality question asks whether the model has measured exposure correctly. The permission question asks who may place the trades, within which mandate limits and at what size.

Investment firms already split these permissions. The portfolio manager owns the investment decision, pre-trade compliance checks the order against mandate and regulatory limits, the dealing desk executes, and middle office reconciles the fills against the order and the custodian’s records.

Map the six fields to the trade

1. Purpose. The model supports duration management against a stated benchmark. Credit selection, currency positions and changes to the benchmark sit outside its purpose.
2. Authority. The portfolio manager approves. The head of fixed income can suspend the model’s use. The dealing desk holds execution rights only for approved orders, and any delegated execution is limited to named portfolios.
3. Evidence boundary. Backtests cover a defined period and rate regime. A model calibrated in stable rates says little about conditions like the September 2022 gilt market stress, when liability-driven investment funds sold long-dated gilts and the Bank of England intervened.
4. Risk. Tracking error limits, maximum trade size as a share of average daily volume, issuer concentration limits and liquidity thresholds.
5. Reversibility. Offsetting trades can reverse the position. Transaction costs, market impact and any realised losses remain.
6. Verification path. Confirm fills against the order, settlement through the custodian, and post-trade exposure recalculated from actual positions. The model’s own estimate of the result does not count as verification.

Route the trade by consequence

Routine rebalances inside tracking error limits, and small relative to market liquidity, could run through delegated execution. Pre-trade compliance supplies the conditions the system can check before acting.

Trades above a size threshold, in thinner markets or during periods of market stress return to the portfolio manager. A breach of a hard mandate limit blocks the order and triggers the agreed fallback.

Firms with a mature order management system already encode much of this. Hard and soft compliance rules map directly onto the three routing cases: inside the boundary, uncertain, and breached. The design task is to connect the model’s output to those existing controls, so it cannot reach execution by a separate path.

Run the rollout on the desk

Start in shadow mode for a full quarter. Log every model recommendation alongside the portfolio manager’s actual decision, and review the divergences with the head of fixed income.

Move to advisory use once the divergences are understood and the evidence boundary is documented. Grant narrow delegated execution only for named portfolios, below a defined trade size, with suspension authority assigned before the first automated order.

Walk one trade through the chain

Follow a single recommendation from model output to closed decision. On a Tuesday morning, the risk model flags that a sterling aggregate portfolio sits 0.4 years long against its benchmark after a rally in long-dated gilts.

1. Recommendation. The model proposes selling £25 million of 2053 gilts and buying an equivalent duration-weighted amount of 2031 gilts. The output records the model version, the positions it read and the timestamp of the market data it used.
2. Approval. The trade exceeds the delegated size limit for this portfolio, so it routes to the portfolio manager. She accepts the sale, reduces the purchase to keep a small residual position she wants for a separate view, and records the reason for the change.
3. Pre-trade compliance. The order management system checks the amended order against mandate limits, issuer concentration and the liquidity threshold. It passes, and the check result attaches to the order.
4. Execution. The dealing desk works the order across three counterparties. The execution record holds fills, prices, timestamps and counterparties.
5. Verification. Middle office reconciles fills against the order. Settlement confirms through the custodian on the following day. Exposure is recalculated in the order management system from settled positions and shows duration within the approved tolerance of the target.

The record now answers four separate questions: what the model proposed, what the portfolio manager permitted, what the desk did, and what the settled positions show. Each answer comes from a different owner and a different source of evidence.

The amendment matters most. The portfolio manager changed the recommendation, and the record preserves both versions. A later review can measure model quality against the original proposal and measure the outcome against the approved order, without confusing the two.

If settlement had failed on one counterparty, verification would hold the decision open. The case would return to middle office and the portfolio manager with the unsettled amount recorded, rather than closing on the strength of the desk’s execution report.

The banking version

Banks face the same question under formal model risk rules. In the UK, the Prudential Regulation Authority’s supervisory statement SS1/23, originally published in May 2023 and updated in April 2026, sets five model risk management principles, including model identification, governance, independent validation and model risk mitigants. Its expectations apply directly to UK banks, building societies and PRA-designated investment firms with internal model approval for regulatory capital, and it addresses machine learning in modelling. In Australia, APRA’s CPS 230, in force since 1 July 2025, sets operational risk requirements, including for critical operations and material service providers.

Take a credit decisioning model that recommends limit increases for small business customers. Validation examines whether the model ranks risk accurately. Authority to increase a limit sits in credit policy, which sets delegated lending authorities: limits a model-supported process may approve, limits requiring a credit officer, and limits requiring a credit committee.

The three-figure problem appears here too. A backtest may show reduced expected loss, a pilot may show a lower default rate on a sample portfolio, and finance may project an annual impairment saving. Each supports a different statement, and none grants lending authority.

Reversibility also has limits. A bank can reduce a credit limit, but it cannot recall funds the customer has already drawn. That residual consequence belongs in the approval decision, in the same way a reserve shortfall that cannot be corrected in time belongs in a grid operator’s.

Figure 8. Three domains, one structure.

Make the next authority decision explicit

The NESO figures provide three different forms of evidence. The review’s governance recommendations identify separate work on assurance, approval, thresholds and rollout.

For a regulated operating model, begin with one decision and document its recommendation, approval, execution and verification paths.

Identify any point where an output can trigger action without an explicit grant of authority. Close that route or approve it within a defined boundary before expanding use.

The same discipline holds on a trading desk and in a credit function. The grid, the portfolio and the loan book carry different consequences, and each one needs recommendation, approval, execution and verification recorded as separate steps.

Grant permission for a defined action under defined conditions. Reassess it when those conditions change.