Arrow 
Expand an AI pilot only when the agreed task meets its quality criteria, the complete workload is understood, and the operating team can own failures. Revise a testable weakness; stop or pause when the required evidence or controls are missing.
Decide against the brief, not the demonstration
A pilot review should answer the decision written into the original scope. A persuasive demonstration does not establish that routine requests are handled well, that private information stays within its access boundary, or that the receiving team can recover from failure.
Bring the approved pilot brief, acceptance results, and review-effort measurement to the same review. Identify what is observed, what is assumed, and what remains untested. A missing measurement should remain missing rather than become a positive result.
Use three outcomes with explicit conditions
Continue, revise, and stop are useful only when each leads to a concrete next step. There is no universal success percentage that resolves all three decisions.
| Outcome | Evidence to look for | Next action |
|---|---|---|
| Expand within a defined boundary | The agreed acceptance criteria are met, full task effort is understood, and an operating owner accepts the work. | Name the next team or request type, its review rules, and the cases that must be retested. |
| Revise and test again | A specific source, routing, review, or integration weakness explains the failures and can be tested. | Assign a change owner, preserve the baseline, and define the evidence needed at the next review. |
| Stop or pause | Essential access controls, evidence, ownership, or recovery behavior are absent, or the workflow adds unacceptable work. | Return eligible work to the manual process and record the conditions required to reconsider. |
These are proposed operating criteria. The accountable team must choose its release rules before judging the results, especially where a failure could have significant consequences.
A worked decision record for a limited expansion
Illustrative decision: a support drafting pilot answers one product family's documentation questions. Reviewers find that current manuals support the eligible questions, but requests about an older product repeatedly require correction. The team keeps the older product out of scope.
The decision is to expand the existing scope to a second group of trained specialists, with the same product boundary and review rule. It is not permission to send replies automatically, access additional customer records, or support the older product. Those changes would create new assumptions to test.
The record names the business owner, the reviewed case set, the configuration version, unresolved issues, the receiving team, and the next review trigger. It also records who can pause the workflow and how staff continue work during the pause. No customer outcome or measured gain is implied by this constructed example.
Make stopping and restarting operationally clear
Stopping a pilot should leave work in a known state. Disable new automated intake where appropriate, identify queued tasks, and reconcile any uncertain actions before staff retry them. Retain only the evidence required by the agreed access and retention policy.
Write down what would justify reopening the decision: a missing source has an owner, an access boundary has been corrected, or a receiving team can now support the process. Avoid restarting merely because a newer model is available.
The NIST AI RMF Playbook organizes voluntary suggestions across Govern, Map, Measure, and Manage. This decision record is our practical planning aid, not a complete risk-management assessment. Before routine use, complete the handoff checklist and return to the cluster reading path when the scope changes.
Sources and editorial scope
Original operational guidance by Arrow AI, published and reviewed October 19, 2026. Worked examples and calculations are synthetic, not customer results or forecasts. Adapt the method to the workflow and its operating context.
The Arrow read
Build the whole pilot decision
Follow the AI pilots and operations collection, from scope through acceptance and measurement to handoff. Explore the pilot planning approach for the wider sequence.