Deploy Purchased AI Safely
A practical go-live framework for purchased AI systems after contract award.
Direct answer
What should happen before an AI system goes live? Reconfirm the evaluated production configuration, restrict access and agent permissions, validate data boundaries, establish meaningful human oversight, test representative workflows and failure cases, enable logging, prepare incident routes, train users and record a production baseline against which future changes can be assessed.
Award is not the end of AI procurement. Deployment is where assumptions about the supplier meet the buyer's real users, data, integrations and operating environment.
1. Reconfirm the system boundary
Before go-live, confirm the exact model, service tier, hosting arrangement, integrations, subprocessors and data flows that will be used in production. Check that these match the evaluated and contracted configuration.
2. Configure access correctly
Apply least-privilege access, appropriate authentication, role separation and administrative controls. Where the AI can take actions through tools or agents, restrict those actions to what is necessary for the approved use case.
3. Validate data boundaries
Test which data users can submit, what connected sources the system can retrieve, whether outputs may reveal restricted information and how data is logged or retained. Confirm production settings reflect the agreed training and data-reuse restrictions.
4. Establish human oversight
Define where users must review or approve outputs. Make escalation routes visible. Where users are expected to challenge AI outputs, provide enough context and training for that review to be meaningful.
5. Test real workflows
Run acceptance testing using representative workflows, including difficult cases and known failure scenarios. Do not rely only on supplier demonstrations or benchmark results.
Record:
- test cases;
- expected outcomes;
- actual outcomes;
- issues found;
- mitigations;
- go-live decision.
6. Enable logging and monitoring
Confirm that the buyer can investigate incidents and material errors. Relevant logs may include user actions, model requests, tool calls, administrative changes, security events and model/version information, subject to privacy and proportionality.
7. Prepare incident response
Make sure users know how to report unsafe, incorrect or suspicious behaviour. Establish supplier escalation contacts and internal ownership. The NCSC secure AI guidance highlights secure infrastructure, model protection, incident procedures and responsible release.
8. Train users
User guidance should explain:
- approved and prohibited uses;
- data that must not be submitted;
- known limitations;
- when human review is mandatory;
- how to report problems;
- how outputs may need checking or attribution.
9. Use controlled rollout where appropriate
For material services, consider a pilot, limited user cohort or phased rollout. This provides operational evidence before exposure is widened.
10. Record the production baseline
At go-live, capture the model/service version, material settings, subprocessors, data locations, controls, owners and acceptance results. This creates a baseline against which future changes can be assessed.
Go-live decision
A production approval should state: 1. the approved use case; 2. the evaluated configuration; 3. residual risks; 4. compensating controls; 5. monitoring requirements; 6. review date; 7. material-change triggers.