Specify an AI Procurement

Direct answer

What should an AI procurement specification contain? It should define permitted use, system and model transparency, data handling, security, performance tests, human oversight, material-change controls, incident obligations, audit evidence and exit requirements. Requirements should be testable and evidence-based rather than asking suppliers only to describe their approach.

An AI specification should turn the planning work into requirements that can be tested, evidenced and enforced. Avoid requirements that merely ask suppliers to describe their approach; ask for evidence and define what good looks like.

1. Intended use and boundaries

State the permitted use cases, user groups and operating context. If there are prohibited uses, list them. Require the supplier to identify assumptions, limitations and known failure modes relevant to the use case.

2. Model and system transparency

Require enough information to understand what is being bought:

• whether the service uses proprietary, third-party or open models;
• material model providers and dependencies;
• whether prompts or customer data are used for training or service improvement;
• where inference and storage occur;
• how retrieval, tools, agents or external APIs are used;
• how model or provider changes are governed.

Transparency should be proportionate to risk and should not require disclosure of legitimate trade secrets where alternative evidence is sufficient.

3. Data protection and confidentiality

Specify data classes, permitted processing, retention, deletion, international transfers, subprocessors, access controls and training-use restrictions. Require the supplier to support the buyer's privacy and impact-assessment obligations where applicable.

4. Security requirements

Apply normal cyber-security requirements plus AI-specific controls. The NCSC Guidelines for secure AI system development cover secure design, development, deployment and operation. Relevant evidence may include threat models, secure-development controls, supply-chain management, incident procedures, logging, access controls and model-specific protections.

5. Testing and performance

Define the metrics that matter for the intended task. Require suppliers to explain test datasets, evaluation methodology, known limitations and the conditions under which performance may degrade.

Where the buyer's context materially affects results, include a buyer-side evaluation or pilot using representative data and workflows.

6. Human oversight

Specify where human approval is required, what information must be shown to the reviewer, what override controls exist and how disputed outputs are handled.

7. Change control

AI services can change without a traditional software upgrade. Require notification and, where appropriate, approval for material changes to:

• model provider or model family;
• safety or moderation controls;
• data-processing location;
• subprocessors;
• retrieval sources;
• significant functionality;
• terms affecting training or data reuse.

Define what counts as a material change and when re-evaluation is required.

8. Incident and continuity requirements

Require clear incident notification, escalation contacts, investigation support, evidence preservation and recovery expectations. For critical services, specify failover, degraded-mode operation and exit options.

9. Auditability

Specify the logs, reports and evidence the buyer needs to verify compliance and investigate issues. Requirements should address access to relevant operational records without forcing disclosure of unrelated customer or proprietary information.

10. Exit and portability

Define what happens when the contract ends: data export, deletion, retention exceptions, model-specific assets, prompts, embeddings, fine-tuned components, integrations and transition support.

Public-sector transparency

Where relevant, review PPN 017, which provides optional questions to identify AI use in procurements and delivery of government services.

Specification output

A strong AI procurement specification should leave evaluators able to answer three questions: 1. What exactly must the service do? 2. What evidence must the supplier provide? 3. What would cause the buyer to reject, condition or re-evaluate the service?

Next: Evaluate AI suppliers.