AI Security Due Diligence

Quick answer

AI security due diligence examines whether a specific AI service can be used securely in the buyer's environment. Review its architecture, access controls, model and API dependencies, data exposure, prompt-injection and tool-execution risks, testing, incident response and monitoring. Ask for product- and deployment-specific evidence, not just policy statements or badges. Check gaps against the intended use and revisit security assumptions after material changes.

AI security due diligence should combine conventional SaaS security with AI-specific risks. The goal is not to collect badges; it is to establish whether the proposed service is securely designed, deployed and operated for the buyer's intended use.

Architecture and trust boundaries

Request a current view of hosting, model providers, retrieval systems, data stores, APIs, tools, agents and privileged integrations. Identify which components sit outside the supplier's direct control.

Identity and access

Check authentication, privileged access, tenant isolation, service accounts, secrets, administrative controls and least-privilege design. Agentic services need explicit limits on tool permissions and actions.

Secure development and supply chain

Review secure-development practice, dependency management, vulnerability handling, software/model supply chain, change control and relevant testing.

AI-specific threats

Depending on architecture, assess prompt injection, retrieval manipulation, unsafe tool execution, data leakage, poisoned content, insecure output handling, excessive agency and model/provider compromise.

Logging and investigation

Determine whether material user actions, administrative changes, tool calls, security events and model/version information can be investigated without creating disproportionate privacy risk.

Testing evidence

Relevant evidence can include penetration-test summaries, threat models, secure-development attestations, vulnerability records, red-team results and remediation evidence. Ensure tests match the actual production architecture.

Incidents and maintenance

Check notification, escalation, evidence preservation, patching, lessons learned and how material model/provider changes are handled after award.

The NCSC Guidelines for secure AI system development provide a useful lifecycle reference for secure design, development, deployment and operation.

Frequently asked questions

What is different about AI security due diligence compared with ordinary SaaS reviews?

Conventional checks still matter, including identity, access, vulnerabilities, suppliers and incident response. AI services may also expose model-specific risks, prompt injection, retrieval manipulation, data leakage and tool actions that cross trust boundaries. The NCSC secure AI procurement guide links these checks to lifecycle security.

What evidence should an AI supplier provide about prompt injection?

Ask how the product handles untrusted instructions in documents, webpages, retrieved context and tool outputs. Relevant evidence may include architecture and threat models, permission boundaries, realistic adversarial tests, monitoring and incident handling. Test the proposed deployment rather than assuming a model-level safety claim covers integrations.

Is a penetration test or security certificate enough to approve an AI product?

Not on its own. Verify the test or certificate's scope, date, architecture and findings. Consider model dependencies, data flows, permissions, operational controls and unresolved vulnerabilities. Use the supplier evidence checklist to record evidence strength and gaps.

When should AI security due diligence be repeated?

Reassess after material model, hosting, tool, integration, permission or data-use changes, and after relevant incidents or new critical threats. Set review triggers, evidence access and notification obligations in the AI procurement contract.

How AI TrustMark fits

AI TrustMark can independently verify supplier security evidence within a defined product scope. It does not create NCSC accreditation or guarantee that a service is vulnerability-free.

Organisation-specific security questionnaires, technical evidence reviews, testing scopes, procurement matrices and remediation plans remain professional services.

Return to the AI Procurement Knowledge Base or read the NCSC Secure AI procurement guide.