Procuring Agentic and Autonomous AI
A buyer guide to the additional controls needed when AI systems can plan, call tools or take actions.
Agentic AI changes the procurement problem because the system may not only generate content; it may plan, call tools, access systems, retain memory or take actions. Buyers should therefore assess authority, permissions and reversibility as well as model quality.
Define the action boundary
Document what the agent can read, write, send, approve, purchase, delete, execute or change. Distinguish advisory suggestions from actions performed without further approval.
Apply least privilege
Tool and system access should be limited to the minimum needed for the approved use case. Separate read-only, draft, approval and execution permissions where possible.
Design approval gates
High-impact, external-facing or irreversible actions may require human approval. Verify that approval cannot be silently bypassed by workflow design, retries or delegated agents.
Assess prompt and tool injection risk
Agentic systems can be influenced by untrusted content retrieved from email, webpages, documents or connected systems. Security testing should consider prompt injection, malicious tool instructions, unsafe output handling and privilege escalation.
Examine memory and state
Ask what the agent remembers, where memory is stored, how it is scoped between users or tenants, how long it persists and how inaccurate or sensitive state can be corrected or deleted.
Require traceability
For material actions, logs should allow investigators to reconstruct the initiating user or event, model/agent decision, tool calls, approvals, resulting action and relevant version/configuration.
Set stop and recovery controls
Buyers should understand how an agent can be paused, credentials revoked, queued actions cancelled and erroneous changes reversed.
Monitor autonomy after go-live
Changes in models, tools, permissions or workflow logic can increase autonomy after the original procurement. Define reassessment triggers for material changes.
How AI TrustMark fits
AI TrustMark can independently examine agent architecture, authority boundaries, tool permissions, logging and operating evidence within a defined scope. It does not make the buyer's final risk decision.
Organisation-specific agent threat models, permission design, test plans, tender requirements and deployment assurance remain professional services.
Return to the AI Procurement Knowledge Base or read AI Security Due Diligence.