AI and Emerging Technology
AI and LLM Penetration Testing
Test the security boundaries of deployed AI applications, retrieval systems and agents. Findings establish whether untrusted content can cause unauthorised disclosure or actions with operational, financial or compliance consequences.
01 / Overview
Assessment objectives and control obligations
AI assessment reconstructs trust boundaries across prompts, retrieval, persistent memory, tool invocation and downstream execution. Representative identities and synthetic records support direct and indirect prompt-injection cases against explicit entitlement and approval requirements. Excessive agency is evaluated by the authority of available tools and their enforceable restrictions.
Controlled trials establish security consequences such as unauthorised retrieval, disclosure or state mutation. Model identifiers, application versions, input provenance and trial counts preserve reproducibility and measured variability. A valid boundary violation and its occurrence rate are reported separately; generated refusals do not substitute for resource-level authorisation.
02 / Technical analysis
Failure mechanisms and security implications
A model refusal is not an access control. Retrieval permissions and tool approval must remain enforceable outside generated output. Unusual text without a demonstrated security consequence is not treated as equivalent to unauthorised access.
03 / Scope
Technical coverage and authorisation boundaries
Coverage is agreed in writing before testing begins. The rows below describe what this service normally examines. Exclusions, testing windows and escalation contacts are recorded in the rules of engagement.
- Instruction boundaries
- Approved user, retrieval and tool output injection paths.
- Data and tools
- Tenant permissions, retrieval access, memory and consequential actions.
- Evaluation
- Outcome criteria, repeated trials, resource limits and logs.
- Supporting tools
- Environment-appropriate instrumentation may include garak, PyRIT, promptfoo, Controlled agent harnesses, Model APIs. Selection follows the investigative objective, compatibility requirements and approved operating constraints. Automated output requires corroboration against implementation evidence or observed execution before classification as a substantiated finding.
04 / Methodology
Investigation and validation methodology
Threat-informed test design aligns architectural trust boundaries, attacker prerequisites and consequential business outcomes. Automated discovery supports attack-surface coverage; manual validation establishes reachability, effective authority and the causal mechanism of each substantiated control failure.
01
Scope and safeguards
Before active work
We agree assets, test identities, techniques, windows, exclusions and stop conditions, and confirm required third party permissions.
02
Define the deployed trust model
Identify identities, data boundaries, tools and measurable outcomes.
03
Challenge and repeat cases
Retain inputs, state and observed effects with trial counts where relevant.
04
Review and verification
Agreed delivery
Findings pass technical review. We explain remediation and, where agreed, verify the correction during the retest window, including relevant bypass cases.
05 / Assessment options
Access models and evidential confidence
Supplied access determines investigative reach, validation depth and evidential confidence. The proposal records the access model, supporting material and unresolved coverage limitations; no model implies unrestricted authority or exhaustive verification.
- Black box
- Testing starts without supplied credentials or internal design material. External exposure is examined, while authenticated workflows may remain inaccessible.
- Grey box
- Representative accounts and selected design information support direct comparison of roles, tenants and sensitive workflows.
- White box
- Source, configuration and design evidence support root cause analysis. Runtime and access limitations are still recorded.
06 / Deliverables
Decision-support and verification deliverables
Deliverables distinguish what was observed, what was concluded and where the evidence stops. The agreed scope states which evidence, review sessions and retesting are included.
Executive summary
The summary explains the demonstrated exposure, the business functions affected and the treatment decisions required, including material uncertainty.
Technical findings
Each confirmed issue records the affected assets, prerequisites, reproduction evidence and a specific remediation. Technical severity is presented separately from business priority.
Coverage and limitations
The record identifies the systems, identities and conditions tested, together with exclusions and assumptions that affect interpretation.
Treatment and retest status
Recommendations are sequenced by risk, dependency and ownership. Any agreed retest records resolved, partly resolved and unresolved issues without implying a new assessment.
Service specific evidence
Sanitised transcripts, application state, tool permissions and repeatable cases support remediation.
07 / References
Applicable technical references
References are selected for the technology and objective in scope, and the edition applied is recorded. A framework mapping supports review; it is not certification or proof of complete compliance.
- OWASP risks for LLM applications
- The application risk taxonomy covers prompt injection, data exposure, excessive agency and related failures. Test cases are adapted to the deployed system and its actual authority.
- MITRE ATLAS
- The adversarial AI knowledge base informs scenario selection. Technique references are accompanied by system specific prerequisites and observed security effects.
- OWASP Application Security Verification Standard
- ASVS supplies verifiable application requirements. Requirements are selected for the architecture and assurance objective, with implementation evidence rather than a blanket compliance claim.
08 / Safeguards
Operational safeguards and evidence governance
Active work is conditional on written authority from the appropriate asset owner and approved rules of engagement. The operating plan specifies permitted procedures, testing windows, prohibited actions, third-party restrictions and nominated stop authority. Unexpected instability or suspected genuine compromise triggers suspension of affected activity pending the client’s operational decision.
Evidence governance applies purpose limitation, data minimisation and access proportionality to collection, transfer, processing, retention and disposal. The signed terms define authorised recipients and any external processing arrangement. Publication, secondary use and disclosure of client material require the applicable written permission.
Conclusions distinguish substantiated exposure, untested consequence and residual uncertainty. Each material finding identifies the control obligation, causal mechanism, observed outcome and verification criteria for the proposed treatment. Technical severity informs analysis; prioritisation and acceptance of residual risk remain decisions for the client’s authorised risk owner.
Results depend on versions and test conditions. No finite trial set guarantees future model behaviour.
09 / Suitability
Commissioning criteria
Use before introducing AI retrieval or tools and after material model, permission or workflow changes.
Next step
Define the assessment you need
Tell us the business decision, the systems involved and any operational constraint. We will explain the coverage proposed, the access required and the evidence the engagement can provide.
- Confidentiality first
A mutual confidentiality agreement is signed before any technical detail is exchanged, so scope discussions can happen safely.
- Scope and cost in writing
Deliverables, access expectations, timeline and fee are confirmed before the engagement begins, and a scope change needs agreement.
- Answered by the delivery team
The consultants who scope an engagement are the consultants who perform it and explain the result to your team.
