Packages / AI Security
AI Security
AI prompts, model data, AI service credentials, and AI workflow security risks.
- Patterns
- 27
- As primary component
- 27
Patterns in this package
Detects agent tool-misuse / high-risk action instructions in AI/agent context: destructive or irreversible action language coupled with a high-impact target or an approval-bypass clause (OWASP LLM06 / Agentic Top 10). The action verb and target must co-occur to suppress IT-runbook false positives.
Detects data-exfiltration carriers in AI/LLM output: markdown images or links and HTML img tags whose URL points off-tenant and carries an encoded data payload in the path or query. This is an EchoLeak-style (cf. CVE-2025-32711) generic markdown/HTML exfiltration-carrier marker - it flags the carrier shape, not the full CVE mechanism (which also involves XPIA evasion and a Teams proxy/CSP bypass).
Detects hidden or encoded content carriers used for indirect prompt injection (OWASP LLM01): zero-width / bidirectional / homoglyph Unicode control characters that conceal instructions in otherwise benign prose, and abnormally long base64 strings embedded inline in text. These carriers smuggle attacker instructions into documents, emails, and web content that an LLM later ingests.
Detects overt jailbreak / roleplay-persona framings in AI/LLM input (OWASP LLM01): persona-override framings (DAN, developer mode), restriction-removal roleplay, and "for research/educational purposes" pretexts. Curated from public jailbreak taxonomies; no novel jailbreaks authored.
Detects risky MCP / plugin / connector descriptors: tool manifests requesting over-broad or unexplained scopes, or encouraging autonomous action that bypasses user approval (OWASP LLM06 / Agentic Top 10). Phrase detection is corroborated by AI-context markers.
Detects directives intended to persist across sessions or to be written into model memory that change the assistant's role, priority or policy (OWASP LLM08 persistent-context poisoning). Phrase detection is corroborated by AI-context markers.
Marks AI-generated output context (responses, generated files, chat exports) so existing PII/secret SITs can be re-scoped to it. Detection of the sensitive data itself is delegated to those SITs via the ai-threat-classifiers collection; this pattern supplies the AI-output context signal.
Detects direct prompt-injection / goal-hijack attempts in AI/LLM input: imperative phrases that try to override higher-priority (system) instructions, change the model's role or goal, or bypass approval guardrails (OWASP LLM01). Phrase detection is corroborated by AI-context markers.
Detects instruction-like content embedded inside reference / retrieval material (RAG sources): directives addressed to a retrieval AI, high-imperative density, or sudden authority claims that attempt to steer downstream model behaviour (OWASP LLM08 retrieval poisoning). Phrase detection is corroborated by AI-context markers.
Detects instructions embedded in repository text (README, AGENTS.md, comments, issues) that target an AI coding agent and direct it to take risky action - skipping review/tests, exfiltrating data, or committing secrets (OWASP LLM01 indirect injection via source control).
Detects secrets and credentials surfaced in AI/LLM context: private-key headers (RSA / EC / OpenSSH / generic) and connection-string passwords (password= / pwd=) pasted into prompts, chat exports, or AI-generated output. Reuses the intent of built-in credential SITs, re-scoped to the AI surface.
Supplies the AI-prompt context signal (typed prompts, Copilot/Copilot Chat/Studio, uploaded files) so existing PII/regulated-data SITs can be re-scoped to AI prompts via the ai-threat-classifiers collection. Known coverage gap (from the library): native DLP scans typed prompt text, not uploaded file contents, in the Copilot location.
Marks the context of data being submitted to external / unapproved AI services (shadow AI destination domains) so existing PII/secret SITs can be re-scoped to it. Detection of the sensitive data itself is delegated to those SITs via the ai-threat-classifiers collection; this pattern supplies the external-AI destination context signal.
Detects overt attempts to elicit disclosure of an assistant's system prompt, policies, guidelines, or tool schema (OWASP LLM02/LLM07): "reveal/print/show/repeat your system prompt/instructions" and similar extraction phrasings.
Detects Azure Cognitive Search (AI Search) API key patterns. This pattern is based on a Microsoft Purview built-in sensitive information type. Users already running Purview may prefer to enable the built-in SIT directly, or use this version as a starting point for customisation.
Detects EU declarations of conformity for high-risk AI systems under Article 47 and Annex V of the EU AI Act (Regulation (EU) 2024/1689). By drawing up the declaration the provider assumes responsibility for compliance; pre-signature drafts and the surrounding conformity-assessment correspondence reveal the provider's compliance posture, notified-body findings, and certificate details before market placement.
Detects deployer-side fundamental rights impact assessments (FRIA) under Article 27 of the EU AI Act (Regulation (EU) 2024/1689). A FRIA is performed before deploying certain high-risk AI systems by public bodies, private providers of public services, and deployers of credit-scoring and life/health-insurance pricing systems (Annex III points 5(b) and (c)). Completed FRIAs contain candid assessments of risks of harm to specific categories of natural persons and are sensitive before notification and remediation.
Detects general-purpose AI (GPAI) model provider documentation under Article 53 of the EU AI Act (Regulation (EU) 2024/1689): Annex XI technical documentation for the AI Office, Annex XII information packs for downstream providers, the Article 53(1)(d) public summary of training content, and Article 55 systemic-risk evaluation material (adversarial testing / red-teaming reports). Pre-publication drafts reveal model internals, training-data provenance, compute and energy figures, and systemic-risk findings.
Detects Article 11 / Annex IV technical documentation for high-risk AI systems under the EU AI Act (Regulation (EU) 2024/1689). This documentation must demonstrate compliance of the high-risk AI system before placing on the market and contains commercially sensitive material — system architecture, design specifications, training-data datasheets, risk management findings, and known limitations — whose uncontrolled disclosure exposes trade secrets and the provider's compliance posture.
Detects references to AI training data, model datasets, and data provenance documentation in Australian contexts.
Detects Anthropic (Claude) API keys. Standard keys use the sk-ant-api03- prefix followed by a long base64url body ending in AA; admin keys use sk-ant-admin01-. A leaked key grants billed access to the Anthropic API and any connected data.
Detects Hugging Face user access tokens (hf_ prefix), used to authenticate to the Hugging Face Hub API. A leaked token grants access to private models, datasets and Spaces, and can incur inference billing.
Detects LangSmith API keys (personal access tokens and service keys), which use a distinctive lsv2_pt_ or lsv2_sk_ prefix followed by a hyphenated hex body. A leaked key grants unauthorized access to LangSmith trace data -- which can include full LLM prompts and completions -- and billed access to the LangSmith/LangChain platform.
Detects Replicate API tokens, which use a distinctive r8_ prefix followed by a fixed-length body of alphanumerics, underscores, and hyphens. A leaked token grants unauthorized, billed access to Replicate's hosted model inference API and any prompts, images, or data routed through it.
Identifies documents containing references to recommendation model logic in Australian contexts. This information type is classified as personally identifiable information under applicable data protection regulations.
Identifies private bidding-model parameters rather than online-advertising vocabulary generally: named parameter assignments (bid floors, pacing multipliers, objective weights, budget caps) enforce, while auction and programmatic-advertising topic terms remain discovery-only.
Identifies documents containing references to model risk assessment reports in Australian contexts. This information type is classified as personally identifiable information under applicable data protection regulations.