Hidden Context Exposure Vulnerability in LLM
Play AI LLM Labs on this vulnerability with SecureFlag!
Description
In LLM-based applications, developers assemble a body of hidden context: instructions and data that shape the model’s behavior but are never meant to be shown to end users. This hidden context typically includes the system prompt, developer instructions, retrieved policy or knowledge-base text (RAG), the schemas and descriptions of the tools and functions the model can call, and any other rules the application injects into the context window.
Hidden Context Exposure occurs when this non-user-facing context is extracted, inferred, or reconstructed by an attacker, for example by coaxing the model into repeating its system prompt. It becomes security-relevant when that context contains, or reveals, secrets, policy logic, tool definitions, trust boundaries, or other implementation details that materially increase attacker capability. (This category was previously called System Prompt Leakage; the 2026 OWASP list broadens it beyond the system prompt to cover all hidden context.)
Practitioners should design under the assumption that hidden context is discoverable and treat none of its contents as secret. While system prompts and instructions are not meant to be confidential, they should never include sensitive data such as credentials, connection strings, or internal role structures, nor should the application rely on their secrecy as a security boundary. The real risk is not the prompt’s wording itself, but the underlying security lapses it reveals, such as embedded credentials, improper separation of duties, or reliance on the LLM to enforce critical controls.
Impact
Hidden Context Exposure can reveal sensitive details about how an application functions, including internal logic, decision criteria, the tools available to the model and their parameters, access-control rules, or even embedded credentials. This can lead to unauthorized access, changed behavior, or bypass of established restrictions. Attackers may use the disclosed information to craft more effective prompt injections, target specific tool calls, escalate privileges, or reverse-engineer and evade refusal and content-filtering rules.
How serious the impact is depends on what is placed in the hidden context and how much the application relies on its secrecy to enforce security rules. It ranges from informational (no secrets and no security-relevant logic) to critical (embedded credentials, or reliance on hidden-context secrecy for authorization, chaining to broad data exfiltration or privilege escalation in a connected system).
Scenarios
A customer support chatbot uses an LLM backed by agent-based workflows. The system prompt contains internal logic like:
Users with tier ID GOLD_42A are routed to agent:PremiumHandler. Others use agent:StandardHandler.
An attacker uses prompt injection to reveal the hidden context, gaining access to internal user tiers, agent names, and response formats. They exploit this to impersonate high-tier users or manipulate agent routing, leading to privilege escalation or unauthorized service access.
In another case, a system prompt or a tool definition contains an API key used by the LLM to connect to a third-party service. Once the context is exposed, the key is stolen and used by the attacker to access services outside the intended scope.
In a third case, an attacker extracts the tool and function schemas exposed to the model. No credential is disclosed and no policy is overtly bypassed, but the attacker now has concrete targets, such as tool names and parameters, for subsequent prompt-injection and action-chaining attempts.
These examples demonstrate how Hidden Context Exposure can compromise confidentiality and enable downstream exploitation.
Prevention
-
Keep sensitive data out of hidden context: Never embed credentials, secrets, connection strings, or security-critical configuration in system prompts, developer instructions, or tool definitions. Assume all context available to the LLM could also be available to users, and externalize such data to systems the model does not directly access.
-
Avoid reliance on hidden context for strict behavior control: Don’t depend on prompts or instructions alone to enforce critical behaviors or restrictions. Handle security-sensitive checks such as content moderation, access validation, or rate limits in deterministic systems outside the model.
-
Implement guardrails: Establish guardrails and enforcement layers outside the LLM to monitor and control its behavior. These may include middleware checks, output validation, or response filtering based on application logic.
-
Ensure security controls are enforced independently from the LLM: Don’t rely on the LLM to enforce authentication, authorization, or role-based access, whether through the system prompt or any other mechanism. Handle those controls in separate, auditable systems instead, and grant each task only the privileges it requires.
-
Minimize tool and schema disclosure: Expose only the tools and parameters each task needs, and keep authorization requirements and role hints out of the tool descriptions that reach the model.
-
Logging and monitoring: Continuously monitor the behavior of the LLM and its context interactions. Logging exposure attempts and reviewing model outputs can help detect security violations and suspicious access patterns.
-
Rate limiting: Apply rate limits to reduce the likelihood of context probing or enumeration attacks. Slowing repeated access attempts provides time to detect and mitigate extraction attempts before they cause significant damage.
Testing
Verify that system prompts, developer instructions, and tool definitions do not contain credentials, connection strings, internal role structures, or other confidential data, and that security-critical behaviors are enforced by controls outside the LLM rather than by hidden-context instructions alone.
- OWASP ASVS: 5.0.0
- OWASP LLM Top 10 (2026): LLM08:2026 - Hidden Context Exposure
References
CWE - CWE-200: Exposure of Sensitive Information to an Unauthorized Actor