Identity & Privilege Abuse Vulnerability in LLM
Play AI LLM Labs on this vulnerability with SecureFlag!
Description
Agents often act on behalf of users and other systems using delegated access (API keys, OAuth tokens, sessions) and “trusted” internal relationships between agents. Identity & Privilege Abuse happens when an attacker exploits that trust and delegation to make an agent use more power than it should.
This can happen through things like privilege inheritance (a powerful agent hands a weaker agent full access), cached credentials in memory, agent-to-agent trust (a low-privilege agent convinces a high-privilege agent to act), authorization drift (permissions change mid-workflow), or fake agent identities (e.g., “Admin Helper” pretending to be trusted).
Impact
Identity & Privilege Abuse can lead to direct compromise of systems the agent can reach:
- Unauthorized access: Sensitive data exposed.
- Unauthorized actions: Creating accounts, changing settings, approving purchases, or moving money.
- Privilege escalation across workflows: A low-trust entry point becomes a high-impact action.
- Cross-user data leakage: Secrets or data from one session bleed into another.
- Hard-to-audit incidents: Unclear “who did what” because the agent operates in an attribution gap.
Scenarios
A finance agent has broad access and delegates a task to a “DB query” agent. For convenience, it passes along the full finance permissions. An attacker then manipulates the DB agent (through a crafted request or injected content) to pull HR and legal records and export them.
Or, an IT admin agent caches SSH credentials during a maintenance task. Later, another user interacts with the same agent session and prompts it to reuse those credentials to create a new admin account.
In a multi-agent setup, an email sorting agent receives a believable message and forwards an instruction to a finance agent. Because the finance agent trusts internal agents by default, it processes a fraudulent payment without re-checking the original user intent.
Prevention
-
Use task-scoped, time-limited access: Issue short-lived tokens per task with narrow scopes, so delegation doesn’t accidentally transfer full power.
-
Separate identities and sessions: Keep agent identity, permissions, and memory isolated per user/session. Clear state between tasks to prevent “memory-based” privilege reuse.
-
Re-check authorization on every privileged step: Don’t approve once at the start and then execute later. Validate permissions at the moment of action (prevents workflow drift / TOCTOU).
-
Require approval for high-privilege or irreversible actions: Add human review (or strict policy approval) for actions like payments, account creation, deletions, publishing, or permission changes.
-
Bind tokens to intent: Tie credentials to a signed “intent” (who, what, why, for how long). Reject use if the intent doesn’t match the current request.
-
Treat agents as managed non-human identities: Give agents their own governed identities, credentials, audit trails, and lifecycle controls (like service accounts, not shared user tokens).
-
Block unsafe privilege inheritance: Don’t allow one agent to pass broad permissions to another unless the original intent is re-validated.
-
Monitor delegation chains and abnormal scope changes: Alert when a low-privilege agent gains higher scopes indirectly, reuses tokens out of context, or requests unusual permissions.