Page 382 - CyberSecurityEssentialsEnglish

Video: Security of Agents and Autonomous

Systems
A specialist session reviewing the risks of agents

that plan, use tools, and carry out real steps on an
organization’s behalf.
Duration: about 60 minutes | Level: Advanced

Learning objective: Tell conversational risk apart
from executive-capability risk, and see why

identity, permissions, and memory matter.

Source: OWASP GenAI Security Project [19].
https://www.youtube.com/watch?v=-vXoC0UvpjY
Verified: June 6, 2026

Threat Modeling for Generative and Agentic Systems
Threat modeling begins by drawing the assets, the boundaries, and the flows: which data or

secrets are we protecting? Who are the users and the potential adversaries? Which tools
and external sources are involved? Where does trust pass from one component to another?
In a retrieval system, for example, content may enter from an external document into an
index and then into the prompt; in an agent, text may turn into a decision and then into a

tool call. Every transition creates a different attack surface.

Assume three things. Untrusted content may carry hidden instructions. An output may reach
a system that does not expect harmful text. And memory may keep information that should

not stay. Controls include separating data from instructions, filtering inputs and outputs,
validating structure, isolating execution, allowlisting tools and destinations, managing
secrets outside prompts, and logging significant events.

No wording can prevent every prompt injection. A system is therefore built on the principle

of minimizing the impact of a successful attack: if the model is deceived, what is the most it
can do? When permissions are limited, writes to systems require approval, transactions have
ceilings, and destinations are restricted, a failure turns from a wide incident into a content

error that can be contained.

The incident plan must include agent-specific scenarios: ending a session, revoking tokens,
suspending tools, preserving the execution chain, establishing which actions were actually
carried out, and notifying the owners of affected systems. Deleting the conversation is not

enough; the agent may have sent a message, modified a record, or created an external
commitment.

381