Page 174 - CyberSecurityEssentialsEnglish

FIGURE 4-5: PROMPT INJECTION AND JAILBREAKING ARE NOT THE SAME ATTACK

What to notice: Both end in the system doing something it should not, but injection is
reduced by distrusting content and jailbreaking by limiting capability.

Source: Prepared by the author.

● Remember: Section summary
• Key idea: Connecting a model to external content and tools turns text errors into
security and execution risks.
• Practical decision: Treat both inputs and outputs as untrusted data, and reduce the

permissions.
• Mistake to avoid: Relying on text instructions inside the prompt as though they were
complete security protection.

4.7 Discern: How Do You Verify Facts and Sources?

◆ Key idea: verify the claim, not your impression of the text.
Verification begins by turning the output into checkable claims: a name, a figure, a date,
a causal relation, a quotation, a reference, or a recommendation. Each claim has its own
appropriate kind of evidence. Do not ask the model alone to testify for itself; return to a
primary source, an independent body, or an expert according to the impact.

Hallucination is the production of information that looks plausible but is unsupported or
untrue. It may be an entirely invented reference, the merging of two facts, an added figure,

or the filling of a gap in a document. This chapter does not redefine the phenomenon
theoretically; it focuses on detecting it and preventing it from turning into a decision.
Begin by extracting the claims. The sentence “complaints fell by 30 percent after the

initiative” contains a figure, a temporal relation, and possibly a causal one. Ask: where is the
original count? What is the period? What is the definition of a complaint? Was the decline

173