Page 87 - CyberSecurityEssentialsEnglish

● Remember: Section summary
• Key idea: A structure that validates is not the same as content that is correct.
• Practical decision: Check the shape of the output programmatically and the values of
the output against the source.
• Mistake to avoid: Accepting a record because it matches the schema, or asking for
depth without defining the end point.

2.10 Critique, Sycophancy, and the Limits of Self-Review

◆ Key idea: turn critique into criteria and evidence.
Asking a system to “review your answer” may produce cosmetic changes or introduce a
new error. Stronger review supplies a criterion, a correct example, a source, or an
external test, and asks for the gap to be identified before anything is rewritten.

The system can be used in the role of a critic, but the production of a draft must be separated
from its evaluation. Ask first for an assessment of the output against a clear rubric: accuracy,
coverage, clarity, fidelity to the source, and feasibility. Then ask it to state where the defect

lies and what its effect is, without rewriting immediately. Only after you have reviewed the
diagnosis should you ask for the revision.

Research on intrinsic self-correction indicates that models may be unable to correct
reasoning errors from their own capabilities alone, and that results can sometimes
deteriorate when review is requested without external feedback [12]. Tie critique, therefore,

to a source, a reference solution, a test, or an expert. Self-critique is a tool for exploration,
not a certificate of quality.

TABLE 2-13: DEGREES OF STRENGTH IN REVIEWING AN OUTPUT

Type of review Example Its strength

Weak; it identifies neither the
General impression Review and improve.
defect nor success.

Better for diagnosis, but a
Rate accuracy, coverage, and clarity
Defined criteria rating produced by the model is
from 1 to 5 with evidence.
not a final judgment.

Comparison with a Compare every claim with the text and Strongest when a reliable
source identify the page. reference exists.

Run the calculation, the code, or the Strongest for measurable
External test
test cases. results.

Specialist human Review suitability, context, and Necessary in high-impact
reviewer consequences. decisions.

86