Page 80 - CyberSecurityEssentialsEnglish

The design of your test should therefore state how much variety is acceptable, rather than
assuming that every task needs a literally identical result.

Part of this variation comes from settings that control how the next unit of text is selected.
Temperature adjusts how much randomness enters that selection, and top-p limits how

many candidates are considered at all: lower values make the output more repeatable and
more conservative, and higher values make it more varied. Some tools also expose a seed

that makes a run reproducible. Chapter One explained that generation is probabilistic, and
these settings are the dial on that probability. It is important to understand what they do
not do: they change the diversity of the wording, not the knowledge available to the model.

Lowering the temperature does not make an incorrect statement correct, and raising it does
not add a source.

Test the prompt several times and on varied cases: an easy case, a routine case, an edge
case, a case with missing information, and a case that invites guessing. Watch whether the

facts and constraints stay fixed, whether only the style changes, or whether the substantive
result changes. Record the rate of compliance with the requirements and the type of failure,
instead of relying on an impression of the best output you happened to see.

If you need high consistency, reduce the open space in the task: use an approved list of

categories, boundary examples, a policy for missing values, a structured schema, and a
validation tool. If the goal is creativity, define the domain, the constraints, and the criteria

for variety, then deliberately ask for different alternatives. Stability is not an absolute virtue;
it is the matching of the degree of variation to the function of the output.

Do not tune a prompt on a single example until it becomes excellent for that example and
fails on everything else. This resembles overfitting to your own cases. Keep a local evaluation

set that you have not used in every round of editing, and test the candidate version on it
before adopting it. When the model or the tool changes, test again, because a prompt that
worked before may behave differently.

● Remember: Section summary
• Key idea: Language, medium, decision support, and variation are all part of the

specification, not decoration.
• Practical decision: State the Arabic and file decisions explicitly, separate facts from
inference in any comparison, and match the degree of variation to the function of the
output.
• Mistake to avoid: Assuming the model read every part of a file, or believing that a lower
temperature makes an answer true.

79