Page 114 - CyberSecurityEssentialsEnglish

This is why you should record the identity of the system, not only the name of the model.
That identity includes: the application or API endpoint, the model name or alias, the version
or date when available, the region or subscription plan, the tools that are enabled, and the

retention policy. If the provider does not publish the details, record what you can measure:
the date of the test, the visible functions, the file types, the behavior, and the limits.

A silent update is an operational risk. The language may improve while the JSON formatting
weakens, the refusal behavior may change, or table handling may differ. Repeated tasks

therefore need small regression tests that are rerun after an update. The purpose is not to
freeze the technology, but to detect the change before it spreads into hundreds of outputs.

▣ Practical example: a change no one announced
A meeting transcription tool began separating speakers differently after an update. The
product name did not change, but the task extraction system failed because the field

names changed. The correct test was on the whole chain: audio, transcription, speaker
separation, and the structured output, not on text quality alone.

● Remember: Section summary
• Key idea: The product is a system of layers, and the model is only one of them.
• Practical decision: Record the identity of the system and keep small regression tests for
repeated tasks.

• Mistake to avoid: Reading every difference in behavior as evidence about the
intelligence of the model.

3.3 Why Do Results Differ? Probability and Context

◆ Key idea: sampling explains only part of the variation.
Generative language is probabilistic, but the difference does not come from sampling
alone. The version, the context, the system instructions, the tools, the search, the file
handling, and the platform policies all enter into the result.

Tokens, the Probability Distribution, and Decoding
Language models divide text into tokens, and a token may be a word, part of a word, or a
punctuation mark. At each step the model computes a probability distribution over the next

token given the current context, and a decoding mechanism then applies a rule to select one
token. The process repeats until the output is complete.

113