Page 127 - CyberSecurityEssentialsEnglish

TABLE 3-7: WHERE DID THE RETRIEVAL SYSTEM FAIL?

The likely
The symptom The diagnostic test The remedy
layer

Improve cleaning,
The right Check recall by hand on
Indexing or metadata, query
document never questions whose
the query handling, and hybrid
appears answers are known.
search.

The document Compare the ordering Use a reranker, or
Retrieval or
appears but ranks before and after filters on domain and
reranking
poorly reranking. date.

The passage is right Present the passage Constrain the answer,
Generation or
but the answer is alone and ask for a improve the quotation,
instructions
wrong constrained answer. or change the model.

The source is right Check every claim Ask for a specific
but the claim is Attribution against the supporting quotation and reduce
broader sentence. the room for inference.

Unauthorized Apply access control
Retrieval Test accounts with
information before retrieval and log
permissions different roles.
appears access.

Ask a question whose Add effective dates and
An outdated Knowledge
answer differs between priority, and retire
version is used management
two versions. revoked versions.

Fine-Tuning and LoRA

Fine-tuning updates some or all of a model’s weights using new examples. It helps when you
want repeated behavior: classification in a consistent style, sector-specific language, a
response format, or a specialized task with representative examples. But it requires good

data, a separation between training and evaluation, and monitoring for bias and for
regression in other capabilities.

LoRA is an efficient tuning method that freezes the original weights and adds trainable low-

rank matrices, which reduces the number of parameters and the memory required
compared with full fine-tuning [7]. This does not turn the process into a simple button; data

quality, evaluation, licensing, hosting, and updating all remain fundamental issues.

If the problem is that a regulation changes every month, retrieval is usually more appropriate
than training a model on a version that will become outdated. If the problem is that the

126