Page 146 - CyberSecurityEssentialsEnglish

The problem: the team proposed using a general chatbot and uploading the regulations to
it. An initial test showed good language, but it mixed two versions and answered a question
the documents do not cover. The problem is not the prompt alone; it is version

management, retrieval, attribution, and permissions.

The alternatives: conventional search inside the website provides the source but depends
on wording and may return many pages. A general chat model explains quickly but cannot

guarantee that the regulations are current. A ready specialized tool may reduce build time
but requires testing for Arabic, integration, and data policy. A custom retrieval system allows
control over documents, versions, and citation, but it requires operation, monitoring, and a

responsible team.

The gates: it was determined that the data is public, but that the system may not make a
final decision in a student’s case, and may not answer without a page or a link. It must retain
a log of the question and the retrieved passages, and allow an old version of a regulation to

be retired. Right-to-left layout must work in both the interface and the export.

The retrieval design: the regulations are collected from an official source, and each file is
given a version number, an effective date, and a scope. The documents are divided by article
and heading rather than by a fixed character count alone, and the page, the issuing body,

and the version are stored in the metadata. Retrieval combines legal keywords with semantic
search, and the passages are then reranked. The model is instructed to answer only from

the passages, to show the article and the page, and to write “the available documents are
not sufficient” when evidence is absent.

The evaluation set: cases were built in standard Arabic and in dialect, with abbreviated
phrasings, questions that depend on a table, questions whose answer lies in the middle of a

long file, questions where an old and a new version conflict, and questions outside the scope.
A specialist member of staff reviews the answer and the source. Retrieval accuracy, citation
faithfulness, the rate of correct abstention, review time, and language and layout problems

are all measured.

The decision: the system is not launched as an automated academic adviser. It is adopted as
a research and first-line guidance assistant with official links and a clear disclaimer, while
individual cases, decisions, and exceptions remain with the specialist member of staff. The

launch begins with a limited group, failing questions are reviewed weekly, and the index is
rebuilt when a new regulation is issued.
The lesson: the model was not the decisive element on its own. Success came from source

and version management, the design of retrieval, the abstention rule, Arabic testing, and the
review process. A mid-sized model inside a good system could well outperform a larger
model working from disorganized files.

145