Page 139 - CyberSecurityEssentialsEnglish
P. 139
Total Cost of Ownership and the Cost of Quality
Total cost of ownership includes the subscription or the calls, integration, storage, security,
training, support, monitoring, and leaving the vendor. The cost of quality includes the time
spent verifying, correcting, and reworking, and the cost of failure cases.
A model may cost more per call yet cost less across the process if it reduces review, and a
cheap tool may become expensive when it requires manual reformatting or provides no log
or export. Compare the cost of completing an acceptable task, not the unit price alone.
● Remember: Section summary
• Key idea: A fair comparison needs a local set, repeated runs, blind scoring, and an error
taxonomy.
• Practical decision: Report the distribution and the worst case, not the average alone,
and count review time in the cost.
• Mistake to avoid: Declaring a winner from a demonstration, or from a one-point
difference on three cases.
Video: Evaluating a Generative Model in Practice
A practical demonstration of evaluation concepts,
reference data, measures, and comparison
between versions and configurations.
Duration: about 15 minutes | Level: Intermediate
Learning objective: Turn evaluation from an
impression into a documented, repeatable
protocol.
Source: Google Cloud Tech [26].
https://www.youtube.com/watch?v=AUSunZXC2rg
Verified: June 6, 2026
3.9 Testing Arabic Seriously
◆ Key idea: Arabic support is not a yes-or-no question.
You must test standard Arabic, dialects, code switching, terminology, names, numbers,
right-to-left layout, and files, inside the real task.
What Do the Published Benchmarks Measure?
Published benchmarks give useful indications but do not replace local testing. ALUE
measures a set of Arabic language understanding tasks, and ORCA brings together sixty
datasets across seven families of understanding tasks and covers several Arabic varieties
[11][12]. ArabicMMLU offers forty tasks and 14,575 multiple-choice questions in standard
Arabic drawn from different educational levels and Arab countries [13].
138
