Page 46 - CyberSecurityEssentialsEnglish

● Remember: Section summary
• Key idea: Capability is uneven, so two tasks that look similar to a person can sit on
opposite sides of the limits of a system.
• Practical decision: Test the specific task with your own cases instead of generalizing
from a success on another task.
• Mistake to avoid: Asking whether the model is powerful rather than whether this task,

with this data and context, falls inside its capability.

1.9 Why Does Human Judgment Remain Essential?

◆ Key idea: People define the goal, review the output, and remain

responsible for the decision.
The human role begins before the system is used. People define the quality standard and
the boundaries of what can be delegated, then examine the evidence and context.
Human review is not a ceremonial approval; it requires time, knowledge, and the
authority to revise or reject the output.

People decide what is worth solving in the first place. A system can optimize a stated
objective, but it cannot automatically decide whether that objective is fair or beneficial. If

asked to reduce customer-service time, it may shorten interactions at the expense of
understanding. If asked to increase the number of accepted applicants, it may neglect the
quality of fit. A strong system cannot compensate for a poorly defined objective.

People understand context that data does not fully capture: relationships among parties, the
sensitivity of timing, unwritten information, and values that cannot be reduced to a number.
They can also interpret the significance of an exception. A model may treat a rare case as

statistically unimportant, while an expert recognizes that it involves a serious risk or a right
that must not be ignored.

People remain accountable when a decision has human consequences. AI can summarize a
file or suggest questions, but a physician, teacher, researcher, or manager does not give up
responsibility because a system recommended an action. The division of work must

therefore be explicit: what does the system produce? What does a person review? Who
approves the decision? What happens when there is uncertainty?

Two opposing risks exist. The first is automation bias: accepting an output because the
system presents it confidently. The second is rejecting all automated assistance even when
evidence shows that it helps. A professional approach is to trust conditionally, on the basis

of evidence. Skill erosion is another concern: if people stop practicing analysis, writing, or
calculation, their ability to detect system errors may decline.

45