TAI #37: Do We Really Understand Risk Control Effectiveness?

Dear colleagues:

In many risk files, effectiveness of risk control measures is documented with a reference to design verification and/or design validation. The control was implemented, testing passed, the product was validated—and the control is considered effective.

In some cases, that may be entirely appropriate. When a risk control measure directly addresses a well-defined product safety requirement, such as an applicable IEC 60601 requirement with established test conditions and acceptance criteria, passing the required test can provide sufficient evidence of effectiveness for that narrowly defined control.

But many individual risks of harm are far more complex.

Under ISO 14971, we manage the risk of harm, not simply the risk of failure. A failure may initiate the sequence, but harm may occur only after a cascade of events and conditions creates a hazardous situation. Multiple risk control measures may act at different points along that pathway: one may prevent an initiating event, another detect a developing condition, another interrupt progression, and labeling or training may influence what happens next.

Yet our risk analyses do not always make that failure trajectory visible. FMEA may identify individual failure modes and effects, but it can leave the sequence connecting failure, hazardous situation, and harm fragmented across the analysis.

And that creates a much larger effectiveness problem.

If we do not understand how events progress toward harm, we cannot clearly define what the risk control strategy is expected to change. If we cannot define that expected change, meaningful effectiveness criteria become difficult to establish.

Risk control effectiveness begins with understanding what the control strategy is expected to change in the pathway to harm.

Once that expected effect is explicit, effectiveness becomes much more tangible. We can define what should be prevented, detected, interrupted, or mitigated; establish measurable criteria; and determine what evidence is needed to demonstrate that the combination of controls produces the risk reduction relied upon in the residual-risk conclusion.

The same logic extends into post-market monitoring. If we know where and how the control strategy is expected to alter the trajectory, we can identify signals that its effectiveness may be weakening. Without that understanding, surveillance is often left looking mainly for downstream failures, complaints, or harms—after the control strategy may already have begun to behave differently from what we assumed.

A design validation reference may therefore demonstrate that the product was validated. It does not automatically demonstrate, at the level of an individual risk of harm, why the control strategy is effective, how its effectiveness should be measured, or how loss of effectiveness should be detected over time.

So let’s think about the following question:

👉For your most critical individual risk of harm, can you explain the failure trajectory and show exactly how the combination of risk control measures interrupts it?

As an example, consider the following prompt in the LTR Risk Coach:

Select one significant individual risk of harm from our ISO 14971 risk file. Help me distinguish the initiating failures or events from the hazardous situation and harm, then map the sequence of events connecting them. Identify where each risk control measure acts on that trajectory, what assumptions or dependencies affect its performance, and what evidence demonstrates that the combined control strategy is effective—not merely implemented or verified.

You might also enjoy thinking about these ideas: