Democracy’s eternal vulnerability: Part 2. WHY have we been unable to solve the global Problematique since it was first identified in 1970?

WHY have we been unable to solve the problem?

As described in Part 1, global society faces a staggering array of high-impact common good problems that have gone unsolved for fifty years or more. This class of problems, identified by the Club of Rome in 1970 as the global Problematique, includes these eight highly interconnected problems:

1. Environmental sustainability (including climate change)

2. Backsliding from democracy to authoritarianism

3. High inequality of income and wealth

4. War

5. Endemic government corruption

6. Recurring large recessions and depressions

7. Systemic discrimination of many kinds

8. The power of large corporations

Fifty-six years later, none of these problems have been solved. WHY is this?

It is certainly not for lack of trying. Nor is it for lack of brilliance. These problems have attracted many of the sharpest minds on the planet, who have worked tirelessly for decades to solve them. They have educated the public about the magnitude and urgency of the problems, and what solutions would work if implemented. They have lobbied and cajoled politician after politician. They have organized and written countless articles and books, many based on in-depth well-managed research projects, with the three editions of The Limits to Growth in 1972, 1992, and 2004 being the prime example.

But none of this has worked. So again, WHY has society been unable to solve the problem?

The paper argues the reason is extreme problem complexity. The complexity of this class of problems vastly exceeds the analytical capacity of current problem solving methods, which rely mainly on intuition and expert judgement, supplemented with tools like simulation modeling, correlation analysis, case studies, and experimentation. This approach works well for less complex problems. However, it fails when faced with problems of ultra-high complexity. This failure has been total. Not a single one of the eight problems has even a glimmer of a successful solution in progress. Most are getting worse, particularly the environmental sustainability problem. This caused the authors of the third edition of Limits to Growth in 2004 to sound a dire warning:

… we are much more pessimistic about the global future than we were in 1972. It is a sad fact that humanity has largely squandered the past 30 years in futile debates and well-intentioned, but halfhearted responses to the global ecological challenge. We do not have another 30 years to dither. Much will have to change if the ongoing overshoot is not to be followed by collapse during the twenty-first century.
(p xvi. The authors are rounding, since 2004 - 1972 = 32 years.)

Where the researchers behind the Limits to Growth project erred was this key assumption: When the world was presented with a simulation model convincingly showing that unless environmental impact was radically reduced, collapse is unavoidable, world leaders would take appropriate action. All activists needed to do was raise awareness of the problem and prove the crisis exists, and solution would follow.

That was a false assumption, because it ignored systemic change resistance (as described in Part 1). Because of this the system resisted change and humanity squandered not just the 30 some years leading from 1972 to 2004, but the 54 years from 1972 to 2026. The problem solving process used, system dynamics simulation modeling driven by intuition, expert opinion, and systems thinking, was inadequate because the process could not accommodate the problem’s ultra-high complexity.

If you can’t solve a problem, turn to those who can

Fortunately, problems of high complexity have long been solved by industry. At the core of their approach lies the powerful tool of root cause analysis (RCA). RCA solves causal problems by finding and resolving their root causes. A causal problem occurs when problem symptoms have causes, such as illness or a car that won’t start. Examples of non-causal problems are information search problems (such as scientific discovery or crime solving), math problems, and puzzle solving. The golden rule of RCA is All causal problems arise from their root causes.

RCA works by starting at problem symptoms and asking “WHY does this occur?” until the root causes are found. A root cause is the deepest cause in a causal chain (or the most basic cause in a feedback loop structure) that can be resolved with practical solutions, without side effects that create other equal or bigger problems. Resolved means the problem will probably not recur due to that root cause. RCA is the systematic practice of finding, resolving, and preventing recurrence of the root causes of causal problems.

While the business world uses RCA mainly to maximize profits, RCA is applicable to any problem type. NASA uses it to increase safety. Hospitals use it to improve quality of care. Social scientists have used it for finding the root causes of health inequity (Weinstein et al., 2017), finding the root causes of racial disparity (Conley et al., 2024), and finding the root causes of biodiversity loss (Wood et al., 2000).

However, social problem application is rare and results have been meager. None of the three social science applications listed above led to major impact. Why? As you might have guessed, because no RCA was actually used, even though all claimed to be finding root causes.

For example, The Root Causes of Biodiversity Loss (Wood et al., 2000), a book reporting on a World Wildlife Fund project, was the closest we came to finding a true application of RCA to a social problem. It utilized “a framework for analyzing socioeconomic root causes of biodiversity loss” (p11) with these steps:

1. Perform a literature review to find the first hypothesis of the root causes.

2. Develop a first iteration conceptual model of the problem using these root causes.

3. Collect data to complete the model.

4. Revise the model as needed when new data disagrees with the model until the model is stable and complete.

The book defined root cause as “the set of factors that truly drive biodiversity loss, but whose distance from the actual incidence of loss, either in space or time, makes them a challenge to identify and remedy.” (p3) But nowhere in this definition or the four steps is the Five Whys practice of asking “WHY does this occur?” until the root causes are found. Instead, analysts are searching for “factors that truly drive biodiversity loss,” with no method of finding those factors other than intuition and “perform a literature review.” If a problem has not been solved, how can analysts realistically expect to find the correct root causes in the literature? All they are likely to find is intermediate causes or wrong root causes.

That was the case here. Page 62 contains a table of the 13 “common causes” found. These are population growth, poverty, immigration, inequality, isolation/marginalization, cultural changes, macroeconomic policies, international trade policy, policy failures, domestic market factors, poor environmental law/weak enforcement, unsustainable development projects, and lack of local control over resources.

True RCA requires inspection of the actual system, not the literature. The causes found are a fairly arbitrary collection of intermediate causes and were intuitively derived, despite the process and many conceptual model diagrams. One cause, policy failures, cannot be a cause because policies are solutions, not causes. Saying policy failures is a root cause is the same as saying “Our policies are failing because of policy failures,” which is the circular reasoning fallacy.

All efforts to apply RCA to social problems we examined were similarly weak. Why? Because none used an RCA wrapper process that was tailored to fit difficult large-scale social problems and performed true RCA. The 4-step process above is not true RCA and is not tailored to social problems.

Wrapper processes

The basic RCA process is generic and must be wrapped in a process suitable for a particular class of problems. Examples of widely used wrapper processes are:

1. Six Sigma for process control (Pyzdek, 2003). Six Sigma uses statistical techniques and other tools to identify and resolve root causes of process defects. Six Sigma is so popular it’s been institutionalized in the form of international standards, certifications, and levels of practitioner competency: white belt (awareness), yellow belt (foundational), green belt (project leadership), black belt (change leader), and master black belt (enterprise strategy).

2. Lean for manufacturing (Womack et al., 1990). Lean is a direct copy of the Toyota Production System, widely acknowledged as the most advanced RCA-based large-scale process in the world.

3. MECE issue trees, the tool used by the world’s top three business managements consultancies (McKinsey, Bain, and BCG) for the world’s toughest business management problems (Chevallier, 2016). MECE stands for mutually exclusive, collectively exhaustive, and is a diagram-based decision tree process for systematically searching a problem for its root causes.

4. Fault tree analysis (FTA) for system failure incidents (Ericson, 1999). FTA is a form of failure analysis, such as airplane accidents, in which the undesired state of a system is methodically examined to pinpoint the root causes of failure. FTA can also be used in system design for failure prevention, and is widely used in aerospace, nuclear power, chemical and process, pharmaceutical, petrochemical and other high-hazard industries.

RCA-based processes have proven so effective that Six Sigma, invented at Motorola in 1986, is used by 100% of aerospace, motor vehicle, electronics, and pharmaceutical companies in the Fortune 500 and 82% of all companies in the Fortune 100 (Marx, 2007). Lean, based on the Toyota Production System perfected by Toyota beginning in the 1950s, has become the global best practice for large-scale manufacturing (Nguyen, 2018).

Large scale industry’s commitment to RCA-based processes is so total that we can draw this insightful conclusion:

An extraordinary claim

That’s an extraordinary claim. The high-quality portion of the claim follows logically. Governments and large corporations are both in the business of doing the same thing: creating high-quality low-cost products and services of any type at large scale for their customers. For governments the customer is citizens. Creating products and services at large scale requires a production process. The quality of a process is measured by defects per opportunity to please the customer. All defects arise from root causes and only from root causes. As the root causes of defects are resolved a process is continuously improved, until extraordinary high levels of quality are achieved. This outcome is guaranteed if the RCA-based process is properly applied, and ultra-high levels of quality are routinely achieved. From this follows the high-quality portion of the claim.

The low-cost portion follows historically. Formal RCA originated with the “King of Japanese Inventors,” Sakichi Toyoda (1876-1930), in the early twentieth century when he formalized how he applied RCA with the now ubiquitous Five Whys method (Imai, 1986, p. 50). Use of RCA in Japan spread and began to mature, and received an enormous boost with arrival of W. Edwards Deming in 1947, who introduced a well-structured component for continuous process improvement (Gabor, 1990, p. 20,74). This was the PDCA cycle (Plan, Do, Check, Act), aka the Deming Cycle or Shewhart Cycle, with RCA occurring mainly in the Plan step (Patel & Deshpande, 2017).

Below is “Toyota’s practical problem-solving process”, from The Toyota Way, by Jeffrey Liker, 2004. p 256. As simple as the diagram is, it summarizes the heart of the most impactful production process ever invented: the Toyota Production System.

The combination of RCA and PDCA is now known as modern process control, aka RCA-based continuous process improvement. This was the new process that solved industry’s most difficult problem at the time: how to consistently mass produce complex products and services of very high quality and low cost, a problem that had existed since the dawn of the industrial revolution. The problem had never been solved in the West, even with its large-scale mass production systems and Henry Ford’s perfection of the high-volume moving assembly line. Combining RCA with PDCA allowed process control theory to at last move from its fourth stage of process maturity, statistical quality control using control charts and other techniques, to the all-important fifth stage, total quality control (Feigenbaum, 1991, pp. 15–17). In the fifth stage, the PDCA cycle integrated statistical quality control with RCA to drive specific continuous process improvements in a manner so efficient and effective that mass production of very high-quality products and services at low cost was now possible. This had never been achieved before and was revolutionary. The new process was swiftly adopted by industry after industry, beginning with the automobile manufacturing in the form of lean.

No other method has been found that comes remotely close to the quality and cost results obtained by RCA-based processes. From this follows both the high-quality and low-cost portions of the claim.

The invention of RCA and the Five Whys

How RCA and the Five Whys were invented by Sakichi Toyoda and how the Five Whys work is described by Taiichi Ohno in Toyota Production System: Beyond Large-Scale Production, 1988. This is deeply insightful material, so we cover it at length. As you read, notice how the Five Whys is generic. This explains why a wrapper process is required for each problem type.

While still a young man, Toyoda developed the habit of consciously asking WHY something occurred until he arrived at its true root cause. The habit appeared in his youth when he was 20 years old. “Sometimes, I would spend all day watching the grandmother next door weaving. I came to understand the way the weaving machine worked.” (p77) Toyoda wasn’t understanding how the machine worked in the normal sense. He was silently asking himself WHY certain things happened. WHY, for example, was hand looming necessary? Couldn’t it be replaced by machine looms?

Toyoda went on to invent the most efficient machine loom in the world because he kept asking WHY. Early power looms were plagued by tread breakage, which necessitated one operator per machine. Toyoda asked WHY they were breaking. The causes were too numerous to completely control, so he asked a deeper WHY question: WHY is so much material and operator time wasted once thread breakage occurs? The correct answer, because the machine did not automatically stop, was a technical breakthrough. That insight, combined with additional loom improvements based on other WHY questions, quickly led to a 20-fold increase in productivity in Toyoda’s power looms compared to all previous looms.

Toyoda’s method became known as the Five Whys. The method asks “WHY does this occur?” until the root cause(s) of a problem is found. “When a problem arises... we repeatedly ask why. This is the scientific basis of the Toyota system.” (p18) The beauty of the Five Whys is it can be applied by anyone, anytime, to any causal problem. The trick is to not stop until you’ve found the true root cause(s). This rule is so crucial that Taiichi Ohno, in the Toyota Production System, famously begins chapter two (p17) this way:

Evolution of the Toyota Production System

Repeating WHY Five Times

When confronted with a problem, have you ever stopped and asked why five times? It is difficult to do even though it sounds easy. For example, suppose a machine stopped functioning:

1. Why did the machine stop?
There was an overload and the fuse blew.

2. Why was there an overload?
The bearing was not sufficiently lubricated.

3. Why was it not lubricated sufficiently?
The lubrication pump was not pumping sufficiently.

4. Why was it not pumping sufficiently?
The shaft of the pump was worn and rattling.

5. Why was the shaft worn out?
There was no strainer attached and metal scrap got in.

Repeating why five times, like this, can help uncover the root problem and correct it. If this procedure was not carried through, one might simply replace the fuse or the pump shaft. In that case, the problem would recur within a few months.

To tell the truth, the Toyota Production System has been built on the practice and evolution of this scientific approach. By asking why five times and answering it each time, we can get to the real cause of the problem, which is often hidden behind more obvious symptoms.

Why RCA works so well on complex problems

The Gordian knot of interrelationships in large complex social systems leads to the phenomena of dynamic complexity. This causes counterintuitive system behavior that is so unpredictable the system cannot be understood without identifying its essential cause-and-effect structure. John Sterman, in Business Dynamics: Systems Thinking and Modeling for a Complex World, 2000, examines this behavior. Page 22 describes how dynamic complexity arises because complex social systems are constantly changing, tightly coupled, governed by feedback loops, nonlinear, history dependent, self-organizing, and endlessly adaptive.

These characteristics work together to produce problems so complex and counterintuitive it is little surprise they cannot be correctly analyzed without using an RCA-based problem solving process to identify the essential cause-and-effect structure of the system. This changes one’s mental model of the problem from a black box (where cause and effect is impossible to predict) to a glass box (where solutions have predictable effects), which allows correct decisions.

Cause-and-effect models are built by using RCA to identify the essential causal structure of the problem. Essential causal structure is that portion of the causal structure of a system that contains the nodes, relationships (arrows), and feedback loops needed to identify the causal chains explaining what is required to solve a problem by resolving its root causes. These chains must include symptoms, intermediate causes, root causes, leverage points, and solutions. Everything not essential to these requirements is ignored or relegated to supporting artifacts (such as simulation models), allowing causal diagrams that are easily read by all.

All problem solution decisions are based on mental models. The better the model explains problem behavior, the better the decisions. Three general stages of causal model maturity are recognized as shown below.

Stage 1. A black box model of a system knows only the relationships between inputs (causes, solutions) and outputs (effects, symptoms). Everything else inside the black box is unknown, and is treated as too difficult, too expensive, or impossible to model correctly. For example, society has long known you must eat to survive. But until modern biology explained how food provided the nutrients and energy needed by the body and how these processes worked, no one knew why we had to eat to survive. Or they had shallow intuitively derived theories that were wrong.

Stage 2. A gray box model knows some of the relationships between inputs and outputs. Some of the essential causal structure is known but not all, so much walking of the solution landscape using trial-and-error is still required. Simulation modeling usually builds a gray box model by starting at symptoms and building a model that mimics those symptoms “for the right reasons.” These models always include some of the intermediate causes, because otherwise they would be unable to generate symptom behavior. However, they tend to not include the root causes because root cause analysis is not employed. Despite its sophistication and innovation, the Limits to Growth simulation model is a stage 2 model.

Stage 3. In a glass box model you can “see” the entire essential causal structure of a problem. For example, after Newton discovered gravity and the mathematical laws governing the movement of bodies, astronomers had a glass box model of the universe’s motion behavior. They could now accurately predict where heavenly bodies would be in the future (the effect) given their present location, speed, and other bodies whose presence affected them (the causes). As a second example, once a doctor starts with a patient’s symptoms, asks assessment questions, performs an examination, runs tests as needed, and determines the root causes, they have a glass box model of the problem and can confidently proceed with treatment. A glass box model provides a correct and sufficiently complete explanation of the relationship between causes and effects. The analysis and simulation model in the paper this article reports on is a stage 3 model, though further research is needed.

RCA works well on complex problems because RCA is explicitly designed to build glass box models with ease. Once the paradigm of RCA-based problem solving is grasped, you will easily see why all large-scale industries have turned to RCA-based processes, and why all those working on difficult large-scale common good problems should do the same.

Next article

RCA is not a panacea. Thoughtful, methodical application is required. Crucially, so is a process wrapper that fits the problem type since basic RCA is generic. The next article presents an RCA wrapper process specifically designed for difficult large-scale social problems.

References

Chevallier, A. (2016). Strategic Thinking in Complex Problem Solving. Oxford University Press.

Conley, R., Hall, K., & Scott, L. (2024). Racial Disparity Root Cause Analysis for the Department of the Air Force. RAND.

Ericson, C. (1999). Fault Tree Analysis: A History. 17th International System Safety Conference. https://web.archive.org/web/20110723124816/http://www.fault-tree.net/papers/ericson-fta-history.pdf.

Feigenbaum, A. (1991). Total Quality Control. McGraw Hill.

Gabor, A. (1990). The Man Who Invented Quality: How W. Edwards Deming Brought the Quality Revolution to America. Penguin Books.

Imai, M. (1986). Kaizen: The Key to Japan’s Competitive Success. McGraw Hill.

Marx, M. (2007). Six Sigma Saves a Fortune. Six Sigma Magazine.

Nguyen, H. (2018). Is Lean Manufacturing Still Relevant Today? TRG International. https://blog.trginternational.com/is-lean-manufacturing-still-relevant-today.

Patel, P., & Deshpande, V. (2017). Application Of Plan-Do-Check-Act Cycle For Quality And Productivity Improvement—A Review. IJRASET.

Pyzdek, T. (2003). The Six Sigma Handbook: A Complete Guide for Green Belts, Black Belts, and Managers at All Levels. McGraw Hill.

Weinstein, J., Geller, A., & Negussie, Y. (2017). Communities in Action: Pathways to Health Equity. The National Academies Press.

Womack, J., Jones, D., & Roos, D. (1990). The Machine that Changed the World: The Story of Lean Production. Harper Collins.

Wood, A., Stedman-Edwards, P., & Mang, J. (2000). The Root Causes of Biodiversity Loss. Earthscan Publications.