Democracy’s eternal vulnerability: Part 3. Applying root cause analysis to social problems with Social Force Diagrams

As Part 2 explained, applying RCA to complex problems requires a wrapper process for each problem type. For difficult large-scale social problems, no suitable wrapper was found in the literature or examination of industrial practices so we were compelled to develop one. This is a common occurrence on novel problems. For example:

After extensive review, NASA found that none of the commercially available tools and methods would support a comprehensive root cause analysis of all the unique problems and environments NASA faces on the Earth, in the ocean, in the air, in space, and on moons and planetary bodies. Existing tools were designed for a specific domain (e.g., aviation), a specific type of activity, a specific type of human error (e.g., errors of omission) or had a limited set of cause codes. The NASA RCAT [Root Cause Analysis Tool], a paper-based tool with companion software (now available free to government Agencies and contractors), was designed to address the shortcomings identified in existing tools. (Uusitalo, 2011, p. 73)

In our case, the wrapper must support clear understanding of the important causal forces in difficult complex large-scale social problems. Other fields have the same requirement for their problems: What are the important forces at play? Some fields pose a second question: How can the key forces best be visually represented? The answers are illustrated below for Newtonian physics and quantum reactions.

Sir Isaac Newton’s pioneering discovery of the three laws of motion and universal gravity accomplished far more than laying the foundation for physics. It also demonstrated the need to think in terms of standard forces for developing standard approaches to solving problems. On the left, a free body diagram of combined forces shows how a friction force (F) may be calculated, given the coefficient of friction (μ), a body’s mass (m), the gravitational constant (g), and the inclined plane angle (θ). The formula is F = μ x m x g x cos θ.

The right diagram shows how a Feynman diagram may be used to explain how an electron (e-) and a positron (e+) annihilate when meeting, producing a photon (γ), which becomes a quark-antiquark pair (q and q̄), after which the antiquark radiates a gluon (g). Feynman diagrams allowed theoretical physicists to rapidly think through problems that previously could only be approached with pages of arcane calculations.

In the hands of a postwar generation, [Feynman diagrams were] a tool intended to lead quantum electrodynamics out of a decades-long morass. … With the diagrams’ aid, entire new calculational vistas opened for physicists. Theorists learned to calculate things that many had barely dreamed possible before World War II. It might be said that physics can progress no faster than physicists’ ability to calculate. Thus, in the same way that computer-enabled computation might today be said to be enabling a genomic revolution, Feynman diagrams helped to transform the way physicists saw the world and their place in it. (Kaiser 2005)

Feynman shattered the stereotype of the stuffy out-of-touch-with-reality scientist. He moved to Rio de Janeiro in 1951 for his sabbatical just to learn samba and the bongos, and marched in samba parades. While at the Manhattan Project in Los Alamos from 1941 to 1945, he ingeniously learned how to open safes containing top secret material, and then performed prank after prank. At the Challenger disaster hearing in 1986, while TV cameras were rolling he astonished all by asking for a glass of ice water and forceps, and then demonstrated how the rubber O-ring seal leak was easily caused by near freezing air temperatures, which indeed had occurred during launch. His ingenuity knew no bounds!

The process wrapper must do what the above examples did: visually portray the key standard forces of a problem in such a manner that solving the problem becomes a matter of methodically applying the tool to identify the standard forces at play.

Below is a simple problem analyzed using SFDs. The essential causal structure has been identified. SFDs show the most important forces in a problem, which here are S, F, and R. (Later more complex diagrams show a fourth force, New R.)

Suppose you have the problem of “My car won’t start.” Root cause analysis (RCA) works by asking “WHY does this occur?” until you find the root causes.

RCA always starts with a problem’s symptoms, such as My car won’t start. I turn the key and all I hear is a click. WHY is that? You hear a click, but the engine won’t turn over.

Therefore, the cause must be a dead battery. That’s really the intermediate cause. But you assume it’s the root cause and decide to fix the battery. You then either buy a new battery or recharge the old one. Now the car runs. Problem solved.

But superficial solutions only solve a problem partially or temporarily. A week later the car won’t start again. WHY is that? You have no idea, because replacing or recharging the battery is all you know how to do. The key point of SFDs is that if it’s a difficult problem, the fundamental layer is normally hard to see. Without the right tool all you can see is what’s on the superficial layer.

In this problem the fundamental layer is easy to see with the right tool: a volt meter. A friend shows up with a tool you don’t have: a volt meter. She jump starts your car by hooking up your battery to hers. Then, while the car is running, she checks how many volts are charging the battery. Not enough! So the real cause must be a defective alternator, so the battery is not charged enough. That’s the root cause. Now the high leverage point is obvious: fix the alternator, such as by have a mechanic replace the alternator. Now the problem is permanently solved, at least due to that root cause.

Each of us uses some form of RCA every day to solve causal problems. But we usually don’t use RCA terms and a formal problem solving process using a tool. This works fine on easy problems. But when we apply the same intuitive approach to difficult problems it almost always fails. That’s why tools like SFDs are required for difficult causal problems.

Now let’s analyze a complex problem, one that’s been solved. There is much to learn from analyzing past problems because they illustrate the pitfalls of the wrong problem-solving process, and offer many educational insights into the structure of how complex social systems work.

One of history’s most intractable problems was the Autocratic Ruler Problem. After thousands of years of rule by warlords, dictators, and kings, a period characterized by countless superficial solutions like revolutions, uprisings, assassinations, and coups, the shift to a fundamental solution began with the signing of the Magna Carta in 1215. This introduced the high leverage point concept that people have rights that must be respected. This innovative concept steadily diffused and eventually reached critical mass. The invention of modern democracy in 1776 in the US and 1789 in France signaled the beginning of pushing in a comprehensive manner on the high leverage point. The fundamental solution resolved the root cause so well that a historic mode change occurred and democracy swept much of the world.

The superficial solutions force is especially instructive. The low leverage point is forced replacement of a bad ruler with a good one. This failed to permanently solve the problem because it did nothing to change the system. More bad rulers appeared to replace the good ones, or good rulers went bad once in power. By contrast the fundamental solution of democracy changed the system so that a permanent new mode resulted, where far fewer bad rulers naturally appeared.

SFDs are powerful tools. The diagram shows at a glance why superficial solutions failed to solve the problem for so long, why the fundamental solution worked, and why, once the mode change occurred, political systems tended to stay in the new mode due to the right new balancing feedback loop.

Unfortunately the new mode is being challenged by Problematique problem number 8, democratic backsliding. WHY is that? Because something is going wrong with the fundamental solution’s Voter Feedback Loop, as this article series will later explain.

For analyzing difficult large-scale social problems using RCA we created the standard SFD template. The analyst starts with the template, which uses RCA terminology and a fill-in-the-blanks format. The diagram organizes a problem’s essential causal structure into the four forces found in all problems of this class. This allows the analyst(s) to rapidly apply the tool by working at the highest level of abstraction possible, while still identifying all of a problem’s essential causal structure.

Social force diagrams are organized into two layers:

1. The superficial (symptomatic) layer of the problem, where intermediate causes are so easy to see they are erroneously assumed to be root causes, and

2. The deeper fundamental layer, where by understanding the problem’s deeper structure its true root causes may be found.

The two layers allow avoidance of the Superficial Solutions Trap. The trap occurs when problem solvers unknowingly assume intermediate causes are root causes, and then develop solutions based on that false assumption. This leads to solutions directed toward intermediate causes rather than root causes. Superficial solutions can never resolve root causes because root cause forces exert a greater force on intermediate causes than superficial solutions, regardless of how well funded, managed, or promoted those solutions are. As the next article in this series will show, all popular solutions to the global Problematique fell into this trap.

It’s an easy trap to fall into because root causes can be deceptively hard to identify, as the sage of system dynamics, Jay Forrester, describes: (Italics added)

The intuitively obvious ‘solutions’ to social problems are apt to fall into one of several traps set by the character of complex systems. ...people are often led to intervene at points in a system where little leverage exists and where effort and money have but slight effect.

...social systems are inherently insensitive to most policy changes that people select in an effort to alter behavior. In fact, a social system draws attention to the very points at which an attempt to intervene will fail. Human experience, which has been developed from contact with simple systems, leads us to look close to the symptoms of trouble for a cause. But when we look, we are misled because the social system presents us with an apparent cause that is plausible according to the lessons we have learned from simple systems, although this apparent cause is usually a coincident occurrence that, like the trouble symptom itself, is being produced by the feedback loop dynamics of a larger system. (Forrester 1971, pp 94-95)

Forrester’s “apparent cause” is what root cause analysis calls the intermediate cause. “Little leverage exists” if people assume the apparent cause is the root cause because that leads to pushing on low leverage points.

Social force diagrams are built by starting at problem symptoms and identifying the causal chain with “WHY does this occur?” questions until the root causes are found. As this is done, why past superficial solutions failed is diagrammed. This is critical knowledge, as it indicates the intermediate causes are indeed intermediate rather than root causes, and triggers insights about how to dig deeper. After the superficial layer of the problem is understood, the analyst follows the causal chain down into the fundamental layer to find the problem’s true root causes, and finally the fundamental solutions.

The superficial layer contains one or more intermediate causes. Some problems require multiple diagrams, since they contain multiple subproblems (defined by multiple symptoms) and thus multiple root causes. Difficult problems usually require construction of a feedback loop model to analyze the fundamental layer. Without analysis of the fundamental layer, difficult problems tend to stay stuck in the superficial layer for a long time, as the Autocratic Ruler Problem did for thousands of years.

Failed solutions are powerful clues. Knowledge of the superficial layer and why past solutions failed is required for solving difficult problems, because as Popper (1999) explains (italics in the original):

We are always learning a whole host of things through falsification. We learn not only that a thing is wrong; we learn why it is wrong. Above all else, we gain a new and more sharply focused problem, and a new problem, as we already know, is the starting point for a new development in science.

After the superficial layer is built a new problem that could not be seen before comes into sharp focus: What is the feedback loop structure that identifies the root cause of the lowest intermediate cause in the superficial layer? What is the high leverage point for resolving the root cause? What practical solutions can push on the high leverage point in a manner so well-engineered that the root cause stays resolved and the mode change is relatively permanent? Because each question is so sharply focused, the answer landscape is relatively small and quickly searched.

Social force diagrams focus on understanding four key forces: S, F, R, and new R. Superficial solutions (force S) fail because force S is always less than root cause forces (force R), indicated by S < R. By contrast, if fundamental solutions (force F) are properly designed (especially their impact on feedback loop structure), force F can exceed force R, indicated by F > R. This leads to a systemic mode change, during which the old R is replaced by a new R, and the problem is solved. The new R must be engineered to be strong enough and self-managing enough to permanently hold the system in the solved mode, due to the way force F fundamentally changes critical feedback loop structure and loop dominance.

If analysis shows no F > R exists (no resolvable root cause is found), the problem is unsolvable as defined. In this case problem definition (problem symptoms) can sometimes be relaxed to make the problem solvable, such as raising the maximum allowable global temperature rise for climate change to make that problem solvable. Otherwise, the problem should be declared unsolvable.

Once all four forces are understood and key assumptions have been tested by measurement or experimentation, the analyst has a sufficiently complete scientific theory of the problem. Each force provides an explanatory tenet of the theory. This gives the four requirements for a comprehensive theory of a difficult large-scale social problem. The theory must identify the four forces and explain them in this manner:

1. Social Force S – Why past solutions failed (because S < R).

2. Social Force R – Why the problem occurs (because R is unresolved).

3. Social Force F – Why fundamental solutions can be expected to succeed in causing the desired mode change (because F > R).

4. New Social Force R – Why the mode change will be relatively permanent (because the new R contains self-sustaining feedback loops).

This suggests that any comprehensive theory of how to solve a difficult social problem must adequately explain all four forces. The above list thus serves as the four requirements for a comprehensive theory of a difficult large-scale social problem. The theory must identify the four forces and explain their causal structure. Anything less and we would disappoint Newton and Feynman!

Different analysts will produce different diagrams and supporting simulation models, and reach different policy conclusions. This is the well-known problem of model uncertainty/variation (Young 2009). The standard solution is to specify model requirements and use that for model evaluation, a process step especially important for complex models (Höge, Wöhling, and Nowak 2018). The above four requirements provide this for the class of difficult large-scale social problems.

Meeting the four requirements is challenging. Fortunately, as analysts iteratively mature their social force diagram(s) and simulation model(s) using RCA, measurement, experimentation, calibration, evidence of past solutions that failed, and rigorous testing, they will tend to converge toward each other and the requirements because in the real world only one true causal mechanism exists. But since all models are simplifications of reality and alternative solutions are possible due to multiple solutions for the same high leverage point, multiple useful models and effective solutions are possible. This matters little, as long as an analysis fulfills its purpose. If large crucial differences exist, they can be reconciled by rigorous application of the process. Large differences have not been a problem in industrial RCA.

The best way to learn how to use SFDs is to practice analyzing simple problems you are familiar with, and then gradually escalate to more complex problems. Who knows, you might solve some agonizing problem that’s have been frustrating you or your friends for years! If so, I’d be delighted to hear about it.

This article presented Social Force Diagrams, explained how they work, and illustrated their usefulness with a simple and then a retrospective example. Further examples are here.

The next article begins presentation of the results of analyzing the global Problematique using SFDs. If the proof-of-concept analysis is reasonably correct and can serve as a starting point for serious large-scale research, then perhaps humanity will not squander another 54 years, as explained in Part 2. Or another 550 years, which for the Autocratic Ruler Problem was the time from when problem solvers first began intuitively pushing on the high leverage point (in 1215 with the Magna Carta) to when they intuitively invented a fundamental solution that worked (in 1776 and 1789 with modern democracy and its Voter Feedback Loop).

Forrester, Jay (1971), World Dynamics, Wright-Allen Press.

Höge, Marvin, Thomas Wöhling, and Wolfgang Nowak. 2018. “A Primer for Model Selection: The Decisive Role of Model Complexity.” Water Resources Research 54(3): 1688–1715.

Kaiser, D. (2005). Physics and Feynman’s Diagrams: In the hands of a postwar generation, a tool intended to lead quantum electrodynamics out of a decades-long morass helped transform physics. American Scientist, 93(2).

Popper, Karl. 1999. All Life Is Problem Solving. Routledge.

Uusitalo, I. (2011). Review of security testing tools. Diamonds Consortium. https://publica-rest.fraunhofer.de/server/api/core/bitstreams/3062e987-a93a-420e-8cf1-e7679835aff8/content

Young, Cristobal. 2009. “Model Uncertainty in Sociological Research: An Application to Religion and Economic Growth.” American Sociological Review 74(3): 380–97.