ABM fundamentalism

image: Chernobyl control room

Quite a few recent posts have examined the power and flexibility of ABM models as platforms for simulating a wide range of social phenomena. Joshua Epstein is one of the high-profile contributors to this field, and he is famous for making a particularly strong claim on behalf of ABM methods. He argues that “generative” explanations are the uniquely best form of social explanation. A generative explanation is one that demonstrates how an upper-level structure or causal power comes about as a consequence of the operations of the units that make it up. As an aphorism, here is Epstein's slogan: "If you didn't grow it, you didn't explain it." 

Here is how he puts the point in a Brookings working paper, “Remarks on the foundations of agent-based generative social science” (link; also chapter 1 of Generative Social Science: Studies in Agent-Based Computational Modeling):

"To the generativist, explaining macroscopic social regularities, such as norms, spatial patterns, contagion dynamics, or institutions requires that one answer the following question:

"How could the autonomous local interactions of heterogeneous boundedly rational agents generate the given regularity?

"Accordingly, to explain macroscopic social patterns, we generate—or “grow”—them in agent models." (1)

And Epstein is quite explicit in saying that this formulation represents a necessary condition on all putative social explanations: "In summary, generative sufficiency is a necessary, but not sufficient condition for explanation." (5).

There is an apparent logic to this view of explanation. However, several earlier posts cast doubt on the conclusion. First, we have seen that all ABMs necessarily make abstractive assumptions about the behavioral features of the actors, and they have a difficult time incorporating "structural" factors like organizations. We found that the ABM simulations of ethnic and civil conflict (including Epstein's own model) are radically over-simplified representations of the field of civil conflict (link).  So it is problematic to assume the general applicability and superiority of ABM approaches for all issues of social explanation.

Second, we have also emphasized the importance of distinguishing between "generativeness" and "reducibility" (link). The former is a claim about ontology -- the notion that the features of the lower level suffice to determine the features of the upper level through pathways we may not understand at all. The latter is a claim about inter-theoretic deductive relationships -- relationships between our formalized beliefs about the lower level and the feasibility of deriving the features of the upper level from these beliefs. But I argued in the earlier post that the fact that A is generated by B does not imply that A is reducible to B. 

So there seem to be two distinct ways in which J. Epstein is over-reaching here: he is assuming that agent-based models can be sufficiently detailed to reproduce complex social phenomena like civil unrest; and second, he is assuming without justification that only reductive explanations are scientifically acceptable.

Consider an example that provides an explanation of a collective behavior that has explanatory weight, that is not generative, and that probably could not be fully reproduced as an ABM.  A relevant example is Charles Perrow's analysis of technology failure as a consequence of organizational properties (Normal Accidents: Living with High-Risk Technologies). An earlier post considered these kinds of examples in more detail (link). Here is my summary of organizational approaches to the explanation of the incidence of accidents and system safety:

However, most safety experts agree that the social and organizational characteristics of the dangerous activity are the most common causes of bad safety performance. Poor supervision and inspection of maintenance operations leads to mechanical failures, potentially harming workers or the public. A workplace culture that discourages disclosure of unsafe conditions makes the likelihood of accidental harm much greater. A communications system that permits ambiguous or unclear messages to occur can lead to air crashes and wrong-site surgeries. (link)

I would say that this organizational approach is a legitimate schema for social explanation of an important effect (the occurrence of large technology failures). Further, it is not a generativist explanation; it does not originate in a simplification of a particular kind of failure and demonstrate through iterated runs that failures occur X% of the time. Rather, it is based on a different kind of scientific reasoning, based on causal analysis grounded in careful analysis and comparison of cases. Process tracing (starting with a failure and working backwards to find the key causal branches that led to the failure) and small-N comparison of cases allows the researcher to arrive at confident judgments about the causes of technology failure. And this kind of analysis can refute competing hypotheses: "operator error generally causes technology failure", "poor technology design generally causes technology failure", or even "technological over-confidence causes technology failure". All these hypotheses have defenders; so it is a substantive empirical hypothesis to argue that certain features of organizational deficiency (training, supervision, communications processes) are the most common causes of technological accidents.

Other examples from sociology could be provided as well: Michael Mann's explanation of the causes of European fascism (Fascists), George Steinmetz's explanation of variations in the characteristics of German colonial rule (The Devil's Handwriting: Precoloniality and the German Colonial State in Qingdao, Samoa, and Southwest Africa), or Kathleen Thelen's explanation of the persistence and change in training regimes in capitalist economies (How Institutions Evolve: The Political Economy of Skills in Germany, Britain, the United States, and Japan). Each is explanatory, each identifies causal factors that are genuinely explanatory of the phenomena in question, and none is generativist in Epstein's sense. These are examples drawn from historical sociology and institutional sociology; but examples from other parts of the disciplines of sociology are available as well.

I certainly believe that ABMs sometimes provide convincing and scientifically valuable explanations. The fundamentalism that I'm taking issue with here is the idea that all convincing and scientifically valuable social explanations must take this form -- a much stronger view and one that is not well supported by the practice of a range of social science research programs.

Or in other words, the over-reach of the ABM camp comes down to this: the claims of exclusivity and general adequacy of the simulation-based approach to explanation. ABM fundamentalists claim that only simulations from units to wholes will be satisfactory (exclusivity), and they claim that ABM simulations can always be designed for any problem that are generally adequate to grounding an explanation (general adequacy). Neither proposition can be embraced as a general or universal claim. Instead, we need to recognize the plurality of legitimate forms of causal reasoning in the social sciences, and we need to recognize, along with their strengths, some of the common weaknesses of the ABM approach for some kinds of problems.

Causal diagrams and causal mechanisms

There is a long history of the use of directed causal diagrams to represent hypotheses about causation. Can the mathematics and graphical systems created for statistical causal modeling be adapted to represent and evaluate hypotheses about causal mechanisms and outcomes?

In the causal modeling literature the structure of a causal hypothesis is something like this: variable T increases/ decreases the probability of the occurrence of outcome E. This is the causal relevance criterion described by Wesley Salmon in Scientific Explanation and the Causal Structure of the World. It is a fundamentally statistical understanding of causality.

Here is a classic causal path model by Blau and Duncan indicating the relationships among a number of causal factors in bringing about an outcome of interest -- "respondent's first job".

This construction aims at joining a qualitative hypothesis about the causal relations among a set of factors with a quantitative measurements of the correlations and conditional probabilities that support these causal relations. The whole construction often takes its origin in a multivariate regression model.

Aage Sørensen describes the underlying methodological premise of quantitative causal research in these terms in his contribution to Frontiers of Sociology (Annals of the International Institute of Sociology Vol. 11):

Understanding the association between observed variables is what most of us believe research is about. However, we rarely worry about the functional form of the relationship. The main reason is that we rarely worry about how we get from our ideas about how change is brought about, or the mechanisms of social processes, to empirical observation. In other words, sociologists rarely model mechanisms explicitly. In the few cases where they do model mechanisms, they are labeled mathematical sociologists, not a very large or important specialty in sociology. (370)

My question here is whether this scheme of representation of causal relationships and the graphical schemes that have developed around it are useful for the analytics of causal mechanisms.

The background metaphysics assumed in the causal modeling literature is Humean and "causal-factor" based; such-and-so factor increases the probability of occurrence of an outcome or an intermediate variable, the simultaneous occurrence of A and B increase the probability of the outcome, etc. Quoting Peter Hedstrom on causal modeling:

In the words of Lazarsfeld (1955: 124-5), "If we have a relationship between x and y; and if for any antecedent test factor the partial relationships between x and y do not disappear, then the original relationship should be called a causal one." (Dissecting the Social: On the Principles of Analytical Sociology)

The current iteration of causal modeling is a directed acyclic graph (DAG). Felix Elwert provides an accessible introduction to directed acyclic graphs in his contribution to Handbook of Causal Analysis for Social Research (link). Here is a short description provided by Elwert:

DAGs are visual representations of qualitative causal assumptions: They encode researchers’ expert knowledge and beliefs about how the world works. Simple rules then map these causal assumptions onto statements about probability distributions: They reveal the structure of associations and independencies that could be observed if the data were generated according to the causal assumptions encoded in the DAG. This translation between causal assumptions and observable associations underlies the two primary uses for DAGs. First, DAGs can be used to prove or disprove the identification of causal effects, that is, the possibility of computing causal effects from observable data. Since identification is always conditional on the validity of the assumed causal model, it is fortunate that the second main use of DAGs is to present those assumptions explicitly and reveal their testable implications, if any. (246)

A DAG can be interpreted as a non-parametric structural equation model, according to Elwert. (Non-parametric here means simply that we do not assume that the data are distributed normally.) Elwert credits the development of the logic of DAGs to Judea Pearl and Peter Spirtes, along with other researchers within the causal modeling community.

Johannes Textor and a team of researchers have implemented DAGitty, a platform for creating and using DAGs in appropriate fields, including especially epidemiology (link). A crucial feature of DAGitty is that it is not solely a graphical program for drawing graphs of possible causal relationships; rather, it embodies an underlying logic which generates expected statistical relationships among variables given the stipulated relationships on the graph. Here is a screenshot from the platform:

The question to consider here is whether there is a relationship between the methodology of causal mechanisms and the causal theory reflected in these causal diagrams. 

It is apparent that the underlying ontological assumptions associated with the two approaches are quite different. Causal mechanisms theory is generally associated with a realist approach to the social world, and generally rejects the Humean theory of causation. The causal diagram approach, by contrast, is premised on the Humean and statistical approach to causation.  A causal mechanisms hypothesis is not fundamentally evaluated in terms of the statistical relationships among a set of variables; whereas a standard causal model is wholly intertwined with the mathematics of conditional correlation.

Consider a few examples. Here is a complex graphical representation of a process understood in terms of causal mechanisms from McGinnes and Elandy, "Unintended Behavioural Consequences of Publishing Performance Data: Is More Always Better?" (link):

Plainly this model is impossible to evaluate statistically by attempting to measure each of the variables; instead, the researchers proceed by validating the individual mechanisms identified here as well as the direction of influence they have on other intermediate outcomes. The outcome of interest is "quality of learning" at the center of the graph; and the diagram attempts to represent the complex structure of causal influences that exist among several dozen mechanisms or causal factors.

Here is another example of a causal mechanisms path diagram, this time representing the causal system involved in drought and mental health by Vins, Bell, Saha, and Hess (link).

Here too the model is not offered as a statistical representation of covariance among variables; rather, it is a hypothetical sketch of the factors which play in mechanisms leading from drought to depression and anxiety in a population. And the assessment of the model should not take the form of a statistical evaluation (a non-parametric structural equation model), but rather a piecemeal verification of the validity of the specific mechanisms cited. (John Gerring argues that this is a major weakness in causal mechanisms theory, however, in "Causal Mechanisms? Yes, But ..." (link).)

It seems, therefore, that the superficial similarity between a causal model graph (a DAG) and a causal mechanisms diagram is only skin-deep. Fundamentally the two approaches make very different assumptions about both ontology (what a causal relationship is) and epistemology (how we should empirically evaluate a causal claim). So it seems unlikely that it will be fruitful for causal-mechanisms theorists to attempt to adapt methods like DAGs to represent the causal claims they want to advance and evaluate.

What is methodology?

As social science researchers, we would all like to have an excellent methodology for carrying out the tasks we confront in our scientific work. But what precisely are we looking for when we aspire to this goal? What is a methodology, and what is it intended to allow us to do?

A methodology is a set of ideas or guidelines about how to proceed in gathering and validating knowledge of a subject matter. Different areas of science have developed very different bodies of methodology on the basis of which to conduct their research. We might say that a methodology provides a guide for carrying out some or all of the following activities:

• probing the empirical details of a domain of phenomena
• discovering explanations of surprising outcomes or patterns
• identifying entities or forces 
• establishing patterns
• providing predictions
• separating noise from signal
• using empirical reasoning to assess hypotheses and assertions

Here is what Andrew Abbott has to say about methods in Methods of Discovery: Heuristics for the Social Sciences:

Social scientists have a number of methods, stylized ways of conducting their research that comprise routine and accepted procedures for doing the rigorous side of science. Each method is loosely attached to a community of social scientists  for whom it is the right way to do things. But no method is the exclusive property of any one of the social sciences, nor is any social science, with the possible exception of anthropology, principally organized around the use of one particular method. (13)

So a method or a methodology is a set of recommendations for how to proceed in doing scientific research within a certain domain. Sometimes in the history of philosophy there has been a hope that science could proceed on the basis of a pure inductive logic: collect the data, analyze the data, sift through the findings, report the strongest regularities found in the data set. But scientific inquiry requires more than this; it requires discovery and imagination.

What form might a methodology take? The simplest idea is that a methodology is a recipe for arriving at justified scientific statements with respect to a domain of empirical phenomena. A recipe is a set of instructions for treating a number of ingredients in a sequential way and producing a specific kind of output -- a soufflé or a bowl of pad thai. If you follow the recipe, you are almost certain to arrive at the soufflé. But it is clear that scientific methodology cannot be as prescriptive as a recipe. There is no set of rules that are certain or likely to lead to the discovery of compelling hypotheses and explanations.

So if a scientific methodology isn't a set of recipes, then what is it? Here is another possibility: a methodology consists of a set of heuristics that serve to guide the activities, data collection, and hypothesis formation of the scientist. A heuristic is also a set of rules; but it is weaker than a recipe in that there is no guarantee of success. Here is a heuristic for consumers: "If you are selecting a used car to purchase, pay attention to rust spots." This is a good guide to action, not because rust spots are the most important part of a car's quality, but because they may serve as a proxy for the attentiveness to maintenance of the previous owner -- and therefore be an indication of hidden defects.

Andrew Abbott mentions several key topics for specification through methodology -- "how to propose a question, how to design a study, how to draw inferences, how to acquire and analyze data" (13), and he shows that we can classify methods by placing them into the types of question they answer.

types of data gathering	data analysis	posing a question
History	Direct interpretation	Case study analysis
Ethnography	Quantitative analysis	Small-N comparison
Surveys	Formal modeling	Large-N analysis
Record-based analysis

Abbott suggests that these varieties can be combined into five basic approaches:

• ethnography
• historical narration
• standard causal analysis
• small-N comparison
• formalization

And he arranges them in a three-dimensional space, with each dimension increasing from very particular knowledge at the origin to more abstract knowledge further out the axis. (Commonsense understanding of the facts lies at the origin of the mapping.) The three axes are formal modeling (syntactic program), pattern finding (semantic program), and cause finding (pragmatic program) (28). 

Abbott is a sociologist whose empirical and theoretical work is genuinely original and important, and we can learn a lot from his practice as a working researcher. His meta-analysis of methodology, on the other hand, seems fairly distant from his own practice. And I'm not sure that the analysis of methodology represented here provides a lot of insight into the research strategies of other talented social scientists (e.g. Tilly, Steinmetz, Perrow, Fligstein). This perhaps illustrates a common occurrence in the history of science: researchers are not always the best interpreters of their own practice. 

It is also interesting to observe that the discovery of causal mechanisms has no explicit mention in this scheme. Abbott never refers to causal mechanisms in the book, and none of the methods he highlights allow us to see what he might think about the mechanisms approach. It would appear that mechanisms theory would reflect the pragmatic program (searching for causal relationships) and the semantic program (discovering patterns in the observable data).

My own map of the varieties of the methods of the social sciences suggests a different scheme altogether. This is represented in the figure at the top of the post.

Causal concepts

source: D. Little, “Causal Explanation in the Social Sciences,” Southern Journal of Philosophy (1995) (link)

It may be useful to provide a brief account of some of the key ideas that are often invoked in causal explanations in the social sciences. (Here is an earlier post that summarized some current issues in causation research; link. And here are several earlier articles on causal explanation; link, link, link.)

The general idea of a social cause (X causes Y) goes along these lines: X is a structure or feature of social life that varies across social settings and whose presence increases the likelihood of occurrence of Y. The presence of X (perhaps in the presence of Y and Z as well) contributes to processes leading to Y.

This simple formulation contains several hidden assumptions -- most importantly, that outcomes have causes, that causes retain their characteristics over time and across instances, and that there are processes or dynamics within the domain of things and processes that convey with some form of necessity one set of circumstances and events onto another.

An example

For example, consider this hypothetical narrative describing a riot in a European city with a large community of impoverished immigrant people:

• (C1) simmering resentment by immigrant youth of joblessness and low social esteem
• (C2) heat wave creating discomfort and misery in crowded neighborhoods
• (C3) chronic disrespectful and rough police treatment of immigrant youth
• (I) forceful arrest of mis-identified young person in a city park, leading to serious injury of the youth
• (O) several days of rioting occur

The associated causal hypothesis goes along these lines: In the context of simmering resentment by immigrant youth and a pattern of mistreatment by police, feelings in the community were unusually elevated by the heat wave. When the arrest occurred a small protest began in the park, which spread to other blocks in the city and eventuated in the burning of cars, smashing of shop windows, and multiple further arrests.

Conditions Ci are standing conditions that played a causal role in the occurrence of the riot. The arrest incident was the instigating event, the match that ignited the social "gasoline". If any of C1, C2, C3 had been changed six months earlier, it is unlikely that O would have occurred. Each was necessary for I leading to O in the circumstances of the day.  If C1, C2, C3 are present, it is likely that some instigating event will occur in the normal hustle-bustle of urban life. I was the instigating condition. For researchers seeking general explanations of urban unrest, C1 and C3 appear to be strong candidates for common causes across many examples of urban riots. Two mechanisms are invoked here: a mechanism having to do with the individual's propensity to engage in protest ("resentment and mistreatment elevates propensity to protest") and a mechanism having to do with the spread of protest ("a small disturbance between a few teenagers and the police escalates through direct contact with other disaffected individuals through the neighborhood").

Here are brief discussions of many of the concepts that are commonly invoked in discussions of social causation.

Causal narrative

An organized and temporally directed account of the occurrence of an event or change, identifying the conditions, circumstances, and events that were causally relevant to its occurrence. A narrative needs to provide empirical evidence for its empirical claims and theoretical justification for the causal mechanisms and processes it postulates.

Standing condition

A condition or circumstance that persists through an extended period of time and that serves as part of the necessary causal background of a given causal process or mechanism. Persistent racial isolation is a standing condition in many explanations of the effects of inner city poverty.

Instigating event

An instigating event is an occurrence, including change of state of some background property, that triggers a change in some other property or process. The early-morning arrest by patrons of a blind pig (unlicensed tavern) in Detroit was the instigating event of the 1967 Detroit riot/uprising.

Necessary condition

A condition that must be present in order for a given causal interaction to occur. "If X had not been present, the outcome O would not have occurred."

Sufficient condition (conjunction of conditions)

A condition (or conjunction of conditions) whose presence suffices to bring about the outcome. "If X&Y&Z were present, then O would have occurred."

Counterfactual statements

It is worth underlining the point that necessary and sufficient conditions invoke counterfactual statements: If X had not occurred, Y would not have occurred. The logic of counterfactuals (modal logic) has a controversial and unresolved history. But given that causal language always implies some kind of necessity, we cannot dispense with counterfactuals and still have an adequate causal vocabulary.

INUS condition (J. L. Mackie)

J.L. Mackie's work on causation in The Cement of the Universe: A Study of Causation brought to closure a long line of thought about the logic of causal relations, culminating in his concept of INUS conditions. Consider this complex causal statement about the circumstances causing P:

'All (ABC or DGH or JKL) are followed by P' and 'All P are preceded by (ABC or DGH or JKL)' (Mackie, 62)

Mackie then defines an INUS condition:

Then in the case described above the complex formula '(ABC or DGH or JKL)' represents a condition which is both necessary and sufficient for P: each conjunction, such as 'ABC', represents a condition which is sufficient but not necessary for P. Besides, ABC is a minimal sufficient condition:  none of its conjuncts is redundant: no part of it, such as AB, is itself sufficient for P. But each single factor, such as A, is neither a necessary nor a sufficient condition for P. Yet it is clearly related to P in an important way: it is an insufficient but non-redundant part of an unnecessary but sufficient condition: it will be convenient to call this … an inus condition. (62)

To simplify:

A is an INUS condition for P if for some X and Y, (AX v Y) is a necessary and sufficient condition for P, but A is not sufficient for P and X is not sufficient for P.

Causal mechanism

An interlocked series of events and processes that, once initiated by some set of conditions, [usually] brings about a given outcome O. The idea that there are real mechanisms embodied in the "stuff" of a given domain of phenomena provides a way of presenting causal relations that serves as a powerful alternative to the "regularity" view associated with Hume. "Poor performance on standardized tests by specific groups is caused by the mechanism of stereotype threat" (Claude Steele, Whistling Vivaldi: How Stereotypes Affect Us and What We Can Do (Issues of Our Time)). This mechanism is a hypothesized process within the cognitive-emotional system of the subjects of the test. (James Mahoney's survey article on the mechanisms literature is a good introduction to the debate; link.)

Causal powers

The idea that certain kinds of things (metals, gases, military bureaucracies) have internal characteristics that lead them to interact causally with the world in specific and knowable ways. This means that we can sometimes identify dispositional properties that attach to kinds of things. Metals conduct electricity; gases expand when heated; military bureaucracies centralize command functions. (Harre and Madden, Causal Powers: Theory of Natural Necessity)

Probabilistic causal relation

A relationship between A and O such that the occurrence of A increases/decreases the likelihood of the occurrence of O. This can be stated in terms of conditional probabilities: P(O|A) ≠ P(O) [the probability of O given A is not equal to the probability of O]. For a causal realist, the definition is extended by a hypothesis about an underlying causal mechanism. [Smoking is a probabilistic cause of lung cancer [working through physiological mechanisms X,Y,Z]. This is equivalent to Wesley Salmon's criterion of causal relevance (Scientific Explanation and the Causal Structure of the World).

Causal explanation of a singular event

When we are interested in the explanation of a single event, a causal narrative leading up to that event is generally what we are looking for. What led to the outbreak of World War I? Why did Khomeini come to power in Iran in 1979? There are generally two difficult problems facing a proposed causal-narrative explanation of a singular event. First, we need to somehow empirically validate the claims about causal mechanisms and processes that are invoked in the narrative. But since this is a singular event, we do not have the option of using experimental methods to empirically test the claim "X leads by mechanism M to Y" that the narrative proposes. This is one important reason why mechanism theorists have generally required that specified mechanisms have roughly similar causal properties in a range of circumstances. Circumstances embodying the core features of a public goods problem usually lead to elevated levels of free riding -- whether in public radio fundraising, strikes, classroom discussions, or rebellions. Second, there is the problem of alternative realizability and multiple causal pathways leading to the same outcome. If the conditions leading to World War I were sufficiently ominous, then whether the assassination of the Archduke or some other event brought it about is of less explanatory importance. Given that potential instigating events occur with a certain probability, some event would have occurred within those few months that led to war. So it is better to identify the standing conditions that made war likely as the causes, rather than the assassination of the Archduke.

Generalizations about the causes of a kind of social entity or event

We are often interested in answering causal questions about classes of events: Why do peasant rebellions occur? Why does corruption rise to such high levels in many cities? Why do democracies not wage war against each other? Here we are looking for common conjunctions of causal factors that can be shown to be causally relevant in many such events. It is possible that we will discover that peasant rebellions do not have a single set of causal antecedents; rather there are multiple profiles of peasant rebellions, each with a set of causal conditions significantly different from the other profiles.

Methods of causal inquiry

How can social researchers identify causal relations among social events and structures? There are several groups of methods that social scientists and historians have employed: statistical-causal models, small-N models based on Mill's methods of similarity and difference (link, link), and case studies and process-tracing methods through which researchers seek to identify and confirm causal relations in individual cases. In each case the method derives from fundamental ideas about the nature of causation: the idea that causal relations between several factors give rise to statistical regularities when we have a large number of cases; the idea that we can use the features of necessary and sufficient conditions to select cases in order to include or exclude certain factors as causally related to the outcome; and the idea that causal mechanisms and processes can often be observed fairly directly in the historical record (Alexander George and Andrew Bennett, Case Studies and Theory Development in the Social Sciences).

Causal inference and random trials

image: Tamil Nadu nutrition study

Nancy Cartwright has spent much of her career probing the assumptions scientists make about causation. She has helped to demonstrate that the Humean assumptions about causation that philosophers (including Carl Hempel) carried into twentieth century philosophy of science don't come close to answering the question correctly, and she has provided many reasons to take seriously the ideas of causal powers and mechanisms rather than governing causal regularities. How the Laws of Physics Lie is an important contribution to the philosophy of science and to realist theory.

Her current book Evidence-Based Policy: A Practical Guide to Doing It Better (with Jeremy Hardie) provides a different critical perspective on causal inference, this time in the context of social policy reasoning. The design and implementation of public policies rest upon a fundamental premise: that we can have evidence-based reasons for predicting what the effects of the policy tool will likely be. But what kind of evidence might that be? The dominant form of evidence favored in the policy science field is random controlled trials: specify the policy intervention P, choose a domain of cases to apply the intervention P to, randomly select cases to receive the intervention (versus the control group that does not), and measure the value of the outcome of interest. If there is a significant difference in the value of the outcome between test group and control group, then we have evidence that P had an effect.

In a nutshell, C&H take issue with the conviction that random controlled trials  (RCT) -- the gold standard of causal inference and experiment in clinical medicine -- provide a basis for expecting that a given policy intervention will have similar effects in the future. Their book can be read as a critique of an excessively statistical understanding of social causality, without realistic analysis of the underlying mechanisms and processes. As Cartwright and Hardie state repeatedly, RCT evidence shows only that the policy worked on the circumstances tested in the study. Instead, they argue that we need to offer evidence about two additional considerations: whether the "causal principle" associated with P will remain the same in new circumstances; and whether the associated conditions necessary for the operation of this principle will be present in the new circumstances.

Here is a fundamental statement of what they mean by a causal principle:

We suppose that causes do not produce their effects by accident, at least not if you are to be able to make reliable predictions about what will happen if you intervene. Rather, if a cause produces an effect, it does so because there is a reliable, systematic connection between the two, a connection that is described in a causal principle. (22)

The statement, "El Nino causes wet winters in North America," is a causal principle. But causal principles are neither universal nor exceptionless:

The fact that causal principles can differ from locale to locale means that you cannot read off that a policy will work here from even very solid evidence that it worked somewhere else. (23)

Here is a more extensive description of this idea:

Causal principles are not universal. They differ from place to place and from time to time. That means that it is not enough for you to know that the policy worked somewhere or even that it has worked at some time here. “It worked there”; it played a positive causal role there. So it was one of the factors from a causal principle that holds there. To predict that it will work here, you need to know that it is one of the factors from a causal principle that holds here. That is what ensures that it can play a positive causal role for you. (50)

Cartwright and Hardie look at causation along the lines of J. L. Mackie's analysis of INUS conditions in The Cement of the Universe: a factor is a cause if it is an "Insufficient but Necessary part of an Unnecessary but Sufficient condition for producing a contribution to the effect" (23). The evidence of favorable CRT studies for a given policy intervention doesn't show that this policy will work in the new circumstances of the new proposed application. In order to draw this inference we need to have confidence that the treatment will play the same causal role in the new setting, and that the necessary conditions will be present in that setting. In other words, we need a more detailed causal analysis of the past and the proposed future.

Here is a sketch of the argument that C&H suggest we need to provide in order to project favorable RCT studies onto future applications:

1. x works there (i.e., x genuinely appears in the causal principle that governs the production of y there post-implementation). 
2. Here and there share that causal principle post-implementation. 
3. The support factors necessary for x to contribute under that principle are present for at least some individuals here post-implementation. 
4. Conclusion. x works here (i.e., x genuinely appears in the causal principle that governs the production of y here post-implementation and the support factors necessary for it to contribute to y are present for at least some individuals here post-implementation). (41)

One way of offering support for premise 2 is to engage in the method of process tracing:

This method confirms the existence of a causal connection between start and finish by confirming, one-by-one, a series of smaller causal steps in between. (38)

Cartwright doesn't put her case in these terms, but I would say that the heart of her intuition is that social outcomes are different from medical outcomes because of their inherent causal heterogeneity. In the social world outcomes like teen pregnancy rates or high school dropout rates are the result of a bundle of conjunctural causal processes. So projecting the results of past random controlled trials into the future requires that we first confirm that the same causal influences and important background conditions are at work. And this is rarely the case. So the fundamental underlying prescription is a pragmatic causal realism about social processes: in order to design and implement policies, we need to have a well developed map of the real causal processes and mechanisms that are underway in the production of the effect we would like to change. In other words, we need to be causal realists if we are to be effective policy makers.

(It is worth observing that this book is deliberately different in tone and specialization from Cartwright's other monographs in the philosophy of causation. The book is designed to be useful for real practitioners of public policy, and it offers clear advice about how to gain the understandings needed in order to validate the idea that a given policy will have desired effects in a novel setting.)

Current issues in causation research

This week's conference on Causality and Explanation in the Sciences in Ghent was an unusually good academic meeting (link). Participants gathered from all over Europe, as well as a few from North America, Australia, and South Africa, to debate the logic and substance of causal interpretations of the world. Among other things, it provided all participants with a very good sense of the ideas about causation that are generating the most discussion today.

A general perception that emerges from the gestalt of papers at the conference is that there are three large focus areas in current research on scientific causation. First, there is interest in specifying what causal assertions and concepts mean in scientific explanations. What are the logical, conceptual, and pragmatic issues associated with causal assertions and explanations?

Second, there is a large body of work focusing on the methods we can use to support causal inference in the sciences. Every field of science produces volumes of data about variables and events over time. What methods exist to permit inferences about causal relationships among the observed variables and entities? This includes causal modeling statistical methods, but also comparative methods deriving from Mill's methods of difference and similarity.

Third, there is a group of philosophers and scientists who are primarily interested in the ontology of causation in various parts of the sciences. How do various factors exercise causal powers in ecology, the social sciences, or complex systems? Researchers in these areas need provisional answers to questions raised by the first two groups, but their focus is on substantive causal processes rather than the logic of causal statements.

It is useful to inventory half a dozen approaches that were repeatedly cited. This survey is impressionistic but gives an idea of the current landscape.

The mechanisms approach. The idea that we can explicate causation through the idea of a mechanism has been rising in importance over the past twenty years. The idea here is that the fundamental causal concept is that of a mechanism through which X brings about or produces Y. This is argued to be key to causation from single-case studies to large statistical studies suggesting a causal relationship between two or more variables. Peter Hedstrom and other exponents of analytical sociology are recent voices for this approach for the social sciences, though expositions of this approach don't usually go into the level of detail expected by philosophers like Woodward and Cartwright. An important paper by Peter Machamer, Lindley Darden and Carl Craver, "Thinking about Mechanisms", sets the terms of current technical discussions; their view is referred to as the MDC theory. A common concern is that the approach hasn't been as clear as it should be about what precisely a mechanism is. James Mahoney made this criticism in 2001 in "Beyond Correlational Analysis" reviewing Charles Ragin, Fuzzy-Set Social Science and Peter Hedstrom and Richard Swedberg, Social Mechanisms: An Analytical Approach to Social Theory (link), and we still need a more generally recognized specification of the idea. (See an earlier post on this approach; link.)

The manipulability account. Jim Woodward is perhaps the leading exponent of the manipulability (or interventionist) account. He develops his views in detail in his recent book, Making Things Happen: A Theory of Causal Explanation. The view is an intuitively plausible one: causal claims have to do with judgments about how the world would be if we altered certain circumstances. If we observe that the concentration of sulphuric acid is increasing in the atmosphere, we might consider the increasing volume of H2SO4 released by coal power plants from 1960 to 1990. And we might speculate that there is a causal connection between these facts. A counterfactual causal statement holds that: If X (increasing emissions) had not occurred, then Y (increasing acid rain) would not have occurred. The manipulability theory adds this point: if we could remove X from the sequence, then we would alter the value of Y. And this in turn makes good sense of the ways in which we design controlled experiments.

Difference-making. Another strand of thinking about causation focuses on the explanations we are looking for when we ask about the cause of some outcome. Here philosophers note that there are vastly many conditions that are causally necessary for an event but do not count as being explanatory. Lee Harvey Oswald was alive when he fired his rifle in Dallas; but this doesn't play an explanatory role in the assassination of Kennedy. Crudely speaking, we want to know which causal factors were salient; which factors made a difference in the outcome. Michael Strevens provides a detailed and innovative explication of this set of intuitions in his recent book Depth: An Account of Scientific Explanation, where he introduces his theory of "Kairetic" explanation.

Contrastive analysis as a theory of explanation. When we seek an explanation of something, we generally have something specific in mind: why X rather than X'? And an explanation that keys off the wrong contrast will fail, even though its premises are correct. Bas van Fraassen (1980), The Scientific Image, is often cited in this context. A conference participant, Petri Ylikoski, develops a contrastive counterfactual theory in his dissertation (link). This body of work seeks to clarify pragmatic issues concerning explanation, including understanding and explanatory relevance. If we ask for an explanation for why X occurred, we are usually presupposing a question like this:

Why did X occur [rather than Y]?

• Why is John carrying his umbrella [rather than not]?
• Why is John carrying his umbrella [rather than his raincoat]?
• Why is John carrying his umbrella [rather than his assistant Harry]?

These all demand different answers:

• Because he expects rain;
• Because it is too warm for a raincoat;
• Because Harry is carrying three heavy suitcases.

Here is a much-cited review article by Nancy Cartwright on van Fraasen's work (link), and here is a discussion of contrastive explanation by Jonathan Schaffer (link).

Causal modeling theory. This topic refers to the large body of statistical theory devoted to identifying potential causal relationships among observable variables in a large data set. Hubert Blalock is a founder of this approach (Causal Inferences in Nonexperimental Research; 1964) with his statistical models for causal path analysis. (Here is a short account of the history of path analysis in genetics.) Judea Pearl has contributed a great deal to the method of structural equation modeling (SEM) in Causality: Models, Reasoning and Inference and elsewhere. Here is a handbook article in which he explains the method and its causal relevance (link). Pearl maintains a research blog on causality here. Granger causality is a specific technique for assessing causal relationships within time series data: X Granger-causes Y if variations in X and Y together do a better job of predicting Y than variations in Y by itself.

Prior foundations of philosophical theories of causation. Two older discussions of causality also received some notice in these papers: J. L. Mackie on INUS conditions and causal fields (The Cement of the Universe: A Study of Causation) and Wesley Salmon on the causal structure of the world (Scientific Explanation and the Causal Structure of the World).

Nancy Cartwright's "Causation: One Word, Many Things" provides a very good contemporary review of the varieties of approaches that are currently being taken to the idea of causation (link).

Much of the intellectual vitality of this group of philosophers is captured in the major work recently edited by Phyllis McKay Illari, Federica Russo, and John Williamson, Causality in the Sciences. The book contains a very wide range of disciplines and approaches in its treatment of the topic.

Causal realism for sociology

The subject of causal explanation in the social sciences has been a recurring thread here (thread). Here are some summary thoughts about social causation.

First, there is such a thing as social causation. Causal realism is a defensible position when it comes to the social world: there are real social relations among social factors (structures, institutions, groups, norms, and salient social characteristics like race or gender). We can give a rigorous interpretation to claims like "racial discrimination causes health disparities in the United States" or "rail networks cause changes in patterns of habitation".

Second, it is crucial to recognize that causal relations depend on the existence of real social-causal mechanisms linking cause to effect. Discovery of correlations among factors does not constitute the whole meaning of a causal statement. Rather, it is necessary to have a theory of the mechanisms and processes that give rise to the correlation. Moreover, it is defensible to attribute a causal relation to a pair of factors even in the absence of a correlation between them, if we can provide evidence supporting the claim that there are specific mechanisms connecting them. So mechanisms are more fundamental than regularities.

Third, there is a key intellectual obligation that goes along with postulating real social mechanisms: to provide an account of the ontology or substrate within which these mechanisms operate. This I have attempted to provide through the theory of methodological localism (post) -- the idea that the causal nexus of the social world is constituted by the behaviors of socially situated and socially constructed individuals. To put the claim in its extreme form, every social mechanism derives from facts about institutional context, the features of the social construction and development of individuals, and the factors governing purposive agency in specific sorts of settings. And different research programs target different aspects of this nexus.

Fourth, the discovery of social mechanisms often requires the formulation of mid-level theories and models of these mechanisms and processes -- for example, the theory of free-riders. By mid-level theory I mean essentially the same thing that Robert Merton meant to convey when he introduced the term: an account of the real social processes that take place above the level of isolated individual action but below the level of full theories of whole social systems. Marx's theory of capitalism illustrates the latter; Jevons's theory of the individual consumer ss a utility maximizer illustrates the former. Coase's theory of transaction costs is a good example of a mid-level theory (The Firm, the Market, and the Law): general enough to apply across a wide range of institutional settings, but modest enough in its claim of comprehensiveness to admit of careful empirical investigation. Significantly, the theory of transaction costs has spawned major new developments in the new institutionalism in sociology (Mary Brinton and Victor Nee, eds., The New Institutionalism in Sociology).

And finally, it is important to look at a variety of typical forms of sociological reasoning in detail, in order to see how the postulation and discovery of social mechanisms play into mainstream sociological research. Properly understood, there is no contradiction between the effort to use quantitative tools to chart the empirical outlines of a complex social reality, and the use of theory, comparison, case studies, process-tracing, and other research approaches aimed at uncovering the salient social mechanisms that hold this empirical reality together.

The historian's task

What are the intellectual tasks that define the historian's work? In a sense, this question is best answered on the basis of a careful reading of some good historians. But it will be useful to offer several simple answers to this foundational question as a sort of conceptual map of the nature of historical knowing.

First, historians are interested in providing conceptualizations and factual descriptions of events and circumstances in the past. This effort is an answer to questions like these: “What happened? What was it like? What were some of the circumstances and happenings that took place during this period in the past?” Sometimes this means simply reconstructing a complicated story from scattered historical sources – for example, in constructing a narrative of the Spanish Civil War or attempting to sort out the series of events that culminated in the Detroit race riot / uprising of 1967. But sometimes it means engaging in substantial conceptual work in order to arrive at a vocabulary in terms of which to characterize “what happened.” Concerning the disorders of 1967 in Detroit: was this a riot or an uprising? How did participants and contemporaries think about it?

Second, historians often want to answer “why” questions: “Why did this event occur? What were the conditions and forces that brought it about?” This body of questions invites the historian to provide an explanation of the event or pattern he or she describes: the rise of fascism in Spain, the collapse of the Ottoman Empire, the great global financial crisis of 2008. And providing an explanation requires, most basically, an account of the causal mechanisms, background circumstances, and human choices that brought the outcome about. We explain an historical outcome when we identify the social causes, forces, and actions that brought it about, or made it more likely.

Third, and related to the previous point, historians are sometimes interested in answering a “how” question: “How did this outcome come to pass? What were the processes through which the outcome occurred?” How did the Prussian Army succeed in defeating the superior French Army in 1870? How did Truman manage to defeat Dewey in the 1948 US election? Here the pragmatic interest of the historian’s account derives from the antecedent unlikelihood of the event in question: how was this outcome possible? This too is an explanation; but it is an answer to a “how possible” question rather than a “why necessary” question.

Fourth, often historians are interested in piecing together the human meanings and intentions that underlie a given complex series of historical actions. They want to help the reader make sense of the historical events and actions, in terms of the thoughts, motives, and states of mind of the participants. For example: Why did Napoleon III carelessly provoke Prussia into war in 1870 (David Baguley, Napoleon III and His Regime: An Extravaganza)? Why has the Burmese junta dictatorship been so intransigent in its treatment of democracy activist Aung San Suu Kyi (Nicholas Farrelly, Burma's General Objectives)? Why did northern cities in the United States develop such profound patterns of racial segregation after World War II (Thomas Sugrue, The Origins of the Urban Crisis: Race and Inequality in Postwar Detroit)? Why did young men in the 1910s and 1920s prefer dangerous, noisy internal combustion automobiles to safe, quiet electric vehicles (Gijs Moms, The Electric Vehicle: Technology and Expectations in the Automobile Age)? Answers to questions like these require interpretation of actions, meanings, and intentions – of individual actors and of cultures that characterize whole populations. This aspect of historical thinking is “hermeneutic,” interpretive, and ethnographic.

And, of course, the historian faces an even more basic intellectual task: that of discovering and making sense of the archival information that exists about a given event or time in the past. Historical data do not speak for themselves; archives are incomplete, ambiguous, contradictory, and confusing. The historian needs to interpret individual pieces of evidence; and he/she needs to be able to somehow fit the mass of evidence into a coherent and truthful story. So complex events like the Spanish Civil War present the historian with an ocean of historical traces in repositories and archives all over the world; these collections sometimes reflect specific efforts at concealment by the powerful (for example, Franco's efforts to conceal all evidence of mass killings of Republicans after the end of fighting); and the historian's task is to find ways of using this body of evidence to discern some of the truth about the past.

The photo above gives a small glimpse of the challenges the historian faces. In order to interpret the photo as "a moment in the Spanish Civil War", the historian needs to provide a careful interpretation of its provenance and content. Who are these soldiers? Where is the fighting taking place? Was the photo staged? What, if anything, does it tell us about the social conflicts and military circumstances of the Civil War? How can it help the reader of history to come to a better understanding of the experience of civil war?

In short, historians conceptualize, describe, contextualize, explain, and interpret events and circumstances of the past. They sketch out ways of representing the complex activities and events of the past; they explain and interpret significant outcomes; and they base their findings on evidence in the present that bears upon facts about the past. Their accounts need to be grounded on the evidence of the available historical record; and their explanations and interpretations require that the historian arrive at hypotheses about social causes and cultural meanings. Historians can turn to the best available theories in the social and behavioral sciences to arrive at theories about causal mechanisms and human behavior; so historical statements depend ultimately upon factual inquiry and theoretical reasoning. Ultimately, the historian's task is to shed light on the what, why, and how of the past, based on inferences from the evidence of the present.

MacIntyre and Taylor on the human sciences

There is a conception of social explanation that provides a common starting point for quite a few theories and approaches in a range of the social sciences. I'll call it the "rational, material, structural" paradigm. It looks at the task of social science as the discovery of explanations of social outcomes; and it brings an intellectual framework of purposive rationality, material social factors, and social structures exercising causal influence on individuals as the foundation of social explanation. Rational choice theory, Marxian economics, historical sociology, and the new institutionalism can each be described in roughly these terms: show how a given set of outcomes are the result of purposive choices by individuals within a given set of material and structural circumstances. These approaches depend on a highly abstracted description of human agency, with little attention to deep and important differences in agency across social, cultural, and historical settings. "Agents like these, in structures like those, produce outcomes like these." This is a powerful and compelling approach; so it is all the more important to recognize that there are other possible starting points for the social sciences.

In fact, this approach to social explanation stands in broad opposition to another important approach, the interpretivist approach. On the interpretive approach, the task of the human sciences is to understand human activities, actions, and social formations as unique historical expressions of human meaning and intention. Individuals are unique, and there are profound differences of mentality across historical settings. This "hermeneutic" approach is not interested in discovering causes of social outcomes, but instead in piecing together an interpretation of the meanings of a social outcome or production. This contrast between causal explanation and hermeneutic interpretation ultimately constitutes a major divide between styles of social thinking. (Yvonne Sherratt provides a very fine introduction to this approach; Continental Philosophy of Social Science.) Max Ringer, one of Weber's most insightful intellectual biographers, places this break at the center of Weber's development in the early twentieth century (Max Weber's Methodology: The Unification of the Cultural and Social Sciences). (See earlier discussions of two strands of thought in the philosophy of social science; link, link, link.)

On this approach, all social action is framed by a meaningful social world. To understand, explain, or predict patterns of human behavior, we must first penetrate the social world of the individual in historical concreteness: the meanings he/she attributes to her environment (social and natural); the values and goals she possesses; the choices she perceives; and the way she interprets other individuals' social action. Only then will we be able to analyze, interpret, and explain her behavior. But now the individual's action is thickly described in terms of the meanings, values, assumptions, and interpretive principles she employs in her own understanding of her world.

Most of the arguments in support of interpretive approaches to the human sciences have come from the continental tradition -- Dilthey, Ricoeur, Gadamer, Habermas. So let's consider two philosophers who have made original contributions to the historicist and interpretivist side of the debate, within the Anglo-American tradition. Consider first Alasdair MacIntyre's discussion of the possibility of comparative theories of politics ("Is a science of comparative politics possible?" in Alan Ryan, ed., The Philosophy of Social Explanation). MacIntyre poses the problem in these terms: "I shall be solely interested in the project of a political science, of the formulation of cross cultural, law-like causal generalizations which may in turn be explained by theories" (172). And roughly, MacIntyre's answer is that a science of comparative politics is not possible, because actions, structures, and practices are not directly comparable across historical settings. The Fiat strike pictured above is similar in some ways to a strike against General Motors or Land Rover in different times and places; but the political cultures, symbolic understandings, and modes of behavior of Italian, American, and British auto workers are profoundly different.

MacIntyre places great emphasis on the densely interlinked quality of local concepts, social practices, norms, and self ascriptions, with the implication that each practice or attitude is inextricably dependent on an ensemble of practices, beliefs, norms, concepts, and the like that are culturally specific and, in their aggregate, unique. Thus MacIntyre holds that as simple a question as this: "Do Britons and Italians differ in the level of pride they take in civic institutions?" is unanswerable because of cultural differences in the concept of pride (172-73).

Hence we cannot hope to compare an Italian's attitude to his government's acts with an Englishman's in respect of the pride each takes; any comparison would have to begin from the different range of virtues and emotions incorporated in the different social institutions. Once again the project of comparing attitudes independently of institutions and practices encounters difficulties. (173-74)

These points pertain to difficulties in identifying political attitudes cross-culturally. Could it be said, though, that political institutions and practices are less problematic? MacIntyre argues that political institutions and practices are themselves very much dependent on local political attitudes, so it isn't possible to provide an a-historical specification of a set of practices and institutions:

It is an obvious truism that no institution or practice is what it is, or does what it does, independently of what anyone whatsoever thinks or feels about it. For institutions and practices are always partially, even if to differing degrees, constituted by what certain people think and feel about them. (174)

So interpretation is mandatory -- for institutions no less than for individual behavior. So MacIntyre's position is disjunctive. He writes:

My thesis . . . can now be stated distinctively: either such generalizations about institutions will necessarily lack the kind of confirmation they require or they will be consequences of true generalizations about human rationality and not part of a specifically political science. (178)

Now turn to Charles Taylor in another pivotal essay, "Interpretation and the sciences of man" (Philosophical Papers: Volume 2, Philosophy and the Human Sciences). Taylor's central point is that the subject matter of the human sciences -- human actions and social arrangements -- always require interpretation. It is necessary for the observer to attribute meaning and intention to the action -- features that cannot be directly observed. He asks whether there are "brute data" in the human sciences -- facts that are wholly observational and require no "interpretation" on the part of the scientist (19)? Taylor thinks not; and therefore the human sciences require interpretation from the most basic description of data to the fullest historical description.

To be a full human agent, to be a person or a self in the ordinary meaning, is to exist in a space defined by distinctions of worth. . . . My claim is that this is not just a contingent fact about human agents, but is essential to what we would understand and recognize as full, normal human agency. (3)

Thus, human behaviour seen as action of agents who desire and are moved, who have goals and aspirations, necessarily offers a purchase for descriptions in terms of meaning what I have called "experiential meaning". (27)

One way of putting Taylor's critique of "brute data" is the idea that human actions must be characterized intentionally (34 ff.) in terms of the intentions and self understanding of the agent and that such factors can only be interpreted, not directly observed.

My thesis amounts to an alternative statement of the main proposition of interpretive social science, that an adequate account of human action must make the agents more understandable. On this view, it cannot be a sufficient objective of social theory that it just predict . . . the actual pattern of social or historical events. . . . A satisfactory explanation must also make sense of the agents. (116)

Taylor's discussion of ethnocentricity is important, since it provides a way out of the hermeneutic circle. He believes it is possible to interpret the alien culture without simply covertly projecting our categories onto the alien; and this we do through meaningful conversation with the other (124-25). This is a point that seems to converge with Habermas's notion of communicative action (The Theory of Communicative Action, Volume 1: Reason and the Rationalization of Society).

It isn't entirely clear how radically Taylor intends his argument. Is it that all social science requires interpretation, or that interpretation is a legitimate method among several? Is there room for generalizations and theories within Taylor's interpretive philosophy of social science? What should social science look like on Taylor's approach? Will it offer explanations, generalizations, models; or will it be simply a collection of concrete hermeneutical readings of different societies? Does causation have a place in such a science? (He says more about the role of theory in "Neutrality in political science"; Philosophical Papers: Volume 2, Philosophy and the Human Sciences, 63.)

Both MacIntyre and Taylor are highlighting an important point: human actions reflect purposes, beliefs, emotions, meanings, and solidarities that cannot be directly observed. And human practices are composed of the actions and thoughts of individual human actors -- with exactly this range of hermeneutic possibilities and indeterminacies. So the explanation of human action and practice presupposes some level of interpretation. There is no formula, no universal key to human agency, that permits us to "code" human behavior without the trouble of interpretation.

This said, I would still judge that the "rational, material, structural" paradigm with which we began has plenty of scope for application. For some purposes and in many historical settings, it is possible to describe the actor's state of mind in more abstract terms: he/she cares about X, Y, Z; she believes A, B, C; and she reasons that W is a good way of achieving a satisfactory level of attainment of the goods she aims at. In other words, purposive agency, within an account of the opportunities and constraints that surround action, provides a versatile basis for social action. And this is enough for much of political science, Marxist materialism, and the new institutionalism.