Social science and policy

One of the important reasons that we value scientific knowledge is the possibility that it will allow us to intervene in the world to solve problems that we care about. Good climate science allows us to have high confidence in the causes of global climate change; and it also provides a sound basis for policy interventions to help to mitigate the pace of climate change. Good cellular biology permits a better understanding of autoimmune disease; and it also suggests avenues for prevention and treatment. There is thus an important component of pragmatism in our esteem for scientific knowledge.

In the social sciences we would like to assume that something similar is possible. If we have good sociological understanding of the causes of teen pregnancy or gang violence, perhaps that understanding will also provide a basis for designing effective interventions that reduce the incidence of the social problems we study. In other words, perhaps we can count on social science to provide a valuable and effective basis for the design of social policy.

The philosophy of social science that I've developed in this blog and in New Directions in the Philosophy of Social Science raises some challenges to that hope. It is argued here that the social world is contingent, heterogeneous, plastic, and conjunctural. In the words of Roy Bhaskar, social causation takes place in an open system in which we cannot arrive at confident predictions of particular social outcomes. In place of general theories and comprehensive social laws, it is argued here that we are best advised to seek out particular causal mechanisms that underlie various social outcomes of interest. And it is emphasized that it is difficult to make predictions in particular circumstances even when we have an idea of some of the operative social mechanisms, because of the perennial possibility of contingent interventions by additional factors.

So the hard question is this: to what extent is it at all possible for social science research to provide a confident basis for the design and implementation of social policies to address important social problems?

One approach that does not seem promising is the methodology of random controlled trials (RCT). The logical shortcomings of this approach when applied to social phenomena have been highlighted by Nancy Cartwright and Jeremy Hardie in Evidence-Based Policy: A Practical Guide to Doing It Better, and I discuss these problems In an earlier post (link). So it does not seem promising to expect that we will be able to isolate causal mechanisms (for example, "provide after-school tutoring") and use the method of RCT to demonstrate the efficacy of this mechanism in reducing a given social harm (say, "high school absenteeism").

The problem of establishing a strong relationship between theory and policy has been considered in several areas of social research. One such study is in the field of international relations. Stephen Walt's 2005 article, "The relationship between theory and policy in international relations", is an extended treatment of the topic (link). Here is the abstract to Walt's paper:

Policy makers pay relatively little attention to the vast theoretical literature in IR, and many scholars seem uninterested in doing policy-relevant work. These tendencies are unfortunate because theory is an essential tool of statecraft. Many policy debates ultimately rest on competing theoretical visions, and relying on a false or flawed theory can lead to major foreign policy disasters. Theory remains essential for diagnosing events, explaining their causes, prescribing responses, and evaluating the impact of different policies. Unfortunately, the norms and incentives that currently dominate academia discourage many scholars from doing useful theoretical work in IR. The gap between theory and policy can be narrowed only if the academic community begins to place greater value on policy-relevant theoretical work.

Fundamentally the article raises the question of whether there is a useful relationship between international relations theories and the practice of diplomacy and foreign policy. Can IR theory guide the construction of a successful foreign policy?

Here are some of the ways that Walt believes theory can be used to support policy analysis. Walt believes that theory can assist policy analysis in four important ways, including diagnosis, prediction, prescription, and evaluation. Unfortunately, none of the examples that he offers provide much confidence in any of these capabilities in a significant way. Diagnosis comes down to classification; but given that the idea of a social kind is suspect, we do not add much to our knowledge by classifying a given regime as "fascist", because we know that there is substantial variation across the group of fascist states. Prediction (as Gandhi said about Western civilization) would be nice; but it is almost never attainable in real social situations. Prescription requires a sound knowledge of the likely causal dynamics of a situation; but the open nature of social reality implies that we cannot have such knowledge in any comprehensive way. And evaluation is subject to similar issues. Walt assumes we can evaluate the success of a policy in a quasi-experimental way -- observe the cases where the intervention took place and measure the frequency of the desired outcome. But unfortunately this quasi-experimental method is also suspect.

An important drawback of Walt's treatment is the fairly traditional view that Walt takes with regard to the content of scientific knowledge. There is an underlying preposition of a fairly Humean view of cause and effect.

Policy makers can also rely on empirical laws. An empirical law is an observed correspondence between two or more phenomena that systematic inquiry has shown to be reliable. (25)

But in fact, there are very few useful "empirical laws" in the social realm that might serve as a basis for simple cause-and-effect policy design.

At present, then, there is a still a significant gap between an empirically supported social theory and a well designed social intervention. Unfortunately social causation is rarely as simple and regular as the empiricist framework presupposes. This is indeed disappointing, because it is certainly true that we most urgently need guidance in designing strategies for solving important social problems. (Here is an earlier post that offers a somewhat more positive assessment of the relevance of theory to policy; link.)