"Rigorous" sociology

There is sometimes an inclination within the social sciences to unify and "improve" the methodologies of the social sciences to allow them to be "fully scientific" in the way that chemistry or physics were thought to be in the neo-positivist phase of the philosophy of science. With something like these ambitions Klarita Gërxhani, Nan D. de Graaf, and Werner Raub's recent Handbook of Sociological Science: Contributions to Rigorous Sociology (2022) purports to be a "handbook for rigorous sociology" of all stripes.

Thomas Voss puts the perspective of the "scientific sociology" framework in these terms:

The core features of a scientific approach to sociology as described in this Handbook (see the chapter by Raub, De Graaf & Gërxhani) are as follows: sociology and social science in general is an explanatory empirical science – at least it is the goal to establish such a science. The aim of science is the explanation of regularities that have been established by systematic observation. Theories specify causal relationships and in conjunction with boundary conditions imply testable hypotheses. There are obviously some contrasts between the natural and the social sciences. However, scientific sociology is based on the idea of the unity of science, the conviction that there are no fundamental differences with respect to the methodological rules and criteria of evaluating theories between the sciences, such as physics or biology, and the social sciences. (492)

The preferred model of the structure of sociological knowledge expressed here is familiar from the philosophy of science of the 1950s and the writings of Carl Hempel. It is the hypothetico-deductive model of scientific knowledge, explanation, and confirmation. Scientific knowledge (in a given area of research) ideally consists of a set of abstract hypotheses about the way the world works in this area; logical-mathematical deductions from those hypotheses, along with supporting statements of boundary conditions, leading to "testable" predictive consequences for observable social facts. A social outcome or regularity is "explained" when the scientist succeeds in deducing its occurrence from a set of empirically supported theoretical hypotheses. This is familiar within the philosophy of science; it is the view of scientific knowledge that emerged when the verificationist and radical empiricist versions of philosophy of science associated with the Vienna Circle collapsed. Carl Hempel, Ernest Nagel, and Richard Rudner were central voices of this approach (link). And invoking "the unity of science" is harmful, since it brings with it a host of assumptions -- including reductionism -- that are positively harmful for our framework of thinking about the intellectual work needed in sociology. The social world is not unified, and neither are the sciences (link, link).

Another component of their view of core tools for "rigorous sociology" is the use of sophisticated statistical techniques to sort out large data sets of sociological data. They are also favorable towards computational social science and the use of tools like agent-based models and simulations.

Excessive empiricism is one shortcoming of the "rigorous sociology" framework. There is a second shortcoming of the conception of sociological knowledge that emerges from the volume. In spite of the statements of openness to theoretical and methodological diversity, the editors and many of the contributors are in fact committed to very specific theoretical and methodological ideas. The introduction of the volume is explicit: the best explanations in the social sciences conform to the assumptions of methodological individualism; and the editors clearly prefer micro- to macro-explanations as "most scientific". Coleman's boat is a central tool for their philosophy of science: explanations proceed down the strut from "social context" to "individuals acting and interaction", and up the strut from "individuals" to "macro-conditions". From these assumptions, the priority of rational-choice sociology, analytical sociology, and other individual-grounded approaches is all but unavoidable. John Goldthorpe, James Coleman, and Peter Hedström are cited repeatedly as examples of "good sociology". 

There is an obvious and important relationship between these ideas about rigorous sociology and the manifestos of analytic sociology. Gianluca Manzo draws out this close connection in his contribution to the volume, "Analytical sociology". The editors emphasize that they don't mean to propose that the social sciences should reflect a unified set of foundational theories or research methods; they are all for "diverse approaches" to the study of the social world. But they emphatically advocate for a core commitment to an agreed-upon core of methods of evaluation for scientific hypotheses in the social sciences. On their view, only such a core set of commitments about confirmation and falsification can provide a basis for "cumulative knowledge formation in the social sciences".

More interesting than general calls for "rigorous" verification of sociological claims is Ivan Ermakoff's contribution to the volume, "Validation strategies in historical sociology". Ermakoff's work falls broadly within the fields of historical sociology, and his views about the use of evidence and validation are specific and helpful. He considers a number of works in historical sociology, including Michael Mann's Dark Side of Democracy, and his perspective is a long way from the apparent positivism of the introductory essay. He considers a range of techniques used by historical sociologists to empirically evaluate their hypotheses and theories about mid-level social processes. He uses the umbrella term of "validation" rather than the loaded ideas of confirmation and verification as the crucial link between hypothesis and evidence. He writes, 

Validation is the linchpin of scientific rigor. Claims relying on arguments by seeing, embedding themselves in self-validating discursive serious, or dodging critical assessments undercut the prospect of sound and cumulative knowledge. A significant stake of therefore attached to clear-cut validation yardsticks. (196)

Ermakoff proposes seven different kinds of validation strategies for evaluating hypotheses in historical sociology. (He doesn't suggest the list is exhaustive.)

1. Descriptive fit
2. Probing observable implications of casual hypotheses
3. Counterfactuals
4. Natural experiments
5. Inductive comparisons
6. Process tracing
7. Simulation 

This is a much more diverse set of ideas about how to consider the relation between hypothesis, evidence, and inference than is often offered by empiricist theorists, and deserves careful study. The implications of the idea of "conjunctural causation" comes in for very useful discussion. And he emphasizes the importance to having a clear and coherent set of working ideas about social causation as well.

Descriptive claims are especially important in historical sociology, just as they are within the discipline of history itself. Consider the strategy of analysis and argument pursued in McAdam, Tarrow, and Tilly's Dynamics of Contention. Their goal is to discover some important mid-level generalizations about social contention in a range of settings. However, they do not seek to establish generalizations about the high-level categories of social contention -- civil war, ethnic violence, inter-state war, or riots, which they believe to be unattainable in principle. Instead, they purport to identify an open-ended list of "mechanisms of contention" that occur and recur in a variety of instances and episodes.  ("We search for mechanisms that appear variously combined in al these forms of contention and in others as well. A viable vision of contentious politics, we claim, begins with a search for causal analogies: identification of similar causes ni ostensibly separate times, places, and forms of contention" (74).) In Dynamics they consider a range of episodes of contention from many places and times and seek to describe them in sufficient detail to allow them to identify some of the mechanisms of mobilization, escalation, and repression that occurred in some of these cases. 

The three sequences sketched in Chapters 1 and 2 represent distinctive and well known varieties of contentious politics in the western tradition. Our treatment of them raised standard questions concerning mobilization, actors, and trajectories. In the course of contentious politics: (1) What processes move people into and out of public, collective claim making, and how? (2) Who's who and what do they do? (3) What governs the course and outcomes of contentious interaction? In each case, we found that the standard social movement agenda -- social change, mobilizing structures, opportunity -- provided a disciplined way of asking questions about the events, but pointed to unsatisfactory answers. The answers were unsatisfactory because they were static, because they provided accounts of single actors rather than relations among actors, and because at best they identified likely connections rather than causal sequences. (72)

McAdam, Tarrow, and Tilly are equally interested in asserting the causal efficacy of various kinds of social mechanisms, based on their analysis of the episodes. What would be involved in "validating" or empirically supporting the account provided in Dynamics? Ermakoff's seven approaches provide a good beginning to answering this question, and it would appear that McAdam, Tarrow, and Tilly have done a credible job of providing the right kinds of evidence and arguments that would serve to support their account. We would like to know whether the authors have provided reasonably accurate synopses of the episodes they have considered, and whether their accounts  are based on substantiated historical evidence (primary or secondary). Have they taken appropriate steps to avoid the kinds of biases in historical accounts about which Ermakoff warns us? Second, we would like to know whether the social mechanisms (issue escalation, for example) that they attribute to the episodes under review are reasonably clear and well defined. And third, we would want to explore the degree to which their claims of causal relevance for these mechanisms are justified by the available historical record. Here the techniques associated with process tracing and paired comparisons are most relevant to their arguments; they attempt to show in historical detail how various mechanisms worked in the given historical circumstances. This also makes "natural experiments" a credible basis for their causal reasoning as well. If, for example, the mechanism of issue escalation requires a moderate to high level of density in local social networks, and if it emerges that cases A, B, and C had the causal preconditions of issue escalation but showed substantial variation of network density, then the fact that low-density C did not experience issue escalation while high-density A and B did experience issue escalation, then this looks a lot like a natural experiment evaluating the causal efficacy of issue escalation.

MTT do not use other methods mentioned by Ermakoff. They do not use statistical measures to validate claims they make about the contentious episodes they consider. And they do not show any interest in computational simulations or "generative systems" that would permit them to predict outcomes. "Will current popular unhappiness in China about environmental degradation develop into organized demonstrations and demands against the state for greater environmental regulation?" -- this is not the kind of question that McAdam, Tarrow, and Tilly would find interesting or fruitful, because they are fundamentally aware of the contingency and path dependency that characterizes the emergence of unrest in most settings.

In short, Ermakoff's analysis of descriptive rigor and justification of causal claims seem to be rich enough to provide a basis for classifying the empirical practices of historical sociologists McAdam, Tarrow, and Tilly. And, significantly, these historical sociologists measure up: their work should be classified as "rigorous sociology".

The Perestroika debate in political science

A debate has been raging in the discipline of political science for at least a decade, over the nature of the scientific status and methods of the discipline. Fundamentally, the "dissidents" argue that a narrow and "scientistic" conception of what good political science research ought to look like has reigned and has repressed other, more pluralistic approaches to political science research. The formal methods of rational choice theory, game theory, and statistical analysis prevail, and the more narrative approaches associated with comparative research, area studies, and qualitiative research have been marginalized. And, the critics maintain, the flagship journals of the discipline and the tenure committees of the leading departments converge in maintaining this orthodoxy within the discipline. (Kristen Renwick Monroe has edited a valuable collection that gives the reader a pretty good understanding of the origins and faultlines of the debate; Perestroika!: The Raucous Rebellion in Political Science).

One of the central issues is this: what should a science of politics involve? What form of knowledge should political science produce? What is the role of universal laws or regularities in political science? How important are predictions?

Another key issue, related to the first, is the issue of the methodology of research that ought to be favored. Should quantitative methods be preferred? Should stylized assumptions be offered as the basis for formal rational-choice models of various forms of political behavior? What role should ethnographic research or case-study research play in the discovery of social-science knowledge?

Sanford Schram identifies some of the strands of the Perestroika critique in these terms: "Some focus on the overly abstract nature of much of the research done today, some on the lack of nuance in decontextualized, large-sample empirical studies, others on the inhumaneness of thinking about social relations in causal terms, and still others on the ways in which contemporary social science all too often fails to produce the kind of knowledge that can meaningfully inform social life" (Monroe : 103).

One of the most useful contributions to the Monroe book mentioned above is an essay by David Laitin. He takes issue with Bent Flyvbjerg's book, Making Social Science Matter: Why Social Inquiry Fails and How it Can Succeed Again, and his advocacy of "phronesis". Laitin characterizes the method of phronesis as one that is sensitive to context and that pays close attention to the singular and specific features of a particular social process -- for example, the positioning that occurs as a city decides on its economic development strategy. So the method of phronesis is intentionally not aiming to discover regularities across a set of instances, but rather to uncover some specific features of a particular ongoing process.

Laitin argues that this approach is too narrow a foundation for social-science knowledge. He assimilates the phronesis method to what he calls a "narrative" approach; and he argues that good social science needs to use a three-fold methodology. Investigators should make use of the tools of narrative analysis; but they also need to use statistical methods (quantitative analysis across cases) and formal modeling (models of complex social situations based on assumptions along the lines of rational choice theory). Laitin refers to this approach as a "tripartite" method of comparative research.

Where does the philosophy of social science fit into this debate? I suppose that the philosophy of social science I have advocated has quite a bit in common with the criticisms raised by the Perestroikans. My views emphasize the contingency of social processes, lack of social regularities, multiple conjunctural causes at work, plasticity of social institutions, the value of ethnographic work, and the need for a plurality of methods of inquiry and explanation in the social sciences. And these views are at odds with the natural-science assumptions about how social phenomena ought to be investigated that the Perestroika group is criticizing. And some of the researchers whom I admire most deeply -- James Scott, Charles Tilly, Benedict Anderson, Theda Skocpol, or Susanne Rudolph -- are cited in the original Perestroika manifesto! At the same time, I am committed to the idea of empirical rigor, causal explanation, and making a connection between social science knowledge and practical social problems -- a set of views that are post-positivist but still in the tradition of enlightened empiricism, and opposed to the currents of post-modern jargon that are sometimes mixed into the debate.

So the task is clear: to formulate a conception of social-science research and knowledge that preserves the values of empirical rigor and theoretial clarity, while embracing a pluralism that will permit the formulation of social-science knowledge adequate to the social world and social problems we find around ourselves. The Perestroika debate is an important one, and can help us better in the task of understanding society.

How does philosophy help guide the sciences?

Philosophy observes the sciences. But it has also played a role in the formation of the sciences. And this is especially true in the case of the social sciences.

The idea here is an elusive one. It is that the founders of the social sciences – perhaps similar to all intellectual or creative founders – possessed framing assumptions, presuppositions, or intuitions about what their eventual product ought to look like. Various ideas capture some of this: presupposition, paradigm, guiding framework, tacit knowledge, or “style”. A style of technology or architecture is a “mindset” that guides the creator into affirming one set of choices and denying another; ruling out certain solutions to a problem while favoring others. And many of these ideas derived from philosophy -- for example, empiricism, rationalism, deductivism, atomism, or reductionism.

The nineteenth- and twentieth-century founders of the social sciences had a set of intellectual interests that led them to ask questions about the way that society works. They were led to engage in careful, disciplined study of social phenomena. But how to proceed? What should the results look like? What modes of explanation should be pursued? What should they expect to find? None of the founders proceeded with a “blank slate”. Instead, they were guided by specific intellectual hunches and presuppositions about what a scientific treatment of a subject ought to involve. The histories of physics, chemistry, and biology were very well known to the founders, and the chief logical characteristics of the science of these domains were also well understood. The “stylized facts” about what a domain of inquiry is and what a scientific study of a domain involves were fairly specific. It turns out that these facts were misleading in deep and broad ways when applied to the social world. And contemporary sociology continues to bear the imprint of these early presuppositions.

We might be tempted to call these assumptions about domain, method, and theory a “paradigm”, but it is better to think of them as constituting a “proto-paradigm”. “Paradigm” describes a more advanced stage of the formation of a field of knowledge. The framing ideas that guided the founders were less specific; they represented high-level, abstract presuppositions about the nature of science and the nature of any subject matter that is amenable to scientific study and explanation. They constitute a framework of advanced commonsense about the subject matter. We might describe this framework as a “folk philosophy of knowledge” that is to some extent unexamined but that guides the pursuit of knowledge, the form that it takes, and the ways in which it is evaluated.

Given these historical circumstances, naturalism as a "proto-paradigm" for the social sciences is unsurprising—even though it is profoundly misleading. The strongest — really, the only — examples of scientific achievement in the nineteenth century were in the fields of chemistry, physics, medicine, and biology. There was a developed “proto-theory” of nature that was the object of scientific study (the characteristics and metaphysics of law-governed natural phenomena). The natural world was conceived of as a system of law-governed events and processes. And the logical characteristics of natural science theory were reasonably well understood as well: induction, discovery of laws and regularities, explanation through assimilation within a set of natural laws, confirmation.

There is a fundamental problem with this set of "naturalistic" presuppositions: social phenomena are constituted by a fundamentally different ontology. "Agents in structures" are the fundamental "molecules" of social life -- and this ontology should not be expected to give rise to strong regularities. Instead, we should expect a substantial amount of heterogeneity and plasticity among social entities and processes, and we should expect contingency and path-dependency in the unfolding of social phenomena.

Empirical constraints on sociological theories

What makes sociology "scientific"? An important component of a reply is that assertions, hypotheses, and theories are subject to the test of empirical evidence. Hypotheses need to be evaluated in terms of observations of how the real world behaves. We should evaluate our assertions in terms of their fit with the empirical facts. This is the "empiricist" constraint.

Post-positivist philosophers of science have noticed that these simple ideas raise many of puzzles, however. Consider these points:

• No set of observable facts guarantees the truth of a scientific assertion.
• There is no sharp distinction between observation and theory; our observations of the empirical facts commonly depend upon the assumption of some elements of scientific theory. Observations are "theory-laden".
  
• Even the empirical "facts" are subject to multiple interpretations; it is often possible to redescribe a set of observations in a way that appears to support contradictory hypotheses.

In the social sciences there are additional complexities about how to arrive at empirical observations and measurements.

• Social observations require us to "operationalize" the empirical facts we want to observe. For example, we may want to observe the standard of living of the working class. But we cannot achieve this directly. Instead, we need to arrive at "proxies" that are plausibly indicative of the property in question. So the wage basket that can be purchased with a given average money wage may be the index we use for measuring the standard of living. But there are other defensible ways of operationalizing the standard of living, and the various criteria may yield results that behave differently in given times and places.
  

• Social observation requires aggregation of measurements over a diverse group of individuals. We have to make judgments and choices when we arrive at a process for aggregating social data -- for example, the choice of using the gini coefficient rather than the share of income flowing to the bottom 40 percent as a measure of income inequality, or using the median rather than the mean to observe changes in income distribution. These choices must be made -- and there are no decisive empirical reasons that would decide the issue.

• Social concepts are needed to allow us to break down the social world into a set of facts. But there are plausible alternative conceptual schemes through which we can understand the nature and varieties of social phenomena. So, once again, we cannot hold that "observation" determines "theory".

These are familiar logical difficulties with the basic requirement of empiricism. However, they are not fatal difficulties. At bottom, it remains true that there is such a thing as social observation. It is necessary to accept that observations are theory-laden; that no observation is uncontrovertible; and that empirical evaluation depends upon judgment. All this accepted, there is a range of social observation that is relatively close to the ground and to which we can attribute some degree of epistemic warrant. Finally, there is available to us a coherence epistemology that permits a holistic and many-sided process of conveying warrant.

My view, then, is that the situation of sociology is less like physics (highly dependent on long chains of reasoning in order to assess empirical warrant) and more like journalism (grounded in careful and reasoned constructions of observations of the social world). The social world is reasonably transparent. We can arrive at reasonably confident observations of a wide range of social facts. And we can provide a logical analysis of the degree of credibility a given sociological theory has, given a fixed set of (corrigible) observations. Much of sociology is closely tied to descriptive inquiry, and the epistemic challenges come in at the stage of building our observations rather than our theories.

Moreover, the common views that natural science theories are "under-determined" by all available evidence (so that multiple theories can be equally well supported) and that scientific theories can only be supported or undermined as wholes (with no separate confirmation for parts of theories) appear to be largely inapplicable to the social sciences. Rather, social theories are more commonly of the "middle range", permitting piecemeal observation, testing, and empirical evaluation.

This also means that the celebrated hypothetico-deductive model of confirmation is less crucial in the social sciences than the natural sciences. The key explanatory challenge is to discover a set of causal processes that might explain the observed social world. And sophisticated observation is often the bulk of what we need.

(See "Evidence and Objectivity in the Social Sciences" for a little more on this topic. Ian Shapiro's recent book, The Flight from Reality in the Human Sciences, is a tough critique of excessive formalism and theor-ism in the social sciences.)