"Rigorous" sociology

There is sometimes an inclination within the social sciences to unify and "improve" the methodologies of the social sciences to allow them to be "fully scientific" in the way that chemistry or physics were thought to be in the neo-positivist phase of the philosophy of science. With something like these ambitions Klarita Gërxhani, Nan D. de Graaf, and Werner Raub's recent Handbook of Sociological Science: Contributions to Rigorous Sociology (2022) purports to be a "handbook for rigorous sociology" of all stripes.

Thomas Voss puts the perspective of the "scientific sociology" framework in these terms:

The core features of a scientific approach to sociology as described in this Handbook (see the chapter by Raub, De Graaf & Gërxhani) are as follows: sociology and social science in general is an explanatory empirical science – at least it is the goal to establish such a science. The aim of science is the explanation of regularities that have been established by systematic observation. Theories specify causal relationships and in conjunction with boundary conditions imply testable hypotheses. There are obviously some contrasts between the natural and the social sciences. However, scientific sociology is based on the idea of the unity of science, the conviction that there are no fundamental differences with respect to the methodological rules and criteria of evaluating theories between the sciences, such as physics or biology, and the social sciences. (492)

The preferred model of the structure of sociological knowledge expressed here is familiar from the philosophy of science of the 1950s and the writings of Carl Hempel. It is the hypothetico-deductive model of scientific knowledge, explanation, and confirmation. Scientific knowledge (in a given area of research) ideally consists of a set of abstract hypotheses about the way the world works in this area; logical-mathematical deductions from those hypotheses, along with supporting statements of boundary conditions, leading to "testable" predictive consequences for observable social facts. A social outcome or regularity is "explained" when the scientist succeeds in deducing its occurrence from a set of empirically supported theoretical hypotheses. This is familiar within the philosophy of science; it is the view of scientific knowledge that emerged when the verificationist and radical empiricist versions of philosophy of science associated with the Vienna Circle collapsed. Carl Hempel, Ernest Nagel, and Richard Rudner were central voices of this approach (link). And invoking "the unity of science" is harmful, since it brings with it a host of assumptions -- including reductionism -- that are positively harmful for our framework of thinking about the intellectual work needed in sociology. The social world is not unified, and neither are the sciences (link, link).

Another component of their view of core tools for "rigorous sociology" is the use of sophisticated statistical techniques to sort out large data sets of sociological data. They are also favorable towards computational social science and the use of tools like agent-based models and simulations.

Excessive empiricism is one shortcoming of the "rigorous sociology" framework. There is a second shortcoming of the conception of sociological knowledge that emerges from the volume. In spite of the statements of openness to theoretical and methodological diversity, the editors and many of the contributors are in fact committed to very specific theoretical and methodological ideas. The introduction of the volume is explicit: the best explanations in the social sciences conform to the assumptions of methodological individualism; and the editors clearly prefer micro- to macro-explanations as "most scientific". Coleman's boat is a central tool for their philosophy of science: explanations proceed down the strut from "social context" to "individuals acting and interaction", and up the strut from "individuals" to "macro-conditions". From these assumptions, the priority of rational-choice sociology, analytical sociology, and other individual-grounded approaches is all but unavoidable. John Goldthorpe, James Coleman, and Peter Hedström are cited repeatedly as examples of "good sociology". 

There is an obvious and important relationship between these ideas about rigorous sociology and the manifestos of analytic sociology. Gianluca Manzo draws out this close connection in his contribution to the volume, "Analytical sociology". The editors emphasize that they don't mean to propose that the social sciences should reflect a unified set of foundational theories or research methods; they are all for "diverse approaches" to the study of the social world. But they emphatically advocate for a core commitment to an agreed-upon core of methods of evaluation for scientific hypotheses in the social sciences. On their view, only such a core set of commitments about confirmation and falsification can provide a basis for "cumulative knowledge formation in the social sciences".

More interesting than general calls for "rigorous" verification of sociological claims is Ivan Ermakoff's contribution to the volume, "Validation strategies in historical sociology". Ermakoff's work falls broadly within the fields of historical sociology, and his views about the use of evidence and validation are specific and helpful. He considers a number of works in historical sociology, including Michael Mann's Dark Side of Democracy, and his perspective is a long way from the apparent positivism of the introductory essay. He considers a range of techniques used by historical sociologists to empirically evaluate their hypotheses and theories about mid-level social processes. He uses the umbrella term of "validation" rather than the loaded ideas of confirmation and verification as the crucial link between hypothesis and evidence. He writes, 

Validation is the linchpin of scientific rigor. Claims relying on arguments by seeing, embedding themselves in self-validating discursive serious, or dodging critical assessments undercut the prospect of sound and cumulative knowledge. A significant stake of therefore attached to clear-cut validation yardsticks. (196)

Ermakoff proposes seven different kinds of validation strategies for evaluating hypotheses in historical sociology. (He doesn't suggest the list is exhaustive.)

1. Descriptive fit
2. Probing observable implications of casual hypotheses
3. Counterfactuals
4. Natural experiments
5. Inductive comparisons
6. Process tracing
7. Simulation 

This is a much more diverse set of ideas about how to consider the relation between hypothesis, evidence, and inference than is often offered by empiricist theorists, and deserves careful study. The implications of the idea of "conjunctural causation" comes in for very useful discussion. And he emphasizes the importance to having a clear and coherent set of working ideas about social causation as well.

Descriptive claims are especially important in historical sociology, just as they are within the discipline of history itself. Consider the strategy of analysis and argument pursued in McAdam, Tarrow, and Tilly's Dynamics of Contention. Their goal is to discover some important mid-level generalizations about social contention in a range of settings. However, they do not seek to establish generalizations about the high-level categories of social contention -- civil war, ethnic violence, inter-state war, or riots, which they believe to be unattainable in principle. Instead, they purport to identify an open-ended list of "mechanisms of contention" that occur and recur in a variety of instances and episodes.  ("We search for mechanisms that appear variously combined in al these forms of contention and in others as well. A viable vision of contentious politics, we claim, begins with a search for causal analogies: identification of similar causes ni ostensibly separate times, places, and forms of contention" (74).) In Dynamics they consider a range of episodes of contention from many places and times and seek to describe them in sufficient detail to allow them to identify some of the mechanisms of mobilization, escalation, and repression that occurred in some of these cases. 

The three sequences sketched in Chapters 1 and 2 represent distinctive and well known varieties of contentious politics in the western tradition. Our treatment of them raised standard questions concerning mobilization, actors, and trajectories. In the course of contentious politics: (1) What processes move people into and out of public, collective claim making, and how? (2) Who's who and what do they do? (3) What governs the course and outcomes of contentious interaction? In each case, we found that the standard social movement agenda -- social change, mobilizing structures, opportunity -- provided a disciplined way of asking questions about the events, but pointed to unsatisfactory answers. The answers were unsatisfactory because they were static, because they provided accounts of single actors rather than relations among actors, and because at best they identified likely connections rather than causal sequences. (72)

McAdam, Tarrow, and Tilly are equally interested in asserting the causal efficacy of various kinds of social mechanisms, based on their analysis of the episodes. What would be involved in "validating" or empirically supporting the account provided in Dynamics? Ermakoff's seven approaches provide a good beginning to answering this question, and it would appear that McAdam, Tarrow, and Tilly have done a credible job of providing the right kinds of evidence and arguments that would serve to support their account. We would like to know whether the authors have provided reasonably accurate synopses of the episodes they have considered, and whether their accounts  are based on substantiated historical evidence (primary or secondary). Have they taken appropriate steps to avoid the kinds of biases in historical accounts about which Ermakoff warns us? Second, we would like to know whether the social mechanisms (issue escalation, for example) that they attribute to the episodes under review are reasonably clear and well defined. And third, we would want to explore the degree to which their claims of causal relevance for these mechanisms are justified by the available historical record. Here the techniques associated with process tracing and paired comparisons are most relevant to their arguments; they attempt to show in historical detail how various mechanisms worked in the given historical circumstances. This also makes "natural experiments" a credible basis for their causal reasoning as well. If, for example, the mechanism of issue escalation requires a moderate to high level of density in local social networks, and if it emerges that cases A, B, and C had the causal preconditions of issue escalation but showed substantial variation of network density, then the fact that low-density C did not experience issue escalation while high-density A and B did experience issue escalation, then this looks a lot like a natural experiment evaluating the causal efficacy of issue escalation.

MTT do not use other methods mentioned by Ermakoff. They do not use statistical measures to validate claims they make about the contentious episodes they consider. And they do not show any interest in computational simulations or "generative systems" that would permit them to predict outcomes. "Will current popular unhappiness in China about environmental degradation develop into organized demonstrations and demands against the state for greater environmental regulation?" -- this is not the kind of question that McAdam, Tarrow, and Tilly would find interesting or fruitful, because they are fundamentally aware of the contingency and path dependency that characterizes the emergence of unrest in most settings.

In short, Ermakoff's analysis of descriptive rigor and justification of causal claims seem to be rich enough to provide a basis for classifying the empirical practices of historical sociologists McAdam, Tarrow, and Tilly. And, significantly, these historical sociologists measure up: their work should be classified as "rigorous sociology".