How to think about deliberate social change

What is involved in trying to create a better world? That is: what is involved in being an activist, a reformer, a radical, a revolutionary, or – for that matter – a reactionary? And how do the various forms of knowledge provided by areas of research in the social sciences play into this question? Is reforming the social world more like dreaming or more like building a cabinet?

This is an interesting set of questions for several reasons. One has to do with a very basic and crucial human feature – our capacity for rational-intentional manipulation of our various environments. Marx (and Hegel) put this feature of human capacity at the center of a rich conception of human “species being” – homo faber, the human tool-user, the creator through intelligent labor. The essence of “labor” is the use of one’s skills and knowledge to transform materials into objects that satisfy our needs, wants, and aesthetic values. And we would like to suppose that we can use our intelligence, our abilities to cooperate, and our practical skills at building social institutions, to construct a better world.

When we put this question in terms of transformation of the material world, we are talking about technology. We are led into a reflection on the knowledge provided by the natural sciences and the ways in which these forms of knowledge permit human beings to transform and build the natural environment; we are led to the disciplines of craftsmanship and technology. We are led to a consideration of practical intelligence and the artful transformation of nature. But what about purposive social change? How should we use knowledge and intelligence to bring about a better social world?

An important preliminary question is the scope of the possible: to what extent can we remake either the natural world or the social world? On the side of the natural world, the answer is fairly clear; humanity has thoroughly transformed the surface of the earth through its use of technology and scientific knowledge. (What is less clear is whether we have done so in ways that serve our collective interests, and whether unanticipated consequences have often derailed the benefits we sought.) An aircraft is the result of an integrated design process; a city is the result of a combination of plans, designs, accidents, and uncoordinated choices. So an aircraft is a much more intentional artifact than a city.

In the realm of the social world, the evidence is less clear when it comes to intentional transformation. There is ample evidence of social change, of course; but to what extent is that purposive or intentional social change? To what extent, perhaps, is largescale social change more like an ecological system than a building designed by an architect; more the result of many small and often unintended interventions rather than the result of a blueprint?

We can also ask the question: what sorts of things are we thinking to change, when we yearn for change? Here are several important possibilities: basic social structures; intermediate institutions; patterns of human behavior; mentalities and forms of social consciousness; social practices; and patterns of exploitation and domination.

So where do the social sciences come in when we consider the project of social reform? There are at least three locations within this story:

• in the theory of the present (why does contemporary America embody racial inequalities? What are the current social mechanisms?);
• in the theory of an ideal future (description of feasible institutions that produce equality); and
• in the theory of a strategy of change (description of feasible actions that can be taken over time leading to the establishment of new social arrangements).

What this comes down to is three rather different applications of the empirical and theoretical findings of the social sciences. First, the social sciences can provide the basis for a causal-institutional analysis of the way that the current system works. Second, the social sciences can allow us to “test” the viability of an institutional change and a new set of institutions, to try to estimate the way that these new institutions will function. This is a bit like simulating the behavior of a new device under a set of test conditions. Third, the social sciences can provide an inventory of a large number of mechanisms and processes of change – protest, demonstrations, armed struggle, lobbying, public relations campaigns, … This application permits us to attempt to evaluate the credibility of a proposed strategy of change.

So how should we think about purposive social change – especially change in the direction of novelty, systemic change, and large change of behavior and mentality? Careful reflection on the nature of the social suggests that social arrangements and processes are always conditioned by contingency, heterogeneity, and plasticity; so there are no “iron laws of history,” no general theories that tell us how social processes work. Instead, good social science theories shed light on specific, mid-level social mechanisms and processes – very ably described by McAdam, Tarrow, and Tilly in Dynamics of Contention. So we can forget about the social sciences providing a general “physics” or “mechanics” that might provide a blueprint for social design – as our understanding of the natural sciences does in fact permit in the design of material technologies. Instead, we can derive from the various areas of the social sciences some pragmatic rules of thumb about how mid-level social processes work.

How does this help to answer our question? Because we can use the open-ended totality of the learning that research in the social sciences has made possible about how the social world actually works, to help guide our thinking as we think about social reform and social change. We can attempt to mentally model the way that new institutions would work. We can attempt to foresee some of the glitches that will arise as we undertake a process of extended social reform. We are forced to be humble: prediction is very limited, there are always confounding factors that we haven’t incorporated into our mental models, and contingency runs deep. But we can make a start at evaluating a change strategy – the likely features of functioning that a given institutional design will display; the failures that may arise as a process of communication or coordination is undertaken; and the forms of corruption, weakness of the will, freeriding, and gaming that we can expect in any large process of social change.

A good example worth considering is the inadequate efforts that were undertaken towards reform of the financial industry following the 2008 financial crisis. These efforts were not highly successful, for several identifiable reasons: industry capture of the process, inadequate understanding of complex interactions among rules and regulatory institutions, and failure to anticipate innovative strategies to circumvent the rules. But equally difficult questions arise every kind of social reform – the design of a new set of tenure procedures in a university, the design of a system of regulation to end disparate use of force by US police departments against racial minorities, or the design of single-payer health insurance system.

Several ideas seem to come to the top out of these reflections.

(1) Designing social change is difficult, in part, because it requires us to imagine and create social arrangements that don’t yet exist. So we have to exert our analytical and empirical intelligence to try to estimate the way these arrangements might work – and how ordinary people might live within them. Are they feasible? And do they produce the intended social effects?

(2) The social sciences set some loose limits on our reasoning as we think about both systems and pathways; but social science will never permit precise calculations about the future. There is a very broad range of possible outcomes for almost any social intervention.

(3) The kinds of atrociously bad outcomes that twentieth-century “modernizing projects” have led to have occurred as a result of aggressive and misplaced confidence in social theories and models. This is a deeply important lesson. There is a very wide gap between theory and reality. Marxism and neoliberal purism alike have created enormous human suffering in the past century.

(4) This being recognized, social innovators ought to be risk-aware: take small steps, evaluate, examine; and proceed further. Just as designers of nuclear power plants need to design for worst-case scenarios and soft landings, so social innovators should be cautious as they push forward their reforms.

(5) All these cautions properly acknowledged, we have no choice but to attempt to create a better social future for humanity. And this means respecting the constraints of democracy even as we struggle for substantively better social arrangements.

(Here are several earlier discussions relevant to this topic; link, link, link, link.)

Greenblatt's new historicism

Stephen Greenblatt is a pathbreaking literary critic. But since the 1990s I've also looked at Greenblatt as a genuinely innovative and insightful contributor to the historical social sciences as well. He poses questions that are enormously important for anyone trying to make sense of humanity in history. What is a cultural identity? How do background social, cultural, and ideological conditions shape the individual's mental frameworks?

Greenblatt's advocacy and formulation of the "New Historicism" is an important contribution to literary criticism. But it is likewise a very important contribution to a topic of current interest to me, the question of the making of human cultures in concrete and heterogeneous terms. Nasrullah Mambrol does an excellent job of providing an exposition of the main ideas of the new historicism in a series of essays in Literary Theory and Criticism, including "New Historicism" (2020; link), "New Historicism: A Brief Note" (2016; link), and "Stephen Greenblatt and New Historicism" (2017; link).)

Of special interest to current philosophy of social science is the heterogeneity and plasticity that the "New Historicism" presupposes concerning cultures and worldviews. Here is how Mambrol (2017) summarizes this idea:

In contrast with this earlier formalism and historicism, the New Historicism questions its own methodological assumptions, and is less concerned with treating literary works as models of organic unity than as “fields of force, places of dissension and shifting interests, occasions for the jostling of orthodox and subversive impulses.” New Historicism also challenges the hierarchical distinction between “literary foreground” and “political background,” as well as between artistic and other kinds of production. It acknowledges that when we speak of “culture,” we are speaking of a “complex network of institutions, practices, and beliefs.”

Mambrol highlights this feature of "world-view heterogeneity" in his "New Historicism" piece (2020):

Unlike the prewar historicists, they [the New Historicists] refused to assume that Renaissance texts mirrored, from a safe distance, a unified and coherent world-view that was held by a whole population, or at least by an entire literate class. (Mambrol, 2020)

Culture and worldview are not "unitary" entities; rather, there are important differences across a population and within a generation with respect to very important aspects of identity. This is relevant to the interpretation of literature because it implies that the critic needs to try to work out how the author related to "the complex network of institutions" of culture that were prevalent in his or her lifetime. Here is Mambrol:

[Greenblatt] argues that the scene in which his authors lived was controlled by a variety of authorities— institutions such as the church, court, family, and colonial administration, as well as agencies such as God or a sacred book—and that these powers came into conflict because they endorsed competing patterns for organizing social experience. From Greenblatt’s New Historicist perspective, the rival codes and practices that these authorities sponsored were cultural constructions, collective fictions that communities created to regulate behavior and make sense of their world; however, the powers themselves tended to view their customs as natural imperatives, and they sought to represent their enemies as aliens or demonic parodists of genuine order. (Mambrol 2020)

To put it simply, there was no unitary mentality of sixteenth-century English culture; rather, there were multiple narratives and texts with rather different implications for the individual. We might ask, though Mambrol does not, whether all such variants were equally powerful, or whether one or more were "hegemonic"; and the answer seems obvious: powerful institutions like the church and the state also had powerful means for conveying the lineaments of their preferred ideology. But hegemony does not imply univocal acceptance, and as Gramsci insisted, resistant voices and thoughts are possible even within "hegemonic" ideological forces. This means, as Greenblatt is very eloquent in discovering, that subversive and critical ideas in literature can emerge from a contentious set of ideologies, some of which are "hegemonic". (See James Scott's similar view about "hidden transcripts" in Domination and the Arts of Resistance: Hidden Transcripts; link.)

Mambrol suggests that Greenblatt's 1987 essay "Towards a Poetics of Culture" is an important place where readers can find clarification and amplification of the purpose and ideas of the New Historicism (link). The essay is published with a few minor changes as chapter one of H. Aram Veeser's The New Historicism. Greenblatt is emphatic about several points. The New Historicism is not a new "theory" of criticism or of the relationship between literature and the social environment; it is not a manifesto. Instead, it is a call for recognition of the fluidity of culture and worldview, and the many-many relationships that exist between the many cultures of a time and the literatures and artistic creations of the time. Literature emerges from concrete social and mental arrangements in society; but the relationships that exist are not deterministic, and -- crucially -- different authors respond to different versions of the mental frameworks of their time. Here is Greenblatt:

I propose that the general question addressed by Jameson and Lyotard—what is the historical relation between art and society or between one institutionally demarcated discursive practice and another?—does not lend itself to a single, theoretically satisfactory answer of the kind that Jameson and Lyotard are trying to provide. Or rather theoretical satisfaction here seems to depend upon a utopian vision that collapses the contradictions of history into a moral imperative. (kl 370)

His critique of both Jameson (New Marxism) and Lyotard (Post-structuralism) is that both systems are flawed precisely because they hope that a single abstract theory of capitalism can answer the question of the relationship between literature and society. No! Rather, historical inquiry -- inquiry that is sensitive to detail, to contingency, and to the heterogeneity of social and structural realities -- will suffice to provide a partial answer to a question such as: How did Shakespeare's cultural and mental world influence his plays? Historical research, not theoretical development and application, can allow for meaningful and illuminating answers to particular questions like these; and there are no general theoretical answers to the big question: How do social institutions influence or determine artistic and literary creation?

These comments have to do with literary interpretation. But the point is relevant for social scientists for a similar reason: rather than asking, "how do white working class men think about Donald Trump?" or "how do Latina women relate to the Democratic Party?", we need to be receptive to the idea that there is no single answer to the question. There is variation across each of these groups. At the same time, the ideological struggle between "white supremacy", "Reaganite rejection of government power", and "adherence to liberal democratic values" reflects a hugely important battleground of identities and mental frameworks in the contemporary United States. Greenblatt's masterful analysis of the currents of authoritarianism in contemporary US politics, and the Trump presidency in particular, through his analysis of Shakespeare's tyrants, is brilliant (link). And the aptness of Greenblatt's treatment of Richard III as a foil for Donald Trump illustrates an important point: tyranny depends on shaping values and expectations in the public. And for mysterious reasons, it is hard to dispute the fact that Trump and his acolytes shifted the terms of public thinking about the state and political legitimacy. What seemed like unhinged bravado when candidate Trump uttered these words now seems only an obvious truth today (as quoted in The Guardian in 2018): "US Republican frontrunner Donald Trump is so confident in his support base that he said he could stand on New York’s Fifth Avenue “and shoot somebody” and still not lose voters."

What is the relevance of "New Historicism" to the idea that humanity is self-creating when it comes to culture? The connection is straightforward. Greenblatt demonstrates in his criticism and his more reflective writings that literary creation emerges from an intellectual and moral world that is full of difference, change, and contradiction; that the ambient ideas of a time are sometimes horrible, sometimes dominant, but never univocal; and that writers like Shakespeare have the ability to create something new for human beings to contemplate as they consider morality, social existence, and the tasks of living together. This implies a capacity for creativity that extends beyond literature and into morality and narratives of identity.

Albert Hirschman on uncertainty

Albert Hirschman was a particularly important non-conformist in 20th-century social science. (Here is an earlier discussion of Jeremy Adelman's biography of Hirschman (link).) Two of the things I admire most about him are his unwillingness to be bound by disciplinary divisions and his deep understanding of the uncertainty of virtually all social-science predictions. The social world is too complex, there are too many competing causal and agential factors influencing outcomes, to permit us to have confidence in the precise outcomes of social interventions in the future. The pristine mathematical theories of economics and the rational-choice models of political science alike provide a semblance of predictive precision; but upon examination, we discover that we can have little confidence in those predictions. The social world is orderly but contingent, and local differences in circumstances matter. (Here is an earlier post on social contingency (link), and another on path-dependence (link).)

Michele Alacevich's brilliant intellectual biography of Hirschman (Albert O. Hirschman: An Intellectual Biography) provides new focus on these important insights from Hirschman's intellectual itinerary. Alacevich is an expert on the history of World Bank policies and practices, and this leaves him well situated to assess Hirschman's evolving views of the nature of economic development policy and large strategies of social and political reform. Simon Torracinta provides an outstanding and extensive review of the biography in Boston Review (link). As both Alacevich and Torracinta point out, Hirschman's insights are in danger of being lost in the forest of ideas we have about the power and limitations of the social sciences, so it is worthwhile highlighting several of those ideas. Both Alacevich's book and Torracinta's review essay reward a close reading, but here I will pull out several central ideas that they highlight.

Alacevich places particular importance on Hirschman's own experience in the field in projects aimed at stimulating economic development in Latin America (Columbia in particular). Hirschman witnessed the mismatch that so often developed between the goals and predictions associated with the grand strategies of development, and the actual experience as a particular project played out. Hirschman developed a deep skepticism about comprehensive blueprints of change, to be applied uniformly to the circumstances of various regions or countries. Rather, Torracinta emphasizes the aspects of pragmatism and piecemeal adjustment that underlay Hirschman's view of how social progress could occur. "Try this, adjust, then try that." Here is how Torracinta paraphrases Hirschman's approach:

“A priori deductions,” Hirschman wrote in an assessment of Italian reconstruction in 1947, “while instructive, can only yield extremely rough guesses and are not able to replace as yet the method of trial and error.” He added, in a sentence that could just as well have been written by heterodox analysts of post-pandemic recovery, that looking for the “correct” aggregate volume of investments in reconstruction was a “futile search.” Instead, “one should concentrate upon locating those investments which permit the breaking of important bottlenecks and will thereby lead to increases of output and improvements of performances out of proportion to the investment itself.”

This is "pragmatic", in the sense that it involves a process of informed trial-and-error, followed by assessment of the consequences; and it is piecemeal, in that it advises the reformer to engage in an extended step-by-step process involving adjustment and course-correction along the way. This involves an extensive reliance on decentralized decision-making, with -- once again -- the emergence of deep uncertainties about the consequences of various choices. Social change always involves uncertainty.

Here is how Torracinta sums up Hirschman's intellectual legacy:

What are we to make of this complex legacy? There remain a few Hirschmanian figures still scattered across the academy (the probing economist Dani Rodrik comes to mind). But in retrospect, ambitious balanced and unbalanced growth programs had more in common with each other than with the ideas that succeeded them: consider the socially devastating “reforms” imposed on developing countries by the IMF’s structural adjustment programs in the 1980s. Ironically, given the abeyance into which they fell in that period, many foundational insights of high development theory have now been reincorporated since then—in appropriate mathematical form—into the models of development economics in recent decades. The great inflation debate of 2021 makes it clear, however, that no matter how sophisticated or powerful they may be, models remain a highly contested feature of contemporary economics. Given the theoretical rigidity, mathematical formalism, and fierce professional hierarchy of the mainstream discipline today, Hirschman’s early skepticism of these trends looks more prescient than tragic.

So Torracinta believes that much of the valuable insight offered by Hirschman about the policy process and the possibilities of guided reform has been lost -- once again, deferring to the false confidence offered by formal economic models and rational-choice formalizations of political processes.

Alacevich offers a penetrating account of Hirschman's legacy that emphasizes the degree of contingency, creativity, and uncertainty that exists in the social world:

Hirschman’s emphasis on the concept of possibilism is arguably the most explicit statement of what he thought was his contribution to the deliberative process as reformist activism, and to the study of it as social science. Most social scientists, Hirschman noted, focus on explaining the regularities of social dynamics, and this is obviously an important task. But Hirschman emphasized the opposite type of endeavour: “to underline the multiplicity and creative disorder of the human adventure, to bring out the uniqueness of a certain occurrence, and to perceive an entirely new way of turning a historical corner.” This was particularly promising in seeking to explain the process of social change, for, he added, unless “novelty, creativity, and uniqueness” take place, large-scale social change cannot occur. In the first place, if all elements of social dynamics were already known, reactionary forces could easily foresee and preempt them. Second, he wrote, “radical reformers are unlikely to generate the extraordinary social energy they need to achieve change unless they are exhilaratingly conscious of writing an entirely new page of human history.” (250-251)

Hirschman’s attention to the possible over the probable, to the conjunctural over the structural, is the basis of yet another of his deep-rooted predilections and a fundamental element of his cognitive style—that is, the importance of history. In diametric opposition to the standards of social analysis that took shape after World War II, Hirschman considered the study of history an enormously rich and ineluctable source for understanding social change. (252)

There are several especially powerful ideas embedded here: "attention to the possible over the probable, to the conjunctural over the structural" and "unless 'novelty, creativity, and uniqueness' take place, large-scale social change cannot occur." Both ideas emphasize key aspects of the social world and of social change: heterogeneity, contingency, and the importance of agency. To this we might add the importance of a pragmatic approach to social change that recognizes the limitations of abstract utopian theories of the future. (Ironically, in another recent issue of Boston Review Martin O'Neill reviews Ed Miliband's GO BIG: How To Fix Our World, under the title "Against Incrementalism: Center-left parties should learn that small-bore solutions are a waste of time" (link). It is interesting to consider whether the impatience that many have with "incrementalism" is consistent with the valid insights and critiques offered by Hirschman of the ability of theory to guide comprehensive processes of change.) 

Hirschman was a singular contributor to the social sciences, and his work rewards close reading. Alacevich's biography is an important contribution to understanding Hirschman's legacy and his continuing importance for our understanding of the nature of the social sciences and social change.

(I took particular pleasure in meeting Albert Hirschman while presenting a seminar at the Center for International Studies at Princeton in the 1990s. I presented an early version of my research on what became The Paradox Of Wealth And Poverty: Mapping The Ethical Dilemmas Of Global Development. Hirschman was enormously generous and stimulating with his comments, and he was especially supportive of the goal of bringing normative thinking back into the field of development economics. It was a memorable intellectual pleasure to have spent half an hour discussing these ideas with him.)

Five easy pieces (for the social sciences)

Social scientists are generally interested in "explaining" social outcomes: why did such-and-so take place as it did? Why did the Indochina War occur, and why did it end in the defeat of two modern military powers? Why did the French fail so miserably at Dien Bien Phu? Why was the Tet Offensive so consequential for US military plans in Vietnam? Here are some fundamental questions surrounding the search for social explanations:

1. What is involved in "explaining" a social event or circumstance?
2. In what sense is there a kind of "order" in the social world?
3. Can we reconcile the idea that some social events are "explained" and others are "stochastic"?
4. Are there general and recurring causes at work in the social world?
5. Is there any form of "unity" possible in the social sciences?

It is tempting to hold that many social events are more akin to the white noise of wind in the leaves than planets moving around the sun. That is, we might maintain that many social events are stochastic; they are the result of local contingencies and conjunctions, with little or no underlying order or necessity. This is not to say that the stochastic event is uncaused; rather, it is to say that the causes that led to this particular outcome represent a very different mix of conditions and events from the background of other similar events, so there is no common and general explanation of the event. 

Why should we think that social events often have this high degree of underlying stochasticity or contingency? One reason is the general ontological fact that social events are the result of the actions, decisions, interactions, and mental frameworks of specific individual actors, from anonymous consumers to business entrepreneurs to "social media influencers" to political leaders. The actors all have their own motivations and circumstances, so the strategies and actions that they choose are likely enough to be highly particular. And yet those strategies and actions eventually aggregate to social outcomes that we would like to understand: for example, why one midwestern town in the 1930s became a thriving manufacturing center, while another became a sleepy county seat with two traffic lights and a diner. William Cronon's marvelous account in Nature's Metropolis: Chicago and the Great West of the rise of Chicago as the major metropolis in the middle of the country illustrates the deep level of contingency associated with this history (link).

Beyond the contingencies created by the fact of varying individual motivations and strategies, there is the fact that social outcomes are generally "conjunctural": they are the result of multiple causal influences, and would have developed differently if any of those influences had been significantly different. The dramatic growth of intercontinental trade in the 1960s and later decades depended on several independent factors -- liberalization of trading regimes in many countries and regions, technology change in shipping (containerization), manufacturing companies that were legally able to "off-shore" production of consumer goods, and the like. Each of these factors has its own history, and substantial change in any one of them would presumably have had great consequences for the volume of trade during those decades.

Instead of imagining that all social outcomes should be amenable to simple, general explanations, we should instead take a pluralistic view of the structural and social circumstances that sometimes propel individual actors to one kind of outcome rather than another. It was not inevitable that Chicago would become "nature's metropolis" in the midwest; but the fact that it had easy access to the Great Lakes for cheap transportation, and to the farmlands of the midwest for ample sources of grain and meat made Chicago a more likely place for opportunistic actors to establish the makings of a great city than Springfield, Illinois. Likewise, once the routes of the great east-west railways were established (reflecting their own forms of contingency and struggle among actors), Chicago emerged as a more promising location for business and trade than Oshkosh, Wisconsin.

I referred to this kind of explanation as an "institutional logic" explanation in The Scientific Marx (link). And this kind of explanation has much in common with the explanatory framework associated with the "new institutionalism" (link).

So let's return to the questions posed above:

1. What is involved in "explaining" a social event or circumstance?

We explain a social event when we show how it arose as a result of the actions and interactions of multiple social actors, engaged within a specified social, economic, political, and natural environment, to accomplish their varied and heterogeneous purposes. Sometimes the thrust of the explanation derives from discovering the surprising motives the actors had; sometimes it derives from uncovering the logic of unintended consequences that developed through their interactions; and sometimes it derives from uncovering the features of the institutional and natural environment that shaped the choices the actors made.

2. In what sense is there a kind of "order" in the social world?

There is one underlying foundation (unity) of all social outcomes -- the fact of the composition of outcomes from the actions and mental frameworks of multiple social actors. However, given the heterogeneity of actors, the heterogeneity of the institutions and natural environments in which they act, and the pragmatic question of whether it is the institutional background or the actors' characteristics that are of the greatest explanatory interest, there is no basis for expecting a single unified substance and form for social explanations.

3. Can we reconcile the idea that some social events are "explained" and others are "stochastic"?

One way of making this distinction is to highlight the generality or particularity of the set of conditions that were thought to bring about the event. If O was the result of A,B,C,D,E and each of A-E was contingent and unrelated to the occurrence of the other conditions, then it would be natural to say that O was stochastic. If O was the result of A-E and A,B,C were longstanding conditions while D and E were conditions known to recur periodically, then it would be natural to say that O was explained by the joint occurrence of D and E in the presence of A,B,C. To explain an event is to claim that there is some underlying "necessity" leading to its occurrence, not just a chance conjunction of independent events.

4. Are there general and recurring causes at work in the social world?

There are recurring causes in the social world -- institutional and natural circumstances that shape actors' choices in similar ways in multiple settings. Examples include technological opportunities, economic geography, available methods of warfare, available systems of taxation and governance, and the range of institutional variations that are found in every society. It is the work of historical and social research to discover these kinds of factors, and the discovery of a common causal factor is simultaneously the discovery of a causal mechanism.

5. Is there any form of "unity" possible in the social sciences?

The social sciences cannot be unified around a single theory -- microeconomics, rational choice theory, hermeneutics, Marxism. Instead, social scientists need to approach their work with a diverse toolbox of theories and mechanisms on the basis of which to construct hypotheses about the explanation of diverse social phenomena; they should expect heterogeneity rather than underlying unity and homogeneity. Theoretical pluralism is necessary for a correct understanding of the workings of the social world.

(This is a topic that I've returned to a number of times over the past fifteen years. Here is an argument for why we should not expect to find a unified theory of society (link) from 2008, and here are discussions of "what can be explained in the social world" from 2008 (link) and 2016 (link). In 2014 I discussed the idea of "entropic social processes" (link).)

Explaining large historical change

Great events happen; people live through them; and both ordinary citizens and historians attempt to make sense of them. Examples of the kinds of events I have in mind include the collapse of communism in Eastern Europe and the USSR; the rise of fascism in Europe in the 1930s; the violent suppression of the Democracy Movement in Tiananmen Square; the turn to right-wing populism in Europe and the United States; and the Rwandan genocide in 1994. My purpose here is to identify some of the important intellectual and conceptual challenges that present themselves in the task of understanding events on this scale. My fundamental points are these: large-scale historical developments are deeply contingent; the scale at which we attempt to understand the event matters; and there is important variation across time, space, region, culture, and setting when it comes to the large historical questions we want to investigate. This means that it is crucial for historians to pay attention to the particulars of institutions, knowledge systems, and social actors that combined to create a range of historical outcomes through a highly contingent and path-dependent process. The question for historiography is this: how can historians do the best job possible of discovering, documenting, and organizing their accounts of these kinds of complex historical happenings?

Is an historical period or episode an objective thing? It is not. Rather, it is an assemblage of different currents, forces, individual actors, institutional realities, international pressures, and popular claims, and there are many different “stories” that we can tell about the period. This is not a claim for relativism or subjectivism; it is rather the simple and well-understood point for social scientists and historians, that a social and historical realm is a dense soup of often conflicting tendencies, forces, and agencies. Weber understood this point in his classic essay “’Objectivity’ in Social Science” when he said that history must be constantly re-invented by successive generations of historians: “There is no absolutely “objective” scientific analysis of culture—or put perhaps more narrowly but certainly not essentially differently for our purposes—of “social phenomena” independent of special and “one-sided” viewpoints according to which—expressly or tacitly, consciously or unconsciously—they are selected, analyzed and organized for expository purposes” (Weber 1949: 72). Think of the radically different accounts offered of the French Revolution by Albert Soboul, Simon Schama, and Alexis de Tocqueville; and yet each offers insightful, honest, and “objective” interpretations of part of the history of this complex event.

We need to recall always that socially situated actors make history. History is social action in time, performed by a specific population of actors, within a specific set of social arrangements and institutions. Individuals act, contribute to social institutions, and contribute to change. People had beliefs and modes of behavior in the past. They did various things. Their activities were embedded within, and in turn constituted, social institutions at a variety of levels. Social institutions, structures, and ideologies supervene upon the historical individuals of a time. Institutions have great depth, breadth, and complexity. Institutions, structures, and ideologies display dynamics of change that derive ultimately from the mentalities and actions of the individuals who inhabit them during a period of time. And both behavior and institutions change over time.

This picture needs of course to reflect the social setting within which individuals develop and act. Our account of the "flow" of human action eventuating in historical change needs to take into account the institutional and structural environment in which these actions take place. Part of the "topography" of a period of historical change is the ensemble of institutions that exist more or less stably in the period: cultural arrangements, property relations, political institutions, family structures, educational practices. But institutions are heterogeneous and plastic, and they are themselves the product of social action. So historical explanations need to be sophisticated in their treatment of institutions and structures.

In Marx's famous contribution to the philosophy of history, he writes that "men make their own history; but not in circumstances of their own choosing." And circumstances can be both inhibiting and enabling; they constitute the environment within which individuals plan and act. It is an important circumstance that a given time possesses a fund of scientific and technical knowledge, a set of social relationships of power, and a level of material productivity. It is also an important circumstance that knowledge is limited; that coercion exists; and that resources for action are limited. Within these opportunities and limitations, individuals, from leaders to ordinary people, make out their lives and ambitions through action.

On this line of thought, history is a flow of human action, constrained and propelled by a shifting set of environmental conditions (material, social, epistemic). There are conditions and events that can be described in causal terms: enabling conditions, instigating conditions, cause and effect, ... But here my point is to ask you to consider whether uncritical use of the language of cause and effect does not perhaps impose a discreteness of historical events that does not actually reflect the flow of history very well. It is of course fine to refer to historical causes; but we always need to understand that causes depend upon the structured actions of socially constituted individual actors.

A crucial idea in the new philosophy of history is the fact of historical contingency. Historical events are the result of the conjunction of separate strands of causation and influence, each of which contains its own inherent contingency. Social change and historical events are highly contingent processes, in a specific sense: they are the result of multiple influences that "could have been otherwise" and that have conjoined at a particular point in time in bringing about an event of interest. And coincidence, accident, and unanticipated actions by participants and bystanders all lead to a deepening of the contingency of historical outcomes. However, the fact that social outcomes have a high degree of contingency is entirely consistent with the idea that the idea that a social order embodies a broad collection of causal processes and mechanisms. These causal mechanisms are a valid subject of study – even though they do not contribute to a deterministic causal order.

What about scale? Should historians take a micro view, concentrating on local actions and details; or should they take a macro view, seeking out the highest level structures and patterns that might be visible in history? Both perspectives have important shortcomings. There is a third choice available to the historian, however, that addresses shortcomings of both micro- and macro-history. This is to choose a scale that encompasses enough time and space to be genuinely interesting and important, but not so much as to defy valid analysis. This level of scale might be regional – for example, G. William Skinner’s analysis of the macro-regions of China. It might be national – for example, a social history of Indonesia. And it might be supra-national – for example, an economic history of Western Europe. The key point is that historians in this middle range are free to choose the scale of analysis that seems to permit the best level of conceptualization of history, given the evidence that is available and the social processes that appear to be at work. And this mid-level scale permits the historian to make substantive judgments about the “reach” of social processes that are likely to play a causal role in the story that needs telling. This level of analysis can be referred to as “meso-history,” and it appears to offer an ideal mix of specificity and generality.

Here is one strong impression that emerges from the almost any area of rigorous historical writing. Variation within a social or historical phenomenon seems to be all but ubiquitous. Think of the Cultural Revolution in China, demographic transition in early modern Europe, the ideology of a market society, or the experience of being black in America. We have the noun -- "Cultural Revolution", “European fascism”, “democratic transition” -- which can be explained or defined in a sentence or two; and we have the complex underlying social realities to which it refers, spread out over many regions, cities, populations, and decades.

In each case there is a very concrete and visible degree of variation in the factor over time and place. Historical and social research in a wide variety of fields confirms the non-homogeneity of social phenomena and the profound location-specific variations that occur in the characteristics of virtually all large social phenomena. Social nouns do not generally designate uniform social realities. These facts of local and regional variation provide an immediate rationale for case studies and comparative research, selecting different venues of the phenomenon and identifying specific features of the phenomenon in this location. Through a range of case studies it is possible for the research community to map out both common features and distinguishing features of a given social process.

What is the upshot of these observations? It is that good historical writing needs to be attentive to difference -- difference across national settings, across social groups, across time; that it should be grounded in many theories of how social processes work, but wedded to none; and that it should pay close attention to the evolution of the social arrangements (institutions) through which individuals conduct their social lives. I hope these remarks also help to make the case that philosophers can be helpful contributors to the work that historians do, by assisting in teasing out some of the conceptual and philosophical issues that they inevitably must confront as they do their work.

Contingency and explanation

Social change and historical events are highly contingent processes, in a specific sense: they are the result of multiple causal influences that "could have been otherwise" and that have conjoined at a particular point in time in bringing about an event of interest. Contrast this situation with what we are looking for when we seek an explanation of a change or event. When we explain an event, we show how and why it was not random or accidental; we identify a set of circumstances that made it necessary or likely in the given circumstances. Contingency and explanation therefore seem to be in tension with each other: a wholly contingent world is perhaps one in which explanation of particular occurrences is impossible.

The appearance of contradiction lessens when we realize that "contingent" is not the same as "random" or "uncaused". (See an earlier post for an effort to disentangle a number of related causal concepts; link.) When a uranium atom decays at a particular moment, this is a truly random event. There is no underlying cause that brought about the decay of the nucleus at this particular moment. When a race riot occurs in in Detroit on July 23, 1967, this was a contingent occurrence -- it did not have to happen; but it was not random, spontaneous, or uncaused. Rather, there were multiple causal factors and processes, along with a number of accidental and spontaneous events, leading to a pathway of social actions that resulted in largescale confrontation, arson, violence. Here is how the Kerner Commission described the occurrence of major race riots in the United States (link):

Disorder did not erupt as a result of a single "triggering" or "precipitating" incident. Instead, it was generated out of an increasingly disturbed social atmosphere, in which typically a series of tension-heightening incidents over a period of weeks or months became linked in the minds of many in the Negro community with a reservoir of underlying grievances. At some point in the mounting tension, a further incident--in itself often routine or trivial--became the breaking point and the tension spilled over into violence.

We can understand this account as depending on a distinction between proximate and distal causes; distal causes (a pattern of police brutality, say) set the stage for racial tension, which makes an outbreak of violence more likely; and a precipitating (proximate) event triggers the outburst. The point in this paragraph is that the triggering cause is not the sole cause, or even the most important cause. But all these factors are causally relevant to the outcome. We say that the riot was contingent because there are many ways in which the tensions created by the background conditions could have been defused -- a progressive mayor could have enacted a police reform along with a jobs program, a charismatic leader like Dr. King could have emerged in Detroit who helped to channel tension into electoral politics rather than an outbreak of violence, the Federal government could have been more successful in its civil rights reforms and its War on Poverty. Or the raid on the blind pig could have happened in a driving rainstorm, with the result that no crowd gathered. So the outcome was not preordained. It was contingent, but it was caused.

So it is not the case that a contingent world is one in which nothing can be explained. A chaotic and random world has that property; but contingency is not chaos. Rather, for many historical and social events we can identify a set of background or standing conditions that elevated the probability of the event, we can sometimes identify independent causal processes underway at the same time that interact to further elevate the probability of the event; and we can identify one or more unrelated and random events that served as a trigger to the occurrence of the event of interest.

This is one reason why the strategy of seeking out causal mechanisms in the social world is an appealing approach to social explanation. Appeal to causal mechanisms allows us to make sense of both important features of the social world: that processes and events are contingent, and that many processes and events are amenable to causal explanation.

When researchers set their goals on identifying general causes for groups of social phenomena, they often have in mind the idea that there are similarities in the background standing causal conditions that serve to increase the likelihood of a certain kind of event -- revolution, riot, economic crisis, or period of rapid innovation. And indeed, there are credible hypotheses about such conditions; this is the underlying rationale for the application of Mill's methods to causal reasoning in the social sciences. It is indeed perfectly credible that there are pervasive social conditions that make certain kinds of social events more likely -- a good university system and rapid technological innovation, a defeat in war and political turmoil, the pervasiveness of Protestantism and the hockey stick of market activity.

This insight is closely related to the distinction that Bhaskar and critical realists draw between closed and open systems. In an open system we cannot predict future states of the system because we cannot achieve causal closure; there is always the possibility of another kind of causal influence or mechanism that can offset the workings of the known mechanisms.

The page from the Washington Times above draws attention to an event that was itself highly contingent and yet explicable (the bungled but eventually successful effort to assassinate Archduke Ferdinand); leading to an important historical event (the outbreak of World War I) which was also both contingent and explicable.

Complexity and contingency

One of the more intriguing currents of social science research today is the field of complexity theory. Scientists like John Holland (Complexity: A Very Short Introduction), John Miller and Scott Page (Complex Adaptive Systems: An Introduction to Computational Models of Social Life), and Joshua Epstein (Generative Social Science: Studies in Agent-Based Computational Modeling) make bold and interesting claims about how social processes embody the intricate interconnectedness of complex systems.

John Holland describes some of the features of behavior of complex systems in these terms in Complexity:

• self-organization into patterns, as occurs with flocks of birds or schools of fish  
• chaotic behaviour where small changes in initial conditions (‘ the flapping of a butterfly’s wings in Argentina’) produce large later changes (‘ a hurricane in the Caribbean’)  
• ‘fat-tailed’ behaviour, where rare events (e.g. mass extinctions and market crashes) occur much more often than would be predicted by a normal (bell-curve) distribution  
• adaptive interaction, where interacting agents (as in markets or the Prisoner’s Dilemma) modify their strategies in diverse ways as experience accumulates. (p. 5)

In CAS the elements are adaptive agents, so the elements themselves change as the agents adapt. The analysis of such systems becomes much more difficult. In particular, the changing interactions between adaptive agents are not simply additive. This non-linearity rules out the direct use of PDEs in most cases (most of the well-developed parts of mathematics, including the theory of PDEs, are based on assumptions of additivity). (p. 11)

Miller and Page put the point this way:

One of the most powerful tools arising from complex systems research is a set of computational techniques that allow a much wider range of models to be explored. With these tools, any number of heterogeneous agents can interact in a dynamic environment subject to the limits of time and space. Having the ability to investigate new theoretical worlds obviously does not imply any kind of scientific necessity or validity— these must be earned by carefully considering the ability of the new models to help us understand and predict the questions that we hold most dear. (Complex Adaptive Systems, kl 199)

Much of the focus of complex systems is on how systems of interacting agents can lead to emergent phenomena. Unfortunately, emergence is one of those complex systems ideas that exists in a well-trodden, but relatively untracked, bog of discussion. The usual notion put forth underlying emergence is that individual, localized behavior aggregates into global behavior that is, in some sense, disconnected from its origins. Such a disconnection implies that, within limits, the details of the local behavior do not matter to the aggregate outcome. Clearly such notions are important when considering the decentralized systems that are key to the study of complex systems. Here we discuss emergence from both an intuitive and a theoretical perspective. (Complex Adaptive Systems, kl 832)

As discussed previously, we have access to some useful “emergence” theorems for systems that display disorganized complexity. However, to fully understand emergence, we need to go beyond these disorganized systems with their interrelated, helter-skelter agents and begin to develop theories for those systems that entail organized complexity. Under organized complexity, the relationships among the agents are such that through various feedbacks and structural contingencies, agent variations no longer cancel one another out but, rather, become reinforcing. In such a world, we leave the realm of the Law of Large Numbers and instead embark down paths unknown. While we have ample evidence, both empirical and experimental, that under organized complexity, systems can exhibit aggregate properties that are not directly tied to agent details, a sound theoretical foothold from which to leverage this observation is only now being constructed. (Complex Adaptive Systems, kl 987)

And here is Joshua Epstein's description of what he calls "generative social science":

The agent-based computational model— or artificial society— is a new scientific instrument. 1 It can powerfully advance a distinctive approach to social science, one for which the term “generative” seems appropriate. I will discuss this term more fully below, but in a strong form, the central idea is this: To the generativist, explaining the emergence2 of macroscopic societal regularities, such as norms or price equilibria, requires that one answer the following question:  

The Generativist's Question 

*     How could the decentralized local interactions of heterogeneous autonomous agents generate the given regularity?  

The agent-based computational model is well-suited to the study of this question since the following features are characteristics. (5)

Here Epstein refers to the characteristics of heterogeneity of actors, autonomy, explicit space, local interactions, and bounded rationality. And he believes that it is both possible and mandatory to show how higher-level social characteristics emerge from the rule-governed interactions of the agents at a lower level.

There are differences across these approaches. But generally these authors bring together two rather different ideas -- the curious unpredictability of even fairly small interconnected systems familiar from chaos theory, and the idea that there are simple higher level patterns that can be discovered and explained based on the turbulent behavior of the constituents. And they believe that it is possible to construct simulation models that allow us to trace out the interactions and complexities that constitute social systems.

So does complexity science create a basis for a general theory of society? And does it provide a basis for understanding the features of contingency, heterogeneity, and plasticity that I have emphasized throughout? I think these questions eventually lead to "no" on both counts.

Start with the fact of social contingency. Complexity models often give rise to remarkable and unexpected outcomes and patterns. Does this mean that complexity science demonstrates the origin of contingency in social outcomes? By no means; in fact, the opposite is true. The outcomes demonstrated by complexity models are in fact no more than computational derivations of the consequences of the premises of these models. So the surprises created by complex systems models only appear contingent; in fact they are generated by the properties of the constituents. So the surprises produced by complexity science are simulacra of contingency, not the real thing.

Second, what about heterogeneity? Does complexity science illustrate or explain the heterogeneity of social things? Not particularly. The heterogeneity of social things -- organizations, value systems, technical practices -- does not derive from complex system effects; it derives from the fact of individual actor interventions and contingent exogenous influences.

Finally, consider the feature of plasticity -- the fact that social entities can "morph" over time into substantially different structures and functions. Does complexity theory explain the feature of social plasticity? It does not. This is simply another consequence of the substrate of the social world itself: the fact that social structures and forces are constituted by the actors that make them up. This is not a systems characteristic, but rather a reflection of the looseness of social interaction. The linkages within a social system are weak and fragile, and the resulting structures can take many forms, and are subject to change over time.

The tools of simulation and modeling that complexity theorists are in the process of developing are valuable contributions, and they need to be included in the toolbox. However, they do not constitute the basis of a complete and comprehensive methodology for understanding society. Moreover, there are important examples of social phenomena that are not at all amenable to treatment with these tools.

This leads to a fairly obvious conclusion, and one that I believe complexity theorists would accept: that complexity theories and the models they have given rise to are a valuable contribution; but they are only a partial answer to the question, how does the social world work?

Path dependency in formation of academic disciplines

The topic of the historicity of academic disciplines has come up numerous times in this forum. It is a conviction of mine that disciplines demonstrate a great deal of path dependency over time in their evolution. We can think of a discipline as being constituted at a time by some or all of these elements:

• a definition of important questions for research
• a definition of appropriate methods of research and analysis
• a model of explanation in the field
• some key examples of what theories and hypotheses ought to look like
• institutions for supporting, organizing, and directing research efforts
• institutions for validating and disseminating research findings
• institutions for training young researchers in the key elements of the discipline

This sounds a lot like Kuhn's idea of a paradigm, Lakatos's idea of a research community, or the definitions of scientific enterprise offered by historians and sociologists of science and researchers in the tradition of STS studies (link). An academic discipline is an assemblage of ideas, networks of individuals, institutions, and locations (libraries, laboratories, research institutes).

If this is a reasonable approximation to the social reality of an academic discipline, what does it suggest about contingency and path-dependency in the development of the discipline? For one thing, it suggests multiple sources of contingency both internal to the intellectual enterprise and external to it. Internally, a discipline like philosophy or a sub-discipline like the philosophy of mind is driven in part by a somewhat logical process of attack on existing problems -- what Kuhn referred to as "normal science", and partly by large, compelling breakthroughs by individuals or small groups (for example, the Vienna Circle). Externally, it is straightforward to identify political and institutional influences that shape the research agenda at various times in various disciplines -- the preference for positivism in sociology that was advanced by considerations of the Cold War, for example. And within the institutional setting of the disciplines there are contingencies as well -- for example, a strong editor of a leading journal or research laboratory can set the agenda for theory and methodology in a discipline for a generation. (Andrew Abbott describes this kind of influence in Department and Discipline: Chicago Sociology at One Hundred.)

Almost every element in this list is itself visibly dependent on historical circumstances in multiple ways. Take the issue of defining the important questions for research. There are political and governmental influences on the definition of research problems -- witness the influence of the Cold War on the development of the social sciences, the role that is played by governmental funding agencies like the NSF or NIH, and the occasional intrusion of political pressure into scientific fields like environmental science and sociology.

Within the community of individuals currently pursuing the discipline or proto-discipline there is a range of levels of talent and innovation, on the one hand, and prestige and influence, on the other. (The two categories don't necessarily correlate perfectly.) One charismatic individual or local group (Wittgenstein, say) may exert influence over the direction of a sub-field through charisma and the power of his or her ideas. Another may exert influence over the strategic placement he or she occupies in the institutions of influence -- major graduate schools or prominent journals, for example. And in each case, the discipline moves to a new phase with new questions and ideas.

Bourdieu's theoretical construct of the field (link) is very relevant to these forms of influence on the development of a given academic discipline. By locating various individuals within the network of institutions, scholars, and funding sources it is possible to attempt to piece together the ways in which their own research agendas unfolded (responding to incentives created by their field) and the influence they exerted on other scholars. Neil Gross's sociological biography of Richard Rorty illustrates this kind of analysis (link), as does much of George Steinmetz's research on the development of sociology as a discipline in France, Germany, and the US.

What all of this seems to support is the idea that the academic disciplines are in fact highly contingent in their development, and that there is no reason to expect convergence around a single "best" version of the discipline. The history of disciplines should better be understood in analogy to the brachiation and differentiation associated with the evolution of species and sub-species over time -- lots of contingency, with a consequent specialization of the intermediate results to the demands of a particular point in time. This implies that a discipline like sociology or political science could have developed very differently, with substantially different ideas about research questions and methods. And this seems to be true for similar reasons in the humanities as well as the natural sciences and mathematics. Finally, this suggests that there is no end-point -- no "universal sociology," no "final philosophy," no "complete mathematics." Instead, every discipline in its search for knowledge and new ideas is charting new intellectual space.

Social contingency?

Image: Mason-Pfizer monkey virus (link)

Image: organization chart of General Motors

What does it mean to say that the social world is contingent? Several things. First, it means that social changes and patterns are not strongly law governed. Outcomes are the result of intersecting chains of causal mechanisms and stochastic happenings, so there is no sense in which outcomes are predetermined or confidently predictable. Social outcomes are the result of conjunctural causation, with indeterminate conjunctions of causal processes and conditions proceeding from independent background circumstances. And accidents and random events make a difference in the outcomes as well. This is true at a full range of scales, from large happenings like the outbreak of war to the growth of a corporation to the emergence of a new set of values about gay marriage. So historical processes and sequences are contingent, and we need to pay close attention to the path dependency of social happenings.

Another key kind of contingency has to do with the composition of social entities. In the natural world there are some formations that are necessary. H2O and protein molecules have a specific topology and arrangement that follows strictly from the physical properties of the constituents, and these properties, we would like to assert, are fixed by nature. So it is a necessary fact that H2O molecules all have the same topology -- this topology follows from physical laws. But likewise, large proteins have only a small number of stable geometries as well, given the physical characteristics of the atoms that compose them.

The situation is different for social compounds. They are composed of individuals. But their properties are not fixed by the laws of psychology or any other consistent realm. Rather, there is substantial path dependency in the formation of a particular social formation, and the properties of actual social formations are contingent relative to the properties of the individuals who constitute it.

To say that social phenomena are contingent is not to imply that they are random or unpatterned. In fact, a large part of the task of the social sciences is to identify and explain important social patterns -- for example, regularities of urbanization and habitation. G. William Skinner found that the cities, towns, and hamlets of Sichuan conformed to a pattern of nested hexagons (link); he offered the mechanisms associated with central place theory as the basis for an explanation of this fact. The combined workings of transportation cost and cost-sensitive individual decision-makers imply the hexagonal geometry that Skinner discovers. But there is vast contingency embedded within this account -- reasons why certain locations may be avoided or reasons why a given center may come to have higher-level commercial or military functions than would have been expected, for example. So the regularities that we observe can be explained by the workings of several social mechanisms that favor habitation choices; while extraneous factors can disrupt or distort the pattern that would be normally expected.

So there are conditions and influences that often create identifiable patterns of social activity. This is the chief reason why the study of social mechanisms is so fruitful in the social sciences: there is an open-ended plurality of causal mechanisms at work in the social space. These can be investigated and understood. And we can then use our ability to identify the workings of social mechanisms to provide explanations of both singular occurrences and intriguing social patterns. But at the same time, we are forced to recognize that particular social processes -- economic development, urbanization, political crisis, ethnic conflict, or changes in values systems in a population, for example -- are driven by multiple sub-processes that are themselves contingent, and that interact in contingent ways.

The evolution of species as described by classical Darwinian theory is a good example of the some of aspects of contingency that I believe are characteristic of social developments as well. The large pressures within a given ecological environment are those that affect reproduction and longevity. Variations occur within the genetic information constituting organisms at a certain time, and natural selection favors the proliferation of some of those variations into the population as a whole. But the longterm evolution of the X group of organisms is not pre-determined; X's may invest in better vision, better mobility, greater lethality as predators, greater ability to conceal from predators, or dozens of other possible lines of evolutionary change. So all we can predict is that the assortment of groups of organisms will evolve towards higher levels of reproductive fitness or will disappear; and we can explain, in hindsight, the emergence of some of the physiological characteristics of X's in terms of the reproductive advantage that this feature confers on the organism. So there is nothing in the antecedent habitat that preordains that giraffes will have long necks.

There is an important analogy here with social change. We can identify some of the features that influence the development of organizations and political institutions in a variety of historical settings: the need for states to extract revenues and to exert coercive power, for example. But we cannot predict with confidence what form those adaptations will take. So the theatre state of Bali looks very different from the feudal monarchy in France, even though both states succeed in the central functions of states.

Large predictions in history

To what extent is it possible to predict the course of large-scale history -- the rise and fall of empires, the occurrence of revolution, the crises of capitalism, or the ultimate failure of twentieth-century Communism? One possible basis for predictions is the availability of theories of underlying processes. To arrive at a supportable prediction about a state of affairs, we might possess a theory of the dynamics of the situation, the mechanisms and processes that interact to bring about subsequent states, and we might be able to model the future effects of those mechanisms and processes. A biologist's projection of the spread of a disease through an isolated population of birds is an example. Or, second, predictions might derive from the discovery of robust trends of change in a given system, along with an argument about how these trends will aggregate in the future. For example, we might observe that the population density is rising in water-poor southern Utah, and we might predict that there will be severe water shortages in the region in a few decades. However, neither approach is promising when it comes to large historical change.

One issue needs to be addressed early on: the issue of determinate versus probabilistic predictions. A determinate prediction is one for which we have some basis for thinking that the outcome is necessary or inevitable: if you put the Volvo in the five million pound laboratory press, it will crush. This isn't a philosophically demanding concept of inevitability; it is simply a reflection of the fact that the Volvo has a known physical structure; it has an approximately known crushing value; and this value is orders of magnitude lower than five million pounds. So it is a practical impossibility that the Volvo will survive uncrushed. A probabilistic prediction, on the other hand, identifies a range of possible outcomes and assigns approximate probabilities to each outcome. Sticking with our test press example -- we might subject a steel bridge cable rated at 90,000 pounds of stress to a force of 120,000. We might predict that there is a probability of failure of the cable (40%) and non-failure (60%); the probability of failure rises as the level of stress is increased. But there is a range of values where the probabilities of the two possible outcomes are each meaningfully high, while there are extreme values where one option or the other is impossible.

In general, I believe that large-scale predictions about the course of history are highly questionable. There are several important reasons for this.

One reason for the failure of large-scale predictions about social systems is the complexity of causal influences and interactions within the domain of social causation. We may be confident that X causes Z when it occurs in isolated circumstances. But it may be that when U, V, and W are present, the effect of X is unpredictable, because of the complex interactions and causal dynamics of these other influences. This is one of the central findings of complexity studies -- the unpredictability of the interactions of multiple causal powers whose effects are non-linear.

Another difficulty -- or perhaps a different aspect of the same difficulty -- is the typical fact of path dependency of social processes. Outcomes are importantly influenced by the particulars of the initial conditions, so simply having a good idea of the forces and influences the system will experience over time does not tell us where it will wind up.

Third, social processes are sensitive to occurrences that are singular and idiosyncratic and not themselves governed by systemic properties. If the winter of 1812 had not been exceptionally cold, perhaps Napoleon's march on Moscow might have succeeded, and the future political course of Europe might have been substantially different. But variations in the weather are not themselves systemically explicable -- or at least not within the parameters of the social sciences.

Fourth, social events and outcomes are influenced by the actions of purposive actors. So it is possible for a social group to undertake actions that avert the outcomes that are otherwise predicted. Take climate change and rising ocean levels as an example. We may be able to predict a substantial rise in ocean levels in the next fifty years, rendering existing coastal cities largely uninhabitable. But what should we predict as a consequence of this fact? Societies may pursue different strategies for evading the bad consequencs of these climate changes -- retreat, massive water control projects, efforts at atmospheric engineering to reverse warming. And the social consequences of each of these strategies are widely different. So the acknowledged fact of global warming and rising ocean levels does not allow clear predictions about social development.

For these and other reasons, it is difficult to have any substantial confidence in predictions of the large course of change that a society, cluster of institutions, or population will experience. And this is a reason in turn to be skeptical about the spate of recent books about the planet's future. One such example is Martin Jacques' provocative book about China's future dominance of the globe, When China Rules the World: The End of the Western World and the Birth of a New Global Order: Second Edition. The Economist paraphrases his central claims this way (link):

He begins by citing the latest study by Goldman Sachs, which projects that China's economy will be bigger than America's by 2027, and nearly twice as large by 2050 (though individual Chinese will still be poorer than Americans). Economic power being the foundation of the political, military and cultural kind, Mr Jacques describes a world under a Pax Sinica. The renminbi will displace the dollar as the world's reserve currency; Shanghai will overshadow New York and London as the centre of finance; European countries will become quaint relics of a glorious past, rather like Athens and Rome today; global citizens will use Mandarin as much as, if not more than, English; the thoughts of Confucius will become as familiar as those of Plato; and so on.

This is certainly one possible future. But it is only one of many scenarios through which China's future may evolve, and it overlooks the many contingencies and strategies that may lead to very different outcomes.

(I go into more detail on this question in "Explaining Large-Scale Historical Change"; link.)

Greenblatt on civilization

Steven Greenblatt's recent book, The Swerve: How the World Became Modern, is an ambitious widening of Greenblatt's intellectual palette. The title of the book refers to one of the central ideas in the metaphysics of Epicurus: the idea that atoms sometimes deviate from their straight courses, permitting collisions. This fact, according to Epicurus, is the only possible source of freedom of the will. So swerving is what takes determinacy out of nature and human action.

That is why it is necessary to admit the same thing for the atoms, namely, that there is another cause of motion besides blows [from collisions] and weight, which is the source of our inborn capability [to act freely], since we see that nothing can come from nothing. For the weight of the atoms prevents it from being the case that everything happens as a result of the blows [of collisions], which are like an external force. But that the mind itself does not have an internal necessity in all its actions, and that it is not forced, as though in chains, to suffer and endure, that is what this tiny swerve of the atoms, occurring at no fixed time or place, accomplishes. (The Epicurus Reader: Selected Writings and Testimonia, kl 1610)

Greenblatt's book shows a great deal of regard for the philosophy of Epicurus and the voice of Lucretius. Lucretius's poem was a passionate follower's presentation of the philosophy of Epicurus. Lucretius aimed to present the materialism, empiricism, ethics, and atheism of Epicurus to Roman literati in the most convincing way possible, because he was himself a devoted follower of Epicurean philosophy. Greenblatt gives a good exposure to what some of those ideas and values were, and he believes that its rediscovery in the early fifteenth century had a major influence on the development of western culture from that point forward.

But this isn't primarily a treatment of Epicurus or Lucretius or philosophical ideas.  Instead, it is an account of the fragility and contingency of books and ideas in history.  Greenblatt relates the circumstances of the rediscovery of Lucretius's The Nature of Things by Poggio Bracciolini in the early fifteenth century. It was Poggio whose obsession with ancient words and books prevented the permanent loss of this work. Poggio isn't romanticized. But it was his passionate commitment to Latin rhetoric, grammar, and thought, and his conviction that the current times were a corrupt deviation from the ideals of the ancient world, that led him to his pursuit of forgotten manuscripts throughout Europe.

Greenblatt paints a vivid picture of the intellectual environment of educated secular men like Poggio in the circumstances of early-Renaissance Rome and Florence and the complicated windings of the Catholic Church within that world. This is fascinating and absorbing all by itself, similar to Greenblatt's work in Will in the World: How Shakespeare Became Shakespeare.

But even more fascinating is a persistent subtext, unspoken but visible: human civilization itself took a disastrous "swerve" when Christianity took hold and the values, knowledge, and secular lives of the ancient world were all but extinguished. The humanism of Greek and Roman philosophers was eclipsed and largely forgotten, and a harsh and punitive religious orthodoxy took its place. The vile and bloody murder in 415 of Hypatia by a Christian mob in Alexandria captures well this transition (kindle loc 1435). Books, independence of thought, the love of learning for its own sake -- these values so prominent in the ancient world were suppressed and forgotten.

Nothing more emblemizes this swerve than the clarity of Lucretius and the near extinction of his thought in the emerging world of Europe after the dark ages. The rediscovery of the Lucretius book permitted the reintroduction of a revolutionary set of philosophical ideas into the Renaissance world that had enormous and emancipating effect. And yet it was a near thing: without Poggio's determination to rescue ancient books, this manuscript might well have eventually have turned to dust.

What seems particularly interesting about this episode is the force it gives to the idea that every aspect of human history is contingent. There is no teleology; there is no tendency towards better and better systems and values; there is often not even a cumulativeness of human achievement; there is only the helter-skelter randomness of key events that push developments in this direction or that.

Here is a good article by David Konstan on Epicurus in the Stanford Encyclopedia of Philosophy; link. There are affordable Kindle editions available for the writings of Epicurus and Lucretius, as well as Greenblatt's own book.