Paired comparisons

Sidney Tarrow is a gifted and prolific student of comparative politics. (Listen to my interview with Professor Tarrow.) He has spent much of his career trying to understand social movements, contentious politics, and the causes of differences in political behavior across national settings. And one of his special contributions is his ability to think clearly about the methods that social scientists use.

Tarrow attaches a lot of weight to the idea of "paired comparisons" as a method of research and discovery: Locate a few cases that are broadly similar in many respects but different in a way that is important, interesting, or surprising. Then examine the cases in greater detail to attempt to discover what explains the difference between the two cases. (One of his early books that employs this method is From center to periphery: Alternative models of national-local policy impact and an application to France and Italy.)

Nothing special turns on "pairs" here; what Tarrow is describing is really the logic of small-N comparative research. The point about the broad similarity that is the basis for choosing the cases follows from the logic of causation: if we presuppose that the outcome P is caused by some set of antecedent social and political conditions and we know that C1 and C2 have different outcomes -- then the more factors we can "control" for by finding cases in which these factors are constant, the better. This is so, because it demonstrates that none of the constant factors in the two cases are the cause of variation in outcome. And this limits our investigation of possible causes to the factors in which the cases differ.

If this sounds like Mill's methods of similarity and difference, that's appropriate -- the logic is the same, so far as I can see. Here is Mill's method of difference:

A B C D -E => P
A B -C D -E => -P

And in this case -- making the heroic assumption that A,B,C,D,E exhaust all possible causes of P, and that the cause of P is deterministic rather than probabilistic -- then we can infer that the presence of C causes P.

This reasoning doesn't take us to a valid conclusion to the effect that C is the only factor that is causally relevant to the occurrence of P; it is possible, for example, that there is a third case along these lines:

-A B -C D -E => -P

This would demonstrate that A is a necessary condition for the occurrence of P; withhold A and P disappears. And each of the other factors might also play a role as a necessary condition. So it would be necessary to observe as many as 32 cases (2^5) in order to sort out the status of A through E as either necessary or sufficient conditions for the occurrence of P. (The logic of this kind of causal reasoning is explored more fully in my essay, "An Experiment in Causal Reasoning," which is also published in Microfoundations, Methods, and Causation.)

But I don't think that Tarrow is intending to advance the method of paired comparison as a formal method of causal inference, along the lines of inductive or deductive logic. Instead, I think he is making the plausible point that this method should be understood as a part of an intelligent research strategy. Social processes are complex. We are interested in explaining variation across cases. And we probably have the best likelihood of discovering important causal relationships if we can reduce the number of moving parts (the other kinds of variation that occur across the cases).

Tarrow gives an example of the application of the method of paired comparisions in the context of his early study of the political fortunes of the Italian Communist Party (PCI) in the south of Italy. In this case the paired comparison involves northern Italy and southern Italy. Both are subject to the same national political structures; both populations speak Italian; both populations have an Italian national identity. However, the PCI was fairly successful in mobilizing support and winning elections based on a militant political program in the north, and was persistently unsuccessful in doing these things in the south. What explains the difference?

As Tarrow explains his reasoning, his expectation in conducting the research was a "structural" one. He expected that there would be large structural factors in post-war Italy -- features of economic and political institutions -- that would explain the difference in outcome for PCI political activism. And there were indeed large structural differences in social organization in the two regions. Northern Italy possessed an economy in which industrial labor played a key role and constituted a substantial part of the population. Southern Italy was agrarian and backward, with a large percentage of exploited peasants and only a small percentage of industrial workers.

But, very significantly, Tarrow now believes that these "structural" expectations are probably too "macro" to serve as the basis of social explanation. Instead, he favors the importance of looking at the dynamics of social processes and the specific causal mechanisms that can be discovered in particular social-historical settings. This means looking for causal factors that work at a more strategic and meso level. In terms of the southern Italian PCI outcome that he was interested in explaining thirty years ago -- he now believes that the causal mechanism of "brokerage" would have shed much more light on the political outcomes that were of interest in Italy. (This is the heart of the approach that he takes, along with Doug McAdam and Chuck Tilly, in Dynamics of Contention.)

This finding doesn't invalidate the heuristic of paired comparisons. But it probably does invalidate the expectation that we might discover large "structure-structure" patterns of causation through such comparisons. Instead, what the method facilitates is a more focused research effort on the part of the comparativist, in the context of which he/she can search out the lower-level causal mechanisms and processes that are at work in the various settings under study.

Is network analysis inconsistent with agent-centered explanation?

Quite a few researchers who study dynamic social processes are making use of some of the tools of network analysis. And it is sometimes maintained that this approach is inconsistent with an agent-centered approach to social processes. Some of these researchers take the view that "it's not what is in the heads of various actors, but rather their relationships in networks that provide the causal underpinnings of social change." And they sometimes maintain that the actor's psychological states can't even be identified in isolation from his/her social relationships. So, once again, explanation cannot rest upon facts about individuals alone. And this sort of finding is thought to cast doubt on methodological individualism in particular, and agent-centered explanatory strategies more generally. (Chuck Tilly and co-authors sometimes take a view along these lines; for example, Doug McAdam, Sidney Tarrow and Charles Tilly, Dynamics of Contention.)

There is something right about the intuition that we can't ground social explanations on assumptions that are too narrowly confined to features of individual psychology. Individuals are socially constucted and socially developed, and our explanations of social processes need
to reflect this fact. This is why I prefer the phrase "methodological localism" to "methodological individualism." But both ontologies are agent-centered. So the question remains: does the causal salience of social networks demonstrate that agent-centered accounts are inherently incomplete -- or even worse, inherently unworkable (because we can't even specify the individual agent's powers and motives independently of his/her networks)?

I don't think so, for several reasons. First, what is a network but a set of socially constructed agents in concrete relations with each other -- communication, coordination, power, subordination, and recognition? The facts about the network are exhausted by a description of the social beliefs of the relevant actors and their material relations to each other.

Second, it is certainly true that an agent's possibilities for exercising power are a function of facts beyond his/her own psychological characteristics. So Albert, the peasant activist in the tiny Breton village, is much more empowered than his psychological twin across the border in Normandy, by the fact that he alone has strong relationships with leaders in both the Catholic Church and the wine-growers' guild. His social networks permit him to amplify the scope of action and effect he may attempt. What this means is that Albert's social networks are a causal component in his ability to wield influence. In this sense it is reasonable as well to attribute causal status to the network and to characterize this standing as being independent of Albert as an individual.

But it remains true that all of the causal powers associated with the network depend on the states of agency of the many persons who make it up. We therefore need to be able to provide an agent-centered account of the network's causal powers, distributed over the many agents who make it up. We must have "microfoundations" for the claim that the network exercises social influence. If the actors who constitute nodes within the network didn't have the right mental frameworks, motivational dispositions, or bodies of knowledge, then they would not in fact behave in a way that was sustaining of the network's social-causal properties.

So, it seems inescapable that, when we say that "Albert's power as a peasant activist depends upon the social fact that he is part of such-and-so networks" -- that we have only uncovered another field of research where more agent-centered research is needed. The network's social-causal properties must themselves disaggregate onto a set of facts about the agents who constitute the network. The current causal properties of the network and the agents who make it up are the complex and iterative result of many inter-related actions and alliances of prior generations of agents.

And this in turn demonstrates that network analysis is by no means inconsistent with an agent-centered approach to social explanation.

(See "Levels of the social" for more on this subject.)

The flea market analogy

Is the flea market a helpful analogy for understanding the social world ("The Dis-unity of Science")? Does it serve to provide a different mental model in terms of which to consider the nature of social phenomena?

What it has going for it is heterogeneity and contingency, and an obvious share of agent-dependency. The people who show up on a given Saturday are a contingent and largely disorganized mix of humanity. And the products that wind up on the jumble tables too are highly disorderly and random. Each has its own unique story for how it got there. There is no overall guiding design.

But there is also a degree of order underlying the apparent chaos of the jumble tables. All is not random in a flea market. The participants, for example: there are regular vendors, street people, police officers, health inspectors, jugglers, and pickpockets -- as well as regular shoppers, tourists, school children, and occasional shoppers looking for a used toaster or a single kitchen chair. In most cases there are reasons they are there -- and the reasons are socially interesting. Moreover, the ethnographer of the flea market is likely enough to spot some seasonal or social patterns in the products and people present in a certain month or time of year. So -- a blend of chaos and order.

But the order that can be discerned is the result of a large number of overlapping, independent conditions and processes -- not the manifestation of a few simple forces or a guiding system of laws.

Both accident and order are characteristic of the larger social world as well. The helter-skelter of the flea market is in fact highly analogous to many aspects of social phenomena -- army recruitment, incidents of crime, mortgage defaults. But it is also true that there are other social phenomena that aren't so accidental. So the jumble sale is perhaps less good as an analogy for highly organized and managed social processes -- a tight administrative hierarchy, an orchestrated campaign event, or a coordinated attack in battle.

This addresses the "accidental conjunction" part of the analogy. What about the "composite order" part of the analogy? This element too works pretty well for many examples of social phenomena. When students of the professions discover that there are interesting patterns of recruitment into accountancy or the officer corps, or discover that there are similarities in the organizations of pharmacists and psychotherapists -- they also recognize that these patterns result from complex, intertwined patterns of strategic positioning, organizational learning, and economic circumstances. In other words, the patterns and regularities are themselves the result of multiple social mechanisms, motives, and processes. And these processes are in no way analogous to laws of nature.

So, all considered, the analogy of the flea market works pretty well as a mental model for what we should expect of social phenomena: a degree of accident and conjunction, a degree of emerging pattern and order that results from many independent but converging social processes, and an inescapable dimension of agent-dependency that refutes any hope of discovering an underlying, law-governed system.

The "dis"-unity of social science

One of the central goals of Vienna Circle philosophy of science was the idea of the unity of science. The idea included at least two separable parts: methodological unity and unity of content under a single system of laws. On the methodological side there was the idea that the logic of explanation and confirmation should be the same in all the empirical sciences. If there were to be differences across disciplines, these should be heuristic rather than epistemic differences. (Jordi Cat provides an extensive discussion of the unity of science doctrine in an article in the Stanford Encyclopedia of Philosophy.)

The more basic goal for unity was the idea of a single comprehensive theory that would, in principle, provide the foundation for the theories of all the special sciences. Physics was the intended foundation, and the goal was to demonstrate that all the fields of the natural sciences could be derived in principle from the laws of physics. For example, the hope was that the properties and laws of chemical elements and molecules should be derivable from fundamental physics.

The reasons for wanting to see a unified physical theory were a preference for parsimony and simplicity and a metaphysical conviction that all of nature must really derive from a single set of fundamental laws.

The fate of the unity of science doctrine can be pursued elsewhere. Here the question is whether there is a similar aspiration for the social sciences. The parallel principle could be stated along these lines: there should be some set of basic facts about individuals in social interactions that is sufficient to permit one to derive all varieties of social behavior, given relevant knowledge about context and boundary conditions.

The attractions of such a unified social science are the same as in the natural science case: parsimony, simplicity, and comprehensiveness. And, in fact, unifying theories for social explanation are sometimes advanced. The most thorough-going attempt is the effort by rational choice theory and microeconomics to unify all social action as the consequence of preference-maximizing individual rationality within constraints.

Attractive as this effort might be from an abstract or aesthetic perspective, it is profoundly misguided when if comes to understanding society. Social phenomena are not the law-governed consequences of a few underlying facts and features of individuals. Rather, they are the contingent and mixed results of an inherently heterogeneous set of motives, psychologies, and institutions. The fundamental problem is that the social world is not a system at all in the natural science sense. Instead, it is the contingent and dynamic sum of a variety of shifting processes and contexts.

A better metaphor for the social world -- better than the metaphor of a table of billiard balls governed by the laws of mechanics -- is a large urban flea market. The wares on sale on a particular Saturday are simply the sum of the accidents of circumstance that led a collection of sellers to converge on that particular day. There are some interesting regularities that emerge over time -- in the spring one finds more used lawnmowers and in times of dearth one finds more family treasures. These regularities require explanation. But they do not derive from some governing "law of flea markets" that might be discovered. Instead, the flea market and the larger society are, alike, simply the aggregate result of large numbers of actions, motives, circumstances, and structures that turn kaleidoscopically and produce patterned but non-lawlike outcomes.

So where does this take us with regard to "unified social science"? It leads us to expect something else entirely: rather than unity, we should expect eclectic theories, piecemeal explanations, and a patchwork of inquiries at a range of levels of description. Some explanatory theories will turn out to be more portable than others. But none will be comprehensive, and the social sciences will always remain open-ended and extensible. Instead of theoretical unification we might rather look for a more and more satisfactory coverage, through a range of disciplines and methods, of the aspects of the social world we judge most interesting and important. And these judgments can be trusted to shift over time. And this means that we should be skeptical about the appropriateness of the goal of creating a unified social science.

(See an earlier posting on "Coverage of the Social Sciences" for more relevant comments on this topic.)

Politics and science

In the idealized version of science, the enterprise of scientific knowledge discovery follows its own logic without extraneous non-cognitive or non-rational influences. But this is unrealistic. Science is a social activity, conditioned by institutions, governments, and other social forces. So it is worth considering how politics influences the course of science and how these influences affect the rationality or veridicality of the enterprise. Does the fact of political influence on science make science either less rational or less true?

There are at least two ways in which science can be affected by social or political factors: in terms of the formulation of research questions and priorities, and in terms of the content of the findings of scientific research. Pernicious examples of the second kind of influence are easy to find in the history of science. For example, Stalin's insistence on the correctness of Lysenko's adaptationist theory of species led Soviet biology into a biological science that was profoundly untrue: organisms do not evolve according to the processes or mechanisms attributed to them by Lysenkoism. So the political imperative from Stalin to the adoption of a particular scientific hypothesis led to erroneous science. It was also irrational science (because it depended on criteria of acceptance that were political rather than empirical).

This form of influence of politics on science is clearly anti-scientific and anti-rational. And, regrettably, we appear to have clear instances of this kind of substitution of political expediency for rational scientific judgment in the behavior of the current US administration, in the form of its efforts to control the content of scientific judgments about climate change (article).

So one form of political influence on science is clearly anti-scientific. When politicians substitute their wishes for the judgment of capable scientific researchers, it is inevitable that the result will be bad science. And, of course, bad science is a bad basis for future problem-solving.

But consider the other form of influence mentioned here: the setting of priorities for scientific research, especially through funding strategies. Is this kind of influence inherently inconsistent with the empirical and rational claims of science?

It is obvious that science is subject to this kind of influence. When the National Institutes of Health decides to give higher priority to one kind of cancer rather than another, or to diabetes research over Alzheimer's research, this national institution is setting the agenda for university researchers throughout the country. When the US government gave priority to space exploration research over atmospheric or oceanic research in the 1960s, it likewise gave encouragement to certain scientific disciplines and inhibition to others. And, predictably, there was more progress in the scope and depth of scientific knowledge in some disciplines than others, following the spending priorities. Science requires resources, and one of the duties of a democratic government is to decide about priorities in the expenditure of public moneys.

What this kind of influence does not do, is to dictate the content of the findings. Once the resources are committed, it is essential that the normal processes of science -- empirical study, peer review, experimentation, theory development -- should take place without interference from political or social pressure.

So we might say that the "priority-setting" influence of politics on science is benign from the point of view of scientific rationality. The political decision-makers decide what scientific problems are most important from the public's point of view; and the scientists, following the funding sources, then do their best to understand and solve those problems. But this means that one of our crucial political goals ought to be to create and defend institutions that assure the political independence of scientists, so their research findings are the result of empirical investigation and theory formation rather than pressured conformance to the state's expectations.

Now let's bring these ideas back to the social sciences. Social science research has been subject to both kinds of political influence in American history. There have sometimes been intense pressures on social scientists and historians about the content of their research -- for example, the field of China studies during the period of McCarthyism. When social scientists arrive at unpalatable truths, they are sometimes subjected to shameful political pressures. But second, the social sciences have certainly been shaped in the past forty years by the spending priorities of federal and non-profit funding agencies. This influence isn't necessarily bad, or anti-scientific. In fact, it is unavoidable. But, in the social sciences especially, it may have the insidious effect of pushing social science research away from some difficult or controversial topics; and this may be so, even when those topics turn out to be particularly important for arriving at a better understanding of where our society is going.

What was E P Thompson up to?

Let's think about E P Thompson. His 1963 book, Making of the English Working Class, transformed the way that historians on the left conceptualized "social class." But what, precisely, was it about?

Whereas other Marxist historians focused particularly on the large structures of capitalism, Thompson's eye was turned to the specific and often surprising details of artisanal and working culture in pre-industrial England, the many ways in which the working people at the bottom of English society conceived of themselves and created their own organizations for education and politics in the last half of the eighteenth century. Neither peasant nor middle class, the many segments of working people in England were socially organized by trade and skill, and with remarkably distinct cultural traditions, songs, and political repertoires. They were not, in fact, a "class". And yet, they became a class -- this is the "making" that Thompson's title refers to.

(Harvey Kaye's British Marxist Historians offers an excellent survey of the major British Marxist historians -- Hobsbawm, Hilton, Dobbs, Thompson, and others.)

Commentators often describe Thompson's central contribution as being the provision of a detailed understanding of "class consciousness" in counterpart to Marx's conception of a "class in itself" -- a group of people defined in terms of their relation to the system of property relations. On this line of interpretation, Thompson provided one of the missing links within Marxist theory, by demonstrating how the transition from "class in itself" to "class for itself" was accomplished.

This is too simplistic a reading of Thompson, however. For one thing, Thompson's book demonstrates the very great degree of contingency that attached to the historical construction of the English working class when we consider this process in cultural detail. But to find that the process is contingent, is also to negate the Marxist idea that there is a necessary and direct connection between a group's structural position in the property system and its social consciousness. For another and related reason, Thompson's story goes well beyond Marx's in its emphasis on the independent agency of English working people. Their organizations, their ideas, and their political strategies were not simply derivative of the structural situation of "labor and capital", but rather were the result of specific acts of leadership, creativity, and popular mobilization.

So let's consider the main elements of Thompson's historiography. What was his goal as a historian of this period of England's social history? In writing the book, Thompson took a huge step forward in creating the field of social history, and he established a paradigm of historical writing that guided a generation of historians. His goal is almost ethnographic: he wants to discover the many threads of thought and culture that passed through the many segments of English working people. He takes ideas and ideology very seriously -- and recognizes that the ideas of English Methodism and the rhetoric of liberty were profoundly important in these segments of English society. In particular, the ideas and the modes of organization that were associated with Methodism, were deeply formative for the laborers' and artisans' consciousness that was being forged.

Just as important as these elements of "high" culture, Thompson articulates his concept of the "moral economy" of the crowd -- the idea that there is a shared set of norms in popular culture that underlie social behavior. He identifies popular disturbance -- riots, strikes, and expressions of grievances of various kinds -- as a crucial indicator of political behavior and popular consciousness. And he tries to demonstrate that the popular disturbances of the eightheenth and nineteenth centuries were governed by a set of norms that were popularly observed and enforced -- about price, about social obligation, and about justice. The "bread riot" was not a chaotic or impulsive affair. And this becomes an important theme in the consciousness of the working class that Thompson describes: a consciousness that denounces political oppression as deeply as it decries exploitation.

In other words, Thompson's version of working class consciousness invokes liberty and justice as much as it does deprivation and material factors. "In the end, it is the political context as much as the steam-engine, which had most influence on the shaping consciousness and institutions of the working class" (197). "The people were subjected simultaneously to an intensification of two intolerable forms of relationship: those of economic exploitation and of political oppression" (198).

The culmination of this retelling of the multi-threaded histories of English working people is indeed "a working class consciousness" -- a more or less coherent social and political philosophy that supported a political program and a morality of equality and solidarity. "Thus working men formed a picture of the organization of society, out of their own experience and with the help of their hard-won and erratic education, which was above all a political picture. They learned to see their own lives as part of a general history of conflict between the loosely defined 'industrial classes' on the one hand, and the unreformed House of Commons on the other. From 1830 onwards a more clearly defined class concsiousness, in the customary Marxist sense, was maturing, in which working people were aware of continuing both old and new battles on their own" (712).

Thompson's book remains an innovative and pathbreaking classic -- and one that can continue to provide new ideas about how to understand society.

(See this post on ChangingSociety for more discussion of E. P. Thompson.)

Agent-based modeling as social explanation

Logical positivism favored a theory of scientific explanation that focused on subsumption under general laws. We explain an outcome by identifying one or more general laws, a set of boundary conditions, and a derivation of the outcome from these statements. A second and competing theory of scientific explanation can be called "causal realism." On this approach, we explain an outcome by identifying the causal processes and mechanisms that give rise to it. And we explain a pattern of outcomes by identifying common causal mechanisms that tend to produce outcomes of this sort in circumstances like these. (If we observe that patterns of reciprocity tend to break down as villages become towns, we may identify the causal mechanism at work as the erosion of the face-to-face relationships that are a necessary condition for reciprocity.)

But there are other approaches we might take to social explanation and prediction. And one particularly promising avenue of approach is "agent-based simulation." Here the basic idea is that we want to explain how a certain kind of social process unfolds. We can take our lead from the general insight that social processes depend on microfoundations at the level of socially situated individuals. Social outcomes are the aggregate result of intentional, strategic interactions among large numbers of agents. And we can attempt to implement a computer simulation that represents the decision-making processes and the structural constraints that characterize a large number of interacting agents.

Thomas Schelling's writings give the clearest exposition to the logic of this approach Micromotives and Macrobehavior. Schelling demonstrates in a large number of convincing cases, how we can explain large and complex social outcomes, as the aggregate consequence of behavior by purposive agents pursuing their goals within constraints. He offers a simple model of residential segregation, for example, by modeling the consequences of assuming that blue residents prefer neighborhoods that are at least 50% blue, and red residents prefer neighborhoods at least 25% red. The consequence -- a randomly distributed residential patterns becomes highly segregated in an extended series of iterations of individual moves.

It is possible to model various kinds of social situations by attributing a range of sets of preferences and beliefs across a hypothetical set of agents -- and then run their interactions forward over a period of time. SimCity is a "toy" version of this idea -- what happens when a region is developed by a set of players with a given range of goals and resources? By running the simulation multiple times it is possible to investigate whether there are patterned outcomes that recur across numerous timelines -- or, sometimes, whether there are multiple equilibria that can result, depending on more or less random events early in the simulation.

Robert Axelrod's repeated prisoners' dilemma tournaments represent another such example of agent-based simulations. (Axelrod demonstrates that reciprocity, or tit-for-tat, is the winning strategy for a population of agents who are engaged in a continuing series of prisoners' dilemma games with each other.) The most ambitious examples of this kind of modeling (and predicting and explaining) are to be found in the Santa Fe Institute's research paradigm involving agent-based modeling and the modeling of complex systems. Interdisciplinary researchers at the University of Michigan pursue this approach to explanation at the Center for the Study of Complex Systems. (Mathematician John Casti describes a number of these sorts of experiments and simulations in Would-Be Worlds: How Simulation is Changing the Frontiers of Science and other books.)

This approach to social analysis is profoundly different from the "subsumption under theoretical principles" approach, the covering-law model of explanation. It doesn't work on the assumption that there are laws or governing regularities pertaining to the social outcomes or complex systems at all. Instead, it attempts to derive descriptions of the outcomes as the aggregate result of the purposive and interactive actions of the many individuals who make up the social interaction over time. It is analogous to the simulation of swarms of insects, birds, or fish, in which we attribute very basic "navigational" rules to the individual organisms, and then run forward the behavior of the group as the compound of the interactive decisions made by the individuals. (Here is a brief account of studies of swarming behavior.)

How would this model of the explanation of group behavior be applied to real problems of social explanation? Consider one example: an effort to tease out the relationships between transportation networks and habitation patterns. We might begin with a compact urban population of a certain size. We might then postulate several things:

• The preferences that each individual has concerning housing costs, transportation time and expense, and social and environmental environmental amenities.
• The postulation of a new light rail system extending through the urban center into lightly populated farm land northeast and southwest
• The postulation of a set of prices and amenities associated with possible housing sites throughout the region to a distance of 25 miles
• The postulation of a rate of relocation for urban dwellers and a rate of immigration of new residents

Now run this set of assumptions forward through multiple generations, with individuals choosing location based on their preferences, and observe the patterns of habitation that result.

This description of a simulation of urban-suburban residential distribution over time falls within the field of economic geography. It has a lot in common with the nineteenth-century von Thunen's Isolated State analysis of a city's reach into the farm land surrounding it. (Click here for an interesting description of von Thunen's method written in 1920.) What agent-based modeling adds to the analysis is the ability to use plentiful computational power to run models forward that include thousands of hypothetical agents; and to do this repeatedly so that it is possible to observe whether there are groups of patterns that result in different iterations. The results are then the aggregate consequence of the assumptions we make about large numbers of social agents -- rather than being the expression of some set of general laws about "urbanization".

And, most importantly, some of the results of the agent-based modeling and modeling of complexity performed by scholars associated with the Santa Fe Institute demonstrate the understandable novelty that can emerge from this kind of simulation. So an important theme of novelty and contingency is confirmed by this approach to social analysis.

There are powerful software packages that can provide a platform for implementing agent-based simulations; for example, NetLogo. Here is a screen shot from an implementation called "comsumer behavior" by Yudi Limbar Yasik. The simulation has been configured to allow the user to adjust the parameters of agents' behavior; the software then runs forward in time through a number of iterations. The graphs provide aggregate information about the results.

"Folk" sociology

All of us are sociologists, at some level. We have social concepts in terms of which we analyze the social world around us -- "boss," "working class guy," "politician," "evangelical", "millennial generation". (Stereotypes of groups defined in terms of race and class probably fall in that category.) We operate on the basis of stylized schemata about social causes -- what sorts of things influence what other things. And we operate with some stylized social facts. ("Bad economic times make people more suspicious," "Big cities are more unsafe than towns," "Elections are decided by big campaign contributions," "Midwestern people are more socially conservative than Californians.") Putting all these sorts of assumptions together, we can say that we possess a conceptual framework and causal theory of the social world, which helps us to navigate the social relationships, conflicts, and needs that we have in ordinary life. Action proceeds on the basis of a representation of the world.

What this comes down to is the obvious point that humans are cognitive beings who undertake to conceptualize and explain the world around them; they come up with conceptual schemes and causal hypotheses about how things work, and they construct their plans and actions around these frameworks. We are "cognitive" -- we undertake to represent the world around us, based on observation and the creation of organizing concepts. And, of course, many of those concepts and hypotheses are badly grounded; they don't divide the world in a way that is really illuminating, or they offer stereotypes about how things work that aren't actually true. ("Don't bet on red -- it's come up four times in a row, so it's not likely to come up next time." That's a false statement about a series of randomly generated red and black events, and the player who follows this rule will lose to the player who is guided by probability theory.)

This sort of everyday social cognition is similar to what philosophers of psychology call "folk psychology" -- the ordinary categories of thought and action that we attribute to each other in order to describe and explain each other's behavior (intention, belief, pain, anger, ...). And philosophers have asked whether there is any relation at all between folk psychology and scientific psychology. (Ian Ravenscroft treats this issue in the philosophy of psychology in his article in the Stanford Encyclopedia of Philosophy.) Some philosophers have argued that the most fundamental and scientifically satisfactory explanations of individual behavior will be couched in terms that bear no relation at all to the concepts of ordinary mentalistic psychology.

So what is the status of folk sociology? We can ask several questions about this common sense framework of social cognition and expectation. First, where does it come from? What are the social processes of learning through which we arrive at the specifics of the social assumptions and concepts that we employ? Second, to what extent are there important differences across individuals with respect to the features of their social frameworks? (For example, we can explore whether there are cultural and national differences, gender and race differences, or generational differences across different groups and cohorts.) Third, we can examine the degree to which these categories and assumptions are rigid, or whether they are open to modification through additional experience -- "learning".

A different question, though, is also important: What is the relationship between these ordinary sociological frameworks and scientific sociology? Is there a relationship at all? Can scientific sociology learn from common sense? And can common sense improve its grasp of the social world through interaction with scientific sociology? Might we speculate that ordinary common sense does a fairly good job of picking out the salient features of the social world? Or, on the contrary, might we judge that the categories of "folk" sociology are about as misleading as pre-modern, magical concepts of nature? Or perhaps, might we say that rigorous scientific sociology can serve to refine and improve upon our "folk" concepts of the social world -- lead us to abandon categories such as race, for example, in our efforts to understand Obama, Michael Jordan, and DuBois?

The example of the natural sciences would lead us to one set of answers on these questions: "folk" knowledge of the natural world was not in fact a good guide to scientific physics, and the concepts of modern physics bear little intelligible relationship to common sense concepts of ordinary experience of tables and chairs. One way of putting this is to say that physics concepts are "theoretical", whereas common sense concepts are "phenomenological" (based on immediate experience).

Whether that is a valid distinction or not in physics, it probably is not a valid distinction in the social sciences. Social life is more transparent than the physical world; so our best scientific understanding of the social ought to bear some understandable relationship to the categories of ordinary social cognition. Common sense may not be highly specific in theorizing the concept of "power" in social life; but the phenomena of power are in fact fairly visible, and ordinary common sense captures these phenomena reasonably well. It is possible to paraphrase virtually any esoteric sociological thesis about power, in terms that are understandable in ordinary social experience. And likewise for exploitation, alienation, disaffection, racism, prejudice, discrimination, and affinity groups (to list a grab bag of sociological concepts): each of these concepts can be related to ordinary experiences and ordinary, common sense categories of social interaction.

So here is a possible answer to our original question -- how do ordinary social concepts relate to those of scientific sociology? We can say that there ought to be a critical but intelligible relationship between the two sets of concepts. Scientific sociology can point out the limitations and blind spots of ordinary ways of representing the social world. But ordinary social observation and conceptualization constitute the real content of sociological hypothesis and theory. So both systems of social knowledge fruitfully interact with each other, and -- ideally -- lead to a rising level of competence in cognizing and understanding society.

A normative aspect to power

It sometimes seems as though there is a normative dimension to our concept of power. What if we defined "power" in these terms: an agent exercises power when he/she undertakes to compel individuals or groups to act in ways they prefer not to act, against their interests and without the justification of a legitimate state underwriting the compulsion. Notice that the last qualification entails that the exercise of power is by definition "illegitimate"; legitimate authority compels behavior but does not exercise power. So on this definition, there is a behavioral element and a normative element in the proposition that "A exercises power over B": A compels behavior by B and A does not have a legitimate political or moral right to do so.

This might appear to be largely a semantic question: what do we mean by "power"? Is the exercise of the enforcement of law within a procedurally and substantively just polity an exercise of "power"? Or is it rather the exercise of rightful authority?

Certainly it is correct to observe that the behavioral aspect of involuntary compulsion is present in both types of cases. The criminal who is imprisoned for his crimes is treated coercively, in that he is confined against his will; so the state has the ability to "compel individuals to act contrary to their will". If we took the element of compulsion and coercion as uniquely central, then both the lawful state and the extortion gang are exercising power -- over criminals and innocent citizens respectively. If, on the other hand, we think that coercively backed authority is something different from "power", then a democratically established legal authority cannot be said to be exercising power over its citizens (though it may do so over its international adversaries).

If we go down this road in analyzing power, then there is a close relationship between power, social justice, and democracy.

How would we decide this question? And does it have any importance for the purposes of social and political explanation, or for the design of social policies?

Structure, psychology, power

Political and social power involves the exercise of social resources to compel various kinds of unwilling behavior by others. What creates power in society? What are the sorts of social and structural factors that permit individuals to exercise power? And what features of personality lead a given individual to choose to use the instruments of power to achieve his/her will? In short -- how does power pertain to "structure" and "agency"?

This is one of the categories in social analysis that requires that we bring together both agency and structure. Individuals wield power; but they only do so on the basis of resources and advantages that are conferred upon them by existing social relations. The enduring social relations that exist in a society -- for example, property relations, administrative and political relations, or the legal system -- constitute a structure within which agents act, and they determine the distribution of crucial social resources that become the raw materials on the basis of which agents exercise power over other individuals and groups. So the particular details of a social structure are crucial in determining the forms of power that exist in the society. For example, a privileged position within the property system -- the possession of significant income and wealth -- confers a resource advantage on people in that position. They can use their wealth to solicit powerful allies; they can purchase media outlets; they can influence politicians -- all with an eye to achieving their goals in spite of the contrary wishes or interests of others.

Likewise, a privileged position in the communications system -- a television news producer or newspaper publisher, for example -- can use his/her position to alter the way in which stories are presented in such a way as to change the way the public thinks about the issues; and these changes in thought can lead to changes in behavior. And an elected official can exercise power by setting the agenda for others -- by including or excluding various options from consideration.

So one's position within these various social structures -- systems of social relations -- determines the volume of social resources upon which one can call in the effort to constrain or compel the actions of others. Position determines one's capacity for power. But it does not determine the exercise of power. To be said to exercise power, it is necessary to have the goal of compelling people to do things they don't want to do. This is where agency or the "will to power" comes in. It is possible for a person with access to great power resources to nonetheless behave in ways that do not make use of power but rather depend on building consent and consensus. We might contrast Churchill with Stalin in mobilizing society for war; Churchill persuaded the British people to sacrifice in support of the war effort, whereas Stalin used the coercive power of the state to achieve his war mobilization goals.

This fact suggests that we need to consider something of the psychology of power. This is a topic that Adorno and other critical theorists invoked through the concept of the "authoritarian personality" -- an idea invoked largely in an effort to understand fascism. Others might attempt to assimilate the willingness to use power under the category of "opportunistic" or "instrumentalist" decision-making: coercion is considered as simply one out a menu of feasible strategies for achieving one's will. (This is perhaps the foundation of Hobbes's understanding of the pursuit and use of power.) And here we might speculate that the "democratic personality" is a set of dispositions to behavior that lead the agent to seek out persuasion and consensus rather than force, deception, and coercion as instruments through which to achieve one's goals. (Taken to the limit, we might say that a proper democracy creates an environment in which there is neither opportunity nor impulse towards the exercise of power.)

On this way of laying out the landscape of power, there are several dimensions to be considered: the social arrangements that make it possible for some individuals to pressure, coerce, and compel other individuals to do their bidding; the social arrangements that create profound conflicts of interest in the context of which the incentive to wield power naturally arises; and the circumstances of social psychology and personality that lead some individuals to choose to make use of resources of power to coerce, while others choose strategies that depend on willing consent to achieve collective purposes.

Large social forces?

We often analyze the world around us in terms of large social forces and trends -- globalization, the rise of ethnic identities, the spread of global capitalism, the rise of China as a coming super-power. These large forces are the "folk theories" through which we try to make sense of the world as it changes around us. But do these constructs actually make sense from an analytical, social scientific point of view? Or are they more akin to the large supra-historical categories advanced by pseudo-historians such as Toynbee or Spengler?

One reason for drawing that last conclusion is a justified skepticism about large impersonal social causes. We need to know what the microfoundations are, the concrete local mechanisms through which any social force works -- and merely postulating that "the forces of globalization are producing more social unrest", for example, doesn't provide the necessary illumination at that level.

This critical concern comes along with another: the observation that "folk" concepts of large social forces (for example, "spread of extremism" or "globalization") often encompass a pretty wide and heterogeneous set of lower-level processes. Presumably globalization works differently in Australia, Kenya, and London. So the term is more of an umbrella than a specific theory of how the world works.

But all of this said -- is there any rigorous and scientifically justifiable use for the idea of large social forces?

I think there is such a use. The concerns just mentioned are entirely valid. But they don't exclude the possibility that there are some processes of change in the world that are large and systemic, and that do have the requisite degree of microfoundations at the disaggregated levels. The examples I can think of come largely from the economic realm, but I am sure we could come up with cultural and social examples too.

--- The global effects of the Great Depression in the 1930s. The stock market crash of 1929 created financial and business consequences all over the United States. And the alterations in demand and price for commodities such as cotton or coal had consequences throughout the world. Small peasant farmers in North China were forced to reconfigure their cropping regimes as a result of events that took place on Wall Street. So we can say that the Great Depression was a large social force and one whose consequences were global in scope.

--- The Asian financial crisis of 1997 likewise created a cascading series of effects that were eventually felt everywhere in the world.

--- The SARS epidemic of 2002-2003 was an example of the possibility of global pandemic caused by the rapid transmission of exposure from China to Canada through the air travel system.

In each case we have an example of a large social cause. And we have a good understanding of the "transmission belt" through which these local events can have global consequences. It is the degree of integration among separate agents and groups that is created by national and global markets, communications systems, and transportation systems. Markets have foundations that are both global and local. They signal future events to millions of independent actors throughout space. These actors modify their behavior. And these shifts in turn create new market situations. So the rapid transmission of information, people, and products constitutes the microfoundational account that is needed for asserting the large social force.

There are similar stories that can be told for the transmission of ideas and cultural variations -- e.g. new forms of Christian or Muslim activism. (Stephen Greenblatt has some great examples of the transmission of cultural ideas in Marvelous Possessions: The Wonder of the New World.)

So the forces of globalization are real enough. And they depend upon mechanisms of transmission that create a systemic interdependence of behavior in widely separated places. Moreover, the examples give us an idea about how to characterize the idea that "globalization is changing the world." We can break this claim down into the idea -- an empirical one -- that asserts that there is a rising level of integration across societies, achieved by communications systems, transportation systems, and economic interdependency, through which the actions of people in Indonesia and the UK are much more systemically interconnected than they were a century ago. The analytical task is to be as specific as possible in identifying the pathways through which global influence of a factor is achieved, and not to engage in lazy thinking about big social forces that we really don't understand.

Social progress in India?

How much social progress has India made since Independence sixty years ago? According to economist V. K. Ramachandran, not very much when it comes to life in the countryside. (Hear my interview with Ramachandran on iTunes and on my web page.) Ramachandran gives a profile of the social problems faced by India at the time of Independence -- depths of income poverty, illiteracy, avoidable disease, and the worst forms of caste, class and gender oppression in the world -- and then judges that, appallingly, these same problems continue in the countryside without significant change. And this failure derives from the country's failure to solve its agrarian question. Poverty, inequality, and deprivation continue to be rampant in rural society. And this persistence derives from the failure to address the fundamental relations of property and power in the countryside. Moreover, the processes of globalization and liberalization have, if anything, intensified these problems.

Professor Ramachandran is a research professor at the Indian Statistical Institute in Kolkata, and is the author of Wage Labour and Unfreedom in Agriculture: An Indian Case Study.

Professor Ramachandran takes issue with the view of India that appears to be emerging in Japan and the United States -- as a country with shopping malls, hi-tech companies, and rapid economic growth. These images are true of some places in India -- but they have little relevance to conditions in rural India. (And the population of India continues to be at least 70% rural and agricultural.) The progress that has occurred in the countryside is meaningful -- agriculture has increased its productivity significantly since 1960, and India is now grain-self-sufficient. India is no longer locked into a "ship-to-mouth" existence. But these changes in the productivity of agriculture have not been associated with changes in the basic institutions present in the countryside -- what Ramachandran refers to as the "agrarian structure." And these social relations continue to create a system that entrenches inequality and deprivation for peasants and agricultural workers. Ramachandran maintains that three "new" inequalities have emerged -- inequalities between regions, inequalities between crops, and inequalities between classes. (As an expert on agricultural workers, Ramachandran is in a good position to observe what has happened for this segment of India's rural population.)

Ramachandran is an activist-scholar, and he is involved in a large collaboration with other scholars to provide a review of conditions in villages in a growing list of states in India. Ramachandran underlines the point that there is great variation across the map of India. The goal of these studies is to provide a detailed snapshot of the social conditions in the villages -- studies of the oppressed classes, tribes, and women; the state of village amenities (sewerage, clean water, roads, education). Over a number of years the goal of the research effort is to arrive at a more nuanced description of the conditions of rural life across many states in India. This research is highly valuable, since it permits disaggregation of descriptions of the countryside that are often based on aggregated data.

An interesting feature of this research project is the fact that it is deliberately linked to the activist organizations of peasants, workers, and women. The researchers consult with the agrarian activists to discover what the most important issues are -- and then to focus research effort on discovering the social details associated with these issues. And Ramachandran is emphatic in saying that the rigor of scientific investigation can and should be combined with this collaboration with the activist organizations. In fact, he indicates that the organizations themselves are insistent about this point. "Don't lose your academic rigor," the leaders of the organizations insist.

There is a lot more in the interview. But the bottom line is that Ramachandran offers a really good example of the engaged scholar. And the kind of social research that he and his colleagues are doing is well designed to help to diagnose some of the changes and public policies that are needed in India.

Reasoning about agents

Rational choice theory usually advances a highly abstract theory of decision-making -- utility-maximizing choice among discrete options -- and then draws deductive conclusions. But actual human reasoners don't look much like this abstract ideal. It is interesting to consider how much one can explain while weakening the heroic assumptions about agent rationality. It turns out that it is possible to explain quite a bit of social behavior on the basis of a theory of agency that incorporates only a few qualitative assumptions about practical agency: purposiveness, deliberation, comparison, and choice -- without the specific mathematics of utility theory. And this gives a basis for a “practical agency” type of social explanation that doesn’t fall directly under the many criticisms that are offered of formal rational choice theory. And it represents a framework of analysis that corresponds fairly directly to Aristotle’s conception of deliberation.

First, purposiveness. We assume that agents have purposes -- states of affairs that they want to bring about. Their actions are intended to bring about their goals. (This is the basic assumption of means-end rationality.)

Second, deliberation. We assume that agents collect information about the courses of action that are feasible in the moment. And they collect information about the likely consequences of the various actions that they are considering. Within a deliberative process they choose an action to pursue.

Third, comparison. We assume that the agent deliberates by considering the advantages and disadvantages of each action (costs and benefits) and the degree to which the possible or probable effects of the choice serve the set of purposes the agent pursues. Further, the agent may compare choices on the basis of the nature of the action itself rather than solely on the basis of consequences -- that is, the agent may combine deontological considerations with utilitarian considerations (this represents an element of Kant’s theory of moral decision-making). That is, the agent chooses on the basis of full comparison of the choices.

So far we have described purposive deliberation without invoking utilities. We have invoked the idea of preference in this account, because we have assumed that the agent prefers outcomes that better fulfill his/her purposes. And we have likewise invoked comparison and the idea of “more and less” of something in ranking outcomes. The apparatus of utility theory is one way of articulating or modeling these features -- but we are not forced to attribute a full apparatus of utility measurement and aggregation in order to attribute purposiveness and comparison. It will suffice if the agent can judge "X is better than Y in fulfilling my purposes."

So let's refer to this less abstract description as a “broad theory of purposive agency”. The question here is a simple one: does this description give us enough to get a social explanation going? I believe that it does, and that this demonstrates that the conception of agents as purposive decision-makers is more durable than any particular formal theory of decision-making.

I support this observation with a single inconclusive illustration. We can reproduce the public goods problem, and the derivation of the rationality of free-riding, using only the comparative resources of the reduced theory of agency. And we can simultaneously prepare a theoretical location for future empirical research for those instances where free-riding does not materialize as expected.

First the derivation of the tendency toward free riding in the presence of public goods. The agent has preferences among the possible outcomes of possible collective action. He/she recognizes that the probability of realizing the collective good is not significantly altered by participation. Agent further recognizes that there is a cost to participation. Agent decides to not participate, based on costs and outcomes: the benefit of participation and non-participation is equal and the cost of participation is greater, so non-participation is preferred.

But now consider what happens when we turn on a deontological component of the comparison. Agent notes that non-participation involves taking unfair advantage of the actions of others. Agent prefers actions that are fair. Agent therefore chooses participation.

Finally, we might further complicate things by allowing that agent allows tradeoffs between the cost-benefit comparison and the deontological comparison. In some instances the weight of fairness prevails and agent chooses the less-good alternative. In other cases the situation reverses. Agent sacrifices minor unfairness and chooses the better outcome.

There are many, many examples of explanations in sociology, anthropology, and history that account for an outcome based on analysis of the situation of deliberation confronting a hypothetical set of purposive agents; and substantial explanations ensue. (As one example, consider Jean Ensminger’s explanation of Kenyan cattle-tending and bride price practices as a solution to a principal-agent problem between cattle owners and cattle tenders; Making a Market: The Institutional Transformation of an African Society.)

Power and violence in China

Several recent postings on this blog have focused on power. Ultimately power depends upon a threat of violence. And recent reports from China have thrown the spotlight on the use of violence against innocent citizens who are challenging one aspect of power or another. The photo at left is taken from a news story reporting the results of homeowners' protests in a Beijing neighborhood against unplanned buildings on the greenspace of their neighborhood. (Reportage and more photos are available here.)

What this incident reflects is a disturbing and apparently growing incidence of the use of violence by private security companies against groups of citizens in China who are engaging in a variety of efforts to protect their interests or advance their claims. These incidents of violence also occur at the hands of Chinese police -- for example, the beating death of a newspaper editor that was reported in an article in the New York Times a few years ago.

These reports are somewhat rare -- not because the behavior is rare, but because it is very difficult for journalists to do the sort of investigating and reporting that would be needed in order to uncover these outrages. (Remember Amartya Sen's theory that a free press in India is the best explanation of the absence of famine since Independence; think how different China's political scene would be with a practice of unfettered investigative journalism.)

But what these reports suggest is a very sober reality: that under current conditions in China, ordinary groups of Chinese people are subject to the imposition of serious violence if they come together to press their claims. There are reports of violence by paid thugs concerning migrant workers, factory workers, peasants subject to land confiscation, and ordinary poor people who have somehow gotten into conflict with the authorities.

What this also suggests is that the forging of an effective and binding legal order in China -- one that clearly articulates and defends the civil and individual rights of citizens that are valid even in circumstances of conflict between the powerful and the powerless -- creation of this legal order is a profoundly important goal for Chinese society. Only on the basis of these kinds of legal guarantees can civil society be an arena in which citizens and groups can advocate for their interests in a peaceful and progressive manner.