Sunday, August 17, 2014

Making sense of the ASA


The annual meeting of the American Sociological Association is taking place in San Francisco this week, and it is a tsunami of ideas, methodologies, research strategies, sections, and fields of study. Begin with the program: it is vast. Over five thousand names are listed in the index as presenters, discussants, and chairs; there are 600 sessions, numerous "tables" for smaller presentations, and an endless stream of "authors meet critics." And so many of the books, papers, talks, and discussions seem worth reading and thinking about.

But that poses a difficult problem: no one can possibly read all the books, think about all the methods, consider all the fascinating questions and problems raised. So how are we to think of the role of this annual meeting? Is it an effective way of bringing the profession together, nationally and internationally, senior and junior? Or is it more like a super-sized hardware store where you pretty much need to know what you need before you enter the door, or you will be lost among the nail guns, paints, nuts and bolts, plumbing equipment, and sand paper that confront you? Can we think of the ASA annual meeting as a place for genuine "browsing", where specialists can get exposed to innovative and unfamiliar thinking from other parts of the profession? Or is it a place where the Bourdieusians, the quants, the ethnographically inclined, and the historical comparativists largely keep to themselves, like a segmented social media flow? (One person tunes into the DemocracyNow flow with its associated twitter feeds and followers while the other gets her news from the firehouse of Fox and its online choirs.) Or in other words, does the ASA feed segment into the subdisciplines and their groups of practitioners, so that everyone gets more of what he or she already knows very well?

How would a real student of the discipline of sociology begin to make sense of this vast slice of the mosaic of the discipline? What does Andrew Abbott think? (He's here -- he could be asked.) One possibility is to look at the situation historically. Take the programs of the past ten years and see how they have changed, in a variety of ways: topics, status composition of the panels, gender and race composition of the presenters, heterodox and orthodox approaches to the methodology of sociology, etc. This could be done. We might be able to produce a number of frequency graphs -- the fall and rise of Marxism, the rise of feminist sociology, the frequency of mention of the emotions in titles and abstracts, ... 

Another approach might be to look at the program as the expression of the "field" of the extended discipline of sociology as a field of power and prestige (a sort of Bourdieusian approach). Here we might abstract from content and ask the question, what can we infer about power in the discipline by inspecting the program? What features do the officers and executives of the association have? Do a small number of research institutions seem to play a disproportionate role in decision making and the selection of topics (Berkeley, Yale, Michigan)? Here again questions of gender, race, status, and privilege come into the picture. 

We might also consider the question of the absolute size of the program -- the 4+ days and 600 sessions that were included. Is this a good size for the profession as a whole -- if so, why so? Does it give career opportunities for younger scholars, to help them get their work into the thought space of the more senior practitioners? Does it serve the intellectual goal of broadening the intellectual scope of the discipline and its practitioners? Is the jumbo size of the meeting really about the association's need to put together an annual meeting that is a big success in terms of attendance and revenue? What would invalidate the notion of reducing the program to 2+ days and 300 sessions, with only 2000+ presenters? Would this be seen as a retreat of the significance and health of "sociology" within American research universities?

Finally, what might be possible by way of mapping the knowledge, research, and theory represented by this ocean of research? It seems possible to do this, but it would be a daunting task. The most general possibility is to create a database of all sessions, papers, presenters, and books, and code each entry by several properties. For individuals the characteristics might include affiliation, rank, gender, and race/ethnicity/nationality. (If we could add the individual's PhD institution that would be useful as well.) For papers and books the characteristics might include: author [linked to author entry], topic, primary references, method, ASA section. Topic codes would need to be hierarchical -- perhaps something like this: "race / segregation / urban core poverty / Chicago".

With such a database we could pose a number of interesting descriptive questions: 

What topics and methods are most frequent? 
What is the distribution of contributors by gender and race? 
What is the representation of senior high-prestige researchers relative to junior middle-prestige researchers? 

We might also imagine representing this body of data in terms of network maps. We could present a network graph of papers linked by topics, which would presumably create a number of clusters of papers organized by affinity of research themes. This map might well provide a different outline of the "continents" of the ASA, relative to the mapping by sections of the organization. (That is, there may be many papers addressing race from different methodological points of view and deriving from different research traditions.)

In short, it seems that the annual meeting of a large discipline like sociology, political science, or psychology can potentially provide the basis for some very interesting analysis of how the discipline works and changes within the field of academic life.

(Here is a partial remedy for the problem of audience self-selection: make random, unannounced and surprising substitutions on the program at the last minute. Several hundred people came to an early-morning session on "Bourdieu, culture, and empirical research" expecting to hear Loic Wacquant -- a sympathetic practitioner of Bourdieu's sociology -- and found instead the substitute personage of Michael Burawoy, a sympathetic but orthogonal reader of Bourdieu. Burawoy offered a lively reconsideration of Bourdieu's framework in relation to that of Marxist historical materialism and theory of class. His central thrust was the difference in the ways that Bourdieu, Gramsci, and Marx think about class consciousness. Who, when, and how are the social participants who can come to understand the conditions of their domination? I very much enjoyed his presentation, not least because it was a surprise.)

Monday, August 11, 2014

Realism and methodology


Methodology has to do with the strategies and heuristics through which we attempt to understand a complicated empirical reality (link). Our methodological assumptions guide us in the ways in which we attempt to collect data, the kinds of data we collect, the explanatory hypotheses we bring forward for that range of empirical findings, and the ways we seek to validate our findings. Methodology is to the philosophy of social science as historiography is to the philosophy of history.

Realism is also a set of assumptions that we bring to empirical investigation. But in this case the assumptions are about ontology -- how the world works, in the most general ways. Realism asserts that there are real underlying causes, structures, processes, and entities that give rise to the observations we make of the world, natural and social. And it postulates that it is scientifically appropriate to form theories and hypotheses about these underlying causes in order to arrive at explanations of what we observe.

This description of realism is couched in terms of a distinction between what is observable and what is unobservable but nonetheless real -- the "observation-theoretic" distinction. But of course the dividing line between the two categories shifts over time. What was once hypothetical becomes observable. Extra-solar planetary bodies, bosons, and viruses were once unobservable; they are now observable using various forms of scientific instrumentation and measurement. So the distinction is not fundamental; this was an essential part of the argument against positivist philosophy of science. And we might say the same about many social entities and structures as well. We understand "ideology" much better today than when Marx theorized about this idea in the mid-19th century, and using a variety of social research methods (public opinion surveys, World Values Survey, ethnographic observation, structured interviews) we can identify and track shifts in the ideology of a group over time. We can observe and track ideologies in a population. (We may now use a different vocabulary -- mentality, belief framework, political values.)

There are several realist methodologies that are possible in the social sciences. The methodology of paired comparisons is a common part of research strategies in the historical social sciences. This is often referred to as "small-N research." (Here is a description of the method as practiced by Sid Tarrow; linklink.) The method of paired comparisons is also based on realism and derives from causal ideas; but it is not specifically derived from the idea of causal mechanisms.  Rather, it derives from the simpler notion that causal factors function as something like necessary and/or sufficient conditions for outcomes. So if we can find cases that differ in outcome and embody only a small number of potential contributing causal factors, we can use Mill's methods (or more general truth-table methods) to sort out the causal roles played by the factors. (Here is a discussion of some of these concepts; link.) These ideas contribute to methodology at two levels: they give the investigator a specific idea about how to lay out his/her research ("seek out relevantly similar cases with different outcomes"), and they embody a method of inference from findings to conclusions about causal relations (the truth-table method). These methods allow the researcher to arrive at statements about which factors play a role in the production of other factors. (This is a logically similar role to the use of multiple regression in quantitative studies.)

Another possible realist approach to methodology is causal mechanisms theory (CM). It rests on the idea that events and outcomes are caused by specific happenings and powers, and it proposes that a good approach to a scientific explanation of an outcome or pattern is to discover the real mechanisms that typically bring it about. It also brings forward an old idea about causation -- no action at a distance. So if we want to maintain that class privilege causes ideological commitment, we need to be able to tell an empirically grounded story about how the first kind of thing conveys its influence to changes in the second kind of thing. (This is essentially the call for microfoundations; link.) Causal mechanisms theory is more basic than either paired comparisons or statistical causal modeling, in that it provides a further explanation for findings produced by either of these other methods. Once we have a conception of the mechanisms involved in a given social process, we are in a position to interpret a statistical finding as well as a finding about the necessary and/or sufficient conditions provided by a list of antecedent conditions for an outcome.

It is an interesting question to consider whether realism in ontology leads to important differences in methodology. In particular, does the idea that things happen as the result of an ensemble of real causal mechanisms that can be separately understood lead to important new ideas about methodology and inquiry?

Craver and Darden argue in In Search of Mechanisms: Discoveries across the Life Sciences that mechanisms theory does in fact contribute substantially to contemporary research in biology, at a full range of levels (link). They maintain that the key goal for much research in contemporary biology is to discover the mechanisms that produce an outcome, and that a central component of this methodology is the effort to explain a given phenomenon by trying to fit one or more known mechanisms to the observed process. So working with a toolbox of known mechanisms and "problem-solving" to account for the new phenomenon is an important heuristic in biology. This approach is both ontological and methodological; it presupposes that there are real underlying mechanisms, and it recommends to the researcher that he/she be well acquainted with the current inventory of known mechanisms that may be applied to new settings.

I think there is a strong counterpart to this idea in a lot of sociological research as well. There are well understood social mechanisms that sociologists, political scientists, and other researchers have documented -- easy riders, prisoners dilemmas, conditional altruism -- and the researcher often can systematically explore whether one or more of the known mechanisms is contributing to the complex social outcomes he or she is concerned with. A good example is found in Howard Kimeldorf's Reds or Rackets?: The Making of Radical and Conservative Unions on the Waterfront. Kimeldorf compares two detailed case histories and strives to identify the concrete social mechanisms that led to different outcomes in the two cases. The mechanisms are familiar from other sociological research; Kimeldorf's work serves to show how specific mechanisms were in play in the cases he considers.

This kind of work can be described as problem-solving heuristics based on application of a known inventory of mechanisms. It could also be described as a "normal science" process where small theories of known processes are redeployed to explain novel outcomes. As Kuhn maintains, normal science is incremental but creative and necessary in the progress of science.

A somewhat more open-ended kind of inquiry is aimed at discovery of novel mechanisms. McAdam, Tarrow and Tilly sometimes engage in this second kind of discovery in Dynamics of Contention -- for example, the mechanism of social disintegration (kl 3050). Another good example of discovery of mechanisms is Akerlof's exposition of the "market for lemons" (link), where he lays out the behavioral consequences of market behavior with asymmetric knowledge between buyer and seller.

So we might say that mechanisms theory gives rise to two different kinds of research methodology -- application of the known inventory to novel cases and search for novel mechanisms (based on theory or empirical research).

Causal-mechanisms theory also suggests a different approach to data gathering and a different mode of reasoning from both quantitative and comparative methods. The approach is the case-studies method: identify a small set of cases and gain enough knowledge about how they played out to be in a position to form hypotheses about the specific causal linkages that occurred (mechanisms).

This approach is less interested in finding high-level generalizations and more concerned about the discovery of the real inner workings of various phenomena. Causal mechanisms methodology can be applied to single cases (the Russian Revolution, the occurrence of the Great Leap Forward famine), without the claim to offering a general causal account of famines or revolutions. So causal mechanisms method (and ontology) pushes downward the focus of research, from the macro level to the more granular level.

The inference and validation component associated with CM looks like a combination of piecemeal verification (link) and formal modeling (link). The case-studies approach permits the researcher to probe the available evidence to validate specific hypotheses about the mechanisms that were present in the historical case. The researcher is also able to try to create a simulation of the social situation under study, confirm as much of the causal internal connectedness as possible from study of the case, and examine whether the model conforms in important respects to the observed outcomes. Agent-based models represent one such set of modeling techniques; but there are others.

So the methodological ideas associated with CM theory differ from both small-N and large-N research. The search for causal mechanisms is largely agnostic about high-level regularities -- either of things like revolutions or things like metals. It is an approach that encourages a more specific focus on this case or that small handful of cases, rather than a focus on finding general causal properties of high-level entities. And it is more open to and tolerant of the possibility of a degree of contingency and variation within a domain of phenomena. To postulate that civil disorders are affected by a group of well-understood social mechanisms does not imply that there are strong regularities across all civil disorders, or that these mechanisms work in exactly the same way in all circumstances. So the features of contingency and context dependence play an organic role within CM methodology and fit badly in paired-comparisons research and statistical modeling approaches.

So it seems that the ontology of causal-mechanisms theory does in fact provide a set of heuristics and procedures for undertaking social research. CM does have implications for social-science methodology.

Friday, August 8, 2014

The status of women in India


Sociologists are often interested in making sense of processes of change that radiate along the axes of the great tectonics of social life, including class, race, and gender. These features of social life are particularly fundamental because they denote powerful determinants of opportunity, life-course, and personal outcomes for all of us. The positions into which individuals are born within the property system have great influence on the ways their lives unfold. The social filigree of race and ethnicity, and the ways in which these categories are socially constructed and projected, likewise creates determinative pathways of development and action for individuals in many social settings. And the social freight of gender and family creates opportunities and obstacles, expectations and stereotypes, for boys and girls, women and men. There are other large dimensions of social embeddedness that might be brought forward as well -- religion, culture, normative communities, power and authority, for example. But class, race, and gender are especially profound. And each has given rise to movements of emancipation in reaction to the oppressions that they represent for specific groups in society.

Kenneth Bo Nielsen and Anne Waldrop have assembled an excellent volume on the status of women in contemporary India in their recent book Women, Gender, and Everyday Social Transformation in India. The collection brings together contributors who approach the topic using the tools of ethnographic research to better understand the everyday realities of experience of women in India today. The volume focuses on three large areas of life in contemporary India -- technology and work, political institutions, and feminist activism. There is a valuable effort to attempt to understand the current upsurge of public violence and rape against women in terms of the social changes that are occurring in these areas.

The editors introduce the focus of the volume in these words:
The pace of socioeconomic transformation in India over the past two and a half decades has been formidable. In this volume we are concerned with examining how these transformations have played out at the level of everyday life to influence the lives of Indian women, and gender relations more broadly."
Readers of Understanding Society will find the volume especially interesting because it approaches the realities of gender, class, and caste in a micro- and meso-level way, looking at specific individuals and groups of women within the concrete social relations within which they find themselves. A rich understanding of the socially constituted realities of "agency" and "structure" plays an important role in the approaches these investigators bring to their research. 
People can never act outside of the multiplicity of social relations in which they are enmeshed; ... the exercise of individual agency aiming for change is thus always conditioned, and human agency is both enabled and limited by those social structures within which change is sought. (7)
Several chapters are especially interesting.

Kenneth Nielsen's contribution on the Singur movement in West Bengal against the proposed Tata factory favored by the ruling CP/M government of the state is very interesting. He illustrates how the status rise of a caste group (Chasi Kaibartta in transition to Mahishya) also reflected a fairly deliberate effort to redefine and control the role of women. "Notably, the Mahishya caste movement for higher status entailed an appropriation of upper-caste norms and values including the Hindu elite male concern for harnessing female sexuality" (207). This account suggests a Bourdieu-like struggle within a "field", making strategic adjustments so as to gain advantage over time. Intra- and inter-caste struggles played out into a redefinition of the nature and role of women in village society. Nielsen's central interest is the micro-processes through which women in these villages made the transition from their traditional roles to the role of part-time activist. This is a familiar question from social movements research. (Here is a causal diagram representing the recruitment process taken from Doug McAdam, "Beyond Structural Analysis" (link).)


But Nielsen's treatment is not theoretical; it is ethnographic, attempting to discover through interviews and participant-observation the concrete steps that occurred from household to street demonstration for some of these women.

Another interesting observation Nielsen offers concerns the question of “issue escalation”. It is often observed in the study of contentious politics that a movement around a specific issue often broadens its focus to include a more comprehensive set of issues. In this instance the natural progression would be from the specific issue of land confiscation to a broader concern for gender equity in rural society. But this escalation did not occur in the case of the West Bengal/Singur movement.
In contrast [to movements in Karnataka and Maharashtra], the Singur movement did not spawn reflections on the gendered nature of landownership and inheritance, even though inheritance patterns in Singur followed the broader West Bengal pattern where land is fragmented and customarily inherited through the line of male descendants. (215)
And Nielsen seems to have uncovered the root of an explanation for why this escalation did not occur: the involvement of women in the Singur movement was actually quite consistent with the patriarchal values governing women's roles that were current in rural society in West Bengal. "The mobilization of women into the Singur movement was facilitated by the fact that it was successfully embedded in a gendered discourse and imagery in which Shantipara's Mahishya women appeared as defenders of the common good of the family and the village" (215). (Kimberly Springer describes the complicated relationship between feminist activism and the US civil rights movement in Living for the Revolution: Black Feminist Organizations, 1968–1980.)

Turn now to another contribution that also focuses on West Bengal, Sirpa Tenhunen's "Gender, Intersectionality and Smartphones in Rural West Bengal." Tenhunen's question is how this innovative communication technology affected the opportunities and agency of women in this region. (One thing I appreciate about the volume as a whole is the care that the researchers take to make it clear that their findings do not represent "India" as a whole.)
I argue that to understand how mobile phone use is shaped by and reshapes gender, it is necessary to explore how the physical properties of phones determine technical affordances -- that is, the possibilities for action -- according to the users' gender, but also such parameters of identity as class and education. (33)
Tenhunen's ethnography focuses on women's use of the technology -- the conversations they conduct in specific times and places. She had observed this village in 1999 (before the cell network had been built out), and again in 2005, 2007-08, 2010, and 2012-13. So her observations permitted her to have a perspective on the socially constructed ways in which adoption of telephony was carried out. One of her most basic observations is that the adoption of telephony initially followed caste lines (37), with upper castes adopting cell phones earlier than other caste groups. She also finds that cell phone ownership tended to be concentrated among male villagers in the early period, and that diffusion to women was often the consequence of marriage relationships. "Fathers and brothers gave phones to ensure that women could stay in touch after marriage" (37). And she finds that women's calling patterns are different from male users in the village. Here is a table of calling patterns based on call diaries from 2008.



This is interesting material, and could be interpreted as a map of the patriarchal and domestic orbits of women and men in the village. Tenhunen argues that cell phones facilitated a broadening of social contacts that had favorable effects on gender opportunities for women in the village.
In rural India, the introduction of phones offered rural women a new, unobstrusive avenue to extend their contacts and space without moving out of their neighbourhood. Women's increased contact with their natal village and female relatives in other villages forms part of the broader changes evident in the village's gender relationships. The most pronounced of these changes are the increase in education for females, women becoming visible in formal politics, and a few high-caste women taking up white-collar jobs. (41-42)
What is particularly interesting about this research is the strength of the case Tenhunen establishes for the differential adoption and consequences of the new cellular technology. "Technology is gendered" -- this is a slogan in the sociology of technology, but it is a simple and important reality in this study.

These are only two of the fifteen chapters of Women, Gender and Everyday Social Transformation in India (Anthem South Asian Studies). But every chapter is worth reading, and the book represents an important contribution to understanding the dynamics under way in India today for half of its population.

Wednesday, August 6, 2014

What is methodology?



As social science researchers, we would all like to have an excellent methodology for carrying out the tasks we confront in our scientific work. But what precisely are we looking for when we aspire to this goal? What is a methodology, and what is it intended to allow us to do?

A methodology is a set of ideas or guidelines about how to proceed in gathering and validating knowledge of a subject matter. Different areas of science have developed very different bodies of methodology on the basis of which to conduct their research. We might say that a methodology provides a guide for carrying out some or all of the following activities:
  • probing the empirical details of a domain of phenomena
  • discovering explanations of surprising outcomes or patterns
  • identifying entities or forces 
  • establishing patterns
  • providing predictions
  • separating noise from signal
  • using empirical reasoning to assess hypotheses and assertions
Here is what Andrew Abbott has to say about methods in Methods of Discovery: Heuristics for the Social Sciences:
Social scientists have a number of methods, stylized ways of conducting their research that comprise routine and accepted procedures for doing the rigorous side of science. Each method is loosely attached to a community of social scientists  for whom it is the right way to do things. But no method is the exclusive property of any one of the social sciences, nor is any social science, with the possible exception of anthropology, principally organized around the use of one particular method. (13)
So a method or a methodology is a set of recommendations for how to proceed in doing scientific research within a certain domain. Sometimes in the history of philosophy there has been a hope that science could proceed on the basis of a pure inductive logic: collect the data, analyze the data, sift through the findings, report the strongest regularities found in the data set. But scientific inquiry requires more than this; it requires discovery and imagination.

What form might a methodology take? The simplest idea is that a methodology is a recipe for arriving at justified scientific statements with respect to a domain of empirical phenomena. A recipe is a set of instructions for treating a number of ingredients in a sequential way and producing a specific kind of output -- a soufflé or a bowl of pad thai. If you follow the recipe, you are almost certain to arrive at the soufflé. But it is clear that scientific methodology cannot be as prescriptive as a recipe. There is no set of rules that are certain or likely to lead to the discovery of compelling hypotheses and explanations.

So if a scientific methodology isn't a set of recipes, then what is it? Here is another possibility: a methodology consists of a set of heuristics that serve to guide the activities, data collection, and hypothesis formation of the scientist. A heuristic is also a set of rules; but it is weaker than a recipe in that there is no guarantee of success. Here is a heuristic for consumers: "If you are selecting a used car to purchase, pay attention to rust spots." This is a good guide to action, not because rust spots are the most important part of a car's quality, but because they may serve as a proxy for the attentiveness to maintenance of the previous owner -- and therefore be an indication of hidden defects.

Andrew Abbott mentions several key topics for specification through methodology -- "how to propose a question, how to design a study, how to draw inferences, how to acquire and analyze data" (13), and he shows that we can classify methods by placing them into the types of question they answer.

types of data gathering

data analysis
posing a question
History
Direct interpretation

Case study analysis
Ethnography
Quantitative analysis

Small-N comparison
Surveys

Formal modeling
Large-N analysis
Record-based analysis





Abbott suggests that these varieties can be combined into five basic approaches:
  • ethnography
  • historical narration
  • standard causal analysis
  • small-N comparison
  • formalization
And he arranges them in a three-dimensional space, with each dimension increasing from very particular knowledge at the origin to more abstract knowledge further out the axis. (Commonsense understanding of the facts lies at the origin of the mapping.) The three axes are formal modeling (syntactic program), pattern finding (semantic program), and cause finding (pragmatic program) (28). 


Abbott is a sociologist whose empirical and theoretical work is genuinely original and important, and we can learn a lot from his practice as a working researcher. His meta-analysis of methodology, on the other hand, seems fairly distant from his own practice. And I'm not sure that the analysis of methodology represented here provides a lot of insight into the research strategies of other talented social scientists (e.g. Tilly, Steinmetz, Perrow, Fligstein). This perhaps illustrates a common occurrence in the history of science: researchers are not always the best interpreters of their own practice. 

It is also interesting to observe that the discovery of causal mechanisms has no explicit mention in this scheme. Abbott never refers to causal mechanisms in the book, and none of the methods he highlights allow us to see what he might think about the mechanisms approach. It would appear that mechanisms theory would reflect the pragmatic program (searching for causal relationships) and the semantic program (discovering patterns in the observable data).

My own map of the varieties of the methods of the social sciences suggests a different scheme altogether. This is represented in the figure at the top of the post.



Monday, August 4, 2014

System safety engineering


Why do complex technologies so often fail, and fail in such unexpected ways? Why is it so difficult for hospitals, chemical plants, and railroads to design their processes in such a way as to dramatically reduce the accident rate? How should we attempt to provide systematic analysis of the risks that a given technology presents and the causes of accidents that sometimes ensue? Earlier posts have looked at the ways that sociologists have examined this problem (link, link, link); but how do gifted engineers address the issue?

Nancy Leveson's current book, Engineering a Safer World: Systems Thinking Applied to Safety (2012), is an outstanding introduction to system safety engineering. This book brings forward the pioneering work that she did in Safeware: System Safety and Computers (1994) with new examples and new contributions to the field of safety engineering.

Leveson's basic insight, here and in her earlier work, is that technical failure is rarely the result of the failure of a single component. Instead, failures result from multiple incidents involving the components, and unintended interactions among the components. So safety is a feature of the system as a whole, not of the individual sub-systems and components. Here is how she puts the point in Engineering a Safer World:
Safety is a system property, not a component property, and must be controlled at the system level, not the component level. (kl 263)
Traditional risk and failure analysis focuses on specific pathways that lead to accidents, identifying potential points of failure and the singular "causes" of the accident (most commonly including operator error). Leveson believes that this approach is no longer helpful. Instead she argues for what she calls a "new accident model" -- a better and more comprehensive way of analyzing the possibilities of accident scenarios and the causes of actual accidents. This new conception has several important parts (kl 877-903):
  • expand accident analysis by forcing consideration of factors other than component failures and human errors
  • provide a more scientific way to model accidents that produces a better and less subjective understanding of why the accident occurred
  • include system design errors and dysfunctional system interactions
  • allow for and encourage new types of hazard analyses and risk assessments 
  • shift the emphasis in the role of humans in accidents from errors ... to focus on the mechanisms and factors that shape human behavior
  • encourage a shift in the emphasis in accident analysis from "cause" ... to understanding accidents in terms of reasons, that is, why the events and errors occurred
  • allow for and encourage multiple viewpoints and multiple interpretations when appropriate
  • assist in defining operational metrics and analyzing performance data
Leveson is particularly dissatisfied with the formal apparatus in use in engineering and elsewhere when it comes to analysis of safety and accident causation, and she argues that there are a number of misleading conflations in the field that need to be addressed. One of these is the conflation between reliability and safety. Reliability is an assessment of the performance of a component relative to its design. But Leveson points out that systems like automobiles, chemical plants, and weapons systems can all consist of components that are highly reliable and yet that give rise to highly destructive and unanticipated accidents.

So thinking about accidents in terms of component failure is a serious misreading of the nature of the technologies with which we interact every day. Instead she argues that safety engineering must be systems engineering:
The solution, I believe, lies in creating approaches to safety based on modern systems thinking and systems theory. (kl 88)
One important part of a better understanding of accidents and safety is a recognition of the fact of complexity in contemporary technology systems -- interactive complexity, dynamic complexity, decompositional complexity, and nonlinear complexity (kl 139). Each of these forms of complexity makes it more difficult to anticipate possible accidents, and more difficult to assign discrete accident pathways to the occurrence of an accident.
Accidents are complex processes involving the entire sociotechnical system. Traditional event-chain models cannot describe this process adequately. (kl 496)
Leveson is highly critical of iterative safety engineering -- what she calls the "fly-fix-fly" approach. Given the severity of outcomes that are possible when it comes to control systems for nuclear weapons, the operations of nuclear reactors, or the air traffic control system, we need to be able to do better than simply improving safety processes following an accident (kl 148).

The model that she favors is called STAMP (Systems-Theoretic Accident Model and Processes; kl 1059). This model replaces the linear component-by-component analysis of technical devices with a system-level representation of their functioning. The STAMP approach begins with an effort to identify crucial safety constraints for a given system. (For example, in the Union Carbide plant at Bhopal, "never allow MIC to come in contact with water"; in design of the Mars Polar Lander, "don't allow the spacecraft to impact the planet surface with more than a maximum force" (kl 1074); in design of public water systems, "water quality must not be compromises" (kl 1205).) Once the constraints are specified, the issue of control arises; what are the internal and external processes that ensure that the constraints are continuously satisfied? This devolves into a set of questions about system design and system administration; the instrumentation that is developed to measure compliance with the constraint and the management systems that are in place to ensure continuous compliance.
Also of interest in the book is Leveson's description of a new systems-level way of analyzing the hazards associated with a device or technology, STPA (System-Theoretic Process Analysis) (kl 2732). She describes STPA as the hazards analysis associated with the risks identified by STAMP:
STPA has two main steps:
  1. Identify the potential for inadequate control of the system that could lead to a hazardous state.
  2. Determine how each potentially hazardous control action identified in step 1 could occur. (kl 2758)
Here is an example of the process through which an STPA risk analysis proceeds for NASA (kl 2995).


It would be very interesting to see how an engineer would employ the STAMP and STPA methodologies to evaluate the risks and hazards associated with swarms of autonomous vehicles. Each vehicle is a system that can be analyzed using the STAMP methodology. But likewise the workings of an expressway with hundreds of autonomous vehicles (perhaps interspersed with less predictable human drivers) is also a system with complex characteristics.



Each individual vehicle has a hierarchical system of control designed to ensure safe transportation of its passengers and the vehicle itself; what are the failure modes for this control system? And what about the swarm -- given that each vehicle is responsive to the other vehicles around it, how will individual cars respond to unusual circumstances (a jack-knifed truck blocking all three lanes, let's say)? It would appear that autonomous vehicles create the kinds of novel hazards with which Leveson begins her book -- complexity, non-linear relationships, emergent properties of the whole that are unexpected given the expected operations of the components. The fly-fix-fly approach would suggest the deployment of a certain number of experimental vehicles and then evaluate their interactions in real-world settings. A more disciplined approach using the methodologies of STAMP and STPA would make systematic efforts to identify and control the pathways through which accidents can occur.

Here is a simulated swarm of autonomous vehicles:

But accidents happen; neither software nor control systems are perfect. So what would be the result of one disabling fender-bender in the intersection, followed by a half dozen more; followed by a gigantic pileup of robo-cars?

Saturday, August 2, 2014

Classifying mechanisms by location


If we are going to take social mechanisms seriously, we need to be able to say more about what they are. Earlier posts have opened the possibility of offering a scheme of classification for social mechanisms (link, link). Here I want to briefly explore a different idea: to group mechanisms according to which part they play within the space of social influence postulated by the idea of methodological localism (link). I introduced the idea of methodological localism in "Levels of the Social" (link) as an ontological alternative to both methodological individualism and methodological holism. That specification of the nature of social reality suggested a small handful of fundamental questions. Here I want to experiment with classifying a number of mechanisms according to which of these questions they answer. Here is the relevant statement from "Levels of the Social" (link):

According to methodological localism, the social is constituted by socially situated individuals, nested within social relations and institutions that have only an intermediate degree of persistence and permanence.

The socially situated individual finds herself within a concrete set of social relationships, networks, and institutions. This complex serves to socialize and provide incentives, as well as to constrain. The approach of methodological localism supports as well the reality that institutions often have extra-local scope, geographically, demographically, and administratively. So, we can legitimately describe institutions with broader scope as being “higher-level” institutions. 

This approach suggests six large areas of focus for social science research:
  • What makes the individual tick? [action mechanisms]
  • How are individuals formed and constituted? [social constitution mechanisms] 
  • What are the institutional and organizational factors that motivate and constrain individuals' choices? [institutional mechanisms on individual behavior]
  • How do individual agents' actions aggregate to higher-level social patterns? [aggregative mechanisms]
  • How do macro-level social structures influence other macro-level social structures? [meso-meso mechanisms]
These questions imply eight "zones" of activity for social mechanisms:
0 neuro-cognitive system
1 action and deliberation
2 identity formation
3 institutional influence on individuals
4 aggregation from individual to social
5 social action and collective action
6 hierarchy and control
7 meso-meso influences
I have represented these eight zones in the messy figure above.

This is a "functional" taxonomy of mechanisms; it classifies social mechanisms according to what they do. A different scheme would be to group mechanisms according to how they work: rational choice, game theoretic, social network, sub-cognitive, group dynamics, collective action, coercion, epidemiological, .... If we adopted both schemes, then we would arrive at a two-dimensional classification including both functional location and mode of activity.

So how does this scheme mesh with the mechanisms singled out in my earlier post? Here is a grouping of the mechanisms included in the catalogue presented there according to the current scheme:

0.00           NEURO-COGNITIVE SYSTEM

1.00           ACTION AND DELIBERATION

1.01              Altruistic enforcement
1.02              Conditional altruism [individuals reason on the basis of conditional willingness to act in support of collective good]
1.03              Reciprocity [individuals act for other individuals in expectation of return favors in future; successful only in specific social conditions]
1.04              Social appropriation
1.05              Stereotype threat

2.00           IDENTITY FORMATION

2.01                Boundary activation
2.02                Certification
2.03                norm inculcation

3.00           INSTITUTIONAL INFLUENCE ON INDIVIDUALS

3.01              Audit and accounting [organization establishes rules and roles to oversee compliance with policies]
3.02              Broadcast
3.04              Contract
3.05              Employee training [organization establishes training for employees to encourage or create desired forms of behavior]
3.06              Framing [leaders communicate issues and demands to followers in favorable ways]
3.09              Morale building
3.10              Norms [normative community influences individual action and choice]
3.11              Selective benefits [organization or club offers benefits to those who contribute to joint actions]
3.12              Selective coercion [group, leaders! or members impose sanctions on members to enforce compliance with group rules]
3.14              Supervision
3.15                Regulatory organizations

4.00           AGGREGATION OF INDIVIDUAL TO SOCIAL

4.02                Auction
4.03                Cyclical voting
4.04                Democratic decision making
4.05                Erosion
4.06                Flash trading
4.07                Imitation
4.08                Influence peddling
4.09                Interlocking mobilization
4.10                Interpersonal network
4.01                Market
4.13                Market for lemons
4.15                Producers' control
4.16                Rumor
4.17                Subliminal transmission

5.00           SOCIAL ACTION AND COLLECTIVE ACTION

5.01                Agenda setting
5.01                Brokerage [leaders negotiate coordinated action with other groups/leaders]
5.02                Convention [individuals coordinate action around conspicuous patterns or rules]
5.03                Coordinated action
5.04                Escalation [group and leaders promote broader action alliance or elevate level of action]
5.05                Free rider behavior
5.06                Prisoners' dilemma [result of strategic action among two or more players]
5.07                Log rolling
5.08                Person-to-person transmission

6.00           HIERARCHY AND CONTROL

6.01                Control of communications systems
6.02                Deception
6.03                Informers
6.03                Charisma
6.04                Propaganda
6.05                Secret police files
6.06                Spectacular use of force
6.07                Leadership
6.08                Ministry direction

7.00           MESO-MESO INFLUENCE

7.01                Competition for power [groups and leaders take steps to improve their power position]
7.02                Diffusion [example of collective action spreads to other locales and groups and issues]
7.03                Non-linear effects within social networks
7.04                Overlapping systems of authority (Brenner)
7.05                Transport networks
7.06                Soft budget constraint

Interestingly enough, here is a rather similar diagram (in structure, anyway) that is provided by Thornton, Ocacio, and Lounsbury in their presentation of the field of "institutional logics" (The Institutional Logics Perspective: A New Approach to Culture, Structure and Process):

If we understand each of the arrows as a group of mechanisms, extending influence from one zone to the other, the diagram is very similar in its logic to the one provided above.