How organizations adapt

Organizations do things; they depend upon the coordinated efforts of numerous individuals; and they exist in environments that affect their ongoing success or failure. Moreover, organizations are to some extent plastic: the practices and rules that make them up can change over time. Sometimes these changes happen as the result of deliberate design choices by individuals inside or outside the organization; so a manager may alter the rules through which decisions are made about hiring new staff in order to improve the quality of work. And sometimes they happen through gradual processes over time that no one is specifically aware of. The question arises, then, whether organizations evolve toward higher functioning based on the signals from the environments in which they live; or on the contrary, whether organizational change is stochastic, without a gradient of change towards more effective functioning? Do changes within an organization add up over time to improved functioning? What kinds of social mechanisms might bring about such an outcome?

One way of addressing this topic is to consider organizations as mid-level social entities that are potentially capable of adaptation and learning. An organization has identifiable internal processes of functioning as well as a delineated boundary of activity. It has a degree of control over its functioning. And it is situated in an environment that signals differential success/failure through a variety of means (profitability, success in gaining adherents, improvement in market share, number of patents issued, ...). So the environment responds favorably or unfavorably, and change occurs.

Is there anything in this specification of the structure, composition, and environmental location of an organization that suggests the possibility or likelihood of adaptation over time in the direction of improvement of some measure of organizational success? Do institutions and organizations get better as a result of their interactions with their environments and their internal structure and actors?

There are a few possible social mechanisms that would support the possibility of adaptation towards higher functioning. One is the fact that purposive agents are involved in maintaining and changing institutional practices. Those agents are capable of perceiving inefficiencies and potential gains from innovation, and are sometimes in a position to introduce appropriate innovations. This is true at various levels within an organization, from the supervisor of a custodial staff to a vice president for marketing to a CEO. If the incentives presented to these agents are aligned with the important needs of the organization, then we can expect that they will introduce innovations that enhance functioning. So one mechanism through which we might expect that organizations will get better over time is the fact that some agents within an organization have the knowledge and power necessary to enact changes that will improve performance, and they sometimes have an interest in doing so. In other words, there is a degree of intelligent intentionality within an organization that might work in favor of enhancement.

This line of thought should not be over-emphasized, however, because there are competing forces and interests within most organizations. Previous posts have focused on current organizational theory based on the idea of a "strategic action field" of insiders and outsiders who determine the activities of the organization (Fligstein and McAdam, Crozier; link, link). This framework suggests that the structure and functioning of an organization is not wholly determined by a single intelligent actor ("the founder"), but is rather the temporally extended result of interactions among actors in the pursuit of diverse aims. This heterogeneity of purposive actions by actors within an institution means that the direction of change is indeterminate; it is possible that the coalitions that form will bring about positive change, but the reverse is possible as well.

And in fact, many authors and participants have pointed out that it is often enough not the case that the agents' interests are aligned with the priorities and needs of the organization. Jack Knight offers persuasive critique of the idea that organizations and institutions tend to increase in their ability to provide collective benefits in Institutions and Social Conflict. CEOs who have a financial interest in a rapid stock price increase may take steps that worsen functioning for shortterm market gain; supervisors may avoid work-flow innovations because they don't want the headache of an extended change process; vice presidents may deny information to other divisions in order to enhance appreciation of the efforts of their own division. Here is a short description from Knight's book of the way that institutional adjustment occurs as a result of conflict among players of unequal powers:

Individual bargaining is resolved by the commitments of those who enjoy a relative advantage in substantive resources. Through a series of interactions with various members of the group, actors with similar resources establish a pattern of successful action in a particular type of interaction. As others recognize that they are interacting with one of the actors who possess these resources, they adjust their strategies to achieve their best outcome given the anticipated commitments of others. Over time rational actors continue to adjust their strategies until an equilibrium is reached. As this becomes recognized as the socially expected combination of equilibrium strategies, a self-enforcing social institution is established. (Knight, 143)

A very different possible mechanism is unit selection, where more successful innovations or firms survive and less successful innovations and firms fail. This is the premise of the evolutionary theory of the firm (Nelson and Winter, An Evolutionary Theory of Economic Change). In a competitive market, firms with low internal efficiency will have a difficult time competing on price with more efficient firms; so these low-efficiency firms will go out of business occasionally. Here the question of "units of selection" arises: is it firms over which selection operates, or is it lower-level innovations that are the object of selection?

Geoffrey Hodgson provides a thoughtful review of this set of theories here, part of what he calls "competence-based theories of the firm". Here is Hodgson's diagram of the relationships that exist among several different approaches to study of the firm.

The market mechanism does not work very well as a selection mechanism for some important categories of organizations -- government agencies, legislative systems, or non-profit organizations. This is so, because the criterion of selection is "profitability / efficiency within a competitive market"; and government and non-profit organizations are not importantly subject to the workings of a market.

In short, the answer to the fundamental question here is mixed. There are factors that unquestionably work to enhance effectiveness in an organization. But these factors are weak and defeasible, and the countervailing factors (internal conflict, divided interests of actors, slackness of corporate marketplace) leave open the possibility that institutions change but they do not evolve in a consistent direction. And the glaring dysfunctions that have afflicted many organizations, both corporate and governmental, make this conclusion even more persuasive. Perhaps what demands explanation is the rare case where an organization achieves a high level of effectiveness and consistency in its actions, rather than the many cases that come to mind of dysfunctional organizational activity.

(The examples of organizational dysfunction that come to mind are many -- the failures of nuclear regulation of the civilian nuclear industry (Perrow, The Next Catastrophe: Reducing Our Vulnerabilities to Natural, Industrial, and Terrorist Disasters); the failure of US anti-submarine warfare in World War II (Cohen, Military Misfortunes: The Anatomy of Failure in War); and the failure of chemical companies to ensure safe operations of their plants (Shrivastava, Bhopal: Anatomy of Crisis). Here is an earlier post that addresses some of these examples; link. And here are several earlier posts on the topic of institutional change and organizational behavior; link, link.) 

An evolutionary view of research frameworks

It was noted in prior posts that there is a great diversity of research frameworks in the social sciences, and that there is much to be gained by attempting to understand the social processes through which these frameworks change and develop over time (link, link).

Is it possible to get more specific about how the various research frameworks are related to each other, how similar or different they are, and how far apart they are in the space of the total universe of scientific research frameworks?

Here is an intriguing idea. We might think of a research framework as the compound of a set of "genes" in the form of a collection of ideas and practices through which scientists approach their studies of the world. And we might define the similarity or difference between two research frameworks in the way that geneticists define the similarity of two mammalian species, in terms of the degree of shared genetic material they possess.

On this approach, the research framework (methodology) constitutes a "code" which the young scientist learns through advanced training in the discipline. The resulting body of theory and research findings is the phenotype that results from the expression of the genotype through the activities of individuals and groups of scientists within the social and political environment of the research community.

The genotype of a methodology might consist of items like these:

• ontological assumptions
• investigatory strategies
• practices of scientific collaboration
• experimental designs
• ideas about good explanation
• procedures for evaluating evidence

On this view, scientific innovation takes place when researchers modify one or more elements of the code -- background ontological beliefs, investigative practices, modes of confirmation, regulative ideas about theory and explanation. A given innovation may confer greater efficacy on the research framework by leading to more discoveries, more publications, or more funding.

By invoking the ideas of genotypes, phenotypes, and ecologies into the domain of research methodologies it is tempting to consider whether other aspects of the evolutionary paradigm are applicable as well, including the idea of evolution through selection.

Key here is the question of the mechanisms of fitness and selection that might be at work in the field of science. Do methodologies compete for survival and reproduction? Are there selection pressures at work in the domain of epistemology? Do scientific research frameworks "evolve" through small adjustments at the "gene" level (components of methodology at the level of theory and practice)? If so, do those selection processes lead to the evolution of methodologies better suited to discovering truth, or do they lead instead to methods that better serve the proximate interests of researchers and micro research communities?

One might argue that the two possibilities -- selection pressure and veridicality -- converge. Researchers use and refine methodologies based on their effort to generate publications, influence, and funding. One methodology is more "fit" than another insofar as it contributes to comparative advantage in these related outcomes. But methodologies generate influence, publication, and funding in proportion to the collective judgment of the expert community that they are creating new insight and more truthful representations of the world. So selection processes lead to greater veridicality.

This would be an agreeable outcome -- the concrete practice of science leads generally to greater levels of truthful representation of the natural and social world. But is it plausible? Or does the history and sociology of science suggest that the factors that select for research methodologies are less epistemic and more situational or political? Does the process of science favor innovations that align with received views? Do scientific careers depend more on non epistemic factors than epistemic qualities? Does the process of science favor politically acceptable findings ("fracking is harmless" rather than "fracking causes micro-seismic activity")? Are there contrarian "predator" practices at foot that actively contrive to push scientific findings away from truth (climate deniers, smoking industry advocates; Naomi Oreskes, Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming)?

There are visible differences between biological evolution and the evolution of complexes of ideas and methods, of course. Biological evolution takes place through random variation at the genetic level, whereas theories and methods are subject to design by the scientists who develop them. Functional adaptation is the result of a blind process of selection in the biological world, whereas it is an intended consequence in the realm of theories and methods. And it is possible to define ecological fitness more precisely and mathematically in the biological realm than it is in the realm of science and culture.

It will be noted that this approach has a lot in common with the evolutionary approach to economics and the evolution of firms. A key text in this approach is Richard Nelson and Sidney Winter, An Evolutionary Theory of Economic Change. Richard Dawkins explored a similar view in The Selfish Gene when he considered "memes" as cultural counterparts to genes in the biological world.

The moral sentiments

One of Adam Smith's contributions to the study of philosophical ethics is his book, The Theory of Moral Sentiments. It is an interesting work, one part descriptive moral psychology, one part theory of the emotions.  Here is the opening paragraph (link):

How selfish soever man may be supposed, there are evidently some principles in his nature, which interest him in the fortune of others, and render their happiness necessary to him, though he derives nothing from it except the pleasure of seeing it. Of this kind is pity or compassion, the emotion which we feel for the misery of others, when we either see it, or are made to conceive it in a very lively manner. That we often derive sorrow from the sorrow of others, is a matter of fact too obvious to require any instances to prove it; for this sentiment, like all the other original passions of human nature, is by no means confined to the virtuous and humane, though they perhaps may feel it with the most exquisite sensibility. The greatest ruffian, the most hardened violator of the laws of society, is not altogether without it.

So Smith asserts as a matter of empirical fact that there are common moral emotions and feelings -- sympathy, pity, compassion -- that underlie human social and moral behavior.  And the most basic kinds of morally motivated behavior -- altruism in particular -- are explained by the workings of these natural emotions of empathy with other human beings.  So Smith posed a fundamental question: is there an innate human moral psychology, beyond the reach of training and teaching, that accounts for our willingness to give to others and sometimes sacrifice important interests for the good of others?  Why do firemen rush into the highly dangerous environment of a large fire in order to rescue the people inside?

Now fast-forward to the post-Darwinian world; look at the human organism from the point of view of the study of primate behavior; and ask this key question: Is there an evolutionary basis for social behaviors? Are there emotions supporting cooperation that were selected for through our evolutionary history? Is a moral capacity hardwired?

Philosophers have treated this question in the past.  Allan Gibbard's Wise Choices, Apt Feelings: A Theory of Normative Judgment is a particularly good example. Here is how Gibbard describes the situation.

Consider now human beings evolving in hunting-gathering societies.  We could expect them to face an abundance of human bargaining situations, involving mutual aid, personal property, mates, territory, use of housing, and the like.  Human bargaining situations tend to be evolutionary bargaining situations.  Human goals tend toward biological fitness, toward reproduction.  The point is not, of course, that a person's sole goal is to maximize his reproduction; few if any people have that as a goal at all.  Rather, the point concerns propensities to develop goals.  Those propensities that conferred greatest fitness were selected; hence in a hunting-gathering society, people tended to want the various things it was fitness-enhancing for them to want.  Conditions of primitive human life must have required intricate coordination--both of the simple cooperative kinds involved, say, in meeting a person, and of the kind required for bargaining problems to yield mutually beneficial outcomes. Propensities well coordinated with the propensities of others would have been fitness-enhancing, and so we may view a vast array of human propensities as coordinating devices.  Our emotional propensities, I suggest, are largely the results of these selection pressures, and so are our normative capacities. (67)

One of Gibbard's key points is an analytical one. He argues against the idea of there being specific moral content, ethical principles, or moral emotions that are embodied in the central nervous system (CNS) as a result of variation and selection. Instead, he argues for there being a hardwired set of more abstract capacities that have CNS reality and selection advantage: the ability to learn a norm and to act in accordance with it.  (Richard Joyce makes a similar point: "Evolutionary psychology does not claim that observable human behavior is adaptive, but rather that it is produced by psychological mechanisms that are adaptations.  The output of an adaptation need not be adaptive" (5).)

This is the part that seems counter-intuitive from a simple Darwinian point of view. Wouldn't an organism possessing a genetically determined disposition to act contrary to its mortal interests almost necessarily have less reproductive success? So shouldn't such a gene quickly lose out to a more opportunistic alternative? Gibbard considers the evolutionary arguments surrounding the topic of altruism (including Richard Dawkins' Selfish Gene), and concludes -- not necessarily.  It is possible to mount an evolutionary argument that establishes the fitness-enhancing characteristics of some specific kinds of altruistic behavior.

So what does the current research on this topic add to what we already knew?  And, can we draw any interesting connections back to the venerable Smith?

In fact, there seems to be a new surge of interest in the topic.  A number of philosophers and psychologists are now interested in treating moral psychology as an empirical question, and they are interested in working back to the evolutionary environment in which these human capacities emerged.   (For example, Richard Joyce, The Evolution of Morality and Walter Sinnott-Armstrong, ed., Moral Psychology, Volume 1: The Evolution of Morality: Adaptations and Innateness.)  Particularly interesting is research by Michael Tomasello and his collaborators.  Tomasello is the co-director of the Max Planck Institute for Evolutionary Anthropology.  He argues that human beings are hardwired for cooperation, empathy, and social intensionality in a very interesting recent book, Why We Cooperate.  A great deal of his research has to do with experiments and observations of human children (9-24 months) and of young non-human primates.  He finds, essentially, that infants and children display a range of behaviors that seem to reveal a natural readiness for altruism, sharing, coordination, and eventually following of norms.  "I only propose that the kinds of collaborative activities in which young children today engage are the natural cradle of social norms of the cooperative variety.  This is because they contain the seeds of the two key ingredients" (89-90).  He presents a range of experimental data supporting these ideas:

• Human infants have a pre-cultural disposition to be helpful and empathetic (12-14 months) 
• Human toddlers adjust their cooperative and normative behavior to be more attentive to the behavior of others: generous to the generous and not to the ungenerous. 
• Human infants and toddlers have a precultural disposition to absorb and enforce norms. 
• The emotions of guilt and shame to be hardwired to conformance to norms. 
• Infants appear to take a "we" intentional stance without learning.  They are able to quickly figure out what another agent is trying to do.
• Chimps differ from human infants in virtually each of these areas. 

Here is a particularly interesting piece of evidence that Tomasello offers in support of the idea that human evolution was shaped by selection pressures that favored social coordination: the whites of the eyes in the human being.  Almost all non-human species have eyes that are primarily dark; whereas human eyes feature a large and conspicuous circle of white (the sclera).  The whites of the eyes permit an observer to determine what another individual is looking at -- allowing human individuals to achieve a substantially greater degree of shared attention and coordination.  "My team has argued that advertising my eye direction for all to see could only have evolved in a cooperative social environment in which others were not likely to exploit it to my detriment" (76).

So does this recent work on the evolutionary basis of moral emotions have anything to do with Smith and the moral sentiments?  What the two bodies of thought have in common is the idea that there is a psychological foundation to moral behavior, cooperation, altruism, and helping.  Pure maximizing rationality doesn't get you to "helping"; rather, there needs to be some psychological impulse to improve things for the other person.  Where evolutionary psychology differs from Smith is precisely in the nature of the explanation that is offered for this moral psychology; we have the advantage of having a pretty good idea of how natural selection works on biological traits, and we are therefore in a better position than Smith was to explain why human beings possess moral sentiments.  What we cannot yet answer is the question of the nature of the mechanism at the level of the central nervous system or the cognitive system, of how these moral sentiments are embodied in the human organism.

(It is interesting to contrast this line of argument with that of Tom Nagel in The Possibility of Altruism.  Nagel argues against the moral psychology of Hume -- very similar to that of Smith -- and argues that altruism is actually a feature of rationality.  We behave altruistically, fundamentally, because we have a rational representation of the reality of the external world and of other persons; and to recognize the reality of another person is immediately to have a reason to help the other person.  So no "motor" of moral emotion is needed in order to explain altruistic behavior.  On this approach, we don't need to postulate moral sentiments to explain moral behavior; all we need is a rich conception of practical rationality.)

Social change and natural selection?

Are there any valid analogies between the evolution of species and various kinds of social change?

Here's the basic argument for the evolution of species in Darwinian theory: individual organisms transmit traits to offspring; there is a low but positive rate of mutation of traits; there is no inheritance of acquired characteristics; traits influence the overall reproductive success of the organism (phenotype); organisms with a higher level of reproductive fitness proliferate their traits more frequently; and, in time, the novel trait prevails within the population. Over long periods of time, the species becomes more adapted to the particular features of its environment through the accumulation of fitness-enhancing traits. An important conceptual issue that needed resolution in the development of modern evolutionary theory is the unit of selection: is it the individual, the gene, the kinship group, the extended group, or the species as a whole? For the mechanism of natural selection to work, we need to identify the individual as the unit of selection, not the extended group, because the mechanism depends on changing the frequency of a given gene within the population as a whole. Group selection is not a valid Darwinian mechanism of evolution. (This topic gained greater clarity through debates over altruism in animal behavior; G. C. Williams, Adaptation and Natural Selection.)

In general, I don't find that the analogy of natural selection has proven very useful as a basis for explaining social change. "Social evolution" appears to be a misnomer. In order for the analogy to work as a possible explanation of any kind of social change, we would need to be able to identify several specific things: the unit of selection; the features or traits that constitute the "phenotype" of the unit; the substrate in which the coding for traits exists (genotype); the mechanism of transmission by which "parent" transmits the trait to "offspring"; and the connection that is postulated between current traits and future reproductive success. We would also need a theory about the mechanism of mutation at the level of the coding of social traits, and -- if we are to stick with Darwinian evolutionary theory -- we would need to assure ourselves that acquired traits are not transmitted. Finally, the social situation would need to be one in which there are numerous independent "individual units" whose independent reproductive success over time leads to a change in the frequency of traits over time.

Partly the unsuitability of the analogy for explaining social change derives from the problem of the unit of selection: what are the units over which differential rates of survival are to be measured? The earliest social borrowers from Darwin chose to compare whole societies. The idea was that certain characteristics made one society more viable than another, and over time this advantage would lead to the dominance of the more fit society. But this isn't really a natural selection argument at all; there aren't enough societies, and the concepts of reproduction and fitness are not well defined.

A second sort of effort tried to draw a more exact analogy by singling out a kind of social organization which existed in multiple "phenotypes" in society. For example, evolutionary economics considers models in which the unit of selection is the business firm; the measure of reproductive success is profitability; and "reproduction" is simply survival from one period to the next. Putting the analogy simply: some firms have traits that make them more efficient and effective; these firms will be more profitable as a result; they will grow larger and stronger; and they will drive weaker competitors out of business. The market represents the selection mechanism. Over time, then, the superior business organization will become the standard -- until a next round of innovation occurs.

But this analogy isn't really a good one either, for a host of reasons. A firm is constituted by the ideas and behaviors of the individuals who make it up; there is no underlying genotype; participants actively redesign the traits of the organization (why else would there be consulting firms?); there is no real analogy to the transmission of traits through reproduction; and there would certainly be the possibility of transmission of acquired traits. So "evolution" towards more efficient or profitable forms of business organization seems to be simply a metaphor, not a theory of social change grounded in a mechanism of social reproduction and selection. If anything, the theory that seems to best describe the process identified here is "design, build, redesign" -- create a form of organization, observe its functioning, and modify the organization so as to achieve higher efficiency. In other words -- not natural selection but intelligent design.

A recent effort to use the theoretical machinery of natural selection as a scientific basis for social explanation is a paper by Paul Ehrlich, director of Stanford University's Center for Conservation Biology. Ehrlich uses available data about the timing and places of innovation in Polynesian canoe design to make an argument about cultural evolution. In an interview with Wired he puts his finding this way: "Features of culture that are tested against the environment will evolve at different rates from those not tested against the environment." Only the abstract is available online, so the full text isn't available; but it would appear that the argument goes something like this. The unit of selection is the local canoe-building tradition; the "genotype" is the hypothetical blueprint of the canoe this tradition builds; mutations are local innovations in design; and the winners are those canoe designs that are most seaworthy -- thus permitting their builders to populate more islands and build more canoes.

As a theory of technological change, I'm not convinced that the "selection" mechanism is doing much work here. Natural selection is a blind process; the finch doesn't notice that a longer beak would be a big advantage in digging out bugs from the bark. Rather, the "long beak" mutation occurs, and the long-beak finches out-reproduce the short-beak finches. In the canoe case, on the other hand, it is entirely credible that the skilled canoe builders are themselves observing and applying the test of nature; they are correcting the design so as to address the seaworthiness problems involved in the current iteration. So, as has turned out to be the case in so many other instances of social evolutionary thinking, the evolutionary metaphor doesn't seem to help much.

This isn't to say that it isn't possible to explain the diffusion of technical innovations over time and place as some kind of feedback process. It is certainly true that certain design traditions out-perform their rivals, and it is likely that these winners will eventually produce all the canoes within a certain domain (or steam engines, or silicon chips). Rather, the point is an explanatory one. We need to look for the concrete social mechanisms through which artifacts and traditions of skill change over time in response to the features of the environment and the society. And these mechanisms are almost certainly more intentional than the blind process of mutation and selection would suggest.