Deficiencies of practical rationality in organizations

Suppose we are willing to take seriously the idea that organizations possess a kind of intentionality -- beliefs, goals, and purposive actions -- and suppose that we believe that the microfoundations of these quasi-intentional states depend on the workings of individual purposive actors within specific sets of relations, incentives, and practices. How does the resulting form of "bureaucratic intelligence" compare with human thought and action?

There is a major set of differences between organizational "intelligence" and human intelligence that turn on the unity of human action compared to the fundamental disunity of organizational action. An individual human being gathers a set of beliefs about a situation, reflects on a range of possible actions, and chooses a line of action designed to bring about his/her goals. An organization is disjointed in each of these activities. The belief-setting part of an organization usually consists of multiple separate processes culminating in an amalgamated set of beliefs or representations. And this amalgamation often reflects deep differences in perspective and method across various sub-departments. (Consider inputs into an international crisis incorporating assessments from intelligence, military, and trade specialists.)

Second, individual intentionality possess a substantial degree of practical autonomy. The individual assesses and adopts the set of beliefs that seem best to him or her in current circumstances. The organization in its belief-acquisition is subject to conflicting interests, both internal and external, that bias the belief set in one direction or the other. (This is the central thrust of experts on science policy like Naomi Oreskes.) The organization is not autonomous in its belief formation processes.

Third, an individual's actions have a reasonable level of consistency and coherence over time. The individual seeks to avoid being self-defeating by doing X and Y while knowing that X undercuts Y. An organization is entirely capable of pursuing a suite of actions which embody exactly this kind of inconsistency, precisely because the actions chosen are the result of multiple disagreeing sub-agencies and officers.

Fourth, we have some reason to expect a degree of stability in the goals and values that underlie actions by an individual. But organizations, exactly because their behavior is a joint product of sub-agents with conflicting plans and goals, are entirely capable of rapid change of goals and values. Deepening this instability is the fluctuating powers and interests of external stakeholders who apply pressure for different values and goals over time.

Finally, human thinkers are potentially epistemic thinkers -- they are at least potentially capable of following disciplines of analysis, reasoning, and evidence in their practical engagement with the world. By contrast, because of the influence of interests, both internal and external, organizations are perpetually subject to the distortion of belief, intention, and implementation by actors who have an interest in the outcome of the project. And organizations have little ability to apply rational rational standards to their processes of belief, intention, and implementation formation. Organizational intentionality lacks overriding rational control.

Consider more briefly the topic of action. Human actors suffer various deficiencies of performance when it comes to purposive action, including weakness of the will and self deception. But organizations are altogether less capable of effectively mounting the steps needed to fully implement a plan or a complicated policy or action. This is because of the looseness of linkages that exist between executive and agent within an organization, the perennial possibility of principal-agent problems, and the potential interference with performance created by interested parties outside the organization.

This line of thought suggests that organizational lack "unity of apperception and intention". There are multiple levels and zones of intention formation, and much of this plurality persists throughout real processes of organizational thinking. And this disunity affects both belief, intention and action. Organizations are not univocal at any point. Belief formation, intention formation, and action remain fragmented and multivocal.

These observations are somewhat parallel to the paradoxes of social choice and various voting systems governing a social choice function. Kenneth Arrow demonstrated it is impossible to design a voting system that guarantees consistency of choice by a group of individual consistency voters. The analogy here is the idea that there is no organizational design possible that guarantees a high degree of consistency and rationality in large organizational decision processes at any stage of quasi-intentionality, including belief acquisition, policy formulation, and policy implementation. 

The place for thick theories of the actor in philosophy

image: Bruegel, The Dutch Proverbs (1559)

When philosophers of the social sciences take seriously the importance of individual action within the social realm, we often look in the direction of methodological individualism and the methods of "aggregation dynamics". That is, agent-centered theorists are usually interested in finding ways of climbing the upward strut of Coleman's boat through efforts at modeling the interactive dynamics of purposive individuals and the social outcomes they produce. This leads to an interest in applying microeconomics, game theory, or agent-based modeling as ways of discovering the aggregate consequences of a certain theory of the actor (purposive, calculating, strategic rationality). We then get a ready basis for accounting for causal relations in the social world; the medium of causal powers and effects is the collection of purposive actors who live in social relationships and institutions with a fairly homogeneous form of agency.

This is a perfectly valid way of thinking about social causation and explanation. But the thrust of the argument for thick descriptions of actors, coming from microsociologists, ethnomethodologists, and historical sociologists, is that the abstractions associated with thin theories of the actor (goal-directed behavior driven by rational-self-interest) are often inadequate for understanding real social settings. If we attach weight to sociologists like Goffman and Garfinkel, some of the most interesting stuff is happening along the bottom edge of Coleman's boat -- the interactions among socially situated individuals. So how should we think about the challenge of incorporating a richer theory of the actor into the project of supporting an adequate set of ideas about social inquiry and social explanation?

One approach is to simply acknowledge the scientific relevance and importance of research into the mentality of real social actors. This approach accepts the point that we cannot always make use of the very sparse assumptions of thin theories of the actor if we want to understand social phenomena like the politics of hate, the rise of liberal democracy, or the outbreak of ethnic violence. We can then attempt to address the theoretical and methodological problems associated with research into more nuanced understanding of historically and socially situated persons in specific circumstances involving the phenomena of interest. We can give attention to fields like cultural sociology and ethnography and attempt to offer support for those research efforts. This approach also permits the possibility of attempting to formulate a conception of social explanation that fits thick theories of the actor.

This approach seems to lead most naturally to a conception of explanation that is more interpretive than causal, and it suggests that the hard work of social research will go into the effort to find evidence permitting the researcher to form a theory of the attitudes, beliefs, and mental frameworks of the actors involved in the social setting of interest. The example of Robert Darnton's study of "The Great Cat Massacre" illustrates the value and difficulty of this kind of inquiry (link). And it highlights the crucial role that concrete historical and documentary evidence play in the effort (link). At the same time, the explanations offered are almost inevitably particular to the case, not generalizable.

Is there an approach to social explanation that makes use of a thick theory of the actor but nonetheless aspires to providing largescale social explanations? Can thick theories of the actor, and rich accounts of agency in specific circumstances, be incorporated into causal theories of specific kinds of social organization and change? Can we imagine a parallel masterpiece to Coleman's Foundations of Social Theory, which incorporates the nuances of thick sociology and points towards a generalizing sociology?

Yes and no. Yes, in that important and large-scale works of comparative historical sociology depend directly on analysis of the thick mentalities of the actors who made up great events -- e.g. Mann on fascism (demobilized soldiers), Steinmetz on German colonialism (professional administrators), Frank Dobbin on French technology planning (technocrats), or Charles Sabel on Italian mechanical culture (machinists versus engineers). And this kind of social research depends upon its own kind of generalization -- the claim to identify a cultural type that was current in a given population at a certain time. This is the project of discovering a historically concrete mentalité (link). But no, if we think the primary mode of social explanation takes the form of system models demonstrating the genealogy of this social characteristic or that.

This sounds a bit like the heart of the methodenstreit of the last century, between historicists and nomological theorists. Does the social world admit of generalizing explanations (nomothetic), or is social explanation best understood as particular and historically situated (idiographic)? Fortunately we are not forced to choose. Both kinds of explanation are possible in the social realm, and some problems are more amenable to one approach or the other. Only the view that insists on the unity of science find this dilemma unacceptable. But for a methodological pluralist, this is a perfectly agreeable state of affairs.

The research university

Where do new ideas, new technologies, and new ways of thinking about the world come from in a modern society? Since World War II the answer to this question has largely been found in research universities. Research universities are doctoral institutions that employ professors who are advanced academic experts in a variety of fields and that expend significant amounts of external funds in support of ongoing research. Given the importance of innovation and new ideas in the knowledge economy of the twenty-first century, it is very important to understand the dynamics of research universities, and to understand factors that make them more or less productive in achieving new knowledge. And, crucially, we need to understand how public policy can enhance the effectiveness of the university research enterprise for the benefit of the whole of society.

Jason Owen-Smith's recent Research Universities and the Public Good: Discovery for an Uncertain Future is a very welcome and insightful contribution to better understanding this topic. Owen-Smith is a sociology professor at the University of Michigan (itself a major research university with over 1.5 billion dollars in annual research funding), and he brings to his task some of the most insightful ideas currently transforming the field of organizational studies.

Owen-Smith analyzes research universities (RU) in terms of three fundamental ideas. RUs serves as source, anchor, and hub for the generation of innovations and new ideas in a vast range of fields, from the humanities to basic science to engineering and medicine. And he believes that this triple function makes research universities virtually unique among American (or global) knowledge-producing organizations, including corporate and government laboratories (33).

The idea of the university as a source is fairly obvious: it is the idea that universities create and disseminate new knowledge in a very wide range of fields. Sometimes that knowledge is of interest to a hundred people worldwide; and sometimes it results in the creation of genuinely transformative technologies and methods. The idea of the university as "anchor" refers largely to the stability that research universities offer the knowledge enterprise. Another aspect of the idea of the university as an anchor is the fact that it helps to create a public infrastructure that encourages other kinds of innovation in the region that it serves -- much as an anchor tenant helps to bring potential customers to smaller stores in a shopping mall. Unlike other knowledge-centered organizations like private research labs or federal laboratories, universities have a diverse portfolio of activity that confers a very high level of stability over time. This is a large asset for the country as a whole. It is also frequently an asset for the city or region in which it is located.

The idea of the university as a hub is perhaps the most innovative perspective offered here. The idea of a hub is a network concept. A hub is a node that links individuals and centers to each other in ways that transcend local organizational charts. And the power of a hub, and the networks that it joins, is that it facilitates the exchange of information and ideas and creates the possibility of new forms of cooperation and collaboration. Here the idea is that a research university is a place where researchers form working relationships, both on campus and in national networks of affiliation. And the density and configuration of these relationships serve to facilitate communication and diffusion of new ideas and approaches to a given problem, with the result that progress is more rapid. O-S makes use of Peter Galison's treatment of the simultaneous discovery of the relativity of time measurement by Einstein and Poincaré in Einstein's Clocks and Poincaré's Maps: Empires of Time.  Galison shows that Einstein and Poincaré were both involved in extensive intellectual networks that were quite relevant to their discoveries; but that their innovations had substantially different effects because of differences in those networks. Owen-Smith believes that these differences are very relevant in the workings of modern RUs in the United States as well. (See also Galison's Image and Logic: A Material Culture of Microphysics.)

Radical discoveries like the theory of special relativity are exceptionally rare, but the conditions that gave rise to them should also enable less radical insights. Imagining universities as organizational scaffolds for a complex collaboration networks and focal point where flows of ideas, people, and problems come together offers a systematic way to assess the potential for innovation and novelty as well as for multiple discoveries. (p. 15)

Treating a complex and interdependent social process that occurs across relatively long time scales as if it had certain needs, short time frames, and clear returns is not just incorrect, it's destructive. The kinds of simple rules I suggested earlier represent what organizational theorist James March called "superstitious learning." They were akin to arguing that because many successful Silicon Valley firms were founded in garages, economic growth is a simple matter of building more garages. (25)

Rather, O-S demonstrates in the case of the development of the key discoveries that led to the establishment of Google, the pathway was long, complex, and heavily dependent on social networks of scientists, funders, entrepreneurs, graduate students, and federal agencies.

A key observation in O-S's narrative at numerous points is the futility -- perhaps even harmfulness -- of attempting to harness university research to specific, quantifiable economic or political goals. The idea of selecting university research and teaching programs on the basis of their ROI relative to economic goals is, according to O-S, deeply futile. The extended example he offers of the research that led to the establishment of Google as a company and a search engine illustrates this point very compellingly: much of the foundational research that made the search algorithms possible had the look of entirely non-pragmatic or utilitarian knowledge production at the time it was funded (chapter 1). (The development of the smart phone has a similar history; 63.) Philosophy, art history, and social theory can be as important to the overall success of the research enterprise as more intentionally directed areas of research (electrical engineering, genetic research, autonomous vehicle design). His discussion of Wisconsin Governor Scott Walker's effort to revise the mission statement of the University of Wisconsin is exemplary (45 ff.).

Contra Governor Walker, the value of the university is found not in its ability to respond to immediate needs but in an expectation that joining systematic inquiry and education will result in people and ideas that reach beyond local, sometimes parochial, concerns. (46-47)

Also interesting is O-S's discussion of the functionality of the extreme decentralization that is typical of most large research universities. In general O-S regards this decentralization as a positive thing, leading to greater independence for researchers and research teams and permitting higher levels of innovation and productive collaboration. In fact, O-S appears to believe that decentralization is a critical factor in the success of the research university as source, anchor, and hub in the creation of new knowledge.

The competition and collaboration enabled by decentralized organization, the pluralism and tension created when missions and fields collide, and the complex networks that emerge from knowledge work make universities sources by enabling them to produce new things on an ongoing basis. Their institutional and physical stability prevents them from succumbing to either internal strife or the kinds of 'creative destruction' that economist Joseph Schumpeter took to be a fundamental result of innovation under capitalism. (61)

O-S's discussion of the micro-processes of discovery is particularly interesting (chapter 3). He makes a sustained attempt to dissect the interactive, networked ways in which multiple problems, methods, and perspectives occasionally come together to solve an important problem or develop a novel idea or technology. In O'S's telling of the story, the existence of intellectual and scientific networks is crucial to the fecundity of these processes in and around research universities.

This is an important book and one that merits close reading. Nothing could be more critical to our future than the steady discovery of new ideas and solutions. Research universities have shown themselves to be uniquely powerful engines for discovery and dissemination of new knowledge. But the rapid decline of public appreciation of universities presents a serious risk to the continued vitality of the university-based knowledge sector. The most important contribution O-S has made here, in my reading, is the detailed work he has done to give exposition to the "micro-processes" of the research university -- the collaborations, the networks, the unexpected contiguities of problems, and the high level of decentralization that American research universities embody. As O-S documents, these processes are difficult to present to the public in a compelling way, and the vitality of the research university itself is vulnerable to destructive interference in the current political environment. Providing a clear, well-documented account of how research universities work is a major and valuable contribution.