Organizations as open systems

Key to understanding the "ontology of government" is the empirical and theoretical challenge of understanding how organizations work. The activities of government encompass organizations across a wide range of scales, from the local office of the Department of Motor Vehicles (40 employees) to the Department of Defense (861,000 civilian employees). Having the best understanding possible of how organizations work and fail is crucial to understanding the workings of government.

I have given substantial attention to the theory of strategic action fields as a basis for understanding organizations in previous posts (link, link). The basic idea in that approach is that organizations are a bit like social movements, with active coalition-building, conflicting goals, and strategic jockeying making up much of the substantive behavior of the organization. It is significant that organizational theory as a field has moved in this direction in the past fifteen years or so as well. A good example is Scott and Davis, Organizations and Organizing: Rational, Natural and Open System Perspectives (2007). Their book is intended as a "state of the art" textbook in the field of organizational studies. And the title expresses some of the shifts that have taken place in the field since the work of March, Simon, and Perrow (link, link). The word "organizing" in the title signals the idea that organizations are no longer looked at as static structures within which actors carry out well defined roles; but are instead dynamic processes in which active efforts by leaders, managers, and employees define goals and strategies and work to carry them out. And the "open system" phrase highlights the point that organizations always exist and function within a broader environment -- political constraints, economic forces, public opinion, technological innovation, other organizations, and today climate change and environmental disaster.

Organizations themselves exist only as a complex set of social processes, some of which reproduce existing modes of behavior and others that serve to challenge, undermine, contradict, and transform current routines. Individual actors are constrained by, make use of, and modify existing structures. (20)

Most analysts have conceived of organizations as social structures created by individuals to support the collaborative pursuit of specified goals. Given this conception, all organizations confront a number of common problems: all must define (and redefine) their objectives; all must induce participants to contribute services; all must control and coordinate these contributions; resources must be garnered from the environment and products or services dispensed; participants must be selected, trained, and replaced; and some sort of working accommodation with the neighbors must be achieved. (23)

Scott and Davis analyze the field of organizational studies in several dimensions: sector (for-profit, public, non-profit), levels of analysis (social psychological level, organizational level, ecological level), and theoretical perspective. They emphasize several key "ontological" elements that any theory of organizations needs to address: the environment in which an organization functions; the strategy and goals of the organization and its powerful actors; the features of work and technology chosen by the organization; the features of formal organization that have been codified (human resources, job design, organizational structure); the elements of "informal organization" that exist in the entity (culture, social networks); and the people of the organization.

They describe three theoretical frameworks through which organizational theories have attempted to approach the empirical analysis of organizations. First, the rational framework:

Organizations are collectivities oriented to the pursuit of relatively specific goals. They are "purposeful" in the sense that the activities and interactions of participants are coordinated to achieve specified goals..... Organizations are collectivities that exhibit a relatively high degree of formalization. The cooperation among participants is "conscious" and "deliberate"; the structure of relations is made explicit. (38)

From the rational system perspective, organizations are instruments designed to attain specified goals. How blunt or fine an instrument they are depends on many factors that are summarized by the concept of rationality of structure. The term rationality in this context is used in the narrow sense of technical or functional rationality (Mannheim, 1950 trans.: 53) and refers to the extent to which a series of actions is organized in such a way as to lead to predetermined goals with maximum efficiency. (45)

Here is a description of the natural-systems framework:

Organizations are collectivities whose participants are pursuing multiple interests, both disparate and common, but who recognize the value of perpetuating the organization as an important resource. The natural system view emphasizes the common attributes that organizations share with all social collectivities. (39)

Organizational goals and their relation to the behavior of participants are much more problematic for the natural than the rational system theorist. This is largely because natural system analysts pay more attention to behavior and hence worry more about the complex interconnections between the normative and the behavioral structures of organizations. Two general themes characterize their views of organizational goals. First, there is frequently a disparity between the stated and the “real” goals pursued by organizations—between the professed or official goals that are announced and the actual or operative goals that can be observed to govern the activities of participants. Second, natural system analysts emphasize that even when the stated goals are actually being pursued, they are never the only goals governing participants’ behavior. They point out that all organizations must pursue support or “maintenance” goals in addition to their output goals (Gross, 1968; Perrow, 1970:135). No organization can devote its full resources to producing products or services; each must expend energies maintaining itself. (67)

And the "open-system" definition:

From the open system perspective, environments shape, support, and infiltrate organizations. Connections with "external" elements can be more critical than those among "internal" components; indeed, for many functions the distinction between organization and environment is revealed to be shifting, ambiguous, and arbitrary.... Organizations are congeries of interdependent flows and activities linking shifting coalitions of participants embedded in wider material-resource and institutional environments.  (40)

(Note that the natural-system and "open-system" definitions are very consistent with the strategic-action-field approach.)

Here is a useful table provided by Scott and Davis to illustrate the three approaches to organizational studies:

An important characteristic of recent organizational theory has to do with the way that theorists think about the actors within organizations. Instead of looking at individual behavior within an organization as being fundamentally rational and goal-directed, primarily responsive to incentives and punishments, organizational theorists have come to pay more attention to the non-rational components of organizational behavior -- values, cultural affinities, cognitive frameworks and expectations.

This emphasis on culture and mental frameworks leads to another important shift of emphasis in next-generation ideas about organizations, involving an emphasis on informal practices, norms, and behaviors that exist within organizations. Rather than looking at an organization as a rational structure implementing mission and strategy, contemporary organization theory confirms the idea that informal practices, norms, and cultural expectations are ineliminable parts of organizational behavior. Here is a good description of the concept of culture provided by Scott and Davis in the context of organizations:

Culture describes the pattern of values, beliefs, and expectations more or less shared by the organization’s members. Schein (1992) analyzes culture in terms of underlying assumptions about the organization’s relationship to its environment (that is, what business are we in, and why); the nature of reality and truth (how do we decide which interpretations of information and events are correct, and how do we make decisions); the nature of human nature (are people basically lazy or industrious, fixed or malleable); the nature of human activity (what are the “right” things to do, and what is the best way to influence human action); and the nature of human relationships (should people relate as competitors or cooperators, individualists or collaborators). These components hang together as a more-or-less coherent theory that guides the organization’s more formalized policies and strategies. Of course, the extent to which these elements are “shared” or even coherent within a culture is likely to be highly contentious (see Martin, 2002)—there can be subcultures and even countercultures within an organization. (33)

Also of interest is Scott's earlier book Institutions and Organizations: Ideas, Interests, and Identities, which first appeared in 1995 and is now in its 4th edition (2014). Scott looks at organizations as a particular kind of institution, with differentiating characteristics but commonalities as well. The IBM Corporation is an organization; the practice of youth soccer in the United States is an institution; but both have features in common. In some contexts, however, he appears to distinguish between institutions and organizations, with institutions constituting the larger normative, regulative, and opportunity-creating environment within which organizations emerge.

Scott opens with a series of crucial questions about organizations -- questions for which we need answers if we want to know how organizations work, what confers stability upon them, and why and how they change. Out of a long list of questions, these seem particularly important for our purposes here: "How are we to regard behavior in organizational settings? Does it reflect the pursuit of rational interests and the exercise of conscious choice, or is it primarily shaped by conventions, routines, and habits?" "Why do individuals and organizations conform to institutions? Is it because they are rewarded for doing so, because they believe they are morally obligated to obey, or because they can conceive of no other way of behaving?" "Why is the behavior of organizational participants often observed to depart from the formal rules and stated goals of the organization?" "Do control systems function only when they are associated with incentives ... or are other processes sometimes at work?" "How do differences in cultural beliefs shape the nature and operation of organizations?" (Introduction).

Scott and Davis's work is of particular interest here because it supports analysis of a key question I've pursued over the past year: how does government work, and what ontological assumptions do we need to make in order to better understand the successes and failures of government action? What I have called organizational dysfunction in earlier posts (link, link) finds a very comfortable home in the theoretical spaces created by the intellectual frameworks of organizational studies described by Scott and Davis.