The standard of living across time and space

A very basic question for historians is how to measure and compare the standard of living experienced by people in different historical settings. Is it possible to arrive at credible estimates of the standard of living in the Roman Empire, medieval Burgundy, nineteenth-century Britain, and twentieth-century Illinois? Can we say with any confidence that Romans had a higher (or lower) standard of living than a twelfth-century Burgundian?

One part of the problem is conceptual. What do we mean by the standard of living? Is there a specific set of characteristics that are constitutive of the standard of living -- say, nutrition, income, access to health remedies and education, quality of housing, personal security? And how should we take account of the unequal distribution of these characteristics across a given population? Should we be content with an estimate of an average level of nutrition -- even though this may reflect a misleading impression of the circumstances of the poorest segment of society? Should we hope to be able to arrive at an estimate of the standard of living of certain typical social actors -- landless workers, skilled laborers, merchants?

The second major problem we must confront is the availability and quality of historical data about wages, prices, and consumption. The series of wages and prices that are available in different countries are, of course, incomplete. And, more importantly, the commodities that satisfy basic nutritional needs are different in different countries and regions. So it is necessary to make assumptions about the nutritional equivalents in different cultures before we can begin to arrive at estimates of relative standard of living.

Different approaches to these problems have been offered in the past fifty years. One logical approach is to consider a list of "necessities of life" and to estimate the degree to which these necessities are available to people of various stations in various times and places. Nutrition, housing, and clothing are close to the core for much of the history of humanity, and for much of that time, these goods have been available largely through the market at a price. So a standard approach has been to define a wage basket; measure the prices of the goods in the basket; and measure the typical earnings of people in historical settings of interest. This allows us to calculate the subsistence rate -- the percentage of the population whose income is more than sufficient to purchase the items in the wage basket. What this leaves out is "self production" (for peasant farmers, for example) and state provision. Another logical approach is to look at the human results -- the overall health status of people at various times and places. This can be estimated by contemporary data -- height, longevity, and age information collected by the military, for example -- or by analysis of skeletal evidence centuries later. (Amartya Sen's The Standard of Livingreviews many of the complexities of defining the standard of living and offers his own rationale in terms of capabilities and functionings.)

An important step forward is now possible in our ability to estimate and compare historical living standards, thanks to the research by an international group of scholars in Living Standards in the Past: New Perspectives on Well-Being in Asia and Europe, edited by Robert Allen, Tommy Bengtsson and Martin Dribe. The introduction to the volume by Allen, Bengtsson, and Dribe does a great job of providing an overview of the issues. All the essays are first-rate, and particularly noteworthy are contributions by Kenneth Pomeranz, Li Bozhong, and Robert Allen. (Here is a link to the table of contents of the volume, which gives an idea of the breadth and rigor of the research.)

This group has concentrated their efforts around the current controversy about European and Asian growth patterns in the early modern period. This has several parts: first, careful comparison to determine whether there was a significant difference in the standard of living between Europe and Asia (as held by Smith, Malthus, and Marx); and second, to attempt to determine the timing and causes of differences as they emerged. The editors describe the group's purposes in the introduction in these terms:

How did the standard of living in Europe and Asia compare in the seventeenth and eighteenth centuries? (Kindle loc 230)

The main concern of this book is to assess when the gap between the East and the West emerged and to not only take economic perspectives into consideration but social and demographic ones as well. (Kindle loc 159)

The researchers bring three methodologies to bear on these questions: economic analysis of prices and wages to estimate the real wage; demographic analysis of biometric features such as height, longevity, and fertility to estimate relative standard of living; and historical population analysis to observe the severity of adaptation (mortality, fertility, migration) created in a population by shortterm economic stress, including especially food prices. Here the reasoning is that a population that is close to the edge of subsistence in normal times may be expected to have higher mortality, lower fertility, and greater out-migration than a population with a more comfortable standard of living. So demographic change can be used as an indirect measurement of a population's standard of living. Using these three reasonably independent instruments of observation, it is reasonable to expect that we will gain a fairly accurate idea of the standard of living in a region and how it compares to other regions. The last approach is probably the most innovative:

There were demographic responses ... to high food prices. In the worst case, high prices caused death for those unable to buy enough to eat. In less extreme situations, people resorted to demographic strategies in response to high food prices. These included postponed marriages, migration, and delayed births. Studies of the correlation of death, migration, marriage, and childbearing with food prices, therefore, provide a new approach to the measurement of the standard of living. When aggregate data show that high food prices raised mortality or reduced fertility, one can conclude that the bulk of the population had a low standard of living. (Kindle loc 288)

Cameron Campbell and James Lee make use of this approach in their contribution, "Living standards in Liaoing, 1749-1909: Evidence from demographic outcomes," to assess the standard of living of the bulk of the population in Liaoning in northeast China. They find that the mortality and fertility responses to changes in rice prices essentially disappeared in the north and south of Liaoning in 1780-1850 -- which leads them to infer that the standard of living and nutrition had risen over the past century. (James Lee and others return to this kind of reasoning in Prudence and Pressure: Reproduction and Human Agency in Europe and Asia, 1700-1900, by Noriko Tsuya, Wang Feng, George Alter, and James Lee.)

Robert Allen attempts to establish something like an empirical baseline for the real wage in different parts of Europe and Asia in his contribution, "Real wages in Europe and Asia: A first look at the long-term patterns". He compiles a large dataset of wage data for a number of European cities, and he makes careful inferences about comparable data for India, Japan, and China.

Wages reveal the standard of living if they are compared to the price of consumer goods. This is the interpretation that matters in assessing the prosperity of Asia vis-a-vis Europe. Provided low Asian wages were matched by low consumer goods prices, the standard of living of workers could have been the same at both ends of Eurasia even though Asian manufacturers had a competitive advantage in the textile industry. (Introduction to "Real Wages")

Allen notes that it is necessary to make a number of adjustments in order to estimate the cost of a wage basket in Asia, because of large differences in diet. Allen stipulates 143 kgs/year of rice for the Asian basket versus 208 kgs/year of bread in the European basket. And he converts prices and wages into silver to permit comparison of prices across Europe and Asia. Here is one of Allen's summary graphs comparing laborers' real wages in Japan (farm), Kyoto, England (farm), Oxford, and London.

The graph indicates a significant premium for laborers' wages in London, whereas Japanese and English farm wages are fairly similar throughout most of the period. And it indicates a "take-off" for London laborers' wage beginning in the mid-nineteenth century -- not paralleled by a similar take-off in Kyoto.

Here are Allen's conclusions:

The wage comparisons undertaken in this paper support several important conclusions about living standards in pre-industrial Europe and Asia.

First, wages expressed in grams of silver were lower in China and India than in Europe. The views of the eighteenth century observers cited by Parthasarathi are confirmed. This is important since it was the proximate cause of Asia’s competitive advantage in textiles and luxury manufactures and was, thus, the basis for Asian-European trade in the early modern period. Why these differentials persisted for hundreds of years is an important question in international and monetary economics that must be addressed to explain the dynamics of the world economy in this period.

Second, low Asian silver wages were matched by low Asian prices with the result that living standards in Asia were similar to those in many parts of Europe. Farm workers in Europe and urban workers in central and southern Europe did not enjoy higher living standards than their counterparts in Asia.

Third, some parts of Europe did generate higher real wages than we find in Asia. When real wages were at their peak following the Black Death, most Europeans had a higher standard of living. But this was a transitory condition for most of the continent. High wages persisted only in the commercial centres of the northwest – London and the Low Countries generally. During the eighteenth century, the provincial towns of England were drawn into the same high wage orbit, but agriculture was left behind. This dynamic, urban economy was the engine of growth in early modern Europe, and Asia appears to have had no counterpart. It is possible, of course, that a more extensive Asian database would reveal a parallel: the absence of information on urban Chinese wages is particularly troubling in this regard. However, neither the Japanese cities nor the capital of the Moghul Empire had particularly high wages. The evidence at hand suggests that Asia lacked Europe’s engine of growth. (Conclusion of "Real Wages")

Contributions by Kenneth Pomeranz and Li Bozhong take up the issue of the supposed backwardness and stagnation of the Chinese rural economy at the beginning of the eighteenth century. In The Great Divergence: China, Europe, and the Making of the Modern World Economy Pomeranz argued that England and the Yangzi Delta region had roughly comparable levels of productivity and similar levels of standard of living for poor people (laborers and peasants). In his contribution to this volume he pushes this argument forward with more empirical analysis of the standard of living in China. He attempts to handle the "commensurability" problem mentioned above by converting subsistence food commodities to calorie equivalents. He finds that Chinese data for seventeenth century laborers indicate a daily diet of 2,800 calories for adults in the eighteenth century (Kindle loc 532).

Overall, then, the food component of the standard of living seems generally comparable in eighteenth-century China and Europe, and in the most advanced regions of both. (Kindle loc 626)

So Pomeranz's research here broadly confirms the view he advanced in The Great Divergence, that the standards of living in comparable regions of Europe and China were roughly the same; and he also confirms a significant decline in the standard of living for the bulk of the Chinese population in the nineteenth century.

Li Bozhong takes up the stagnation issue from a different point of view, a careful consideration of farm labor productivity in the Lower Yangzi region. This extends his important work in Agricultural Development in Jiangnan, 1620-1850. And his central finding is an important one as refutation of the standard involution interpretation of China's economic history; he finds that agricultural productivity rose from 1620 to 1850.

The central theme of this chapter is that labour productivity on farms did improve in Jiangnan between 1620 and 1850. The region of Jiangnan, located in east China and consisting of eight late Imperial Chinese prefectures in the Yangzi Delta, has been the most economically and culturally advanced area in China for centuries. ... In some sense, this region is the best 'window' through which we can clearly see economic changes in China before the arrival of the modern west. (Kindle loc 1038)

The issue of productivity is key to an assessment of the standard of living, because flat or declining productivity in a region with a rising population implies a falling standard of living -- the general "theorem" of Malthus. So Pomeranz's finding would be difficult to support if we were forced to conclude that agricultural productivity was constant or falling. But Li's careful and data-rich analysis indicates, to the contrary, that there was substantial gain in productivity from the Ming to the mid-Qing.

The 'trinity pattern' is the optimal pattern in the Jiangnan peasant economy because under this pattern higher yields per mu can be achieved with lower inputs. As I point out above, in the early seventeenth century, a farm ran, on average, 15 mu of cultivated land with a multi-cropping index of 140%, the second crop being wheat. The yield per mu was 1.7 shi for rice and 1 shi for wheat. If all the 15 mu of land were planted in rice, this farm would harvest 26 shi of rice and 6 shi of wheat, together equivalent to 30 shi of rice. In contrast, in the mid-nineteenth century, the average farm size was 9 mu with a multi-cropping index of 170&. Per mu yield was 2.5 shi of rice and 1 shi of wheat. Farm output was 23 shi of rice and 6 shi of wheat, totalling 27 she of rice, 10% below the early seventeenth-century figures. However, if we calculate labor productivity according to the number of workers, output per worker would be 15 shi of rice in the early seventeenth century and 27 shi of rice in the mid-nineteenth century respectively. That is, the figure for the late Ming period is only 55% of the mid-Qing figure. (Kindle loc 1160)

So labor productivity had risen significantly from 1650 to 1850. Li's conclusion is clear:

Since labour productivity and the standard of living are inseparably linked, the rise in farm labour productivity in Jiangnan implies an increase in the peasants' standard of living. ... There is little doubt, therefore, that real incomes of peasants did improve considerably in Jiangnan at this time. Second, the quality of the peasants' diet also improvied in Jiangnan during the period. Fang Xing suggested that ordinary Jiangnan peasants ate more fish, meat, and tofu, drank more tea and wine, and consumed more sugar than ever before. ... The improved standard of living can also be seen in the increase in consumption, not only of 'ordinary goods' like cotton cloth, but also of 'luxury goods' such as silk, wine, tobacco, and opium. (Kindle loc 1221)

What is most valuable about this project is the empirical grip it provides on these important questions: What have been the dynamics of the standard of living across Eurasia from the middle ages to the twentieth century? And, eventually, what economic and demographic forces account for the inflection points and persistent differences in different regions that are documented? And, as all the contributors agree, one of the key discoveries is the fact of variation at every level, from regions of England to regions of Europe to the ends of Eurasia.

The contributions of this book show the highly complex and diverse pattern of the standard of living in the pre-industrial period. The general picture emerging from these studies is not one of a great divergence between East and West during this period, but instead one of considerable similarities. These similarities not only pertain to economic aspects of standard of living but also to demography and the sensitivity to economic fluctuations. In addition to these similarities, there were also pronounced differences within the East and within the West -- differences that in many cases were larger than the differences between Europe and Asia. This clearly highlights the importance of analysing several dimensions of the standard of living, as well as the danger of neglecting regional, social, and household specific differences when assessing the level of well-being in the past. (Kindle loc 455)

Thinking cities darkly

Image: frame from West of the Tracks

Cities capture much of what we mean by "modern," and have done so since Walter Benjamin's writings on Paris (link). But unlike the eighteenth or nineteenth centuries, much of our imagining of cities since the early twentieth century has been dark and foreboding. A recent volume edited by Gyan Prakash, Noir Urbanisms: Dystopic Images of the Modern City, offers a collection of recent work in cultural studies that attempts to decode some of this dark imagery.

Several things are particularly interesting about the volume. Most basically, it represents an interesting conjunction of humanities perspectives and sociology. The articles are individually very good. And as a group they pose a series of important questions. How does a film set in Los Angeles or Shanghai serve to depict the city? Is there sociological content in a film that can contribute to a better sociology of the city? But also -- what can we say about the cultural currents that produce a particular vision of the city? Are there post-modern sensibilities and fears that lead filmmakers to turn the ambience dark?

The volume treats cities and their depictions in many parts of the world -- China, South Africa, Mexico, India, Europe, and the United States.  What is unusual about the volume is the fact that it is not a collection in "film studies" or in "urban studies", but rather a series of contributions taking seriously representation and the represented.  Moreover, there is no effort to force the perspectives taken into a common theory of "noir representation"; there are common themes that emerge, but each contributor brings forward a singular perspective, informed by the specifics of the region and genre that he/she studies. It is a project on the nexus between imaginative representation and existing social realities.

Prakash's excellent introduction begins with these observations:

As the world becomes increasingly urban, dire predictions of an impending crisis have reached a feverish pitch. Alarming statistics on the huge and unsustainable gap between the rates of urbanization and economic growth in the global South is seen to spell disaster.  The unprecedented agglomeration of the poor produces the specter of an unremittingly bleak "planet of slums." Monstrous megacities do not promise the pleasures of urbanity but the misery and strife of the Hobbesian jungle.  The medieval maxim that the city air makes you free appears quaint in view of the visions of an approaching urban anarchy.  Urbanists write about fortified "privatopias" erected by the privileged tow all themselves off from the imagined resentment and violence of the multitude. Instead of freedom, the unprecedented urbanization of poverty seems to promise only division and conflict.  The image of the modern city as a distinct and bounded entity lies shattered as market-led globalization and media saturation dissolve boundaries between town and countryside, center and periphery. From the ruins of the old ideal of the city as a space of urban citizens there emerges, sphinx-like, a "Generic City" of urban consumers.

As important as it is to assess the substance of these readings of contemporary trends in urbanization, it is equally necessary to examine their dark form as a mode of urban representation. This form is not new.  Since the turn of the twentieth century, dystopic images have figured prominently in literary, cinematic, and sociological representations of the modern city. In these portrayals, the city often appears as dark, insurgent (or forced into total obedience), dysfunctional (or forced into machine-like functioning), engulfed by ecological and social crises, seduced by capitalist consumption, paralyzed by crime, wars, class, gender, and racial conflicts, and subjected to excessive technological and technocratic control What characterizes such representations is not just their bleak mood but also their mode of interpretation, which ratchets up a critical reading of specific historical conditions to diagnose crisis and catastrophe. (1)

All the essays are interesting and insightful, but I was particularly interested in the Asian contributions -- India, China, and Japan.

First is Li Zhang's treatment of some current treatments of the dark side of Chinese cities (Shanghai and Shenyang) in "Postsocialist Urban Dystopia?".  She treats the Sixth Generation and New Documentaries movements in contemporary Chinese filmmaking, focusing on two recent works (Wang Bing's West of the Tracks, about the decline of a rust-belt city in the Northeast, and Lou Ye's Suzhou River, about the lives of poor and disaffected people in Shanghai).

Both works serve as powerful examples of "noir urbanism" in a Chinese context.  West of the Tracks is a nine-hour documentary capturing the lives and declining prospects of working class people in Shenyang following the reform of Chinese industry in the 1990s.  (C. K. Lee describes this process in Against the Law: Labor Protests in China's Rustbelt and Sunbelt.) Here is a link to West of the Tracks, well worth viewing.  And Suzhou River captures some of the gritty, squalid aspects of life in contemporary Shanghai, but also dwells on the moral shift that China is undergoing, towards a consumerist, wealth-oriented corrupt society.  Here is a clip from Suzhou River:



Zhang combines her own anthropological fieldwork in Chinese cities with her reading of these films, giving her essay a multiple sense of authority.  Here is a brief description of West of the Tracks that illustrates the intersection of criticism and fieldwork:

While capturing the "raw and the real" experiences of workers, West of the Tracks offers a subtle yet powerful critique of the postsocialist state and its neoliberal turn.  What is so striking in the story told here is the lack of government help and the indifference of society toward workers' dilemmas. (137)

She refers to the bleak setting of the break room at the factory:

In their daily conversations in the break room, smelting workers frequently talk about how the managers and cadres of the factories steal public money to line their own pockets by taking kickbacks at the expense of the enterprise.  The management and bosses rarely appear in the film.  The longest presence is a banquet gathering at a local restaurant where factory managers and cadres talked about the imminent total privatization.  They are well dressed in leather and wool coats with fur collars. (137)

So there are several key themes here: First, there is a critical perspective on the rising inequalities and dispossession of ordinary people that have followed from China's growth policies; this is the documentary aspect of the films she discusses, and plainly reflects the filmmakers' interest in capturing an important and disturbing contemporary social reality in China.  And second, there is a critical vision of the moral dislocations that China has undergone, from Maoist egalitarianism to capitalist and consumerist pursuit of wealth.  Zhang captures this element of contemporary China in her discussion of both films, but especially in Suzhou River.  There is squalor and poverty, to be sure, but more pervasive is the sense of moral ungroundedness.

Moneymaking, market exchange, and pleasure-seeking are the dominant forces of everyday life.  For example, the power of money erodes Mardar's blossoming love for Mudan and eventually destroys her, the symbol of innocent, unpolluted love.  Human greediness corrupts souls and drives violent acts such as kidnaping and murder. (139)

Zhang's summary is explicit:

During market liberalization, Chinese society has irrevocably changed into a mass consumer society in which money increasingly controls people's lives and determines their lifestyles. (139)

Another fine contribution to the volume shifts focus to India's cities.  Ranjani Mazumdar's "Friction, Collision, and the Grotesque: The Dystopic Fragments of Bombay Cinema" focuses on the mental urban landscape -- the way in which an Indian city is perceived by its residents, and the ways in which the residents are impaired by the city.  Mazumdar discusses three "urban fringe" films, Dombivli Fast, Being Cyrus, and No Smoking.  Here is a clip from Dombivli Fast:



Dombivli Fast is quite different from the films discussed by Zhang. It is reflective of the current social realities of Mumbai -- meaningless work, endless commuting on super-crowded trains.  But it is more personal and introspective than the Chinese works, in that it focuses on one man and his family; it attempts to reveal his inner anxieties and thoughts.  The dystopia here is not crushing poverty -- Madhav Apte and his family live a middle-class life in Mumbai.  Here the dystopia is the pressure, stress, and callous injustice of society that drives Madhav to the breaking point.

Madhav Apte does not go back home for three days after he explodes. Armed with a cricket bat, Apte acquires a menacing persona as he moves through a city that is almost fated to collapse because of corruption, inequality, and indifference. In his journey across Bombay's deadly streets, Madhav becomes an active figure whose rage makes him see the city with a heightened perception. (159)  

(There are clips from Being Cyrus and No Smoking on Youtube as well.  This is one of the fascinating realities of reading the volume: it is possible for us non-specialists to view segments from most of the films that are discussed.)

David Ambaras takes up Tokyo in its cultural representations in "Topographies of Distress: Tokyo, c. 1930." He too highlights the discrepancy between official, ideological expressions of the city, and the underlying grinding reality that modern cities often represent.

Yet despite this ebullience, to many contemporaries, urban modernity signaled the destruction of Japanese social values by Western materialism and individualistic hedonism, of which the modern girl served as the prime example. (188)

Ambaras doesn't work through cinema, but rather what he calls "slum discourse" and graphic pictorial representations of urban life.  He highlights the popular and journalistic literature of the 1870s through the early 1900s as a barometer of the anxieties Tokyo residents experienced about their changing city.  Stories of disease, child murder, beggars, and abject poverty permeate this literature.

These various forms of representation, ... had combined to produce in the Iwanosaka case a set of images that both shocked the sensibilities of readers and investigators and were necessary to their understanding of themselves as part of a modern metropolitan social formation. They reinforced the sense, common to many interpretations of the modern condition, that modernity was best apprehended through contrasts -- between, for example, utopian promise and dystopian reality -- or in terms of dark mysteries concealed beneath the surface of social relationships, and that the modern (urban) subject was compelled to navigate anxiously between these two positions, ever unsure as to which was the "truth" or in which direction he/she was being led. (210-11)

It is worth sorting out the different perspectives on social knowledge represented in this volume.  First, there is the question of knowledge of the object, the contemporary city.  Does cinema shed light on the current social realities of Shanghai or Mumbai?  Can cinema contribute to urban sociology?  Second is the question of the mentality of a place and time; the way that contemporary Mumbai-ers or Shanghai-ers think of themselves and their society.  Can cinema accurately capture some strands of social consciousness and anxiety that are real threads in the social landscape?  Is cinema a legitimate form of ethnography?  And third is the mentality and intentions of the creative class itself -- the filmmakers.  Can the critic discover threads in the filmmaker's work that sheds important light on the preoccupations of this slice of contemporary society?

Finally, we can ask the question of perspectivalism: how many Shanghai's are there?  Zhang refers to the Maoist preference for social realism or socialist romanticism; there are the entertainment-oriented Shanghai thrillers; there is the global Shanghai as an exotic backdrop to drama; and there is the noir representation of the social problems of the city.  Can we say that one depiction is more veridical than the other?  Or perhaps, can we say that several of these perspectives are compatible with the truth of Shanghai; and that optimism and pessimism are equally distorting frames for social perception?

(I note that several of the essays refer to Mike Davis's Planet of Slums; this is worth reading.)

Ngram anomalies

Now that I've played with the Google Ngrams tool a little, I continue to think it's a powerful window into a lot of interesting questions. But I also see that there are patterns that emerge that are plainly spurious, and surely do not correspond to real changes in language, culture, or collective interest over time. It is easy to find examples of search terms that very plainly indicate that there is some kind of "instrument error", an observation that emerges because of an artifact of the method rather than a real pattern in the underlying behavior.

Fortunately it is possible to probe these areas of anomaly with the goal of figuring out what they mean. So let's see what happens when we pick out a set of common words that are not freighted with a lot of culturally specific significance. This will let us see more clearly how the instrument itself works.

Consider the color words red, green, yellow, blue, black. Let's graph the frequency of these terms in American English from 1800 to 2000. Before looking at the Ngram graph, let's consider what we would expect ex ante. Color words occur in books to designate -- color. Color terms are common words, so we might expect that they would remain fairly constant in frequency over time. So here is the null hypothesis about the frequency of common color terms: without a change in culture about color, we should expect the color words would remain roughly constant in frequency (flat curve). And the usage patterns for each term should be independent from the others. So we should expect a degree of independent random fluctuations in the frequencies of each color word, where "blue" bumps up in frequency in a given year and "red" bumps down.

(Why should we expect a degree of independence in the random fluctuations between "red" and "blue"? Because, fundamentally, there is no common mechanism that would link their behavior.)

Here are some ways in which the actual behavior of color terms might deviate from the null hypothesis.  Some colors may be more in style than others at a time -- there may be a cultural preference for red over blue, so the frequency of red may be greater than the frequency of blue. And the frequencies may change as cultural preferences change; so blue may become more frequent than red in a later generation. More generally, literary taste may change by becoming more descriptive overtime -- with more frequent use of color terms -- or more formal, with less use of color terms. So it would be possible to explain persistent differences in frequency of color terms; shifting frequencies across different color words; and even a longterm rise or decline in the whole family of color words.

So ex ante, for this group of common color words we would expect a graph of flat lines for the five terms, with uncorrelated fluctuations in each line.

Now let's look at the actual graph of these word frequencies (link).

Here we can see behavior that flatly contradicts these reasonable ex ante expectations. First, there are stretches of time in which the color words covary extremely closely, to the extent that the graphs look identical in shape. This is true, for example, in the neighborhood of 1820. This is impossible to explain as anything else than an artifact of some sort. It is impossible to believe that the frequencies of several color words would fluctuate up and down with this degree of synchrony.

Here is another aspect of the graph that is also suggestive of artifact: the long wave of rise and fall in the frequency of all the color words between 1810 and 1920. It is not impossible that "color" became more important in literary language and then declined; but that seems improbable. So this long wave coordinated behavior of the color words seems to be more likely the effect of a database anomaly than a manifestation of a real trend.

Is there any reliable information in this graph?  Yes.  There is one feature of this graph that appears to have real significance, and that is the change in the behavior of "black" after 1960. Prior to that year the term behaves pretty much like all the other color words. After that year it takes off on a very different trajectory. And this abrupt and accelerating increase in the frequency of "black" seems to have everything to do with a real social and cultural change in the 1960s and forward -- the abrupt increase in those decades in the salience of race. There is a similar divergence between the behavior of "black" and all the other color words in 1860; the frequency of the word increases for a few years following the American civil war.

More tantalizingly, it may be significant that "blue" moves up from "yellow" to "green" in frequency over time.  This is one element of the graph where the terms are not correlated with each other; instead, "blue" changes its position relative to other color frequencies.

This example shows that we need to be careful about the inferences we draw from the patterns that appear from Ngram searches. We need to always ask: "Does this pattern really correspond to a fact about underlying collective linguistic behavior, or is it the result of an artifact?" More fundamentally, we need to understand the sources of the artifacts we are able to detect -- spurious correlations, inexplicable long-wave changes in frequency, and others still to be discovered. And, finally, we should seek out techniques that can be applied to the results that serve to filter out the artifacts and focus on the real variations the data contain. We need some signal processing here to separate signal from noise. The Ngram tool is powerful, but we need to use it critically and intelligently.

A new tool for intellectual history

Google's NGram Viewer is a really amazing new tool for researchers in literature and the humanities (link, link, link, link).  What is perhaps not quite so evident is the power it may have for people interested in the evolution of the social science disciplines.

Basically the concept is a simple one.  The Google Book project has now scanned and OCR'd millions of books published over the past several centuries.  With the announcement of NGram Viewer, Google has shared a vast indexed database of words and phrases included in this full corpus.  And its Viewer tool allows users to provide specific phrases and see a graph of the frequency of that phrase over time (1800-2000 by default).  It is important to bear in mind that the project measures the frequency of the phrase, not the absolute number of occurrences; so as the volume of words increases decade by decade, a phrase with a .005% share has an increasing number of occurrences.

Now consider this: to what extent is it possible to track themes, concepts, and theories in the social sciences by making use of the tool and a well-chosen list of terms or phrases?  For example, the classical economists (Smith, Ricardo, Marx) referred to their science as "political economy," whereas the economists defining the marginalist revolution (Jevons, Walras, Menger) tended to refer to their work as "economics." Classical economists operated under a fairly distinctive set of assumptions, including the labor theory of value, that differed sharply from the assumptions associated with the marginalist revolution and the creation of neoclassical economics.  So is it possible that we can we track the decline of classical economics and the rise of neoclassical economics by tracking "political economy" and "economics"?  Here is what that graph looks like:

The "political economy" line shows a gradually rising trend from 1800 to 1890, and then declines through about 1950.  Only in the 1980s does the term become more popular again -- perhaps corresponding to a resurgence of interest in the classical foundations of economic thought (Smith, Ricardo, Marx, Sraffa).  The term "economics" has virtually no use prior to 1880; it then begins to increase in frequency, passing "political economy" in frequency in 1910, and climbing rapidly into the 1980s.  The frequency of "economics" appears to be roughly eight times the frequency of "political economy" by 1980.

So how about other areas of the social sciences?  We might take a look at the development of anthropology using this approach; for example, how have the terms "anthropology" and "ethnography" changed in frequency in books during the past two centuries?  Here's the graph:

Through 1860 both terms are very infrequent.  After 1860, however, "anthropology" takes off, and "ethnography" continues at a low level through 1960, roughly at the time when Geertz and other path-breaking anthropologists began redefining the field in the direction of "local knowledge" or ethnography. From that point on both terms increase in frequency (with a large dip for "anthropology in the 1970s), with anthropology remaining a 3:1 favorite.

So now let's try out a somewhat more general question: how did political theory develop during the twentieth century?  We might probe this question by picking out a few important political theorists and examine the pattern of frequency that they demonstrate throughout the century; perhaps this will tell us something about them individually, as well as something interesting about the waxing and waning of interest in political theory more generally during the period.  Suppose we look at Hobbes, Rousseau, and Locke as the central founders of modern political theory.  Here's what that graph looks like:

This is interesting in many ways.  First, individually: Rousseau comes on very strongly in the 1930s, whereas Locke is fairly flat during the period.  Second, comparatively: Locke and Rousseau are consistently more frequent in the book literature than Hobbes -- in spite of our tendency to think of Hobbes as the ultimate founder of modern political theory.  ("Life in the state of nature is nasty, brutish, solitary, and short.")  And finally, there is a striking periodicity in the three graphs.  There are long waves of rise and fall of frequency over the century, and even more surprisingly, these waves are fairly closely correlated.  This implies one of two things: either political theory as a field waxed and waned through the century, with the frequency of references to the founders following along; or else this correlation is a symptom of a large artifact of the method itself.  We could test this latter idea by examining a larger set of authorities, including other areas of intellectual work.  So let's look at a final graph, including the three political theorists as well as Darwin, Newton, and Einstein:

Naturally enough, Einstein starts slowly in the early twentieth century, but then begins to rise in frequency after 1918 or so.  He passes Hobbes for good in 1938 or so.  Darwin, surprisingly enough, shows a falling trend through the century, recovering a bit after 1975.  But we were looking for evidence of artifact, and this graph seems to provide it; to the eye, there is a similar periodicity in the frequency of all these authors throughout the century, with a low point in 1945 (the end of World War II), and for all but Newton, another peak around 1965 followed by a trough around 1980.  So it isn't "political theory" that is oscillating in the ocean of words included in the universe of books, but intellectuals, philosophers, and scientists generally.  And this suggests that there's something going on that may have more to do with the measurements than the reality; more investigation is needed.

In short, it seems promising to think that there are some very interesting patterns to be teased out using this tool, and they are patterns that never could have been identified without the databases that Google has created.  A new form of our ability to probe and represent the growth of knowledge is made possible.

Hate as a social demographic

Every democracy I can think of has a meaningful (though usually small) proportion of citizens who fall on the extreme right by any standard: racist, White supremacist, hateful, anti-immigrant, anti-Semitic, anti-Muslim, nativist, nationalist, or violently anti-government individuals and groups. In the United States we have many, many organizations that are basically racist and potentially violent hate groups. They provide a basis for cultivating, recruiting and mobilizing like-minded followers, and they are sometimes co-opted by opportunistic politicians for their own narrow purposes. The Southern Poverty Law Center (link) and the Anti-Defamation League (link) do a great job and a needed service in tracking many of these organizations. (For example, SPL monitors 26 hate groups in the state of Michigan; link.)  The umbrella term for these organizations and individuals is "hate groups" -- individuals and organizations who organize their views of the social world around intolerance of other groups and a motivation to harm or subordinate those other people.

A recent report by the Institute for Research and Education on Human Rights (link) provides a detailed snapshot of how some of these racist groups have shown up in the Tea Party movement (link). The NAACP made a very careful statement about racist statements and provocations that had occurred at Tea Party protests in 2009 and 2010 (link), including the egregious incident that occurred in Washington in which Representatives Emanuel Cleaver, John Lewis, and Barney Frank were showered with vitriol. And this temperate and careful statement was derided by Tea Party leaders. The IREHR study goes a long way to document the concerns raised in the NAACP statement.

The maps of Tea Party membership are genuinely interesting. Here is an aggregate map including six factions of Tea Party organizations around the country (p. 14):

Here is a summary finding from the IREHR report:

Tea Party organizations have given platforms to anti-Semites, racists, and bigots. Further, hard-core white nationalists have been attracted to these protests, looking for potential recruits and hoping to push these (white) protestors towards a more self-conscious and ideological white supremacy. One temperature gauge of these events is the fact that longtime national socialist David Duke is hoping to find money and support enough in the Tea Party ranks to launch yet another electoral campaign in the 2012 Republican primaries. (7)

What I find really worth considering is the question of the fact of the very existence of these pockets of virulent racists and anti-semites in our society. I'm not thinking here of garden-variety racial stereotyping and prejudice, which is surely much more widespread, but of a kind of racism that extends to overt hostility and sometimes violence against the other group. It is hard to estimate the percentage of our society that falls in this category, though there are some public opinion surveys that help us make a crude estimate (Howard Schuman, Charlotte Steeh, Lawrence Bobo, Racial Attitudes in America: Trends and Interpretations, Revised Edition and Schuman and Bobo, "Survey-Based Experiments on White Racial Attitudes towards Residential Integration" (link); Pew Research Center, "Race, Ethnicity & Campaign '08" (link)).

But here is the important question: why do a certain number of people in modern societies have these attitudes in the first place? Is it just a sort of basic fact about our population that a certain percentage of us fall in this mindset? Are there specific features of our informal system of acculturation that creates this minority of hate-disposed people? Are these the result of a sort of sub-culture of militia encampments, prison gangs, and biker groups who are somehow able to promulgate their hatred to new recruits? Is it ignorance, disaffection, and economic uncertainty that brings out these qualities in otherwise decent people? In short -- is it the organizations that produce the hate in some people, or is it the hateful individuals who create the organizations?

Michael Mann's The Dark Side of Democracy: Explaining Ethnic Cleansing is one kind of empirical study of a related question, the occurrence of murderous ethnic cleansing. Here are a few of his hypotheses.

Murderous cleansing is modern, because it is the dark side of democracy. Let me make clear at the outset that I do not claim that democracies routinely commit murderous cleansing. Very few have done so. Nor do I reject democracy as an ideal – I endorse that ideal.Yet democracy has always carried with it the possibility that the majority might tyrannize minorities, and this possibility carries more ominous consequences in certain types of multiethnic environments. (2)

Ethnic hostility rises where ethnicity trumps class as the main form of social stratification, in the process capturing and channeling class-like sentiments toward ethnonationalism.

The danger zone of murderous cleansing is reached when (a) movements claiming to represent two fairly old ethnic groups both lay claim to their own state over all or part of the same territory and (b) this claim seems to them to have substantial legitimacy and some plausible chance of being implemented. (2-9)

These are political-structural theories of the conditions that stimulate outbreaks of hate-based mobilization and murderous ethnic cleansing.  But actually, Mann never addresses my question head-on: what accounts for the existence of a certain small percentage of haters in a given society.  Mann is interested rather in the behavior and the occurrence of mass mobilization for ethnic violence in certain times and places -- the involvement of thousands of citizens in acts of violence and murder against their fellow citizens. And he doesn't believe that the actors are exceptional; rather, the circumstances elicit the terrible violence from people much like all of us. Here is how he categorizes the perpetrators:

There are three main levels of perpetrator: (a) radical elites running party-states; (b) bands of militants forming violent paramilitaries; and (c) core constituencies providing mass though not majority popular support. (9)

And he doesn't think there is anything distinctive about the third group:

Finally, ordinary people are brought by normal social structures into committing murderous ethnic cleansing, and their motives are much more mundane. To understand ethnic cleansing, we need a sociology of power more than a special psychology of perpetrators as disturbed or psychotic people – though some may be. (9)

This "ordinary people as killers" theory (c) doesn't really seem to do justice to the problem at issue here. And the political opportunists who play the hate card aren't too hard to understand (a). It's really the people Mann includes under (b) that are of concern to me -- the true believers, the extremist White supremacists or virulent anti-Semites who become the local activists and paramilitaries that I'd like to understand better. And this population seems to be defined by the attitudes, motives, and ideologies that they bring with them, rather than the inter-group dynamics and political opportunism that Mann focuses on. So, once again, where does this mentality or psychology of activist hatred come from in our society (or in other contemporary societies)?

The easy answers are ready to hand. We might postulate a strand of political culture and thought in American society that reproduces hate and racism in some of the young people who are exposed to it. (Hate on the Internet falls in this category.)  Or we might postulate a recessive "racist personality" type that is a portion of the human psyche (along the lines of the theory of the authoritarian personality). Or we might postulate that lack of opportunity and an enduring situation of defeated expectations pushes some young people into hate (skinheads in Britain or Germany).

But I don't find any of the theories very convincing by itself.  So we seem to have an important theoretical issue here that is unresolved: where does "hate" come from?

Diagrams and economic thought

source: The Paretian System (link)

The most vivid part of any undergraduate student's study of economics is probably the diagrams.  Economists since Walras, Pareto, and Marshall have found it useful to express their theories and hypotheses making use of two-axis diagrams, allowing for very economical formulation of fundamental relationships. Supply-demand curves, production functions, and a graph of diminishing marginal product all provide a way of making geometrical sense of a given economic principle or hypothesis.  They allow us to visualize the relationships that are postulated among a set of factors.

Mark Blaug has made a long and fruitful career out of his remarkable ability of placing economic thought into its context (Economic Theory in Retrospect (1962), The Methodology of Economics: Or, How Economists Explain (1992)).  Now he has collaborated with Peter Lloyd to produce Famous Figures and Diagrams in Economics (2010), and the book is a marvelous contribution.

The book is organized into several large sections: Demand and supply curve analysis; Welfare economics; Special markets; General equilibrium analysis; Open economies; Macroeconomic analysis; and Growth and income distribution.  Experts have been recruited to write short, technical but accessible essays on some 58 topics, including discussion of about 150 diagrams.  

The figures that the book considers pretty much reproduce the history of modern economic thought.  And, indeed, some figures have been repeatedly rediscovered; Blaug attributes the "Marshallian cross" to Cournot (1838), Rau (1841), Dupuit (1844), Mangodt (1863), and Jenkin (1870).  Almost all the examples are drawn from the history of orthodox neo-classical economics; rare exceptions are Joan Robinson's "graph of discrimination" and August Losch's "market areas".  The main insights of classical economics are equally amenable to presentation through diagrams, so it is interesting that the classical economists (including Marx) were not particularly inclined to use them.  Here is a diagram not included in the book, representing Michio Morishima's effort to express some of Ricardo's central economic intuitions:

source: Michio Morishima, Ricardo's Economics: A General Equilibrium Theory of Distribution and Growth, p. 38

It is worth thinking a bit about what a diagram is, from a cognitive point of view.  To start, it is not a data graph; a diagram does not generally provide a summary of actual economic variables over time, such as unemployment.  But generally an economic diagram is not simply a graph of a given mathematical function either, plotting the value of a function over part of the domain of the independent variable.  We need more than a graphing calculator to create a useful economic diagram.

Rather, an economic diagram is a stylized representation of the behavior and interaction of (often) several variables in a range of interest.  Take the most fundamental diagram of neoclassical economics, the supply-demand diagram.  We are asked to consider "supply" and "demand" over a range of "price".  One curve represents the quantity of the good that will be produced at a low price through high price; the other curve represents the quantity of the good that will be purchased at the range of prices.  The intersection of the curves is the point of interest; it is the equilibrium at which quantity demanded equals quantity supplied.  The shape of the curve is significant; a straight line represents the view that supply and demand are linear with respect to price, whereas a curved line represents a non-linear relation between quantity and price.  (Each increment in price stimulates a smaller change in quantity.)

Here are some of the uses of diagrams in economics that Blaug and Lloyd mention in their introduction:

Figures and diagrams have been used in economic theory in several ways.  They have been used as a device to discover economic results; theorems or properties of models; or comparative static propositions and dynamic propositions. They have been used to prove some results. And they have been used as an expository device. (5)

They go on to quote Marshall:

It happens with a few unimportant exceptions all the results which have been obtained by the application of mathematical methods to pure economic theory can be obtained independently by the method of diagrams.  Diagrams represent simultaneously to the eye the chief forces which are at work, laid out, as it were, in a map; and thereby suggest results to which attention has not been directed by the use of methods of mathematical analysis. (5)

We might imagine that economic diagrams are purely mathematical constructs, and we might suppose that we have little choice in the way that a diagram is constructed.  But Edward Tufte has quite a bit to say on this subject in a series of books beginning with The Visual Display of Quantitative Information.  Essentially Tufte's message is that quantitative ideas can be conveyed in better and worse ways, and that much of the communication we do about quantities is misleading.  Conveying a quantitative relationship through a diagram can be done more or less insightfully; it is up to the economist to find a concise way of representing the relationships he/she is interested in exploring.

source: EJ Marey's train schedule, Paris to Lyons, in Edward Tufte,  The Visual Display of Quantitative Information

Blaug and Lloyd take some note of the "presentation aesthetics" of economic diagrams when they discuss modern methods of presentation:

In many areas of economic theory, the way in which economists understand economic concepts and propositions is through figures and diagrams.  What teacher of economic theory has not seen the dawn of understanding come over students when, failing to understand an exposition of some complex model in algebra or calculus, they are presented with a simple illustration? ... 

One can comprehend relationships among a number of variables (as in the box diagrams) or the effects of shifting curves or multiple equilibria more readily than in the corresponding algebra. This advantage has been increased by modern technologies.  Textbooks today use multi-coloured diagrams to great effect and the delivery of diagrams in classroom from computer-based programs allow overlays and other graphical techniques that aid the exposition of complex ideas. (8-9)

One of my favorite economic diagrams is the one introduced by Mark Elvin to represent his theory of a high-level equilibrium trap in agricultural development in The Pattern of the Chinese Past.

This diagram represents several different kinds of historical change in one compact figure: gradual technical progress along a production curve, shift of production curves through technical innovation, and the maximum production possibility curve that lies above each of these.  The axes represent "total output" and "rural population." The concave shape of each curve has a very specific economic and demographic meaning: as population grows within a given mix of techniques, output grows more slowly; so average output per capita approaches the subsistence line OS.  The HLET is graphically and laconically indicated on the upper right quadrant of the graph; there is no further room for technical improvement, and population has increased to the point where there is no surplus to fund radical technological innovation.  (Elvin's theory of the high-level equilibrium trap is discussed in my Microfoundations, Methods, and Causation; link.)

India's Naxalites

India is the world's largest democracy.  It also is home to one of the more persistent and deadly Maoist insurgencies in the world, the Naxalite movement in eastern India (Communist Party of India (Maoist) (CPI/M)).  The Naxalites were a splinter group that separated from India's Communist Party in the 1960s, and their hallmarks have been a commitment to violent revolution and a determined effort at mobilizing India's most disadvantaged rural people.  Here is a review article about the movement from The Economist in 2006 and a review in the New York Times in 2009, and here are several updates from The Economist (link, link, link).  (The Hindu provides frequent coverage of the Naxalite insurgency; for example, here.)

The 2006 Economist article was occasioned by a deadly attack against a police camp by Naxal guerillas in Chhattisgarh State. The attack signalled the fact that this movement continued to possess organization, followers, and deadly intent.  This map (reproduced from the Wikipedia article on the Naxalites) indicates the regions where the Naxalites currently have organizational presence; it is a large swath of rural India.  Estimates of the number of active fighters range from 14,000 to 20,000, with several times these numbers of local supporters and militias.

Naxalite attacks have continued since 2006.  Here is a chronology of attacks since 2006 compiled by NDTV (link).  Particularly widely noted in the world press was an April 2010 attack against Central Reserve Police Force, killing 75 police personnel, also in the forests of Chhattisgarh (link).  And the insurgency appears to be growing; in 2009 just under 1000 people were killed in Naxalite attacks -- almost triple the 2008 figures.

The leadership of CPI(M) has focused its mobilization efforts almost exclusively at the poorest of the poor and the most disadvantaged people within the caste system; and they have had particular success among tribal people living in India's forests.  But extending back to the 1960s the movement also has had some appeal to educated elites in cities and universities -- people on the left who believe that India has made no serious efforts to ameliorate poverty or caste.

What is surprising about this 40-year history is the fact that a violent revolutionary movement has managed to survive and conduct operations within the interstices of a democratic India.  Atul Kohli provided some context for this question in his 1991 Democracy and Discontent: India's Growing Crisis of Governability.  Here is how Kohli summarized his assessment twenty years ago:

India is still, of course, a functioning democracy, but increasingly it is not well governed.  The evidence of eroding political order is everywhere.  Personal rule has replaced party rule at all levels -- national, state, and district.  Below the rulers, the entrenched civil and police service have been politicized.  Various social groups have pressed new and ever more diverse political demands in demonstrations that often have led to violence.  The omnipresent but feeble state, in turn, has vacillated; its responses have varied over a wide range: indifference, sporadic concessions, and repression.  Such vacillation has fueled further opposition.  The ineffectiveness of repression, moreover, has highlighted the breakdown of the civil machinery intended to enforce the law and maintain order. Inorder to protect themselves, citizens in some parts of the country have begun organizing private armies. The growing political violence has periodically brought the armed forces into India's political arena, whereas the armed forces were considered apolitical. (3)

The main national parties in India, Congress and the Hindu nationalist Bharatiya Janata Party (BJP), have contested for control of the national government.  But India's states have consistently demonstrated a wider range of parties and politics.  West Bengal has been governed for decades by the Communist Party Marxist (CPM).  Kerala has been governed through left-coalitions including the Communist Party of India (Marxist) (CPI(M)).  Both these parties have renounced revolutionary violence and pursue a poverty-oriented set of economic and political policies.  Other states are governed by right and center-right coalitions, with a rhetoric that is much more focused on Hindu nationalism and pro-market ideology.

Kohli's study of governability focuses on West Bengal, Gujarat, and Bihar, and his goal is to identify some of the political and social processes that have led to a national and regional governability crisis.  Kohli finds substantial inter-state variability in governability across the three states he studies.

The situation of total breakdown of order in Bihar, for example, has resulted from corrosion of the authority vested in the social structure and absence of cohesion in political structures.  Increasing power struggles in the society and a highly factionalized elite have provided a combustible political mixture that ignites periodically, and in Bihar even the forces of repression are ineffective. Political violence in Gujarat has had a more purposive quality than in Bihar.  It results not primarily from a breakdown of social order but from the calculated mobilization strategies employed by competing elites.

In contrast to the situations of these two states, the case of West Bengal demonstrates how the presence of a cohesive party can bring stability even in a highly mobilized political environment: Strong leaders and disciplined ruling parties -- forces that can impart a degree of cohesion to state structures -- can help moderate the impact of corrosion of authority in the social structure.  The emergence of incoherence in both state and social structures, however, is a sure recipe for a breakdown of order. (15)

So how is it possible for a violent rural insurgency to survive and grow in a modern democracy?  There seem to be several primary factors.  One is the familiar fact that insurgents establish their bases of operations in remote, lightly-policed regions.  The Naxalite movement has managed to secure its "base areas" in forests and other poorly developed rural areas.

(The Hindu)

A second factor is the fact that Indian rural society is generally only lightly policed by professional police forces.  India is a "weak state" when it comes to local presence in the countryside.  The local presence of police is very thin on the ground, and easily overrun by trained guerillas.  Third, the Naxalites direct their mobilization efforts to the most disadvantaged and disaffected segments of rural Indian society; so their anti-state message is relatively well received.  And, finally, it would appear that the ideology of class and class antagonism has its own resonance in Indian political culture, far beyond the violent extremism of the Naxalite party.  This movement was a splinter from the Communist Party of India (Marxist) (CPM), which now governs West Bengal on the basis of its electoral successes (at least up until the present).  But CPM rhetoric sounds many of the same themes of anti-capitalism and anti-globalization that the Naxalites advocate.  So it is possible for elements of the left to become frustrated with the electoral program of CPM and develop new resonance with the radical program of the Naxalites.

The consensus view seems to be that the national and state governments in India have failed in efforts to suppress the Naxalite insurgency to date out of a failure of will, a degree of conflict of political interests in several states, and a failure to arrive at a winning strategy of police and military force combined with significantly more successful delivery of rural development successes.  Villages that lack roads, clean water, education, and health clinics are certainly more likely to find a radical program of change more appealing, and India's states have had little success in improving the quality of life for India's poorest rural people.

Granularity

If we think back over the history of sociology and political science since 1800, one thing that is striking is the trend away from the most macro-level frameworks of thought in the direction of more situated and proximate social and political arrangements. There was a tendency among the founders -- Montesquieu, Spenser, Comte, for example -- to pose their questions at the civilizational level. The largest social constructs were identified as worthy of study -- religions, kinds of governments, cultures. By the time we get to writers like Tocqueville and John Stuart Mill in mid-nineteenth century, we have thinkers approaching modern social life with a more limited gaze. They are more interested in particular, differentiating social and political arrangements, and they are interested in the particulars of human behavior within those arrangements. And they are much more interested in the empirical details, rather than general philosophical theories of democracy or religion.

Twentieth century thinking in the social sciences has generally continued this trend. What makes social science "scientific" is rigorous attention to empirical characteristics of the social world.  There is generally even less patience with large philosophical theories of society and history.  A good theory isn't of much interest unless it can be closely tied to particular bodies of empirical observation. It is hard to think of a 20th-century sociologist who took up the Comtean project of comparing the course of civilizations. And there is a growing consensus that the focus of social research needs somehow to capture the behavior of situated actors within socially specific arrangements -- in other words, a refinement of focus towards the more particular arrangements.

A partial exception to this trend is the field of comparative politics (Lichbach and Zuckerman, eds., Comparative Politics: Rationality, Culture, and Structure). This is a paradigm that undertakes to discover similarities and differences in large political institutions across settings -- with the implication that there is some reason to expect that "developmental states" will have some important similarities whether in Brazil, Nigeria, or Syria. How does "electoral competition" work in the United States, Mexico, and Poland? Hw do anti-corruption regimes work in Indonesia, Singapore, and Hong Kong? Comparative politics specialists have accepted the need for careful empirical investigation, to be sure, and they are mindful of differences as well as similarities. But the field is more macro than many other areas of contemporary social science.

Another exception seems to be "world systems theory", which locates social-historical analysis at the global level.  But as argued earlier (link), Wallerstein was in fact sensitive to historically specific "granularities" within the world system.  Here's what I concluded there:

But Wallerstein's practice as a sociologist is far more defensible than this language would suggest. He was in fact sensitive to causal heterogeneity, contingency, and variation in the systemic relations he meant to capture -- particularity as well as universality. So he doesn't actually treat the modern world system as if it were analogous to a set of gravitational objects governed by fixed laws of nature.

So we might say that two of the large features of modern social science research paradigms are these: the conviction that a researcher needs to formulate questions that permit careful empirical observation and assessment; and the idea that the scope of research should be tailored to capture social behavior and organization at an intermediate level. We need to formulate research questions that capture enough social behavior to be interesting, while at the same time permitting careful, rigorous empirical investigation and assessment.

This is where the metaphor of granularity comes in. It suggests the idea of a deliberate choice of levels of analysis, a deliberate selection of the zone of focus we choose for the investigation. Go too fine and you lose sight of the structures that will turn out to be interesting. Go too gross and, once again, those structures disappear. Then there is the important idea of heterogeneity -- the idea that stuff is not uniform in structure all the way down (link); so it matters what level we choose to focus on.

The scanning electron microscope image above (center) is illustrative of this fact with regard to the natural world. Carbon appears uniform at a gross level (anthracite), and it is also uniform at an atomic level.  But there is an intermediate level of resolution where there are structures that are very important for the physical properties of the stuff; and we won't find these structures unless we look at this intermediate level.

Something very much like this seems to be true about the social realm as well.  If we ask questions about "citizens" or "consumers", we have characterized the actor in a highly abstract way, with no indication or recognition of the more specific circumstances in which the actor conducts his/her life.  If we provide more institutional, organizational, and motivational context, however -- that is, if we increase the level of resolution in our depiction of the social actor -- we are likely to find more illuminating features of behavior, and we are likely to find discriminating explanations for differences in behavior.

Perhaps an illustration is found in the kinds of social action that Erving Goffman investigates -- the stylized behaviors and interactions of restaurant workers, for example (link). Go too deeply into the personal stories of the persons involved and you lose what is distinctive about the social context of the restaurant. Aggregate facts about service workers everywhere, and once again, the distinctive behaviors disappear.  And, if we abstract too severely from the specifics of the social setting of the restaurant worker -- that is, if we zoom our microscope out to the level of pure rational choice theory -- then we will overlook the differentiating social circumstances that influence behavior in real hotels and restaurants.

We might say that all of this is pointing towards a certain kind of sociological research that focuses on the proximate social settings within which people act and interact. This takes us towards what I call methodological localism (link). But it also conforms to the sociological frameworks of the new institutionalism (link, link) and even with the emerging emphasis on explanations based on social causal mechanisms (link).  

In seeking out the concept of granularity here, I've actually invoked several different contrasts: abstract to concrete, apriori to empirical, and macroscopic to microscopic.  If we put all these together, we might say that a "granular" approach to a social phenomenon is one that achieves a degree of specificity in conceptual definition, a robust degree of empirical detail, and an intermediate analytical and representational perspective that permits discovery of the mid-sized structures that influence the behavior of the whole.

(This issue has come up previously under the topic of "macro, meso, micro" (link).)