What is methodology?

As social science researchers, we would all like to have an excellent methodology for carrying out the tasks we confront in our scientific work. But what precisely are we looking for when we aspire to this goal? What is a methodology, and what is it intended to allow us to do?

A methodology is a set of ideas or guidelines about how to proceed in gathering and validating knowledge of a subject matter. Different areas of science have developed very different bodies of methodology on the basis of which to conduct their research. We might say that a methodology provides a guide for carrying out some or all of the following activities:

• probing the empirical details of a domain of phenomena
• discovering explanations of surprising outcomes or patterns
• identifying entities or forces 
• establishing patterns
• providing predictions
• separating noise from signal
• using empirical reasoning to assess hypotheses and assertions

Here is what Andrew Abbott has to say about methods in Methods of Discovery: Heuristics for the Social Sciences:

Social scientists have a number of methods, stylized ways of conducting their research that comprise routine and accepted procedures for doing the rigorous side of science. Each method is loosely attached to a community of social scientists  for whom it is the right way to do things. But no method is the exclusive property of any one of the social sciences, nor is any social science, with the possible exception of anthropology, principally organized around the use of one particular method. (13)

So a method or a methodology is a set of recommendations for how to proceed in doing scientific research within a certain domain. Sometimes in the history of philosophy there has been a hope that science could proceed on the basis of a pure inductive logic: collect the data, analyze the data, sift through the findings, report the strongest regularities found in the data set. But scientific inquiry requires more than this; it requires discovery and imagination.

What form might a methodology take? The simplest idea is that a methodology is a recipe for arriving at justified scientific statements with respect to a domain of empirical phenomena. A recipe is a set of instructions for treating a number of ingredients in a sequential way and producing a specific kind of output -- a soufflé or a bowl of pad thai. If you follow the recipe, you are almost certain to arrive at the soufflé. But it is clear that scientific methodology cannot be as prescriptive as a recipe. There is no set of rules that are certain or likely to lead to the discovery of compelling hypotheses and explanations.

So if a scientific methodology isn't a set of recipes, then what is it? Here is another possibility: a methodology consists of a set of heuristics that serve to guide the activities, data collection, and hypothesis formation of the scientist. A heuristic is also a set of rules; but it is weaker than a recipe in that there is no guarantee of success. Here is a heuristic for consumers: "If you are selecting a used car to purchase, pay attention to rust spots." This is a good guide to action, not because rust spots are the most important part of a car's quality, but because they may serve as a proxy for the attentiveness to maintenance of the previous owner -- and therefore be an indication of hidden defects.

Andrew Abbott mentions several key topics for specification through methodology -- "how to propose a question, how to design a study, how to draw inferences, how to acquire and analyze data" (13), and he shows that we can classify methods by placing them into the types of question they answer.

types of data gathering	data analysis	posing a question
History	Direct interpretation	Case study analysis
Ethnography	Quantitative analysis	Small-N comparison
Surveys	Formal modeling	Large-N analysis
Record-based analysis

Abbott suggests that these varieties can be combined into five basic approaches:

• ethnography
• historical narration
• standard causal analysis
• small-N comparison
• formalization

And he arranges them in a three-dimensional space, with each dimension increasing from very particular knowledge at the origin to more abstract knowledge further out the axis. (Commonsense understanding of the facts lies at the origin of the mapping.) The three axes are formal modeling (syntactic program), pattern finding (semantic program), and cause finding (pragmatic program) (28). 

Abbott is a sociologist whose empirical and theoretical work is genuinely original and important, and we can learn a lot from his practice as a working researcher. His meta-analysis of methodology, on the other hand, seems fairly distant from his own practice. And I'm not sure that the analysis of methodology represented here provides a lot of insight into the research strategies of other talented social scientists (e.g. Tilly, Steinmetz, Perrow, Fligstein). This perhaps illustrates a common occurrence in the history of science: researchers are not always the best interpreters of their own practice. 

It is also interesting to observe that the discovery of causal mechanisms has no explicit mention in this scheme. Abbott never refers to causal mechanisms in the book, and none of the methods he highlights allow us to see what he might think about the mechanisms approach. It would appear that mechanisms theory would reflect the pragmatic program (searching for causal relationships) and the semantic program (discovering patterns in the observable data).

My own map of the varieties of the methods of the social sciences suggests a different scheme altogether. This is represented in the figure at the top of the post.