Barley's CT Scanner Study and the Role of Ethnography in Organizational Research (by Keith Rollag for Soc 260 Class)

This paper discusses ethnography (and specifically participant observation) and it's limited but important role in organizational research. Stephen Barley's ethnographic study of how new technology affected the organizational structure and social order of two radiology departments (Barley, 1986) is reviewed as a strong example of how traditional ethnography can be adapted to address quantitative limitations of the method.

Ethnography: Definition and History

Ethnography is the "empirical account of culture and social organization of particular human populations" (Ellen, ed. 1984). Ethnographers try to describe another culture as accurately as they can based on the "natives perspective" (Werner & Shoepfle, 1987). While the traditional ethnographer primarily uses participant observation and interviews, ethnographies often include both archival research and questionnaires in their analysis.

Participant observation is usually defined as direct observation of actor behavior supported by conversation with the actor to better understand their perspective and intent concerning the behavior (Spradley, 1980). While much participant observation is somewhat unobtrusive, ethnographers often become more involved in their research environment, sometimes as full participants (e.g., Burawoy, 1989).

Ethnography was created in the 1840's during the formation of ethnology (the predecessor to anthropology). The science of participant observation coalesced primarily through the efforts of Malinoski in Britain and Boas in the US during the early 1900's (Ellen, ed. 1984). Later anthropologists such as Mead and Radcliff-Brown further embedded participant observation into the toolbox of anthropologists.

Ethnography in Organizational Research

From the beginning, participant observation has been a useful tool for organizational researchers. The second issue of Administrative Science Quarterly in 1956 contains participant observations of psychiatric hospitals (Caudill, 1956) and merchant ships (Richardson, 1956).

To loosely measure the usage of ethnography and participant observation in organizational research, I examined the methodology sections of all 548 articles in Administrative Science Quarterly from 1975-1994. Based on my codification, there were 36 published ethnographic studies ( 3% of all articles) during the past 20 years.

The percent of ASQ articles using participant observation has increased somewhat over this time period (see Figure 1 and Table A in the Appendix). I suspect that the heightened interest in organizational culture research in the early 1980's spurred the use of ethnographic methods. Van Maanen's efforts at MIT to promote and train others in organizational ethnography may have also generated better quality research (Van Maanen, 1988, Special Issue on Qual. Methods in ASQ 1979).

Figure 1 (omitted, sorry!)

Advantages and Disadvantages of Ethnography and Participant Observation

Unlike other qualitative methods, participant observation allows confirmation of actor behavior, not merely espoused behavior found in interviews and questionnaires. Furthermore, direct observation lends a sense of "truth" to the ethnographers conclusions not present in indirect quantitative methods. Ethnographic analysis is usually narrative and only mildly statistical, which makes the research often more approachable to other researchers and especially laypersons.

Still, the difficulties of ethnographic research make it much less popular than questionnaires, interviews, or database analysis (see Figure 1). Since ethnographies typically include only one organization, conclusions are not easily generalizable. The lack of rigorous quantitative data is problematic for organizational researchers who see "truth in numbers".

More importantly, ethnographic research is simply hard to do. Thorough observation in the anthropological tradition can take months, often incompatible with the other academic demands such as teaching and administration. Translating fieldnotes into data suitable for analysis is as much art as science. Qualiative research is not as popular or academically pervasive as quantitative research. These disadvantages hinder it's use beyond the doctoral dissertation.

Barley's CT Scanner Study: An Example of "Quantitative Ethnography"

Despite the various biases against conducting ethnographies in organizational research, Barley's study of two radiology units in Boston (Barley,1986) has received much praise and attention from more "mainstream" researchers. It has been cited 83 times since 1986 in both qualitative and quantitative studies (SSCI, 1986-1995).

Barley observed the impact of CT scanner technology on the social order of two Boston radiology departments. Through participant observation and interviews at both sites for almost a year, Barley witnessed the evolution of work relationships between radiologists and technicians caused by the interpretive challenges of the new equipment. Barley codified the interaction between radiologist and technician into scripts and used statistical analysis of script frequency to demonstrate that behavior (and thus relationships) significantly changed as CT scanner technology was incorporated into the work flow of the hospital (Barley, 1986).

I believe there are several reasons for the significant impact of Barley's study in organizational research. First, the study directly examines the dynamic interactions between structure and action, and supports Gidden's structuration theories (Giddens, 1979). Barley's direct observation of action and structure also gives tremendous "face validity" to his conclusions.

Unlike other ethnographic studies, the CT scanner study also appears "scientific". Barley studied the impact of identical technologies on two similar organizations, thus effectively controlling for the main variable. The results then apparently derive from a true "experiment" than merely long-term observation.

Finally, Barley developed a "script" methodology to translate his observations into quantifiable data. This allowed him to use statistics of script frequency rather than verbal arguments to support his arguments. His quantitative approach to a qualitative research method helped legitimize the study among other researchers (Barley, personal conversation).

Barley followed his 1986 paper with another quantitative look at the same study, focusing on the change in informal network structure resulting from the new technology (Barley, 1989).

My main concern with Barley's study is that it is difficult to generalize his conclusions beyond the realm of the radiology department, other than to conclude that technology can affect organizational structures in different ways. The technology effect can also be somewhat explained simply by differences in initial technician competency level and individual training styles among radiologists.

Finally, though his use of scripts does make it more palatable for quantitative-based researchers, his personal codification of the data does introduce the potential of experimenter bias.

Still, his study did match the overall trends seen in role differentiation among radiologists and technicians with the advent of new scanning technologies. Unlike x-rays, the new technologies require some image interpretative ability to operate. Since these interpretive skills disrupt the traditional hierarchy of technicians (as image producers) and radiologists (as image interpreters), new organizational structures are required (Barley, 1995 seminar).

Conclusion

Ethnography remains as a intensive but important tool in organizational research. While long-term participant observation is time-consuming and somewhat limited in scope, it allows direct confirmation of individual intent and action and facilitates discovery of underlying social processes often undetectable through interviews and questionnaires.

Ethnographers like Barley who successfully integrate quantitative techniques into the basic ethnographic method help legitimize the use of ethnography in mainstream organizational research. Participant observation and other ethnographic techniques can provide the "reality check" for more quantitative analyses.

References

Administrative Science Quarterly, issues from 1974-1995

Barley, S. R. (1986), "Technology as an occasion for structuring: evidence from observations of CT scanners and the social order of radiology departments," Administrative Science Quarterly, 31: 78-108

Barley S. R. (seminar). Interdepartmental seminar conducted by Barley on November 2nd, 1995.

Barley S. R. (1990), "The alignment of technology and structure through roles and networks, Administrative Science Quarterly, 35: 61-103

Burawoy, M (1979), Manufacturing Consent, University of Chicago Press,

Caudill, W. (1956) "Perspectives on Administration in Psychiatric Hospitals", Administrative Science Quarterly, 1:2 155-170

Ellen, R. F. ,ed. (1984), Ethnographic Research: A Guide to General Conduct, Academic Press, London

Giddens, A. (1979) "Agency, structure" In Central Problems in Social Theory, University of California Press, 1979, 49-72.

Richardson, S. A (1956), "Organizational Contrasts on British and American Ships", Administrative Science Quarterly, 1:2 189-209

SSCI, 1986-1995, Social Science Citation Index

Van Maanen, J. (1988), Tales of the Field, On Writing Ethnography, University of Chicago Press

Werner, O., & Shoepfle, G. M. (1987), Systematic Fieldwork: Foundations of Ethnography and Interviewing, Sage Publications, Newbury Park