Pub. Date:
Princeton University Press
Seven Rules for Social Research / Edition 1

Seven Rules for Social Research / Edition 1

by Glenn Firebaugh


View All Available Formats & Editions
Current price is , Original price is $35.0. You
Select a Purchase Option (New Edition)
  • purchase options
    $30.62 $35.00 Save 13% Current price is $30.62, Original price is $35. You Save 13%.
  • purchase options

Product Details

ISBN-13: 9780691135670
Publisher: Princeton University Press
Publication date: 01/03/2008
Edition description: New Edition
Pages: 272
Product dimensions: 6.00(w) x 9.25(h) x 0.70(d)

About the Author

Glenn Firebaugh is Distinguished Professor of Sociology and Demography at Pennsylvania State University. He is the author of The New Geography of Global Income Inequality.

Read an Excerpt

Seven Rules for Social Research

By Glenn Firebaugh Princeton University Press
Copyright © 2008
Princeton University Press
All right reserved.

ISBN: 978-0-691-12546-6

Chapter One The First Rule


Social research differs fundamentally from advocacy research. Advocacy research refers to research that sifts through evidence to argue a predetermined position. Lawyers engage in advocacy research in attempting to prove the guilt or innocence of defendants. Special interest groups engage in advocacy research when trying to influence the vote of lawmakers on a particular bill. The aim of advocacy research is to convince others (juries, lawmakers) to think or act in a given way. Hence advocacy research presents evidence selectively, focusing on supporting evidence and suppressing contrary or inconvenient evidence.

Social research, in contrast, does not suppress contrary or inconvenient evidence. In the social sciences we may begin with a point of view, but (unlike advocacy research) we do not want that point of view to determine the outcome of our research. Rule 1 is not intended to suggest that you need to check your idealism at the door when you embark on a social research project. Far from that-if you don't want to change the world for the better, you might not have the doggedness needed to do good social research. But rule 1 is intended to warn that you don't want to be blinded by preconceived ideas so that you fail to look for contraryevidence, or you fail to recognize contrary evidence when you do encounter it, or you recognize contrary evidence but suppress it and refuse to accept your findings for what they appear to say.

You must be ready to be surprised by your research findings. It is the uncertainty that makes social research exciting and rewarding. If you always knew the answer beforehand, what is the point of doing the research?

For first-time researchers, the surprise very often is in the finding of small effects where large effects were expected. Rarely do new researchers obtain findings that were as strong as they anticipated. Sometimes results are even opposite to expectations. Perhaps you were absolutely certain, for example, that in the United States women are more likely to approve of abortion than men are (not true: see exercises at the end of this chapter), or that younger adults tend to be happier than older adults are (which is also false: see exercises at the end of chapter 2).

Although you might at first be disappointed in your results, it is important to keep in mind that a noneffect is not a nonfinding. Any result is a finding. Finding no effect could be more interesting than finding a big effect, especially when conventional wisdom holds that there is a big effect. Often the most interesting results in social research are those that fly in the face of conventional wisdom. So you should not be disappointed when you do not find the big effect you expected, or when your results are inconsistent with "what everyone knows."

Selecting a Research Question

The first step in a research project is to decide upon a research question. A research question in the social sciences is a question about the social world that you try to answer through the analysis of empirical data. The empirical data may be data that you collected yourself, or data collected by someone else.

There are two fundamental criteria for a research question: interest and researchability. In selecting a research question, your aim is to find a question that is (1) researchable and (2) interesting to you and to others in your field.

Researchable Questions

Some research questions in the social sciences are not researchable because they are simply unanswerable (see Lieberson 1985, chap. 1). In that respect economics, political science, psychology, and sociology are the same as astronomy, biology, chemistry, and physics. Some questions-such as the existence of God-are inherently unknowable with the scientific method. Other questions might be unanswerable as the questions are currently conceived. Other questions might be unanswerable with the knowledge and methods we currently have.

If you are a student, some questions that are answerable in principle might nonetheless be beyond your reach. The Minneapolis Domestic Violence Experiment, for example, sought to determine whether repeat incidents of domestic violence would decline if police arrested accused abusers on the spot (Sherman 1992; Sherman and Berk 1984). Typically students do not have the time, resources, or credentials required to carry out an experiment of this magnitude. The possibilities for student projects are not as restricted as one might think, however, because there are a surprising number of data sets that are available to students for analysis (the student exercises at the end of this chapter use just such a data set, the General Social Survey). Even if you plan to collect your own survey data, you should consult such prior surveys early in your project, if only for guidance on how to word your questions.

Other research is ruled out because it is unethical. To jump ahead to an example that is central in a subsequent discussion of causality (chapter 5), we did not know, until a few decades ago, whether or not smoking causes lung cancer-some experts said it did, some said that the evidence was inconclusive. Scientifically, the best way to settle the dispute would have been to assign individuals randomly to one of two groups, a smoking group and a nonsmoking group. In terms of science, then, the question could be answered by doing an experiment. In practice, however, such an experiment would have raised serious ethical questions regarding an individual's right to smoke (or not smoke). The important point here is that ethical considerations are important-important enough to render some types of research undoable.

What, then, does a researchable question look like? Researchable questions in the social sciences tend to be questions that are neither too specific (for example, about a specific individual or event) nor too grand ("Why are there wars?"). The question "Why is my uncle an alcoholic?" is not a good research question because in the final analysis it is impossible to predict the behavior of a given individual; social laws are not deterministic. (Besides, the question isn't very interesting to anyone outside your family.) It is possible to make that question researchable by generalizing it to alcoholics in general. We could ask, for example, why some people tend to be more prone to alcoholism than others are. To address that question, we first ask what characteristics distinguish alcoholics from nonalcoholics. In other words, we ask what individual or community traits correlate with alcoholism.

In the social sciences what questions generally are easier to answer than why questions. As Lieberson (1985, p. 219, italics removed) puts it, "Empirical data can tell us what is happening far more readily than they can tell us why it is happening." Consider alcoholism again. With appropriate data it is a straightforward matter to find the correlates of alcoholism, that is, to find the characteristics that distinguish alcoholics from others. Whether these correlates are causes of alcoholism is another matter (we take up the causality question under rule 5, "Compare like with like"). But that does not make the correlates uninformative. Even if we want to determine causes, generally the first step is to find correlates, since knowing the what gives us clues about the why.

Because "mere description" is sometimes devalued in the social sciences, it is important to underscore the point that "what" questions come before "why" questions. Facts come first, as Sherlock Holmes stated famously in his warning to Dr. Watson that "It is a capital mistake to theorize in advance of the facts" ("The Adventure of the Second Stain"). To punctuate the detective's point, later I describe an instance where social theorizing went astray by attempting to account for "rising global inequality" when in fact global inequality was not rising. Obviously we need to get the facts right about what there is to explain before we concoct an explanation. As the first maxim of Galileo's Discors states, "description first, explanation second" (see Pearl 2000, pp. 334-35).

Getting the facts right is critical, second, because accurate description sometimes is itself the end goal. Consider the problem of motion. Medieval attempts to understand motion grappled with the nature and origins of motion. Current understanding takes motion as given and attempts to describe it precisely and accurately. As Andrew Abbott (2003, p. 7) points out:

[Isaac Newton] solved the problem of motion by simply assuming that (a) motion exists and (b) it tends to persist. By means of these assumptions (really a matter of declaring victory, as we would now put it), he was able to develop and systematize a general account of the regularities of motion in the physical world. That is, by giving up on the why question, he almost completely answered the what question.

In this instance progress was stalled until the question was switched from the unknowable (why things move) to the knowable (how they move). We have returned, then, to the central point of this section: that a research question must first of all be answerable. The second requirement is that it is interesting.

Interesting Questions

Besides finding a question that is answerable, you want a question that is interesting-to you and to others. Research involves entering into a conversation. The issue is how your research will contribute to that conversation. A research question might be interesting because of its scientific relevance: In addressing this issue, you join an ongoing discussion in the social sciences. Or a question might be interesting because it is socially important; perhaps it bears on some important local or national social problem. Or a question might be interesting because of its timeliness; a question relating to Americans' attitudes about gun control might be particularly interesting, for example, when important legislation is pending on the subject.

"The heart of good work," Abbott (2003, p. xi) writes, "is a puzzle and an idea." An interesting research question encompasses both. The puzzle is the issue about the social world that bears investigation. The idea is the new twist that you bring to the investigation.

How do you find an interesting research question? Often personal experiences, or the experiences of others we know, provoke our curiosity about some aspect of human behavior. You can also obtain ideas by reading the research of others. New findings often lead to new questions, and research reports often conclude by noting further research that is needed, or by suggesting directions that subsequent research might take.

Unfortunately, there is no foolproof recipe for cooking up an interesting research question. In that light, it is important to keep in mind that the ideas below are intended to stimulate your thinking, not to provide a roadmap to discovery. No one knows where lightning will strike, but it is still a good idea to avoid sitting on a high metal roof in a thunderstorm. Inspiration is the same way: We cannot be certain where it will strike, but we know that the probabilities are greater in some places than in others.

If you have trouble thinking of a good question for a research project, most likely the problem is either that you have an idea that you like but others don't, or you cannot come up with an idea in the first place.

Suppose you have an idea for a project that enthuses you but no one else. One reaction-perhaps most common among students doing their first research project involving actual data analysis-is to ignore what others, such as your professors, think. That is generally not a good idea. The interest criterion for a research question dictates that you find an issue that is interesting to you and to others in your field. The "others in your field" requirement is important because the "so what?" question is the first challenge you will face when writing up the results of your research. Early in the report you must be able to explain, in a few sentences, why your results are of interest to your readers. Perhaps the question you address relates to an ongoing discussion in your field. Perhaps it extends prior research in new directions, or replicates findings using a new population. Or perhaps it sheds light on an empirical puzzle. The point is that you need some rationale for doing your research. The fact that you cannot get anyone else interested in your research is a telltale sign that you need to rethink the project. Rethinking the project does not necessarily mean that you need to abandon your idea. But it does mean that you need to rework your question to make it interesting to others.

The issue, then, is how to revise your research question to make it more interesting to others. (I assume here that you have done the necessary preliminary review of other research on the topic, so you have a basic idea of what others have found before you.) First you should discuss the project with others to find out why it is uninteresting to them. Most likely they will say that your question is uninteresting either because we already know the answer, or because we would not care about the answer even if we knew it. That is:

The no-surprises objection: We won't learn anything from the research because the answer either is already well documented by prior research or is preordained by the research method. In other words, the research is gratuitous, since we know the answer before we do the research. The "so what" objection: The answer to the research question either has no relevance for social science theory or for everyday life, or matters to such a small handful of people that the question is trivial.

"Why is my uncle an alcoholic?" is a good example of a trivial research question. No matter how important the answer might be to you, the question is trivial to social scientists because the answer matters to such a small handful of people. To make the question interesting we must generalize it. Instead of asking why your uncle is an alcoholic, we ask what general traits distinguish alcoholics from others. So the triviality problem generally is solved by casting a wider net in the research question.

Irrelevance is also solved by casting a wider net. In truth, of course, few questions are completely irrelevant; relevance is a matter of degree. As editor of the American Sociological Review I found that one of the most frequent complaints of reviewers was "Why should I care about the results of this paper?" As social scientists we care about research when the results speak to ongoing conversations and debates in our field. The more such connections we can make, the more interesting our results tend to be. So it is important to show how our research question links to theories and other empirical work.

The other objection (no-surprises) is consequential because, as rule 1 states, surprise is a hallmark of social research. It is the possibility of surprise that distinguishes social research from advocacy research. So if your social research project is uninteresting because it lacks the element of surprise, you need to make the case that the answer is not known beforehand. Sometimes results can be known ahead of time (making the research uninteresting) because the results are built in by the way concepts are measured. So if others say your project is uninteresting, you first need to make sure that your method is not to some degree preordaining your result. In examining the effect of marital satisfaction on overall life satisfaction, for example, it is important to find independent measures of the two concepts, so that the measures used do not guarantee a positive relationship between the two.


Excerpted from Seven Rules for Social Research by Glenn Firebaugh
Copyright © 2008 by Princeton University Press. Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

Table of Contents

Preface xi

Chapter 1: The First Rule

There Should Be the Possibility of Surprise in Social Research 1

Selecting a Research Question 2

Researchable Questions 2

Interesting Questions 4

Selecting a Sample 18

Samples in Qualitative Studies 23

Is Meaningful Social Research Possible? 26

Summary 29

Student Exercises on Rule 1 31

Chapter 2: The Second Rule

Look for Differences That Make a Difference, and Report Them 36

You Can't Explain a Variable with a Constant 37

Maximizing Variance to Find the Effect of a Cause 39

Size versus Statistical Significance 41

Comparing Effects Where There Is a Common Metric 42

Calibration: Converting Explanatory Variables to a Common Metric 44

Substantive Profiling: The Use of Telling Comparisons 46

Visual Presentation of Results 51

Policy Importance 53

Importance for Theory 54

Conclusion 56

Student Exercises on Rule 2 58

Chapter 3: The Third Rule

Build Reality Checks into Your Research 64

Internal Reality Checks 65

Reality Checks on Data—Dubious Values and Incomplete Data 65

Reality Checks on Measures—Aim for Consistency in Conceptualization and Measurement 69

Reality Checks on Models—The Formal Equivalence Check 71

External Reality Checks: Validation with Other Data and Methods 76

Using Causal-Process Observations to Test Plausibility of Results 77

Using Ethnographic Data to Help Interpret Survey Results 79

Other Examples of Multiple-Method Research 81

Concluding Remark 82

Student Exercises on Rule 3 84

Chapter 4: The Fourth Rule

Replicate Where Possible 90

Sources of Uncertainty in Social Research 91

Overview: From Population to Sample and Back to Population 93

Measurement Error as a Source of Uncertainty 100

Illustration: Two Methods for Estimating Global Poverty 101

Toward a Solution: Identical Analyses of Parallel Data Sets 105

Meta-analysis: Synthesizing Results Formally across Studies 106

Summary: Your Confidence Intervals Are Too Narrow 109

Student Exercises on Rule 4 111

Chapter 5: The Fifth Rule

Compare Like with Like 120

Correlation and Causality 121

Types of Strategies for Comparing Like with Like 129

Matching versus Looking for Differences 130

The Standard Regression Method for Comparing Like with Like 131

Critique of the Standard Linear Regression Strategy 132

Comparing Like with Like Through Fixed-Effects Methods 134

First-Difference Models: Subtracting Out the Effects of Confounding Variables 134

Special Case: Growth-Rate Models 138

Sibling Models 140

Comparing Like with Like through Matching on Measured Variables 146

Exact Matching 146

Propensity-Score Method 147

Matching as a Preprocessing Strategy for Reducing Model Dependence 151

Comparing Like with Like through Naturally Occurring Random Assignment 152

Instrumental Variables: Matching through Partial Random Assignment 153

Matching Through Naturally Occurring Random Assignment to the Treatment Group 158

Comparison of Strategies for Comparing Like with Like 159

Conclusion 162

Student Exercises on Rule 5 165

Chapter 6: The Sixth Rule

Use Panel Data to Study Individual Change and Repeated Cross-section Data to Study Social Change 172

Analytic Differences between Panel and Repeated Cross-section Data 173

Three General Questions about Change 175

Changing-Effect Models, Part 1: Two Points in Time 176

Changing-Effect Models, Part 2: Multilevel Models with Time as the Context 182

What We Want to Know 183

The General Multilevel Model 183

Convergence Models 185

The Sign Test for Convergence: Comparing Your fs and ds 186

Convergence Model versus Changing-Effect Model 191

Bridging Individual and Social Change: Estimating Cohort Replacement Effects 195

An Accounting Scheme for Social Change 197

Linear Decomposition Method 198

Summary 201

Student Exercises on Rule 6 203

Chapter 7: The Seventh Rule

Let Method Be the Servant, Not the Master 207

Obsession with Regression 209

Naturally Occurring Random Assignment, Again 209

Decomposition Work in the Social Sciences 218

Decomposition of Variance and Inequality 220

Decomposition of Segregation Indexes 222

The Effects of Social Context 226

Context Effects as Objects of Study 227

Context Effects as Nuisance 230

Critical Tests in Social Research 231

Conclusion 235

Student Exercises on Rule 7 236

References 241

Index 253

What People are Saying About This

Christopher Winship

The audience for this book is great. Most graduate programs require second- or third-year students to write some type of research paper. This book is perfect for the task.
Christopher Winship, Harvard University

Dalton Conley

Anyone who wants to learn how to do social research better read this book. Written for the new student and the seasoned researcher alike (one is never too old, after all), Seven Rules for Social Research hits that sweet but till-now-neglected spot between overly simplified methods texts and advanced statistical manuals. Stick with Firebaugh's seven rules and you won't go wrong.
Dalton Conley, New York University

David Strang

A valuable contribution. Firebaugh masterfully surveys a wide variety of key issues at the intersection of statistical theory, research design, and empirical analysis. His book can help improve the quality of social scientific research.
David Strang, Cornell University

Customer Reviews

Most Helpful Customer Reviews

See All Customer Reviews