Albany College of Pharmacy Mastering Metrics D in D Questions .) CLO1h. Identify an example of a natural experiment discussed in MM Ch 5, where a differenc

Albany College of Pharmacy Mastering Metrics D in D Questions .) CLO1h. Identify an example of a natural experiment discussed in MM Ch 5, where a difference-in-difference (D-in-D) in means is a plausible causal effect. Explain how an interaction model automates estimation of a D-in-D estimate. Explain why a difference-in-means analysis risks presenting a biased estimate of the effect of the policy.2.) Is microdata or aggregate data used in calculating the statistics presented in MM Ch 1 Table 1.1 ? What type of data is used in the Richardson and Troost (2009) study of monetary policy, introduced on p. 182 of Mastering Metrics?3.) Who was John Snow? Can you identify a D-in-D estimate from the data presented in MM p. 206? Mastering ’Metrics
Mastering ’Metrics
The Path from Cause to Effect
Joshua D. Angrist
and
Jörn-Steffen Pischke
PRINCETON UNIVERSITY PRESS
▪
PRINCETON AND OXFORD
Copyright © 2015 by Princeton University Press
Published by Princeton University Press, 41 William Street, Princeton, New Jersey 08540
In the United Kingdom: Princeton University Press, 6 Oxford Street, Woodstock, Oxfordshire OX20 1TW
press.princeton.edu
Jacket and illustration design by Wanda Espana
Book illustrations by Garrett Scafani
All Rights Reserved
Library of Congress Cataloging-in-Publication Data
Angrist, Joshua David.
Mastering ’metrics : the path from cause to effect / Joshua D. Angrist, Jörn-Steffen Pischke.
pages cm
Includes index.
Summary: “Applied econometrics, known to aficionados as ’metrics, is the original data science.
’Metrics encompasses the statistical methods economists use to untangle cause and effect in human
affairs. Through accessible discussion and with a dose of kung fu-themed humor, Mastering ’Metrics
presents the essential tools of econometric research and demonstrates why econometrics is exciting and
useful. The five most valuable econometric methods, or what the authors call the Furious Five—
random assignment, regression, instrumental variables, regression discontinuity designs, and
differences in differences-are illustrated through wellcrafted real-world examples (vetted for
awesomeness by Kung Fu Panda’s Jade Palace). Does health insurance make you healthier? Randomized
experiments provide answers. Are expensive private colleges and selective public high schools better
than more pedestrian institutions? Regression analysis and a regression discontinuity design reveal the
surprising truth. When private banks teeter, and depositors take their money and run, should central
banks step in to save them? Differences-in-differences analysis of a Depression-era banking crisis offers
a response. Could arresting O.J. Simpson have saved his ex-wife’s life? Instrumental variables methods
instruct law enforcement authorities in how best to respond to domestic abuse. Wielding econometric
tools with skill and confidence, Mastering ’Metrics uses data and statistics to illuminate the path from
cause to effect. Shows why econometrics is important Explains econometric research through humorous
and accessible discussion Outlines empirical methods central to modern econometric practice Works
through interesting and relevant real-world examples”—Provided by publisher.
ISBN 978-0-691-15283-7 (hardback : alk. paper)—
ISBN 978-0-691-15284-4 (paperback : alk. paper)
1. Econometrics. I. Pischke, Jörn-Steffen. II. Title.
HB139.A53984 2014
330.01′5195—dc23
2014024449
British Library Cataloging-in-Publication Data is available
This book has been composed in Sabon with Helvetica Neue Condensed family display using ZzTEX by
Princeton Editorial Associates Inc., Scottsdale, Arizona
Printed on acid-free paper. ♾
Printed in the United States of America
1 3 5 7 9 10 8 6 4 2
CONTENTS
List of Figures vii
List of Tables ix
Introduction xi
1 Randomized Trials 1
1.1 In Sickness and in Health (Insurance) 1
1.2 The Oregon Trail 24
Masters of ’Metrics: From Daniel to R. A. Fisher 30
Appendix: Mastering Inference 33
2 Regression 47
2.1 A Tale of Two Colleges 47
2.2 Make Me a Match, Run Me a Regression 55
2.3 Ceteris Paribus? 68
Masters of ’Metrics: Galton and Yule 79
Appendix: Regression Theory 82
3 Instrumental Variables 98
3.1 The Charter Conundrum 99
3.2 Abuse Busters 115
3.3 The Population Bomb 123
Masters of ’Metrics: The Remarkable Wrights 139
Appendix: IV Theory 142
4 Regression Discontinuity Designs 147
4.1 Birthdays and Funerals 148
4.2 The Elite Illusion 164
Masters of ’Metrics: Donald Campbell 175
5 Differences-in-Differences 178
5.1 A Mississippi Experiment 178
5.2 Drink, Drank, … 191
Masters of ’Metrics: John Snow 204
Appendix: Standard Errors for Regression DD 205
6 The Wages of Schooling 209
6.1 Schooling, Experience, and Earnings 209
6.2 Twins Double the Fun 217
6.3 Econometricians Are Known by Their … Instruments 223
6.4 Rustling Sheepskin in the Lone Star State 235
Appendix: Bias from Measurement Error 240
Abbreviations and Acronyms 245
Empirical Notes 249
Acknowledgments 269
Index 271
FIGURES
1.1 A standard normal distribution
1.2 The distribution of the t-statistic for the mean in a sample of
size 10
1.3 The distribution of the t-statistic for the mean in a sample of
size 40
1.4 The distribution of the t-statistic for the mean in a sample of
size 100
2.1 The CEF and the regression line
2.2 Variance in X is good
3.1 Application and enrollment data from KIPP Lynn lotteries
3.2 IV in school: the effect of KIPP attendance on math scores
4.1 Birthdays and funerals
4.2 A sharp RD estimate of MLDA mortality effects
4.3 RD in action, three ways
4.4 Quadratic control in an RD design
4.5 RD estimates of MLDA effects on mortality by cause of death
4.6 Enrollment at BLS
4.7 Enrollment at any Boston exam school
4.8 Peer quality around the BLS cutoff
4.9 Math scores around the BLS cutoff
4.10 Thistlethwaite and Campbell’s Visual RD
5.1 Bank failures in the Sixth and Eighth Federal Reserve Districts
5.2 Trends in bank failures in the Sixth and Eighth Federal Reserve
Districts
5.3 Trends in bank failures in the Sixth and Eighth Federal Reserve
Districts, and the Sixth District’s DD counterfactual
5.4 An MLDA effect in states with parallel trends
5.5 A spurious MLDA effect in states where trends are not parallel
5.6 A real MLDA effect, visible even though trends are not parallel
40
41
42
42
83
96
103
108
149
150
154
158
161
166
167
168
172
177
184
185
186
198
198
199
5.7
6.1
6.2
6.3
6.4
John Snow’s DD recipe
The quarter of birth first stage
The quarter of birth reduced form
Last-chance exam scores and Texas sheepskin
The effect of last-chance exam scores on earnings
206
230
230
237
237
TABLES
1.1 Health and demographic characteristics of insured and
uninsured couples in the NHIS
1.2 Outcomes and treatments for Khuzdar and Maria
1.3 Demographic characteristics and baseline health in the RAND
HIE
1.4 Health expenditure and health outcomes in the RAND HIE
1.5 OHP effects on insurance coverage and health-care use
1.6 OHP effects on health indicators and financial health
2.1 The college matching matrix
2.2 Private school effects: Barron’s matches
2.3 Private school effects: Average SAT score controls
2.4
2.5
3.1
3.2
3.3
3.4
3.5
4.1
5.1
5.2
5.3
6.1
6.2
6.3
6.4
School selectivity effects: Average SAT score controls
Private school effects: Omitted variables bias
Analysis of KIPP lotteries
The four types of children
Assigned and delivered treatments in the MDVE
Quantity-quality first stages
OLS and 2SLS estimates of the quantity-quality trade-off
Sharp RD estimates of MLDA effects on mortality
Wholesale firm failures and sales in 1929 and 1933
Regression DD estimates of MLDA effects on death rates
Regression DD estimates of MLDA effects controlling for beer
taxes
How bad control creates selection bias
Returns to schooling for Twinsburg twins
Returns to schooling using child labor law instruments
IV recipe for an estimate of the returns to schooling using a
single quarter of birth instrument
6.5 Returns to schooling using alternative quarter of birth
5
7
20
23
27
28
53
63
66
67
76
104
112
117
135
137
160
190
196
201
216
220
226
231
instruments
232
INTRODUCTION
BLIND MASTER PO:
Close your eyes. What do you hear?
YOUNG KWAI CHANG CAINE:
MASTER PO:
Do you hear your own heartbeat?
KWAI CHANG CAINE:
MASTER PO:
I hear the water, I hear the birds.
No.
Do you hear the grasshopper that is at your feet?
KWAI CHANG CAINE:
Old man, how is it that you hear these things?
Young man, how is it that you do not?
Kung Fu, Pilot
MASTER PO:
Economists’ reputation for dismality is a bad rap. Economics is as exciting
as any science can be: the world is our lab, and the many diverse people in it
are our subjects.
The excitement in our work comes from the opportunity to learn about
cause and effect in human affairs. The big questions of the day are our
questions: Will loose monetary policy spark economic growth or just fan the
fires of inflation? Iowa farmers and the Federal Reserve chair want to know.
Will mandatory health insurance really make Americans healthier? Such
policy kindling lights the fires of talk radio. We approach these questions
coolly, however, armed not with passion but with data.
Economists’ use of data to answer cause-and-effect questions constitutes the
field of applied econometrics, known to students and masters alike as ’metrics.
The tools of the ’metrics trade are disciplined data analysis, paired with the
machinery of statistical inference. There is a mystical aspect to our work as
well: we’re after truth, but truth is not revealed in full, and the messages the
data transmit require interpretation. In this spirit, we draw inspiration from
the journey of Kwai Chang Caine, hero of the classic Kung Fu TV series.
Caine, a mixed-race Shaolin monk, wanders in search of his U.S.-born half-
brother in the nineteenth century American West. As he searches, Caine
questions all he sees in human affairs, uncovering hidden relationships and
deeper meanings. Like Caine’s journey, the Way of ’Metrics is illuminated by
questions.
Other Things Equal
In a disturbing development you may have heard of, the proportion of
American college students completing their degrees in a timely fashion has
taken a sharp turn south. Politicians and policy analysts blame falling college
graduation rates on a pernicious combination of tuition hikes and the large
student loans many students use to finance their studies. Perhaps increased
student borrowing derails some who would otherwise stay on track. The fact
that the students most likely to drop out of school often shoulder large
student loans would seem to substantiate this hypothesis.
You’d rather pay for school with inherited riches than borrowed money if
you can. As we’ll discuss in detail, however, education probably boosts
earnings enough to make loan repayment bearable for most graduates. How
then should we interpret the negative correlation between debt burden and
college graduation rates? Does indebtedness cause debtors to drop out? The
first question to ask in this context is who borrows the most. Students who
borrow heavily typically come from middle and lower income families, since
richer families have more savings. For many reasons, students from lower
income families are less likely to complete a degree than those from higher
income families, regardless of whether they’ve borrowed heavily. We should
therefore be skeptical of claims that high debt burdens cause lower college
completion rates when these claims are based solely on comparisons of
completion rates between those with more or less debt. By virtue of the
correlation between family background and college debt, the contrast in
graduation rates between those with and without student loans is not an other
things equal comparison.
As college students majoring in economics, we first learned the other things
equal idea by its Latin name, ceteris paribus. Comparisons made under ceteris
paribus conditions have a causal interpretation. Imagine two students
identical in every way, so their families have the same financial resources
and their parents are similarly educated. One of these virtual twins finances
college by borrowing and the other from savings. Because they are otherwise
equal in every way (their grandmother has treated both to a small nest egg),
differences in their educational attainment can be attributed to the fact that
only one has borrowed. To this day, we wonder why so many economics
students first encounter this central idea in Latin; maybe it’s a conspiracy to
keep them from thinking about it. Because, as this hypothetical comparison
suggests, real other things equal comparisons are hard to engineer, some
would even say impossibile (that’s Italian not Latin, but at least people still
speak it).
Hard to engineer, maybe, but not necessarily impossible. The ’metrics craft
uses data to get to other things equal in spite of the obstacles—called selection
bias or omitted variables bias—found on the path running from raw numbers
to reliable causal knowledge. The path to causal understanding is rough and
shadowed as it snakes around the boulders of selection bias. And yet, masters
of ’metrics walk this path with confidence as well as humility, successfully
linking cause and effect.
Our first line of attack on the causality problem is a randomized
experiment, often called a randomized trial. In a randomized trial,
researchers change the causal variables of interest (say, the availability of
college financial aid) for a group selected using something like a coin toss. By
changing circumstances randomly, we make it highly likely that the variable
of interest is unrelated to the many other factors determining the outcomes
we mean to study. Random assignment isn’t the same as holding everything
else fixed, but it has the same effect. Random manipulation makes other things
equal hold on average across the groups that did and did not experience
manipulation. As we explain in Chapter 1, “on average” is usually good
enough.
Randomized trials take pride of place in our ’metrics toolkit. Alas,
randomized social experiments are expensive to field and may be slow to
bear fruit, while research funds are scarce and life is short. Often, therefore,
masters of ’metrics turn to less powerful but more accessible research designs.
Even when we can’t practicably randomize, however, we still dream of the
trials we’d like to do. The notion of an ideal experiment disciplines our
approach to econometric research. Mastering ’Metrics shows how wise
application of our five favorite econometric tools brings us as close as
possible to the causality-revealing power of a real experiment.
Our favorite econometric tools are illustrated here through a series of wellcrafted and important econometric studies. Vetted by Grand Master Oogway
of Kung Fu Panda’s Jade Palace, these investigations of causal effects are
distinguished by their awesomeness. The methods they use—random
assignment, regression, instrumental variables, regression discontinuity
designs, and differences-in-differences—are the Furious Five of econometric
research. For starters, motivated by the contemporary American debate over
health care, the first chapter describes two social experiments that reveal
whether, as many policymakers believe, health insurance indeed helps those
who have it stay healthy. Chapters 2–5 put our other tools to work, crafting
answers to important questions ranging from the benefits of attending private
colleges and selective high schools to the costs of teen drinking and the
effects of central bank injections of liquidity.
Our final chapter puts the Furious Five to the test by returning to the
education arena. On average, college graduates earn about twice as much as
high school graduates, an earnings gap that only seems to be growing.
Chapter 6 asks whether this gap is evidence of a large causal return to
schooling or merely a reflection of the many other advantages those with
more education might have (such as more educated parents). Can the
relationship between schooling and earnings ever be evaluated on a ceteris
paribus basis, or must the boulders of selection bias forever block our way?
The challenge of quantifying the causal link between schooling and earnings
provides a gripping test match for ’metrics tools and the masters who wield
them.
Mastering ’Metrics
Chapter 1
Randomized Trials
What happens in a man’s life is already written. A man
must move through life as his destiny wills.
KWAI CHANG CAINE:
Yet each is free to live as he chooses. Though they seem
opposite, both are true.
Kung Fu, Pilot
OLD MAN:
Our Path
Our path begins with experimental random assignment, both as a framework
for causal questions and a benchmark by which the results from other
methods are judged. We illustrate the awesome power of random assignment
through two randomized evaluations of the effects of health insurance. The
appendix to this chapter also uses the experimental framework to review the
concepts and methods of statistical inference.
1.1 In Sickness and in Health (Insurance)
The Affordable Care Act (ACA) has proven to be one of the most
controversial and interesting policy innovations we’ve seen. The ACA requires
Americans to buy health insurance, with a tax penalty for those who don’t
voluntarily buy in. The question of the proper role of government in the
market for health care has many angles. One is the causal effect of health
insurance on health. The United States spends more of its GDP on health care
than do other developed nations, yet Americans are surprisingly unhealthy.
For example, Americans are more likely to be overweight and die sooner than
their Canadian cousins, who spend only about two-thirds as much on care.
America is also unusual among developed countries in having no universal
health insurance scheme. Perhaps there’s a causal connection here.
Elderly Americans are covered by a federal program called Medicare, while
some poor Americans (including most single mothers, their children, and
many other poor children) are covered by Medicaid. Many of the working,
prime-age poor, however, have long been uninsured. In fact, many uninsured
Americans have chosen not to participate in an employer-provided insurance
plan.1 These workers, perhaps correctly, count on hospital emergency
departments, which cannot turn them away, to address their health-care
needs. But the emergency department might not be the best place to treat,
say, the flu, or to manage chronic conditions like diabetes and hypertension
that are so pervasive among poor Americans. The emergency department is
not required to provide long-term care. It therefore stands to reason that
government-mandated health insurance might yield a health dividend. The
push for subsidized universal health insurance stems in part from the belief
that it does.
The ceteris paribus question in this context contrasts the health of someone
with insurance coverage to the health of the same person were they without
insurance (other than an emergency department backstop). This contrast
highlights a fundamental empirical conundrum: people are either insured or
not. We don’t get to see them both ways, at least not at the same time in
exactly the same circumstances.
In his celebrated poem, “The Road Not Taken,” Robert Frost used the
metaphor of a crossroads to describe the causal effects of personal choice:
Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;
Frost’s traveler concludes:
Two roads diverged in a wood, and I—
I took the one less traveled by,
And that has made all the difference.
The traveler claims his choice has mattered, but, being only one person, he
can’t be sure. A later trip or a report by other travelers won’t nail it down for
him, either. Our narrator might be older and wiser the second time around,
while other travelers might have different experiences on the same road. So it
is with any choice, including those related to health insurance: would
uninsured men with heart disease be disease-free if they had insurance? In
the novel Light Years, James Salter’s irresolute narrator observes: “Acts
demolish their alternatives, that is the paradox.” We can’t know what lies at
the end of the road not taken.
We can’t know, but evidence can be brought to bear on the question. This
chapter takes you through some of the evidence related to paths involving
health insurance. The starting point is the National Health Interview Survey
…
Purchase answer to see full
attachment