Tufte Without Tears - University of Washington

Tufte Without Tears - University of Washington
Tufte Without Tears
Flexible tools for visual exploration
and presentation of statistical models
Christopher Adolph
Department of Political Science
and
Center for Statistics and the Social Sciences
University of Washington, Seattle
Chris Adolph (UW)
Tufte without tears
1 / 106
Edward Tufte & Scientific Visuals
In several beautifully illustrated books Edward Tufte gives the following advice:
1
Assume the reader’s interest and intelligence
Chris Adolph (UW)
Tufte without tears
2 / 106
Edward Tufte & Scientific Visuals
In several beautifully illustrated books Edward Tufte gives the following advice:
1
Assume the reader’s interest and intelligence
2
Maximize information, minimize ink & space: the data-ink ratio
Chris Adolph (UW)
Tufte without tears
2 / 106
Edward Tufte & Scientific Visuals
In several beautifully illustrated books Edward Tufte gives the following advice:
1
Assume the reader’s interest and intelligence
2
Maximize information, minimize ink & space: the data-ink ratio
3
Show the underlying data and facilitate comparisons
Chris Adolph (UW)
Tufte without tears
2 / 106
Edward Tufte & Scientific Visuals
In several beautifully illustrated books Edward Tufte gives the following advice:
1
Assume the reader’s interest and intelligence
2
Maximize information, minimize ink & space: the data-ink ratio
3
Show the underlying data and facilitate comparisons
4
Use small multiples: repetitions of a basic design
Tufte’s recommended medical chart illustrates many of these ideas:
Chris Adolph (UW)
Tufte without tears
2 / 106
This chart is annotated for pedagogical purposes
Lots of information; little distracting scaffolding
A model that can be repeated once learned. . .
Chris Adolph (UW)
Tufte without tears
3 / 106
A complete layout using small multiples to convey lots of info
Elegant, information-rich. . . and hard to make
Chris Adolph (UW)
Tufte without tears
4 / 106
What most discussions of statistical graphics leave out
Tufte’s books have had a huge impact on information visualization
However, they have two important limits:
Modeling Most examples are either exploratory or very simple models;
Social scientists want cutting edge applications
Tools Need to translate aesthetic guidelines into software
Social scientists are unlikely to do this on their own—
and shouldn’t have to!
Chris Adolph (UW)
Tufte without tears
5 / 106
Key problem Ready-to-use techniques to visually present model results:
for many variables
Chris Adolph (UW)
Tufte without tears
6 / 106
Key problem Ready-to-use techniques to visually present model results:
for many variables
for many robustness checks
Chris Adolph (UW)
Tufte without tears
6 / 106
Key problem Ready-to-use techniques to visually present model results:
for many variables
for many robustness checks
showing uncertainty
Chris Adolph (UW)
Tufte without tears
6 / 106
Key problem Ready-to-use techniques to visually present model results:
for many variables
for many robustness checks
showing uncertainty
without accidental extrapolation
Chris Adolph (UW)
Tufte without tears
6 / 106
Key problem Ready-to-use techniques to visually present model results:
for many variables
for many robustness checks
showing uncertainty
without accidental extrapolation
for an audience without deep statistical knowledge
Not covered here The theory behind effective visual display of data
Visual displays for data (not model) exploration
For these and other topics, and a reading list, see my course at
faculty.washington.edu/cadolph/vis
Lots of examples. . . Too many if we need to discuss methods in detail
Chris Adolph (UW)
Tufte without tears
6 / 106
Who votes in American elections?
Source: King, Tomz, and Wittenberg
Method: Logistic regression
What do alligators eat?
Source: Agresti
Method: Multinomial logit
How do Chinese leaders gain power?
Source: Shih, Adolph, and Liu
Method: Bayesian model of partially observed ranks
When do governments choose liberal or conservative central bankers?
Source: Adolph
Method: Zero-inflated compositional data model
What explains the tier of European governments controlling health policies?
Source: Adolph, Greer, and Fonseca
Method: Multilevel multinomial logit
Chris Adolph (UW)
Tufte without tears
7 / 106
Presenting Estimated Models in Social Science
Most empirical work in social science is regression model-driven,
with a focus on conditional expectation
Our regression models are
full of covariates
often non-linear
usually involve interactions and transformations
If there is anything we need to visualize well, it is our models
Yet we often just print off tables of parameter estimates
Limits readers’ and analysts’ understanding of the results
Chris Adolph (UW)
Tufte without tears
8 / 106
Coefficients are not enough
Some limits of typical presentations of statistical results:
Everything written in terms of arcane intermediate quantites
(for most people, this includes logit coefficients)
Little effort to transform results to the scale of the quantities of interest
→ really want the conditional expectation, E(y |x)
Little effort to make informative statements about estimation uncertainty
→ really want to know how uncertain is E(y |x)
Little visualization at all, or graphs with low data-ink ratios
Chris Adolph (UW)
Tufte without tears
9 / 106
Voting Example (Logit Model)
We will explore a simple dataset using a simple model of voting
People either vote (Votei = 1), or they don’t (Votei = 0)
Many factors could influence turn-out; we focus on age and education
Data from National Election Survey in 2000. “Did you vote in 2000 election?”
[1,]
[2,]
[3,]
[4,]
[5,]
[6,]
[7,]
[8,]
[9,]
[10,]
...
vote00 age hsdeg coldeg
1 49
1
0
0 35
1
0
1 57
1
0
1 63
1
0
1 40
1
0
1 77
0
0
0 43
1
0
1 47
1
1
1 26
1
1
1 48
1
0
Chris Adolph (UW)
Tufte without tears
10 / 106
Logit of Decision to Vote, 2000 Presidential NES
est.
Age
Age2
High School Grad
College Grad
Constant
0.074
−0.0004
1.168
1.085
−3.05
s.e.
0.017
0.0002
0.178
0.131
0.418
p-value
0.000
0.009
0.000
0.000
0.000
Age enters as a quadratic to allow the probability of voting to first rise and
eventually fall over the life course
Results look sensible, but what do they mean?
Which has the bigger effect, age or education?
What is the probability a specific person will vote?
Chris Adolph (UW)
Tufte without tears
11 / 106
An alternative to printing eye-glazing tables
1
Run your model as normal. Treat the output as an intermediate step.
Chris Adolph (UW)
Tufte without tears
12 / 106
An alternative to printing eye-glazing tables
1
Run your model as normal. Treat the output as an intermediate step.
2
Translate your model results back into the scale of the response variable
I
Modeling war? Show the change in probability of war associated with X
I
Modeling counts of crimes committed? Show how those counts vary with X
I
Unemployment rate time series? Show how a change in X shifts the
unemployment rate over the following t years
Chris Adolph (UW)
Tufte without tears
12 / 106
An alternative to printing eye-glazing tables
1
Run your model as normal. Treat the output as an intermediate step.
2
Translate your model results back into the scale of the response variable
3
I
Modeling war? Show the change in probability of war associated with X
I
Modeling counts of crimes committed? Show how those counts vary with X
I
Unemployment rate time series? Show how a change in X shifts the
unemployment rate over the following t years
Calculate or simulate the uncertainty in these final quantities of interest
Chris Adolph (UW)
Tufte without tears
12 / 106
An alternative to printing eye-glazing tables
1
Run your model as normal. Treat the output as an intermediate step.
2
Translate your model results back into the scale of the response variable
I
Modeling war? Show the change in probability of war associated with X
I
Modeling counts of crimes committed? Show how those counts vary with X
I
Unemployment rate time series? Show how a change in X shifts the
unemployment rate over the following t years
3
Calculate or simulate the uncertainty in these final quantities of interest
4
Present visually as many scenarios calculated from the model as needed
Chris Adolph (UW)
Tufte without tears
12 / 106
A bit more formally. . .
We want to know the behavior of E(y |x) as we vary x.
In non-linear models with multiple regressors, this gets tricky.
The effect of x1 depends on all the other x’s and β̂’s
Generally, we will need to make a set of “counterfactual” assumptions:
x1 = a,
x2 = b,
x3 = c, . . .
Choose a, b, c, . . . to match a particular counterfactual case of interest or
Hold all but one of the x’s at their mean values (or other reference
baseline), then systematically vary the remaining x.
The same trick works if we are after differences in y related to changes in x,
such as E(yscen1 − yscen2 |xscen1 , xscen2 )
Chris Adolph (UW)
Tufte without tears
13 / 106
Calculating quantities of interest
Our goal to obtain “quantities of interest,” like
Expected Values: E(Y |Xc )
Differences: E(Y |Xc2 ) − E(Y |Xc1 )
Risk Ratios: E(Y |Xc2 )/E(Y |Xc1 )
or any other function of the above
for some counterfactual Xc ’s.
For our Voting example, that’s easy—just plug Xc into
E(Y |Xc ) =
Chris Adolph (UW)
1
1 + exp(−Xc β)
Tufte without tears
14 / 106
Getting confidence intervals is harder, but there are several options:
For maximum likelihood models,
simulate the response conditional on the regressors
See King, Tomz, and Wittenberg, 2000, American Journal of Political
Science, and the Zelig package for R or Clarify for Stata.
These simulations can easily be summarized as CIs: sort them and take
percentiles
For Bayesian models, usual model output is a set of posterior draws
See Andrew Gelman and Jennifer Hill, 2006, Data Analysis Using
Hierarchical/ Multilevel Models, Cambridge UP.
Once we have the quantities of interest and confidence intervals, we’re ready
to make some graphs. . . but how?
Chris Adolph (UW)
Tufte without tears
15 / 106
Here is the graph that King, Tomz, and Wittenberg created for this model
How would we make this?
Chris Adolph (UW)
Tufte without tears
16 / 106
We could use the default graphics in Zelig or Clarify (limiting, not as nice
as the above)
Or we could do it by hand (hard)
Chris Adolph (UW)
Tufte without tears
17 / 106
Wanted: an easy-to-use R package that
1
takes as input the output of estimated statistical models
2
makes a variety of plots for model interpretation
3
plots “triples” (lower, estimate, upper) from estimated models well
4
lays out these plot in a tiled arrangement (small multiples)
5
takes care of axes, titles, and other fussy details
Chris Adolph (UW)
Tufte without tears
18 / 106
Wanted: an easy-to-use R package that
1
takes as input the output of estimated statistical models
2
makes a variety of plots for model interpretation
3
plots “triples” (lower, estimate, upper) from estimated models well
4
lays out these plot in a tiled arrangement (small multiples)
5
takes care of axes, titles, and other fussy details
With considerable work, one could
coerce R’s basic graphics to do this badly
or get lattice to do this fairly well for a specific case
But an easy-to-use, general solution is lacking
Chris Adolph (UW)
Tufte without tears
18 / 106
The tile package
My answer is the tile package, written using R’s grid graphics
Some basic tile graphic types:
scatter
lineplot
ropeladder
Scatterplots with fits, CIs, and extrapolation checking
Line plots with fits, CIs, and extrapolation checking
Dot plots with CIs and extrapolation checking
Each can take as input draws from the posterior of a regression model
A call to a tile function makes a multiplot layout:
ideal for small multiples of model parameters
Chris Adolph (UW)
Tufte without tears
19 / 106
Column Title 1
Column Title 2
Column Title 3
Plot Title 1
Plot Title 2
Plot Title 3
Top Axis 2
1 5 10
20
35
1
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
1
0
5 10 20 35
X Axis 1
Under Title 1
Under Title 2
Plot Title 4
Top Axis 4
1 5 10
20
Plot Title 5
35
1
0.4
0.2
0
Under Title 4
Chris Adolph (UW)
0.4
0.2
0
1
5 10 20 35
X Axis 3
Under Title 3
Plot Title 6
35
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0
Right Axis 6
Row Title 2
Right Axis 4
0.6
0.6
Top Axis 6
1 5 10
20
1
0.8
1
0.8
Right Axis 2
Row Title 1
Y Axis 1
1
0.8
0.2
1
0
5 10 20 35
X Axis 5
Under Title 5
Tufte without tears
Under Title 6
20 / 106
Plot simulations of QoI
Generally, we want to plot triples: lower, estimate, upper
We could do this for specific discrete scenarios, e.g.
Pr(Voting) given five distinct sets of x’s
Recommended plot: Dotplot with confidence interval lines
Chris Adolph (UW)
Tufte without tears
21 / 106
Plot simulations of QoI
Generally, we want to plot triples: lower, estimate, upper
We could do this for specific discrete scenarios, e.g.
Pr(Voting) given five distinct sets of x’s
Recommended plot: Dotplot with confidence interval lines
Or for a continuous stream of scenarios, e.g.,
Hold all but Age constant, then calculate Pr(Voting) at every level of Age
Recommended plot: Lineplot with shaded confidence intervals
Chris Adolph (UW)
Tufte without tears
21 / 106
This example is obviously superior to the table of logit coefficients
But is there anything wrong or missing here?
Chris Adolph (UW)
Tufte without tears
22 / 106
1
College
Probability of Voting
0.8
0.6
High School
0.4
Less than HS
0.2
Logit estimates:
95% confidence
interval is shaded
0
20
30
40
50
60
70
Age of Respondent
80
90
18 year old college grads?! And what about high school dropouts?
Chris Adolph (UW)
Tufte without tears
23 / 106
1
College
Probability of Voting
0.8
0.6
High School
0.4
Less than HS
0.2
Logit estimates:
95% confidence
interval is shaded
0
20
30
40
50
60
70
Age of Respondent
80
90
tile helps us systematize plotting model results,
and helps avoid unwanted extrapolation by limiting results to the convex hull
Chris Adolph (UW)
Tufte without tears
24 / 106
Three steps to make tile plots
1
Create data traces. Each trace contains the data and graphical
parameters needed to plot a single set of graphical elements to one or
more plots.
Chris Adolph (UW)
Tufte without tears
25 / 106
Three steps to make tile plots
1
Create data traces. Each trace contains the data and graphical
parameters needed to plot a single set of graphical elements to one or
more plots.
I
Could be a set of points, or text labels, or lines, or a polygon
Chris Adolph (UW)
Tufte without tears
25 / 106
Three steps to make tile plots
1
Create data traces. Each trace contains the data and graphical
parameters needed to plot a single set of graphical elements to one or
more plots.
I
Could be a set of points, or text labels, or lines, or a polygon
I
Could be a set of points and symbols, colors, labels, fit line, CIs, and/or
extrapolation limits
Chris Adolph (UW)
Tufte without tears
25 / 106
Three steps to make tile plots
1
Create data traces. Each trace contains the data and graphical
parameters needed to plot a single set of graphical elements to one or
more plots.
I
Could be a set of points, or text labels, or lines, or a polygon
I
Could be a set of points and symbols, colors, labels, fit line, CIs, and/or
extrapolation limits
I
Could be the data for a dotchart, with labels for each line
Chris Adolph (UW)
Tufte without tears
25 / 106
Three steps to make tile plots
1
Create data traces. Each trace contains the data and graphical
parameters needed to plot a single set of graphical elements to one or
more plots.
I
Could be a set of points, or text labels, or lines, or a polygon
I
Could be a set of points and symbols, colors, labels, fit line, CIs, and/or
extrapolation limits
I
Could be the data for a dotchart, with labels for each line
I
Could be the marginal data for a rug
I
All annotation must happen in this step
Chris Adolph (UW)
Tufte without tears
25 / 106
Three steps to make tile plots
1
Create data traces. Each trace contains the data and graphical
parameters needed to plot a single set of graphical elements to one or
more plots.
I
Could be a set of points, or text labels, or lines, or a polygon
I
Could be a set of points and symbols, colors, labels, fit line, CIs, and/or
extrapolation limits
I
Could be the data for a dotchart, with labels for each line
I
Could be the marginal data for a rug
I
All annotation must happen in this step
I
Basic traces: linesTile(), pointsile(), polygonTile(),
polylinesTile(), and textTile()
I
Complex traces: lineplot(), scatter(), ropeladder(), and
rugTile()
Chris Adolph (UW)
Tufte without tears
25 / 106
Trace functions in tile
Primitive trace functions:
linesTile
Plot a set of connected line segments
pointsTile
Plot a set of points
polygonTile
Plot a shaded region
polylinesTile Plot a set of unconnected line segments
textTile
Plot text labels
Complex traces for model or data exploration:
lineplot
ropeladder
rugTile
scatter
Chris Adolph (UW)
Plot lines with confidence intervals, extrapolation warnings
Plot dotplots with confidence intervals, extrapolation warnings,
and shaded ranges
Plot marginal data rugs to axes of plots
Plot scatterplots with text and symbol markers,
fit lines, and confidence intervals
Tufte without tears
26 / 106
Three steps to make tile plots
1
Create data traces. Each trace contains the data and graphical
parameters needed to plot a single set of graphical elements to one or
more plots.
2
Plot the data traces. Using the tile() function, simultaneously plot all
traces to all plots.
Chris Adolph (UW)
Tufte without tears
27 / 106
Three steps to make tile plots
1
Create data traces. Each trace contains the data and graphical
parameters needed to plot a single set of graphical elements to one or
more plots.
2
Plot the data traces. Using the tile() function, simultaneously plot all
traces to all plots.
I
This is the step where the scaffolding gets made: axes and titles
I
Set up the rows and columns of plots
I
Titles of plots, axes, rows of plots, columns of plots, etc.
I
Set up axis limits, ticks, tick labels, logging of axes
Chris Adolph (UW)
Tufte without tears
27 / 106
Three steps to make tile plots
1
Create data traces. Each trace contains the data and graphical
parameters needed to plot a single set of graphical elements to one or
more plots.
2
Plot the data traces. Using the tile() function, simultaneously plot all
traces to all plots.
3
Examine output and revise. Look at the graph made in step 2, and
tweak the input parameters for steps 1 and 2 to make a better graph.
Chris Adolph (UW)
Tufte without tears
28 / 106
R Syntax for Lineplot of Voting Logit
# Using simcf and tile to explore an estimated logistic regression
# Voting example using 2000 NES data after King, Tomz, and Wittenberg
# Chris Adolph
# Load libraries
library(RColorBrewer)
library(MASS)
library(simcf)
library(tile)
# Download simcf and tile packages
# from faculty.washington.edu/cadolph
# Load data (available from faculty.washington.edu/cadolph)
file <- "nes00.csv"
data <- read.csv(file, header=TRUE)
Chris Adolph (UW)
Tufte without tears
29 / 106
R Syntax for Lineplot of Voting Logit
# Set up model formula and model specific data frame
model <- vote00 ~ age + I(age^2) + hsdeg + coldeg
mdata <- extractdata(model, data, na.rm=TRUE)
# Run logit & extract results
logit.result <- glm(model, family=binomial, data=mdata)
pe <- logit.result$coefficients # point estimates
vc <- vcov(logit.result)
# var-cov matrix
# Simulate parameter distributions
sims <- 10000
simbetas <- mvrnorm(sims, pe, vc)
Chris Adolph (UW)
Tufte without tears
30 / 106
R Syntax for Lineplot of Voting Logit
# Set up counterfactuals: all ages, each of three educations
xhyp <- seq(18,97,1)
nscen <- length(xhyp)
nohsScen <- hsScen <- collScen <- cfMake(model, mdata, nscen)
for (i in 1:nscen) {
# No High school scenarios (loop over each age)
nohsScen <- cfChange(nohsScen, "age", x = xhyp[i], scen = i)
nohsScen <- cfChange(nohsScen, "hsdeg", x = 0, scen = i)
nohsScen <- cfChange(nohsScen, "coldeg", x = 0, scen = i)
# HS grad
hsScen <hsScen <hsScen <-
scenarios (loop over each age)
cfChange(hsScen, "age", x = xhyp[i], scen = i)
cfChange(hsScen, "hsdeg", x = 1, scen = i)
cfChange(hsScen, "coldeg", x = 0, scen = i)
# College grad scenarios (loop
collScen <- cfChange(collScen,
collScen <- cfChange(collScen,
collScen <- cfChange(collScen,
over each age)
"age", x = xhyp[i], scen = i)
"hsdeg", x = 1, scen = i)
"coldeg", x = 1, scen = i)
}
Chris Adolph (UW)
Tufte without tears
31 / 106
R Syntax for Lineplot of Voting Logit
# Simulate expected probabilities for all scenarios
nohsSims <- logitsimev(nohsScen, simbetas, ci=0.95)
hsSims <- logitsimev(hsScen, simbetas, ci=0.95)
collSims <- logitsimev(collScen, simbetas, ci=0.95)
# Get 3 nice colors for traces
col <- brewer.pal(3,"Dark2")
# Set up lineplot traces of expected probabilities
nohsTrace <- lineplot(x=xhyp,
y=nohsSims$pe,
lower=nohsSims$lower,
upper=nohsSims$upper,
col=col[1],
extrapolate=list(data=mdata[,2:ncol(mdata)],
cfact=nohsScen$x[,2:ncol(hsScen$x)],
omit.extrapolated=TRUE),
plot=1)
Chris Adolph (UW)
Tufte without tears
32 / 106
R Syntax for Lineplot of Voting Logit
hsTrace <- lineplot(x=xhyp,
y=hsSims$pe,
lower=hsSims$lower,
upper=hsSims$upper,
col=col[2],
extrapolate=list(data=mdata[,2:ncol(mdata)],
cfact=hsScen$x[,2:ncol(hsScen$x)],
omit.extrapolated=TRUE),
plot=1)
collTrace <- lineplot(x=xhyp,
y=collSims$pe,
lower=collSims$lower,
upper=collSims$upper,
col=col[3],
extrapolate=list(data=mdata[,2:ncol(mdata)],
cfact=collScen$x[,2:ncol(hsScen$x)],
omit.extrapolated=TRUE),
plot=1)
Chris Adolph (UW)
Tufte without tears
33 / 106
R Syntax for Lineplot of Voting Logit
# Set up traces with labels and legend
labelTrace <- textTile(labels=c("Less than HS", "High School",
"College"),
x=c( 55,
49,
30),
y=c( 0.26, 0.56,
0.87),
col=col,
plot=1)
legendTrace <- textTile(labels=c("Logit estimates:", "95% confidence",
"interval is shaded"),
x=c(82, 82, 82),
y=c(0.2, 0.16, 0.12),
plot=1)
Chris Adolph (UW)
Tufte without tears
34 / 106
R Syntax for Lineplot of Voting Logit
# Plot traces using tile
tile(nohsTrace,
hsTrace,
collTrace,
labelTrace,
legendTrace,
width=list(null=5),
limits=c(18,94,0,1),
xaxis=list(at=c(20,30,40,50,60,70,80,90)),
xaxistitle=list(labels="Age of Respondent"),
yaxistitle=list(labels="Probability of Voting"),
frame=TRUE
)
Chris Adolph (UW)
Tufte without tears
35 / 106
Chomp! Coefficient tables can hide the punchline
Agresti offers the following example of a multinominal data analysis
Alligators in a certain Florida lake were studied, and the following data
collected:
Principal Food
1 = Invertebrates,
2 = Fish,
3= “Other” (!!! Floridians?)
Size of alligator
Sex of alligator
in meters
male or female
The question is how alligator size influences food choice.
We fit the model in R using multinomial logit and get . . .
Chris Adolph (UW)
Tufte without tears
36 / 106
Chomp! Coefficient tables can hide the punchline
Multinomial Logit of Alligator’s Primary Food Source
Intercept
Size
Female
ln(πInvertebrates
/πFish )
ln(πOther
/πFish )
4.90
(1.71)
−2.53
(0.85)
−0.79
(0.71)
−1.95
(1.53)
0.13
(0.52)
0.38
(0.91)
Direct interpretation?
Chris Adolph (UW)
Tufte without tears
37 / 106
Chomp! Coefficient tables can hide the punchline
Multinomial Logit of Alligator’s Primary Food Source
Intercept
Size
Female
ln(πInvertebrates
/πFish )
ln(πOther
/πFish )
4.90
(1.71)
−2.53
(0.85)
−0.79
(0.71)
−1.95
(1.53)
0.13
(0.52)
0.38
(0.91)
Direct interpretation?
We could do it, using odds ratios and a calculator.
Invertebrates vs Fish: A 1 meter increase in length makes the odds that an
alligator will eat invertebrates rather than fish exp(−2.53 − 0) = 0.08 times
smaller.
Chris Adolph (UW)
Tufte without tears
37 / 106
Chomp! Coefficient tables can hide the punchline
Multinomial Logit of Alligator’s Primary Food Source
Intercept
Size
Female
ln(πInvertebrates
/πFish )
ln(πOther
/πFish )
4.90
(1.71)
−2.53
(0.85)
−0.79
(0.71)
−1.95
(1.53)
0.13
(0.52)
0.38
(0.91)
Invertebrates vs Other: A 1 meter increase in length makes the odds that an
alligator will eat invertebrates rather than “other” food
exp(−2.53 − 0.13) = 0.07 times smaller.
Chris Adolph (UW)
Tufte without tears
38 / 106
Chomp! Coefficient tables can hide the punchline
Multinomial Logit of Alligator’s Primary Food Source
Intercept
Size
Female
ln(πInvertebrates
/πFish )
ln(πOther
/πFish )
4.90
(1.71)
−2.53
(0.85)
−0.79
(0.71)
−1.95
(1.53)
0.13
(0.52)
0.38
(0.91)
Invertebrates vs Other: A 1 meter increase in length makes the odds that an
alligator will eat invertebrates rather than “other” food
exp(−2.53 − 0.13) = 0.07 times smaller.
No reader is going to do this
Chris Adolph (UW)
Tufte without tears
38 / 106
Chomp! Coefficient tables can hide the punchline
Multinomial Logit of Alligator’s Primary Food Source
Intercept
Size
Female
ln(πInvertebrates
/πFish )
ln(πOther
/πFish )
4.90
(1.71)
−2.53
(0.85)
−0.79
(0.71)
−1.95
(1.53)
0.13
(0.52)
0.38
(0.91)
Fish vs Other: A 1 meter increase in length makes the odds that an alligator
will eat fish rather than “other” food exp(0 − 0.13) = 0.87 times smaller.
There has to be a better way
Chris Adolph (UW)
Tufte without tears
39 / 106
Food choice by alligator size and sex
1
1
Pr(Food preference)
Invertebrates
0.8
0.8
Fish
0.6
Fish
0.6
0.4
0.4
Other
0.2
0
Invertebrates
1
2
Other
0.2
3
4
Size of alligator
0
Male
1
2
3
Size of alligator
4
Female
A simple lineplot made with tile shows the whole covariate space
Much more dramatic than the table
Chris Adolph (UW)
Tufte without tears
40 / 106
Food choice by alligator size and sex
Pr(Food preference)
1
Invertebrates
1
Fish
0.8
0.8
0.6
0.6
0.4
0.4
Other
0.2
0.2
0
0
1
2
3
Invertebrates
4
Fish
Other
1
2
3
Size of alligator
Size of alligator
Male
Female
4
We can also add easy to interpret measures of uncertainty
Above are standard error regions
Chris Adolph (UW)
Tufte without tears
41 / 106
Food choice by alligator size and sex
Pr(Food preference)
1
Invertebrates
1
Fish
0.8
0.8
0.6
0.6
0.4
0.4
Other
0.2
0.2
0
0
1
2
3
Invertebrates
4
Fish
Other
1
2
3
Size of alligator
Size of alligator
Male
Female
4
Easy to highlight where the model is interpolating into the observed data
. . . and where it’s extrapolating into Hollywood territory
Chris Adolph (UW)
Tufte without tears
42 / 106
When simulation is the only option: Chinese leadership
Shih, Adolph, and Liu investigate the advancement of elite Chinese leaders in
the Reform Period (1982–2002)
Explain (partially observed) ranks of the top 300 to 500 Chinese Communist
Party leaders as a function of:
Demographics
Education
Performance
Faction
age, sex, ethnicity
level of degree
provincial growth, revenue
birth, school, career, and family ties to top leaders
Bayesian model of partially observed ranks of CCP officials
Model parameters difficult to interpret: on a latent scale
and individual effects are conditioned on all other ranked members
Only solution:
Simulate ranks of hypothetical officials as if placed in the observed hierarchy
Chris Adolph (UW)
Tufte without tears
43 / 106
0
< High School
Female
High School
Han
Non−Princeling
Average Member
Graduate School
College
Jiang Zemin Faction
Male
Relative GDP Growth +1 sd
Princeling
Party Exp +1 sd
Relative Fiscal Growth +1 sd
Minority
Hu Jintao Faction
Deng Faction
Age +1 sd
Expected percentile
40
60
80
100
Black circles
show
expected
ranks for
otherwise
average
Chinese
officials with
the
characteristic
listed at left
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
356
Chris Adolph (UW)
20
285
214
142
Expected rank
Tufte without tears
71
0
44 / 106
0
< High School
Female
High School
Han
Non−Princeling
Average Member
Graduate School
College
Jiang Zemin Faction
Male
Relative GDP Growth +1 sd
Princeling
Party Exp +1 sd
Relative Fiscal Growth +1 sd
Minority
Hu Jintao Faction
Deng Faction
Age +1 sd
Expected percentile
40
60
80
100
●
●
●
●
●
Thick black
horizontal
lines are 1 std
error bars,
and thin lines
are 95% CIs
●
●
●
●
●
●
●
●
●
●
●
●
●
356
Chris Adolph (UW)
20
285
214
142
Expected rank
Tufte without tears
71
0
45 / 106
0
< High School
Female
High School
Han
Non−Princeling
Average Member
Graduate School
College
Jiang Zemin Faction
Male
Relative GDP Growth +1 sd
Princeling
Party Exp +1 sd
Relative Fiscal Growth +1 sd
Minority
Hu Jintao Faction
Deng Faction
Age +1 sd
Expected percentile
40
60
80
100
●
●
●
Blue triangles
are officials
with random
effects at ±1
sd; how much
unmeasured
factors matter
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
356
Chris Adolph (UW)
20
285
214
142
Expected rank
Tufte without tears
71
0
46 / 106
0
< High School
Female
High School
Han
Non−Princeling
Average Member
Graduate School
College
Jiang Zemin Faction
Male
Relative GDP Growth +1 sd
Princeling
Party Exp +1 sd
Relative Fiscal Growth +1 sd
Minority
Hu Jintao Faction
Deng Faction
Age +1 sd
Expected percentile
40
60
80
100
●
●
●
●
●
Helps to sort
rows of the
plot from
smallest to
largest effect
●
●
●
●
●
●
●
●
●
●
●
●
●
356
Chris Adolph (UW)
20
285
214
142
Expected rank
Tufte without tears
71
0
47 / 106
Change in
Rank Percentile
40
GDP Growth +1 sd
0
−20
College Degree
'82
Change in
Rank Percentile
Fiscal Revenue +1 sd
20
'87
'92
'97
'02
'82
'87
Ethnic
Minority
40
'92
'97
'02
'82
'87
'92
'97
'02
Female
20
0
−20
Graduate Degree
'82
'87
'92
'97
'02
'82
'87
'92
'97
'02
'82
'87
'92
'97
'02
Shih, Adolph, and Liu re-estimate the model for each year, leading to a large
number of results
A complex lineplot helps organize them and facilitate comparisons
Chris Adolph (UW)
Tufte without tears
48 / 106
Change in
Rank Percentile
40
GDP Growth +1 sd
20
0
−20
College Degree
'82
Change in
Rank Percentile
Fiscal Revenue +1 sd
'87
'92
'97
'02
'82
'87
Ethnic
Minority
40
'92
'97
'02
'82
'97
'02
'82
'87
'92
'97
'02
Female
20
0
−20
Graduate Degree
'82
'87
'92
'97
'02
'82
'87
'92
'87
'92
'97
'02
Note that these results are now first differences:
the expected percentile change in rank for an otherwise average official who
gains the characteristic noted
Chris Adolph (UW)
Tufte without tears
49 / 106
Change in
Rank Percentile
40
Mao Faction
20
0
−20
Long Marcher 2nd Field Army Vet
Change in
Rank Percentile
'82
40
'87
'82
Zhao Ziyang
Faction
'87
'82
Jiang Zemin
Faction
'87
Deng Faction
'82
'87
'92
'97
Hu Jintao Faction
Hu Yaobang Fact'n
'02
'82
'87
Princeling
20
0
−20
'87
'92
'92
'97
'02
'97
'02
'82
'87
'92
'97
'02
Over time, officials’ economic performance never matters, but factions often
do
Runs counter to the conventional wisdom that meritocratic selection of
officials lies behind Chinese economic success
Chris Adolph (UW)
Tufte without tears
50 / 106
Robustness Checks
So far, we’ve presenting conditional expectations & differences from
regressions
But are we confident that these were the “right” estimates?
The language of inference usually assumes we
correctly specified our model
correctly measured our variables
chose the right probability model
don’t have influential outliers, etc.
Chris Adolph (UW)
Tufte without tears
51 / 106
Robustness Checks
So far, we’ve presenting conditional expectations & differences from
regressions
But are we confident that these were the “right” estimates?
The language of inference usually assumes we
correctly specified our model
correctly measured our variables
chose the right probability model
don’t have influential outliers, etc.
We’re never completely sure these assumptions hold.
Most people present one model, and argue it was the best choice
Sometimes, a few alternatives are displayed
Chris Adolph (UW)
Tufte without tears
51 / 106
The race of the variables
Model 1
Model 2
Model 3
Model 4
My variable
of interest, X1
X.XX
(X.XX)
X.XX
(X.XX)
X.XX
(X.XX)
X.XX
(X.XX)
A control
I "need"
X.XX
(X.XX)
X.XX
(X.XX)
X.XX
(X.XX)
X.XX
(X.XX)
X.XX
(X.XX)
A control
I "need"
X.XX
(X.XX)
X.XX
(X.XX)
X.XX
(X.XX)
X.XX
(X.XX)
X.XX
(X.XX)
A candidate
control
A candidate
control
X.XX
(X.XX)
X.XX
(X.XX)
X.XX
(X.XX)
Alternate
measure of X1
Chris Adolph (UW)
Model 5
X.XX
(X.XX)
X.XX
(X.XX)
Tufte without tears
52 / 106
Robustness Checks
Problems with the approach above?
1
Lots of space to show a few permutations of the model
Most space wasted or devoted to ancillary info
Chris Adolph (UW)
Tufte without tears
53 / 106
Robustness Checks
Problems with the approach above?
1
Lots of space to show a few permutations of the model
Most space wasted or devoted to ancillary info
2
What if we’re really interested in E(Y |X ), not β̂?
E.g., because of nonlinearities, interactions, scale differences, etc.
Chris Adolph (UW)
Tufte without tears
53 / 106
Robustness Checks
Problems with the approach above?
1
Lots of space to show a few permutations of the model
Most space wasted or devoted to ancillary info
2
What if we’re really interested in E(Y |X ), not β̂?
E.g., because of nonlinearities, interactions, scale differences, etc.
3
The selection of permutations is ad hoc.
Chris Adolph (UW)
Tufte without tears
53 / 106
Robustness Checks
Problems with the approach above?
1
Lots of space to show a few permutations of the model
Most space wasted or devoted to ancillary info
2
What if we’re really interested in E(Y |X ), not β̂?
E.g., because of nonlinearities, interactions, scale differences, etc.
3
The selection of permutations is ad hoc.
We’ll try to fix 1 & 2.
Objection 3 is harder, but worth thinking about.
Chris Adolph (UW)
Tufte without tears
53 / 106
Robustness Checks: An algorithm
1
Identify a relation of interest between a concept X and a concept Y
Chris Adolph (UW)
Tufte without tears
54 / 106
Robustness Checks: An algorithm
1
Identify a relation of interest between a concept X and a concept Y
2
Choose:
I
a measure of X , denoted X ,
Chris Adolph (UW)
Tufte without tears
54 / 106
Robustness Checks: An algorithm
1
Identify a relation of interest between a concept X and a concept Y
2
Choose:
I
a measure of X , denoted X ,
I
a measure of Y, denoted Y ,
Chris Adolph (UW)
Tufte without tears
54 / 106
Robustness Checks: An algorithm
1
Identify a relation of interest between a concept X and a concept Y
2
Choose:
I
a measure of X , denoted X ,
I
a measure of Y, denoted Y ,
I
a set of confounders, Z ,
Chris Adolph (UW)
Tufte without tears
54 / 106
Robustness Checks: An algorithm
1
Identify a relation of interest between a concept X and a concept Y
2
Choose:
I
a measure of X , denoted X ,
I
a measure of Y, denoted Y ,
I
a set of confounders, Z ,
I
a functional form, g(·)
Chris Adolph (UW)
Tufte without tears
54 / 106
Robustness Checks: An algorithm
1
Identify a relation of interest between a concept X and a concept Y
2
Choose:
3
I
a measure of X , denoted X ,
I
a measure of Y, denoted Y ,
I
a set of confounders, Z ,
I
a functional form, g(·)
I
a probability model of Y , f (·)
Estimate the probability model Y ∼ f (µ, α), µ = g(vec(X , Z ), β).
Chris Adolph (UW)
Tufte without tears
54 / 106
Robustness Checks: An algorithm
1
Identify a relation of interest between a concept X and a concept Y
2
Choose:
I
a measure of X , denoted X ,
I
a measure of Y, denoted Y ,
I
a set of confounders, Z ,
I
a functional form, g(·)
I
a probability model of Y , f (·)
3
Estimate the probability model Y ∼ f (µ, α), µ = g(vec(X , Z ), β).
4
Simulate the quantity of interest, e.g., E(Y |X ) or E(Y2 − Y1 |X2 , X1 ), to
obtain a point estimate and confidence interval.
Chris Adolph (UW)
Tufte without tears
54 / 106
Robustness Checks: An algorithm
1
Identify a relation of interest between a concept X and a concept Y
2
Choose:
I
a measure of X , denoted X ,
I
a measure of Y, denoted Y ,
I
a set of confounders, Z ,
I
a functional form, g(·)
I
a probability model of Y , f (·)
3
Estimate the probability model Y ∼ f (µ, α), µ = g(vec(X , Z ), β).
4
Simulate the quantity of interest, e.g., E(Y |X ) or E(Y2 − Y1 |X2 , X1 ), to
obtain a point estimate and confidence interval.
5
Repeat 2–4, changing at each iteration one of the choices in step 2.
Chris Adolph (UW)
Tufte without tears
54 / 106
Robustness Checks: An algorithm
1
Identify a relation of interest between a concept X and a concept Y
2
Choose:
I
a measure of X , denoted X ,
I
a measure of Y, denoted Y ,
I
a set of confounders, Z ,
I
a functional form, g(·)
I
a probability model of Y , f (·)
3
Estimate the probability model Y ∼ f (µ, α), µ = g(vec(X , Z ), β).
4
Simulate the quantity of interest, e.g., E(Y |X ) or E(Y2 − Y1 |X2 , X1 ), to
obtain a point estimate and confidence interval.
5
Repeat 2–4, changing at each iteration one of the choices in step 2.
6
Compile the results in a variant of the dot plot called a ropeladder.
Chris Adolph (UW)
Tufte without tears
54 / 106
Robustness Checks: A central banks example
In The Dilemma of Discretion, I argue central bankers’ career backgrounds
explain their monetary policy choices
Central bankers with financial sector backgrounds choose more conservative
policies, leading to lower inflation but potentially higher unemployment
I argue more conservative governments should prefer to appoint more
conservative central bankers, e.g., those with financial sector backgrounds
Chris Adolph (UW)
Tufte without tears
55 / 106
Robustness Checks: A central banks example
I test this claim using data on central bankers setting monetary policy over 20
countries and 30 years
Conservatism of central bankers is measured by percentage share in
“conservative” careers, and summarized by an index CBCC
Partisanship of government is measured by PCoG:
higher values = more conservative Partisan“Center of Gravity”
I estimate a zero-inflated compositional regression on career shares
and find the expected relationship:
conservative goverments pick conservative central bankers
Chris Adolph (UW)
Tufte without tears
56 / 106
Robustness Checks: A central banks example
Conservative goverments pick conservative central bankers
Is this finding robust to specification assumptions?
In my case, the statistical model is so demanding it’s hard to include many
regressors at once.
So try one at a time, and show a long “ropeladder” plot. . .
Chris Adolph (UW)
Tufte without tears
57 / 106
Robustness Ropeladder: Partisan central banker appointment
Estimated increase in Central Bank Conservatism (CBCC) resulting from . . .
Control added
Shifting control from
Shifting PCoG from
low (µ-1.5 sd)
From left gov (µ-1.5 sd)
to high (µ+1.5)
to right gov (µ+1.5 sd)
[None]
Office appointed to
CBI (3 index avg.)
CBI (Cukierman)
Lagged inflation
Lagged unemployment
Trade Openness
Endebtedness
Financial Sector Employment
Financial Sector Score
Time Trend
Central Bank staff size
-0.4
-0.2
0.0
0.2
0.4
0.6 -0.4
-0.2
0.0
0.2
0.4
0.6
Chris8.4:
AdolphPartisanship
(UW)
Tufte
without tears
Figure
of central banker
appointment:
Robustness, part 2. Each row presents a58 / 106
Anatomy of a ropeladder plot
I call this a ropeladder plot.
The column of dots shows the relationship between Y and a specific X under
different model assumptions
Each entry corresponds to a different assumption about the specification, or
the measures, or the estimation method, etc.
Chris Adolph (UW)
Tufte without tears
59 / 106
Anatomy of a ropeladder plot
I call this a ropeladder plot.
The column of dots shows the relationship between Y and a specific X under
different model assumptions
Each entry corresponds to a different assumption about the specification, or
the measures, or the estimation method, etc.
If all the dots line up, with narrow, similar CIs, we say the finding is robust, and
reflects the data under a range of reasonable assumptions
If the ropeladder is “blowing in the wind”, we may be skeptical of the finding.
It depends on model assumptions that may be controversial
Chris Adolph (UW)
Tufte without tears
59 / 106
Anatomy of a ropeladder plot
I call this a ropeladder plot.
The column of dots shows the relationship between Y and a specific X under
different model assumptions
Each entry corresponds to a different assumption about the specification, or
the measures, or the estimation method, etc.
If all the dots line up, with narrow, similar CIs, we say the finding is robust, and
reflects the data under a range of reasonable assumptions
If the ropeladder is “blowing in the wind”, we may be skeptical of the finding.
It depends on model assumptions that may be controversial
The shaded gray box shows the full range of the point estimates for the QoI.
Narrow is better.
Chris Adolph (UW)
Tufte without tears
59 / 106
Why ropeladders?
1
Anticipate objections on model assumptions, and have concrete answers.
Avoid: “I ran it that other way, and it came out the ‘same’.”
Instead: “I ran it that other way, and look —it made no substantive or
statistical difference worth speaking of.”
Or: “. . . it makes this much difference.”
Chris Adolph (UW)
Tufte without tears
60 / 106
Why ropeladders?
1
Anticipate objections on model assumptions, and have concrete answers.
Avoid: “I ran it that other way, and it came out the ‘same’.”
Instead: “I ran it that other way, and look —it made no substantive or
statistical difference worth speaking of.”
Or: “. . . it makes this much difference.”
2
Investigate robustness more thoroughly.
Traditional tabular presentation would have run to 7 pages, making
comparison hard and discouraging a thorough search
Chris Adolph (UW)
Tufte without tears
60 / 106
Why ropeladders?
1
Anticipate objections on model assumptions, and have concrete answers.
Avoid: “I ran it that other way, and it came out the ‘same’.”
Instead: “I ran it that other way, and look —it made no substantive or
statistical difference worth speaking of.”
Or: “. . . it makes this much difference.”
2
Investigate robustness more thoroughly.
Traditional tabular presentation would have run to 7 pages, making
comparison hard and discouraging a thorough search
3
Find patterns of model sensitivity.
Two seemingly unrelated changes in specification had the same effect.
(Unemployment and Financial Sector Size)
Turned out to be a missing third covariate. (Time trend)
Chris Adolph (UW)
Tufte without tears
60 / 106
81
Robustness for several QoIs at once
Change in inflation, five years after +1 s.d. in . . .
FinExp
Specification
FMExp
CBExp
GovExp
CBCC
Baseline
Robust Estimation
Add π world
Use Cukierman CBI
Omit Imports/GDP
Add Exchange Regime
Add % Left-Appointees
Add Partisan CoG of Aptees
Add % with Econ PhDs
−3
−1
1
−3
−1
1
−1
1
3
−1
1
3
−3
−1
1
Figure 3.7: The career-inflation link under alternative specifications. Each plot shows the five-year
first difference in inflation resulting from a one standard deviation increase in a career variable (FinExp,
Each ropeladder,
or column,
GovExp, etc.), given the specification noted at left. See text for a description of the baseline model and
Circles
anddifferent
squares indicate
point estimates
of the
first difference, and horizontal lines 90
shows alternatives.
the effect
of a
variable
on the
response
That
percent confidence intervals. The results marked with a square are the same as those used in Figure 3.6.
The shaded areas highlight the range of point estimates across all alternatives. Effects of controls not
(of thoseacross
considered,shows
only π worldthe
had results
a significantfrom
effect on
is,shown
reading
a inflation).
single model
Reading down
shows the results for a single question across different models
A similar pattern of robustness emerges if we keep the baseline model in place and
instead vary the construction of the key explanatory variables. Most central banks have
Chris Adolph (UW)
Tufte without tears
61 / 106
Chinese Officials Redux
Earlier, I showed the relationship between many different covariates and the
expected political rank of Chinese officials
I also showed how these relationships changed over time, making for a
complex plot
But our findings were controversial: countered the widely accepted belief that
Chinese officials are rewarded for economic performance
Critics asked for lots of alternative specifications to probe our results
I can use tile to show how exactly what difference these robustness checks
made using overlapping lineplots
Chris Adolph (UW)
Tufte without tears
62 / 106
Change in
Rank Percentile
40
GDP Growth +1 sd
0
−20
College Degree
'82
Change in
Rank Percentile
Fiscal Revenue +1 sd
20
'87
'92
'97
'02
'82
'87
Ethnic
Minority
40
'92
'97
'02
'82
'87
'92
'97
'02
Female
20
0
−20
Graduate Degree
'82
'87
'92
'97
'02
Original Model
'82
'87
'92
'97
'02
'82
'87
'92
'97
'02
Factional Tie Requires Job Overlap
Some critics worried that our measures of faction were too sensitive,
so we considered a more specific alternative
This didn’t salvage the conventional wisdom on growth. . .
Chris Adolph (UW)
Tufte without tears
63 / 106
Change in
Rank Percentile
40
Mao Faction
20
0
−20
Long Marcher 2nd Field Army Vet
Change in
Rank Percentile
'82
40
'87
'82
Zhao Ziyang
Faction
'87
'82
Jiang Zemin
Faction
Deng Faction
'87
'82
'87
'92
'97
Hu Jintao Faction
Hu Yaobang Fact'n
'02
'82
'87
Princeling
20
0
−20
'87
'92
'92
Original Model
'97
'02
'97
'02
'82
'87
'92
'97
'02
Factional Tie Requires Job Overlap
But did (unsuprisingly) strengthen our factional results
(Specific measures pick up the strongest ties)
Chris Adolph (UW)
Tufte without tears
64 / 106
Change in
Rank Percentile
40
Mao Faction
20
0
−20
Long Marcher 2nd Field Army Vet
Change in
Rank Percentile
'82
40
'87
'82
Zhao Ziyang
Faction
'87
'82
Jiang Zemin
Faction
'87
Deng Faction
'82
'87
'92
'97
Hu Jintao Faction
Hu Yaobang Fact'n
'02
'82
'87
Princeling
20
0
−20
'87
'92
'92
'97
'02
'97
Original Model
'02
'82
'87
'92
'97
'02
Surprise Performance, ARMA(p,q)
Other critics worried about endogeneity or selection effects flowing from
political power to economic performance
We used measures of unexpected growth to zero in on an official’s own
performance in office—which still nets zero political benefit
Chris Adolph (UW)
Tufte without tears
65 / 106
Change in
Rank Percentile
40
GDP Growth +1 sd
0
−20
College Degree
'82
Change in
Rank Percentile
Fiscal Revenue +1 sd
20
'87
'92
'97
'02
'82
'87
Ethnic
Minority
40
'92
'97
'02
'82
'97
'02
'82
'87
'92
'97
'02
Female
20
0
−20
Graduate Degree
'82
'87
'92
'97
'02
'82
'87
'92
Original Model
'87
'92
'97
'02
Surprise Performance, ARMA(p,q)
But is there a more efficient way to show that our results stay essentially the
same?
Chris Adolph (UW)
Tufte without tears
66 / 106
Change in
Rank Percentile
GDP Growth +1 sd
40
0
−20
College Degree
'82
Change in
Rank Percentile
Fiscal Revenue +1 sd
20
'87
'92
'97
'02
'82
'87
Ethnic
Minority
40
'92
'97
'02
'82
'97
'02
'82
'87
'92
'97
'02
Female
20
0
−20
Graduate Degree
'82
'87
Original Model
'92
'97
'02
'82
'87
'92
Ignore exact ACC ranks
'87
'92
'97
'02
Job Overlap Only
7 models controlling Surprise Performance, Five Factor Perform. & Schools
In our printed article, we show only this plot, which overlaps the full array of
robustness checks
Chris Adolph (UW)
Tufte without tears
67 / 106
Change in
Rank Percentile
40
Mao Faction
20
0
−20
Long Marcher 2nd Field Army Vet
Change in
Rank Percentile
'82
40
'87
'82
Zhao Ziyang
Faction
'87
'82
Jiang Zemin
Faction
'87
Deng Faction
'82
'87
'92
'97
Hu Jintao Faction
Hu Yaobang Fact'n
'02
'82
'87
Princeling
20
0
−20
'87
Original Model
'92
'92
'97
'02
'97
'02
Ignore exact ACC ranks
'82
'87
'92
'97
'02
Job Overlap Only
7 models controlling Surprise Performance, Five Factor Perform. & Schools
Conveys hundreds of separate findings in a compact, readable form
No knowledge of Bayesian methods or partial rank coefficients required!
Chris Adolph (UW)
Tufte without tears
68 / 106
Bonus Example: Allocation of Authority for Health Policy
Adolph, Greer, and Fonseca consider the problem of explaining whether local,
regional, or national European government have power over specific health
policy areas and instruments
Areas: Pharamceuticals, Secondary/Tertiary, Primary Care, Public Health
Instruments: Frameworks, Finance, Implementation, Provision
Each combination for each country is a case
Chris Adolph (UW)
Tufte without tears
69 / 106
Bonus Example: Allocation of Authority for Health Policy
Adolph, Greer, and Fonseca consider the problem of explaining whether local,
regional, or national European government have power over specific health
policy areas and instruments
Areas: Pharamceuticals, Secondary/Tertiary, Primary Care, Public Health
Instruments: Frameworks, Finance, Implementation, Provision
Each combination for each country is a case
Fiscal federalism suggests lower levels for information-intensive policies
and higher levels for policies with spillovers or public goods
Also control for country characteristics and country random effects
With 3 nominal outcomes for each case, need a multilevel multinomial logit
Chris Adolph (UW)
Tufte without tears
69 / 106
Bonus Example: Allocation of Authority for Health Policy
Covariates:
Policy area
Policy instrument
Regions old or new
Country size
Number of regions
Mountains
Ethnic heterogeneity
Nominal
Nominal
Binary
Continuous
Continuous
Countinuous
Continuous
Tricky part to the model:
some cases have structural zeros for regions (when they don’t exist!)
Chris Adolph (UW)
Tufte without tears
70 / 106
How to set up counterfactuals?
We could set all but one covariate to the mean, then predict the probability of
each level of authority given varied levels of the remaining covariate
We should do this separately for countries with and without regional
governments
Let’s fix everthing but policy instrument to the mean values, then simulate the
probability of authority at each level for each instrument
We show the results using a “nested” dot plot, made using ropeladder() in
the tile package
Chris Adolph (UW)
Tufte without tears
71 / 106
Two−tier countries
0
Probability of allocation
0.2
0.4
0.6
0.8
Three−tier countries
1
Probability of allocation
0.2
0.4
0.6
0.8
0
state
state
region
Provision
●
local
local
state
state
region
Implementation
●
local
local
state
state
region
Finance
●
local
local
state
state
Framework
region
●
local
0
Chris Adolph (UW)
1
local
0.2
0.4
0.6
0.8
Probability of allocation
1
Tufte without tears
0
0.2
0.4
0.6
0.8
Probability of allocation
1
72 / 106
Special plots for compositional data
Probabilities have a special property: they sum to one
Variables that sum to a constraint are compositional
We can plot a two-part composition on a line,
and a three-part composition on a triangular plot
This makes it easier to show more complex counterfactuals, such as every
combination of policy area and instrument
But we also need to work harder to explain these plots
Chris Adolph (UW)
Tufte without tears
73 / 106
Lo
ca
l
Probability of allocation of authority by policy type
Pro
vis
ion
Local
●
•
Provision
•
0.6
0.4
0.2
0.8
Public Health
Primary Care
Secondary/Tertiary
Pharmacueticals
Finance
0.6
•
0.4
me
nta
t
Fin
a
ion
nc
e
● ●
Implementation
Framework
0.2
0.4
0.6
0.8
State
•
0.8
Im
ple
•
0.2
(Ph
ceu arma
tica ls)
ew
ork
Fra
m
Sta
te
●
Region
Above holds country characteristics at their means
Chris Adolph (UW)
Tufte without tears
74 / 106
Lo
ca
l
Probability of allocation of authority by country type
0.2
Local
0.8
Mountainous
Ethnically
Diverse ●
Post-1973
Regions
0.4
Many
Small
Regions
Pre-1973
Regions
Typical
Three
Tier
•
Few Large
Regions
Ethnically
Diverse
Mountainous
Country
Large
Country
Sta
te
Large
Country
0.2
Ethnically
Homogeneous
0.8
●
Flat Country
Typical Two-Tiered
•
Flat
Country
0.6
Ethnically
Homogeneous
0.6
0.4
Small
Country
Small
Country
0.2
0.4
0.6
0.8
State
Region
Above holds policy area and instrument “at their means”
Chris Adolph (UW)
Tufte without tears
75 / 106
Lo
ca
l
Residual country effects
Local
●
0.2
Finland
Lithuania ●
0.4
0.8
0.6
•
Czech Republic
Latvia ●
Greece ●
Portugal ●
Ireland
Slovenia ●
Cyprus
●
Luxembourg ●
Netherlands
●
●
Slovakia
●
•
France
● Bulgaria
● Romania
●
••
Switzerland
•
Italy
Austria
Belgium
Germany
••
•
Denmark
●
United Kingdom
Sweden
0.2
0.4
0.6
State
0.8
Sta
••
Spain
Poland
te
Malta
0.2
• 0.4
0.8
●
0.6
Hungary
Estonia
Region
Looking at the country random effects might suggest omitted variables
Chris Adolph (UW)
Tufte without tears
76 / 106
Lo
ca
l
Residual country effects
Local
●
0.2
Finland
Lithuania ●
0.4
0.8
0.6
•
Czech Republic
Latvia ●
Greece ●
Portugal ●
Ireland
Slovenia ●
Cyprus
●
Luxembourg ●
Netherlands
●
●
Slovakia
●
•
France
● Bulgaria
● Romania
●
••
Switzerland
•
Italy
Austria
Belgium
Germany
••
•
Denmark
●
United Kingdom
Sweden
0.2
0.4
0.6
State
0.8
Sta
••
Spain
Poland
te
Malta
0.2
• 0.4
0.8
●
0.6
Hungary
Estonia
Region
On a previous iteration, mountainous countries clustered as high Pr(Regions)
Chris Adolph (UW)
Tufte without tears
77 / 106
Lessons for practice of data analysis
Simulation + Graphics can summarize complex models for a broad audience
You might even find something you missed as an analyst
And even for fancy or complex models, we can and should show uncertainty
Payoff to programming: this is hard the first few times, but gets easier
Code is re-usable, and encourages more ambitious modeling
Chris Adolph (UW)
Tufte without tears
78 / 106
Teaching with tile
tile helps clarify data and models in research
Also helps in teaching statistical models
I incorporate this software throughout our graduate statistics sequence
Greatly aids intuitive understanding of models
Find out much more, and download the software, from:
faculty.washington.edu/cadolph
Chris Adolph (UW)
Tufte without tears
79 / 106
Building a scatterplot: Supplementary exploratory example
In my graphics class, I have students build a scatterplot “from scratch”
This helps us see the many choices to make, and implications for:
1
perception of the data
2
exploration of relationships
3
assessment of fit
A good warm up for tile before the main event (application to models)
See how tile helps follow Tufte recommendations
Chris Adolph (UW)
Tufte without tears
80 / 106
Building a scatterplot: Redistribution example
Data on
political party systems
and
redistributive effort from various industrial countries
Source of data & basic plot:
Torben Iversen & David Soskice, 2002,
“Why do some democracies redistribute more than others?” Harvard
University.
Chris Adolph (UW)
Tufte without tears
81 / 106
Building a scatterplot: Redistribution example
Concepts for this example (electoral systems and the welfare state):
Effective number of parties:
Chris Adolph (UW)
Tufte without tears
82 / 106
Building a scatterplot: Redistribution example
Concepts for this example (electoral systems and the welfare state):
Effective number of parties:
# of parties varies across countries
electoral rules determine #
I
Winner take all (US) →∼ 2 parties.
I
Proportional representation → more parties
To see this, need to discount trivial parties.
Chris Adolph (UW)
Tufte without tears
82 / 106
Building a scatterplot: Redistribution example
Concepts for this example (electoral systems and the welfare state):
Effective number of parties:
# of parties varies across countries
electoral rules determine #
I
Winner take all (US) →∼ 2 parties.
I
Proportional representation → more parties
To see this, need to discount trivial parties.
Poverty reduction:
Chris Adolph (UW)
Tufte without tears
82 / 106
Building a scatterplot: Redistribution example
Concepts for this example (electoral systems and the welfare state):
Effective number of parties:
# of parties varies across countries
electoral rules determine #
I
Winner take all (US) →∼ 2 parties.
I
Proportional representation → more parties
To see this, need to discount trivial parties.
Poverty reduction:
Percent lifted out of poverty by taxes and transfers.
Poverty = an income below 50% of mean income.
Chris Adolph (UW)
Tufte without tears
82 / 106
100
90
80
povredct
70
60
50
40
30
20
10
0
0
2
4
6
8
enpart
10
12
14
Initial plotting area is often oddly shaped (I’ve exaggerated)
Plotting area hiding relationship here. Sometimes can even exclude data!
Filled circles: okay for a little data; open is better when data overlap
Chris Adolph (UW)
Tufte without tears
83 / 106
70
povred
60
50
40
30
20
2
3
4
5
6
7
enpart
Sensible, data based plot limits
Appears to be a curvilinear relationship. Can bring that out with. . .
Chris Adolph (UW)
Tufte without tears
84 / 106
70
povredct
60
50
40
30
20
0.3
0.4
0.5
0.6
logpart
0.7
0.8
Log scaling.
But what logarthmic base? And why print the exponents?
Chris Adolph (UW)
Tufte without tears
85 / 106
70
povredct
60
50
40
30
20
2
3
4
5
6
7
8
9
10
enpart
Combine a log scale with linear labels. Now everyone can read. . .
Next: Axis labels people can understand
Chris Adolph (UW)
Tufte without tears
86 / 106
Percent lifted out of poverty by taxes and transfers
70
60
50
40
30
20
2
3
4
5
6
7
Effective number of political parties (log10 scale)
8
9
10
So what’s going on the data? What are those outliers?
Chris Adolph (UW)
Tufte without tears
87 / 106
Percent lifted out of poverty by taxes and transfers
Belgium
70
Norway
Netherlands
Sweden
60
50
France
United Kingdom
Germany
Italy
Australia
40
30
Denmark
Finland
Canada
20
Switzerland
United States
2
3
4
5
6
7
8
Effective number of political parties (log10 scale)
9
10
With little data & big outliers, show the name of each case
Now we can try to figure out what makes the US and Switzerland so different
Chris Adolph (UW)
Tufte without tears
88 / 106
Percent lifted out of poverty by taxes and transfers
80
Belgium
Denmark
70
Netherlands
Norway
Finland
Sweden
60
France
50
United Kingdom
40
30
Germany
Italy
Australia
Electoral system
Majoritarian
Proportional
Unanimity
Canada
20
10
Switzerland
United States
2
3
4
5
6
Effective number of political parties (log10 scale)
Chris Adolph (UW)
Tufte without tears
7
8
9
10
89 / 106
% lifted from poverty by taxes & transfers
80
Majoritarian
Proportional
Unanimity
Belgium
Denmark
Netherlands
Norway
Finland
Sweden
60
France
United Kingdom
40
Germany
Italy
Australia
Canada
20
Switzerland
United States
0
2
3
4
Effective number of parties
5
6
7
Plot redone using scatter (tile package in R)
Chris Adolph (UW)
Tufte without tears
90 / 106
% lifted from poverty by taxes & transfers
80
Majoritarian
Proportional
Unanimity
Belgium
Denmark
Netherlands
Norway
Finland
Sweden
60
France
United Kingdom
40
Germany
Italy
Australia
Canada
20
Switzerland
United States
0
2
3
4
Effective number of parties
5
6
7
Scatterplots relate two distributions.
Why not make those marginal distributions explicit?
Chris Adolph (UW)
Tufte without tears
91 / 106
% lifted from poverty by taxes & transfers
80
Majoritarian
Proportional
Unanimity
Belgium
Denmark
Netherlands
Norway
Finland
Sweden
60
France
United Kingdom
40
Germany
Italy
Australia
Canada
20
Switzerland
United States
0
2
3
4
Effective number of parties
5
6
7
Rugs accomplish this by replacing the axis lines with the plots
We could choose any plotting style: from the histogram-like dots. . .
Chris Adolph (UW)
Tufte without tears
92 / 106
% lifted from poverty by taxes & transfers
80
Majoritarian
Proportional
Unanimity
Belgium
Denmark
Netherlands
Norway
Finland
Sweden
60
France
United Kingdom
40
Germany
Italy
Australia
Canada
20
Switzerland
United States
0
2
3
4
Effective number of parties
5
6
7
. . . to a strip of jittered data. . .
Chris Adolph (UW)
Tufte without tears
93 / 106
% lifted from poverty by taxes & transfers
80
Majoritarian
Proportional
Unanimity
Belgium
Denmark
Netherlands
Norway
Finland
Sweden
60
France
United Kingdom
40
Germany
Italy
Australia
Canada
20
Switzerland
United States
0
2
3
4
Effective number of parties
5
6
7
. . . to a set of very thin lines marking each observation
Because we have so few cases, thin lines work best for this example
Chris Adolph (UW)
Tufte without tears
94 / 106
% lifted from poverty by taxes & transfers
80
Majoritarian
Proportional
Unanimity
Belgium
Denmark
Norway
Netherlands
Finland
Sweden
60
France
United Kingdom
40
Germany
Italy
Australia
Canada
20
Switzerland
United States
0
2
3
4
Effective number of parties
5
6
7
Let’s add a parametric model of the data: a least squares fit line
tile can do this for us
Chris Adolph (UW)
Tufte without tears
95 / 106
% lifted from poverty by taxes & transfers
80
Belgium
Majoritarian
Proportional
Unanimity
Denmark
Norway
Netherlands
Finland
60
Sweden
France
United Kingdom
40
Germany
Italy
Australia
Canada
20
Switzerland
United States
0
2
3
4
Effective number of parties
5
6
7
But we don’t have to be parametric
A local smoother, like loess, often helps show non-linear relationships
Chris Adolph (UW)
Tufte without tears
96 / 106
% lifted from poverty by taxes & transfers
80
Belgium
Majoritarian
Proportional
Unanimity
Denmark
Netherlands
Norway
Finland
Sweden
60
France
United Kingdom
40
Germany
Italy
Australia
Canada
20
Switzerland
United States
0
2
3
4
Effective number of parties
5
6
7
M-estimators weight observations by an influence function to minimize the
influence of outliers
Chris Adolph (UW)
Tufte without tears
97 / 106
% lifted from poverty by taxes & transfers
80
Belgium
Majoritarian
Proportional
Unanimity
Denmark
Netherlands
Norway
Finland
Sweden
60
France
United Kingdom
40
Germany
Italy
Australia
Canada
20
Switzerland
United States
0
2
3
4
Effective number of parties
5
6
7
Even with an M-estimator, every outlier has some influence
Thus any one distant outlier can bias the result
Chris Adolph (UW)
Tufte without tears
98 / 106
% lifted from poverty by taxes & transfers
80
Majoritarian
Proportional
Unanimity
Belgium
Denmark
Netherlands
Norway
Finland
Sweden
60
France
United Kingdom
40
Germany
Italy
Australia
Canada
20
Switzerland
United States
0
2
3
4
Effective number of parties
5
6
7
A robust and resistant MM-estimator, shown above, largely avoids this
problem
Only a (non-outlying) fraction of the data influence this fit.
rlm(method="MM")
Chris Adolph (UW)
Tufte without tears
99 / 106
% lifted from poverty by taxes & transfers
80
Majoritarian
Proportional
Unanimity
Denmark
Belgium
Netherlands
Norway
Finland
Sweden
60
France
United Kingdom
40
Germany
Italy
Australia
Canada
20
Switzerland
United States
0
2
3
4
Effective number of parties
5
6
7
In our final plot, we add 95 percent confidence intervals for the MM-estimator
A measure of uncertainty is essential to reader confidence in the result
Chris Adolph (UW)
Tufte without tears
100 / 106
Syntax for redistribution scatter
library(tile)
# Load data
data <- read.csv("iver.csv", header=TRUE)
attach(data)
Chris Adolph (UW)
Tufte without tears
101 / 106
Syntax for redistribution scatter
# First, collect all the data inputs into a series of "traces"
# The actual scattered points
trace1 <- scatter(x = enp, # X coordinate of the data
y = povred, # Y coordinate of the data
labels = cty, # Labels for each point
# Plot symbol for each point
pch = recode(system,"1=17;2=15;3=16"),
# Color for each point
col = recode(system,"1=’blue’;2=’darkgreen’;3=’red’"),
# Offset text labels
labelyoffset = -0.03,
# on npc scale
# Fontsize
fontsize = 9,
Chris Adolph (UW)
Tufte without tears
102 / 106
Syntax for redistribution scatter
# Marker size
size = 1, # could be vector for bubble plot
# Add a robust fit line and CI
fit = list(method = "mmest", ci = 0.95),
# Which plot(s) to plot to
plot = 1
)
# The rugs with marginal distributions
rugX1 <- rugTile(x=enp, type="lines", plot = 1)
rugY1 <- rugTile(y=povred, type="lines", plot = 1)
Chris Adolph (UW)
Tufte without tears
103 / 106
Syntax for redistribution scatter
# A legend
legendSymbols1 <- pointsTile(x= c(1.8,
y= c(78,
pch=c(17,
col=c("blue",
fontsize = 9,
size=1,
plot=1
)
1.8,
1.8),
74,
70),
15,
16),
"darkgreen", "red"),
legendLabels1 <- textTile(labels=c("Majoritarian",
"Proportional",
"Unanimity"),
x= c(2.05,
2.05,
2.05),
y= c(78,
74,
70),
pch=c(17,
15,
16),
col=c("blue", "darkgreen", "red"),
fontsize = 9,
plot=1
)
Chris Adolph (UW)
Tufte without tears
104 / 106
Syntax for redistribution scatter
# Now, send that trace to be plotted with tile
tile(trace1,
# Could list as many
rugX1,
# traces here as we want
rugY1,
# in any order
legendSymbols1,
legendLabels1,
# Some generic options for tile
RxC = c(1,1),
#frame = TRUE,
#output = list(file="iverson1", width=5, type="pdf"),
height = list(plot="golden"),
# Limits of plotting region
limits=c(1.6, 7.5, 0, 82),
Chris Adolph (UW)
Tufte without tears
105 / 106
Syntax for redistribution scatter
# x-axis controls
xaxis=list(log = TRUE,
at = c(2,3,4,5,6,7)
),
# x-axis title controls
xaxistitle=list(labels=c("Effective number of parties")),
# y-axis title controls
yaxistitle=list(labels=c("% lifted from poverty by taxes & transfers"
# Plot titles
plottitle=list(labels=("Party Systems and Redistribution"))
)
Chris Adolph (UW)
Tufte without tears
106 / 106
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement