Disentangling Disadvantage : Can We Distinguish Good Teaching from Classroom Composition?

This paper focuses on the use of teacher value-added estimates to assess the distribution of effective teaching across students of varying socioeconomic disadvantage. We use simulation methods to examine the extent to which different commonly used teacher-value estimators accurately capture both the rank correlation between true and estimated teacher effects and the distribution of effective teaching across student characteristics in the presence of classroom composition effects. Varying the amount of teacher sorting by student characteristics, the within-teacher variability in classroom composition, and the amount of student learning decay, we compare aggregated residuals, teacher random effects, and teacher fixed effects models estimated in both levels and gains, with and without controls for classroom composition. We find that models estimated in levels more accurately capture the rank correlation between true and estimated teacher effects than models estimated in gains, but levels are not always preferable for recovering the correlation between teacher value-added and student achievement. For recovering that correlation, aggregated residuals models appear preferable when sorting is not present, though fixed effects models perform better in the presence of sorting. Because the true amount of sorting is never known, we recommend that analysis incorporate contextual information into their decisions about model choice.