Abstract:

A fundamental question for education policy is whether outcomes-based accountability, including comprehensive educator evaluations and a closer relationship between effectiveness and compensation, improves the quality of instruction and raises achievement. We use synthetic control methods to study the comprehensive teacher and principal evaluation and compensation systems introduced in the Dallas Independent School District (Dallas ISD) in 2013 for principals and 2015 for teachers. Under this far-reaching reform, educator evaluations that are used to support teacher growth and determine salary depend on a combination of supervisor evaluations, student achievement, and student or family survey responses. The reform replaced salary scales based on experience and educational attainment with those based on evaluation scores, a radical departure from decades of rigid salary schedules. The synthetic control estimates reveal positive and significant effects of the reforms on math and reading achievement that increase over time.  From 2015 through 2019, the average achievement for the synthetic control district fluctuates narrowly between -0.27 s.d. and -0.3 s.d., while the Dallas ISD average increases steadily from -0.28 s.d. in 2015 to -0.08 s.d. in 2019, the final year of the sample. Though the increase for reading is roughly half as large, it is also highly significant.

Abstract:

With the expansion of high-stakes evaluations, there is growing interest in understanding how they affect school quality and leadership composition. This paper studies a reform in a large urban public school district that ties principal compensation to their performance. Using a regression discontinuity design, I investigate the effect of categorical performance ratings on principal composition and future school leadership effectiveness for equally productive principals. Results indicate that principals with rating scores that place them just below the higher rating cutoffs are more likely to turnover, exit the district and are less likely to be promoted. However, I do not find much evidence of changes in future leadership productivity in the schools where principals fall below the cutoffs. Principal transitions around the rating boundaries may be driven by the label effect (changes in principal motivation, district decisions and signal sent to other employers) or the salary effect. Exploiting the information that crossing the higher rating threshold does not result in a salary change for all principals, I document that the transition effects are concentrated within the group that does experience a salary change at the rating boundaries. This highlights the significance of the role salary plays in transition decisions. Findings from this paper have important policy implications regarding performance-based incentive programs.

Abstract:

Evidence that teacher effects on both cognitive and non-cognitive skills contribute to longer-term academic, social and labor-market outcomes highlights potential limitations of a singular focus on achievement and the importance of measuring and identifying educator effects on the development of a range of skills. This likely holds even more for school leaders than for teachers. In this study, we use administrative data from the state of Texas to estimate middle-school principal effects on cognitive and non-cognitive skill development and the relationships between these skills and attachment to post-secondary schooling, work and engagement with the criminal justice system. Estimates that use leave-one-out and panel data methods to account for unobserved student and school influences show highly significant effects of middle school principals on the probabilities of future engagement with the criminal justice system, attending and persisting in college and working. The impacts on the probability of engagement with the criminal justice system come through effects on the development of noncognitive skills, while impacts on post-secondary schooling and work come through effects on the development of both cognitive and noncognitive skills. The heterogeneity analysis that estimates separate effects on the probability of an arrest or guilty outcome for Black and nonblack males shows that principal value added to noncognitive skills measured by the
probability of receiving an out of school suspension is more than twice as large for Black males.

Abstract:

While an extensive literature investigates value-added models and a smaller literature studies supervisor observations, to date there is little evidence on young student surveys, despite their use in practice. In this paper, I explore the influence of classroom composition on survey scores a teacher receives and examine the distribution of these scores by teacher experience. The results reveal that except for lagged achievement of the class, classroom composition does not play a very significant role in the survey response a teacher receives. Using a within-teacher returns-to-experience framework, I also find that the survey scores improve for the first five years, remain stable for the next ten years, and then experience a downward trajectory. This paper provides some new information on the student survey measure.

Plan:

Although leadership quality is often cited as key to organizational success, difficulties separating the impact of leaders from other organizational components complicate the estimation of their productivity and effectiveness. In this paper we will estimate the variation in the effectiveness of school leaders using administrative panel data on the State of Texas and City of Chicago public schools. We will also combine the quantitative data with principal, teacher, and student surveys to investigate the contributions of instructional leadership and data management skills to principal effectiveness.