This study examines the impact of a high-dosage tutoring program, characterized by low tutor-to-student-ratio, on the literacy achievement of students in grades two through five in a midsized suburban school district in the southeastern United States. Using a student-level randomized controlled trial, 333 students were randomly assigned to either receive tutoring during the intervention period of the school day or continue with business as usual. While the average treatment effect on literacy achievement is not statistically significant across the full sample, analysis reveals a significant positive impact of 0.2 standard deviations for students with lower baseline achievement, those most in need of high-dosage tutoring, accompanied by a negative impact for the higher achieving students. These findings underscore the importance of tailoring high-dosage tutoring interventions to student needs and suggest that differentiated implementation may be important to maximizing benefits of such programs across diverse learner profiles.

This study examines the impact of using instructionally aligned literacy tutoring with students in kindergarten through third grade under a Response to Intervention framework. We conducted a randomized controlled trial to evaluate the impact on literacy assessment scores for 296 students in four schools in a large suburban school district in the southeastern United States. Students in the treatment group received tutoring where strategies and materials were aligned with core instruction, while those in the control group received tutoring that used supplemental strategies and materials that were distinct from core instruction. We find that students in the treatment group score an average of 0.12 standard deviations higher than the students in the control group. Exploratory analyses reveal that instructional alignment appears to have a greater impact on boys and lower-performing students. Additional exploratory analyses suggest the treatment effect is stronger when delivered in groups of four and by tutors who do not hold a master’s degree.

Abstract:

This paper estimates the relative contributions of subjective (supervisor observations and student surveys) versus objective (test score value-added) evaluation measures in capturing teacher effectiveness at increasing cognitive and non-cognitive skills. I use data from a large urban public school district where teacher compensation is tied to the evaluation measures. Estimates reveal that the subjective measures provide information about teacher effects on short-run achievement and absences as well as long-run achievement in ways that test score value-added does not. However, test score value-added remains the most significantly related to long-term achievement and absences along with short-term achievement.

Abstract:

A fundamental question for education policy is whether outcomes-based accountability, including comprehensive educator evaluations and a closer relationship between effectiveness and compensation, improves the quality of instruction and raises achievement. We use synthetic control methods to study the comprehensive teacher and principal evaluation and compensation systems introduced in the Dallas Independent School District (Dallas ISD) in 2013 for principals and 2015 for teachers. Under this far-reaching reform, educator evaluations that are used to support teacher growth and determine salary depend on a combination of supervisor evaluations, student achievement, and student or family survey responses. The reform replaced salary scales based on experience and educational attainment with those based on evaluation scores, a radical departure from decades of rigid salary schedules. The synthetic control estimates reveal positive and significant effects of the reforms on math and reading achievement that increase over time.  From 2015 through 2019, the average achievement for the synthetic control district fluctuates narrowly between -0.27 s.d. and -0.3 s.d., while the Dallas ISD average increases steadily from -0.28 s.d. in 2015 to -0.08 s.d. in 2019, the final year of the sample. Though the increase for reading is roughly half as large, it is also highly significant.

Abstract:

With the expansion of high-stakes evaluations, there is growing interest in understanding how they affect school quality and leadership composition. This paper studies a reform in a large urban public school district that ties principal compensation to their performance. Using a regression discontinuity design, I investigate the effect of categorical performance ratings on principal composition and future school leadership effectiveness for equally productive principals. Results indicate that principals with rating scores that place them just below the higher rating cutoffs are more likely to turnover, exit the district and are less likely to be promoted. However, I do not find much evidence of changes in future leadership productivity in the schools where principals fall below the cutoffs. Principal transitions around the rating boundaries may be driven by the label effect (changes in principal motivation, district decisions and signal sent to other employers) or the salary effect. Exploiting the information that crossing the higher rating threshold does not result in a salary change for all principals, I document that the transition effects are concentrated within the group that does experience a salary change at the rating boundaries. This highlights the significance of the role salary plays in transition decisions. Findings from this paper have important policy implications regarding performance-based incentive programs.

Abstract:

Using rich Texas administrative data, we estimate the impact of middle school principals on post-secondary schooling, employment, and criminal justice outcomes. The results highlight the importance of school leadership, though striking differences emerge in the relative importance of different skill dimensions to different outcomes. The estimates reveal large and highly significant effects of principal value-added to cognitive skills on the productive activities of schooling and work but much weaker effects of value-added to noncognitive skills on these outcomes. In contrast, there is little or no evidence that middle school principals affect the probability a male is arrested and has a guilty disposition by raising cognitive skills but strong evidence that they affect these outcomes through their impacts on noncognitive skills, especially those related to the probability of an out-of-school suspension. In addition, the principal effects on the probability of engagement in the criminal justice system are much larger for Black than for nonBlack males, corresponding to race differences in engagement with the criminal justice system.

Abstract:

While an extensive literature investigates value-added models and a smaller literature studies supervisor observations, to date there is little evidence on young student surveys, despite their use in practice. In this paper, I explore the influence of classroom composition on survey scores a teacher receives and examine the distribution of these scores by teacher experience. The results reveal that except for lagged achievement of the class, classroom composition does not play a very significant role in the survey response a teacher receives. Using a within-teacher returns-to-experience framework, I also find that the survey scores improve for the first five years, remain stable for the next ten years, and then experience a downward trajectory. This paper provides some new information on the student survey measure.

Plan:

Although leadership quality is often cited as key to organizational success, difficulties separating the impact of leaders from other organizational components complicate the estimation of their productivity and effectiveness. In this paper we will estimate the variation in the effectiveness of school leaders using administrative panel data on the State of Texas and City of Chicago public schools. We will also combine the quantitative data with principal, teacher, and student surveys to investigate the contributions of instructional leadership and data management skills to principal effectiveness.