Evaluation of the DC Opportunity Scholarship Program: Impacts After Three Years

Evaluation of the DC Opportunity Scholarship Program: Impacts After Three Years

by Patrick Wolf, Babette Gutmann, Michael Puma, Brian Kisida
     
 

The District of Columbia School Choice Incentive Act of 2003, passed by Congress in
January 2004, established the first federally funded, private school voucher program in the United States.
As part of this legislation, Congress mandated a rigorous evaluation of the impacts of the Program, now called the DC Opportunity Scholarship Program (OSP). This report… See more details below

Overview

The District of Columbia School Choice Incentive Act of 2003, passed by Congress in
January 2004, established the first federally funded, private school voucher program in the United States.
As part of this legislation, Congress mandated a rigorous evaluation of the impacts of the Program, now called the DC Opportunity Scholarship Program (OSP). This report presents findings from the evaluation of the impacts 3 years after families who applied were given the option to move from a public school to a participating private school of their choice.
The evaluation is based on a randomized controlled trial design that compares the outcomes
of eligible applicants randomly assigned to receive (treatment group) or not receive (control group) a
scholarship through a series of lotteries. The main findings of the evaluation so far include:
• After 3 years, there was a statistically significant positive impact on reading test
scores, but not math test scores. Overall, those offered a scholarship were performing
at statistically higher levels in readingequivalent to 3.1 months of additional
learningbut at similar levels in math compared to students not offered a scholarship
(table 3). Analysis in prior years indicated no significant impacts overall on either
reading or math achievement.
• The OSP had a positive impact overall on parents’ reports of school satisfaction
and safety (figures 3 and 4), but not on students’ reports (figures 3 and 4). Parents
were more satisfied with their child’s school (as measured by the percentage giving the
school a grade of A or B) and viewed their child’s school as safer and more orderly if
the child was offered a scholarship. Students had a different view of their schools than
did their parents. Reports of safety and school climate were comparable for students in
the treatment and control groups. Overall, student satisfaction was unaffected by the
Program.
• This same pattern of findings holds when the analysis is conducted to determine
the impact of using a scholarship rather than being offered a scholarship. Fourteen
percent of students in our impact sample who were randomly assigned by lottery to
receive a scholarship and who responded to year 3 data collection chose not to use their
scholarship at any point over the 3-year period after applying to the Program.1 We use
a common statistical technique to take those “never users” into account; it assumes that
the students had zero impact from the OSP, but it does not change the statistical
significance of the original impact estimates. Therefore, the positive impacts on
reading achievement, parent views of school safety and climate, and parent views of
satisfaction all increase in size, and there remains no impact on math achievement and
no overall impact on students’ perceptions of school safety and climate or satisfaction
from using an OSP scholarship.
• The OSP improved reading achievement for 5 of the 10 subgroups examined.2
Being offered or using a scholarship led to higher reading test scores for participants
who applied from schools that were not classified as “schools in need of improvement”
(non-SINI). There were also positive impacts for students who applied to the Program
with relatively higher levels of academic performance, female students, students
entering grades K-8 at the time of application, and students from the first cohort of
applicants. These impacts translate into 1/3 to 2 years of additional learning growth.
However, the positive subgroup reading impacts for female students and the first
cohort of applicants should be interpreted with caution, as reliability tests suggest that
they could be false discoveries.
• No achievement impacts were observed for five other subgroups of students,
including those who entered the Program with relative academic disadvantage.
Subgroups of students who applied from SINI schools (designated by Congress as the
highest priority group for the Program) or were in the lower third of the test score
distribution among applicants did not demonstrate significant impacts on reading test
scores if they were offered or used a scholarship. In addition, male students, those
entering high school grades upon application, and those in application cohort 2 showed
no significant impacts in either reading or math after 3 years.

Read More

Product Details

ISBN-13:
2940148886167
Publisher:
ReadCycle
Publication date:
11/24/2013
Sold by:
Barnes & Noble
Format:
NOOK Book
File size:
1 MB

Customer Reviews

Average Review:

Write a Review

and post it to your social network

     

Most Helpful Customer Reviews

See all customer reviews >