The Alabama Revenue Department has made public another study by the University of Alabama assessing performance of students attending private schools on Accountability Act scholarships in comparison to students in public schools.  Grades were reviewed from the 2016-17 academic year..

Bottom line: Scholarship students do no better academically than public school students.

The report states,” …comparison between the scholarship students and the students attending public schools in Alabama generally indicate that the two groups continue to fall short of meeting the benchmarks on standardized tests.”

To see the entire report, go here.

Following is the executive summary of the report.

Points that are especially noteworthy:

  •  34 percent of students were “zoned” to attend a failing school.  This means the percentage who actually may have attended a failing schools was significantly less.
  • 73 percent of students had been in their private school for at least three years.
  • On average, over time, participating in the scholarship program was not associated with significant improvement on standardized tests scores”.

Once again we have to question why we are diverting $30 million annually from the Education Trust Fund to support a program yielding such limited results.

Executive Summary

This report fulfills the evaluation requirements of the 2013 Alabama Accountability Act by reporting on the academic achievement of the 2016-2017 scholarship recipients.

The report focuses on three objectives: 1. Describe the academic achievement of students in the scholarship program. 2. Compare scholarship recipients to Alabama public school students. 3. Assess changes in achievement across time.

Scholarship Granting Organizations provided demographic information and achievement test scores for scholarship recipients. Achievement test score information for Alabama public school students was retrieved from the State Department of Education website.

Some challenges were encountered in conducting the evaluation:

The lack of a uniform achievement test among schools constrained the description of the achievement of scholarship recipients and the comparisons that could be made to Alabama public school students.

Norm-referenced tests (e.g., the Stanford Achievement Test) and criterion-referenced tests (e.g., ACT Aspire) are based on different standards and cannot be directly compared.

Some achievement tests were used by only one school or included only a small number of students, making analyses unreliable.

The test score information available from the Alabama State Department of Education only includes the percentage of students in proficiency groups based on ACT Aspire and ACT College Entrance Exam scores, which limited the types of analyses that could be conducted.

Inconsistencies in test score reporting from schools and missing test data limited the number of students who could be included in the evaluation sample.

The evaluation was based upon test scores from 1,991 scholarship recipients attending 114 schools in 43 counties. This represented 76% of the scholarship recipients in the grades for which testing was required. These students varied in their demographic characteristics:

Number of years receiving a scholarship: o 15% were first time scholarship recipients. o 11% were two-time scholarship recipients. o 51% were three-time recipients. o 22% were in their fourth year. • 90% were eligible for free/reduced lunch subsidies. • 34% were zoned to attend a failing school. • 62% were Black/African American, 20% were White/Caucasian, and 11% were Hispanic..

Although this report can show trends for this subsample of scholarship recipients, due to the necessity of excluding a significant proportion of scholarship recipients (24%) from analyses, findings may not be representative of all of the scholarship recipients.

Findings for Objective 1: Describe the academic achievement of students in the scholarship program. • On norm-referenced tests, scholarship recipients generally performed below the average U.S. student at their grade level. • On criterion-referenced tests, the majority of scholarship recipients failed to meet benchmark proficiency scores. • Outcomes were even poorer for African-American participants who made up the majority of scholarship recipients (65%). • These findings are similar to those of the National Assessment of Educational Progress for students attending public schools in Alabama.

Findings for Objective 2: Compare the learning achievement of scholarship recipients to students scholarship recipients and students attending Alabama public schools.  .• There were very few subject areas in which more than 50% of the students met proficiency standards for either group of students. • For the ACT Aspire, comparisons did not present a clear pattern across subjects and grade levels to indicate that one group performed better or worse than the other. • Overall, scholarship recipients in the 11th grade performed about the same as their public school counterparts on the ACT.

Findings for Objective 3: Assess changes in achievement across time. • On average, over time, participating in the scholarship program was not associated with significant improvement on standardized tests scores. • On the ACT Aspire, students were more likely to remain in a non-proficient category than to improve. Although proficiency rates for 2016-2017 were higher for scholarship students than those of Alabama poverty students, the majority of students in both groups did not meet proficiency benchmarks. • The overall lack of change over time follows the same pattern seen in public school students in Alabama and is likely not attributable to participation in the scholarship program.

Issues for Future Evaluations: • Drawing conclusions regarding the academic achievement of scholarship recipients relative to students attending public schools depends on the number of schools with scholarship recipients that use tests that are utilized by ALSDE in the future. • ALSDE discontinued the use of the ACT Aspire for the 2017-2018 academic year, and may change achievement tests again in 2018-2019. This will further constrain comparisons between attending Alabama public schools over time.