Another Look At NAEP scores

Several days ago I wrote about a senator taking to the floor of the senate to denounce the latest Alabama scores on the National Assessment of Educational Progress.  As I pointed out, he did not bother to look at trend lines and instead, only singled out scores from one test year.

Most of us have tried to lose weight at some time.  (But for me, not lately.)  Let’s say that we’ve worked hard so far in 2016 and have now lost 20 pounds.  However, we had a wee too much to eat last Easter Sunday and when we hopped on the scales Monday, we were one pound heavier than the day before.  Should we pitch a fit about just that one day and declare that our diet is a total and complete failure?

I suppose only if we are in the Alabama Senate and trying to make the numbers tell a pre-determined story (that our public schools are going backwards.)

So I looked at our NAEP scores again and looked again at how we are doing on our diet since the first of the year, not just on the Monday after a big Sunday dinner. I found that if you look at Alabama scores for 4th grade reading and math back to 1992 and at 8th grade reading back to 1992 and 2002 (as far back as info on the national NAEP site goes for 8th grade reading) you find that GAINS in Alabama have EXCEEDED national gains in all four cases.

In 4th grade math, we went up 23 points, nationally the gain was 21 points.  In 8th grade math, we went up 15 points, while the national increase was 14 points.  For 4th grade reading, Alabama increased 10 points, the national gain was 6 points.  And for 8th grade reading, we gained 6 points and the gain nationally was 3 points.

And here is something especially interesting.  The proposed RAISE/PREP Act says we will use something called VAM (Value Added Model) process to determine how good our teachers are and how we can adjust education to make more rapid gains.

The first VAM was created by Dr. William Sanders in Tennessee and was put into use in the Volunteer State in 1992.  The proponents of this very inexact methodology want us to believe it is the best thing since sliced bread.  And since Tennessee has been using it for 14 years, they must be blowing our doors off down here in Alabama.

Well, not so fast.  Truth is that when you compare NAEP scores in Alabama to those in Tennessee you see that reading scores for our 4th graders have risen more from 1992 until 2015 than their counterparts in Tennessee.  Same for 8th grade reading scores from 2002 to 2015.

Of course, this hardly fits the narrative of those non-educators wanting to tell our teachers and school how to do things.  But then, facts can be troublesome at times and often get in the way of political agendas.


Please Don’t Let Facts Get In The Way

Given that some in Alabama are hell bent to show how bad our public schools are it is hardly a surprise when folks take things out of context and twist them any way the want to.

Take the National Assessment of Educational Progress (NAEP) tests that a senator recently railed about on the floor of the Senate.  He was upset that Alabama scores dropped slightly from 2013 to 2015.

But here is what he did not say.  NAEP is often called the “gold standard” of testing because it is a way to compare schools across the nation.  This test is given every two years for fourth grade reading and math and for eight-grade reading and math.  Students get no grade and therefore, have no incentive to perform well.

Students and schools are picked at random.  Tests are not aligned to new standards.  About 2,500 students in Alabama are tested.  Test is 75 minutes long.  About 30 students are tested per school.  Some of them must be students with disabilities and English language learners.

There are 730,000 students in our public schools.  So we are judging all of them on the performance of just .003 of them.

Had the senator bothered to do his homework he would have found that fourth grade math scores were down in 16 states and up in only four in the 2015 testing cycle.  He would have found that eighth grade math scores were down in 22 states.  So what happened here was not an anomaly.

Had he bothered to look on the NAEP web site, he would have learned that the change is Alabama scores is not even considered statistically significant.

Had he bothered to look he would have found that since Alabama started NAEP in 1992, we have narrowed the gap between our scores and national scores in fourth grade reading and math, as well as in eighth grade reading and math.

So the truth is that we are doing better–not worse.

But then, why let the truth get in the way when you are trying to prove an invalid point?

Never Let The Facts Get In The Way Of A Good Story

I know nothing about the Alabama Policy Institute.  Don’t know what purpose they supposedly serve, how they pay their bills, etc.  But with a name such as they have, one would think they should be all about research and policy statements and educating the public.

However, all I ever see are articles in some media outlet, such as this one, that is clearly about pushing an agenda, instead of laying all the facts on the table.

In this case they try to make the case that there is precious little relationship between resources spent on education and student performance.  To do this, they talk about how the state of Alabama ranks in comparison to other states on the most recent National Assessment of Educational Progress (NAEP) scores.

For instance, the article states, “Since 2000, rankings in math for fourth and eight grade students fell from 35th and 32nd place, respectively, to 51st and 50th place in 2015.”

The implication clearly is that our students are doing worse and whatever money we are spending on education is being wasted.  The only problem, they are talking about state rankings–not the actual scores of students.

So let’s look at what is meaningful, how students score on NAEP.

Fourth-grade math scores in 2000 for Alabama were 217.  In 2015, they were 231–an increase of 14 points.  That was exactly what was done nationally, an increase of 14 points.  Eighth-graders increased five points.  The national increase was nine points.

In reading, fourth grade scores jumped ten points from 2002 to 2015 (the national increase was four points) and eighth grade scores rose six points (the national average of one point).  So in four measures, we beat the national average in two and tied it in one.

The state of Massachusetts is often considered to have the top school system in the nation.  Alabama showed a bigger gain in math scores for both fourth and eighth grade in this time period than they did.

Yet the folks at Alabama Policy Institute discard the truth and want us to believe our students are going backwards–not forward.

The article above says, “Alabama’s rankings on the NAEP in math and reading have largely collapsed.”  The only thing I see collapsing is the credibility of the organization when they put out such as this.