For months and months someone advising Senator Del Marsh about his RAISE/PREP Act insisted that using student test scores to evaluate teacher effectiveness is a grand idea. Usually referred to as VAM, research has shown time after time that this methodology is suspect at best. Some education researchers refer to it as “junk science.”
As time went along in all the back and forth about the legislation, VAM stuck in the craw of many educators who bothered to look at research. But RAISE/PREP proponents (most of whom had very limited education experience) were dug in and unyielding. The result was Senator Marsh finally announcing that he would not bring the bill to the Senate floor this session.
Here is info from yet another research project showing that VAM is unreliable.
The National Center for Education Evaluation and Regional Assistance (NCEE) conducts unbiased large-scale evaluations of education programs and practices supported by federal funds; provides research-based technical assistance to educators and policymakers; and supports the synthesis and the widespread dissemination of the results of research and evaluation throughout the United States.
This NCEE Technical Methods paper addresses likely error rates for measuring teacher and school performance in the upper elementary grades using student test score gain data and value-added models.
Type I and II error rates for comparing a teacher’s performance to the average are likely to be about 25 percent with three years of data and 35 percent with one year of data.
This means that in a typical performance measurement system, 1 in 4 teachers who are truly average in performance will be erroneously identified for special treatment, and 1 in 4 teachers who differ from average performance by 3 to 4 months of student learning will be overlooked.
These results strongly support the notion that policymakers must carefully consider system error rates in designing and implementing teacher performance measurement systems based on value-added models, especially when using these estimates to make high-stakes decisions regarding teachers (such as tenure and firing decisions).
The moral of this story. The next time Senator Marsh tells his staff to write a cookbook, he should ask if any of them know how to cook.
Slowly, but surely, the Education Trust Fund budget is working its way through the legislative process. It has now passed both the House and Senate. However, the Senate made some changes to the House version and this bill will now go to a conference committee to come up with a version that both bodies can agree to.
Senator Del Marsh added an amendment to the Senate bill last week that will impact how educators are compensated for holding advanced degrees. The amendment states in part, “Commencing on January 1, 2017, additional compensation for an advanced degree shall only be provided if the advanced degree is earned in the same field of study as the subject matter the person is teaching…..” Then he goes on to list a number of exemptions, which leaves the door to interpretation swinging in the wind.
And as seems to be the case nine times out of ten when this legislature injects itself into education issues without first consulting with real educators, this amendment creates more questions than answers.
For instance, here is feedback from honest-to-goodness folks with experience in education.
January 1, 2017 seems to be way too soon to start this.
Fewer teachers will want to earn advanced degrees if they won’t earn the master pay; therefore, impacting student enrollment for colleges and universities.
Defining areas of teacher shortage is complex. For example, Shelby County may have 200 applicants for a position in English, while Marengo County has none.
If I am teaching in a critical shortage area this year but next year it is no longer a critical shortage area, is my salary cut?
If a teaching assignment changes from year to year, will compensation also change from year to year?
I have no clue as to why the President Pro Tem added this amendment. He certainly is not adverse to spending taxpayer dollars on salaries as was the case when his chief of staff got a $95,000 boost in five years.
We Southerners are well-versed in Bless his heart. Often it is intended to mean that someone simply can’t help what they do. As in, “Bless his heart, his daddy was an alcoholic too, so what did you expect?”
Given Senator’s Marsh’s track record on legislation that benefits public education (Alabama Accountability Act, charter schools, RAISE/PREP Act), a Bless his heart seems appropriate when discussing this amendment.
There were smiles a yard wide in the education community when reporters tweeted two night ago that Senate President Pro Tem Del Marsh was “shelving” the PREP bill for this session. One of those smiles belonged to me. But at the same time, I remember in 2013 when a conference committee of the legislature went into session carrying a small cherry tomato only to emerge a few hours later with a giant watermelon called the Alabama Accountability Act.
This bill was also the handiwork of the same Senator Marsh.
So perhaps we have heard the last of PREP this session. But never take anything at face value with the crowd now running the statehouse.
This bill should be dead. In fact, it should have never come to life. It was bad from the beginning and stayed that way through change after change. Not only were most of the ideas it wanted to implement weak and not backed by sound research, it was sloppily crafted and left too many questions to be answered.
For example: the bill sets up the Alabama Teacher Recruitment Fund and appropriates $5 million to it. This would be used for $1,000 per year bonuses for teachers in a number of situations. One being a teacher in a school with 80 percent of more free-reduced lunch students.. We had 364 such schools in 2014 according to data from the state department of education. They have about 130,000 students and more than 6,000 teachers. A $1,000 bonus for each would require $6 million. This is more than the appropriation.
Had this bill been turned in as a class assignment by any high school student in Alabama, chances are they would have gotten an F for their grade.
But then, what do you expect when people with no education credentials are writing education policy?
And this was the REAL issue about RAISE/PREP from the beginning. Senator Marsh choice to surround himself with “experts” who were not that at all; instead of real, bonafide education professionals. It was as if he asked some folks who don’t know how to cook to write him a cookbook.
On the bright side of all of this is the fact that for the first time in a long time, the education community pushed back. Hard. Senators got the message loud and clear from teachers, superintendents and concerned citizens that PREP was bad legislation and the public was watching carefully to see who would support it and put politics above the welfare of public school students.
Senator Marsh was apparently surprised by the resistance and attempted to castigate those who disagreed with him, such as the School Superintendents of Alabama. But his argument that superintendents do not want accountability for their systems or their teachers is ridiculous. He might as well have said that a car dealer doesn’t care how many cars he sells. Every superintendent in the state is doing everything within their power to improve both their teachers and their schools. And they know that the last thing they need is a piece of legislation that takes away their flexibility to make the changes and adjustments they need and locks them into going to Montgomery with their hat in their hand to beg permission from the Legislature to do so.
Maybe PREP really is dead for this session. Let’s hope so. And maybe Senator Marsh has realized that educators want to be at the table when education policy is being made–not just on the menu as they have been in recent years.
I have no doubt that educators will be more than happy to sit down with Senator Marsh in the future and partner on policy that makes sense and will actually benefit our students. But that is a decision only he can make.
The most recent version of RAISE/PREP became public last week. Senate sponsor Del Marsh says he is through negotiating about the details of the bill and will move it to the senate floor on Tuesday for a vote.
Does he have the votes to pass it? That’s the $64 question at this point. Many feel he does not. However, these same folks know that Marsh is a very astute politician and a master at deal-making, so you knows what rabbit he may pull out of the hat to sway an undecided vote.
But one thing is certain. This was a bad bill when it first came to light and it remains so today. There are simply too many unanswered questions as to what it requires, who will pay for these requirements, how parts will be interpreted, etc.
For instance, versions prior to the most recent defined STUDENT GROWTH MODEL as “A statistical growth model used to isolate the effect and impact of a teacher on student learning, controlling for preexisting characteristics of a student including, but not limited to, prior achievement”
This is nothing but the very controversial VAM (Value Added Model) process used to evaluate teachers. The same one the American Statistical Association and the American Education Research Association has great reservations about.
In the new bill, STUDENT GROWTH is defined as “The change in achievement for an individual student between two or more points in time as approved by the State Board of Education.”
So VAM is gone. Or is it really?
On page 17, line 6-17 we find that this version requires the State Department of Education to “Develop, implement, and publicly disseminate a method for measuring student achievement and student growth for purposes of teacher evaluations in order to standardize student academic measures and ensure teachers are evaluated according to the impact they have on student achievement and student growth in the classroom or school, for teachers and teachers serving as principal or assistant principals. The department shall work with the Alabama Longitudinal Data System Center created by an act of the Legislature during the 2016 Regular Session. Nothing in this subdivision shall be construed to usurp or diminish the authority of the department in administering and implementing this act.”
For one, at this writing legislation creating the Longitudinal Data Center has NOT been passed this session. Secondly, I sent the above info to 10 educators I highly trust and asked what it means. Most said that it is simply VAM in a new set of clothes.
And let us not forget that the same folks pushing PREP are the same folks who gave us the Alabama Accountability Act in 2013. The bill that said up front that it was all about helping students in “failing schools.” And it was the same cast of characters who came back in 2015 to amend the Alabama Accountability Act so that it is now all about “school choice” instead of helping “failing schools.”
And we are now supposed to believe them?
On page 28 we find the bill creating the Alabama Teacher Recruitment Fund and putting $5 million in it to come from the Education Trust Fund. The intent is to recruit teachers who serve primarily high-poverty schools and teach subjects that are hard to staff.
So who can argue with this?
But as always, the devil is in the details and they are lacking. The bill says a qualifying school is any one where 80 percent or more of students receive free-reduced lunches. Alabama had 364 such schools in 2014 according to info from the state department of education. They had 130,000 students which means they had about 6,500 teachers. If all of them get a $1,000 bonus, that’s $6.5 million, which is more than has been set aside to give bonuses.
So the money is gone before all the other categories of teachers eligible for bonuses are even considered. And no where can I find in the language of this section WHO will decide which teachers get a bonus.
As I’ve said before, we ain’t got enough lipstick in Alabama to make this pig look pretty. And trust me, having grown up on a farm I know what pigs look like.
The latest and greatest version of the infamous RAISE/PREP Act surfaced last week. Once again, changes had been made. And some, such as the Alabama Association of School Boards, who had previously opposed the legislation switched sides. But do not put Jennifer Brown, current Teacher of the Year, in that column She remains very opposed to this bill, perhaps even more so than ever.
Here is a statement she put out late last week.
“After proctoring the 10th grade ACT Aspire at Vestavia Hills high were I teach, I cannot support an evaluation system, whether it comes from the Legislature or the Alabama State Department of Education, which ties teacher evaluations to ACT Aspire scores for the following reasons:
1) The students have had no preparation for this, and there are no computerized practice problems that I am aware of (please correct me if I’m wrong). My math colleague shared that one of her students said to her during class, “For the questions that asked us to WRITE an answer, I didn’t know how to do it, so I just skipped all of them.”
2) Students are told in the directions that this test means nothing to them. Do we really expect our students to take the test seriously after telling them it’s no big deal?
3) Several of the students in my room finished within the first 10 minutes – it was a 60 minute test. Yes, we are supposed to report it if we THINK they selected random answers. How can we PROVE it?
I have many more examples of our students across the state who didn’t see the need to take this test seriously. My Vestavia Hills City Schools Design Team has been working on a model that does not include ACT Aspire scores because we were told the following:
-Teachers will demonstrate student engagement by using ONE or more of the following measures: surveys, benchmark assessments, ASPIRE, district and/or school goals
-Teachers will demonstrate student growth through data sources that indicate a master of a subject or grade-level standard.
According to the PREP Act (Senate Bill 316), schools must use ACT Aspire data for a portion of evaluations. As a teacher, I cannot support an evaluation system that holds teachers accountable for test scores without accountability for students.
The only fair way to use testing data is to go to a pre-test/post-test system, but I want to remind everyone that this means more high-stakes tests because we will have to test EVERY subject at the beginning and the end of the year.
Teachers are not opposed to accountability, but it has to be fair. I do not think that tying our evaluations to a test that students may or may not take seriously is a fair way to evaluate us.
Senator Marsh’s Office has stated that that there will be no further negotiations on this bill.
I am asking everyone to call and email ALL SENATORS to vote NO next Tuesday, which is currently when the PREP Act is slated to be on the Senate floor.
I am also asking you to email the Alabama State Board of Education. They need to know that the ACT Aspire test may be a great test, but if there is no accountability for students taking the test, we should not be legislatively mandated to be held accountable for their results either.”
My friend Dr. Brittany Larkin at Auburn University took a look at Del Marsh’s interview on APTV’s Capitol Journal about the PREP bill. She minces no words in in her comments about what she heard:
“The biggest issue is Senator Marsh seems to believe the failure of Alabama public schools is the fault of the teachers. Thus he believes to improve the schools we have to evaluate teacher, remediate the “bad” ones, and ultimately get rid of the unfit ones. I get it. We want good teachers. We want to have an evaluation system of our teachers. I’ll agree to that. But here is the giant missing piece.
“Teachers have been reduced to factory workers by legislation and rules telling them what curriculum they must use, at what pace to teach, at what rate of growth is acceptable, what acceptable evaluations of growth can be used, etc. etc etc.. So, if we really want to evaluate how effective a teacher is, we have to give them the autonomy to be a teacher. This was their trained profession. Allow them to use their training both in formal schooling and in the job trainings to teach children in developmentally appropriate ways. Then you will see the good teachers from the bad. But putting teachers in shackles of what non-educators (legislators) deems appropriate and successful, then evaluating them on their performance is ridiculous.
“On top of this the legislators have not fully funded education in years. Every time, the state department of education is conservative with their funds in hopes to fully fund programs and services that the legislature requires but doesn’t fund, they get that money swiped away. Are you kidding me! So, you want to tell teachers and school how to do their job, but not give them the money to do it adequately, and then evaluate how well they can do it!
“I’d be willing to bet, the results of the state assessments is a direct result of the states effort to “provide equitable” education services. They think that equitable means giving everyone the same thing (i.e. the scripted curriculum and pacing guides to follow) will produce the same results. This is so far from reality. Reality is kids are not the same, communities are not the same, they don’t start in the same place, they don’t grow and learn at the same rate, they don’t demonstrate their knowledge in the same way. So, you can’t prescribe education as if every student is the same. Period.
“My best solution…
1) the state can provide standards created by a base of experts in our state;
2) give teachers, school systems, etc. the autonomy to do their job! Let them teach the standards the way their students need, using the curriculums, assessments, remediation, etc. to meet the goals;
3) then you can evaluate how effective a teacher/system is at moving students toward mastery of those standards;
4) Once a underperforming teacher is identified, evidence of support and remediation is provided, then a clear and supportive path for termination must be made.
5) Also, the principal must have a very influential, if not total control, over the teachers/staff hired in their building. We cannot evaluate a principal on a teacher’s performance who was “placed” in the school by the district.”
In other words, if you can’t cook, don’t try to write a cook book.