Information Age Education Blog
If you enjoy reading the Information Age Education Blog entries, please subscribe. The subscription is free. When a new blog entry is published (currently about one per week) you will receive email notification. To subscribe, click on Subscribe to blog near the top of the page. Note that IAE does not sell use of its mailing lists.
"Not everything that can be counted counts, and not everything that counts can be counted." (Albert Einstein; German-born theoretical physicist and 1921 Nobel Prize winner; 1879–1955.)
My career has included being a university Department Head, starting and running the International Society for Technology in Education which was a $3 million a year organization when I left it, and running a large number of funded projects. In these professional activities, I felt that I was not particularly skillful at evaluating the staff working under me.
I recently read an article about “value added” methods for teacher evaluation. Quoting from the Wikipedia:
Value-added modeling (also known as value-added analysis and value-added assessment) is a method of teacher evaluation that measures the teacher's contribution in a given year by comparing the current test scores of their students to the scores of those same students in previous school years, as well as to the scores of other students in the same grade. In this manner, value-added modeling seeks to isolate the contribution, or value added, that each teacher provides in a given year, which can be compared to the performance measures of other teachers. VAMs are considered to be fairer than simply comparing student's achievement scores or gain scores without considering potentially confounding context variables like past performance or income. It is also possible to use this approach to estimate the value added by the school principal or the school as a whole.
William Eger is a high school math teacher and insightful writer. In his article, Education Is Not 'Moneyball': Why Teachers Can't Trust Value-Added Evaluations Yet (8/27/2014), Eger compares the process of evaluating professional baseball players with the current value added approaches to evaluating teachers.
Current Teacher Evaluation Systems
According to the teacher evaluation systems currently being used, very few teachers are performing at a “needing improvement” or “ineffective” level. This is in marked contrast to what is typically being reported in the mass media or talked about by politicians.
Quoting from Teachers Highly Rated Under Hawaii, Delaware Systems (Sawchuk, 8/22/2014):
About 40 states have put in new evaluation systems to replace previous ones that had been largely pro forma.
If you've been reading Education Week's coverage of teacher-evaluation reform—for example, here and here, then this won't come as a huge surprise. As in many other states, among them Michigan, Florida, and Indiana, only a small fraction of teachers are getting low ratings.
Hawaii's breakdown, per the Star-Advertiser: 16 percent of teachers in 2013-14 were deemed highly effective and 81.7 percent were effective. Just 2.1 percent needed improvement and only 0.2 percent were ineffective.
Delaware’s teachers scored even better: Just under half of teachers got the top rating, and 51 percent got an effective rating. One percent needed improvement and not a single teacher was considered ineffective.
In my opinion, these results should be viewed with considerable suspicion. The teacher evaluation systems being used do not appear to me to adequately identify teachers who need help to appropriately help their students meet the student standards that are in place and/or are being implemented.
Evaluating Professional Baseball Players
Over the past couple of decades, the method of evaluating professional baseball players has undergone a great deal of change. Quoting from Eger (8/27/2014):
About 20 years ago, “sabermetricians” revolutionized baseball by analyzing hitters not in terms of isolated stats like home runs or batting average but in terms of their overall value. To try and define value they used regression analysis to create a metric called “Value Over Replacement Player” (VORP), the predecessor of the now ubiquitous Wins Above Replacement (WAR).
The idea behind VORP and WAR is simple: The objective is to calculate how many “wins” a batter will generate above a hypothetical standardized backup. For example, this season, catcher Buster Posey has provided 3.5 WAR for the San Francisco Giants, implying that if the Giants had started a typical backup in lieu of Posey, the Giants would likely have won about four fewer games.
WAR has proven to be a much more effective way of evaluating a baseball player than the observational, personal opinion, scouting reports, etc., methods used 20 years ago.
Eger argues that our current teacher rating systems are rather like the systems baseball used 20 years ago, and have a long way to go to catch up to the current baseball WAR system. Quoting from his (8/27/2014) article:
While sabermetricians were perfecting VORP [and WAR] calculations, education researchers were rethinking teacher performance. By employing regression analysis in a similar manner, researchers were able to predict a student's expected growth on end-of-year state tests. The difference between the prediction and the student’s actual performance would be the "value" that a teacher either added or subtracted during the year.
The resulting systems, known as “Value-Added Models” or VAM, are as conceptually simple as WAR. For example, if a student scores higher than 30 percent of her peers on her 9th grade state math test, she would be predicted to score at least that well on her 10th grade state math test. If after taking the test she scored higher than 50 percent of her peers, then her math teacher, the theory goes, is credited with the 20 percent increase.
Eger then goes on to point out some desirable traits of having a WAR-type system being used to evaluate teachers, and the inherent difficulties of developing such a system. A key argument he makes is that professional baseball games are played in parks that are relatively the same, and that the rules of baseball are precisely defined. Such is not the case for teachers. Quoting from his (8/22/2014) article:
At its core, teaching is the summation of subtle teacher-student interactions. These interactions are shaped by both the teacher and the class composition. A class with two innately strong helper students, for example, will likely do better than a class without such students, regardless of the teacher. Strong classes can be derailed by a single broody adolescent in ways that value-added measurement is unable to foresee. Of course excellent teachers can create helper students or inspire a morose teenager. But instead of rewarding exceptional performance, value-added models will view this teacher as merely on par with another teacher who was luckily assigned more diligent students—or who is “lucky” enough to teach at a school brimming with helpful students and student emotional support structures.
Ultimately value-added models fall short of WAR in terms of effectiveness because teachers, unlike baseball players, don’t control their “on-field” performance. In a statement cautioning educators about too quickly implementing value-added models, the American Statistical Association reviewed the academic literature and found that teacher quality only explains between 1 and 15 percent of the variations found in VAM.
In brief summary, we need teacher evaluation systems that are helpful in improving the quality of education our students are receiving. Good systems can help drive the creation of better preservice and inservice preparation of teachers (Eger & Zuckerman, 3/1/2014). We are a very long way from having such teacher evaluation systems in widespread use.
What You Can Do
Spend some time thinking about your unique characteristics as a teacher that make you different from many or most of your colleagues. For example, you may be extra strong in peaking your student’s curiosity and getting them to pose insightful questions. Perhaps you are particularly skilled in areas such as texting and use of social networking systems, and you take advantage of this in your teaching. How do you, personally, evaluate the effectiveness of your unique characteristics? How might a teacher evaluation system that is designed to be used with large numbers of teachers be able to “catch” your unique, highly effective teaching characteristics? What can you do to help your professional colleagues gain added strength in your areas of strength?
References and Suggested Readings
Crawford, A. (9/2/2014). Teacher effectiveness – A race to nowhere redux (the urgency of now). Getting smart. Retrieved 9/2/2014 from http://gettingsmart.com/2014/09/teacher-effectiveness-race-nowhere-redux-urgency-now/. Quoting from this docuent:
Two years ago I wrote a column about the misguided emphasis of the Teacher Effectiveness and Accountability movements. I highlighted the fact that, in America, we focus on measurement rather the systemic changes required to support teachers in becoming effective at reaching all students in a more rigorous college-and-career-ready-standards environment. Sadly, it feels like we’re on an endless treadmill. Little has changed in these past two years, and the need for change holds true.
Districts need to take responsibility for building systems of support for teachers and principals. And we need accountability placed on districts to overhaul professional development to actually impact instructional practices. According to the research, less than 1% of PD today impacts student achievement, and we spend ridiculous amounts of money on the traditional lecture/workshop-style professional development favored by school districts. We preach blended learning for kids, so why not begin to think about blended professional learning for teachers and principals? These blended learning scenarios can address not only the needs for more intensive acquisition of knowledge and skills, but they can also support teachers’ ongoing, informal knowledge sharing and virtual learning communities of fellow connected educators.
(Eger, W. (8/27/2014). Education is not 'Moneyball': Why teachers can't trust value-added evaluations yet. Education Week. Retrieved 8/29/2014 from http://www.edweek.org/tm/articles/2014/08/27/fp_eger_valueadded.html. If you are a serious major league baseball fan, read about "moneyball" at http://en.wikipedia.org/wiki/Moneyball. Quoting from the cited website:
Moneyball: The Art of Winning an Unfair Game is a book by Michael Lewis, published in 2003, about the Oakland Athletics baseball team and its general manager Billy Beane. Its focus is the team's analytical, evidence-based, sabermetric approach to assembling a competitive baseball team, despite Oakland's disadvantaged revenue situation. A film based on the book starring Brad Pitt was released in 2011.
Eger, W., & Zuckerman, M. (3/1/2014). There's a cheaper, more effective way to train teachers. The Atlantic. Retrieved 8/29/2014 from http://www.theatlantic.com/education/archive/2014/01/theres-a-cheaper-more-effective-way-to-train-teachers/282778/.
Sawchuk, S. ( 8/22/2014). Teachers highly rated under Hawaii, Delaware systems. Education Week. Retrieved 8/29/2014 from http://blogs.edweek.org/edweek/teacherbeat/2014/08/the_two_latest_states_to.html.
Suggested Readings from IAE
Moursund, D. (August, 2014). Empowering learners and teachers. IAE-pedia. Retrieved 8/29/2014 from http://iae-pedia.org/Empowering_Learners_and_Teachers.
Moursund, D. (8/10/2014). Four parables about educational reform. IAE Blog. Retrieved 8/29/2014 from http://i-a-e.org/iae-blog/entry/three-parables-about-educational-reform.html.
Moursund, D. (6/29/2014). Adequacy of teacher preparation. IAE Blog. Retrieved 8/29/2014 from http://i-a-e.org/iae-blog/entry/adequacy-of-teacher-preparation.html.
Moursund, D. (8/25/2013) Stop picking on teachers. IAE Blog. Retrieved 8/29/2014 from http://i-a-e.org/iae-blog/entry/stop-picking-on-teachers.html.
Moursund, D. (11/30/2012). High school graduation rates are only one measure of educational success. IAE Blog. Retrieved 8/29/2014 from http://i-a-e.org/iae-blog/entry/high-school-graduation-rates-are-only-one-measure-of-educational-success.html.
Moursund, D. (November, 2010). Assessing our schooling system. IAE Newsletter. Retrieved 8/29/2014 from http://i-a-e.org/newsletters/IAE-Newsletter-2010-54.html.
Nocera, Joe (8/29/2014). Imagining successful schools. The New York Times. Retrieved 9/6/2014 from http://www.nytimes.com/2014/08/30/opinion/joe-nocera-imagining-successful-schools.html?emc=edit_th_20140830&nl=todaysheadlines&nlid=50637717&_r=1. Quoting from this article:
What should teacher accountability look like?
We know what the current system of accountability looks like, and it’s not pretty. Ever since the passage of No Child Left Behind 12 years ago, teachers have been judged, far too simplistically, based on standardized tests given to their students — tests, as Marc S. Tucker points out in a new report, Fixing Our National Accountability System, that are used to decide which teachers should get to keep their jobs and which should be fired. This system has infuriated and shamed teachers, and is a lot of the reason that teacher turnover is so high, causing even many of the best teachers to abandon the ranks.
All of which might be worth it if this form of accountability truly meant that public school students were getting a better education. But, writes Tucker, “There is no evidence that it is contributing anything to improved student performance.” Meanwhile, he adds, test-based accountability is “doing untold damage to the profession of teaching.”