School Leader

Popham on formative vs. summative assessment

In this short video segment, assessment giant W. James Popham gives a brief but thoughtful description of formative and summative assessments and how they differ.  This might be a helpful resource in training teachers on the purpose and value of formative assessment (forgive the cheesy music intro...):

 

Popham describes the research base supporting formative assessment, and notes that, "whereas formative assessment intends to improve on-going instruction, summative assessment tries to answer the question, 'Was instruction effective?'"

I find that, as much as teachers are talking about formative assessment in recent years, there's still a lack of real understanding about these differences.

Popham also notes that, while most statewide standardized accountability tests are considered "summative," most are not particularly good at measuring the real impact of teaching on student learning (what he calls a tests "instructional sensitivity;" for more on this see the final chapter of Popham's book Transformative Assessment).  For this reason, well-designed and implemented formative assessment techniques are one of the best tools for measuring the actual impact teachers are making on student performance.   School leaders should take this into serious consideration, not only for supporting meaningful classroom-level assessment, but also in evaluating teacher performance.

Report on teaching quality ranks Kentucky 41st in nation

After Kentucky's encouraging leap to 14th place nationally in Education Week's annual Quality Counts survey a few weeks ago, this week comes a new report ranking the Commonwealth 41st in teacher quality.

The National Council on Teacher Quality released its 2011 State Policy Yearbook on Wednesday and while Kentucky showed slight improvements in some areas, nevertheless earned a D+ overall in teacher quality.  Read the Associated Press summary of the report here, and the full Kentucky report from NCTQ here.

The teacher quality report criticized Kentucky on several counts, including the preparation of special education teachers, the rather weak link between teacher evaluation and student performance, financial threats to the state's teacher retirement program, and other concerns. 

What to make of the differences between the Quality Counts report and the Teacher Quality report?  For one thing, the two surveys use very different criteria for assessing educational quality.  Quality Counts focuses on a broad set of measures, including standards and accountability structures, school financing, and long-term student outcomes, while the Teacher Quality report focuses much more narrowly on teacher preparation, development, compensation, and evaluation. 

Many of Teacher Quality indicators reflect state-level policies related to certification requirements, teacher training, tenure guidelines, etc., and as such are largely beyond the control of most school-level teachers and administrators (except to the extent we can influence policy through our professional organizations).  Kentucky Educational Professional Standards Board director Phil Rogers indicated to the Associated Press that the state would consider the report's recommendations carefully.

From my perspective, the most important implications of the Teacher Quality report for school-level leaders is the emphasis on teacher mentoring, development, and evaluation - all of which are ultimately local concerns and responsibilities.  The state's recently-approved-but-still-in-development teacher evaluation overhaul (which, if implemented, would undoubtedly raise Kentucky's ranking on the Teacher Quality report) is designed to build much greater capacity in schools for improving teacher practice and the link to student achievement.  I have no problems with the what the state is proposing (though I am skeptical they'll get the necessary inter-rater reliability to establish the system's validity), but I don't think such structures can really yield strong results without extremely high-quality instructional leadership at the school level.

Teacher quality is definitely a statewide problem, but the secrets to solving it lie in the interactions and conversations between principals and teachers in the classrooms, hallways, and meeting rooms of schools in our own neighborhoods.

Charter advocates rally in KY capitol

Many of my students and readers of this blog know that I'm an advocate for expanding school choice.  After 16 years working in the trenches as a teacher and administrator in both public and private school contexts, I've become convinced that something is terribly wrong with our system when people have more choices of toothpaste at their local convenience store than in educational options for their children. 

I believe real innovation, meaningful accountability, and sustainable gains in student learning will only come when we've vastly expanded the marketplace of educational choices.  Many teachers and administrators fear school choice, but I'm convinced it's the pathway to greater autonomy, professionalism, and job satisfaction for educators and vastly superior learning alternatives for students and their families.

One mechanism for expanding school choice is charter schools.   Kentucky remains one of only nine states that don't allow some form of charters and yesterday pro-charter advocates rallied in Frankfort to ask state legislators to make room for new educational choices in the Commonwealth.  One of the organizing groups is Kentuckians Advocating for Reforms in Education (KARE), which recently launched an ad campaign promoting charter schools in the state.  KARE's second television ad debuted this week: 

I don't necessarily agree with some of the rhetoric of these ads.  Many communities in the state have excellent public schools (as schools go).  But the value of expanding school options is still very relevant and important, even in such communities.

This is National School Choice Week, and Reason Foundation education analyst Lisa Snell offers three good reasons for why school choice is on the rise:

 

Task, Text, Talk: Teaching Kids "News Literacy"

The January 2012 issue of ASCD's Education Update (available to members only) features an article on The News Literacy Project.  This initiative is designed to actively engage undergraduate college students in critically analyzing current events reporting from a wide variety of media.  The goal is to promote critical thinking on the part of students, and especially the capacity to identify bias and discriminate among credible (and suspect) sources of news information.

The News Literacy Project's resources are available to all educators, and Education Update writer Willona M. Sloan recommends schools incorporate these materials into their civics, government, social studies, and media courses.  The video below highlights how the pr0gram is being used in three urban districts:

 

I haven't used these materials and so I can't comment directly on their value, but I am reminded of Mike Schmoker's sweeping recommendations for improving social studies instruction in his 2011 book Focus: Elevating the Essentials to Radically Improve Student Learning.  Schmoker suggests a major shift toward a "task, text, talk" approach that spends a heavily portion of every class with students reading, analyzing, and arguing about various real-world texts.  The News Literacy Project may be one of several sources social studies teachers can use to build their libraries of materials for just such activities.

Danielson and other teaching frameworks found to help distinguish effective and ineffective teachers

Dovetailing with the study I discussed earlier this week, the Gates Foundation has released new research confirming that teaching frameworks like those designed by Charlotte Danielson and others accurately predict teacher effectiveness as measured both by student perceptions and student achievement scores.

Read the full study here and a summary of the study's findings from Education Week here.

There are multiple layers to the results, all worth pondering for their practical implications (including the unsurprising finding that the frameworks were far more accurate depending how many observers evaluated teachers using them).  Perhaps the most significant finding, however, was the level of impact of high- versus low-performing teachers as measured by the frameworks on student achievement:

For instance, the study estimated that the difference in learning between students taught by teachers in the top and bottom quartiles of effectiveness, using just the Framework for Teaching as a measure, amounted to about 2½ months of growth in learning. But when the additional factors were included, the spread between top and bottom teachers’ effectiveness grew to 7½ months of learning, or almost a full year of schooling.

All of this is perhaps further confirmation that the school principal's single-biggest responsibility is to identify the performance level of teachers and then reward, celebrate, and encourage the best, providing ample structures for the middle performers to improve, and assist the rest in finding a new professional role.  To do this, of course, principals need to be well-versed in the frameworks for effective teaching themselves and willing to engage in deep, meaningful dialogue with teachers about their practice on a daily basis. 

I'll look forward to discussing these ideas and more in class this semester.

How much value does value-added add? A lot.

Last week the New York Times reported on a ground-breaking new study from researchers at Harvard and Columbia Universities that shows the impressive aggregate impact high-quality teachers make on their students and on society as a whole.

The study looked at various "value added measures" of student achievement (the impact a teacher makes on a student's achievement from the beginning to the end of the year in which the teacher instructs that student), gathering data from 2.5 million students nationwide and examining various outcomes for those students over a 20 year period.  The authors confess to being value-added skeptics prior to their research, doubting that such measures had much validity or long-term impact on student achievement.  They were wrong.

Data from the study indicated that while the impact of high-valued-added teachers on individual students appear relatively modest, when you consider the aggregate impact of these teachers on their students as a whole, the results are unequivocally positive.  What's more, the differences of impact between average and low value-added teachers were also striking:

Replacing a poor teacher with an average one would raise a single classroom’s lifetime earnings by about $266,000, the economists estimate. Multiply that by a career’s worth of classrooms.

“If you leave a low value-added teacher in your school for 10 years, rather than replacing him with an average teacher, you are hypothetically talking about $2.5 million in lost income,” said Professor Friedman, one of the coauthors [all of whom are economists]...

Students with top teachers are less likely to become pregnant as teenagers, more likely to enroll in college, and more likely to earn more money as adults, the study found.

The results of this study shed new light on various value-added measures of teacher effectiveness, suggesting that, while these metrics are sometimes imperfect, in the aggregate they do yield tangible evidence of teacher impact on learning.  As the authors suggest, districts should pay closer attention to student outcomes and consider regularly non-renewing contracts of lower performing teachers.

Kentucky General Assembly: Charter bill may get hearing; more

Last week Kentucky House Education Chair Carl Rollins (D-Midway) indicated he may let proposed charter school legislation (such as that proposed by Rep. Brad Montell, R-Shelbyville) come to the committee for discussion.  In the past, charter legislation has been blocked from debate.

This is a positive step.  Rollins is pushing his own bill to create schools and districts "of innovation," which would create an application process whereby schools could seek charter-like status, free of many prevailing educational regulations .  Kentuckians Advocating Reform in Education, a pro-charter group running a statewide advertising campaign, opposes Rollins' idea.

Personally I have no problems with Rollins' proposal, but it is not an acceptable substitute for real charter schools (namely because most students in "schools of distinction" - like traditional public schools - have no other choices if they are unhappy with the education their child is receiving).

At any rate, these are all positive developments for school choice in Kentucky - and the accountability and innovation that come with it.

One last note: research about the performance of students in charter and voucher schools versus traditional schools always comes up when this topic is discussed.  Most research studies show mixed results (though other researchers argue otherwise).  But a recent review shows that most of these research studies are methodologically flawed, and those that aren't show positive results for students who exercise school choice.

Ed Leadership to encourage discussion on school choice debate

A couple of weeks ago I blogged about James Harvey's piece in the current issue of Educational Leadership magazine.  I sent my comments to the editors, and they offered to publish my reaction in the March 2012 issue, along with a response from Dr. Harvey, and to post both on the ASCD website and encourage reader discussion and debate.  I appreciate the editors' reply and commend their willingness to foster more dialogue on this topic.  I'll post more details as they come available.

Kentucky soars on Ed Week's "Quality Counts" report; lots of work still to come

Education Week released its 16th annual Quality Counts report yesterday, which grades states on educational quality in six key areas.  This year Kentucky's rank rose from 34 to 14, earning a grade of C+ (no states received an "A").

Read Governor Steve's Beshear's press release detailing Kentucky's performance here.  You can find the executive summary here, view state-by-state results here, and see other Quality Counts resources and information here. 

Education Commissioner Terry Holliday credited Kentucky's rise to the state's passage and implementation of Senate Bill 1, which overhauled curriculum, assessment, and accountability systems.  It's hard to parse out the data to verify this claim, but Kentucky did score an "A" - its highest category grade - in "Standards, Assessment, & Accountability."

For all the frustration educators have felt this year with the implementation of new curricula in reading and math, and all the uncertainty attached with a new statewide testing system, it's affirming to know that outside observers view these changes in a positive light. 

That said, teachers in the trenches could be forgiven for feeling a little less than enthusiastic.  The enormous attention paid to the new curriculum this year, and justified anxiety about the new test (and even murkier components like program reviews) mask serious underlying challenges facing our schools.  Massive work still needs to be done in developing, unpacking, and prioritizing the curriculum - and that's just in reading and math.  Most schools still had major work to do in social studies, science, and other areas (and many are postponing making any changes pending possible national standards in those areas).  In other words, the most important work of finding what Mike Schmoker calls "focus" is on hold throughout the state.

I'm supportive of most of the reforms in Senate Bill 1 (if not the timeline for implementation), but anyone who thinks these top-down changes are our silver bullet for sucess might be very disappointed, the good news from Quality Counts notwithstanding.

Students: The missing piece in formative assessment

Larry Ferlazzo writes for the Classroom Q&A blog at Education Week.  Earlier this week he posted in response to a reader's question about good strategies for involving students in the formative assessment process.  Ferlazzo draws from his own work and contributions from several other authors and readers to offer a somewhat disconnected but nevertheless helpful collection of ideas and reflections on this crucial, often missing, piece of formative assessment.  Suggestions include having students regularly self-assess their own learning progress, develop their own learning goals, and use technology to make recordings to evaluate their reading or speaking fluency, just as one example. 

These are all good ideas.  I would just add that W. James Popham has written about as brilliantly as anyone on the topic, especially in his book Transformative Assessment.  Popham's framework makes it clear that assessment that only informs the teacher about student progress (even when it leads to real instructional adjustments) is not bad, but comes nowhere close to realizing its full potential.  Popham sees optimal formative assessment as a process, which begins with clear, student-friendly learning targets that engage students in clearly understanding and measuring their own mastery and that guide every aspect of instruction and assessment.  Optimal formative assessment, by definition, makes students a full partner in the learning process:

Formative assessment is a planned process in which assessment-elicited evidence of students' status is used by teachers to adjust their ongoing instructional procedures or by students to adjust their current learning tactics.

The best set of simple strategies for actively engaging students in formatively assessing their own learning is found in Susan Brookhart's Formative Assessment Strategies for Every Classroom (2nd edition).  Many of the tools in Brookhart's book need to be (and easily can be) adapted for different subject areas and grade levels.  I've used this book several times in facilitating Formative Assessment Academies, and strongly recommend it, especially if your interest is in rediscovering the essential student component of formative assessment.