How does Kentucky's education system stack up against other states?

It seems like every few months a new report emerges ranking the states' educational systems.  In several over the last few years Kentucky appears to have shown improvement.  This week a report from Johns Hopkins University's Civic Education and Everyone Graduates Center celebrates Kentucky's seeming success in improving graduation rates, especially for students of poverty.  I'll address that report in a future post. 

In this post, I want to challenge the idea that it's so easy to make these comparisons in the first place, and to share some data that suggest Kentucky isn't really making as many huge strides as the headlines would suggest.  This is not meant as a criticism of the hard-working teachers, school administrators, and education officials who are trying valiantly to improve student learning.  I've concluded that we have deep systemic problems in education that actually prevent innovative educators from having the impact they could otherwise, and you can find my thoughts about how to address that challenge in other posts and in future posts.  Here I just want to make the case that Kentucky still has a long, long way to go in improving student learning outcomes. 

My thinking on this is greatly shaped by the work of Richard Innes, education analyst for the Bluegrass Institute.  Dick makes an excellent effort to explore data trends giving much greater insight into how our schools are actually doing than most ranking systems or single scores of achievement can convey (by way of disclosure, I serve on the Board of Scholars for the Bluegrass Institute, so I follow Dick's work closely and help advise the Institute on education policy proposals).

The first point to make is that finding consistent metrics for cross-state comparison isn't easy.  States calculate graduation rates in multiple ways, don't always test all their students, and use a wide variety of testing systems.  The National Assessment of Educational Progress (NAEP, also known as "the nation's report card") is considered one of the best measures because it is well respected and administered nationwide. 

But even doing state-to-state comparisons with the NAEP is difficult for two reasons.  First, the statistical margin of error on the NAEP can be large enough to wipe out differences between states that a simple ranking of average statewide performance can otherwise suggest.  And second, the demographic make-up of states is also extremely variable and we know differences in student achievement across demographic categories is one of the most difficult educational challenges to address.  For this very reason, the National Center for Educational Statistics (producer of the NAEP) has cautioned since 2005 that overall state achievement comparisons are problematic, and suggests comparisons of specific demographic groups as a more reliable method.

This is exactly the approach Dick Innes has used in a recent (and on-going) analysis of 2015 NAEP data. To address the demographic differences across states, Dick is comparing the achievement of only Kentucky's white students to white students in other states.  A similar analysis could be done with any demographic group, but since whites are the largest racial category in Kentucky, it makes sense to start here.  Dick is also comparing the achievement of white students who were eligible for free and reduced lunch with their counterparts from other states on the 2011 and 2013 NAEP, but since the advent of community-based free/reduced lunch eligibility, those data are now somewhat meaningless for 2015 (a big problem for educational research that is thoughtfully discussed here).

As one example of the charts Dick has created using the NAEP Data Explorer tool, see how Kentucky's white 8th graders compared to whites in other states on the 2015 NAEP math test.  They outperformed their counterparts in only two other states (Alabama and West Virginia).  We've hovered in this same range since 2011.

Meanwhile our white students receiving free/reduced lunch lost ground to their peers in other states between 2011 and 2013 on the 8th grade NAEP math assessment (remember that there is no reliable data on this demographic group for 2015).

Other findings from Dick's analysis:

• In 4th grade math, while it looks like Kentucky's white students improved relative to other states in 2015, our scale scores actually stayed flat, while the national average score for this demographic group declined.  So we're better in comparison to other states, but not really in comparison to our historical performance.
• 4th grade reading scores for Kentucky whites did improve in 2015, but Kentucky white students receiving free and reduced lunch declined from 2011 to 2013.
• Despite those real improvements in 4th grade reading, 8th grade reading scores have been in decline compared to other states since 2011.

I'll share more details as Dick posts these analyses on the Bluegrass Institute blog.

It's important to note that, comparisons with other states aside, Kentucky 4th and 8th graders have in fact improved their performance on all NAEP subtests over the last few decades.  But here's the thing: the increment of improvement is so small that, at this rate, it would take about 50 years to reach 80% proficiency in 4th grade reading, and over 100 years to reach the same level of proficiency in 8th grade math.

That rate of improvement is just unacceptable given the economic and social challenges Kentucky - and the United States in general - now faces.  Much more dramatic changes in the nature and structure of education policy and delivery are in order.

Usual disclaimer: Opinions expressed on this blog are mine alone and don't reflect the views of Western Kentucky University or the Kentucky Board of Education.