Heather Hill is a leading researcher on professional learning, teacher evaluation, high-quality instruction, and mathematics teaching. She is the Jerome T. Murphy Professor in Education at the Harvard Graduate School of Education and a fellow of the American Educational Research Association (AERA).

In addition to serving on the editorial boards of several education journals, she writes a column for Education Week called “What Works, What Doesn’t,” which explores how to apply education research to practice. She recently shared insights about professional learning research with The Learning Professional.

Q: Educators and policymakers often make blanket statements about whether professional learning works or doesn’t work. Your research has shown that it’s more complicated. What are the most important factors to look at when assessing the impact of professional learning?
A: In a perfect world, I would look for two things to assess impact: whether the professional learning has improved instruction in classrooms and whether it has resulted in students who perform better with respect to the content the professional learning is designed to address.

Because we are in an imperfect world, it’s very challenging to determine whether outcomes improve as a result of teacher professional learning, in part because the standardized tests used in many states are not sensitive to academically rigorous instruction.

Instead, educators may have to look at what’s in professional learning to make a judgment about the elements that are likely to affect teaching and learning. For example, my study with Kathleen Lynch and others at Harvard found that STEM professional learning programs that focus on helping teachers learn to use curriculum materials were more effective (in terms of improving student achievement) than programs that featured professional learning alone. Combining professional learning with curriculum materials was also more effective than programs in which teachers simply received new curriculum materials.

Why programs that combine curriculum materials with professional learning are more effective is an interesting question. It’s possible that the professional learning helps teachers fine-tune how they use the curriculum materials, maybe by focusing on the underlying STEM content or on the specific teaching techniques the curricula contain.

Another possibility is that this professional learning may encourage teachers to use the curriculum materials, improving fidelity and the overall quality of instruction, versus a situation in which teachers are cobbling together lessons from sites such as Teachers Pay Teachers.

Still another possibility is that professional learning without curriculum materials leaves teachers without much structure for actually changing what they do on a day-to-day level in the classroom.

Our study also found that programs aimed at improving teachers’ content knowledge and pedagogical content knowledge tended to outperform those that did not focus at all on this topic. Programs also benefited from what we called “implementation meetings” — chances for teachers to come back together after the beginning of the program to talk about successes and challenges using the program’s teaching techniques. And, surprisingly, programs that contained a summer workshop did better than those without.

Teachers have long said that they want professional learning that is immediately useful and helps them teach better. My takeaway from this study is that they are right: Professional learning that digs deep into the heart of teaching and gives teachers either content knowledge or concrete strategies for implementing new practices makes a difference for students.

Q: Can you talk about your research on implementation fidelity — that is, how well educators use an intervention as it was designed and intended? What should educators take away from this research?
A: My co-author Anna Erickson and I looked at the relationship between implementation fidelity and student outcomes in 76 federally funded studies. These studies evaluated many different kinds of programs — for instance, new math curricula, professional development focused on social-emotional learning, and writing programs, and most captured both implementation fidelity and student outcomes. As expected, we found that stronger implementation fidelity increased the odds of seeing positive results on student outcomes.

What was surprising to us is that moderate and high-fidelity implementation produced more positive student outcomes at about the same rate. It was only low-fidelity implementation that led to weaker student outcomes. In line with my comments above, we also found that programs that featured new curriculum materials as a major component tended to report better fidelity.

Q: Teachers have lamented for years that a lot of professional learning is not worthwhile because it is short-term and not connected to teachers’ practices in the classroom. How can we weed out ineffective sit-and-get workshops and improve the overall quality and effectiveness of professional learning?
A: I wrote an EdWeek article that reviewed research on teacher professional learning, which explains that coaching and curriculum-focused professional learning programs have, on average, positive results (Hill, 2020).

Weeding out sit-and-get type programs will probably take a few things: changes in state laws that require that teachers cover specific content or take professional learning in a specific format each year; states and districts to refrain from pushing new policies out via professional learning; and regular time for teachers to engage with coaches or colleagues during their regular day or week.

Q: It seems that one important strategy for improving the quality of professional learning is for districts to research how well their professional learning approaches are working. Yet capacity to do this kind of research is limited. What advice do you have for districts about learning from their own work in order to improve it
A: You’re right, capacity is really limited, especially now with COVID straining district resources. To do research on the efficacy of professional learning (or any new program, in fact), districts need to have three things.

The first is the ability to compare a group of classrooms receiving the new program to those that do not. Sometimes comparison groups can be hard for districts to arrange — everyone needs to receive the new program at the same time.

But if there’s a chance to pilot the program with a randomly selected set of teachers or release the new program to half of the district’s schools in one year and then the other half in the next, this can provide the kind of comparison group that is needed to identify the effect of the program.

The second thing they need is measures that can track change in teachers’ classroom instruction and changes in student outcomes. This is a little more tricky. In my field, STEM education, there are lots of classroom observation and student assessment instruments for use in research, but there hasn’t been a ton of progress toward instruments that are both easy to use and accurate when used “live” in school settings.

The best option might be to cobble together measures from the teacher evaluation rubrics used in districts and interim assessments given to students. But these often aren’t fully aligned with new programs. For the classroom observations, you will also need more observations per classroom than teacher evaluation systems typically collect, and they must be timed around the beginning and end of the professional learning program.

The third thing is capacity to collect and analyze the data. If you’re looking to augment analytic capacity, two strategies to leverage are research-practice partnerships and doctoral students. The former is a way for academics and districts to convene research projects aimed at answering common questions. Regional Education Laboratories and some foundations provide financial support to these partnerships.

The federal government has also had grant programs that provide similar support. And doctoral students are golden. Harvard has a doctoral fellowship that supports partnerships with districts and states to understand the effects of new policies and programs.

Q: In one of your Education Week columns, you argued that many grade-level team meetings set to analyze student data focused on noninstructional reasons for students’ lack of progress. What would it take to shift the focus of data conversations to instructional strategies, and would you expect to see a bigger impact on teaching and learning outcomes?
A: I’m a big believer in the power of routines. Effective routines can focus attention and energy on what matters — and make difficult conversations much easier — because everyone knows what to expect.

I’d advise educators to design routines that focus on not just what content students don’t know, but also on examining curriculum materials, so that teachers can compare students’ unfinished learning to what’s present in the materials and the way that content was taught.

Teachers may discuss the ways that the idea the child missed was or wasn’t covered during instruction and hypothesize about ways to enhance that content within the parameters of the curriculum materials. If teachers feel shaky on the content of the lessons, routines can support reviewing that content or practicing how to teach it.

These routines also should be flexible enough to lead grade-level teams to other problems in classrooms, such as students who have disengaged from instruction or classroom management problems that take away from learning time. From my recent experience watching mathematics instruction, I would bet that student disengagement is a major driver of unfinished learning.

Q: What does research tell us about online professional learning, such as online coaching, that we can use to help educators in this time?
A: My work with Kathleen Lynch showed that professional learning programs that contained an online element were not as effective as professional learning programs that were entirely face-to-face. However, with COVID, online is part of the new normal.

We’ve run online math coaching (MQI Coaching) for years, and have found it to be an effective way to change instruction. One silver lining of this pandemic is that online coaches won’t need to ship cameras to teachers this fall — they can easily peek in on instruction that occurs over Zoom or Google Hangouts. MQI Coaching offers a set of guiding principles for online coaching.

Q: What have you learned from the response to your Education Week column? Has it changed how you think about the way educators use research and what they need from research?
A: I have studied research use by practitioners, so nothing surprises me. I guess my learning this year has been that teachers and school leaders are faced with an impossible job — trying to reconcile what research says with more local evidence about what works and doesn’t in their own practice.

A colleague, Lauren Yoshizawa, observed as teachers and coaches tried to process research recommendations, for instance, for reciprocal teaching all the while knowing that it had been tried and hadn’t worked in their classrooms due to problems with implementation.

There are also certainly schools for which the study of student data has worked to improve outcomes. These findings raise interesting questions about whether research or local knowledge should be prioritized in funding programs and federal legislation.

Principles for online coaching

Access MQI Coaching’s guiding principles for online coaching at: mqicoaching.cepr.harvard.edu/files/mqi-coaching/files/mqi_covid_resource.pdf