The debate about how to measure and quantify the impact of professional learning on teachers and students is often peppered with statements about how we don’t know what works. In fact, many studies that examine a specific strategy or program, often as a component of a larger professional learning system, find measurable positive effects. These studies are important to educators, professional learning advocates, and the field as a whole.
Understanding both the specificity and the generalizability of such studies helps build an awareness that professional learning is critical and has impact, especially when we carefully consider which strategies work for which educators and students under what conditions.
Learning Forward is working to bring precision to conversations about the outcomes of professional learning, especially by examining the specific components that drive outcomes. You can see this focus in our publications, the networks we facilitate focused on educator-identified outcome measures, and our efforts to compile and share data from districts and affiliates.
While large-scale randomized controlled trial studies are critical to understanding professional learning and its impacts, studies that are small, qualitative, and focused on teacher responses are equally as important to drive this goal of precision and, in turn, inform decisions about funding and support for professional learning.
A recent study by Debra McKeown and colleagues is important not only because it highlights professional learning about a model of writing instruction of interest to many educators, but also because it examines teachers’ reflections about their experiences in professional learning.
Building on a larger quantitative study, this qualitative study elicited teachers’ insights about the elements of the professional learning they found beneficial for instruction and student outcomes.
A professional learning approach to improve writing
McKeown and colleagues looked at how 2nd- and 3rd-grade teachers in three rural schools experienced and responded to professional development related to a writing instruction method called self-regulated strategy development (SRSD).
SRSD is a complex instructional approach that includes “active, discourse-based, scaffolded, and explicit learning of: strategies for genre-specific and general writing employed across the writing process” (p. 757). SRSD instruction includes as core principles engaging students as active collaborators, attention to social and emotional learning, and a commitment to teacher adaptations.
The SRSD strategy is well-researched and recognized by the What Works Clearinghouse as an evidence-based practice. More than 100 studies by multiple research teams have found that SRSD achieves significantly higher effect sizes than other instructional approaches in writing.
The current qualitative study built on a larger mixed-method research study on the impact of SRSD professional learning on teacher practice and student outcomes. In that study, teacher teams were randomly assigned to professional development in one of two writing areas selected by the educators to align with local and state goals: opinion essay writing or story writing.
Each option formed the control group for the other, because previous studies have shown that the story writing professional learning does not impact essay writing and vice versa. This allowed the researchers to use a randomized controlled trial without withholding professional learning from any teachers.
Teachers engaged in 12-14 hours of professional learning over two days about one week apart, as well as readings, outside preparation, and collaborative discussion. The professional learning incorporated characteristics supported by the research about the elements of effective professional learning — intensive, collaborative, focused on outcomes, and aligned to the curriculum — which align with the Learning Forward’s Standards for Professional Learning (Learning Forward, 2011).
The researchers noted that they intentionally mirrored the stages of SRSD in their professional development with these teachers: Develop background knowledge, discuss it, model it, support it, and demonstrate independent performance.
At the beginning of the professional learning, teachers described their previous experiences in writing instruction, professional learning, and preservice preparation, as well as their goals for students and current challenges, using student work examples. Teachers then reviewed SRSD techniques and sample lesson plans, then the professional learning leaders modeled lessons.
Teachers created lesson plans based on their classroom contexts and student needs and practiced instructional techniques in pairs. They reflected and received feedback from their peers and the professional learning leaders. They discussed components of the SRSD approach, such as student self-assessment, and how to adapt lessons to specific student needs. Teachers also made plans for regular peer meetings to reflect on implementation and challenges.
Observers visited classrooms after every three or four lessons to provide feedback and support to teachers and measure implementation fidelity. In addition, the professional learning leaders offered ongoing informal support and sent weekly emails sharing ideas and observations from their research team meetings.
Going beneath the surface with focus groups
In a previously published quantitative report, researchers identified that the SRSD and associated professional learning resulted in significant and positive outcomes for students in both the opinion essay and story writing conditions.
Implementation of SRSD specifically related to opinion essay writing resulted in significant and meaningful improvements in the number and quality of opinion essay elements as well as in student writing overall. Students who received whole-class, teacher-implemented SRSD instruction in story writing demonstrated significant and meaningful improvements in elements and quality of story writing. In both groups, the intended outcomes were achieved.
The sub-study conducted by McKeown and colleagues built on these findings and mined teachers’ reflections to gain further insight about both teachers’ and students’ experiences. McKeown and colleagues conducted and recorded focus groups with 14 of the 20 participating teachers, then coded and analyzed the themes, using well-established methods for inter-rater agreement.
During the focus groups, teachers discussed their evaluations of the components and adequacy of the professional learning, how the professional learning might be enhanced in the future, how they implemented SRSD, changes or adaptations they made to meet students’ needs, changes they would make in the future, and how they believed the professional learning could be improved.
Teachers’ observations and reflections augmented the quantitative findings about improved student outcomes. Teachers were able to observe specific student behaviors in ways that the quantitative study did not allow. Teachers identified that students became more independent writers, better able to organize ideas to leverage in opinion writing and persuasive writing, more able to write to real-world situations, in part because they became more comfortable with how to start an assignment and formulate arguments.
Teachers also observed increased confidence in writing for some students. Students were quicker getting to task and enjoyed writing more than before the implementation. One teacher said, “I had one (student) who wrote zero in the beginning and wrote a persuasive paragraph in the end. He grew a lot and he was able to write and stay focused and keep on track, check all his parts, had his rocket (graphic organizer), and the whole bit” (p. 778).
Teachers identified a mix of social, emotional, and academic outcomes. There was consensus among teachers that this strategy “allowed kids to grab on to something that had been missing in other kinds of writing instruction” (p. 780).
Professional learning components
Teachers also reflected on the components of professional learning and their impact on their teaching and their students. Teacher data provided important insights about perceptions, evaluations, recommendations about the professional learning, and suggestions for improvement.
Overall, teachers said they were happy with the SRSD professional learning and offered specific reflections about what worked and what did not:
- Active, interactive learning and practice were perceived as helpful and supportive.
- Cognitive modeling and using “self” statements were challenging.
- Teachers reflected about the ideal size of learning group — six to 10 — a small but important detail for professional learning design. Smaller groups helped teachers feel “safe” and “made us try.” Because of the size of the group, one teacher said, “I got to do it, not just say I saw someone do it.”
- Most teachers found the detailed model lesson plans to be an important learning tool and a helpful reference for planning instruction.
Teachers also shared feedback about the specific instructional aspects of SRSD that they learned. For example, many teachers and their students were initially challenged by the strategy of using “self-talk” about what to write and how, but grew more comfortable with it over time.
Balancing fidelity and adaptation
Fidelity of implementation was high throughout the study, but teachers had been encouraged to adapt the strategies they learned as needed by their students. Teachers recognized the need to adapt when students became disengaged, for instance, or did not have the expected background knowledge, but were concerned with fidelity to both the SRSD design and the research project parameters.
In addition, teachers reflected on specific tools and strategies such as mnemonic devices, graphic organizers, and model lesson plans. These detailed notes would be helpful for educators especially interested in the SRSD instructional model.
Generalizability and Connection to the Standards
This study speaks to several Standards for Professional Learning, including the Learning Designs standard. The way the teacher voice and reflections highlight which elements of the professional learning have a positive or challenging effect on students is critical information for the design of future professional learning.
For instance, teachers sharing that a strategy is not having the intended impact can lead to an adjustment or tweak to the overall approach, including classroom-tested strategies that offer a range of options for other educators. In addition, research like this provides detail and insights into Learning Forward’s focus on the intersection of high-quality instructional materials and professional learning.
These findings are a reminder that qualitative data can be simple and telling, revealing that materials are too time-consuming, or difficult for students to engage with for a long period of time. These are the details that can provide design tweak ideas and midcourse corrections.