In the spirit of self-reflection for the purposes of improvement, Learning Forward often engages with evaluation partners to look closely at its own efforts to strengthen professional learning through networks and communities of practice. A recent independent evaluation by WestEd of Learning Forward’s What Matters Now Network looked at the ways in which a structured, facilitated network provided support for state-level coalitions working with districts, schools, and educators to improve instruction and make progress toward achieving content standards.

The What Matters Now Network was based on a research report by the National Commission on Teaching and America’s Future (NCTAF, 2016) that explored the conditions needed to support great teaching and learning. The network was facilitated by Learning Forward, used and taught the principles of improvement science, and was funded by the Carnegie Corporation of New York and the William and Flora Hewlett Foundation.

The network operated from 2016 to 2021 and included multirole coalitions from three states: Maryland, Ohio, and Rhode Island. WestEd, an independent research firm, conducted a three-year evaluation of the What Matters Now Network to examine the impact of the network’s design and facilitation as well as the results of data-informed improvement cycles focused on increasing teacher and student learning.

Methodology

WestEd grounded its evaluation in the network’s driver diagram, which identified intermediate and long-term aims as well as the drivers that would move the coalitions to improved teacher and student outcomes: effective job-embedded professional development; identification and implementation of high-quality curriculum and instructional materials; and leadership, policy, and resources for sustaining and scaling continuous improvement in these areas.

WestEd collected and synthesized information to assess the impact of the network on professional learning and teacher practice as well as successes and challenges related to three levels: network support, progress against goals or aims, and participant results. The evaluation team observed and gathered data at the convenings, conducted role-alike focus groups, surveyed participants, and looked at artifacts from teacher collaborative meetings and student work.

WestEd researchers also conducted two rounds of interviews with network members, including state department of education staff and district and school administrators and teachers. In addition to questions related to the specifics of the project (such as about the use of improvement science strategies and the structure of the state coalitions), some of the main research questions included:

To what extent and in what ways does the network leadership foster shared purpose, collaboration, capacity-building, and common language and inquiry approaches?

To what extent and in what ways is the network progressing toward its aim of teachers accessing and engaging in effective job-embedded professional learning grounded in the use of high-quality curriculum and instructional materials?

What are the successes and challenges encountered by state coalitions in pursuing their aims?

The evaluation also looked at the goals and outcomes for each state, within that state’s area of focus: Maryland focused on increasing teacher capacity to identify and implement Next Generation Science Standards-aligned professional learning and instructional resources; Ohio focused on using collaborative learning teams to strengthen teacher practices for pre-K-3 literacy outcomes; and Rhode Island focused on increasing teacher engagement in examining student data and reflective instructional practice to meet identified student needs.

Because the network operations were impacted by the COVID-19 pandemic and school closings, additional questions were added to interview protocols in winter 2021 to gather information about the impact of the COVID-19 pandemic on the group’s work in the states.

Findings

The evaluation found that the participants valued the network’s design and facilitation and that the focused, data-informed improvement cycles increased teacher and student learning. Engaging in the coalition and network activities resulted in more focused and productive discussions among educators, as well as more teacher engagement in the processes of implementing the curriculum.

The evaluation also found that educators’ capacity to effectively use student data increased, and classroom practice and student outcomes improved. Educators used collaborative planning time more effectively and targeted interventions more effectively than before the network engagement. According to one participant, “Teacher professional learning time has moved beyond compliance to become efficient and purposeful — focused on looking at data, reflecting on practice, and observing and supporting peers.”

Coalitions were able to identify and institute new professional learning strategies in response to the needs surfaced in the collaborative discussions. For instance, one coalition began principal walk-throughs to observe whether professional learning was addressing the priorities identified by teachers’ analyses, while another coalition began peer observations and supportive discussions about how to discuss feedback with fellow educators.

Participating in the network had a positive impact on educators, in general, according to the evaluation findings. One principal reported, “Teacher practice is more intentional and purposeful. I see them reflecting and discussing, ‘How do we make this fit the need of this kid and also this kid?’ It is way more explicit and systematic than it has been in the past.”

Engaging in the network and coalition activities also improved teachers’ knowledge and skills, including how to identify what lessons and materials were aligned to content standards and on grade level, and provided opportunities to ensure consistency of content across school teams. A new teacher from the Maryland coalition noted, “It really helps a lot because I had never heard of the Next Generation Science Standards. Now I know how to do data analysis to find out how to fill gaps for children.”

The final, post-COVID pandemic shutdown interviews found that the structures, protocols, and relationships developed during the What Matters Now Network supported educators during the sudden transition to online learning. According to one participant from a state department of education, “Districts are collaborating and having PLCs, and data is rolling in like we’re not in a pandemic.”

Implications

The What Matters Now Network was an initiative that incorporated the Standards for Professional Learning as a framework alongside the principles of improvement science. The careful, collaborative analysis of student and educator data informed the tailoring of professional learning content and pacing, demonstrating some of the key elements of the Learning Designs standard. In addition, the way in which participants and stakeholders advocated for and provided resources for the support teachers said they needed is a strong example of the Implementation standard.

The What Matters Now Network has concluded, but the network’s structure, protocols, and evaluation design continue to be helpful to other Learning Forward networks and communities. There is also the potential to develop a toolkit to leverage the tools developed and tested by the three coalitions, since they successfully strengthened professional learning communities and curriculum-based professional learning.