Professional learning is vital to address the multiple issues facing educators today. Recovering from declines in student achievement brought about by the COVID-19 pandemic requires educators to prioritize learning goals and develop new ways to accelerate students’ learning progress. Political discord, social tensions, and global conflicts compel teachers to guide students in exploring diverse perspectives and engaging in constructive, respectful dialogues. Furthermore, the rapid evolution of technology, particularly artificial intelligence (AI), necessitates teachers and school leaders to consider entirely new paradigms in instructional methods and assessment.

At the same time, there is a great need for credible evidence on the effectiveness of professional learning. Rachel Garrett and her colleagues (2021) conducted an extensive meta-analysis of high-quality studies of teacher professional learning published between 2010 and 2020. The aim was to establish connections between features outlined in Learning Forward’s Standards for Professional Learning (Learning Forward, 2022) and teacher instruction as well as student learning.

To be eligible, studies needed to “(1) include teachers in Grades K-12, (2) examine the impacts of professional learning on instruction measured through classroom observation, and (3) have enough information to compute effect sizes” (p. 5). Among the hundreds of studies identified, only 48 met these modest criteria for inclusion.

This combination of a significant demand for professional learning and a paucity of credible evidence yields two critical implications. First, it highlights the pressing need for enhanced methods of collecting sound evidence on the impact of professional learning. While many professional learning initiatives may indeed have a positive influence on teaching and student learning outcomes, there has been a lack of concerted effort to adequately document these effects. Second, it underscores the necessity of using trustworthy evidence to elevate the quality of professional learning experiences for all educators. This entails more systematic and purposeful evaluations of professional learning initiatives.

A LONG HISTORY

These two implications have long been recognized. I recall discussions with Dennis Sparks, Stephanie Hirsh, Shirley Hord, and Susan Loucks-Horsley during my initial involvement with Learning Forward, then called the National Staff Development Council. I had just graduated with a degree in educational measurement and evaluation, and these influential leaders sought my advice on developing new and better methods for gathering evidence on the effects of professional learning.

Initially, I proposed a model developed by Donald Kirkpatrick in 1959 for evaluating training programs in business and industry. Kirkpatrick’s four-level model included trainees’ reaction (initial satisfaction), learning (knowledge, skills, and attitudes), behavior (on-the-job performance), and outcomes (increased productivity). However, early attempts to apply the model to professional learning in education were challenging and proved unsuccessful. It was apparent that something unique to the field of education was missing.

Analyses of the lack of success in applying Kirkpatrick’s model of evaluation consistently pointed to organizational factors. Professional learning leaders were successful from a training perspective, but challenges arose when participants returned to organizations that either did not sufficiently support implementation of new ideas or posed barriers to implementation.

In response to these organizational challenges, I adapted Kirkpatrick’s model to create a five-level model that included organization support and change. This modification recognized the crucial role of organizational support in the success of professional learning initiatives. By incorporating this additional level, the model aimed to address the unique dynamics of educational settings and the importance of aligning organizational structures and culture with the goals of professional learning.

A MODEL FOR EVALUATING PROFESSIONAL LEARNING

The new evaluation model I proposed presents a comprehensive framework for assessing the effectiveness and impact of professional learning initiatives in education. It can be applied to event-driven activities like workshops and seminars, as well as to the wide range of ongoing, job-embedded professional learning activities such as study groups, action research, collaborative planning, curriculum development, structured observations, peer coaching, mentoring, and so on.

The model provides a systematic approach to evaluation, emphasizing the importance of not only measuring participants’ reactions, but also delving deeper into the outcomes and changes in teacher practices and student learning.

The model comprises five levels, each focusing on a specific aspect of evaluation. At the first level, participants’ reactions is gauged by collecting feedback from educators who have experienced the professional learning. This initial step acknowledges the importance of participant perceptions and engagement and recognizes that satisfaction contributes to the overall success of the effort.

The second level explores participant learning, assessing the knowledge and skills acquired by participants during professional learning. This emphasizes that professional learning is a purposeful endeavor with intended learning goals for participants. Evidence gathered at this level must be aligned with the activity’s learning goals to offer meaningful feedback on areas of strength and shortcomings, facilitating continuous enhancement of professional learning.

Moving beyond individual participants, the third level considers organization support and change. This level concentrates on the conditions and resources necessary for high-quality implementation. A key element at this level is strong leadership that supports individuals in navigating the complexities of change. A culture of collaboration and open communication is paramount.

In addition, adequate resources and infrastructure coherence are crucial for success. This includes the provision of adequate time, access to current teaching materials, technology, and ongoing follow-up opportunities. When educators have the tools, resources, and support they need, it empowers them to confidently embrace and implement new practices.

The fourth level, participant use of new knowledge and skills, emphasizes the practical application of newly acquired knowledge and skills within school and classroom settings. This level recognizes that successful implementation goes beyond mere acquisition and understanding; it involves the effective use of new strategies and practices.

The concept of “mutual adaptation” (McLaughlin, 1976) describes the dynamic nature of this process. Participants are not only expected to adjust to the use of new strategies but also to adapt these innovations to suit their unique educational settings, highlighting the need for a flexible and context-sensitive approach. This level is crucial for understanding the practical application and relevance of the professional learning, bridging the gap between theory and practice.

The fifth level focuses on the ultimate goal of professional learning activities: impact on student learning outcomes. This level addresses critical questions about the effectiveness of the changes in practices on students. Did these changes lead to improvements in students’ learning? Did they influence students’ attitudes, dispositions, or behaviors? The assessment of student learning outcomes is tailored to the specific goals of the professional learning, reflecting a forward-looking perspective that underscores the importance of achieving lasting and transformative change in education.

The fifth level also recognizes that different stakeholders trust different forms of evidence. Because district and school leaders tend to rely on large-scale standardized measures, while teachers put more trust in classroom assessments and observations (Guskey, 2007), diverse measures of student learning are essential to comprehensively evaluate impact. In addition, multiple sources of evidence help capture the potential for unintended outcomes.

For instance, suppose a group of elementary educators organizes a study group to find ways to enhance students’ writing skills and sees a significant increase in writing scores. A deeper analysis reveals, however, that over the same time, students’ math scores declined, potentially resulting from a reallocation of instructional time. This shows the need for a holistic approach to assessing student outcomes, ensuring that possible unintended consequences are identified and addressed.

The insights gained from this level serve as a crucial guide for refining all aspects of professional learning, including activity design, implementation, and follow-up. By linking changes in practice to tangible improvements in student learning, educators can continually refine their approaches to maximize positive outcomes. Additionally, evidence on student learning outcomes provides a basis for estimating the comparative cost-effectiveness of professional learning initiatives.

The figure on p. 31 describes these five levels in detail. Focusing on these five levels reflects a commitment to evaluating the pragmatic impact of professional learning, acknowledging the complexities of the process, and promoting a continuous cycle of improvement in educational practices.

 IMPLICATIONS OF THE MODEL

Since initially described in the 1998 article “The age of our accountability” (Guskey, 1998), this model has emerged as a foundational framework for evaluating professional learning worldwide. Evaluating Professional Development (Guskey, 2000), which explains in detail the model’s five levels, has been cited in over 6,200 scholarly articles and served as the foundation for more than 40 doctoral dissertations (e.g., Newman, 2010; Ross, 2010). This widespread use demonstrates the model’s relevance and applicability across diverse educational contexts. Crucial to this work are three primary implications that emanate from the model.

1. Each level is important.

The model emphasizes that each of the five levels is crucial in evaluating educators’ professional learning experiences, and no level can be neglected. Each level represents a different stage or aspect of the professional learning process, and each requires a different form of evidence. Taken together, these five levels represent a holistic approach to evaluation, considering the various facets and stages of the professional learning process.

2. Each level builds on the levels that come before.

While achieving success at a lower level is often a necessary foundation for success at a higher level, it does not guarantee it. Breakdowns in the effectiveness of professional learning can occur at any point along the way. And once it breaks down, progress ends. This emphasizes the need for a comprehensive understanding of the entire process and the potential challenges that may arise during the transition from one level to the next.

3. When planning professional learning, the order of levels must be reversed.

The third and perhaps the most crucial implication is the importance of a reversed approach to planning. Instead of planning forward from the initial stages of professional learning to student learning outcomes, effective planning is “backward” (Guskey, 2001). It means starting with the desired end goal of improvements in student learning, then working backward to determine the necessary steps and components at each preceding level. This deliberate approach aims to ensure that the entire professional learning process is aligned with the primary goal of improving student learning outcomes.

Backward planning starts by clearly defining the desired student learning outcomes and deciding what evidence best reflects those outcomes (level 5). This could include goals such as improving reading comprehension, enhancing problem-solving skills, boosting confidence in learning situations, or fostering better collaboration among classmates. The identification of these outcomes is informed by critical analyses of relevant data from classroom and large-scale assessments, examples of student work, and school records.

Once the desired student learning outcomes are determined and evidence sources established, determine the instructional practices and policies that are supported by pertinent research evidence to achieve those outcomes (level 4). Ask critical questions about this evidence, considering factors such as its reliability, applicability to specific contexts, and whether it is research-based rather than simply opinion-based. Be especially cautious about adopting popular trends without solid research backing.

Next, consider what aspects of organizational support need to be in place for those practices and policies to be implemented (level 3). Active engagement and support from school leaders will be vital for success. Sufficient planning time and necessary resources must be in place. In addition, aspects of the organization that pose barriers to implementation must be identified and revised. Certain policies regarding student discipline and grading, for example, may determine teachers’ options in dealing with students’ behavioral or learning problems. A big part of planning involves ensuring that organizational elements are in place to support the desired practices and policies.

Then, decide what knowledge and skills the participating professionals must have to implement the prescribed practices and policies (level 2). What must they know and be able to do to successfully adapt the innovation to their specific context and bring about the sought-after change?

Finally, consider what set of experiences will enable participants to acquire the needed knowledge and skills (level 1). Workshops and seminars, especially when paired with collaborative planning and structured opportunities for practice with feedback, action research projects, organized study groups, and a wide range of other professional learning strategies, can all be effective, depending on the specified purpose of the professional learning initiative.

What makes this backward planning process so important is that the decisions made at each level are interconnected. For instance, the choice of specific student learning outcomes directly influences the selection of instructional practices and policies. Similarly, the chosen practices and policies have implications for the required organizational support or changes. This interconnectedness in the backward planning process shows that 90% of the crucial questions in any professional learning evaluation are addressed before any activities begin. It also emphasizes the need for a holistic and purposeful approach to planning.

The complexity of professional learning is further compounded by its context-specific nature. What works effectively in one context with a specific community of educators and students may not yield the same results in a different setting with diverse participants. This recognition of context-specific challenges highlights the difficulty in developing comprehensive best practices that can be applied universally across various educational settings.

The backward planning process is essential because it recognizes the interconnectedness of decisions at different levels and navigates the challenges posed by the context-specific nature of professional learning. The dynamic interplay between student learning outcomes, instructional practices, organizational support, and the unique context underscores the need for a tailored and strategic approach to professional learning planning and evaluation.

SHIFTING TOWARD PURPOSEFUL EVALUATION

Traditionally, evaluating professional learning has not been a priority for educators. Many view evaluation as a cumbersome and resource-intensive process that diverts attention from more immediate tasks such as planning, implementation, and follow-up. Others believe that they lack the requisite skills and expertise to engage meaningfully in evaluations.

Contrary to these reservations, effective evaluations don’t need to be complex or resource-draining. They simply require thoughtful planning, the ability to pose pertinent questions, and a fundamental understanding of how to obtain valid answers. When approached purposefully and strategically, evaluations yield meaningful information that can be instrumental in making informed and responsible decisions about professional learning processes and their effects.

In addition, the shift toward more purposeful evaluations aligns with the evolving landscape of education. As the demand for accountability and evidence-based practices grows, educators are recognizing the need to assess the impact and efficacy of professional learning. This shift in mindset involves viewing evaluation not as an isolated, burdensome task, but as an integral part of the continuous improvement cycle within education.

While there may be challenges associated with evaluating professional learning efforts, the benefits far outweigh the perceived drawbacks. A mindset shift toward embracing evaluation as a valuable tool for improvement can empower educators to make informed decisions, enhance the quality of professional learning experiences, and ultimately contribute to better outcomes for both teachers and students.