March 2014


Evaluating professional learning: Measuring educator and student outcomes


image of Evaluation Chart




When high-stakes changes put education in the spotlight, stakeholders want to see evidence that their investments of time, money, and effort are not wasted. Educators want to know that the professional learning tied to implementing new initiatives is effective; system and state leaders want evidence that their planning and funding are producing results; and community members want to know that educators are spending their time and tax dollars wisely and, most importantly, achieving results for students. Evaluating professional learning can satisfy these needs and help educators make the connection between professional learning and better teaching.

Not all evaluation of professional learning requires rigorous, academic evaluation. On the contrary, when professional learning is focused on changing educator practice and improving student outcomes, all educators have the ability — and the responsibility — to gather evidence and participate in the evaluation of professional learning as part of a system of continuous improvement.

 

     

Outputs are not outcomes

It can be easy to confuse outputs and outcomes.

Outputs can be a learning plan or data about which teachers attended sessions. These do not indicate whether educators accomplished their goals.

 

Achieving outcomes indicates progress or success in achieving goals, such as changing educator practice and improving student achievement.


What to measure

Because the goals of standards-based professional learning are to change educator practice and improve student learning, those are the outcomes to measure in an evaluation. The two are tied together. Changes in educator practice that do not lead to better learning for students aren’t contributing to the ultimate goal. Conversely, changes in student learning without clearly defined changes in educator practice offer no evidence of a link to professional learning. 

 

Educators must avoid the urge to measure attendee perceptions of learning elements unrelated to outcomes, such as their impressions of the presenter, the comfort of the room, or the quality of the snacks. When educators stay focused on educator and student learning benchmarks, they gather solid evidence to understand the value of their professional learning.  

 

Practitioner responsibility

Because effective professional learning improves educator practice and is fundamental to student learning, all educators have an obligation to seek out and participate in effective professional learning. However, educators often leave the responsibility for evaluating that learning to others. The cyclical nature of effective learning systems – beginning with goals based on data, continuing with careful planning, and including constant monitoring --  means that educators at all levels have the ability to assess the impact of professional learning on their practice and student results.

 

Evaluation steps

The data or evidence to use for measuring the impact of professional learning can vary according to context, such as school or system goals, measurement tools available, and content. However, effective professional learning systems share the core elements needed for effective evaluation no matter what the context. Learning Forward offers the following seven steps for tapping into those core elements to evaluate professional learning. Ideally educator teams will work together through these steps as part of overall continuous improvement processes.


Step 1: Analyze student, educator, and system data.

Where are students and educators in relation to school or district goals?

 

Learning needs are tied to school improvement goals. Whether schools are focused on implementing content standards, closing achievement gaps, or serving all students equitably, these improvement goals serve as a starting point to determine what changes are needed.

 

Learning Forward belief: All educators have an obligation to improve their practice.

As outlined in the Data standard of the Standards for Professional Learning, educators determine what student, educator, and system data are available to guide the planning, implementation, and evaluation of professional learning. Student data include informal and formal assessments, daily classroom work and assessments, and annual and end-of-course exams. Educator data include classroom performance, student results, and data from individual evaluations. System data include information about what resources are available and how they are allocated and relevant system policies and procedures.

Learning how to analyze data is a critical skill for teachers, school leaders, and system leaders. Teachers and school leaders need ongoing opportunities to examine data formally and informally, and in formative and summative forms.

 

Step 2: Set learning goals for students and educators

Based on the data from Step 1, what do students and educators need to learn?

 

Only after educators understand student needs can they determine their own learning needs. With specific student gaps in mind, educators identify what knowledge and skills they need to be able to help students improve. Using a goal- setting process that allows educators to pinpoint specific student learning targets is critical so they can both monitor progress and identify the specific learning needs they have.

 

For example, educators might establish a SMART goal (Specific, Measurable, Attainable, Results-based, Time-bound) that outlines a higher level of achievement in one mathematical concept for third graders by the end of the year. Such a goal would help them focus on the related content and pedagogical knowledge and skills they need as teachers to help students progress.

 

Step 3: Establish benchmark indicators

What changes in knowledge and skills, practices, and student learning will indicate progress toward the learning goals from Step 2?

 

Educators identify specific changes in practice and student performance to look for in moving toward attaining goals. These might represent phases of learning or steps of implementation. Educators consider both quantitative and qualitative data from multiple sources for a comprehensive look at student and educator progress.

 

For example, educators might state that they will identify and use two new instructional strategies within three months time. This benchmark builds on their own learning with the intention of leading to better understanding for students.

 

Step 4: Establish and implement a learning plan

How will educators and teams move toward the goals identified in Step 2?

 

In the cycle of continuous improvement, educators select learning designs that match their content and pedagogical needs and schools and systems ensure ongoing learning support for individuals and learning teams.

 

Step 5: Conduct formative evaluations

How will educators gauge progress and make adjustments?

 

Based on the benchmark indicators from Step 3, educators periodically assess progress towards learning goals and make changes as needed. For example, their supervisors might use walk-throughs to answer questions about new practices, and educators might look at informal assessment data weekly to gauge student progress. Overall, they look for evidence of change in knowledge and skills, practices, and student learning to create a body of evidence to formulate and support claims about impact.

 

At this stage, educators and teams make adjustments based on the progress they see – or don’t see. If their practices didn’t change as intended, they identify what knowledge and skills they need to improve. If their practices changed but students didn’t advance, they adjust accordingly.

 

Step 6: Conduct a summative evaluation

Has the learning led to achievement of student and adult learning goals?

 

At the end of the professional learning initiative, or at predetermined phases, educators gather evidence to determine whether or not students and educators met learning and performance goals. Educators have already set the goals they intended to reach and identified how to get there. At this stage, they gather the data that show what worked and what didn’t.

 

Step 7: Formulate conclusions about successes and areas for further improvement

What can educators learn from the evidence they gathered in Step 6, and what improvements can they make?

 

This step requires deliberate analysis and reflection to determine the success of the professional learning and what changes to make in the future to improve results.

 

The learning goals determine the questions that guide the evaluation of professional learning. Questions to consider are:

  • Did the professional learning lead to achievement of the goals established at the beginning?
  • Were there other factors that the team didn’t anticipate?
  • What is the overall effectiveness of the professional learning?
  • What changes will improve the results?
  • Will the school or system expand, continue, or stop the professional learning?

 

Not all evaluations of professional learning have to involve education researchers or additional expenses. The recursive and embedded nature of effective professional learning that is focused on student learning and educator practice means that educators at all levels can participate in the evaluation process.