Evaluating professional learning from the district\'s perspective
An interview with Dawn F. Wilson
Evaluating professional learning means much more today than it did 10-12 years ago. Back then, I knew how to plan and deliver professional development, but we weren\'t as focused on implementing and evaluating the impact at educator, school, and district levels. Today, evaluating professional learning is a focused, intentional process that we use to determine the value of the professional learning that is delivered.
We want to be responsive to certain basic intentions, such as did the learning achieve the intended results, were those results better than before, and was the method we used more effective than other learning designs? For example, we have implemented lesson study over the last several years. We compared it to traditional lesson planning, examined if the teachers were more effective in using data to plan lessons and assess impact on their practice and student learning. We also looked at the monetary implications, such as was the lesson study model worth the cost and allocation of resources. What value did lesson study add?
Measure at different levels
Our theory of action, or as I have recently started saying, our theory in action, is that quality professional learning can enhance educator knowledge and skills, instructional practice, and ultimately student performance. To measure the quality of that professional learning, we systematically collect and analyze data on educator knowledge, educator practice, and student achievement. We look at a variety of sources for our data. The use of multiple sources of data offers a balanced and more comprehensive analysis of student, educator, and system performance.
To evaluate our professional learning at the educator level, we analyze student data. These data give educators direction and help them understand where their students are, compare their progress to our standards, compare their practice to our professional standards, and identify their focus for their individual professional development plans. Student data can include formal and informal assessments, achievement data, state tests, benchmark data, course data, daily classroom work, classroom assessments, and other forms like demographics, attendance, student perceptions, and behavior or discipline data. These data are all important in painting a picture of who a child is and how educators can support that child. Educators know individual student needs and use this information to guide and inform instructional decisions.
We know that if there are changes at the student and educator level they are best sustained when school and system leveraging occur simultaneously. At the school level, they collect data, analyze that data, and determine what is needed. For example, if a teacher is participating in a change, such as lesson study, the principal expects to see evidence of that in lesson planning, data analysis, lesson delivery, and how the teacher assesses students.
At the school level, administrators observe in classrooms to see if teachers use student data to assess effectiveness of the application of the learning. When a teacher designs assessments or scoring guides and analyze the results themselves, they can get important information about the effect of their learning on students. Our intent is that through individual professional development plans, the teachers learn how to acquire knowledge and skills and pick areas of professional learning that are connected to student outcomes. Then teachers can measure their progress toward meeting their goals throughout the year.
At the district level, data analysis is important to us. Until just recently, we had a dedicated evaluator of professional learning on staff. According to state law, 10% of professional development programs must be formally evaluated, so she conducted formal program evaluations over several years - lesson studies, learning communities, our teacher induction program, reading competencies, our coaching model, etc. The results of these studies informed how we adjusted our professional learning. We looked at what worked well and what we needed to eliminate.
Challenges in evaluating professional learning
Shifting from district-delivered to school-based, job-embedded learning is a challenge. How can schools build internal capacities and resources when their educators are used to district-delivered learning? We have to consider whether or not systems and structures are in place to support that change. For example, are sufficient resources allocated to the school level to support the change?
Resource allocation is another challenge. Systems must invest in new technologies to give educators the evidence they need to discuss the impact of professional learning. The challenge is in linking data to improved teacher practice and student outcomes. A comprehensive, integrated, intuitive 24/7/365 system can give stakeholders the resources they need to manage continuous improvement. The system needs to support data collection and analysis, instructional planning and resources, and rapid reporting - all tied together in one easy-to-access system. My favorite analogy is shopping for data. I liken many systems to shopping in an old downtown square where shoppers might go in and out of several stores to get what they want. This is different from shopping at a mall, where there is a single point of entry and shoppers can get everything they need under one roof - much like a single sign-on for a technology system. Districts are data rich, but many systems haven\'t yet made the data connections so that educators can easily access multiple data resources in one central system. To get there, resources need to be allocated.
Three system-level actions
There are three things we can do at the system level to help with evaluating professional learning. First, we need to understand the Standards for Professional Learning and Innovation Configuration maps. The revised standards are comprehensive, and essential tools for evaluating professional learning. Connecting professional learning to outcomes is throughout the standards, but is really evident in the Implementation, Data, and Outcomes standards. Data has assessing progress and evaluation. Implementation emphasizes the support required to implement professional learning, and the Outcomes standard connects adult and student learning. The Innovation Configuration maps paint a picture of what the standards look like in practice. These two tools are foundational for assessing professional learning in districts.
Second, districts are wise to develop professional learning plans. Our district plan is a repository of what is essential for effective professional learning. You can see our current professional development plan on our website. The plan includes references to three research studies that every district needs to inform their evaluation model. Thomas Guskey\'s Five Levels of Professional Development Evaluation, Joyce and Showers\' Professional Development Components and Attainment of Outcomes chart, and Darling-Hammond\'s report Learning in the Learning Profession Professional Learning: A Status Report on Teacher Development in the U.S. and Abroad. These three studies can guide districts because they clarify what works in professional development and provide guidance for program evaluation. A district plan gives coherence and clarity to professional development system keeps the focus on results. The plan becomes our district\'s \"Rosetta Stone\" for professional learning.
Third, systems can build and share a professional learning evaluation plan so decision makers can make informed judgments about the impact of professional learning on teaching and learning. One of the best models for this is Joellen Killion\'s workbook, Comprehensive Professional Learning System: A Workbook for States and Districts. This workbook offers information and tools to walk educators through eight planning steps, from data analysis to setting goals to identifying learning designs to monitoring impact.
Dawn F. Wilson (WilsonD@duvalschools.org) is a Learning Forward 2010 Academy graduate, coach of Learning Forward Academy 2015 class, and director of professional development for Duval County Public Schools (Fla.).