Professional learning research is essential for helping education leaders make informed, strategic decisions about how to build staff members’ knowledge and skills. That’s why applying data and evidence is a theme throughout Standards for Professional Learning and an operating principle for all of Learning Forward’s work.

One of the first steps in this process of data-informed decision-making is knowing the current state of a system’s professional learning. Understanding the way teachers experience professional learning and what impact professional learning is having (or not having) can shape the approaches that systems start, stop, and continue and can increase the likelihood that future strategies and programs achieve their intended outcomes.

Learning Forward has developed a tool to provide this relevant, educator-level data. The Standards Assessment Inventory (SAI) is designed to help systems of all kinds — states, districts, schools, provinces, and organizations — gather and track data about the professional learning their educators experience. The framework for the inventory, which is administered in survey format, is the evidence-based set of 11 Standards for Professional Learning, which describe the conditions, processes, and content of high-quality, systemic professional learning that are theorized to lead to improvements for educators and students.

Realizing the promise of the evidence-based concepts and strategies outlined in the standards starts with assessing how well the professional learning educators experience is aligned to those standards. Then, bringing the school or system’s professional learning closer to the full realization of the standards will improve educators’ knowledge, skills, and capacity, ultimately resulting in better outcomes for students (Garrett et al., 2021).

How the SAI measures professional learning 

The SAI is a confidential, 30-minute, web-based instrument for teachers, coaches, and other professional learning stakeholders that helps systems assess the quality of the professional learning their teachers experience. Familiarity with the standards isn’t required or assumed.

The first iteration of the SAI was created in 2003. Learning Forward recently created a new and significantly improved version to align with the latest version of the standards, which were released in 2022. The new version also includes more detailed data and an easier-to-navigate platform for reviewing the results.

In the new SAI, educators respond to 74 items, choosing from five rating categories that range from “never” to “always.” Respondents also can answer “don’t know.” Following are some selected illustrative items:

• Professional learning in my school builds my capacity to implement my curriculum with integrity.
• I have the opportunity to build discipline-specific content knowledge and expertise through professional learning at my school.
• The professional learning I experience at my school is relevant to my work.
• Professional learning at my school includes conversations about how cultural and historical barriers can impact student learning.
• Teachers in my school receive ongoing support (e.g., coaching, co-teaching, peer feedback) in various ways to improve teaching.
• My school system has structures and procedures that support collaborative educator learning.

Respondents’ privacy is protected, as SAI data is anonymized and aggregated in all reports to the system or school administrator. School and system leaders use the SAI data to better understand their educators’ collective experiences and whether there is a gap between their vision and strategies for professional learning and what educators experience. If the SAI is administered pre- and post-initiative or at the beginning and end of an academic year, administrators can look across data to see if any changes have taken place.

How the new SAI was developed and tested

The SAI itself embodies several important themes that run throughout the standards because it supports a culture of continuous improvement and elevates the voices and true experiences of educators. The items underwent extensive internal and external reviews and a rigorous external study to ensure that the SAI yields data that accurately shows how well teachers’ professional learning is aligned with the evidence-based content in the 2022 standards.

The updating process began with a close review of the previous version of the SAI. Learning Forward interviewed professional learning experts who have administered the SAI and analyzed data over many years with multiple systems to get their feedback about items, reflections on how well the data represents systems’ professional learning, and insights on how they use the results. Learning Forward revised the SAI items, in collaboration with professional learning content experts, during 2021-22 to align with the revised Standards for Professional Learning released in April 2022.

The American Institutes for Research (AIR) then partnered with Learning Forward to conduct a validity and reliability study to ensure the items measure the standards as intended. Learning Forward administered the revised SAI to a sample of volunteer participants, consisting of 259 teachers and instructional coaches in 166 schools from 148 districts, and AIR analyzed the data.

Learning Forward recruited the participants from their professional networks. The participants came from diverse professional backgrounds with varying roles, years of experience, years at their current school, and school settings. Most of the respondents had never completed the SAI previously (71%) or were unsure about whether they had (23%). The majority were female (86%) and non-Hispanic (95%). Sixty-four percent were white, 26% were Black or African American, 3% were Asian, and 7% identified as other races. Participants had high levels of education experience, with 86% having five or more years of experience, but nearly half (47%) had spent four or fewer years at their current school. Most respondents were either content-area teachers (45%) or instructional coaches (35%), and they worked in elementary school (56%), middle school (28%), or high school (29%).

Based on the data collected by Learning Forward, AIR conducted psychometric analyses to determine whether the revised SAI items map onto the standards as intended, examine measurement quality of the revised SAI, and examine item consistency. In addition, AIR performed multiple group analyses and differential item functioning analyses to examine whether the SAI, overall and by each item, may have potential measurement bias for certain teacher groups. AIR also examined item nonresponse to determine if any items had a nonresponse rate of 25% or greater. For items with nonresponse rates of 25% or greater, AIR assessed potential bias by comparing the characteristics of respondents, nonrespondents, and the entire sample.

These analyses found that the revised SAI is a valid measure of all of the 2022 Standards for Professional Learning. The items that were designed to measure each standard produced consistent information and align to those standards as intended. In addition, AIR’s analyses showed that educators at different schools and of different racial backgrounds scored similarly (although scores were higher among teachers than instructional coaches for some standards), indicating that the SAI does not have issues with measurement bias.

To complement the qualitative analysis, AIR analyzed qualitative data from cognitive interviews and focus groups with teachers, instructional coaches, and school and district leaders about their experiences taking or using the SAI in their settings. These qualitative data provided insight into the experience of completing the survey and considerations for continued revisions to ensure that the revised SAI is relevant to districts and schools as they assess their professional learning systems. Overall, teachers, instructional coaches, and district leaders shared positive feedback on the SAI and believed that the revised SAI captures key professional learning experiences and will yield valuable information about professional learning systems.

Next steps

Learning Forward is exploring the feasibility of conducting a predictive validity study to determine the extent to which the SAI predicts future outcomes for teachers and students. However, establishing such correlations in a way that is rigorous and does not fall prey to the limitation of self-report surveys requires a very large sample and considerable expense.

A future article in The Learning Professional will share details about how SAI data can be used and applied to improving professional learning in systemic ways. We’ll illustrate the tool’s useable format and delve into how it can lead the way toward systemic improvement of educators’ learning and practice.

To view tables download pdf here.