Across diverse sectors of the economy, standardized processes and shared tools have allowed professionals to manage complexity at enormous scale. Without that access to accurate information, the organizations we rely on would look very different.

Imagine a large retail business where executives have to call store managers to send employees up and down aisles with clipboards to answer basic questions about inventory. Imagine an investment bank where employees spend hours manipulating data in spreadsheets whenever they need to know the value of key assets. It’s hard to envision these businesses operating so inefficiently.

Yet, too often, schools lack the kind of access to timely information and predictable processes that are taken for granted in other work settings, especially when we consider what happens in educators’ direct work with students. Consider the experience of a student who begins the 7th grade reading at a 3rd-grade level. The instructional response to that student’s challenges is determined largely by the class and teacher to which the student happens to be assigned.

In many districts and schools, there is no defined protocol for supporting the student. Decisions about the curriculum, instructional tools, and types of reading assessments are often made by individual teachers or administrators. The processes for those decisions are not transparent, and professional learning to help educators carry them out is inconsistent. The result is that the student’s likelihood of learning to read proficiently is left to chance.

This absence of common tools and standard processes happens in many aspects of education, including professional learning design, and it leads to large, unintended, and entirely preventable variations in student opportunity and outcomes. The negative impact falls most heavily on those students with learning challenges, whose communities have been systematically marginalized, and who have the least external resources. This is not a failure of individual educators, but of a system to provide the infrastructure to build educators’ capacity so they are responsive to all students in a large-scale, complex environment.

It is this type of problem that New Visions for Public Schools aims to solve as part of our longstanding goal to improve graduation rates across all of the 71 New York City schools in our network. Today, these schools have an average graduation rate of 91.5%, but at the time of our founding in 1989, the graduation rate in New York City had hovered for decades at around 50%.

Our work for two decades creating effective small schools changed this trend and began a dramatic increase in high school outcomes. Despite this, by the end of the small school reform era, there were still too many students who were not graduating and many schools where the graduation rate was unacceptably low.

In 2014, we realized that students entering their senior year of high school with similar academic profiles across our network had dramatically different graduation outcomes. New York City has some of the most complex graduation requirements in the country, offering multiple diploma types and pathways, each of which includes many specific course requirements and subject-specific exit examinations.

This can lead to wide variation in graduation outcomes for students with similar profiles who attend different schools. For example, when we compared students who began their senior year of high school with 30-plus credits and having passed two state subject exams, there was so much variation that one school graduated 100% of those students while another graduated only 19%, and other schools fell at many points in between.

Through many conversations with administrators and staff at our partner schools, we came to better understand key obstacles to graduation. We also came to realize that schools needed the key resources that those retail businesses and investment banks described above have — easy access to useful data — and opportunities to learn how to use it.

Harnessing data to improve student outcomes

We recognized that to continue to consistently and dramatically improve graduation rates, we needed to enable educators to have easy access to integrated data in useful formats so that they could track who was graduating, who wasn’t, and why. We also needed to develop and support student planning processes aligned to the data that educators could implement at scale so that no students would fall through the cracks.

We ultimately created a structure we call strategic data check-ins. The check-ins are protocol-driven conversations that take place at defined points during the school year in which educators use student-level data to learn about students’ trajectories and make important decisions about how to support students so they’ll be prepared to graduate. Strategic data check-ins provide schools with a framework for making decisions and sharing those decisions and action steps in a transparent way to avoid the all-too-common haphazard approach to student support.

We created strategic data check-ins for several aspects of student experience and achievement that affect the likelihood of graduation. The core check-ins and their associated strategies are:

• Graduation planning: Identify aspirational graduation date (e.g., June 2022) and diploma type (e.g., Regents);
• Attendance: Match all students with less than 90% attendance to an appropriate attendance support and monitor progress;
• Credit gaps: Identify courses students need to take to keep them on-track to actualize their graduation and postsecondary plan;
• Marking period grade analysis: Match all students who underperform in courses at midterm to academic support(s) and monitor their progress; and 
• Regents planning and prep: Plan and schedule the Regents exams (state exams in specific content areas) students need to achieve their graduation plans and college readiness. Once exams are planned, identify students needing additional support to achieve success on Regents exams, match them to an appropriate preparation activity, and monitor their progress.

The New Visions data portal 

When we created the strategic data check-ins, we recognized that they alone would not change educators’ practice and that a number of other components would have to work in concert with the data protocols.

For starters, the data required by the check-ins existed but were not quickly and readily available to administrators and counselors. If gathering the data from multiple sources and preparing it in a digestible way takes a researcher with strong spreadsheet skills three days of work, it’s not going to be helpful to teachers in a meaningful, real-time way.

To understand and address this problem, we sat with educators and watched their struggles and workarounds to compensate for the absence of tools designed around their actual workflows. With a better understanding of their needs, we developed more sophisticated spreadsheet tools, which we piloted immediately at significant scale given our longstanding partnerships.

Over time, we were able to use the spreadsheet tools as a testing ground for what became a web application called the New Visions Data Portal. The portal automatically pulls reports from multiple New York City Public Schools data sources into a single place so that educators can easily identify students who are in need of attention.

It also provides space for schools to engage in and record student-level planning. Updated on a daily basis, the portal simplifies the gathering and analysis of data to allow school staff to focus their time and attention on planning and supporting students.

Initially, the portal was only available to schools in the New Visions network, but in response to demand, the New York City Public Schools expanded access to a growing group of schools, the Renewal Schools, a program initiated by the Department of Education to support a group of the most underperforming K-12 schools across the city. We engaged educators in professional learning about using the portal to conduct strategic data check-ins in a routinized manner. Eventually, the portal was rolled out to all K-12 public schools in the city.

Evidence over the last nine years (see table on p. 63) suggests that the citywide implementation of strategic data check-ins using the portal supported progress on key academic indicators including attendance, credit accumulation, state exam performance, graduation rates, and college readiness rates. The improvement is most marked in the most underperforming schools, but improvement has occurred across the city.

 The use of the portal and implementation of strategic data check-ins not only allowed our organization to hit and exceed our longstanding goal of an 80%-plus graduation rate, but also led to a significant reduction in graduation rate disparities across ethnicities.

College readiness Network for School Improvement 

In summer 2018, we embarked on a new effort that would leverage our learning about the check-ins and the portal to dramatically increase the number of low-income students of color graduating from high school on time and prepared to succeed in college and career. The Bill & Melinda Gates Foundation selected New Visions to participate in Networks for School Improvement, a group of schools that would work together to use continuous improvement to solve common problems they face.

Through both national research and an analysis of data from our own network, we found that a GPA of 80-plus on a 100-point scale is the single most predictive indicator of college enrollment and persistence. We also learned that our students’ GPAs at the end of 9th grade were highly predictive of GPAs at the end of high school. Among those who had lower than an 80 GPA in 9th grade, only 8% would go on to exceed the 80 threshold at the end of high school. This led us to narrow the aim of our network: to improve the percentage of Black and Latinx students achieving an 80-plus GPA by the end of 9th grade.

To support schools in monitoring yet another metric, we had to integrate it into the routines that schools already had in place and provide them with the key data they needed to make decisions about academic support for students.

With that in mind, we leveraged what we learned from our initial set of check-ins and engaged in inquiry with educators to determine foundational strategies we could build on. We realized that we could expand on the marking period grade analysis check-in to include a review of students who weren’t failing courses but were passing with a GPA lower than 80. These students are typically overlooked because they aren’t struggling so much that they require a deep intervention, nor are they high achievers being primed for advanced coursework.

The image on p. 64 shows a screenshot of where we added new functionality to the portal so that educators can easily monitor progress toward an 80-plus GPA. (All names and data are fictional.) Here, schools can see not only the student’s current GPA but what their projected GPA will be if their grades remain the same for the rest of the term.

Schools can see what the change in overall GPA would be if the projected GPA becomes actualized, the count of courses the student is not passing, courses they are passing but below the 80-plus threshold, and those where they already have an 80-plus — all of which help schools support students academically. Educators can use this information to either deploy existing student support strategies or engage in professional learning about how to support students in their challenge areas.

We knew that if we didn’t build capacity and create routines at the school level, the new GPA metric could easily fall by the wayside. When we engage educators in learning how to use the system and then consistently support them to review data at key points in the year, over time it becomes part of the way the team works.

Each time marking period data were released, our coaches helped educator teams at each school in the network use the portal to review GPA outcomes. As the years passed and teams became more experienced, we noticed that coaches needed to do less prompting and some of our teams included routines for reviewing gradebook data between formal marking periods. It stopped feeling like an add-on and became a common way that the teams worked.

Impact on GPA outcomes 

The table on p. 65 shows the 80-plus GPA outcomes for our Black and Latinx students over the last four years. Although year-to-year changes have been inconsistent, which we believe is largely due to COVID-19 disruptions, the percentage of Black and Latinx students across our network achieving an 80-plus GPA has increased by 7.5% over four years. We attribute much of this to schools having routines for consistently reviewing this data and intervening when students were falling behind in courses.

Approach and lessons 

New Visions believes that, for institutions to be successful, they must have tight systems and reliable routines. Creating a tool and naming routines for using that tool are not sufficient. We needed to provide consistent support and repeatedly return to the same routines over multiple school years to build the capacity of team members.

The establishment of these routines not only provides stability to the school and ensures consistency in the quality of student plans, but also provides the space and time for schools to be more innovative, all of which have contributed to improved student outcomes over time.

Download pdf here.