Menu

FOCUS

‘Data’ shouldn’t be a dirty word

By Jim Knight and Michael Faggella-Luby
Categories: Coaching, Continuous improvement, Data
April 2022

Data is an inescapable part of our lives. We ask Siri the temperature before we decide which coat to wear when we leave the house in the morning, we keep an eye on the speedometer as we drive to and from work, and we might even use a sleep app to gauge how effectively we sleep at night. Data is so deeply woven into the fabric of our lives that it is next to impossible to imagine what a data-free life would be like.

But despite the centrality of data in everyone’s personal lives, when people talk about data in schools, their comments are often negative. For example, aware of the negative feelings the word “data” evokes for many, coaches often try to find other ways to refer to it.

A certain degree of distaste for the word is understandable. In educational systems with an intense focus on achievement scores, teachers can get the sense that numbers are all that matters and that learning is not nearly as important as test scores.

In some instances, educators feel that data isn’t useful because standardized test results arrive too late in the school year to change practice for the group of students the data represents. And in worst-case scenarios, leaders sometimes share data in a way that erodes teacher morale, especially when scores are lower than hoped and teachers don’t know what they can do to improve results.

But we believe “data” should not be a dirty word. In fact, we believe that gathering, interpreting, and sharing data are all essential parts of effective instructional coaching. Data should be embraced, not shunned, because it can improve student learning and well-being, accelerate professional growth, and build teacher morale.

Data can help teachers see their students’ learning and well-being needs more clearly. For example, data on something as simple as the percentage of noninstructional time occurring in a classroom can lead to important insights into how learning is and isn’t occurring during lessons. When teachers reduce noninstructional time by 20%, for example, that is comparable to adding a full-day of learning for students each week.

Data also accelerates professional learning by revealing what is and isn’t working for students. A simple exit ticket collected at the end of a lesson that asks students to respond to a content-related question can help teachers see whether they need to adjust their instruction — for example, by providing more feedback, modeling new skills, or giving students more opportunities to practice with a partner — so that more students succeed. When students and teachers both see progress, real learning can occur.

Finally, the way data is shared during coaching can build morale by fostering hope. As psychologist Shane Lopez (2013) has explained, hope has three dimensions: goals; pathways to the goals, which involve monitoring progress and making adjustments; and agency, our “perceived ability to shape our lives” (p. 25).

Data is central to all three dimensions. That is, data helps teachers identify preferred futures or goals by making the invisible visible. Data helps people see if their pathway to a goal is working or needs to be modified. And finally, data helps build agency (in both students and teachers) by helping people see their progress as they move closer and closer to identified goals.

As Teresa Amabile and Steven Kramer (2011) write, “Of all the things that can boost emotions, motivation, and perceptions during a workday, the single most important is making progress in meaningful work.” In a very real sense, when they share data effectively, coaches build hope in others.

Data rules

Data takes many forms in schools, but if teachers and coaches intend to reap the benefits described above during coaching, schools must gather and share data effectively; sadly, too often, they do not. For the past two years, we have been studying how data can be used by teachers and coaches to increase learning and hope. We’ve summarized our findings in the following data rules.

Teachers should choose the types of data to collect and analyze. Teachers are more likely to embrace data and, therefore, benefit from it, when they choose it themselves. Coaches can ask teachers the simple question, “How will you know you’ve hit the goal?” as a helpful starting pointing for identifying what data to gather. If data is going to increase hope, it needs to be the kind of data that teachers see as important.

Measures must be valid. Data isn’t helpful if it measures the wrong thing. Asking people to complete a multiple-choice quiz to assess their ability to ride a bike isn’t nearly as effective as actually watching them attempt to ride a bike. Coaches and teachers should use the appropriate kind of assessment for the learning that is being assessed. For example, quizzes and checks for understanding might work well for assessing student knowledge, but rubrics are likely more effective for assessing skills.

Data gathering should be reliable. Reliability means that different people get the same results when they gather the data. Within coaching, this means that the coach and teacher get the same results regardless of who collected the data. This is important because when people engage in dialogue about data, they need to be certain they are talking about the same things. Miscommunication is almost inevitable when data is not gathered reliably or when those involved have different definitions for the data.

Coaches and teachers should prioritize objective over subjective data. Data is most powerful when it is objective. Although subjective data can be helpful, most of us find it easier to process factual data rather than someone’s opinion. Furthermore, research suggests that observers often overestimate the validity and reliability of their observations (Buckingham & Goodall, 2019), so our subjective observations may not be as helpful as we assume.

Data is more useful when it is gathered frequently. One of the most important uses of data is to help teachers see whether what they are doing is working, and frequent data collection helps them make real-time connections between their practice and students’ learning. Teachers who frequently give students exit tickets with questions about content, for example, are able to see if their teaching is working or if they need to make adjustments to how they teach so students master the content before moving on to new content. Also, data only builds hope when it shows teachers and students that they are making progress. For these reasons, we suggest data is gathered at least once a week.

Data should be easy to gather. Since assessments need to be conducted frequently, they need to be as easy to implement as possible. There are many excellent assessments that yield excellent data for teachers but aren’t helpful because they are too difficult to implement regularly and reliably.

Data is most powerful when teachers are involved in gathering it. When teachers, rather than coaches or others, gather data, it cuts out the middle person, and data becomes more immediate and powerful. For example, if teachers watch video recordings of their lessons and code what types of questions they ask in class, it’s more powerful than when coaches observe lessons and code the data and share it with teachers. When teachers see for themselves the impact of their teaching, they are much more likely to believe they and their students can achieve their goals.

Kinds of data coaches can gather

The data rules can be applied to the gathering of many different types of data, but during instructional coaching, three types are especially important: student achievement, student engagement, and instructional practices.

Student achievement

We usually describe achievement data as either summative or formative. Teachers give summative assessments at the end of a learning period to get an overall measure of achievement and give formative assessments to measure student understanding at regular intervals during instruction.

Formative assessment is most useful for guiding professional learning since it provides timely data for instructional decision-making. Formative data can help coaches and teachers consider the following types of achievement.

Knowledge. Formative assessments of knowledge can include daily or semiweekly short quizzes or checks for understanding, such as exit tickets, response cards, Likert-scale responses, or other types of assessments that teachers can use quickly and informally during teaching.

Other options include prompts being embedded within a set of Google slides or requiring a written response via an index card or sheet of paper placed on students’ desks that the teacher can collect at the end of the lesson.

Skills. Checks for understanding are valid measures of learning, but they’re less effective for measuring skills. Teachers and coaches are better off using rubrics to assess the extent to which students have mastered skills. The three essential components of a rubric include a set of evaluative criteria, quality explanations of criteria at specific levels, and a means of scoring.

Rubrics can be holistic, analytical, or single point in nature. Holistic rubrics employ a single scale with multiple criteria (e.g. capitalization, organization, punctuation) to assign a single score. Analytical rubrics provide criteria for each component of the assignment separately, leading to a total score but also clear individual component feedback. Finally, single-point rubrics are used to guide student self-assessment.

Big ideas. Teachers and coaches can use all the types of data described above to measure whether students have learned big concepts or principles. Analytic rubrics, in particular, can be used to assess more complex forms of student writing.

Student engagement

Instructional coaches may not immediately think of engagement data as important, but it is, in fact, essential, because lack of engagement is the main reason students drop out of school (see Knight, 2019, for more information on coaching and engagement). Unfortunately, engagement is often mistakenly seen as a single element. However, it is helpful to view the whole student and gather data on the behavioral, cognitive, and emotional elements.

Behavioral engagement is the most directly observable type of engagement due to external behaviors during learning. For example, physical responses associated with engagement are easily observed when students are on task, using appropriate materials, challenged but not overwhelmed, and able to ask or answer relevant questions. Teachers can collect observable data on student disruptions or time on task during a particular learning activity (e.g. during circle time). Teachers might also use a clipboard with a seating chart to keep track of which students respond to questions and engage in group discussions.

Cognitive and emotional engagement is more challenging to observe. For example, it is difficult to observe how much mental investment (cognitive engagement) a student puts into a given activity before giving up. Similarly, it is tough to gauge the range of positive or negative emotions students experience during different parts of a lesson, including interest, anxiety, or happiness.

However, there are ways to get inside student thinking and feeling before, during, and after learning experiences. One excellent source of data on cognitive engagement is exit tickets, which ask students to share their thinking and, therefore, provide formative feedback. Typically, students respond to a prompt on a slip of paper, index card, or prepared worksheet (digital solutions are also helpful).

Similarly, admit slips, used to assess students’ attitudes when they enter class, prompt students to share their thoughts about their interest in the lesson, their emotional response, or other helpful points of clarification before the start of class for immediate teacher attention.

Another alternative is the use of interactive journals, in which teachers and students write back and forth to each other. Finally, to learn more about their students’ social-emotional experiences and reactions, teachers may use engagement interviews, probing students’ belonging and connection, sense of safety, and hope.

Instructional practices

Gathering data that assesses student learning and engagement keeps the focus where it needs to be — on students — but teachers frequently want to adapt how they teach to help students succeed, and that requires teachers to look at their own practices. Two ways to do this include coding the questions teachers ask and recording the amount of teacher talk throughout a lesson. Teachers can gather this type of data by reviewing video of their lessons, and coaches can do it while observing teachers’ lessons or reviewing data.

Teacher questions are vital components of classroom talk. When they are asked effectively, questions can support students’ thinking and provide teachers with ongoing assessments of students’ understandings. Data collected should include the type, kind, and level of questions asked. The two major types of questions are open questions — those that involve infinite correct answers (Who is your favorite author?) — or closed questions with specific, discernable answers (Who wrote The Pearl?).

Questions can be further categorized by kind: opinion questions that don’t have personal or individual answers or right/wrong questions that must be answered correctly. Finally, teachers can assess levels of thinking required to answer a question by noting if the question focuses on student knowledge, skills, or big ideas.

Teacher talk refers to the verbal interactions that take place between a teacher and a student, or group of students, and is usually intended to facilitate learning. Coaches and teachers can assess three types of teacher talk: teacher monologue, involving no interaction with students; teacher-initiated interactive talk, involving segments of verbal interaction with students as initiated by the teacher; and student-initiated interactive talk, involving segments of verbal interaction with students as initiated by the students.

Alternatively, student-initiated interactions may arise from individual student questions about the material and take the form of either prompted or unprompted opportunities from the teacher. Student responses may be individual or shared. To ensure that teacher talk does not dominate the learning environment, data can be collected to note timeframes for teacher talk only, guided practice with teacher-led student talk, and student talk only.

The power of data

All data is imperfect, and none of the forms of data discussed here is without limitations. However, moving forward with a little bit of light is better than moving forward in darkness. Data fosters student learning and well-being, guides teachers’ professional learning, and builds hope. Data is not a dirty word — just the opposite. When coaches partner with teachers and gather data effectively, data is powerful, positive, and empowering.


References

Amabile, T. & Kramer, S. (2011). The power of small wins. Harvard Business Review. hbr.org/2011/05/the-power-of-small-wins

Buckingham, M. & Goodall, A. (2019). The feedback fallacy. Harvard Business Review, 97(2), 92-101.

Knight, J. (2019). Students on the margins. The Learning Professional, 40(6), 28-32.

Lopez, S. (2013). Making hope happen: Create the future you want for yourself and others. Atria Books.


Image for aesthetic effect only - Jim-knight-150x188-1
Senior Partner at Instructional Coaching Group | + posts
Image for aesthetic effect only - Michael Faggella-luby-scaled-e1648225445554
+ posts

Michael Faggella-Luby (m.faggella-luby@tcu. edu) is a professor at Texas Christian University and director of the Alice Neeley Special Education Research & Service (ANSERS) Institute.


Categories: Coaching, Continuous improvement, Data

Search
The Learning Professional


Published Date

CURRENT ISSUE



  • Recent Issues

    TAKING THE NEXT STEP
    December 2023

    Professional learning can open up new roles and challenges and help...

    REACHING ALL LEARNERS
    October 2023

    Both special education and general education teachers need support to help...

    THE TIME DILEMMA
    August 2023

    Prioritizing professional learning time is an investment in educators and...

    ACCELERATING LEARNING
    June 2023

    Acceleration aims to ensure all students overcome learning gaps to do...

    Skip to content