Do You See What I See?

District designs learning plan to develop a clear vision of effective instruction

By Learning Forward
Categories: Educator evaluation, School leadership
December 2015
As Larry Gerardot, a principal in Fort Wayne (Indiana) Community Schools, sat in front of a computer, he had no idea how the new project in which he had been asked to participate would affect his work and the work of other principals. Yet he knew that Fort Wayne Community Schools had decided that the district would approach inter-rater reliability as professional learning, starting with the principals. Though he was uncertain of the outcome of this work, he was intrigued with the power of principals working and learning together on the RISE Indiana Teacher Effectiveness Rubric — the district’s instrument for evaluating instructional practice — and improving his practice in supporting teacher learning. The district leadership team of Fort Wayne Community Schools, Indiana’s largest school

Read the remaining content with membership access. Join or log in below to continue.

Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem.

Log In


Kay Psencik, C. Todd Cummings, and Larry Gerardot

Kay Psencik ( is a Learning Forward senior consultant. C. Todd Cummings ( is manager of the System of Support and Compensation and Larry Gerardot ( is principal of the Career Center at Anthis in Fort Wayne Community Schools in Indiana.

Steps In The Process


Guiding Assumptions

  • Establishing clear guidelines and thorough, intense practice through experience with peers strengthens inter-rater reliability and observer agreement. If observers and raters have clear and concise instructions about how to rate behavior and can come to agreement about this rating, this agreement increases consistent ratings across the district.
  • When using qualitative data using two or more observers, developing inter-rater reliability and observer agreement ensures that results generated will be useful in understanding the effectiveness of all teachers based on common vocabulary and can be used to design professional learning.
  • If even one of the observers is erratic on his or her scoring system, the entire system may be jeopardized as perceptions of others may interfere with its effectiveness.
  • Developing inter-rater reliability and observer agreement is more about having clear distinguishing descriptors, exemplars, and conversations than about simple agreement.


Bill & Melinda Gates Foundation. (2012). Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains. Seattle, WA: Author.

Graham, M., Milanowski, A., & Miller, J. (2012, February). Measuring and promoting inter-rater agreement of teacher and principal performance ratings. Center for Educator Compensation Reform. Available at

Hirsh, S., Psencik, K., & Brown, F. (2014). Becoming a learning system. Oxford, OH: Learning Forward.

Kimball, S. & Milanowski, A. (2009). Examining teacher evaluation validity and leadership decision making within a standards-based evaluation system. Educational Administration Quarterly, 45(1), 34-70.

+ posts

Learning Forward is the only professional association devoted exclusively to those who work in educator professional development. We help our members plan, implement, and measure high-quality professional learning so they can achieve success with their systems, schools, and students.

Categories: Educator evaluation, School leadership

The Learning Professional

Published Date


  • Recent Issues

    February 2024

    How do you know your professional learning is working? This issue digs...

    December 2023

    Professional learning can open up new roles and challenges and help...

    October 2023

    Both special education and general education teachers need support to help...

    August 2023

    Prioritizing professional learning time is an investment in educators and...

    Skip to content