Reformer ToolboxLogin



CancelLost your password?

Transitioning to Student-Centered Learning: Evaluating Impact


• Karla Phillips

While working with states to implement innovative student-centered learning policies and programs, ExcelinEd has identified policies and practices that hinder new student-centered learning models. Transitioning to Student-Centered Learning: Policy Solutions for States is a policy brief series dedicated to addressing these challenges.

The Challenge of Evaluating Progress and Impact

All new education programs and initiatives are subject to scrutiny from policymakers, media and parents. However, the level of scrutiny state innovation and pilot programs face is often heightened since these innovative programs are, by definition, untested.

Because most existing state innovation or pilot programs are relatively new, they don’t have a track record of success yet. In fact, many schools and districts participating in these programs are still in the design and planning phase so they don’t have a track record at all. As such, little empirical data is available for states to reference when developing a program or measuring the full impact of innovation and pilot programs on students and their outcomes.

Though challenging, it’s critical that states can evaluate, report and communicate the progress and impact of their innovation and pilot programs.

“States [should] have a plan in place for knowing if or when and how to scale up. States can identify processes for evaluating the design and impact of small-scale pilots to determine what aspects of the innovations are working, under what conditions, and for which students. Conversely, the evaluation could also be designed to identify innovations that do not meet their promise, even after adequate time to adjust implementation to improve results.”

Taken from Achieve’s report Creating Innovation Zones to Advance the Promise of Competency-Based Pathways

Recommendation: Develop a Thoughtful Evaluation Plan

States must understand that the transition to student-centered learning will not happen overnight. As states begin this transition, there are practical steps they can take to ensure they develop thoughtful evaluation plans to support their long-term vision for innovative pilots and programs.

Evaluation Plan Objectives

A thoughtful evaluation plan informs parents, teachers, the community, policymakers and more about what is working and what isn’t. The evaluation plan should:

  • Supply local leaders with the information necessary for continuous improvement and building community support.
  • Meet legislative expectations by providing the information needed to build continued support and potentially expand the program.
  • Identify the issues schools are encountering and the resources needed to overcome them.
  • Highlight policy obstacles and potential solutions.
  • Provide as much transparency as possible, building trust in the program and evaluation process.

Guiding Questions to Develop a Thoughtful Evaluation Plan

Developing an evaluation plan may seem intimidating, but it really comes down to four questions:

  1. What are the goals or desired outcomes of the program?

Most programs have clearly stated outcomes either in legislation or in applications. But if this isn’t the case, goals and desired outcomes should be defined at the outset, so the right data is collected and participating schools have clear guidance.

  1. How often will reports be produced?

There may be formal statutory reporting deadlines, but more frequent interim and more informal reports should be included. As soon as possible, establish a timeline with due dates for all involved.

  1. Who will perform the evaluation?

Progress reports can be formal reports from the state department of education or selected presentations by school leaders at a state board meeting. However, if there is a specific time frame for implementation, a more formal external analysis would carry more weight.

  1. What data will be needed?

States should identify early on what data will need to be collected to accurately evaluate the impact of innovation and pilot programs as they mature.

State summative assessments will be able to provide some of the data necessary, but participating schools should be asked to identify what formative and interim assessments will be used.

State longitudinal data systems must be set up to identify students participating in state programs and for which period they participated. This would make it possible to evaluate the impact of programs on long-term student outcomes such as high school graduation and postsecondary success.

Full participation in data collection and evaluation activities should be made clear on any state program applications.

Indicators to Measure Program Quality

It is important to identify the correct indicators and the timing for the associated data collection to appropriately measure state innovation pilot and program quality. While student outcomes should remain at the core of evaluating the impact of state programs, many of the recommended indicators below are particularly valuable in measuring innovation pilot and program effectiveness when student outcome data is limited.

Program Quality Indicators*

Leading Indicators

During Implementation

Student Engagement

  • Absenteeism rates
  • Student surveys
Instructional Practices

  • Teacher survey of instructional practices
  • Administrative observations
  • Teacher turnover by teacher effectiveness
  • External observation for fidelity of implementation
  • Trend in students needing added instructional support/interventions

Primary Indicators

Once Fully Implemented for at Least Three Years

Performance on Local Formative and Benchmark Assessments

  • Percent of students demonstrating at least a year’s worth of improvement or more in core subject areas
  • Percent of students demonstrating proficiency at a specific level in core subject areas
  • Percent of students on-track to be college/career ready by the end of high school in core subject areas
Performance on State Standardized Assessments

  • Percent of students making at least a year’s worth of growth in math and ELA
  • Percent of students performing at proficient or above in math and ELA
  • Percent of students on-track to graduated college/career ready by the end of high school in math and ELA

Long-Term Indicators

Student Outcomes Post-Graduation

High School Success

  • Percent of students graduating in 4-years or less
  • Percent of student graduating with a college/career ready diploma (if offered in state)
  • Percent of students who have completed a rigorous high school curriculum:
    • Math—Completed Trigonometry or higher
    • Science—Completed Biology, Chemistry or Physics
  • Percent of students proficient in a specific vocational/technical skill
  • Percent of students demonstrating college/career readiness:
    • Percent of students performing at the college ready benchmarks on college admissions tests (SAT/ACT)
    • Percent of students having earned college credit in high school
    • Percent of students having earned an industry certification
Postsecondary Success

  • Percent of students enrolled in 2- or 4-year college within two years of graduation
  • Percent of students who persisted from their 1st to 2nd year of college within 3 years of graduation
  • Percent of students in 2- or 4-year college who enrolled in at least one remedial course
  • Percent of students with an industry certification
  • Percent of students not enrolled in college who have a full-time job with benefits

* All outcomes should always be disaggregated by subgroups.


About the author


Karla Phillips

Karla@ExcelinEd.org

Karla is Policy Director for Personalized Learning at ExcelinEd. Previously, she served as Special Assistant to the Deputy Superintendent of Policy and Programs at the Arizona Department of Education. Karla also served as the Education Policy Advisor for Governor Brewer and as the Vice-Chair of Arizona’s Developmental Disabilities Planning Council. Her experience includes serving as Director of State Government Relations for Arizona State University (ASU) and as a senior policy advisor for Arizona’s House of Representatives. Karla received her B.A. from Indiana University and an M.P.A from Arizona State University.