The impact of critical policy evaluation professional development on education policy practitioner's knowledge

Shifts in the sociopolitical environment further spotlighted the need for reforming our nation’s educational inequity problems (Hernández, 2020). As policy change or policy reforms target inequities, the goal is to improve student outcomes and access to opportunity. Policy evaluation in education systems is an important and often underutilized tool in reforming public problems (Golden, 2020). Policy evaluation illuminates the efficacy of policy reforms, clarifying what is working and why it works (Golden, 2020). Today, policymakers are using policy reforms as a tool to combat inequities, without supporting policy practitioners, analysts, and researchers with the capacity, mindsets, and infrastructure for the effective evaluation of the impacts of policy reforms, thus limiting public accountability and effective policy learning opportunities (Golden, 2020). This limits the progress of equity-focused policy reforms in public schools because society never realizes the full possibilities for impacting student outcomes and dismantling structures of injustice (Golden, 2020). As equity gaps persist in our public education systems, there is a need for a professional development training and policy evaluation tool that: (1) Expands policy practitioners’ evaluative thinking skills and policy evaluation best practices. (2) Builds on the tenants of evaluation culture to grow evaluation capacity. (3) Analyze equity-based reforms through an outcomes-based lens. This study utilizes a mixed-methods approach to understand how policy practitioners engage with the methodology and tools of professional development and critical outcomes-based policy evaluation tool and the impact that learning and using the evaluation tool had on their evaluative thinking. The research explored the following questions (1) In what ways do critical outcomes-based policy evaluation tools and methods impact policy practitioners’ knowledge? (2) How do practitioners engage with a critical outcomes-based policy evaluation tool? (3) How do leadership conditions and structures support practitioners in engaging in critical outcomes-based policy evaluation? The study found that the professional development component improved practitioners’ knowledge, skills, and mindsets with evaluative thinking and evaluation culture. The pretest to post scores highlighted the efficacy of the training and professional development component with practitioners. The impact of the tool was inconclusive because of the lack of a full implementation among the participants. However, the findings revealed that policy practitioners engaged practically with the evaluation tool and implemented parts and sections of the tool in their practice that were aligned to current needs and work streams and found that favorable given the demands of their roles. Leadership conditions were found to be a significant factor in the tool’s success’s implementation and use. With the role of partisan politics ever-present and shaping the policy priorities of some evaluation organizations and agencies, the actions and acceptance of leadership favorable to these types of practices will promote the use of this type of evaluation. The recommendations for this work moving forward are increasing the focus of the intervention on the organizational leadership in policy organizations, policy practitioners getting more support in political navigation, and additional versions of the policy evaluation tool that support feasible and practical use.