Accountability model program evaluation

Download PDF. Share Full Text for Free. Web of Science. Let us know here. System error. Please try again! How was the reading experience on this article? The text was blurry Page doesn't load Other:. Details Include any more information that will help us locate the issue and fix it faster for you. Thank you for submitting a report! Submitting a report will send us an email through our customer support system.

Submit report Close. Recommended Articles Loading References School counselors and school reform: New directions. Counseling centers and accountability: Immoveable objects, irresistible forces.

School counseling in the 21st century: Personal and professional reflections. Comprehensive school counseling programs: A review for policymakers and practitioners.

Accountability: A M. Toward the development of a program evaluation business model: Promoting the longevity of counselling in schools. Accountability practices of school counselors: national survey. Evaluation of counseling services: Accountability in a rural elementary school.

Accountability strategies for school counselors: A baker's dozen. Making school reform work. Standards: Here today, here tomorrow. Assessing outcomes in practice settings: A primer and example from an eating disorders program. Improving school guidance programs: A framework for program, personnel, and results evaluation. Interpreting these linkages is useful in understanding the implicit theory of change of the ASCA National Model and consequently in designing appropriate evaluation studies.

It also is helpful to scan backward in the logic model to identify how changes in outcomes are expected to occur. For example, student achievement and gap reduction is linked to six model outputs student change, parent involvement, teacher competence, school policies and processes, school counselor competence, and school counseling program improvements. Increased program resources are connected to three model outputs school counselor competence, school counseling program improvements and administrator support.

Systemic change and school improvement also can be connected to three outputs teacher competence, school policies and processes, and school counseling program improvements. Both of these inputs were identified as being important in the delivery of all six activities.

The first linkage was a unidirectional arrow leading from management processes to program evaluation in the activities column. This arrow was intended to represent the tight connection between management processes and evaluation activities that is evident in the ASCA National Model.

Relatedly, a unidirectional arrow leading from the school counseling program evaluation activity to the program advocacy activity was added.

This arrow was intended to represent the many instances of the ASCA National Model suggesting that program evaluation activities should be used to generate essential information for program advocacy. The final additional link was a recursive arrow leading from the increased program resources outcome to the program resources input.

Logic models portray the implicit theory of change underlying a program and consequently facilitate the evaluation of the program Frechtling, Overall, the theory of change for an ASCA national program could be described as follows: If school counselors use the foundational elements of the ASCA National Model and have sufficient program resources, they will be able to develop and implement a comprehensive program characterized by activities related to direct services, indirect services, school counselor personnel evaluation, management processes, program evaluation and relatedly program advocacy.

If these activities are put in place, several outputs will be observed, including the following: student changes in academic behavior, increased parent involvement, increases in teacher competence in working with students, better school policies and processes, increased competence of the school counselors themselves, demonstrable improvements in the school counseling program, and increased administrator support for the school counseling program.

If these outputs occur, then the following outcomes should result: increased student achievement and a related reduction in the achievement gap, notable systemic improvement in the school in which the program is being implemented, and increased program support and resources. If these additional resources are reinvested in the school counseling program, the effectiveness of the program will increase.

Logic models can be used for a number of purposes including the following: enhancing communication among program team members, managing the program, documenting how the program is intended to operate and developing an approach to evaluation and related evaluation questions Frechtling, The present study was conducted in order to develop a logic model for ASCA National Model programs so that these programs could be more readily evaluated, and based on the results of these evaluations, the ASCA National Model could then be improved.

Evaluations can focus on the question of whether or not a program or components of a program actually result in intended changes. At the most global level, an evaluation can focus on discovering the extent to which the program as a whole achieves its desired outcomes.

At a more detailed level, an evaluation can focus on discovering the extent to which the components i. In both types of evaluations, it is important to use a design that allows some form of comparison.

In the simplest case, it would be possible to compare outputs and outcomes before and after implementation of the ASCA National Model. In more complex cases, it would be possible to compare outputs and outcomes of programs that have implemented the ASCA National Model with programs that have not.

In these cases, it is essential to control for the confounding effects of extraneous variables e. If the level of implementation of the ASCA National Model program as a whole can be measured, it is even possible to use multivariate correlation approaches to examine whether the level of implementation of the program is related to desired outcomes while simultaneously controlling statistically for potential confounding variables.

These same correlational procedures can be used to examine the relationships between the more discrete activities of the program and their corresponding outputs. At the most global level, it is important to evaluate the extent to which the implementation of the ASCA National Model results in the following: increases in student achievement and associated reductions in the achievement gap , measurable systemic change and school improvements, and increases in resources for the school counseling program.

No evaluations to date have examined whether ASCA National Model implementation results in systemic change and school improvement or in an increase in program resources. It also is important to evaluate the extent to which specific program activities achieve their desired outputs. Table 2 contains a list of sample evaluation questions for each activity. Within these questions, evaluation is focused on whether or not components of the program result in overall benefits.

In addition to examining program-related change, it is important to evaluate whether a basic assumption of the ASCA National Model bears out in reality. Qualitative evaluations of the relationships between inputs and quality of the activities are necessary to determine what levels of inputs are necessary for full implementation. Constructing a logic model retrospectively is inherently challenging and complex.

This is especially true when the program for which the logic model is being created was not initially developed with reference to an explicit, coherent theory of action.

In the present study, the authors approached the work systematically and are confident that others following similar procedures would generate similar results.

Engaging individuals who were involved in the development and implementation of the ASCA National Model in dialogue might have resulted in a richer logic model with even more utility in directing evaluation of the ASCA National Model. As a follow-up to the present study, the authors intend to continue this inquiry by asking key individuals involved with the ASCA National Model to evaluate the present logic model and to suggest revisions and extensions.

Even given this limitation, the current study has potential immediate implications for improving practice that go beyond its role in providing focus and direction for ASCA National Model evaluation. A potentially fertile testing ground for the implementation of the logic model is present within the RAMP Award process.

Currently, schools provide evidence data and create narratives regarding how they have successfully met RAMP criteria. Twelve independent rubrics are scored and totaled to determine whether a school receives a RAMP Award.

At least two contributions of the logic model for improving the RAMP process seem feasible. Second, the logic model may also help improve the RAMP process by highlighting clearer links between activities, outputs and outcomes. In future revisions of the RAMP process, more attention could be paid to the documentation of benefits achieved by the program in terms of both outputs i.

In this vein, the authors hope that the logic model developed in this study will help to improve the RAMP process for both practitioners and RAMP evaluators.

Retrospective logic models map a program as it is. In that sense, they are very useful in directing the evaluation of existing programs. Prospective logic models are used to design new programs. Using logic models in program design or redesign has some distinct advantages. Kellogg Foundation, , p. When programs are planned with the use of a logic model, greater opportunities exist to explore foundational theories of change, to explore issues or problems addressed by the program, to surface community needs and assets related to the program, to consider desired program results, to identify influential program factors e.

Kellogg Foundation, The authors hope that logic modeling will be incorporated prospectively into the next revision process of the ASCA National Model. Basing future editions of the ASCA National Model on a logic model that comprehensively describes its theory of action should result in a more elegant ASCA National Model with a clearer articulation between its components and its desired results.

Such a model would be easier to articulate, implement and evaluate. The authors hope that the development of a retrospective logic model in the present study will facilitate the prospective use of a logic model in subsequent ASCA National Model revisions.

It is a good starting point for reconsidering such questions as how the model should operate, whether the outcomes are the right outcomes, whether the activities are sufficient and comprehensive enough to lead to the desired outcomes, and whether the available program resources are sufficient to support implementation of program activities. American School Counselor Association. Alexandria, VA: Author.

Astramovich, R. Program evaluation: The accountability bridge model for counselors. Training school counselors in program evaluation. Professional School Counseling , 9 , 49— Carey, J. Professional School Counseling , 16 , — A statewide evaluation of the outcomes of the implementation of ASCA National Model school counseling programs in high schools in Utah.

Professional School Counseling , 16 , 89— Clemens, E. The school counseling program implementation survey: Initial instrument development and exploratory factor analysis.

Professional School Counseling , 14 , — Dimmitt, C. Evidence-based school counseling: Making a difference with data-driven practices. Frechtling, J.

Logic modeling methods in program evaluation. Gysbers, N. Developing and managing your school guidance program 3rd ed. Heppner, P. Performance measurement is a way to continuously monitor and report a program's progress and accomplishments, using pre-selected performance measures. By establishing program measures, offices can gauge whether their program is meeting their goals and objectives.

Performance measures help programs understand "what" level of performance is achieved. Measurement is essential to making cost-effective decisions. We strive to meet three key criteria in our measurement work:. A program sets performance measures as a series of goals to meet over time. Program evaluations assess whether the program is meeting those performance measures but also look at why they are or are not meeting them. For example, imagine you bought a new car that is supposed to get 30 miles per gallon.

But say, you notice that you are only getting 20 miles per gallon. That's a performance measurement. You looked at whether your car was performing where it should be. So what do you do next? You would take it to a mechanic. The mechanic's analysis and recommendations would be the program evaluation because the mechanic would diagnose why the car is not performing as well as it should. You need performance measures to know whether your program or car is performing where it should be, and you do a program evaluation or go to the mechanic to find out the reason why it is not meeting those expectations.

Skip to main content. Contact Us. Popular Tools. What is program evaluation?