Carol Tomlinson
Lori Bland
University of Virginia
Charlottesville, VA
Calls for the evaluation of programs stem from a strong belief that the evaluation process will generate data useful in the process of program improvement and development. However, evaluation data and reports that are not used serve no purpose whatsoever. Hence, recent studies at the University of Virginia have addressed the issue of evaluation utilization in programs for the gifted. The first study (Hunsaker & Callahan, 199 1) examined current trends in evaluation of gifted programs. A review of 70 evaluations collected from school districts across the nation reveal several distinct trends.
- Most gifted program evaluations use summative evaluation.
- Administrators determine the evaluation questions.
- Data primarily reflects opinions about the program and are gathered solely through questionnaires.
- Evaluation designs do not reflect current thinking about what constitutes effective gifted program evaluation.
These trends are at variance with recommended practice for evaluation utilization.
To understand and describe which factors do contribute to evaluation utilization in gifted programs was the purpose of an extension of the descriptive study of Hunsaker and Callahan. First, through a review of the general and gifted education evaluation utilization literature (Tomlinson, Bland, & Moon, 1993) two sets of factors affecting utilization were identified:
- Factors which are not under the evaluator’s control (such as economic and political conditions).
- Factors which evaluators can control (such as design, audience identification, dissemination, and quality of evaluation).
Then, through a series of case studies of 12 school districts selected on the basis of complete and thorough evaluation reports, we were able to conclude that an intent to evaluate and a prescribed evaluation procedure result in the use of the findings for positive program change. However, a continuum of differences in utilization exists based upon the extent to which good evaluation practices are used. From these findings, several suggestions follow:
- Make evaluation a part of planning from the earliest stages of program development.
- Develop clear program descriptions and goals.
- Provide adequate funding for evaluations and adequate time for evaluation procedures to be followed.
- Prepare staff for conducting and analyzing the results of the evaluation.
- Clearly identify all audiences that have an interest in or need for evaluation results, and involve them in the full evaluation process.
- Ask questions which are well focused to provide information about the goals, structures, and activities of the program being evaluated–questions which will aid in making significant program improvements.
- Use multiple data sources in order to understand the values and perspectives of varied groups of stakeholders.
- Develop evaluation designs which address complex issues of measurement in programs for the gifted.
- Use a variety of data gathering methods designed to reflect the unique structure and goals of programs for gifted learners.
- In evaluation reports, fully describe procedures for data collection and interpretation so that audiences understand processes which were followed and conclusions which were drawn.
- Disseminate to all appropriate audiences reports which are timely and designed to encourage follow-through in translating findings into action. Develop a specific plan for turning findings into positive program growth as an essential part of each evaluation, including roles which various program personnel, evaluators, and stakeholders will play in that plan.
Also emerging from this study is the need for training of personnel in gifted education program evaluation.
For more complete descriptions of these studies see: