E. Jean Gubbins
University of Connecticut
Storrs, CT
Periodically, we initiate an information inventory of products resulting from our research studies and commissioned papers. We revisit abstracts, executive summaries, and full-length monographs and assess the evolving knowledge base since the beginning of The National Research Center on the Gifted and Talented (NRC/GT) in 1990 (Gubbins, 1995). We pose questions, such as the following, to ensure that we are fulfilling our original mission:
- What topics have received considerable attention?
- What topics need further elaboration?
- What questions are suggested by practitioners, researchers, parents, and students?
- What information is requested via letters, e-mail, web site, and fax?
- What resources are responsive to information requests?
- What additional resources need to be created or adapted?
In response to these questions, we determine recurring topical areas. Identification and programming are at the top of the list. We took the liberty of adding evaluation to the list, due to its importance. Our information inventory resulted in a discussion of resource toolkits, consisting of a collection of products responsive to frequently asked questions.
When people pose questions about identification, programming, and evaluation, they want to know about instruments and procedures. Some questions are very specific and technical; others are more general. We refer people to selected NRC/GT products, annotated bibliographies, or other resources available from the United States Department of Education, National Association for Gifted Children, ERIC Clearinghouse, State Directors of Gifted and Talented Education, and Council for Exceptional Children, just to name a few. As readers of the NRC/GT Newsletter, we thought a description of three resource toolkits would be useful.
Almost daily, we are asked about identification. Questions focusing on characteristics of gifted and talented students and assessment procedures predominate. Historical and current perspectives are available in Toward a New Paradigm for Identifying Talent Potential (Frasier & Passow, 1994). Moving the identification paradigm from a single indicator to a multifaceted approach is a central tenet of this monograph. Test scores, teacher nominations, rating scales, observation data, or work samples provide valuable information about students’ skills and abilities. In A New Window for Looking at Gifted Children (Frasier et al., 1995), an observation form, known as Panning for Gold, is accompanied by sample case studies to be used in training teachers how to document the traits, aptitudes, and behaviors of young people. Pulling all this information together as an individual case study is aided by the Frasier Talent Assessment Profile. Assessment data are recorded on a matrix and additional information is sought to ensure advocacy for each child. The final section of the profile reorients the screening and selection committee, as they move from a data matrix to additional descriptive information to a visual of a circle (the child) in the middle of a rectangle. Each quadrant of the rectangle is completed by summarizing the child’s needs: programming options; curricular needs; counseling needs; and goals/outcomes evaluations.
In the appendices of A New Window for Looking at Gifted Children, you can review the annotated bibliography of tests, rating scales, product, and process measures. These annotations will help you understand the purpose of various instruments, the scoring format, the age appropriateness of measures, and the availability of reliability and validity data. The annotations also delineate the relationship to the traits, aptitudes, and behaviors listed in the Panning for Gold instrument: motivation, interests, communication skills, problem-solving ability, memory inquiry, insight, reasoning, imagination/creativity, and humor.
Educators and parents alike describe the behavioral characteristics of young people or ask about traditional and nontraditional assessment procedures. We often suggest a search of Mental Measurement Yearbooks, Tests in Print, ERIC/AE Test Locator Service (www.ericae.net/testcol.htm), and the University of Virginia repository of identification and evaluation instruments. The Mental Measurement Yearbooks summarize the purposes and characteristics of instruments and provide critiques of the test’s strengths and weaknesses. However, you need access to the series of yearbooks to find information about tests developed at different time periods, since each yearbook is noncumulative. Therefore, it is helpful to have the companion reference, Tests in Print, which is a comprehensive listing of tests across all Mental Measurement Yearbooks. If these resource books are not easily available, then consider a search of computer databases from ERIC/AE Test Locator Service that includes all tests from the Mental Measurement Yearbooks and Tests in Print.
You may request a customized computer search of instrument-related information. The NRC/GT at the University of Virginia conducted an extensive search of available identification and evaluation instruments and created a repository. Information from several databases can be customized according to specific criteria. For example, you may request test reviews on specific categories of giftedness: mathematical/logical aptitude, scientific aptitude, acting ability, or task commitment/motivation. A complete summary of the processes used to create the repository is available in the monographs by Callahan, Tomlinson, Hunsaker, Bland, and Moon (1995) and Callahan, Hunsaker, Adams, Moore, and Bland (1995).
Understanding different perspectives on how to identify gifted and talented students is important as educators, parents, and policymakers assess the extent to which challenging educational opportunities are available. Looking at the individual needs of students and available programs and services is the first step in determining the educational match. The educational match should also be viewed in light of existing legislation. Two books that are a must among our resources are The 1996 State of the States Gifted and Talented Education Report (Council of State Directors of Programs for the Gifted, 1996) and State Policies Regarding Education of the Gifted as Reflected in Legislation and Regulation (Passow & Rudnitski, 1993). State directors of programs provide extensive survey data on topics, including:
- state mandates and regulations,
- funding,
- state agency staffing,
- state definitions and identification of students,
- programming,
- program accountability, and
- teacher endorsement and preparation.
There is a wealth of information in tabular, graphic, and narrative formats. Information is easily accessible and comparisons can be made of state or regional data.
A few years ago, Passow and Rudnitski requested state-level documents describing identification and programming strategies and practices. All but one state provided documents, consisting of legislation, regulations, rules, handbooks, and resource materials. All documents were reviewed and analyzed. Illustrative information on topics such as identification, programming, differentiated curriculum and instruction, and counseling and support services provides readers with an overview of existing policies and procedures. In many ways the information can be used as a possible template for improving local or state policies.
Identifying special populations or underserved populations is another topic of great interest. Parents request information about students with dual exceptionalities. They are often well-schooled in their child’s disability, understand interventions that address specific needs, and note the emphasis on their child’s learning difficulties, rather than learning strengths. Depending on the specific question, we often recommend resources on high ability students with behavior disorders (Reid & McGuire, 1995), attention deficit hyperactivity disorder and creativity (Cramond, 1994), high potential students with cerebral palsy (Willard-Holt, 1994), and high ability students with learning disabilities (Reis, Neu, & McGuire, 1995).
What are the characteristics of effective programs and services? The question of “what works” is difficult to answer from a distance. Quality programs and services for gifted and talented students must be carefully connected to the needs of students and the school district (USDE, 1993). What talents and abilities of students are nurtured and challenged? What talents and abilities need to be addressed? Asking such questions moves the conversation to the schoolroom. Obviously, recognizing existing programs and services throughout the school district is the first step in developing a comprehensive continuum of services. We often share a continuum of services at elementary, middle, and secondary levels outlined by Renzulli (1994). Some of the options are:
- general classroom enrichment
- within and across grade pull-out groups by targeted ability and interest areas
- non-graded cluster grouping by skill level
- internships
- mentorships
- magnet school
- special school
- honors classes. (p. 78)
Delcourt, Loyd, Cornell, and Goldberg (1994) and Delcourt and Evans (1994) conducted quantitative and qualitative longitudinal studies, respectively, of different programming options: special school, special class, pull out, and within class. In the qualitative study of learning outcomes in elementary schools, Delcourt and Evans identified key traits consistent across exemplary program models: leadership; atmosphere and environment; communication curriculum and instruction; and student needs. A strong administrative voice characterizes exemplary models (Delcourt, 1995). The leader ensures that staff and community members understand the program’s purposes and view it as a critical program component of school community. Establishing this connection requires clear and frequent communication with parents, students, teachers, and administrators concerning program activities and student performance. Recognizing students’ needs and providing quality programs and services are central goals of excellent school systems (USDE, 1993).
Focusing discussions on service delivery options is certainly not the first decision to be made after determining the academic, affective, or artistic needs of gifted and talented students. However, potential options do have programmatic, personnel, resource, space, financial, and other implications. Understanding students’ needs leads to discussions about the appropriate content match. Some related resources for the programming toolkit include: reading (Jackson & Roller, 1993); mathematics (Sheffield, 1994); science (Brandwein, 1995); arts (Clark & Zimmerman, 1994); curricular options for high-end Learning (Gavin et al., 1994); and thinking skills (Burns, 1993).
What is the best time to develop an evaluation plan?
- end of the first year of program implementation
- after three years of program implementation
- before new programs and services are added
- during initial program planning
If you answered a, b, or c, you are not alone. People often pose tactical questions about program evaluation after programs and services are operational for a few years. They want to be sure that their plans are fully incorporated before they are assessed. Actually, the most appropriate answer is d, since you need to know what has been accomplished and what must be accomplished.
One way to initiate an evaluation during the early stages of program implementation is to conduct a self-evaluation, as described by Fetterman (1993). The assessment may involve questions such as the following:
- Are the identification, screening, and selection criteria appropriate for the program in operation?
- Does the program operate in accordance with its own philosophy?
- Does the curriculum reflect the philosophy and goals of the school program?
- Are students engaged? Is there any observation, product, interview, or other documentation of critical and creative thinking in the program? (pp. 6-7)
Another approach is to use the Program Profile Form designed by Delcourt and Evans (1994) for their qualitative evaluation of four programs representing one of each service delivery model (i.e., separate class, special school, pull-out program, within-class program). The Program Profile Form consists of four parts. Part I requires that you provide an overview of your program (e.g., philosophy/mission statement, needs/belief statements, definition of giftedness/talent, systems/models, and program options. Part II delineates various categories of information needed to document the identification procedure, including type of instrument, selection criteria, special population provisions, and decision making protocol. Part III requires curriculum/student assessment information on program objectives, evidence of scope and sequence of activities, staff development system, and parent, teacher, student, administrator communication systems. Finally, Part IV addresses components of program evaluation, namely focus, design, information sources, and data gathering methods. As you review your program and document the information for each section of the Program Profile Form, you can visually determine which sections lack information or are not well-articulated. What aspect of your program needs attention? What sections illustrate sound identification, programming, and evaluation principles?
Callahan and Caldwell (1995) prepared a guide to evaluating programs for the gifted. They introduce practitioners to the language of the evaluation field, discuss evaluation designs responsive to programmatic questions, describe how to select or construct instruments, and provide pointers on synthesizing data for appropriate audiences. Evaluation should be an ongoing approach as programs and services are designed and implemented. Evaluation questions are posed, instruments are created or selected, data are collected and summarized, and results are reported to appropriate audiences. Evaluation is a process of decision-making (Renzulli, 1975). Resulting data should be used to modify, extend, or create appropriate programs and services.
Identification, programming, and evaluation toolkits are part of our professional library. We constantly look for sources of information responsive to people’s questions. Our collection of favorite resources may change periodically, but we often find that certain key resources always provide critical information for multiple audiences.