A Local Guide to Continuous Program Improvement (and Accreditation Reports)
Oh, sorry, did you say rubric?
Table of Contents: The AAAAA List
This Guide is designed to assist COE departments and programs in putting together assessment plans and accreditation reports. We hope other units find it useful, as well.
FIVE STEP PLAN
Adapted from a presentation by Douglas Walcerz and Eric Gibbs, True Outcomes, May 2008, UHM
1. ACCOUNT (page 3)
Anchor your program (provide your context) in COE. (What is it all about, Alfie?)
2. ALIGN (5)
Agree on what you are trying to accomplish. (Yes, this does mean your faculty.)
3. ASSESS (7)
Accumulate evidence. (Use technology to avoid perishing in piles of paper.)
4. ANALYZE (10)
Aggregate; disaggregate. (And aggravate just about everyone in the process.)
5. ACT (11)
Apply what you find to help students learn better. (Worth it.)
Appendix A Template Workbook (12)
Appendix B COE Program Examples (16)
Oh, yes . . . NCATE Focused Visit in Fall 2009
Step 1. ACCOUNT
Anchor your program (provide your context) in COE
Describing your program comes first, including the usual suspects—who, what, when, where, why, how. Set the context for your program, students, faculty, and program standards or outcomes as an essential part of the College of Education. Remember to link what your program does to the COE vision, mission, and conceptual framework, shown here.
College of Education
Motto: Preparing educators to contribute to a just and democratic society
Vision: The COE consists of educators who provide innovative and cutting-edge research and teaching in an effort to further the field of education and prepare educators to contribute to a just and democratic society.
Mission: The mission of the College is to work as a diverse and democratic community to:
• prepare new educators and provide ongoing professional development in education (teaching), • increase the knowledge base in education and related fields through the production and application of research related to teaching, learning, and assessment (research), and
• serve as partners and leaders for excellence in education (service).
Philosophy, Purposes, and Goal: The College is committed to the fundamental goal of employing and preparing education professionals who are knowledgeable, effective, and caring. These individuals demonstrate these core values through their knowledge, skills, and dispositions.
COE programs that submit program reports (e.g., Special Education, Secondary Mathematics, Educational Technology) to the National Council for Accreditation of Teacher Education (NCATE) Specialty Professional Associations (SPAs) for review must provide the following information in narrative form. Other programs can use these topics as a guide.
Template 1. Program Context (All programs should complete #3 - #8)
Description of any state of institutional policies that may influence the application of program standards.
Description of the field and clinical experiences required for the program, including the number of hours for early field experiences and the number of hours/weeks for student teaching or internships.
Description of the criteria for admission, retention, and exit from the program, including required GPAs and minimum grade requirements for the content courses accepted by the program.
Description of the relationship of the program to the unit’s conceptual framework.
Indication of whether the program has a unique set of program assessments and the relationship of the program’s assessments to the unit’s assessment system.
Program of Study Sheet (outline of courses and experiences required to complete program).
Candidate and Completer Chart (provide three years of data, beginning with most recent).
Faculty Information (COE will collect and make this available to you electronically).
Template 1a: Program of Study
Courses and Other Requirements
Template 1b: Candidate and Completer Chart
(Provide for last 3 years)
Number of candidates enrolled in the program
Number of program completers
The Unit (COE)
Our assessment system calls for each program to specify requirements at four transition points (entry, during program, completion, and follow-up) Requirements must align with the COE Conceptual Framework that calls for helping our students become knowledgeable, effective, and caring education professionals. Our conceptual framework matches the NCATE requirements to demonstrate evidence of students’ knowledge, skills, and dispositions. See the next section (Step 2: Align) for more information.
Unit (COE) Assessment System
Program Requirements (Use specific, descriptive titles)
Knowledgeable, Effective, Caring
Step 2. ALIGN
Agree on what you are trying to accomplish
The first important alignment for COE programs is identifying requirements for all students at selected transition points in their program of study. The four transition points we use as a unit (college) are:
Entry (admission criteria)
During program (important assignments/projects during the program)
Completion (work required by the end of the program)
Follow-up (licensing exams; surveys of graduates, mentors, employers)
COE uses a grid to show program requirements at the four transition points (template 2) aligned with the conceptual framework of helping our students become knowledgeable, effective, and caring professionals. Our framework aligns precisely with the NCATE requirement to show evidence of our students’ knowledge, skills and dispositions.
Template 2: Show Requirements at Transition Points (use specific, descriptive titles)
Show separately for programs that have different assessments
The entry transition point consists of admissions criteria, such as applications, GPA, and interviews. The during program and completion transition points should include six to eight (not more than six to eight total) direct assessments of student work (e.g., projects, papers, presentations) that students complete to move from one program phase to the next. Follow-up consists of state licensing requirements, where applicable, and surveys of graduates, employers, and others who can provide program feedback.
As noted, the transition points should be aligned with the conceptual framework of helping students become knowledgeable, effective, and caring professionals, thereby demonstrating their knowledge, skills, and dispositions. Step 3 provides more information on selecting and writing assessments of student work.
In addition to specifying requirements at transition points, programs should identify their important outcomes or standards (i.e., professional, national, state), and show how they link with the program of study (courses and other requirements). Outcomes (often called student learner outcomes or SLOs) are what we want students to know, be able to do, and care about. Accrediting bodies often call these outcomes or standards, but terms such as objectives, goals, benchmarks, and performance indicators also are used in various contexts. Some accrediting bodies (e.g., NCATE SPAs) specify the standards and outcomes that programs must meet.
Don’t get sidetracked by terminology. Focus on what you want your students to learn and do. For COE, we must show links to the conceptual framework/KSD and any other national or state standards that apply to our programs (template 3).
Template 3: Align Standards/Outcomes, Conceptual Framework, and Program of Study
What we want students to know, do, and care about
applicable professional standards
You many want to use a simple check () to show the major emphases in your curriculum in Template 3. The Secondary Education Program uses a more detailed system to provide further information about various courses:
A Apply to Practice
P Portfolio Artifact
Template 3, which is a curriculum map or matrix, can help reveal gaps or overemphasis in various areas. Faculty members need time to work together to agree on the basic “must have” requirements for each course, while still leaving room for interests, talents, and creativity that individual faculty members bring to teaching and learning experiences. Think of the course must-have requirements as minimum qualifications for the course. The individuality that each instructor brings makes up the desirable qualifications. Good courses need both.
Some accrediting bodies, such as the Western Association of Schools and Colleges (WASC) require programs to show student learner outcomes in course syllabi. Faculty members should ensure that these outcomes are genuine and doable for themselves and their students. No one needs any more jargon these days.
Next, faculty members must decide how to determine whether students really are learning what they think and hope students are learning. COE programs find that six to eight key assessments (e.g., papers, projects, assignments, exams, internship evaluations) provide the evidence they seek. Practical advice: Begin with assignments that students already complete as part of the program. Using assignments that are embedded in courses makes sense. Instructors or advisors who teach the courses or chair papers/projects can conduct the assessments, or programs might want to assign different or multiple assessors for students’ work. NCATE also requires COE to describe the processes we have adopted to ensure that assessment procedures are fair, accurate, consistent, and free of bias (template 6). The assessments you select to showcase your program (template 4) also go into the during program and completion transition points in Step 2 (template 2).
Template 4: List of 6-8 Key Program Assessments to demonstrate that students are knowledgeable, effective, and caring education professionals
Type or form of assessment
When assessment is administered
Who assesses the work
Once you select your assessments, another grid can help show whether all of your outcomes or standards are examined adequately in you plan (template 5).
Template 5: Relation of Program Standards/Outcomes (template 3) to the 6-8 Key Program Assessments (template 4)
What we want students to know, do, and care about
Template 6: Processes to ensure that program assessment procedures are fair, accurate, consistent, and free of bias
What processes has the program used to ensure that assessment procedures are fair, accurate, consistent, and free of bias? (What, when, how, and with what results?)
Further describing each assessment in detail is important. The College of Education uses the following format, (template 7) in accordance with NCATE SPA program review: