Indirect Assessment Methods
Description: Perception, opinion or attitude of students.
| INDIRECT METHODS | DESCRIPTION | BENEFITS | DRAWBACKS |
|---|---|---|---|
| Interviews | Face-to-face, one-to-one discussions or question/answer session | Provides rich, in-depth information and allows for tailored follow-up questions "Stories" and voices can be powerful evidence for some groups of intended users | Trained interviewers needed Transcribing, analyzing, and reporting are time consuming |
| Focus Group Interviews | Face-to-face, one-to-many discussions or question/answer session | Provides rich, in-depth information and allows for tailored follow-up questions The group dynamic may spark more information--groups can become more than the sum of their parts "Stories" and voices can be powerful evidence for some groups of intended users | Trained facilitators needed Transcribing, analyzing, and reporting are time consuming |
| Percent of time or number of hours/minutes spent on various activities related to a student learning outcome | Students' self-reports or observations made by trained observers on time spent on various activities (e.g. co-curricular; homework; cultural activities) | Information about co-curricular activities and student habits can help programs make sense of results and/or guide them in making decisions about program improvement | Retrospective self-reports may not be accurate |
| Grades given by professors that are not based on explicit criteria directly related to a learning outcome | Grade point averages or grades of students in a program | Data relatively easy to collect | Impossible or nearly impossible to reach conclusions about the levels of student learning |
| Job Placement Data | The percent of students who found employment in a field related to the major/program within one year | Satisfies some accreditation agencies' reporting requirements | Tracking alumni may be difficult |
| Enrollment in higher degree programs | The number or percent of students who pursued a higher degree in the field | Satisfies some accreditation agencies' reporting requirements | Tracking alumni may be difficult |
| Institutional Research Data | Information such as the following: Registration or course enrollment data; Class size data; Graduation rates; Retention rates; Grade point averages | Can be effective when linked to other performance measures and the results of the assessment of student learning (using a direct method) |
| DIRECT METHODS | DESCRIPTION | BENEFITS | LIMITATIONS |
|---|---|---|---|
| Licensure or certification | National exams designed to assess knowledge, skills, and abilities in a field. Passing these exams allow students to practice in their field of study (e.g. NCLEX) | National comparisons Reliability and validity monitored by test developers External organization administer and evaluate | Faculty may be unwilling to make changes to their curriculum if students score low (reluctant to "teach to the test") Test may not be aligned with the program's intended curriculum and outcomes Information from test results is too broad to be used for decision making |
| National exams or standardized test (e.g. CAAP, GRE) | National exams or tests that focus on student learning (tend to be focused on general education) | National comparisons Reliability and validity monitored by test developers External organization administer and evaluate | Students may not take exam seriously Faculty may be unwilling to make changes to their curriculum if students score low (reluctant to "teach to the test") Test may not be aligned with the program's intended curriculum and outcomes Information from test results is too broad to be used for decision making Can be expensive The external organization may not handle administration and evaluation |
| Local Exams | Exams created within an institution (not conducted within a course) | Faculty typically more willing to make changes to curriculum because local exam is tailored to the curriculum and intended outcomes | Students may not take exam seriously They are not motivated to do their best Campus or program is responsible for test reliability, validity, and evaluation |
| Course embedded testing or quizzes | Tests or quizzes created for and administered to particular courses (e.g. Psych 101 final exam) | Students motivated to do well because test/quiz is part of their course grade Evidence of learning is generated as part of normal workload | Faculty members may feel that they are being overseen by others, even if they are not |
| Course embedded assignment | The program selects course assignments ("signature assignments") that can provide information on a student learning outcome. Students complete these assignments as a regular part of the course. Assignments are scored using criteria or a scoring rubric and these scores are used for program-level assessment. | Students motivated to do well because assignment is part of their course grade Faculty members more likely to use results because they are active participants in the assessment process Online submission and review of materials possible Data collection is unobtrusive to students | Faculty members may feel that they are being overseen by others, even if they are not Faculty time required to develop and coordinate, to create a rubric to evaluate the assignment, and to actually score the assignment |
| Portfolio | A collection of student work such as written assignments, personal reflection, and self-assessments | Provides a comprehensive, holistic view of student achievement and/or development over time Students can see growth as they collect and reflect on the products in the portfolio Students can draw from the portfolio when applying for graduate school or employment Online submission and review of materials possible | Amount of resources needed: costly and time consuming for both students and faculty Students may not take the process seriously (collection, reflection, etc.) Accommodations need to be made for transfer students (when longitudinal or developmental portfolios are used) |
| Pre/Post-Test | Students take the pre-test as part of a required, introductory course and take the post-test during their senior year (often in a required course or capstone course) | Provides "value-added" or growth information | Increased workload to evaluate students more than once Designing pre- post-tests that are truly comparable at different times is difficult Statistician may be needed to properly analyze results |
| Employer's or internship supervisor's direct evaluations of student's performances | Evaluation or rating of student performance in a work, internship, or service-learning experience by a qualified professional | Evaluation by a career professional is often highly valued by students Faculty members learn what is expected by community members outside this institution | Lack of standardization across evaluations may make summarization of the results difficult |
| Observation of student performing a task | Professor or an external observer rates each students' classroom discussion participation using an observation checklist | Captures data that is difficult to obtain through written texts or other methods | A trained, external observer (not the course instructor) to collect data is recommended, which may cost money and/or require the willingness of faculty members to observe colleagues' courses and allow observations of their class Some may believe observation is subjective and therefore the conclusions are only suggestive |
| Culminating project: capstone project, senior theses, senior exhibit, senior dance performance | Students produce a piece of work or several pieces that showcase their cumulative experiences in a program | Provides a sophisticated, multi-level view of student achievement Students have the opportunity to integrate their learning | Creating an effective, comprehensive culminating experience can be challenging Faculty time required to develop evaluation methods (multiple rubrics may be needed) |
| Student publications or conference presentations | Students present their research to an audience outside their program | Gives students an opportunity to practice being a professional and receive feedback from career professionals or community members | Cost of travel, time, resources |
Adapted from University of Hawaii at Manoa.