Advances Vol. 2, No. 2 Assessment of Student Learning in SocialWork Education: The Indiana Model Barry R. Cournoyer Abstract: In this paper, the author discusses assessment of student learning in light of evolving accreditation standards. The author describes the Indiana Model—a com- prehensive approach that includes: (a) a Course-Learning Objectives (CLO) classifi- cation system to organize and analyze the total array of course learning objectives addressed in a curriculum, (b) a direct Assessment of Student Learning system to demonstrate student learning outcomes, and (c) an indirect Assessment of Student Learning system to provide for the perspectives of consumers and other stakeholders. When integrated, the three systems may be used for curriculum analysis and devel- opment, assessment of student learning, and program evaluation—particularly in terms of student learning outcomes. The proposed integrated approach to student learning assessment addresses both university and professional accreditation stan- dards. Keywords: Assessment, student learning, curriculum analysis, social work educa- tion, accreditation Many schools and departments of social work struggle with issues related to curriculum analysis, program evaluation, and assessment of student learning. Accreditation standards of the Council on Social Work Education (CSWE) require programs to “specify the outcomemeasures andmeas- urement procedures that are to be used systematically in evaluating the program, and that will enable it to determine its success in achieving the desired objectives” (Commission on Accreditation [COA], 1994). The recently adopted Educational Policy and Accreditation Standards (EPAS) suggest that academic programs become even more active in assessment and evaluation. Consider Accreditation Standard 8: Program Assessment and Continuous Improvement : • The program has an assessment plan and procedures for evaluating the outcome of each program objective. The plan specifies the measurement procedures and methods used to evaluate the outcome of each program objective. • The program implements its plan to evaluate the outcome of each pro- gram objective and shows evidence that the analysis is used continuously to affirm and improve the educational program (CSWE, 2001). 128 Barry R. Cournoyer, D.S.W., L.C.S.W. is Professor, Indiana University School of Social Work, Indianapolis, Indiana 46202-5156. Copyright© 2001 Advances in SocialWorkVol. 2 No. 2 (Fall 2001) 128-151. Indiana University School of SocialWork. 129 National and regional university accreditation associations have also become more rigorous in their requirements for the assessment of student learning out- comes. All university regional accreditation bodies emphasize the assessment of student learning and the incorporation of quality improvement principles in edu- cational and program evaluation. Many social work programs throughout the country have begun to consider the means to assess student learning. Indeed, most programs are sincerely motivated to develop and implement processes bywhich to evaluate progress toward accom- plishment of their mission and goals, including those that refer to student learn- ing, and then to use the results to improve outcomes. The issues may be captured in the question: “How do we assess students’ learning in an efficient manner that coincides with accreditation standards, helps us improve the quality of our cur- riculum and instruction, and enables us to document optimal student learning outcomes?” In this paper, the author addresses this question by describing: • The elements of aCourse-LearningObjectives (CLO) classification system that facilitates organization and analysis of the total array of course learn- ing objectives addressed throughout a social work curriculum. • A direct Assessment of Student Learning system to document student- learning outcomes. • An indirect Assessment of Student Learning system to gather consumer and stakeholder generated information. • Themeans bywhich theCourse-LearningObjectives (CLO) classification, and the direct and indirect Assessment of Student Learning systems may be integrated to contribute to the evaluation of academic programs, assessment of outcomes, and as part of the means by which to address CSWE and university accreditation standards. • The implications of the comprehensive and integrated approach for social work programs. Cournoyer/ASSESSMENT OF STUDENT LEARNING Figure 1. Illustrates the main components of the Indiana Model—an integrated approach to the assessment of student learning outcomes1. 130 ADVANCES IN SOCIALWORK COURSE-LEARNINGOBJECTIVES (CLO) CLASSIFICATION SYSTEM: CONCEPTUALOVERVIEW All social work schools and departments have goals and objectives. Whether explicit or implicit, their missions, visions, values, and goals become manifest by the words and the actions of the faculty and staff. Some goals are highly abstract (e.g., statements of mission and vision) and others are quite descriptive and oper- ational (e.g., course learning objectives). The goals of BSW, MSW, or doctoral pro- grams (i.e., program objectives) tend to fall approximately in the middle—not as abstract as a school or university mission but not as descriptive or operational as individual course learning objectives. Ideally, all goals and objectives should reflect conceptual congruence and logical interrelationships. A review of a curriculum, for example, should readily reveal connections from the most abstract to the most descriptive goals and objectives, and vice-versa—from the most concrete to the most general. Optimally, a review- er should be able to link all individual course-learning objectives to at least one program objective, at least one school or departmental goal, and at least one dimension of the organizational mission. Conversely, several lower level goals and objectives (sub-goals) for each highly abstract goal should be apparent. This is, of course, a daunting challenge. However, it is worth undertaking because the major indication of an organization’s success or failure centers upon the degree to which it achieves its goals and objectives. Optimally, when students achieve a course- learning objective, their learning should simultaneously contribute to the achieve- ment of one ormore program objectives and some aspect of the school or depart- mental mission. To support this claim, however, each course-learning objective must clearly link to and support higher-level goals and objectives. A course-learning objectives (CLO) classification system represents a means by which social work programs may analyze their curriculum in light of higher-level goals. For example, suppose a school of social work offered educational programs leading to the BSW, MSW, and Ph.D. degrees. Each program offers several class- room or practicum courses, or other educational experiences that address several learning objectives. A school that offers a full continuummight offer 50 or 60 dis- tinct courses or seminars, each of which might have seven to 12 learning objec- tives. Collectively, the school might have as many as 500 or 600 discrete course- learning objectives. Ideally, all professors, students, and relevant stakeholders should carefully read and reflect upon themeaning and implication of each learning objective offered in all courses and seminars. A simple “eyeballing” process can be extraordinarily revealing! However, a formal classification system facilitates organization and sys- tematic analysis. A small group of faculty and stakeholders could classify each dis- crete learning objective according to higher-level program objectives, school goals, and other relevant factors. Computer software programs (e.g., database, spreadsheet, or some statistical packages) may facilitate the classification process and, of course, contribute to subsequent data analysis. In light of emerging accreditation standards and growing expectations for greater accountability, a programmight classify each course-learning objective in terms of the following dimensions: 131 • School or Department Mission-Related Goals • ProgramObjectives (BSW,MSW [foundation & concentration], Ph.D.) • CSWE Content Areas (EPAS) • CSWE Foundation Objectives (EPAS) • ASWB Examination Content Domains • Principles of Cultural Competence • Levels of Bloom’s Taxonomy Mission Related Goals Schools and departments of social work typically create and publish formal state- ments of mission. A mission statement is a declaration of an organization’s raison d’être. A good mission statement answers key questions such as “What is our pri- mary purpose?” “What is our reason for being?”What are our primary activities?” Written in the present tense, themission statement describeswhat is and serves as a general guide for action and decision-making. The primary reason that most schools and departments of social work exist is to promote learning, especially by students, but also by faculty, that enables gradu- ates to provide high quality social services. The vision statement addresses the same questions albeit in the future tense. The vision describes what the organiza- tion wants to become in the future—typically five or so years hence. Some organ- izations also establish a set of values or principles that serve as moral and ethical guideposts for operational activities. Together, the mission and vision statements lead to school or departmental goals. Derived from the mission and vision state- ments, organizational goals are anticipated outcomes or accomplishments rather than activities or processes. Although described in outcome fashion, organiza- tional goals remain fairly general. These abstract statements and goals help clarify the major purposes and func- tions of the organization aswell as its direction for the future. Typically, statements of mission, vision, values, and goals relate to aspects of the well-known trinity of academic life: teaching, service, and research or scholarship. In schools and departments of social work, student learning usually receives a great deal of prominence within the teaching dimension. A mission statement might include reference to an aspect of student learning such as, “We educate students for competent, ethical, and effective social work service in the 21st century.” A vision statement might refer to student learning in this manner: “We aspire to offer educational experiences that prepare students to adapt to emerging knowledge and changing circumstances through continuous processes of learning, unlearning, and learning anew.” An organizational goal might include reference to both mission and vision statements by indicating that graduating students demonstrate the abilities to: • Think critically and analyze contemporary research studies for applica- tion in practice. • Engage in self-assessment of their learningneeds, developpersonal learn- ing plans, and implement those plans in an active, self-directedmanner. Cournoyer/ASSESSMENT OF STUDENT LEARNING 132 ADVANCES IN SOCIALWORK The school or departmental goals link directly to and support the mission and vision. In social work education, university, and campus expectations, the policies and standards of CSWE, professional principles and values, student and faculty aspirations, and the nature of community needs also inform the identification of organizational goals—which become a fundamental component of the Course Learning Objectives classification system. Program Objectives Each academic program (e.g., BSW,MSW, Ph.D.) develops program objectives that guide curriculum planning, development, and implementation. They link directly to one or more mission-related organizational goals. The specific standards and policies of CSWE (COA, 1994; CSWE, 1992, 2001) serve to guide, but not necessar- ily constrain, the development of BSWandMSWprogramobjectives. Althoughnot required, some social work doctoral programs refer to CSWE policies as part of the process of identifying higher-level program objectives. Academic program objectives tend to be less abstract andmore descriptive than organizational goals, but not nearly as specific as course learning objectives. They, too, appear as outcomes or accomplishments rather than as activities or process- es. Faculty in a baccalaureate program might, for example, identify the following as a program objective: “Graduates of the program are able to provide competent, ethical, effective, and ethnically-sensitive generalist social work services to a diverse range of individuals, families, groups, organizations, and communities.” CSWE Content Areas The CSWE Curriculum Policy Statement (1992) and the Standards for Accreditation (1994) require social work programs to address certain content areas. The newly adopted EPAS2 include the following: 1. SocialWorkValues and Ethics 2. Diversity 3. Populations-at-Risk and Social and Economic Justice 4. Human Behavior and the Social Environment 5. SocialWelfare Policy and Services 6. SocialWork Practice 7. Research 8. Field Education. (CSWE, 2001) Although many social work educational programs organize their curriculums into sequences that address required content areas (e.g., the policy sequence, the practice sequence, the research sequence),many donot. Someprograms integrate knowledge, values, and skills from several content areas within each course, sem- inar, practicum, or other learning experience. Indeed, even within highly struc- tured, formally sequenced curriculums, many course-learning objectives con- tribute to students’ learning in several CSWE-required content areas. Virtually all social work practice courses include information about human behavior and the social environment, and many incorporate research content and skills as well. Most courses consider at-risk populations, social and economic justice, and some aspects of diversity. Furthermore, knowledge about values, ethics, laws, and the skills of critical thinking and ethical decision-making are integral to courses throughout the social work curriculum. Classifying course-learning objectives by CSWE content area may be particularly useful as programs experimentwith innovative teaching and learning approaches. Integrative orientation, field practicum, and capstone experiences frequently enable students to learn throughout many and sometimes all of the eight content areas. Programs that clearly and precisely identify the curriculum location of expected learning within these domains may easily demonstrate coverage of CSWE-required foundation content. CSWE Foundation Objectives The EPAS (CSWE, 2001) require BSW and MSW social work programs to address certain foundation program objectives: The professional foundation, which is essential to the practice of any social worker, includes, but is not limited to, the following program objec- tives. Graduates demonstrate the ability to: 1.Apply critical thinking skills within the context of professional social work practice. 2.Understand the value base of the profession and its ethical standards and principles, and practice accordingly. 3.Practicewithout discrimination andwith respect, knowledge, and skills related to clients’ age, class, color, culture, disability, ethnicity, family structure, gender,marital status, national origin, race, religion, sex, and sexual orientation. 4.Understand the forms andmechanisms of oppression and discrimina- tion and apply strategies of advocacy and social change that advance social and economic justice. 5.Understand and interpret the history of the social work profession and its contemporary structures and issues. 6.Apply the knowledge and skills of generalist social work practice (or, for MSW graduates, “a generalist social work perspective”) with systems of all sizes. 7.Use theoretical frameworks supported by empirical evidence to under- stand individual development and behavior across the life span and the interactions among individuals and between individuals and fami- lies, groups, organizations, and communities. 8.Analyze, formulate, and influence social policies. 9.Evaluate research studies, apply research findings to practice, and eval- uate their own practice interventions. 10.Use communication skills differentially across client populations, col- leagues, and communities. 133Cournoyer/ASSESSMENT OF STUDENT LEARNING 134 ADVANCES IN SOCIALWORK 11.Use supervision and consultation appropriate to social work practice. 12.Functionwithin the structure of organizations and service delivery sys- tems and seek necessary organizational change. (CSWE, 2001) These programobjectives constitute the required foundation learning objectives for both BSW andMSW academic programs. Although accredited MSW programs educate students for advanced practice within an area of concentration, they also must help all studentsmaster the foundation curriculum. Therefore, classification of course-learning objectives by foundation objectives is useful for both BSW and MSW programs. Social work doctoral programs may find them useful as a stimu- lus for the development ofmore advanced program and learning objectives. As do a large number of BSW and MSW programs, many D.S.W. or Ph.D. programs in social work develop sets of objectives, competencies, or abilities that graduating students are expected to demonstrate. These program-specific objectives comple- ment those required by CSWE and contribute to the unique identity and mission of the school or department. For purposes of CLO classification, they may be added to the array of foundation (and MSW concentration) objectives or consid- ered within a separate dimension. ASWB Examination Content Domains Almost all states, one territory, and one Canadian province have adopted the Association of SocialWork Boards (ASWB) sponsored social work examinations for the purposes of licensing or certification. Although there are many areas of com- mon interest, the goals of academic programs vary somewhat from those of pro- fessional associations and licensing boards. These divergent functions provide healthy tension and often lead to useful conversations among the organizations. Of course, schools and departments of social work should never “teach to the test.” Nevertheless, some programs may decide to classify course-learning objectives according to the content domains addressed within the ASWB Basic or Intermediate Examinations in order to provide additional perspective. The pub- licly disseminated domains reflect the findings from large studies of current prac- tices of professional social workers from throughout the United States (and soon fromat least oneprovince of Canada aswell).The resulting content outlines, based as they are upon studies of practicing social workers, may complement the poli- cies of CSWE. The ASWB Basic Examination is typically required of BSW graduates; the Intermediate Examination of MSW graduates. Advanced or Clinical Examinations are required of MSW graduates with supervised post-graduate practice experi- ence. The current ASWB Basic Examination addresses the content domains (ASWB, 2001)3 listed in Table 1. Principles of Cultural Competence in SocialWork The changing demographics, composition, and globalization of society require that all social workers understand, value, and demonstrate cultural competence in their service to others. Because of the extraordinary significance of multi-cultural abili- ties, many social work programs may decide to emphasize their importance 135 through the classification of course learning objectives according to certain dimensions of cultural competence. Anderson, Richardson, and Leigh (Leigh, 1998) identified seven principles of the culturally competent social worker. They posit that the culturally competent social worker would agree with the following statements: 1. I accept the fact that I have much to learn about others. 2. I have an appreciation of the regional and geographical factors related to people of color and contrasting cultures, how the individual may vary from the generalizations about their regional and geographical group, and how regional groups vary from the total cultural group. 135Cournoyer/ASSESSMENT OF STUDENT LEARNING 010 Human development and behav- ior 011 Theoretical approaches to under- standing individuals, families, groups, communities, and organi- zations 012 Human growth and development 013 Human behavior in the social environment 014 Impact of crises and changes 015 Abnormal and addictive behav- iors 016 Dynamics of abuse and neglect 020 Diversity 021 Effects of diversity 030 Assessment in social work prac- tice 031 Social history and collateral data 032 Use of assessment instruments 033 Problem identification 034 Effects of the environment on client behavior 035 Assessment of client strengths and weaknesses 036 Assessment of mental and behav- ioral disorders 037 Indicators of abuse and neglect 038 Indicators of danger to self and others 040 Social work practice with individ- uals, couples, families, groups, and communities 041 Theoretical approaches and mod- els of practice 042 The intervention process 043 Components of the intervention process 044 Matching intervention with client needs 045 Intervention techniques 046 Intervention with couples, fami- lies, and groups 047 Intervention with communities 048 Professional use of self 049 Use of collaborative relationships in social work practice 050 Interpersonal communication 051 Theories and principles of com- munication 052 Techniques of communicating 060 Professional social worker/client relationship 061 Relationship concepts 062 Relationship practice 070 Professional values and ethics 071 Responsibility to the client 072 Responsibility to the profession 073 Confidentiality 074 Self-determination 080 Supervision in social work 081 Educational functions of supervi- sion 082 Administrative functions of super- vision 090 Practice evaluation and the uti- lization of research 091 Methods of data collection 092 Research design and data analysis 100 Service delivery 101 Client rights and entitlements 102 Implementation of organizational policies and procedures 110 Social work administration 111 Staffing and human resource management 112 Social work program manage- ment Table 1: Association of Social Work Boards (ASWB) Basic Examination Content Domains 136 ADVANCES IN SOCIALWORK 3. I follow the standard that knowledge is obtained from the person in the situation and add tomy learning about the situation from that per- son before generalizing about the group-specific person. 4. I have the capacity to form relationships with people from contrasting cultures in social, work, and professional relationships. 5. I can engage in a process characterized by mutual respect and con- scious effort to reduce power disparities between myself and persons of minority status. 6. I have the ability to obtain culturally relevant information in the pro- fessional encounter. 7. I have the ability to enter into a process of mutual exploration, assess- ment, and treatment with people of contrasting culture and minority status in society. (Leigh, 1998, pp. 173-174) These seven principles could be converted into characteristics, attitudes, or abilities and incorporated within the course learning objective classification scheme. As a supplement or substitution for Anderson, Richardson, and Leigh’s principles, programsmay prefer to adopt the recently publishedNASWStandards of Cultural Competence in SocialWork (2001) as part of their classification system. NASW organizes their conception of cultural competence into 10 standards that could represent categories for classification: 1.Ethics andValues 2.Self-Awareness 3.Cross-Cultural Knowledge 4.Cross-Cultural Skills 5.Service Delivery 6.Empowerment and Advocacy 7.DiverseWorkforce 8.Professional Education 9.Language Diversity 10.Cross-Cultural Leadership Bloom’s Taxonomy of Cognitive Educational Objectives In addition to organizational,mission-related goals, academic programobjectives, CSWE content areas, foundation (and concentration) objectives, alongwith program specific competencies or abilities, ASWB examination content domains, and principles of cultural competence, each course-learning objectivemaybe classified according toBloom’sTaxonomyofCognitive EducationalObjectives. Several decades ago, Benjamin Bloom and a group of educational psychologists (Bloom & Krathwohl,1956)developedataxonomyofcognitive learningobjectives.Thetaxonomy remains pertinent today, andmay be used formultiple purposes4. The six hierarchical levels represent a useful scheme for classification of course learning objectives within a socialwork curriculum. In ascendingorder of cognitive complexity, the six levels are: 137 Level One: Recollection. The ability to recall is the basic level of learning and refers simply to the ability to remember material such as facts and basic theoreti- cal terms and concepts. Level Two:Comprehension. The ability to comprehend refers to an understand- ing of the material. This is often demonstrated by providing an explanation, sum- mary, or interpretation of the material. Level Three: Application. The ability to apply knowledge refers to use of the material in a particular situation. In social work, for example, this ability might be demonstrated through the application of a practice skill in service to a client. Or, a social worker might use a theoretical concept to better understand a particular person-and-situation dynamic. Application refers to the use of rules, methods, and principles outlined in the material. Level Four: Analysis. The ability to analyze involves the careful examination of the various elements of the material. Relationships among and between compo- nents are critically considered in terms of organizational structure and internal coherence. Level Five: Synthesis. The ability to synthesize includes pulling together ele- ments in a new way to form an innovative structure. The creation of a new con- ceptual model could be a form of synthesis. Level Six: Evaluation. The ability to evaluate involves the determination of the relative value of knowledge for a defined purpose. Typically, this would include the creation, adoption or adaptation, and application of evaluative criteria. COURSE-LEARNINGOBJECTIVES CLASSIFICATION (CLO) SYSTEM: APPLICATION Tables 2 through 4 reveal how an individual course objective may be identified, coded, and categorized within a CLO classification system as described above. Table 5 reflects how the classified course objective appears as a row within a spreadsheet or database. Of course, all course objectives throughout the entire curriculum require classification and entry into the system. Furthermore, pro- grams may add additional categories to meet organizational needs. Table 6 illustrates a hypothetical distribution of an academic program’s Course LearningObjectives as classified in accordancewith Bloom’s taxonomic levels.The table reveals the potential value of the CLO classification for curriculum analysis and development. The example (see Table 6) suggests that 82.5% of the classified course objectives address the first (recall), second (comprehension), and third (application) levels of Bloom’s taxonomy. According to Table 6, the curriculum reflects only modest attention to higher level cognitive abilities (analysis, synthe- sis, and evaluation) that aremost associated with critical thinking. If the classifica- tion system is reasonably valid and reliable, faculty may decide to add or revise selected course-learning objectives in order to strengthen students’ learning at higher levels of cognitive learning. Assessment of Student Learning Emerging professional (e.g., CSWE) and regional university accreditation stan- dards (e.g., New England Association of Schools and Colleges, Middle States Cournoyer/ASSESSMENT OF STUDENT LEARNING 138 ADVANCES IN SOCIALWORK Association of Colleges and Schools, North Central Association of Colleges and Schools, Western Association of Schools and Colleges, Northwest Association of Schools and Colleges, Southern Association of Colleges and Schools) require aca- demic programs to evaluate student-learning outcomes. Clearly, student-learning assessment is a major component of accreditation expectations. However, our conceptions of learning and student-learning assessment require elaboration. Consider the following as working definitions: Student learning is growth in knowledge, values, and skills that occurs as the result of learning activities and experiences. Student-learning assessment includes the tools and processes used, and the findings obtained from intentional efforts to appraise students’ growth in Course Course Course Sect. Year Term Status Program Learning Title No. No. Level Objective Fall=1 BSW=1 Statement Spring=2 Req’d.=1 MSW Found=2 Summer 1=3 Elect.=2 MSW Conc=3 Summer2=4 MSW Conc=4 MSW Conc=5 MSW Conc=6 MSW Conc=7 Ph.D.=8 01. Social 501 342 01 1 1 2 Understand Work the Practice I fundamental values, ethics, and legal obligations of the profession Table 2: Course Learning Objectives (CLO) Classification System Framework—Part One Course Learning Syllabus Unique CLO School Goal Program Objective Objective Learning Number Statement Objective 1xx=BSW Number 2xx=MSW 3xx=Ph.D. 01. Understand the 01 5013420111201 4. Prepare ethical, 205. Demonstrate fundamental values, effective social commitment to ethics, and legal workers that the values and obligations of the reflect personal ethics of social work. profession. and professional integrity in all aspects of their service to others. Table 3: Course Learning Objectives (CLO) Classification System Framework—Part Two 139 knowledge, values, and skills that occurs as the result of learning activities and experiences. Students reflect evidence of learning when, for example, they becomemore pro- ficient in interviewing, assessment, intervention, and evaluation skills as a result of social work educational experiences. Professors engage in a student-learning assessment when they examine students’ videotaped interviews and apply a well- designed scoring rubric to evaluate students’ performance of clearly identified interviewing skills. Student-learning assessment occurs at the conclusion of their classroom or practicum courses when students complete questionnaires about the extent to which they have gained the knowledge and skill necessary for com- petent professional service. Feedback from focus groups of graduates’ employers represents a form of student-learning assessment. Programs engage in student- learning assessment when, for instance, they analyze the individual and aggregat- ed results of a qualifying examination that all students complete at the conclusion Cournoyer/ASSESSMENT OF STUDENT LEARNING Course CSWE CSWE ASWB Principle Level of Learning Content Foundation Content of Cultural Bloom’s Objective Area Objective Domain Competence Taxonomy Statement (01. 1. Social 2. Understand 070. 3. I follow 2. Understand Understand Work Values the value base Professional the standard the the and Ethics. of the values and that meaning fundamental profession and ethics. knowledge and values, ethics, its ethical is obtained relevance and legal standards and from the of obligations of principles, and person in knowledge the profession. practice the situation and accordingly. and add to information my learning (e.g., about the comprehend, situation interpret, from that explain, and person summarize). before generalizing about the group- specific person. Table 4: Course Learning Objectives (CLO) Classification System Framework—Part Three Unique School Program CSWE CSWE ASWB Principle Level of CLO Goal Objective Content Foundation Content of Bloom’s Number Area Objective Domain Cultural Taxonomy Compe- tence 5013420111201 4 205 1 2 070 3 2 Table 5: Course Learning Objectives (CLO) Data Analysis System—Spreadsheet, Statistical Package, or Database 140 ADVANCES IN SOCIALWORK of their foundation studies. Student-learning assessmentmay occur during a cap- stone seminar, when students and faculty systematically evaluate the quality of social work portfolios submitted as a requirement for graduation. Schools and departments may view assessment of student learning as a form of program evaluation, educational assessment, quantitative or qualitative research, or an aspect of Continuous Quality Improvement or Total Quality Management. The nature and forms of assessment are wide and surprisingly flexible. Programs may focus on inputs, outputs, or outcomes andmay do so through direct and indi- rect forms of assessment (see Table 7). Input Assessment tends to focus on the resources or “raw materials” of the school or program (e.g., quality of faculty, incoming students, curriculum design, or programmatic resources). Output Assessment usually addresses the productiv- ity of the organization (e.g., numbers of graduating students and graduates, grad- uation and retention rates, number of teaching awards, amount of research). Outcome Assessment involves consideration of the effects of organizational activ- ities upon those intended to benefit from them. In the context of social work edu- cation, we hope that successful students who complete the coursework and earn degrees benefit in some tangible way from those experiences (e.g., increase their knowledge and skill, secure employment, feel competent in their professional lives, and effectively serve clients). Most schools and programs are familiar with the collection and analysis of inputs and outputs. Programs regularly report on the average SAT or GRE scores of admitted students, acceptance and retention rates, the numbers of graduates, GPA averages, the amount of external funding, the faculty-to-student ratios, the number and kind of faculty publications, the amount of physical space, the num- ber of library volumes, the amount of secretarial support, and other aspects of inputs and outputs. Assessment of student learning outcomes is less common. However, it is precisely the area that accrediting bodies and other stakeholders increasingly emphasize. Indeed, assessment of student learning outcomes appeals to many social work faculty who regularly teach students to evaluate clients’ progress toward goal achievement. Such perspectives are highly compat- ible with quality improvement initiatives. Direct Assessment of student-learning outcomes involves examining students’ or graduates’ knowledge, attitudes, and skills (e.g., depth or breadth and expertise) through observation and evaluation, or through valid and reliable measures. Indirect Assessment also seeks to determine students’ or graduates’ knowledge Total Level 1: Level 2: Level 3: Level 4: Level 5: Level 6: Total Number Recall Compre- Apply Analyze Synthesize Evaluate Classi- hend fied Objective 200 80 150 100 60 5 5 400* Number (20%) (37.5%) (25%) (15%) (1.2%) (1.2%) (100%) and Percent Table 6: Hypothetical Example of Number and Percent of a Program’s Course Learning Objectives (CLO) Classified by Bloom’s Taxonomy of Cognitive Learning 141 and expertise but does so through the collection of data presumed to be associ- ated with actual understanding and performance (e.g., self-reports, faculty and field instructor evaluations of students, letter grades, focus groups or surveys of employers’ views of graduates’ performance, evidence of graduates’ practice effectiveness in service to clients). Direct Assessment of Student Learning Direct assessment of learning usually involves the systematic evaluation of the performance or products of students or graduates. There are many kinds and forms of available evidence. Doctoral, masters’, or senior theses; qualifying exam- inations; standardized tests; scores on licensing examinations; videotapes of actual or simulated interviews; results of single-subject research; and social work portfolios represent forms of evidence that may be used to assess student learn- ing outcomes. Ideally, the products and performances relate to learning goals at one or more organizational levels (e.g., school or departmental goals, program objectives, or course learning objectives). Three forms of direct assessment that may be especially useful for social work programs are: 1) entry and exit essays, 2) pre- and post-exams, and 3) social work learning portfolios. Entry and Exit Essays Many programs require applicants to prepare an essay, perhaps in the formof an autobiographical statement, as part of the admissions process. Such essaysmaybe used by screening committees to consider the readiness of the candidate for the nature and rigor of the academic program. They may also become useful for assessment purposes in that applicants complete them before beginning the pro- gram. A similar kind of essay could be expected just before graduation. Comparisons between the two essays might well reveal areas of growth, knowl- edge, and expertise. The nature of the essay assignment and the assessment criteria used to assess them should be well constructed to match school goals and program objectives to best serve the function of student-learning assessment. For instance, rather than an autobiographical statement, applicants might be provided a case sce- nario to analyze. Towards the end of the program, graduating students could be asked to repeat the process with an analogous case situation. Pre- and Post-Instruments In addition to or insteadof entry and exit essays, pre- andpost-instrumentsmight also be adopted (e.g., the Baccalaureate Educational Assessment Package [BEAP]) or developed for use as evidence of growth and learning. The instruments would require careful analysis in order to determine validity, reliability, and relevance to Cournoyer/ASSESSMENT OF STUDENT LEARNING Assessment Direct Indirect Inputs Outputs Outcomes Table 7: Student Learning Assessment Matrix the program objectives. And, assessment guidelines should match the program’s purposes, goals, and focus. SocialWork Portfolios Portfolios are becoming increasingly popular both within social work and other disciplines as well. Portfolios may be course specific or may apply to an entire program of study (e.g., BSW, MSW, Ph.D.). Program-oriented portfolios reflect tremendous potential for assessment of student learning because they incorpo- rate samples of students’ work throughout various stages of the educational process. As a central part of the development of program-oriented portfolios, some schools and departments encourage students to prepare learning con- tracts, learning goals, and learning plans at the beginning of their studies. The contracts then help students and faculty to guide the selection of courses, and the emphasis that is placed within the classroom and practicum experiences. They also serve as contexts for the assessment of the final portfolios submitted as a requirement for graduation. The portfolios include carefully selected learning products that serve as tangible evidence for the direct assessment of learning. In the aggregate, evaluation of graduating students’ portfolios would ideally reveal that most, if not all, demonstrate mastery of program objectives as well as reflect progress toward achievement of their individual learning goals (Cournoyer, 2001; Cournoyer & Stanley, 2002). Indirect Assessment of Student Learning Indirect forms of student-learning assessment also have considerable value, especially when used in conjunction with direct evidence. Indirect assessment tends to yield opinions about, rather than demonstration of, knowledge, values, and skills. Nonetheless, the views and experiences of consumers and stakehold- ers are at least as important and sometimes more important than scores on exams or grades on papers. Most schools and departments of social work have faculty that are well trained to develop surveys, conduct focus groups, and collect and analyze data. These are well-established forms of indirect assessment. The famous, or infamous, end-of-semester “course evaluation” is one that might be adapted for the indirect assessment of student learning. Professors’ and instruc- tors’ performance tend to be the focus of the items presented in traditional course evaluations. However, they can be easily adapted to assess student learning. Assessing Student Learning through Course Evaluations Traditionally, students enrolled in schools and departments of social work have completed end-of-semester questionnaires to evaluate the general quality of courses and their instruction. Although the format varies somewhat from univer- sity to university, course evaluations are widely used—both for personnel evalu- ation purposes (i.e., promotion, tenure, salary increments) and as a measure of student satisfaction. Course evaluations may take many forms. In some pro- grams, professors design their own instruments. In others, professors may select items from a “cafeteria” system provided by the university. Some programs require that professors use a standard or “common” course evaluation, which enables the social work school or department to analyze students’ opinions of courses and professors through comparison to average scores. 142 ADVANCES IN SOCIALWORK Common Items. Programs derive many benefits from the use of standard or common items in end-of-semester evaluation instruments. Students in all social work courses respond to these standard items, potentially yielding a rich data set for statistical analysis. Traditionally, most common items within course evalua- tion questionnaires tend to elicit students’ opinions about the characteristics and behaviors of the professor and the nature of the course. Items that refer to the professor’s accessibility, preparedness, fairness in grading, and ability to commu- nicate are typical, as are items related to the utility of textbooks, examinations, and assignments. Most course evaluation instruments include few, if any, items about students’ learning. Table 8 contains items that are representative of those commonly used in universities throughout North America. All items in this instrument (Table 8) refer to the course or the instructor. Indeed, they are quite useful if faculty or administration want data for personnel or performance evaluation. The items are consistent with traditional pedagogical approaches to education where the focus is more upon the quality of teaching performance than on the nature or quality of student learning. [See Barr & Tagg (1995) for a comparison of the “teaching” and “learning” paradigms in higher education]. However, programs may easily convert end-of-semester course eval- uation forms into an assessment of student learning instruments. Instead of or in addition to asking students what they think about the professor and the course, we could inquire about their learning. Items could reflect a greater emphasis upon student learning. Consistent with research findings about effective teaching and learning (Chickering & Gamson, 1987) and trends toward active, adult learn- ing, programs might develop end-of-semester student-learning assessment instruments that contain items such as those presented in Table 9. Programs could also add to these general items those that correspond to the course learning objectives for each social work classroom or practicum course or seminar offered throughout the school or department. In effect, this requires the development of individualized Assessment of Student Learning instruments for each course or seminar. However, it provides students an opportunity to indicate the degree to which they believe they learned what they might reasonably expect to learn—based upon the learning objectives outlined in the course or seminar syllabus. Table 10 illustrates how learning objectives from a social work course might appear as items within an end-of-semester course evaluation instrument. Students, of course, respond to these items based on their perceptions of learn- ing. Undoubtedly, various factors (e.g., nature of the course, rigor, grading poli- cies, and the characteristics of the instructor) affect their responses. This approach clearly represents an indirect rather than a direct measure of learning. Nonetheless, when combined with direct evidence of student learning (e.g., stan- dardized testing, student portfolios, qualifying examinations, theses, or scores on licensing exams), they represent a powerful source of assessment information from the consumers’ perspective. In a manner consistent with the principles of Continuous Quality Improvement (CQI), students assess the degree of their own learning. 143Cournoyer/ASSESSMENT OF STUDENT LEARNING 144 ADVANCES IN SOCIALWORK Statistical analysis of aggregated responses to both common items (CI) and course-learning objective (CLO) items may yield pertinent information about students’ perceptions of learning by course, sequence, academic status (i.e., jun- ior, senior, MSW-foundation, MSW-concentration), and other pertinent factors. For example, Table 11 reflects the Common Item Scale (CIS) scores (i.e., aggregat- ed averages) of a hypothetical set of common items (such as those presented in Tables 8 or 9) by course and year. Scores may range from 1 (reflecting strong dis- agreement) to 5 (reflecting strong agreement). In this example, all items appear in an affirmative fashion so that higher scores consistently reflect stronger agree- ment. The table illustrates the kind of assessment information that various con- stituencies may receive. The data indicate that some courses reflect similar rat- ings over a four-year period, while others vary considerably. Of course, the analy- sis of findings and the way they are, or are perceived to be, used are central to the success or failure of any assessment process. In general, readers should view “average” ratings with caution. Despite the limitations, descriptive statistics serve a function as “indicators” of courses or curriculum areas that deserve closer attention. For example, in Table 11, the CIS scores associated with Social Work Course #1 reveal a substantial drop in students’ evaluation of the course during the 1996 and 1997 years and a fair recovery during 1998. The CIS scores associat- ed with Social Work Course #2 reveal a continuously declining trend from 1995 through 1998, while those connected with SocialWork Courses #3 and #5 reflect a more favorable, upward trend during the same period. CIS scores for SocialWork Course #4 are consistently positive throughout the entire period. 1 SA A U D SD This course is well-described. 2 SA A U D SD This course has clearly stated goals. 3 SA A U D SD The course assignments contribute to the quality of the course. 4 SA A U D SD The course text(s) are well-chosen. 5 SA A U D SD The course contributes to my professional development. 6 SA A U D SD The instructor is knowledgeable about course content. 7 SA A U D SD The instructor is well-prepared for class. 8 SA A U D SD The instructor is organized. 9 SA A U D SD The instructor clearly explains the grading system. 10 SA A U D SD The instructor assigns grades fairly. 11 SA A U D SD The instructor is excited about the subject. 12 SA A U D SD The instructor is a good teacher. 13 SA A U D SD The instructor communicates well. 14 SA A U D SD The instructor treats students with respect. 15 SA A U D SD The instructor is accessible for consultation. 16 SA A U D SD I would recommend this course to others. 17 SA A U D SD I would recommend this instructor to others. Table 8: Typical Items in Traditional “Instructor” Oriented Course Evaluation Instruments 145 Course Learning Objective (CLO) Related Items. Because they involve the stu- dents’ (i.e., consumers’) view of their attainment of specific course objectives, rat- ings of CLO items are especially useful for the assessment of student learning within the context of accreditation standards—if all course objectives clearly link to program objectives, organizational goals, and ultimately, the mission of the school or department. Students’ aggregated CLO ratings represent indirect evi- dence of learning for discrete items, courses, sequences, or programs. Students’ responses to all CLO items associated with a particular course may be averaged in the form of a Course Learning Objectives Scale (CLOS) score to provide gener- al indications of the degree of overall learning in a particular course. In many social work programs, professors agree to use identical learning objectives in all sections of the same course.When this occurs, programsmay aggregate students’ responses from multiple course sections and analyze them by semester or year. The CLOS score reflects an average of students’ ratings of the learning objective- related items for each course. Since each itemmay be rated on a one-to-five basis, the lowest possible CLOS score would be one and the highest five. Table 12 reflects the CLOS scores (i.e., aggregated averages) of students’ responses to a hypothetical set of course-learning objective-related items (such as those presented in Table 7) by course and year. Scores may range from one Cournoyer/ASSESSMENT OF STUDENT LEARNING 1 SA A U D SD Learners held high expectations for one another. 2 SA A U D SD Learners spent a lot of time and energy undertaking learning experiences and assignments. 3 SA A U D SD Learners interacted frequently with other learners including the professor. 4 SA A U D SD Learners engaged in active learning experiences. 5 SA A U D SD Learners participated in one or more cooperative learning teams. 6 SA A U D SD Learners gave prompt and constructive feedback to other learners. 7 SA A U D SD Learners respected diverse talents and ways of learning. 8 SA A U D SD Learners assumed individual and collective responsibility for learning. 9 SA A U D SD Learners cooperated and collaborated with one another. 10 SA A U D SD Learners actively sought out, discovered, and constructed rele- vant information. 11 SA A U D SD Learners applied information to understand, assess, analyze, and address real issues. 12 SA A U D SD I learned a great deal in this course. 13 SA A U D SD I became more proficient in the social work knowledge base. 14 SA A U D SD I developed critical thinking abilities. 15 SA A U D SD I learned much that will help me as a practicing professional social worker. 16 SA A U D SD I became a more competent social worker. Table 9: Typical Items in a “Student Learning” Oriented Evaluation Instrument 146 ADVANCES IN SOCIALWORK (reflecting strong disagreement) to five (reflecting strong agreement). In this example, all items appear in an affirmative fashion so that higher scores consis- tently reflect stronger agreement. Information such as that presented in Table 12 provides useful insight into stu- dents’ perceptions of learning. Scores in the four-to-five range (“agree” to “strong- ly agree”) suggest that students believe that they learned what their social work instructors hoped they would learn—based upon the course learning objectives. Such a table also reflects changes across semesters as well as students’ ambiva- lence about the extent of their learning in certain courses. As such, program administrators and faculty may attend to certain courses or sequences within the curriculum. For example, in reviewing Table 12, faculty members might well decide to investigate the factors associated with the drop in CLOS scores and the increase in variability as indicated by the standard deviations for the SW9 cours- es from the 1995 and 1996 levels to the 1997 and 1998 levels. Integrating the CLOClassification andAssessment of Student Learning Systems Theuseof end-of-semester evaluation instruments that emphasize student learning in both general common items (CI) and course learning objective (CLO) related items represent a substantial contribution to the demonstration of progress toward achievement of program goals and the EPAS standard that requires pro- 1 SA A U D SD Understand the fundamental values, ethics, and legal obliga- tions of the social work profession. 2 SA A U D SD Apply social work values, ethics, and legal obligations in processes of ethical decision-making. 3 SA A U D SD Understand the skills associated with each of phase of social work practice (i.e., preparing, beginning, exploring, assessing, contracting, working and evaluating, and ending). 4 SA A U D SD Apply social work skills in interviews with real or simulated clients. 5 SA A U D SD Apply knowledge and understanding of self in interactions with and service to others. 6 SA A U D SD Apply social work knowledge and skills differentially to avoid discrimination and demonstrate respect for persons of diverse backgrounds and characteristics, and populations-at-risk. 7 SA A U D SD Prepare clear and well organized professional social work case records (e.g., intake and social histories; social work assess- ments of person-issue-situation, including strengths as well as problems; contracts and plans, including clear specification of intervention goals; progress and evaluation notes; and closing summaries). 8 SA A U D SD Assess one’s strengths, limitations, and learning needs, includ- ing evaluation of the quality and appropriateness of social work skill selection and application. Table 10: CLO-Related Items in an Assessment of Student Learning Instrument Please use the enclosed five-point (Strongly Agree to Strongly Disagree) scale to rate the degree to which, as a result of the learning experiences you completed in this course, you learned to: gram assessment. They are consistent with the principles of Continuous Quality Improvement. When direct assessment indicators are added (e.g., entry and exit essays, pre- and post-instruments, or program oriented student portfolios), an array of data about student learning outcomes become available. However, maximum benefit occurs when assessment processes are integrated with the Course LearningObjectives Classification system. If each course-learning objective offered throughout a curriculum is classified according to pertinent dimensions—including school or department goals and program objectives, and if direct and indirect forms of assessment are geared toward evaluation of progress toward achievement of those goals and objectives, then programs have the capac- ity to analyze data according to any or all of the classified categories. For example, suppose a program classifies its course objectives according to CSWE foundation program objectives. The program may then, with the aid of computer software, separate and statistically analyze students’ responses to CLO items that pertain to each objective. The programmay also develop evaluation rubrics to directly assess learning products (e.g., essays, papers, examinations, portfolios) in relation to those objectives as well. For purposes of both continuous improvement and to address requirements for accreditation, the program could then describe the nature and extent of student learning for each competency, ideally over a period of several years to reveal patterns or trends. Table 13 presents data from students’ ratings of course learning objective relat- ed items by competency and year. If they were actual, rather than hypothetical data, the programmight be concernedwith students’ (especially those completing end-of-semester instruments during 1998) perception of learning in regard to their ability to “apply critical thinking skills within the context of professional social work practice” (Foundation ProgramObjective #1). Theymight be especially 147Cournoyer/ASSESSMENT OF STUDENT LEARNING 1995 1996 1997 1998 Course Mean SD N Mean SD N Mean SD N Mean SD N SW6 4.68 .42 24 4.50 .86 85 4.50 .50 109 4.42 .57 111 SW7 4.46 .72 42 4.35 .78 32 4.60 .49 45 4.52 .66 49 SW8 4.67 .38 36 4.65 .44 42 4.73 .42 36 4.53 .53 43 SW9 4.41 .54 124 4.34 .70 119 3.84 1.09 107 3.78 1.12 92 Table 12: Course Learning Objective Scale (CLOS) Scores by Social Work Course and Year 1995 1996 1997 1998 Course Mean SD N Mean SD N Mean SD N Mean SD N SW1 4.68 .42 24 3.88 1.15 99 3.86 .99 149 4.13 .85 125 SW2 4.56 .61 70 4.08 .80 62 3.98 .96 61 3.60 .97 55 SW3 3.65 .91 48 3.94 1.01 72 4.37 .67 67 4.68 .55 76 SW4 4.24 .88 94 4.37 .67 88 4.31 .73 96 4.45 .75 110 SW5 3.67 .99 60 3.83 1.12 100 4.50 .50 109 4.49 .66 11 Table 11: Common Item Scale (CIS) Scores by Social Work Course and Year concerned if direct assessment of performances or products also revealed weak- nesses in critical thinking abilities among a substantial number of graduating students. Similarly, there might be concern about students’ learning during 1997 in rela- tion to the ability to “apply the knowledge and skills of generalist social work prac- tice (or, for MSW graduates, ‘a generalist social work perspective’) with systems of all sizes” (Foundation Program Objective #6). Professors might be particularly curious about what may have happened during 1995 in relation to students’ per- ceptions of their ability to “evaluate research studies, apply research findings to practice, and evaluate their own practice interventions” (Foundation Program Objective #9) and their ability to “use communication skills differentially across client populations, colleagues, and communities” (Foundation ProgramObjective #10) during 1998. Although far from definitive, these data help programs identify how well their students think they learn within certain areas of a curriculum. Direct assessment of students’ performance and learning products may serve to substantiate or refute findings from indirect forms of assessment. Similarly, infor- mation gained through indirect means may be supported, or challenged, by evi- dence generated by direct forms of assessment. Although Table 13 illustrates data that relates to one classification dimension (i.e., CSWE Foundation Program Objectives), programs may conduct similar analyses based upon other factors. Analyses based upon organizational goals help support a school or department’s claim that they accomplish their mission. Analyses based upon Bloom’s Taxonomy may indicate that students are learning how to analyze, evaluate, and synthesize professional knowledge, values, and skills. These higher order cognitive skills constitute essential aspects of critical thinking—one of the hallmarks of professionalism. IMPLICATIONS FOR SOCIALWORK EDUCATION Social work deans, program directors, and faculty members are likely to confront numerous challenges as they implement processes for assessing student learning and educational effectiveness. Faculty may anticipate dilemmas such as the following: • At a time when programs are asked to do “more with less,” programs may lack sufficient resources to support extensive assessment processes. Personnel are needed to develop and administer instruments, and to collect and analyze data. Higher costs may be expected during developmental phases when the program determines an assessment philosophy and decides what, how, and how much to assess. Administrators may anticipate the need for release time for personnel undertaking these activities. Some programs might benefit from faculty and staff development programs that address the topic andmethods of assessment and evaluation. Finally, programs may need to invest in some equipment (e.g., scanners, computers) and computer software programs through which to organize and analyze data. • Within some university contexts, social work educators may be successful in implementing a sound assessment process but lack authority or resources to use findings for decision-making. For example, a program director may obtain assessment data suggesting that students in Professor “X’s” Social Policy class- 148 ADVANCES IN SOCIALWORK es appear to learn a great deal from the course. Both direct and indirect sources of evidence indicate that students achieve most of the course objectives. In Professor “X’s” Human Behavior and the Social Environment (HBSE) courses, however, the assessment data suggest that students tend to achieve few of the course objectives. Although it might seem reasonable to assign Professor “X” to teachmore social policy courses and fewer, if any, HBSE courses, a directormay not possess the authority or have sufficient faculty resources to reassign pro- fessors based upon evidence of student learning. • Individuals or committees that implement assessment programs face ques- tions related to the validity, reliability, and relevance of assessment processes and outcomes. Assessment approaches that rely primarily or exclusively upon student (i.e., consumer) feedbackmay be strongly challenged—especially if the results are usedmore to evaluate personnel performance than to assess student learning and program quality. At a time when social work students are being confronted with multiple roles (e.g., parent, full-time worker, caregiver of older parents), demanding learning environments may influence the way courses, instructors, and the overall quality of the program are perceived. Nonetheless, when both direct and indirect assessment processes are used and the findings converge, serious challenges to the accuracy and utility of the information are less likely. • Social work educators should anticipate how various stakeholders might inter- pret and use assessment findings—especially during the early phases. For instance, some university administrators may be quite uncomfortable with assessment results because they sometimes highlight significant issues and lead to difficult decisions. Some officials may find it easier, safer, or more com- fortable to deny, minimize, or ignore findings that call for decisive, unpopular, or costly action. 149Cournoyer/ASSESSMENT OF STUDENT LEARNING 1995 1996 1997 1998 Objec- Mean SD N Mean SD N Mean SD N Mean SD N tive 1 4.04 .55 83 4.01 .68 96 3.99 .57 109 3.73 .86 65 2 4.41 .65 122 4.29 .75 145 4.36 .56 140 4.34 .53 108 3 4.22 .67 115 3.96 .84 126 4.29 .59 160 4.16 .71 112 4 4.20 .60 108 4.27 .56 132 4.28 .57 179 4.29 .65 177 5 4.39 .64 139 4.37 .70 176 4.09 .74 102 4.07 .62 68 6 4.28 .62 94 3.96 1.03 169 3.71 1.00 199 4.36 .79 80 7 4.34 .57 104 4.33 .63 134 4.44 .57 131 4.15 .91 131 8 4.23 .63 97 4.36 .69 97 4.01 .98 131 4.65 .46 104 9 3.07 1.01 105 4.27 .76 113 4.18 .76 119 4.21 .77 103 10 4.19 .68 75 4.56 .45 120 4.27 .47 105 3.63 1.65 90 11 4.75 .32 34 4.67 .47 52 4.61 .48 44 4.51 .94 38 12 4.18 .71 112 4.26 .79 120 4.42 .69 106 4.29 .66 98 Table 13: Aggregated Average CLO Ratings by CSWE Foundation Program Objective • Within many academic contexts, programs “compete” with one another for resources (e.g., funding from the university, “faculty lines,” or even students). At times, negative assessment findings place a programat a disadvantage vis-à-vis other programs that engage in little, if any, assessment or evaluation activities. Directors and faculty members should consider how, for example, higher administration might respond to a program that implements a strong assess- ment system that yields negative findings about student learning. In some con- texts, programs may be recognized and credited for conducting assessment, and providing encouragement and resources to make improvements based upon the findings. In other circumstances, unfavorable results may be used to punish the program that conducted the assessment activities. During the early phases of implementation, such punitive responses send a clear message that genuine assessment is unsafe and probably unwanted. During the next decade or two, social work programs will face many challenges. Certainly, demands for greater accountability will continue to increase. Various stakeholders will expect answers to questions such as the following: Do your stu- dents learn what you say you teach? Does your faculty genuinely help students learn? Does your curriculum truly meet CSWE and university accreditation stan- dards? Are your learning expectations and experiences progressively more intel- lectually, academically, and professionally challenging? In other words, do you expect more of seniors than you do of juniors, and more of juniors than you do of sophomores? Do you require more of MSW students than you do of undergradu- ates, and more of MSW concentration students than MSW foundation students? Social work educators will also be asked to produce evidence of their program’s effectiveness in pursuing its mission and accomplishing its goals. In attempting to provide such evidence, they may be challenged from various sources. Higher administration, students, parents, and some organizations (e.g., legislatures and social service agencies) may demand more and better indications of program quality and effectiveness. Some faculty colleagues may also question the purpos- es, validity, reliability, and relevance of the assessment processes—particularly if the results are used primarily for personnel evaluation decisions (e.g., for promo- tion and tenure decisions) rather than for enhancing faculty development or improving program quality and educational effectiveness. We suggest that the IndianaModel—which incorporates both direct and indirect processes, and integrates Course Learning Objectives Classification and Assessment of Student Learning systems—represents a strategy by which to address some of these questions.We fully recognize that data obtained exclusive- ly from direct or from indirect forms of assessment have finite value. As important as the consumer voicemight be, and as appealing as a valid and reliable standard- ized examinationmight be, feedback from one source alone is simply insufficient. Indeed, we strongly recommend the use of multiple indicators of both an indirect and direct nature in order that findings may be subject to multidimensional con- sideration. 150 ADVANCES IN SOCIALWORK Endnotes 1 This approach to student learning assessment originated at Indiana University School of Social Work. Other schools and programs of social work have begun to refer to “The Indiana Model” to capture the general thrust of this form of assessment. 2 The Educational Policy and Accreditation Standards (EPAS) (2001) of the Council on Social Work Education reorganized the content areas by combining (a) populations-at-risk and (b) social and eco- nomic justice. These domains were separate in the 1992 CPS and 1994 Standards. 3 Note: The items have been renumbered to facilitate classification. 4 Bloom’sTaxonomy is used by the Association of SocialWork Boards (ASWB) to organize test items on the standardized social work licensing examinations. References Association of SocialWork Boards. (2001).Basic examination:Content domains. Association of SocialWork Boards. [On-line]. Retrieved August 21, 2001, from theWorldWideWeb: www.aswb.org. Barr, R. B., & Tagg, J. (1995). From teaching to learning: A new paradigm for undergraduate education. Change, 27(6), 13-25. Bloom, B. S., & Krathwohl, D. R. (1956). Taxonomy of educational objectives: The classification of educa- tional goals, by a committee of college and university examiners. Handbook I: cognitive domain. New York: Longmans, Green. Chickering, A.W., & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. AAHE Bulletin, 39(7), 3-7. Commission on Accreditation. (1994). Handbook of accreditation standards and procedures. Alexandria, VA: Commission on Accreditation of the Council on SocialWork Education. Council on SocialWork Education (1992). Curriculum policy statement. Alexandria, VA. Author. Council on SocialWork Education. (2001). Educational policy and accreditation standards (EPAS). Council on Social Work Education. [On-line]. Retrieved August 21, 2001, from the World Wide Web: http://www.cswe.org. Cournoyer, B. R. (2001, November 2). The social work portfolio. Paper presented at the Annual Meeting of Baccalaureate Program Directors, Denver, Colorado. Cournoyer, B. R., & Stanley, M. J. (2002). The social work portfolio: Planning, assessing, and documenting lifelong learning in a dynamic profession. Pacific Grove, CA: Brooks/Cole. Leigh, J.W. (1998). Communicating for cultural competence.NeedhamHeights, MA: Allyn and Bacon. National Association of Social Workers. (2001). NASW standards of cultural competence in social work. Washington, D.C.: Author. Author’s Note: Address correspondence to: Barry R. Cournoyer,D.S.W., IndianaUniversity School of SocialWork, 902West NewYork Street, Indianapolis, IN 46202-5156 USA, bcourno@iupui.edu. 151Cournoyer/ASSESSMENT OF STUDENT LEARNING