ASSESSING AND DEVELOPING STUDENT SKILLS USING A GROUP EXERCISE Developments in Business Simulation and Experiential Learning, Volume 34, 2007 387 ASSESSING AND DEVELOPING STUDENT SKILLS USING A GROUP EXERCISE Marian W. Boscia King’s College marianboscia@kings.edu R. Bruce McAfee Old Dominion University rmcafee@odu.edu ABSTRACT Colleges of Business are now under increasing pressure from AACSB to assess student learning and to engage students in collaborative learning experiences. This paper presents an experiential learning exercise used at King’s College that is designed to accomplish both of these objectives. In addition, the exercise helps to develop student skills in several areas: critical thinking, oral and written communication, technology, quantitative reasoning, and information literacy. INTRODUCTION AACSB International, the accreditation agency for colleges of business, requires that schools develop measures that assess the extent to which students have reached the intended learning goals. It is not sufficient to teach students, give regular exams, and declare that students have learned course content. Rather, once an institution establishes a mission statement and specific learning goals for students, an assessment must be made. These assessments must be “regular, systematic, and sustained (AACSB, 2006, p. 61).” In addition AACSB recommends that students should have opportunities to engage in collaborative learning. They argue that, “each student is a resource who brings unique experience and knowledge to combined tasks. Students need to acknowledge their responsibilities to their fellow students by actively participating in group learning experiences (AACSB, 2006, p. 56).” This paper presents a group exercise that has multiple purposes. Its primary purpose is to assess student skills on six different competencies: critical thinking, effective writing, library and information literacy, oral communication, technology, and quantitative reasoning. Secondarily, the exercise is designed to simultaneously enhance these same skills. Since the exercise is done by student groups, this exercise also helps to meet AACSB’s recommendation that students should engage in collaborative learning. Learning assurance and assessment have long been a concern of ABSEL members. This topic was the focal point for a panel discussion at ABSEL 2004 presented by Khushwant Pittenger, Joshua Mott, and Constance Savage from Ashland University, Sandra Morgan from the University of Hartford, and Diana Page from the University of West Florida. This discussion shared ideas for experimentation with student assessment (Pittenger, et. al. 2004). One year later, Gretchen Vik from San Diego State University, Douglas Micklich from Illinois State University and Monique Forte from Stetson University presented a panel at ABSEL 2005 to share their experience implementing various assessment systems at their universities (Vik, Micklich, and Forte, 2005). Other ABSEL members have proposed assessment methods. Most recently, Hall and Ko (2006) developed a model to be used with simulations to measure and assess learning in executive management degree programs. This paper continues ABSEL’s long tradition of supporting and investigating assessment of student learning and competency. THE SOPHOMORE-JUNIOR DIAGNOSTIC ASSESSMENT PROGRAM OVERVIEW The Sophomore-Junior Diagnostic Assessment Program is a requirement for all accounting majors and minors in the College of Business at King’s College. The assessment is conducted during class for all students taking Intermediate Accounting II, at the end of their sophomore year. Approximately 50 students take part in the program each year, and, although revised each year, the program has been in place for about 10 years. Students work in teams of three or four to complete the project. Each individual student’s performance, however, is evaluated, accounting for 20% of the final grade. The program requires that student teams analyze the Annual Report and 10K filing of a publicly traded corporation along with other company and industry information. Each group selects a company that has not previously been chosen by another group. The company must be pre-approved by the course professor. After analyzing their company’s Annual Report and related information, each student is responsible for presenting a portion of the analysis, submitting a full sentence outline describing the content of his/her section, and appropriate citations for sources used in the presentation. Each group also submits an EXCEL spreadsheet containing mailto:marianboscia@kings.edu mailto:rmcafee@odu.edu Developments in Business Simulation and Experiential Learning, Volume 34, 2007 388 the financial statements and ratio computations and a bibliography of outside resources used to develop the content of the presentation. During each group’s presentation, each student is assessed in terms of the six core competencies mentioned above. INTRODUCING THE EXERCISE Students are told that in order to evaluate their understanding of the use of financial statements in decision- making, they are to assume the role of a financial analyst. Their group will study the profitability, stability, solvency, and potential for growth of a publicly-held company. They are to tell the story of what their company has been doing for the past three years and where they think it will be going in the future. They are to remember that their audience is investors who want to know whether they should buy the company’s stock and creditors who are making lending decisions. They are told that before they begin to create a written outline and undertake the analysis of a company’s financial statements, they will need to compile the relevant information. They are given a list of documents that are intended to serve as guides to help them accumulate useful information and to organize that information effectively. They should note that not every item listed is relevant to their company and that they may wish to consult additional outside resources as well. PRESENTATION REQUIREMENTS Students are told that individual presentations and outlines for each group should focus on the specific elements given below. Groups have approximately 25 minutes to make their presentation. No two members of a group are allowed to present the same information. Ideally, the group, as a whole, will adequately address all of the following: Level 1 -- Business and Industry Analysis The first part of this level of analysis (no more than five minutes of the presentation) should focus on their company’s background information. The second part should be a short discussion of the company’s operating activities. For the final part, students should include information about the size and profitability of the company’s industry. Some suggested items to be included in this section are listed below. Company Background: The following basic facts should be presented. Most of this information can be found in the investor relations portion of the corporate web site. 1. Legal name of the company 2. URL of company Web site 3. State and year of incorporation 4. Stock ticker symbol and stock exchange where shares are traded 5. Company’s independent auditor. Has there has been a recent change in the auditor? What fees have been paid to the auditors for audit fees, audit related fees (probably tax-related), and any other type of service such as information technology or consulting? 6. Composition of the company’s Board of Directors (i.e., age, gender, ties to the company, etc.) 7. Names of major company officers and the Chair of the Board of Directors 8. Company’s fiscal year-end (month and day) 9. Overview of company’s key financial data, such as total revenues and expenses, total assets and liabilities, numbers of share of stock outstanding, etc. Company Operating Activities: The SEC Form 10-K and a financial service, such as Morningstar, Money Central (MSN), Yahoo! Finance, MarketWatch.com, or StreetInsider.com can be used to gather information. 1. Industry segments – how many, which are the largest 2. The main company products/services, the means by which products are sold or delivered, and the target customers for those products/services. 3. Relevant financial information by segment or product line, such as revenue, assets, expenses, gross margin, etc. 4. Company’s primary and secondary SIC or NAICS codes 5. Company’s competitive condition and its major competitors 6. Company size based on: a. Dollar amount of assets, sales/revenues, and net income b. Market share c. Earnings per share (diluted) and number of common shares outstanding 7. Research and development efforts 8. Proposals or topics submitted for stockholder vote in the most recent proxy statement. Level 2 -- Accounting Analysis and Financial Highlights For this level of analysis, students need to present financial highlights. They should also examine the company’s financial statements in order to determine the degree to which the firm’s accounting captures its underlying business reality. As part of this analysis, one should evaluate key accounting policies and the quality of the disclosures, looking for any potential red flags or indications of accounting distortions. This section should take no more than 8 to 10 minutes to present. Some suggested items to be included in this section: The Company’s Financial Highlights: This topic is covered in the annual report to shareholders and the SEC form 10-K. 1. Track changes in the following items over the prior three to five years: a. Total assets and total liabilities b. Short-term debt and long-term debt c. Revenues, expenses, and net income d. Dividends per share e. Basic and diluted earnings per share 2. Discuss any problems such as the acquisition of raw materials, seasonality of business, or impairment of assets. Developments in Business Simulation and Experiential Learning, Volume 34, 2007 389 3. Cash flows are important because investors want cash dividends or they want cash reinvested to increase future earnings. Creditors want debt to be repaid and to receive interest payments. From the investors’ and creditors’ perspective, students should discuss where cash comes from and how it is used. Items that one may want to present include: a. The method that the company uses to report cash flows -- direct method or the indirect method b. The change in the cash balance over the past few years 4. A comparison net cash flow from operating activities to net cash flow from investing activities. Were operating cash flows used to finance the purchases of operating assets or was it necessary to incur debt to purchase assets? 5. The company’s major strengths, weaknesses, opportunities, and challenges that promote or impede its growth and profitability. Significant accounting policies: This topic is generally covered in the notes to the financial statements. 1. Students should briefly explain the accounting policies for: a. Inventories b. Deprecation c. Income taxes, including the effective tax rate d. Employee benefit plans e. Contingencies 2. If a new financial accounting standard affects the financial statements during the current year, one should briefly explain the potential affects on the company's financial statements. 3. Students might want to discuss risk management from the Management’s Discussion and Analysis section of the annual report. Executive compensation: Information on executive compensation is usually included in Form 10-K. Related information may be presented in the proxy statement. 1. For the firm’s highest paid executive, total the dollar amounts paid for salaries and bonuses that were received in each of the last three years. If this amount seems high, compare this to the executives of your company’s competitors. 2. Does the trend in stock returns match the trend in compensation being paid to the company’s top executive? a. Compare the increase or decrease in executive compensation to the stock performance graph near the end of the proxy statement. b. Compare your company’s stock price performance to the S&P 500, NASDAQ average or the Dow average. Level 3 -- Analysis of Historical Financial Data Using the financial ratios, students are required to analyze the firms’ profitability, solvency, and operating efficiency. The information discussed above should provide support for any conclusions based on ratios. Compare your company’s ratios across at least three years and also compare them with at least one competitor or to industry averages. Which ratios one chooses to discuss in the presentation will depend on which issues are critical to one’s conclusions. This section should take no more than 8 to 10 minutes to present and at a minimum the presentation should cover the following: Liquidity Ratios: Current Ratio or Working Capital Ratio Quick Ratio Cash flow adequacy ratio Times Interest Earned Ratio or Interest Coverage Profitability Ratios: Gross Profit Percentage Return on Investment (ROI) or Return on Financing (ROF) Return on Assets (ROA) and Asset Turnover DuPont Analysis of Return on Equity Leverage Ratios: Debt to Total Assets Debt-to-Equity Times Interest Earned or Fixed Charge Coverage Efficiency Ratios: Accounts Receivable Turnover and Average Collection Period Inventory Turnover Ratio and Average Days Sales Are in Inventory Stock Ratios: Basic and Diluted Earnings per Share Price Earnings Ratio (PE ratio) Book Value per Share Dividend Payout Ratio or Dividend Yield Financial Recommendations: In this section, each group should make recommendations to investors and creditors. Should the stock be purchased by investors? Should a bank be willing to lend money to the firm? A member of each group should spend 2-5 minutes summarizing the key points made by other group members to support the group’s recommendations. ASSESSING STUDENT SKILLS The content of each student’s presentation is evaluated based on answers to the questions above. They are assessed on the degree to which they have mastered the subject matter of the assignment. Their critical thinking skills, technological competency, information literacy, quantitative reasoning, and their oral and written communication skills are evaluated using the evaluation form below based on the following guidelines: 1) Critical Thinking Competency a) The student should understand the general components of the Balance Sheet, Income Statement, Statement of Cash Flows, and Statement of Retained Earnings (the General Purpose Financial Statements of a Corporate Annual Report) b) The student should be able to analyze the individual Developments in Business Simulation and Experiential Learning, Volume 34, 2007 390 Liability and Equity accounts of a Balance Sheet by attesting to the types of transactions that flow through each account. c) The student should be able to assess the general financial condition of a Corporation through item (b) above. 2) Effective Writing Competency The student should be able to prepare a written analysis of a corporation’s general financial condition based upon a thorough review of its Annual Report. 3) Library and Information Literacy Competency The student should be able to research and identify the relevant ARB, APB, and FASB guidelines that a corporation uses to prepare its Annual Report with accompanying footnotes. 4) Oral Communication Competency The student should be able to present an effective presentation related to a corporation’s general financial condition based upon a thorough review of its Annual Report. In addition, the student should be able to involve the audience as active participants in a presentation as well as react to audience questions. 5) Technology Competency The student should be able to conduct research on the Internet relevant to a corporation’s Annual Report. In addition, the student should be able to use Microsoft Word, Excel Worksheet, PowerPoint, and other relevant Windows computer applications. 6) Quantitative Reasoning Competency The student should be able to perform trend and ratio analysis based on a corporation’s Annual Report. In addition, the student should demonstrate an understanding of common-size financial statements in time-series and cross-sectional formats. DISCUSSION While the Sophomore-Junior Diagnostic Assessment Program was developed primarily to meet the AASCB requirements, we believe that from an education perspective its major use is to develop student’s “higher order thinking” skills. Bloom (1956) in his well-known skills taxonomy contends that there are six major cognitive objectives that range in increasing order of complexity as follows: 1. Knowledge-skills that require recall of information; remembering. 2. Comprehension-skills that require one to establish relationships, categorize, or classify information. 3. Application-skills that require one to apply general principles, ideas, or theories to concrete situations. 4. Analysis-skills that require one to break apart abstractions into their constituent elements and show the relationship among those elements. 5. Synthesis-skills that create new expressions or innovative patterns. 6. Evaluation-skills that require one to make judgments about ideas, methods, values, or solutions. While the Project does require students to both recall information (knowledge component) and to establish relationships among financial data (comprehension component), its major focus is on developing application, analysis, synthesis, and evaluation skills. One of the major requirements of the Project is for students to state whether they would recommend that an investor buy the firm’s stock or lend the firm money. While this requirement focuses specifically on the “evaluation” component of Bloom’s taxonomy, students cannot make an investment decision without applying general financial principles to their specific firm (application component), analyzing financial ratios (analysis component) and looking for patterns among the financial ratios (synthesis component). CONCLUSION As Page and Mukherjee (2006) noted at last year’s ABSEL meeting, the amount of information pertaining to different fields has grown rapidly. Faculty now feel increasing pressure to pack more and more factual content into their classes. This has resulted in a lack of emphasis on developing critical thinking skills even though AACSB suggests that “reflective thinking skills” are an important outcome for business schools (AACSB, 2006). The Sophomore-Junior Diagnostic Project described above attempts to satisfy AACSB’s requirement for Colleges of Business to assess student learning. Simultaneously, it is designed to enhance student’s higher order thinking skills or “reflective thinking skills.” Since all Colleges of Business are under pressure to assess student learning, it would appear that colleges can approach this task is two different ways. One approach is to conduct these assessments without making any attempt to enhance student skills at the same time. For example, student learning could be assessed by evaluating how students perform on specific test questions asked as part of regular examinations. The other approach, and the one described here, is to assess student skills while simultaneously enhancing those same skills. While both approaches have their own strengths and weaknesses, we believe that a strong case can be made for a multi-purpose approach, given that the major goal of most universities is to educate students. REFERENCES AACSB International (2006). Eligibility Procedures and Standards for Business Accreditation. AACSB International-The Association to Advance Collegiate Schools of Business, Tampa, FL, http://www.aacsb.edu. Bloom, B. S. (Ed.). (1956). Taxonomy of Educational Objectives: The Classification of Education Goals, Handbook 1: Cognitive Domain, New York: McKay. Hall, O. P. & Ko, K. (2006). Learning Assurance Using Business Simulations Applications to Executive Management Education. Developments in Business Developments in Business Simulation and Experiential Learning, Volume 34, 2007 391 Simulation & Experiential Exercises, Volume 33, 1-5. Page, D. & Mukherjee, A. (2006). Using Negotiation Exercises to Promote Critical Thinking. Developments in Business Simulation and Experiential Learning, 33, 71- 78. Pittenger, K. S., Morgan, S., Mott, J., Page, D. & Savage, C. (2004). Experimentation With Assessment Techniques: A Proposal For Panel Discussion. Developments in Business Simulation and Experiential Learning, Volume 31, 146-147. Vik, G., Micklich, D. & Forte, M. (2005). How Simulations and Experiential Learning Fit As We Comply With Legislative and AACSB Assessment Guidelines: How To Develop Academically Sound Courses That Also Meet Stakeholder Needs. Developments in Business Simulations and Experiential Learning, Volume 32, 315- 318. Developments in Business Simulation and Experiential Learning, Volume 34, 2007 392 INDIVIDUAL PERFORMANCE EVALUATION Name ______________________________ Company ______________________________ Date ______________________________ Grading Scale: On a 6-point scale, the individual is scored on the extent to which he or she met the evaluation criteria. The scale is as follows: To a To a Completely great extent satisfactory extent Inadequately Not at all 5 4 3 2 1 0 Criteria Points CRITICAL THINKING 1. Your descriptions of issues were comprehensive 5 2. Your description of component parts of the Annual Report was accurate 5 3. Your analysis of the company’s information was thorough 5 4. Your summary reflected a clear understanding of the company characteristics 5 EFFECTIVE ORAL COMMUNICATION 5. Your presentation was delivered at the time scheduled and was an appropriate length 5 6. You were dressed appropriately for a professional presentation 5 7. You were poised, controlled, expressive and maintained eye contact 5 8. You related your part of the presentation to presentations by other team members 5 TECHNOLOGICAL COMPETENCY 9. Your team used EXCEL cell formulas to perform ratio calculations 5 10. Your team’s EXCEL spreadsheet was visually attractive 5 11. You used PowerPoint to develop a visual aid for your presentation 5 12. Your PowerPoint presentation enhanced your presentation 5 EFFECTIVE WRITTEN COMMUNICATION 13. You provided a complete full-sentence written outline of your presentation 5 14. Your outline was well organized 5 15. Your outline was grammatically correct and used clear language 5 16. You used technical vocabulary accurately 5 QUANTITATIVE REASONING COMPETENCY 17. You presented quantitative information that was relevant to your presentation 5 18. You correctly interpreted the quantitative information 5 INFORMATION LITERACY 19. Your evaluation of the company’s condition was based on sound evidence 5 20. You provided a bibliography that was properly formatted 5 Total score (100 points possible) 100 Comments: Table of Contents Volume 34, 2007 Experiential Teaching May Lead To Experiential Learning The Role Of Learning Versus Performance Orientations When Reacting To Negative Outcomes In Simulation Games: Further Insights An Analysis Of The Interaction Of Firm Demand And Industry Demand In Business Simulations Consistency Of Participant Simulation Performance Across Simulation Games Of Growing Complexity Assessing And Incentivizing Learner Contribution And Performance Excellence: Creating Win-Win Evaluation Options The Technological Impact Analysis: A Research-Based Exercise To Heighten Learners' Technological Sensitivity Beginnings: How We Start the Semester and Individual Classes : A Roundtable Discussion Simulation Performance and Its Effectiveness As A PBL Problem: A Follow-Up Study The Use of Multimedia Learning Tools to Facilitate Online Learning of Business Statistics Forecasting Accuracy And Learning: A Key To Measuring Business Game Performance Panel Discussion: Alternative Ways of Using the Internet for Business Simulations to Input Decisions, Process, and Present Financial and Economic Data Output Absel Research -- One Additional Perspective On Where We Are And Where We Have Come From Assessing participant learning in a business simulation From Case Presentation To Case Facilitation: How Assessment Changed The Capstone Course The Use Of Computer-Assisted, Interactive Role-Play Simulation In Hong Kong Corporate Positioning: A Business Game Perspective Demonstration Of A Computer-Assisted Global Business Simulation Applying .NET Remoting To A Business Simulation Assessing Emotional Intelligence: The EQ Matrix Exercise Outcomes And Observations Of An Extended Accounting Board Game The Use Of Learning Styles Questionnaire In Hong Kong Forming Teams For Classroom Projects Online Budgeting And Marketing Control With The Proforma Analysis Package The Effect Of Experiential Learning Experiences On Management Skills Acquisition Students' Perceptions on the Individual Managerial Performance Orderness: A New Definition Of Alignment An Exploratory Study Of The Efficacy Of Servsafe® Online Team Behavior And Team Success Results From A Board Game Simulation Telecommuting Internships - Do They Work? The WEE GAME: A Pre-Game Effects On Learning When Students Have Information About Games And Their Outcomes When Playing Them Assessment And Simulations: Measuring The Academic Learning Compacts Within Dangerous Business: An Interactive Ethics Case Activity The Paradise Islands Assessing And Applying fiAppropriatefl Conflict Management Styles Distance Learning In Communication: Blended Or On-Line? Developing An On-Line Advanced Communication Course World Café: Simulating Seminar Dialogues in a Large Class The Programming Game: An Exploratory Collaboration Between Business Simulation And Instructional Design The Canary Principle: An Alternative Model for Providing Real-Time Coaching in an On-Line Discussion Environment Assessing Character Development Using A Direct-Approach Business Ethics Exercise Persistence Of Decisions By Simulation Game Participants The Invalidity of Profit = f(Product Quality) PIMS Validation of Marketing Games A Study of The Applicability of The Perceptions of Organizational Politics Scale (POPS) For Use in The University Classroom The ROI Of Connected Projects: A Payroll Example Initiation of Research on Gaming Simulation in Japan Inappropriate Use of Citations and Corrupting the Body of Knowledge: Accepting Urban Legends as Truth An Exploration Of The Perceived Value Of Highly Socio-Inductive Learning How "Whole" Is Whole Person Learning? An Examination Of Spirituality In Experiential Learning Modeling Outsourcing and Strategy Alignment into a Business Game Successful Integration Of Webct Into A Small Business School The Thinking Steps Model In Game Theory: A Qualitative Approach In Following The Rules The Application Of Means-End Theory To Understanding The Value Of Simulation-Based Learning Armchair Travel: An active learning approach to increasing global awareness and participant self-efficacy Goldratt's Thinking Process: Is there a place for it in the Total Enterprise Simulation Assessing And Developing Student Skills Using a Group Exercise Computer Business Simulation Design: Novelty & Complexity Issues