A FRAMEWORK FOR THE IDENTIFICATION OF MODERATED AND MEDIATED PERFORMANCE CONSEQUENCES OF PEDAGOGICAL ALTERNATIVES


Developments In Business Simulation & Experiential Exercises, Volume 17, 1990 

 30

A FRAMEWORK FOR THE IDENTIFICATION OF MODERATED AND MEDIATED 
PERFORMANCE CONSEQUENCES OF PEDAGOGICAL ALTERNATIVES 

 
Alvin C. Burns, Louisiana State University 

 
ABSTRACT 

 
Researchers have utilized a stimulus-response paradigm to assess the 
efficacy of pedagogical alternatives, but this is a black box procedure. A 
better approach is to measure learning and determine the extent to which it 
mediates pedagogy’s effects on performance. Similarly, we should consider 
the influences of individual difference factors, which moderate these 
effects. A framework which does both is presented, and an example of its 
use is provided. Finally, three challenges associated with this framework are 
described. 

INTRODUCTION 
 
It is well known that a business educator has a wide variety of pedagogical 
strategies from which to choose as a means of fostering learning in students. 
Often these strategies appear as alternatives, and due to time, energy and 
other constraints, the educator must select one over its rivals. Even when 
he/she decides on a pedagogy, there is invariably a multitude of options 
within that approach. The selection process is not easy because of the 
myriad of considerations, which should be mulled over (Wolfe, Gentry, and 
Burns, 1991). This situation has fostered empirical studies, and as 
academicians, we must be vigilant to the progress of research, which 
addresses the relative efficacy of these pedagogical alternatives under 
various conditions. With these points in mind, this paper is intended to 
assist the scientific process, which seeks to reveal those pedagogical 
alternatives, which are most effective. 
 
The standard approach to these investigations has been to select certain 
independent variables and to test their effects on key dependent variables. 
To be sure, this strategy embodies adroit experimental design philosophy 
and is an accepted means of ultimately discovering truth. However, there 
are criticism of this method when it is applied to complex subject matter 
such as individual learning and performance. Fortunately, an alternative 
framework has recently been espoused by those working with mental 
constructs, which link stimuli and human response under circumstances 
where individual differences abound. So this paper has five purposes: (1) to 
identify the shortcomings of the typical approach; (2) to posit a framework 
which will lead us to better model and understand our subject matter; (3) to 
describe some of the analytical procedures associated with this framework; 
(4) to provide an illustration; and (5) to identify some issues to resolve for 
those who adopt it. 
 

THE STANDARD APPROACH TO DETERMINING 
PERFORMANCE EFFECTS 

 
The typical investigation of the effects of pedagogical’ alternates on student 
performance subsumes an honored research paradigm. That is, a researcher 
normally adopts a stimulus-response (S-R) frame of reference when 
designing experiments or otherwise undertaking investigations. To take the 
example of simulation gaming (where much research has taken place), we 
might identify a reasonable predictor, such as group size and hypothesize 
that it will have an effect on a criterion variable such as team performance. 
The experimenter then can separate student teams into two treatments, say 
large groups and small groups, and test for differences in the (e.g.) 
profitability of their respective companies at the end of a simulation game 
experience. The researcher conducts the experiment and appropriately uses 
a t-test for the significance of the difference between the group means as the 
statistical’ test. If the large group size treatment results in greater average 
profitability than the small group size treatment as hypothesized, the 
researcher is satisfied to have proven that the abundant resources of several 
students work better than the sparse resources of few students as regards 
simulated company performance. 

Two recent reviews can be used as evidence of the widespread subscription 
for the S-R model by business pedagogy researchers. Upon assessing a host 
of research studies on business gaming, Wolfe (1991) admonishes 
researchers to address process factors as they relate to dependent (Learning) 
variables. Also, Gosenpud (1991) reviews several studies on experiential 
learning where the dependent variable is either cognitive learning or 
behavioral change (e.g., skill acquisition). Both Gosenpud’s (1991) tables 
and Wolfe’s (1991) comments clearly show that virtually all studies 
embody the S-R, or independent-dependent variable, paradigm either as a 
formal experimental design or implicitly as a function of the statistical 
model utilized to test hypothesized relationships. 
 
While it is apparently universally used, there are criticisms of the S-R 
approach. First, it suffers from the “black box” condemnation. Although we 
may have found differences, we have no real understanding of why the 
predictor effected the criterion. In the example just cited, greater 
profitability may have been facilitated by group dynamics, strong 
leadership, more representation of bright students in large teams, a better 
grapevine to last semester’s students, or any combination of several unseen 
factors. The S-R model inevitably leaves us with thc question of "why" 
because the box is closed, and the process by which the response is created 
is not illuminated for us. 
 
There is a second, and equally disconcerting, problem associated with S-R 
models. This traditional method implicitly disallows individual differences 
as a confounding factor in our experiment. That is, we are well aware that 
students represent unique characteristics in terms of demographics, ability, 
willingness, circumstances, and situations. Similarly, educators, (i.e., 
researchers using themselves or others to generate research data) have 
idiosyncratic features which have potential’ to interact with experimental 
treatments and effect different consequences. Even subtle differences such 
as number of office hours, time in the day a particular class is taught, or the 
availability of graduate assistant help uniquely characterize educators and, 
conceivably, fall differentially on treatmcnt groups. 
 
In reality, pedagogical research is saturated with these individual difference 
factors, and the only way the S-R model can handle these is by portraying 
them as additional stimuli. There are two reasons why this treatment is 
incorrect. First, an individual difference variable such as sex cannot be 
controlled by the researcher. Granted, it can be randomly assigned to 
treatment groups, but this tactic differs from explicitly addressing the 
variable’s influence and testing it under experimental conditions. Second, 
when a researcher casts an individual difference factor as a predictor 
variable, he will only took at its association with or effects on the criterion 
variable. Gosenpud (1991) rcvicws several “contingency’ studies where the 
researchers do precisely this type of analysis. They are not disposed to look 
at impacts on the relationship between predictor and criterion. 
 
The following example will serve to illustrate these points. If grade point 
avcragc (i.e., “intelligence’) were suspected to be an individual difference 
variable operating on our large group-small group experiment described 
above, we could determine the average GPA of each team and separate our 
teams into high and low GPA groups based on a median split. Now, we one 
could test performance by GPA and perhaps find that no significant 
relationship existed; however, by doing two separate tests, one for each 
treatment group, it is possible that the researcher would find (e.g.) that high 
GPA groups significantly outperform low GPA groups in large teams, but 
not in small tcams. The inspection of such differences within treatment 
groups is very rare, and even covariate analysis, which is sometimes used, 
extracts the influence of the individual difference variable across treatment 
groups rather than testing for them within each group. 
 

THE MODERATOR-MEDIATOR-CONSEQUENCES FRAMEWORK 
 
We hasten to point out that our criticisms are not completely original. In 
fact, they stem from Baron and Kenny’s (1986) recommendations 
concerning the conceptualization and statistical analysis of latent constructs 
in  



Developments In Business Simulation & Experiential Exercises, Volume 17, 1990 

 31

social-psychological research. Baron and Kenny (1986) also distinguish 
sharply between predictor, moderator, mediator and consequences variables, 
and we adopt their terms throughout this paper. In applying their paradigm, 
experimental treatments under the control of thc rcsearchcr (stimuli) should 
be cast as indcpendent variables. Criterion measures of performance effects 
are idcntificd as dependent variables (consequences or responses), while 
some cognitive process must be positioned as mediating the effects of 
predictor variables on the criterion variables. Thus, the mediator serves as a 
cognitive process bridge between the predictor and the criterion variables. 
That is, something must happen in subjects’ hcads in order for them to 
respond. Moderators, on thc other hand, affect the strength of the 
relationships between predictor, mediator, and/or criterion. Thus, individual 
differences in academic ability, sex, or some other state variable which 
diffcrcntiates subjects and/or their circumstances qualify as potential 
moderators. 
 
Accordingly, Figure 1 illustrates thc general approach wc advocate for 
rcscarch on cxpcriential pedagogy alternatives. We will use Figure 1 to 
describe the Baron and Kenny (1986) framework as it can be applied to 
pedagogical research. As can be seen, moderators are those trait, state, or 
other differentiating conditions which are believed to classify students or 
educators in such ways that learning will uniquely and systematically. 
Again, a moderator is a factor outside thc control of the rescarchcr but with 
the potential to confound the relationships and/or results of the experiment. 
Predictor (independent) variables, on the other hand, are those factors 
manipulated to test various performance effects. As noted earlier, setting up 
teams m different sizes is a predictor variable candidate. Researchers 
investigate predictors as function of whatever controllable pedagogical 
factors arc of interest. Their effects, in turn, constitute the criterion 
(dcpcndcnt) variable set. In thc case of simulation gaming, thc effects 
typically measured are profitability, market share, ROI, or gains. With 
experiential exercises, they pertain to skill acquisition or behavioral 
outcome. Finally, wc arc confronted by the need to identify a mediating 
variable which connects the predictor to the criterion. In our view, lcarning 
serves as a reasonable cognitive process mediating factor operating to 
accentuate or attenuatc thc effects of a manipulated independent variable on 
a dependent variable. 
 
We are fully aware that this last contention appears to run counter to those 
who view learning as the salient criterion variable. But, despite our claim, 
we are in perfect philosophical agreement with this stance that wc are in thc 
“lcarning business.” In fact, we will next describe the statistical procedures 
described by Baron and Kenny (1986), and it will become apparent that 
learning must be shown to be an effect of thc predictors, albeit couched as a 
mediator to performance. 
 
Given this conceptual framework, it is appropriate to relate thc general 
statistical rcquiremcnts necessary to demonstrate moderator and mediator 
effects in our pedagogical studies. We will only describe thc simple cases 
and refer interested readers to Baron and Kenny (1986) for more complex 
cases. Causal analysis is assumed throughout thc following discussion. If 
both the moderator and the independent variable are measured as simple 
dichotomies, and the depcndcnt variable is continuous, the recommended 
analysis is a 2X2 ANOVA, and moderation is signified by a significant 
intcraction. That is, the differcnt states of the modcrator variable must be 
associated with difffcrcnt lcvcls for the dcpcndent variable across levels of 

the independent variable. This result infers differcnt strengths or directions 
of relationship bctwcen the independent and dependent variables as a 
function of the state of the modcrator. We interpret this approach as the 
essence of individual differences analysis applied to relationships. 
 
With mediation, on the other hand, the assumption is that the mediator in 
some way transforms the relationship between a predictor variable and its 
criterion variable. Using separate regression analyses cast as path analyses, 
three steps are required to substantiate mediation (Baron and Kenny 1986). 
We have identified these regression paths in Figure 1 as “a b,” and “c.” 
First, the path between predictor and mediator (“a”) must be statistically 
significant. If not, the claim of mediation is not supported. Second, the path 
between predictor and criterion (“b”) must be statistically significant. 
Otherwise, thcrc is no basis for an effect of the predictor on the criterion. 
Finally, when using both predictor and mediator as indepcndent variables, 
the predictor-criterion path ("b") should be significantly reduced, while the 
mediator-criterion (“c”) path should be significant. Thus, the amount of 
reduction in the size of the predictor-criterion (“b") path in the third 
regression relative to the second one is a surrogate measure of the potency 
of the mediator. If this path becomes nonsignificant while the other 
conditions hold, Baron and Kenny (1986) claim perfect mediation has been 
demonstrated. 
 
Figure 1 implies this analytical approach to moderator and mediator effects. 
The solid arrows connecting independent variable to mediator and mediator 
to dependent variable are meant to signify the suspected presence of a 
mediator cffcct which translates the independent-dependent variables 
relationship. The dashed arrow connecting indepcndcnt and dependent 
variables, accordingly, signifies that this demonstrated direct path should 
diminish, ideally to zero, when thc mediator is added to the equation. 
Finally, the modcrator effects are represented by a curved double-headed 
arrow. Moderator effects are thus portrayed as pervasive to the entire 
indcpendent-mediator-dcpcndent system, for relationship sizes and/or 
directions should exhibit change across diffcrcnt levels of the modcrator. 
 

AN ILLUSTRATION OF THE APPLICATION 
OF THIS FRAMEWORK 

 
To illustrate the use of this framework, we will present simulated data 
which pertains to thc use of a stand-alone simulation exercise where a small 
class was taken to a computer lab, and cach student was provided with a 
computer disk and instructed to make five period decisions on price, 
promotion, and distribution. The rcsults were cumulative over thc five 
pcriods. There were two treatments, and students were randomly assigned 
to either: (1) a highly volatile environment or (2) a very stable environmcnt. 
All other game factors were held constant. The company’s earnings per 
share (eps) at the and of the five periods was the performance measure. In 
addition, the students were given an objective test at the end of their 
simulation experiences. 
 
Using the S-R approach, a researcher would simply look for differences 
between the two groups, and indeed, with a t-test, it was found that the 
stable environment group’s average eps was higher (2.29 versus -2.41, p< = 
.001). However, with the moderator-mediator-consequences approach, the 
researcher first inspects for moderator effects. Table 1 presents the analysis 
of variance results where students’ sex is suspected to be a modcrator. It 
reveals a significant interaction between thc game environment and sex, and 
Figure 2 further reveals that male students performed significantly lower in 
thc volatile environment than they did in the stable environment. Female 
students, on the other hand, performed about the same regardless of the 
environment. Females outperformed males overall, as wcll.. 



Developments In Business Simulation & Experiential Exercises, Volume 17, 1990 

 32

 
To test for a mediator, the objective test was used as the measure of 
learning, while eps was used as the criterion. The three regressions were run 
as described above with the following standardized betas (path coefficients) 

as results: (1) .35 for environment to test (p= .07); (2) .34 for environment 
to eps (p=. 08), and (3) .67 for test to eps (p=.002) while thc environmcnt to 
eps path became nonsignificant (p = .48). (Space constraints disallow tables 
here.) Thus, the objective test measuring the learning affected by the 
computer exercise has been shown to mediate the game environmcnt 
treatment and the eps consequences. The stable environment engendered 
more learning and subsequent better performance, and we know that it was 
more effective with male students. 
 

SOME ISSUES TO RESOLVE 
 
Unfortunately, this framework is a panacea only if researchers are more 
diligent than is ordinarily the case. At least three issues must be confronted 
once one subscribes to this approach: (1) the proper measurement of 
learning; (2) the complexity presented by group activities; and (3) the 
choice of moderators. 
 
With respect to the measurement of learning, the researcher must devise 
tests, which assess the specific learning attributable to the experiential 
stimulus. This task may wcll prove frustrating since many experiential 
exercises and simulations embody a host of concepts and touch on a variety 
of underlying principles. The appeal of using a performance consequence as 
a surrogate for learning is great, and undoubtedly has no small part in 
perpetuating the S-R model’s attractiveness. Nonetheless, it is incumbent on 
educators who adopt thc framework prescribed here to develop tests which 
assess the cognitive changes (i.e., learning) which take place as students 
undergo experiential episodes in their classes. 
 
Thc second issue is bothersome when student groups are used. While the S-
R model can easily be applied to group performance, the modcrator and 
mediator aspects of the Baron and Kenny (1986) framework require careful 
attention to individual differences and individual cognitive processes. A 
possible solution is to somehow evaluate individual-level performance apart 
from or as a function of team performance. For instance, in a team situation 
where profitability is the criterion, an instructor may have team members 
rate each other’s contributions and then assess those ratings against the 
team’s performance to yield individual performance measures. This 
approach may be satisfactory as long as the validity of the team members’ 
ratings is not in question. In any case, some mechanism must be applied to 
effectively identify individual performance. 
 
Finally, which modcrator(s) to use must be addressed. The use of 
demographics (sex, foreign-nonforeign student, GPA, etc.) is at best an 
obvious and coarse separation mechanism. Certainly, more subtle 
distinctions exist, and it may well be a major undertaking to ferret out the 
salient traits or circumstances of students which moderate thc outcomes wc 
observe and otherwise naively assume to be uniquely attributable to the 
pedagogy we have chosen to use at the time. 

REFERENCES 
 
Baron Reuben M. and David A. Kenny (1986), “The Moderator-Mediation 

Variable Distinction in Social Psychological Research: Conceptual, 
Strategic, an Statistical Considerations,” Journal of Personality and 
Social Psychology, 51 (6), 1173-1182. 

 
Gosenpud, Jerry (1991), "Evaluation of Experiential Learning," in Guide to 

Business Gaming and Experiential Learning, James Gentry (ed), 
Nichols/GP Publishing, East Brunswick, 301-329. 

 
Wolfe, Joseph (1991), “The Evaluation of Computer-Based Business 

Games: Methodology, Findings and Future Needs," in Guide to 
Business Gaming and Experiential Learning. James Gentry (ed.), 
Nichols! GP Publishing, East Brunswick, 279-300. 

 
Wolfe, Joseph, James Gentry and Alvin Burns (1991), “A Cornucopia of 

Considerations in Evaluating thc Effectiveness of Experiential 
pedagogies,’ in Guide to Business Gaming and Experiential Learning, 
James Gentry (ed.), Nichols/GP Publishing, East Brunswick, 253-278.  


	Table of Contents
	Volume 19, 1992
	The Pedagogical Utility of a Management Simulation Game in a Business Policy Course
	Modeling Economic Environments in Business Simulations: Some Comparisons and Recommendations
	Experiential Learning and TQM Principles: Teaching Behavioral Science in a Business School
	The Use of Poster Presentations as a Final Project in the Business Policy Course
	The Device Business: A Management Simulation for Strategy Formulation
	The Advantages of Experiential learning in the Auditing curriculum
	A Framework for the Identification of Moderated and Mediated Performance consequences of Pedagogical Alternatives
	Simulating Qualitative Research Relating to Values and Lifestyle Segmentation
	A New Market Demand Model for Business Simulators
	Interactive Optimization Using the Method of Relative Improvement Preferences: Methodology and Empirical Evaluation
	The Service Trainer Simulation
	Benefits of Internet Computer Networks for ABSEL Members
	How Should We Measure Experiential Learning?
	An Assessment of Simulation Usage in Management Accounting Courses
	The Influence of Myers-Briggs Type and Group Dynamics on Simulation Performance
	Effective Leadership Behavior in the Desert Storm Arena: An Application of the Vroom-Yetton-Jago Model
	Installing and Consolidating Work-Team Values: The Effects of a Multicultural Outdoors Experiential Program
	Key Determinants and Decision performance in a Business Simulations and Experiential Learning environment
	Multi-Cultural Experiential Learning: A Computer Simulation in Indonesia
	Attitudes Toward and Emotions Related to Women as Managers: A Replication and Beyond
	Scenario Approach to Simulating Consumer Expenditures: A Cross-Cultural Analysis
	Insights into Ethical Decision making Activities and Organizational Performance: A Management Simulation Analysis of College Students and Managers
	Evaluating a Business Simulation Program for Joint Venture Negotiation and Management
	Teaching Business Interviewing Strategies with an Experiential Approach
	Cooperative Learning Across the Business Curriculum
	Modeling Total Quality Elements into a Strategy-Oriented Simulation
	Teaching Business Decision making using a Simulator 
	Extending the Educational Utility of a Simulated Competition within the Confines of an Established Undergraduate Marketing Curriculum
	Expert Systems Versus Traditional Methods for Teaching Accounting Issues
	Confidence Extremes Diminish Quality Performance in a Total Enterprise Simulation
	Can Ethics Be Taught? A Simulation Tests a Traditional Ethics Pedagogy
	Through The Looking Glass, Inc.: Organizational Climate Research as Experiential Pedagogy
	The use of Cluster Analysis for Business Game Performance Analysis
	Power and Ethnicity: An Experiential Learning Exercise (How to Sensitize Students to Diversity)
	Directed Development of Critical and Creative Thinking Skills for Case Analysis
	Implementing Total Quality Management in a Computerized Business Simulation
	Product Quality in Business Simulations
	Use of Simulation for Ethics Education in Management
	Satisfying the University's Customers through Total Quality Management Instruction: A Case Study
	Personality Characteristics and Group Performance in Total Enterprise Simulations
	Concepts of Interval Estimation and Quality Control Charts via Computer Simulated Sampling
	An Examination of the Effect of Team Cohesion, Player Attitude, and Performance Expectations on Simulation Performance Results
	The Effectiveness of Inventory Management and Production Scheduling Training in a Total Quality Management Environment
	Peer Group Indicators of the External Validity of Business Games: A five-year Longitudinal Study
	Political Strategies and Personal Actions
	Does Practice make Perfect? Observations on Simulation Trial
	The Use of A Non-Business Computer Simulation to Teach Marketing Management
	BankPro Commercial Bank Simulation
	The Production Game
	Pursuing Excellence: Work Strengths, Assets and relevant Values (An Exercise)
	Test of a Short Outward Bound Experience for College Students
	What is it that we want Student To Learn?
	Using Two TQM Philosophies when Playing Blackjack
	The Lagged Effects of Decision Variables on Financial Performance Measures Used in Two business Simulations
	The Development of an Experiential Exercise for Career Planning and Effective Job search Performance
	Measuring Quality in Management of Business Pedagogy
	Exploring Quality and Productivity Improvement: Using and Experiential Process
	Toward a Generalized Architecture for Intelligent Reactive Management Systems
	The Quality Game
	TQX: Using Expert Systems to Improve Training in Total Quality Management
	A Simulation of the Effect of the Medicaid Payment Lag on the Financial Position of Community Pharmacies
	The Use of a Board of Directors to Evaluate and Validate Decisions in a Competitive Graduate Management Simulation Course
	The Impact of Academic Dishonesty on Business Simulations and Experiential Learning Activities
	Picture Project: An Experience of Icebreaking and/or Decision-Making
	An Effective Role-playing Exercise for Teaching Requisite TQM Supervisory Attitudes/Behavior
	Computerized Management Simulations and Some Correlates of Students' Satisfaction
	The Timing and Stability of Reactions to Market Structure in a Single Player Simulation Environment
	A Graphics Application Extension for a Simulated Decision Support System Environment
	Aspects of a Group Project Utilizing Actual Business data and a Computerized Accounting System
	The Role of Universities' Extended Learning Department in Assisting Organizations Implement Total Quality Management
	A three-dimensional Learning Experience to Develop Total Quality Management Skills
	Estimating Quality Costs by Computer Simulation
	Applications and Examples of Quality Control Software
	Assessing Business Pedagogy
	A Demonstration of an Experiential Process for Exploring Quality and Productivity Improvement Processes
	An Action-Ethics Dilemma: A Demonstration
	Org Sim Jr.: A 2-3 Hour Version of the 2-Day Blanchard/Murrell Organization Simulation
	Building Buildings: An Experiential Exercise in Organizational Structure, Communication, Leadership and Group Dynamics
	Expert Systems for Organization Design: A Demonstration
	A Demonstration of Product Quality in a Business Simulation: Version 2 of CEO
	Two Revolutions, Total Quality Leadership, and the Baldy: The Story of Milliken
	Total Quality after the Award - The Xerox Story
	Building a Competitive Advantage through Customer Satisfaction and Reengineering
	Human Resource Planning: Managing the Only Renewable Resource for a Competitive Advantage
	Using Simulations to Teach International Issues: An analysis of the Multinational Management Game's Learning Environment
	Quality Function Deployment: A Tutorial
	Care and Nurturing of Teams
	Making a Good Thing Better: Adding TQM to Participative management
	Empowering Organizations to Redesign and Transform Themselves - Eastman Chemical
	Using Simulations in Field Management Development of an International Life Insurance Company
	Reinforcing the TQ Environment via Simulation
	Transforming a Business College into a Total Quality College