Lesgold, A. (2016, September 14). It takes a village of mentors. In S. Tobias, J. D. Fletcher, & D. C. Berliner 
(Eds.), Acquired Wisdom Series. Education Review, 23. http://dx.doi.org/10.14507/er.v23.2139 

 
 

It Takes a Village of Mentors 
Alan Lesgold 

 
 

Reflecting on an extremely rewarding 

career, I am struck by how much it 
depended on a series of  mentors – and of  
course on some lucky breaks. In this 
chapter, I first provide some personal 
history and then consider lessons I have 
learned along the way. 
 
Early Life and Schooling 

I provide some details about my 
early life for two reasons. First, it will be 
evident that many fortunate accidents 
combined to help me get a good start. 
Second, it probably was easier for those 
accidents to occur back then than today. My 
major concern has been to help our 
education systems find ways to restore the 
kind of  supportive environment that 
offered me opportunities in spite of  being 
from a family that was not at all wealthy.  
I am a third generation American. My 
grandparents all grew up in the Pale of  
Settlement, the western edge of  the Russian 
Empire where Jews were permitted to live. 
Almost all Jews in that region were poor, 

with no social support network beyond their 
Jewish neighbors, and they were 
discriminated against substantially. One of  
my grandparents lived with a relative in a 
different town than her parents and some 
siblings, because my great grandparents 
simply could not support all of  them. My 
father’s parents came to America in 1907, 
settling briefly in Delafield, Wisconsin and 
then moving to Milwaukee. My mother’s 
father was subsidized to come to the U.S. in 
1912 by Jacob Schiff, a wealthy New York 
businessman who feared that Jews would 
suffer if  they became too concentrated in 
New York. So, Zaide (my grandfather) 
entered through Galveston in order to get 
part of  his way paid by Mr. Schiff. The 
Russian Revolution and World War I 
intervened before he could bring my 
grandmother and uncle over in 1918. 
Eventually, they also ended up in Milwaukee 
where my parents met and I was born. 

Milwaukee was a city run by 
socialists until I turned 15. As a result of  
that, we had two years of  kindergarten and a 
general atmosphere of  socially shared 
responsibility. There were boxes of  sand and 
ashes on street corners, with shovels in 
them. On icy days, people were expected to 
get out of  their cars and shovel a little grit 
on any slick spots to protect their neighbors. 
Within both the family and the community, 
I had the good fortune to be born in a very 
nurturing place. 

My father was the first person in 
his family to attend college, and he struggled 
at first. He worked more than full time while 
also in school full time, and that did not 
work out. So, he had to leave school for a 
year. Quite possibly, the added stress and 
financial need created by my recent birth 
contributed to all of  that. But, he returned 
to school and graduated in 1948 from the 



Acquired Wisdom/Education Review                                                                                                             2 

 
University of  Wisconsin with a degree in 
electrical engineering. 

My entry into the world was not 
without some stresses. I had a genetic 
problem, hypertrophic pyloric stenosis (an 
overgrowth of  muscle constricting the exit 
valve of  the stomach), and I ended up being 
one of  the first children to have that fixed 
surgically in the months shortly after my 
birth. When I caught pneumonia after 
surgery, I became one of  the first civilians 
to be saved by penicillin, which had been 
reserved for military use during World War 
II. The advanced (for its time) health care 
that saved my life was affordable back then; 
today a poor child with the same problem 
could die, even though the treatment is now 
more routine than it was in January 1946. 
While my good fortune started early, it was 
not financial good fortune. My parents, two 
brothers, grandfather, and I all lived in a 
three-bedroom house for most of  my 
childhood, so my two brothers and I shared 
a single small bedroom. It was good 
preparation for the population density of  
dormitory life at college. 

From the beginning, my father 
taught me that one could solve large 
problems if  one persisted. Indeed, to this 
day I feel a little guilty if  I hire an electrician 
or plumber to do something that he taught 
me to do myself. Even at the age of  3, I was 
recruited to help my father build our first 
TV set, shortly after Milwaukee got its first 
television station. My job was to find the 
right resistors from a large pile. Resistors 
were coded with colored stripes that 
indicated their resistance value. My father 
was color blind, so I had to match the colors 
associated with a particular resistance value 
to the stripes on the resistors and then hand 
him the one he needed next. The resulting 
TV set worked throughout my childhood; I 
occasionally was sent to buy replacement 
vacuum tubes. As an adolescent, when I 
briefly was interested in amateur radio, my 
father and I built a radio transmitter rather 
than buying one. Dad had become an expert 
in magnetics and power supplies; he even 
designed a few components for the Lunar 
Lander that still remain on the moon 

(fortunately, there is an extra lander in the 
Smithsonian so I could show my kids the 
parts grandpa designed). I was no 
electronics whiz, but I did feel that I 
understood the environment in which I 
lived and could shape it to my needs. 

My parents were very nurturing, 
provided considerable intellectual 
stimulation, and also were fully committed 
to public education. When I was offered a 
full scholarship to the best private high 
school in the region; they refused to send 
me there because they didn’t see us as being 
among the privileged. They did send me to a 
variety of  extracurricular opportunities that 
broadened my formation substantially. So, 
for example, I was in an after-school 
program for high school students that 
exposed me to top scientists periodically. I 
especially remember, for example, hearing a 
presentation on peaceful uses of  nuclear 
energy by Edward Teller at which he 
showed movies to support his call for use 
of  nuclear devices to dig a bigger canal 
across Central America. As I saw the side of  
a mountain rippling during a buried nuclear 
test, I remember thinking of  the line from 
Psalm 114: “the mountains danced like 
rams.” While uneasy about Teller’s specific 
goal, I certainly grew up thinking that 
science and engineering could be extremely 
powerful forces for improving our lives. 

My interest in research also was 
stimulated by a summer National Science 
Foundation (NSF) workshop experience. At 
the age of  15, between my junior and senior 
years of  high school, I attended an NSF 
program at Oregon State University. My 
parents had no concerns about me flying 
alone to Corvallis and being in that program 
for a few weeks, over 2000 miles from 
home, because daily life back then assumed 
that people took care of  one another, 
including strangers. The NSF program gave 
me an intuitive grounding for calculus, and 
exposure to the beginnings of  computer 
science (I had the chance to build simple 
computers from relays taken out of  slot 
machines that the local sheriff  confiscated 
from bars). It was the first of  several lucky 
breaks that set the stage for me to use 



It Takes a Village of Mentors                                    3 

 
information technology throughout my 
career and later to teach myself  something 
about artificial intelligence.1  

When it came time to go to college, 
my initial plan had been to attend the 
University of  Wisconsin in Madison. 
However, my parents discouraged this (in 
their later years, they insisted that they did 
not). As a result, another positive force of  
my upbringing, having a wonderful 
girlfriend, eventually was lost, as she did go 
to UW. As this drama was unfolding, I 
received a letter from Michigan State 
University (MSU), informing me that they 
had established a number of  scholarships 
under the National Merit program that were 
available to students who specified MSU as 
their first choice. This was part of  an effort 
to attract strong students there. I made them 
my first choice, and ended up enrolling at 
MSU. This had two major effects on my life. 
First, the woman who became my wife was 
similarly attracted to MSU from Kingman, 
Arizona, eventually leading to us becoming 
one of  the first pair of  Merit scholars to be 
crossbred. 

Second, I entered the MSU Honors 
College, which was in its eleventh year. This 
was important because Honors College 
students at MSU could design their own 
academic programs, subject only to approval 
of  an assigned advisor. This allowed me to 
take a mix of  graduate and undergraduate 
courses, skip such things as introductory 
psychology as I pursued a psychology major, 
and otherwise learn a lot in a more rigorous 
environment. It also probably primed me to 
seek a student job that had better prospects 
for affording learning opportunities than the 
first on-campus job that I had. 

Knowing that I had two younger 
brothers and that college costs were a strain 
for my parents, I worked throughout my 
undergraduate years. Initially, MSU provided 
a job in the dormitory food service. It was 
my privilege to operate the world’s largest 
garbage disposal. For this work, I was paid 

                                                           
1 Recently, I learned from Edward Reingold, who attended the Oregon State program with me, that at 
least seven attendees that year became either professors or scientists in research institutes (some attendees 
could not be located after over 50 years, so the total could be higher).  

95 cents per hour. After a few weeks, I was 
pretty sure that I had learned about as much 
as I would from this particular job, though it 
really was not all that bad (the young are less 
prone to dishpan hands). One day, though, I 
was reading the student newspaper and saw 
a small ad placed by Prof. Charles Wrigley, a 
political psychologist who had formed the 
Computer Institute for Social Science 
Research. The ad indicated that there were 
jobs for Honors College students in which 
they would be trained to do computer 
programming. Moreover, these jobs paid 
$2.50 per hour. Needless to say, this seemed 
like something I should check out. 

I remember a few things about my 
initial interview with Charles. He was from 
New Zealand and had a British education, 
but his knowledge of  how MSU students 
lived was a little sketchy. When I told him I 
worked in the food service, he said “so, your 
job is to take pies out from the kitchen to 
the students?” I filled him in on the 
technological depths I actually had reached. 
He then explained that he was hoping that I 
could join some graduate students in the 
task of  writing programs to do multivariate 
statistical analyses. Specifically, he was 
interested in doing factor analyses of  a large 
body of  data collected on early Peace Corps 
volunteers. He then took a pad of  paper, 
and explained, to someone who had no 
background in statistics or linear algebra, the 
basic ideas behind factor analysis. 
Amazingly, it all made sense and I was able 
to follow his explanations. 

For the remainder of  my 
undergraduate days, three and one-half  
years, I worked in Prof. Wrigley’s institute. 



Acquired Wisdom/Education Review                                                                                                             4 

 
The core task we pursued – and I was one 
of  many involved – was to write a package 
of  software to do multivariate statistics 
(programs like SPSS came a few years after I 
graduated; we essentially built a precursor 
from scratch). So, my undergraduate job 
ended up teaching me computer 
programming and multivariate statistics at 
the cost of  limiting my expertise in quickly 
removing peanut butter from dishes before 
they went through a dishwashing machine. 
My mentor from day to day in this effort 
was Frank Sim, a graduate student in 
sociology. I also learned a lot about scaling 
from Louis McQuitty, who at the time was 
dean of  social sciences and a co-founder of  
the Computer Institute for Social Science 
Research. And of  course, Charles Wrigley 
was a wonderful first mentor. 

There was another experience that 
occurred as a result of  my work in the 
Institute. In 1964, when I was a sophomore, 
I was asked to teach a course for high 
school students in FORTRAN 
programming. This was my first teaching 
experience, and it went reasonably well, I 
think. A couple years later, a national 
magazine had an article suggesting that this 
was the first programming class ever for 
high school students, but I am pretty sure 
someone at MIT had done it earlier. Overall, 
Michigan State provided incredible 
opportunities. I got a good grounding in 
psychology, including some graduate 
courses. I learned the essentials of  
multivariate statistics and scaling. I learned 
both how to program and that I was quite 
capable of  getting computers to do things. 

Computers were harder to use back 
then. For example, in order to carry out a 
number of  statistical analyses on the Peace 
Corps database or the U. S. Census database, 
we needed to develop algorithms that were 
efficient in their relative demands on huge 
data tapes versus very limited core memory. 
Reading data from tape was extremely slow, 
so we developed schemes to pre-screen the 
set of  analyses that a user wanted to 
perform. Our program would figure out 
how many analyses could be done on a 
single pass through the data tape and then 

accumulate the various sums of  squares and 
cross-products needed for those analyses 
into core memory all at once rather than 
doing the analyses one at a time. By limiting 
the number of  passes through tape (actually, 
for the census it was a set of  several large 
tapes that had to be mounted by hand one 
after another), we made the program run a 
lot faster. Years later when I took an 
operations research class and studied formal 
techniques for optimization, I learned that I 
could have done better at this than my 
intuitions suggested at the time. I also 
learned from experience with people who 
used the statistical programs that it was 
worthwhile, before printing them, to sort 
the analyses to match the original order 
requested by the researcher. These 
experiences remind me that many students 
can stretch far beyond what we routinely ask 
of  them. Fortunately, Charles Wrigley 
invested in stretching me. 

It is somewhat amusing as a dean 
of  education to recall that MSU also 
provided the only teaching certification I 
have ever received. There was, at the time, a 
requirement that students complete three 
recreational physical education classes in 
order to graduate. My sporting interests 
back then were limited. So, I succumbed to 
the temptation to take, as one of  my 
requirements, a course in trap shooting. This 
actually provided a lot of  exercise, since the 
firing range was a couple miles away from 
the campus, in a direction without mass 
transit. I learned how to hit the broad side 
of  a barn, and while many of  the clay 
pigeons released when I was shooting were 
able to be recycled, I apparently 
accomplished what was required in the 
course. To my surprise, about a month later, 
the State of  Michigan issued me credentials 
as a certified hunter safety instructor, which 
remains my only government-issued 
teaching credential. 

Another important set of  
experiences at MSU was the opportunity to 
take several courses in logic and philosophy 
of  science from Gerald Massey. I was 
probably in Jerry’s first undergraduate class, 
and he taught me a lot. Later, when I got to 



It Takes a Village of Mentors                                    5 

 
the University of  Pittsburgh, Jerry was my 
colleague and chaired Pitt’s Department of  
Philosophy. Eventually, when I helped 
found Pitt’s Intelligent Systems Program, I 
helped hire Jerry’s first PhD student, Bruce 
Buchanan. In some ways, the academy is a 
small world. 

In 1966, during my junior year, I 
began to think about next steps after MSU. 
With advice from Charles Wrigley and 
others, I ended up applying to psychology 
programs at Michigan, Stanford, Penn, and 
Wisconsin. That was about the order of  my 
preference as well. Michigan had strengths 
in quantitative psychology and also in the 
emerging space of  information processing 
approaches to cognition, precursors to the 
cognitive psychology that would be evolving 
as I went through graduate school. Stanford 
was in the midst of  its era of  mathematical 
psychology. Initially, I planned to attend the 
University of  Michigan. I was intrigued by 
the thinking of  Walter Reitman and also had 
connected with Clyde Coombs, so both 
information processing ideas and 
quantitative methods were attractions there. 

Once again, as in my choice of  
undergraduate institutions, chance 
intervened. Sharon and I were planning to 
be married just before graduation, so the 
choice was now a two-person matter. Just 
before the deadline for accepting offers, 
Stanford wrote Sharon indicating that she 
had been awarded a Shell Fifth Year Merit 
Scholarship. Now, after four years at MSU 
that included many subzero days and a 
three-foot snowfall, this Arizona girl really 
was ready for someplace warmer. In my 
mind, the opportunities at Michigan and at 
Stanford were both exciting, so at the last 
minute we decided to go to Stanford. The 
graduate student who had been assigned to 
work with me initially in Charles Wrigley’s 
Institute, Frank Sim, was already at Stanford 
as an assistant professor of  sociology. So, I 
was going to Stanford already knowing 
someone there. 

Frank soon asked whether I would 
like a summer job at Stanford before I 
started my studies there in September. That 
seemed like a great idea, and what was 

needed there was to translate and expand 
statistical software programs to run on 
Stanford’s new computer, the IBM 360. 
Back in those days, changing computers 
often meant rewriting software substantially. 
Especially noteworthy that summer was the 
opportunity to get to know Sanford 
Dornbusch. Sandy, a fine sociologist, was in 
the midst of  his study of  high school 
students at the time, and one of  his findings 
has stayed with me since then. He found 
that students doing less well in school 
believed that they were working harder and 
longer on their homework than the better 
students, but that in fact the better students 
were putting in more time. The lesson for 
me was not the one that would emerge later 
in Carol Dweck’s work (that less successful 
students often believe that learning is due to 
aptitude rather than effort) but rather that 
students doing poorly really don’t have a 
good idea of  how much they are learning or 
of  how hard it will be to learn more. 
Decades later, while studying how children 
learn to read (or don’t), I would return to 
this basic idea that students who struggle 
need help both in recognizing their 
successes and in gauging how much effort 
to invest in learning. 

A second, more poignant, lesson 
came that summer as well. One day, Frank 
told me that he was going to leave Stanford 
to go to Penn State. Now, it takes very little 
time for someone from the upper Midwest 
to figure out that Stanford is part of  
paradise. So, I asked Frank what had 
prompted him to leave The Farm for Happy 
Valley. What he told me was another 
important life lesson that I have passed on 
several times as dean: “Well, Alan, when you 
walk into your chairman’s office and tell him 
you just got a great offer, be prepared for 
him to shake your hand and wish you well.” 
Painfully, Frank died quite young. I’ve always 
wondered whether he would have done 
better being treated at Stanford Hospital. 

Late in the summer, it seemed 
appropriate to get to know some of  the 
people in psychology at Stanford, since I 
was about to begin my graduate studies with 
them. I learned that the faculty and graduate 



Acquired Wisdom/Education Review                                                                                                             6 

 
students in mathematical psychology held a 
weekly research seminar on Friday 
afternoons, so I decided I should start going 
to that. As luck would have it, I picked a 
Friday when the faculty were having a 
farewell lunch for a visiting colleague. So, 
they all showed up quite mellow from that 
event. Gordon Bower, truly one of  the 
finest and most humane mentors I have had, 
spotted me and inquired most exuberantly, 
“Who the hell are you?” I introduced myself  
and mentioned that he had been assigned as 
my advisor. So it was that I met the person 
who really taught me to be a professor. 

Working with Gordon Bower was 
an incredible experience. He was, of  course, 
an exceptional scientist, something 
recognized later with a National Medal of  
Science and other major accolades. He also 
was an exceptional writer and 
communicator. And, he was a great model 
of  good research strategy. While I started 
out biased to do complex experiments that 
dealt with every possible subtlety that might 
arise, Gordon taught me to focus first on 
making sure that there was a core 
phenomenon that could be established 
strongly and without doubt and then to 
focus in on details. Regrettably, in the 
current age of  micromanagement of  
research strategy in the grant making 
process, this lesson is sometimes lost. On 
the other hand, better statistical and data 
management tools now do allow initial 
studies to be larger in scale and 
consequently able to answer more questions. 

Mathematical psychology at 
Stanford back then was an intense 
intellectual effort. Absent powerful 
computers, theories were expressed as 
complex sets of  equations which then were 
solved by hand. To be a good mathematical 
psychologist, one had to be an exceptional 
mathematician. Gordon Bower was one 
such person, and there were others, 
including Bill Estes, Dick Atkinson, Walter 
Kintsch, and especially Duncan Luce. Some 
were continuing members of  the Stanford 
faculty and others were visitors. The overall 
level of  mathematical analysis skill in 

psychologists at the time was quite 
incredible. 

Stanford had a rule that every 
doctoral student had to complete four 
courses outside of  the degree-granting 
department. This resulted in me taking 
courses from Pat Suppes, Mike Arbib, and 
some math and operations research faculty, 
filling in a lot of  gaps in my mathematical 
expertise and further preparing me to learn 
about artificial intelligence. Stanford also 
required that the milestone exams – 
comprehensives and dissertation – be 
chaired by someone outside the degree-
granting department. My comps, as a result, 
allowed me to meet a bright young 
computational linguist, Roger Schank, from 
whom I also have learned on and off  over 
the decades. 

The years I spent at Stanford, 1967 
to 1971, were the period in which 
experimental psychologists were beginning 
to move from an associationist viewpoint 
anchored in behavior theory to an 
information processing viewpoint that 
would become the basis for cognitive 
science. It was a privilege to be doing this 
learning myself  along with my teachers. 
Another student of  Gordon Bower’s, John 
Anderson, helped shape my thinking and 
that of  many others during that period. 
John made the critical observation that 
knowledge could be represented by rules 
that captured what mental process should 
take place as a function of  what content was 
currently active in memory. In essence, he 
moved the behavioral notion of  
connections between external stimuli and 

motor responses into the head, where the 
connections were between working memory 
content and mental (or physical) acts.  

In a paper that few have read but 
that was of  great importance, Pat Suppes 



It Takes a Village of Mentors                                    7 

 
(1969) showed that the rules Anderson was 
working with could capture the capability of  
any finite automaton. Any thinking that 
could be carried out by a computer could be 
captured by the system of  rules Anderson 
used to capture cognition, and thus the task 
of  cognitive psychology in large part was to 
establish how rules get learned. Moreover, 
Pat Suppes’ theorem suggested that what 
can be understood about human thinking 
can be embodied in computer programs. 
From this beginning came the best of  
current learning theory as well as the 
foundations for intelligent tutoring systems. 

Those Friday seminars were my 
first introduction to Stanford’s psychology 
department and were continued throughout 
my four years there. Each week, a student or 
faculty member or visiting scholar would 
present their latest work, affording all of  us 
another view of  the evolution from 
statistical learning theory to cognitive 
psychology. It was the ultimate school for 
scholars as well as an intellectual paradise, 
even on the not-too-infrequent days when I 
could not follow the complex mathematical 
arguments fast enough and took most of  
the next week to figure out, even partially, 
what I had heard. 

There were certain bonuses to 
working with Gordon Bower. He was an 
incredible crafter of  both journal articles 
and oral presentations. And, Sharon Bower, 
his wife, was also available to bring the 
perspective of  a drama coach to bear on our 
rehearsals before our debut presentations at 
professional meetings. Sharon’s advice still 
comes to mind as I prepare and give 
presentations, especially her ability to focus 
on each sentence and ask: “what will the 
listener understand from what you just 
said.” To this day, my only decent 
presentations are those I prepare using what 
she taught me. 

Gordon had incredible writing 
skills in addition to his amazing 
mathematical and statistical ability. He 
would eyeball spreadsheets of  data we 
showed him and, if  he had trouble getting 
to sleep that night, he would appear the next 
day with a complete results section written, 

asking us to check the statistics, which were 
always very close to being perfect. He 
eyeballed the data during the day, wrote the 
results up without notes at night, and 
seldom was off, even on the detailed 
statistics, by more than a few percent. I did, 
after about twenty years of  research 
experience, discover a few tricks to doing 
this, but the bottom line is that Gordon’s 
understanding of  distributions and 
probabilities was superb. At the same time, 
Gordon also modeled the importance of  
taking the time to properly prepare a 
presentation or article. He could have 
winged it on many occasions, but he didn’t. 
We often got early drafts of  presentations 
and papers for comments, and his questions 
about our presentations also modeled the 
kind of  reflection needed to communicate 
well. 

It would be nice to be able to say 
that by the time I left Stanford, I had 
learned both to do good research and to 
write it up well, but in reality it took me 
another decade to get even halfway decent 
at this, and it took additional mentors. It 
also is important to recognize that others at 
Stanford shaped my development as a 
scholar, including Dick Atkinson and Herb 
Clark, who also were on my doctoral 
committee and visiting scholars like Walter 
Kintsch and Jim Greeno. Each taught me a 
lot. Finally, there were other students, both 
of  Gordon and of  other faculty, who were 
an incredible source of  wisdom. A number 
of  them were Stanford classmates who later 
became colleagues in Pittsburgh at Pitt or 
Carnegie Mellon, including John Anderson, 
Pat Carpenter, Marcel Just, Lynn Cooper, 
Bobbi Klatzky, and Ruth Day. I could go on 
recalling additional classmates, but I surely 
would miss some. The quick summary is 
that the combination of  incredible faculty 
and amazing fellow students made it much 
more likely that I would emerge from the 
graduate student experience on the path to 
doing some decent work. 

There were other strokes of  luck 
along the way as well. Recall that while I was 
a graduate student, the United States was 
fighting a war in Vietnam. In the midst of  



Acquired Wisdom/Education Review                                                                                                             8 

 
my studies, lottery numbers were issued by 
the Selective Service. One classmate, Gary 
Olson, got a low number, which meant he 
would be drafted. He was able to get a 
commission in the Navy and served his time 
as an officer at the New London submarine 
base. This allowed him to continue doing 
research part of  the time and also to be 
certified as a hard hat diver. 

My luck was a little different. I 
drew a borderline number that turned out to 
be low enough to be a problem. But, I was 
lucky. I had been a pretty chunky guy as an 
undergraduate, but early in my graduate 
studies, I began to lose weight. When called 
for a physical, I probably was eligible to 
serve. However, the person who measured 
my height concluded that I was about three 
inches shorter than I believe I was at the 
time (of  course, the Army is always right, so 
I can’t be sure). As luck would have it, my 
weight was a few pounds too high for a 
short person. I was worried that they would 
call me back and reach a conclusion more 
consistent with my own observations, so 
before leaving the place where they gave 
physicals, I got the Navy representative to 
endorse my physical with a statement that it 
was sufficient for a non-line officer’s 
commission. Periodically, over the next few 
years, my draft board would ask me to have 
a physician report my weight. This was done 
accurately each time. However, the system 
never felt the need to reassess my height. So, 
I was able to complete my studies 
uninterrupted – something that probably 
was extremely helpful to any success that 
may have come along later. 

As I approached the end of  my 
studies at Stanford, I had two job interviews. 
One was at Oklahoma State, and they loved 
me – I suppose that I could have ended up 
later with fellow Stanford alumnus Bob 
Sternberg as my provost had I gone there. I 
also interviewed at Columbia. I fared less 
well there, perhaps because I gave the wrong 
answer when they asked me how much I 
liked New York. Then, some interactions 
began with the University of  Pittsburgh. I 
forget the order of  events, but within a 
short period, Jim Voss, who was on 

sabbatical from Pitt, stopped by to chat and 
I was invited by Bob Glaser to interview at 
Pitt. My interview was scheduled 
immediately before a conference being held 
at Pitt’s Learning Research and 
Development Center (LDRC), and I was 
invited to stay for the conference. The 
resulting book from that meeting, edited by 
Tulving and Donaldson (1972), recorded the 
transition from associationist theories of  
learning to a cognitive view, at least for a 
major chunk of  the experimental 
psychology world. Needless to say, I left the 
interview pretty convinced that Pitt was an 
important node in the experimental 
psychology research world. Among the 
researchers at that meeting, incidentally, was 
Gordon Bower. Given all these occurrences, 
it probably was inevitable that I would end 
up at Pitt. Like the accident of  going to 
Stanford, this lucky break played a major 
role in whatever successes I have had as a 
scholar. 

Having spent my graduate years 
amid the shift from associationist thinking 
to cognitive science, I have spent my 
professorial years in an institution that has 
become truly great and that is evolving to 
make important contributions to our world. 
In contrast to the dusty image of  a never-
changing professoriate, my experiences have 
been with people who were strong scholars, 
respected the accumulated knowledge of  
earlier colleagues, but also were ready to 
challenge their own beliefs and to search for 
ways to make what they learned useful.  

The early years at Pitt were 
extremely helpful. Bob Glaser was an 
excellent mentor and gave me conviction 
that research could indeed improve 
schooling. He also supported my sense of  
adventure (and modeled such a sense 



It Takes a Village of Mentors                                    9 

 
himself), which helped me make some of  
the shifts in my scholarly work over the later 
years. Bob’s own protégé, Lauren Resnick, 
also taught me a lot. She, too, took 
educational research seriously and shared 
with Bob a strong sense of  responsibility for 
taking what one learns and helping the 
world make use of  it. While earlier mentors 
taught me that I could push myself  to think 
really deeply, Bob and Lauren taught me 
that with further effort I could push my 
ideas far enough to be useful in learning and 
training. They also taught me a lot about the 
leadership of  academic enterprises (I was 
their associate director for fourteen years 
before becoming a dean), which helped 
years later when I became a dean. 

Before discussing some of  my 
work and the lessons to be drawn from it, I 
should add one more historical note that 
might be helpful to future colleagues. While 
Pitt was an adventurous place, especially 
within the LDRC, no university operates 
with uniformity in any respect. When it was 
time to be considered for tenure, my record 
was adequate but probably not the strongest 
to pass through Pitt. Nonetheless, the 
faculty in the psychology department, where 
my tenure stream appointment was located, 
voted clearly in favor of  tenure. However, 
the chair turned in a negative 
recommendation, believing that further 
movement of  the department toward 
applied research was a bad idea (and I 
certainly had made clear by then my interest 
in seeing my work have impact in real 
educational settings). So, I was not granted 
tenure. Bob Glaser was kind enough to keep 
me on as a research assistant professor at 
LRDC.  

Several years later, the University of  
Delaware offered me a tenured full 
professorship. By then, Pitt’s psychology 
department had a new chair. I turned down 
his offer of  a tenured associate professor 
position, and was about to accept the offer 
from Delaware when Pitt made me a full 
professor with tenure. So, I have never been 
an associate professor. Had I gone to 
Delaware, my work would have been more 
heavily in educational technology, but it  

Learning and Research Development Center 
(LDRC) 

 
probably would not have been as 
adventurous as it became at Pitt with a joint 
appointment in the LRDC.  

 
Lessons Learned Beginning My Career 

My assignment in this chapter is 
not only to provide some background about 
my career but also to pass on lessons I have 
learned. Perhaps the most important lesson 
is to seek and learn from mentors. Good 
mentors are sometimes tough to deal with. 
If  they are really good, they challenge one’s 
work just when one is getting happy with it. 
At the same time, good mentors tend to 
build two-way relationships with their 
students/colleagues. I always felt that my 
mentors and I were engaged in a common 
enterprise of  building publicly shared 
understanding.  

A second lesson from my early 
years, and also later on, is that accidents are 
opportunities. Whether it was several years 
without tenure at Pitt or being at Stanford 
instead of  Michigan or having to work while 
a student at MSU, there were ways to grow 
presented by each accident. I report only the 
ones I was wise enough to respond to; I’m 



Acquired Wisdom/Education Review                                                                                                             10 

 
sure there were many other opportunities 
squandered. 

Related to this is another lesson, 
which is that it is worth investing a lot of  
effort in the occasional adventure. I could 
easily have persuaded myself  that I was not 
qualified to work for Prof. Wrigley. Instead, 
I took the chance, and that experience has 
shaped much of  my career. On a number of  
occasions described below, I decided to be 
adventurous. One could argue that I simply 
exhibited excessive hubris. I think the 
difference between hubris and adventure is 
the extent to which one decides that doing 
something new will take a lot of  hard work. 

A third lesson really has two parts. 
The first part is that scholarly success 
involves a huge investment of  effort, 
whether it is work in junior high school or 
preparing a conference presentation or 
responding to editorial review of  a journal 
article. For the scholarly world, though, 
there is another part to the lesson, which I 
have tried to pass on to my students. 
Scholarly success does not come from 
having brilliant ideas. Rather, it comes from 
being able to explain simply the ideas one 
has developed after extended effort. 
Developing those explanations, the art of  
scholarly writing, also requires huge effort. 
At least for me, the task keeps getting 
harder as I become more able to assess what 
I have written and to assess who will and 
who will not understand that writing. 
 
Research on Learning and on Reading 

I turn now to the work I have done 
since completing my doctorate. My work for 
the first ten years at Pitt was initially on the 
role of  syntax and other structural 
constraints on what is learned from texts, on 
techniques for enhancing recall, and, near 
the end of  that period, on the course of  
learning to read. In this work, I had some 
great colleagues. I did several years of  work 
jointly with Charles Perfetti and also did a 
longitudinal study of  reading acquisition 
with Lauren Resnick. Each line of  work 
taught me important lessons. 

In the work with Perfetti on 
comprehension of  texts, I eventually 

realized that not everyone learns from texts 
the same way. Now, in this age of  often 
unsupported claims of  differing learning 
styles, that may not seem all that striking a 
conclusion. However, what I saw in our 
work was a bit different from current 
assertions about learning from pictures 
versus learning from texts. In a conference 
presentation for which I have lost the 
reference, I suggested a three-fold path to 
learning from text. Some people, I argued, 
have so much prior knowledge of  the topic 
being addressed by a text that even if  their 
reading ability is limited, they can figure out 
what is going on. Others primarily leverage 
incredibly facile ability to decode individual 
words and sentences and to keep their 
content in mind long enough to get 
connected to the content of  the next 
sentences encountered. Still others have a 
repertoire of  general comprehension tactics 
that allow them to make sense of  texts even 
when they don’t fully understand everything 
that is being discussed. Of  course, really 
good readers leverage all three of  these 
sources of  reading capability, but different 
students often are weak on one or another 
of  them.  

This is important because it means 
that unified process models of  reading that 
are purely sequential and that assume 
specific results from each element of  the 
reading system will not easily be a good fit 
to all readers. For some, early stages will be 
incomplete in what they generate, while for 
others, the earliest stages of  the process will 
drive the entire outcome. Prior knowledge 
may result in content being guessed by one 
reader while being extracted explicitly from 
text by another. There are no simple 
universal sequential process models of  
effective reading, though there are certainly 
normative patterns worthy of  attention. 

This lesson that not everything is 
simple came home to me in another way. 
When Lauren Resnick, Kathy Hammond, 
and I did our longitudinal study of  the first 
four years of  reading acquisition (Lesgold, 
Resnick, & Hammond, 1985), we were 
interested in the relative efficacy of  whole 
word versus phonics instruction. So, we 



It Takes a Village of Mentors                                    11 

 
developed structural equations models that 
allowed us to assess the extent of  phonics 
instruction and the extent of  whole word 
instruction and then to see which was a 
bigger driver of  eventual reading success. 
The master plan was that we would follow 
some students in a reading program that 
emphasized phonics and others in one that 
emphasized the whole word approach and 
see which group made faster progress in 
learning to read. We did this, and we 
developed a structural equations model 
based on all of  our data. It showed relatively 
similar contributions of  the two approaches. 

Given my long experience with the 
kinds of  statistical ideas behind structural 
equations modeling (going back to my years 
of  writing multivariate statistics programs 
with Frank Sim), I was worried about 
making strong claims about the results of  
such modeling without digging a lot deeper. 
I first built about 70 different models, 
testing each. I just wanted to be sure that 
some tiny change in the model wouldn’t lead 
to very different conclusions. Then, I 
decided to model separately each of  the two 
samples, the students given a lot of  phonics 
and those given a lot of  whole word 
preparation. I discovered that students who 
got heavy phonics had phonics as the 
primary determinant of  their overall reading 
acquisition success, while students who got 
a whole-word approach had measures of  
whole word recognition as the main 
determinant of  overall reading competence. 

Now, it could have been that both 
approaches work but that they teach 
different ways to read. That would have 
been consistent with the earlier idea of  the 
three-fold path to reading success. However, 
it turned out that another hypothesis also 
explains the data really well. Students who 
get phonics training perform more 
consistently on phonics tasks, having 

                                                           
2 Since then, some rather nice brain research data has shown the importance of  phonics-type training 
(see, for example, Yoncheva, Wise, & McCandliss, 2015). 
3 Dornbusch described his finding to me orally during that summer before I started graduate study. It was 
mentioned in a Stanford alumni magazine later on, but I have not found it stated completely explicitly, 
though it was part of  Dornbusch’s broader study of  high school students (see Natriello & Dornbusch, 
1984, P. 56). 

become more practiced; there is less error 
variance in measures of  their phonics skills. 
Students who get whole-word training 
become more reliable and consistent in 
whole word recognition; they end up with 
less variance in measures of  whole word 
recognition. When a measure has higher 
“error” variance, it will have a lower 
correlation with other measures. In essence, 
it is entirely possible that some of  what gets 
learned from either method is how to 
perform reliably on measures of  the part 
skills being taught and thus to assure a 
higher correlation of  those part skills with 
overall reading comprehension.2 The lesson 
to be learned is that it is dangerous to use 
any complex analytic approach without 
deeply understanding the underlying 
mathematics and without extensive testing 
to make sure that what initial analyses seem 
to be saying is really what is going on. 

In working with students who 
struggled to learn to read, I noticed that 
while some faced special barriers, often they 
were, like the students studied by Sandy 
Dornbusch,3 not accurate in their 
perceptions that they were working harder 
than their peers. I ended up deciding that I 
could only understand what it was like to 
struggle in school if  I undertook myself  to 
learn something that was a struggle for me. 
Professors tend to come from among those 
who find school learning easy, so I needed 
to find something to learn for which I had 
no reservoir of  skill on which to build. I 
chose oriental ink painting. I found a 
wonderful teacher, Louisa Yee, and had 
lessons from her at least once a month for 
three years.  

I practiced painting, probably less 
than great painters but probably more often 
than it should take to become good. To 
emulate the ink-spreading characteristics of  
high quality Asian papers, I practiced on 



Acquired Wisdom/Education Review                                                                                                             12 

 
rolls of  the paper doctors spread on 
examining tables. It is cheap and unsized – 
ink runs on it just as it does on high quality 
rice papers. At the end of  the three years, I 
felt that I had learned nothing. I mentioned 
this to my wife, who then dug out all the 
pieces of  paper on which I had practiced 
over three years and showed me that there 
was a clear progression toward better results. 
This was a shock to me, and even today I 
get emotional when I think about it. I 
learned an important lesson. People evaluate 
their capability with reference to competent 
performance that they see around them. It is 
easy for them to notice when they are 
substantially behind their skilled peers. It is 
much harder for them to notice the tiny 
incremental improvements that come each 
day with practice. Many years later, when 
thinking about a possible motivation 
research center in the school of  education 
that I lead, this lesson helped shape my 
thinking about the scope of  work the center 
should embody. 

 Another important experience 
occurred during those early years. In trying 
to understand how learning and memory 
processes might work, I became interested 
in the possibility of  using artificial 
intelligence to build cognitive models. More 
broadly, I had become interested in the 
possibility that computer power could 
extend human power in theorizing. This led 
me in directions that eventually proved 
central to the best work of  my career. Jim 
Greeno and I began more or less 
independently to explore some of  the rule-
based systems that were emerging as the 
field of  artificial intelligence began to 
develop.  

With the help of  Harry Pople and 
Jack Myers, two Pitt colleagues, we arranged 
access to a specialized system that ran at 
Stanford on a computer that Joshua 
Lederberg and colleagues had acquired. At 

                                                           
4 There is an interesting tidbit in this story. Since Pitt did not yet have connections to the 
Arpanet/Internet, Jim and I were given accounts at CMU. However, eventually CMU decided to create 
the domain name of  pitt.edu for the several of  us from Pitt whom they hosted. With Jim’s retirement and 
the unfortunate passing of  Harry Pople and Jack Myers, I am pretty certain that I have the oldest email 
address at Pitt. 

that time, in the late 70’s and early 80’s, 
personal computers were not available and 
nothing small was anywhere near powerful 
enough to support rule-based systems. The 
SUMEX-AIM system that ran at Stanford 
primarily supported work there by Bruce 
Buchanan, Ted Shortliffe and others on 
MYCIN, a system for diagnosing infectious 
diseases. But, there was enough spare 
capacity that it could be used for other 
intelligent system and modeling efforts. The 
recently-developed ARPANET connected 
Carnegie Mellon University to Stanford, and 
Jim and I partnered with the Pople-Myers 
Internist project (an expert system for 
internal medicine diagnosis) to rent a high-
speed telephone line from Pitt to CMU, 
which in turn connected to the ARPANET 
and thus to Stanford.4 

If  nothing else, this connection 
allowed me to teach myself  enough about 
artificial intelligence to later do some useful 
work. I did a lot of  modeling of  the 
learning processes on which I was doing 
experimental work, although the models 
themselves did not rise initially to being 
worth publishing on their own. What 
became clear very quickly was that artificial 
intelligence tools would afford the 
opportunity for significant change in how 
learning and thinking were studied. There 
was another lesson learned through this 
involvement. From time to time, I would 
need one or another utility program (e.g., a 
sorting function) on the SUMEX-AIM 
system for my work. I would search for the 
utility on the system and access its 
documentation. Often, I found that the 
tools I needed had been written by Prof. 
Lederberg. Another lesson: great scientists 
make their own tools rather than being 
constrained by what is easily available. 

By 1982, a few important things 
had happened that shaped a big chunk of  
my career as well as much of  the field of  



It Takes a Village of Mentors                                    13 

 
cognitive science. At the time, the primary 
language for software that supported 
artificial intelligence was Lisp. But, most 
computers in general use had been built to 
support numerical computation. So, in 
laboratories at MIT and at the Xerox Palo 
Alto Research Center (PARC), people were 
working to develop specialized hardware to 
run Lisp (routine computer power was 
barely up to this task, so specialized 
hardware was necessary). The Learning 
Research and Development Center at Pitt 
had acquired a VAX computer in 1979 to 
support cognitive science work, but even 
that machine was barely up to handling 
serious machine intelligence. Eventually, 
both MIT and Xerox PARC developed Lisp 
machines, and it was clear that with this kind 
of  tool, serious modeling of  cognitive 
processes could now be done on a scale 
previously unimaginable. The machines 
were small enough to fit in someone’s office 
and designed as personal machines (the 
smallest was the size of  a two-drawer file 
cabinet).5 

But, as some of  us involved in this 
work reflected on the future, we saw a 
chicken-egg problem. To do the work that 
these new tools enabled, one had to get one 
of  the new machines. But, there were many 
barriers to that happening. First, while 
Xerox PARC had invented potentially 
commercial versions of  the Lisp machine, 
Xerox as a company really saw the 
platforms only as supporting office uses, not 
being specialized for artificial intelligence. 
Second, there was no maintenance 
infrastructure for the machines, and they 
had the kind of  failure rate that required 
keeping technicians continually on call. 
Third, they were relatively costly, more than 
20 times the cost today of  a very high end 
desktop computer. That meant that 
researchers needed to be able to get grants 
to cover the costs. But, researchers didn’t yet 

                                                           
5 The Lisp machines did generate a lot of  heat, though. Years later, on a cold winter day long after 
desktop computers were powerful enough for artificial intelligence work, I discovered that one of  my 
colleagues had resurrected one of  the machines to provide extra warmth in her office. 
6 Among other things, the supply chain involved literally a little old lady in San Jose whose TV repair shop 
always seemed to have the right components in stock. 

know all that much about the equipment, 
and funding agencies didn’t yet have a basis 
for evaluating proposals. Fourth, there was 
need for considerable training before a cadre 
of  researchers would be able to make 
serious use of  the new tools. 

The training problem began to be 
addressed via periodic training courses that 
Xerox and major AI centers would offer 
during summers. That still left the chicken-
egg issue of  funding and infrastructure. So, 
somewhere around 1982, Henry Halff  of  
the Office of  Naval Research (ONR), John 
Seely Brown of  Xerox PARC, and I had a 
brief  chat in a hotel room while attending a 
research conference. This led to a plan to 
place the new Lisp machines in a number of  
laboratories around the country. Researchers 
in those places were selected based upon 
their recent publication and funding history. 
ONR provided the funding (from 1983 until 
1990, when specialized machines were no 
longer necessary). Pitt provided the central 
home and an ONR-funded small support 
team that researchers could call when 
machines failed. Xerox PARC created a 
somewhat-informal6 arrangement that 
assured that components of  the Lisp 
machines could be purchased if  a part failed 
in one of  the units. I remain extremely 
grateful for the flexible thinking and 
imaginative work that Drs. Halff  and Brown 
provided to bootstrap the beginnings of  a 
serious use of  artificial intelligence tools in 
cognitive theorizing in the U.S. Shortly 
thereafter in the UK, Rank Xerox stimulated 
a similar deployment of  Lisp machines in 
European labs. 

There is another lesson here. All of  
the effort to stimulate the development of  a 
cognitive science anchored in artificial 
intelligence efforts that could be empirically 
studied required a huge level of  trust. Xerox 
was not focused on selling Lisp machines, 
so John Seely Brown had to be trusted to 



Acquired Wisdom/Education Review                                                                                                             14 

 
stay with the effort long enough to make it 
work. The Office of  Naval Research had to 
trust that researchers who were given access 
to these machines would put in the time to 
master them and would use them to do 
better work than otherwise would have been 
possible. Both parties had to trust the Pitt 
group to be impartial and altruistic in their 
distribution and support of  the Lisp 
machines. This was not that different from 
Milwaukeeans trusting that neighbors would 
put ashes on slippery ice patches in the 
winter or my parents trusting that someone 
would meet my plane in Oregon when I 
went to the summer NSF workshop. It is, 
though, quite different from the model of  
social goals that infects much of  our 
political life today. It also is different from 
the much more conservative funding 
approach now favored in the federal 
government generally. End of  sermon. 

This early start in using artificial 
intelligence tools fueled my desire eventually 
to develop a technology of  intelligent 
tutoring systems to teach complex problem 
solving, and of  course it also accelerated the 
desire of  my former classmate John 
Anderson and his colleagues to develop 
intelligent tutoring systems for teaching 
mathematics. But, in my case, some work on 
expertise intervened between this early start 
and my eventual development of  intelligent 
coached apprenticeship systems. 
 
Research on Expertise 

 Around 1980, there was quite a bit 
of  work emerging on expertise, stimulated 
by the seminal work of  Chase and Simon 
(1973). I was beginning to view reading as a 
form of  expertise, and I was wondering if  I 
could understand more about how to instill 
high levels of  expertise by looking at some 
other domains. Bob Glaser and I discussed 
the question of  what area of  expertise to 
explore, and we decided to focus on an area 
that involved some level of  perception, 
since perceptual expertise had not been 
explored much. We considered several 
topics, narrowed our choices to 
crystallography and radiology because of  

local expert availability, and ended up 
deciding to look into radiological expertise.  

We did some very interesting work 
on radiology expertise (Lesgold, 1984), and 
some of  it has continued to be cited as time 
passed. One of  our findings taught me 
another lesson. We had focused some of  
our work on interpretation of  very 
anomalous x-ray images. For example, we 
had a chest x-ray of  a person who had a 
lung lobe removed quite a bit earlier. 
Because the lungs behave a bit like balloons, 
when one lobe is removed, there is a 
considerable amount of  unoccupied chest 
space that eventually becomes occupied as 
other components shift under pressure from 
expanding and contracting lobes that are still 
there. In the case I recall, all of  the organs 
shifted around because of  a missing upper 
right lobe, so everything was out of  its usual 
place in this person, in addition to signs of  
the specific new disease process the person 
had. Images like this were difficult enough 
to allow us to see differences among three 
groups, first and second year residents, third 
and fourth year residents, and experts with 
over ten years of  experience. 

Because of  the limited number of  
such films, we used them in multiple studies. 
Indeed, by accident we used a few with the 
same radiologists at both earlier and later 
stages of  their residency training. We found 
that some images that were correctly 
diagnosed by early-stage residents were 
sometimes misdiagnosed by the same 
residents a year or two later, after they had 
acquired experience with thousands of  
additional images. Our seasoned experts did 
as well or better than the newer residents, so 
we spoke of  a U-shaped learning curve that 
dipped at intermediate levels of  expertise. 
We concocted a reasonable explanation for 
this phenomenon, namely, that residents 
gain, with experience, the ability to consider 
more possible causes for the body 
arrangements seen in an x-ray image. 
However, when they first gain this ability to 
consider more alternatives, they have not yet 
become able to manage the memory load 
imposed by this deeper reasoning. To make 
a reasoned decision, they had to keep a 



It Takes a Village of Mentors                                    15 

 
difficult-to-perceive image in view and 
simultaneously consider multiple body 
arrangements that could produce that image 
(remember that x-ray images are projections 
into two dimensions from three and that 
they show only the total density of  a slice 
through the body, not all the edges of  
individual body components). That is, they 
had learned to think deeper, but they did 
not yet have enough of  the deeper thinking 
capability automated to make it work 
reliably. 

We made a lot of  this U-shaped 
learning curve in at least one paper 
(Lesgold, 1984). Shortly thereafter, Dave 
Klahr sent me a note pointing out that U-
shaped developmental curves were not all 
that uncommon in cognitive development 
and that perhaps I should become 
acquainted with the relevant developmental 
literature. He was right, of  course, and that 
allowed this finding from a niche corner of  
psychology to be connected to a much 
wider range of  better-understood 
phenomena in children. The lesson is 
simple. It really helps to be widely 
acquainted with the literature in a field. 
Getting overly focused can be helpful at 
times, since research is hard and presses 
against mental load limits. However, breadth 
of  knowledge also is helpful in assuring that 
we end up with theories that are as simple 
and broad as possible rather than piles of  
unconnected fragmentary hypotheses. 
 
The Intelligent Systems Program 

All through the work on expertise, 
I kept in the back of  my mind that I wanted 
to do more cognitive modeling and to 
leverage the tools of  artificial intelligence 
more broadly in my research. One problem, 
though, was that the students attracted to 
graduate study in psychology seldom had 
the range of  knowledge and experience to 
be able to acquire quickly the basics of  
artificial intelligence at the same time they 
were learning cognitive psychology. 
Discussions around the Pitt campus 
revealed that several colleagues in other 
disciplines who were interested in artificial 
intelligence faced similar difficulties finding 

students with adequate 
computational/formal/mathematical skills. 
Moreover, there were students already on 
campus in areas like computer science who 
really wanted to focus their doctoral work 
on a research problem related to artificial 
intelligence.  

Initially, Harry Pople, a faculty 
member in the School of  Business, worked 
with me to develop a preliminary plan for a 
doctoral program in artificial intelligence. 
Harry eventually left the University for a 
government-related job. So, Rich 
Thomason, a philosopher and 
computational linguist, and I developed the 
final proposal for Pitt’s Intelligent Systems 
Program. There were students and faculty in 
the health sciences as well as in arts and 
sciences who were interested, so we 
designed the program to leverage faculty 
with appointments in existing departments 
who would do some of  their teaching and 
doctoral advising in the intelligent systems 
program. This low-overhead approach has 
served quite well. The program was 
proposed during a period of  leadership 
transition at Pitt, and that made getting 
started a little complex. However, with the 
support and advice of  Tom Detre, the head 
of  Pitt’s health sciences, we were able to 
launch the program in 1986.  

In many ways, Allen Newell at 
Carnegie Mellon University was the 
godfather for the program. We didn’t always 
agree, but in the end, Allen convinced me 
of  many things that I first saw differently. 
For example, my initial model was to hire a 
flock of  assistant professors and see which 
ones we wanted to keep after a few years. 
Allen convinced me (during a meeting in his 
office that started at about 10pm one 
Sunday night and continued into Monday 
morning) that we needed a strong senior 
person more central to the computational 
side of  artificial intelligence than me or 
Rich. So, Bruce Buchanan, who had been at 
Stanford in the SUMEX-AIM lab, was soon 
recruited as our first externally hired faculty 
member, and the program was launched. It 
remains to this day and continues to be 
anchored partly in computer science and 



Acquired Wisdom/Education Review                                                                                                             16 

 
partly in biomedical informatics, with 
additional participants in psychology, 
information science, education, and other 
units.7 And, we now attract plenty of  
students with rigorous preparation in both 
artificial intelligence and another discipline. 
The lesson learned in this effort was that 
building a strong team makes a huge 
difference and that attending to the needs 
of  the broader unit in which an enterprise 
will function really pays off. 
 
Developing Intelligent Tutors – Sherlock  

The work on radiological expertise, 
combined with thinking about cognitive task 
analysis that was triggered by my early 
artificial intelligence efforts, led to the next 
important accident in my career. Our 
radiological expertise work happened to be 
noticed by Dr. Sherrie Gott and her 
colleagues at the Air Force Human 
Resources Laboratory (AFHRL). The folks 
at AFHRL began to wonder whether 
training for Air Force technical jobs might 
be improved if  those jobs were studied 
using the same methods we had used for 
radiology. At this time, the field of  cognitive 
task analysis was developing quickly, and in 
essence the Air Force wanted LRDC to do 
task analyses of  some technical jobs. Bob 
Glaser and I were pretty overbooked at the 
time, but we had a colleague who had a lot 
of  talent but who was short on research 
funding, and we asked him if  he would like 
to take on the project. He agreed and the 
project proceeded quite well. Bob and I 
contributed to some of  it, too. However, 
when it came time to brief  the general who 
was funding the work, it turned out that our 
colleague wasn’t very good at preparing for 
or giving short, focused briefings.  

I felt responsible for the project, 
having convinced the Air Force to fund it, 
so I stepped in, did the needed further 
analyses to support the tight briefing, gave 
it, and got us back into the good graces of  
AFHRL. Soon thereafter, Dr. Gott asked if  

                                                           
7 As I write this, it appears that the intelligent systems program, the department of  computer science, and 
the School of  Information Science at Pitt soon will be more integrated. 
8 Avionics is a term that refers to the electronics needed for aircraft navigation and operations. 

LRDC could do some further task analysis 
work and if  Bob and I could lead it. By 
then, I was thinking hard about the 
possibility of  intelligent tutoring systems, so 
I said I would join Bob in the work if  I 
could also build an intelligent tutor for one 
of  the job specialties we were to study. The 
Air Force agreed, so the seeds planted years 
earlier in the cognitive modeling efforts that 
used Joshua Lederberg’s computer were 
about to germinate. There is a lesson here as 
well. On the one hand, we did what the Air 
Force wanted us to do. On the other hand, I 
also was able to persevere in what I most 
wanted to do – to build an intelligent 
tutoring system. It is worth working hard to 
deliver on promises to funders and also to 
stick with the problems that provide the 
challenges that keep us doing research. 

Initially, we analyzed two job roles, 
jet engine mechanics and avionics8 
technicians. We learned some important 
lessons about mastery learning from these 
efforts. Using the standard instructional 
systems design procedures of  the time 
(Branson et al., 1975; Gagné & Briggs, 1974; 
van Merriënboer, 1997), training in the Air 
Force was developed by decomposing the 
target knowledge into small, coherent, 
teachable units, teaching each unit 
separately, and not moving to the next until 
a mastery test had been passed on the unit 
being taught. 

 We found, though, that these bite-
sized units could be learned fast but also 
could be forgotten fast. For example, even 
though all the technicians we interviewed 
for our task analysis work had “mastered” 
what a capacitor is, many, when asked later 
to describe the function of  one in a circuit, 
said that they were fuses and could no 
longer recall what they had learned in the 
mastery curriculum. In daily work, 
capacitors only came to their attention when 
one burned out, so thinking they are fuses 
was a reasonable experience-based 
hypothesis, but it showed complete loss of  



It Takes a Village of Mentors                                    17 

 
the earlier definition they had “mastered.” 
Today, military training is assessed more 
often in terms of  valued performances, 
though there remain vestiges in the training 
world of  content briefly memorized, 
regurgitated, and then forgotten since it has 
never been used in real work contexts.  

There is a lesson here for those 
who develop online educational systems. 
Quite often, those systems also divide the 
content to be taught into tiny pieces, teach 
each piece, test it immediately, and then 
decide that it has been mastered if  an 
immediate and minimal-context test has 
been passed. Being able to define a concept 
or even to apply it in a very limited context 
does not tell us much about whether it is 
understood deeply enough to come to mind   
when relevant to new tasks that might 
present themselves. 

While doing the task analysis work, 
I realized that in the information age, 
machines can do almost anything that we 
can teach using the old instructional systems 
design approach or its school-based variants. 
Humans, increasingly, are valued for their 
ability to apply old knowledge to emergent 
situations and to bridge between different 
bodies of  knowledge resting on different 
ontologies. In schooling terms, life is an 
unfair test. Knowing something in the 
artificial context used when it is taught in 
school does not imply knowing it well 
enough to be able to use that knowledge in 
new situations. If  we consider it unfair to 
test a student’s knowledge using novel 
situations, we preclude knowing whether the 
knowledge that student acquired is usable or 
is “inert” (Whitehead, 1929). Realizing this 
led me to develop an intelligent tutor to 
prepare people for situations that were not 
explicitly presented as part of  the training. 
In the terms of  traditional experimental 
psychology, I wanted to do training for far 
transfer. 

As luck would have it, while most 
of  flight-line engine maintenance involved 
standard routines, with an emphasis on 

                                                           
9 In some of  our work, we did even further far transfer studies, looking at the ability to repair failures of  a 
different system than the one on which the training occurred. 

explicit protocols for testing and 
replacement of  parts that would not 
function as specified, the avionics 
technicians struggled when encountering 
failures of  the tools that allowed their day-
to-day work to be routinized. These failures 
generally were novel or at least relatively 
novel. In years on the job, a technician was 
unlikely to see any given failure, though of  
course a few failures would occur and had 
to be handled. Still, given the low failure 
frequency, the limited training time, and the 
need to be prepared for whichever failures 
did occur, there was a clear need for training 
that produced “far transfer”9 in the sense 
that the specific circumstances in which the 
effects of  training would be realized were 
not likely to be the situations on which 
training was provided. 

So, we embarked on the 
development of  a system to train 
technicians not in their daily work but in 
handling of  rare events for which they were 
not prepared explicitly. The domain of  our 
work was the F-15 avionics test station. This 
was a tool used by technicians for the 
routine work of  diagnosing and repairing 
electronic modules from the cockpit of  F-
15 fighter planes. When a pilot reported a 
problem from a flight just completed, the 
technician would remove the module 
implicated by the problem report and carry 
out a series of  routine tests. To do those 
tests, the suspect module was connected to 
the test station, which then was 
programmed via switch settings to carry out 
prescribed tests documented in a manual. 
That was the routine work, and there were 
tight procedures for that work to be 
regimented by the test stations and their 
programming. But, the test station itself  
could fail. There was no routine and no test 
station testers if  that happened. Test station 
failures had to be diagnosed without 
routinized processes. They required explicit 
problem solving that had not specifically 
been practiced. 



Acquired Wisdom/Education Review                                                                                                             18 

 
One can get a sense of  the amount 

of  experience arising in the field with test 
station failures from an interesting bit of  
data we accumulated in our work. We had 
reported that about 25 hours of  training 
with our intelligent tutor produced as much 
learning as four years on the job, so I 
wondered how much experience with test 
station failures the average technician gets in 
four years. It turned out that the best 
estimate we could get was that technicians 
get perhaps 10-12 hours of  experience 
diagnosing and repairing test stations in four 
years. So, we were preparing technicians for 
crucial roles that are exercised rarely but are 
crucial when needed, just as pilots practice 
in the simulator for events that probably will 
not occur during their working careers but 
that have high costs attached if  they are not 
handled well. 

We started our work with an 
idealized view of  how training would occur. 
We thought we would have an expert model 
that could solve every problem, compare 
trainee performance to that model, and use 
the comparisons to develop appropriate 
hints if  the technician was stuck. At the 
time, though, and to some extent today, 
systems simply weren’t powerful enough to 
develop perfect diagnoses and perfectly 
tailored hints, nor was our pedagogical 
content knowledge complete enough for us 
to know exactly what knowledge gap was 
the immediate barrier when a trainee got 
stuck on a problem. This led us to an 
important conclusion, which is that when a 
trainee was sitting in front of  our computer 
system, the smartest intelligence in the room 
was the trainee, even though we were trying 
to get them to learn something new. 

So, our tutor operated by making 
relevant information available to technicians 
and then letting them select from several 
levels of  hints. At one extreme, a trainee 
could simply ask to have their prior activity 
on the problem summarized and 
recapitulated. At the other extreme, they 
could be told exactly what to try next. In 
between, technicians could get pointed to 
the right general area of  the system that 
they should be thinking about or get some 

help with approaching diagnosis in that area. 
In this training situation, where technicians 
are highly motivated to become experts to 
keep planes in the air and win wars, 
technicians did a good job of  not asking for 
more help than they needed. Conceivably, in 
other applications, incentives would be 
needed to encourage students to invest 
cognitive effort and not just rely on the 
machine for guidance. 

We learned an important lesson 
from our experience with hints. Over half  
of  the time, when trainees asked for hints, 
they were able to proceed further after being 
given only a systematic listing of  what steps 
they already had taken – with no new 
information. The work on mental load was 
just beginning at this point, but our 
experience certainly is consistent with a view 
that cognitive resources are limited and that 
sometimes lack of  practice in doing 
complex thinking can limit one’s ability to 
keep in mind all that is needed to do the 
required reasoning – even if  one knows 
each element required to do that reasoning. 
Scaffolding often requires only a little bit of  
help. Had we developed perfect systems to 
search for trainee knowledge gaps, those 
systems probably would have provided too 
much scaffolding and have led to less 
transfer potential. 

While many other things were 
learned from our intelligent tutoring systems 
work, they are documented elsewhere (Gott 
& Lesgold, 2000; Lesgold, 2012; Lesgold & 
Nahemow, 2001), so I present only a few 
details that may give a sense of  how my 
career has been shaped by interesting 
opportunities and responses to those. One 
opportunity arose when we were designing 
displays to be available from the intelligent 
tutor. We wanted to have all relevant 
information available to trainees while they 
solved problems, and that meant storing 
gigantic schematic circuit diagrams in the 
system and providing an interface that could 
make any part of  the diagrams available on 
demand. But, the whole diagram could not 
be displayed at once (because of  limited 
screen space), so the question arose of  what 
part to display when the technician wanted 



It Takes a Village of Mentors                                    19 

 
to consult the circuit diagrams. Initially, we 
fussed with all kinds of  indexing systems, 
but eventually we came up with a basic 
design principle that I suspect has much 
wider applicability. 

We decided that we would display 
only the part of  the system schematics that 
an expert would probably want to see during 
the course of  work on a diagnosis problem 
up to that moment. That wasn’t all that 
difficult to determine. The expert model 
would have rules that would be triggered by 
the current problem state. Those rules 
referred to system elements. The system 
elements as well as the “goal stack” of  
higher level elements currently in focus, 
would allow us to present the “right” part 
of  the display. But, that still left the question 
of  how to make the display appear. 

That turned out to be extremely 
simple. Our representations of  the test 
station were hierarchical – they consisted of  
a few main system elements, which 
contained subelements, which contained 
subsubelements, etc. We used this 
hierarchical structure to drive the generation 
of  displays in real time. For each system 
element or subelement, there was a simple 
routine that could display it. That routine 
would be told what part of  the screen it 
could use, with elements that the expert 
model was focused on getting more of  the 
display space. The element’s display routine 
then would “instruct” its subelements to 
display themselves, telling each what 
proportion of  the element’s display space it 
was allowed to occupy. Display space was 
allocated as a function of  how central a 
subelement was to the current “state of  
mind” of  the expert model. If  a subelement 
was allocated only a little space, it could not 
be further decomposed while generating the 
display. Overall, then, the display would 
show detail where the expert model was 
focused and collapse to a higher level of  
abstraction where the expert model was not 
focused. One of  the team members was 
able to produce this whole scheme with less 
than 15 lines of  code once we specified the 
circuit as a hierarchical object collection. 

But, recall the design principle 
stated above that the smartest person in the 
room is the trainee. Because of  that, we 
added a feature that clicking on any element 
of  the displayed circuit diagram would open 
it up to display its subelements and their 
interconnections. So, if  a trainee felt it 
important to check more details in the 
schematic diagrams than we had provided, 
he could just click once to see that detail. I 
have come to refer to this approach as 
intelligent shallow hypertext – the system 
displays what it thinks you want to see but 
makes it easy to see other things should it 
guess wrong. 

As mentioned above, the approach 
I have taken in building the intelligent tutors 
was not to try to diagnose exactly what 
concept the technician was missing but 
rather to provide just enough cognitive 
support so that they could keep practicing 
and getting better at stretching their 
knowledge to handle new situations 
effectively, situations that were not explicitly 
studied. I believe that a modest caricature of  
the Common Core State Standards is that 
they have added to the curriculum more 
practice in applying what is taught to new 
situations, i.e., being able to stretch and 
broadly use what is being learned. If  that is 
the case, then perhaps the approach we used 
has broader applicability. I should note that 
largely because of  this distinction between 
deeply diagnostic intelligent tutoring 
systems and the kinds of  systems colleagues 
and I developed, our second sponsor, Intel 
Corporation, insisted on referring to our 
systems as intelligent coached 
apprenticeship environments. They were 
probably right to do so. 

I should comment briefly on how 
the initial Air Force system was named. Our 
later systems usually were just called ICA for 
intelligent coached apprenticeship, but the 
ones developed with Air Force support were 
called Sherlock. We were pushed by the Air 
Force to come up with a name for our 
system, and I’m just not much into names. 
So, I left the task to my colleagues, and one 
of  them had worked with a pretty good 
artist. Between them, they came up with the 



Acquired Wisdom/Education Review                                                                                                             20 

 

Sherlock name and related images. The 
reason the pictures of  Sherlock we used 
displayed a pipe was to divert attention from 
a certain amount of  within-project joking 
that the system was really the equivalent of  
Holmes’ “7% solution,” the cocaine that he 
injected when he was really stumped by a 
case. This detail was one reflection of  a 
broader principle that I pass on to graduates 
every year when I address them at 
graduation: take your work seriously, but 
remember that life goes better if  you don’t always 
take yourself  too seriously (I then put a red ball 
on my nose prior to recessing from the 
platform). 

We developed two generations of  
intelligent tutors with Air Force 
sponsorship. By the time we were done, we 
had a technology that worked well for the 
kinds of  jobs that were major training 
problems for the Air Force. Moreover, we 
had demonstrated substantial far transfer as 
well as persistence of  the acquired 
competence over extended time. However, 
we still were working in what I felt was too 
simple a domain. Test stations of  the kind 
we focused on used old mechanical logic. 
Almost everything that happened in them 
was either directly observable (as the 
movement of  mechanical elements) or at 
least easily envisioned even if  it occurred 
inside a closed box. Moreover, the basic 

conceptual knowledge needed for any aspect 
of  the diagnostic work either was already 
taught well in classrooms before technicians 
ever came on the job or else was in the 
realm of  common sense reasoning. Even 
concepts such as the intelligent shallow 
hypertext displays might have worked only 
because the domain was so simple. So yes, 
Sherlock did provide in 25 hours the 
equivalent of  four years of  on-the-job 
training, but there are much harder cognitive 
tasks to be learned, so we had not really 
done such a good job demonstrating the 
power of  intelligent coached apprenticeship. 

The Air Force was interested in 
funding further work, but I decided that we 
could not afford to continue. The work 
increasingly was being funded by short-term 
contracts issued only when spare cash 
became available to one or another general. 
Contracting rules prevented us from 
charging any costs incurred before each 
contract was signed, but we needed to 
maintain a team of  people who understood 
the work. We simply could no longer afford 
to cover the costs incurred when one short-
term contract ended and before the next 
one started. We could handle small amounts 
of  that with the risky move of  taking fixed 
price contracts (which meant that Pitt was 
on the hook if  any piece of  the work took 
longer than anticipated), working harder and 
faster than originally planned, and investing 
any “profit” in covering gaps until the next 
little contract appeared. That was not an 
adequate strategy in the end, and it also 
bumped up against lawyers who worried 
about whether the “profits” were taxable. At 
the time I decided not to take further Air 
Force funding, the Learning Research and 
Development Center at Pitt was spending 
about $150,000 per year plus my own salary 
in sustaining the Sherlock work between 
contracts. The Air Force, at least at an 
institutional level, could not grasp this. 
 
The Intel Tutors 

Cutting the ties with the Air Force 
turned out to be a very positive move in the 
end. Shortly after we made the decision, 
Paul Chan at Intel contacted me to ask if  we 



It Takes a Village of Mentors                                    21 

 
would be interested in teaming with Intel 
experts to further develop the intelligent 
coached apprenticeship concept. Because 
their interest was in maintenance of  
equipment used to make computer chips, 
which are much more complex than the test 
stations, this was very appealing. We could 
learn two things: whether we could make 
tutor development cheap enough to be 
financially viable and whether the approach 
could work for jobs involving much more 
complex technical knowledge. Throughout 
our Air Force work, AFHRL had an 
economist studying the financial viability of  
intelligent tutors for training. That person 
confused the likely unit cost once the 
technology was developed with the cost of  
inventing the technology and implementing 
it the first time – sort of  as if  people had 
written off  light bulbs because Edison spent 
a lot of  money figuring out how to make 
the first one. 

There is a lesson to be learned here, 
too. The first version of  Sherlock cost about 
$2.5 million. This was the cost of  a multi-
year effort that invented the entire 
approach, implemented it for a specific job 
area, and evaluated whether or not the 
approach worked. We developed two 
versions of  tutors with the Air Force and 
then three more versions with Intel. The 
cost of  the final version was about $70,000. 
Each generation cost less than half  that of  
its predecessor. It is extremely unusual for 
any funding pattern to be sustained long 
enough to gain these levels of  cost 
reduction. Generally, we rely on the market 
to turn research into a venture capital target. 
It is not surprising then that venture 
capitalists focus only on developing 
products that are either mundane 
applications of  well-known understanding 
(e.g., almost all education start-up 

                                                           
10 Of  course, it could be argued that if  I had chosen to move into business, I might have attracted enough 
venture funding to do the refinement that was accomplished through the five generations of  research 
funding. Potentially, a funder could have been found, though going that route would have produced less 
understanding of  what intelligent learning tools can accomplish, since market-related research is not quite 
the same as inquiry unbiased by financial motive. Also, current venture money for software tends to focus 
on applications where consumers are the market, not institutions, since institutional markets can always 
find a way to copy a product at lower cost. So, the market path really is not viable for intelligent tutors 
that will likely be sold to institutions (see Friend, 2015). 

companies) or exploitation of  an idea that 
has received strong verification and does not 
involve any novel production methodology 
(e.g., a new drug that can be sold at a high 
price to a large market). On occasion, it may 
be worthwhile for federal and other sources 
to fund multiple generations of  refinement 
before hoping that the venture capital world 
will take over.10 

All three generations of  tutor 
development with Intel proceeded with the 
same basic organizational structure. We were 
set up like an Intel working group, and a 
representative of  the plant management 
community was the project manager – that 
is, the project was led administratively by a 
customer for the system. Marty Nahemow 
and I were the intellectual leaders, and the 
project manager helped assure that we 
listened to both the experiences and the 
ideas of  the potential users of  the training 
systems being built. We are seeing many 
calls today for similar organizations in the 
preparation of  teachers and even in research 
on how better to prepare teachers. This 
makes sense, but it takes effort to develop 
modes of  operation that are comfortable 
for all participants and on the academic side 
we happened to have people who were 
amenable to the ego-free approach valued in 
the business world. 

I also should say a little bit about 
my colleague in the Intel work, Marty 
Nahemow. Marty had been a manufacturing 
physicist at Westinghouse. As Westinghouse 
came on hard times, he decided to take a 



Acquired Wisdom/Education Review                                                                                                             22 

 
retirement deal. Shortly thereafter, he joined 
us at LRDC and we embarked on a state-
funded project to develop a high school 
curriculum that could be linked to 
apprenticeship in the machine tool industry. 
That work was a lot of  fun. Unfortunately, 
most superintendents were deathly afraid of  
any programming that didn’t aim at four-
year college, so only one district, 
Williamsport, adopted the approach we 
built. The project was, however, 
documented in Smithsonian Magazine in 
1993 (Kiester, 1993).  

Working with Marty was extremely 
rewarding. He is exceptionally smart. In fact, 
while at Westinghouse, he invented the 
screw-in fluorescent light bulb. 
Westinghouse sold the patent to Philips for 
very little, but all of  us have benefited from 
the invention in recent years. Marty knew a 
lot of  physics, and he quickly picked up a lot 
of  cognitive science. His presence at LRDC 
allowed me to take some chances that might 
otherwise have been a bit riskier. 

This was true from the beginning 
of  the relationship with Intel. One day, 
while I was still mourning the need to stop 
taking Air Force funding, I got a call from 
Paul Chan at Intel. He told me that Intel had 
determined that the first intelligent coached 
apprenticeship system they wanted to build 
with us would be one that taught technicians 
how to repair machines used to do ion 
deposition – putting layers onto computer 
chips. I agreed that we would like to 
collaborate in this effort as a way to refine 
and test our ideas about intelligent coached 
apprenticeship. But, I did not have a clue 
about what ion deposition was or what 
knowledge might be required to diagnose 
and repair ion deposition devices. 

So, I walked over to Marty’s office 
and told him that I had just agreed that 
LRDC would develop an intelligent coached 
apprenticeship system to teach diagnosis 
and repair of  an ion deposition system and 
that I would love his help with this. Marty 
pulled his beard once or twice and then said 
that while his patents on ion deposition 
were for a different application, he thought 
he understood the process and would be 

able to provide expert advice as we built the 
intelligent training system. I breathed a sigh 
of  relief  and set out to learn from Marty. All 
in all, we did a pretty good job of  building 
that first system, and it produced good 
results. We also cut the cost of  development 
in half  yet again, and of  course we had 
moved from a domain with limited 
conceptual content (relay-based switching 
systems like the Air Force test stations) to 
one that involved a lot of  physical 
chemistry, silicon chemistry, and physics. 
Note that the technicians we trained had 
perhaps two years of  community college 
coursework past high school, so the extent 
of  science content required to learn from 
our tutors was an important issue. 

Given the success of  the first 
project with Intel, I eventually got another 
call from Paul Chan. This time, he was 
interested in a training system for repairing 
ion beam implant devices, the devices that 
“write” circuits on layers in computer chips. 
Once again, I agreed to the project even 
though I had no clue about what ion beam 
implant processes were. Once again, Marty 
pulled his beard and indicated that his 
patents on ion beam implant were for a 
different application but that he thought he 
could handle the project. Once again, we 
built an extremely effective intelligent 
coached apprenticeship system (Lesgold & 
Nahemow, 2001). Needless to say, I could 
not have done the work I did on intelligent 
coached learning environments without 
Marty. 

The reader might assume that 
Marty only helped me with the physics. In 
fact, he absorbed a lot of  cognitive science 
understanding and came up with an 
important element of  our later tutoring 
systems, which we called the Process 
Explorer (Lesgold & Nahemow, 2001). 
Recall that I mentioned above that the test 
stations were pretty simple. It was not 
necessary to learn a lot of  new science to  



It Takes a Village of Mentors                                    23 

 
understand how they work. In contrast, chip 
making equipment leverages a lot of  
science, including silicon chemistry (a 
domain almost as complex as organic  
chemistry, except that it has to do with 
silicon rather than carbon compounds), 
quantum physics (involved in explaining 
where ions end up when they are “shot” at a 
chip), and other content not likely within the 
grasp of  someone two years out of  high 
school. 
 The Process Explorer was, in 
essence, a just-in-time delivery engine for 
little bits of  conceptual knowledge that 
might help in understanding particular 
occurrences in the failure of  chip making 
equipment. Many things can go wrong in 
such complex equipment, and the system 
failures manifest themselves in many 
different ways. Marty figured out how to 
handle just-in-time delivery of  appropriate 
conceptual coaching. Consider a very large 
matrix, perhaps with hundreds of  thousands 
of  cells. The columns of  this matrix are  
possible symptoms that might arise when 
diagnosing the failure of  a chip-making 
machine. The rows are possible causes for 
those symptoms. Each cell of  this gigantic 
matrix refers to how a given cause can 
produce a given symptom. What we put into 
the matrix cell was an explanation of  that 
causal relationship. So, when a symptom was 
noted by a technician, he could get a brief  
explanation of  each possible cause of  that 
symptom. This was feasible because almost 
all of  the cells of  this gigantic matrix are in 
fact empty. To make relevant just-in-time 
explanations available to the trainee was a 
simple matter. The symptoms described in 
the data that framed the problem statement 
defined a small number of  matrix columns, 
and the remaining columns need not be 
displayed. The relevant columns had entries 
in only a small number of  rows, the rest 
were empty for the columns still active, so 
they also did not need to be displayed. For 
any given problem, therefore, there was a 
small matrix of  symptoms and causes that 
could be extracted from the theoretically 
huge matrix. 

Figure 1 shows an example of  the 
process explorer in the middle of  a problem 
solution by a trainee. What are shown in the 
columns of  the matrix are the three 
measures for which the problem included 
indications of  a deviation from normal. The 
rows indicate variables in the chip making 
process that could be responsible for 
deviations in those measures. The individual 
cells show whether the deviation that might 
have occurred would result in a small or 
large increase or decrease in the measure in 
question. Clicking on a cell gives an 
explanation of  how such a change could be 
caused. In the example shown in the figure, 
the trainee has clicked on the cell showing 
that a substantial increase in stress on the 
chip wafer could be produced by an increase 
in RF power to the chip making machine. 
That click produced the explanation shown 
in the bottom right of  the figure. Each 
explanation we offered consisted of  a 
reduction of  a complex phenomenon to a 
more or less linear relationship that could be 
explained readily. 

The Process Explorer is another 
example of  a recurrent experience in my 
career. What looked theoretically like a 
terribly complex and intractable task turned 
out to be quite manageable empirically. Yes, 
the machines in question depend upon all 
kinds of  complicated science. However, the 
bits of  that science that are relevant to the 
role being trained are relatively few and 
readily indexed. You could argue that it 
would be better if  the trainees had complete 
courses in all of  the relevant physics and 
chemistry. However, that is not practical. 
Doctoral candidates in those domains would 
not be happy spending their work days in 
bunny suits fixing the same small number of  
machines, nor could companies afford to 
pay salaries commensurate with years of  
graduate study. If  we can deliver self-
contained and understandable chunks of  
domain knowledge that stretch a person’s  
competence, that is a worthwhile thing to 
do, since it broadens their social value and 
earning capacity.  

It is interesting that during the first 
years of  my intelligent tutoring systems  



Acquired Wisdom/Education Review                                                                                                             24 

 

 
 
work, a colleague in Computer Science 
worked hard to prove that building such 
systems was computationally impossible. He 
would have been correct if  it had turned out 
that complete knowledge of  each relevant 
domain needed to be part of  the tutor’s 
intelligence and that it needed to fully 
understand all of  the thought processes a 
technician might be engaged in while 
solving one of  the problems it posed. 
However, what was impossible in principle 
turned out to be quite possible in fact.  

This lesson has had to be relearned 
many times since people argued that the 
Enigma code was not crackable by a 
computer (Stripp, 1993). In principle, 
reverse engineering the Enigma design was 
beyond the computational capability of  
computer devices of  that time, which gave 
the German military confidence that it 
could not be cracked. However, Enigma was 
not used perfectly, and the actual ways in 
which it was used only partly tapped its 
potential. Cracking a perfect machine used 
imperfectly was a lot easier than the  

Figure 1. Example of  Process Explorer. 
 

theoretical task of  cracking the machine if  it 
were reprogrammed as regularly as it could  
have been and if  no mistakes had been 
made conveying the new Enigma 
configuration to all stations when that 
reprogramming occurred.  

With our intelligent training system 
work, it was the limited relevance of  large 
parts of  the underlying science that made 
the task of  just-in-time explanation 
tractable. And, of  course, it took Marty 
Nahemow’s brilliance both to discover that 
the process explorer was feasible and to 
manufacture simple and understandable 
explanations of  many complicated 
phenomena. 

The most satisfying part of  the 
intelligent training system development 
work for me was the fifth generation. In that 
generation, Intel staff  did almost all the 
system development, and again the cost was 
cut in half  from Generation 4. We had 
shown that not only did our approach work, 
but it also could be realized outside the 
research laboratory. In fact, with an average 



It Takes a Village of Mentors                                    25 

 
savings of  $80,000 (due to shorter 
downtime) when one of  the people we 
trained used their new knowledge for the 
first time, the entire cost of  Generation 5 
($70,000) was recovered because of  the 
reduced time that a manufacturing line was 
down and then restored by that trainee. 

We had one other thing on our side 
as we proceeded, Moore’s Law. It is quite 
possible that even five generations would 
not have been sufficient to produce a viable 
approach if  computers were not getting 
faster during this period. From the 
beginnings of  Sherlock to the last work with 
Intel, fifteen years elapsed. Given Moore’s 
Law, we can estimate that the amount of  
computer power available for a given price 
doubled ten times over that period, giving 
us, for the same level of  investment, over 
1000 times the capability at the end of  that 
period compared to the beginning. Had it 
not been for the industrial realities captured 
by Moore’s Law, that computer scientist 
who was proving that my goal was 
impossible to attain might have turned out 
to be right, at least temporarily. The lesson 
to be learned is that sometimes it is sensible 
to aim beyond what is possible today, 
provided that there is reason to believe that 
needed elements for solving a problem will 
become available faster than the thinking 
needed to deploy those elements. So, either I 
was visionary or I was slow enough that the 
world moved faster and became able to 
adapt to the limits of  my thinking. Either 
account matches the data. 

Quite a number of  students, 
programmers, and post-docs worked on the 
intelligent coached apprenticeship 
environments we developed, and I learned 
from all of  them, including Sandra Katz and 
Susanne Lajoie, both of  whom have 
developed strong research careers. There is 
another lesson learned about which I have 
written before (Lesgold, 2012). The fact that 
we had developed and refined a practical 
and powerful new learning approach did not 
mean that it would instantly be deployed. In 
fact, no one has fully used the technology 
we developed beyond the specific efforts in 
our five generations of  work, although 

partial uses are becoming more common of  
late. While the work has influenced some of  
the recent increase in use of  intelligent 
tutoring systems methodology, its influence 
has been slow and applied only partially. To 
understand why this could happen, even 
when the cost of  deployment can be 
recovered when one person uses the 
knowledge the system teaches, it is useful to 
look at the medical world for a moment.  

Medicine has a long tradition of  
basic and applied research, and the national 
investment in both has been huge. Still, it is 
not unusual that well-established applied 
results do not automatically lead to changes 
in practice. Procedures for avoiding 
hospital-induced infections are a simple 
example. We know that certain infections 
often are transmitted in hospital settings. We 
also know that the chances of  transmittal 
are decreased markedly when hospital staff  
wash hands before and after seeing each 
patient. Nonetheless, those findings, by 
themselves, did not sufficiently influence 
hospital practice for quite a while. Even 
today, infection reduction is a work in 
progress, but it is now clear that a variety of  
specific institutional interventions are 
needed for it to occur, including placing 
needed equipment and supplies right at 
hand in front of  each patient room, 
counseling patients to remind staff  to wash 
and glove, and providing financial incentives 
to hospitals for reducing infection rates. 

The research needed to get human 
systems ready to use accumulated applied 
research is called translational research. In 
medicine, major centers have been 
established to conduct such research, and it 
is starting to pay off. In education, such 
efforts have been started, but really good 
translational research is still rare. For 
example, the Institute of  Education 
Sciences has a category of  research support 
which they have called “Goal 4.” It is for 
efforts to take an applied result that is well 
grounded in past research and to show how 
to get school systems to improve their 
outcomes using that research. Very few 
submissions have come in for that funding 
category, and extremely few proposals have 



Acquired Wisdom/Education Review                                                                                                             26 

 
reached the quality level needed for funding. 
Certainly, in my work on intelligent tutoring 
systems, it became clear that real payoff  
would require substantial translational 
efforts. Possibly, as machine intelligence is 
better absorbed into our culture, the gap 
between the work we did and wider 
adoption will be closed. 
 
Tools for School Learning 
 After the work on intelligent 
tutoring systems, I did a bit of  work on 
tools to support argumentation. I was 
influenced in this work by a number of  
factors. First, it was clear from research of  
others (cf. Sieg, 2007) that intelligently 
coached graphical environments might be 
very useful in helping students learn to 
develop arguments more completely and to 
collaborate in doing so. This led to a project 
focused both on developing graphical tools 
to represent arguments and on building 
coaching systems that could “observe” 
students collaborating to build such a 
graphical representation and coach their 
collaborative efforts (Suthers et al., 2001; 
Tothet al., 2002). The basic goal was to build 
a system that allowed students to represent  

an argument graphically. For example, when 
we gave a couple students a text about the  
possible causes of  the demise of  dinosaurs, 
a piece of  the argument diagram they 
developed is shown in Figure 2. The figure 
shows only the very beginning of  a 
discussion by the students, with the 
arguments being developed represented  
graphically in the diagram. The diagram 
allowed them to indicate hypotheses, data, 
and relationships of  knowledge elements to 
one another. We called the system 
Belvedere, since it graphically displayed an 
overview (the translation from Italian of  
belvedere is overview) of  a developing 
argument. 

The overall system was rather 
interesting. It included a representation of  
the specific pieces of  knowledge added by 
students, the overall accumulated diagram, 
and an intelligent coach that tracked the 
ongoing development of  the argument 
diagram. The coach “observed” the pattern 
of  student contributions and provided 
advice based upon that pattern. 

 
 

Figure 2. From Suthers et al., 2011. 
 



It Takes a Village of Mentors                                    27 

 

  
shows an example of  the advice that could 
be offered at the point where the diagram 
had progressed to the state shown in Figure 
3. 

The system had some interesting 
computational characteristics. For example, 
the coach ran on a different computer than 
the argument display environment, 
observing the stream of  student activity and 
generating suggestions based upon how the 
activity progressed. This was done for 
multiple reasons. It had the effect of  
allowing the coaching system to run on a 
more powerful computer while the 
diagramming system could run in multiple 
low-cost student computers. This reflected 
our belief  that we could not predict the 
kinds of  computers that might be used in 
schools for different purposes, except that 
networking would get easier and computer 
power would get cheaper. It also meant that 
arguments could be developed by groups of  
students distributed geographically while 
having a coach in a central machine. In a 
sense, we anticipated architectures like 
Google Voice that are partly in a small 
computer (e.g., a phone) and partly in the 
“cloud.” In moments of  extended 
optimism, I imagined students in different 
countries developing arguments about 
international affairs with neutral coaching 
from the cloud based upon the structure of   

 

Figure 3. From Suthers et al., 2011. 
 
those arguments. This was optimistic for 
multiple reasons, including the likelihood 
that an argument’s adequacy may depend 
upon meanings of  terms that might not be 
shared cross-culturally. Still, systems like this 
could be quite powerful. 

Certain members of  the team 
involved with the Belvedere effort have 
continued to do related work independently, 
including Dan Suthers, Patrick Jermann, and 
Amy Soller. It was a real blessing to work 
with them. I suspect that the approach that 
we took could generalize to good effect. 
Schooling historically has involved students 
working independently on structured 
assignments. But, schooling in the future 
will include more student project activity 
where less of  the specific knowledge being 
engaged by a student will be predictable, 
students may work in groups, and aspects of  
the structure and perhaps the content of  
their work will be represented using various 
tools. Perhaps it will be useful to work 
toward intelligent systems that can keep an 
eye on student work and occasionally 
comment in productive ways even if  the 
details of  what students might be thinking 
or might be trying to do are not always 
evident.  

 
 
 



Acquired Wisdom/Education Review                                                                                                             28 

 
Preparing Educators 

While my career has taken a 
number of  turns over the years (several such 
turns are omitted from this account, though 
they contributed ideas that later influenced 
my work), there has been a continuing 
theme throughout. I wanted to learn how to 
make more learning happen when it matters. 
Sometimes, that meant figuring out what it 
meant to be an expert, and sometimes it 
meant figuring out how to support the 
practice that leads to expertise. Believing in 
the importance of  research to improve 
education, I was continually forced to realize 
that finding out important things about 
learning didn’t necessarily mean that better 
teaching would occur, or that students 
would learn more. My early immersion in 
medical expertise prompted many 
reflections over the years on how the 
teaching of  medicine has progressed 
compared to the teaching of  teachers and 
their students in pre-college education. 
Eventually, I came to believe that if  I kept 
on doing the kind of  work I had done, it 
might not have any lasting impact on 
schooling or training, even though what I 
was learning was highly relevant to 
educational improvement. 

Concretely, this was manifest in my 
university in a disjunction between LRDC, 
where I had pursued my career, and the 
School of  Education, which I felt was not 
teaching some of  the most important 
concepts that came out of  work by my 
LRDC colleagues. This was not an issue 
unique to Pitt.11 There were many good to 
great schools of  education that were not 
translating certain major findings of  basic 
and applied researchers into tested 
approaches to school-based practice. 
Eventually, I came to feel that it would be 
better to work on this problem than simply 
to complain about it. I flirted with one or 
two other possibilities but could not bring 
myself  to leave the University of  Pittsburgh, 
which had supported my efforts even when 

                                                           
11 The Deans for Impact group recently produced an account of  the principles of  learning that it believes 
all prospective teachers should master. It can be downloaded from 
http://deansforimpact.org/pdfs/The_Science_of_Learning.pdf   

they stretched the boundaries and goals of  
individual units to which I was appointed 
(LRDC, Psychology, and Intelligent 
Systems). When the deanship opened in 
Pitt’s School of  Education, I decided to be a 
candidate for it. 

Many people wondered if  I knew 
what I was doing, including the Chancellor 
and the Provost. They came to realize that I 
was fully aware of  the costs of  being a dean 
(I really wasn’t, but I was as close as anyone 
is who has never had the job), and I was 
fortunate to be appointed dean in July, 2000. 
It has been a wonderful experience.  It has 
been a privilege to help such a fine group of  
colleagues achieve great things. 

In the course of  my deanship, I 
realized that simply bringing the kind of  

Two years after I became 
dean, I also became, for 30 months, the 
president of a large Reform Jewish 
congregation. I learned a lot from that 
experience, including that a room can be 
too hot and too cold, services can be too 
short and too long, and liturgy can be too 
traditional and too contemporary, all at 
once. I served during a rough moment for 
the congregation, and it taught me that one 
can come through rough times with the 
help of friends and that one’s opponents 
often are well meaning but that 
organizations still need to choose among 
conflicting paths that all have justification.  

I remain indebted to those who 
kept me going on some really dark days, 
and I deeply respect those who disagreed 
with the directions I took. Much as I 
learned, though, I do not recommend 
holding two jobs requiring pastoral work at 
the same time. Both roles do have in 
common the reality that if you do them 
long enough, you have to deal with some 
extremely self-destructive behaviors. 



It Takes a Village of Mentors                                    29 

 
cognitive science that LRDC excelled in to 
the School of  Education was not enough. 
Pitt has always been a stronghold of  the 
best in subject-specific pedagogy. While 
teacher preparation in many schools of  
education then, and in quite a few even now, 
included generic teaching methods courses 
and weak subject-area majors, we at Pitt 
always had subject-specific teacher 
preparation and also required a full 
undergraduate major as a pre-requisite. To 
become a math teacher at Pitt, for example, 
one not only has to have a bachelor’s degree 
in mathematics, one also takes methods 
courses specific to the teaching of  the 
various hard topics in math, such as rational 
numbers. Those courses are taught by 
education faculty, many of  whom can and 
sometimes do also teach in disciplinary 
departments. I still believe that this is 
extremely important. Different subjects 
have what Lee Schulman (2005) has called 
different signature pedagogies. One does 
not teach writing the way one teaches 
mathematics (though there are common 
elements in coaching learning-by-doing in 
the two subjects). 

Being dean of  a school of  
education, though, forced me to spend more 
time looking at schools and schooling. I 
came to feel that the subject-specific 
pedagogy work at which Pitt excelled was 
improving the education of  privileged 
children substantially but was not working 
for many children whose families were less 
wealthy, including many children of  color. 
To benefit from all we know about 
pedagogy, students need to be immersed in 
the learning opportunities that teachers 
engineer. That means they need to come to 
school, engage the learning situations 
available there, and persist in tasks that are 
presented to them. 

This sounds simple, but it is not. 
When I went to school, the community 
from which the students and the teachers 
came was quite homogeneous. Students 
were all from a white community, were all 
from families of  roughly the same wealth 
(middle and working class), and were taught 
by teachers from similar backgrounds. 

Everything outside of  school supported 
everything that went on inside. This 
included a pervasive social belief  both in the 
American dream that one can advance one’s 
circumstances by working hard and in the 
major role of  education in supporting that 
advance. Those around us who had been 
successful in life routinely conveyed the 
message that working hard in school was 
important to their success. And, of  especial 
importance, there were plenty of  good jobs 
for people who had worked hard in school, 
learned to show up on time, and learned to 
follow directions, even if  their overall 
performance on tests was less than stellar. 

Today, this is not the case. The 
schools around me in Pittsburgh, like 
schools in most cities and rural areas, serve a 
wide range of  students. Some come from 
impoverished families while others may 
come from better circumstances. Those well 
off  generally go to different schools than 
those who are poor, though all are taught by 
teachers from the middle class. Students of  
color often are taught by white teachers 
whose lives are dramatically different. There 
are several important differences between 
urban public schools today compared to 
when I attended one in the middle of  the 
20th century. First, many students have no 
experience with adults who succeeded in life 
because they did well in school. In addition 
to multigenerational damage from 
inadequate schooling, other factors 
including race have created a situation in 
which some communities, based accurately 
on their experiences, do not share a 
common belief  that schooling is important 
to success in life. In addition, while white 
children from wealthy families spend their 
days in bright, cheery school environments, 
many children of  color experience schools 
that signal that society does not care about 
them except to keep their behavior from 
causing problems. Finally, many schools, 
especially those struggling in low-income 
areas, do not provide education in key 
knowledge and skill requirements for the 
information age.  

It would be easy to decide that all 
that needs to be added to what worked in 



Acquired Wisdom/Education Review                                                                                                             30 

 
my generation is social justice, but that will 
not be sufficient. For one thing, giving every 
child exactly what I got no longer is enough. 
Machines do most of  the things that average 
and below-average students learned to do in 
the schools I attended, even though our 
society has yet to adapt to this fully. So, part 
of  the problem is that children not well 
served by schooling are making decisions to 
disengage that are grounded in assumptions 
that are at least partly true. Any effort to 
change this must include a focus both on 
justice and on aiming high enough in our 
educational goals. In addition, regardless of  
how enticing we might be able to make 
schooling, it also is likely that the total effort 
required of  today’s students if  they are to 
have good lives is higher than was needed 
for my generation. 

This calls, I believe, for a 
multipronged strategy. First, we need to do a 
lot of  research on motivation, persistence, 
and engagement. Regardless of  what else 
happens and even if  schooling becomes 
perfectly fair tomorrow, we have generations 
of  families who have learned about 
schooling from experience, and what they 
have learned may not be optimal for 
assuring community support for student 
success. We need to understand how to get 
students more engaged and how to induce 
them to persist in the work of  learning. 

Second, we need to document and 
clearly explicate the differences between 
schools for the privileged and schools for 
the underprivileged. While selfishness, as 
embodied in suburban flight and other 
forces that make education separate and 
unequal, will not disappear, the effects can 
be ameliorated if  more of  the voting public 
understands how bad the situation is and 
how it might change. And, less privileged 
families need to understand what it takes for 
schooling to make a difference in their 
children’s lives. Parents need to be pushing 
for schools that prepare their children for 
good lives, which may require more than 
assuring the delivery of  diplomas that may 
not be coupled with deep preparation to use 
school-taught knowledge to solve novel 
problems. Some of  that preparation should 

include practice in collaborating with very 
different people than a student may be used 
to, for example, and parents need to learn 
why that could be important. 

Third, we need to help students 
learn to lead healthy, active lives. For a while 
now, recess and physical education has been 
sacrificed in many schools, often to allow 
just a little bit more drill to prepare for 
standardized tests that capture only part of  
what our children need to learn. We know 
that students who are more fit physically 
achieve more in school (Donnelly & 
Lambourne, 2011; Van Dusen, Kelder et al., 
2011). We also know that personal behavior 
decisions impact both health and the cost of  
health care. Part of  schooling needs to be 
about helping children learn to live healthy 
lives.  

These three areas – motivation, 
education for the less privileged, and 
education in healthy living – have been a 
focus of  my tenure as a dean. I have had the 
good fortune to be surrounded by great 
colleagues who are making impressive 
progress on these foci. I saw my role as 
supporting their efforts and helping assure 
that they were eventually joined by even 
more talented peers. This part of  being a 
dean I feel pretty comfortable about. We 
have remained strong in subject-specific 
pedagogy, and Pitt overall has remained a 
major force in the cognitive science of  
learning, but in addition, we also are focused 
on these three important areas. 

I am just starting to understand 
what schools of  education should be doing 
in different areas. Schooling as we know it 
today was dramatically influenced by the 
industrial age. Sometimes, this influence is 
associated solely with extreme prescriptions 
of  the sort advocated for industrial jobs by 
Frederick Taylor (1911). Taylor saw much of  
human labor as inefficient, and he pioneered 
time studies of  work that could lead to 
more production per worker. A benign 
example of  this is the specification of  
delivery routes for UPS drivers that 
eliminate left turns, which take a lot of  time 
because oncoming traffic has precedence 
over the person turning. A less desirable 



It Takes a Village of Mentors                                    31 

 
result was redesign of  assembly line jobs so 
that workers kept repeating a small number 
of  movements and thus could grind out a 
few more widgets each hour. 

Unfortunately, much of  the Taylor 
approach in education has focused on 
scripting teacher performance to eliminate 
variation that was seen as inefficient. 
Sometimes, such scripting resulted in small 
increases in test scores, but it also interfered 
with making major changes in goals and 
processes of  schooling needed for the 
information age. And, as in factories, it 
demoralized the workers, in this case both 
teachers and students.  

Even when teachers have been 
given a lot of  freedom, the world of  
education has been organized like a 
corporate bureaucracy. In addition to the 
bureaucracy of  school systems, additional 
layers of  bureaucracy exist at the state and 
federal level as well. At a minimum, very 
simple accountability measures have been 
used to assess how well school systems, 
schools, and teachers are performing. 
Instead of  money, which is the 
accountability measure of  the business 
world, education uses standardized tests. 

A simple accountability measure 
has strengths and weaknesses. The focus on 
profits has worked well for shareholders, but 
it has not always served social purposes nor 
has it assured the long-term health of  
businesses. After all, sometimes 
shareholders profit most when a company is 

stripped of  its assets, even if  workers are 
left in the lurch. Also, customers are not 
always helped by such measures – anyone 
who has taken a plane trip in recent years 
understands that airline profits do not map 
onto customer comfort. All in all, our 
economy works better because we attend to 
profits, but it could serve society better still 
if  we had additional measures of  corporate 
contribution. 

The currency of  test scores has 
worked similarly. Overall, test scores tend 
primarily to predict initial success in college, 
generally the freshman year, when grades 
come largely from mass testing similar in 
form to the tests used to measure school 
success. To the extent that achievement tests 
correlate highly with aptitude tests, 
standardized tests also predict, to some 
extent, the quality of  life in one’s later years 
(Holahan & Sears, 1995). At the same time, 
there are a variety of  pathologies introduced 
when schools focus solely on boosting mean 
test scores. Deep engagement of  content is 
sacrificed to drill. Teachers focus on 
students in the “bubble” who can, with 
minor gains, advance from one score 
category to another (e.g., from basic to 
proficient), ignoring the students who need 
the most substantial help. Socialization in 
kindergarten is sacrificed to have time for 
drill on accurately filling in multiple choice 
test forms. Art, music, and physical 
education, which are not tied to test score 
improvement in the bureaucratic mind, are 



Acquired Wisdom/Education Review                                                                                                             32 

 
sacrificed to make room for a few more 
minutes per day of  drill. Students disengage 
and drop out because classroom life is so 
regimented that it parallels the soulless life 
of  assembly line workers at the height of  
the “scientific management” era. 

Society is demanding that we 
change this, though policy makers struggle 
to figure out how the change can happen. 
Parents revolt against testing when their real 
concern is about the depressing nature of  
some classrooms. Moreover, in addition to 
demand for change, we also see flight from 
regimented, test-driven schooling. Children 
of  the wealthy may get paid tutors who help 
them to do better on the SAT, but the 
schools they attend throughout childhood 
are not driven by standardized test scores. 
Instead, students get large projects to work 
on that afford opportunities to practice the 
21st century skills linked to creativity, 
collaboration, communication, and using old 
knowledge to attack novel problems. They 
give up large chunks of  class time to 
rehearsals for the annual school play or 
musical. They also write more even if  
competent writing is not captured very well 
by multiple choice tests. 

Those of  us who prepare educators 
should, I believe, be working to bring the 
schooling that serves the wealthy into 
universal public use. To do this, we need to 
do two very big things. First, we need to 
prepare teachers, principals, superintendents, 
and policy makers to work in school 
environments that are healthy and that 
provide students with multiple, meaty 
patterns of  activity as they develop their 
knowledge and competence for life in our 
increasingly automated world. Students need 
to learn how to work in teams on complex 
problems for which no one person is fully 
prepared. They need to learn how to learn 
new skills quickly, since any job will 
eventually be a candidate for automation. 

                                                           
12 In recent years, some schools have put aside time for students to make artifacts that interest them. 
Making a complex object or computer application requires practicing the skills mentioned above: 
creativity, collaboration, communication, and applying old knowledge to novel situations. The trend 
toward this kind of  activity in schools and in after-school settings has been referred to as the “maker” 
movement. 

Rather than getting all their practice on 
small performances for which they are 
deeply drilled, they need to practice 
confronting tasks that might not have been 
imagined even a few years ago and at 
investing effort even when there is some 
chance of  failure. This is a matter both of  
specific skills – managing student projects, 
stimulating “maker” activity,12 etc. – and 
deep understanding of  how the rise of  
machine intelligence is changing what makes 
people valuable to their neighbors. 

Second, we need to develop 
alternative ways of  assessing how well 
learning is proceeding. For starters, just as 
foods in the store and increasingly even in 
restaurants contain nutritional labeling, we 
need to develop simple schemes for labeling 
the educational nutrition value of  games, 
apps, projects, cyber school lessons, and 
other learning opportunities. My father ate 
his meals during army service in a mess that 
was regimented like the factories of  that 
time. An expert figured out what should be 
eaten each day, and everyone got the same 
thing. Today, parents make choices for their 
children, sometimes guided by nutritional 
information that food products must 
provide. The schools of  the future, in order 
to rehearse for life in the future, will need to 
be places where teachers and students 
choose from a range of  possible activities, 
guided by indices of  their “nutritional” 
value, and also get periodic assessments of  
their learning health just as we get physical 
examinations and feedback on our physical 
health. We will, of  course, need to help 
parents and teachers learn to use education’s 
equivalent of  nutritional information, and 
we will need to learn how to assess the 
“nutritional” content of  activities for the 
mind. 

As noted above, much of  this 
scenario exists for the wealthy, though the 
measures are mostly personal intuitions 



It Takes a Village of Mentors                                    33 

 
enforced by the power of  personal wealth. 
The wealthy would benefit from the 
assessment approach just suggested. The 
less wealthy would benefit not only from 
such assessments but also from well-trained, 
imaginative teachers taught to proceed 
fearlessly in making learning opportunities 
for their students.  

We have, in recent years, seen a 
great deal of  deconstruction of  our public 
education system.  Without efforts by the 
educational research and teaching 
community, this likely will continue. And, it 
will work for many children, but not for all. 
Parents who received a strong education and 
who are adapted to the information age will 
be able to curate the learning opportunities 
available to their children, and those 
children will be served well. But, not all 
children have the benefit of  parents with a 
solid education along with family resources 
sufficient to support access to many 
learning opportunities. Those children face a 
very real probability of  not being prepared 
to fare well in an individualistic world where 
the ability to do more than cheap software 
can do is the key to economic success. The 
number of  such children already is high, 
because we have failed many children for 
over a generation and they now are parents. 

With increased economic division 
between great wealth and bare subsistence 
poverty, the effects of  past failings of  our 
education system likely will grow and 
certainly won’t decrease. And, more routine 
jobs are disappearing because of  
automation, along with jobs requiring only a 
modicum of  intelligence. The education 
schools of  the future will need to attack this 
problem in diverse ways. Some will try to 
recreate universal public education. Some 
will try to assure that an open marketplace 
of  educational options is partly disciplined 
by improved ability to assess the likely utility 
of  various offerings. Some may even 

                                                           
13 A student-run start-up company being incubated by the University of  Pittsburgh has developed a 
computer system that can observe a person’s knee movements and automatically diagnose the results of  
physical therapy after ACL injuries, for example. Currently doctors and well-trained physical therapists do 
this work, but the automated system sometimes beats the best of  them, and the student team is just 
getting started. 

develop their own offerings in the cyber 
world. The stakes are high. If  we don’t 
attack these fundamental issues, our value 
for society as educators will diminish, and 
education schools as we know them may 
disappear. 

I believe that among the 
educational opportunities that should exist 
in the future are systems like the intelligent 
tutors that I built. Those tutors allowed 
technicians to take knowledge acquired 
earlier and apply it to really tough problems, 
thus learning how to make use of  what they 
had learned before. Surely, students should 
have similar opportunities.  Some of  today’s 
players of  computer games do have such 
opportunities. However, the games in which 
they immerse themselves are not designed 
to be optimal teachers. Perhaps we need 
games that have built-in tools like the 
Process Explorer that Marty Nahemow 
developed. Such tools could provide just-in-
time simple explanations of  the science or 
other understanding behind a particular 
game situation. Perhaps more of  the games 
that are available should allow students to 
tackle authentic problems that society faces.  

Finally, some of  what we need to 
be doing in the world of  educational 
scholarship amounts to a new kind of  
educational philosophy. We need to keep 
asking ourselves, as industry keeps 
producing greater computational power, 
what the role of  humans is in the age of  
smart machines. The answers may change 
over time. Assuming essentially infinite 
computational power, simply identifying a 
human role clearly makes it a target for 
automation. Even the professions will 
struggle (see Susskind & Susskind, 2015) as 
computers take over every day professional 
work like drafting routine contracts, quickly 
scanning medical test results for signals of  
disease, and even evaluating progress in 
recovering from knee injuries.13  



Acquired Wisdom/Education Review                                                                                                             34 

 
At present, at least, there are some 

key human roles. Humans beat machines in 
attacking novel problems, emergent 
situations for which we have not prepared 
intelligent machines. Humans usually beat 
machines at bridging between bodies of  
knowledge that rest on differing ontologies. 
This includes diplomacy, but it also includes 
mundane things like mediating between 
consumers and insurance companies. Also, 
humans still excel at learning to do new jobs 
during the brief  period before they are 
displaced by new computer systems, though 
that niche may be eroded as machines get 
smarter. 

Note that none of  these three 
broad areas of  human dominance are readily 
assessed by today’s standardized tests. None 
are covered by the standard curriculum. The 
Common Core State Standards brush 
against some of  this unsupported space of  
needed learning, but not as completely as 
might be optimal. Perhaps the next 
generation of  educational philosophy will 
address this mismatch.  

Education schools that persist as 
they have existed for about a century likely 
will shrivel or transform. At the same time, 
no one who stays aware of  how learning 
occurs and what learning is needed will be 
short of  work in the decades ahead. Thus, 
the future for educators and educational 
researchers looks pretty positive. It has been 
extremely gratifying to be involved in 
strategic planning and subsequent action of  
a school of  education at this moment of  
great change.  

 
Broad Lessons Learned 

Throughout this chapter, I have 
tried to identify some important lessons that 
can be learned from the patterns of  my 
experience. I conclude now by restating a 
few of  them more generally. 

First, we live in a time of  rapid 
change. This creates both problems and 
opportunities. The key problem for 
education is that what we need to know 
keeps changing. We are pretty good at 
measuring the acquisition of  specific 
knowledge, but we have not fully mastered 

assessing the ability to learn new content or 
to apply old knowledge to novel situations. 
With so much educational policy shaped by 
a focus on accountability, we push schools 
to teach what can be measured. This will be 
a problem until we learn how to measure 
the ability to figure out what new 
competence is needed to deal with an 
emergent life situation, to learn what is 
needed to have that competence, and to 
collaborate with others when no one person 
has all needed competence. Without the 
right measures, accountability driven by 
inadequate measures produces 
pathological results – small changes in 
what can be measured that do not 
represent progress toward the real goal. 
This does not mean that measures have to 
be perfect. We steer our cars remarkably 
well without ever investing effort in a 
perfect decision about how next to move 
the steering wheel. At the same time, 
steering fails if  we don’t look at the road. 

While that is mostly a lesson 
learned by watching others, there is a related 
personal lesson as well. Life in a time of  
change requires a sense of  adventure. There 
are dangers to undisciplined adventure, and 
some adventures end in failure, but there 
also are dangers in not taking risks. Indeed, 
sometimes not taking risks is the riskiest 
behavior of  all. As I look back on the 
careers of  my colleagues and my own career, 
I see some people who stuck to what they 
had initially learned to do and others who 
took on new problems when they felt well 
placed to tackle them, even if  they didn’t 
know everything they needed to know and 
even if  it wasn’t always the safest thing to do 
in terms of  success in their social situations. 
Those who contribute the most in times 
of  rapid change and who are most 
successful take measured risks when 
great opportunities arise and try to have 
fallback plans if  they fail in their 
adventures. 

A really important lesson comes 
from my childhood, as noted near the 
beginning of  this chapter. Getting a strong 
formation and education requires three 
things: good schools, community 



It Takes a Village of Mentors                                    35 

 
support not only for the school but also 
for general social formation, and 
personal effort. The lessons I learned 
watching people shovel sand over slippery 
spots to help their neighbors were as 
important as the social lessons learned in 
kindergarten and later schooling. We need to 
learn how to teach children by example that 
a strong system of  free enterprise can only 
survive if  it teaches its next generations that 
neighbors take care of  each other even as it 
rewards business adventures that succeed in 
improving people’s lives. 

A broad lesson from my career, 
beyond just valuing adventure, was that 
setbacks also are opportunities. Things 
have not gone smoothly all the time, but 
even painful events like not getting tenure 
provided a basis for the start of  good 
things. Related to this lesson is another – 
challenges are opportunities. By finding a 
way to teach myself  some basic artificial 
intelligence methods using the computer at 
Stanford, I created some 
opportunities for 
collaboration that led to 
creation of  the 
Intelligent Systems 
Program. That, in turn, 
provided a flow of  students with talents that 
greatly expanded the work I could do. This 
is an instance of  another lesson – great 
things can be done when people work 
together and complement each other’s 
skills. 

Another lesson I have learned is 
that a reputation for honesty really pays 
off. When we don’t have perfect measures 
of  desired results, we necessarily must rely 
on honesty of  the various partners in new 
efforts. The Lisp machine project required 
that people at Xerox PARC, people in the 
government, and people at Pitt could all 
trust each other. Without that trust, 
adventurous action was not possible. With 
it, we were able to do some good. While I 
hope I have a reputation of  being as good 
as my word, I deeply believe that for me 
honesty has been highly rewarding; I could 
not have had the wonderful experiences I 
have had without it. 

I should make one final point. 
There were no miracles in any of  the work I 
have done. Like every other academic, I had 
periods of  stress from working very hard. I 
like to think that I worked on things that 
matter, but I’m sure that decisions about 
what to work on were satisficing decisions 
and that some of  the time, I could have 
skimped on something that in the end did 
not matter. I hope my family didn’t suffer 
too much because of  that.  

Some of  what looks a bit more 
audacious, like finding a way to bring 
artificial intelligence hardware to a number 
of  researchers and putting some of  my time 
into supporting that, required a level of  
faith in the belief  that it was worth the 
effort to support advances by many 
colleagues rather than just staying focused 
on my own research plans. Thinking beyond 
one’s immediate goals has more long-term 
than short-term payoff. But, we humans live 
a long time, and preparing for the long term 

may be wise. 
What looks 
audacious to 
some may look 
scary to others. 
Nothing I have 

been able to accomplish required much 
courage. What it did require is the 
conviction that if  I act as if  the scholarly 
world is a group endeavor and that we work 
for each other as well as for ourselves, then 
others will share that conviction. This is not 
true universally, but it’s true enough. This 
leads to my last lesson – good things 
become possible not because everyone is 
ready to work hard on them but rather 
because enough people are.  

In closing, I again thank all my 
mentors and companions in the adventures 
of  scholarship. I hope I was as useful to all 
of  you as you were to me. I owe a lot to so 
many people, including Charles Wrigley, 
Frank Sim, Gordon Bower, Dick Atkinson, 
Herb Clark, Chuck Perfetti, Allen Newell, 
Robert Glaser, Lauren Resnick, Marty 
Nahemow, and my many other Pittsburgh 
colleagues. Some were critical influences at 
one moment while others provided years of  



Acquired Wisdom/Education Review                                                                                                             36 

 
advice. All were important sources of  
wisdom, direction, and confidence.

 
 

 
References 
 
Branson, R. K., Rayner, G. T., Cox, J. L., Furman, J. P., King, F. J., & Hannum, W .H. (1975). 

Interservice procedures for instructional systems development. (Vols. 1-5) TRADOC Pam 350-30, 
NAVEDTRA 106A. Ft. Monroe, VA: U.S. Army Training and Doctrine Command. 

Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive psychology, 4(1), 55-81. 
Donnelly, J. E., & Lambourne, K. (2011). Classroom-based physical activity, cognition, and 

academic achievement. Preventive Medicine, 52, S36-S42.  
Friend, T. (2015, May 18). Tomorrow’s Advance Man: Marc Andreessen’s plan to win the future. 

The New Yorker. 
Gagné, R. M. & Briggs, L J. (1974). Principles of Instructional Design (2nd ed.). Holt, Rinehart, and 

Winston.  
Gott, S. P., & Lesgold, A. M. (2000). Competence in the Workplace: How Cognitive 

Performance Models and Situated Instruction Can Accelerate Skill Acquisition. In R. 
Glaser (Ed.), Advances in instructional psychology. Hillsdale, NJ: Erlbaum.  

Holahan, C. K., & Sears, R. R. (1995) The Gifted Group in Later Maturity. Stanford, CA: Stanford 
University Press. 

Kiester, E., Jr. (1993, March). Germany prepares kids for good jobs; we were preparing ours for 
Wendy's (apprenticeship program). Smithsonian, 23(12), 44+. Retrieved from 
http://go.galegroup.com/ps/i.do?id=GALE%7CA13566287&v=2.1&u=upitt_main&it
=r&p=AONE&sw=w&asid=f96301aabf6dd395684e82c39d24f720 

Lesgold, A. M. (1984). Acquiring expertise. In Anderson, J. R., and Kosslyn, S. M. (eds.), Tutorials 
in Learning and Memory. San Francisco: Freeman, San Francisco. 

Lesgold, A. M. (2012). Practical issues in the deployment of new training technology. In P. J. 
Durlach and A. M. Lesgold (Eds.), Adaptive Technologies for Training and Education. New 
York: Cambridge University Press. 

Lesgold, A. M., & Nahemow, M. (2001). Tools to assist learning by doing: Achieving and 
assessing efficient technology for learning. In D. Klahr & S. Carver (Eds.), Cognition and 
instruction: Twenty-five years of progress. Mahwah, NJ: Erlbaum.  

Lesgold, A. M., Resnick, L. B., & Hammond, K. (1985). Learning to read: A longitudinal study of 
word skill development in two curricula. In G. Waller and E. MacKinnon (Eds.), Reading 
Research: Advances in Theory and Practice. New York: Academic Press.  

van Merriënboer, J. J .G. (1997). Training Complex Cognitive Skills: A Four-Component Instructional 
Design Model for Technical Training. Englewood Cliffs, New Jersey: Educational Technology 
Publications.  

Natriello, G., & Dornbusch, S. M. (1984). Teacher evaluative standards and student effort. New York: 
Longman.  

Sieg, W. (2007). The AProS project: Strategic thinking and computational logic. Logic Journal of 
IGPL, 15(4), 359-368. 

Shulman, L. S. (2005). Signature pedagogies in the professions. Daedalus, 134(3), 52-59. 
Stripp, A. (1993). The Enigma machine: Its mechanism and use. In F. H. Hinsley & A. Stripp, 

Codebreakers: The Inside Story of Bletchley Park. (Pp. 83-88). Oxford: Oxford University 
Press. 

Suppes, P. (1969). Stimulus-response theory of finite automata. Journal of Mathematical Psychology, 
6(3), 327-355. 

http://www.amazon.com/Principles-Instructional-Design-Robert-Gagne/dp/0534582842/bigdogsbowlofbis/
http://www.amazon.com/Training-Complex-Cognitive-Skills-Four-Component/dp/0877782989/bigdogsbowlofbis
http://www.amazon.com/Training-Complex-Cognitive-Skills-Four-Component/dp/0877782989/bigdogsbowlofbis


It Takes a Village of Mentors                                    37 

 
Susskind, R., & Susskind, D. (2015). The Future of the Professions: How Technology Will Transform the 

Work of Human Experts. OUP Oxford.  
Suthers, D., Connelly, J., Lesgold, A., Paolucci, M., Erdosne Toth, E., Toth, J., Weiner, A. 

(2001). Representational and advisory guidance for students learning scientific inquiry. In 
K. Forbus and P. Feltovich (Eds.), Smart machines in education. Menlo Park, CA: AAAI 
Press.  

Taylor, Frederick Winslow (1911). The Principles of Scientific Management, New York, NY, USA and 
London, UK: Harper & Brothers. 

Toth, E. E., Suthers, D. D., and Lesgold, A. M. (2002). Mapping to know: The effects of 
representational guidance and reflective assessment on scientific inquiry skills. Science 
Education, 86, 264-286. 

Tulving, E., & Donaldson, W. (1972). Organization of memory. New York: Academic Press. 
Van Dusen, D. P., Kelder, S. H., Kohl, H. W., Ranjit, N., & Perry, C. L. (2011). Associations of 

Physical Fitness and Academic Performance among Schoolchildren. Journal of School 
Health, 81(12), 733-740. 

Whitehead, A.N. (1929). The aims of education and other essays. Free Press, New York.  
Yoncheva, Y. N., Wise, J., & McCandliss, B. (2015). Hemispheric specialization for visual words 

is shaped by attention to sublexical units during initial learning. Brain and language, 145, 
23-33. 

 
 
 
 

 
 
 

 
 

  



Acquired Wisdom/Education Review                                                                                                             38 

 
 

 

 

 About Acquired Wisdom 

This collection began with an invitation to 

one of the editors, Sigmund Tobias, from 

Norman Shapiro a former colleague at the 

City College of New York (CCNY). Shapiro 

invited retired CCNY faculty members to 

prepare manuscripts describing what they 

learned during their College careers that 

could be of value to new appointees and 

former colleagues. It seemed to us that a 

project describing the experiences of 

internationally known and distinguished 

researchers in Educational Psychology and 

Educational Research would be of benefit to 

many colleagues, especially younger ones 

entering those disciplines. We decided to 

include senior scholars in the fields of adult 

learning and training because , although 

often neglected by educational researchers,  

their work is quite relevant to our fields and 

graduate students could find productive and 

gainful positions in that area.  

 

 

 

 

 

 

 

 

Junior faculty and grad students in 

Educational Psychology, Educational 

Research, and related disciplines, could learn 

much from the experiences of senior 

researchers. Doctoral students are exposed 

to courses or seminars about history of the 

discipline as well as the field’s overarching 

purposes and its important contributors. .  

A second audience for this project include 

the practitioners and researchers in 

disciplines represented by the chapter 

authors. This audience could learn from the 

experiences of eminent researchers—how 

their experiences shaped their work, and 

what they see as their major contributions—

and readers might relate their own work to 

that of the scholars. The first issue, prepared 

by Tobias as a sample chapter, was intended 

for illustrative purposes. Authors were 

advised that they were free to organize their 

chapters as they saw fit, provided that their 

manuscripts contained these elements: 1) 

their perceived major contributions to the 

discipline, 2) major lessons learned during 

their careers, 3) their opinions about the 

personal and 4) situational factors 

(institutions and other affiliations, 

colleagues, advisors, and advisees) that 

stimulated their significant work. 

We hope that the contributions of 

distinguished researchers receive the wide 

readership they deserve and serves as a 

resource to the future practitioners and 

researchers in these fields. 

 

  



It Takes a Village of Mentors                                    39 

 

 
 

Acquired Wisdom is 

Edited by 

          Sigmund Tobias                       J. D. Fletcher                 David C. Berliner 

University at Albany             Institute for Defense Analyses         Arizona State 

University 

    State University of New York              Alexandria VA          Tempe AZ 

 
Advisory Board Members 

Gustavo Fischman, Arizona State University 
Arthur C. Graesser III, Memphis State University 

Teresa l. McCarty, University of California Los Angeles 
Kevin Welner, Colorado State University 

 

Education Review/Reseñas Educativas/Resenhas Educativas is supported by the edXchange 
initiative’s Scholarly Communications Group at the Mary Lou Fulton Teachers College, 

Arizona State University. Copyright is retained by the first or sole author, who grants right of 
first publication to the Education Review. Readers are free to copy, display, and distribute this 
article, as long as the work is attributed to the author(s) and Education Review, it is 
distributed for non-commercial purposes only, and no alteration or transformation is made 
in the work. More details of this Creative Commons license are available at 
http://creativecommons.org/licenses/by-nc-sa/3.0/. All other uses must be approved by 
the author(s) or Education Review. Education Review is published by the Scholarly 
Communications Group of the Mary Lou Fulton Teachers College, Arizona State University.  

 
Please contribute reviews at http://www.edrev.info/contribute.html. 
 
Connect with Education Review on Facebook 

(https://www.facebook.com/pages/Education-Review/178358222192644) and on Twitter 
@EducReview 

 
 
 
 
 
 
 
 

 

http://www.edrev.info/contribute.html
https://www.facebook.com/pages/Education-Review/178358222192644