Please log your edits Content and Pedagogical Knowledge in Colorado Teachers’ Mathematics Exams at the Turn of the 20th Century Robert M. Capraro Lynn M. Burlbaw Texas A and M University Linda Reichwein Zientek Sam Houston State University Abstract During the late 1800s and early 1900s, several published reports recommended changes in teaching mathematics. For this paper, items from the Colorado county arithmetic tests (1878 – 1912) form the basis of investigating teacher’s content knowledge and the degree to which education was adhering to these recommendations. The purpose of this paper is to investigate whether or not recommendations proposed by the Report of the Committee of Ten (NEA, 1893) and the Report of the Committee of the Chicago Section (1899) made an impact on teacher certification exams in the state of Colorado. Certification of Teachers By portraying teachers as young women graduates of the 8 th grade who then went on to teach children younger than themselves, television shows such as “Little House on the Prairie” have reinforced a view that 19 th and 20 th century teachers lacked sufficient content and pedagogical knowledge. The idea that becoming a teacher requires little education beyond what was taught in the public school remains viable in some peoples’ minds. The idea that an 8 th grade education was sufficient to teach affirms a belief that teaching requires little specific knowledge, either in content or pedagogy. This idea is further substantiated with accelerated programs which give a minimal amount of training prior to assuming full classroom responsibility (Hallinan and Khmelkov, 2001). Consider the following questions: A square and a triangle contain an equal area, and the base of the triangle is 36.1 feet and its altitude is 5 feet, what is the side of the square? A man’s expenses are 30 percent of his income, and 33 percent of his income equals 10 percent of his property which is valued at $27,000. What is his income? What are his expenses? Examination questions such as these may seem rather commonplace on a mathematics test. Consider though, a question such as, “When it is 1 o’clock a.m. on the first day of January, 1889, at Bangor, Maine, 68º 47’ west, what was the time at the City of Mexico, 99º 5’ west? might not routinely appear on a mathematics test. These questions (Peavey 1896, 288, 283, 288), and others like them, were routine test items for people seeking to become elementary level teachers in Colorado between 1878 and 1923. A vision of the certification process in turn-of-the-century Colorado has been presented by Burlbaw (2002, 2003a, 2003b) and Burlbaw and Bozeman (2003). These papers show that becoming a certified teacher was not automatic 159 and that teachers receiving county certificates were continuously studying and completing exams. Some authors (cf. Askey 2005; 2006a; 2006b; 2007; nd.) have examined contents on teachers’ exams in California and Michigan, questioning whether current teachers could pass those exams. This paper focuses on the nature of the content of the arithmetic exams teachers completed between 1878 and 1912, examining the expected knowledge through a modern lens of modern national standards. To contextualize teacher preparation (1) Types of Certificates, (2) Content of Exams, (3) Criteria for Class (Grade) of Certificate, (4) Teacher Performance on Exams, and finally (5) Teaching in the 20th Century are discussed. Types of Certificates To teach in the state of Colorado in the late 1800s, one had to have one of three types of certificates: a county certificate, a state certificate or a normal school certificate. The latter, a life- time certificate qualified a person to teach at any public school in the state and was awarded once a person completed a course of study at one of the state teachers’ colleges. A second type of certificate, the state certificate, also a life-time certificate to teach at any school in the state, was awarded once a person passed the "state" teachers exam. In order to sit for the “state” exam, a candidate had to possess a valid “first grade” county certificate. Beginning in 1895, a Normal Institute Certificate, another lifetime certificate, was issued to teachers who attended summer institutes, successfully passed the required number of courses during the summer sessions and paid tuition for university credit (Craig 1906a). The Normal Institute Certificate and program was an attempt to address the lack of preparation of teachers “who had very little or no professional training” and provided the “minimum of continued professional training required by law for those who would continue to be certified” (Strayer and Englehardt 1920, 18, 34). In the period from 1880-1920, county certificate exams were not viewed as a way around what today would be considered a “traditional” normal school certification process but the quickest and most efficient mode to provide teachers. With a growing population, counties needed teachers and a way to ensure quality instruction. In 1891, 86.6 percent of school age children under 16 were enrolled in school; these 66,750 students were largely taught by 1877 county certificate teachers; 419 of whom had First Grade Certificates; 719 Second Grade Certificates and 739 Third Grade Certificates (Coy 1893). In his first Biennial Report, W. C. Lothrop (1872), Superintendent of Public Instruction of the Territory of Colorado wrote: The highest discretion should be used in the employment of a teacher for any position. It is a popular fallacy to suppose that anyone who is a fair scholar can teach school, especially doest this false notion prevail in regards to schools composed of small children. . . Let me here suggest to county superintendents the necessity for great care in the examination of teachers and discrimination in issuing certificates of qualification. . . . To county superintendents then, we must look for the means of preventing the employment of incompetent teachers. (7-8) The University of Colorado at Boulder opened its first education school in 1881; the first institution dedicated to teacher preparation, the State Normal School, at Greeley, was founded in 1889 and opened its doors to students in 1890. A second normal school at Gunnison (today 160 Western State College) was authorized in 1901 but did not open its doors until September 12, 1911. As late as 1919, 48 percent of the teachers teaching in one-teacher schools were teaching under county certificate authority. In two-teacher and three or more-teacher schools and high schools, the percentages were 33 percent, 11 percent and 9 percent respectively (Bradford 1921). In 1925, 41 percent of the teachers in the state were county certified while 39% had normal school certificates and only 16 percent had state certificates (Bradford 1927). According to a 1911 report (Updegraff 1911), Colorado was one of 16 states that used a state-county certification model; only in 17 states was the sole certification authority the state board of education (138-142). Thus, for over 50 years, county exams, as used in many other states, were vital in ensuring quality teachers were available for Colorado public schools. After an applicant met the criteria for a given certification on the state prepared test the county superintendent of schools issued the certificate. During the 1880s, the county test was administered quarterly, usually at the county seat although provisions were made for testing in another larger city in the county. Although the First Biennial Report indicates that tests were administered quarterly (Lothrop 1878), the exact dates do not appear until the Third Biennial Report where the dates are listed as the “last Friday of February, May, August and November (Cornell 1883). This was a departure from the earlier practice of giving the exam on Saturdays. L. S. Cornell, State Superintendent of Public Instruction, suggested to county superintendents that they consider administering the test over two days, 6 subjects each day, in a five to six hour session (Cornell 1883). By 1898, tests were being administered only three times a year (March, August, and December) on dates designated by the State Superintendent of Public Instruction. County certificates were valid for a specific period of time, ranging from six months to three years. Three grades or classes of certificates were awarded: First, Second and Third. The term grade did not refer to school grades but hierarchy of certification type. Therefore, for this paper we will use the term class to refer to the different levels of certification. First class certificates entitled the holder to teach for three years in any school in Colorado. The other certificates for lesser periods of time as explained below in the section, Criteria for Class (Grade) of Certificate. Content of Exams. With the exception of removing botany and adding civil government in 1895 to the subjects tested, the content remained consistent over time (Peavey 1896). A review of the published exams reveals the following subjects were tested each time: arithmetic, reading, U.S. history and constitution; physiology/health, orthography, school law, natural science, grammar, theory and practice, geography, and penmanship/writing. From 1878 to 1882, the subjects tested were arranged into two branches with different passing scores required for each branch: The school law requires examination in the “Elements of the Natural Sciences.” As this is a new departure it was thought best to fix the standard of these requirements in these branches a little lower than in those branches in which applicants had formerly been examined. (Lothrop 1878, 20) Table 1 shows the division of the subjects into two branches; the division into branches was discontinued in 1885 (Cornell 1887). 161 Table 1: Subjects Tested in Each Branch Branch One Branch Two Arithmetic U. S. History and Constitution Reading Orthography Grammar Theory and Practice Physiology and Laws of Health School Law Botany Other Natural Sciences Penmanship (Lothrop 1878, 19) The order of the subjects on the exams varied by administration but each was tested each time. In the early testing years, specific lengths of time were allotted for candidates to complete each content test. For example the time allotment, for arithmetic (10 questions) was 40 minutes; physiology and health (5 questions), 25 minutes; and theory and practice (10 questions), 35 minutes (Lothrop 1878). In later years, the test was extended to a two day session. Three hours in the morning and three hours in the afternoon were allotted for completing assigned test sections. According to Shattuck (1884), “There is work for two days of five or six hours each for the average applicant and I recommend a two days’ session . . . Applicants should have time to do themselves justice. . . . For the first day use Nos. 1 to 6 inclusive; second day, Nos. 7 to 12 inclusive” (29). For many years, both the county and the state certification tests were published in the Biennial or Annual Superintendent's report (cf. Lothrop 1878; Cornell 1883, 1887; Coy 1893; Craig 1908; Grenfell 1900). Criteria for Class (Grade) of Certificate Because all candidates took the same exam, distinction between certificates was determined by a teacher’s performance on the exam, overall and on individual parts. Teachers had to achieve a certain score on each subject of the exam and a specific cumulative average was required for each certificate. The higher certificates required both a higher subject and overall score. Table 2 provides a picture of the subject scores and test averages needed for each of the certificates at various times between 1878 and 1923. Beginning with the first quarter administration in 1885, subjects were no longer differentiated by branch and a single scoring standard was used for all certificates. In 1898, the minimum criterion score for the third class certificate was raised. Table 2: Required Averages and Scores for Certificates, 1878 – 1923 1878; 1882 to 1884 Administrations 1879 to 1881 Administrations 1885 to 1897 Administrations 1898 and after Administrations Certificate Grade Branch a Avg b Min c Ave b Min c Ave b Min c Ave b Min c First One 90 75 90 75 90 70 90 70 Two 75 60 75 40 Second One 75 60 75 60 80 60 80 60 Two 60 40 50 40 Third One 60 50 60 50 70 50 70 60 Two 50 40 50 40 Note. . a After 1885, subjects were no longer divided into branches; a single minimum score applied to all subjects. b Average = average of all scores received on each subject on the exam. c Minimum = lowest allowable score on any subject on the exam to receive that grade of certificate. 162 For example, in 1895, a teacher whose scores on the subject exams were at least 70 but whose overall average on the 12 subject exams was in excess of 90, would receive a First class certificate. If each subject exam was at least 70 but overall average was less than 90 or if on one subject the score was below 70, the teacher would be awarded a Second class certificate. In addition to achieving a high score on the test, First class certificate teachers had to demonstrate success in teaching before taking the test that qualified them for the First class certificate (Lothrop 1878; Cornell 1883; Craig 1908; Wixson 1912). Second class certificates were issued to those scoring within a certain band, below that of a First class certificate, on the exam and qualified a teacher to teach for 12 to 18 months at any school in the county of issuance with no transferability to another county. The Third class certificate, depending on the time issued, was valid for either six or nine months and also not transferable to another county. Holders of a Third class certificates scored within the lowest band of acceptable scores. A teacher could only receive a third class certificate twice; after that a teacher could not be certified to teach without attaining a score high enough to warrant a First or Second class certificate. Beginning in 1901, the State Superintendent of Public Instruction office published a listing of those who had received a Third class certificate that was distributed to all county superintendents (cf. Grenfell 1900, 1902; Craig 1906b, 1908; Wixson, 1912). Teacher Performance on Exams Information on how teachers performed on the various exams can be found in two types of reports. The first, and most general, is tabular information found in the Biennial Reports. These tables report on how many teachers received First, Second, and Third class certificates in each county and how many teachers were teaching on each certificate type among other things. The second source of information, a richer and more detailed record, is also found in the Biennial Reports. This source is a listing of the test items which appeared on each of the offered exams. The items from the arithmetic tests (1878 – 1912) form the basis for this paper’s analysis of teacher content knowledge. The practice of publishing the exams was not limited to Colorado. The Superintendent of Public Instruction in Michigan (1900) also published the exams as did E. L. Kellogg and Company which published test items from the New York State Uniform Examination (Kellogg 1889, 1890). Influences on Mathematics Education In the period between 1890 and 1900, several national reports were issued and changes within Colorado’s educational administration were occurring. The 1893 Report of the Committee of Ten and the 1899 Report of the Committee of the Chicago Section made recommendations regarding what teachers should know and how students should be taught (as cited in Bidwell and Clason 1970/2002). In the 1893 report, the committee divided the special reports into four subheadings: (a) arithmetic, (b) concrete geometry, (c) algebra, and (d) formal geometry. The committee recommended that arithmetic be both abridged and enriched with an elimination of the majority of commercial arithmetic problems. Recommendations were given on the importance of accuracy and speed scholars should be able to perform the four fundamental operations. Practical applications should be extended to real- world applications that are interesting to students. The 1899 Report of the Committee of the Chicago Section advocated different presentation styles and believed more would be gained by 163 presenting problems in more than one way. Arithmetic problems should be confined to (a) the fundamental processes, (b) factorizations, (c) prime numbers, (d) denominate numbers, (e) operations in fraction, (7) money, (8) decimals, (9) simple interest, and (10) method of analysis for simple and compound proportion (Bidwell and Clason 1970/2002). The Committee believed high school teachers should have sufficient mathematics knowledge through calculus as well as the theory of equations. In addition to content knowledge, the Committee advocated pedagogical content knowledge acquired under the supervision of an experienced mathematics teacher. While the 1899 report provided insights into mathematics such as how it should be taught, what should be taught, and when in school specific content should be taught it did not provide a clear framework for analyzing the exams of the day. “Various method of teaching mathematics are in vogue. The good teacher will not tie himself to any one method, but, on occasion, make use of the good features of every one. . . . The committee recommends no single method above all others, . . . the aim should always be to cultivate independent thinking to the part of the pupil. A method that . . . permits rote work or mechanical manipulations , is radically wrong.” (196). From these excerpts it seems the use of current NCTM standards as an analytic framework is logical and justified even though, as we know, mathematics education took a very different turn with the rise of behaviorism at the turn of the century. Purpose of Paper The purpose is to report on our investigation into whether either of two conditions had an effect on the content of the certification exams in the state of Colorado at the turn of the 20 th century; hence, influencing what and how it was taught: 1. Did the recommendations of the 1893 Report of the Committee of Ten, the 1899 Report of the Committee of the Chicago Section make an impact on teacher certification? 2. Did the effect of the leadership of Helen Grenfell, State Superintendent of Public Instruction between 1899 and 1905, who was known for her support for pedagogical knowledge, have an impact on teacher certification exams? The topics discussed at the turn of the 21 st century were some of the same topics discussed at the turn of the 20 th century. Therefore, the NCTM content and process strands were used as a lens to classify and contextualize the problems and to examine how they changed in regard to content and pedagogy at the turn of the 20 th century. Helen Grenfell - a former classroom teacher - was elected in 1899 to the position of State Superintendent of Public Instruction. Grenfell’s support for pedagogical knowledge appears to align with the pedagogical recommendations of the 1899 Report. Grenfell served from 1899 to 1905 and was the former president of the Colorado Education Association. In 1903, during her tenure as State Superintendent, she was elected the first female Vice-President of the National Education Association. Grenfell was an ardent supporter of improving public education and entered the public school system of Colorado as a teacher in Eire, Colorado, in 1880. In 1882 she received a Master’s Degree at State Normal School in Albany, New York. She later taught in Blackhawk, Colorado, eventually being appointed 164 County Superintendent in 1896. Grenfell continued her studies in education and campaigned for school improvement throughout her life (Grenfell 1939). Methodology A historical synthesis and data reduction of mathematics teacher certification exams was conducted on Colorado tests from 1878-1912. In the Biennial Report to the governor on the status of education in Colorado, the State Superintendents of Public Instruction published the items used on each of the tests during the previous biennium. These tests, published in the Biennial State Superintendent Bulletins, were located in the Denver Public Library and Colorado State Archives and were photocopied. The certification exam items for the arithmetic section from 1878 – 1912 were coded. Each question was coded by (a) year, (b) process strand: problem solving, reasoning and proof, communication, connections, or representation, (c) content strand: number and operations, algebra, geometry, measurement, and data analysis or probability, (d) according to real- world problems: business, life, or neither, and (e) as pedagogy problems. If a problem was coded as business or commercial, it was examined to determine if reform suggestions from the Report of the Committee of Ten (National Education Association 1893) were implemented. The NCTM process strands were used as markers to determine changes in the Colorado educational system to match to recommendations set forth by mathematical organizations and educational entities of that time period. Further, problem difficulty, depth of content knowledge required to successfully answer the question, and the scope of topics covered was classified. Therefore, the use of process strands served as a proxy for influences on the state educational system. Problem Coding The criteria for coding an item as problem solving were: (1) several obvious solution strategies and (2) higher order thinking skills required. Reasoning and proof problems were problems in which formulas could be applied to the problem or an application of basic order of operations could result in the correct answer. Communication problems asked the prospective teacher to explain or demonstrate a concept. Connection problems asked the prospective teacher to convert or reduce a number to another form. Representation problems asked the prospective teacher to illustrate or show concretely the problem solution. Data were coded as “prior to” and “1893 and after” to determine the effects of the Report of the Committee of Ten. Because of the complexity of some items, multiple codings were required. A problem-solving item that required proficiency with numbers as well as conceptual understandings of geometry would, therefore, be coded into three categories: problem-solving, geometry, and numbers and operations. In contrast, items may have made incidental use of two content strands such as geometry or measurement but these skill concepts were not instrumental in solving the item; hence, items were only coded according to the skills necessary to complete the problem. For example, an item may have used units of measure but a successful answer did not require manipulation or transformation between groups of units for successfully completing the question. In this case, the item was not coded as measurement. Two raters knowledgeable of the coding scheme coded all items. After coding, overall agreement was .85. The codes were later 165 reviewed by two other raters not involved in the study who examined the codes for accuracy of embodiment of the intent of the process and content strands. Agreement between the two external reviewers was .98 and .91 with the original coders. Because interest lies in the comparison of the proportion of items in any given year corresponding to the codes, arithmetic means were reported to provide insights into educational reform issues of the time. To determine if the report of 1893 had an impact on the items appearing on the teacher certification exams, correlations were calculated between process and content strands and real-world applications and year. Kruskal-Wallis test for independent samples with Bonferroni corrections were conducted between types of application problems (i.e., business or no real-world applications) administered before and after 1893. Year was dummy coded as “0” prior to 1893 or “1” 1893 or later. To determine if the instillation of the first woman superintendent impacted teacher certification, correlations were calculated between years 1900 and 1905 in algebra and pedagogy, and Kruskal-Wallis tests were conducted (a) between the time period 1900 to 1905 and algebra and pedagogy and (b) between the time period prior to and after 1900 and the NCTM process strands. Relationships between pedagogy and real-world application problems and content and process strands were qualitatively analyzed. The relationship between algebra problems and business problems were also qualitatively analyzed. Examples of problems and coding are contained in Table 3. Results Spearman Rho correlations indicated a statistically significant relationship between the year the exam was administered and business (   = .131, p =.002) and commercial problems (   = - .085, p =.043) meaning that as years increased so did business problems but commercial problems decreased. There was a statistically significant relationship between problem solving and reasoning (   = -.602, p < .001) which is indicative of increases in problem solving resulted in a decrease in reasoning items. A statistically significant relationship existed between problem solving and algebra (   = .237, p < .001), geometry (   = -.097, p = .021), and measurement (   = -.096, p = .023). Regression results indicated the year did not depend upon content and process strands but year did depend upon real-world application problems (R 2 = .054, p < .001). Influences of the 1983 Report of the Committee of Ten. A statistically significant relationship existed between business problems prior to and after 1893 (   = -.099; p = .019). A Kruskal- Wallis test for independent samples was conducted between types of application problems (i.e., business or no real-world applications) administered before and after 1893. Statistically significant differences existed at p <.025 (alpha set by Bonferroni correction) between time before 1893 and business ( 2 (1, n = 561) = 5.75, p = .016) and no real world applications problems ( 2 (1, n = 561) = 11.20, p = .001). As shown in Table 4, investigations of arithmetic means by year indicate there was an increase in business applications and a decrease in application problems not related to real-world problems after 1893. Figure 1 illustrates how both the percent of business problems and the percent commercial non-arithmetic business problems steadily increased over time. 166 Table 3 Sample Problems and Codes Code Year Item R and P B but not CA 1901 What will be the expense of plastering a room 18 feet long, 15 feet wide, and 8 feet high, at 30 cents a square yard, allowing 150 square feet for doors, windows, etc.? R and P B and CA 1897 Find the difference between the true discount and the bank discount of $1,000, for 1 year, 3 months, 15 days at 7%. Problem Solving 1902 A, B and C are to share $1,200 in the proportion of 3, 4, and 5, respectively. B dies. How should the whole sum be divided between A and C? Pedagogy 1878 Explain to a child how to add three digit numbers. Pedagogy 1905 Tell how you would develop one of the processes of finding the area of a circle, triangle, parallelogram or trapezoid to a pupil. Pedagogy 1900 In division of decimals how would you lead a child to understand where in the quotient the decimal point should be placed? Illustrate. Pedagogy 1902 Divide 0.75 by 17 5/8 by 4/5 of 0.035, giving the answer the form of a decimal number. Show by discussion and by examples how to teach pupils to place the decimal point correctly in the quotient. Pedagogy 1905 Tell how you would develop one of the processes of finding the area of a circle, triangle, parallelogram or trapezoid Connections and not- business problem 1897 What is the difference between figures and numbers? What is a complex decimal? What is a perfect power? What is the difference between an arithmetical and geometrical progression? What is a scalene triangle? Communication 1898 What is the mental effect of the study of arithmetic? Note. Items prior to 1900 asked prospective teachers to explain concepts, whereas items after 1900 asked prospective teachers to apply reasoning and proof skills and required them to make connections between various mathematical concepts. Codes: R and P= Reasoning and Proof; B= Business problem; CA=Commercial Arithmetic. 167 Table 4: Reported Mean Testing of Items Y ear N u m b er o f E x am s R easo n in g an d P ro o f P ro b lem S o lv in g C o m m u n icatio n C o n n ectio n s R ep resen tatio n s B u sin ess L ife P ro b lem s N o B u sin ess o r L ife C o m m ercial P ed ag o g y 1878 2 .550 .050 .300 .000 .050 .300 .200 .500 .100 .050 1881 1 .600 .300 .000 .100 .100 .300 .200 .500 .300 .000 1883 1 .600 .200 .100 .100 .000 .300 .100 .600 .200 .000 1888 2 .850 .050 .100 .000 .000 .200 .100 .700 .150 .000 1889 1 .800 .200 .000 .000 .000 .100 .000 .900 .000 .000 1890 2 .750 .050 .150 .000 .050 .400 .250 .350 .100 .000 1891 4 .650 .075 .175 .075 .025 .333 .325 .425 .125 .000 1892 4 .675 .125 .075 .125 .000 .425 .250 .433 .125 .000 1895 4 .775 .150 .000 .050 .025 .425 .375 .200 .150 .000 1896 3 .433 .233 .233 .067 .033 .267 .267 .467 .000 .000 1897 3 .700 .100 .067 .100 .033 .400 .167 .433 .167 .000 1898 3 .667 .033 .233 .033 .033 .267 .300 .433 .133 .000 1899 3 .700 .133 .067 .067 .033 .467 .267 .267 .000 .000 1900 3 .833 .167 .050 .000 .033 .500 .200 .300 .167 .067 1901 3 .733 .167 .100 .033 .000 .400 .233 .333 .133 .133 1902 5 .600 .220 .120 .080 .020 .360 .240 .400 .060 .040 1903 1 .600 .200 .200 .000 .000 .300 .400 .300 .100 .100 1904 3 .667 .200 .100 .033 .000 .500 .233 .267 .100 .033 1905 3 .667 .200 .100 .112 .000 .433 .167 .400 .033 .033 1906 2 .700 .150 .100 .050 .000 .500 .250 .250 .100 .000 1908 1 .500 .200 .100 .100 .100 .600 .000 .300 .000 .000 1910* 1 .636 .182 .090 .000 .000 .545 .182 .182 .090 .090 1912 1 .700 .300 .000 .000 .000 .400 .500 .100 .000 .000 Note. Each exam had 10 problems except the 1910 exam; * The 1910 exam had 11 problems 168 Figure 1. The Increase in Business Problems and the Percent of Business Problems Categorized as Not Commercial and Commercial Arithmetic. Algebra problems usually related to business problems (80%) with the majority of these consisting of investment problems. Probability and statistics problems were almost non-existent - only two problems on the entire set of exams. Both of these problems had prospective teachers compute the arithmetic mean. Forty-two percent (n = 237) of the questions related to the content strand numbers and operations. The problems related to numbers and operations were complex by today’s standards and tested knowledge on multiple concepts such as fractions, percentages, and order of operations. A problem from a test administered in 1904 (Grenfell 1904, 115) is given below 4 3 5 2 16 3 8 3 69 3 2 9 2 8 3 1 5 1 2 6 5 1 2 1 22 8 3            of 169 Twenty-three of the 56 exams (41%) contained definition problems. There does not appear to be a relationship between year and the number of definition problems; however, there does appear to be relationship between year and the type of definition problems proposed. Prior to 1893, nine of the 10 exams (90%) contained definitions relating to mathematical terms typical in a mathematics classroom such as define an abstract number or a prime number [1888 test]. After 1893, definitions evolved from classical mathematics terms to terms used in the real- world. On the 13 exams containing definitions after 1893, seven of the exams (54%) contained definitions typical to a traditional mathematics course. Almost half - six of the remaining exams (46%) - asked definitions of terms such as bond, ad valorem, duty [1902 test] or define promissory or times notes [1908 test]. Influence of Changes in Leadership Non-parametric tests (i.e. Kruskal-Wallis) indicated statistically significant relationships between the number of algebra ( 2 (1, n = 561) = 6.75, p = .009) and pedagogy ( 2 (1, n = 561) = 6.10, p = .014) problems before, after, and during the time period from 1900 to 1905. The years from 1900 to 1905 were influenced by superintendent Helen Grenfell and the publication of the Report of the Chicago Section of the American Mathematical Society. Prior to 1900, few pedagogical items appeared. However, after 1900 not only did the number of pedagogical problems increase, but the nature of the pedagogical problems appeared to have evolved from basic understanding to in-depth pedagogical knowledge. Items prior to 1900 asked prospective teachers to explain concepts; whereas items after 1900 had prospective teachers apply reasoning and proof skills and required them to make connections between various mathematical concepts. For example, a problem in 1878 asked a prospective teacher to explain to a child how to add three digit numbers whereas a problem in 1905 asks the prospective teacher to tell how they would develop one of the processes of finding the area of a circle, triangle, parallelogram or trapezoid to a pupil (Grenfell, 1906). No statistically significant relationship existed between the time period before and after 1900 in regards to process strands. Discussion As educators, we must examine the history of education. Investigating our past gives us a glimpse of who we were and helps us determine how to be more effective in the future. Studying the impact of recommendations by major organizations and changes in leadership helps us identify effective methods of instilling changes in our system that will lead to improved teaching and an increase in student achievement. Analysis of Colorado teacher exams indicated educators attempted to follow many of the recommendations set forth by mathematical organizations and educational entities at the turn of the 20th century. The Committee of Ten (1893) was unanimous in implementing a radical change in the teaching of arithmetic. They recommended omitting subjects “which perplex and exhaust the student without affording any really valuable mental discipline, and enriched by a greater number of exercises in simple calculation, and in the solution of concrete problems” (Bidwell and Clason, 1970/2002, 23) and suggested omitting or curtailing topics such as compound proportion, cube root and a large part of commercial arithmetic. The committee recommended that algebra, with options for bookkeeping or commercial arithmetic, be taught in grades ten and eleven (Willis, Schubert, Bullough, Kridel and Holton 1994). 170 Business Problems Business problems were coded as commercial arithmetic or not commercial arithmetic. According to our analysis of the Colorado teaching exams, there was a modest attempt to curtail commercial arithmetic problems. Review of the arithmetic means of commercial arithmetic problems in Table 3 shows a modest decline in the number of commercial problems. As was noted by the committee, there was the consequence of people desiring a system suited to be more practical to commercial and business life, which could explain the only modest decline in commercial arithmetic items and the slight increase in business life items. The committee did recommend that if commercial arithmetic items were to prevail, pupils should master definitions regarding the business terms or else inexperienced students would struggle with questions they could not comprehend. A statically significant relationship existed between business and commercial problems and the year, as illustrated in Figure 1, and to business and real- world application problems before and after the 1893 Report of the Committee of Ten with an increase in the number of business problems. The educational system did take heed of the committees warning regarding students’ understanding of terms with a noticeable increase in the number of definition problems pertaining to business on the teacher certification exams after 1893. Problem Solving and Reasoning and Proof Problem solving is the medium in which children can develop mathematical ideas. Reasoning problems represent the shift from memorization to doing mathematics and is “the logical thinking that helps us solve problems and decide if and why our answers make sense” (Van de Walle 2001, 7). Van de Walle expanded his thoughts on problem solving to afford that “[i]f problem solving is the focus of mathematics, reasoning is the logical thinking that helps us solve problems and decide if and why answers make sense” (8). There was an inverse relationship between problem solving and reasoning and proof indicating that as the number of problem solving questions increased, the number of reasoning problems decreased. Most of the questions focused on reasoning and proof without giving prospective teachers the opportunity to apply their problem solving skills. Despite the lack of problem solving items, each exam did contain items which required prospective teachers to use problem solving skills and there were multiple items requiring reasoning and proof skills. The tests just lacked the opportunity for teachers to simultaneously apply these skills. The problem solving strand was positively correlated to algebra, geometry, and measurement, which is indicative of being able to apply higher order thinking skills. Pedagogy Problems From 1900 until 1905, there was a statistically significant increase in mathematical pedagogy problems and a change in the complexity of these problems. The 1899 election of a former classroom teacher, Helen Grenfell, to the position of State Superintendent of Public Instruction and the 1899 Report of the Chicago Section of the American Mathematical Society appears to have influenced the increase in pedagogical items. While there was a gradual decrease in pedgagogy problems after 1905, educators did not fall back into the pattern of completely eliminating pedagogy problems. Today, NCTM and teacher educations support the need to understand pedagogy and prepare teacher exams that reflect this mode of thought. 171 The 1899 Report of the Chicago Section of the American Mathematical Society advocated the different presentation styles and believed more would be gained by presenting problems in more than one way (Bidwell and Clason 1970/2002). The 1899 Report stated that "The good teacher will not tie himself to any one method, but on occasion, will make use of the good features of every one. . . A method which encourages, or even permits, rote work, or mechanical manipulations is radically wrong” (Bidwell and Clason, 196). The committee supported different presentations with the goal of cultivating independent thinking. The pedagogy problems on the Colorado teacher exams usually related to the communication process strand. The pedagogy problems after 1900 differed from the items prior to 1878 in regards to complexity. Since the exam in 1878 is assumed to be representative of the population, one can concur that most exams contained problems similar to the ones exhibited in our sample. Whereas, the 1878 exam had prospective teachers explain to a child an arithmetic problem; the exams after 1900, evolved into more complex descriptions regarding proving how formulas were derived. While explaining problems is important, showing that prospective teachers understand the underlying concepts is an important focus in current teacher preparation programs. Our results shows that the turn of the 20 th century did begin this transformation in Colorado by changing from mainly assessing prospective teachers’ procedural skills to assessing both procedural skills and conceptual knowledge. Content Strands. Of the content strands, probability and statistics was almost nonexistent and when represented consisted of computing arithmetic means. This is not surprising since the first major introduction of probability and statistics in the western world came in the 1960s as part of the new mathematics curriculum (Truran, 2001). The large percentage of operations and number items was also not surprising because the understanding of numbers and operations is instrumental in understanding other content strands. The lack of problems pertaining to algebra (24%) and the large number relating to business and finance was a surprise since algebra has been heralded as an important subject for everyone to learn (Usiskin, 1995) and is an important topic in today’s curriculum. Conclusion Today’s major goal of producing qualified teachers whose students achieve a high degree of success are similar to those voiced in the late 1800s and early 1900s. The 1893 Report of the Committee of Ten had three recommendations for better trained teachers: a) utilize agencies already in existence to practicing teachers by professional development opportunities and financial support to pursue these endeavors, b) colleges and universities should give stated courses of instruction in elementary and secondary subjects, and c) a mentoring system in which the best teacher in each department gives part of his/her time towards “. . .helping the other teachers by inspecting and criticizing their work, and showing them, both by precept and example, how to do it better” (Bidwell and Clason 1970/2002, 54). The committee recommended that secondary subjects should be taught the same to every pupil no matter what their probable destination or when their education might cease. The goal of secondary schools was not to prepare boys and girls for college but was “to prepare for the duties of life that small proportion of all the children in the country . . . who show themselves able to profit by an education prolonged to the eighteenth year” (Bidwell and Clason 1970/2002, 51). 172 The rigor of the 1904 mathematics problem presented above exemplifies the complexity of items for those certifying to teach and are seemingly more demanding procedurally than those items appearing on present day first through eighth grade teacher certification exams. Although some might continue to argue that if one possesses the ability to solve problems one probably possesses sound process and conceptual knowledge, ; this is not reflected in contemporary practice. Today, preparation of teachers focuses on mathematical processes and conceptual development rather than limiting preparation to algorithmic procedures and processes. However, this study identified differences in the level of rigor on procedural problems. Maybe, as educators we should revisit the mode of thought prior to and around 1900 and create assessments that contain more rigorous and demanding mathematical problems as well as problems emphasizing conceptual knowledge. This study indicates that educational entities in charge of determining who would teach in the classroom adhered to the recommendations by leading mathematical organizations. This study also showed how teacher testing changed in Colorado around the turn of the 20 th century, and aids in dispelling the myth that 19 th and 20 th century teachers lacked sufficient content or pedagogical knowledge. Input from experts, accompanied by changes in leadership within the school system, cause revision in teacher testing, which in turn, had an impact on classroom instruction. In conclusion, this study suggests that reports produced by people who were experts in their field of study were considered valuable to the educational community and had an impact upon teacher certification in the state of Colorado. References Askey, R. (Fall 2005). Mathematics for teaching: Then and Now. Teacher Educator, 23. Retrieved September 28, 2005 from http://www.aft.org/pubs-reports/ american_educator/issues /fall2005/Askey.pdf Askey, R. (Spring 2006a). Mathematics for teaching: Then and now. Teacher Educator, np. Retrieved September 28, 2006 from http://www.aft.org/pubs-reports/ american _educator/issues /fall2005/Askey.pdf Askey, R. (2006b). “Mathematical content knowledge of teachers – a view from 1875,” Wisconsin Teacher of Mathematics, 56 (2), 31-35. Askey, R. (nd). Learning from Assessment. Mimeograph paper obtained from the author. Askey, R. (2007). Learning from Assessment. Chapter 9 in Title: Assessing mathematical proficiency / edited by Alan H. Schoenfeld. Publisher: Cambridge ; New York : Cambridge University Press, 2007. Pages 125-136. Bidwell, J. K., and R. G. Clason, (Eds.). (1970/2002). Readings in the history of mathematics education. Reston, VA: National Council of Teachers of Mathematics. Bradford, Mary C. C. (1921). Twenty-third biennial report of the superintendent of public instruction of the state of Colorado for the official biennium ending November 30, 1920. Denver: Eames Brothers. Bradford, Mary C. C. (1927). Twenty-fifth biennial report of the superintendent of public instruction of the state of Colorado for the official biennium ending November 30, 1926. Denver: Bradford-Robinson Ptg. Co. Burlbaw, Lynn M. (2002, September). Teacher testing – could you pass the test? Paper presented at Twenty-fourth Mid-America Conference on History, Fayetteville, AR. Burlbaw, Lynn M. (2003a, September). In context of the times: Historical, geographic, and political content in elementary certification exams at the turn of the century Colorado teachers’ exams. Paper presented at Twenty- 173 fifth Annual Mid-America Conference on History, Memphis, TN. Burlbaw, Lynn M. (2003b, October). In Context of the times: Pedagogical exams for turn of the century Colorado teachers. Paper presented at Midwest History of Education Society meeting, Chicago, IL. Burlbaw, Lynn M., and Bozeman, Dane. (2003, October). A Scientifically literate profession – the knowledge base of county certified elementary teachers in the state of Colorado, 1880-1912. Paper presented at Midwest History of Education Society meeting, Chicago, IL. Cornell, Leonidas S. (1883). Third Biennial Report of the Superintendent of Public Instruction of the State of Colorado for the two years ending August 31, 1881 and August 31, 1882. Denver: Times Public Printer. Cornell, Leonidas S. (1887). Fifth Biennial Report of the Superintendent of Public Instruction of the State of Colorado for the two years ending August 31, 1885 and August 31, 1886. Denver: The Collier and Cleaveland Lith. Co. Coy, Nathan B. (1893). Eighth Biennial Report of the Superintendent of Public Instruction of the State of Colorado, December, 1892. Denver: Smith Brooks Printing. Craig, Katherine L. (1906a). Report of the State Superintendent of Public Instruction of the State of Colorado for the years 1905-1906. Denver: Smith-Brooks Printing, Co. Craig, Katherine L. (1906b). Third Grade Certificates issued Aug. 16 and 17. Denver, CO: Smith-Brooks Press. Craig, Katherine L. (1908). Report of the State Superintendent of Public Instruction of the State of Colorado for the years 1907-1908. Denver: Smith-Brooks Printing, Co. Grenfell, E. I. (1939). A brief sketch of the life and works of Helen Thatcher Loring Grenfell. Denver, CO: Smith-Brooks Printing Co. Grenfell, Helen L. (1900). Twelfth Biennial Report of the Superintendent of Public Instruction of the State of Colorado to the Governor. Denver: Smith Brooks Printing. Grenfell, Helen L. (1902). Third Grade Certificates issued December 20 and 21, 1901. Denver, CO: Smith-Brooks Press. Grenfell, Helen L. (1904). Fourteenth Biennial Report of the Superintendent of Public Instruction of the State of Colorado to the Governor. Denver: Smith Brooks Printing. Grenfell, Helen L. (1906). Fifteenth Biennial Report of the Superintendent of Public Instruction of the State of Colorado to the Governor. Denver: Smith Brooks Printing. Hallinan, M., and Khmelkov, V. (2001). Recent developments in teacher education in the United States of America. Journal of Education for Teaching, 27, 175-185. Kellogg, E. L. (March, 1889). New York uniform state examination questions. The Teachers Institute and Practical Teacher,11(7), 221- 223. Kellogg, E. L. (June, 1890). Second and third grades. The Teachers Institute and Practical Teacher, 12(10), 320-322. Lothrop, W. C. (1872). First biennial report of the superintendent of public instruction of the territory of Colorado for the school years ending September 30, 1870, and September 30, 1871. Central City, CO: D. C. Collier. Lothrop, W. C. (1878). First Biennial Report of the Superintendent of Public Instruction of the State of Colorado for the two years ending August 31, 1878. Denver: Tribune Steam Printing House. Michigan (1900). Annual report of the Superintendent of Public Instruction of the State of Michigan: with accompanying documents, for the year 1900. Lansing: The Superintendent. National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: NCTM. National Education Association. (1893). Report of the Committee of Ten on Secondary Schools. Washington, DC: U. S Government Printing Office. Peavey, Angenette J. (1896). Tenth Biennial Report of the Superintendent of Public 174 Instruction of the State of Colorado. Denver, CO: Smith-Brooks Printing. Shattuck, Joseph C. (1884). Fourth Biennial Report of the Superintendent of Public Instruction of the State of Colorado for the years ending August 31, 1883, and August 31, 1884. Denver: Times Printing Company. Strayer, George, and Englehardt, Nate. (1920). The classroom teacher at work in American schools. New York: American Book Company. Truran, J. (2001). Postscript: Researching stochastic understanding-The place of a developing research field in PME. Educational Studies in Mathematics, 45, 9-13. Updegraff, H. (1911). Teachers’ certificates issued under general state laws and regulations. United States Bureau of Education, Bulletin 1911, No. 18. Washington: GPO. Usiskin, Z. (1995). Why is algebra important to learn? American Educator, 32, 30-37. Van de Walle, J. A. (2001). Elementary and middle school mathematics: Teaching developmentally (4th ed.). New York: Addison Wesley Longman. Willis, George, William H. Schubert, Robert V. Bullough, Jr., Craig Kridel, and John T. Holton. (1994). The American curriculum: A documentary history. Westport, CT: Praeger. Wixson, Helen M. (1912). Eighteenth Biennial Report of the State Superintendent of Public Instruction for the two years ending 1912. Denver: Smith-Brooks Printing Co.