Microsoft Word - Steiner Final Version.docx
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
Bringing a Learning Strategies Project to Scale in a First-Year Seminar
Hillary H. Steiner, Ph.D., Kennesaw State University,
hsteiner@kennesaw.edu
Nirmal H. Trivedi, Kennesaw State University
Joshua A. Brown, Kennesaw State University
Abstract: This paper describes the impact of a learning strategies intervention
conducted in first-year seminar courses that, 1) disaggregated components of
academic skills into meaningful components for first-year students, 2) taught
students academic skills within an authentic context, and 3) scaled-up the
intervention for implementation at a programmatic level. This work is grounded in
research on metacognition, self-regulation, and motivation, as well as literature on
the academic transition to college. Results reinforced earlier findings indicating
significant improved use of metacognitive learning strategies, even when the
intervention was expanded to include multiple instructors in multiple course
sections. Further research is needed to determine the precise factors precipitating
improvement when the project was brought to scale.
Keywords: first-year seminar, metacognition, self-regulated learning, learning
strategies
Teaching academic skills in a first-year seminar has been an established practice for
several years. In the National Resource Center’s 2012-2013 National Survey of
First-Year Seminars (NSFYS), developing academic skills was reported as the third
most common course objective after developing a connection to an institution and
orienting students to campus resources (Young & Hopp, 2014). A precise
description of what is meant by “academic skills,” however, varies tremendously,
depending on the course characteristics and institution type. Four-year colleges
often include critical thinking skills, motivation, time management, oral
communication, and faculty-student communication as academic skills. Two-year
colleges might include the development of study skills, academic planning,
grammar, and note-taking (Young & Hopp, 2014). Acquiring these skills is
important for college success; however, designing adequate curricula is challenging
for first-year seminar programs given the vast range of learning skills and study
habits of beginning college students (Balduf, 2009).
The present paper presents the results of an implementation of a learning
strategies project in a three-credit hour, letter-graded first-year seminar conducted
first as a pilot study and then scaled-up for implementation in several course
sections, with an eventual goal of implementation in an entire First-Year Seminar
Program. The Strategy Project assignment (see Appendix) is unique in that it
disaggregates “academic skills” into component parts that enable the first-year
student to understand the differences and relationships between time management,
motivation, study skills, and metacognition. Furthermore, the Strategy Project is
context-specific: to complete the project, students plan, study, and reflect on their
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 28
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
planning for and execution of an exam taken in another course, thus providing
learners with an authentic context for developing specific academic skills. The
research study described herein was guided by the question: Does the Strategy
Project impact first-year students’ self-regulation and metacognition strategies as
measured by the Motivated Strategies for Learning Questionnaire (MSLQ) and self-
reported strategy use when brought to scale for a large first-year seminar program?
Literature Review
Research on self-regulated learning has a long history in the educational psychology
literature, originating from work by Zimmerman, Schunk, Pressley, and others on
goal setting, motivation, and strategy use (Zimmerman, 2008). As Zimmerman
(2008) notes, a self-regulated learner is one who is motivated, reflective, and
proactive in his use of learning strategies. For college students, self-regulation of
learning strategies involves taking the initiative to learn and implement appropriate
strategies for the task, set goals for learning, and reflect on one’s approach,
modifying it for the next task if necessary. Metacognition, most commonly defined
as the knowledge and regulation of one’s own cognition, is closely related to self-
regulation in that a metacognitive student is aware of what she does and does not
know, and plans for, monitors, and reflects on the acquisition of that knowledge.
Students exhibit good metacognitive behaviors when they choose and use
appropriate learning strategies for the task at hand and reflect on the results of
their efforts.
Self-regulating and metacognitive behaviors are proven predictors of academic
success (Pintrich & DeGroot, 1990). However, many first-year students have not
yet developed metacognitive and self-regulated learning strategies that are
sufficient for success in their new academic environment. In college, learning
strategies that go beyond memorization and passive knowledge acquisition are
essential, as students who use higher-level metacognitive strategies tend to be
more successful in their courses (Dunlosky, Rawson, Marsh, Nathan, & Willingham,
2013; Kitsantas, 2002). For example, students who are taught to monitor their
comprehension and evaluate their learning as they read show deeper processing of
the material and better retention than students who passively read the text (Nash-
Ditzel, 2010). In addition, first-year students who successfully manage their time
and seek out existing resources show greater achievement in the first year
(Thibodeaux, Deutsch, Kitsantis, & Winsler, 2017; Tuckman, 2003).
Many students enter college believing that they are prepared for this challenge due
to their success in high school (Balduf, 2009). However, because learning tasks at
the college level require different skills and greater personal responsibility for
managing those skills, many high school students begin to show deficits in these
abilities once they reach the college level (Cohen, 2012). This is not limited to poor
performing students. While some students may have better self-regulation
strategies than their peers in middle and high school (e.g., Zimmerman & Martinez-
Pons, 1990) those same students may reach the ceiling of their strategic ability in
college where deeper processing is required (Balduf, 2009). This suggests the
teaching of self-regulating and metacognitive learning strategies is important for
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 29
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
college students of all achievement levels. Furthermore, when strategies are
embedded in or taught alongside a content area, students are better able to
transfer the strategies to future tasks and reflect on the outcome of that transfer
(Tuckman, 2003).
Ample evidence shows that self-regulated learning strategies, as primary
contributors to academic success (Pintrich & DeGroot, 1990), can and should be
taught (Cohen, 2012; Paris & Paris, 2001; Weinstein, Tomberlin, Julie, & Kim,
2004; Tuckman, 2003), yet faculty attempting to teach them face many challenges.
For example, many students are reluctant to change the learning strategies with
which they were successful in high school (Dembo & Seli, 2004; Dunlosky et al.,
2013). Many students also are unable to judge whether or not they are actually
learning—a key aspect of metacognition—which can lead to overconfidence (Koriat
& Bjork, 2005). The intervention presented in this paper encourages students to
practice confronting their existing habits, monitor their knowledge acquisition, and
reflect on outcomes of learning. Motivation theory suggests that students are more
likely to engage in activities for which there is a high degree of personal value and
little cost (Barron & Hulleman, 2015). The Strategy Project assignment attempts to
maximize value and minimize cost by engaging students in activities that will help
them achieve a higher grade in a class in which they are currently enrolled.
Method and Findings
Both studies described in this paper seek to answer the question: Are first-year
seminar students more likely to show evidence of metacognition and self-regulation
after completing the Strategy Project assignment? In this section, we will describe
the methods and findings for a pilot study, in which one member of the research
team measured the effects of the Strategy Project on her own students, and for the
current study, in which this intervention was brought to scale for a group of trained
instructors who used the Strategy Project in their first-year seminar courses and
were compared to a matched group of instructors who did not use the Strategy
Project. Both Institutional Review Board-approved studies were conducted with
students at a large, public university in suburban Atlanta, Georgia, where a first-
year seminar is required for all first time, full-time students in their first semester
of college.
Strategy Project Assignment
The Strategy Project assignment (see Appendix) is a multistep project requiring
students to plan, monitor, and evaluate their newly learned strategies as they
prepare for a test in a course in which they are currently enrolled. Each step in the
process is meant to disaggregate “learning skills” into meaningful components that
students are empowered to integrate for their individual purpose. In this
assignment, students are directed to complete the following activities related to an
exam in another course that they have chosen for the project. Students must: 1)
create a plan for their study time; 2) interact one-on-one with the instructor of
their chosen course; 3) use active reading and/or active note-taking strategies as
they learn new material; 4) implement additional active learning strategies that are
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 30
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
appropriate for the content area, such as self-quizzing or concept mapping; and 5)
reflect on their exam performance in a final paper.
Students receive instruction on the use of self-regulated learning strategies as well
as research-based evidence on why the strategies work. This lesson is extended to
a discussion on metacognition and the benefits of deep cognitive processing. After
instruction in the various strategies, students identify a test in a course for which
they have ample time to prepare and reflect. Typically, this is the second or third
test in a semester-long course. When given the option to choose their own course
for the project, choices can vary widely, from Psychology to Math to Chemistry;
however, most students choose a course in which they feel less confident and
prepared. Students meet briefly with the first-year seminar instructor to review the
requirements and determine if any substitutions should be made.
Next, the student meets with the instructor of the course they have chosen for the
project. This is a significant piece of the Strategy Project, as it allows students not
only to gain knowledge about how to succeed in the course but also to build a
relationship with a faculty member, an important part of the transition to college
(Goodman, Baxter Magolda, Seifert, & King, 2011). Requiring this interaction as
part of the Strategy Project may help ease the initial discomfort that many new
college students feel in speaking with their professors.
Students then plan their approach to the Strategy Project by mapping out a time-
and task-specific “plan of study.” As an essential component of self-regulation for
college students (Kitsantas, Winsler, & Huie, 2008; Tuckman, 2003), time
management is another important piece of the assignment. With increased task
demands and more responsibility for allocating their time, college students must be
taught strategies for managing their study time.
Specific learning strategies are the next components of the Strategy Project. The
active reading portion allows for practice of textbook annotation, an active strategy
that promotes deep processing and engagement with the material (Simpson & Nist,
1990). In the notetaking portion, students are instructed to take notes for the
duration of the exam-preparation time using the Cornell method of note-taking
(Pauk, 1962) or a similar active note-taking method. Finally, students are directed
to choose two additional strategies from among those discussed that are
appropriate for test preparation in their course. Evidence of good quality textbook
annotations, class notes, and study strategies are presented as a portion of the
final Strategy Project.
Finally, students reflect on the results of their Strategy Project through a written
assignment and oral presentation. Both of these final products empower students to
analyze critically the efficacy of each portion of the project. By sharing these
reflections with classmates, students share peer-driven models of self-regulation
and co-develop discipline-specific strategies that may help their fellow students
succeed in other courses.
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 31
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
Pilot Study
To determine whether the MSLQ (Pintrich, Smith, Garcia, & McKeachie, 1991),
described below, revealed any changes in metacognition and self-regulation after
completion of the Strategy Project assignment, a pilot study was conducted with
fifty-two students recruited from two sections of the first author’s first-year
seminar. Just under 54% percent of the students identified as female and about
42% identified as male. When asked to choose the best description of their
ethnicity, 65% of students indicated White (Caucasian), 12% African-American,
12% Hispanic, 4% Asian, and 2% other. The students’ median age was 18 years.
The study did not include data from students younger than 18.
Pilot Study Procedures. An online pre- and post-test version of the MSLQ
(Pintrich et al., 1991) was distributed via the learning management system during
Weeks 2 (prior to the introduction of the assignment) and 14 (well after the
assignment had been completed) of a 16-week semester. Completion of each
survey counted for 1% of the course grade, respectively. The study was explained
to students before obtaining their consent, and during each survey administration
an alternative assignment was offered to students who were under 18 or who did
not wish to participate in the study.
Measurement Instrument. The MSLQ has been widely used for decades
to investigate cognitive and metacognitive strategies, motivation, and self-
regulation. In both studies described in this paper, the full 81-item version of the
MSLQ, containing fifteen subscales, was used. The subscales included: Intrinsic
Goal Orientation, Extrinsic Goal Orientation, Task Value, Control of Learning Beliefs,
Self-Efficacy for Learning Performance, Test Anxiety, Rehearsal, Elaboration,
Organization, Critical Thinking, Metacognitive Self-Regulation, Time and Study
Environment Management, Effort Regulation, Peer Learning, and Help Seeking.
Participants were instructed to answer the questions based on the course they
chose for their Strategy Project. The survey also included a section where
participants were asked to indicate which strategies they currently used to study for
the chosen course. Participants indicated that they completed one or more of the
following strategies during their study time for the chosen course: read the
textbook; took notes on the textbook; took notes in class; read over class notes;
made flashcards; created a chart, diagram, or concept map; studied with a group;
completed practice problems; or engaged in self-quizzing. In addition, the survey
included questions to gather demographic data, including age, gender, and
ethnicity.
Pilot Study Results. Following recommendations from Jamieson (2004)
and others who suggest Likert scale data be treated as ordinal-level data, non-
parametric statistics were used in the analysis. A Wilcoxon Signed Rank test
revealed a statistically significant pre- and post-Strategy Project difference on the
total scores of the MSLQ subscale pertaining to Metacognitive Self-Regulated
Learning, z = -2.757, p < .01, with a fairly large effect size (r = .40). The median
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 32
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
score on the subscale increased from pre-project (Md = 17.5) to post-project (Md =
18.0). None of the other subscales showed significant differences.
A series of related-samples McNemar’s Tests revealed statistically significant
increases for the use of several self-reported strategies (see Table 1), including
active reading, using flashcards, creating charts/diagrams/concept maps, group
studying, and self-quizzing. No significant differences were found for textbook
reading (passive reading), re-reading class notes (also a passive strategy),
completing practice problems, or taking notes in class.
Table 1
Results of McNemar’s Tests on Study 1 Participants’ (n=52) Use of Study Strategies
over Time
Pre-Strategy Project Post-Strategy Project
Strategies Used n % of group n % of
group
Read textbook 23 44% 26 50%
Took notes on textbook* 10 19% 17 33%
Read over class notes 39 75% 36 69%
Made flashcards*** 2 15% 29 56%
Created chart, diagram or concept
map**
2 4% 11 21%
Group study 12 23% 20 39%
Completed practice problems 23 44% 28 60%
Took notes in class 42 81% 45 87%
Engaged in self-quizzing** 16 31% 30 64%
* pre- to post-project change is significant at p<.05
** pre- to post-project change is significant at p<.01
*** pre- to post-project change is significant at p<.001
Note: Students in this group significantly increased their use of textbook
annotations, flashcards, charts/diagrams/concept maps, and self-quizzing. There
were no significant decreases in strategy use.
Current Study
The current study sought to investigate whether the Strategy Project could be
scaled up by providing targeted faculty development to a select group of instructors
who used the Strategy Project in their first-year seminar courses. These instructors
were matched with a group of instructors who had not previously used the Strategy
Project in their courses.
Participants. Five hundred eight students in thirty-three sections of a first-
year seminar participated in the study. Close to fifty-seven percent of the students
identified as female and 43% as male. When asked to choose the best description
of their ethnicity, 57% of students indicated White (Caucasian), 23% African-
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 33
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
American, 8% Hispanic, 6% Asian, and 6% other. The students’ median age was 18
years. The study did not include data from students younger than 18.
Procedures. Instructors teaching first-year seminar were recruited to
participate in the study via emails describing the Strategy Project and associated
faculty-training workshop. The ten instructors who responded and agreed to attend
the workshop were assigned to the treatment group and were matched on three
variables—years of experience, number of course sections taught, and educational
background—to a comparison group of instructors who also participated in the
study. Both groups included full- and part-time instructors from a variety of
disciplinary backgrounds.
Instructor training. Instructors in the treatment group attended a four-
hour workshop conducted by the researchers on the use of the Strategy Project.
During this workshop, instructors received an overview and background information
about the project, discussed instructional content related to the project including a
timeline and assessment practices, and engaged in individual course planning. After
the workshop, the research team continued to reach out to treatment group
instructors for individual consultations and held multiple drop-in sessions for
instructors to ask questions about the project. Instructors in the comparison group
were told that their students would be taking surveys about their use of learning
strategies but were not given any special direction in curriculum planning.
Strategy Project intervention. The Strategy Project, described above,
was assigned in each of the sections taught by instructors in the treatment group.
Twenty-five percent of the semester grade was allotted to the project in each
section. Instructors used four class periods early in the semester for Strategy
Project-related instruction, following the protocol given during the training
workshop.
Survey implementation. An online version of the MSLQ was distributed
via the learning management system as a pre-test to all students in both groups
during Weeks 2-3 of the semester. A member of the research team made a brief
visit to each course section in both groups to encourage students to complete the
survey. As in the pilot study, completion of the survey counted for 1% of the course
grade. For grading purposes, a member of the research team sent frequent updates
to instructors noting which of their students had completed the survey. In Week 14
of the semester, a post-test version of the MSLQ was given to all students in both
groups. This survey also contributed to 1% of the course grade. During each survey
administration, an alternative assignment was offered to students who were under
18 or who did not wish to participate in the survey.
Instrument. As in the pilot study, the full 81-item version of the MSLQ,
containing fifteen subscales (see description above), was used. Participants were
advised to answer the questions based on the course they chose for their Strategy
Project (treatment group) or to answer the questions based on a course they were
currently taking (comparison group). As in the pilot study, treatment and
comparison group surveys also included a section where participants were asked to
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 34
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
indicate which strategies they currently used to study for the chosen course (see
Table 3 for a list of strategies). In addition, the survey included questions to gather
demographic data, including age, gender, and ethnicity.
Current Study Results. To compare changes on the MSLQ pre- to post-project
between the two groups, change variables were computed for each subscale by
subtracting pre-test subscale scores from post-test subscale scores. As with the
pilot study, a conservative approach was taken with the Likert scale data in the use
of non-parametric statistics, and estimated effect size is labeled as “r” and
interpreted using Cohen’s (1988) criteria.
A series of Mann-Whitney U Tests revealed significantly greater change for the
treatment group as compared to the comparison group on the total scores of four
MSLQ subscales: Task Value, Metacognition and Self-Regulated Learning, Time and
Study Environment Management, and Peer Learning (see Table 2). Effect sizes were
small, ranging from r = .10 to r = .16.
Table 2
Results of Mann-Whitney U Tests Comparing Treatment and Comparison Groups on
Each MSLQ Subscale
MSLQ Subscales n
(T)
n
(C)
Md
(T)
Md
(C)
U z p r
Internal Goal
Orientation
252 251 .333 .333 31052.5 -3.53 .724 ns
External Goal
Orientation
252 252 -.250 -.250 30480 -.781 .435 ns
Task Value 252 252 -.250 -.167 28178.5 -2.189 *.029 .10
Control of
Learning
Beliefs
252 252 .000 .000 30671.5 -.663 .507 ns
Self-Efficacy 252 252 -.125 -.125 28556.5 -1.956 .050 ns
Test Anxiety 252 252 -.200 -.200 31246 -.310 .757 ns
Rehearsal 251 248 .250 .000 28186 -1.830 .067 ns
Elaboration 251 248 .167 .000 29635 -9.26 .354 ns
Organization 251 249 .000 .000 30292.5 -.594 .553 ns
Critical Thinking 251 248 .000 .200 29266.5 -1.155 .248 ns
Metacognition and
Self-Regulated
Learning
251 249 .083 -.083 27311 -2.44 *.015 .11
Time and Study
Environment
Management
251 249 .000 -.250 25496.5 -3.565 **.000 .16
Effort Regulation 251 248 -.250 -.250 29160.5 -1.223 .221 ns
Peer Learning 251 248 .333 .000 27696.5 -2.136 *.033 .10
Help Seeking 255 251 .000 .000 31693.5 -.189 .850 ns
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 35
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
* significant at p<.05
** significant at p<.001
Note: (T) indicates treatment group; (C) indicates comparison group. Estimated
effect sizes were listed for those subscales with significant results; when non-
significant, “ns” was noted.
Finally, a series of McNemar’s tests examining the pre- to post-change of self-
reported strategies between the treatment and comparison groups revealed several
significant differences between the two groups. The comparison group significantly
increased their use of flashcards and group studying, while decreasing textbook
reading and note-taking. In contrast, the treatment group significantly increased
their use of most strategies listed, including note-taking, flashcards, creating
charts/diagrams/concept maps, group studying, completing practice problems, and
self-quizzing. Tables 3 and 4 provide details about these strategy changes.
Table 3
Results of McNemar’s Tests on the Treatment Group’s (n=255) Use of Study
Strategies over Time
Pre-Strategy Project Post-Strategy Project
Strategies Used n % of group n % of group
Read textbook 121 47% 131 51%
Took notes on textbook 91 36% 86 34%
Took notes in class** 220 86% 241 95%
Read over class notes 180 71% 189 74%
Made flashcards*** 79 31% 160 63%
Created chart, diagram or concept
map***
9 4% 35 14%
Group study*** 60 24% 121 47%
Completed practice problems*** 114 45% 154 60%
Engaged in self-quizzing*** 106 42% 166 65%
** pre- to post-project change is significant at p<.01
*** pre- to post-project changes is significant at p<.001
Note: Students in the treatment group significantly increased their use of class
notes, flashcards, charts/diagrams/concept maps, group study, practice problems,
and self-quizzing. There were no significant decreases in strategy use.
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 36
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
Table 4
Results of McNemar’s Tests on the Comparison Group’s (n=253) Use of Study
Strategies over Time
Pre-Strategy Project Post-Strategy Project
Strategies Used n % of group n % of group
Read textbook* 159 63% 136 53%
Took notes on textbook 101 40% 95 38%
Took notes in class* 230 91% 213 84%
Read over class notes 197 78% 185 73%
Made flashcards* 39 15% 56 22%
Created chart, diagram or concept map 19 8% 30 12%
Group study** 49 19% 71 28%
Completed practice problems 136 54% 117 46%
Engaged in self-quizzing 98 39% 106 42%
*pre- to post-project change is significant at p<.05
**pre- to post-project change is significant at p<.01
Note: Students in the comparison group significantly increased their use of
flashcards and group study. They significantly decreased their use of the textbook
and class notes.
Discussion
Both studies discussed in this paper were guided by the question: Does the
Strategy Project impact first-year students’ self-regulation and metacognition
strategies as measured by the MSLQ and self-reported strategy use when brought
to scale for a large first-year seminar program?
Findings from the pilot study and current study indicate that the Strategy Project
does positively affect students’ metacognition and self-regulation, but the effect as
measured by the MSLQ varies. In the current study, students who experienced the
Strategy Project assignment had a statistically significant increase in their scores on
four subscales, all of which reflect various aspects of metacognition and self-
regulation, when compared with their peers: Task Value, which reflects “beliefs
about the importance, utility, and relevance of the task” (Pintrich, 2004, p. 365);
Metacognitive and Self-Regulated Learning; Management of Time and Study
Environment; and Peer Learning. These subscales were closely aligned with our
goals for the project. Of particular interest was the “Management of Time and Study
Environment” subscale, which “is a measure of how well you manage your time and
schedule, and your use of a place to study” (Pintrich et al., 1991), and reflects a
major challenge for first-year students (Tuckman, 2003). Notably, median scores
on this subscale actually decreased for the comparison group during the semester
but increased significantly for students using the Strategy Project. The change in
Peer Learning subscale scores was also significantly different between groups. This
subscale measures the degree to which students report collaborating with peers to
increase learning. Instructors using the Strategy Project encouraged this
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 37
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
collaboration by making explicit the advantages of group studying, and by including
group study as a choice in the metacognitive strategies portion of the assignment.
More impressive, perhaps, is that students in the treatment group significantly
increased their use of metacognitive strategies like creating concept maps,
completing practice problems, and self-quizzing, while the comparison group did
not. Recent research indicates that self-quizzing, in particular, is a highly effective
metacognitive strategy (Dunlosky et al., 2013; Rodriguez et al., 2018). One
explanation for this success lies in the fact that the Strategy Project explicitly
requires students to practice, then observe the effect of, metacognitive and self-
regulation strategies. What is notable is that the post-test measure was taken
during Week 14 (out of 16) of the semester, several weeks after the Strategy
Project itself had taken place, suggesting that students continued to use
metacognitive strategies in the course chosen for the Strategy Project when they
were no longer formally required and encouraged by their instructors. This finding
confirmed the researchers’ intent to empower individual students to become
strategic and self-directing in their use of specific “learning skills.” In other words, it
is likely that students began to themselves disaggregate the utility of various
strategies depending on the task at hand.
Limitations
Like most studies involving the scholarship of teaching and learning, this study was
quasi-experimental due to the impossibility of randomly placing students into the
various sections of first-year seminar. As in most universities, students self-
selected their own course section. However, prior to registration, most sections
were not yet staffed with instructors, and students were not aware which
instructors were participating in the study, nor were they aware to which group
participating instructors belonged. In order to reduce instructor differences as a
confounding variable, the research team was very intentional about matching
instructors for comparison. However, it is possible that the instructors in the two
groups differed in some way that acted as a confounding variable.
It is also possible that the two groups of students differed at the outset of the
study. To investigate this possibility, demographic data for the treatment and
comparison group were compared. A t-test revealed no significant difference in age,
and chi-square analyses revealed no significant difference in gender or ethnicity
between the two groups. Additional comparisons between the comparison and
treatment groups were made on the pre-MSLQ subscale scores and self-reported
strategies to establish that the two groups were equivalent at the beginning of the
study in their approach to metacognition and self-regulated learning. A series of t-
tests did reveal significant differences on some of the pre-test subscales, including
task value, metacognition and self-regulated learning, and management of time
and study environment. A series of chi-square analyses revealed significant
differences on the use of textbook reading, flashcards, and practice problems. In all
of these cases except for the use of flashcards, the comparison group scored higher
than the treatment group on their pre-tests, which meant that the treatment group
often had more room for improvement. The research team could find no plausible
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 38
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
explanation for this initial difference between the groups. Pre-test group differences
such as these are not uncommon in educational research (McGrath, 2016), but it is
a potential moderator for how the key findings of the study are interpreted.
Conclusions and Future Directions
Essential skills like metacognition, time management, and self-regulation are critical
for first-year student success and are often within the purview of a first-year
seminar (Young & Hopp, 2014). First-year students, in particular, need appropriate
guidance from faculty about the component parts that constitute complex concepts
like motivation and “academic skills” so as not to reinforce faulty prior knowledge
about how such academic skills are developed. In a large first-year seminar
program, it can be a struggle to maintain consistency and quality of instruction,
especially for programs that rely on instructors with varying disciplinary
backgrounds and few years of teaching experience. Therefore, there is a need for
high quality, evidence-based practices that can be brought to scale by providing
appropriate faculty development and support.
Based on the evidence described in this paper, the Strategy Project assignment
may be a viable option for first-year programs seeking to scale up students’ use of
time management, learning strategies, “study skills,” or “academic skills” in the
authentic environment of another course in which they are enrolled. Future studies
should establish the replicability of the Strategy Project assignment in other
contexts; one such study is currently being conducted at another institution
(Steiner, Sturges, & Langdon, 2018). In addition, longitudinal studies of students’
strategy use would reveal whether the changes brought about by the Strategy
Project have lasting impact beyond the first semester.
Dunlosky and colleagues (2013) have suggested that students continue to use
ineffective study techniques because they don’t know—and haven’t been taught—
which techniques are most effective, perhaps in part because instructors
themselves don’t know. The project discussed in this paper could serve as a
springboard for both students’ and instructors’ learning about learning. In a post-
project feedback survey, instructors in the treatment group touted the benefits of
the project, indicating that the project made “more of a difference to my students
than any other assignment I've ever taught.” Another appreciated that it was a
“great tool for teaching these necessary study habits and strategies, all rolled into
one project.” This anonymous instructor also said that it sparked his/her own
interest in metacognition and self-regulation: “Study strategies are not my strong
suit so I didn't feel highly equipped, but did the best I could and learned a lot. [I]
may want to become an expert in the future!” When complex assignments like the
Strategy Project are paired with faculty development, both students and faculty
may benefit. The studies described in this paper suggest that these important
benefits can be brought to scale for even the largest first-year seminar programs in
order to maximize student success.
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 39
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
References
Balduf, M. (2009). Underachievement among college students. Journal of Advanced
Academics, 20(2), 274-294.
Barron, K. E., & Hulleman, C. S. (2015). Expectancy-Value-Cost model of
motivation. In J.D. Wright (Ed.), International encyclopedia of the social &
behavioral sciences, 2nd edition (Vol. 8, pp. 503-509). Oxford: Elsevier.
Cohen, J.W. (1988). Statistical power analyses for the behavioral sciences (2nd ed).
Hillsdale, NJ: Lawrence Erlbaum.
Cohen, M. T. (2012). The importance of self-regulation for college student learning.
College Student Journal, 46(4), 892-903.
Dembo, M. H., & Seli, H. P. (2004). Students' resistance to change in learning
strategies courses. Journal of Developmental Education, 27(3), 2-11.
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T.
(2013). Improving students’ learning with effective learning
techniques: Promising directions from cognitive and educational
psychology. Psychological Science in the Public Interest, 14, 4-58.
Everett, M. C. (2013). Reflective journal writing and the first-year experience.
International Journal of Teaching and Learning in Higher Education, 25(2),
213-222.
Goodman, K. M., Baxter Magolda, M., Seifert, T. A., & King, P. M. (2011) Good
practices for student learning: Mixed-method evidence from the Wabash
National Study. About Campus, 16(1), 2-9.
Jamieson, S. (2004). Likert scales: How to (ab)use them. Medical Education, 38,
1212-1218.
Kitsantas, A. (2002). Test preparation and test performance: A self-regulatory
analysis. Journal of Experimental Education, 41, 231–240.
Kitsantas, A., Winsler, A., & Huie, F. (2008). Self-regulation and ability predictors of
academic performance during college: A predictive validity study. Journal of
Advanced Academics, 20(1), 42–68.
Koriat, A., & Bjork, R. A. (2005). Illusions of competence in monitoring one's
knowledge during study. Journal of Experimental Psychology: Learning,
Memory, and Cognition, 31(2), 187-194.
McGrath, A. (2016). Searching for significance in the scholarship of teaching and
learning and finding none: Understanding non-significant results. Teaching &
Learning Inquiry, 4(2).
Nash-Ditzel, S. (2010). Metacognitive reading strategies can improve self-
regulation. Journal of College Reading and Learning, 40(2), 45-63.
Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self-
regulated learning. Educational Psychologist, 36, 89–101.
Pauk, W. (1962). How to study in college. Boston, MA: Houghton Mifflin.
Penuel, W. R., & Fishman, B. J. (2012). Large-scale science education intervention
research we can use. Journal of Research in Science Teaching, 49(3), 281–
304.
Pintrich, P. R., & DeGroot, E. V., (1990). Motivational and self-regulated learning
components of classroom academic performance. Journal of Educational
Psychology, 82(1), 33-40.
Pintrich, P. R., Smith, D., Garcia, T., & McKeachie, W. J. (1991). A manual for the
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 40
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
use of the Motivated Strategies for Learning Questionnaire (Technical Report
91-B-004). The Regents of the University of Michigan.
Pintrich, P. R. (2004). A Conceptual Framework for Assessing Motivation and Self-
Regulated Learning in College Students. Educational Psychology Review,
16(4), 385–407.
Rodriguez, F., Kataoka, S., Janet Rivas, M., Kadandale, P., Nili, A., & Warschauer,
M. (2018). Do spacing and self-testing predict learning outcomes? Active
Learning in Higher Education, 385-407.
Simpson, M. L., & Nist, S. L. (1990). Textbook annotation: An effective and efficient
study strategy for college students. Journal of Reading 34, 122-129.
Steiner, H.H. (2016). The strategy project: Promoting self-regulated learning
through an authentic assignment. International Journal of Teaching and
Learning in Higher Education, 28 (2), 271-282.
Steiner, H.H., Sturges, D., & Langdon, J. (2018, October). Promoting self-regulated
learning in the first-year seminar: Evidence and future directions. Workshop
presented at the annual meeting of the International Society for the
Scholarship of Teaching and Learning. Bergen, Norway.
Thibodeaux, J., Deutsch, A., Kitsantas, A., & Winsler, A. (2017). First-Year College
Students' Time Use: Relations With Self-Regulation and GPA. Journal Of
Advanced Academics, 28(1), 5-27.
Tuckman, B. W. (2003). The effect of learning and motivation strategies training on
college students' achievement. Journal of College Student Development,
44(3), 430-437.
Weinstein, C. E., Tomberlin, T. L., Julie, A. L., & Kim, J. (2004). Helping students to
become strategic learners: The roles of assessment, teachers, instruction,
and students. In J. Ee, A. Chang, & O. Tan (Eds.), Thinking about thinking:
What educators need to know (pp. 282–310). Singapore, China: McGraw-Hill.
Young, D. G., & Hopp, J. M. (2014). 2012-2013 National Survey of First-Year
Seminars: Exploring High-Impact Practices in the First College Year.
Research Reports on College Transitions No. 4. National Resource Center for
The First-Year Experience and Students in Transition. Columbia, SC: National
Resource Center
Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical
background, methodological developments, and future prospects. American
Educational Research Journal, 45, 166–183.
Zimmerman, B. J., & Martinez-Pons, M. (1990). Student differences in self-
regulated learning: Relating grade, sex, and giftedness to self-efficacy and
strategy use. Journal of Educational Psychology, 82, 51-59.
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 41
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
Appendix
Strategy Project
This assignment will help you see the connections between what you are learning in your first-year seminar
and your other courses. In this project, you will apply the strategies and hints that we have learned in
seminar to preparing for a test in another class. The test you select should be between Sept. 30 and Oct. 30,
and should be a test that requires you to study. To complete the strategy project, you will complete and
submit a variety of test preparation strategies along with written reflections about your experience. You will
submit these materials as noted below. All papers should be double-spaced, with 12-point font.
Strategy Project Timeline and Activities:
Contract (5 points). List the details of your chosen test, choose the strategies you plan to use, and mark the
necessary due dates. We will meet briefly in class to ensure your chosen test and strategies are appropriate.
Please bring the printed and signed contract to class on [date]. If you need to make changes to your contract
after this meeting, please notify me. No changes may be made after Week 5 of the semester.
_________________________________________________________________________________________
Pre-Project Survey (5 points). This portion of the project will require you to take the MSLQ survey (found at
this link) or complete an alternative assignment in accordance with IRB policy (see your instructor for details).
If you are younger than 18, please ask your instructor for the alternative assignment. Your instructor will only
grade the survey for completion, not for how you answered the questions.
Pre-Project Reflection (25 points). Please reflect on your current habits in a 2-3 page essay. When writing
your essay, please consider the following questions:
● What do you feel are your strengths when it comes to college success?
● What do you feel are your weaknesses when it comes to college success?
● What are you most worried about when it comes to college studying and learning?
● What are your favorite study strategies, and why? Where did you learn these strategies?
● Is there anything you would like to improve this semester that could lead you to be a
successful college student?
Submit the essay via the Assignments Tool on D2L. The survey and reflection must be completed by [date].
Professor Interaction (15 points). Attend office hours or a drop-in session with the instructor of the course
you have chosen for the project and write a one- or two-paragraph description of the questions you asked,
what you learned, and why attending office hours might be helpful. A list of suggested questions is below,
but please feel free to add your own:
● What is the format of the upcoming test? What do you think are the best ways to study
for this test?
● What do you think differentiates the most successful students in your class from the
least successful students?
● What do you like most about being a college instructor?
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 42
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
● Do you conduct research? If so, what is it about?
Submit this via the D2L Assignments Tool by the due date indicated on your contract.
Plan of Study (15 points). Using the suggestions from your professor interaction as well as what you’ve
learned in KSU 1101, please outline your plan for studying for this test. You should create a detailed study
schedule that describes what you will do and when you will do it, demonstrating your plan to spread out your
studying over at least two weeks, using a variety of active learning strategies. This outline should be created
and submitted via the D2L Assignments Tool by the due date indicated on your contract.
_________________________________________________________________________________________
Metacognitive Study Strategies (60 points). These study strategies should be submitted in one folder or
binder in class on the date indicated on your contract. Please make sure everything is clearly labeled, and
that your signed contract is in the front of the folder. You must choose at least four of these strategies, one
of which must be active notetaking or active reading. Please choose your strategies carefully, taking into
consideration the course, the professor, and the type of test.
Choose at least one:
● Active Notetaking. Use the Cornell or a modified Cornell method to take notes in
your chosen class for at least two weeks during the period leading up to your test,
making sure to set aside space for self-quizzing and daily summaries. Submit these
notes (or a copy of these notes) in your folder.
● Active Reading. Preview and annotate all textbook chapters associated with this
test, paying special attention to the visual aids, vocabulary, and concepts.
Summarize these annotations by distilling all important information into a one-page
study sheet. Copy a 4-page sample of your annotations and include it and your one-
page study sheet in your folder.
Choose three (or two, if you chose both strategies above):
• Complete Practice Problems from an outside source. This is a good strategy for
math, physics, chemistry, and computer science courses.
• Quiz Yourself by creating your own test.
• Create Flashcards. This flashcards must include explanations of the topic in your
own words as well as examples. You must create at least fifteen flashcards.
• Create a Concept Map.
• Meet with a study group and describe your experience in 1-2 paragraphs.
• Attend a tutoring session and describe your experience in 1-2 paragraphs.
• Other strategies? Propose a strategy that is appropriate to your course in
consultation with your instructor.
_________________________________________________________________________________________
Post-Project Survey (5 points). Complete the MSLQ survey (found at this link) or complete an alternative
assignment in accordance with IRB policy (see your instructor for details). Again, your instructor will only
grade the survey for completion, not for how you answered the questions.
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 43
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
Post-Project Reflection (20 points). After you take your test but before you receive it back, please
estimate the grade you think you received. Once you receive your feedback, please write a reflection
paper 2-3 pages in length about your experiences with the strategy project. In particular, please reflect
on which elements of the project you think helped and which did not. Please also reflect on the grade
you received, whether it was an improvement over your last grade (list both grades), and whether it
matched what you thought you’d receive. Also address whether or not your study strategies have
changed since you’ve been in college. Submit your reflection paper via the D2L Assignments Tool.
The survey and reflection must be completed by the date indicated on your contract.
_____________________________________________________________________________________
Strategy Project Presentation--IF REQUIRED BY YOUR INSTRUCTOR (15 points). Please plan to share
your Strategy Project with fellow students in an informal presentation of about 10 minutes in length on
[date]. You may use the doc cam to show elements of your project. This is a great chance for other
students to learn strategies that may be useful to them in the future. In particular, your fellow students
would like to know: What is your course like? Was the professor receptive to your questions? What
elements of the Strategy Project worked in your course?
Bringing a Learning Strategies Project to Scale in a First-Year Seminar 44
Journal of Effective Teaching in Higher Education, vol. 1, no. 2
Strategy Project Contract
Name__________________________________________
Course you will use for this project__________________ Date of Test (between 9/30 and 10/30):
__________________
I would like to choose the following study strategies for my project. Please choose at least four.
☐ Active Notetaking AND/OR ☐ Active Reading
☐ Complete Practice Problems
☐ Quiz Yourself
☐ Create Flashcards
☐ Create a Concept Map
☐ Meet with a study group
☐ Attend a tutoring session
☐ Other (Please Describe):
I understand that my due dates for this project are the following:
_______________ Contract
_______________ Pre-Reflection Paper and Survey
_______________ Professor Interaction (three weeks prior to test)
_______________ Plan of Study (two weeks prior to test)
_______________ Active Notetaking (next KSU 1101 class meeting after your test day)
_______________ Active Reading (next KSU 1101 class meeting after your test day)
_______________ Additional Test Preparation Strategies (next KSU 1101 class meeting after your test day)
_______________ Post-Reflection Paper and Survey (two weeks after your test day)
_______________ Presentation (if required by instructor)
__________________________Student Signature _______________________Instructor Signature