key: cord-272439-9dvvm78i authors: iyer, parvati; chin, lauren; lundergan, william title: remote learning in a periodontal workshop during the covid‐19 pandemic date: 2020-08-09 journal: j dent educ doi: 10.1002/jdd.12358 sha: doc_id: 272439 cord_uid: 9dvvm78i nan social distancing guidelines and shelter in place orders due to the coronavirus disease 2019 ( covid-19) pandemic have made preclinical instruction in periodontics challenging. ordinarily, 18−36 students would be working in the clinic in teams of three, practicing infection control protocols, performing periodontal examinations, practicing periodontal instrumentation, and improving their communication skills. clinical dentistry involves decision-making skills, communication skills, and most importantly, psychomotor skills. how can dental students replicate a periodontal recall visit virtually, in preparation for treating patients? the teaching process in periodontics that had been in place before the pandemic was a product of incremental development. the theme for the current quarter was recall care. once it was decided that the instruction would be virtual, faculty were trained to teach via webex (cisco systems, inc., san jose, ca, usa). a flipped classroom approach was used. every week, students followed a familiar schedule, watched curated educational videos, looked up answers to critical thinking questions prior to the workshop, and engaged in discussions with familiar faculty. ample time was allotted for question and answer or discussion. students were graded on their participation in the seminar. questions that were not answered in class were addressed through email in a timely manner. participant feedback was reviewed for implementation before the subsequent groups rotated through. faculty also offered small group or individual videoconference tutoring as needed. technical support operators were readily available ensuring a smooth virtual experience. attendees were able to participate despite environmental circumstances, differences in time zones, etc. students reviewed the sequence of clinical care and were kept informed of changing guidelines concerning personal protective equipment. small groups discussed various aspects of instrumentation and patient management, addressed broader concerns pertaining to becoming a provider, and received timely grading and feedback. students as a group, reaped the benefit of one person posing a question and having multiple faculty offer responses in real time. students gave positive feedback on this format; they appreciated the pace, efficiency, and the learning effectiveness. a true clinical simulation was not possible because not all students had the necessary equipment and instruments. faculty and students were unable to utilize the camera during lessons due to connectivity issues. faculty could not give very detailed personalized feedback to students. there is an underutilization of videoconferencing applications and other technology to engage students in synchronous and asynchronous teaching. it is important to understand the limitations of remote learning and design learning objectivess accordingly. teaching manual skills is a unique challenge to meet and may need to be separated from the virtual experience, taught in a laboratory environment, or delayed until initial clinical experiences begin. with the possibility of a resurgence of the pandemic in the future, it is imperative that we evaluate this experience and attempt to streamline our objectives. future direction could be to assess students' ability to perform the sequence of clinical care, via recorded video of them role playing a recall periodontal appointment. case-based discussions could provide context and help with the decision-making process. implementation of a flipped classroom educational model in a predoctoral dental course using online periodontal case-based discussions to synchronize theoretical and clinical undergraduate dental education instructional video in e-learning: assessing the impact of interactive video on learning effectiveness exploring the relation between online case-based discussions and learning outcomes in dental education key: cord-280760-jmmvcaey authors: arowoshola, lola title: medical education engagement during the covid-19 era – a student parents perspective date: 2020-07-01 journal: medical education online doi: 10.1080/10872981.2020.1788799 sha: doc_id: 280760 cord_uid: jmmvcaey the covid-19 pandemic has affected the delivery of medical education and has limited the ability of student parents to fully engage with their studies. student parents have been faced with additional challenges such as increased childcare roles and home-schooling responsibilities, splitting their focus. identifying the issues student parents face and adopting workable solutions at all levels, will ensure the best outcomes for these students and better preparedness for the future. the coronavirus (covid-19) pandemic has had far reaching implications worldwide. society and socialising have been significantly changed with the imposition of infection control measures such as social distancing, the use of facemasks and the enforcement of nationwide lockdowns and travel restrictions. work-from-home is now the new normal for the majority of the population in lockdown. educational institutions ranging from nurseries and primary schools all the way to up further education establishments have been closed except to children of key workers [1] . this has led to parents providing round the clock childcare while still working or studying full time. parents have also become impromptu teachers as home-schooling has been adopted. the change in the educational landscape is likely to have significant and unforeseen consequences for years to come [2] . medical education has not been spared in these covid-19 times. in the uk, just prior to the official government lockdown on 23rd march 2020, some medical schools began cancelling clinical placements and face-to-face teaching [3] . at st george's university of london, the teaching and administrative staff developed online teaching content to be delivered via platforms such as big blue button, zoom and microsoft teams. the use of technology to provide online teaching has been shown to achieve high levels of student engagement [4] . the content covered for missed placements, small group tutorials, clinical skills sessions, anatomy sessions as well as lectures. the availability and accessibility of such online resources is however not synonymous with student engagement for some subsets of the student cohort. medical students with parental responsibilities fall into this group. under normal circumstances, student parents face many challenges that affect their educational experience and their ability to engage [5] . specifically, the requirement for care giving and home-schooling during the lockdown has affected my ability to engage fully with my studies. homeschooling can be a very involved process especially for younger children. children in reception/year 1 are just at the start of their education, when it is essential they build the basic foundations for a life of learning. they require consistent teaching and interaction due to the potential for decline in school performance during lockdown [2] . for me, this raised the dilemma where i was not able to focus fully on teaching my children properly as i still had to give attention to my medical education. as such, i struggled to do justice to either. online teaching sessions running for several hours would lead to interruptions and reduced focus due to attending to the caring needs of my children. to compound matters further, the need to prepare for and sit an online exam from home with all the associated stresses and distractions made a hard situation even worse. the medical faculty provided different avenues to help students with caring needs. these ranged from the extreme of taking an interruption of studies to providing extra time to submit any coursework and ensuring all online teaching was recorded, allowing students the option to gain from the session at a more convenient time. during the lockdown period, the department of health named medical students as essential workers [6] . this title has given student parents the opportunity to send their children to school on a full time basis, giving some much needed assistance. in theory, this solution would seem to solve the childcare problem facing student parents. however, access has been mixed as not all nurseries or schools have been able to remain open for key workers children. some schools have required that in two parent households, both parents must be key workers to be allowed to attend. schools are only running standard hours of 9am to 3pm, which do not align with full time attendance and placement hours. this government level solution has unfortunately not fully addressed the needs of all essential workers, which extends to student parents. as lockdown eases in england, the working plan for schools to reopen fully in september, may not be realised [7] . we also have to be wary as the potential for isolation measures to be continued as far as 2022 is a possibility [8] . there is a lot of uncertainty looming. this will invariably continue to impact on student parents ability to engage fully with their course and affect their ability to attain high levels of academic achievement. in times of uncertainty and crisis, challenges can be quickly identified in isolation and the solutions developed and implemented can have limited effect. although the situation around covid-19 is novel and unforeseen, the approach taken to appropriately manage a crisis is essential. what covid-19 has taught us as societies is the need for flexibility and adaptability. this extends from the government, to the medical institutions and to us students as well. it is important that we take the time to identify the issues appropriately. by taking note of the success stories such as online teaching in medical education, we have the opportunity to improve and develop them further. we must also learn from the shortcomings of the solutions adopted, especially those at the government level so we are better prepared for next time. the health service requires medical students to progress and graduate, so the onus is on all parties to ensure that no student is left behind. covid-19: schools set to close across uk except for children of health and social care workers community, work, and family in times of covid-19 let us help'-why senior medical students are the next step in battling the covid-19 using technology to increase student (and faculty satisfaction with) engagement in medical education medical student-mothers london: 2020. statement on clinical placements schools may not fully reopen in september, health secretary hints [internet]. independent projecting the transmission dynamics of sars-cov-2 through the postpandemic period the authors report no conflict of interest. http://orcid.org/0000-0002-5138-0096 key: cord-314605-bnsp4xfj authors: dewart, georgia; corcoran, lynn; thirsk, lorraine; petrovic, kristin title: nursing education in a pandemic: academic challenges in response to covid-19 date: 2020-05-28 journal: nurse educ today doi: 10.1016/j.nedt.2020.104471 sha: doc_id: 314605 cord_uid: bnsp4xfj nan the world is facing unprecedented challenges in the face of a global pandemic. coronoavirus disease 2019 has, to date, killed thousands worldwide (world health organization, who, 2020a) . the data related to numbers of tests administered, positive cases, hospitalizations, and deaths, changes on an hourly basis. mounting concerns regarding the strain on health care systems globally are already occurring and likely to get significantly worse. this health crisis impacts not only frontline staff and clinical leaders but all systems and communities. covid-19 has also already disrupted universities and academic institutions. within the health field, schools of nursing are bracing for unique challenges related to our role in helping develop the next generation of care providers. this article focuses on the unique needs and concerns of nursing educators and nursing students in the face of the covid-19 pandemic. drawing from firsthand experience as nurse educators leading clinical courses in a post-licensure nursing program, we consider some of the early lessons as we face this health crisis. we discuss the concerns that have been shared by students. we question how we will continue to educate nurses in a society facing social distancing, isolation, and quarantine measures, while also needing nurses at the frontline. on march 15, 2020, our academic nursing program made the difficult decision to remove students from clinical practicums. at that point in time, there were fewer than 250 cases of covid-19 across canada (who, 2020b). in the days preceding this decision, numerous questions and concerns were shared in online meetings, over conference calls, and in urgent emails amongst colleagues. athabasca university offers a bachelor of nursing program to licensed practical nurses from across canada in an online environment. while the majority of j o u r n a l p r e -p r o o f learning is conducted online, students in this program must complete in-person clinical placements to fulfill the bachelor of nursing degree requirements. athabasca university did not experience the same challenges that other academic institutions faced with the quick pivot to distance learning in a digital environment. however we, like other nursing programs across canada, had to make difficult decisions on whether or not to continue with our clinical courses. ultimately, the serious concerns related to learner safety were at the forefront of this decision. ethically we had to consider weighing the value of education against the risk and strain to the learner personally and professionally. within our program, most students work regular hours as licensed practical nurses in their respective communities across canada in addition to pursuing their educational commitments, including theory and clinical courses. students often travel hundreds and sometimes thousands of kilometres from their home province to come to alberta to complete clinical courses. when rates of covid-19 increased, there were concerns from both faculty and students on the possible restrictions or limits to travel within canada. concerns regarding travelling focused on the possibility of transmission. specifically, the risk that a nursing student as either an asymptomatic or mildly symptomatic carrier of covid-19 could return to their community and cause a further unconscionable spread of the infection. if students were exposed to covid-19 while in a clinical setting, they risked not only their health but that of their family. we also considered that students could face the financial burden of being quarantined without an income. additionally, as a post-licensure program, if students were exposed, they would be unable to return to work, adding further strain to the health system. students expressed concern about what an interruption in their nursing education would mean for their future careers as registered nurses. many students in clinical placements were in j o u r n a l p r e -p r o o f their final focus clinical practicum, and thus close to successfully completing their program. some learners were only one or two courses away from completion of their bachelor of nursing degree. in addition to cancelling their current clinical placement, they had concerns about progressing in the rest of their program. we did not have all the answers or solutions at the time the decision was made, and are still working through the immediate safety concerns and implications for future clinical placements. some students shared that they felt selfish and were frightened about not reaching the career goals that they have worked so hard to obtain. our approach with students has been to reassure them that the worries regarding their academic path are valid and taken seriously. the risk of acquiring covid-19 as a care provider is significant (bernstein et al., 2020) . however, many students have already disclosed that they would take on that risk if it means they could finish their bachelor of nursing degree. the responses from students highlight a central issue in removing students from clinical: the academic institutions' concern about the risk to the student, and the student's willingness to take on the risk if it means the completion of their program. examples include students who would consider volunteering for pandemic related service if it accumulated clinical hours necessary for degree progression. the rising tension in nursing is palpable, and for many of us, this is unprecedented. our academic programs will provide a workforce with further skills and knowledge to contribute in a direct and meaningful way. registered nurses will play a major role in managing and ultimately defeating this pandemic. these actions signal a concern that there may come a time when the similarly, there may be a point where educators are required to return to the bedside to provide care. in addition to the responsibility to our students, nursing academics, especially those with clinical ties, have a sense of duty to our clinical colleagues. as registered nurses, we are wrestling with guilt from the tension between wanting to urgently join our colleagues on the frontlines and staying in our current roles, recognizing that in the intermediate and longer-term, this will enable the addition of more registered nurses to the workforce. exposure to repeated and disturbing images in the media related to the covid-19 epidemic, such as overworked and understaffed units, can be psychologically distressing for many people (garfin et al., 2020) . nurse academics may also feel a heightened and acute sense of guilt for not contributing as frontline health care workers during the pandemic. there have already been requests to nursing alumni and retired nurses for increased support in the covid-19 response, and the impulse to contribute to direct pandemic care is compelling. however, it would be irresponsible to act without considering the long term implications of a shift in focus. nurse educators will face difficult decisions on supporting the public and their clinical colleagues. however, there also must be a consideration for the needs of current and future nursing students. amid panic and crisis, it is challenging to continue planning for an uncertain future. however, universities and educators have a societal obligation to think long term. how will we recover from the pandemic? will we be able to sustain and maintain a program if the crisis extends long term? is it responsible and ethical to advocate for accelerated programs to licensure, and reduced clinical practicum hours? if educators, who are also nurses, are pulled into providing clinical care and frontline service, how does that affect our responsibility and capability to j o u r n a l p r e -p r o o f educate when, in a year, an exhausted and diminished workforce needs more nurses? at this moment, we have more questions than answers, but sharing these concerns allows for collective action and collaboration. we hope that by disclosing these concerns, we can help frame the academic discussions that all nursing programs will have to face. yet, even with all the fear and anxiety, there are lessons in hope. students' main concerns have been the health of patients and communities. in the week following our removal of students from clinical, we have continued student learning online. in discussion forums, students have shared insightful, articulate, and honest thoughts on the impact of this pandemic worldwide. even without direct clinical education, these nursing students are learning about the role of the registered nurse. their responses have been a salve to the terrifying news cycle; their intelligence and compassion that will be one of the factors that guide us through this crisis. we have also witnessed globally the recognition of the dedication, roles, and significant contributions that nurses play in the health care system. for the students to enter academic programs in the future, they will begin their education aware of risks and challenges that no other cohort could have ever imagined. covid-19 is an ongoing crisis; it is a real-time lesson in equity, leadership, social justice, ethics, and patient care. this pandemic will forever shift the educational landscape; it already has. the healthcare system in italy's lombardy region is so strained from the new coronavirus that officials are asking doctors to come out of retirement and nursing students are being fast-tracked to graduation covid-19 hits doctors, nurses and emts, threatening health system the washington post the novel coronavirus (covid-2019) outbreak: amplification of public health consequences by media exposure. health psychology coronavirus disease 2019 (covid-19) coronavirus disease 2019 (covid-19) key: cord-318722-9nhrd81x authors: sell, naomi m.; qadan, motaz; delman, keith a.; roggin, kevin k.; spain, david a.; phitayakorn, roy; lillemoe, keith d.; mullen, john t. title: implications of covid-19 on the general surgery match date: 2020-05-20 journal: ann surg doi: 10.1097/sla.0000000000004032 sha: doc_id: 318722 cord_uid: 9nhrd81x nan otherwise be immersed within the hospital on core clerkships, including general surgery. while creation of a "virtual" surgical curriculum would be beneficial, 2,3 the exposure to actual surgery cannot be replicated with any available virtual surrogate and as such, surgical disciplines are disproportionally affected by the inability to experience the hospital setting firsthand. strategies to account for these missed opportunities must be developed given the critical importance that core clerkships ultimately have on medical students' decision-making in choosing career specialties. numerous studies have demonstrated the impact that a surgical clerkship rotation has on inspiring career choices for medical students. 4 exposure to the operating room environment, in particular, remains critical to fully appreciate a surgical career, and with current tools it is impossible to completely replicate it virtually. with limited exposure to a hands-on surgical clerkship, medical students may not be as confident or wellinformed in their decision to pursue a 5-7 year residency in surgery, with repercussions for students entering the match in general surgery for the next 1-2 years following the height of the covid-19 pandemic. in addition to the impact on core clerkship rotations, there is a simultaneous effect on students with an expressed interest in surgery with planned sub-internships (sub-is). these rotations provide critical experiences during which students gain a deeper appreciation for the field of surgery to learn if it is an appropriate "fit" for them as a career choice. 5 sub-i's also provide many students foundational knowledge to help prepare them for surgical training. furthermore, the added exposure a student gains to their home department can prove critical in their quest to identify advisors, mentors, and advocates to help them navigate their future copyright © 2020 wolters kluwer health, inc. unauthorized reproduction of this article is prohibited. professional endeavors. reciprocally, these rotations provide an opportunity for faculty to evaluate students' performance, offer them guide advice as to ideal characteristics of residency programs, and generate the content of letters of recommendation (lor). many students undertake multiple sub-is within their desired specialty, which start as soon as april, in order to maximize opportunities prior to submitting their applications. since many surgery residency programs use a rolling residency application deadline, students often feel obligated to submit their applications as soon as the electronic residency application service (eras®) site opens. unfortunately, the current pandemic has placed a halt on sub-is, which has considerable implications for the downstream application components. the questions that therefore arise are how can the educational system compensate for these losses and will the value of a lor be diminished as a result? more importantly, how will the students gather an acceptable portfolio of information about specific disciplines to make informed choices about their optimal career? the decreased availability of sub-i rotations also adversely affects "away" rotations, which are likely to be difficult to arrange even after core clerkship rotations have resumed due to ongoing travel restrictions. for most students, the away rotation is an opportunity to audition at specific programs of interest or to demonstrate their interest in a particular geographic location, and it provides an exploration of "fit" in otherwise unknown programs. 6 this added obstacle will further exacerbate the challenges evolving in students' decision making as access to data about outside programs erodes even further. similarly, for residency programs that utilize these rotations to evaluate prospective applicants from around the country, programs will have to create new opportunities to recruit the best students from across the country and abroad. copyright © 2020 wolters kluwer health, inc. unauthorized reproduction of this article is prohibited. given the above highlighted challenges that may have a substantial impact on downstream resident recruitment for years to come, we propose the following potential solutions: serious consideration should be given to shortening the duration of each core clerkship and/or enabling students to tailor their clerkship schedule to their clinical interests. for example, a student interested in surgery may wish to spend more time on this clerkship and correspondingly less time on another clerkship in which they have less interest (e.g., psychiatry). by delaying the date when programs can receive eras applications from september 15 th to october 15 th , and by delaying the date when the mspe ("dean's letter") becomes available from october 1 st to november 1 st , fourth year medical students would have an additional month to finalize their specialty choice, to complete their desired sub-i rotations, and to obtain the necessary lors for their eras applications. this option would also permit medical students to graduate on schedule without the emotional stress of feeling rushed to choose a specialty or to submit an incomplete application. to meet the need of available sub-i rotations and to still provide appropriate educational experiences, changes to the current structure of sub-is may be necessary. while sub-i rotations are often 1 month in length, it may be beneficial to shorten sub-is to 3 weeks for the copyright © 2020 wolters kluwer health, inc. unauthorized reproduction of this article is prohibited. upcoming academic year. this would still provide fourth year medical students with adequate exposure to the specialty, while allowing for the participation in more rotations in a shorter period of time. additionally, this would increase the availability of positions for other sub-is and visiting students. ideally, this change in length would be adopted by all programs across the country in order to level the playing field and offer maximal benefit to all students. should certain programs not be able to accommodate this, their flexibility would be appreciated in letting their students do this non-traditional rotation and count the credits equally. alternatively, the structure of sub-is could be changed to an "apprentice style" in which the student rotates with just a single faculty member, thereby enabling more students to complete sub-is on a given service. sub-is will also require clearly defined goals and learning objectives with explicit institutional guidelines regarding how covid-19 exposure is managed, including any required testing and/or quarantine time either before or after the sub-i. 7 residency programs should consider hosting a virtual "meet and greet" where potential applicants can ask questions of current residents to obtain answers regarding the culture and "fit" of each program. while this does not make up for the lost audition of the away rotation for some students, it will at least allow students to express interest and can help them assess the "fit" of a given program for them. ideally, these sessions would follow a standardized format to enable the student to make fair comparisons between programs. copyright © 2020 wolters kluwer health, inc. unauthorized reproduction of this article is prohibited. modifying the traditional interview process may be necessary. in congruence with the delayed availability of the eras applications, the interview season could be postponed by 1-2 months, extending even into early february. this short delay would still allow enough time for rank list submission by the end of february and for the match to occur as usual in mid-march. additionally, should pandemic restrictions persist, it may be necessary to transition to virtual interview platforms. programs might even consider an initial round of virtual interviews in october or november as a screening process to choose a smaller number of students to invite later for an in-person interview. flexibility on behalf of residency programs is paramount, as virtual interviews may be consciously or subconsciously evaluated differently than in-person interviews. thoughtfulness in the selection of interview dates will be necessary to minimize overlapping interviews between programs, and consideration to conducting interviews at the conclusion of a student's sub-i rotation should be considered. although this is not an exhaustive list of possible solutions, the importance of anticipating, mitigating, and planning accordingly will allow surgery programs to address the needs of medical students at a time of great uncertainty. it is imperative to maintain continuity of medical student education, engagement, and recruitment into surgery during this difficult time, and we encourage our national surgery education organizations to maintain an active role in this process. important guidance for medical students on clinical rotations during the coronavirus (covid-19) outbreak american college of surgeons acs/ase medical student simulation-based surgical skills curriculum medical student experience in surgery influences their career choices: a systematic review of the literature medical student subinternships in surgery: characterization and needs assessment time to reevaluate the away rotation: improving return on investment for students and schools surgical subinternships: bridging the chiasm between medical school and residency: a position paper prepared by the subcommittee for surgery subinternship and the curriculum committee of the association for surgical education wolters kluwer health, inc. unauthorized reproduction of this article is prohibited nms was supported by the nih t32 research training in aging grant 5t32ag023480-14.the nih had no involvement in study design; collection, analysis or interpretation of data; writing of the report; or decision to submit the article for publication.we also want to acknowledge and thank all the amazing surgery educators who are balancing their clinical responsibilities while still trying to inspire our next generation of physicians during these challenging times.copyright © 2020 wolters kluwer health, inc. unauthorized reproduction of this article is prohibited. key: cord-323621-cw54dfos authors: reuben, jayne s.; crawley, william t.; webb, paris; den brok, koen; woodburn, elizabeth; montemayor, jennifer r.; roberts-lieb, sol; de jong, peter g. m.; dickinson, bonny l. title: iamse meeting report: student plenary at the 24th annual conference of the international association of medical science educators date: 2020-09-25 journal: med sci educ doi: 10.1007/s40670-020-01087-9 sha: doc_id: 323621 cord_uid: cw54dfos nan health professions are rapidly changing in response to advances in technology, modern approaches to information collection and processing, financial pressures, changes in society and demographics, and shifts in political climates. in response to these and other challenges, healthcare providers must become agile and modify their clinical practices to continue to provide exceptional patient care. importantly, these challenges also need to drive and empower educators to evolve in their approaches to training future providers. thus, educators not only need to prepare students to practice in current healthcare environments but also must provide a foundation for practice in the rapidly evolving healthcare environments of the future. while research that gathers students' thoughts, attitudes, and impressions about their education and training is useful, hearing their voices first-hand is essential to identifying and addressing their needs to inform curricular change. with this in mind, iamse held its first student plenary featuring students studying medicine and dentistry in the netherlands and the usa. the title of the session was student voices: envisioning the future of health sciences education across different healthcare professions worldwide. the goal of the session was to engage meeting participants in a discussion of how best to prepare students for the healthcare environments of the future. each presenter gave a 10-min presentation outlining their thoughts about the future of health profession education. an audience question and answer session followed the presentations. the student presentations are summarized below. william tyler crawley, ms4 student, rocky vista university college of osteopathic medicine, usa the field of medical education has long lagged behind other disciplines when it comes to teaching about diversity. as a result, medical providers are not being adequately taught how disease uniquely affects different populations or how to apply evidence-based medicine to treat diverse patient populations. implicit bias is another area requiring further attention in medical curricula. we all develop unconscious attitudes and stereotypes that impact our decision-making. these biases are responsible for the incorrect assumptions we hold that can narrow the options provided to our patients with the effect of limiting their ability to make informed decisions about their own healthcare [1] . research has shown that interpersonal experiences and education can raise awareness of implicit bias, and that this may help to prevent negative impacts on patient care [2] . to address implicit bias, students must partner with educators and institutional officials to change not only what is taught but also how it is taught. educators the 2020 iamse annual meeting introduced a new program feature: the student plenary. in this session, the student perspective on the meeting theme was highlighted by a panel discussion with four students from different health professions programs. in this meeting report, we provide a brief summary of their presentations. working with students and experts in the community should identify opportunities to integrate information about diverse patient groups into the medical curriculum. furthermore, a thorough review of curricula should be conducted to ensure that learning materials and assessments are free from historical, medical, and societal stereotypes so that these biases are not perpetuated. in summary, as universities seek to address diversity, an explicit commitment is needed to provide dedicated time in the curriculum for education on minority populations and diversity topics. these efforts should also extend beyond the classroom to address diversity issues in the faculty, staff, standardized patients, and university policies. opportunities should be sought for institutions to partner with experts in the community who can share their knowledge, serve as speakers, and provide learning resources. finally, these efforts can be facilitated as institutions provide students with a mechanism to report concerns about learning opportunities and to recommend suggestions for improvement. paris webb, d4 student, texas a&m university college of dentistry, usa in response to the covid-19 pandemic, many institutions were forced to switch from a traditional in-class lecture model to a virtual educational format. these efforts demonstrated that virtual learning is possible, although it cannot substitute for the requisite face-to-face learning required to master communication and hand skills that dentistry requires. thus, the pandemic has presented a unique opportunity to rethink how we educate health professions students. another driver of change at texas a&m college of dentistry occurred in response to changes in the national dental board examination [3] . specifically, the institution introduced several curricular changes to provide new educational approaches and assessments to prepare future dentists to excel in the evolving field of dentistry. these changes included reorganization of a siloed first-year curriculum into an integrated systems-based human structure, function, and pathology course that included content from anatomy, histology, physiology, biochemistry, and pathology. in addition, the creation of an integrated practice of dentistry course enabled firstyear students to practice diagnostic techniques as part of the physical assessment of the dental patient, which facilitated early entry of these students into the dental clinics to assist upper-level students. third, a new longitudinal comprehensive pain management curriculum was developed in response to the ongoing opioid epidemic. a final driver of change in healthcare education is the growing diversity of the us population [4] , which highlights the importance of teaching students about cultural competence and social determinants of health in all specialties of healthcare including dentistry. these changes in healthcare education necessitate collaboration between students, educators, and institutions to ensure that the next generation of dental health professionals are well prepared to provide individualized patient care. koen den brok, master 2 (5th year student), radboud university medical center, the netherlands over the years, the practice of dental care in the netherlands has slowly changed due to an aging population with different healthcare demands [5] . at the same time, the number of oral health professionals is declining as many of them are facing retirement in the next 5-10 years. in response to these developments, dentists have begun to collaborate in shared practices and group-clinics [6]. to prepare the dental students of today for this new reality of the future, radboud umc dental faculty developed a master clinic program for 4-6th year students. this master clinic creates an interprofessional workplace simulating a dental clinic. in the master clinic, both dentistry and oral hygienist students learn practical and theoretical dentistry together while providing care to the patients in their respective clinics. entrustable professional activities have been implemented as an effective tool to teach students selfreflection and to follow study progression on clinical and professional activities [7, 8] . students also learn leadership skills as they are now involved in teaching their peers and managing the clinics. in the program, teachers have taken on the role of coaches and guides instead of lecturers. this novel approach is highly valued amongst students and staff and has proved to be a significant addition in preparing and motivating the students for life-long learning. this successful innovation could not have happened without synergy between student representatives and the faculty. finally, the covid-19 epidemic has been an accelerator for the development and implementation of new digital teaching modalities in the dental curriculum. these new teaching modes have enabled students to learn at their own pace and have opened up possibilities for making the study program more flexible. these and other curricular changes have the potential to decrease stress felt by healthcare professions students and should be carried forward into the future. the future of medical education should be forward-focused, patient-centered, and student-driven to graduate professionals who are engaged in their work and positioned to lead the field of healthcare forward. as students have access to an everincreasing amount of information in the form of texts and digital resources from which to learn the basic sciences, there is a need to refocus classroom time on collaborative learning and synthesis of information. with problem-based learning serving as an early foundation, students can translate learning to patient care and scholarship, so they are better stewards of healthcare resources [9] . dedicated independent study time in the curriculum is needed for students to examine medicine from different perspectives and to devise potential solutions for clinical problems they encounter [10] . these changes will encourage students to embrace a growth mindset and create positive change. to achieve these goals, health science educators must also adopt a growth mindset and be open to developing and using novel interventions, methodologies, pedagogies, and learning tools. conflict of interest the authors declare that they have no conflict of interest. informed consent na unconscious bias in academic medicine: overcoming the prejudices we don't know we have contact and role modeling predict bias against lesbian and gay individuals among early-career physicians: a longitudinal study integrated national board dental examination united states census curriculum development for the workplace using entrustable professional activities (epas): amee guide no. 99 assessing trainees and making entrustment decisions: on the nature and use of entrustment-supervision scales problem-based learning and medical education forty years on. a review of its effects on knowledge and clinical performance introducing first-year medical students to product innovation and entrepreneurship publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-028462-f01yt9l1 authors: tenório, kamilla; chalco challco, geiser; dermeval, diego; lemos, bruno; nascimento, pedro; santos, rodrigo; pedro da silva, alan title: helping teachers assist their students in gamified adaptive educational systems: towards a gamification analytics tool date: 2020-06-10 journal: artificial intelligence in education doi: 10.1007/978-3-030-52240-7_57 sha: doc_id: 28462 cord_uid: f01yt9l1 in this paper, we present the results of a case study conducted to validate the effectiveness of our gamification analytics model for teachers proposed in [20]. to conduct this case study, we developed a tool to monitor and adapt gamification designs in gamified adaptive educational systems. employing this tool, the case study was conducted in a real situation, and the findings suggest that the use of our model and tool improves students’ engagement, learning outcomes, and motivation. gamification is pointed out as a valuable approach to improve students' engagement, motivation, and learning outcomes [1, 2, 6, 13, 15] . however, previous studies reported that using gamification in educational technologies does not always assure the expected results' achievement [5, 8, 14, 18] . a promising solution to maximise the gamification benefits is to monitor users' behaviour in the gamified environment and adapt its gamification design when the expected outcomes are not achieved [9, 10] . this approach is named gamification analytics and it was defined by heilbrunn, herzig, and schill [10] as "the data-driven processes of monitoring and adapting gamification designs". nevertheless, there is a lack of studies that apply the gamification analytics approach in education, and, particularly, in the aied field [3, 9, 21] . therefore, we propose a gamification analytics model for teachers to support them in the process of monitoring the impact of gamification in gamified adaptive learning systems, and adapt the gamification design when considered necessary. based on this model, a tool was developed, and a case study was conducted to investigate the impact of the use by teachers of the model through the proposed tool regarding students' engagement, learning, and motivation. in the gamification analytics model, teachers may define interaction goals they expect their students achieve, and monitor, during the learning process, if the interaction goals are being achieved through the visualisation of students' interaction with the system's learning resources and game elements. if the outcome is not as expected, teachers may adapt the gamification design through the creation of missions. gamanalytics is a gamification analytics model-based tool, and the design concepts implemented in the gamanalytics tool were validated with teachers with respect to their needs and opinions [20] . gamanalytics tool is integrated to a gamified adaptive educational environment, named avance (https:// avance.eyeduc.com/). this tool includes a class' dashboard and an individual student's dashboard. in the class' dashboard, there are visualisations shown through descriptive data and graphs for each topic of a course, such as number of students registered in the course; the period expected for students to achieve the interaction goals; the class' progress over time in relation to interaction with learning resources; the number and names of students that achieved or not the interaction goals; the number and names of the students that interacted (with success or not) with each learning resource; the number and names of the students that are in each level of gamification. in the individual student's dashboard, there are more visualisations, such as student' basic info; student's gamification info such as points, current level, and position in the ranking; student's progress over time in relation to interaction with learning resources; and student's interaction with each learning resource (see fig. 1 ). a case study is conducted to explore the impact of the use by teachers of the gamification analytics model through the gamanalytics tool regarding students' engagement, learning, and motivation. ten undergraduate and graduate students of the federal university of alagoas enrolled in the "gamification in education" course are considered in this case study. this study took place for four weeks, which was the expected time for students to master the "framework, models and processes" and the "gamiflow" topics. to conduct the case study, the gamanalytics tool integrated into the gamified adaptive educational environment (avance) was used. first, the teacher defined the interaction goals that he expected students to achieve for the domain of each topic (e.g., it was expected that students interact at least with 60% of the resources of the "gamiflow" topic in 3 weeks). after the teacher's preparation, students completed a demographic questionnaire, and answered the informed consent form. students also answered a pre-test, reviewed by the teacher, of the two topics. pre-tests were planned according to the levels of the revised bloom taxonomy [12] to be balanced with the post-tests. afterwards, students started using avance, and the teacher could visualise students' data through the gamanalytics tool. when the teacher realised that the outcomes were not as expected, he assigned missions to groups or to a specific student through sending emails. in the email, teacher indicated the expected period of time for the mission, reward, and the set of resources that students should interact to achieve the sent mission. after that, he could visualise the impact of the intervention through the gamanalytics. for each topic, teachers created 3 different missions depending on students' interaction. at the end, students answered the post-tests, the imi (intrinsic motivation inventory) [7, 16, 17] and imms questionnaires (instructional materials motivation survey) [11, 19] to measure participants' motivation -questionnaires validated in the portuguese language [4] . to investigate students' engagement, we measured the number of students' interaction with each topic's resources before and after the teacher's intervention (creation of missions). the results (from shapiro-wilk test for normality) indicate that the data concerning the two topics are not from a normal population (first topic: w = 0.594, p-value = 0.000047; w = 0.618, p-value = 0.000091/second topic: w = 0.432, p-value = 0.020; w = 0.432, p-value = 0.000058 -before and after the intervention respectively). a non-parametric wilcoxon signed-rank test was performed to compare the number of students' interaction before and after the intervention. concerning the first topic, the wilcoxon signed-rank test indicates a statistically significant difference (z = −2.121, p-value = 0.034) between the number of interactions before and after the teacher's intervention. for the second topic, the test's results also indicated a statistically significant difference (z = −2.214, p-value = 0.027) between the number of interactions before and after the intervention. therefore, students increased significantly their interaction with the resources of the two topics after the teacher's intervention based on the monitoring of students' information, suggesting that students have improved their interaction with the system after teachers intervention. the results of the pre-and pos-tests taken by students before and after the domain of each topic learned were used to measure the impact on students' learning. results from a shapiro-wilk test show that the data may come from a normal distribution -first topic: w = 0.965, p-value = 0.843 (pre-test); w = 0.932, p-value = 0.473 (post-test)/second topic: w = 0.909, p-value = 0.271 (pre-test); w = 0.916, p-value = 0.325 (post-test). a t-test was performed, which indicates that there is a statistically significant difference between the scores of the first topic (t(9) = −4.116, p-value = 0.003) and of the second topic (t(9) = −2.449, p-value = 0.037). therefore, our results might suggest that students have improved their understanding on both topics of the "gamification in education" course after interacting with resources sent by teachers through missions. at the end of each topic, the imi and imms questionnaires were answered by the participants (7-point likert scale). the internal consistency of all imi and imms questionnaires' subscales was greater than .70. concerning the imi questionnaire, the mean overall intrinsic motivation score for the "frameworks, models and process" topic was 4.52. concerning the second topic, the mean overall intrinsic motivation score for the "gamiflow" topic was 4.63. these results may suggest that students were more intrinsically than extrinsically motivated during the intervention in the two topics. concerning the imms questionnaire, in the first topic, note that the mean overall motivation level score was 5.19. whereas, in the second topic, the mean overall motivation level score during the teaching was 4.95. in summary, our results might suggest that the students were motivated (intrinsically and extrinsically) during the intervention in the "frameworks, models and process" and "gamiflow" topics. in this work, we conducted a case study to validate the impact of a gamification analytics model for teachers to monitor and adapt gamification design for students during the learning process. our results might suggest that a gamification analytics tools based on this model impacts positively on students' learning, engagement, and motivation -which are of utmost importance since it also shows that teachers may be active users of gamified adaptive learning systems with the aid of gamification learning analytics. as teachers may monitor and adapt gamification design according to how students or groups of students interact with an adaptive system, teachers could be more effective to make opportunistic pedagogical decisions (informed by gamification analytics) that may lead to an increase in learning, engagement, and motivation of the students. perceptions of students for gamification approach: kahoot as a case study new challenges for the motivation and learning in engineering education using gamification in mooc medit4cep-gam: a model-driven approach for user-friendly gamification design, monitoring and code generation in cep-based systems gamification of collaborative learning scenarios: an ontological engineering approach to deal with motivational problems in scripted collaborative learning gamifying learning experiences: practical implications and outcomes climbing up the leaderboard: an empirical study of applying gamification techniques to a computer programming class autonomy in children's learning: an experimental and individual difference investigation assessing the effects of gamification in the classroom: a longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance tools for gamification analytics: a survey gamification analytics-methods and tools for monitoring and adapting gamification designs development and use of the arcs model of instructional design a revision of bloom's taxonomy: an overview structuring flipped classes with lightweight teams and gamification comparing success and engagement in gamified learning experiences via kahoot and quizizz what do students do on-line? modeling students' interactions to improve their learning experience perceived locus of causality and internalization: examining reasons for acting in two domains relation of reward contingency and interpersonal context to intrinsic motivation: a review and test using cognitive evaluation theory spendency: students' propensity to use system currency effectiveness of motivationally adaptive computer-assisted instruction on the dynamic aspects of motivation raising teachers empowerment in gamification design of adaptive learning systems: a qualitative research a systematic literature review on the gamification monitoring phase: how spi standards can contribute to gamification maturity key: cord-297384-0urbjy5p authors: bal, irene a.; arslan, okan; budhrani, kiran; mao, zixin; novak, kae; muljana, pauline s. title: the balance of roles: graduate student perspectives during the covid-19 pandemic date: 2020-08-09 journal: techtrends doi: 10.1007/s11528-020-00534-z sha: doc_id: 297384 cord_uid: 0urbjy5p the covid-19 pandemic has impacted personal and professional lives. graduate students juggle a variety of roles and had to quickly adjust. in this article, six graduate students share their reflections regarding the influence of the pandemic on respective stages in their doctoral program. they provide unique personal and professional perspectives that depict their abrupt shift to remote working and remote learning. the intention of this article is to garner an understanding of graduate students’ challenges during the pandemic, capture their strategies for success, and provide a space for further conversation and support about how the pandemic has impacted graduate students. after the covid-19 outbreak, education shifted to remote learning, and daily and professional lives quickly changed (daniel, 2020; liguori & winkler, 2020; zraick & garcia, 2020) . within this change, graduate students had to juggle a variety of professional and personal adjustments beyond their student role. this paper focuses on the experiences of six graduate students on the effect the pandemic had on their lives. the students are at various stages in their programssome are completing their coursework and some are working through their dissertation. each student brings a unique professional and personal perspective of covid-19 and their abrupt shift to remote working and learning. who would have thought that friday 13th, 2020 would be the last day of 'normal?' i am a parent of two school-age children, and on that friday the 13th, i received the announcement from the school district-schools would be closed starting monday with students learning at home. i immediately asked myself, "how will i complete my coursework with the kids staying at home?" during the first 3 weeks, i was struggling and my kids were confused. instructions from the school kept changing and my own tasks were neglected. i needed a strategy. i started by identifying my essential tasks to complete the semester successfully. i chunked those tasks into bite-size tasks and created two categories: tasks that needed deep concentration and those that did not. i also created a daily schedule dividing the day between mornings and afternoons. in the morning, i helped the kids with their schoolwork. while helping them, i could accomplish the tasks that did not need deep concentration (e.g., searching for articles). in the afternoon, the kids conducted their independent activities (e.g., free play and arts) while i worked on tasks that needed deep concentration (e.g., writing). this strategy was not perfect. there were times when i could not complete my daily tasks. although this schedule is not the 'normal' we knew before friday the 13th, this strategy allowed me to finish my spring semester! instructional designer perspective by kae novak i am an instructional designer in colorado and a doctoral student at the university of colorado -denver. spring 2020 was supposed to be a quiet semester for instructional designers and educational technologists like me. there was no learning management system (lms) migration this semester, and all known issues were manageable. nevertheless, my team's quiet semester imploded. every faculty member wanted to change their f2f course into an online course immediately. we were bombarded with emails, phone calls, and drop-ins. all new development of courses stopped, and the next 6 weeks seemed like an endless stream of individual just-in-time training on how to use tools in the lms or how to hold class over webex. we created shortcuts and heuristics to help. we discovered emails were faster to read and respond to than voicemails, so we stopped taking voicemail requests. we had to endlessly explain that technology is not magic, sometimes it does not work. size also matters-there will probably be a delay uploading a four-hour lecture you just recorded into the lms! this did not just spill into my doctoral classes, it consumed them. there was no balance. everything became remote learning! my semester paper ended up focusing on remote learning. in courses, as the only ed-tech doctoral student, i was asked to help figure out every setting on zoom, google documents, and determine what tech we could use for final presentations. i had to forgo any strategy for balance and instead fully immerse. i planned to be away for 10 days, not knowing i would be gone for most of the semester. i am an online lecturer in educational technology at loyola university maryland and an online doctoral student at old dominion university (odu). working and learning in online programs allows me to travel but adding a pandemic in the middle of a semester, while traveling, shifted my life considerably. at first, the challenges were in managing my time between family, learning, and teaching. this didn't seem to be much of a shift since i was already working and learning online, but that quickly changed when stay-at-home orders altered my travel plans, my learners needed more flexibility, and my "embrace the suck!" i could almost hear this motivational speech from my drill sergeant as i worked late into the night. ten weeks had passed since covid-19 overloaded our it department and compared to the 10 weeks i spent at the army basic training, these past 10 weeks went by in a blur of zoom meetings and document revisions. as the associate director of information and faculty technology at our business school, it has not been easy to balance my job responsibilities with my roles as a doctoral candidate, an adjunct, and a father. the pandemic brought a disproportional amount of work, which cut into my time for the dissertation. in order not to stall my dissertation completely, i switched my activities from literature review to reflection. i made this change because i knew that my mind would not be sharp enough for the literature review after work. reflection allowed me to organize what i have already read, come up with questions, and identify gaps in my research. my hope is that when the pandemic blows over, i will be on a clearer path to continue my literature review. work will continue to be very exhausting for the rest of the year at my department as we convert our courses to a hybrid format. it is a challenge that must be met. work overload sucks, but i have to try to make the best of the situation, one zoom meeting at a time. spring break ended with a 30-hour travel across four airports but this time it included frequent handwashing, sanitizer, and face masks. i returned to charlotte from an indian wedding and self-quarantined for 14 days. little did i know that this was the beginning of the 'new normal' of remote learning, working from home, and amazon groceries. upon my return, i was caught in the covid-19 wave of emergency remote learning at the university of north carolina at charlotte. this was a major disruption in my dual roles as full-time instructional designer and doctoral candidate in the middle of the dissertation process. nothing could have prepared me for the scale of rapid upskilling needed to train faculty from every discipline to shift every lesson, activity, assessment, project, or lab online. i felt like a first responder, prioritizing the university needs, while my dissertation moved to the backburner. it became harder to balance my time and remain focused on the dissertation. when asked how i was doing, i often responded that i was "taking it one hour at a time, one day at a time." i drew inspiration from the instructional design community on their ideation and creative solution-finding for what i have come to coin as 'six-feet-apart pedagogy.' i am proud of my colleagues for their empathy, discernment, and swiftness handling the challenges of covid-19. my family and friends continue to share their comfort, love, and support. as an instructional designer and graduate student, plans for the next semesters are undecided and erratic… i am still taking it one hour at a time, one day at a time. as the announcement was sent that the campus was closed and courses were moving to online, i thought, "no big deal!" as a teaching assistant at texas tech university, i am responsible for grading and mentoring students in face-to-face and online graduate-level courses. since one of the courses was already online, i thought i could simply help transform the face-to-face course to an online format. however, i was wrong! both courses had a variety of challenges. due to the influx of communication, students were confused and frustrated. some of them were overwhelmed, managing the coursework and their full-time jobs, failing to turn in assignments, requesting extensions, and even requesting to withdraw. my no-big-deal attitude changed as i shifted the workload of the courses to support and motivate students to persist. in addition to this, i am working on my dissertation. balancing the roles between teaching assistant and dissertation writing was challenging. since i designed my dissertation as an experimental study, i paused my dissertation until there was a clear procedure on conducting such a study during the pandemic. i first focused my teaching assistantship, supporting the adjustment of project deadlines, course requirements, and meetings to lessen the load for the students and to support home situations. i also scheduled one-on-one meetings with the students to keep them on track and address their inquiries, and scheduled out-of-class meetings to provide social and emotional support. after things were settled a bit more with the courses, i met with my dissertation supervisor to forecast the barriers for experiments during this time. eventually, we decided to change the dissertation topic. now, i have a blank page in front of me with a totally new dissertation topic. although it is sometimes frustrating, i still have the motivation and the hope to progress on my doctoral journey. education and the covid-19 pandemic from offline to online: challenges and opportunities for entrepreneurship education following the covid-19 pandemic canceled because of coronavirus: a brief list publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations our dual roles uniquely positioned our perspectives and reactions to covid-19. to establish a community of support, we reflected on our different perspectives, challenges, and strategies in a webinar hosted by the aect graduate student assembly. we encourage students to support one another as we are all in this together! you can view our covid-19 pandemic webinar at https://bit.ly/gsacovid-19 and connect with other fellow students through social media: twitter (twitter.com/gsa_aect), facebook group (facebook.com/ groups/aectgsa), instagram (instagram.com/aectgsa), and slack (aectgsa.slack.com/). key: cord-028450-pll5d3mq authors: sjödén, björn title: when lying, hiding and deceiving promotes learning a case for augmented intelligence with augmented ethics date: 2020-06-10 journal: artificial intelligence in education doi: 10.1007/978-3-030-52240-7_53 sha: doc_id: 28450 cord_uid: pll5d3mq if ai systems are to be used for truly human decision-making in education, teachers will need better support for deciding upon educational interventions and strategies on an ethically informed basis. as indicated by a recent call by the aied society to focus on the fate (fairness, accountability, transparency, and ethics) of ai in education, fundamental issues in this area remain to be explicated, and teachers’ perspectives need to be accounted for. the paper offers examples of how ai may serve to promote learning but at the cost of presenting limited or untruthful information to the student. for example, false information about a student’s current progress may motivate students to finish a task they would otherwise give up; hiding information from the student that is disclosed to the teacher may decrease students’ cognitive load while supporting the teacher’s strategic choices, and deceiving the student as to the actual nature of the task or interaction, such as when using virtual agents, can increase students’ efforts towards learning. potential conflicts between such scenarios and basic values of fate are discussed, and the basis for developing an “augmented ethics” system to support teachers’ decision-making is presented. the importance of ethical issues in aied community motivated a recent call to focus on the fate (fairness, accountability, transparency, and ethics) of ai in education. although fate makes a nice acronym, it blurs the conceptual relations between these topics (e.g. fairness can be seen as one of several ethical concerns, and accountability as a concept which guides ethical considerations). to guide research and practice, it needs to be situated both in an ethic-theoretical context and in empirical research, and to take into account the perspective of the practitionersthe teachers. teachers' knowledge of ai and related ethical issues in school needs to increase, and the literature has not clearly addressed the role of the teacher [1, 2] . if ai is to empower education by augmenting human capabilities, how can ethical standards of human decision-making be ensured? what makes for an ethically informed basis? this paper aims to address the ethical foundation that can guide empirical research on the teacher's practical knowledge needs, when using presently available ai such as adaptive systems, virtual agents and learning analytics. it argues that the constituents of an augmented ethics system require a broader analysis than that of augmented intelligence in the traditional sense. for instance, there are national curricula, treaties and policy documents, such as the general data protection regulation (gdpr) in europe and the un convention on the rights of the child, which must not be neglected to provide useful support for teachers. hence, ethical theory, teaching practice and policies all need to inform the development of a system that effectively "augments" ethics. there are many opportunities for using ai to enhance student learning at the cost of presenting untruthful, partial or misleading information to the studentin other words, systems that lie, hide or deceive. the message is not to condemn the existence or use of such functionsin fact, teachers have always used deliberate (over-)simplifications and factual misrepresentations in order to help students learn, and so has been done since the early days of aied [e.g. 3]but as aied grows in complexity, and becomes more pervasive in the absence of human reflection and judgment, we need refined conceptual tools to identify and assess potential ethical conflicts with basic human values. to what extent teachers need support, and of what kind, for taking a position to ethical dilemmas raised by recent aied, remains an outstanding question. some cases may appear unethical, such as deliberately inducing confusion in students by staging disagreements between agents [4] or presenting students with erroneous examples [5] , but become less problematic for mature learners who are "game", become aware of the manipulations and submit to the pedagogic strategy. then there are systems which may have personal repercussions far beyond what students and/or their teachers may recognize. a prevalent concern is privacy, relating to learning analytics (la) [6, 7] , for example whether the overall improvement of a learning environment is a valid reason to store and share the exact location of students to facilitate collaboration with peers. other examples concern the use of intelligent tutoring systems (its) that match students with virtual tutors on emotional and cognitive parameters. this raises issues as to when students' interactions with non-human systems are preferred to a human being. as noted in one study, "what is true if the teacher and ai do not agree?" [1] . the message then, in line with other recent work [8] , is that ethical use of ai in schools require that teachers' unique human expertise is preserved and promoted. such expertise is needed for deciding when it is warranted to use misrepresentations or a "deceptive" system for a larger good, in order to secure educational benefits and avoid risks for students' well-being. next are some examples of such potential conflicts. lying refers to deliberately presenting information to the student that is incorrect, with reference to the available data. in principle, this concerns all cases where students are presented with incorrect information and requested to correct it, although the ai "knows" the correct answer. but there are more subtle and specific examples. studies on learning curves and motivation suggest that students work longer in a problem domain if they make visible progress and are closer to goal (say, 80%) compared to not progressing, further from the goal (say, 40%). such data can be used for algorithms that truthfullymatch the difficulty of learning tasks to the student's current performance level in a "personally" adaptive system (e.g. sana labs, www.sanalabs.com). would it then be ethically justifiable to present false information about a student's current progress, suggesting that one is closer to the goal than performance indicates, in order to motivate students to finish a task they would otherwise give up? hiding refers to presenting selective, but not untrue, information to the student, while processing more data that is relevant to the task but may be presented at a later time and/or to another person (a peer or a teacher). ai systems that serve to identify what data are important to students are implemented in learning analytics (la) and motivate the separation between student-facing and teacher-facing la [9] . hiding information from the student that is disclosed to the teacher may decrease students' cognitive load while supporting the teacher's strategic choices. should ai therefore be used to determine what data are 'better' communicated to teachers and students, respectively? deceiving refers to presenting the student with tasks that are designed to maintain false beliefs or illusions, without making the actual nature of the task or interaction explicit. a form of voluntary deception occurs in all (educational or other) games which involve an "intelligent" opponent that is technically invincible but adapts to the player's performance. the same can be said about collaborative virtual agents, such as teachable agents that increase students' efforts [10] [11] [12] . an interesting example is bella [13] which employs a "super-agent" to adapt to students' knowledge gaps without actual "teaching" by the student. in research, wizard-of-oz methodologies exploit student expectations for improving upon existing systems by having human actors simulate ai agents. to what extent are such illusions ethically justifiable to maintain? the wide variety of issues and ethical concerns makes it difficult to define which aspects of fate to focus on. from consulting ethical-philosophical expertise and standard works [14] four basic values are identified: privacy, safety, trust and fairness. these values are fundamental in the sense that there is no obvious way of telling which value trumps another one. as to the fate dimensions, one can argue that, for instance, "transparency" is not a fundamental ethical value because it could, at the same time, be a risk and a benefit to safety, and a risk to privacy. "fairness", on the other hand, is a fundamental social value (one cannot be "fair" in isolation), theoretically independent of individual privacy and safety. addressing the multiplicity of concerns is helped by distinguishing between pedagogies on the screen-level, "how individual systems work with a single student", and the orchestration-level, "whereby such systems are deployed in the bigger temporal and spatial context of a whole class" [15, p. 6 ]. the screen/orchestration level distinction thus helps both to direct teachers' attention and to see how accountability is attributed. the results of discussions with teachers can inform ethical guidelines that support decision-making as to what values should be protected, to what costs and benefits. figure 1 offers a simplified categorization grid of ethical concerns that emerge from relating teachers' knowledge needs on the screen-and orchestration levels. it is suggested that the teacher take a stand on two questions: is the concern a screen-level priority? is the concern an orchestration-level priority? it should be emphasized that the yes (✓) or no (✗) to these questions is a deliberate simplification; they are a question of focus rather than exclusion, and they do not definitely tell where concerns belong. on the screen-level, concerns of privacy can be viewed with respect to privacy settings available to the individual but also what data the system stores and what personal data is requested at start. the individual's trust in the system is dependent on how well it functions, both for protecting personal data and for producing the expected outcomes. the teacher can assist students with available privacy settings and data storage but cannot directly influence students' trust and expectations, which can only develop from personal experience of working with a system in relation to (human or ai) alternatives. on the orchestration-level, safety can be viewed with respect to how the teacher assesses and manages the risks and threats for all students in a class (arguably, students may have different preferences of privacy and what information they are willing to share, but they should all have an equal level of safety). safety concerns are about the whole group and the orchestration of all systems used in the classroom. fairness is a value of broader ethical concern than can be addressed by either the student on screen or the teacher beyond her own classroom. issues of fairness must not be ignored, but teachers need to be aware of the complex social, financial, and cultural context in which they are embedded. for example, teachers and policy makers may need to consider gender equality, and whether the use of ai should be mandatory. trust e.g. is it possible that students will trust an aibased system so much as to prefer its company to their teacher and their peers? are ai-based systems more trustworthy than teachers in certain areas of knowledge, e.g. math and computer science? fairness e.g. how should ai resources be distributed to student groups on an equal basis? how do we share teacher and teaching resources fairly among students? fig. 1 . a grid for determining types of ethical priorities for aied, with example questions. in conclusion, this organization of ethical priorities put the theoretical corner stones on which to base ethical positions with respect to the teacher's responsibilities, the system properties and contextual knowledge needs. each of the four values deserves attention in its own right. for understanding their meaning in practice and further development, it is suggested that teachers are involved at an early stage and work together with researchers, such as in workshops, in an iterative process of identifying, analyzing, evaluating and re-evaluating ethical concerns. such a project would have great significance both on a societal level and for covering knowledge gaps on the ethics of aied. critical imaginaries and reflections on artificial intelligence and robots in postdigital k-12 education teacher-supported ai or ai-supported teachers the use of pedagogic misrepresentation in tutorial dialogue inducing and tracking confusion with contradictions during complex learning using erroneous examples to improve mathematics learning with a web-based tutoring system ethical and privacy principles for learning analytics learning analytics: ethical issues and dilemmas international perspectives on the role of technology in humanizing higher education trends and issues in student-facing learning analytics reporting systems research teachable agents and the protégé effect: increasing the effort towards learning cognitive anatomy of tutor learning: lessons learned with simstudent a teachable agent game engaging primary school children to learn arithmetic concepts and reasoning reinforcing math knowledge by immersing students in a simulated learning-by-teaching experience a theory of justice escape from the skinner box: the case for contemporary intelligent learning environments key: cord-286267-pr481uep authors: borsheim, brianna; ledford, chelsea; zitelny, edan; zhao, caroline; blizzard, john; hu, yenya title: preparation for the united states medical licensing examinations in the face of covid-19 date: 2020-06-16 journal: med sci educ doi: 10.1007/s40670-020-01011-1 sha: doc_id: 286267 cord_uid: pr481uep the covid-19 pandemic has led to the canceling and rescheduling of the united states medical licensing examination (usmle) examinations due to the nationwide closure of the prometric testing centers, which poses a significant challenge to medical students. the rescheduling of a high-stakes board examination leads to significant stress and potential burnout. students may need guidance to decrease anxiety and reframe their study plan while maintaining their knowledge. here, we combined board examination coaching tips with specific worked examples to describe how to prevent burnout, give recommendations for scheduling, and suggest practical approaches to usmle and other high-stakes examinations. the covid-19 pandemic has caused uncertainty and anxiety at all levels of education, and medical education is no exception. it has made a remarkable impact during a time of educational transition for medical students, especially for students who are preparing to take the united states medical licensing examination (usmle) licensing examinations. performance on usmle step 1 and 2 examinations is a major consideration for residency programs in selecting medical school graduates [1] [2] [3] . under normal circumstances, medical students who are facing the high-stakes usmle examinations are challenged by the intensity and length of study. for example, in step 1 preparation, it was reported that the average studying time per day is over 11 h for 35 consecutive days [4] . during this period, students' emotional, mental, and physical coping methods are continually challenged. with the initial nationwide closure of prometric centers, partial "soft" openings, and continued glitches of unpredictable cancelations of the test dates, students with pre-scheduled test dates in march and april had to reschedule to later dates with the hope of being able to sit for their respective exams. in addition, the discrepancy between the prometric's official statements and planned operating procedures (nbme.org) forced students to face waves of cancelations of their rescheduled dates in may and june. some students with confirmed dates from the prometric centers arrived to test and were met with cancelation. this prompted an unprecedented acknowledgment from the usmle of prometric's "failure to fulfill its obligation to examinees." as a result, students have to modify-and often extend-their study plans. this unexpected turn of events will likely increase their anxiety and risk of burnout, which can directly impact their performance [5] . furthermore, with more than half of the medical student population already experiencing burnout, estimated by a major multi-institutional study pre-covid-19, the negative impact of covid-19 on student's well-being and burnout cannot be ignored [6] . in addition, burnout has been associated with decreasing empathy [7] . empathy holds one of the core values in physician-patient care. chen and colleagues [8] characterized that there was a declining level of empathy among students as early as the third year of medical school. therefore, it will require multi-faceted approaches to generate practical ways to alleviate stress. specifically, during these delays and cancelations due to covid-19, preventing burnout with modification or reframing of a study plan is of utmost importance. here, we describe the practical applications of the usmle examination strategies, with a focus on study scheduling, burnout prevention, and approaches for the usmle examination in this unprecedented time. the following two practical applications were developed by the corresponding author, yh, and based on the framework of cognitive load theory (clt) and adult learning theory (alt) in medical education [9] [10] [11] . these practical applications specifically focus on the clt components of decreasing intrinsic and extraneous cognitive load, as well as self-directed learning in alt. they were designed to promote a focused, intentional, and outcome-driven framework that can be applied to various learning situations. the self-assessment, identification, and patching (sip) approach is a continuous cycle. self-assessment can be accomplished using any learning resource, such as lecture notes, anki cards, or question banks. it should facilitate the identification of specific areas needing improvement. for example, it may start broadly as "let me see how i am doing in the gastrointestinal (gi) system using a block of questions in gi." identification is achieved from data analysis of the performance on the pre-selected topic, e.g., performance on gi. this type of review may need further narrowing of the focus down to anatomy, biochemistry, and pathology within gi. with the specific question and the assessment data in hand, patching becomes more focused, personalized, and manageable. consequently, the selection of resources and study strategies to patch up these areas becomes more efficient. the second round of self-assessment (e.g., questions of anatomy, biochemistry, and pathology in gi) will demonstrate the effectiveness of the study strategies and selected resources. the method provides a personalized, logical, and focused approach that demands accountability from the student. the goal, means, and outcome (gmo) method is a simplified and effective tool meant to assist the learners in identifying and modifying achievable and measurable goals to reduce anxiety. it was created by the corresponding author (yh, manuscript in preparation). it aids in planning, outlining, and achieving an often intimidating task. when overwhelmed and stressed in preparation for usmle examinations, students may set vague goals without any specific assessment to determine the outcome of their effort, such as "i need to improve biochemistry" for step 1. when asked, "how do you know if you are successful," some students cannot provide an answer. therefore, we need to link the goals with means and outcomes. the goal in gmo can be either broad or focused but must be attainable with a clear and measurable outcome. compare these two goals: "i need to put more effort into reviewing the gi system" versus "i need to put more effort into reviewing gi pathology." the first goal is important, but too broad. thus, time and effort may not be spent on the mostneeded areas. the second goal is more defined and focused; it is likely the result of self-assessment and will lead to more success for the student. means refers to the resources and strategies used to accomplish the goal. a well-defined goal will help drive resource selection. an individualized approach to match the students' preferred learning style with the appropriate resource modality is critical. using the example above, students can focus on the individualized gi pathology resources rather than resources dedicated to the entire gi system. finally, thinking about outcome while deciding on the goals has two purposes. the first is to help define the goal and make it attainable. the second is to drive students to generate their next goal, as defined by the result of their self-assessment, perpetuating a focused cycle of addressing study needs. for students who were preparing for the usmle examinations, the covid-19 pandemic brought uncertainty to a process that, in normal circumstances, is already stressful. scheduling-and by consequence, rescheduling-is one of a myriad of things contributing to the uncertainty that many students are experiencing. studies have shown that stress negatively impacts cognition, including memory retrieval and updating [12, 13] . in a recent survey assessing 4th-year medical student perceptions to changes occurring to the usmle examination as a result of covid-19 at wake forest university school of medicine, 94.8% (out of 144 students, 95 responded to the survey (66%)) reported some level of stress due to rescheduling their usmle step 2 examination date. the stress students are experiencing is related to having to change their examination dates, while also having to alter their study plans, often having to extend them by weeks to months. these same feelings of anxiety can be expected for any other students who are having a delay any high-stakes testing dates and extend their study plans. as the visual aid in fig. 1 indicates, students' study plans would typically follow the blue curve, with the intensity of studying increasing until a few days before the exam date. however, many students-especially those with test dates in late march and april-are now in limbo due to examination delays and rescheduled test dates. as a result, students may feel pressured to complete an extended period of highintensity studying, which would likely contribute to increased student anxiety and increased levels of burnout and could negatively impact performance [5] . instead, we suggest students adopt a study plan similar to the yellow line in fig. 1 , which takes into account whether the prometric testing center closure lasts 30 days and its partial reopening. the study plan does this by implementing a plateau, or a period of consistent, daily studying at a lower intensity level. during this time, we recommend students to continue with the sip method to maintain their current knowledge base and patch up weaker areas. furthermore, students should consider to remain gmooriented to organize their days and facilitate intentional studying effectively. broadly, we suggest this study plan modification using the sip and gmo approaches to prevent burnout, while simultaneously allowing students to maintain-and further develop-their knowledge base. without this approach, students may struggle to find direction and lose motivation as their study period continues to extend (as demonstrated by the plateau in fig. 1 ). here, we provide examples of how to use the sip and gmo methods to create a productive study schedule during this period. with sip, we recommend that students take an initial self-assessment or use a recent assessment (s) to identify (i) areas that need to be patched (p). then, we would advise students to use the gmo method to put the sip plan in place. for example, if a student's self-assessment identified cardiac physiology as an area of weakness, the goal would be to patch up this area. to achieve this, students would review appropriate resources (means) and gauge their effectiveness (outcome) with a set of cardiology-and physiology-specific questions selected from the relevant question bank. overall, we believe that using these methods will provide direction and purpose, as well as allow students to create an effective study schedule during this challenging time. once the prometric testing center test dates are confirmed, the intensity of studying should increase as students prepare for their new exam date. during this time, we recommend that students use assessment tests to determine their next steps. if students are content with their performances but acknowledge room for improvement, we recommend they take their respective step examination as soon as the testing center is available and their schedule allows it. however, if students are not content with their performance, we would advise them to consider delaying their test date while they continue to patch up specific focus areas. burnout is broadly defined as a state of emotional exhaustion that can lead to a reduction in the perceived or actual personal accomplishment [6] . medical students and residents, compared to the general population, have a higher rate of burnout rates [14] . in a recent meta-analysis, frajerman and colleagues [15] showed that 50% of medical students-of the 17,431 students included across 24 studies-are experiencing burnout before reaching residency, with the most prevalent factor leading to burnout being emotional exhaustion. burnout has been studied extensively, with data suggesting that it is associated with increased psychiatric illness and suicidal ideation both during and beyond medical school [7, 8] . in addition, the decline of empathy levels throughout the medical school is detrimental to high-quality patient care [8] . it is essential to develop tools and support for medical students. with exam dates and study schedules uncertain, students are likely to experience emotional exhaustion leading to increased levels of burnout, which, in turn, would lead to deviation from organized study plans and less efficient studying [11] . to combat this and reduce anxiety, we suggest students apply the sip and gmo methods to take control of their learning scope, resources, and daily scheduling. these methods prevent burnout by allowing students to see their progress-big or small-in a timely fashion, thus helping to improve work satisfaction. by giving students the ability to control how small or large their goals are, repeated use of the gmo method would be especially valuable for students struggling with motivation even to start studying. when students fail to use their performance data from self-assessment tools such as question banks or practice examinations to identify knowledge gaps, they are likely to create broad and unachievable goals. for example, they may plan to review all of the biochemistry without a measurable outcome. over time, unfocused efforts may promote a vicious cycle of underperformance, misguided studying, limited improvement, and eventually, burnout. developing concrete goals is the key to avoiding burnout during a time as dynamic as a global pandemic, especially while preparing for high-stakes examinations. effective use of sip and gmo methods relies on adequate self-assessment and the development of clear, attainable goals and allows the student to select appropriate resources to patch knowledge deficits, which leads to targeted and more effective study efforts. using this method will allow students to monitor their improvement, further motivating goal-oriented studying and decreasing the likelihood of burnout by breaking a student's vicious cycle of unfocused studying. for example, if a student completed a block of 20 gastrointestinal-related questions (self-assessment) and realized they got several questions wrong regarding pancreatic enzymes (identification), they can use the resources that best suit their study strategies to address these areas of weakness (patching). by focusing solely on pancreatic enzymes and not digestion, topics become more manageable and achievable, and students remain more motivated to succeed through improving scores on question sets and other quantitative markers of improvement. in this section, we describe two approaches we would recommend during the extended study period. the first approach focuses on how to tackle weaker topics while maintaining stronger ones. the second approach discusses strategies based on the students' perceived level of satisfactory performances on their self-assessment tests. we applied cognitive load theory, which uses instructional techniques to decrease intrinsic and extraneous cognitive load to improve working memory, to medical education. we recommend simplifying the task by dividing it into two categories: a focused bucket and a maintenance bucket (fig. 2) [9] . guided by question bank performance, the lowest-performing subjects or systems should be placed in the focused bucket. to improve the knowledge in these weaker areas (goal), students should be intentional with resource selection (means) and, subsequently, should gauge the effectiveness of their efforts towards their intended goal (outcome). the maintenance bucket will contain the remaining subjects or systems that students have previously mastered. the guiding principle is to maintain current knowledge while continuing to self-assess, identify, and patch weaker areas. as a result, the topics in the focused bucket will improve and transition into the maintenance bucket, and the relatively weaker topics in the maintenance bucket will emerge and be moved into the focused bucket (fig. 2) . this approach provides a robust framework for students to reset their study plans. the next approach uses the students' self-assessment of their current performance and data analysis provided by the assessment test (e.g., comprehensive basic science self-assessment (cbssa) for usmle step 1, clinical science mastery series, national board of medical examiners subject examinations for step 2, or any recent assessment tests). often, it can be challenging to identify areas for improvement when using highly integrated question sets. here, we outline the following steps for this approach: & first, students should consider tallying their missed questions into the appropriate cells in table 1 , general guidance for a more in-depth analysis of cbssa results. this approach will assist students in identifying weaker areas. & second, students should consider dividing the missed questions into either content or test-taking errors. they could also divide missed questions into general principles versus system-specific errors. both steps aim to provide answers to individualized questions and help identify areas for improvement. for students who scored lower than their desired outcome range on assessment tests, we recommend they spend more time on the content in the focused bucket during the plateau period in fig. 1 of the proposed study modification. students should consider blending both sip and gmo methods in the process. for example, assessment tests (self-assessment) should reveal areas for improvement (identification), such as cardiac pharmacology versus just pharmacology. identification of a specific goal will aid in resource selection (means) and guide the content reviewed, and such examples may include anki decks or first aid. once the student successfully patches the knowledge deficit as evidenced by continually achieving their target range (outcome), this content may transition from the focused bucket to the maintenance bucket (shown in fig. 2 ), and overall scores should improve. when deficits have been sufficiently patched and moved to the maintenance bucket, the process can be repeated. without adopting this method may result in unachievable goals and ineffective study efforts. the goal of this method is to transfer the focused content that reaches the desirable performance into the maintenance bucket, without neglecting subjects mastered previously. once the testing date is finalized, students should increase the intensity of their studying and focus more on overall improvement until decreasing a few days before the examination to avoid possible burnout. for students who scored within their desired outcome range on assessment tests, maintenance is the objective (goal) of their study period. however, students may continue to utilize the strategy of focused versus maintenance buckets. in practice, students may do fewer blocks of questions (means) with all topics and systems selected (self-assessment), which allows students to identify relatively weaker areas to move into the focused bucket for further improvement (patching). once the weaker topics are patched and transitioned back into the maintenance bucket, overall performance should improve (outcome). as a result, the sip and gmo methods complement each other and can be applied at any level of performance. during this extraordinary and trying time, we propose the integration of the gmo and sip methods into three challenging situations that students may face: test scheduling, burnout prevention, and study approaches. these have been implemented in our usmle step 1 and 2 coaching process at wake forest school of medicine (manuscript in preparation). here, these methods have been updated with practical approaches and student input to address the uncertainties that students may encounter in preparation for the usmle examinations, the comprehensive osteopathic medical licensing examination, or any other high-stakes examinations during the covid-19 pandemic. fig. 2 the framework of continued study strategies in an extended studying period. the shaded parameter illustrates the recommended proportion of time each student should spend on the content within the respective buckets the association of usmle step 1 and step 2 ck scores with residency match specialty and location selection criteria for residency: results of a national program directors survey do usmle steps, and ite score predict the american board of internal medicine certifying exam results study behaviors and usmle step 1 performance: implications of a student self-directed parallel curriculum test anxiety and united states medical licensing examination scores burnout in medical students: a systematic review association between empathy and burnout among emergency medicine physicians characterizing changes in student empathy throughout medical school cognitive load theory: implications for medical education: amee guide no the adult learner: a mythical species application in continuing education for the health professions: chapter five of "andragogy in action stress and cognition learning and memory under stress: implications for the classroom distress among matriculating medical students relative to the general population burnout in medical students before residency: a systematic review and meta-analysis publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations conflict of interest the authors declare that they have no conflict of interest. key: cord-291526-aau0hnwx authors: ge, fenfen; zhang, di; wu, lianhai; mu, hongwei title: predicting psychological state among chinese undergraduate students in the covid-19 epidemic: a longitudinal study using a machine learning date: 2020-09-17 journal: neuropsychiatr dis treat doi: 10.2147/ndt.s262004 sha: doc_id: 291526 cord_uid: aau0hnwx background: the outbreak of the 2019 novel coronavirus disease (covid-19) not only caused physical abnormalities, but also caused psychological distress, especially for undergraduate students who are facing the pressure of academic study and work. we aimed to explore the prevalence rate of probable anxiety and probable insomnia and to find the risk factors among a longitudinal study of undergraduate students using the approach of machine learning. methods: the baseline data (t1) were collected from freshmen who underwent psychological evaluation at two months after entering the university. at t2 stage (february 10th to 13th, 2020), we used a convenience cluster sampling to assess psychological state (probable anxiety was assessed by general anxiety disorder-7 and probable insomnia was assessed by insomnia severity index-7) based on a web survey. we integrated information attained at t1 stage to predict probable anxiety and probable insomnia at t2 stage using a machine learning algorithm (xgboost). results: finally, we included 2009 students (response rate: 80.36%). the prevalence rate of probable anxiety and probable insomnia was 12.49% and 16.87%, respectively. the xgboost algorithm predicted 1954 out of 2009 students (translated into 97.3% accuracy) and 1932 out of 2009 students (translated into 96.2% accuracy) who suffered anxiety and insomnia symptoms, respectively. the most relevant variables in predicting probable anxiety included romantic relationship, suicidal ideation, sleep symptoms, and a history of anxiety symptoms. the most relevant variables in predicting probable insomnia included aggression, psychotic experiences, suicidal ideation, and romantic relationship. conclusion: risks for probable anxiety and probable insomnia among undergraduate students can be identified at an individual level by baseline data. thus, timely psychological intervention for anxiety and insomnia symptoms among undergraduate students is needed considering the above factors. the 2019 novel coronavirus disease is caused by a variety of coronavirus (sars-cov-2). in march 2020, the world health organization (who) declared covid-19 as a global pandemic. 1 the symptoms of covid-19 are usually nonspecific (eg, fever, cough, and dyspnea). 2 up to now, covid-19 is significantly contagious and no effective treatments or vaccines are available. 3 as of 19 june 2020 (10:00 am cet), over 8.45 million cases have been diagnosed globally with more than 453,000 fatalities. 4 many unprecedented strategies have been taken to cut off the spread of the virus in many countries (eg, china, england, and the united states). for example, the chinese government released some guidance for the general population to self-isolate. specifically, lockdown (eg, in wuhan), temporary closure of schools/factories, and restriction of individuals' activities isolation and quarantine measures have been effective at preventing the spread of covid-19. however, consensus has arisen about their potential mental health. 5, 6 in the severe acute respiratory syndrome epidemic, confirmed cases suffered higher stress levels, poor sleep quality, and depressed mood. 7 recently, brooks' review suggested that the mental health impact of quarantine is wide-ranging and substantial. 5 in the present epidemic of covid-19, higher anxiety level (23.04%) was reported among medical staff. 8 and zhang's study found that more than onethird of medical staff suffered insomnia symptoms. 9 in addition to special groups (eg, medical staff and confirmed cases), wang's study found that about 33.33% of the general population reported moderate-to-severe anxiety. 10 isolated people may experience stressful conditions (eg, social activities and face to face communication were restricted). 11 thus, negative emotions experienced by people may be compounded. 11, 12 undergraduate education is a special stage that is highly specialized, knowledgeable, and continuous. theory teaching and practical teaching are two profiles as well as two taches in the same teaching life-cycle. the epidemic prevented undergraduate students from returning to university. thus, undergraduate students' psychological state may fluctuate. 13, 14 we found that previous studies have some limitations. firstly, most research focuses on the mental health of medical students, and ignores the mental health of students of other disciplines (eg, humanities, engineering, and agricultural). although medical students are a special group, they only constitute a very small part of undergraduate students. secondly, most of the studies have a cross-sectional design, only exploring the psychological state and risk factors of undergraduate students during the covid-19 epidemic. in summary, we aimed to investigate the prevalence rate of probable anxiety and probable insomnia and to confirm the risk factors among undergraduate students during the covid-19 outbreak. finally, we ranked risk factors in the model based on "feature importance." we have obtained the ethics committee of the ocean university of china (2,020,001). an online version of an informed consent form is provided to students before starting the survey. in the form, we explained to the students that participation was voluntary, and refusals would have no negative consequences. we also guaranteed data confidentiality and that only the researchers could access the information. undergraduate students can choose to participate in or reject the survey. if they choose to participate in the survey, they are evaluated via an online platform. if they refuse to take part in the survey, they withdraw from the online platform. we have obtained informed consent from students who accomplished the survey. the target population of the research was undergraduate students at the ocean university of china. the ocean university of china is a government university located in shandong province. the baseline survey (t1) was collected from freshmen. specifically, all freshmen underwent psychological evaluation at two months after entering the ocean university of china. in the presence of the covid-19 epidemic (t2, february 10th to 13th, 2020), we used convenience cluster sampling and invited undergraduate students (n=2500) from four grades (freshman, sophomore, junior and senior) to participate in the survey. finally, 2009 participants completed the webbased survey (response rate: 80.36%). we used the students' id numbers to match the data. basic demographic information was collected at t1 stage. we collected basic characteristics using a self-constructed questionnaire. the questionnaire included gender (female=1, male=2), year of education (first=1, second=2, third=3, fourth=4), family economic (low family economic=1, high family economic=2), upbringing place (metropolitan=1, medium and small cities=2, town=3, country=4) and singlechild families (yes=1, no=2). the ministry of education of the people's republic of china recommended the college students mental health screening scale (csmhss) as a reliable and valid measurement tool to evaluate the mental health of undergraduate students. the scale of csmhss consists of 22 dimensions. specifically, it includes psychotic experiences (4 items), suicidal ideation (3 items), the history of anxiety symptoms (4 items), the history of depression symptoms (5 items), paranoia (4 items), inferiority (5 items), sensitivity (4 items), social phobia (4 items), somatization (4 items), dependence (4 items), aggression (4 items), impulsive (4 items), obsession and compulsion (4 items), internet addiction (5 items), self-injury (4 items), eating problems (4 items), sleeping problem (4 items), school adjustment difficulties (4 items), interpersonal distress (4 items), academic pressure (4 items), employment pressure (4 items) and romantic relationship problems (4 items). the scale uses 4-point likert-scaled items ranging from 1 (not at all like me) to 4 (very like me). the scores in each dimension are added then standardized. the standard score of the total score in each dimension can be categorized at a cutoff of 3 (fang, yuan, hu, deng, and lin, 2018). the scale of general anxiety disorder (gad-7) was used in this research. it was a tool to assess anxiety symptoms. a score ≥7 indicated clinically significant anxiety symptoms. 15,16 gad-7 was well validated and sensitive to the general population. [17] [18] [19] [20] studies had shown that gad-7 has good reliability and validity in china. 21, 22 the cronbach's alpha value was 0.90 for gad-7 in this research. we used the insomnia severity index (isi-7) which is a 7item instrument that evaluates subjective sleep symptoms. each item is scored on a 5-point scale, with higher scores representing more severe insomnia symptoms. a score >14 indicates clinically significant insomnia symptoms. 23 the cronbach's alpha value was 0.86 for isi-7 in this research. descriptive data analysis was implemented in spss 21.0 for windows. xgboost (extreme gradient boosting), which is a machine learning algorithm, was implemented in python 3.70. xgboost is a method for regression and classification problems according to the gradient boosting decision tree. this method has been widely used in all kinds of data fields for regression and classification. 24 the algorithm of xgboost can utilize a cross-validation approach to divide data into a model "training set" and "testing set." in the current research, we used a 5-fold cross-validation method. classification performance was scored with the area under the receiver-operation curve (auc), sensitivity (sen), specificity (spe), and accuracy (acc). finally, there were 2009 undergraduate students who were included in our research. of the 2009 participants, 50.97% were female, 79.99% were from high family economic background, and 25.35% were from metropolitan areas. in the present epidemic of covid-19, the prevalence rate of probable anxiety and probable insomnia symptoms was 12.49% (gad-7≥7) and 16.87% (isi-7>14), respectively. the detailed basic characteristics are shown in table 1 . we integrated the data collected at t1 stage to predict probable anxiety and probable insomnia during the covid-19 epidemic (at t2 stage). the auc of probable anxiety and probable insomnia is 99.00% and 98.00%, respectively. figure 1 shows the aucs for probable anxiety and probable insomnia. according to the aucs and the confusion matrix, we calculated the sensitivity (sen), specificity (spe), and accuracy (acc). the machine learning of xgboost predicted 1954 out of 2009 as either anxiety or no-anxiety and this translated into 97.3% accuracy (97.3% sensitivity and 96.3% specificity). the machine learning of xgboost predicted 1932 out of 2009 as either insomnia or no-insomnia and this translated into 96.2% accuracy (95.5% sensitivity and 100.0% specificity). detailed information is demonstrated in table 2 . feature importance assigned positive coefficients via xgboost, indicating that an increase in probable anxiety included 1) romantic relationship problems, 2) suicidal ideation, 3) the history of anxiety symptoms, and 4) sleep symptoms. it was also indicated that an increase in probable insomnia included 1) aggression, 2) psychotic experiences, 3) suicide ideation, and 4) romantic relationship problems. the confusion matrix and "feature importance" are shown in figures 2a and b and 3a and b, respectively. in the current research, the prevalence rate of probable anxiety and probable insomnia among undergraduate students was 12.49% and 16.87%, respectively. the prevalence rate of probable anxiety is higher than the zhang's study (7.5%) and lower than the cao's study. cao's study found that 24.9% of medical students suffered from anxiety symptoms. 25 the variability of prevalence rates could be explained by medical students being a special group who face more academic and employment pressure. and previous studies found that medical students are more likely to have psychological problems. 26, 27 the prevalence rate of probable insomnia is the similarity to huang's study (18.2%) 28 and lower than the 36.1% reported by zhang's study. 9 this variability of prevalence rates could be explained by the participants, questionnaires, and regions. most relevant variables predicting probable anxiety included romantic relationship problems, suicidal ideation, history of anxiety symptoms, and sleep symptoms. falling in love is a universal behavior among undergraduate students. studies indicate that youths experience romantic relationships of joy and happiness. however, a romantic relationship is not entirely a happy period of life. bajoghli's study found that for youths, falling in love may be also associated with anxiety symptoms. 29 consistent with asselmann's study, we found that the history of anxiety symptoms prior to/at baseline predicted a recurrence of probable anxiety at the time of follow-up. 30 narmandakh's study found that sleep disturbance may precede anxiety symptoms. and anxiety symptoms might be prevented by alleviating sleep disturbance. 31 previous results suggested that the presence of "any anxiety disorder" increases the risk for suicidal ideation among the general population, even after controlling for confounding factors (wilcox et al, 2010 ). in the current study, we found that suicidal ideation can be used to predict probable anxiety. the results may indicate that there is a bidirectional relationship between suicidal ideation and anxiety symptoms among youths. most relevant variables predicting probable insomnia included aggression, psychotic experiences, suicidal ideation, and romantic relationship problems. consistent with previous studies, we also found that insomnia is a consequence of psychotic symptoms. 32 recent studies demonstrate that insomnia also contributed to the development of psychotic symptoms. 33 insomnia symptoms may be one of the top warning signs of suicide in a clinical outpatient setting. 34 suh's study also found that insomnia symptoms were related to concurrent and future ideations of suicide in a population-based longitudinal study. 35 and a meta-analysis showed that sleep disturbances in general, as well as insomnia individually, appear to represent a risk factor for suicidal ideation and behavior. 36 namely, there may be a bidirectional association of insomnia symptoms with psychotic experiences and insomnia symptoms with suicide ideation. falling in love is an emotional occurrence at any age, but for undergraduate students, the feelings might be overwhelming. 37 in addition to being a positive feeling (eg, joy and happiness), a romantic relationship may cause stress and negative effect, especially if the feeling is not reciprocal. 38 kuula's study revealed that romantic relationship is one reason for sleep disturbance in girls and may be associated with symptoms of anxiety in both boys and girls. 39 we have found relatively reliable and accurate predictive models during the covid-19 epidemic. and our models provide useful information about the most relevant variables to predict dovepress probable anxiety and insomnia among college students. the stage of university education is an important period of life development, and it is very necessary to carry out psychological assessment of freshmen who have just entered the university. intervening with students with psychological problems in a timely and effective manner would not only help them recover their mental health, but also help them adjust their state when facing emergencies. stoessel's study found besides being positive feelings, romantic relationships may cause stress and negative effect, especially if the love is not reciprocal. 38 thus, in addition to resolving regular psychological problems, it is necessary to help college students to establish healthy romantic relationships, one of the principal developmental tasks of emerging adulthood. how to effectively organize the mental health services for those undergraduates who have present anxiety and/or insomnia symptoms due to the covid-19 pandemic is also very important. community-based and school-based mental health services care be combined into the national health system. 40 there are some strengths in this research, including 1) this is a longitudinal study and we use the data at the time of enrollment to predict college students' anxiety and insomnia during the outbreak; 2) we integrate data from multiple dimensions; 3) we calculate models for individual classification using machine learning. however, the current research has some limitations, including 1) our participants are from a specific university located in shandong province. and this university does not include medical students. thus, the results cannot be generalized to all chinese undergraduates. 2) we used self-reported questionnaires in this research, so response bias and recall bias may exist considering that undergraduate students may have underreported or overreported their anxiety and/or sleep symptoms. we took some steps to reduce this by keeping uniformity of data collection approach. it is worth mentioning that we found that romantic relationship trouble is an important factor in predicting anxiety and insomnia. 3) we used different questionnaires at t1 stage and t2 stage. thus, it is difficult to directly compare the prevalence rate of anxiety and insomnia at two stages. 4) temperament is stable across the lifespan and mediate adaptive functioning to some extent. and the attachment system may be activated in stressful situations. recently, moccia's research found that some specific affective temperament (eg, cyclothymic and anxious temperaments) and attachment features (eg, need for approval) can be used to predict the burden of mental health. 41 however, information on temperamental and attachment was not collected in our study. thus, it is necessary for researchers to consider temperament and attachment in future studies. this longitudinal research contributes to our understanding of the psychological state of undergraduate students who suffered a sudden public health event. and we found a reliable model to predict anxiety and insomnia during the sudden public health. thus, timely psychological intervention is necessary, not only to help undergraduate students recover their mental health but also to help them face some emergency events. fenfen ge and di zhang are co-first authors. the authors report no conflicts of interest in this work. neuropsychiatric disease and treatment is an international, peerreviewed journal of clinical therapeutics and pharmacology focusing on concise rapid reporting of clinical or pre-clinical studies on a range of neuropsychiatric and neurological disorders. this journal is indexed on pubmed central, the 'psycinfo' database and cas, and is the official journal of the international neuropsychiatric association (ina). the manuscript management system is completely online and includes a very quick and fair peer-review system, which is all easy to use. visit http://www.dovepress.com/testimonials.php to read real quotes from published authors. china coronavirus: who declares international emergency as death toll exceeds 200 a novel coronavirus outbreak of global health concern we signed up for this!" -student and trainee responses to the covid-19 pandemic covid-19): what you need to do the psychological impact of quarantine and how to reduce it: rapid review of the evidence the consequences of the covid-19 pandemic on mental health and implications for clinical practice factors associated with psychosis among patients with severe acute respiratory syndrome: a case-control study survey of insomnia and related social psychological factors among medical staff involved in the 2019 novel coronavirus disease outbreak immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (covid-19) epidemic among the general population in china recommended psychological crisis intervention response to the 2019 novel coronavirus pneumonia outbreak in china: a model of west china hospital psychosocial effects of an ebola outbreak at individual, community and international levels prevalence of depression and its associated factors among clinical-year medical students in eastern province, saudi arabia medical student mobilization during a crisis: lessons from a covid-19 medical student response team patient-reported outcome measures in community mental health teams: pragmatic evaluation of phq-9 a brief measure for assessing generalized anxiety disorder -the gad-7 depressive symptomatology among norwegian adolescent boys and girls: the patient health questionnaire-9 (phq-9) psychometric properties and correlates psychometric properties of the 7-item generalized anxiety disorder scale (gad-7) in a large representative sample of finnish adolescents screening for depression in primary care: a rasch analysis of the phq-9 anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection reliability and validity of the chinese version of the patient health questionnaire (phq-9) in the general population reliability and validity of chinese version of the generalized anxiety disorder 7-item (gad-7) scale in screening anxiety disorders in outpatients from traditional chinese internal department empirical validation of the insomnia severity index in cancer patients xgboost: a scalable tree boosting system the psychological impact of the covid-19 epidemic on college students in china stress and depression among medical students: a cross-sectional study systematic review of depression, anxiety, and other indicators of psychological distress among u.s. and canadian medical students mental health burden for the public affected by the covid-19 outbreak in china: who will be the high-risk group? i love you more than i can stand!" -romantic love, symptoms of depression and anxiety, and sleep complaints are related among young adults the bidirectional association between sleep problems and anxiety symptoms in adolescents: a trails report the role of sleep dysfunction in the occurrence of delusions and hallucinations: a systematic review insomnia, negative affect, and psychotic experiences: modelling pathways over time in a clinical observational study suicidality and sleep disturbances longitudinal course of depression scores with and without insomnia in non-depressed individuals: a 6-year follow-up longitudinal study in a korean cohort meta-analysis of sleep disturbance and suicidal thoughts and behaviors adolescent romantic relationships differences and similarities on neuronal activities of people being happily and unhappily in love: a functional magnetic resonance imaging study emotions relating to romantic love-further disruptors of adolescent sleep mental health during and after the covid-19 emergency in italy affective temperament, attachment style, and the psychological impact of the covid-19 outbreak: an early report on the italian general population key: cord-275074-qyre2hht authors: ortega, pilar; shin, tiffany m.; pérez-cordón, cristina; martínez, glenn a. title: virtual medical spanish education at the corazón of hispanic/latinx health during covid-19 date: 2020-08-19 journal: med sci educ doi: 10.1007/s40670-020-01058-0 sha: doc_id: 275074 cord_uid: qyre2hht nan hispanic/latinx health inequities have been exposed in the context of the coronavirus disease 2019 (covid-19) pandemic. existing health system deficiencies are at the heart and core-el corazón-of the health inequities being magnified during the current public health crisis for racial, ethnic, and linguistic minorities [1, 2] . among the most important factors that influence whether health information is understood and whether individuals can access care is language. individuals with limited english proficiency (lep) have improved health outcomes when they experience languageconcordant health care-that is, when their clinician speaks their language [3] . the lack of medical interpreter availability and increased difficulties with in-person access to care during the covid-19 health crisis have further complicated health care for linguistic minorities [4] . cities such as new york, boston, and chicago are increasingly reporting large clusters of infection rates and disproportionately high death rates in hispanic/ latinx patients [5] [6] [7] . rural hispanic/latinx groups are also at elevated risk given their significant representation in meat and poultry packing plants throughout the midwest [8] . some of the health system deficiencies that may result in a disproportionate impact of covid-19 on language minority communities include a paucity of data regarding patient language and ethnicity, limited availability of languageappropriate services, and the lack of language-concordant health-care professionals [9] . medical spanish education has been proposed as a strategy to increase the language-concordant physician workforce [9] [10] [11] . although many medical schools provide opportunities for students to enhance language skills such as medical spanish [12] , the covid-19 pandemic also has affected the landscape of medical education, causing medical schools to seek virtual educational options for students [13] . existing experiences in which students might have gained medical spanish skills or clinical exposure to minority communities, such as clinical clerkships, away electives, study-abroad opportunities, and service-learning experiences, have in many cases been canceled or postponed in light of pandemic safety restrictions. despite these restrictions, virtual learning platforms may present a new opportunity to effectively teach communication skills to care for linguistic minority populations. building a professionally multilingual physician workforce through medical spanish courses can address the languageconcordant physician deficit that affects a growing number of communities in the usa [14] . according to the most recent population estimates, the usa is home to 56.6 million individuals who identify as hispanic/latinx, representing 17.6% of the population and constituting the largest and fastestgrowing racial or ethnic minority in the usa [15] . moreover, 40 million us residents (13.3% of the population) report speaking spanish at home, and 42.6% of hispanics who speak spanish at home report speaking english less than very well [15] . the number of spanish speakers in the usa represents a 131.2% increase between 1990 and 2015 [15] , making the usa the country with the second most spanish speakers in the world (after mexico) [16] . this trend is expected to continue, exacerbating unmet challenges to health systems and clinicians who, despite legal requirements [17] and guidelines to provide linguistically and culturally appropriate services to all patients [18] , may be unprepared to do so, particularly during patient surges such as the covid-19 pandemic. existing data shows that a majority of medical students-53.2% according to 2013 electronic residency application system data-have some pre-existing spanish skills [19] , but many are being asked to use skills in patient care without appropriate training or confirmation of skills [20] . moreover, a recent analysis of the alignment of non-english languagespeaking resident physicians and the lep patient population geographic distribution demonstrates that spanish is the us language with the lowest ratio of language-concordant resident physicians to patients [21] . despite the public health need to increase the number of competent spanish-speaking health professionals, the intrinsic complexities of medical spanish education have resulted in barriers to medical spanish course implementation. for example, lack of qualified educators and lack of time in the curriculum are two of the main reported barriers to offering medical spanish education in medical schools [12] . additionally, many schools' medical spanish educational opportunities are student-led [12] , a model that challenges long-term sustainability as well as curricular standardization. importantly, concerns about unintentionally promoting medical student "false fluency" by imparting partial knowledge of medical spanish without appropriately teaching limitations and self-assessment skills may also be a reason why schools are reticent to provide formal medical spanish education [22] . virtual access to medical spanish programs can contribute to the democratization of knowledge by bridging previously identified physical and geographic barriers to medical spanish education since educators need not be locally situated. scholars in spanish language pedagogy have made a compelling case that the appropriate use of technology can increase contact with the target language, address the variety of content interests that learners bring to language learning, and augment exposure to comprehensible input [23] . a thoughtful approach to virtual medical spanish education may serve as an effective strategy to teach patient-centered communication skills for a linguistically vulnerable and growing patient population, particularly in schools in which it was not previously accessible or available. for example, some areas of the usa with the highest growth of the hispanic/latinx population are rural areas. north dakota, west virginia, and south dakota are the three states with the highest immigrant population growth between 2010 and 2016 with 48%, 41%, and 39% immigrant population growth, respectively, the majority of whom identify as hispanic/latinx [24] . at least half of the us immigrant population has lep [25] , although this may represent an underestimate due to difficulties in data capture for populations with language needs besides english [9, 14] . medical school programs located in rural environments may experience an influx of patients with lep due to immigrant population growth in rural communities, but their curricula and onsite faculty may not be prepared to provide language skills education such as medical spanish courses. additionally, considering pandemic restrictions suspending or postponing student in-person clinical experiences, opportunities to learn clinical and interpersonal skills through virtual strategies now are being sought widely and actively by medical schools both in rural and urban locations. medical schools should take advantage of the current need to adjust curricula to virtual environments as an opportunity to also ensure that the educational content equitably addresses the linguistic and cultural communication needs of the us patient population. given population trends, this would require intentional inclusion of communication skills with linguistic minority communities, particularly spanish speakers. furthermore, it is critical to acknowledge that the challenge of communication with spanish-speaking patients and other communities with lep should not be addressed by medical students and physicians alone but also should involve other health professions and incorporate the perspectives of language experts. the more flexible online environment may facilitate medical spanish collaboration among institutions and disciplines, and across health professions programs. for example, language educators and clinician experts have been called upon to collaborate in the development of medical spanish educational standards, but their work is often unintentionally siloed in their respective departments [26] . additionally, a virtual classroom in which students from various health programs, such as medicine, pharmacy, nursing, physical therapy, dentistry, and others, could collaborate to learn medical spanish has never been published but could enable medical spanish programs to more widely address the need for a linguistically and culturally prepared healthcare workforce. to understand how a medical spanish course can be transformed into a high-quality virtual educational opportunity, the overall purpose, specific learning objectives, and assessment methods of such a course must be clearly defined. historically, medical spanish courses have lacked consistent curricular structure or objectives, resulting in many variations of course content, teaching activities, student spanish proficiency pre-requisites, and learner objectives [12, 27, 28] . medical spanish in medical school settings has been defined by expert consensus as "the use of spanish in the practice of medicine for communication with patients" [27] . a standardized list of five core learner competencies for medical spanish students in medical school have been proposed and include (1) medical interviewing skills, (2) comprehension of the patient story, (3) patient-centered discussion of the assessment or diagnoses, (4) patient-centered discussion of the plan of care, and (5) self-assessment of confidence and limitations [27] . the fifth competency is a progressive lifelong learning goal for students to continually reassess strengths and weaknesses in communication skills with linguistic minority patients and has also been referred to as global linguistic competency [28] . within the patient-centered learning objectives, medical spanish courses are tasked not only to teach the linguistic aspects of communication but also to address the learner's cultural humility and cultural competency in order to understand the spanish-speaking patient's sociocultural context and health communication needs [29, 30] . teaching these communication skills goes far beyond teaching a vocabulary or terminology list or correcting grammar; in fact, while these activities form part of the background content necessary to teaching medical spanish, experts recommend that medical spanish courses focus on the patientcentered communication rather than on the technical medical vocabulary [31] . synchronous, asynchronous, and blended synchronous/asynchronous strategies can be applied to virtual teaching and learning of medical spanish skills. for example, elements of grammar and vocabulary-focused education may be effectively taught through asynchronous learning methods such as vocabulary and grammar exercises or games that aid memorization and support other cognitive processes that reinforce learning in general and language learning in particular [32] . patient-centered communication skills, however, must be taught, observed, and practiced using more interactive modalities. for instance, a flipped-classroom approach is a type of blended learning in which students are introduced to content through pre-recorded videos and texts (individual learning space), and face-to-face classroom time using video communication solutions is dedicated to interactive activities that require personal engagement such as dialogues, debates, or presentations (group learning space). this approach can be an effective way for students to review and learn material in preparation for class, which would be dedicated to applied language use and faculty-guided reinforcement. a digital voiceboard, through which students can record themselves speaking in response to prompts, is an asynchronous strategy that has been shown to provide students an opportunity to both practice and reflect on responses to patient queries during the medical interview [33] . table 1 summarizes our recommended pedagogical practices and examples for implementation in virtual medical spanish courses, as well as how these practices align with medical spanish course competencies, learning objectives, and expected performance outcomes [27] . transforming medical spanish courses from in-person to virtual learning platforms need not require recreating the curriculum for schools with pre-existing formal medical spanish courses but rather adapting the teaching methods and practices to the virtual environment. adapting courses for virtual education may present a useful opportunity to revisit the course objectives, to maximize the productivity of face-to-face sessions through high-yield clinically relevant interactive methods, and to address some potentially deficient components of the program. for example, the assessment of medical spanish linguistically and culturally appropriate communication skills is a critical element of medical spanish education, yet one that has been lacking in a majority of reported medical spanish courses [12] . for example, arranging virtual role-play pairings by level of proficiency, providing real-time constructive feedback on performance, fostering peer-to-peer reviews and assessments, and guiding students on how to effectively self-reflect on their strengths and limitations can be important ways to continually assess students' performance, confidence, and skill levels. this learner-centered assessment method should ideally not only evaluate but also lead to the progressive improvement of the students' language skills and cultural competence, and to the gradual growth in student confidence commensurate with skills acquisition. a continuous assessment through role-play activities and other interactive classroom experiences may better contribute to this goal than a single final exam as the only learner assessment methodology. moreover, standardized patient (sp) encounters, which are the recommended approach to assessing student skills at the end of a medical spanish course [28] , can be conducted through virtual solutions. the simulated telehealth encounter can be useful for assessing medical student performance in a setting that reflects the realities of current access to care under pandemic circumstances. telehealth communication presents particular challenges to clinicians and patients with lep, since it may be more difficult to interpret gestures, display empathy, or address limited health literacy using video technology [34] . during the covid-19 crisis, health systems are encountering barriers to the technological application of their prior language assistance staff, such as interpreters or patient advocates, to telehealth encounters [34, 35] . further, patients with lep have increased difficulty and decreased comfort with using technology to access health services [36] . as a result, using telehealth sp encounters as part of medical spanish courses may not only be useful for assessing student performance of medical spanish skills but may also serve to educate learners on strategies to communicate empathically and effectively pre-class assignments: students receive a video lecture, podcast, or reading assignment to complete prior to class. conversation sessions: between class virtual sessions, students are assigned to participate in pre-arranged small group virtual meetings to foster conversation focused on specific health topics in spanish. self-study exercises: students independently complete vocabulary and grammar exercises or games to cognitively reinforce new vocabulary or grammar through workbooks or online software (e.g., quizlet or anki). student-led role play: a student must prepare a case related to the class topic and will play the role of the patient while another student is tasked as the clinician interviewing the patient. digital recordings: students are assigned to prepare a video or audio recording in response to a video/audio prompt or a patient scenario as oral/aural practice using digital voiceboards. the faculty reviews and provides constructive feedback. competency 3: patient-centered discussion of the assessment or diagnoses competency 4: patient-centered discussion of the plan of care demonstrate patient-centered explanation of medical diagnoses/assessment. demonstrate patient-centered explanation of treatment/evaluation plan. orally explain findings of the medical evaluation and plan to the patient, adjusted for cultural, emotional, and literacy needs assess patient comprehension of the information provided and address knowledge gaps or concerns guided self-assessment at key intervals: students are guided on how to self-reflect on their personal strengths and limitations in medical spanish at various points throughout the course, such as pre-course, mid-course, and end-course. post-activity guided self-reflection: students respond to a self-reflection prompt following a role play or performance assessment activity, such as a standardized patient encounter. competency 5: self-assessment of confidence and limitations accurately and progressively self-assess confidence and limitations in medical spanish identify when and how to work with a medical interpreter seek a medical interpreter if/when limitations are reached effectively work with a medical interpreter with spanish-speaking patients using video technology. such an educational and assessment strategy for medical students may enhance their preparedness to provide competent language-concordant telehealth care for vulnerable populations. language is an important factor in the health inequities experienced by linguistic minority groups and currently magnified by the covid-19 pandemic. addressing language needs is at the core of health equity for linguistic minorities, and improving virtual access to high-quality medical spanish education is an important strategy to prepare medical students and other health professionals to care for vulnerable communities during the current pandemic and beyond. next steps in enhancing virtual medical spanish education and assessment should include an evaluation of medical school strategies and methodologies in transforming current medical spanish courses into virtual learning environments, collaboration among medical spanish faculty to evaluate and share virtual learning resources, and the development of a national repository of standardized virtual sp encounters and assessment methodology for evaluation and eventual certification of medical student and physician medical spanish skills. code availability not applicable. author contributions the first draft of the manuscript was written by pilar ortega, md, and all of the authors commented on previous versions of the manuscript. all of the authors read and approved the final manuscript. data availability not applicable. conflict of interest p. ortega receives author royalties from saunders (an imprint of elsevier) for a textbook. real-time feedback is provided during synchronous classroom interactive activities such as role plays; feedback may be provided by the faculty member as well as peer-to-peer feedback guided by faculty. simulated case post-assessments: a simulated telehealth encounter is conducted on a virtual platform to assess student skills, and student receives feedback from standardized patient and faculty member. competencies 1-5 evaluate the extent to which the listed course learning objectives and performance objectives have been met for each learner. improve the learner's ability to successfully communicate with spanish-speaking patients better serve spanish-speaking patients by identifying and addressing communicative gaps and limitations the disproportionate burden of covid-19 for immigrants in the covid-19 and racial/ethnic disparities a systematic review of the impact of patient-physician non-english language concordance on quality of care and outcomes when coronavirus care gets lost in translation latinos disproportionately dying, losing jobs because of the coronavirus: 'something has to change coronavirus patients who don't speak english could end up 'unable to communicate in their last moments of life illinois department of public health, covid-19 statistics trump orders meat and poultry plants to remain open despite growing threat of covid-19 infections. latino usa spanish language concordance in u.s. medical care: a multifaceted challenge and call to action increasing the supply and quality of language-concordant physicians for spanish-speaking patients teaching medical spanish to improve population health: evidence for incorporating language education and assessment in u.s. medical schools. health equity national survey of medical spanish curriculum in u.s. medical schools medical student education in the time of covid-19 language-concordant primary care physicians for a diverse population: the view from california profile america facts for features: hispanic heritage month informe del instituto cervantes improving access to services for persons with limited english proficiency. executive order 13166 national standards for culturally and linguistically appropriate services in health and health care: a blueprint for advancing and sustaining clas-policy and practice non-english-language proficiency of applicants to us residency programs medical students' experiences and perspectives on interpreting for lep patients at two us medical schools supply and demand: association between non-english languagespeaking first year resident physicians and areas of need in the usa let's not contribute to disparities: the best methods for teaching clinicians how to overcome language barriers to health care brave new digital classroom: technology and foreign language learning immigrants in the u.s. states with the fastestgrowing foreign-born populations the limited english proficient population in the united states. migration policy institute medical spanish programs in the united states: a critical review of published studies and a proposal of best practices medical spanish standardization in u.s. medical schools: consensus statement from a multidisciplinary expert panel strategies for teaching linguistic preparedness for physicians: medical spanish and global linguistic competence in undergraduate medical education beginning medical spanish: oral proficiency and cultural humility cultural humility versus cultural competence: a critical distinction in defining physician training outcomes in multicultural education words matter: translanguaging in medical communication skills training minds online: teaching effectively with technology spanish in health care: policy, practice and pedagogy in latino health addressing equity in telemedicine for chronic disease management during the covid-19 pandemic hospitals have left many covid-19 patients who don't speak english alone, confused, and without proper care the digital divide in health-related technology use: the significance of race/ethnicity publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-304619-tpv76833 authors: chatterjea, devavani title: teaching immunology as a liberal art date: 2020-07-14 journal: front immunol doi: 10.3389/fimmu.2020.01462 sha: doc_id: 304619 cord_uid: tpv76833 a complex, rapidly evolving biomedical field that is of critical relevance to human health and well-being, immunology provides important and substantive opportunities to practice and teach the central tenets of a liberal arts curriculum. it's one of those "end of semester" days in december-i am looking forward to wrapping up the term, the familiar mix of exhaustion and anticipation in my bones. the junior and senior biology majors in my immunology survey course at an undergraduate liberal arts college in the midwest are setting up their immunology-themed art presentations. a pile of "plushies"-giant stuffed fabric white blood cells decorated with their known surface markers invites tactile exploration, and an impromptu game of toss. an immune cell synapse wired with leds lights up in series as "activation" switches are flicked on. students flock to the edible displays. a towering croquembouche "lymph node" of choux pastries invites them to pull out individual ones to taste-flavored with different fillings, the pastries represent the different cells in a lymph node. as the puffs get eaten, the spun sugar matrix of the tower loses shape, much as a lymph node matrix would without resident cells. the hematopoiesis cookie table is a hit. the student who set it up explains how a basic set of ingredients is flexibly transformed into different kinds of cookies-at which points commitments to certain final products occur and when and how steps become irreversible; class-mates sample some of the finished products and take turns building cookies of different "lineages" with nuts, fruit, chocolate chips, bits of candy sparking a spontaneous discussion about food allergies, routes of exposure and safe handling practices. a student clears their throat and the hum of chatter subsides. a self-described "non-artist, " they have chosen instead to deliver a "testimony to congress" to advocate for robust funding for immunological research inspired by the advocacy of members of the american association of immunologists (1). as standin lawmakers, we listen attentively to the evidence-based arguments for the importance of basic immunology research for a healthy society. there are tough but respectful questions on animal research ethics, a plausible timeline for a universal flu vaccine and the structural inequities of access to cutting edge cancer therapies such as car-t cells. after the q&a, students read each other's artist's statements, take turns trying to sit on the fold-out monocyte chair without falling, and play with the stick and string co-stimulation maze which can only be solved with 3 manipulations in the correct sequence! over 20 years ago, i was an undergraduate in an immunology class, irresistibly drawn to the discipline despite the confounding maze of nomenclatures, the alphabet soup of transgenic tcr names and the flood of cell types and molecules that went over my head. through graduate and post-doctoral work, my love for the field endured and i realized that i wanted an undergraduate liberal arts curriculum to be the canvas for my immunology teaching and research. i don't think that, at the time, i could have foreseen a class period quite like the one i just described: students making the material their own in inventive and surprising ways, going confidently into the heart of foundational and cutting-edge concepts and using their intellectual and practical engagement with the material to connect their study of immunology with their lives. teaching and learning immunology as a liberal art together with my students has been transformative for all of us. macalester college is an urban small undergraduate liberal arts college with ∼2,000 students−30% students of color, 14% international, and 16% first-generation. biology is one of the top 5 majors. i teach an immunology survey course with laboratory for undergraduates who have taken cell biology and genetics. though the human immune system is the primary focus of the course, we study amoeba, social insects, bacteria, plants, and jawless fish to better understand the evolution of protective responses. students write multipledraft review papers with graphical abstracts, volunteer with the immune deficiency foundation, present art/performance works and write weekly reflective essays connecting their immunology learning to other parts of their academic or personal lives. my immunological methods course is embedded in my research program investigating the connections between environmental toxins, allergic responses, and chronic pain; students participate in scientific conversations and critique scaffolded by preparatory writing assignments, map metaarguments from sub-fields of published literature, cooperatively design, and execute experiments, and write a collaborative scientific paper. i use my upper-level seminar courses-neuroimmunology and cancer immunology to teach more advanced students about public communication of science. in our college's first year course program i offer semester-long immunology-themed courses: aids/influenza/malaria -ancient pathogens in a brave new world explores the persistence and reemergence of infections and inflammatory diseases in vulnerable populations around the world and bodies on fire centers on the global pandemic of inflammatory diseases. these courses do not have pre-requisites and are structured around connecting patient/physician memoirs, popular science books, and science journalism with the scientists and scientific discoveries they describe and typically ask students to explore these connections through writing, movement, and art. historian william cronon describes the essence of liberal education as "gaining the power and the wisdom, the generosity, and the freedom to connect"-through the acts of listening, reading, writing, talking, solving puzzles, seeking complex truths, seeing other perspectives, working in a community and being willing to both lead and follow in honest and imaginative ways (2) . structurally, a liberal arts education connects the natural and physical sciences, humanities, social sciences, quantitative thinking, and artistic inquiry. even as they engage deeply with methods and analyses in particular areas of study, students learn to appreciate different ways of making meaning of our world with tools from different disciplines. they learn to recognize and interrogate the societal structures and deep assumptions that drive the ways in which such bodies of knowledge are constructed within and across academic disciplines. immunology is a perfect fit for a liberal arts education. while traditional practices such as variolation and uses of immunomodulatory foods and botanical medicines have existed for thousands of years in societies around the globe, the constructs of cellular and circulating immune mechanisms have been articulated in the context of academic biomedicine only as recently as the late 1800s. and within these 200 years, paradigms have been swiftly proposed, critiqued, modified and transformed into an ever more complex and nuanced understanding of immunity (3). concepts of preservation of self over "non-self " have morphed into understandings of danger, disruption, repair, and memory embedded deep within cell lineages, epigenetic imprints and tissue architectures. mechanisms once described more bluntly as "killing pathogens" are now understood as highly regulated, selective, tunable responses to commensal and non-commensal microbes that constitute the multi-species ecosystems of multicellular hosts. while the immune system gives us critical protection for survival, virtually every global health concern from emerging infections, allergies and asthma, autoimmunity, chronic pain, and other psychiatric, cardiovascular, and metabolic imbalances are all fueled by these same mechanisms of inflammation, shifted by context to become harmful and pathological. author chimamanda ngozi adichie, in her ted talk "the danger of a single story, " warns that assuming a single story about a people leads to dangerous misconceptions, and learning to listen for the many different stories is essential for cross-cultural understanding (4) . immune responses, with their double-edged nature, provide a natural set of case studies in the importance of "many stories." immune responses demand careful contextual analyses, and to study them closely is to learn to grapple with complexity and uncertainty-an essential skill in today's rapidly changing, connected yet divergent world. another advantage of studying immunology is its immediate personal and social relevance. students only have to look at their own bodies, experiences of well-being and illnesses, and their environments for applications of what they learn. for many students, one immunology-related class might be their only sustained experience with the discipline, but the lessons they draw from it have the potential to remain relevant and useful in their lives. as a powerful example of this, i have observed my neuroimmunology students particularly resonate with learning about the role inflammation plays in mental health. students on college campuses are experiencing anxiety and depression at unprecedented levels, and managing neurological diagnoses while removed from their families and support systems (5) . understanding the roles of pathological inflammation intertwined with these mental health conditions, exploring the connections of stress, diet, and rest to these neuro-inflammatory pathways are empowering for students; appreciating the "bodily" basis of psychological challenges appears to make them seem more tractable. while these lessons do not take the place of the counseling and/or psychiatric support they or their peers need and receive, i have observed that students do find this scientific parsing of the mind-body connection to be of practical use. many immunology students are drawn to careers in biomedical research and its applications in the practices of medicine and/or public health. immunological researchdiscovery, translational, academic, clinical, industrial-and its applications in drug development, medical technologies, and public health interventions are at once scientific and social endeavors. countering anti-vaccine movements, crafting community, and government public health responses to disease outbreaks, regulating environmental toxicants in food, water, and air all contain important immunological arguments at their core. being able to understand and speak the language of immunology and tell its stories to specialist as well as general audiences so they can be truly heard is an important skill for students to practice. iteratively learning to read the often dense and technical immunological literature and synthesizing and communicating these findings in their own written and spoken words is both preparation for future work in biomedical fields and a core tenet of a liberal arts education-the importance of listening, reading, speaking, arguing, and writing. these skills are not unique to the study of immunology, but immunology offers undergraduates and their professors in a liberal arts context a rich and pragmatic field within the biomedical sciences in which to practice them. students in my courses and research laboratory write literature reviews, give talks and present posters on their research at conferences, and collaborate with me on writing papers and grant proposals for scientific audiences. however, they also write white papers and reflective essays connecting their learning in immunology to other disciplines, prepare educational materials for community organizations, teach secondary school students and mentor younger peers and, in doing so, practice translating the technical jargon of scientific communication into information that their audiences need and can use. a spacious liberal arts education makes room for multidisciplinary training, provides opportunities for immersive learning and community engagement and asks students to connect their learning to the world in different ways, giving them opportunities to make this complicated and compelling field their own. the perceived "difficulty" of immunology can be deconstructed in this more permissive, integrative environment to allow creative strategies for making meaning and finding purposeful engagement with the subject. immune systems are synergistic wholes of interconnected parts continuously stirred up by new discoveries that complicate existing knowledge and demand new ideas and interpretations; this has been so since paul ehrlich sketched his intricate visions of cells shedding neutralizing anti-toxins and butted heads with ilya metchnikoff 's cheeky but utterly prescient observation that immunity might just look like hungry amoeba out to forage (6). in the last two decades, our view of the immune system has been transformed by newly discovered innate cell subsets, the regulation of immunity by microbial and viral symbionts, the control of immune responses by metabolic switches, and the realization that all cells, not just the ones that we recognize as immune cells, participate in and regulate immune responses of multicellular organisms. this framework of synergistic interactions and multi-factorial outcomes can provide our students with maps and metaphors useful beyond immunology, for broader understandings of complex social and planetary processes. the precarious balance of protective vs. harmful immune responses is a mirror of the collateral costs of inequities, statesanctioned violence, and xenophobia in our societies. chronic inflammation and accompanying adverse health outcomes are materially correlated with lower socio-economic status, lack of access to nutritious foods, stressful living conditions and unstable access to healthcare (7) . that any immune response takes a toll on the living tissue it is trying to protect from real or imagined threats parallels the effect that xenophobic, reactive intolerance, and unregulated violence can have on a community or society at large. just as our own cells and those of our commensal symbionts maintain a collaborative understanding that we disrupt at our peril, our local and global communities are sensitive to the behavior of individuals and cooperation between the diverse populations who live in them. tolerance, balance, homeostasis, repair are technical terms with specific immunological meanings that are just as relevant to our social fabric as they are to our understanding of healthy and disease states of our bodies. and likewise, jingoistic militarized language about the immune system vanquishing pathogens can echo intolerant social rhetoric. the nuance and care required to understand and modulate immune responses and their outcomes serve as object lessons in how we speak and act as individual and collectives in social and political arenas. an immunological framework can also be applied to the relationship of humans with our planet as a whole. humaninduced climate change has driven our planet and its inhabitants to a perilous state of imbalance, with rapid rise in temperature and sea levels, catastrophic weather events, heat stress, food shortages, displacement of peoples, biodiversity loss, emerging pathogens (such as sars-cov2), and exacerbation of poverty and conflict, all of which create negative health outcomes for those who are most vulnerable and have the least access to resources. the united nations intergovernmental panel on climate change (8) advocates for immediate, massive, and collective action to mitigate this crisis if we are to survive. our students are joining their climate activist peers-greta thunberg, isra hirsi, xiye bastida, and others in climate strikes and actions to emphasize the urgency of the situation. the literal health effects of climate change are, and will be marked by inflammatory processes in our individual bodies, and sharp increases in global disease burdens; it is as if the entire planet is in a state of chronic inflammatory distress. everything is connected and what we do individually, and collectively, to our bodies and to our world comes back to us. teaching about our immune systems in integrative, socially relevant ways can help our students make meaningful connections between the content of their learning and the larger global context in which they live. in her book teaching to transgress (9), feminist author and educator bell hooks says: to educate as the practice of freedom is a way of teaching that anyone can learn. that learning process comes easiest to those of us who teach who also believe that there is an aspect of our vocation . . . is not merely to share information but to share in the intellectual and spiritual growth of our students. in information-dense, rapidly evolving fields of study, it is natural to feel overwhelmed by the responsibility to share information as accurately and comprehensively as possible before our limited time with any one group of students comes to a close. i am grateful that immunology-the beautiful, maddening, messy field that it is-keeps me humble and honest about the work i really want to do with my students and the way in which i want to do it. it resolutely refuses to be told as a "single story" and any arcane details memorized for exams are known to have modest shelf lives in any case. so with each passing year, i am challenged to re-imagine how i can best help my students be as prepared as possible to hear and understand all of the immunological stories that have not been written yet-to be able to know the workings of their future bodies and minds a little better, to understand and appreciate why a pandemic coronavirus can ravage one body it infects and leave another unscathed, to be able to use these stories to build healthy lives and communities, and make new discoveries. in her more recent book, teaching community: a pedagogy of hope (10), bell hooks says: "it is imperative that we (teachers) maintain hope even when the harshness of reality suggests otherwise." i take these words to heart. much of western biomedical science has been built around concepts of illness rather than wellness and i wonder whether it is simply too overwhelming to keep coming back to narratives and mechanisms of morbidity, dysfunction, and imbalance. here again, the spaciousness of a liberal arts framework allows both instructors and students to be more open to leavening the difficult topics with moments of beauty and fun. psychologist alison gopnik has demonstrated that children who "pretend play, " in elaborate, unreal scenarios with the aid of language, props and gestures, are able to respond correctly to counterfactual questions about a novel real-world causal relationship (11) . while the evolutionary imperative for play may well be to develop robust cognitive functions, children play because it is a lot of fun. the paradox of play is that in order to be able to reach a variety of practical benefits in the long run, one must be somewhat less concerned with immediate accomplishments of goals in the short run. eating a cardamom and orange cream-filled choux bun pulled out of a patisserie "lymph node" might not immediately seem central to learning about immune systems but it is delicious and it distills the joy of learning and sharing in a way that sticks in our brains and hearts-both my students' and mine. a liberal arts education with its emphasis on connective and integrative inquiry aims to be transformative, to crack the world open a little bit wider for every student with every course of study, every class, every discipline. but it is not only the student who is transformed, it is also the teacher. teaching immunology as a liberal art has made me a more curious, capable and happier immunologist than i had known i could be. the original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author. the goals of a liberal education immunology's coming of age the danger of a single story who world mental health surveys international college student project: prevalence and distribution of mental disorders microbe hunters reissued socioeconomic status and inflammation: a meta-analysis the intergovernmental panel on climate change. global warming of 1.5 0 c teaching to transgress: education as the practice of freedom teaching community: a pedagogy of hope the power of possibility: causal learning, counterfactual reasoning and pretend play the author confirms being the sole contributor of this work and has approved it for publication. i thank my students at macalester college for being my collaborators on this journey since 2006, and my fellow immunologists at liberal arts institutions for their inspiring conversations and support. the author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.copyright © 2020 chatterjea. this is an open-access article distributed under the terms of the creative commons attribution license (cc by). the use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. no use, distribution or reproduction is permitted which does not comply with these terms. key: cord-337350-ysy7vru2 authors: lópez-carril, samuel; anagnostopoulos, christos; parganas, petros title: social media in sport management education: introducing linkedin date: 2020-08-15 journal: j hosp leis sport tour educ doi: 10.1016/j.jhlste.2020.100262 sha: doc_id: 337350 cord_uid: ysy7vru2 social media provide innovative teaching and learning pedagogical frameworks that change means of communication within academic institutions and enable students to develop digital skills that are helpful for a successful professional career. linkedin, a social media tool that focuses on professional networking and career development, has become the most popular professional social network, used by all stakeholders of the sport industry, and can therefore be used by students to stay in touch with experts and the latest trends in the sports industry. the purpose of this article is to define the main features and functionalities of linkedin from a sport management perspective and share guidelines to embrace and introduce it effectively into sport management courses. technology has opened the path for many educational reforms and innovations (tayag, 2020) . for instance, the internet has reshaped the way students, teachers and universities interact (scott & stanway, 2015) , enabling the emergence of new learning models, such as online learning, which, since the beginning of the century, started to experience high growth (goodyear, salmon, spector, steeples, & tickner, 2001) . the expansion of these new pedagogies in the educational fieldwhich include the use of digital technologies and new communication channels, such as social mediaexperienced a boom during the covid-19 pandemic that has impacted the whole educational community. social media platforms and applications have become widely popular across all industries, for a variety of purposes (malik, heyman-schrum, & johri, 2019) . in university environment, however, integration of social media either in the teaching-learning process or as a communication tool to connect with stakeholders, happens rather slowly compared to the wider industry benchmarks (o'boyle, 2014) . this can be mainly attributed to the fact that social media platforms were not designed with the explicit purpose of supporting educational or other learning-related activities (tess, 2013) . furthermore, faculty are naturally reticent to change their teaching and learning methods, and there is often an issue regarding why, how, and what impact the pedagogical change may have concerning the quality of the teaching-learning process and its resulting benefits (salmon, 2005) . therefore, current research has shown that the educational possibilities of the social media are still only partially exploited (manca, 2020) . however, the inherent characteristics of social media in terms of networking and content sharing have been noticed by the academic community, and their use in the educational landscape is also on the rise (malik et al., 2019) . indeed, there is a need and desire to rethink models for teaching and learning and introduce pedagogical innovations in the classroom (manning, keiper, & jenny, 2017) . pedagogical innovation can be viewed as the attitude and process of investigating new ideas, conceptions, proposals, and contributions (imbernón, 1996) , aiming to transform and improve the teaching and learning processes. in this sense, social media offer the opportunity to create learning contexts as well as positive and engaged educational experiences for students (manning et al., 2017) . social media can offer a number of advantages to students, such as providing meaningful experiences related to strategic social media use that will be helpful after graduation (sanderson & browning, 2015) , preparing them for impactful networking in their professional career or business purposes (tess, 2013) and helping them to connect with professionals and follow new trends in the sector (marr & dewaele, 2015) . with particular regard to networking and job-related information seeking, one social media tool -linkedinseems to be an ideal fit. linkedin focuses on professional networking and career development. unlike most social networks, linkedin has been designed to help people make business connections, share their experiences and resumes, and find jobs (linkedin, 2019). with over 650 million users spread across the world, linkedin is the most important channel to distribute business-related information and the preferred tool for marketing and management executives to find relevant quality content. at the time of writing, over 30 million companies are listed on linkedin and over 20 million jobs are offered (linkedin, 2019). due to such peculiar characteristics, linkedin fits perfectly in the sport management course dynamic as it fosters the connections between the university and the professional industry, being a first step for the development of students' professional careers. despite the huge industry interest, there is a clear need for academic literature with a specific focus on linkedin. within the sport management education literature, with the exception of the work of brown and pederson (2019) , twitter is the social media that has generated the most interest (e.g., brown & pederson, 2019; feito & brown, 2018; manning et al., 2017; marr & dewaele, 2015; o'boyle, 2014; sanderson & browning, 2015) . while these studies have focused mainly on providing conceptual frameworks as well as rules and benefits on how and why to use social media in sport management courses, researchers have been asking for additional social media research to establish methodological standards and best practices, as well as with regard to pedagogical experiences (lebel, danylchuk, & millar, 2015; mayr & weller, 2017) . therefore, the objective of the current paper is twofold: a) to provide sport management lecturers with guidelines and working methodologies to successfully introduce linkedin in sport management courses, while inspiring them to develop new pedagogical innovations in the field. b) to strengthen the connection between the academic environment and the professional industry using linkedin as a first step towards the development of student's professional careers. a pedagogical innovation through linkedin should ideally start with an introduction to the history and evolution of linkedin, providing current statistics of users, describing the main functions that the platform offers, emphasizing similarities and differences with other social media and selectively showing real examples of usage by various stakeholders of the sport industry (clubs, federations, sport brands, athletes, etc.). the next step for students would then be to create a linkedin account and learn how to complete the basic functionalities that configure their linkedin's user profile: the profile is probably the most important section for any linkedin user, as it is the image that they project towards the wider linkedin community. a linkedin profile should be updated regularly and in line with the advances in the student's professional career. however, in order to develop a harmonious and coherent profile that will help the students achieve their professional objective and before starting to complete their profile, students should reflect on the following points: 1. academic and professional achievements 2. hard and soft skills 3. field/s of interest within the sport industry (such as sports marketing, coaching, sports psychology, sports facilities management, etc.) 4. job aspirations 5. what they can already offer the linkedin community answering the above points will also allow students to design a content strategy that develops a strong personal brand that draws the attention of other linkedin users and related stakeholders. once the profile is developed, students will be ready to start interacting with the linkedin community. however, given the characteristics of the tool, students should first ask themselves the following question, prior to making a public action in linkedin: do my actions provide value to the community? if the answer is "no", it is advisable not to publish anything to avoid doing any harm to the individual's personal brand. a positive response to this question leads to the considerations of content and network creation, which are described next. students have to know that they should be active in linkedin if they want to become known and gain higher status in their field of interest. there are different types of content a student can publish. in the sport management context and for the purpose of this article, we focus on those that are most relevant and most easily realizable by students (table 1 ). creating a strong network of contacts is one of the main objectives of linkedin as this is the core of the professional relationships with different stakeholders. sport management faculty should help students identify their target users according to their professional goals. students should also be aware that, according to linkedin's contact system, their professional contact network will be composed of: • first-level contacts: the core of the linkedin network. these are the contacts that accepted the student's invitations to connect or those who sent them invitations that the students have accepted. • second-level contacts: the first-level contacts of the students' first-level contacts automatically the students' second-level contacts. content themes that sport management students could develop for their profile content strategy. linkedin is a suitable platform to share ideas, advice, or guidelines related to with a field of interest. thus, one of the basic types of content can be related to topics directly linked to the sport industry, shared in an easy and understandable way (quotes, a brief overview of a topic, etc.). in an increasingly over-informed society, it is valuable to be able to filter the highest quality content and share it with our network of contacts, adding a personal view about the selected topic. it is this personal view that will add extra value to the post. people like to know the latest trends in the sport industry. being on top of such content is well appreciated by the linkedin's community, proving that the student is proactive and up-to-date with advances in the industry. creating "how to…" video tutorials students can make "how to" videos related to the sports industry. these should be clear and short in order to engage quickly with the audience. debates students could demonstrate their knowledge and ability to communicate their views about a specific topic (even controversial ones), leading to a debate in the online community event conclusions or outcomes this include sharing personal conclusions after attending diverse kind of events (conferences, course lessons, workshops, sport matches, etc.) related to sport. this will demonstrate their learning skills as well as their capacity for synthesis. this type of content consists of posting information regarding a business relationship that students experience with a sport entity or with a linkedin individual contact. this enables them to demonstrate their professional experience with success, validating it in public. students can share information regarding a job opening or internship position related to their field of interest and thereby gain value by benefiting others in their online community. a very popular content type in linkedin as it captures the interest of professionals and scholars in their respective field of interest while their shareability positively impacts the visibility of the students' profile. linkedin is a perfect window to showcase professional skills looking for potential clients. students should be aware of this and exploit it during the student stage by disseminating news on product or service launches. • third-level contacts: first-level contacts of the students' second-level contacts automatically the students' third-level contacts students should strategically send invitations to users that add value to their network, considering linkedin's contacts structure. thus, both quantity and quality are important. quantity is an important factor because the larger the network of first contacts, the larger the network of second-and third-level contacts. quality is also important because having influential people or field experts at the first level of contacts will add to the network of second-and third-level contacts people who are equally interesting to their professional field. in this sense, students should focus on adding contacts specialized in their professional fields as they will generate content that will probably be of more interest to them compared with the content that will probably generate a person that works in a completely different sector. at this point, it is essential to advise students not to send a blank connection request to linkedin users they want to connect with, as there will be more chances of being ignored. the advisable idea is to teach them how to create attractive and personalised introductory messages that will be appealing to the potential new contact, providing some justification/context regarding why they are asking to connect to that particular sport industry professional. by proceeding in this way, students will have more options to connect with industry professionals who may be unfamiliar at first. in addition to three levels of contacts, there are also group contacts. a linkedin private group is a group of linkedin users who gather to discuss, share, debate, inform, etc., about a specific topic. the topic could be broad, but the aim is specified in the description of the group as well a set of rules that all the group users must follow. it is important that students identify and aim to belong to groups of a thematic interest to them, as they will then find users who probably belong to a specific professional niche; that is, people worth networking with. for example, for a student interested in fitness centers, it is advisable to identify a group of fitness center owners or fitness trainers, as both will add value to the student's education and networking. apart from sending and receiving contact requests, linkedin offers the option to follow profiles. by following another user, the content produced by that user can be accessed, but not vice-versa. to follow another user, the "follow" option must be clicked in the user's respective profile. this option is very interesting for students because it allows them to access content of even high-profile people who might not otherwise accept a student's contact request. the final step should then be to focus on the creation of a strategy for short, medium, and long-term growth in linkedin. faculty should drive this conversation by giving advice in person, thorough private messages in linkedin, or by making video tutorials explaining more features and possibilities not detailed in this article. as stated before, linkedin allows users to create private groups. a classroom-specific linkedin group can be created that will be a meeting point for developing a pedagogical innovation. students should join the group while the group rules can be set by the sport management faculty or commonly agreed with the class. the following usage possibilities exist: 1. become a more informal medium (compared to email or moodle) to reach and interact with students and make class announcements. 2. publish different type of content by faculty members (see table 1 ) deciding on the timing of publications and adapting it to the course objectives (for example, sharing class material, facilitating resources to expand what is taught in class, create debates, etc.). 3. publish different type of content by students. this allows students to show creativity and take ownership and responsibility. however, especially in larger groups, it is important to clarify a set of rules in order to avoid creating too many posts, which prevents content tracking. in short, the linkedin class group can play a central role of a linkedin pedagogical innovation and the corresponding assignment, as faculty could launch activities that will be assessed, as described next. faculty members should think about the working objectives they want to develop with an educational innovation that uses linkedin as a pedagogical resource rather than introducing linkedin just because it is fun or topical. for the practical proposal described below, the two main objectives are to assist students in: a) developing their personal brand and identifying people and stakeholders related with their desired professional career; and b) creating valuable content that is in line with the course syllabus. the assignment, written specifically in this article addressed to students, is designed to be carried out over a period of roughly three months. in terms of grading, the proposal is divided into two blocks: a. compulsory work, graded with seven out of 10 possible points, consisting of: 1. development of basic aspects of your linkedin profile. note that your entire profile must be in harmony with your professional interest. 1.1. upload a profile and cover photo. 1.2. develop the headline and the "about" (summary). complete the experience, education and licenses & certifications add at least five personal or professional skills & endorsements that you consider you have aspects related to the development of your professional network. 2.1. add 20 classmates and five lecturers to your network. 2.2. endorse the skills to at least five classmates or lecturers (students must only endorse for skills they are absolutely sure the other person possesses) join the linkedin class private group find and join three linkedin groups related to your professional interest connect/follow at least five experts related to your professional interest. 2.6. follow at least five linkedin business profiles of stakeholders of your desired professional field (companies, clubs, federations activity in the linkedin classroom group. 3.1. participate in at least seven activities created by the faculty non-compulsory work, consisting of six tasks and graded with three out of 10 possible points create and publish in public (not in the class group) at least three posts made by you. it is not enough to copy and paste a link to some material (video, news, etc.) or share a publication already created by others linkedin to classroom community: assessing classroom community on the basis of social media usage a practical approach to incorporating twitter in a college course competences for online teaching: a special report en busca del discurso educativo: la escuela, la innovación educativa, el currículum, el maestro y su formación social media as a learning tool: sport management faculty perceptions of digital pedagogies about linkedin use of twitter across educational settings: a review of the literature snapping, pinning, liking or texting: investigating social media in higher education beyond facebook. internet and higher education pedagogical innovations for the millennial sport management student: socrative and twitter incorporating twitter within the sport management classroom: rules and uses for effective practical application think before you collect: setting up a data collection approach for social media studies mobilising social media in sport management education flying not flapping: a strategic framework for e-learning and pedagogical innovation in higher education institutions from the physical to the social: twitter as a pedagogical innovation in the sport communication and sport management classroom tweeting the lecture. how social media can increase student engagement in higher education pedagogical support for blended learning classrooms: interfacing teacher and student perspectives the role of social media in higher education classes (real and virtual) -a literature review the first author of this work is grateful to the support of a predoctoral contract "fpu15/05670" granted by the spanish ministry of science, innovation and universities. supplementary data to this article can be found online at https://doi.org/10.1016/j.jhlste.2020.100262.s. lópez-carril et al. journal of hospitality, leisure, sport & tourism education 27 (2020) 100262 all tasks should be easily verifiable and clearly assessed by the faculty. the only qualitative aspect is point 6 of voluntary block, where teachers make a global qualitative evaluation of the work done by the students, rewarding factors such as having achieved true harmony between the corresponding field of interest and their developed profile and content aspects. in addition, aspects such as participation and interaction with other students/teachers, the quality of contributions (post or comments), or providing additional input than required by the proposal (such as attaching a cv to the linkedin summary, having a multilingual profile, etc.) can also be assessed.the above proposal is only one example of a possible assignment. faculty members should adapt their proposal to the specific context of their course, adjusting it to the main theme of the course (sport marketing, sport business, management of sport facilities, etc.) as well as to the characteristics of the students (age, number, level of knowledge, etc.). the higher education sector should be aware of the industry shifts and popular culture trends, and sport management faculty should train students in media literacy to enable them to be better qualified to pursue their professional aspirations. social media has a relevant role in the current sports industry and are increasingly popular among students. linkedin has a lot of potential and educational value due its own special characteristics, which enhance the development of students' professional academic career, entrepreneurship, and innovation. in addition, linkedin facilitates the development of the sport management curriculum from an engaging and innovative approach. although concerns about the social media introduction in sport management pedagogies could discourage faculty from conducting new pedagogical experiences based on social media (marr & dewaele, 2015) , the benefits and learning opportunities discussed herein outweigh any possible limitations. samuel lópez-carril: conceptualization, writing -original draft. christos anagnostopoulos: supervision. petros parganas: writing -review & editing. key: cord-287470-otsts5c9 authors: kim, so mi; park, seok gun; jee, young koo; song, il han title: perception and attitudes of medical students on clinical clerkship in the era of the coronavirus disease 2019 pandemic date: 2020-08-25 journal: medical education online doi: 10.1080/10872981.2020.1809929 sha: doc_id: 287470 cord_uid: otsts5c9 background: the coronavirus disease 2019 (covid-19) has been placing severe strain on global healthcare systems and medical education programs, leading to growing demands for medical students to assume the role of preliminary healthcare providers. objectives: to assess the perception and attitudes of medical students about clinical clerkship training during the covid-19 pandemic. design: a cross-sectional survey with web-based 3-fields/14-items questionnaire was conducted, from april 7 to 14, 2020, to evaluate their self-assessed perception and attitudes on clerkship training of hospital practice under the covid-19 outbreak and spread among 161 (78 on pre-clerkship course, 83 on clinical clerkship course) medical students at dankook university college of medicine, cheonan, republic of korea. results: of the 151 medical students who completed the survey, 81 students (53.7%) considered themselves familiar with covid-19. although the students were concerned about the spread of the virus during clinical clerkship training, 118 (78.1%) students preferred the clerkship training in a hospital practice. the students in the clinical clerkship program preferred this over those in the pre-clerkship program (85.7% vs. 70.2%, p = 0.03), primarily because a clinical clerkship could not be replaced by an online class during the covid-19 pandemic. in addition, their responses indicated, in order of significance, fear of not completing the clerkship course on time, willingness to participate as a preliminary healthcare provider in pandemic, the potential waste of tuition, and belief that a hospital is rather safe. the change in the academic calendar had not a positive impact on the lifestyles of many students. conclusions: in circumstances such as the covid-19 pandemic, educational strategies to clinical clerkship training for medical students should be developed to provide them with the opportunity to be actively involved in hospital practice under strict safety guidance focused on preventing virus infection and transmission. the coronavirus disease 2019 (covid-19) is currently placing huge pressure on the medical community around the world. since the outbreak of this novel infectious disease by severe acute respiratory syndromecoronavirus-2 (sars-cov-2) in wuhan, china, in late 2019, the status of the disease has changed from 'epidemic' to 'pandemic,' putting countries on high alert to this menace to their national public health [1] [2] [3] . many countries have been devoting all available resources to providing medical healthcare services for the treatment of covid-19-positive individuals, preventing the spread of the virus and community infection, and enacting quarantine-based official restrictions on social activity [4] . however, despite countries' intensive effort to install these healthcare measures, it is unrealistic to expect a reduction in the virus itself or a reduction in the increased healthcare burden until the emergence of an approved antiviral treatment regimen and the development of vaccine against covid-19 [5, 6] . where this pandemic is likely to continue, countries' healthcare systems may struggle to cope owing to a shortage of healthcare personnel, medical facilities, and/or equipment. recently, the availability of hospital-based roles for medical students during the covid-19 pandemic has been recognized as an important issue [7] [8] [9] . rescheduling the medical academic calendar has an important bearing on the current status of the covid-19 pandemic. the korean association of medical colleges (kamc) has strongly recommended the postponement of the start of the educational calendar for several weeks, and the transition to remote learning with online lectures over direct clerkship training within a hospital, to all medical colleges. the association of american medical colleges (aamc) has also instructed medical schools to suspend student clerkships [10] [11] [12] . due to the distinctive characteristics of the medical college curriculum, kamc subsequently stated that medical colleges could reschedule and become more flexible to enable the education of students engaged in clerkship programs at training hospitals while adhering to safety guidance regarding covid-19 infection and transmission. most medical colleges have followed the kamc's guidance [13] . these recommendations seem to be reasonable, as they place the priority on student safety during the covid-19 outbreak; however, it is considered that they do not sufficiently reflect medical students' own opinions about clinical clerkship programs. it is important to understand the concerns and attitudes of medical students surrounding clerkship training in a hospital setting under circumstances such as the covid-19 pandemic. in addition, educational policies such as the development of alternative clerkship programs, new modes of student-patient interaction, and the re-establishment of medical students' roles are vital issues to consider. this study aimed to assess the perception and attitudes of medical students about these changes to clinical clerkship training programs in south korea during the covid-19 pandemic. we enrolled a total of 161 medical students at dankook university college of medicine in this web-based, cross-sectional survey. the students represented four levels of training: 41 were in the first year of medical college, 37 in the second year, 44 in the third year, and 39 students were in the fourth year. due to the outbreak of covid-19, the first day of the new semester was postponed for a few weeks, so it began later than was originally scheduled on the academic calendar. the dankook university authorities deemed it necessary to replace all face-to-face classes with online classes for students in the pre-clerkship stage (first year and second year); clinical clerkship training proceeded as usual for students in the clerkship stage (third year and fourth year). we conducted this survey over 2 weeks between april 7 and 21, 2020, during the 2 weeks following the start of the delayed semester. due to the restrictions imposed by the covid-19 pandemic, face-to-face surveys were not possible. hence, all students were contacted by phone from the administrative team of dankook university college of medicine and asked to participate using an online google docs survey. the link for the survey was informed by a text message. the principles of voluntary participation, no harm to the participants, anonymity and confidential assurances were announced at the beginning of the questionnaire. we developed the questionnaires to evaluate medical students' thoughts on clinical clerkship training during the covid-19 pandemic from the viewpoints of both preliminary healthcare providers and university students. we also prepared both korean-and english-language versions of the questionnaire to enable the inclusion of international students in the survey, and conducted a pilot testing among six students: three pre-clerkship and three clinical clerkship students, with each group including one international student to reduce the likelihood of questions that could lead to misunderstandings or biased answers from participants. the study questionnaires were composed of the following three fields: 1) students' knowledge and perception of covid-19 (three questions); 2) students' attitudes toward clinical clerkship training during the covid-19 pandemic (six questions); and 3) the impact of the change in the academic calendar due to the covid-19 pandemic on students' routines and lifestyles (four questions). enrolled students completely answered the questions addressing their own knowledge of covid-19 based on self-assessment, and the questions about risk perception on the likelihood of contracting the novel coronavirus during the clinical clerkship training. the questions addressing students' attitudes toward clinical clerkship training during the covid-19 pandemic measured their willingness to participate and concerns regarding qualitative changes in the clinical clerkship. furthermore, we evaluated their answers to questions about the impact of the changes to the academic calendar on students' lifestyles by considering both the magnitude and significance of these lifestyle changes and the effectiveness of online classes at home. most of the questionnaires consisted of closed-ended questions to facilitate categorization of the participants' answers. dichotomous questions were scored using a two-point scale, and questions with more than two options were scored based on a five-point likert scale. open-ended questions were also used to invite a variety of opinions from participants. the present study was approved by the institutional review board of dankook university hospital (dkuh 2020-04-054-he001). we displayed all data in raw numbers with percentages and estimated the mean and standard deviation values. we analyzed, between two groups, continuous variables by the unpaired student-t-test and categorical variables by the χ2 test or fisher's exact test. all analyses were performed using spss version 25 software for windows (spss inc., chicago, il, usa). statistical significance was determined at p < 0.05. among the 161 medical students, 151 (93.8%) completed the survey. the baseline characteristics of participants are summarized in table 1 . the 151 medical students who completed the survey were divided into two groups depending on whether they had entered the clinical clerkship course. thus, there were 74 students in the pre-clerkship group (first and second years) and 77 in the clinical clerkship group (third and fourth years). there were no differences in gender or ethnicity between the two groups; the mean age was higher in the clinical clerkship than in the pre-clerkship group (24.3 ± 1.3 vs. 22.2 ± 1.7 years, p = 0.001). more than half of these medical students (53.7%) believed that their knowledge of covid-19 was 'above average' or 'far above average'; there was little difference between the pre-clerkship and clinical clerkship groups (48.7% vs. 58.4%). during the clerkship training, students' perception of the sars-cov -2 transmissibility was higher in the pre-clerkship than in the clerkship group, but not significantly so (transmission possible from patients to students, 60.8% vs. 48.1%, p = 0.086; transmission possible from students to patients, 51.3% vs. 50.6%, p = 0.129) ( table 1) . the students' attitudes toward clinical clerkship training during the covid-19 pandemic are summarized in table 2 . twenty-seven (17.9%) students were not satisfied with the postponement of the academic calendar due to the covid-19 pandemic. although the students were concerned that the virus could spread during clinical clerkship training, 118 (78.1%) students believed that they should take the clerkship training in hospital practice, nonetheless. this preference was greater among students in the clinical clerkship course than those in the pre-clerkship course (85.7% vs. 70.2%, p = 0.03). of particular note, the willingness to participate in the clinical clerkship course was stronger in students in the third year than those in the fourth year ( figure 1 ). students felt they should take the clerkship training during the covid-19 pandemic for the following reasons, in order of significance: concerns that an online class could not replace the clinical clerkship course (86.5% of students on pre-clerkship vs. 92.4% of students on clerkship); a fear of not completing the clerkship course on time (69.2% vs. 72.5%, respectively); a willingness to participate as a preliminary healthcare provider (34.6% vs. 48.5%, respectively); the risk of wasting tuition (36.5% vs. 37.8%, respectively), and the belief that a hospital is rather safe (19.2% vs. 16.6%, respectively). the differences between the two groups were not statistically significant (p = 0.441). in contrast, where students in the pre-clerkship and clinical clerkship groups were reluctant to participate in the clinical clerkship, the reasons were as follows: a need to follow national policies such as social distancing (81.8% vs. 63.6%, respectively); a fear of exposure to sars-cov -2 (63.6% vs. 72.7%, respectively); concerns about the deterioration of the clinical clerkship due to the spread of covid-19 (40.9% vs. 63.6%, respectively); and a trust in online classes as a viable alternative to the clinical clerkship (4.5% vs. 36.3%, respectively). these differences were statistically significant (p = 0.041) ( figure 2 ). approximately one-third of students thought the quality of clinical clerkship training would be reduced due to limitations imposed on various clinical experiences by the covid-19 pandemic. more than two thirds (69.5%) of the 151 medical students surveyed believed that the postponed academic calendar had a significant impact on their way of life. this belief was stronger, but not significantly so, for (table 3 ). in this cross-sectional analysis, most medical students, especially third-year students, despite the covid-19 pandemic, showed a commitment to clinical clerkship training with the thought that it can not be replaced with any class and that they have to participate as a preliminary healthcare provider. in the past, such as during the outbreaks of severe acute respiratory syndrome (sars) caused by the sarsassociated coronavirus in 2003 and of middle east respiratory syndrome (mers) caused by mers coronavirus in 2015, measures of such changes to the academic schedule or partial school closures were implemented, but these were temporary and localized note: * p-value was calculated between the students on pre-clerkship course and on clinical clerkship course. covid-19: coronavirus disease 2019, qc: questionnaire c. [14, 15] . the current global outbreak of covid-19, however, is even more serious, disrupting educational systems worldwide [16, 17] . as the pandemic period of covid-19 has continued, university authorities have postponed academic calendars and recommended alternative education programs. however, these policies, such as those implemented by the national ministry of education, are focused on students' safety and the prevention of covid-19 transmission, and they fail to consider the specific qualities of medical education courses and the attitudes of students toward clinical clerkship. therefore, the results of this study may suggest factors that university authorities should prioritize when making policy decisions for medical education in hospital practice under these circumstances. in this survey, the majority of medical students showed a positive attitude toward hospital practices at this time, despite concerns that clerkship training during the covid-19 pandemic could spread the virus. this demonstrates how important it is for students to complete the educational curriculum in a timely manner, as clinical clerkship training is irreplaceable. additionally, it was possible to reaffirm their identities as doctors through their intention to take part in the covid-19 crisis as a preliminary healthcare provider, not just as a student. it is particularly interesting to note that third-year students were most likely to express an interest in participating in clinical clerkship training, reaching 90%. in the korean medical education curriculum, the third year is an introductory period for hospital practice through the clinical clerkship course. it is a difficult time for students in training due to the high intensity of coursework in major subjects, including internal medicine, general surgery, pediatrics, psychiatry, and obstetrics [18] . therefore, it is considered that curiosity about introductory clinical practice, along with the pressure of the extensive practical education, influenced the higher preference of third-year medical students to participate in clinical clerkship training compared to other-year students. although a minority of students were reluctant to participate in clinical clerkship training, this was primarily due to their belief that clerkship training during the covid-19 outbreak was against national policies such as social distancing, rather than a fear of exposure to sars-cov-2 or a concern about the deterioration of clinical clerkship. these responses show that students support the current korean national policy to prevent the spread of covid-19 and take this view as citizens, not as medical students. furthermore, they might feel guilty or confused about participating in clinical clerkship, as it seems to conflict with national policies that are being applied to other non-medical students to assure that they do not attend classes in person. therefore, these results suggest that university authorities should provide sufficient explanation of the purpose and necessity of the clinical clerkship and ensure that students understand and are able to voice their opinions. they should also ensure that students are not placed in a situation of emotional conflict between the desire to do their clinical clerkship and the requirements of national policy. during the covid-19 pandemic, concerns about sars-cov-2 transmission from patients to students on clerkship and the reduced quality of clinical clerkship training appeared to be greater in the pre-clerkship group than in the clinical clerkship group. these results are likely due to the fact that students on the preclerkship course have had no opportunity to experience various clinical strategies for infection prevention and control within the hospital. however, the expectation of experiencing various clinical practices through the clinical clerkship during the covid-19 pandemic was lower in the clinical clerkship group compared to preclerkship group. students seemed to feel that restrictions on access to patients and on the use of hospital facilities or equipment resulting from changes in the hospital system designed to prevent the spread of sars-cov-2 were not suitable for proper education of students. therefore, it is necessary to develop revised clinical clerkship programs that allow students to be involved in various medical scenarios that can still be experienced during the covid-19 pandemic. in addition, these policies should encourage active participation in clinical clerkship training under the strict control of strengthened safety systems. in this study, a majority of medical students responded that their lifestyles were negatively affected by the postponement of the academic calendar due to covid-19. with regard to the significance of that impact, students' routine lifestyle was influenced more positively in the pre-clerkship group than the clinical clerkship group. the results demonstrate that although all face-to-face classes were replaced with online classes at home for the pre-clerkship group, the changes to the academic schedule did not have a great influence on their day-to-day routine. conversely, even if clerkship training proceeded as usual, there would be many restrictions on social activities outside of training due to covid-19. if the current academic schedule were to be replaced with online classes, students generally felt that their ability to participate and understand the classes would be poor. however, previous studies have shown that online classes were as useful, effective, and even as enjoyable as traditional didactics [19, 20] . therefore, in preparation for situations such as the covid-19 pandemic, it is necessary to develop various learning techniques to replace face-to-face classes that can maintain and encourage students' participation and understanding during non-face-to-face education. this study has several limitations. first, as a survey study, it depended on participants' own subjective, rather than objective, views through self-assessment and self-reporting. however, on the other hand, it also demonstrated participants' willingness and ability to be involved during these difficult circumstances. second, this survey was conducted following the university's decision regarding the clinical clerkship training of medical students; thus, there may have been a timing bias as students expressed their response to the hospital practice. however, this may also provide an opportunity to record their feelings while doing their own hospital practice. in addition, given that medical students taking this survey were selected from one university and the number of participants was relatively small, the generalizability of these findings to other medical students in korea is limited. in conclusion, in the era of the covid-19 pandemic, most medical students have a feeling of confidence in their understanding of covid19 and are concerned about its transmission to both themselves and patients during clinical clerkship training in hospitals. nevertheless, they feel that clinical clerkship is necessary due to the difficulty of replacing hospital clerkships with online lectures, the need for timely achievement of academic goals with minimal disruption, and the desire for participation as required in clinical fields as a preliminary healthcare provider. educational strategies to clinical clerkship training for medical students should be developed that provide the opportunity for active involvement in practical hospital training under strict safety guidance to prevent virus transmission. clinical features of patients infected with 2019 novel coronavirus in wuhan world health organization. statement on the second meeting of the international health regulations (2005) emergency committee regarding the outbreak of novel coronavirus (2019-ncov) secondmeeting-of-the-international-health-regulations -(2005)-emergency-committee-regarding-theoutbreak-of-novel-coronavirus-(2019-ncov) world health organization. statement on the second meeting of the international health regulations (2005) emergency committee regarding the outbreak of novel coronavirus (2019-ncov) rolling updates on coronavirus disease (covid-19) evidence based management guideline for the covid-19 pandemicreview article a review of sars-cov-2 and the ongoing clinical trials current status of epidemiology, diagnosis, therapeutics, and vaccines for novel coronavirus disease farber on medical students can help combat covid-19. don't send them home accessed at www.ama-assn.org/delivering-care/public-health/covid-19-states-call-early-medical-school-grads-bolster-workforce on 5 early graduation could send medical students to virus front lines. the new york times accessed at miller dg, pierson l, doernberg s. the role of medical students during the covid-19 pandemic medical students are not essential workers: examining institutional responsibility during the covid-19 pandemic guidance on medical students' clinical participation: effective immediately association of american medical colleges. important guidance for medical students on clinical rotations during the coronavirus (covid-19) outbreak. press release korean association of medical colleges. guidance on medical student's clinical clerkship training in covid-19 pandemic ? m_cd=15♠2&wr_id=20200130176758079. (korean) medical student's clinical clerkship training in covid-19 pandemic sars and its effect on medical education in hong kong avoiding student infection during a middle east respiratory syndrome (mers) outbreak: a single medical school experience our education, our concerns: medical student education impact due to covid-19 medical student education in the time of covid-19 mental health and coping strategies among medical students the impact of e-learning in medical education the role of e-learning in medical education the authors wish to thank the administrative team of dankook university college of medicine for their technical support for web-based survey. special thanks to all the medical students who have participated in this study. the authors report no conflicts of interest. the authors alone are responsible for the content and writing of the article. il han song http://orcid.org/0000-0003-3975-6342 ihs and smk were involved in the conception and design of the study. smk developed the questionnaires for web-based survey. all authors (smk, sgp, ykj and ihs) contributed to the analysis and interpretation of the data. smk and ihs wrote the first draft of the paper. sgp and ykj contributed to the critical revision of the paper. all authors approved the final manuscript and are accountable for the manuscript. the present study was approved by the institutional review board of dankook university hospital, 11 may 2020 (dkuh 2020-04-054-he001). key: cord-265726-08k9f732 authors: parker, elizabeth u; chang, oliver; koch, lisa title: remote anatomic pathology medical student education in washington state: an early covid-19 experience date: 2020-08-20 journal: am j clin pathol doi: 10.1093/ajcp/aqaa154 sha: doc_id: 265726 cord_uid: 08k9f732 objectives: the coronavirus disease 2019 pandemic has halted in-person medical student education in many large academic centers, including the university of washington. we identified a unique opportunity to bring comprehensive and targeted anatomic pathology training to large numbers of medical students who would not receive it otherwise but also need credited coursework. methods: we developed a comprehensive 2-week remote-learning course encompassing lectures, virtual slides, discussion groups, and unique case-based activities. activities are tailored to the nonpathologist future clinician, emphasizing basic microscopy and pathology terminology. we employ multiple strategies and technologies to increase engagement while distance learning, including screen annotation, “flipped classroom” slide presentations, and repetition of common themes. results: given 13 virtual courses to choose between 13% of students enrolled in our course (70 of our 540 rising thirdand fourth-year students), a nearly 10-fold increase in average pathology rotators. conclusions: this is an unprecedented opportunity to provide tailored anatomic pathology instruction, both helping our medical students continue training during crisis and illuminating the field of pathology for our future colleagues. preliminary results have been overwhelmingly positive regarding understanding of pathology concepts as well as attitudes toward pathology. washington state reported the first cases of coronavirus disease 2019 (covid-19) in the united states. 1 the university of washington school of medicine (uwsom) is headquartered in seattle and serves students in five states, including washington, wyoming, alaska, montana, and idaho, with regional sites in each state. in march 2020, as cases of covid-19 increased in washington state, the uwsom halted all in-person clinical clerkships and advised students to return to their home states. although the uwsom has experience coordinating our medical education curriculum across a variety of geographies and health care systems, this pandemic has brought profound changes for usas well as for our colleagues around the world-as pathology educators. to meet the needs of postgraduate pathology trainees (residents and fellows), the pathology education community has mobilized. the college of american pathologists is live-streaming didactic sessions 5 days a week, 2 and the • to meet medical student needs during the coronavirus disease 2019 pandemic, we developed a 2-week remote anatomic pathology course comprising lectures, virtual slides, discussions, and case-based activities. • given the chance, 13% of our clinical students (third and fourth years) or approximately 20% to 25% of the third-year class have enrolled in the course, a nearly 10-fold increase from our usual monthly rotators. • multiple technologies and strategies allowed us to create a highly interactive course with positive preliminary feedback. us and canadian association of pathologists has made many of its prerecorded continuing medical education workshops available to trainees free of charge. leading pathologists offer webinars and "tweetorials," shared widely in the robust pathology social media community. these resources have been organized in a very short time period and demonstrate a true commitment by pathology leaders to residency education. medical student educators are undertaking similarly herculean efforts in all specialties as departments collaborate to offer remote courses to medical students to preserve graduation timelines. [3] [4] [5] however, medical school pathology instruction is quite distinct from other specialties. most students at our institution do not complete a dedicated pathology rotation (we typically have around 40 nonpathology students complete anatomic pathology clerkships each year). anatomic pathology is usually folded into other topics, limited to describing histology and briefly outlining pathophysiology. this leaves many learners without a comprehensive understanding of what pathologists do and, unfortunately, without an appreciation for how critical our work is to the clinical care of patients. medical students (appropriately) spend a great deal of time perfecting history and physicals, writing "soap" (subjective, objective, assessment, and plan) notes, and honing differential diagnoses-all skills that transfer across many specialties and prepare them for clinical residency training. those students who do go into pathology residency typically have completed a pathology clerkship and therefore obtain more pathology training than general medical students. even so, new pathology residents face a steep learning curve as they transition to residency. this creates a large gap between the level of pathology residency training materials and the level of the nonpathology medical student learner. furthermore, because most medical students do not go into pathology, the goals of medical student pathology education are different. rather than train students in the practice of pathology, our goal is to prepare them for how pathology and pathologists will affect their practice. most clinicians do not need to be able to look at a slide and make a diagnosis, but they do need to understand the language that appears on the pathology report and how to apply it to their patients. for all of these reasons, simply modifying current residency training activities is not an appropriate strategy for medical student anatomic pathology education. conversely, medical students are well suited to join in with laboratory medicine resident activities, which center on basic science principles (polymerase chain reaction [pcr] , microbiology, etc) as applied to patient samples. our colleagues in laboratory medicine have developed a 2-week companion course that allows medical students to attend lectures, plate rounds, call rounds, signouts, and so on. given the extraordinary circumstances in which we find ourselves, this laboratory medicine course gives medical students a truly once-in-a-lifetime opportunity to learn, real time, about laboratory-developed tests, scaling testing capacity, supply chain issues, and reference laboratory logistics as we respond to severe acute respiratory syndrome coronavirus 2. this unprecedented educational opportunity is also harnessed in our anatomic pathology course; we include two afternoon lectures given by our laboratory medicine colleagues focusing on both the anatomic pathology and laboratory medicine aspects of covid-19, including autopsy findings, current laboratory testing techniques, immunologic aspects of the virus, and implications for vaccine development and ascertainment of community immunity. as pathology educators, we have identified the current covid-19 pandemic as an unprecedented opportunity to address several critical needs simultaneously: (1) provide credited coursework to medical students while in-person courses are unavailable, (2) reach a greater audience from which to recruit future pathologists, and (3) provide comprehensive training in the aspects of anatomic pathology that are critical to all specialties and are unfortunately missing from current medical school curricula. in fact, it is the lack of exposure to pathology in medical school that is often cited as the reason for declining numbers of us anatomic pathology residents. anatomic pathology, as a diagnostic specialty, is particularly amenable to remote instruction. 6, 7 with the plethora of high-quality digital platforms and online resources, we see an opportunity to meet students where they are and to support their educational needs by providing a remote course. our first 2-week session began at the end of april, and two additional offerings were slated for later this spring. our course objectives are (1) to provide an interactive, organ system-based review of pathology at the level of the medical student learner; (2) to educate future nonpathologists in the terminology and workflow of pathology as it relates to their future patients; and (3) to elevate pathology in our students' minds, increasing interest in and understanding of pathology as a career. we developed a schedule with a variety of integrated activities ❚table 1❚. the 2-week course consists of 10 business days and is structured to include 2 hours of morning, large-group (20-25 students) didactics and 2 hours of afternoon, small-group work (2 groups of 10-13 students). we designed our 2-week curriculum to provide a tour of anatomic pathology by organ system, including anatomy, normal histology review, and a dive into highyield diagnostic entities. throughout these organ-based modules, we stress concepts that pathologists use daily but are often poorly understood by clinicians. we describe types of tissue reaction, including inflammation, hyperplasia, metaplasia, and atrophy. importantly, we use every lecture to review neoplasia, defining the terms benign, malignant, in situ, and invasive, and applying them to the most common processes in each organ system. we outline how tissue is received, processed, and reviewed. we also include a molecular genetics lecture, discussing the methods for determining molecular information (pcr, next-generation sequencing, fluorescence in situ hybridization) and applying them to previously discussed entities as predictive, prognostic, or diagnostic markers. lectures were developed by e. parker, l. koch, and several pathology resident volunteers using a consistent format and designed to build on each other. the content is aimed at clinical students with little pathology exposure; we integrate histologic findings into clinical scenarios in the manner they will see as clinicians-essentially helping them to "see" the disease on the slide. each afternoon small-group slide session goes over topics discussed in that morning's lecture, providing repetition and reinforcement. a key consideration in online education is student engagement. in a traditional lecture, the presenter can make eye contact, look for confusion or agreement, and ask questions. we use zoom meetings & chat (zoom video communications) to present our lectures; all university of washington members have free access to zoom pro. while this platform is a fantastic tool during social distancing, it does allow for students to take a passive approach. given that this is a new curriculum and new information for most of the students, we were concerned about how to gauge whether it was being delivered effectively. we use several strategies to assess and increase engagement. morning didactics are delivered to the entire group via zoom with powerpoint (microsoft) slides supplemented with virtual slides presented through the online digital platform pathpresenter (pathpresenter.net). lectures are presented by l. koch, e. parker, and several volunteer pathology trainees. in these first iterations, we wanted as much participation as possible. we do not provide the powerpoint slides until after the lecture, to allow students to "be there" with us as we discuss the material in real time. while we may record future sessions of the course, we decided that the existence of recordings would be an excuse for students to not come to class. we will assess student desire for recordings and, if we implement them, will evaluate attendance to determine if recordings will help or harm engagement. while in class, students are encouraged to turn on their video feed. questions are encouraged at any time, through voice or the chat box. we make heavy use of the "annotate" function, which allows students to write or answer questions directly on the screen if they like. in addition, we ask for frequent participation, having the students annotate where on the slide they see a mitosis, outline the neoplasm, or vote for the answer to a question ("if you think this is neoplastic, put a stamp on the left side of the screen; if you think it's reactive, put a stamp on the right"). we are finding that this method allows all students, even those who feel uncomfortable speaking up or turning on their cameras, to be actively engaged. each lecture concludes with a series of "board-style" questions, where students give their responses using the annotation tools. afternoon sessions consist of small-group work led by teaching assistants (tas, our former pathology postsophomore fellows) with support from o. chang, who holds daily training sessions with the tas. afternoon small group work reinforces concepts learned during that morning's didactic sessions. a shared log-in was created for pathpresenter, where students go to preview virtual slide trays curated from pathpresenter's vast public library of whole-slide images. students are each assigned one unknown slide to preview, which they present to their small group in a modified flipped classroom model. engagement is encouraged both by this model as well as by the fact that tas are their peers, providing a more informal environment in which to ask questions and discuss cases. the slides are chosen for maximum-level appropriate educational value; they contain common entities with normal background tissue, allowing students to compare normal to abnormal and learn how to use pathology terminology as they describe the differences. notably, camera usage is much higher during these sessions (l.k., personal observation, 2020). in addition, the small groups are assigned a diagnostically challenging case to work through as a group (the so-called detective case, due to the mystery and team effort required). this detective case is introduced to the small groups at the beginning of the course, and the students work together throughout the duration of the course to solve the mystery. they discuss the case and then "order" tests from the tas, who report the results, including whole-slide images on pathpresenter, imaging studies, and pathology reports, adapted from real patient cases. each group elects a "chief resident," who is in charge of communicating orders. the ta may offer advice to keep them on track, but the bulk of the discussion is led by the students. at the end of the course, they present their diagnosis and reasoning. additional follow-up questions and teaching points are discussed as a large group with the entire class. this case is designed to be relatively solvable and to demonstrate fundamental pathology principles. it centers on carcinoma of unknown primary, requiring students to judiciously order imaging and tissue sampling to make the diagnosis and decide on treatment (surgery vs chemotherapy). examples include a woman presenting with a broken hip (metastatic breast carcinoma) and a man with gross hematuria and anemia (noninvasive papillary urothelial carcinoma and metastatic colon carcinoma). last, students are invited to spend the 2 weeks independently working through a particularly challenging case (the so-called good will hunting case due to its rigor and the independent nature of the workup). students seek (by email) expert advice from pathology consultants (volunteer pathology trainees who do not have access to the case answer key), and course instructors provide requested test results and periodic updates as the patient's status changes. our first good will hunting case was an unusual presentation of fabry disease, requiring physical examination findings, biopsy, enzyme testing, and genetic testing, among other testing modalities. on the final day of the course, students compare their workups, and the diagnosis is revealed and discussed. the small group who arrives at the correct diagnosis in the fewest number of steps "wins." in addition to being a fun, interactive, and challenging exercise, this creates a forum in which to discuss health care utilization, test performance, and laboratory management. in this way, students engage with multiple learning modalities combining lecture, interactive virtual slides, small groups, and independent study. the course is administered through canvas (instructure), the learning management system used by uwsom. the course site includes the schedule, zoom links, small group assignments, reading assignments, additional resources, surveys, and powerpoint slides (following the lecture). lecturers access their powerpoints from the canvas teaching module, which is not accessible to students. course materials are housed permanently in microsoft teams (microsoft), including lectures, detective case presentations and test results, and materials in development. the teams "post" function serves to significantly cut down on email. team members include the three course faculty (authors), tas, and graduate medical education administrators, who help maintain the canvas site. to determine the effectiveness of this remote course, we administer a questionnaire at the beginning and completion of the course to assess cohort shifts in understanding and use of pathology terminology ❚figure 1❚. the questionnaire is anonymous (pre-and postcourse surveys are paired by a course administrator and given to us deidentified). the course is pass/fail; our requirements for passing are attendance and completion of these surveys. in addition, we solicit anonymous feedback at the completion of the course regarding the format of the remote learning sessions as well as changes in attitudes toward pathology, including likelihood of pursuing pathology as a specialty ❚figure 2❚. we will use these data to improve future versions of the course. surveys are also provided and submitted via canvas using microsoft forms (microsoft), which will also be used for data analysis. in march 2020, we discussed this opportunity with uwsom leadership and by early april 2020, we developed a 2-week remote learning anatomic pathology course. early april 2020 was dedicated to curricular and didactic development; late april 2020 was used to finetune content and the small-group cases. see table 1 for a summary of our course content. our remote course was one of 13 courses available to students learning from home. importantly, a concurrent course was being developed by our laboratory medicine colleagues with the goal of initially rolling out separate 2-week courses for students and later combining them into one 4-week course. strikingly, this opportunity has greatly increased the number of medical students completing pathologyspecific coursework. at our institution, we have on average six medical students each month rotating in anatomic pathology across several sites. over the past 3 years, we have hosted between 61 and 68 students annually, including both uwsom students and visiting students (students who are interested in our residency). this is a small footprint given that uwsom accepts 270 students in each entering class. from a total of 13 online course offerings, 70 clinical students enrolled in three sessions of this 2-week course, many of whom are third-year students. this means that, when given the opportunity, 13% of our clinical students (70 out of 540 clinical third-and ❚figure 1❚ students completed a two-part survey prior to starting the course. the first part of the survey asks participants to define basic pathology terminology, and the second part of the survey asks participants to categorize various disease entities as neoplasm, benign, malignant, and/or cancer. after the course, the students were invited to complete the second part of the survey again. ❚figure 2❚ in addition to asking basic demographic questions, we included survey questions about attitudes toward pathology as a specialty as well as confidence in various pathologically relevant skills. fourth-year students) and closer to 20% to 25% of our third-year class signed up for a pathology rotation. this is a nearly 10-fold increase in students per month as we now have nearly 50 students a month who will be rotating with us remotely. this is a tremendous increase in exposure to anatomic pathology and the resultant understanding of anatomic pathology by our graduating medical students. we have completed the first iteration of the course, and preliminary feedback has been overwhelmingly positive. the first group of students rated "the educational quality of this remote course vs an in-person course" an average of 4.5 on a 5-point scale. the content appears to be effective as well; for example, nine of 25 students thought a tubular adenoma was "cancer" or "malignant" prior to the class; after class, only one still did. on a scale of 1 to 5, precourse students rated their confidence in signing a death certificate, on average, at 1.6; this increased to 3.2 after the course. confidence in identifying organ of origin under the microscope went from 2 to 3.3. coincidentally, we had a spot open up in our postsophomore fellowship during the first session, and two of our students applied for a yearlong position beginning in less than 6 weeks; both said our course had made them more interested in pathology. the course evaluation comments included the following: • "i really enjoyed the structure of having lecture in the mornings followed by interactive sessions in the afternoon looking at, describing, and work[ing] through actual slides that helped to apply and ingrain the knowledge from the morning sessions. having all of this followed by an interactive and engaging case study was also terrific!" • "didactics with annotation function were fun and engaging. small group leaders were also engaging and helpful in creating a comfortable learning environment." • "i liked that this rotation was more geared to understanding pathology from a clinical perspective rather than focusing on all the nitty-gritty!" • "i like the structure of doing large group lecture and then small group reading of the slides. i need repetition and practice and this gave me that." while there are limitations to what can be conveyed and experienced via remote teaching, this is an unprecedented opportunity for pathology educators to dramatically increase medical students' exposure to and understanding of the field of pathology. this is an opportunity to demonstrate to medical leadership the value of pathology education to our students in an age when many medical schools are curtailing preclinical contact education for medical students and the number of students going into pathology is dwindling. our ability to provide a meaningful clinical experience for students during this time of limited interpersonal contact and social distancing enhances the visibility of and appreciation for pathology within our institution. technology has been key in engaging our learners. in particular, zoom's annotation function, encouraging web camera use, and virtual slide sessions via pathpresenter were viewed highly by both our instructors and our students. organizational tools, including canvas and microsoft teams, have allowed us to centralize information and documents, as well as to communicate efficiently. the course has resulted in a dramatic increase in the number of students who enrolled in a pathology rotation. given the opportunity, 13% of our clinical students (both third-and fourth-year students) or approximately 20% to 25% of the third-year class have enrolled in our remote pathology course. this is a nearly 10-fold increase in our average number of monthly student rotators. preliminary results indicate the course is effective in both increasing knowledge of pathology and elevating student opinions about anatomic pathology, including enticing two students to apply for our postsophomore fellowship. we have a tremendous opportunity to meet the unique needs of our students during the covid-19 pandemic and to simultaneously elevate pathology understanding in our future nonpathology colleagues. our future work will focus on collaborating with our laboratory medicine colleagues in the creation of a comprehensive 4-week remote anatomic pathology rotation. we envision this future course as a tool to meet the needs of medical students facing nonpandemic obstacles to on-site pathology training, including family obligations, disability, and financial concerns. most important, we remain optimistic that in this time of terrible disease and disruption we can make pathology an accessible and valuable learning experience for all of our students. first case of 2019 novel coronavirus in the united states covid-19 and medical education daily medical education for confined students during covid-19 pandemic: a simple videoconference solution medical student education in the time of covid-19 digital pathology and its role in cytology education whole slide images and digital media in pathology education, testing, and practice: the oklahoma experience key: cord-267076-a4ulg4ck authors: mechili, enkeleint a.; saliaj, aurela; kamberi, fatjona; girvalaki, charis; peto, ela; patelarou, athina e.; bucaj, jorgjia; patelarou, evridiki title: is the mental health of young students and their family members affected during the quarantine period? evidence from the covid‐19 pandemic in albania. date: 2020-07-13 journal: j psychiatr ment health nurs doi: 10.1111/jpm.12672 sha: doc_id: 267076 cord_uid: a4ulg4ck introduction: the use of quarantine method has a significant impact on mental health status. aim: this study aimed to assess the levels of depression among bachelor and master university students (nurses/midwives) and their family members’ during the quarantine period of covid‐19 pandemic. method: a cross‐sectional study was conducted in vlora university, albania. patient health questionnaire (phq‐9) was used. institutional e‐mails of all active students were used for their recruitment. results: in total, 863 students and 249 family members participated in the study. the mean phq‐9 score was 6.220 (sd=5.803) and 6.280 (sd=5.857) for students and family members, respectively. being unsatisfied with covid‐19 prevention measures, beliefs that covid‐19 infection and quarantine process can cause problems on their health status were the key factors for students to be screened positive for mental problems in multiple regression analysis. for parents, the last two factors were found to contribute significantly. discussion: our findings indicate that quarantine measures have a significant impact on the levels of depression among both university students and their family members. broader population studies are necessary to safeguard these results. implications for practice: provision of mental health services during the quarantine period is of paramount significance. mental health first aid trainingand support could benefit both students and their parents significantly. additionally, follow up services after the end of quarantine are recommended. the sars-cov2 or covid-19 first cases were initially detected in wuhan, hubei province in china (ecdc, 2020) . by april 16 2020, more than 2.078.000 cases have been diagnosed while the burden of death is up to 138.100 worldwide. in albania, until june 5, the number of reported cases was 1.197 and 33 fatalities (johns hopkins csse, 2020) . according to the available data (april 14, 2020) , the countries with the highest numbers on positive cases and deaths are the usa, italy and spain (johns hopkins csse, 2020) . in terms of strict measures implemented to prevent the transmission of the virus, china and in particular hubei province was the first to be quarantined. soon after, the european countries were put in a similar situation to china. the albanian government was also among the first, which implemented stringent measures to prevent the spread of covid-19. first, on march 10 2020, schools and universities were closed and a total lockdown was put into force two days after. only supermarkets, bakeries and pharmacies remained open. at the same time, people were allowed to leave their houses only for one hour per day, and only one member per family, from 05:00 am till 13:00 (with 40 hours lockdown during weekends) by keeping physical distance and respecting all the prevention measures. evidence from previous outbreaks data showed that quarantine measures have been effective (dénes & gumel, 2019; gupta et al., 2005; tsang & lam, 2003) while preliminary data after covid-19 outbreak, showed that it is arecommended as an effective way to stop disease transmission (ng et. al., 2020; choi & ki, 2020; tang et. al., 2020) . world health organization (who) has stressed the potential psychological impact of covid-19 outbreak on the population and published key messages for different target groups (who, 2020a) . a recent study in china evaluated the levels of depression and other mental health problems during the covid-19 pandemic. authors concluded that the prevalence of moderate to severe depressive symptoms, anxiety and stress in the general population is at 16.5%, 28.8% and 8.1% respectively (wang et al., 2020a) . another study conducted in china showed no statistical differences in depressive, anxiety and stress scale during the initial outbreak and four weeks later (wang et al., 2020b) . according to haoet. al., (2020) , the depression, anxiety and stress scale-21 during the covid-19 outbreak was significantly higher (8.3 vs. 2.2; p<0.001) on psychiatric patients in comparison to the healthy population. the prevalence of stress, insomnia, depression and anxiety was found at the levels of 1.5%, 2.3%, 3.7% and 3.8% respectively in the study of tan et al., (2020) . however, the authors concluded that this article is protected by copyright. all rights reserved returning to work after the quarantine period was not very stressful. a study among healthcare personnel in singapore and india found that the prevalence of moderate to severe depression symptoms is at 5.3% (chew et. al., 2020) . mental health problems are frequent to students' population, with nursing students being also at risk (cilar et. al., 2019; holm-hadulla & koutsoukou-argyraki 2015; chernomas & shapiro 2013) . the 27% of nursing students reported mild to extremely severe depression levels (chernomas & shapiro 2013) . another study showed that depression prevalence among baccalaureate nursing students was at 35.8% (cheung et. al., 2016) . since universities curfew in albania, students and their families were obligated to stay at home and respect the government rules for stopping the pandemic spread. although the mental health status in the general population has been evaluated during the covid-19 quarantine period (wang et al., 2020a; wang et al., 2020b) there is a lack of data about nursing and midwives students. in addition, albania lacks of data on the mental status both of the general population and the students. therefore, given, the psychological burden caused by similar emergencies in the past, this study aimed to evaluate the levels of depression of students (nursing and midwifery) and their family members' during the quarantine period. this is a cross-sectional study conducted in vlora, albania, with the participation of university of vlora (uv) students from faculty of public health and their parents/family members from march 30 till april 9. after the first ten days of quarantine, uv initiated the use of the googleclasroom platform for online classes. to join the e-classrooms students needed to use their institutional e-mail to log in the platform and participate. all active students of the faculty (both bachelor and master), attending the online classes, were asked to participate in the study. to approach all students, the institutional e-mails of all faculty students were provided to the research team from the information technology (it) sector of the university. twenty days after the quarantine, the link of the online questionnaire was sent to the institutional e-mails of the students to be completed both from them and their family members. general information regarding the purpose of the study, the questionnaires and the process to be followed were also included in the communication. the first reminder for those who had not this article is protected by copyright. all rights reserved responded yet was sent three days after and the second, seven days after the initial e-mail was sent. additionally, university professors encouraged the students to participate in the study and reinforced the importance through the googleclasroom platform. google forms were used to distribute the questionnaire online. we used the patient health questionnaire (phq-9) to measure depression levels and monitor severity (kroenke & spitzer, 2002) . it is a tool translated in the albanian language, validated and systematically used in primary healthcare settings for the annual population check-up (those aged 35-70 years old). the phq-9 questionnaire has been used before for screening depression among university students and the general population (du et. al., 2017; ngasa et. al., 2017; zhang et. al., 2013; yu et. al., 2012; kareklaet. al., 2012; al-busaidi et. al., 2011) . additionally, the online use of phq-9 has been suggested for screening depression among chinese university students (du et. al., 2017) . the participants completing the phq-9 questionnaire were asked to rate the frequency (not at all; several days; more than half the days; nearly every day) at which they had been bothered by any of the following problems the past two weeks: little interest or pleasure in doing things; feeling down, depressed, or hopeless; trouble falling or staying asleep, or sleeping too much; feeling tired or having little energy; poor appetite or overeating; feeling bad about yourself or that you are a failure or have let yourself or your family down; trouble concentrating on things, such as reading the newspaper or watching television; moving or speaking so slowly that other people could have noticed. or the opposite being so figety or restless that you have been moving around a lot more than usual; thoughts that you would be better off dead, or of hurting yourself. each statement was rated from zero to three and then the total score was calculated. higher scores indicated higher depression levels (0-4 indicates minimal depression levels, 5-9 mild depression levels, 10-14 moderate depression levels, 15-19 moderate-severe depression levels and 20-27 severe depression levels). the cut-off score of 10 (phq-9≥ 10) which was previously found to have high sensitivity and specificity for major depression (du et. al., 2017; kroenke et. al., 2001) , was set in the study. furthermore, questions related to gender, age and study programme were added to the students' questionnaire. to control the impact and satisfaction with the measures undertaken, this article is protected by copyright. all rights reserved three more questions were added. 1) do you think that covid-19 can cause problems to your health; 2) do you think that quarantine causes problems to your health status; and 3) are you satisfied with the prevention measures undertaken to stop covid-19 spread. as about the family members' questionnaire except the aforementioned, questions related to their working status were added. to analyse the data, the statistical programme ibm spss-23 was used. the significance levels were defined at the 0.05 level. phq-9 showed high-reliability rate to both populations. cronbach's alpha indicator was 0.885 for students' population and 0.898 for family members' population. with absolute and relative frequencies are presented the categorical variables and as mean and standard deviation the quantitative variables. in both populations, the cut-off point of 10 was used and in this way, we transformed the phq-9 in a dichotomous variable with participants having scored above nine considered depressed and those below ten not. we conduct simple logistic regression with the phq-9 used as the dependent variable and the rest as the independents. in all cases, the enter method was used. dummy coding was done for qualitative variables. multiple regression analysis was done by using the backward conditional method to explore the risk factors for phq-9 positive screening (phq-9≥ 10). the entry point of independent variables was set at 0.05 level and removal point at 0.10. even though some variables didn't show statistical significance during simple logistic regression analysis, we considered of higher importance to include all them in multiple regression analysis. we did not run chisquared test but conducted a multiple logistic regression analysis because this method gave us the possibility to neutralise different confounding factors at the same time. additionally, based on the fact that the number of participants was high and the number of independent variables was low, our model is very much reliable. in both populations, after applying hosmer-lemeshow control, the p-value was found higher than 0.05, something that confirms the suitability of the logistic regression analysis. to some extent, the standard errors found were low, something that means a very low probability of linearity existence. after multiple regression analysis, the nagelkerke r 2 was 0.232 for students' sample and 0.134 for family members' sample. this article is protected by copyright. all rights reserved the ethical committee of the faculty of public health was informed and approved the study. due to the university lockdown, the request and approval were handed to us by e-mail (issue date 26/03/2020). no personal data were recorded and all questionnaires were completed anonymously. due to technical difficulties of using printed material, participants were informed through e-mail that by participating, they provide their convent (students and family members) to the researchers. this information was also shared with them during the online classes when they were first invited to participate in the study. additionally, they were informed that participation in the study was voluntary and participants could withdraw at any moment. furthermore, they were informed that all the data collected will be used only for the current study. all ethical guidelines for medical research were strictly respected. this article is protected by copyright. all rights reserved invitations were sent to 921 students in total. from them, 892 completed the questionnaire (response rate 96.8%). twenty-nine questionnaires were withdrawn as incomplete. finally, data from 863 students were used for the analysis. as about their family members, initially, 253 responded. four questionnaires were withdrawn as incomplete. the final sample of the family members was 249. in both populations, the majority of the participants were female (88.6% among students and 70.8% among family members). the 79.5% of the students are conducting their bachelor studies and 90.3% of them reported being satisfied with the measures undertaken to prevent the spread of covid-19. the 67.4% of the students and 75.2% of family members stated that covid-19 can be harmful to their health. additionally, 27.4% and 28.4% respectively indicated that quarantine can cause health problems for them (table 1) . the mean phq-9 score for the students was 6.220 (sd=5.803) and for the family members was 6.280 (sd=5.857) with a range of 0-27 in both populations. the 25.2% (n=217) of the students and 25.6% (n=64) of the family members were above the threshold of phq-9≥ 10, indicating moderate to severe symptoms of depression. in simple logistic regression analysis for students' population (table 2) can cause problems to their health (p-value <0.001) and if quarantine cause problems to their health status (p-value <0.001) were statistically significant. with regards to the family members population (table 2) , if covid-19 can cause problems to their health (p-value <0.001) and if quarantine cause problems to their health status (p-value <0.001) were statistically significant. in multiple logistic regression analysis for students' population (table 3) , three factors were found to contribute significantly to being screened with an elevated risk for depression. students who were not satisfied with covid-19 prevention measures were more likely to have elevated levels of depression in comparison to those being satisfied (or= 1.802; 95%ci: 1.070-3.036 p-value= 0.027). students who believed that covid-19 can cause problems on their health and students who believed that quarantine can cause problems on their health were more likely to be screened positive for depression (or=1.888; 95%ci: 1.275-2.797 p-value= 0.002 and or= 6.312; 95%ci: 4.450 -8.952 p-value<0.001). as for the family members, participants who consider that covid-19 can cause problems on their health (or=2.471; 95%ci: 1.081 -5.648 p-value= 0.032) and that quarantine can cause problems on their health were more likely to have elevated levels of depression (or=3.492; 95%ci: 1.895 -6.437 p-value<0.001), (table 3) . this article is protected by copyright. all rights reserved to our best knowledge, this is the first study in albania and among the first worldwide to evaluate depression levels among nursing/midwifery students and their family members during the quarantine. our research found that more than a quarter of both populations were screened with moderate to severe depression levels. key factors contributing to high levels of depression were: the perception that covid-19 and quarantine can cause problems on their health (both population) and the satisfaction with prevention measures on students' population. according to albania demographic and health survey (albanian institute of public health and institute of statistics, 2018) thirteen and eighteen percent of albanian females and males aged 15-59 reported feeling depressed a lot of the time or all of the time during the two previous weeks, at the time when the study was conducted. these results differ with the current study but we should take into account the different periods of receiving the data, the differences in representativeness and differences in data collection. in the study of albanian institute of publichealth, data were collected through interviews while in the current study via online questionnaires. usually, participants are more reluctant to report their mental health situation in a face to face interview but easier while completing an anonymous online survey. our results are similar to a recent chinese study in which authors concluded that the prevalence of moderate to severe depressive symptoms is at the levels of 16.5% (wang et al., 2020a) but with no significant differences four weeks after the outbreak (wang et al., 2020b). our study has not re-evaluated the mental health of these populations. findings of a study in university students at thessaloniki, greece reported an increase of depression by 60.9% during the lockdown (kaparounaki et. al. 2020 ). both studies received information via online tools, but in the current study, nursing and midwifery students were included while in the study of thessaloniki university students from all fields were recruited. however, the study of wang et. al. (2011) found no differences among quarantined and non-quarantined students during the h1n1 epidemic. to some extent, the students in the study of wang et. al., (2011) were quarantined only for seven days while now they were quarantined for more than four weeks. prevalence of moderate to severe depressive symptoms differs between the current study and the healthy population in the study of haoet. al. (2020) . differences in the sample size, measuring instruments and data collection are the main reasons for these differences. this article is protected by copyright. all rights reserved differences in the prevalence of depression exist between the current study, the study of chew et. al., (2020) and the study of tan et al., (2020) . these differences can be attributed mainly to the different periods of data collection and the approach used, the differences in culture and on the instrument used, the intensity of the epidemic, the response from governments and healthcare systems as well as the possibility in selection bias in the current study. in comparison to other studies in student populations which used the phq-9 questionnaire, our study reported much higher depression levels (kim et. al., 2019; du et. al., 2017; karekla et. al., 2012) . however, we should take into account that these studies were conducted under normal conditions without the impact of isolation and quarantine, which most probably is the reason for these high levels. similar results with our study with regards to the general population are also reported elsewhere (villarreal-zegarra et. al., 2019; ahn et. al., 2017) . however, since the family members' sample of our study is small, it cannot be considered as representative of the general population and interpretation of the results should be done with caution. most of the participants, in both populations, did not consider that quarantine can cause problems in their health status. probably this shows the low level of awareness on the potential impact of isolation on mental health. in contrast, a relatively small percentage of respondents (lower on family members) considered that covid-19 could not cause problems on their health. most probably, this comes as a result of the high numbers of confirmed cases and deaths in many countries as well as the everyday discussion in the media. the higher rates on family members could also reflect on the fact that covid-19 affects older ages (who, 2020b) . according to chew et. al., (2020) , experiencing physical symptoms during the covid-19 pandemic is significantly associated with depression at healthcare workers. however, our study did not evaluate physical symptoms and comparisons are not feasible. both students and family members who considered that quarantine itself and covid-19 can cause problems on their health were more likely to be have high levels of depression. according to a study among medical students, the presence of a recent major life event or being a student at the clinical level are significantly connected with depression (ngasa et. al., 2017) . we cannot distinguish if they are facing endogenous depression or reactive depression as a result of the situation and further research is needed to confirm these findings. this article is protected by copyright. all rights reserved the level of satisfaction with covid-19 prevention measures in our study, was correlated with mental health problems. improvement of the prevention measures is a key issue that can decrease these health issues significantly. giving more information about the impact of sars-cov-2 is not effective, although updating or developing training programs and communication activities could be more helpful (tran et. al., 2020) . additionally, performing hand hygiene, wearing face masks, improvement of ventilation and school hygiene are some simple actions for the low prevalence of psychiatric symptoms (tan et. al., 2020) . prevalence of depression in the current study is high on both populations, but differences were not found between gender and working status. however, during the period of this study, neither the students nor their family members returned to the university and /or their job positions. a recent study in china confirms that returning to work during the covid-19 outbreak does not increase the prevalence of depression and other mental health symptoms (tan et al., 2020) . probably, our results may possibly change after they go back to the daily routine, and further studies are needed to confirm this. to improve the mental health status of the population the coordination and collaboration of community settings and hospitals is important. the online provision of mental health services by using cognitive behavioural therapy and mindfulness-based cognitive therapy could meliorate the outcome (ho et. al., 2020) . this is the first study to measure the depression levels of nursing/midwifery students and their family members in albania during the covid-19 outbreak and quarantine period. students' response rate was very high, which means that generalizability can be done for this population. however, the results are self-reported and verification by using other screening instruments or by a clinician is recommended. additionally, the sample of family members is low and the use of convenience sampling means difficulties in representativeness in the general population. besides, although students were instructed not to influence the responses of their family members, a bias may have occurred. lack of previous data makes difficult comparisons and determining whether the high percentage found is due to the quarantine situation or is indicative of the permanent mental health parameters of this population. however, these results can be used as a baseline. this article is protected by copyright. all rights reserved the psychological impact of quarantine due to covid-19 outbreak is high among nursing and midwives students' of vlora university as well as among their family members. at a cutoff point of ten (phq-9≥ 10), 25.2% of the students and 25.6% of the family members were screened positive. key factors for this were found perceptions that covid-19 and quarantine can cause problems on health. nursing and midwifery students, as well as their family members, could benefit from mental health support during the quarantine period. offering mental health first aid training and support could be a useful approach to provide help and to ease the symptoms they are experiencing. this is the first study that assesses depression levels of nursing and midwifery students as well as their family members during the quarantine period in albania. this study can be used as a baseline for further assessment of depression levels during and after the end of quarantine. looking after and providing mental health services to nursing and midwives students as well as their family members are recommended. having healthy future healthcare personnel should be a priority for health policymakers. tables table 1. this article is protected by copyright. all rights reserved this article is protected by copyright. all rights reserved prevalence of and risk factors associated with dry eye: the korea national health and nutrition examination survey prevalence of depressive symptoms among university students in oman a multinational, multicentre study on the psychological outcomes and associated physical symptoms amongst healthcare workers during covid-19 outbreak stress, depression, and anxiety among undergraduate nursing students depression, anxiety and symptoms of stress among baccalaureate nursing students in hong kong: a cross-sectional study this article is protected by copyright. all rights reserved accepted article key: cord-301171-1lpd8dh9 authors: davison, robert m. title: the transformative potential of disruptions: a viewpoint date: 2020-05-19 journal: int j inf manage doi: 10.1016/j.ijinfomgt.2020.102149 sha: doc_id: 301171 cord_uid: 1lpd8dh9 i engage with the impact of disruptions on my work life, and consider the transformative potential that these disruptions offer. i focus on four parts of my life: as a researcher, teacher, administrator and editor. in each, i examine the nature of the disruption and the way i deal with it. i also consider how the present disruption may facilitate a transformation of current practices that lead to a better world at the individual and institutional levels. rather than lamenting the inconvenience of a crisis, i prefer to celebrate the opportunity to do better. every so often the more-or-less smooth tenor of our lives is disrupted. we are forced to deal with a new set of challenges and circumstances. the exact situation varies: it may be a local situation or a global one. even when it is global, each of us experiences the disruption in different ways and to different degrees. the underlying characteristics of disruptions vary but include political, economic, environmental, medical and/or social. disruptions don't always come singly. distinct global and local disruptions may co-exist, further adding to the complexity of a situation. i suggest that while these disruptions are undoubtedly inconvenient, not to mention potentially life-threatening, they do offer us an opportunity for transformative change. out of the darkness of disruption we may perceive glimmers of hope: the potential to do things better. in 2008, barack obama's soon-to-be-chief of staff, rahm emanuel observed "you never want a serious crisis to go to waste" 1 . a crisis thus presents an opportunity to further disrupt the status quo and bring about radical changes that might otherwise be inconceivable. in chinese, the two characters (危機) for crisis imply danger (危) and opportunity (機) . as researchers, we have the potential to play a significant role in transforming the opportunity and making the world a better place (davison et al., 2019) . but the operative word is 'potential'! there is never a guarantee that the transformation will actually occur, nor indeed that it will necessarily lead to better things. furthermore, overly hasty reactions are more likely to introduce fresh crises. considerable thought and care needs to go into the design of the transformed processes (or artifacts) before their benefits can be fully reaped. i was asked to write about the impact of covid-19 (c19) on information management research and practice. however, i find that the topic is too narrow. instead, i see the opportunity to consider disruptions more generally, hence the title and the current focus. i first offer some contextual details to help you to make sense of my perspective. i live in hong kong, where we have never (yet) had a lockdown that legally confines most people to their homes most or all of the time, whether for the current situation or any other in living memory. public transport operates, shops are mostly open, social distancing is broadly adhered to, and i can choose to work at home or in the office. my work includes a mix of research, teaching, administration and journal editing. i deal with each of these activities below. disruptions undoubtedly exert a negative impact on my work as a researcher. as a qualitative researcher, i need to observe and interview people. i can still do this online, but it is less effective. one major project has been delayed indefinitely because i need to be in shanghai to collect data, and that is impractical at the moment, simply because even if i could travel to shanghai, i would need to be quarantined for two weeks on arrival and a further two weeks on return to hong kong. the inconvenience of the quarantining is too disruptive to bear. if the current situation persists or returns, both the topics that i choose to investigate and the way i do research will need to change. remote data collection will become normal as we adapt. this will not be limited to remote interviews, but must also include remote site visits. we will need to develop new data collection protocols for instance. but it seems to me that the real problem is that we are trying to replicate our physical world online. we used to have synchronous, face to face meetings; now we have synchronous, virtual meetings. we used to collect data in person; now we try to mediate data collection through technology. that's not transformation in my view. it's replication. we need to transform the way we do things. we need to find a better way to meet, to collect data, to do research. simple things like turning off the video can help because this reduces the number of cues that we need to process. it also prevents us from noticing the existence of video-audio lags that are annoying at the best of times, though it may also impoverish the richness of the medium as some of the paralinguistic cues disappear. creating a natural yet virtual space where both researcher and researchee feel comfortable to engage in a meaningful and efficient conversation is challenging. whether you are more persuaded by media richness theory (daft and lengel, 1986) or media synchronicity theory (dennis et al., 2008) , each of us has to select a medium that balances these various constraints and thus is oriented more toward replication of face to face interaction or transformation into something different. as a regular invited speaker, visiting professor and general globetrotter, i habitually travel extensively. this aspect of my life has also been severely curtailed. from march to june, 2020, i have had to cancel five work trips to eight countries: china, finland, indonesia, morocco, norway, sweden, switzerland, uk (thrice). the objectives to be attained on these trips included a mix of data collection, student recruitment, research seminars and collaboration, and a phd thesis defence. i have been able to hold some seminars online, but the effect is not the same. it is also hard work: you may have read about 'zoom fatigue' 2 . a research visit, for instance, is much more than a seminar. it also involves many one-to-one conversations, brainstorms, insights, and the exchange of ideas, lubricated with laughter, intellectual spice and good cheer. transforming this kind of work is challenging: there is a grave risk that it will be reduced to the functionality of a seminar but without the rich interactions that take place on the side-lines. i have been invited to a seminar + conversation event in late june, 2020, that will see me 'zoom' into umeå university, in northern sweden. it will be interesting to see if the transformation is effective for the audience and productive for the speakers. i fear that a world without the freedom to travel unhindered may be a much less global world. that's probably less of a concern if you live in a big country with extensive domestic travel opportunities. however, while we will miss the global opportunities, we should examine local opportunities more carefully, and appreciate our local contexts. as a teacher, all my classes since mid-january, 2020, have been online using a variety of technologies. while some of my students remain in hong kong, many are elsewhere in china as well as further afield. the vast majority are accessing the internet from home, quite often on slow connections. my guest lecturers are doing the same, and for them it is certainly more convenient than travelling to the university campus. i anticipated that this new environment would be very hard to adapt to, as a teacher, but strangely enough i was wrong. i have a number of principles that help me ensure that students receive as high quality an individual education as is possible in the circumstances. some of these are transformative, but they build on earlier work. a key challenge that i encounter in a normal (face to face) class is the low level of student interaction. perhaps this is cultural, but i find that while a small minority may be willing and able to interact, the vast majority is not. one of the drivers appears to be fear of making mistakes in front of others and thus losing face. a second relates to interacting in a second (or third) language. in an online class, the dynamics change and i find that, with a little effort, i can get 90% of the students to interact without disrupting each other or me. those of you familiar with the research into group support systems (gss) will recall the various benefits associated with this technology, notably the lack of air-time fragmentation and the elimination of dominant individuals (davison and briggs, 2000) . in a gss-facilitated meeting, there is more time for discussion, more even participation, and more interaction and feedback. these meetings might also induce distractions and digressions, and suffer from participants flaming or insulting each other. i have certainly seen the positive effects in my classes. with 60+ students, it is essentially impossible for each of them to have significant contribution time in a regular classroom, especially if one or two dominate the air time. with parallel conversations, it becomes a reality and the primary problem is restraining their creativity and drawing a conversation to a close, so that the class can move onwards. in an online class, judicious use of the 'chat' feature enables students to type to each other or to me as they wish or need to. they can raise questions, make suggestions, provide links to external sources, and so can i. in order to maintain student attention, a lesson that i learned early on was that i need both to slow down my speech and to break it up with frequent interludes where students can ask questions and create ideas. not all students think equally quickly: some need more time to reflect on the content and thus interjecting regular hiatuses into my content delivery enhances access to that content immensely. thus, every 15 minutes or so, i will stop and seek to provoke them with an issue that has no easy answer, where a range of perspectives 2 https://www.bbc.com/worklife/article/20200421-why-zoom-video-chats-are-so-exhausting j o u r n a l p r e -p r o o f can reasonably be identified, and where students are likely to have an opinion. i give the students 10 minutes to brainstorm in the 'chat' feature of whichever software i am using to run the class. another approach is to use the online breakout rooms where students first discuss an idea in small groups before presenting it to the class as a whole. whichever technique is used, both the instructor and other students can provide additional feedback and commentary. over a three-hour class with 50-60 students, i find that these 'chat' interludes generate around 9,000 words of generally high quality comment. i regard this sharing of ideas as essential to individual learning because in order to write sensibly you have to think. i make sure that i read all the comments. i type my own reactions to some while others i react to verbally. i save all the typed comments to a doc file and email it to all the students after the class. all the comments are identified (there is no anonymous function) but this seems reasonable as in a real class they would be identified anyway. moreover, individual students receive credit for their ideas, both from me and from each other. is this transformative? well, it is a small-scale transformation in the way i teach because of the way i deliberately fracture my episodes of speaking into smaller fragments and punctuate them with chat interludes that provide the opportunity for students to voice out their thoughts in parallel. although they can contribute to the chat at any time, most wait for the invitation from me. i do believe that this new teaching-learning protocol transforms the learning process, because students are more actively engaged in the process of learning. they know (because i tell them) that they will be rewarded for their participation (in my classes, i typically award 20% of marks for participation), and when they see others typing in the 'chat', then this seems to create a gentle peer pressure for them to emulate their peers. while some students type just a few words and then submit, others take considerably more care and write a few tens or even hundreds of words. their communication style is transformed by the technology that exerts no time pressure to complete a communication by a hard deadline. these longer comments often attract attention from other students who comment on them in turn, setting up a viral pattern of inter-student learning. i intervene as necessary to minimise digressions, correct misunderstandings, offer an opinion and further challenge their imagination. where assessments are concerned, it is clearly more difficult for students to undertake group work if they lack the luxury of a face to face environment, but virtual teams have been around for a long time and there is no reason why students cannot work virtually. it is a new skill to learn (hardin et al., 2007; davison et al., 2017) that will be of value in the workplace. a blended learning approach might see a mix of synchronous and asynchronous virtual team work. the synchronous events may be psychologically more comfortable and productive, yet simultaneously less convenient because of the need for everyone to be virtually present. the asynchronous states that persist in between the synchronous events will still see work done, even if the intensity is lower (maznevski and chudoba, 2000) . examinations online are also feasible, though there are fears that students will take each and every opportunity to cheat, whether by employing proxies to answer the questions for them, by outsourcing questions to experts on demand, or by some other ingenious means. my preference is to set an exam question that is really much too long (or difficult) for the time available and which requires analysis but not memory, pushing students to the limit of their capabilities and thus rendering cheating that much more complicated. i thus transform the assessment process and provide students with the opportunity to demonstrate what they have learned. it must be said, however, that evaluating these kinds of assessments is challenging in itself. i invest considerable time and mental effort in carefully reading lengthy answers to questions and then justifying the marks awarded. a marking scheme may help, but answers inevitably deviate from the ideal or model answer, and so flexibility is essential. the proof of the effectiveness of the transformation of teaching, learning and assessment will be in the proverbial pudding: student evaluations of my teaching, teacher evaluations of student learning, as approximated via exam answer papers, and the eventual employment into which students enter on graduation. initial feedback from students is positive, but a single cohort of students will not provide sufficient evidence; there are lessons to keep learning on both sides. if online learning is here to stay, i am confident that we can transform both ourselves and it to a high level of effectiveness that will in some measure exceed what is possible in face to face environments. i earlier alluded to the problems of zoom fatigue. i know people who are in back-to-back zoom meetings for hours and days at a stretch. here the replication problem is all too evident. we need to evaluate carefully if a synchronous online meeting is really essential. if it is not, and i feel that we should assume that the vast majority are not, then a move to asynchronous interactions is called for. in effect, this means less meetings, which is surely a good thing! the gss software that i mentioned earlier provides an excellent basis for this kind of interaction. a meeting can be open for a period of hours or days and members should expect to visit on several occasions so as to add remarks, read those of others, offer comments and engage in a prolonged deliberation. this kind of extended meeting requires both a good work ethic and for the meeting organiser to have good facilitation skills, in order that a meeting can be drawn to a productive close in a timely manner without cutting people off. the most challenging aspect of this transformation is accepting that asynchronous interactions can work, and that meetings are really not essential most of the time. thus, instead of forcing the technology to support existing meeting patterns, we should allow the technology to support a transformed meeting arrangement. for myself, i find that the vast majority of administrative tasks can be completed perfectly adequately through email, i.e. asynchronously. response times are generally fast for urgent matters and i see no reduction of effectiveness or efficiency. where group discussion is needed, gss (or a similar technology) may offer a richer environment than zoom, yet not require synchronous presence. as an editor, i see a pair of disparate effects. firstly, many more papers are being submitted. alas, these are not always of the highest quality. many of them relate to the current c19 pandemic and consist of quickly-thrown-together collections of notes with little scientific import or practical value. these are politely rejected. special issues on the impacts of pandemics have also been proposed, equally hurriedly, and they too are rejected. secondly, reviewers of papers tell me that they need more time to complete their reviews and authors currently revising their papers also ask for more time. a month, or three, would be nice! the culture of asking for extensions is rife in our world and it is encouraged by those who grant such extensions, sometimes without even being asked! an extension request that is received well in advance, with some careful argument as to why it is needed, is fine and will be granted. an extension request that is received in the hours (or minutes) before the deadline (for an extra 6 months) will not be entertained. if you wish to work right up to the deadline, that's your choice. but careful time management is always a good idea, and since emergencies will happen, we need to allow time for them. don't assume that deadlines will be extended! any transformation here has to be at the personal level. we can work around disruptions if we want to. for me, it is primarily a matter of time management and learning to say no. quality is still going to be the issue. a disruption does not justify sloppy or haphazard work, or knee-jerk research either. bear in mind that your article is going to be in multiple review-revise cycles for several months, if not longer. we want to publish high quality research that will stand the test of time. that hasn't changed and i don't see it changing. my editing work seems least affected by disruptions, though i admit that it is progressively harder to secure good reviewers willing to complete their assignments on time and to a high quality level. technology has great transformative potential, if we want to transform and to be transformed. but do we? my personal suspicion is that while we teach our students about the value of disruptive technology, we are less keen to be disrupted ourselves, unless it is on terms of our own choosing. punctuated equilibrium theory (eldredge and gould, 1972; gersick, 1989) suggests that disrupting the underlying structures of a stable situation (equilibrium) may create the potential for the introduction of radical changes that enhance the status quo. a pandemic virus, or rather the human reaction to it, is certainly disruptive and is punctuating many of our stable states. do we try to go back to the old stable state or do we accept the transformation challenge? disruptions to the research process are the most difficult to resolve, and i see this as a work in progress. disruptions to teaching, learning and assessment, and administration, on the other hand, are more amenable to transformative action. we will need to plan to teach in a different way, our students will need to accept to learn in a different way, and we will have to create new ways to assess that learning. we may even be able to escape the iron grip of meetings! it can be done, if we have the will. it has been suggested that the current c19 pandemic will disrupt academia in a way that will permanently change it. apparently the top ranked universities are destined to survive while others may disappear 3 . i'm not so sure: in my view, those that thrive will be those that transform to and profit by the new status quo. the ability to transform is no more than survival of the fittest in a new set of circumstances. but this applies as much at the level of the institution as at the level of the unit or the individual. digitising core activities is a start. reinventing the institution (of everything we do and where and how we do it) is down the road. are we ready to transform? my own institution was the first in hong kong to put all classes online several months ago. transformation is often revolutionary, which is why punctuated equilibrium theory is so pertinent. revolutions are not tea parties, and so it may be some time before we can enjoy the next stable state. finally, we need to be careful not to change too quickly. contact-tracing has emerged as one of the tools that can be used to trace possibly infected individuals. however, early reports suggest significant concern about the privacy implications of contact-tracing apps like australia's covidsafe, which was designed and implemented very quickly with less attention to usability issues than would normally be the case 4,5 . digital surveillance, incorporating the tracking and tracing of individual people, is already rather common in many societies and not a new research topic (clarke, 2001) . under the cover of a pandemic, where fear of infection is the lowest common denominator, it is not hard for governments to ramp up surveillance activities in a way that would be firmly rejected in normal times, yet now is broadly accepted! klein 6 underscores the concerns here by pointing out how politicians are working with technology firms "with an emphasis on permanently integrating technology into every aspect of civic life" where "our every move, our every word, our every relationship is trackable, traceable and data-mineable by unprecedented collaborations between government and tech giants". with such a panoptic vision, it seems that revocation of surveillance is not envisaged and so that this is the new norm. this is as much a research issue as a philosophical one: as researchers, we should be critical of actions that further diminish our already eroded right to be left alone. since contact tracing and quarantine are enabled and enforced through information systems applications, these topics are firmly in zone for im researchers. person location and person tracking: technologies, risks and policy implications organizational information requirements, media richness and structural design gss for presentation support: supercharging the audience through simultaneous discussions during presentations establishing effective global virtual student teams call for papers: responsible is research for a better world tasks, and communication processes: a theory of media synchronicity punctuated equilibria: an alternative to phyletic gradualism revolutionary change theories: a multilevel exploration of the punctuated equilibrium paradigm i know i can, but can we? culture and efficacy beliefs in global virtual teams bridging space over time: global virtual team dynamics and effectiveness key: cord-288933-q3b0r5ig authors: bushell, mary; frost, jane; deeks, louise; kosari, sam; hussain, zahid; naunton, mark title: evaluation of vaccination training in pharmacy curriculum: preparing students for workforce needs date: 2020-08-20 journal: pharmacy (basel) doi: 10.3390/pharmacy8030151 sha: doc_id: 288933 cord_uid: q3b0r5ig background: to introduce and evaluate a university vaccination training program, preparing final year bachelor of pharmacy (bpharm) and master of pharmacy (mpharm) students to administer vaccinations to children and adults in community pharmacy and offsite (mobile and outreach) settings. methods: final year bpharm and mpharm students were trained to administer intramuscular vaccinations to adults and children. the education program embedded in pharmacy degree curriculum was congruent with the requirements of the australian national immunisation education framework. the training used a mix of pedagogies including online learning; interactive lectures; and simulation, which included augmented reality and role play. all pharmacy students completing the program in 2019 were required to carry out preand post-knowledge assessments. student skill of vaccination was assessed using an objective structured clinical assessment rubric. students were invited to complete pre and post questionnaires on confidence. the post questionnaire incorporated student evaluation of learning experience questions. results: in both cohorts, student vaccination knowledge increased significantly after the completion of the vaccination training program; pre-intervention and post-intervention mean knowledge score (sd) of bpharm and mpharm were (14.3 ± 2.7 vs. 22.7 ± 3.3; p < 0.001) and (15.7 ± 2.9 vs. 21.4 ± 3.2; p < 0.001) respectively. there was no difference between the bpharm and mpharm in the overall knowledge test scores, (p = 0.81; p = 0.95) pre and post scores respectively. using the osca rubric, all students (n = 52) were identified as competent in the skill of injection and could administer an im deltoid injection to a child and adult mannequin. students agreed that the training increased their self-confidence to administer injections to both children and adults. students found value in the use of mixed reality to enhance student understanding of the anatomy of injection sites. conclusion: the developed vaccination training program improved both student knowledge and confidence. pharmacy students who complete such training should be able to administer vaccinations to children and adults, improving workforce capability. mixed reality in the education of pharmacy students can be used to improve student satisfaction and enhance learning. vaccination and injection skills training has been taught in some australian pharmacy degree curriculums since 2013 [1] . indeed, training was being taught in pharmacy schools before pharmacists were administering vaccinations in the practice setting [1, 2] . the rationale for this was that both the profession and pharmacy schools were anticipating regulation change to expand the scope of practice to enable pharmacist-administered vaccination [1] . teaching and upskilling pharmacy students to vaccinate would enable a work-ready graduate. in 2014, queensland became the first jurisdiction, outside a pilot program, to modify regulations to enable pharmacists to vaccinate [3] . since then, regulations across all australian states and territories have been modified to allow appropriately trained pharmacists to administer vaccinations to adults and more recently children aged 10 and over [4] [5] [6] [7] [8] . many pharmacy students across australia have now completed vaccination training embedded within their university degree; however, until march 2019, training was not formally recognized. that is, students would complete university vaccination training, and then, once registered (provisionally or fully, dependent on jurisdictional regulation), complete training delivered by an external provider (e.g., pharmaceutical society of australia or pharmacy guild of australia) to be certified competent to vaccinate [9, 10] . this resulted in duplication of training for many early career pharmacists and an inherent lag time between original knowledge and skills development and administration of vaccines in the practice setting. in march 2019, the australian pharmacy council (apc), the body responsible for the accreditation of pharmacy education in australia and new zealand, published the standards for the accreditation of programs to support pharmacist administration of vaccines version 1.1 [11] . the apc amended the standard to enable pharmacy students enrolled in an accredited pharmacy degree program, to complete a vaccination training program delivered either within the degree program curriculum or via an external provider, during their period of study [11] . this change enabled universities to train and certify students to vaccinate. however, authorization to administer vaccinations is determined by state and territory legislation; at the time of writing, regulations preclude pharmacy student vaccinations in all australian states and territories. however, the move by the apc to recognize vaccination training embedded in pharmacy degrees removes duplicity of vaccination training and enables students to be ready to vaccinate once they register. the scope of practice of the australian pharmacist vaccinator is constantly evolving to include more vaccinations and expand the age groups that pharmacists can vaccinate to. the eligible age of patients that pharmacists can vaccinate varies across jurisdictions. interestingly, even within a state or territory, the eligible age to vaccinate differs between vaccines. from may 2020, appropriately trained pharmacists across all states and territories can administer the influenza vaccine to children aged 10 and over [5, 6, 8, 12] . in most jurisdictions, pharmacists can administer measles-mumps-rubella (mmr) and whooping cough (dtpa) to individuals 16 and over. while in victoria, pharmacists can administer the mmr and dtpa vaccines to people aged 15 years and over [7] . there is a clear trend to lower the age limit eligibility and increase the type of pharmacist-administered vaccinations, improving accessibility and vaccine uptake. more recently, regulation has been modified to enable pharmacists to administer vaccines outside the pharmacy setting via both mobile and outreach services [7] . therefore, it is appropriate that pharmacy students are trained and certified competent to deliver a vaccination service to both children (aged 10 and up) and adults. to date, most australian pharmacy schools have integrated vaccination training into undergraduate and postgraduate pharmacy degrees, with a focus on administering vaccinations to adults [2, 13] . the vaccination training program developed by the authors and evaluated in this paper, used the learning outcomes for the national immunization education framework for health professionals [14] . this paper describes and evaluates the teaching and learning of vaccination training embedded in the pharmacy curriculum at the university of canberra. a vaccination training program (vtp) was developed in line with the national immunization education framework for health professionals (the framework) [14] . this framework was designed to facilitate the development of nationally consistent, quality education programs for australian health professionals, who are not medical practitioners, who want to be recognized as competent to administer vaccinations within their scope of practice. the university vtp adopted the core learning objectives and outcomes from the framework, and then the teaching team adapted them to be pharmacy specific. to do this, the standards and guidelines specific to pharmacy (professional practice standards, practice guidelines for the provision of immunization services within pharmacy) [15, 16] were considered and integrated where appropriate. vaccination training has been embedded in the bachelor and master of pharmacy degrees at the university of canberra since 2016. the training, co delivered by pharmacists, pharmacy and nursing academics (all authorized immunizers), focused on teaching the knowledge and skills to administer vaccinations to adults. in 2019, to ensure that teaching and learning is congruent with contemporary pharmacy practice, this training was expanded to include content and skills assessment of injections to children. as the pharmacist vaccinators did not have, at that point, experience administering vaccinations to children, a nurse practitioner qualified to provide immunizations, delivered the content, theory, skills training, and assessment related to children. pharmacists work as part of a broader health care team. the developed vaccination training program was taught via an interprofessional teaching team, which included pharmacist and nurse vaccinators. with reference to and consistent with the literature on pharmacy student vaccination training, there were a variety of educational pedagogies used to promote understanding and skill competency [13] . teaching included both face-to-face (internal) and non-face-to-face learning opportunities and delivery of content. see table 1 . students were given access to the online non-face-to-face content at semester commencement. this learning material could be completed by students in an asynchronous fashion prior to the intensive workshops. the face-to face content was delivered over four intensive whole day sessions. students were taught the knowledge and skills to administer both im and subcutaneous (sc) vaccinations and how to appropriately manage anaphylaxis. to simulate environments and prepare students for real experience, the training program used the following: role-plays, mannequins, standardized patients, and mixed reality. students had to role play and administer vaccinations to both a pediatric and adult low fidelity mannequin. a mixed reality simulation technique using the microsoft hololens head-mounted devices along with the gigxr applications holohuman and holopatient were used in the face -to-face delivery. the two applications were used to augment the students understanding of anatomy and physiology and to view a simulated patient who was portraying symptoms of anaphylaxis. holohuman is an anatomy application that allows a student to gain a spatial understanding of anatomy and walk through the holographic body. as the student walks through the holographic image, layers of virtual anatomy peel away to reveal the underlying structures. this provided students with a unique way of identifying landmarks (i.e., deltoid muscle) for intramuscular (im) vaccination. it was used to enable students to visualize the shoulder (synovial) joint and to recognize why a shoulder injury related to vaccine administration (sirva) would occur if given too high. mixed reality has the power to engage the learner in a variety of interactive ways, which until this point have not been possible. students skill competency was assessed using an objective structured skills assessment (osca). see appendix a. an authorized immunizer assessed student skill competency to administer a vaccination to both an adult and child mannequin and provided feedback at the end of the assessment. students completed identical pre-and post-knowledge assessments on the content taught on the topic of vaccination. thirty questions assessed understanding of the topics taught. there were questions that assessed knowledge of the national immunization schedule, immunological principles of vaccination, vaccine preventable diseases, the different types of vaccines and how they elicit an immune response, current legislation and regulations related to pharmacist administered vaccination, vaccine cold chain, how to appropriately administer vaccines, documenting the vaccination service, and managing anaphylaxis. to enable matching of the pre-and post-vaccination knowledge tests, while enabling students to be deidentified, students had to provide an answer to questions, such as who was their first teacher and the day of the month they were born, on both the pre and post-tests. all students completing the vaccination training, embedded in the unit pharmacy practice 4, were invited to participate in the evaluation of the training program by completing a hard copy questionnaire at the completion of the training. participating in the evaluation questionnaire was voluntary and no payment or other incentive was provided. the questionnaire was developed by the authors of this paper. questionnaires were face validated by pharmacy and nursing academics, all authorized vaccinators. each evaluation questionnaire included 19 questions that required students to rate their level of agreement on 5-point likert scale (strongly agree to strongly disagree) and two free text questions. one question asked what the student liked about the vaccination training, the other how the vaccination training could be improved. descriptive statistics were conducted. free text responses were analyzed to identify repeating themes. all subjects gave their informed consent for inclusion before they participated in the study. the study was conducted in accordance with the declaration of helsinki, and the project was approved by the human research ethics committee of the university of canberra (hrec17-138). in total, in 2019, 52 students completed the vaccination training. of this, 34 (65.4%) were enrolled in the final year of bpharm and 18 (34.6%) were enrolled in the final year of the m pharm degree. see table 2 . when combined, 19/52 (36.5%) had a current first aid certificate, 12/52 (23.1%) had a current mental health first aid certificate, and 48/52 (92.3%) were currently working in a pharmacy. see table 3 . there was no association between working in pharmacy, having a current first aid certificate and/or mental health first aid certificate and the mean knowledge score of the pre-test. the only statistically significant finding was that students who held a first aid certificate performed better than students who did not have a first aid certificate on the post-knowledge test (p = 0.014). the mean pre-intervention knowledge score for the cohort was 14/30, while the post intervention knowledge score was 22/30. the difference in mean vaccination knowledge scores pre and post educational intervention was better (p < 0.001) with a large effect size (cohens d = 0.75). see table 2 . the results show that there was no statistically significant difference between the scores for the knowledge assessment between bachelor and master cohorts. bpharm students mean score pre-educational intervention was 15/30, and for master of pharmacy students it was 16/30. the mean score post-intervention was 23/30 for b.pharm students, and 21/30 for m.pharm students (p = 0.95). using the osca rubric, all students (mpharm and bpharm) completing the training program were identified as competent in the skill of injection. all students (n = 52) scored a yes against the 25 criteria of the osca rubric (appendix a). all students could administer an im deltoid injection to a child and adult mannequin. all students (n = 52, 100%) either agreed (16/52, 31%) or strongly agreed (36/52, 69%) that the vaccination training enhanced their knowledge of vaccination. all students (n = 52, 100%) either agreed (6/52, 11%) or strongly agreed (46/52, 89%) that the practical session of administering a vaccine was useful/beneficial. all students (n = 52, 100%) either agreed (9/52, 17%) or strongly agreed (43/52, 83%) that the practical session increased their confidence to administer vaccinations. when asked 'i feel confident that i know the correct vaccination technique for both adults and children', one student (1.9%) responded neutral, 29/52 (58%) agreed and 22/52 (42%) strongly agreed. students voiced value in having the content delivered by an interprofessional teaching team, which included pharmacists and nurses. a sample of students provided simple but positive comments like: "good teaching team" pharmacy student a. students were both satisfied and valued the integration of mixed reality in the vaccination training. students voiced that it helped with the understanding of certain concepts, for example, shoulder injury related to vaccine administration (sirva). from the feedback evaluation form: "walking into the holohuman was really neat. i liked that the layers of the human peeled away and it felt like i looking inside a human layer by layer. it helped my understanding not only of anatomy but the importance of making sure when administering an injection, i administer it in the right spot." pharmacy student b. students' knowledge significantly increased post the educational intervention vaccination training. there was no difference between bpharm and mpharm student knowledge pre or post education intervention. this indicates that delivery of the training program in the final year of both degrees enables comparable understanding of the content and skills taught and a work ready graduate. one finding was that students who had completed a first aid certificate, had higher post vaccination training knowledge scores. this finding is interesting as students did not have a higher mean pre knowledge test score. one possible reason for this is that students complete first aid training as adjunct training, that is, while it is recommended, it is not compulsory for students to complete. students have gone above expectation to complete the training and have demonstrated commitment to continuing education. this attitude to study may be extrapolated to their commitment to the vaccination training and the larger unit in which the training is embedded. many recent studies have shown that an individual's grit, perseverance and passion for long term goals is associated with higher academic grades [17, 18] . the training employed a range of teaching pedagogies to promote student understanding, skill competency and confidence. simulation is a learner-centred educational pedagogy that facilitates learning by exposing the learner to a situation which is based on or mimics a real-life event. simulation includes a broad range of activities. the use of simulation as an educational tool enables experiential learning and constructivism. it provides students with an opportunity to create their own meaning and co-construct knowledge in a safe environment, taking knowledge learnt from the lectures to application of the skill. consistent with the literature, this teaching evaluation shows that student's value and like to learn using simulation when being taught medical skills, such as vaccination [19] . students learnt how to administer both an im vaccination to a child and adult low fidelity mannequin. a recent study showed that the use of high-fidelity simulation mannequins as a teaching tool resulted in lower or equal student performance of clinical skills when compared to low-fidelity simulation mannequins [20] . the use of the different low fidelity mannequins enabled students to learn how to best position themselves to administer the vaccination safely at a 90-degree angle. students were educated to be seated when administering a vaccine to a seated individual. using appropriately sized mannequins showed students why vaccinations given by a standing immunizer to a seated individual are linked with increased risk of being administered too high. using the different mannequins provided students with a safe environment to problem-solve and learn prior to practicing in the professional setting [21] . mixed reality (mr) is an emerging technology in health care education [22] . consistent with previous studies exploring student acceptability, students enjoyed and valued the integration of this teaching tool in the vaccination training [23] . to correctly identify the im deltoid injection site, pharmacy students were taught to use anatomical markers. they were taught to create a 'triangle' over the individuals' deltoid with their fingers, the centre of which the injection is administered. they were educated to locate the acromion process (shoulder tip) and place their index finger (or first finger) along it. then to place the second and third finger down underneath the index finger (the third finger becomes the base of the triangle). the fourth finger is then opened to create a 'side' of the triangle. in the middle (not the top or the bottom) of the triangle, the injection should be made. using mr enabled students to peel away the body to see these important anatomical markers, contextualizing and providing insight as to why the content was taught and the potential outcomes of incorrect vaccine administration. students using the mixed reality anatomical software could visually see, using the 3d animation, that vaccines that are administered too low can be injected into the radial nerve, while vaccines that that are given too far to the side can cause damage to the axillary nerve. this highlights the importance of administering the vaccination into the correct area. role play enabled the students to step through the process of delivering the vaccination service and encouraged both peer learning and formative feedback. using role play, students learnt to communicate appropriate, evidence-based information about benefits and the potential risks of vaccination and obtain valid consent. the active learning approach is widely used in higher education and allows learning across cognitive, psychomotor, and affective domains [24] . pharmacists work as part of a broader health care team. one strength of the vaccination training program was that it used an interprofessional teaching team. in doing so, students were provided with an opportunity for interprofessional learning and practice. this strengthened program delivery and enabled students to see the value of interprofessional collaborative practice, which will continue in the practice setting [25] . while both internationally and nationally there are only a small number of studies published on the delivery and evaluation of pharmacy vaccination training in pharmacy schools, the results of this study are consistent with other published evaluations [2, 26] . across the studies, students' value gaining new knowledge and skills and report confidence to administer vaccinations on completion of training [2, 13, 26] . consistent with this study, the two australian studies that outline the development of a pharmacy vtp also used national pharmacy standards to inform material [2, 26] . in contrast, where this study reports adopting the learning outcomes from the national immunization education framework for health professionals, the other studies did not. this makes direct comparison between the training programs difficult. there are currently varied approaches to when vaccination training is embedded in degree programs. there are pharmacy schools that embed vaccination training in the earlier years of the curriculum, to enable students to see themselves as future 'clinical' health professionals [26, 27] . in the published studies, students report satisfaction and skill acquisition. they also report enabling students to see that they will be touching patients as part of their future professional role when providing care [28] . given pharmacy students in australia, can now complete a university vaccination training program and, when jurisdictional regulation allows, vaccinate without needing to complete additional training by an accredited external provider, the best year level to embed vaccination training should be further researched. in doing so, an evidence-based and consistent decision can be made about the best year level to introduce the training. further, to date, research in australia has not directly compared the delivery, skill acquisition, confidence, and competence of students across different university vaccination training programs, this too should be further explored. students, like authorized immunizers, completed vaccination training that is congruent with the national immunization education framework for health professionals and demonstrated competency. there is a case for jurisdictional regulations to be modified to enable pharmacy student-administered vaccinations. international research demonstrates that pharmacy students, under the supervision of a credentialled vaccinator, can administer vaccines safely [28, 29] . the application of skill acquisition in the clinical setting improves students' self-confidence [27, 30] . having pharmacy students ready to vaccinate on placement would enable a more work-ready graduate and a critical workforce that can be used to promote vaccination uptake. this skill is likely to be of greater use particularly in times when vaccination demand is high, for example a pandemic. if and when the covid-19 vaccine is available, mass immunization is likely to be needed in a relatively short period of time to mitigate the spread of the disease and enable international borders to reopen. pharmacy students could be used to improve workforce capability in australia. the vaccination training program described in this paper was embedded in the final year of mpharm and bpharm curriculum and enabled the same skill competency and knowledge acquisition across cohorts. the training program incorporated a suit of teaching methods, including mixed reality, which had high student acceptability. aligning with the changed scope of practice for australian pharmacists, pharmacy students learnt how to administer vaccinations to both adults and children. international research demonstrates that pharmacy students, under the supervision of a credentialled vaccinator, can administer vaccines safely. given student competency and readiness after completing vaccination training, there is a case for jurisdictional regulations to be modified to enable pharmacy student-administered vaccinations in australia. case for pharmacist administered vaccinations in australia current research: incorporating vaccine administration in pharmacy curriculum: preparing students for emerging roles australia's first pharmacist immunisation pilot-who did pharmacists inject? act government. medicines, poisons and therapeutic goods (vaccinations by pharmacists) direction poisons and therapeutic goods regulation queensland government. health (drugs and poisons) regulation 1996, drug therapy protocol-pharmacist vaccination program victorian pharmacist-administered vaccination program expansion communique northern territory government. qualifications prescribed for pharmacist to supply and administer schedule 4 vaccine. in department of health; northern territory government evaluation of a vaccination training program for pharmacy graduands in australia current research: a shot in the arm: pharmacist-administered influenza vaccine in new south wales standards for the accreditation of programs to support pharmacist administration of vaccines version 1.1 pharmacist-led immunisation in the northern territory: results from the pilot study development and design of injection skills and vaccination training program targeted for australian undergraduate pharmacy students national immunisation education framework for health professionals practice guidelines for the provision of immunisation services within pharmacy national competency standards framework grit: perseverance and passion for long-term goals building grit: the longitudinal pathways between mindset, commitment, grit, and academic outcomes the history of medical simulation high-fidelity is not superior to low-fidelity simulation but leads to overconfidence in medical students a brief history of the development of mannequin simulators for clinical education and training virtual reality in pharmacy: opportunities for clinical exploring the application of mixed reality in nurse education exploring the potential of role play in higher education: development of a typology and teacher guidelines interprofessional collaborative practice for medication safety: nursing, pharmacy, and medical graduates' experiences and perspectives managing vaccine-associated anaphylaxis in the pharmacy an introductory pharmacy practice experience emphasizing student-administered vaccinations integration of first-and second-year introductory pharmacy practice experiences introductory and advanced pharmacy practice experiences within campus-based influenza clinics a literature review of the impact of pharmacy students in immunization initiatives the authors would like to thank louise mcclean for helping to assess students. the authors declare no conflict of interest. the funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results. pharmacy 2020, 8, 151 appendix a table a1 . osce: administration of medication by intra-muscular injection (imi). demonstrates: the ability to administer imi into the deltoid muscle. assessor question: where would you find information regarding the drug class, action and side effects of this drug? key: cord-298778-wnbqqzot authors: hamza, marwa s.; badary, osama a.; elmazar, mohamed m. title: cross-sectional study on awareness and knowledge of covid-19 among senior pharmacy students date: 2020-06-15 journal: j community health doi: 10.1007/s10900-020-00859-z sha: doc_id: 298778 cord_uid: wnbqqzot extraordinary actions have been implemented in an effort to control the rapid spread of the ongoing covid-19 epidemic in egypt. people’s adherence to control measures is influenced by their knowledge, attitudes and practices towards the disease. therefore, in the present study we assessed pharmacy senior students’ knowledge, attitudes and practices towards the covid-19 pandemic. an online questionnaire was created and it consisted of 12 questions testing their knowledge about covid-19 clinical characteristics, transmission routes and prevention and control steps. among senior pharmacy students (n = 238), 70% were females and 63% were living in greater cairo. their main source of information included social media (70%), published articles (48%) and television (48%). the overall correct knowledge score was 83%. most of the students displayed a good covid-19 knowledge level (72.5% of the students). the students were least informed when trying to answer questions about hyper-coagulation, as a major cause for death in patients with severe covid-19, and about the timings on the necessity to wear masks. assessment of students’ attitudes and practices towards covid-19 reflected that 87% of them were confident that health care teams and scientists could win the fight against the virus. in addition, 72% of students agreed that covid-19 will be controlled successfully. the greater the students’ knowledge, the more confident they felt that covid-19 will be controlled successfully (or 2.2, 95% confidence interval [ci] 1.03–4.72). good behavioral practice towards covid-19 control was confirmed when 87% of students answered that they didn’t go out to any crowded place. females were 3.6 times (95% confidence interval [ci] 1.03–3.11) more likely to avoid going out than males. bad behavioral practice became evident when approximately 50% of students admitted that they did not wear masks when they left their house. therefore, more efforts should be taken to protect future pharmacists from this pandemic. an epidemic, triggered by the new coronavirus sars-cov-2, has recently become the focus of the scientific community attention [1] . covid-19 represents the illness generated by this virus. its clinical presentation ranges from being an asymptomatic infection to developing into a severe disease with high mortality rate [2] . at the time of writing (may 16, 2020), over 4,425,485 covid-19 cases and 302,059 death have been registered worldwide [3] . in particular, in egypt, there are 11,228 positive individuals and 592 death cases [4] . now, covid-19 has been classified as a pandemic by the world health organization. it is both a highly contagious and a life-threatening disease [3] . its reproduction ratio (rr) is defined as the expected number of cases generated by a single infected individual within a susceptible population [5] . it varies between 2.4 and 2.7 [6] . these values are high compared to influenza (rr = 1.3-1.8) [7] . currently, a fight is being undertaken against the covid-19 pandemic. adherence to control and prevention steps is paramount for ensuring a complete success over covid-19. lessons were learned from the 2003 sars outbreak. these indicate that fear exists within the population awareness and attitude to infectious diseases. this factor may hinder further efforts to prevent the virus spread. therefore, it is crucial to understand covid-19 public's awareness at this critical stage to facilitate pandemic management [8, 9] . as members of the health care team, pharmacists play an essential role in patient care. this includes contributing to the decision-making process within the multidisciplinary health care team, taking responsibility for medication administration and assessment of patient care [10] . moreover, together with physicians, pharmacists' knowledge, attitude and practices are crucial to prevent and control the disease [11] . health care teams include pharmacists are responsible for providing knowledge, delivering good quality management and protecting individuals from illness during epidemic prevalence period. therefore, pharmacy students' education should include training in precautionary measures, effective treatment and follow-up. this is critical together with their behavior in these fields [12] . normally, training in disaster medicine occupy a very small place in regular medical curricula worldwide [13] . therefore, the present study was carried out to evaluate knowledge about covid-19 as an attempt to identify senior students' degree of awareness and its effects. furthermore, this study objective aims to contribute in developing a fit for purpose education program that creates awareness among future practitioners. to the best of the authors' knowledge, this is the first study directly exploring senior pharmacy students' knowledge, attitudes and practice regarding this topic. this is an institutional based cross sectional study conducted in faculty of pharmacy at the british university in egypt during the period from april 28, 2020 to april 30, 2020. the targeted population included students who had completed ≥ 85% of the credit-hours required for graduation. the sample size was calculated by using the online sample size calculator raosoft®. based on an estimated population of 470 students in the final years (year 4 and 5) and on the anticipated response of 50%, the minimum required sample size was 204 participants with a confidence level of 95% and a 5% margin of error. a pre-validated questionnaire [14] was modified after a comprehensive literature search and based on the most recent available information from the world health organization, the center for disease control and prevention (usa) and the egypt ministry of health. the initial draft was sent to a group of experts, chosen according to their experience and expertise in related fields, to appraise the questions in terms of relativity, simplicity and importance. then, the questionnaire was sent by email, through survey monkey software®. a pilot study was conducted on 10 subjects, to test the questionnaire validity. following a group discussion, the questionnaire was completed. the data from the pilot study was removed from the final analysis. the questionnaire included two sections: demographics, and knowledge, attitudes and practices questions (kap). demographic variables included age, gender and place of current residence (cairo, other governorates in egypt). the first part of the kap represented the knowledge investigation, consisting of 12 questions ( fig. 1 ): 4 questions were about covid-19 clinical presentations (k1-k4), 3 asked about transmission routes (k5-k7), and the following 5 questions quizzed on prevention and control (k8-k12). these questions were answered on a true/false basis, including an additional "i don't know" option. one point was assigned to a correct response, while no points were given for an incorrect/unknown answer. the total score ranged from 0 to 12, with a higher score suggesting better covid-19 awareness. in the pilot sample, the knowledge questionnaire cronbach's alpha coefficient was 0.73, indicating reasonable internal consistency [15] . attitudes towards covid-19 were appraised with 2 questions (a1-a2, fig. 2 ) that asked about the level of agreement on the covid-19 control implementations and the confidence perception for winning the war against this coronavirus. the assessment of students practices was performed through 4 questions (p1-p4, fig. 2 ), that investigated their actions towards going to a crowded place, wearing masks when going out in recent days, maintaining social distance between others and if they agree that patients should disclose their exposure to covid-19. finally, one last question was added to inquire about the students' source of covid-19 information. before study start-up, permission was taken from the ethics committee of the faculty of pharmacy at the british university in egypt. the participants were briefed about the study rationale and were reassured about the confidentiality safeguards for their personal information and responses. the answered survey submission was considered as consent for study participation. respondents' participation was completely consensual, anonymous and voluntary. the completed questionnaires data were analyzed using the statistical package for social science, version 21.0 (spss 20.0, chicago, il). the significance level was set at a value of p < 0.05 for all analyses. initially, all information gathered through the questionnaire was coded into variables. in the pilot study, the cronbach's alpha coefficient test was used to measure internal consistency. data normality was tested using the kolmogorov-smirnov test. descriptive and inferential statistics, involving chi-square test, mann-whitney u test, correlation and logistic regression, were used to analyze the results. fever, tiredness, and dry cough, k2 some patients infected with the covid-19 virus may suffer from aches and pains, nasal congestion, runny nose, sore throat or diarrhea, k3. coagulation dysfunction is one of the major causes for death in patients with severe covid-19, k4. older persons and persons with pre-existing medical conditions (such as high blood pressure, heart disease, lung disease, cancer or diabetes) appear to develop serious illness with covid-19 more often than others, k.5 people with covid-19 will not transmit the virus to others when fever is not present, k6. covid-19 can spread from person to person through small droplets from the nose or mouth which are spread when a person with covid-19 coughs or exhales, k7. only wear a mask if you are ill with covid-19 symptoms or looking after someone who may have covid-19, k8. it is not necessary for children and young adults to take measures to prevent the infection by the covid-19 virus, k9. to prevent the infection by covid-19, individuals should avoid going to crowded places such as public transportation, k10. covid-19 is caused by a virus, so antibiotics do not work. antibiotics should not be used as a means of prevention or treatment of covid-19. they should only be used as directed by a physician to treat a bacterial infection, k11. isolating and treating people with covid-19 infections are effective means of minimizing viral spread, k12. people who have contact with someone who has covid-19 infection should be isolated in a proper place immediately. the observation period is usually 14 days in the present study, the scoring system and categorization were utilized to assess knowledge levels towards the covid-19 pandemic. the questions determination was based on one point for every correct answer and zero points for false answers. by adding the results of all the knowledge questions in the survey, the participants mean knowledge scores (kss) were determined. ks ranged from 0 to 12, with the higher scores demonstrating an increased level of pandemic covid-19 knowledge. knowledge level categories were defined by an 80% cut-off point as any score above 80% is good and any score below 80% is poor [16] . during the course of this study, a total of 248 participants completed the devised online survey questions. after excluding 10 respondents who participated in the pilot study, the final sample consisted of 238 participant students. among these, the average age (measured in years) was 22 ± 1, 166 students (70%) were females. when identifying the place of residence, 149 students (63%) were living in greater cairo while 89 (37%) resided in other governorates (table 1) . with regards to the students' source of information, results disclosed that the main source of information was social media (70%) followed by published articles (48%) and television (48%). in the present study, the main objective was to measure covid-19 related knowledge of pharmacy senior students. the mean score of covid-19 related knowledge was 83% (10 ± 1.2, ranging from 4 to 12) (table 1 ). in particular, 72.5% of the students possessed a good knowledge level, while 23.5% displayed poor knowledge based on an 80% cut-off point [16] . the first 4 survey questions (k1-k4) measured students' knowledge towards covid-19 clinical presentations. the highest correct answer rate (98%) related to the covid-19 related knowledge items identified by 2 questions; 'the main clinical symptoms of covid-19 are fever, tiredness, and dry cough (true)' and 'older persons and persons with pre-existing medical conditions (such as high blood pressure, heart disease, lung disease, cancer or diabetes) appear to develop serious illness with covid-19 more often than others (true)' (fig. 1) . however, the item with the lowest correct-answer rates was: 'coagulation dysfunction is one of the major causes for death in patients with severe covid-19 (true)' (37%) (fig. 1 ). the answer 'i don't know' was selected by 12% of students for the question 'covid-19 may cause aches and pains, nasal congestion, runny nose, sore throat or diarrhea in some patients' (fig. 1) . then the following 3 questions (k5-k7) measured students' knowledge towards the transmission routes. in this case, the percentage of knowledgeable students is high. k.5: no transmission if fever is not present (88%); k6: covid-19 can spread from person to person through small droplets (98%); and k7: wear a mask only if you are ill or looking after someone who may have covid-19 (96%). a similar result was achieved with the next 5 questions (k8-k12), which concerned assessing knowledge about covid-19 prevention and control (fig. 1) . this study showed a significant relation between the knowledge score and published articles as source of information (p < 0.05). the students' attitude was measured with 2 questions. the highest positive result was attained by the following covid-19-related attitude item: 'are you confident that the health care team and scientists can win the fight against the covid-19 virus?' (87% answered yes). however, the lowest positive attitude result corresponded to the following: 'do you agree that covid-19 will be successfully controlled?' (only 72% answered yes) (fig. 2) . the highest performance rate was attained by the following covid-19-related practice item: 'in recent days, have you gone to any crowded place?' (87% answered no). however, the item with the lowest performance rate was: 'in recent days, if you left your house, have you worn a mask when leaving home?' (50.5% gave a negative answer) (fig. 2) . as expected, it was found that the covid-19 practice for going out in crowded place was significantly different between males and females: females displayed a lower tendency to go out than males (p < 0.05). in addition, the practice for keeping social distance between students was significantly different between males and females: females maintained social distance more successfully than males (p < 0.05). it is worth mentioning that significant relations existed between the two practices and the students' source of information. in particular, the practice of wearing a mask when leaving home was directly correlated with using television as source of information (p < 0.05). furthermore, the practice of keeping social distance of at least 2 m from other people was directly associated with utilizing physicians' advice as source of information (p < 0.05). moreover, the correlation coefficient, existing between the major knowledge, attitude and practice variables was analyzed. a weak positive correlation was observed between attitude and practice and this result appeared to be highly significant (r = 0.2; p < 0.01). however, no correlation appeared to exist between knowledge and attitudes or knowledge and practice (table 2) . logistic regression analyses showed that students with a good knowledge score are 2.2 times more optimistic and agreed that covid-19 will be successfully controlled (95% confidence interval [ci] 1.03-4.72). students who live in cairo were 1.7 times more likely to maintain social distance of at least 2 m from other people (95% confidence interval [ci] 0.9-3.2)., and 1.7 times more likely to agree that patients should disclose potential covid-19 exposure, than students who live in other governorates in egypt (95% confidence interval [ci] 0.9-3.3). moreover, female students were 3.6 times more likely to avoid going to any crowded place than male students (95% confidence interval [ci] 1.5-8.6), and 2.3 times more likely to keep their social distance of at least 2 m from other people than male students (95% confidence interval [ci] 1.2-4.4) ( table 3 ). the coronavirus disease 2019 (covid-19) is an emerging contagious respiratory disease caused by a novel coronavirus. it was first detected in december 2019 in wuhan, china [17] . emerging infectious diseases like covid-19 can occur anywhere in the world. by being part of the health care team, pharmacists can be exposed to the risk of infection [18] . here, we present the results of a questionnaire study about the knowledge, attitudes and practices of egyptian senior pharmacy students towards the covd-19 disease. in the present study, one of the most significant findings was that the highest main source of covid-19 information among senior students was social media, television and published articles. the results strongly support similar findings in which the main source of middle east respiratory syndrome (mers) information was reported to be the internet and social media [19] . this clearly indicates that senior pharmacy students are relying consistently on the internet, social media and online information as the principal ways to obtain information, compared with friends and other resources. the egyptian ministry of health and population and the world health organization (who) have provided clusters with covid-19 information through their websites and are recommending everyone to be aware of updates relating to knowledge about covid-19 [3, 4] . similarly, a previous study reported that internet was the main source of information about mers-cov [20] . therefore, it is feasible to suggest that researches should urgently find ways to utilize the internet to promote health, especially in emerging infectious diseases and pandemics. at present, the quality of the information available on the internet is insufficient and needs to be revised. furthermore, a very small percentage of participants reported that healthcare professionals, either physicians or pharmacists, were their primary source of information. this scenario illustrates the lack of healthcare team engagement in increasing the public awareness regarding different aspects of covid-19. the present study showed a significant relation existing between the knowledge score and published articles as source of information (p < 0.05). that may be due to the role played by the who and the egyptian ministry of health and population in providing cluster with covid-19 information through their websites. therefore, these organizations recommend everyone to be aware of updates about covid-19. as mentioned earlier, 238 senior pharmacy students participated in the present study. the overall mean knowledge scored among students was 83% (10 ± 1.2). most of the students possessed a good covid-19 knowledge level (72.5% of the students), possibly owing to the fact that the egyptian ministry of health and population (mohp) and the world health organization (who) are providing a significant amount of covid-19 information through their websites [3, 4] . the present study results showed that 98% of participants were knowledgeable about the main clinical symptoms of covid-19. in addition, the same percentage of students knew that older persons and persons with preexisting medical conditions were more liable to develop serious illness with covid-19 than others. these findings confirm observations from previous studies, which reported that the majority of participants believed that the disease is more dangerous for the elderly and for those suffering from chronic diseases [21, 22] . the students were least knowledgeable towards the question of coagulation dysfunction being one of the major death causes in patients with severe covid-19. suitable remedial actions should be taken since vascular, venous thromboembolism and arterial thrombosis risks have been documented in severe and critically ill covid-19 patients [23, 24] . a perceived lack of information regarding some of the covid-19 clinical presentation or preventive measures highlights the requirement for the inclusion of more training, seminars in their courses. this raises concerns regarding the ability of newly-graduated pharmacists to deal with the covid-19 pandemic disease, or any other potential similar epidemic events. also, the students were least knowledgeable about when to wear masks. masks and sanitizing products use has evolved since the beginning of the coronavirus pandemic, resulting in their rapid depletion. lack of appropriate security measures represents a major concern for health workers. this is a cause of concern in a highly populated country such as egypt, where the public may experience anxiety and even panic feelings, because basic protection measures are unavailable [25] . knowledge is a prerequisite for establishing positive attitudes and promoting positive behaviors. therefore, it makes sense to invest in individuals' cognition towards the infection. regarding the senior students' attitude towards covid-19, in the present study 72% of them agreed that covid-19 will be successfully controlled and 87% were confident that health care teams and scientists can win the fight against the coronavirus. interestingly, it was found that a greater students' knowledge relates to more confidence in covid-19 becoming successfully controlled (or 2.2, 95% confidence interval [ci] 1.03-4.72). correlation between the students' attitude and practice indicates that their positive attitude is affecting their action towards covid-19. this present study results showed that 87% of participants did not go to any crowded place, 76% of the students agreed that patients should disclose their exposure to covid-19 and 73% of the students kept the social distance of at least 2 m from other people. however, the students showed bad practice when using masks as a protective measure (approximately 50%). that was directly related to their lower level of knowledge towards the importance of wearing masks (p < 0.05). that is supported by findings from a previous study, which reported that only about 35% of the participants were willing to wear face masks [21] . interestingly, it was reported that females preferred not to go to any crowded place more than male students (or 3.6, 95% confidence interval [ci] 1. 5-8.6 ). in addition, females preferred to keep the social distance more than male students (or 2.3, 95% confidence interval [ci] 1.2-4.4). moreover, students who live in other governorates kept social distance and agreed that patients should disclose their exposure to covid-19 rather than the students who live in the capital (or 1.7, 95% confidence interval [ci] 0.9-3.2). it is noteworthy that a significant association existed between the practice of wearing masks when leaving home and using television as source of information (p < 0.05). likewise, an association existed between the practice of keeping the social distance at least 2 m from other people and utilizing physicians as a source of information (p < 0.05). this encourages the public to avoid public meetings, including sports, ceremonies, meetings and school classes, in order to prevent a global transmission of coronavirus infection [26] . however, many people ignore the importance of maintaining social distance because of attitudinal issues. it can be concluded that senior pharmacy students are generally knowledgeable about covid-19. they lack knowledge about some of the clinical presentation of severe covid-19 infection such as hyper-coagulation. they also lack knowledge about the indication of wearing face masks. the present study constitutes the first attempt to examine knowledge and perceptions of future pharmacists in egypt on pandemic diseases. finally, the analysis of senior pharmacy students' knowledge and the factors affecting their attitudes and practices towards covid-19 could provide a reference for preventing further spread of this disease. it is recommended that strategies and education programs should be implemented to reduce the risk of infection among future pharmacists. senior pharmacy students should acquire increased awareness regarding the seriousness and consequences of a covid-19 infection. some of the results of this study were compared with research on other new emerging infectious diseases similar to coronavirus, because no analyses have been conducted concerning pharmacy students or students in general. covid-19-navigating the uncharted the loss of igm memory b cells correlates with clinical disease in common variable immunodeficiency covid-19) situation report-117. situation-reports/20200311-sitrep-51-covid-19.pdf?sfvrsn=1ba62e57_10 2020 covid-19 overview pandemic potential of a strain of influenza a (h1n1): early findings the reproductive number of covid-19 is higher compared to sars coronavirus estimates of the reproduction number for seasonal, pandemic, and zoonotic influenza: a systematic review of the literature college students' knowledge, attitudes and adherence to public service announcements on ebola in nigeria: suggestions for improving future ebola prevention education programmes the impact of knowledge and attitudes on adherence to tuberculosis treatment: a case-control study in a moroccan region health care professional development: working as a team to improve patient care knowledge, attitudes, and practices regarding pandemic h1n1 influenza among medical and dental residents and fellowships in shiraz iran awareness of the pandemic h1n1 influenza global outbreak 2009 among medical students in karachi pakistan disaster 101: a novel approach to disaster medicine training for health professionals knowledge, attitudes, and practices towards covid-19 among chinese residents during the rapid rise period of the covid-19 outbreak: a quick online cross-sectional survey the use of cronbach's alpha when developing and reporting research instruments in science education knowledge, attitude and practice regarding dengue fever among the healthy population of highland and lowland communities in central nepal epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in wuhan, china: a descriptive study community pharmacists: on the frontline of health service against covid-19 in lmics knowledge and attitude of healthcare workers about middle east respiratory syndrome in multispecialty hospitals of qassim saudi arabia identification of information types and sources by the public for promoting awareness of middle east respiratory syndrome coronavirus in saudi arabia knowledge, perceptions, and attitude of egyptians towards the novel coronavirus disease (covid-19) covid-19 patients' clinical characteristics, discharge rate, and fatality rate of meta-analysis anticoagulant treatment is associated with decreased mortality in severe coronavirus disease 2019 patients with coagulopathy clinical characteristics of coronavirus disease 2019 in china mass gathering events and reducing further global spread of covid-19: a political and public health dilemma. the lancet the researchers acknowledge our senior pharmacy students who took part in this study. the authors declare that they do not have conflict of interests. key: cord-313140-8043wvpa authors: cain, robert; shafer klink, dane; yunus, samina title: telemedicine implementation in family medicine: undergraduate clerkship during covid‐19 pandemic date: 2020-09-22 journal: med educ doi: 10.1111/medu.14313 sha: doc_id: 313140 cord_uid: 8043wvpa nan the covid-19 pandemic forced cancellation of student clinical activities abruptly in march 2020. clinical patient contact was needed to supplement the online curriculum that was implemented, whereas students were barred from clinical settings. introducing medical students to virtual visits with patients as an alternative to real-time patient contact during the covid-19 crisis. shortly before the onset of the pandemic, our organisation had prioritised the implementation of 'virtual visits' using a proprietary video platform as a method of patient population management. with the onset of the pandemic in our region, these types of visit quickly became the standard format for out-patient visits. we developed a pilot programme to see whether students in their family medicine clerkship would benefit from patient interactions in these visits. telemedicine has become an increasingly adopted part of undergraduate medical training in the united states, 1 but had only been available on an elective basis at our school before this. student volunteers were recruited from the cohort scheduled to begin their clerkship in early april. they worked with clinical faculty conducting virtual visits individually or jointly with the faculty. students worked remotely to ensure their safety. patients were selected in advance and screened for their consent to participate and to ensure they had the appropriate virtual interfaces. students were able to review charts in advance via their remote access to our ehr platform (epic). ongoing feedback was provided by text messages or by the chat function of the ehr. at the conclusion of the visit, students were able to document the visit in the chart, and this was reviewed and attested by the attending. students readily adapted to this type of patient visit. a postparticipation survey (1 = strongly disagree and 5 = strongly agree) of the students revealed: connecting was easy and intuitive (3.80/5), meaningful connection with the patient was established (4.20), format was sufficient to demonstrate clinical skill to preceptor (3.60), student felt confident in ability to thoroughly assess patient's complaints in virtual setting (3.80), and preceptor was able to teach sufficiently using virtual format (4.40). student participants felt that the experience was a meaningful introduction to telemedicine, provided an excellent opportunity to work on interviewing skills and provided a chance to work on a virtual team for the first time. technical obstacles were very minor. they expressed the importance of communication with the attending physician before the clinic, and a post-clinic phone session for feedback and additional learning suggestions. physical examination options were limited in this format. students and faculty alike felt the programme was a valid format for clinical experiences during clerkships and endorsed expanding it to all students training in family medicine at our institution. https://orcid.org/0000-0002-2553-563x telemedicine training in undergraduate medical education: mixed-methods review key: cord-303569-nsh9sv34 authors: mendez-reguera, aniela; lopez cabrera, mildred vanessa title: engaging my gen z class: teaching with memes date: 2020-09-09 journal: med sci educ doi: 10.1007/s40670-020-01078-w sha: doc_id: 303569 cord_uid: nsh9sv34 fostering engagement in a traditional setting with generation z learners was difficult enough; to do so online, while most educators are battling with technology, poses an even harder challenge amid the pandemic. in an unconventional assignment, students were asked to create an internet meme to explain any immunology class themes. members of gen z, true digital natives, have been navigating the internet and social networks, and immersed in video platforms since they were born. they have been described as pragmatic and socially active individuals. most of their interaction has transferred from face-to-face to digital environments which have made them excel in representing complex situations in one-image or slang expression that makes online communication more efficient. every time an element of a movie or expression goes viral, they are present. they live and understand these viral phenomena which shape their everyday lives and create an impact in their psyche. the current pandemic has forced educators, from baby boomers to millennials, to plunge into an online model. if fostering class engagement in a traditional classroom setting with gen z learners was difficult, do so online, while battling with technology teachers sometimes are not used to, poses an even harder challenge during this pandemic. understanding the key features of the learners' generation allows educators to define a strategy to approach them, as well as to understand and implement tools that could benefit the teaching-learning process. in an unconventional effort to promote class engagement and assess student understanding of the content of the immunology class, an unorthodox assignment was given. this assignment was set as optional and had no impact on course grades, and consisted of students creating a meme of their authorship and uploading it to an online discussion board [1] . these sarcastic images, part of the student's environment, have become widely known and part of pop culture and the extent to which they shape the students' perception is still unknown. teachers sometimes overlook the utility they have to create an impact on education and promote students' engagement. the assignment required that the image depict or explained any of the second term themes or content: complement, phagocytosis, inflammation, antigen presentation, natural killer (nk) cells, etc. a positive response resulted in 64 memes designed by 45 students, some of them posting more than once. informal student feedback on the meme activity was positive: "it was an entertaining assignment. through the weeks i was even collecting memes of my social media feed because i thought they would be useful to explain the processes we were discussing in class." one student mentioned that she valued this innovation as an extra-effort that the teacher was making on adapting the class to a full-online setting. another student mentioned that when creating the meme, he worked hard not only on presenting the information but also on how to communicate effectively using visuals. the results were presented in the class web-conference session, and students voted for their favorite. the most voted image was the "two spiderman meme" to explain how the c3 convertase of two complement activation pathways are molecularly different but functionally the same. by making this meme, students showed the understanding that c3 molecules of different complement pathways may be different in composition, such as the classical and lectin pathway c3 convertase composed of c4b+2a molecules and the alternative pathway c3 convertase composed of c3b+ bb molecules, but their main function is the same. the runnerup contribution was a scene in the harry potter saga where professor mcgonagall asks ron, harry, and hermione why they are always getting into trouble. the student inserted over each wizard's character the name of neutrophil, macrophage, and dendritic cell, which are crucial in the initiation of immune responses. this showed that the student achieved an understanding of inflammation and the crucial cells in this process. there were also shrek references, and the recent "coffin dance meme" to represent the last thing a virusinfected cell sees before it wa7.ss killed by an nk cell. although there are many paths nk cells use to destroy infected cells, students identified the ultimate result of an nk cell with an infected or neoplastic cell. students made clever connections using these visual representations which are part of their digital environments, with what was discussed in the classroom. although regardless of their generation, students and educators may prefer different learning modalities for example synchronous discussions, flipped classroom approaches, or journal clubs, but relating academic knowledge and digital resources becomes a strategy in the repertoire of available tools to engage with students after this pandemic. through the lens of popular culture: why memes and teaching are well suited key: cord-311403-rbia88vf authors: crawford, andrew; blitch, alejandro; lindsley, janet e.; dickerson, ty t. title: embracing uncertainty: covid‐19 exploration in real time date: 2020-09-20 journal: med educ doi: 10.1111/medu.14320 sha: doc_id: 311403 cord_uid: rbia88vf nan covid-19 disrupted the standard pathways for learning and professional development in medical education. however, it also created a direct interest in epidemiology, virology and immunology, inspiring a wide breadth of students to develop themselves during this crisis as future clinicians and disseminators of scientific information. additionally, given the likely association between tolerating ambiguity and the psychological well-being of medical students and health care workers, 1 we hypothesised that communally embracing and exploring the uncertainty caused by the pandemic would support students in developing coping skills and preparing for the often uncertain nature of clinical medicine. a four-week remote course titled covid-19 exploration in real time that explicitly acknowledged the limited evidence-based information and evolving nature of the pandemic was created. participants included medical students from every year, graduate students, postdoctoral fellows and faculty members. contrary to traditional medical school courses where there is a definitive dichotomy between student and professor, this course embraced a more collaborative approach. communal exploration of the crisis was encouraged during each class session, as students, physicians and scientists shared to enhance each other's understanding. topics covered in the course included epidemiology, virology, immunology, pathology, clinical management, suppression and mitigation, special populations, mental health and ethics. instructional methods combined pre-session readings, readiness quizzes, online forums and zoom-based, expert-led discussions. additionally, students created and updated their own epidemic curves for utah and new york, requiring them to locate and critique currently available data in order to develop hypotheses about incubation periods, modes of transmission and curve projections. students expressed high levels of gratitude for the opportunity to participate in a course that was designed to face the unfolding and chaotic reality of covid-19 together as a team. an unexpected outcome of the course was the sense of community that was created. the discussions of the current crisis among all participants provided much-needed camaraderie to many students who were experiencing feelings of isolation due to loss of important interpersonal interactions among peers. this became especially apparent after a session in which mental health professionals invited students to share thoughts and feelings about how the current events were affecting them. by exploring difficult topics with a class of interprofessional learners, students were able to gain insight from each other's unique perspectives. due to the direct situational relevance of the crisis to every participant, we predict that students will retain a meaningful and enduring comprehension of concepts that can be extrapolated to their future experiences in medicine and science. upon course completion, students expressed that they felt more prepared to deal with future health care crises. the success of the covid-19 exploration in real time course suggests that medical education initiatives capitalising on current public health events can galvanise student interest and serve as a platform to reinforce important clinical concepts and professional identity formation. portions of the course are currently being adapted to prepare medical students who will soon be transitioning to clerkships and clinical electives. tolerance of ambiguity and psychological well-being in medical training: a systematic review key: cord-028411-vae865vj authors: nawaz, sadia; srivastava, namrata; yu, ji hyun; baker, ryan s.; kennedy, gregor; bailey, james title: analysis of task difficulty sequences in a simulation-based poe environment date: 2020-06-09 journal: artificial intelligence in education doi: 10.1007/978-3-030-52237-7_34 sha: doc_id: 28411 cord_uid: vae865vj task difficulty (td) reflects students’ subjective judgement on the complexity of a task. we examine the task difficulty sequence data of 236 undergraduate students in a simulation-based predict-observe-explain environment. the findings suggest that if students perceive the tds as easy or hard, it may lead to poorer learning outcomes, while the medium or moderate tds may result in better learning outcomes. in terms of td transitions, difficulty level hard followed by a hard may lead to poorer learning outcomes. by contrast, difficulty level medium followed by a medium may lead to better learning outcomes. understanding how task difficulties manifest over time and how they impact students’ learning outcomes is useful, especially when designing for real-time educational interventions, where the difficulty of the tasks could be optimised for students. it can also help in designing and sequencing the tasks for the development of effective teaching strategies that can maximize students’ learning. students' perceptions of tasks can influence their learning behaviours [4, 6] . for example, when a task is challenging yet attainable, students may invest effort and persist at it. in contrast, students may not engage in a task if they repeatedly fail at it [28, 49] . this, then, engenders the question: how can instructors design optimal learning conditions where students get challenged but feel confident in accomplishing the task? to address this question, we analyse the relation of task difficulties (tds) with students' learning outcomes. further, we observe how tds vary in a simulation-based learning environment (e.g., is it more probable for tds to transition from easy to hard or vice-versa). lastly, we assess whether students' sequences of tds can be indicative of their learning outcomes. in this paper, tds are analysed in a digital simulation-based predict-observe-explain (poe) learning environment by using the likelihood statistic (l-stat). the aied community has frequently used l-stat for studying students' affective dynamics [18, 19, 21, 22, 36, 37] . compared to a traditional classroom environment, a benefit of analyzing tds in a digital setting is that students can receive just-in-time support. for instance, the level of tds can be adjusted by the instructors to match student's level of understanding or individual students may also choose and change the level of td in a self-controlled setting [3, 25, 30, 62] . we believe that a better understanding of students' tds will enable interventions to improve students' learning [1, 53, 55] and reduce undesirable behaviours such as gaming the system [2] and disengagement [29] . task complexity and task difficulty (td) are often used interchangeably. however, they are two different constructs [51, 52] . task difficulty refers to a person's subjective judgment on the complexity of a task, whilst task complexity represents the characteristics or cognitive demands of a task [9] . different learners can perceive the same tasks differently [9] . researchers have shown that tds can influence students' motivation [32] and self-regulation [4] . tds can also affect problem-solving strategies and tactics. for example, deloache, cassidy and brown [24] suggest that "problems that are too easy or too difficult are less likely to elicit strategic behaviour than the problems that present a moderate degree of challenge" (1985, p. 125). further, the "law of optimum perceived difficulty" states that, if the tasks are perceived very easy or very hard, they can result in lower levels of engagement than the moderately difficult tasks -which may lead to higher levels of engagement [6] . vygotsky [60] suggested that for instruction to be effective it must be aimed at learners' proximal level of development (where learners can succeed with assistance; a difficulty that is somewhat more challenging than an exact match to a student's skill level, but not so challenging that the student cannot succeed). csikszentmihalyi in his works [14, 58] talks about tds and their influence on emotions. he suggests that a person may feel worried and anxious when presented with overly challenging tasks and may feel bored if the tasks are too easy. however, when the tasks are moderately difficult, or they offer just the right challenge, a positive 'flow' experience may occur [15, 16] . therefore, different emotions can be encountered based on how an individual perceives a given task. this, then raises the question: what relation do tds have to students' learning outcomes? the data is not entirely clear on these theoretical perspectives. some studies report that tds have a negative association with students' self-efficacy and performance [44, 45] , yet [7] states that 'certain difficulties can enhance learning'. several studies have indicated that students can learn from challenges that lead them to identify and articulate their current views, examine their ideas and clarify their misconceptions [34, 35] . to sum up, we investigate the following questions in this paper: rq1: what relation do task difficulties have with students' learning outcomes? rq2: how do task difficulties vary over time? rq3: is there a sequence of task difficulties that is indicative of better learning? this study is built on an underlying educational framework known as the predict-observe-explain (poe) paradigm [61] . poe is a three-phase, iterative design [23] . 1. during prediction, students formulate a hypothesis. they are often asked to provide the reasons as to why they committed to it. 2. during observation, students test their hypothesis by changing parameters or variables in a simulation. they can then see the effects of their manipulations. this phase is especially crucial for those who make incorrect hypotheses, as they can see a mismatch between their predictions and observations [26] . 3. during the explanation phase, clarifications are provided to students detailing the relationship between variables or parameters that represent the conceptual phenomenon under investigation. this phase assists students to reconcile any discrepancies between what they predicted and what they observed in the simulation [31] . poes can be applied in face-to-face, online and computer lab contexts [13] . they can promote student discussion [61] , probe into their prior knowledge and help them update prior conceptions [12, 39, 59] . poe learning designs can make digital environments more engaging [39, 57] . recently, poe environments have been analysed to examine students' affective experience [38] and their behaviours relating to struggle and confusion [47, 48] . to the best of our knowledge, tds have not yet been investigated within poe based environments. understanding how tds manifest over time and how they impact students' learning outcomes is useful, especially when designing for real-time interventions. therefore, it is essential that we examine how tds vary in these environments. the data in this study is taken from an online project-based course called habitable worlds. it aims to introduce the foundational concepts of physics, chemistry and biology [33] . it intends to develop problem-solving and logical reasoning skills in students through immersive and interactive tasks in a guided discovery environment. habitable worlds is built using smart sparrow's elearning platform 1 , which records moment by moment activity of students. this adaptive learning environment allows the provision of feedback based on students' responses or lack of responses. this course is offered to non-science major undergraduate students over a duration of 7.5 weeks, and it consists of 67 interactive modules. the current study focuses on an introductory module called stellar lifecycles. the concept under investigation is the relation between a star's mass and its lifespan. there are several tasks within this module which involve one or more of the following activities: providing free-text answers to a question, watching videos, responding to multiplechoice questions or the 'submissions' associated with simulations. in this module, students follow the prescribed sequence of tasks or activities. occasionally, however, there is pathways adaptivity for the remediation of students who make errors. further, the students cannot proceed onto the next tasks unless the current task is completed. of the 23 tasks within this module, we utilize the following poe based tasks: • prediction: students need to select a hypothesis from five possible choices regarding the relationship between stellar mass and lifespan. then, they need to report their reasons (through free text) for selecting that hypothesis. • observation 1: during the first stage of the observe task, students explore the stellar nursery simulator to create virtual stars, manipulate their mass and run them (as many times as they wish). through this simulator, students can study and hopefully understand the relation between stellar mass and its lifespan. • observation 2: during the second stage of the observe task, students need to create at least three different stars within a specified mass range. they need to record the mass and associated lifespan of these stars. next, given their observations, they need to either accept or reject their earlier proposed hypotheses. • explanation 1: this task is only available to the students who make incorrect predictions and endorse them or those who make correct predictions but reject them. this task can assist students in rectifying their hypotheses. • explanation 2: this task requires the students to report the minimum and the maximum lifespan of seven different stellar classes. students can again create and run stars within the stellar nursery simulator. most students seem to struggle at this task as they need to manipulate several different stellar classes. this struggle is reflected in students making repeated attempts. those who manipulate only one stellar class at a time (more systematic) are more likely to complete this task than those who manipulate more than one stellar classes (less systematic) [48] . • post poe: at the final stage, students are provided with a short lecture-style video to explain to them why low mass stars live longer and how a star's mass and internal pressure contribute in the nuclear fusion process which fuels the burning of stars. the data in this study is taken from the october 2017 offering of the course habitable worlds. a total of 236 non-science major undergraduate students attempted this module. of these students, 50% were females, 46% were males, and 4% did not respond. in terms of age, 33% of students were younger than 20, 46% were between the age range of 21 and 30 both inclusive. the remaining 21% were older than 30. learning outcomes. we analyse students' scores at the transfer task -the stellar applications module, which immediately follows the stellar lifecycles module. it tests students on the concepts that were already introduced to them. the maximum achievable score is ten; with each incorrect attempt, students are penalized by two marks. perceived difficulty during-task. during each phase of the poe tasks, to infer students' perceived difficulty, they are asked to report their levels of confidence and challenge on a 6-point scale: from 1 (not at all) to 6 (extremely). following questions are asked: • how confident are you that you understand the task right now? • how challenging do you find the task right now? perceived difficulty after-task. at the end of the poe sequence, students can again report their confidence and challenge on a 6-point scale when asked these questions: • overall, how confident are you that you understood the material in the preceding tasks? • overall, how challenging was the material in the preceding tasks? the response to these survey items is voluntary. in terms of participation, duringtask, 186 students report their perceived td during the prediction task, 151 and 146 during the observe-1 and observe-2 tasks respectively, 74 and 146 during the explain-1 and explain-2 tasks. lastly, 185 students report their perceived td after-task. for analyzing the td dynamics, we include those students who respond to one or more of the task-based surveys. as mentioned, survey items are related to students' confidence and challenge for a given task. to infer tds, we assign following (3) labels: note that our td labels match with csikszentmihalyi's flow theory [17] . while the flow theory reports on students' affects in terms of their challenge and skills; we use these measures (challenge and confidence) to infer students' perceptions of difficulties. learning outcomes reflect students' scores at the transfer task. the maximum achievable score is 10, and for each repeated attempt at this task two points are deducted. high achieving students are those who score above the mean (m = 9.21, sd = 0.92), while, the students scoring below the mean are considered low achievers (m = 3.64, sd = 4.58). to compare the above two student groups, we perform pearson's chi-square test (or fisher's exact test when the entries in the contingency table are less than 5). comparisons are presented for each level of td and during each phase of the poe cycle. during each phase of the poe tasks, as students report their confidence and challenge, we infer their td sequences. later, we use these td sequences to estimate the likelihood statistics (l-stat) as well as the bigram sequences. calculating l-stat. after obtaining students' td sequences, we compute the likelihoods of transitions between any two possible states using the transition metric l [21] , with self-transitions included in the calculation. this metric specifies the probability of a transition from a level at time t to t + 1, after correcting for the base rate at time t + 1. we can represent this as l (difficulty t → difficulty t+1 ), where difficulty t is the difficulty level at the current task and difficulty t+1 is the difficulty level at the next task: the value of l may vary from −∞ to 1. for a given transition, if l ≈ 0, we say that the transition occurs at chance level, if l > 0, we say that the transition is more likely than chance. finally, if l < 0 then the transition is less likely than chance [20] . for calculations, the l-statistic is computed separately for each student and for each possible transition. the transitions where l is undefined are excluded from further analysis. later, one-sample (two-tailed) t-tests are conducted on the calculated l values to measure whether each transition is significantly more or less likely than chance. next, the benjamini-hochberg (bh) post-hoc correction is applied to control for false positives, as the analysis involves multiple comparisons [36] . generating bi-gram sequences. we process students' td sequences to generate td bigrams. we only consider the students who respond to all task-based surveys and who also attempt the transfer task -there are 63 such students. in this regard, given a sequence: 'easy-medium-medium-hard-hard-easy', the associated bigrams are: 'easy-medium', 'medium-medium', 'medium-hard', 'hard-hard' and 'hard-easy'. after this, we compare the students who report a given bigram sequence versus those who do not report it. for this, we perform t-tests and report the results in terms of p-value statistic and t-value statistic. test result is considered significant if p-value < 0.05 (*) and marginally significant if p-value < 0.10 (·). as the analysis also involves multiple comparisons, bh post-hoc correction is applied. a comparison of perceived difficulties, between the high achieving students and the low achieving students, is presented in fig. 1 . the high achievers are more likely to perceive the tasks as medium or moderately difficult than the low achievers -who seem to perceive the tasks as either hard or easy. overall, the proportion of students who respond during the explain-1 is the lowest, as this task is only available to the incorrect predicting students. further, during the post poe phase, many of the high achievers did not respond to the surveys. therefore, the patterns during this task (where each td category is more likely to be reported by the low achievers) differ from the overall trend. fig. 1 . comparison of tds between the high and low achievers using pearson's chi-square test (or the fisher's exact test when the counts in the contingency table are less than 5). high-achievers tend to report medium tds; in contrast, low-achievers tend to report the tds as either easy or hard. results are significant if p-value < 0.05 (*)and marginally significant if p-value < 0.10 (·). table 1 presents the td dynamics in terms of d'mello's l statistic. for self-transitions, the shift from easy → easy is not significantly more or less likely than chance, in contrast, the shift from hard → hard and from medium → medium are significantly less likely than chance. in terms of increasing tds, a transition from easy → medium is less likely than chance, from easy → hard is more likely than chance and from medium → hard is not different from chance level. finally, in terms of decreasing tds, the transitions from hard → easy and from medium → easy are not different from chance level, however, from hard → medium is more likely than chance. next, we analyze students' perceived difficulty over consecutive tasks. we compare the students who report a given bigram sequence versus those who do not report it. this analysis can assist in analyzing how a sequence of tds may impact students' post-test performance (see table 2 ). from this table, the performance is significantly low for the table 2 . td sequences and their likely association with students' performance. performance seems to be lower for the bigram sequence hard-hard, and it appears to be higher for the sequence medium-medium. students who report the td sequence hard-hard than those who do not report it. in contrast, the students who report the td sequence medium-medium have significantly high scores than those who do not report it. the goal of this study is to analyse the perceptions of difficulties or tds. for analysis, we use three labels namely: easy, medium and hard. rq1. the first research question examines the relationship between students' tds and their learning outcomes. from fig. 1 it is observed that during the poe sequence of tasks, the low achieving students mostly report the tasks as either easy or hard. for the low achievers who report the tasks as hard, it could be that they struggled with the learning content, the environment or both. however, for the students who perceive the tasks as easy and yet achieve poorer learning outcomes, a possible explanation for this could be their self-efficacy beliefs. self-beliefs may influence students' performance [4, 5] . the students with unrealistic and overly optimistic opinions may have difficulty aligning their efforts with the desired performance levels and that can subsequently deteriorate their performance [10, 11, 46] . figure 1 further suggests that the high achieving students mostly report the tds as medium. a plausible explanation for this outcome is that students tend to engage more in the tasks that are perceived moderately difficult than the tasks that are perceived too easy or too hard [6] . therefore, for curricula design, the instructors should plan the tasks that are within the learners' zone of proximal development (zpd) [60] . if learners are taught a skill that is within their zpd, it can lead to better performance than when the skill is not [62] . in this regard, [15] suggests that subjects can perform at their optimal capabilities when they experience 'flow', which is likely to happen when their challenge regarding the tasks matches with their skills (confidence in this case). it is important to mention that students' tds from fig. 1 seem to differ at the start of the poe tasks -the prediction phase, where the high achieving students are more likely (p-value < 0.10) to indicate that the tds are easy. this difference during the prediction task is important as this task probes students' prior knowledge. reporting this task easy could mean that these students have higher prior knowledge or higher confidence in prior knowledge which contributed to their performance [40, 41] . further, in a poe context, the observe phase is crucial, it may provide valuable insights into students' prior held beliefs [26] . confusion may be triggered for students who make incorrect predictions [47] . interestingly, there were more low achievers who made incorrect predictions; yet the low achieving students were more likely to report this task as easy (p-value = 0.08). thus, knowledge of students' tds at specific moments can help identify the students who require interventions. the second research question analyses the dynamics of tds -how students' perceptions of difficulties change within this environment. prior research on task-based instruction suggests that pedagogic tasks should be sequenced in increasing order of their demands or complexity [43, 52, 56] . for example, the cognition hypothesis suggests that a gradual increase in task complexity can prepare students for more advanced problems and can lead them to achieve better performance and development [50] [51] [52] . within the current simulation environment, as the students progressed, the tasks became more complex (in terms of the required actions and activities). the impact of task complexity on tds is presented in table 1 . from this table, the transition from hard → medium is more likely than chance, while from easy → medium is less likely than chance. when the findings from rq1 suggest that medium or moderate difficulty may lead to better learning outcomes, the results from rq2 suggest that harder tasks are likely to be followed by moderate difficulty. this, then raises the question of how we can make all students experience difficulties of moderate level -should we intentionally make harder or complex tasks as they seem to precede tds of medium level? or should we make the follow-up tasks feel easier by comparison? we believe that this question may benefit from further studies where, e.g., we compare two groups, a treatment group may be offered less guidance from the system so that the tasks become more complex. the last research question analyses the association between sequences of tds and students' learning outcomes. research on the sequential effects of tds suggests that a learner's performance on a given task (regardless of whether the task is easy or hard) may be affected by the tds on the preceding task [8, 54] . in their work, schneider and anderson [54] report that when an individual faces a hard task, a greater amount of cognitive resources may be allocated to it, and as they proceed to the next task there may be a depletion in the available resources. hence, the performance in the next task may be affected. to inspect this in more detail, we analyse the impact of td sequences (over consecutive tasks) on students' learning outcomes. from table 2 , the students with perceived difficulty of hard on two or more consecutive tasks are significantly more likely to have poorer learning outcomes than those who do not report such a transition. on the one hand, it could mean that these students are weak and therefore perceive the tasks as hard. on the other hand, it could also mean that perhaps there was a depletion of resources as students progressed from a hard task -which is in agreement with [54] . the next significant finding from table 2 is that the students who report medium difficulty on two or more consecutive tasks are likely to have better learning outcomes than other students. what implications do these findings have for learning design? we find that medium tds may lead to better learning outcomes and they often follow hard tds. however, if tasks get too difficult for students, e.g., reporting hard on two or more consecutive tasks, then it can adversely affect students' performance. a knowledge of such perceptions of tds, early on, may enable us to provide timely interventions to students. in this study, we use task difficulties (tds) as a factor of analysis. researchers [27, 28] have acknowledged that only limited studies have investigated the role of students' tds on their learning outcomes. we examine the effects of increasing as well as decreasing tds on students' performance. students who find the tasks easy or hard generally have poorer learning outcomes. however, if a task is perceived easy and it is the prior knowledge task, it may lead to better learning outcomes. furthermore, in accordance with zpd [60] and the flow theory [15] , we find that tds of medium level can lead to better performance. an implication for aied researchers is that, tds are based on students' subjective judgement of the task rather than task complexity. this creates a possibility of individualized predictions of better paths to learning for each student. an unexpected finding was that the students who find the current task to be hard are more likely to perceive the following task as medium than the students who find the current task to be easy. this suggests that hard and challenging tds have the potential to engage students and lead them to achieve better scores, as well as potentially influencing perception of following tasks. however, when tasks become too hard (difficulty sustains over two or more tasks) then it can adversely affect students' performance. to control for the negative effects of tds, one approach is to detect these difficulties early on so that personalised interventions are provided to enhance students' learning. a potential future direction for this work could be the analysis of students' learning behaviours to see how some students who find the current task to be hard can overcome their challenges and then report the following task to be easy or medium. understanding how task difficulties manifest over time and how they impact students' learning outcomes is useful especially when designing for real-time educational interventions, where the difficulty of the tasks could be optimised for the learners. it can also help in designing and sequencing the tasks, for the development of effective teaching strategies that can maximize students' learning [42] and reduce undesirable behaviours such as gaming the system [2] and disengagement [29] . the impact of animated pedagogical agents on girls' and boys' emotions, attitudes, behaviors and learning off-task behavior in the cognitive tutor classroom: when students "game the system better to be frustrated than bored: the incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments self-efficacy: the exercise of control social learning theory the general strategy hypothesis as applied to cognitive theory in mental retardation desirable difficulties perspective on learning subtraction by addition task complexity: a review and analysis evidence on the performance of accounting students: race, gender and expectations the association between the directional accuracy of self-efficacy and accounting course performance investigating the effectiveness of a poe-based teaching activity on students' understanding of condensation affect and learning: an exploratory look into the role of affect in learning with autotutor beyond boredom and anxiety finding flow: the psychology of engagement with everyday life the flow experience. consciousness: brain and states of awareness and mysticism flow: the psychology of optimal experience confusion and its dynamics during device comprehension with breakdown scenarios modeling cognitive-affective dynamics with hidden markov models antecedent-consequent relationships and cyclical patterns between affective states and problem solving outcomes monitoring affective trajectories during complex learning dynamics of affective states during complex learning practical eteaching strategies for predict -observe -explain, problem-based learning and role plays precursors of mnemonic strategies in very young children's memory modeling learners' cognitive, affective, and social processes through language and discourse the pupil as scientist expectancies, values and academic behaviors motivational beliefs, values, and goals operationalizing and detecting disengagement within online science microworlds challenge point: a framework for conceptualizing the effects of various practice conditions in motor learning a matter of gravity the impact of task difficulty expectations on intrinsic motivation habitable worlds: delivering on the promises of online education designing for productive failure productive failure in learning and problem solving the implications of a subtle difference in the calculation of affect dynamics the case of self-transitions in affective dynamics all roads lead to rome: tracking students' affect as they overcome misconceptions the effect of predict-observe-explain strategy on learners' misconceptions about dissolved salts feedback in written instruction feedback and response confidence the role of perceptions of task difficulty in relation to selfperceptions of ability, intrinsic value, attainment value, and performance three approaches to task-based syllabus design the role of subjective task complexity in goal orientation, self-efficacy, and performance relations effects of objective and subjective task complexity on performance self-efficacy and student performance in an accounting course moments of confusion in simulation-based learning environments struggle town? developing profiles of student confusion in simulation-based learning environments motivation in education: theory, research, and applications cognitive complexity and task sequencing: a review of studies in a componential framework for second language task design task complexity, cognitive resources and syllabus design: a triadic framework for examining task influences on sla task complexity, task difficulty, and task production: exploring interactions in a componential framework the effects of an interactive software agent on student affective dynamics while using an intelligent tutoring system asymmetric switch costs as sequential difficulty effects modeling how incoming knowledge, persistence, affective states, and in-game progress influence student learning from an educational game a cognitive approach to language learning effect of predict-observe-explain strategy on achievement in chemistry of secondary school students university students' emotions in lectures: the effect of competence beliefs, value beliefs and perceived task-difficulty, and the impact on academic performance the process of conceptual change in force and motion during computer-supported physics instruction mind and society: the development of higher mental processes probing understanding towards helping teachers select optimal content for students key: cord-291943-7y11bd9w authors: abram, marissa d.; jacobowitz, william title: resilience and burnout in healthcare students and inpatient psychiatric nurses: a between-groups study of two populations date: 2020-10-21 journal: arch psychiatr nurs doi: 10.1016/j.apnu.2020.10.008 sha: doc_id: 291943 cord_uid: 7y11bd9w background stressful environments can have significant effects on an individual and lead to burnout. psychological consequences of burnout include trauma reactions, depression, anxiety and substance use disorders. resilience, a protective mechanism can mitigate the negative impact of burnout. method this study utilizes an exploratory correlational design to determine whether or not there is 1) a similar correlation between resilience and burnout between psychiatric nurses working in a high stress environment and students who are naive to the high stress academic environment and 2) a higher level of resilience in the psychiatric nurses compared to the healthcare students. results a significant inverse relationship was observed with respect to resilience and burnout in both the professional nurse and student groups. however, the inpatient psychiatric nurse group was not found to have a higher level of resilience than the student group, when controlling for age. conclusion: this suggests that resilience to burnout is not related to the work environment but life experience (age) was found to be a factor related resilience in this sample. demographics such gender, age and length of employment were found to be associated with higher levels of resilience, specifically with female healthcare workers, full-time work versus part-time work and age or being an older employee (gillespie et al., 2009; sull, harland et al., 2015) . however, this is not consistently reported. in other studies, years of experience in the workplace did not increase resilience scores (cooke et al, 2013; rushton et al, 2015) . other demographic factors such as income and level of education did not affect levels of resilience (gillespie, et al., 2009 : wagnild, 2003 . posttraumatic growth is another important consideration when conceptualizing resilience. posttraumatic growth is the transformative positive change that can occur as a result of a struggle with great adversity (maitlis, 2020) . exposure to traumatic events can cause a range of posttrauma growth responses, however individuals who experienced personal trauma versus occupational trauma in practice settings had a greater association with negative outcomes (shamia, thabet & vostanis, 2015) . that being said, when examining the impact of personal trauma, li, cao, cao and liu (2015) found that nursing students who experienced adverse childhood events, emotional intelligence and psychological resilience were associated with posttraumatic growth. emotional exhaustion, fatigue, and depression, (b) the emphasis is on mental and behavioral symptoms more than physical ones, (c) burnout symptoms are work-related, (d) the symptoms manifest themselves in "normal" persons who did not suffer from psychopathology before, and (e) decreased effectiveness and work performance occur because of negative attitudes and behaviors. these factors have been linked to poor levels of psychological resilience which in turn causes difficult psychological adjustments (rees, breen, cusack, & hegney, 2015) . research on burnout has identified six areas of risk in the occupational setting: excessive workload, lack of autonomy, insufficient recognition and reward, lack of support and conflict in the environment, perception of fairness and values or meaningfulness of the work (price, 1991) . treglown, palaiou, zarola and furnham, (2016) found that individuals with moody, emotionally volatile, excitable personalities were at greater risk for burnout due to short-lived enthusiasm for projects and people, as well as a heightened sensitivity to betrayal. they also found that individuals with cautious personalities also presented with a higher risk of burnout due to limited positive social interactions. burnout has been recognized as an occupational hazard to individuals working in healthcare (maslach et al, 2016) . burnout among health care workers is associated with negative health effects, missed work days, high turnover rates and lack of effectiveness in the work setting, as well as low job satisfaction (maslach et al, 2001; portoghese, galletta, coppola, finco, & campagna, 2014; schaufeli, leiter & maslach, 2009 ). specifically, health risks such as cardiovascular diseases, musculoskeletal pain and depression are linked to burnout (salvagioni et al., 2017) . preventing burnout, mitigating stress and promoting retention in the healthcare workforce is well documented in the literature. resilience is a protective mechanism against burnout and has been significantly correlated with less stress, better mental health and improved quality of life (kemper, mo & khayat, 2015; leners, sowers, quinn griffin, & fitzpatrick, 2014) . resilience is linked to high compassion satisfaction, low burnout, and a higher tolerance of both general and clinical uncertainty. resilience is also noted to be negatively associated with burnout, secondary traumatic stress and inhibitory anxiety (cooke, doust, & steele, 2013) . although resilience buffers the impact of negative experiences by improving mental health and preventing burnout, resilience did not mediate the negative effects of burnout in regards to physical health (arrogante & aparicio-zaldivar, 2017) . consequently, medical students experiencing burnout with lower quality of life measures in mental and physical health domains had significantly more thoughts of dropping out of school (dyrbye et al., 2010) . however, in a cohort of student athletes, information and support from their coach moderated the stress burnout relationship (lu et al., 2016) . personal characteristics such as optimism and work-related factors such as organizational support impacted engagement in the healthcare work environment (mache et al., 2014) . other factors were also explored. spiritual well-being was associated with reduced emotional exhaustion and depersonalization and physical well-being was associated with personal accomplishment (rushton et al, 2015) . self-compassion and mindfulness were protective factors that are positively associated with resilience and inversely associated with burnout (olson, kemper, & mahan, 2015) . interventions focused on improving mindfulness and resilience may be helpful in the prevention and treatment of burnout (montero-marin et al., 2015) . the purpose of this study is to address two main research questions. first, is there an inverse relationship between resilience and burnout in inpatient psychiatric nurses and in healthcare students (which would further support the theory that resilience is protective)? second, does a population of inpatient psychiatric nurses working in a high stress environment have higher levels of resilience compared to healthcare students (this might suggest that resilience can be enhanced through work-related experiences)? to test these research questions, four hypotheses were formulated. for purposes of analysis, students were selected from a variety of educational programs to further determine whether there are differences among student types. hypothesis 1: resilience scores are inversely related to burnout scores in the combined student group and inpatient psychiatric rns. hypothesis 2: among the students, the 14-item resilience scale (rs-14) score for the combined bachelors and graduate nursing students is greater than for the combined public health and other students hypothesis 3: the burnout score in inpatient psychiatric nurses is not significantly greater than in the combined student group. hypothesis 4: resilience scores are greater in inpatient psychiatric nurses compared to the combined student group. 1. stressful work environment -working in an icu, inpatient psychiatric unit, or emergency department. 2. healthcare student -attendee of one of the following programs: generic baccalaureate rn, adult gerontological primary care nurse practitioner, masters in public health. 3. non-nursing students -masters in public health students. 4. nursing students -includes both bachelors and masters level nursing students 5. combined student group -members of the bachelors rn, masters in public health, and graduate students in adult gerontology nurse practitioner programs. design the study's design utilized an exploratory correlational design employing a 56-item online questionnaire that was administered to a convenience sample of undergraduate nursing, graduate nursing and graduate public health students at a northeastern us university. for purposes of hypothesis testing, an alpha level of .05 was chosen. sample size estimation was performed using g*power 3.1 (faul, erdfelder, buchner & lang, 2009 ). at power = .80 using a mann-whitney u test with an expected medium effect size in a two-tailed analysis, a sample of 134 was required. in order to facilitate protection of human subjects, this study was reviewed and approved by the irb of the authors' university, the student-subjects' data were anonymous as they were collected using an online platform (survey monkey tm ), and identifying information, including ip addresses, were not solicited or collected. additionally, de-identified data of the inpatient psychiatric nurses from a prior irb approved study conducted by one of the current study's authors with respect to the relationship of resilience and burnout were used to compare to the results of the data collected from the current study (see table 1 for demographic characteristics of the subjects). the sampling procedure consisted of student research assistants soliciting other students in their classrooms to be the subjects of the study. permission from the class instructors was obtained beforehand, and the instructors were not present in the room during the solicitation and consenting processes. the research assistants were not students in the classes where students were solicited, and an implied consent process was employed, i.e., the research assistants read a script describing the study, answered any questions that were posed, and directed the students to access the consent using their smartphones or laptops through a link that the research assistants provided. the students could either exit from the link or continue to the questionnaire depending on their inclination towards participating as subjects. the data were collected in the spring of 2018 from 3 groups of samples: 1) bachelors nursing students; 2) adult gerontology primary care nurse practitioner students; and 3) public health graduate students (mph program, non-clinical) . these data were compared to a dataset from a previous study conducted by one of the authors in the spring of 2012 at a 250-bed free standing psychiatric hospital in a suburban new york location. in that study, data were collected from rns in the locked acute inpatient units. the survey utilized a 15-item demographic author-developed questionnaire. to measure resilience and burnout, the 14-item resilience scale (rs-14) (wagnild, 2009) ; and the burnout scale (a 10-item standardized subscale of the professional quality of life scale) were used (stamm, 2010) . the resilience scale-14 (wagnild, 2009 ) has been used to measure resilience in a variety group. it employs a 7-point likert scale, ranging from strongly agree to strongly disagree and consists of 14-items. published cronbach alphas for the rs-14 range from .93 in a clinical sample to .96 in a college student sample (aiena, baczwaski, schulenberg, & buchanan, 2015) . the burnout scale (bo) (stamm, 2010) has been used to measure burnout in many studies involving clinical samples. it utilizes a 5-point likert scale, ranging from very often to never, and consists of 10-items. cronbach alphas of the burnout scale in clinicians range from .78 to .85 (hemsworth, baregheh, aoun, & kazanjian, 2018) . the data were analyzed for missing statistics and normality of the distributions. due to relatively few missing statistics, data missing at random were replaced using each individual subject's mean score for that scale. data missing not at random were not replaced since it may be the result of subject bias. due to only one out of the eight distributions of the groups of data (4 j o u r n a l p r e -p r o o f groups of subjects x two variables = 8 groups of data) being normally distributed, all analyses were conducted using nonparametric tests, i.e., spearman correlation coefficient, mann-whitney u test and the kruskal-wallis anova test. statistical analyses were performed using ibm-spss version 25. the first step involved assessing the quality of the data. one-hundred and twenty-two (122) students responded, but 3 were excluded due to data missing in a non-random manner (mnar), resulting in a sample size of 119. the nature of the missing data were that they failed to indicate which academic program they were in and they failed to complete one or both of the standardized measurement scales. three (3) students in the generic rn program failed to answer from 1 to 3 questions on one of the measurement scales in a random manner (no observable pattern). in each of the latter cases, the data were replaced using the mean scores for that student with respect to the particular scale. additionally, 3 students indicated their program as 'other'. the previously collected data with respect to the inpatient psychiatric rns consisted of 51 subjects. in total, a combined dataset of 170 subjects was utilized which is in excess of the minimum size of 134 as calculated from the sample size estimation procedure. with the alpha level set at 0.05 (p < .05), significant differences were not observed in comparing the combined student group to the inpatient psychiatric nurse group, except with respect to 5 demographic characteristics. categories of age showed significantly more 21 to 30year old's in the student group compared to the inpatient psychiatric nurses (chi-square, df = 3, p < .0010) which was expected based on the nature of the groups. the category of race indicated significantly fewer white and more asian and 'other' subjects among the students compared to inpatient psychiatric nurses (chi-square, df = 6, p = .013). similarly, ethnicity represented j o u r n a l p r e -p r o o f significantly more latino individuals among students compared to inpatient psychiatric nurses (chi-square, df = 1, p = .03). the category of relationship status, as expected, contained significantly more students living with family members (other than a life partner) compared to inpatient psychiatric nurses (chi-square, df = 2, p = .0010). lastly, students' unemployment was significantly greater than inpatient psychiatric rns, as expected (chi-square, df = 2, p < .0010). of note, 81% of students were employed either full-time or part-time (see table 1 for more details). [insert table 1 here] the four groups of data (bachelors rn, np, mph and inpatient psychiatric rn's) were evaluated for normality of the distributions on the resilience and burnout measures using descriptive statistics, histograms and the kolmogorov-smirnov test. only the distribution of data for the burnout variable for inpatient psychiatric nurses approached normality (see table 2 for more details). as a result, all of the analyses employed non-parametric univariate and bivariate tests. [insert table 2 here] h1: the rs-14 scores are inversely related to bo scores in the combined student group and inpatient psychiatric rns. resilience and burnout scores were tested for correlation in the combined student group using the spearman correlation test. a significant inverse relationship was observed with a large j o u r n a l p r e -p r o o f effect size (cohen, 1992 ; r s = -.61, p = < .001, n = 119). a similar result using the same test was observed among inpatient psychiatric rns with a medium effect size (cohen, 1992 ; r s = -.39, p = .005, n = 51). these results suggest that it is likely there is an inverse relationship between resilience and burnout that is stable in different populations. h2: among the students, the rs-14 score for the combined bachelors and graduate nursing students is greater than for the combined public health and other students. using the non-parametric kruskal wallis anova test for three or more groups, the resilience scores were not found to significantly differ among the student groups (p = .64, n = 119). this result suggests that there is no self-selection process with respect to internal coping ability (resilience) and choice of a stressful career (nursing). of note, a post hoc power analysis suggests that a sample of 156 is necessary for this analysis. given that the actual combined student group size was 119, it is possible that this result is due to insufficient power. however, the p-value of .64 with a group size of 119 subjects suggests that it is quite probable that a significant difference in resilience among the students would not be found even with the suggested minimum sample size. h3: the bo scores in inpatient psychiatric nurses is not significantly greater than in the combined student group. again, the mann whitney u test was used to compare the two groups of subjects. this hypothesis was rejected and burnout scores were found to be significantly higher in combined students (median = 23) compared to inpatient psychiatric rns (median = 19; p < .001; n = 170). it appears that students in this sample experience a significantly greater amount of burnout symptoms compared to inpatient psychiatric rns, but with a range of scores from 10 to 37 j o u r n a l p r e -p r o o f caution in interpreting the meaning of the result needs to be exercised, similar to that discussed related hypothesis 2. h4: the rs-14 scores are greater in inpatient psychiatric nurses compared to the combined student group. using the non-parametric mann whitney u test for comparison of two groups, it was observed that the resilience scores were significantly greater in inpatient psychiatric rns (median = 88) compared to combined students (median = 83.5; p = .002; n = 170). although statistically significant, the range of scores (17 -98) needs to be considered and it is important to recognize that a readily discernible difference in characteristics in these two groups may not be possible. as mentioned earlier in the results section of this paper, 5 demographic variables were found to significantly vary between the inpatient psychiatric rn group and the combined student group. to determine the effects of these demographic variables on the findings for hypotheses 3 and 4, a two-step multiple logistic regression analysis was performed. in step 1, the 5 demographics were regressed against the 'inpatient psychiatric rn/combined student' binary variable. in step 2, burnout scores and the rs-14 scores were regressed (maintaining the demographics as control variables). the results are that 1) age was the only demographic found to be statistically different between the inpatient psychiatric rn and combined student groups (b = -.66, p = .01, or = .51, r 2 = .44); 2) burnout scores remained significantly different between the inpatient psychiatric rn and the combined student groups (b = .133, p = .03, or = 1.14, r 2 = .02); and more importantly 3) rs-14 scores were not significantly different between the inpatient j o u r n a l p r e -p r o o f psychiatric rn and combined student groups. this latter finding suggests that hypothesis 4 is not retained when controlling for the effect of age. the latter findings may be interpreted that age did not have an effect on the difference in burnout between the inpatient psychiatric nurse and combined student groups (students had significantly higher levels of burnout), but that age was a more robust predictor of resilience than work/school environment resulting in the latter no longer being a significant predictor of resilience. this appears to suggest that life experience (age) is more strongly associated with the development of resilience than psychological adjustment to workplace-related stress. [insert table 3] [insert table 4 ] discussion a significant inverse relationship was observed with respect to resilience and burnout in both students and inpatient psychiatric nurses. this suggests that resilience may be protective with respect to the development of burnout. this finding is consistent with the literature (cooke et al., 2013; kemper et al., 2015; leners et al., 2014; zou et al., 2016) . resilience can minimize the impact of a strenuous or harmful environment and enhance the individual's ability to thrive in a stressful environment (mcgowan et al., 2016; rippstein-leuenberger, mauthner, sexton & schwendimann, 2017; sanderson & brewer, 2017) . however, in a recent study of more than 5,000 physicians, researchers found that even with higher levels of resilience, the individual was still susceptible to the effects of burnout (west et al, 2020) . this is important because even though resilience may offer some protection against burnout, it may not prevent it. initial results from this study found that the inpatient psychiatric nurses had higher resilience levels than students. findings such as this, could possibly occur through some aspect of the work environment or work experience that enhances resilience and reduces burnout such as traumatic growth or through attrition. challenges of the work environment reduce the nurse workforce to only those with higher resilience and lower burnout, i.e., healthy worker effect (li & sung, 1999) , or some combination of both of the above processes. one study found that the turnover rate for newly hired registered nurses is estimated to be 17.5% and one in three (33.5%) leave within two years (kovner, brewer, fatehi & jun, 2014) potentially supporting the healthy worker idea. however, results from the current study did not support this, because post hoc tests controlling for significant demographic variables revealed that age had a stronger association with the development of resilience versus the environment. that being said, when examining the effects of stress in the work environment, crane and serle, (2016) found that environments that had challenge stressors, or stressors that created opportunities for growth and development, fostered the development of resilience versus environments with hindrance stressors which inhibited the development of resilience. based on their work, it appears that the environments of the psychiatric setting and the school setting where the research took place contain hindrance stressors that inhibit the development of resilience. this may be due to certain unpredictable aspects to the stressor or the overall amount of stress (overwhelming) that the subjects experience. consistent with other studies in the literature, findings in the current study found that age was significantly predictive of higher levels of resilience (gillespie et al, 2009; gooding, hurst, johnson & tarrier, 2012; sull et al, 2015) . age however, did not have an effect on burnout. this is contrary to findings from other studies, which have found a relationship between younger age j o u r n a l p r e -p r o o f and burnout (gã³mez-urquiza et al., 2017; zeng et al., 2020) . this difference may be due to the unpredictable and overwhelming characteristics of work in an acute psychiatric inpatient setting, and the quantity of stress that students experience who are frequently, working at full or parttime employment while dealing with the challenges of academic workload. students in this study had significantly higher levels of burnout compared to inpatient psychiatric nurses. for students, factors such as self-evaluation and organizational support, perfectionism, lack of academic self-efficacy and the perception of lack of teacher support were linked to burnout (leupold, lopina, & erickson, 2019; onuoha & idemudia, 2017) . understanding the impact that stress and trauma can potentially have on the student, resilience and burnout prevention training should begin in the academic setting. as students transition into potentially professional environments, they should be prepared to navigate challenges successfully with continued support. it is essential that they are trained to know how to develop resilience in response to overcoming obstacles, persevering in a demanding career and interestingly, a review of resilience in the academic health education setting found weak j o u r n a l p r e -p r o o f evidence that resilience was associated with only slightly improved academic performance and decreased burnout (mcgowan & murray, 2016) . overall, resilience training for students was implemented as a preventative measure to entering a stressful career, and not as a response to mitigating student burnout in the academic setting. this evidence points to the need for integration of resilience development and burnout prevention training starting in the academic setting. furthermore, vanhove, herian, perez, harms, and lester, (2016) found that with more formal approaches such as resilience and trauma training, the effects were small and diminished over time with the exception of higher risk individuals. this diminishing effect suggests that training should be ongoing as the individual transitions into the professional environment. finally, resilience was not found to be significantly different among the 4 categories of students. this suggests that there is no self-selection process with respect to resilience and the choice of nursing as a career. this is important as resilience development and burnout prevention training should be taught across the health professions in an interprofessional format. the data reported here are cross-sectional and do not reflect findings stemming from an experimental design, no statements of causality can be inferred from the results. in addition, given subjects were not randomized to the two study groups, issues of lack of comparability and sample bias are possible. this is supported from the fact that 20% of students work in a high stress workplace as defined in this paper, i.e., icu, emergency department or inpatient (jacobowitz et al, 2015) . individual coping mechanisms to support resilience such as developing mindfulness skills, engaging in self-care and seeking emotional support from colleagues (perez et al., 2015) should be encouraged and taught to nurses. furthermore, as the literature shows diminished effects of interventions over time, resilience development and burnout prevention training should occur on a consistent, routine basis. it is important to understand how to reduce burnout and enhance resilience development in students. interventions should be developed to promote the health and well-being of students and measured for outcomes. furthermore, resilience can be measured utilizing a longitudinal study approach. this will shed light on how resilience changes over time as the individual transitions from an academic environment into the workforce. this may also reveal reasons for attrition and identify if and how the healthy worker effect impacts the existing workforce. j o u r n a l p r e -p r o o f j o u r n a l p r e -p r o o f measuring resilience with the rs-14: a tale of two samples nurses' reports of working conditions and hospital quality of care in 12 countries in europe u.s. nursing leadership supports practice/academic partnerships to assist the nursing workforce during the covid-19 crisis burnout and health among critical care professionals: the mediational role of resilience. intensive and critical care nursing a meta-analysis of the relationship between job burnout and secondary traumatic stress among workers with indirect exposure to trauma resilience and psychological distress in psychology and medical students a power primer a survey of resilience, burnout, and tolerance of uncertainty in australian building resilience through exposure to stressors: the effects of challenges versus hindrances factors associated with resilience to and recovery from burnout: a prospective, multi-institutional study of us medical students statistical power analyses using g* power 3.1: tests for correlation and regression analyses concept analysis: resilience. archives of psychiatric nursing the influence of personal characteristics on the resilience of operating room nurses: a predictor study age as a risk factor for burnout syndrome in nursing professionals: a metaanalytic study psychological resilience in young and older adults relationship between resilience, stress and burnout among civil servants in beijing, china: mediating and moderating effect analysis a critical enquiry into the psychometric properties of the professional quality of life scale a meta-analysis of the trait resilience and mental health burnout and alcohol abuse/dependence among us medical students ptsd in psychiatric nurses and other mental health providers: a review of the literature post-traumatic stress, trauma-informed care, and compassion fatigue in psychiatric hospital staff: a correlational study posttraumatic growth at work. annual review of organizational psychology and organizational behavior refining trait resilience: identifying engineering, ecological, and adaptive facets from extant measures of resilience the relationship between resilience and compassion fatiguein counselors. vistas: compelling perspectives on counseling historical and conceptual development of burnout understanding the burnout experience: recent research and its implications for psychiatry surviving workplace adversity: a qualitative study of nurses and midwives and their strategies to increase personal resilience exploring resilience in nursing and midwifery students: a literature review the role of medical students during the covid-19 pandemic mindfulness, resilience, and burnout subtypes in primary care physicians: the possible mediating role of positive and negative affect stressâ�¦at work what factors promote resilience and protect against burnout in first-year pediatric and medicine-pediatric residents personal attributes influencing school burnout among graduating students promoting resiliency among palliative care clinicians: stressors, coping strategies, and training needs burnout and workload among health care workers: the moderating role of job control. safety and health at work healthy work: stress, productivity, and the reconstruction of working life understanding individual resilience j o u r n a l p r e -p r o o f journal pre-proof in the workplace: the international collaboration of workforce resilience model a qualitative analysis of the three good things intervention in healthcare workers burnout and resilience among nurses practicing in high-intensity settings physical, psychological and occupational consequences of job burnout: a systematic review of prospective studies what do we know about student resilience in health professional education? a scoping review of the literature psychological distress, coping behaviors, and preferences for support among new york healthcare workers during the covid-19 pandemic. general hospital psychiatry burnout: 35 years of research and practice. career development international retention of nurses in a rural environment: the impact of job satisfaction, resilience, grit, engagement, and rural fit exposure to war traumatic experiences, post-traumatic stress disorder and post-traumatic growth among nurses in gaza stress, resilience and psychological well-being in chinese undergraduate nursing students the concise proqol manual. pocatello, id: proqol. org, 78 resilience of health-care workers in the uk; a crosssectional survey relationships among perceived psychological growth, resilience and burnout in physicians the dark side of resilience and burnout: a moderation-mediation model psychometric study of depression, anxiety and stress among university students can resilience be developed at work? a meta-analytic review of resilience-building programme effectiveness a review of the resilience scale resilience and successful aging: comparison among low and high income older adults resilience and burnout among physicians and the general us working population relationship between occupational stress and depression among psychiatric nurses in japan prevalence of burnout in mental health nurses in china: a meta-analysis of observational studies. archives of psychiatric nursing correlates of psychological distress, burnout, and resilience among chinese female nurses key: cord-299547-9i8kv8p8 authors: aucejo, esteban m.; french, jacob; araya, maria paola ugalde; zafar, basit title: the impact of covid-19 on student experiences and expectations: evidence from a survey date: 2020-08-27 journal: j public econ doi: 10.1016/j.jpubeco.2020.104271 sha: doc_id: 299547 cord_uid: 9i8kv8p8 in order to understand the impact of the covid-19 pandemic on higher education, we surveyed approximately 1,500 students at one of the largest public institutions in the united states using an instrument designed to recover the causal impact of the pandemic on students’ current and expected outcomes. results show large negative effects across many dimensions. due to covid-19: 13% of students have delayed graduation, 40% have lost a job, internship, or job offer, and 29% expect to earn less at age 35. moreover, these effects have been highly heterogeneous. one quarter of students increased their study time by more than 4 hours per week due to covid-19, while another quarter decreased their study time by more than 5 hours per week. this heterogeneity often followed existing socioeconomic divides; lower-income students are 55% more likely than their higher-income peers to have delayed graduation due to covid-19. finally, we show that the economic and health related shocks induced by covid-19 vary systematically by socioeconomic factors and constitute key mediators in explaining the large (and heterogeneous) effects of the pandemic. the disruptive effects of the covid-19 outbreak have impacted almost all sectors of our society. higher education is no exception. anecdotal evidence paints a bleak picture for both students and universities. according to the american council on education, enrollment is likely to drop by 15% in the fall of 2020, while at the same time many institutions may have to confront demands for large tuition cuts if classes remain virtual. 1 in a similar vein, students face an increasingly uncertain environment, where financial and health shocks (for example, lack of resources to complete their studies or fear of becoming seriously sick), along with the transition to online learning may have affected their academic performance, educational plans, current labor market participation, and expectations about future employment. this paper attempts to shed light on the impact of the covid-19 pandemic on college students. first, we describe and quantify the causal effects of the covid-19 outbreak on a wide set of students' outcomes/expectations. in particular, we analyze enrollment and graduation from college in the fall 2020 semester at more than twice the rate in previous years. historically, 28% of students who fail to re-enroll do not return to asu or another university after 5 years (authors' calculations from asu first-time freshmen transcript data for the 2012-2014 spring semesters), suggesting that the pandemic may have a lasting impact on the educational achievement of current students. we also find that students report a decreased preference for online instruction as a result of their recent experiences. as expected, the covid-19 outbreak also had large negative effects on students' current labor market participation and expectations about post-college labor outcomes. working students suffered a 31% decrease in their wages and a 37% drop in weekly hours worked, on average. moreover, around 40% of students lost a job, internship, or a job offer, and 61% reported to have a family member that experienced a reduction in income. the pandemic also had a substantial impact on students' expectations about their labor market prospects post-college. for example, their perceived probability of finding a job before graduation decreased by almost 20%, and their expected earnings when 35 years old (around 15 years from the outbreak) declined by approximately 2.5%. this last finding suggests that students expect the pandemic to have a long-lasting impact on their labor market prospects, which is qualitatively consistent with the literature on graduating during a recession. for instance, oreopoulos et al. (2012) and schwandt and von wachter (2019) find significant reductions in earnings 5 and 10 years after graduation, respectively, and kahn (2010) finds an even longer-lasting effect on wages. on the other hand, although we are measuring the probability of finding a job before graduating, not unemployment directly, our estimated quantitative effect on students' expectations of finding a job seems to be larger relative to the literature (kahn, 2010; altonji et al., 2016; and rothstein, 2020) . the data also show that while all subgroups of the population have experienced negative effects due to the outbreak, the size of the effects are heterogeneous. for example, compared to their more affluent peers, lower-income students are 55% more likely to delay graduation due to and are 41% more likely to report that covid-19 impacted their major choice. learning, the impact of covid-19 on honors students' academic outcomes is consistently smaller than the impact on non-honors students. finally, we evaluate the extent to which mitigating factors associated with more direct economic and health shocks from the pandemic (for example, a family member losing income due to covid-19, or the expected probability of hospitalization if contracting can explain the heterogeneity in pandemic effects. we find that both types of shock (economic and health) are systematically correlated with students' covid-19 experiences. for example, the expected probability of delaying graduation due to covid-19 increases by approximately 25% if either a student's subjective probability of being late on a debt payment in the following 90 days (a measure of financial fragility) or subjective probability of requiring hospitalization conditional on contracting covid-19 increases by one standard deviation. as expected, the magnitude of health and economic shocks are not homogeneous across the student population. the average of the principal component for the economic and health shocks is about 0.3-0.4 standard deviations higher for students from lower-income families. importantly, we find that the disparate economic and health impacts of covid-19 can explain 40% of the delayed graduation gap (as well as a substantial part of the gap for other outcomes) between lower-and higher-income students. this analysis should be viewed as descriptive in nature and not necessarily causal, since omitted factors that are correlated both with the shocks and the outcomes may be driving these relationships. to our knowledge, this is the first paper to shed light on the effects of covid-19 on college students' experiences. the treatment effects that we find are large in economic terms. whether students are overreacting in their response to the covid-19 shock is not clear. we do find that previous cumulative gpa is a strong predictor of expected semester gpa without covid-19, suggesting that students' reported expectations are meaningful. however, we know a total of 1,564 respondents completed the survey. 5 90 respondents were ineligible for the study (such as students enrolled in graduate degree programs or diploma programs) and were dropped from the sample. finally, responses in the 1st and 99th percentile of survey duration were further excluded, leading to a final sample size of 1,446. the survey took 38 minutes to complete, on average (median completion time was 26 minutes). the first five columns of table 1 show how our sample compares with the broader asu undergraduate population and the average undergraduate student at other large flagship universities (specifically, the largest public universities in each state). relative to the asu undergraduate population, our sample has a significantly higher proportion of first-generation students (that is, students with no parent with a college degree), and a smaller proportion of international students. the demographic composition of our sample compares reasonably well with that of students in flagship universities. our sample is also positively selected in terms of sat/act scores relative to these two populations. the sample may also differ from the student body at other large public schools in that 30% report living on campus, which is not always the norm at other large institutions and may play an important role in how disruptive the pandemic has been. 6 the better performance on admission tests could be explained by the high proportion of honors students in our sample (22% compared to 18% in the asu population). the last four columns of table 1 show how honors students compare with asu students and the average college student at a top-10 university. we see that they perform better than the average asu student (which is expected) and just slightly worse than the average college student at a top-10 university. the share of white honors students in our sample (60%) is higher than the proportion in the asu population and much higher than the proportion of white students in the top-10 universities. overall, we believe our sample of asu students is a reasonable representation of students at other large public schools, while the honors students may provide insight into the experiences of students at more elite institutions. though, it is important to acknowledge that elite institutions we next outline a simple analytic framework that guides the empirical analysis. let ( -19) i o covid be the potential outcome of individual i associated with covid-19 treatment. we are interested in the causal impact of covid-19 on student outcomes: where the first term on the right-hand side is student i 's outcome in the state of the world with covid-19, and the second term being student i 's outcome in the state of the world without covid-19. recovering the treatment effect at the individual level entails comparison of the individual's outcomes in two alternate states of the world. with standard data on realizations, a given individual is observed in only one state of the world (in our case, -19 = 1 covid ). the alternate outcomes are counterfactual and unobserved. a large econometric and statistics literature studies how to identify these counterfactual outcomes and moments of the counterfactual outcomes (such as average treatment effects) from realized choice data (e.g., heckman and vytlacil, 2005; angrist and pischke, 2009; imbens and rubin, 2015) . instead, the approach we use in this paper is to directly ask individuals for their expected outcomes in both states of the world. from the collected data, we can then directly calculate the individual-level subjective treatment effect. as an example, consider beliefs about end-of-semester gpa. the survey asked students "what semester-level gpa do you expect to get at the end of this semester?" this is the first-term on the right-hand side of equation (1 the approach we use in this paper follows a small and growing literature that uses supply. there is one minor distinction from these papers: while these papers elicit ex-ante treatment effects, in our case, we look at outcomes that have been observed (for example, withdrawing from a course during the semester) as well as those that will be observed in the future (such as age 35 earnings). thus, some of our subjective treatment effects are ex-post in nature while others are ex-ante. the soundness of our approach depends on a key assumption that students have well-formed expectations for outcomes in both the realized state and the counterfactual state. since the outcomes we ask about are absolutely relevant and germane to students, they should have well-formed expectations for the realized state. in addition, given that the counterfactual state is the one that had been the status quo in prior semesters (and so students have had prior experiences in that state of the world), their ability to have expectations for outcomes in the counterfactual state should not be a controversial assumption. 7 as evidence that students' expectations exhibit meaningful variation, appendix figure a1 shows that previous cumulative gpa is a strong predictor of expected semester gpa with covid-19. we start with the analysis of the aggregate-level treatment effects, which are presented in table 2 . the outcomes are organized in two groups, academic and labor market (see appendix for example, the average subjective treatment effect of covid-19 on semester-level gpa is a decline of 0.17 points. more than 50% of the students in our sample expect a decrease in their gpa due to the treatment (versus only 7% expecting an increase). additionally, on average, 13% of the participants delayed their graduation,11% withdrew from a class during the spring semester, and 12% stated that their major choice was impacted by while almost no students report planning to drop out due to covid-19, on average they expect to take a break from asu in the fall 2020 semester at nearly twice the historical rate (historically). admittedly, the decision to take a break during a pandemic may be different than in more normal times. however, a substantial increase in the share of students failing to continue their studies is concerning, as historically 28% of students who fail to re-enroll for a fall semester do not return to asu or another university within 5 years. regarding the impact of the pandemic on major choice, students who report that covid-19 impacted their major choice were more likely to be in lower-paying majors before the pandemic; mean pre-covid major-specific annual earnings were $43,053 ($46,943) for students whose major choice was (not) impacted by impacted students were also 9.3 percentage points less likely to be in a science, technology, engineering, or math (stem) major before covid-19. 10 we are only able to observe pre-and post-covid major choices for the subset of students who had switched their major by the date of the survey. 11 within this selected subsample of switchers, students chose to move into higher paying majors, with an average change in first-year earnings of $3,340. these patterns are generally consistent with the finding that students tend to gravitate towards higher-paying majors when exposed to adverse economic conditions when in college (blom et al., 2019). table 2 is that, on average, students are 4 percentage points less likely to opt for online instruction if given the choice between online and in-person instruction due to their experience with online instruction during the pandemic. 12 , 13 however, there is a substantial amount of variation in terms of the direction of the effect: 31% (47%) of the participants are now more (less) likely to enroll in online classes. we explore this heterogeneity in more detail in the next section, but it seems that prior experience with online classes somewhat ameliorates the negative experience; the average treatment effect for students with prior experience in online classes is a 2.4 percentage points decrease in their likelihood of enrolling in online classes, versus a 9.5 percentage points decline for their counterparts (difference statistically significant at the 0.1% level). this large variation in the treatment effects of covid-19 is apparent in several of the other outcomes, such as study hours, where the average treatment effect of covid-19 on weekly study hours is -0.9 (that is, students spend 0.9 less hours studying per week due to . the interquartile range of the across-subject treatment effect demonstrates substantial variation, with the pandemic decreasing study time by 5 hours at the 25th percentile and increasing study time by 4 hours at the 75th. overall, these results suggest that covid-19 represents a substantial disruption to students' academic experiences, and is likely to have lasting impacts through changes in major/career and delayed graduation timelines. students' negative experiences with online teaching, perhaps due to the abruptness of the transition, also has implications for the willingness of students to take online classes in the future. turning to panel b in table 2 , we see that students' current and expected labor market outcomes were substantially disrupted by covid-19. as for the extensive margin of current employment, on average, 29% of the students lost the jobs they were working at prior to the pandemic (67% of the students were working prior to the pandemic), 13% of students had their internships or job offers rescinded, and 61% of the students reported that a close family member had lost their job or experienced an income reduction. the last statistic is in line with findings from other surveys of widespread economic disruption across the us. 14 respondents experienced an there was no change in weekly earnings for 52% of the sample, which again reflects substantial variation in the effects of covid-19 across students. in terms of labor market expectations, on average, students foresee a 13 percentage points decrease in the probability of finding a job by graduation, a reduction of 2% in their reservation wages, and a 2.3% decrease in their expected earnings at age 35. the significant changes in reservation wages and expected earnings at age 35 demonstrate that students expect the treatment effects of covid-19 to be long-lasting. qualitatively, this is broadly consistent with the literature on graduating during recession. oreopoulos et al. (2012) finds that graduating during a recession in which the unemployment rate increases 5% implies an initial loss in earnings of 9%, that decreases to 4.5% within 5 years and disappears after 10 years for a sample of male college graduates in canada. similarly, schwandt and von wachter (2019) find a 2.6% reduction in earnings 10 years after graduation for a 3-percentage point increase in unemployment at graduation, and kahn (2010) finds an even longer-lasting effect on wages. a large literature has investigated the impact of graduating during recessions on unemployment rates. kahn (2010) we next explore demographic heterogeneity in the treatment effects of covid-19. figure 1 plots the average treatment effects across several relevant demographic divisions including gender, race, parental education, and parental income. honors college status and cohort are also included as interesting dimensions of heterogeneity in the covid-19 context. the figure shows the impacts for six of the more economically meaningful outcomes from table 2 (additional outcomes can be found in figure a2 ). at least four patterns of note emerge from figure 1 . first, compared to their classmates, students from disadvantaged backgrounds (lower-income students defined as those with below-median parental income, racial minorities, and first-generation students) experienced larger negative impacts for the academic outcomes, as shown in the first three panels of the figure. 15 the trends are most striking for lower-income students, who are 55% more likely to delay graduation due to covid-19 than their more affluent classmates (0.16 increase in the proportion of those expecting to delay graduation versus 0.10), expect 30% larger negative effects on their semester gpa due to covid-19, and are 41% more likely to report that covid-19 impacted their major choice (these differences are statistically significant at the 5% level). for some academic outcomes, covid-19 had similarly disproportionate effects on nonwhite and first-generation students, with nonwhite students being 70% more likely to report changing their major preference compared to their white peers and first-generation students being 50% more likely to delay their graduation than students with college-educated parents. thus, while on average covid-19 negatively impacted several measures of academic achievement for all subgroups, the effects are significantly more pronounced for socioeconomic groups which were predisposed towards worse academic outcomes pre-covid. 16 the pandemic's widening of existing achievement gaps can be seen directly in students' expected semester gpa. without covid-19, lower-income students 15 the cutoff for median parental income in our sample is $80,000 16 based on analysis of asu administrative data including transcripts, we find that, relative to their counterparts, first-generation, lower-income, and non-white students drop out at higher rates, take longer to graduate, have lower gpas at graduation, and are more likely to switch majors when in college (see appendix table a3) j o u r n a l p r e -p r o o f second, panel (d) of figure 1 shows that the switch to online learning was substantially harder for some demographic groups; for example, men are 7 percentage points less likely to opt for an online version of a course as a result of covid-19, while women do not have a statistically significant change in their online preferences. we also see that honors students revise their preferences by more than 2.5 times the amount of non-honors students. as we show later (in table 4 ), these gaps persist after controlling for household income, major, and cohort, suggesting that the switch to online learning mid-semester may have been substantially more disruptive for males and honors students. while the effect of covid-19 on preferences for online learning looks similar for males and honors students, our survey evidence indicates that different mechanisms underpin these shifts. based on qualitative evidence, it appears that honors students had a negative reaction to the transition to online learning because they felt less challenged, while males were more likely to struggle with the learning methods available through the online platform. 18 one speculative explanation for the gender difference is that consumption value of college amenities is higher for men (however, jacob et al. (2018), find little gender difference in willingness to pay for the amenities they consider). the third trend worth highlighting from figure 1 is that honors students were better able to mitigate the negative effect of covid-19 on their academic outcomes (panels a, b, and c), despite appearing to be more disrupted by the move to online learning (panel d). honors students report being less than half as likely as non-honors students to delay graduation and change their major due to covid-19. extrapolating from these patterns provides suggestive evidence that academic impacts for students attending elite schools-the group more comparable to these honors studentsare likely to have been small relative to the impacts for the average student at large public schools. finally, the last two panels of figure 1 present the covid effect on two labor market expectations and show much less meaningful heterogeneity across demographic groups compared to the academic outcomes in previous panels. this suggests that, while students believe covid-19 will impact both their academic outcomes and future labor market outcomes, they do 17 the difference is significant at 1% in both cases. 18 honors students were as likely as non-honors students to say that classes got easier after they went online but, conditional on saying classes got easier, were 47% more likely to say "homework/test questions got easier." conversely, males were marginally more likely to say classes got harder after they went online (10% more likely, p=0.055) and, conditional on this, were 14% more likely to say that "online material is not clear". the one notable exception to the lack of heterogeneity in panels (e) and (f) of figure 1 are seniors, who on average revised their subjective probability of finding a job before graduation three times as much as other cohorts. figure a3 further breaks down the estimated covid-19 effects by expected year of graduation. perhaps unsurprisingly, the 2020 cohort expects much larger effects on immediate job market outcomes such as reservation wages and probability of finding a job before graduation. while average expected changes to job market outcomes are noisier for academically younger students, perhaps reflecting additional uncertainty about the longer-term impacts of covid-19, they appear to anticipate meaningful changes to their future labor market prospects. conversely, younger students also expected larger disruptions to academic outcomes such as semester gpa and study time. this section presents mediation analysis on the drivers of the underlying heterogeneity in the treatment effects. the covid-19 pandemic serves as both an economic and a health shock. however, these shocks may have been quite heterogeneous across the various groups, and that could partly explain the heterogeneous treatment effects we documented in the previous section. we proxy for the financial and health shocks due to covid-19 by relying on a small but relevant set of covariates which capture more fundamental or first-order disruptions from the pandemic. financial shocks are characterized based on whether a student lost a job due to covid-19, whether a student's family members lost income due to covid-19, the change in a student's monthly earnings due to covid-19, and the likelihood a student will fail to fully meet debt payments in the next 90 days. to measure health shocks, we consider a student's belief about the likelihood that they will be hospitalized if they contract covid-19, a student's belief about the likelihood that they will have contracted covid-19 by summer, and a student's subjective health table 3 reports summary statistics of the different economic and health proxies by demographic group. given the results in figure 1 , the remainder of the analysis will focus on three socioeconomic divisions: parental income, gender, and honors college status. our data indicate that lower-income students faced larger health and economic shocks as compared to their more affluent peers. in particular, they are almost 10 percentage points more likely to expect to default on their debt payments compared to their higher-income counterparts. additionally, lower-income students are 16 percentage points more likely to have had a close family member experience an income reduction due to covid-19. regarding the health proxies, lower-income students rate their health as worse than higher-income students and perceive a higher probability of being hospitalized if they catch the virus. finally, the differences in economic and health shocks between lower and higher-income students, as summarized by the principle components of the selected proxy variables, are statistically significant. columns (5)(7) of table 3 show that both economic and health shocks are larger for non-honors students. in fact, the average differences in the principal component scores for both the economic and health factors is larger for these two groups than for the income groups. likewise, the last three columns of the table show that women experienced larger covid-19 shocks due to economic and health factors. these differences are partly driven by the fact that, in our sample, females are more likely to report that they belong to a lower-income household than males (50% vs. 42%). in short, table 3 makes clear that the impacts of covid-19 on the economic well-being and health of students have been quite heterogeneous, with lower-income and lower-ability students being more adversely affected. to investigate the role of economic and health shocks in explaining the heterogeneous 19 eigenvalues indicate the presence of only one principal component for each of the shocks. table 4 shows estimates of equation (2) for four different outcomes (appendix table a2 shows the estimates for additional outcomes). for each outcome, five specifications are reported ranging from controlling for only demographic variables in the first specification to controlling for both economic and health factors in the fourth specification. finally, the last column includes only the principal component of each shock to provide insight about overall effects, given that certain shock proxies show high levels of correlation (see appendix table a4 for the correlations within each set of proxies). several important messages emerge from table 4 . first, both shocks are (economically and statistically) significant correlates of the covid-19 effects on students' outcomes. in particular, f-tests show that the financial and health shock proxies are jointly significant across almost all specifications. 20 this is also reflected in the statistical significance of the principal components. moreover, the fact that the effect of key proxy variables remains robust when we simultaneously control for both shocks demonstrates the robustness of our results. for example, we find that a 50 percentage point increase in the probability of being late on debt payments is associated with an increase in the probability of delaying graduation and switching majors due to covid-19 of 6.9 and 6.4 percentage points, respectively. these effects are large given that they represent more than 20 the only exception is the financial shock when explaining changes in the probability of taking classes online. journal pre-proof half of the overall covid-19 treatment effect for these variables. similarly, we find that an analogous increase in the probability of hospitalization if contracting covid-19 is associated with a 6 and 5 percentage points increase in the probability of delaying graduation and switching majors due to covid-19. second, in terms of labor market expectations, we find that the change in the expected probability of finding a job before graduation strongly depends on having a family member that lost income (which is also correlated with the student himself losing a job). in particular, the size of this effect represents 32% of the overall covid-19 treatment effect. therefore, this finding suggests that students' labor market expectations are driven in large part by personal/family experiences. third, although the proxies play an important role in explaining the pandemic's impact on students, there is still a substantial amount of variation in covid-19 treatment effects left unexplained. across the four outcomes in table 4 , the full set of proxies explain less than a quarter of the variation in outcomes across individuals. appendix figure a4 visualizes this variation by plotting the distribution of several continuous outcomes with and without controls. while the interquartile range noticeably shrinks after conditioning on the proxy variables, these plots highlight the large amount of variation in treatment effects remaining after conditioning on the proxies. finally, our results show that the financial and health shocks play an important role in explaining the heterogeneous effects of the covid-19 outbreak. in particular, columns (4) and (9) demonstrate that economic and health factors together can explain approximately 40% and 70% of the income gap in covid-19's effect on delayed graduation and changing major respectively. the gap between honors and non-honors students is likewise reduced by 27% and 39% for the same outcomes. taken together, these results imply that differences in the magnitude of covid-19's economic and health impact can explain a significant proportion of the demographic gaps in covid-19's effect on the decision to delay graduation, the decision to change major, and preferences for online learning. these results are important and suggest that focusing on the needs of students who experienced larger financial or health shocks from covid-19 may be an effective way to minimize the disparate disruptive effects and prevent covid-19 from exacerbating existing achievement gaps in higher education. journal pre-proof this paper provides the first systematic analysis of the effects of covid-19 on higher education. to study these effects, we surveyed 1,500 students at arizona state university, and present quantitative evidence showing the negative effects of the pandemic on students' outcomes and expectations. for example, we find that 13% of students have delayed graduation due to covid-19. expanding upon these results, we show that the effects of the pandemic are highly heterogeneous, with lower-income students 55% more likely to delay graduation compared to their higher-income counterparts. we further show that the negative economic and health impacts of covid-19 have been significantly more pronounced for less advantaged groups, and that these differences can partially explain the underlying heterogeneity that we document. our results suggest that by focusing on addressing the economic and health burden imposed by covid-19, as measured by a relatively narrow set of mitigating factors, policy makers may be able to prevent covid-19 from widening existing achievement gaps in higher education. notes: bars denote 90% confidence interval. notes: data in columns (2), (3) and (8) *significant at 10%, **5%, ***1%. a it refers to the mean of the first factor of a pca that uses the measures in the corresponding panel. b 1 through 5 scale where higher numbers mean better health. notes: p-value columns report the p-value of a difference in means test between the two columns indicated by the numbers in the heading. notes: data winsorized below 5% and above 95%. controls include cohort fixed effects, major fixed effects, and the economic/health proxies in table 3 *significant at 10%, **5%, ***1%. figure 1 notes: table reports correlation matrix for indicated variables key: cord-309201-c1awh48y authors: elzainy, ahmed; el sadik, abir; al abdulmonem, waleed title: experience of e-learning and online assessment during the covid-19 pandemic at the college of medicine, qassim university date: 2020-10-22 journal: j taibah univ med sci doi: 10.1016/j.jtumed.2020.09.005 sha: doc_id: 309201 cord_uid: c1awh48y objectives: during the covid-19 pandemic, academic institutions are promptly shifting all educational activities to the e-learning format. the present work describes concurrent procedures for online teaching and assessment performed at the college of medicine, qassim university, ksa. we also explored the impact of e-learning and assessment on the performance of students and faculty, and the challenges to their sustainability. methods: in this descriptive cross-sectional study, we recorded the number and duration of different online educational activities during the covid-19 pandemic. training sessions for various procedures of virtual classrooms and online assessments were organised for teachers and students. a newly established e-assessment committee arranged different online assessments. a comparison between the mean problem-based learning (pbl) grades of the same students was conducted either face-to-face or online. a student satisfaction survey and online staff focus group about the online learning experiences were conducted, and weekly staff perception reports were prepared. the results obtained were then analysed. results: a total of 620 virtual classrooms were successfully implemented over 994 h including theoretical lectures, pbl sessions, seminars, and tutorials. a significant increase in the mean pbl grades was observed for female students during the online sessions. out of the basic year students and staff, 58.82% reflected their high satisfaction towards virtual classrooms, online assessment, and online workshops. conclusion: the present study elaborates on the benefits of e-learning and assessment. we observed higher student achievements and promising staff perceptions with obvious improvement in their technological skills. these findings support the shift towards future implementation of more online medical courses. ‫ﺖ‬ ‫ﻟ‬ ‫ﺠ‬ ‫ﻨ‬ ‫ﺔ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﻘ‬ ‫ﻴ‬ ‫ﻴ‬ ‫ﻢ‬ ‫ﺍ‬ ‫ﻹ‬ ‫ﻟ‬ ‫ﻴ‬ ‫ﻜ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻭ‬ ‫ﻧ‬ ‫ﻲ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﻨ‬ ‫ﺸ‬ ‫ﺄ‬ ‫ﺓ‬ ‫ﺣ‬ ‫ﺪ‬ ‫ﻳ‬ ‫ﺜ‬ ‫ﺎ‬ ‫ﻣ‬ ‫ﺨ‬ ‫ﺘ‬ ‫ﻠ‬ ‫ﻒ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﻘ‬ ‫ﻴ‬ ‫ﻴ‬ ‫ﻤ‬ ‫ﺎ‬ ‫ﺕ‬ ‫ﺍ‬ ‫ﻹ‬ ‫ﻟ‬ ‫ﻴ‬ ‫ﻜ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻭ‬ ‫ﻧ‬ ‫ﻴ‬ ‫ﺔ‬ . ‫ﺗ‬ ‫ﻢ‬ ‫ﻣ‬ ‫ﻘ‬ ‫ﺎ‬ ‫ﺭ‬ ‫ﻧ‬ ‫ﺔ‬ ‫ﻣ‬ ‫ﺘ‬ ‫ﻮ‬ ‫ﺳ‬ ‫ﻂ‬ ‫ﺩ‬ ‫ﺭ‬ ‫ﺟ‬ ‫ﺎ‬ ‫ﺕ‬ ‫ﻧ‬ ‫ﻔ‬ ‫ﺲ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻄ‬ ‫ﻠ‬ ‫ﺒ‬ ‫ﺔ‬ ‫ﺃ‬ ‫ﺛ‬ ‫ﻨ‬ ‫ﺎ‬ ‫ﺀ‬ ‫ﺟ‬ ‫ﻠ‬ ‫ﺴ‬ ‫ﺎ‬ ‫ﺕ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﻌ‬ ‫ﻠ‬ ‫ﻢ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻘ‬ ‫ﺎ‬ ‫ﺋ‬ ‫ﻢ‬ ‫ﻋ‬ ‫ﻠ‬ ‫ﻰ‬ ‫ﺣ‬ ‫ﻞ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﻌ‬ ‫ﻀ‬ ‫ﻼ‬ ‫ﺕ‬ ‫ﺳ‬ ‫ﻮ‬ ‫ﺍ‬ ‫ﺀ‬ ‫ﻋ‬ ‫ﻘ‬ ‫ﺪ‬ ‫ﺕ‬ ‫ﻭ‬ ‫ﺟ‬ ‫ﻬ‬ ‫ﺎ‬ ‫ﻟ‬ ‫ﻮ‬ ‫ﺟ‬ ‫ﻪ‬ ‫ﺃ‬ ‫ﻭ‬ ‫ﺗ‬ ‫ﻢ‬ ‫ﻋ‬ ‫ﻘ‬ ‫ﺪ‬ ‫ﻫ‬ ‫ﺎ‬ ‫ﺇ‬ ‫ﻟ‬ ‫ﻜ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻭ‬ ‫ﻧ‬ ‫ﻴ‬ ‫ﺎ‬ . ‫ﻛ‬ ‫ﻤ‬ ‫ﺎ‬ ‫ﺗ‬ ‫ﻢ‬ ‫ﻋ‬ ‫ﻤ‬ ‫ﻞ‬ ‫ﻭ‬ ‫ﺗ‬ ‫ﺤ‬ ‫ﻠ‬ ‫ﻴ‬ ‫ﻞ‬ ‫ﺍ‬ ‫ﺳ‬ ‫ﺘ‬ ‫ﺒ‬ ‫ﺎ‬ ‫ﻧ‬ ‫ﺔ‬ ‫ﺭ‬ ‫ﺿ‬ ‫ﺎ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻄ‬ ‫ﻼ‬ ‫ﺏ‬ ، ‫ﻭ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﻘ‬ ‫ﺎ‬ ‫ﺭ‬ ‫ﻳ‬ ‫ﺮ‬ ‫ﺍ‬ ‫ﻷ‬ ‫ﺳ‬ ‫ﺒ‬ ‫ﻮ‬ ‫ﻋ‬ ‫ﻴ‬ ‫ﺔ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﺼ‬ ‫ﻮ‬ ‫ﺭ‬ ‫ﺃ‬ ‫ﻋ‬ ‫ﻀ‬ ‫ﺎ‬ ‫ﺀ‬ ‫ﻫ‬ ‫ﻴ‬ ‫ﺌ‬ ‫ﺔ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﺪ‬ ‫ﺭ‬ ‫ﻳ‬ ‫ﺲ‬ ‫ﺑ‬ ‫ﺎ‬ ‫ﻹ‬ ‫ﺿ‬ ‫ﺎ‬ ‫ﻓ‬ ‫ﺔ‬ ‫ﺇ‬ ‫ﻟ‬ ‫ﻰ‬ ‫ﺧ‬ ‫ﺒ‬ ‫ﺮ‬ ‫ﺍ‬ ‫ﺕ‬ ‫ﺟ‬ ‫ﻠ‬ ‫ﺴ‬ ‫ﺔ‬ ‫ﻣ‬ ‫ﺠ‬ ‫ﻤ‬ ‫ﻮ‬ ‫ﻋ‬ ‫ﺔ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻛ‬ ‫ﻴ‬ ‫ﺰ‬ ‫ﻋ‬ ‫ﻦ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﻌ‬ ‫ﻠ‬ ‫ﻢ‬ ‫ﺍ‬ ‫ﻹ‬ ‫ﻟ‬ ‫ﻴ‬ ‫ﻜ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻭ‬ ‫ﻧ‬ ‫ﻲ‬ . ‫ﺍ‬ ‫ﻟ‬ ‫ﻨ‬ ‫ﺘ‬ ‫ﺎ‬ ‫ﺋ‬ ‫ﺞ‬ : ‫ﺗ‬ ‫ﻢ‬ ‫ﺗ‬ ‫ﻨ‬ ‫ﻔ‬ ‫ﻴ‬ ‫ﺬ‬ ٦٢٠ ‫ﻓ‬ ‫ﺼ‬ ‫ﻼ‬ ‫ﺍ‬ ‫ﻓ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﺍ‬ ‫ﺿ‬ ‫ﻴ‬ ‫ﺔ‬ ‫ﺑ‬ ‫ﻨ‬ ‫ﺠ‬ ‫ﺎ‬ ‫ﺡ‬ ‫ﻋ‬ ‫ﻠ‬ ‫ﻰ‬ ‫ﻣ‬ ‫ﺪ‬ ‫ﻯ‬ ٩٩٤ ‫ﺳ‬ ‫ﺎ‬ ‫ﻋ‬ ‫ﺔ‬ ‫ﺷ‬ ‫ﻤ‬ ‫ﻠ‬ ‫ﺖ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﺤ‬ ‫ﺎ‬ ‫ﺿ‬ ‫ﺮ‬ ‫ﺍ‬ ‫ﺕ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻨ‬ ‫ﻈ‬ ‫ﺮ‬ ‫ﻳ‬ ‫ﺔ‬ ، ‫ﻭ‬ ‫ﺟ‬ ‫ﻠ‬ ‫ﺴ‬ ‫ﺎ‬ ‫ﺕ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﻌ‬ ‫ﻠ‬ ‫ﻢ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻘ‬ ‫ﺎ‬ ‫ﺋ‬ ‫ﻢ‬ ‫ﻋ‬ ‫ﻠ‬ ‫ﻰ‬ ‫ﺣ‬ ‫ﻞ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﻌ‬ ‫ﻀ‬ ‫ﻼ‬ ‫ﺕ‬ ، ‫ﻭ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻨ‬ ‫ﺪ‬ ‫ﻭ‬ ‫ﺍ‬ ‫ﺕ‬ ‫ﻭ‬ ‫ﺣ‬ ‫ﻠ‬ ‫ﻘ‬ ‫ﺎ‬ ‫ﺕ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻨ‬ ‫ﻘ‬ ‫ﺎ‬ ‫ﺵ‬ . ‫ﻻ‬ ‫ﺣ‬ ‫ﻈ‬ ‫ﻨ‬ ‫ﺎ‬ ‫ﺯ‬ ‫ﻳ‬ ‫ﺎ‬ ‫ﺩ‬ ‫ﺓ‬ ‫ﺫ‬ ‫ﺍ‬ ‫ﺕ‬ ‫ﺩ‬ ‫ﻻ‬ ‫ﻟ‬ ‫ﺔ‬ ‫ﺇ‬ ‫ﺣ‬ ‫ﺼ‬ ‫ﺎ‬ ‫ﺋ‬ ‫ﻴ‬ ‫ﺔ‬ ‫ﻓ‬ ‫ﻲ‬ ‫ﻣ‬ ‫ﺘ‬ ‫ﻮ‬ ‫ﺳ‬ ‫ﻄ‬ ‫ﺪ‬ ‫ﺭ‬ ‫ﺟ‬ ‫ﺎ‬ ‫ﺕ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻄ‬ ‫ﺎ‬ ‫ﻟ‬ ‫ﺒ‬ ‫ﺎ‬ ‫ﺕ‬ ‫ﺃ‬ ‫ﺛ‬ ‫ﻨ‬ ‫ﺎ‬ ‫ﺀ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺠ‬ ‫ﻠ‬ ‫ﺴ‬ ‫ﺎ‬ ‫ﺕ‬ ‫ﺍ‬ ‫ﻹ‬ ‫ﻟ‬ ‫ﻜ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻭ‬ ‫ﻧ‬ ‫ﻴ‬ ‫ﺔ‬ ‫ﻟ‬ ‫ﻠ‬ ‫ﺘ‬ ‫ﻌ‬ ‫ﻠ‬ ‫ﻢ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻘ‬ ‫ﺎ‬ ‫ﺋ‬ ‫ﻢ‬ ‫ﻋ‬ ‫ﻠ‬ ‫ﻰ‬ ‫ﺣ‬ ‫ﻞ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻤ‬ ‫ﻌ‬ ‫ﻀ‬ ‫ﻼ‬ ‫ﺕ‬ . ‫ﻋ‬ ‫ﻜ‬ ‫ﺲ‬ ‫ﺃ‬ ‫ﻛ‬ ‫ﺜ‬ ‫ﺮ‬ ‫ﻋ‬ ‫ﺪ‬ ‫ﺩ‬ ‫ﻣ‬ ‫ﻤ‬ ‫ﻜ‬ ‫ﻦ‬ ٥٢.٨٢ ٪ ‫ﻣ‬ ‫ﻦ‬ ‫ﻃ‬ ‫ﻼ‬ ‫ﺏ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺴ‬ ‫ﻨ‬ ‫ﻮ‬ ‫ﺍ‬ ‫ﺕ‬ ‫ﺍ‬ ‫ﻷ‬ ‫ﺳ‬ ‫ﺎ‬ ‫ﺳ‬ ‫ﻴ‬ ‫ﺔ‬ ‫ﺑ‬ ‫ﺎ‬ ‫ﻹ‬ ‫ﺿ‬ ‫ﺎ‬ ‫ﻓ‬ ‫ﺔ‬ ‫ﺇ‬ ‫ﻟ‬ ‫ﻰ‬ ‫ﺃ‬ ‫ﻋ‬ ‫ﻀ‬ ‫ﺎ‬ ‫ﺀ‬ ‫ﻫ‬ ‫ﻴ‬ ‫ﺌ‬ ‫ﺔ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﺪ‬ ‫ﺭ‬ ‫ﻳ‬ ‫ﺲ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺮ‬ ‫ﺿ‬ ‫ﻰ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻌ‬ ‫ﺎ‬ ‫ﻟ‬ ‫ﻲ‬ ‫ﺗ‬ ‫ﺠ‬ ‫ﺎ‬ ‫ﻩ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻔ‬ ‫ﺼ‬ ‫ﻮ‬ ‫ﻝ‬ ‫ﺍ‬ ‫ﻻ‬ ‫ﻓ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﺍ‬ ‫ﺿ‬ ‫ﻴ‬ ‫ﺔ‬ ، ‫ﻭ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﻘ‬ ‫ﻴ‬ ‫ﻴ‬ ‫ﻢ‬ ‫ﺍ‬ ‫ﻹ‬ ‫ﻟ‬ ‫ﻴ‬ ‫ﻜ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻭ‬ ‫ﻧ‬ ‫ﻲ‬ ‫ﻭ‬ ‫ﻭ‬ ‫ﺭ‬ ‫ﺵ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻌ‬ ‫ﻤ‬ ‫ﻞ‬ ‫ﺍ‬ ‫ﻹ‬ ‫ﻟ‬ ‫ﻜ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻭ‬ ‫ﻧ‬ ‫ﻴ‬ ‫ﺔ‬ . ‫ﺍ‬ ‫ﻻ‬ ‫ﺳ‬ ‫ﺘ‬ ‫ﻨ‬ ‫ﺘ‬ ‫ﺎ‬ ‫ﺟ‬ ‫ﺎ‬ ‫ﺕ‬ : ‫ﺃ‬ ‫ﻭ‬ ‫ﺿ‬ ‫ﺤ‬ ‫ﺖ‬ ‫ﻫ‬ ‫ﺬ‬ ‫ﻩ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺪ‬ ‫ﺭ‬ ‫ﺍ‬ ‫ﺳ‬ ‫ﺔ‬ ‫ﻓ‬ ‫ﻮ‬ ‫ﺍ‬ ‫ﺋ‬ ‫ﺪ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﻌ‬ ‫ﻠ‬ ‫ﻢ‬ ‫ﺍ‬ ‫ﻹ‬ ‫ﻟ‬ ‫ﻜ‬ ‫ﺘ‬ ‫ﺮ‬ ‫ﻭ‬ ‫ﻧ‬ ‫ﻲ‬ ‫ﻭ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﺘ‬ ‫ﻘ‬ ‫ﻴ‬ ‫ﻴ‬ ‫ﻢ‬ . ‫ﻻ‬ ‫ﺣ‬ ‫ﻈ‬ ‫ﻨ‬ ‫ﺎ‬ ‫ﺍ‬ ‫ﺭ‬ ‫ﺗ‬ ‫ﻔ‬ ‫ﺎ‬ ‫ﻋ‬ ‫ﺎ‬ ‫ﻓ‬ ‫ﻲ‬ ‫ﺇ‬ ‫ﻧ‬ ‫ﺠ‬ ‫ﺎ‬ ‫ﺯ‬ ‫ﺍ‬ ‫ﻟ‬ ‫ﻄ‬ ‫ﻼ‬ ‫ﺏ‬ ‫ﻭ‬ ‫ﺗ‬ ‫ﺼ‬ ‫ﻮ‬ ‫ﺭ‬ ‫ﺍ‬ ‫ﻭ‬ objectives: during the covid-19 pandemic, academic institutions are promptly shifting all educational activities to the e-learning format. the present work describes concurrent procedures for online teaching and assessment performed at the college of medicine, qassim university, ksa. we also explored the impact of e-learning and assessment on the performance of students and faculty, and the challenges to their sustainability. methods: in this descriptive cross-sectional study, we recorded the number and duration of different online educational activities during the covid-19 pandemic. training sessions for various procedures of virtual classrooms and online assessments were organised q7 for teachers and students. a newly established e-assessment committee arranged different online assessments. a comparison between the mean problem-based learning (pbl) grades of the same students was conducted either face-to-face or online q8 . a student satisfaction survey and q9 online staff focus group about the online learning experiences were conducted, and weekly staff perception reports were prepared. the results obtained were then analysed q10 . results: a total of 620 virtual classrooms were successfully implemented over 994 h including theoretical lectures, pbl sessions, seminars, and tutorials. a significant increase in the mean pbl grades was observed for female students during the online sessions. out of the basic year students and staff, 58.82 q11 % reflected their high satisfaction towards virtual classrooms, online assessment, and online workshops. the present study elaborates on the benefits of e-learning and assessment. we observed higher student achievements and promising staff perceptions with obvious improvement in their technological skills. these findings support the shift towards future implementation of more online medical courses. the covid-19 pandemic has led to the global disruption of medical education which necessitated working online. 1 urgent response to the current situation required an increase in medical educators' awareness towards online teaching. 2 several researches determined the effectiveness of digital technologies for life-long e-learning and continuous professional development. 3 e-learning has been established worldwide in response to the shortage of health educators and the need to switch into teal q13 . 4 ,5 e-learning has several advantages q14 , such as encouraging students for self-directed learning 6 and updating the curricula. 7 the college of medicine at qassim university, established in 2001, adopted the pbl system as an interactive educational strategy. 8 the college shifted into digitalised pbl materials for one year, which was a good preparation for complete online pbl sessions. recent technologies allowed the progressive innovation of e-learning. 9, 10 several studies have investigated the benefits of these technologies in medical education, especially the pbl system. 11, 12 official online platforms, mainly through the blackboard learning management system (lms) version 9.1 (blackboard, washington, dc), are used in qassim university to conduct educational sessions including lectures, tutorials, pbl sessions, seminar presentations, and open discussion forums. online formative assessments, through the blackboard, were also performed. these assessments reflect the nature of online learning and give the students more responsibility for their q15 learning. 13 online assessment allows the learners to demonstrate their capabilities in critical thinking and solving problems, which are the key benefits of shifting from traditional teaching to e-learning where the teacher is mainly a facilitator. 14 the present study described the procedures performed to facilitate the urgent transition to e-learning and online assessment during the covid-19 pandemic and to highlight its expected benefits and impact on student and staff satisfaction and performance. it also aimed to compare the scores of male and female students during both face-to-face and online pbl sessions, and to explore the expected challenges of this experience to sustain its future implementation after the covid-19 pandemic. this was a descriptive observational study conducted q16 over 65 days. during this period, four weeks of pbl sessions were accomplished for the basic year students. the study collected data from all the undergraduate students, involved in this e-learning experience, of preclinical phase (n ¼ 425) (male students ¼ 252; female students ¼ 173) and clinical phase (n ¼ 249) (male students ¼ 155; female students ¼ 94), and 120 instructors (47 basic and 73 clinical). data of 23 students were not included as they dropped out from both the basic and clinical phases q17 . the aim and procedure of the study were clarified to the participating students and staff, and their consents were obtained. students' identity will not be disclosed for ethical reasons. the confidentiality of the information obtained was maintained. all theoretical activities, including lectures, pbl, tutorials, and seminar sessions, in college of medicine at qassim university were adapted to the e-learning modality through the blackboard, version 9.1 (blackboard, washington, dc) and zoom cloud meetings . the authors contributed to the rescheduling of timetables for all these activities. all practical or clinical sessions were postponed. webinars about virtual classrooms and online assessments were presented for staff and students. an e-assessment committee was established for the first time consisting of thirteen membersdnine basic scientists and four clinical staff. this committee formulated, revised, and uploaded the online assessments with their logistics and evaluated the results using item analysis. the educational strategies for the live streaming sessions, during the covid-19 pandemic, at the college of medicine at qassim university included lectures, pbl sessions, tutorials, and seminar presentations to achieve the course objectives. comparisons of the mean pbl marks during control (face-to-face) and online (virtual) sessions were done for male and female students of the first and third years. secondyear students were excluded as they started a new block with the shift to e-learning. the students completed an online satisfaction survey on their perceptions of the e-learning experience, with a 5-point likert scale. weekly reports concerning staff perceptions on the effectiveness of live streaming activities were collected with a 3-point likert scale. the cronbach's alpha test was used for testing the internal consistency and reliability of the students' and staff's perceptions q20 . kendall's tau b, a nonparametric measure of association that exists between two variables, was used to test the correlation of the items of the students' survey. an online focus group for the staff, using the focus-groupit software (https://www.focusgroupit.com/), was performed. the questions were presented in the form of a swot analysis (figure 1 ). the group was composed of one moderator, one observer, and seven participants (the supervisors of basic sciences departments and one radiology staff). the two-hour-long online synchronous focus group discussion was recorded. the analysed data were reviewed and interpreted by an independent investigator. the data were analysed using the statistical package for the social sciences (spss) software, version 21 (ibm corp., armonk, ny). descriptive statistics (percentages, mean, q21 and standard errors of the mean) were used to describe the quantitative variables with their analysis through pairedsamples (to compare the male and female mean pbl marks) and independent t-tests (to compare the mean marks of the face-to-face and online pbl sessions). a p-value of <0.05 was considered significant. weekly reports though the official learning management systemdblackboarddregarding the number, duration, and modality of different educational activities including the live streaming sessions and students' attendance rate were collected from the course organisers in coordination with the e-learning unit, phase coordination, and e-assessment committees. evaluation of students during their e-learning experience was done based on their assessment during the online pbl sessions. tables 1 and 2 present the details regarding the virtual classrooms, conducted mostly through blackboard, of the preclinical and clinical phases q22 . they represent the numbers of e-learning activities successfully implemented. significant increase in the mean pbl marks of the female students of both first and third years was observed during the online sessions than in the face-to-face sessions of the relevant year. additionally, there was a significant increase in the mean pbl marks of the female students of the first and third years than that of the male students of the relevant year in both face-to-face and online sessions (tables 3 and 4) . two hundred and fifty q23 students of the preclinical phase (58.82%) completed the perception survey. students' satisfaction towards the new modality of e-learning and online assessment was reported (table 5 ). the items covered in the students' survey included the following: the success of elearning in compensating for the urgent suspension of faceto-face teaching during the covid-19 pandemic, efficiency of instructions announced before the online teaching, staff's resistance and experience in e-learning requirements, and effectiveness of online assessment in testing their knowledge and skills levels. the cronbach's alpha test performed for all items of this survey resulted in an overall score of q24 0.67. kendall's tau b was used to test the correlation of these items. the correlation coefficient ranged from 0.134 to 0.394. open-ended comments were received from 225 out of 250 students (90%), which were classified into two categoriesdone for teaching activities and the other for online assessment. eighty-seven per cent q25 (n ¼ 195) of the students reflected their enthusiasm towards the e-learning modality: for example, 'very amazing; online teaching experience is the future of learning' and 'online lectures are super-satisfying' q26 . they expressed their satisfaction in having the same staff delivering the lectures for both male and female students. e-learning provided a chance for the shy studentsdas they could participate freely through online chattingd and motivated them for verbal discussions: 'initially, i was shy and shared my opinions during the open discussions only through chat; by the time i was encouraged to share orally q27 '. they expressed satisfaction with the rescheduling of educational activities and their allocated time and duration; for example, 'modification of the timetable allows more time for educational and recreational activities, thus allowing to live like a normal human'. the students expressed that some of the staff lacked adequate experience in conducting open discussions during online teaching. seventy per cent (n ¼ 158) of the students reported that frequent online quizzes motivated them to study; for example, 'online quizzes are very helpful to improve my grades'. they appreciated the conduction of the mock quiz; for example, 'mock exam was very helpful for training before the online exams'. electronic assessment ensured fewer errors carried 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 out by the students while filling out bubbles in q28 their answer sheets. lastly, they recommended that some courses could be implemented online in the future. staff satisfaction in virtual classrooms was also observed (tables 6 and 7) , which reflected that the live streaming sessions were very effective. the online focus group discussion was categorised into four themes based on the swot analysis. concerning the strengths, most of the staff agreed that the new arrangement of educational activities efficiently compensated for the suspension of face-to-face teaching q29 . they were satisfied with the university's efforts to improve staff awareness regarding virtual classrooms and online assessment through webinars q30 . the majority of them appreciated the tremendous shift to implement online summative assessments; for example, 'online assessment assured the staff about students' achievement of learning outcomes' and 'conduction of live oral exams based upon clear standardised checklist was highly effective'. the concerns expressed by the participants included some staff's limited online teaching experience and q31 the insufficient number of it technicians which interferes with proper digitalisation. regarding the opportunities that could be gained, the staff acknowledged the constitution of the e-assessment committee in the focus group; 'such committee was an urgent requirement to tackle the full first 114 27 231 3634 37 91 13 second 108 7 216 3512 31 72 12 third 81 34 189 1979 19 84 12 total 303 68 636 9125 87 247 37 bb: blackboard. a. elzainy et al. 4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 responsibility for online assessments'. they reflected the improvement of their expertise towards the novel pedagogical e-learning techniques and their enthusiasm to construct online courses; for example, 'adoption of electronic courses allows better arrangement of educational activities and gives more chances for students' self-directing learning'. two staff members reported the inconvenience of some students in taking the online quizzes, especially during the initial period, while the others specifically talked about q32 those who live in distant rural areas with insufficient internet coverage. four staff members emphasised the limited capabilities of the official version of blackboard, particularly in conducting the morning sessions. advanced technologies emerged during the covid-19 pandemic to sustain world productivity. the horizon 2020 teaching and learning report highlighted the role of advanced technology in medical education. the present study represents the major change in the educational culture. e-learning was highly beneficial for competent educators as it decreased the needs for in-class attendance. 15 in agreement with mccoy et al., 16 students and staff reflected that live streaming lectures efficiently compensated the suspension of face-toface teaching and provided more chances for open discussions. nomination of the same staff to virtually present each topic for both male and female students encouraged more peer sharing and competition among the students q33 . the rescheduled educational activities were more convenient to the students, as reflected in their attendance. recent technologies resolved the lack of physical attendance and increased learning effectiveness. 17, 18 the horizon 2020 report highlighted the effectiveness of online teaching in overcoming the restrictions such as shortage of venues for large group lectures. however, one of the main challenges of e-learning, as reflected in the current work, is in teaching the psychomotor, practical, and clinical skills efficiently. murphy 19 recently reported that most medical schools suspended q34 the clinical settings during the covid-19 pandemic. this could be overcome by using virtual-reality simulators. 20, 21 unfortunately, most of the students were unsatisfied with how some staff members practiced e-learning. a previous study performed in a similar culturedunited arab emiratesdobserved that the teachers felt worried about the shift into a new educational strategy. psychological assurance was recommended to encourage them to deal with the unknown consequences. multiple webinars about proper virtual classrooms and peer sharing of experiences between the staff members solved this problem. goh et al. 18 claimed that live streaming applications will improve the technological skills of the educators. pivot meded 22 appreciated the free elearning webinars for health professional educators worldwide and the national coordination between medical schools in sharing such training courses. successful collaborative online learning demanded the support of it technicians, as previously reported. 16, 23 the cronbach's alpha test implied that the survey tool had a good level of internal consistency and reliability for both the students' survey and staff's overall satisfaction in the live streaming experience. the correlation coefficient indicated that the items in the students' survey were well correlated. however, the application of kendall's tau b test on the staff perception was non-feasible since it measures the association between two variables, unlike the current study which focused on the staff perception on the effectiveness of virtual classrooms. the higher achievement level detected in the mean marks of online pbl compared with face-to-face sessions could be attributed to the easier access to the explanation of the phenomena. the students' assessment during the pbl sessions in this work was based upon their commitment, team spirit, interaction with peers and tutors, presentation skills, and ability for brain storming and analysing the phenomena. the students' assessment during the pbl sessions was based upon their performance during the session rather than their achievements through other summative assessment methods such as multiple choice questions which are associated with higher chances of cheating q35 . 24 these findings are in agreement with the previous study 25 which observed that online pbl enhanced critical thinking and fulfilled the intended learning objectives. therefore, online pbl could enhance the metacognitive skills, ability to solve problems, and team working. collaborative interaction in the online environment helped enhance peer sharing. 26 after the sars epidemic, one medical school in china adopted online pbl as an educational strategy for the subsequent years. 1 the higher pbl scores in female students could be attributed to the difference in the style of thinking, learning, and capabilities of problem-solving. 27 makonye 28 detected that female students have more ability for knowledge perception and reflection of their own ideas, and a higher competitive attitude. additionally, this gender variation in pbl scores could be due to a higher commitment of female students in attending different educational activities. 29 however, ajai and imoko 30 observed equal performance of male and female students in pbl sessions, and q36 recommended further studies to examine the underlining causes. validity and reliability of assessment should be established to ensure students' achievement of the learning objectives. 14 the ability to solve any technical obstacle met during the online mock exam helped the e-assessment committee to manage the subsequent exams appropriately. 25 results of the online assessments and their item analysis represented evidence-based high-quality evaluation. this perception was supported by jawaid et al. 31 and bandele et al. 32 who reported that students expressed a more positive attitude towards online exams. similar findings were also observed by martin et al. 33 who emphasised that ideal assessment was based upon optimal evaluation strategies. the online tests were revised by the e-assessment committee, in coordination with the supervisors of the relevant departments, to estimate sufficient time for each exam. redistribution of students' grades towards more objective assessment strategies such as pbl, seminar presentations, and oral assessments was kept in consideration. this minimised the subjectivity of grading, as expressed in the feedback gained from the focus group, and discrepancy in students' grades, in agreement with ozden et al. 34 . the procedures implemented during the online exams to minimise the chance of cheating and unauthorised collaboration with peers included scheduling of brief exams of not more than 30 min with a timer set for the whole exam and also for each individual q37 question. formulation of a considerable number of scenario-based questions was implemented focusing on higher-order critical thinking, following the bloom's taxonomy. 35 questions and answer choices were randomised; each question was presented on a separate page with forced completion of these questionsdstudents were not allowed to return to the previously-submitted questionsdin agreement with fontanillas et al. 36 . a backup version of the questions with the same difficulty index was prepared for those students who experienced technical difficulties during submission. the answers were not displayed to the students until the examiner's permission was obtained and all participants had completed answering q38 . the potential use of teal in medical educationdthe future learning eradis expected after the resolution of the covid-19 pandemic. analysis of the benefits of e-learning in the present study could help the decision-makers of educational policies and committees of curriculum reform to implement them in the future. goh and sandars 11 pointed out that the medical educators worldwide, being deeply involved in the current tremendous shift towards e-learning, have to adapt to the current major educational challenges. appropriate e-learning resources should be available to guarantee the implementation of these enormous changes. more use of technological tools will enable the medical schools to establish the active process of e-learning. 37 the horizon 2020 report focused on 'learning engineering' such as the virtual reality simulators to facilitate the rapid development of teaching and learning. online courses form one of the key success factors to achieve the program learning outcomes of medical graduates. the current work explored the benefits of the shift towards e-learning and online assessment, which is a promising strategy with great educational potentials, after the who's declaration of the covid-19 pandemic. this successful digital learning environment was observed in terms of student and staff satisfaction, achievement, and improvement of technological educational skills. the present study recommends a sustained monitoring and updating of the e-learning resources, particularly the official lms, and availability of sufficient number of information technology personnel. multiple webinars and workshops suggested increasing the student and staff awareness of online teaching and assessment via q39 improving the current faculty development program. furthermore, the adoption of teaching through complete and partially online courses, and a total shift from physical attendance for pbl sessions to online ones are encouraged in the future. all data are available from the corresponding author upon reasonable request. researchers would like to thank the deanship of scientific research, qassim university for funding the publication of this project. the authors have no conflict of interest to declare. ethical approval was taken from the ethical committee of qassim university. written informed consents were obtained from all the participants. waa conceived and designed the study, conducted research, provided research materials, and collected and organised the data. ae and aes analysed and interpreted the data. all authors wrote the initial and final drafts of the article, and provided logistic support. all authors have critically reviewed and approved the final draft and are responsible for the content and similarity index of the manuscript. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 jtumed653_proof ■ 22 october 2020 ■ 6/7 covid-19 and medical education online learning during the covid-19 pandemic: what do we gain and what do we lose when classrooms go virtual? internet-based learning in the health professions: a meta-analysis world health organization. human resources for health and implementation of the outcomes of the united nations' high-level commission on the role of e-learning in medical education the impact of e-learning in medical education analysing undergraduate medical curricula: experience from a saudi medical college ethical priority setting for universal health coverage: challenges in deciding upon fair distribution of health services assessing the role of online technologies in project-based learning the use of e-learning in medical education: a review of the current situation meta-analysis of inquiry-based learning effects of guidance classroom assessment in web-based instructional environment: instructors' experience students' perceptions of e-assessment at saudi electronic university using online digital tools and video to support international problem-based learning developing technologyenhanced active learning for medical education: challenges, solutions, and future directions evaluation of e-learning for medical education in low-and middle-income countries: a systematic review a vision of the use of technology in medical education after the covid-19 pandemic covid-19: how the virus is impacting medical schools virtual patient design and its effect on clinical reasoning and student experience: a protocol for a randomized factorial multi-center study learning or technology enhanced learning in medical educationdhope, not hype problem-based learning in the digital age formative and summative assessment of the problem based learning tutorial session using a criterion referenced system the effects of online and face-to-face problem based learning environments in mathematics education on student's academic achievement e-assessment process: giving a voice to online learners gender-related differences in learning in student-led pbl tutorials the enactment of problem based approaches in pre-service mathematics and the levels of performance of teacher students in problem projects of teacher students in problem projects female students get more marks as compared to male students: a statistical study gender differences in mathematics achievement and retention scores: a case of problem-based learning method computer based assessment (cba): perception of residents at dow university of health sciences opinions of undergraduates on the use of electronic examination in a nigerian university award-winning faculty online teaching practices: course design, assessment and evaluation, and facilitation students' perceptions of online assessment: a case study taxonomy of educational objectives e-assessment process: giving a voice to online learners exploring the implementation of an electronic record into a maternity unit: a qualitative study using normalisation process theory the authors would like to acknowledge the college of medicine, qassim university for the generous facilitation of the required learning resources. key: cord-274544-mxkputbd authors: gautam, ritu; sharma, manik title: 2019-ncov pandemic: a disruptive and stressful atmosphere for indian academic fraternity date: 2020-04-11 journal: brain behav immun doi: 10.1016/j.bbi.2020.04.025 sha: doc_id: 274544 cord_uid: mxkputbd nan brain, behavior, and immunity journal homepage: www.elsevier.com/locate/ybrbi the zoonotic virus (2019-ncov) has terrifically affected the world that it becomes even hard to breathe. the sharp pang of this pandemic (2019-ncov) is exponentially sweeping across the world and is triggering chaos, fear, anxiety, and stress among the people (montemurro, 2020) . it may cause pernicious impacts on cognitive functions, and also inflict extensive neurological disruption (wu et al. 2020) . by april 7, 2020, more than one million (1,383,436) persons have been globally infected due to the convergence of this uncontrollable infectious disease. most of the global population has been depressed and threatened due to the exponential growth of infection and the increasing number of fatalities (covid-19 coronavirus pandemic, 2020) . to avoid the mass spreading of this pandemic virus, the decision regarding nationwide lockdown has been taken. no doubt, this will save the masses of life. however, this lockdown is also creating chaos and huge difficulties for the people (sharma et al., 2020) . the 2019-ncov has shattered the lifestyle, daily routine, business, stock market and even the education system of the globe. the indian academic fraternity has been badly affected by this pandemic. due to scrupulous isolation measures and closedown of universities and colleges, academic fraternity is under insurmountable mental pressure which is raising the prevalence and rate of stress, anxiety and depression among them (charnsil and chailangkarn, 2020) . primarily, there is a great threat of being infected. the concern of family and friends is also intensifying the stress rate. furthermore, due to the dilemma of lockdown and to maintain the social distancing, the authorities have instructed the teaching fraternity to take their classes online (choudhury et al., 2020) . during this pandemic, it is very tough to prepare and deliver the quality lectures to the students and the situation gets worse when the teaching is online as most of the teachers have not been trained for the effective use of web resources for teaching. additionally, the teachers who are specialized in conventional teaching found it more difficult to cope with online methods. moreover, the level of frustration during this pandemic would be on the higher side for the old-aged and psychologically-disordered teachers as they are more vulnerable to contamination. the liability of online classes will further intensify the stress as they are not comfortable in using electronic gadgets and online resources. above all, some of the students are not serious in online teaching as the teachers have negligible control over students in online teaching. additionally, the teachers residing in the remote areas are more frustrated and depressed because due to unavailability or poor internet connection, they are not able to complete their tasks (zhang, 2020) . most of the adhoc and contractual teachers of different colleges and universities are also worried about their job and salary. the teachers employed in small-scale institutes don't get the proper salary and to manage the daily needs of their family they normally indulge in extra works like tuition or part-time job (evening shifts). this pandemic has shut down all the sources of their income. additionally, the level of stress and frustration is on the peak for the guest lecturers as they were paid on the basis of the lectures taken per day. the interruption in research activities during this outbreak is also being a reason of stress for the teaching community. due to the lockdown of colleges, universities the teachers and students (particularly science faculty) are not able to use the facilities of their laboratories. moreover, most of the teachers are not able to access online journals as they have institutional web access only. this pandemic has also significantly affected the mental state of the students. they are also in the dilemma of being infected with this unfortunate pandemic virus. the massive transmission of the fake news over social sites (whatsapp, twitter, facebook) and media has created chaos and stressful atmosphere for the students. the scary atmosphere is affecting the concentration level and the learning ability of the students. the training students (mca/mba/b.tech) who have joined their internship in different companies are not able to get the hand-on experience of the live projects. some of the research scholars (life and applied sciences) who have been carrying out their experiments in their institutional labs for the last couple of weeks and were about to finish their studies feel more frustrated as they have to re-conduct their experiments due to this lockdown. furthermore, different kinds of examinations have been postponed due to this zoonotic virus and there is complete uncertainty about the examination policies i.e. how and when it will be conducted. in addition to the regular examination, most of the competitive examination has also been aborted or postponed for which students were preparing for the last couple of months or even a year. the postponement of the examinations is also causing frustration and stress among the students. these different kinds of tensions disrupt the sleep time of the students which eventually decreases the body's immunity and hence makes them more susceptible to infection. some of the students have taken educational loan for their higher studies in abroad. the restricted transmission has delayed their joining process however; the cycle of monthly instalments is going on. therefore, there will be an excessive financial burden on the students which will indirectly be a cause of stress or anxiety among them. some of the researchers who have got the opportunity of post-doctorate fellowship in different international universities are under tremendous pressure about their future. several such scholars are under high financial burden as most of them have resigned their current jobs for this fellowship. the long prevalence of this pandemic may create different types of psychological disorders among teachers and students. the consequences of this pandemic can be worsened for psychologically weak students and teachers. finally, this pandemic has taught us that the subject of online teaching needs to be incorporated at the primary and higher level of education. additionally, all educational institutes need to periodically organize the workshops related to the use of online learning and teaching. likewise, a short course on stress management needs to be mandatory for all the students so that they can beat the stress in similar catastrophic events. above all, there is a need to be post-traumatic stress disorder and related factors in students whose school burned down; cohort study working from home under social isolation: online content contributions during the coronavirus shock covid-19 coronavirus pandemic the emotional impact of covid-19: from medical staff to common people a chaotic and stressed environment for 2019-ncov suspected, infected and other people in india: fear of mass destruction and causality nervous system involvement after infection with covid-19 and other coronaviruses thoughts on large-scale long-distance web-based teaching in colleges and universities under novel coronavirus pneumonia epidemic: a case of chengdu university supplementary data to this article can be found online at https:// doi.org/10.1016/j.bbi.2020.04.025. key: cord-320361-lld2oib2 authors: jackman, deirdre; konkin, jill; yonge, olive; myrick, florence; cockell, jim title: crisis and continuity: rural health care students respond to the covid-19 outbreak date: 2020-09-22 journal: nurse educ pract doi: 10.1016/j.nepr.2020.102892 sha: doc_id: 320361 cord_uid: lld2oib2 the covid-19 outbreak in winter (2020) has caused widespread disruption for health sciences students undergoing clinical placements—vital periods of experiential learning that cannot be substituted with distance alternatives. for students placed in rural areas, already coping with isolation, precarious supply chains and shortages of essential personnel, the effects of the covid-19 outbreak may have far-reaching implications for psychosocial wellness, self-efficacy and clinical judgment. four nursing and eight medical students (n = 12) supplied photographs and commentary documenting the experience of withdrawing suddenly from clinical sites in rural alberta. collaborative, thematic analysis revealed continuities between preand post-outbreak life, both for the students and their rural hosts. social determinants of health such as seclusion, environmental hazards, and health-seeking behaviors carried over and compounded the effects of the outbreak on the placement communities and clinical sites. other continuities included the reliance on technology for clinical and social connectivity, and capitalizing on natural settings to cope with isolation and confinement. prolonged liminality, lack of closure, and the loss of team identity were the greatest stressors brought on by the suspension of clinical activities. however, the participants felt well equipped to deal with these circumstances through the resilience, adaptability, and community ethos acquired during their placements. on march 5, 2020, alberta reported its first confirmed case of covid-19. over the subsequent two weeks, large sectors of the provincial economy came to an abrupt halt as public services, schools, retailers, restaurants, tourism, sporting, arts, and entertainment facilities were shuttered-a pattern repeated around the globe. among the millions of lives disrupted were health sciences students undergoing clinical placements-a vital period of hands-on, experiential learning for which there can be no long-distance alternative (stokes, 2020) . while no one would wish to jeopardize the safety of patients, students and health care providers in hospitals, the experience of being pulled abruptly from an extended, immersive placement was undoubtedly jarring. for nursing and medical students placed in rural and remote alberta communities-already struggling with tenuous supply lines and shortages of essential personnel-the covid-19 outbreak could be considered the latest and most dramatic test of resolve, adaptability, and the capacity to draw clinical and holistic insights from a crisis. since 2016, our research team has collaborated with consecutive cohorts of nursing and medical students, undertaking placements in rural alberta, to construct compelling, multimedia stories around the challenges and rewards of rural, clinical education and practice. we use participatory action modalities-photovoice and digital storytelling-to put these students in control of their own narratives, and to help them translate their clinical experiences into knowledge. our winter 2020 cohort began much as anticipated, with photographs and commentary around the rural setting; social determinants of health; acclimating to the local community; and being welcomed into the rural, interdisciplinary health care teams. everything changed in mid-march, when the faculties of nursing and medicine suspended clinical experiences until further notice, while the local hospitals implemented covid-19 protocols and braced for a potential onslaught of cases. faced with an early end to our project, we opted instead to present our participants with a new research question: what is the story of withdrawing from a rural, clinical placement in the midst of a public health emergency? in canada, the closest precedent to the current situation was the sars outbreak of 2003, experienced most acutely in and around the toronto metropolitan area, with over 14,000 persons quarantined (cava et al., 2005) . nurses and other frontline health care workers were found to be especially vulnerable to burnout and post-traumatic stress (brooks et al., 2020; marjanovic, greenglass & coffey, 2007; maunder et al., 2003; reynolds et al., 2007) . similar patterns were observed in taiwan (bai et al., 2004; pan et al., 2005) and the people's republic of china (prc) (wu et al., 2009) . amongst the most significant stressors affecting hospital staff during this time were isolation, stigma and anxiety, owing to the risk of infecting others (brooks et al., 2020; marjanovic, greenglass & coffey, 2007; maunder et al., 2003; pan et al., 2005) . maunder et al. (2003) note that even workers not at bedside, such as administrators, or staff taking days off, felt the dissonance of being cut off from the frontline team at a critical time (p. 1249). prior to the covid-19 outbreak, opportunities to study the psychosocial effects of quarantine on a large scale were limited. reynolds et al. (2007) suggest that health care workers bore the brunt of the 2003 sars outbreak, the 10-day quarantine and its psychological sequelae, in comparison to quarantined members of the general public who j o u r n a l p r e -p r o o f did not share their sense of burden and anxiety about infecting others. wang et al. (2010) found no significant, short-term psychological effects of a seven-day quarantine on undergraduate students during the 2009 outbreak of h1n1 flu in hangzhou, zhejiang province, prc. by contrast, cava et al. (2005) found the stress of quarantine was linked to public stigma and the availability of up-to-date information, during the 2003 sars outbreak in toronto. the widespread perception that quarantine was "an outdated public health measure… [evoking] images of a bygone era," (p. 398) may have driven anxiety amongst those affected, as well as depressing overall compliance. as the potential scale of the covid-19 pandemic was just becoming apparent, brooks et al. (2020) reviewed the literature on mass quarantine, concluding that its imposition and duration must be weighed against its psychological consequences. suspense and confusion owing to lack of information-or competing sources of information-are primary lockdown stressors which public health officials must strive to mitigate. another major area of concern, already playing out in daily headlines, is the effect of seclusion and economic shutdown on individuals already on the socioeconomic margins (cava et al., 2005) . brooks et al. observe that "coordination for provision of supplies should ideally occur in advance, with conservation and reallocation plans established to ensure resources do not run out" (p. 918). rural and remote communities, already at the mercy of precarious supply chains, volatile commodity prices and shortstaffed hospitals, must be counted amongst the most vulnerable regions as covid-19 increasingly dominates our public health discourse. in the revenue-driven, us health care sector, the pandemic financial shock alone may drive over 350 rural hospitals out of business (kacik, 2020) . j o u r n a l p r e -p r o o f whether placed in rural or urban settings, health sciences students confront an equally uncertain future, at least for the short term. the suspension of all clinical learning, especially for students nearing licensure, may engender hopelessness, helplessness, and long-term psychosocial harm (shaw, 2020) . helplessness -the learned response that "their fate is out of their control" (p. 2) -has the most troubling implications, insofar as it may shatter self-confidence and self-reliance, carefully nurtured over months by preceptors and other staff members in the clinical environment. while many post-secondary institutions are making a hasty (and overdue) shift to distancelearning in nursing (jowsey et al., 2020) and medicine (stokes, 2020) , pre-licensure students who have completed all but their clinical education face a unique predicament. they must somehow stay focused on their career goals, while shut out indefinitely from the one setting necessary to advance them. stokes (2020) advocates the responsible integration of senior medical students into health care sites struggling to prepare for increased patient volumes. in the uk, emergency policy initiatives are already underway to create paid, clinical placements for senior nursing students, as well as re-certifying practitioners who have moved on from their careers in health care (mitchell, 2020) . since 2016, we have carried out an iterative, participant action research initiative based on a hybridization of photovoice and digital storytelling methodologies. this project, entitled [omitted for blind peer review], was designed 1) to give nursing and medical j o u r n a l p r e -p r o o f students co-equal status with the researchers in collection, analysis and dissemination of data pertaining to rural preceptorship; 2) to supplement the experiential learning of health care preceptorships, by giving these students an opportunity to translate their clinical experiences into knowledge through digital media; 3) to give nursing and medical students, placed side-by-side, opportunities for interprofessional learning and democratic dialogue; and 4) to produce research output accessible to educators, students, clinicians and policymakers, as a means of advocating for rural health care careers and policy reform. as the covid-19 outbreak developed and eventually brought a halt to all clinical rotations in mid-march, 2020, we asked participants to focus their remaining data collection and reflections to the experience of withdrawing suddenly from the rural clinical setting, and its immediate aftermath. ethical approval for human research was granted by our employing institution's human research ethics board. all participants were apprised of their right to withdraw from the project and gave signed, informed consent, with the understanding that their anonymity could not be guaranteed owing to the nature of photographic data. pseudonyms are nonetheless employed in this article. in winter 2020, through purposive sampling, we recruited four, senior baccalaureate nursing students and eight third year medical students, all of whom had opted to undertake clinical placements in rural alberta. for the nursing students, this entailed a j o u r n a l p r e -p r o o f 10-week preceptorship in a rural acute care setting, just prior to graduation. the medical students were already onsite, having begun their 42-week, longitudinal integrated clerkship the previous fall. nursing and medical students in rural communities but not coplaced with one another were excluded from the sample. recruitment was carried out by the project manager (not a faculty member) in cooperation with the students' clinical placement coordinators. the project manager delivered a 30-minute project orientation, covering photovoice and digital storytelling methodology and ethics, to the four nursing students during classroom time, and to the eight medical students via videoconference link. the participants (n=12) were placed at acute-and community care sites in four rural alberta communities, with populations ranging from 3,100 to 10,500, between 90 km and 740 km distant from the metropolitan site of the faculties. two third year medical students and one fourth year nursing student were placed in each community. in addition to practicing in hospitals with 21-54 acute care beds, the students accompanied their preceptors on clinic visits to outlying communities, some barely accessible. the medical students also spent significant time in the community clinics of their preceptors. the students carried out photovoice and digital storytelling activities in connection with their clinical work. photovoice data collection began with a five-week acclimation period, wherein students documented the rural context and community through a minimum of six photographs and attached commentary, posted to a group discussion forum on voicethread.com. the project manager facilitated a mid-project teleconference j o u r n a l p r e -p r o o f wherein students were able to discuss their photographs in real time. the second and final five-week phase of photovoice-wherein students were to have explored interprofessionalism and clinical-community engagement-was interrupted by the covid-19 outbreak. once it became apparent that students would not return to the rural clinical setting, we suggested they complete the project offsite, focusing on the personal and community impact of the public health crisis. a final teleconference was held as originally scheduled, and the four nursing students submitted summative, audio reflections as a means of tying their photographs together into digital stories (medical students were invited but not required to complete this stage). once the term was complete for nursing students and their grades had been assigned by nursing faculty, the project manager invited all students to take part in the study by agreeing to submit their photovoice and digital storytelling content to the project dataset, with no further participation required. all co-placed nursing and medical students thereby had the benefit of experiential learning and knowledge translation through the photovoice methodology, without being coerced to take part in the study. we nonetheless obtained the signed consent of all students who contributed photographs and commentary to the voicethread.com forum. participant engagement in all stages of research-collection, analysis and dissemination-is a central priority of participant action. the advent of web 2.0 and user-based content platforms, such as voicethread.com, have made the latter two stages much more achievable, particularly where large distances and asynchronous schedules j o u r n a l p r e -p r o o f are involved. the group discussion forum and teleconferences provided students and researchers a safe, democratic space to explore issues such as clinical conflict and rural/urban disparities, while building consensus around major themes cutting across the entire dataset. we in turn based our own codebook on these themes. the hybrid, inductive-deductive approach to thematic analysis outlined by fereday and muir-cochrane (2006) has served us well over multiple cohorts since 2016. once we built our dataset from the content submitted by participants, we carried out our own substantive coding using nvivo11, both to test the fittingness of the participant analysis-based codebook, and to augment it with codes arising from our own analysis. this iterative, back-and-forth process of harmonizing themes ultimately brought us to the findings detailed below. our project output likewise bifurcates into participant-authored digital stories posted to our research site [omitted for blind peer review] and the scholarly articles and conference presentations authored by our research team. in our view, the simultaneous dissemination of participants' narratives with our own findings also provides a ready means for readers to validate the latter, by corroborating and legitimizing them against the former. while the covid-19 outbreak shows all the hallmarks of a historic, global inflection point, our participants' early reactions have been measured and thoughtful-very much in keeping with the tone and tenor of their pre-outbreak photographs and remarks. they have focused on the continuity of their pre-and post-outbreak lives, and the lives of the j o u r n a l p r e -p r o o f host communities with which they have come to identify. mental discipline, sharpened through clinical and community engagement, has helped them cope with the sudden change in personal circumstances, and it has kept their attention on the wellbeing of others more vulnerable to the pandemic and its psychosocial sequelae. george, a nursing student placed 240 km northeast of the nearest metropolitan center, took a picture of an iv bag ( figure 1 ) during his first few weeks onsite. the bag, he explained, contained platelets needed by a patient with a rare clotting disorder, and it had taken three and a half days for three units to arrive from the city. "it felt like handling gold, because they were so valuable," george recalled. "it's so hard to get, out in [a rural] community… it really impressed upon me how easy it is to take for granted the resources that are available to you, when you're working in a large center." the iv picture resurfaced during our covid-19 discussion, when supply chains had suddenly taken on new significance. "i don't think people are lining up to donate blood at the clinic right now," said george. "it takes six units of donated blood to create one unit of platelets. and this man had to receive three." beyond the clinical setting, distance, remoteness and infrastructure were already prominent social determinants of health, well before pandemics entered the current public discourse. bradley, a medical student placed in northern alberta, 450 km from the nearest city, photographed an ancient barge crossing a river (figure 2 aside from seclusion, other rural, social determinants of health were seen to carry over from pre-to post-pandemic, where they compounded its effects. from the outsets of their placements, students observed that influenza-like illness and other respiratory symptoms were common amongst rural clients. "a lot of people are farmers, and there is all the stuff that's floating around in the air," said murray, a medical student. "obviously lung issues, or copd, or asbestosis, or other stuff, are really well-seen and probably over-represented here, in hospitals and rural communities. i don't see that changing any time soon." both leia (nursing) and lyle (medicine) photographed emissions from the local pulp and sawmill in a neighbouring town ( figure 3) ; "it seems like almost every patient some j o u r n a l p r e -p r o o f days, in some way, is connected to one of the lumber mills in the area here, and it's a huge part of the community," said lyle. even before covid-19, the perceived volume of patients presenting in the local emergency room, with non-urgent respiratory complaints, concerned at least one participant. "i found that the emergency department was being taken advantage of as a walk-in clinic, and had many people coming in with cough and cold-like symptoms, or for prescription refills," recalled sophia, a nursing student. "this led to an increase in wait time, and some people even would leave the emergency department waiting room, or become upset with us, even though we were taking care of more acute or ill patients." as news of the pandemic first hit, these behavioural patterns intensified. "we saw a large influx of patients presenting with influenza-like illness, and i believe this was because of misinformation in the media, and the need for reassurance from a health professional," said sophia. george (nursing) observed the same panic in his placement site: "a lot of the people who came in with [influenza-like illness had] almost anxiety-induced symptoms. and so it kind of started to overload the healthcare team with cases that weren't really cases. just because there was so much misinformation, at the beginning especially." other longstanding rural habits proved hard to break in the age of covid-19. social distancing runs contrary to small-town community ethos, where everyone knows everyone else. after a brief stop in the city, sophia returned to her hometown (population 10,000) only to find herself struggling to adjust. "in the city, i was aware of how people will circle around each other to maintain that distance, but that almost didn't apply to people living rurally. the warnings are there for a reason; i just didn't realize how j o u r n a l p r e -p r o o f difficult it would be to follow them." at last, she resorted to hiking in the mountains ( figure 4 ) as a way of keeping her distance and her mental equilibrium. while the reluctance to distance may be a rural public health hazard for the time being, our participants have been unanimous in pointing to community solidarity as the greatest asset in their rural placements. pictorially, they connected this solidarity with agencies and services familiar to most canadians, but having an outsized role in rural community leadership. leia (nursing) photographed the loading dock of the local food bank ( figure 5 ), commenting, "to me this represents what rural context is all about… it demonstrates how close-knit and interconnected rural communities are, as they all support and help each other." she moreover observed that cashiers at the local grocer frequently invited her to make food bank donations, further evidence of a community ethos unique to the rural setting, and a shared responsibility to take care of those most vulnerable in times of crisis. participants also highlighted the leadership role of rural hospitals and health care practitioners in promoting holistic community wellness, both through healthy, active lifestyle choices (all the participants spent a significant portion of their downtime at the local fitness centers, outdoor recreation facilities and backwoods trails) and exemplifying community values. linda (nursing) photographed a promotion for pink shirt day ( figure 6 ), an anti-bullying campaign spearheaded by the hospital: "i just remember specifically seeing the hospital administration and managers put [this] out, kind of push us to participate and be leaders in our community-to open that door for other people in the j o u r n a l p r e -p r o o f community to also participate." as covid-19 broke out, these communities were predisposed to turn to their health care providers for guidance. for the students, rural preceptorships were as much about constructing identities as acquiring clinical skills. not only did participants become personally invested in the health care teams with whom they worked, but they also developed affective attachments as community insiders. abrupt and unceremonious withdrawal was difficult. "i feel robbed of experiencing that placement to the end," acknowledged sophia (nursing). "i hadn't prepared mentally or emotionally to leave the people i met behind, and when i got back [home] it was almost hard to enjoy being back with friends and family." participants who took part in the final, photovoice teleconference all agreed that the denial of fulfilment-so close at hand-was most frustrating. "that was what we were building up to, and then it's just suddenly being cut off, and there's no closure," leia (nursing) remarked. "i built relationships with [my preceptors]. i liked working with them, and we just expected that we were going to have longer to work together." while the nursing faculty agreed to waive the few remaining shifts for each nursing student, it was small comfort for leia. "i feel like going through-graduating-we didn't do the full thing, right? it doesn't feel completed." for many participants stuck at home, complying with provincial social-distancing orders, this lack of fulfilment has been compounded with the struggle to find purpose. suddenly being out of the loop at a critical juncture, after weeks or months on placement, amounts to a kind of psychosocial limbo. "i don't really know what's going on at the hospital because i haven't been in there for a week now, which feels very weird," said bradley (medicine). the shock of disconnection was heightened by the perception that their placement settings were rapidly transforming into unfamiliar places. linda (nursing) recalled, "after i packed, i went to drop my key at the admission desk. i noticed they were also doing the full [personal protective equipment] at the door, screening everyone who came in." furthermore, the indefinite length of provincial restrictions on postsecondary institutions and clinical sites has left some students with the sense they have regressed to a more dependent, directionless stage of life. "today, my parents are gone to work, and i'm just sitting at home, trying to do some schoolwork; and i don't know what to do with myself, 'cause i'm used to being very busy," said leia (nursing). studying for the postponed provincial nursing licensure exam; term papers; photovoice; reading; baking and housework are a few ways students have sought to occupy their time. cognizant of the strain on themselves, they speculated on the wider implications of quarantine on public wellness. "it makes me wonder what kind of impact this pandemic will have on people's mental health, physical health, financial situations, and general trust in one another," sophia (nursing) remarked. a few students have been fortunate enough to find clinical roles in one form or another, post-outbreak. kristen (medicine) mentioned, "they're trying to get med students doing the contact tracing… so i'm going to try to join in with one of the groups." this work, while minimally employing the skillset of a third year medical student, is nonetheless vitally important in pandemic control, and kristen seemed grateful to have found a way to contribute. other medical and nursing students, mindful of the burden on healthcare providers, were organizing a babysitting service as we were concluding our j o u r n a l p r e -p r o o f data collection. and leia (nursing) had already leveraged her nursing studies into a paid, student position at her small, hometown hospital months earlier (figure 7) . "i'm actually at work right now, today, because they needed me," she put in during our final teleconference. while her duties did not afford the same broad, clinical coverage as the preceptorship, the job was a welcome source of purpose, and a link between her pre-and post-pandemic life. leia's undergraduate nursing role illustrates that alberta's rural hospitals have made do with auxiliary and casual help long before the covid-19 outbreak, and it is one example of many post-outbreak continuities pointed out by the students, pertaining to challenges and coping capacities. technological connectivity and virtual gathering spaces have been used to support rural, clinical work for years, and the onsite assets impressed the students in their sophistication, bradley (medicine) photographed the telehealth station in his placement setting (figure 8) , commenting, "it's hard to [overstate] how much this has revolutionized rural health care in canada… i will keep telehealth in my mind throughout my practice, whether or not i work in a rural community, and remember how important a difference it can make." a few weeks later, he found himself speculating on the same technology as a last resort to continue his involvement in clinical experiences: "there's some hope right now that we might be able to participate in a telehealth perspective, but it's really unsure right now if that's gonna work, or what educational validity or service that will be." linda (nursing) also noted wryly that she used facetime while she keenly missed this interaction upon her return home, she found a creative way to maintain her fitness regimen and the social element attached to it, as she explained with a picture of the frozen-over glenmore reservoir (figure 11 ). "my friend and i are doing a physical distancing stair set. we are on the phone and both start at the bottom together. my stair set in calgary is shorter than her stair set in edmonton… so i have to do squats at the top of my stairs until she gets to the top of the stairs in edmonton. then we chat while we are walking down." as of this writing, in early may, 2020, alberta's rural hospitals appear to have been spared the worst-case scenario-a surge of critical covid-19 cases beyond capacity. the medical students are slowly being reintegrated into the clinical environment. in the coming months and years, the global, psychosocial aftermath of the outbreak on frontline health care workers will no doubt be compared and contrasted with the recent outbreaks of sars, h1n1, and ebola, with many calls for policy reform. we could well expect to see a spike in post-traumatic stress amongst nurses and physicians in the hardest-hit regions (bai et al., 2004; brooks et al., 2020; marjanovic, greenglass & coffey, 2007; maunder et al., 2003; pan et al., 2005; reynolds et al., 2007; wu et al., 2009 ). this is not to suggest that rural communities and hospitals in alberta are insulated from the effects of the pandemic. physician and nurse attrition-already a major concern in the wake of recent cutbacks-may accelerate with the inevitable crash of the provincial economy (bellefontaine, 2020) . as public funds become increasingly scarce, our province may be at risk for a wave of rural hospital closures (kacik, 2020) . and as our data illustrate, rural and remote areas are already among the hardest hit by pandemic-related disruptions to supply lines, which may prove fatal for the most vulnerable community members. our participants' photographs and comments reveal how abrupt disconnection from the healthcare frontline, at a critical moment, may itself be disruptive and traumatic (reynolds et al., 2008) . preceptorship is a time of capacity-building for self-reliance, clinical confidence, and interprofessional identity (author, 2018). shaw (2020) draws attention to the vulnerability of trainees whose sense of self-efficacy hinges on positive clinical experiences and reinforcement by other team members. the covid-19 outbreak has had the effect of undercutting these experiences, pushing students back to watching passively from the sidelines. the longer this situation is drawn out, the more cause we have for concern that months of clinical gains may be compromised. meaningful roles in j o u r n a l p r e -p r o o f public health support-such as contact tracing and casual relief-are already making a significant difference for some of our participants' sense of purpose. it seems reasonable to expand such opportunities for the nursing students who are just short on their qualifying, clinical hours (mitchell, 2020; stokes, 2020) , especially in understaffed rural acute care settings. several students were prescient in drawing attention to onsite, distance-treatment and learning resources such as telehealth, weeks before covid-19 became a reality. in this regard, many rural health care sites have been ahead of the curve, integrating such technologies into daily practice over years of development. as clinical science educators rush to implement online components into their curricula, it is telling that much of the relevant research comes from australia and new zealand-global leaders in educational and clinical connectivity with rural and remote locations (jowsey et al., 2020) . however, the limits of this technology are such that the completion of clinical objectives for our participants, in the current circumstances, appears implausible. for our participants, the pandemic quarantine has resulted in a state of prolonged liminality, with no clear resolution in sight. evans and kevern (2015) note that degree programs in the health sciences-and especially clinical placements-are liminal spaces wherein students transit between one identity and another. "socialisation into a professional role is itself a rite of passage and includes a period of personal challenge, uncertainty and adjustment," the authors observe, moreover raising concerns regarding "the personal cost of this process for the student, and also the risk that they might avoid the challenge by dropping out; by retreating into the communitas of the student group; or by 'going through the motions' without properly internalizing the role they are required j o u r n a l p r e -p r o o f to adopt" (p. 5). our data suggest that extending this fragile state of crisis indefinitely, without the closure of a summative experience or a ceremonial farewell (such as celebrating with the staff) leaves students feeling unfulfilled, helpless (shaw, 2020) and less certain of their status, at a time when they would normally be preparing to re-enter society as newly minted health care professionals. communitas is the fellowship that arises among co-travelers in the transitional, liminal landscape (barton, 2007; evans & kevern, 2015) . notwithstanding evans and kevern's (2015) caution that undergraduate communitas may distract from the ultimate goal of professional caregiver status, the concept has become crucial as our participants-indeed, all persons confronting uncertain futures as a result of covid-19-seek to cope with their circumstances. seclusion, confinement and suspense, intrinsic to quarantine life (cava, 2005; brooks, 2020) , were daily realities confronted by the students during their rural placements, and they coped through forms of communitas, such as facetime with peers and subzero runs with fellow interprofessional team members. even solitary outdoor activities, such as backwoods hiking, were connected with a sense of social solidarity, as students felt they were buying into the values of their host communities (hansen-ketchum et al., 2011) . simply put, rural placements improved the students' capacities for technological and metaphysical connectivity, which continue to serve them as they navigate the suspenseful, unfamiliar territory of a global pandemic. amongst allied health caregivers, the classes of 2020 and subsequent years will build practices shaped by their covid-19 experiences, for better or worse. lingering anxiety over status, and the sense of a clinical education cut short, may weigh on their interprofessional relationships and clinical judgment. alternatively, their practices may benefit from the reflexive abilities to adapt, to empathize, to capitalize on circumstance, and to retain a holistic perspective in times of crisis. the data from our 2020 photovoice cohort suggest that rural placements-even left unfinished-equipped nursing and medical students to handle their post-outbreak situations with equanimity and solidarity. as alberta's rural communities and hospitals confront post-outbreak, existential threats from shortages, elimination of services, and precipitous socioeconomic decline, this generation of caregivers should be welcomed and supported as a key asset. j o u r n a l p r e -p r o o f survey of stress reactions among health care workers involved with the sars outbreak student nurse practitioners -a rite of passage? the universality of van gennep's model of social transition alberta premier jason kenney starts second year in office dealing with 3 crises the psychological impact of quarantine and how to reduce it: rapid review of the evidence the experience of quarantine for individuals affected by sars in toronto liminality in preregistration mental health nurse education: a review of the literature demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development strengthening access to restorative places: findings from a participatory study on engaging with nature in the promotion of health blended learning via distance in pre-registration nursing education: a scoping review rural hospitals face dire financial picture, with many at risk of closing the relevance of psychosocial variables and working conditions in predicting nurses' coping strategies during the sars crisis: an online questionnaire survey the immediate psychological and occupational impact of the 2003 sars outbreak in a teaching hospital former and future nurses step up in coronavirus battle voices from the frontline: nurses' impact and coping during the 2003 sars outbreak in southern taiwan understanding, compliance and psychological impact of the sars quarantine experience hopelessness, helplessness and resilience: the importance of safeguarding our trainees' mental wellbeing during the covid-19 pandemic senior medical students in the covid-19 response: an opportunity to be proactive is quarantine related to immediate negative psychological consequences during the 2009 h1n1 epidemic? the psychological impact of the sars epidemic on hospital employees in china: exposure, risk perception, and altruistic acceptance of risk key: cord-291525-yjhgj438 authors: nic dhonncha, e.; murphy, m. title: learning new ways of teaching and assessment – the impact of covid‐19 on undergraduate dermatology education date: 2020-07-03 journal: clin exp dermatol doi: 10.1111/ced.14364 sha: doc_id: 291525 cord_uid: yjhgj438 covid‐19 has resulted in unprecedented global disruption. as this global pandemic persists with no end in sight, we must not ignore its impact on undergraduate medical education. dermatology undergraduate exposure is variable, limited and often suboptimal. it has been shown that medical students and qualified doctors are not adequately confident in their ability to assess and manage skin conditions, and many feel that their undergraduate dermatology teaching was not satisfactory. in march 2020, irish universities were closed and clinical rotations suspended. this presented us with an immediate need to adapt our teaching methods to limit the negative impact on undergraduate dermatology exposure. we report our experience of undergraduate dermatology teaching during this extraordinary time. we adopted the platform microsoft teams to deliver online teaching to medical students. this collaboration tool facilitated video lectures with active participation from students, real time discussion, and document sharing. teaching sessions included didactic lectures, interactive tutorials, and student-led case and topic presentations. remote assessment is challenging, as all online assessments from home are essentially openbook. we sought to devise a rigorous tool to assess students' dermatology knowledge. we designed an online 30-question multiple-choice question examination using the online learning management platform, canvas, with each question based on a clinical image. we felt that the use of clinical images would offset any potential benefit of additional information resources available to students in the home environment, and a maximum time of 30 minutes was allocated to complete the exam. students were allowed to take the exam at any 30-minute period over a 24-hour window, in order to accommodate time differences for overseas students, who had returned home upon closure of the university. studies have demonstrated that university students have experienced increased symptoms of anxiety and depression due to the covid-19 pandemic, which may in part be attributable to the effect on their studies. 1, 2 in an effort to alleviate anxiety related to their dermatology rotation, we allocated a specialist registrar in dermatology as a point of contact and encouraged students to reach out with any concerns related to their rotation. on reflection, we successfully substituted scheduled in-person teaching and assessments with online alternatives, but didn't implement a suitable alternative to clinical patient interaction. as we slowly return to a new-normal with video-consultations, virtual clinics and a gradual increase in in-person consultations, we must consider how best to incorporate our dermatology students into this new clinical setting. social distancing is likely to dictate our practice, with a significant impact and reduction in clinical rotations likely for the months and potentially years ahead. in addition to our current teaching methods as outlined above, we will invite medical students to participate in our virtual weekly journal club, departmental clinical and multidisciplinary meetings, which are now established on zoom. we plan to upload short videos on canvas demonstrating surgical techniques and procedures. we also plan to develop dermatology podcasts for medical students. in order to protect the invaluable experience of patient encounters whilst facilitating the requisite social distancing, participation of medical students' in video consultations and virtual patient visits via online applications will also be explored. while virtual learning cannot replace hands-on clinical experience and patient exposure, the unique circumstances of covid-19 have promoted innovation in medical education. we must continue to develop alternative learning streams to the best of our ability to ensure adequate preparation of the next generation of doctors in an era where clinical rotations are likely to be limited for the foreseeable future. evaluation of educations methods in dermatology and confidence levels: a national survey of uk medical students dermatology and junior doctors: an evaluation of education, perceptions and self-assessed competencies university students' mental health amidst the covid-19 quarantine in greece the psychological impact of the covid-19 epidemic on college students in china accepted article key: cord-294832-i831swu4 authors: o’leary, noreen; salmon, nancy; clifford, amanda m. title: ‘it benefits patient care’: the value of practice-based ipe in healthcare curriculums date: 2020-11-12 journal: bmc med educ doi: 10.1186/s12909-020-02356-2 sha: doc_id: 294832 cord_uid: i831swu4 background: practice-based interprofessional education (ipe) is essential to prepare students for collaborative working. pockets of practice-based ipe are integrated into healthcare curriculums in some regions. yet practice-based ipe is not globally valued as a key element of healthcare curriculums. as students and clinical educators are key stakeholders, this study presents a case example of their experiences in a country where practice-based ipe is at an emergent stage. their experiential knowledge generated important insights into how practice-based ipe is perceived. this learning can be applied, both locally and further afield, by those seeking to embed practice-based ipe in their placement curriculums. methods: a qualitative case study was conducted at a school of allied health and partner placement sites in ireland. data collection comprised two participant observations, 13 interviews and 12 document analyses. inductive thematic analysis and deductive framework analysis, underpinned by activity theory and hofstede’s cultural dimensions, informed data analysis and interpretations. results: participants are grappling to establish the value of practice-based ipe, illustrated in three themes: clarifying the concept of practice-based ipe, mapping ipe activities and diversifying interprofessionalism. first, ambiguous conceptualisation of why and how to implement practice-based ipe was identified. highlighting how practice-based ipe improved patient care and safety created a clear rationale for implementation. it was also helpful to demonstrate how adaptations to existing practice education models, rather than entirely new models, could achieve high-quality practice-based ipe. second, the positioning of practice-base ipe in the placement curriculum was unclear. overt mapping of practice-based ipe activities onto learning outcomes within assessment tools enhanced its value within practice education. third, varying levels of professional engagement were noted, perpetuating stereotypes. creating diverse educator networks and embedding practice-based ipe in organisational strategy may incentivise engagement across a greater range of professions. conclusions: implementing these recommendations could enhance the value of practice-based ipe and optimise student preparation for collaborative working. practice-based ipe remains a complex model and the trajectory of embedding in healthcare curriculums will differ globally. supplementary information: the online version contains supplementary material available at 10.1186/s12909-020-02356-2. interprofessional collaboration (ipc) is necessary for optimal patient care and outcomes [1] . therefore, students require appropriate preparation to enter the workforce as collaborative-ready, patient centred practitioners. there are many ways of preparing students for ipc, subsumed by the umbrella term interprofessional education (ipe). ipe can be broadly categorised as classroombased, simulated and practice-based. practice-based ipe requires students from two or more professions working and learning together at the same placement site [2] . location at clinical sites provides unique learning opportunities [3] as students apply theory to practice [4] , experience ipc first-hand [5] and commence socialisation into clinical teams [6, 7] . indeed, healthcare professionals whose training included ipe cite practice-based ipe as the most meaningful ipe input in terms of clinical practice [8, 9] . however, understanding of student and clinical educator experiences as practice-based ipe becomes embedded in a curriculum is relatively limited. therefore, it is critical to explore this process in depth, to advance integration of practice-based ipe and optimise student preparation for ipc. there are challenges specific to integrating practicebase ipe that differ from those relating to classroom ipe. beyond the well documented logistical complexities [10] , practice-based ipe involves tackling sensitive issues such as professional stereotypes and role boundaries in often demanding clinical settings [11] where patient safety and wellbeing are the primary focus [7] . educators at clinical sites are primarily practicing clinicians [12] and can sometimes lack educator specific training even uniprofesionally [13, 14] . ipe facilitation is perceived as a complex role for educators [15] and targeted training is rare [3, 16] . consequently, clinical educators may be reluctant to become involved in practice-based ipe. additionally, all practice education must ensure students achieve competencies required by their professional regulatory body [17] . as such, practice-based ipe is a complex practice education model. furthermore, practice-based ipe occurs at the interface of education and frontline health services, both of which are influenced by the social and cultural context [18] . therefore, experiences of embedding practicebased ipe likely differ internationally. for instance, interprofessional training wards at acute hospitals are well established in scandinavian countries [19] , while rural and remote healthcare activities are often reported in australia [20] . geographical [21] and specific healthcare needs and resources [22] likely influenced the approach taken in these regions. globally, long-term funding for practice-based ipe is an on-going challenge [23] and many practice-based ipe projects do not extend beyond pilot or short-term initiatives [24] . this has stimulated growing interest in relatively low resource activities such as case-based tutorials [25] [26] [27] . currently, practicebased ipe is not cohesively integrated into healthcare curriculums globally [28] . theory provides a crucial anchor when seeking a nuanced understanding of how students and clinical educators experience this complex model [29] . activity theory is suitable for unpicking the interacting factors influencing practice-based ipe, as it focuses on how people engage within rule-governed systems and use tools to achieve objectives in real-life circumstances [30] . during practice-based ipe, distinct students and clinical educator activity systems temporarily coalesce [31] . within and across these activity systems tensions can arise, for example between differing objectives [32] (further detail can be found in additional file 1). given the seismic changes occurring in health and education spheres globally due to the covid-19 pandemic [33] , it is perhaps more crucial than ever to analyse how national sociopolitical contexts intersect with implementing changes to healthcare education models such as practice education [34] . hofstede's cultural dimensions theory [35] offers one interpretation for how national culture can influence values and behaviours [34] . hofstede posits that as people are exposed to national cultures from birth, these traits are more ingrained than workplace culture, which is more transient and acquired later in development [36] . cultural trends considered by hofstede include attitudes to democracy, individualism or collectivism, tradition and achievement as well as long and short term planning and enjoyment of life [35] (further information can be found in additional file 2). regarding practice-based ipe this theory can contribute to understanding how and why ipe has evolved differently across countries. the aim of this research was to develop an in-depth treatise of student and clinical educator experiences while seeking to embed practice-based ipe in the curriculum. to this end the following objectives were developed: to document the practice-based ipe experiences of students and clinical educators affiliated with one university. to explore the context in which these activities developed. to develop recommendations supporting sustainability and growth of practice-based ipe activities with applicability beyond the research site. as such this paper will contribute to the discussion on how to embed practice-based ipe as a valued aspect of health professions education, providing signposts for stakeholders including clinical educators and accrediting bodies. this qualitative case study facilitated in-depth exploration of practice-based ipe within the parameters of a specific case [37] , consisting of practicum sites connected to an irish university. five allied health professional qualification programmes are offered by the university. students attend diverse placements including hospital, community care and rehabilitation sites. this research forms one phase of a larger doctoral study at the same site. a previous study has explored the experiences of university-based educators involved in developing and coordinating practice-based ipe (under review). ethical approval was provided by the university and placement site research ethics committees. the standards for reporting qualitative research were used to report key features of the research process [38] (additional file 3). data collection occurred from november 2019 to april 2020. however, the foundations for this phase, including familiarity with placement structures and access to potential gatekeepers, were in place from previous research at the site, which began in 2017. methodological triangulation was used to enhance data collection validity [39] and credibility of findings [40] . participant observations were conducted to allow the researcher to develop a first-hand and socially contextualised understanding of practice-based ipe [41] . using a specifically designed template (additional file 4), the first author observed interprofessional tutorials (n = 2) over 5 h. participants included seven clinical educators and 17 students. five professions were represented -nursing, occupational therapy, physiotherapy, radiography and speech and language therapy. semi-structured interviews (n = 13) were carried out by the first author to facilitate exploration of individual experiences and perspectives [42] . interview length ranged from 26 to 42 min, with a median length of 33 min. participants were clinical educators (n = 4), current students (n = 7) and recent graduates (n = 2). four professions were represented -occupational therapy, physiotherapy, dietetics, and speech & language therapy. interview guides were informed by observations, literature, and theory (additional file 4). relevant documents (n = 12) were analysed in conjunction with observations and interviews [40] to facilitate comparison of stated policy and guidelines with participant experiences [43] and to generate further lines of inquiry. documents included profession-specific competency forms and interprofessional education resources. observation, interview and documentary data were imported into nvivo12 software to support data management [44] . thematic and framework analyses were used to interpret data as per fig. 1 . analytical pluralism was adopted to achieve more nuanced data interpretations than would be achieved through use of either approach singularly [45] and to limit interpretive bias [46] . thematic analysis was used to inductively code and interpret participant data and develop initial themes [47] . a deductive framework analysis was then used to analyse participant data using a priori codes [48] from activity theory [49] and hofstede's cultural dimensions [35] . initially, the first and second author individually coded a subset of three transcripts. this enhanced the comprehensiveness of the initial inductive coding framework and refined the application of the theory-based deductive framework. sample data analysis can be found in additional file 5. the approaches chosen were philosophically compatible [50] , as neither is aligned to a specific epistemological perspective and both focus on generating themes [48, 51] . reflexive memoing [52] along with ongoing author and advisory panel discussions enabled exploration and resolution of divergent interpretations [48] . we begin this section by framing the context in which participants reported they experienced practice-based ipe. students from each programme typically complete four blocks of placement. between placements, students complete five interprofessional academic modules, designed to establish foundations for collaborative working. for example, shared attendance at sessions on topics such as professional documentation, infection prevention and control and ethics. students subsequently engage in interactive interprofessional modules where they develop interprofessional management plans for hypothetical complex cases. operationally, placement timetables were aligned to maximise opportunities for practice-based ipe. students engage in practice-based ipe at any stage of their programme. the experiences included in this study captured all placement stages, from initial to final placements. the content and format of practicebased ipe is decided locally by clinical educators, guided by clinical needs and facilitation resources available at the placement site. placement handbooks outline opportunities for practice-based ipe and signpost clinical educators to useful resources, such as a practice-based ipe resource pack developed by the university practice education team. those involved in practice-based ipe depend on the professions available at the site at the time and agreement by educators from programmes to enable students to become involved. participant data reflected a situation of fledgling practice-based ipe at an early stage of integration into the practice education curriculum. participants are grappling with cultivating the value of practice-based ipe, represented in three key themes ( fig. 2 clarifying the concept of practice-based ipe mapping practice-based ipe activity diversifying interprofessionalism in activity theory terms, these themes reflect sources of tension within the systems of practice education as participants sought to embed practice-based ipe. conceptualising practice-based ipe. at a conceptual level, participants reported equivocation regarding two key issues, the rationale for practicebased ipe and the process of establishing it. both students and educators experienced uncertainty about why practice-based ipe was needed: i was baffled as to why it's required, or who these people are. [student 7] some of the nurses didn't even know what ipe was. [clinical educator 1] for example, educators reported that many colleagues perceived practice-based ipe as a purely educational activity and did not link it to improved clinical practice and patient outcomes: the one thing that made [the nurses] open their eyes a little bit was when we said, "no actually there's evidence, they say it benefits patient care and patient outcomes"… it wasn't, 'all students think it's great' … this is what the benefit is. in a similar vein, when students experienced practicebased ipe it deepened their understanding of its contribution to patient care: i think now i have an understanding of how important interprofessional working is, i could advocate for that a bit more, having seen it. [student 3] activity theory highlights that activity is objective driven. activity that aligns with core objectives of healthcare is likely to be perceived as valid. as in this example, spotlighting the impact of practice-based ipe on improved patient safety and care added validity. students and educators expressed concern that practice-based ipe was resource intensive, creating additional work for clinical educators and reducing student time for uniprofessional activity. i think it would be a mistake to make it [ipe] a big job because i think it would turn people off and it feels forced then, when it should just be kind of a case discussion. [clinical educator 1] indeed, feedback from graduates and educators who experienced practice-based ipe illustrated that small-scale activities, building on existing clinical activity provided impactful learning opportunities. as a case example, during an acute hospital placement two graduates each worked with a student from another profession, to jointly assess a patient, develop an interprofessional management plan and present their findings to their clinical educators: what we did for our project, it wasn't overly complicated. it had nice structure to it, but it wasn't complicated. [ from the activity theory perspective, the template provided a tool for students to divide labour in pursuit of the shared objective of patient care while also prompting critical reflection. small-scale activities such as this can lead to meaningful outcomes, in terms of student's interprofessional interactions. for example, clinical educators and students experienced increased interprofessional communication following a two-hour interprofessional tutorial: i actually had a number of students approaching me … and say, "can i ask you a question about this patient?... i don't think she'd have approached me without having done those sessions. i haven't come across that before. [clinical educator 2] students themselves reflected that it was working together during tutorials which facilitated future communication and interactions i was less cautious about approaching the other professions, so i really noticed that actually after the ipe tutorial … it kind of broke down the barrier. in this theme we explore how practice-based ipe is currently mapped onto placement curriculums. overall students and educators asserted that while ipe was important, their priorities, and thus activity, during practice education is guided by the competency forms on which students are graded. on placement, you're being marked, you're being graded and it's worth a lot to your degree. [ student 2] there was a prevailing sense of ambiguity about the place of practice-based ipe in the placement curriculum and assessment: i think it is kind of an unwritten rule that on your placements you will do sessions with other professionals [student 8] each profession is assessed using a different competency tool. most competencies refer to uniprofessional activities with some lending themselves to practice-based ipe. however, the wording of the latter competencies allows considerable interpretative latitude, for example: contributes effectively as a team member; build collaborative working relationships [53] this was confirmed by student recollections of variable practice-based ipe experiences regarding type and level of interprofessional collaboration: my first placement wasn't a multidisciplinary setting … [so] you were graded on your communication with everyone else … even with the receptionist and everybody else in general, they looked at that as a whole. [student 4] the most common practice-based ipe opportunities were acute placement sites, supported by co-location of professions and patient needs. this took the form of interprofessional tutorials, case presentations and joint assessments. many students identified missed opportunities for practice-based ipe: on my last placement, there were other students there … i think there was one day a week we were in the same building … even if there was a half an hour a week just set aside for group talk or something like that... talk over or plan something. [student 8] across the board there were variable interpretations as to how interprofessional activity informed student assessment. for example, the following two students reported contrasting experiences of the same practicebased ipe activity and its link to their assessment: the practice educator said before the sessions you're not being assessed on this. [student 5] the educator was observing [the tutorial] and she even drew back to that when we were completing the form then that she'd seen me recognize the role of the other professionals. [ student 3] to begin addressing these inconsistencies clinical educators reflected that making explicit links between practice-based ipe and professional competency assessment strengthened alignment between the activity and assessment and created a clear rationale for the activity, thus enhancing its value: we're very clear and we can tell them beforehand, these are the competencies, that it's going to help you to progress in … there's a good reason why we're asking you to do this. [clinical educator 3] participants acknowledged a lack of guidance from higher level bodies such as the professional regulator regarding practice-based ipe contributed to ambiguity: indeed, from the lens of activity theory, articulating practice-based ipe expectations more explicitly within the regulatory and competency tools mediating placement activity could support integration of practice-based ipe. moreover, clearly mapping interprofessional activities onto competencies increases clarity about the function of the activity and the intended results. maximising clarity is a useful approach when introducing any new practice. this strategy is especially beneficial in countries where uncertainty avoidance is culturally important. this in turn may increase the perceived value of practice-based ipe. we found that involving the full range of professions in practice-based ipe is challenging. both students and educators noted variable professional involvement: we don't have any collaboration with medics. thus, involvement across professions relies on individual educators rather than being an integrated expectation across practice education: a medic involved in the medical school here he was really keen on it but then he left. [clinical educator 3] the absence of certain professions may leave professional stereotypes unchallenged. for example, medical students or educators were not involved in interprofessional tutorials observed for this research. during a group activity to develop a patient care plan one participant commented: then the medic comes in and says discharge. [interprofessional tutorial observation 1] the implied meaning was medics override other professions and the group response of laughter, and head nodding indicated agreement with this perspective. in their absence, the 'us/them' stereotype regarding one profession was perpetuated between other professions. furthermore, student reflections highlighted that it was collective participation in practice-based ipe activities that established communication bridges with students from other professions: i never asked a question to one of the medical [students]. i don't know if them being at the interprofessional sessions would have made them seem like real life people ...they were in the same room at lunch, they're in the same building, but i never talked to them. [student 6] without a guided opportunity to initially engage with other professions, shared presence in clinical and social spaces did not translate to interprofessional communication and working. educators noted there can be a hesitancy to become involved if ipe is perceived to be the property of specific professions or people: if it's all coming from me then people are always going to be a bit suspicious … why are they doing this now and what's the agenda here? [clinical educator 2] in terms of activity theory, there appeared to be poorly developed communities to support practice-based ipe. while practice education staff at the university are a clearly defined unit, this differs at clinical sites. educators work within their own professions, links with educators in other professions are developed ad hoc by individuals: i met with x and she was very keen, like myself, so we decided we'd do it [ipe] and we did. [clinical educator 3] participants felt that innovations such as practice-based ipe would be perceived as having greater value if initiated and supported by management within the healthcare organisation: we're just two tutors. whereas, if someone said, "oh actually, we're the new managers in student education in the hospital" … then everyone is like, "this is someone who maybe can get us things or get stuff done for us" … i think if you're sending an email from a person like that, at least there's a bit of buy in. [clinical educator 1] viewing this through hofstede's cultural dimensions of individualism and achievement orientation, if educators can see the benefit of involvement to their profession within their organisation they may be more positively predisposed to involvement. as such integrating practice-based ipe as an organisational priority may be advantageous in promoting practice-based ipe as valued activity across professions. based on the findings reported above, fig. 3 provides an overview of how practice-based ipe can attain greater value at clinical sites and thus become more embedded in practice. practice-based ipe offers a powerful opportunity to prepare students for future collaborative practice [8] . nevertheless, development of practice-based ipe lags behind classroom and simulation ipe [54] and requires a firmer footing in healthcare curriculums [55] . this study identified conceptualisation of practice-based ipe, mapping of ipe activities and interprofessional diversity as key features of embedding practice-based ipe. drawing on these findings we make recommendations to enhance the value of practice-based ipe in clinical settings. the goal is not to develop a universal practice-based ipe model. this is neither practicable nor desirable given the inherent variability across placement sites [56] . rather, learning from experiences thus far can inform future practice-based ipe initiatives and clarify the hallmarks of embedded practice-based ipe in healthcare curriculums. in this study practice-based ipe primarily occurred at acute sites, mirroring international trends [28] . physical co-location of students at these sites, in conjunction with the diverse clinical needs among patients likely support practice-based ipe. however, development of practice-based ipe at the level of existing primary healthcare sites could materially extend capacity and scope of practice-based ipe [57] . moreover, considering international aims to optimise community-based care [58] , it is timely to develop opportunities for community practice-based ipe. while co-location with other students can be challenging at community sites, activities such as interprofessional case discussions could be conducted with students at other locations using secure communication platforms. during the covid-19 pandemic, healthcare educators developed greater facility with online platforms [59] and positive outcomes in terms of ipe are emerging [60] . clarifying why and how to implement practice-base ipe enhances its perceived value. maintaining patient care and safety is a key activity objective for clinical educators and students. therefore, activity that is framed as supporting this objective is likely to be perceived as having greater value. to this end practice-based ipe may benefit from a greater focus on its role in improving patient safety and quality of care [61] , in addition to the educational benefits as this is an objective with high value for healthcare staff and students. a common misconception reported was that practice-based ipe requires novel, time-intensive activities. however, our findings indicate that brief activities such as interprofessional tutorials or a joint assessment session with appropriate resources to support interprofessional learning can have a meaningful impact. this aligns with recommendations that practice-based ipe is more sustainable if it can be achieved efficiently without requiring substantial resource allocation [62] . applying the cultural lens, initially adopting small-scale projects may address hesitancy round moving away from traditional models (uncertainty avoidance) [35] . participants in this study noted that 'passive observation' of other students would have been less impactful than activity guided by the interprofessional observation template, which focused their attention. this echoes previous graduate feedback that effective interprofessional learning during placement needed structure and focus [8] . consequently, two key features are extrapolated for educators seeking to develop sustainable practice-base ipe. first, liaise with other educators to consider what reasonable adaptions could be made to support authentic ipe opportunities during student's placement day. culturally, this can allay concerns regarding relinquishing established placement practices (uncertainty avoidance), whilst ensuring activity is meaningful for both students and patients. second, utilise appropriate tools to guide and capture learning from these activities. while templates from a range of countries are available [63] , developing or adapting tools in conjunction with clinical educator colleagues and the placing university can ensure alignment with locally available opportunities and assessment tools. this can evidence the learning gained from practice-based ipe in real time. pedagogically, guided activity and reflection creates a robust learning experience and may be particularly beneficial in cultures where there is a preference for achieving outcomes relatively quickly (short-term orientation) [35] . currently the link between practice-based ipe and learning outcomes is tenuous, as competencies relating to practice-based ipe are broadly framed. placement providers do require flexibility to deliver practice education in line with specific programme requirements and local capacity. however, ambiguous phrasing of expectations can lead to a policy-practice chasm between what is perceived to be occurring based on formal documents and what is actually happening in practice [64] . the danger with this situation is that complacency may set in, with the rhetoric of practice-based ipe in the absence of meaningful integration into curriculums. based on current research it is recommended that dedicated practice-based ipe competencies and guidelines are developed. while this would require collaborative working and national level agreement by regulators, higher educational institutions, and placement providers, it would represent significant progress in embedding practicebased ipe in healthcare curriculums. embedding detailed expectations in documents with regulatory approval may help educators justify this activity [65] . most ipe educator research to date has focused on university faculty [66, 67] . however, clinical educators represent a more diverse group [68] . they continue to hold core clinical roles and are not centrally organised as an educational team. developing clinical teaching teams introduces the idea that educators across professions could contribute to student education [69] , promoting educator networks at clinical sites. this may help address the issue of reliance on individuals or small groups of champions for practice-based ipe, creating a community of educators who can share the division of labour. involvement of organisational leadership in developing these networks could provide essential support for increasing visibility and status of practice-based ipe. in practical terms offering interprofessional facilitation training would both support educators to work with students from other professions [3] , while also evidencing organisational investment and value in practice-based ipe. furthermore, placement sites often host students from different institutions, which may facilitate opportunities for inter-institutional practice-based ipe [70] . while this would require agreement at a national level between host universities and placement providers, it may broaden opportunities for practice-based ipe and diversity of professional involvement [70] . initially, convening an inter-institutional, interprofessional steering group is recommended, with student, university, regulatory and placement-provider representatives. this group could develop governance guidelines and support an initial action plan for trialling this type of practice-based ipe [71] . beyond the level of individuals and local placement sites, national cultural preferences can shed light on how practice-based ipe may be perceived and valued [34] . this in turn may help tailor the approach to integrating practice-based ipe on a country-by-country basis. in this research adapting existing practice education activities was preferred over introducing wholly new models for practice-based ipe. hofstede reported that irish culture tends to prefer normative and traditional ways of operating. similarly, bonello and morris [72] considered the introduction of ipe to maltese healthcare curriculums through the lens of hofstede's cultural dimensions. they found that participant data reflected the national preference for uncertainty avoidance, which was useful to account for when implementing ipe. while data from individuals or groups cannot be assumed to represent overall culture [73] and cultural tendencies should not be perceived as predictive [74] , they can draw attention to less visible factors impacting the integration of models such as practice-based ipe across countries [75] . limitations in the breadth of data from which recommendations were generated warrants consideration. educators were from one clinical site and student experiences of practice-based ipe was primarily at this site. there did not appear to be factors significantly differentiating this site from typical healthcare placement sites. however, considering the cultural research orientation it cannot be discounted that site specific or local factors were influential. the context of the study allowed for immersion in staff and students experiences and detailed analytical consideration of embedding practicebased ipe, which is appropriate for a case study. two other sites were to be included but this was not feasible due to covid-19 pandemic. subsequent studies could build on this research to include other acute and community sites, to develop a comprehensive profile of practice-based ipe and understand core features required for establishing culturally relevant practice-base ipe. at the time of writing the ongoing covid-19 global crisis has highlighted the need for a flexible and collaborative workforce [76] , however, it does not automatically resolve pre-existing challenges and may perpetuate some issues [77] . regarding practice-based ipe, there may a risk of reverting to uniprofessional silos to achieve perceived core uniprofessional competencies. future planning for practice-based ipe may require even closer collaboration with placement providers. practice-based ipe offers authentic opportunities to develop collaborative working skills [5] . this paper draws on student and clinical educator experiences to offer recommendations for enhancing the value and sustainability of practice-based ipe. clarifying the concept of practice-based ipe, clearly mapping activities on measurable competencies, and developing diverse educator networks would support embedding of this model and add to its value would support embedding of this model and add to its value. prevailing local and national cultures should be considered when developing implementation strategies [72] . crucially, impactful practice-based ipe does not necessitate overhauling practice education. rather, thoughtful and explicit adaptations to existing practices can lead to meaningful outcomes for students and sustainable models of practice-based ipe. working on working together. a systematic review on how healthcare professionals contribute to interprofessional collaboration teaching teamwork: an evaluation of an interprofessional training ward placement for health care students interprofessional education in the clinical setting: a qualitative look at the preceptor's perspective in training advanced practice nursing students key elements for interprofessional education interprofessional education and teamworking: a view from the education providers the hospital clinical preceptor: essential preparation for success communities of clinical practice: the social organization of clinical learning recommendations from recent graduates in medicine, nursing and pharmacy on improving interprofessional education in university programs: a qualitative study physiotherapy new graduate self-efficacy and readiness for interprofessional collaboration: a mixed methods study interprofessional education in primary health care for entry level students: a systematic literature review intra-and interprofessional practices through fresh eyes: a qualitative analysis of medical students' early workplace experiences attributes of effective interprofessional placement facilitation a clinical education and practice placements in the allied health professions: an international perspective better prepared, better placement: an online resource for health students clinical supervisors' experiences of using an interprofessional clinical supervision model in an acute care setting bridging the training-practice gap in interprofessional student supervision defining and understanding the relationship between professional identity and interprofessional responsibility: implications for educating health and social care students can dimensions of national culture predict cross-national differences in medical communication? a review of interprofessional training wards: enhancing student learning and patient outcomes mapping the interprofessional education landscape for students on rural clinical placements: an integrative literature review students' experiences and perceptions of interprofessional education during rural placement: a mixed methods study interprofessional education and practice guide: developing interprofessional communitybased clinical experiences pre-registration interprofessional clinical education in the workplace: a realist review bumping along': a qualitative metasynthesis of challenges to interprofessional placements schultz j-h exploring early interprofessional socialization: a pilot study of student's experiences in medical history taking short duration clinically-based interprofessional shadowing and patient review activities may have a role in preparing health professional students to practice collaboratively: a systematic literature review sustainable interprofessional learning on clinical placements: the value of observing others at work interprofessional education placements in allied health: a scoping review studying complexity in health services research: desperately seeking an overdue paradigm shift tensions and opportunities: an activity theory perspective on date and storage label design through a literature review and co-creation sessions improving management of student clinical placements: insights from activity theory implementing interprofessional learning curriculum: how problems might also be answers how covid-19 opened up questions of sociomateriality in healthcare education the role of national culture in shaping health workforce collaboration: lessons learned from a case study on attitudes to interprofessional education in malta cultures and organizations: software ofthe mind: intercultural cooperation and its importance for survival dimensionalizing cultures: the hofstede model in context case study research: design and methods standards for reporting qualitative research: a synthesis of recommendations how to conduct a mini-ethnographic case study: a guide for novice researchers doing excellent social research with documents. london: routledge problematising short-term participant observation and multimethod ethnographic studies ethnographic methods for process evaluations of complex health behaviour interventions evidence based guidelines or collectively constructed "mindlines?" ethnographic study of knowledge management in primary care transparency and coherence in a doctoral study case analysis: reflecting on the use of nvivo within a 'framework' approach qualitative research and anomalous experience: a call for interpretative pluralism a reflexive exploration of two qualitative data coding techniques reflecting on reflexive thematic analysis using the framework method for the analysis of qualitative data in multi-disciplinary health research activity theory as a framework for analyzing and redesigning work analytical pluralism in qualitative research: a meta-study using thematic analysis in psychology the eye regards itself: benefits and challenges of reflexivity in qualitative social work research competency based fieldwork evaluation for occupational therapists. nelson education limited: canada interprofessional learning in practice sustaining interprofessional education in professional award programmes interprofessional education: implications and development for medical education community health placements for junior medical and nursing students for interprofessional learning gearing up' to improve interprofessional collaboration in primary care: a systematic review and conceptual framework virtual bedside teaching rounds on patients with covid-19 virtual workshops to preserve interprofessional collaboration when physical distancing models of interprofessional education for healthcare students: a scoping review a practice model of interprofessional education developing sustainable and embedded interprofessional education: threshold learning outcomes as a potential pathway making it real: the institutionalization of collaboration through formal structure new approaches to interprofessional education and collaborative practice: lessons from the organizational change literature facilitators' perceptions of delivering interprofessional education: a qualitative study building and expanding interprofessional teaching teams a survey of the practice and experience of clinical educators in uk secondary care teaching in the clinical workplace: looking beyond the power of 'the one forming inter-institutional partnerships to offer pre-registration ipe: a focus group study cultivating interprofessional practice in new zealand: an inter-sectorial approach to developing interprofessional education institutionalizing interprofessional education in small states: perspectives from faculty and key stakeholders in malta implementing the reimbursement scheme: views of health and social care staff in six high performing sites claiming too much, delivering too little: testing some of hofstede's generalisations cultural dimensions in the transition of overseas medical graduates to the uk workplace caring for children and adolescents with eating disorders in the current coronavirus 19 pandemic: a singapore perspective what we learn in time of pestilence publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations the authors would like to thank marie o'donnell and mairéad cahill at school of allied health, university of limerick for their review of preliminary findings.authors' contributions nol, ns and amc developed the research concept and designed the research study. nol coordinated the research, collected, analysed and synthesised the data. ns analysed a sample of data. nol led the drafting of the paper. ns and amc contributed to writing each draft of the paper. nol, ns and amc contributed to, reviewed and approved the final draft of the paper. the online version contains supplementary material available at https://doi. org/10.1186/s12909-020-02356-2.additional file 1.additional file 2.additional file 3.additional file 4. abbreviations ipe: interprofessional education; ipc: interprofessional collaboration the authors declare that they have no competing interests.received: 15 july 2020 accepted: 4 november 2020 key: cord-318102-jrd09hia authors: mccleary, daniel f.; clark, frankie j.; dawes, jillian; flowers, jaime m.; ellis-hervey, nina title: going digital to address the school psychologist shortage date: 2020-11-02 journal: contemp sch psychol doi: 10.1007/s40688-020-00327-4 sha: doc_id: 318102 cord_uid: jrd09hia given the shortage of school psychologists, especially in rural areas, school psychology programs are beginning to emerge that provide students with the opportunity to attend classes through distance education (de) methods (i.e., virtually). despite many school psychology students and faculty holding negative perceptions of de methods, the use of de is expanding within the field of school psychology. this article provides an overview of existing de programs and existing de research, specific to school psychology, followed by a brief history of a de option at stephen f. austin state university that began in fall 2018. solutions to common issues encountered with de are also presented. the field of school psychology continues to experience a shortage of practitioners and trainers (brock 2015; castillo et al. 2014; fagan 2004; reeves 2016; savage 2016) . dixon (2017) began a series of articles in the communique asking other school psychology practitioners to discuss the role online education could play in addressing these shortages. institutions such as eastern washington university (ewu), the university of calgary, wayne state university (wsu), fort hays state university, and the university of houston-victoria (dixon et al. 2020; drefs et al. 2015) already provide instruction to school psychology candidates through a variety of predominately distance education (de) methods (e.g., online, hybrid/blended). the purpose of this article is to provide a brief history of de programs, a summary of published de research findings within the field of school psychology, information about a de option provided by stephen f. austin state university (sfa), and recommendations for common issues associated with de. although this paper was originally written for programs contemplating incorporating de into their program, searching for ways to address the school psychology shortage, or address low enrollment numbers the information is also useful for programs forced into providing de options due to terminology de and its different modalities (online, hybrid/blended) come with a slew of terms that may be unfamiliar or, at times, difficult to discern differences between. for an orientation to concepts and common terms related to de, readers are encouraged to familiarize themselves with the concise work of fischer et al. (2018) , moy et al. (2017) , and moy et al. (2018) . in this paper, the term "face-to-face in-person" is used to refer to instruction physically occurring in the same room. the term "face-to-face virtually" refers to instruction wherein the instructor and some students physically meet in the classroom while other students attend the same classroom through virtual means (e.g., zoom). the first known school psychology program to offer de was ewu, which began offering a certificate in 2000 before transitioning to an online ed.s. program in 2015 (ruby et al. 2017) . although an online program, ewu requires students to attend functions on campus to interact with faculty and other students (for a full description, see waldron-soler et al. 2019 ). in 2007, canada's university of calgary began an m.ed. online school psychology program that also required students to be on campus for specific components of the program (for a full description, see drefs et al. 2015) . fort hays state university started a school psychology program in 2012 that only requires students to physically attend a five-day workshop (dixon et al. 2020; fort hays state university n.d.) . since at least 2013, the university of houston-victoria has provided individuals an opportunity to earn a specialist in school psychology (ssp) degree by completing approximately 62% of the coursework in an online format and the other courses either face-to-face in-person or through interactive television (dixon et al. 2020; hendricker and viola 2019) . beginning in 2018, wsu began offering a m.a. in school and community psychology online via synchronous classes that include on-campus students. like other de programs, wsu requires students to attend campus about one evening per semester (dixon et al. 2020; wayne state university n.d.) . hendricker et al. (2017) surveyed school psychology program directors about de perspectives. they reported 32-35% of school psychology programs provide instruction through some form of de (i.e., online, hybrid/blended) despite it not being advertised in program handbooks or on websites. of those programs, 45% provided de in theory courses and 57% reported de occurred in courses contained within the program but taught by non-program faculty. furthermore, the overwhelming majority of programs containing some form of de reported that neither the quality of the courses nor the program outcomes (i.e., praxis scores, graduation rates, employment rates) decreased after switching to de content delivery methods. viola et al. (2019) surveyed school psychology graduate students regarding their perceptions of the use of de within school psychology graduate programs. in general, students expressed more favorable opinions on hybrid/blended courses than online courses. only 40% of the participants perceived predominately online courses to be as beneficial as traditional face-to-face classes, whereas 75% believed hybrid/blended courses to be as beneficial as traditional classes. participants who had taken online and hybrid/blended courses reported disadvantages to include lack of interaction with others, fewer mastered skills, and reduced quality of instruction. nonetheless, participants who had taken online and hybrid/ blended courses in school psychology reported appreciation for the flexibility and convenience remote instruction allows, which provided them the opportunity to reach their educational goals. more recently, fischer et al. (2020) surveyed school psychology program faculty and found approximately 79% of school psychology programs include some form of de instruction. this is a notable increase from the 32-35% de instruction reported by hendricker et al. in 2017. similar to the results found by hendricker et al. (2017 ), fischer et al. (2020 reported de courses were more likely to be theorybased than skill-based (e.g., assessment) and faculty were more favorable of hybrid/blended courses than online courses. it is also worth noting that approximately 34% of school psychology faculty reported having plans for additional online course instruction within the next four years. despite expressed concerns from program directors, faculty, and students, published data from the university of calgary, university of houston-victoria, and ewu indicate students and alumni of de programs are adequately trained and prepared for the workforce (drefs et al. 2015; hendricker and viola 2019; ruby et al. 2017; schroeder 2019; waldron-soler et al. 2019 ). the school psychology program at stephen f. austin state university (sfa) has both an m.a. and ph.d. program. sfa is a regional university that is located in nacogdoches, tx, near the border of louisiana (i.e., east texas, tx). nacogdoches is in a rural area with approximately 33,000 residents (suburban stats 2020). the city is 132 miles from houston, tx, and 165 miles from dallas, tx. sfa has an overall student population of approximately 13,000. the school psychology program at sfa is unique and needed due to the rural location. there are a total of 10 school psychology programs in the state of texas and the nearest to the sfa campus is approximately 76 miles away (grad schools 2020). the location also has a shortage of clinical and school mental health professionals; however, many of the students admitted to the program express interest in either serving in the local community or neighboring rural communities. this allows the program to create opportunities for the community to expand the reach of mental health professionals, specifically qualified to work in school systems. the m.a. in school psychology program was authorized by the texas higher education coordinating board (thecb) in 2000. it was first approved by the national association of school psychologists (nasp) in 2006. the master's in school psychology program adheres to the state requirements for licensure as a licensed specialist in school psychology (lssp) and the national training standards for certification as a national certified school psychologist (ncsp). the training requires 63 semester hours, which includes an internship. the mission of the program is to ensure students learn to be ethical, responsible, and competent school psychologists. the program approach and teaching are that of the practitionerscientist/scholar. success refers to accomplishment of not only immediate goals but also long-range goals such as maximizing personal potentials, social integration, meaningful work, and contributing to society (stoltenberg et al. 2000) . faculty have various backgrounds and credentials including those holding the lssp, ncsp, bcba, and/or licensure as psychologists. faculty are dedicated to ensuring candidates develop high levels of interpersonal and collaborative skills and apply cultural competence and sensitivity to diversity as they fulfill their professional, legal, ethical, and social roles in education and society. students also have access to, and some training in, clinics and centers associated with the school psychology program including the human neuroscience laboratory, the janice a. pattillo early childhood research center, and the school psychology assessment center. due to the program location and need for more school psychologists in the local community and neighboring rural communities, the sfa school psychology program created a de option in 2018 that is delivered via zoom. it was designed for students at least 50 miles away from the sfa nacogdoches campus. applicants must meet the same admission criteria as face-to-face students in the m.a. program and have the ability to attend courses via zoom video conferencing (i.e., strong internet connection, webcam, microphone). students attend classes via synchronous de. classes are physically conducted on the sfa campus with face-to-face students and de students attending virtually through zoom. in our experience, having de courses within the traditional classroom with face-to-face students attending allows live discussion to occur as it traditionally would. in addition to the standard program admission requirements, applicants must meet the criteria to be eligible for admission. the first requirement is that applicants' permanent address must be a minimum of 50 miles distance from the university campus. additionally, applicants agree that if accepted, they will be able to meet the following criteria: & students must attend courses in real-time, with face-toface virtual conferencing via zoom. & some courses and meetings may be required on campus. students will be notified at the start of each semester dates they will be required to be on the university campus. & some courses require assessment materials. for these courses, students must be able and willing to work with an education region service center in texas (texas education agency n.d.) or local schools for access to testing kits or travel to campus to check out and return the materials from the program. & students will be responsible for all course assignments and program requirements, including thesis, practica, and internship. & in the event of a technology issue of the instructor's/ university's fault, the instructor will work with the student to ensure appropriate instruction is provided. if a technology issue of the student's fault occurs, the course policy for class absences will be followed according to the syllabus. students are admitted to cohorts that include campus-based and distance learning students. during the course of the program, de students are required to attend select activities on campus. these requirements are intended to familiarize de students with program faculty, on-campus peers, and to provide face-to-face in-person experiences for activities that are difficult to complete remotely, consistent with established de programs in school psychology at other institutions. while the activities reported below have been required by the sfa school psychology program in the past, the program is moving or considering moving many of these activities to de during the fall 2020 semester due to the covid-19 pandemic. it is unclear what the effects of this change may be. the first of these requirements is the annual student orientation, held prior to the beginning of each fall semester. requiring on-campus attendance at orientation helps connect de and campus-based students in the program to each other and to program faculty. this assists de students in establishing relationships with faculty and peers so that they feel connected to the program (waldron-soler et al. 2019 ). an additional on-campus requirement is in the fall semester for first-year full-time students. the program holds an annual research event, where students in all research methods courses in the school psychology program present a poster. in addition to student posters, there is an invited paper presentation by university faculty or another professional in a relevant field. this event is highly publicized and well attended by students, faculty, and administrators within the university. requiring all students to present on campus provides an additional opportunity for faculty and peer connections, as well as professional development and networking experiences. while both of these events could be attended via de, program faculty decided to require some on-campus presence by de students to help support them in building relationships with students and faculty in the program. faculty believe this is important for students to gain a professional identity and establish a professional network. student feedback has been overwhelmingly positive in regard to these on-campus requirements. students are also required to attend campus for three days to practice and demonstrate skills related to assessment. these designated days are noted in the assessment course syllabus and students are informed when they will be required to attend campus prior to the semester starting. the university has purchased zoom accounts for all faculty and students. additionally, school psychology program faculty, as well as students enrolled in clinical experience courses each semester, have accounts that are upgraded to be health insurance portability and accountability act (hipaa; and family educational rights and privacy act (20 u.s.c. § 1232g) compliant. this allows additional encryption so that meetings are secure. additionally, several classrooms in the building that house the school psychology program are deemed "refreshed classrooms," as they have been updated with the technology needed to provide de instruction. the refreshed classrooms on campus have updated zoom technology, desktop computers (including mouse and keyboard), web cameras that have the ability to move and swivel, projection technology (overhead), external audio speakers, and microphones (at the instructor's presentation desk). in most campus buildings these classrooms offer the basic settings including lights, desks (some may be re-arranged), and a prominent desk at the head of the classroom for the course instructor. however, the campus also has some classrooms that are deemed "zoom classrooms" or "distance learning classrooms" that were created to help facilitate a seamless zoom experience. these classrooms have many of the same setup options as the refreshed classrooms; however, they also have a larger space at the head of the classroom designated for instructors, televisions that project lessons and course presentations, more sound-proofing to avoid distractions and less seating than other classrooms due to the expectation that teaching and learning will mostly occur by way of the online platform. the university has invested in distance learning technology and has made available training and troubleshooting resources for faculty and students. zoom technology is used for all class meetings with de students, as well as for additional virtual meetings between students and faculty (e.g., advising). during the covid-19 crisis, the university addressed concerns that many students and employees have limited access to the internet. wireless coverage was added in various outdoor areas of campus and extended in range to support safe options to utilizing the internet while continuing to attend distance courses. furthermore, the university continues to provide students with guidance on emergency grant funding, including the coronavirus aid, relief, and economic security (cares) act: higher education emergency relief fund (2020), which has awarded grants to 7661 students, to support them in gaining resources they need to continue their education. this includes computers and other materials that may assist in creating a more successful student experience with distance learning. there has also been an effort to ensure students have discounts for these resources through the university. all program courses are offered as both de and campusbased. a separate section is opened for each type of student, and then the sections are co-listed, so that campus-based and de students attend courses together as a cohort. all students are expected to attend classes in real-time and to participate in class discussions. course requirements are identical for all students, with the exception of the method of class attendance. additionally, students and faculty utilize zoom for other meetings, including group project work, virtual office hours, and more. one possible area of difficulty in designing and offering a de program is in courses that include use of assessment materials and those where the primary purpose is field-based work, such as practicum and internship. our program has addressed this with two options for students: using community resources or using on-campus resources. there is incredible community support from the texas education region service centers for de students admitted to our program. the program faculty have established relationships with many educational administrators that lead to our de students' ability to use resources of education region service centers in their area. these resources can be used in assessment courses so that students are able to access assessment materials to use in learning and practice and for assessment course requirements. for de students living in areas where the program does not have an ongoing relationship, the student has two options: work with program faculty to establish a relationship between the program and the education region service center and/or travel to campus to check out and return materials required for use in the courses. to date, all distance learning students have been able to work with their local education region service centers to use materials. practicum and internship requirements for campus-based and de students are identical. while most campus-based students complete practicum requirements at a site near the university, it is more common for students completing internship to obtain an internship at a site farther from campus. therefore, the program had already established de guidelines for the internship. these were then generalized to practicum courses for de students and include the following: & students attend all practicum/internship class meetings in real-time via zoom & students obtain at least the minimum face-to-face supervision required by nasp and the texas state board of examiners of psychologists from the field supervisor & students complete all required hours and activities/assignments, as required by the program & university supervisors utilize hipaa and ferpacompliant zoom, as well as university email, to communicate with the student and the field supervisor and to complete site visits and field supervisor interviews to date, we have had one de student complete practicum (this student was the first student admitted to the program, we have yet to experience attrition with de students). feedback from the student and field supervisors have indicated it was a positive experience and that the student was well prepared for the experience and to continue on to internship. this student was the only person admitted through the de option in the 2018-2019 academic year, the 2019-2020 cohort included three students, and the 2020-2021 cohort currently includes nine students. as so few students have matriculated through the program at this time, we feel uncomfortable sharing specific outcome data, as it could readily be identified. however, our experience has been that de students receive grades, comprehensive exam scores, and practicum evaluations that lead to the successful completion of requirements toward the degree. given the widespread shortage of lssps in texas (barbre 2019), we receive numerous emails and calls asking for current students and alumni to fill positions in the public schools each week. the region 8 education service center (texas education agency n.d.) reached out to sfa four years ago offering to provide equipment, proctors, and rooms for us to provide de through their facility for school psychology training. they expressed concern over being understaffed, having many of their lssp's close to retirement, and being unable to bring new psychologists into their area. they report much of the problem to be the rural nature of their districts, with schools being many miles apart and having no colleges close enough for people interested in the field to attend. the region 8 service center is more than two hours away from sfa and even further away from other lssp training programs. they stated they felt they needed to "grow our own," noting that often the only way to get applicants who want to be in their service center area is for the applicant to be from the area, wanting to come home, or wanting to stay close to their childhood homes. the region 8 education service center in conjunction with sfa school psychology faculty and interns provided meetings in their facility and numerous flyers, brochures, and phone calls were used to invite people to attend. we had several attendees but no follow through with applicants. as a faculty, we explored options and decided that designing a program through zoom for the m.a. degree might be an avenue to address the practitioner shortage as well as the declining number of applicants to our existing program. this was advertised through the 20 region education service centers in texas. after this, we were approached by region 15 education service center about participating in a new project from the texas education agency, because of our online option (zoom) to get more evaluators in texas public schools. they wrote a small and rural schools grant funded by the texas education agency (https://tea4avoswald.tea.state.tx.us/ grantopportunities/forms/grantprogramsearch.aspx). the region center expressed a desire to work together to provide partial and potentially full funding for individuals who work in a small or rural district and are willing to commit to staying in the district for at least three years. they presented their grant information along with their partners in a training at a regional education conference. de courses, whether online or hybrid/blended, can cause faculty numerous issues. online teaching is like learning a new sport; a new sport can be time-consuming and onerous. we strongly encourage faculty considering teaching a de course to work with the center of teaching and learning, or similar office, on their campus. the center for teaching and learning at sfa requires instructors to complete a semester-long online instructor certification program before teaching any online course. we took this class in preparation for working with face-to-face virtual students and frequently collaborate with each other to troubleshoot issues or share new tips and techniques we have learned. faculty also regularly attend the center for teaching and learning workshops on new instructional strategies that use technological services of benefit to face-to-face in-person and face-to-face virtual students. instructors should invest a significant amount of time becoming comfortable with virtual learning modalities and interfacing with other individuals on these platforms before engaging in virtual teaching. while such courses are vital, some situations are difficult to anticipate and are learned through lived experiences. we wish to present the common issues we have experienced in this process and possible solutions to these issues (see table 1 ). one of the main complaints of many faculty teaching an online course is passive students (lieberman 2019) . it is difficult to deliver lecture material to the quiet static of muted students without video, not to mention engage in discussion. face-toface in-person instruction allows faculty to read the faces of their students. the quiet space of the online classroom can be daunting. zoom offers a solution to this issue. zoom (2020) allows real-time polling during instruction. other learning management systems offer similar options. faculty can poll students on material as a simple check for understanding or as an opinion poll to increase participation and discussion. the polling can be set to anonymous, which allows students a safe space to voice opinions. this type of active student responding during instruction has a positive relationship with achievement (ellis et al. 1994; greenwood et al. 1984; rosenshine 2012) . another strategy is requesting a check for understanding during lectures. ask students to verbally state or use the chat box "thumbs up" or "thumbs down" to indicate they either understand the material or need more review of the material. the instructor can provide them the option of private messaging if they do not wish to publicly state they do not understand. this technique can promote engagement during online lectures (rosenshine 2012). zoom (2020) also allows the instructor to use a whiteboard. here, the instructor can draw or write in real time to the student. the instructor can also give control of the whiteboard to a student to answer a problem or demonstrate a skill. besides zoom, there is other software to help increase engagement in the online classroom. kahoots (2020) and quizizz (2020) allow the instructor to make a game out of quiz questions, allowing students to compete against each other. these platforms have the added benefit of allowing students to replay the quizzes at a later time to help prepare for exams. flipgrid (2020) allows the instructor to ask a question and students answer with a video message, creating a social media feel within the classroom. in a course with both face-to-face in-person and face-toface virtual learners, you can utilize these practices for both groups. you can embed polling into your powerpoints for traditional learners and use zoom (2020) polling for distance learners, and you can have the traditional learners physically give a thumbs up or thumbs down during lecture while the face-to-face virtual learners use the chatbox. struggling to get students to engage in discussion boards and projects in an online classroom is common. many students take very little action until the end of the semester (lieberman 2019) . this can be time-consuming and frustrating for the instructor. the instructor can model the behavior they wish to see from the student. for discussion boards and projects, the instructor can provide an example of an exceptional student from a previous semester or complete the project themselves to show exactly what is expected. this will also help alleviate some of the questions about the task. another recommendation is to provide choice during these discussion boards and projects. this way the students can engage in self-directed learning and develop interests within the course content. abrami et al. (2011) found in de classrooms that providing students with personally meaningful work, which students had some sort of choice in created greater levels of intrinsic motivation. allowing students to personalize their learning experience will create a better distance experience. another potential concern with online classrooms is an increase in email communication with questions from students. one recommendation from garrison and vaughan (2007) is to view these emails as part of the instruction. if an instructor receives a question in an email such as "do i need the latest edition of the book?", they should assume many students are wondering this and post the answer as an announcement. another recommendation is to create a discussion thread with the title questions for the instructor. this way every student will be able to benefit from the instructor's answers to the questions, and it may reduce the number of emails asking similar questions. however, it is also important to clearly communicate to students that digital correspondence, particularly in the absence of face-to-face in-person interaction, is important and valued. it is difficult sometimes in face-to-face in-person classrooms to create collaboration, and it becomes even more difficult in the face-to-face virtual classroom. zoom (2020) has breakout rooms. the instructor can choose to place students into smaller groups. during these breakout sessions, the instructor can join different breakout groups to check-in and engage the subgroup in the discussion or activity. when time is up, the instructor is able to give the students a warning timer for when the breakout rooms will end and everyone will return to the larger classroom. this creates a similar environment to the face-to-face in-person classroom, allowing students to work together in small groups. at sfa, when students in the faceto-face in-person classroom were broken up for group work the instructor was able to place the face-to-face virtual students into a breakout room so they could have equivalent experience. depending on the number of face-to-face inperson and face-to-face virtual students present, creating groups the aforementioned way may be problematic and the instructor may desire to arrange people in groups based on other factors. in this case, face-to-face in-person students can use their phone or laptop to join the virtual breakout rooms. the virtual learning environment inevitably comes with technical difficulties. anything can happen from video not working to full wi-fi failure. a few tips to help with this is to hold a tech session prior to the start of class. the instructor and students can work out difficulties prior to the semester beginning. this also can alleviate the anxiety for less tech-savvy students. providing documents for students to troubleshoot technical issues is also helpful. often a university's technology office has already created one that can simply be linked to the course. online teaching requires much flexibility. during the first class meeting the instructor should discuss with the student how technical difficulties will be handled (i.e., what will happen if the instructor loses the internet during the allotted course time or what will happen if a student loses the internet and cannot attend). there also should be a document in the course content with an explanation of these possibilities (e.g., handbook, syllabus, learning management system). with a little planning and a few extra steps, instructors can make the online classroom engaging and easy to navigate for any student. despite being ranked as the 36th best job (u.s. news and world report n.d.), the field of school psychology continues to face a national shortage (castillo et al. 2014; fagan 2004) . one way to address the shortage is for school psychology programs to consider providing instruction via de (dixon 2017; dixon 2018; moy et al. 2017) . to address the national, state, and local shortage of school psychology practitioners and trainers the faculty at sfa began offering a de option to students outside a 50-mile radius from campus. although we started with only one student it has expanded with the assistance of regional service centers and district leaders interested in a "grow your own" model. although the sfa de option is too young to provide outcome data, there is evidence from existing de programs that students are adequately trained and prepared for the workforce (dixon et al. 2020; drefs et al. 2015; hendricker and viola 2019; ruby et al. 2017; schroeder 2019; waldron-soler et al. 2019) . this is in contrast to the majority perception held by school psychology faculty and graduate students that de training options are inadequate. program faculty at sfa were also skeptical of de courses, especially skills-based courses. however, anecdotally, we have not found the de students' knowledge or skills to be inferior to face-to-face students in our program. course grades and other outcomes, such as comprehensive exams, are also commensurate between de and campus-based students, although the sample size is small. while de has the potential to increase the recruitment reach of school psychology training programs, provides flexibility to students, and has evidenced similar outcomes between students in de and non-de courses (hendricker et al. 2017) , there are some limitations to consider. one limitation is the perspectives of students and applicants. some graduate students report a lack of interaction and reduced skill acquisition when completing de courses ). the decreased face-to-face in-person interaction also has the potential to impair the quality of mentorship that can be provided. in particular, it may be more difficult for the faculty member to write strong letters of recommendation when they have limited face-to-face in-person interaction with the person in order to judge their professionalism and ability to interact with others in a live setting. another potential limitation is faculty perspectives. luongo (2018) completed a survey with fulltime and part-time faculty teaching de courses at a university. results indicate that about 47% of respondents disagreed that they were as satisfied with teaching de courses as using other modalities and that about 63% of participants indicated their workload is higher with de courses than courses using other modalities. over 70% of respondents indicated they either disagreed or strongly disagreed that they missed face-to-face contact with students when teaching de courses. in addition, luongo (2018) discusses several factors related to faculty investment and performance in de, including a lack of administrative support, lack of compensation, and inaccessibility of training and resources. accessibility to resources is not limited to a faculty barrier. a number of resources are needed to make de viable as a course modality option. these resources include, but are not limited to, technology and training for faculty and students. access to equipment and technology can somewhat be overcome by the university providing equipment and hot spots to faculty and students; however, there can continue to be issues with equity of content delivery, particularly for those students living in areas with limited broadband speed capabilities who are not able to travel to provided hot spot locations. sisneros and sponsler (2016) report 53% of individuals living in rural areas have inadequate bandwidth speed, compared with 8% of individuals living in urban areas. additionally, they report that as household income decreases, so does the likelihood of individuals having a bandwidth subscription, with lower-income black and hispanic households being even less likely to purchase bandwidth subscriptions than white low-income households. these data indicate limitations of access and equity across region, income, and race and ethnicity that need to be considered when planning for a de instructional model. access to training and support for faculty and students also need to be considered. a survey by viola et al. (2020) indicated students in graduate school psychology courses report using a number of technologies in de courses, including learning management systems (blackboard, canvas, etc.), real-time video applications (skype, zoom, etc.), and shared drives. many of these technologies may not have been used by faculty or students prior to the de course experience, requiring adequate training for best use of technology by faculty and usability for students. universities can provide this training through professional development opportunities and support services. students in a "grow your own" environment often have restricted supervisor options, which is another limitation de programs may face when trying to identify and select qualified supervisors for de students. providing training to supervisors and developing strong bilateral communication with supervisors about course requirements and evaluation of student performance is also a challenge from a geospatial distance. while additional limitations will no doubt be experienced by programs transitioning to a de model, faculty and student perspectives, technology access and equity, and training and support appear to be most frequently cited in the current literature. while these limitations may result in barriers to effective de course delivery, many can be overcome with careful planning and resource allocation. programs considering de should work with university administration to prepare for and prevent these barriers to the extent possible prior to offering de courses. school psychology de courses and de programs have received increased attention in professional outlets (i.e., the communique, journal articles, special issues) recently. although we anticipated this trend to continue before the pandemic, we more firmly believe de options will become more prevalent as faculty and student have been provided opportunities to experience and become more comfortable with this format. nonetheless, much more research is warranted and the discussion of how best to prepare students in skill-based classes is needed. data availability not applicable. conflict of interest the authors are affiliated with the program featured in this paper. ethical approval not applicable. consent to participate not applicable. code availability not applicable. interaction in distance education and online learning: using evidence and theory to improve practice a report on the state of school psychology in texas schools: 2017-2018 school year. research and practice in the schools where are the school psychologists? personnel needs in school psychology: a 10-year follow-up study on predicted personnel shortages and economic security act: higher education emergency relief fund online education: let's start the conversation online school psychology: blueprint to higher education conversation extending the university walls: using technology to prepare future practitioners canadian innovation: a brief history of canada's first online school psychology graduate program effective teaching principle and the design of quality tools for educators school psychology's significant discrepancy: historical perspectives on personnel shortages online content instruction in school psychology training faculty perceptions of distance education in school psychology training online school psychology program blended learning in higher education: framework, principles, and guidelines school psychology programs and degrees in texas opportunity to respond and student academic performance using distance education with nontraditional school psychology students trends and perceptions of distance learning in school psychology kahoot. (version 6.0.1) [computer software discussion boards: valuable? oversued? discuss. inside higher ed an examination of distance learning faculty satisfaction levels and self-perceived barriers opportunities and challenges for online instruction in school psychology programs key concepts and terminology in online instruction: a primer for school psychology programs quizizz. (version 4.27). [computer software principles of instruction: research-based strategies that all teachers should know quality distance learning to address the school psychology shortage: a model for online respecialization school psychology matters preparation for a premaster's internship in a blended school psychology program: a case study. training and education in professional psychology broadband access and implications for efforts to address equity gaps in postsecondary attainment training models in counseling psychology: scientist-practitioner versus practitioner-scholar current nacogdoches, texas population, demographics and stats in 2019 education service centers distance education in graduate training programs: lessons learned from school psychology students instructional technology in graduate psychology distance education: trends and student preferences learning from graduate training in related fields: a model online school psychology program master of art in school and community psychology zoom. (version 5.0.1) [computer software publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations mccleary is an associate professor at stephen f. austin state university. his primary interests include math fluency and the scholarship of teaching and learning clark is a clinical professor at stephen f. austin state university. she has 40 plus years in private practice in deep east texas. currently she is the director of the apa accredited sfa charter school psychology internship program dawes is now an assistant professor at the citadel. her interests include math fluency and behavior analysis flowers is an assistant professor at stephen f. austin state university. her interests include bullying and behavior analysis ellis-hervey is an associate professor at stephen f. austin state university. her interests include selfesteem, locus of control, and personal presentation difficulties encountered by women of color key: cord-295670-jl8aouq9 authors: rozenshtein, anna; griffith, brent d.; ruchman, richard b. title: residency match during the covid-19 pandemic: the clear and present danger of the remote interview date: 2020-10-12 journal: j am coll radiol doi: 10.1016/j.jacr.2020.10.005 sha: doc_id: 295670 cord_uid: jl8aouq9 nan in the coming fall, graduate medical education will be adapting to the disruptions in resident recruitment caused by the covid-19 pandemic. out of concern for the safety of medical students, the upcoming recruitment season will be conducted remotely. the benefits of remote interviews are obvious -an elimination of the risk of contagion during travel and face-to-face interviews. additionally, remote interviews are inexpensive and convenient --and therein lies the problem. remote interviews will almost certainly exacerbate the problem of over-application which has plagued the match since the introduction of eras. before the advent of eras, the cumbersome process of applying to residency programs limited the number of applications medical students were willing to send. however, with eras medical students had the ability to apply to all programs of their choice with a click of the mouse. because modest increases in numbers of applications tended to increase the applicants' chances of matching, it made sense to apply widely. as a result, since 2001 the average number of applications per applicant through eras has increased every year in every participating specialty [1] . in radiology, the number of applications per us medical school senior has more than doubled and the number of applications per training program more than tripled ( figure 2 ). faced with a deluge of applications, residency programs increasingly relied on the united states medical licensing examination (usmle) step 1 score filter, leading to its own set of problems. to remedy the situation, the association of american medical colleges (aamc) initiated the "apply smart" web page which allowed medical students to determine the point of diminishing returns for each subsequent application depending on their uslme score. still, the number of applications per person continued to rise. weissbard et al. explained the phenomenon by the prisoner's dilemma (table 1) , predicting that as long as j o u r n a l p r e -p r o o f their peers are not limited in number of applications, medical students will try to stay ahead of the competition by applying to more and more programs [2] . it is likely that remote interviews will further exacerbate the problem by taking the cost of time and travel off the table. fogel and colleagues reported that 41% of medical students declined some residency interviews for financial reasons [3] . we predict that transition to remote interviews will result in increased numbers of interview requests (eras applications) and higher interview acceptance rate. in the nearly costless scenario medical students lose nothing from every additional encounter while improving their interviewing skills. because the most desirable students are usually invited first, we foresee that the competitive cohort is likely to displace other qualified applicants who would have been granted an interview in prior years. if this comes to pass, programs will interview the same applicants, resulting in a smaller rank pool and increase in the number of unfilled positions and unmatched applicants. radiology would be particularly affected, because of its frequent use by applicants as a "backup" specialty ( figure 2 ). the higher the percentage of such applicants in a given specialty, the more it must compete with other specialties for qualified candidates. if the cost constraints of inperson interviews are removed, top tier applicants using a specialty for "backup" may further displace qualified candidates for whom that specialty is the first or only choice. j o u r n a l p r e -p r o o f immediate solutions: the calls by programs for hard application caps [2] are unacceptable to medical students, while the calls by medical student groups for program-specific data on characteristics of matched applicants are unacceptable to programs. attempts at making the application process less generic, such as the 2015 requirement of a program-specific paragraph introduced in the otolaryngology match may have contributed to a decline in applications and increase in unmatched programs [5] . regardless, none of these steps correct the fundamental problem because they do not allow applicants to determine their competitiveness and signal their preferences to training programs. all program directors receive numerous expressions of interest from applicants, but such declarations are of little value since programs cannot judge the sincerity of such an expression. we recently surveyed radiology program directors as to whether they would consider an "early action" period, whereby for a short time (e.g., two weeks) at the beginning of the interview season eras could allow medical students to apply to a small number (e.g., ten) of programs of their choice, thus allowing a clear indication of special interest in a program. over three quarters of respondents were willing to participate in such a program. unfortunately, the authors' repeated attempts at persuading eras to pilot the solutions did not gain traction. it is possible that the aamc, the parent of eras, did not see this to be a problem for medical students. however, medical students clearly saw it as a problem, and recently, appear and nrmp that "business as usual" cannot continue and urgent reform must take place. in conclusion, while the nrmp algorithm continues to fulfill its promise of a strategy-free residency match, the current application/interview process is increasingly flawed. we worry that the covid-19 pandemic, with its transition to remote interviews, will exacerbate the problem. in the coming match season residency administrators should keep in mind that a rise in the number of applications and a higher interview acceptance rate from highly qualified applicants do not mean greater interest in either their program or radiology as a specialty. rather, many desirable candidates may use the additional screen time to hone their interview skills. some training programs grant more interviews to truly interested and qualified students with less emphasis on eras filters such as usmle scores and in-state location resulting in a more diverse resident body. students are less constrained by in-state location. both groups of students benefit. group 2 students who applied only to their top choice programs are more likely to fail in the match. group 1 students benefit by overapplication. group 2: students apply to as many programs as possible group 1students who applied only to their top choice programs are more likely to fail in the match. group 2 students benefit by over-application. training programs are overwhelmed with applications and filter them by the usmle score and in-state location. qualified students with lower uslme scores cannot get an interview, resulting in a less diverse resident body. students are more constrained by in-state location. both groups of students are harmed by over-application. j o u r n a l p r e -p r o o f j o u r n a l p r e -p r o o f relationship between the number of residency applications and the yearly match rate: time to start thinking about an application limit? the urology match as a prisoner's dilemma: a game theory perspective the economic burden of residency interviews on applicants is the program-specific paragraph responsible for declining application numbers? a commentary nrmp results and data 2020 main residency match table 13a key: cord-306415-n8klypke authors: banerjee, snehasish title: to capture the research landscape of lecture capture in university education date: 2020-09-30 journal: comput educ doi: 10.1016/j.compedu.2020.104032 sha: doc_id: 306415 cord_uid: n8klypke the use of lecture capture has been burgeoning in the higher education sector. scholarly interest on the topic is also on an upward trajectory. hence, the goal of this paper is to clarify the literature on the use of lecture capture in higher education through a systematic review, which involved 71 articles that came from scopus and education resources information center (eric). the systematic review revealed that most research focused on university education in the us, the uk and australia. the most studied science, technology, engineering and mathematics (stem) subject area in the sample is biological sciences. in contrast, the most studied non-stem subject area is business/economics/management. in terms of methods, descriptive and exploratory research dominates the research landscape while causative evidence is relatively limited. the literature includes two overarching research streams: while one focuses on the benefits of lecture capture, the other focuses on its drawbacks. apparently, the assumption among most students is that lecture capture is helpful for learning, and among staff is that it takes a toll on attendance. future research needs to reconcile students’ perception of lecture capture, instructors’ perception of lecture capture, and the reality of lecture capture. on the practical front, students should be explicitly told about the benefits and the drawbacks of lecture capture. this would ensure that they make the most of the technology. educators are urged to reflect on why students should be attending the face-to-face sessions, and thereafter also watch the recorded lectures. digital technology has been making rapid inroads in curriculum design. specifically, lecture capture is increasingly becoming popular all across the globe (stokel-walker, 2019) . consistent with its surging popularity among practitioners, scholarly interest on the topic has also been on the rise in recent years. the diversity in the literature calls for synthesising the research landscape of lecture capture (nordmann et al., 2019) . as a response to the call, the goal of this paper is to clarify and consolidate the accumulated state of knowledge regarding the use of lecture capture in higher education. the focus is particularly on higher education due to two reasons. first, the use of lecture capture is burgeoning in this sector. over 1,000 higher education institutions worldwide use it to record more than 100 hours of lecture video every month (stokel-walker, 2019) . in countries such as the us and the uk, more than 4 in 5 institutions utilise lecture capture (newland, 2017; panopto, 2020) . the technology has grown in stature to such an extent that it is now noted by its absence rather than presence in higher education (cramer, 2015; ibrahim et al., in press ). second, university students are more independent than their junior counterpart such as students in k-12 (ary et al., 2002; belland and drake, 2013) , who are likely to experience greater parental mediation-strategies that parents use to monitor and regulate children's use of digital media (warren, 2001) . therefore, lecture capture use by the former is likely to be more self-regulated and self-paced. as a result, the literature on the use of lecture capture in the higher education setting is likely to be more nuanced than that in the k-12 realm. to achieve the research goal, a systematic literature review was conducted with the following objectives: (1) to identify the contexts that lecture capture research has explored in higher education, (2) to recognise the methods that lecture capture research in higher education has employed, and (3) to analyse the overarching research streams in the current body of the lecture capture literature in higher education. the paper is significant on three fronts. first, the systematic review enables educators to better understand how university students use lecture capture. second, it contributes to the academic discourse on the use of lecture capture in higher education by identifying several under-investigated research contexts, research methods, and research themes. third, it dovetails extant systematic reviews on the broader theme of technology-enhanced learning. for one, kay (2012) reviewed the literature on the use of vodcasts-also called video podcasts-in education including secondary school and elementary school. in contrast, this paper casts the spotlight specifically on the use of lecture capture in university education. more recently, o'callaghan et al. (2017) conducted a systematic review on the use of webbased lecture technologies such as lecture capture, podcasts, and video streaming in australia. in contrast, this paper seeks to offer a more in-depth review of lecture capture only but without limiting the geographical scope. in an even more recent work, clunie et al. (2018) reviewed the literature on technology-enhanced learning tools in anatomy education-a field that is known to be at the forefront of integrating innovative technologies in the curriculum. in contrast, this paper does not restrict the review to any discipline or sub-discipline. the paper proceeds as follows: section 2 offers a conceptualisation of lecture capture. section 3 describes the methods employed to conduct the systematic review. section 4 discusses the findings. finally, section 5 highlights the paper's contributions, limitations, implications for practice, and opportunities for further research. lecture capture, as the name suggests, essentially consists of recorded or captured lectures that are then used as a learning resource by students. the recording can take place either in vitro-where lectures are delivered without any audience just for the purpose of being captured-or in vivo-where live face-to-face lectures in front of students are captured in real time (pale et al., 2014) . the scope of this paper is trained on the latter. meanwhile, a live face-to-face lecture in front of students can take place in three formats. in one, the instructor interacts with students by writing on a traditional blackboard. no digital audio-visual element is involved. in the second format, the instructor uses the lecture-room projector to present a slideshow to the students. other audio-visual learning materials including smart boards and videos can also be displayed. the third format is one that includes a combination of the first two. for the purpose of this paper, lecture capture encompasses recording the activities on the lecture-room projector during a live face-to-face lecture, which takes place based on either the second or the third format, for subsequent dissemination among the students enrolled in the course. the recording notwithstanding, the students are expected to attend the faceto-face sessions. some versions of such a lecture capture system record the lecture-room projector along with the instructor's voice. other more encompassing versions record the instructor along with the students-both speech and video (mccunn and newton, 2015; newton et al., 2014) . the recordings are often timestamped for ease of navigation, and can be accompanied with captions (gorissen et al., 2015) . given the increasing smartphone penetration in people's everyday lives, lecture capture provides students, even those with a range of disabilities, with on-demand any time-any place access to lectures (ibrahim et al., in press; mccunn and newton, 2015) . the online recordings serve as long-lasting digital footprints of live lectures that are otherwise ephemeral. watching lecture capture is a key step in contemporary education. it allows students to revisit lectures that they had either missed or failed to understand, thereby promoting inclusivity-a theme that has been attracting much attention among higher education leaders and policy makers. inclusive higher education seeks to not only engage every student without exclusion but also modernise the learning environment to meet contemporary learning needs (blessinger et al., 2018; thomas and may, 2010) . it also fits well with the 2030 agenda for sustainable development of leaving no-one behind (united nations, 2015) . clearly, the use of lecture capture seems to be a step in the right direction in this digital economy insofar as achieving inclusivity and equal access to higher education. using lecture capture is however a social process that sees the confluence of not only technology but also the behaviour of students and educators (kirschner et al., 2004) . hence, it is important to cast the spotlight on lecture capture through the lens of learning theories. the two key paradigms that lie at opposite ends of the spectrum are behaviourism and constructivism (bichelmeyer and hsu, 1999; radianti et al., 2020) . behaviourism assumes objectivity and the existence of a single reality. students acquire knowledge passively from instructors who aim to transfer the correct behavioural response to external stimuli. learning motivation is extrinsic, involving positive and negative reinforcement. there is an emphasis on repetition and rote learning (ertmer and newby, 1993) . in contrast, according to constructivism, there are multiple realities that are subjectively constructed. students, who are self-regulated, actively construct their own understanding of reality by linking new information to their prior experiences (ertmer and newby, 1993) . instructors serve as learning facilitators, rather than knowledge transmitters, who emphasise on problem solving (bichelmeyer and hsu, 1999) . since knowledge acquisition depends on how information is received and assimilated, digital technology can be particularly helpful for constructivist learning design (radianti et al., 2020) . watching lecture capture after attending the face-to-face lectures is likely to help students in knowledge j o u r n a l p r e -p r o o f building from half-baked notions, thereby enabling them to gradually develop a grasp over complex problem solving (scardamalia and bereiter, 2010) . given this conceptualisation of lecture capture that fits with constructivism, the following section presents the methods of the systematic review. the findings can be beneficial for designing technology-enhanced constructivist learning through the use of lecture capture. to collect relevant papers in english language as exhaustively as possible, a literature search protocol was necessary. for this purpose, pilot searches were conducted on scopus, the largest database of peer-reviewed literature (maflahi and thelwall, 2016) . three observations arose. first, relevant papers used at least one of the following six alternative phrases: "lecture capture," "lecture recording," "lecture streaming," "recorded lectures," "screencasting," or "screencast." second, relevant papers in the context of higher education used at least one of the following three alternative phrases: "university education," "higher education," or "university teaching." third, when the search was applied on full texts, several irrelevant articles were retrieved and the noise exceeded the data. the volume of noise was possible to reduce considerably by restricting the search on titles, abstracts and keywords. guided by these initial observations, the following search query was used: ("lecture capture" or "lecture recording" or "lecture streaming" or "recorded lectures" or "screencasting" or "screencast") and ("university education" or "higher education" or "university teaching"). the use of such boolean operators is common in systematic literature reviews (pal and chua, 2016; zhang et al., 2015) . the search query was applied to titles, abstracts and keywords. two databases were used for searching: scopus and education resources information center (eric). scopus was chosen because it is well known for its comprehensive coverage (maflahi and thelwall, 2016) . it indexes articles retrievable through several other independent academic databases such as emerald and springer. eric was chosen because it is a repository dedicated to the education literature (clunie et al., 2018) . general google searches were avoided to exclude grey literature that may not have undergone scientific peer-review (yli-huumo et al., 2016) . the peer-reviewed literature retrievable through google are anyway expected to be obtained via scopus and/or eric. the search was restricted to journal articles as the inclusion criterion. this was necessary for quality control as journal articles undergo more rigorous peer-review compared with either conference proceedings or book chapters. no date restriction was applied because the use of lecture capture is not an age-old phenomenon, and is therefore self-limited to the last two decades. by conducting the search in april 2020, 185 articles were retrieved. figure 1 depicts the literature screening process that involved three steps. in the first step, the initial corpus of 185 articles (65 from scopus + 120 from eric) was checked for duplicates. a total of 46 duplicates were identified, and eventually removed. in the second step, the relevance of the remaining 139 unique articles was checked by reading their titles and abstracts. forty articles that were thematically irrelevant were removed. whenever the relevance of an article was not possible to determine with certainty, it was retained. seven articles related to lecture capture were also excluded. these either described the authors' personal experiences (newton et al., 2014) , did not report any empirical work (zandona et al., 2016) , or focused on the technological implementation of lecture capture (valor miró et al., 2014) -which are beyond the scope of this systematic review. taken together, 47 articles were excluded. in the third step, the relevance of the remaining 92 articles was checked through analyses of their full-texts. twenty-one articles were further excluded because they did not conform to the paper's conceptualisation of lecture capture-recording of live lectures for students who are also expected to attend the sessions face-to-face (cf. section 2). for example, loch et al. (2014) focused on video recordings that were not recorded during live lectures. bahnson and olejnikova (2017) looked into recorded lectures but did not require students to attend the live sessions face-to-face. all such instances had to be removed. finally, 71 articles were admitted for the systematic literature review. figure 2 shows their year-wise distribution. the coding and analysis involved two steps. in the first step, the 71 articles admitted for the systematic literature review were carefully inspected. the researcher used a microsoft excel spreadsheet to record the following data points: study details (e.g., year of publication, geographical location of the study), research methods (e.g., qualitative / quantitative / mixed, correlational / causative), sample characteristics (e.g., subject area, undergraduate / postgraduate), lecture capture characteristics (e.g., length of recordings, number of recordings), and finally thematic focus of the paper-as identified through iterative rounds of coding (e.g., students' attendance, academic performance) coupled with the finding with respect to the theme. in the second step, another coder, who was a full-time postgraduate student in a large university in the uk, independently inspected all the articles. the goal was to cross-check the entries in the spreadsheet obtained from the previous step. all inconsistencies were resolved through discussion. thus, the final spreadsheet reflected full inter-coder agreement between the researcher and the independent coder. the involvement of the researcher, who dons the hat of an instructor with experience of leading courses that are lecture captured, along with a student in the coding procedure was particularly necessary to minimise any bias. after all, lecture capture divides instructors and students-as reflected later in the findings too. j o u r n a l p r e -p r o o f as depicted in figure 3 , among the 71 articles, much of the scholarly attention has been trained on the us (18 articles, 25.35%), the uk (18 articles, 25.35%), and australia (15 articles, 21.13%). beyond these top three countries that account for more than 70% of the sample, other sparingly studied research contexts include canada, the netherlands, malaysia, qatar, taiwan, austria, germany, turkey and the uae. in the meantime, recent statistics suggest that more than 1,000 higher education institutions worldwide use lecture-recording tools such as panopto to record more than 100 hours of lecture video every month (stokel-walker, 2019). thus, the geographical distribution of the articles does not seem to be in line with the current trends of lecture capture usage. only one article reported a comparative analysis between two countries, namely, australia and the uk (trenholm et al., 2019) . some discrepancies were identified. for example, in australia, regular and irregular lecture capture users differed in terms of age. however, such a difference was non-significant in the uk. given the limited research, this paper calls for more cross-country and cross-cultural investigations that can offer insights into differences in students' motives, behaviours and attitudes regarding lecture capture. as shown in figure 4 , the most studied science, technology, engineering and mathematics (stem) subject area in the sample is biological sciences (29 articles, 40.84%). informed by nightingale et al. (2019), biological sciences in this paper include biochemistry, health j o u r n a l p r e -p r o o f science, nursing, midwifery, medicine, pharmacy, and psychology. in contrast, the most studied non-stem subject area is business/economics/management (16 articles, 22.54%). most (58 articles, 81.69%) of the 71 articles studied lecture capture from a student-only perspective while only three had a staff-only perspective (freed et al., 2014; germany, 2012; joseph-richard et al., 2018) . the remaining 10 articles offered a greater degree of triangulation by shedding light on the two perspectives in tandem. thus, students were involved as research participants in 68 articles (58 + 10). furthermore, staff members were involved as research participants in 13 articles (3 + 10). among the 68 articles involving students, the spotlight was predominantly on those pursuing only undergraduate degrees (53 articles) even though they were not always clearly differentiated in terms of their year of study. only one article specifically focused on executive students (farooq et al., 2017) . another two focused exclusively on postgraduate students (baker et al., 2018; saunders and hunt, 2015) . yet another four studied a mixture of undergraduate and postgraduate students (dommett et al., 2019; dona et al., 2017; gosper et al., 2010; taplin et al., 2011) . the scope of caglayan and ustunluoglu (in press) was trained on not only undergraduate and postgraduate students but also what are referred as associate students. in addition, there are a few instances where the degree pursued by the students was not clarified explicitly (e.g., groen et al., 2016) . nevertheless, it is clear that lecture capture research is limited in the contexts of postgraduate and executive education. future research needs to plug this contextual gap in the literature. undergraduate students are expected to have a relatively lower level of maturity than either postgraduate or executive students. therefore, findings gleaned from the former cannot be generalised to the latter. it is conceivable that the use of lecture capture throughout the world may differ in terms of factors such as length of recordings, number of recordings, and availability of recordings. findings cannot be assumed to be the same regardless of these contextual factors. yet, several articles refrained from providing such details (e.g., cooke et al., 2012; germany, 2012; mccunn and newton, 2015) . among the handful of articles that included such a level of details, length of the lectures captured was mostly reported as one hour or less (e.g., brooks et al., 2014; elliot and neal, 2016; williams et al., 2016) . in three articles, duration of the lectures varied from one to two hours (chapin, 2018; jones and olczak, 2016; trenholm et al., 2019) . in yet another three articles, lectures were as long as three hours (bollmeier et al., 2010; chen and lin, 2012; owston et al., 2011) . the number of lecture recordings varied drastically from as low as two (hadgu et al., 2016) to as high as 47 (guy et al., 2018) . few articles stated exactly when the lecture recordings were released, and how long they remained available to students. a notable exception is bollmeier et al. (2010), according to which students had access to lecture recordings for 72 hours following the live lectures. lecture capture research does not seem to suffer from mono-method bias. this is evident from the widespread use of mixed methods. of the 71 articles, 27 (38.03%) employed quantitative and qualitative methods in conjunction. among the rest, purely quantitative works (40 articles, 56.34%) outnumber those that are purely qualitative (4 articles, 5.63%). even in terms of data source, several articles complemented the use of surveys and interviews with focus group discussions as well as tracking of student-specific data. in particular, 45 articles (63.38%) reportedly administered surveys. student-specific dataranging from attendance and academic performance in assessments to lecture capture usage via online logs-were tracked in 40 articles (56.34%). the use of focus group discussions was evident in 13 articles (18.31%). experiments were employed in 11 articles (15.49%). seven articles (9.86%) conducted interviews. as the widespread use of lecture capture continues to spawn a growing corpus of digital data, the scholarly community is also starting to employ novel analytical techniques on student-specific data. three such examples are illustrated as follows: brooks et al. (2014) used unsupervised machine learning algorithms on students' data. luttenberger et al. (2018) applied latent class analysis. more recently, o'brien and verma (2019) applied cluster analysis. all of these essentially leveraged data analytics to make sense of students' lecture capture utilisation patterns. nevertheless, correlational, descriptive and exploratory research seems to dominate the research landscape of lecture capture while causative evidence, which is possible to glean only through experiments, is still relatively limited. this could be attributed to the logistical difficulty in arranging randomized samples, random assignments, and controlling extraneous variables in university education. drouin (2014) and traphagan et al. (2010) are among the few exceptions that used quasi-experimental research designs to infer causation. moreover, the methods of most articles were driven by practical concerns, and not by theory. only a handful of works were explicitly rooted in theory. for example, farooq et al. (2017) and nair et al. (2015) were rooted in the unified theory of acceptance and use of j o u r n a l p r e -p r o o f technology. however, none of the articles were situated within any pedagogical or technology-mediated learning frameworks. in biological sciences-the most studied stem subject area (29 articles), the distribution of research methods is as follows: 9 mixed methods + 19 quantitative + 1 qualitative. at a granular level, this body of research contains 18 surveys, 17 studies that involved tracking student-specific data, five focus group discussions, five experiments, and one interview. in business/economics/management-the most studied non-stem subject area (16 articles), the distribution of research methods is as follows: 5 mixed methods + 11 quantitative. at a granular level, this literature encompasses 10 surveys, 11 studies that involved tracking student-specific data, one focus group discussion, and two experiments. appendix a summarises the findings of the paper corresponding to research context (objective 1) as well as research methods (objective 2). reading of the lecture capture literature highlighted two overarching research streams that almost seem to collide head-on. while one research stream focuses on the benefits of lecture capture, the other focuses on its drawbacks. this is perhaps unsurprising as lecture capture tends to polarise views. some consider it a priceless adjunct to promote inclusive education (blessinger et al., 2018; mallinson and baumann, 2015) whereas others view it as a perfect recipe for non-attendance (aldamen et al., 2015) . still, the benefits of lecture capture (table 1 and appendix b) that this systematic review identifies outnumber the drawbacks (table 2 and appendix c). the top five benefits of lecture capture are as follows: it facilitates students' learning and/or revision (41 articles, 57.75%), positively impacts students' academic performance (39 articles, 54.93%), allows students a better work-life balance (31 articles, 43.66%), helps students fill in lecture notes (21 articles, 29.58%), and aids students in revisiting difficult concepts (19 articles, 26.76%). in contrast, the top five drawbacks of lecture capture are as follows: it has an adverse impact on lecture attendance (20 articles, 28.17%), engenders technical difficulties (19 articles, 26.76%)-for both students (e.g., al nashash and gunn, 2013) as well as instructors (e.g., germany, 2012), falls flat due to students' reluctance to watch the lecture recordings (12 articles, 16.90%), attenuates instructor-student interaction (10 articles, 14.08%), and encourages surface learning and/or procrastination among students (9 articles, 12.68%). articles that feature in neither appendix b nor appendix c are those that fail to offer empirical support for any benefits or drawbacks of lecture capture. for example, bollmeier et al. (2010) found no correlation between lecture capture usage and either attendance or academic performance. even more recently, works such as brackenbury (in press), hadgu et al. (2016) , as well as sarsfield and conway (2018) did not detect any relationship between lecture capture viewing and academic performance. looking at the bigger picture, a conundrum is evident. on the one hand, students as research participants almost unanimously voice their support in favour of lecture capture, which they expect will facilitate learning as well as revision-particularly when difficult concepts are involved. these in turn, students believe, will have a positive impact on academic performance. on the other hand, instructors as research participants almost unanimously express concerns about waning attendance caused by lecture capture. in other words, the assumption among most students is that lecture capture is helpful for learning, and among staff is that it takes a toll on attendance. this is why facilitation of learning and reduction in attendance emerged as the most widely documented benefit and drawback of lecture capture respectively. however, research suggests that lecture capture, even if it facilitates learning and/or revision among students, does not always translate to better academic performance. works such as franklin et al. (2011) found students to believe that lecture capture improves grades, but no significant difference was detected in reality. likewise, marchand et al. (2014) found students to believe that lecture capture facilitates learning but instructors saw j o u r n a l p r e -p r o o f little impact on students' performance. in fact, drouin (2014) as well as edward and clinton (2019) identified a negative relationship between lecture capture use and academic performance. drouin (2014) further revealed that even though students had a positive attitude toward lecture capture, they seldom viewed the actual lecture recordings. in addition, despite the general proclivity of the scholarly community to report only significant results and under-report those that are non-significant (kerr, 1998; yukhymenko, 2011) , it is telling that several works have indicated a non-significant association between lecture capture viewing and academic performance (bollmeier et al., 2010; chapin, 2018; euzent et al., 2011; ford et al., 2012; hadgu et al., 2016; leadbeater et al., 2013; mallinson and baumann, 2015; sarsfield and conway, 2018; traphagan et al., 2010) . with respect to lecture attendance, leadbeater et al. (2013) confirmed a negative relationship even though students thought lecture capture would not affect their willingness to attend lectures. similarly, according to marchand et al. (2014) , students believed that lecture capture would not affect their attendance but instructors noted increased absenteeism. nevertheless, several works also reported non-significant relations between lecture capture and attendance (baker et al., 2018; bollmeier et al., 2010; davis et al., 2009; franklin et al., 2011; groen et al., 2016; gysbers et al., 2011mclean and suchman, 2016; nordmann et al., 2019; shaw and molnar, 2011; von konsky et al., 2009; wiese and newton, 2013; williams et al., 2016) . a possible way to reconcile the conundrum lies in finding ways to maximise the benefits of lecture capture while minimising its drawbacks as far as possible. to this end, a few works offered useful insights. bos et al. (2016) showed that students often end up using lecture capture as a substitute for face-to-face lectures. this in turn takes a toll on lecture attendance. edward and clinton (2019) suggested that overreliance on lecture capture as a replacement for lecture attendance can be problematic for student learning. lecture capture seems to breed a sense of dependence among students, who then start to undermine the value of attending the lectures face-to-face. ominously, even lowperforming students have been shown to rely on lecture capture as a substitute rather than a supplement to live lectures, thereby taking a toll on their attendance (aldamen et al., 2015) . nevertheless, according to brooks et al. (2014) , students who watched lecture recordings throughout the term outperformed those who did not. sarsfield and conway (2018) alluded to the fact that high-achieving students tend to view the recordings for learning whereas low-achieving students view only during the exam phase. williams et al. (2012) found a positive correlation between lecture capture and academic performance, but it was evident only among students who also went to the lectures. similarly, von konsky et al. (2009) suggested that higher-achieving students have a proclivity to leverage both live and recorded lectures to reinforce their learning. according to dommeyer (2017), lecture capture has a positive impact on academic performance but it works best when it supplements rather than replaces a lecture. interpreting these insights in conjunction with one another, lecture capture seems to work well if and only if students use the technology as a supplementary resource-rather than an alternative-to live lectures (bos et al., 2016; brooks et al., 2014; dommeyer, 2017; edward and clinton, 2019; sarsfield and conway, 2018; von konsky et al., 2009; williams et al., 2012) . it is probably time that this point is clearly communicated to students. instructors have a huge role to play too. if they would like students to attend lectures as well as view the recordings, the course content should be strategically designed so that there is sufficient incentive on both fronts. with academia almost en route to becoming netflixised, the reason why students should be attending the face-to-face sessions needs to be carefully thought through. this paper sought to clarify the literature on the use of lecture capture in university higher education. a systematic literature review was conducted with 71 articles in the final sample. the findings suggest that most works focus on the us, the uk and australia. the most studied stem subject area in the sample is biological sciences. in contrast, the most studied non-stem subject area is business/economics/management. in terms of methods, descriptive and exploratory research dominates the research landscape while causative evidence is relatively limited. the literature includes two overarching research streams: while one focuses on the benefits of lecture capture, the other focuses on its drawbacks. the most widely documented benefit is that lecture capture facilitates learning, and the most widely documented drawback is that it hampers attendance in face-to-face lectures. the boundary of these findings is however limited to the commonly studied contexts of the us, the uk, and australia-particularly for undergraduate students who study biological sciences and business/ economics/management. caution is advocated in generalising the findings beyond these boundary conditions. by conducting a systematic review of lecture capture at a point in time when technology is increasingly infiltrating face-to-face lectures, the paper makes two key contributions. first, the critical insights that the review offers will enable educators to better understand how students use lecture capture. instructors will have a richer understanding of how lecture capture is helpful on some fronts, and counter-productive on others. instructors are urged to reflect on why students should be attending the face-to-face sessions, and thereafter also watch the recorded lectures (cf. section 5.1 for detailed implications for practice). second, it contributes to the academic discussion on the use of lecture capture in higher education by identifying several under-investigated contexts (e.g., cross-country analysis), methods (e.g., experiments), and themes (e.g., instructors' lecture capture usage). with an attempt to shift existing paradigms, the paper hopes to ignite a body of research that can potentially reconcile students' perception of lecture capture, instructors' perception of lecture capture, and the reality of lecture capture (cf. section 5.2 for detailed recommendations for future research). these contributions, however, need to be viewed in light of the limitation that only articles in english were considered in this systematic review. hence, the paper does not incorporate the essence of other works on lecture capture in institutions where the medium of instruction is not english, in which case the results stand a good chance to have been published in non-english outlets. the paper has implications for teaching practices in university education. the systematic review offers a better understanding of how students use lecture capture than what any single empirical study would afford. as such, instructors and educators could use this understanding as a basis for a more effective integration of lecture capture in teaching. they could use the findings on what is helpful (cf. table 1 and appendix b) and what is counterproductive (cf. table 2 and appendix c) pertaining to the use of lecture capture to better utilise the technology as an educational resource. furthermore, students should be explicitly told about the benefits and the drawbacks of lecture capture. one cannot assume students to be aware of the best practice by default. it is the responsibility of instructors and educators to raise awareness among students that lecture capture needs to be utilised as a supplementary resource rather than an alternative to face-to-face lectures. this would ensure that they make the most of the technology while obviating its drawbacks. if students are required to attend face-to-face lectures and also watch the recordings for knowledge building, watching lecture capture might as well be added to their personalised timetables outside the face-to-face contact hours. this may offer them a sense of clarity and structure regarding what they should be doing with the recorded lectures. for scholars to conduct lecture capture research using online logs, there are implications for lecture capture technology too. it is misleading that students who merely click on a link are considered to have accessed a lecture recording (chapin, 2018) . perhaps, tools such as panopto need to track the duration for which students are active and inactive. it could also capture which parts of the video are viewed once, which parts are viewed repeatedly, and which parts are skipped altogether. these will constitute useful data for future inquiry. the paper calls for further research to expand the contextual boundaries of the extant lecture capture literature beyond the us, the uk and australia. this will help bring the literature more in line with the present trends of lecture capture usage worldwide, thereby bridging the literature-practice gap. more studies involving postgraduate students, executive students, and part-time students will also be helpful, particularly in subject areas that are currently under-represented in the literature (cf. figure 4) . where possible, lecture capture research needs to consider the perspectives of both students and staff in conjunction. this will help paint a more holistic picture and afford a greater degree of triangulation compared with current works, most of which myopically focus on only students (cf. appendix a). for research involving staff, care should be taken to recruit not only junior academics but also experienced instructors-whose voices are not widely echoed in the current literature. research comparing data from multiple institutions is particularly recommended to better understand how lecture capture is received by a diverse range of students and staff, and whether any systematic differences exist as a function of contextual factors. for this purpose, cross-institutional collaborations among academics could be an ideal way forward. cross-national analysis has also been far and few hitherto. as indicated earlier, only one article reported a comparative analysis between two countries (trenholm et al., 2019) . plugging this research gap may require wider cross-country collaborations among academics. in addition, works such as bos et al. (2016) have made their data available under a creative commons by-nd 3.0 licence. this could also be a step in the right direction in order to foster cross-country and cross-cultural lecture capture research. all in all, crossinstitutional collaborations, cross-country collaborations, and open data sharing are poised to add a real impetus to the lecture capture research landscape, without which the question of how culture shapes lecture capture usage and attitudes will remain unanswered. moreover, scholars interested to publish empirical works in the field of lecture capture are urged to provide as much contextual details as realistically possible. peer-reviewers of journals too have a crucial role to play as gatekeepers. since the field is very much sensitive to contextual factors (caglayan and ustunluoglu, in press; nordmann et al., 2019) , lack of sufficient details thwarts a systematic interpretation of the results. for example, several articles in the systematic review did not specify the duration of lecture recordings, the number of lectures or the shape of the academic year (e.g., semester or trimester), and undergraduate student categorisation by year of study where applicable. findings cannot be assumed to be the same regardless of such factors. in fact, these might as well moderate the relation between lecture capture usage and academic performance as well as that between lecture capture usage and attendance-a significant omission in the current literature that warrants scholarly attention. as technology continues to evolve and students keep on adapting, this paper recognises a need for replication in lecture capture research, especially the quantitative studies with small sample size. currently, owing to the relatively limited number of studies with little methodological consistency-that too with occasionally insufficient contextual details, the true picture of lecture capture remains blurred. in this vein, both close replication and differentiated replication could be pursued (uncles and kwok, 2013) . close replication facilitates verifying previous results in almost similar settings. differentiated replication allows for variations at conceptual, methodological and/or substantive levels to test the generalisability of previous results. such replications should be situated within broader pedagogical theories and frameworks. this is important because existing works mostly present descriptive and/or exploratory research fuelled by practical concerns rather than theoretical debates. the literature will be enriched by an understanding of lecture capture through the lenses of various learning paradigms such as behaviourism, cognitivism and constructivism (bichelmeyer and hsu, 1999; ertmer and newby, 1993; radianti et al., 2020) . furthermore, the replications should control for all possible contextual factors that the original studies might have missed out. even if the replications yield statistically nonsignificant results, they should not be treated as theoretically insignificant. otherwise, the j o u r n a l p r e -p r o o f practice of reporting only significant results will never allow the true picture of lecture capture to emerge. given the often-held apprehension that journals are hesitant to publish sole replications, scholars are encouraged to augment the replicative efforts with additional studies to pass muster. on the quantitative front, experimental research is encouraged so as to obtain causative evidence. big data analyses and computational techniques could be explored too. if algorithms can predict academic performance from lecture capture usage, it would be a helpful tool for educators to identify at-risk students on the fly and support them accordingly. on the qualitative front, more in-depth interviews could be conducted. emerging data collection approaches such as screencast videography could be employed to better appreciate students' lived digital experience of viewing lecture recordings (kawaf, 2019). nuances could be teased out between when lecture recordings are watched on laptops versus mobile devices. differences between lower-achieving and higher-achieving students as well as those between frequent attendees and non-frequent attendees are worth investigating. at the undergraduate level, longitudinal studies-either quantitative or qualitative-will also be useful to shed light on how students mature from their first year of study to the final year. furthermore, the paper calls for more lecture capture research involving instructors. here are a few possible research questions that are under-explored thus far: to what extent do instructors utilise lecture capture to reflect on their teaching practices? how does instructors' teaching performance (e.g., student feedback as proxy variable) with lecture capture differ from that without lecture capture? in what ways do instructors' use of lecture capture for teaching purposes evolve over time? how does instructors' lecturing style, personality and other individual differences affect their teaching performance as well as students' lecture capture usage? are there any differences in attitudes toward lecture capture between junior and senior academics, given that the latter lies relatively more toward the wrong side of the digital divide? to what extent do instructors' teaching performance and students' lecture capture usage vary when the technology captures only audio versus both audio and video? in what ways do instructors' and students' use of lecture capture differ when the technology is rolled out under regulatory pressure versus when it is embraced voluntarily? by identifying these research opportunities, the paper hopes to pave the way to reconcile students' perception of lecture capture, instructors' perception of lecture capture, and the reality of lecture capture. table 3 summarises the dominant research trends and the future research directions in light of the three objectives that this systematic review sought to achieve. finally, as the covid-19 pandemic and the social distancing measures continue to accelerate the push toward online teaching and learning, how instructors and students adapt their use of lecture capture-both in vitro and in vivo (pale et al., 2014)-is worth investigating. instructors, who were once reluctant to adopt lecture capture, may become more open to using the technology. students' attitudes, motivations, and behavioural engagement may also change in unprecedented ways. to better understand the educational upheaval linked with the pandemic, pre-versus mid-versus post-covid-19 comparative j o u r n a l p r e -p r o o f studies on lecture capture are essential. this could be a particularly exciting research direction among scholars who have already accumulated a wealth of data prior to the covid-19 outbreak. j o u r n a l p r e -p r o o f to conclude, while the issues raised by lecture capture are complex, the researcher would like to argue that its benefits (cf. appendix b) outweigh its drawbacks (cf. appendix c). nevertheless, students need to be explicitly told how to make the best use of lecture capture. in addition, instructors need to plan content and delivery in such a way so that either of lecture attendance or lecture capture viewing cannot steal the thunder from each other. that said, the paper invites interested scholars to challenge this argument forwarded by the researcher. in the wake of the covid-19 outbreak, if it manages to stir a healthy debate among the scholarly community regarding the use of lecture capture in universities, it would have served its purpose. the 71 notes. 1 = more than one institute involved. 2 = more than one country involved. 3 = site not explicitly specified but inferred from the author(s)' affiliation. fgd = focus group discussion. stem = science, technology, engineering and mathematics subject. the last column indicates the number of data points within parentheses for each data source, provided it was clearly indicated. lecture capture technology and student performance in an operations management course the netflixisation of academia': is this the end for university lectures?, the guardian students' satisfaction and valuation of web-based lecture recording technologies the impact of lecture capture on student performance in business courses inclusive learning and teaching in higher education. the higher education academy impact of class lecture webcasting on attendance and learning investigating undergraduate mathematics learners' cognitive engagement with recorded lecture videos designing research with in-built differentiated replication transforming our world: the 2030 agenda for sustainable development evaluating intelligent interfaces for postediting automatic transcriptions of online video lectures in words and deeds: parental involvement and mediation of children's television viewing use of lecture capture in undergraduate biological science education the impact of online lecture recordings on student performance lecture capture podcasts: differential student use and performance in a large introductory course online lecture recordings and lecture attendance: investigating student preferences in a large first year psychology course where is current research on blockchain technology?-a systematic review lecture capture with real-time rearrangement of visual elements: impact on student performance students' interest in social studies and negotiation self-efficacy: a meta-analysis of the globaled project should lecture recordings be mandated in dental schools? two viewpoints quality of health information for consumers on the web: a systematic review of indicators, criteria, tools, and evaluation results appendix c j o u r n a l p r e -p r o o f• most lecture capture research has been conducted in the us, the uk, and australia.• biological sciences & business/economics/management are well-studied subject areas. • theory-driven research gleaning causative evidence through experiments is limited.• the assumption among most students is that lecture capture is helpful for learning. • the assumption among most staff is that lecture capture takes a toll on attendance. this is a single-authored work. the author thanks chaturi liyanage who was the independent coder for this research. key: cord-272792-gmsm0lv0 authors: ripoll, vanessa; godino-ojer, marina; calzada, javier title: teaching chemical engineering to biotechnology students in the time of covid-19: assessment of the adaptation to digitalization date: 2020-11-11 journal: nan doi: 10.1016/j.ece.2020.11.001 sha: doc_id: 272792 cord_uid: gmsm0lv0 with the global outbreak of covid-19 in march 2020, there was an immediate shutdown of face-to-face classes and a sudden shift to on-line learning. confinement required finding innovative approaches to teaching and student assessment. this paper aims to share the experience of adapting the course in biochemical engineering, part of the biotechnology program at francisco de vitoria university (madrid, spain), to remote learning. a sequence of collaborative learning activities, with active student participation, was designed to replace the traditional mid-term exam. activities were carefully implemented, considering the range of learning styles. engineering skills, transversal competences and higher-order thinking skills were fostered through these activities. the analysis of the teaching/learning experience was based on teacher observations, academic performance and student surveys. all indicators showed that the adopted methodology had a positive impact of student performance. student participation, especially among those repeating the course, also improved. furthermore, students gained a more accurate and positive perception of the link between chemical engineering and biotechnology, which may have a favourable impact on the teaching of bioreactors in the coming academic year. these courses are the "biotechnological tools" module within the degree in biotechnology at the ufv, with biochemical engineering serving as an introductory course to provide students with the foundation necessary for the bioreactors course. the aim of the biochemical engineering course is to help students (ufv, 2019) i) to interpret and apply the fundamentals of enzyme kinetics, understanding their importance and applications in industry, ii) to understand the basic fundamentals of engineering in the design of biotechnological processes, and iii) to interpret and apply relevant parameters regarding transport phenomena and mass and energy balances in bio-industrial processes. student assessment was as follows:  50% of the final mark will be the average of the two mid-term exams. all students must take the mid-term exams as part of their continuous assessment. students not sitting these exams will be assessed through a final exam.  25% of the final mark will be the results of practical, laboratory work.  25% of the final mark will be from their continuous assessment, that is:  15%: the average mark from continuous assessment where 70-80% corresponds to the best marks among the different course assignments.  10%: the higher mark of the first and second mid-term exams. students not sitting these exams will lose this part on their final mark. underlying educational principles the course is based on a constructivist and connectivist pedagogical model. the course content and assignments are designed to expose students to a variety of tasks (problemsolving, case studies, video presentations, etc), based on a constructivist methodology; that is, through a series of tasks and assignments, students acquire new knowledge through a flexible and interactive learning process. with guidance from the teacher, students build on their previous knowledge and experiences, connecting with and learning from their fellow classmates in a process of acquiring new skills and competences (torras, 2015) . by using a broad variety of tasks and assignments, students are assessed as individuals, considering their different levels of motivation, attitudes and ways of learning (felder and brent, 2005) . the methodology of the course is based on a view of learning as a collective and collaborative process (torras, 2015) , incorporating a connectivist approach which understands learning as a j o u r n a l p r e -p r o o f consequence of building nodes and links between experts, knowledge repositories (databases, libraries or other information sources) and learners (siemens, 2004; guerrero and flores, 2008 ). within the context of the covid-19 crisis, the teacher takes on a larger role, beyond that of a learning facilitator, instructing students in their learning objectives (stephenson & sangrà, 2013) . the collateral effects of covid-19 and the experience of lockdown demanded that teachers take a more active role in guiding and accompanying students along their learning path. students must be the protagonists of their own learning process; that is, playing an active role and making their own decisions throughout the course (stephenson & sangrà, 2013) . biochemical engineering was initially designed as a face-to-face course within the biotechnology degree, and the conventional methodology of previous years needed to be replaced to address the realities of remote learning. students maintained their active role in developing higher-order thinking skills, based on bloom's taxonomy (analyse, evaluate, create; etc) (nikolić & dabić, 2016) , given the strong connection between cooperative learning and the development of higher-order thinking skills (davison et al, 2014) . looking forward, and regarding the acquisition of practical competences, it is essential to foster the development of these higher-order skills among students, including the attitudes and habits of thought to be expected among scientists and engineers. teachers must create and maintain the proper environment which encourages students to develop these skills (felder & brent, 2004 a & b) . the present work describes how teaching and student assessment methods in the biochemical engineering course were adapted to the remote learning environment during the covid-19 pandemic. this involved the development of a wide range of learning activities and a new assessment system to meet the demands of online learning. the sequence of course activities was designed to achieve the following goals: i) to change student perception of the difficulty of chemical engineering and so increase student interest j o u r n a l p r e -p r o o f and engagement, ii) to motivate students to study and facilitate the understanding of concepts through constant feedback, and iii) to seize the attention and boost the participation of repeating students who often do not attend lectures and drop out of the course very early. the activities described were designed to reinforce student knowledge, skills and competences using a practical, hands-on and collaborative methodology (torras, 2015; stephenson & sangrà, 2013) . the aims of these learning activities were as follows: 1 students were asked to work in groups, inventing a problem where the fundamentals of mass balances are applied. collaborative learning helps students to connect their knowledge and experience in biotechnology with chemical engineering, building their own knowledge working as part of a team while helping their peers to learn. 3 students were asked to complete concept quizzes in order to check their level of conceptual understanding and assess their calculation skills and knowledge. groups of 4 students were given a problem to be solved within a limited period of time, developing their ability to discuss, defend and communicate their ideas effectively, while applying appropriate mathematical methods to problem-solving. finally, students were asked to record the solution to the problem in order to assess their understanding, the application of mass balances equations and problem-solving capacity, as well as their ability to synthesise and summarise information. the participants of this work were undergraduate students of biotechnology enrolled in the compulsory course of biochemical engineering during the 2019/2020 academic year. the methodology was applied in two classes (1 and 2). class 1 consisted of students of the degree in biotechnology (37.5%) and students attending a double degree in biotechnology and j o u r n a l p r e -p r o o f pharmacy (62.5%). class 2 consisted entirely of students in the degree in biotechnology (100%). a description of the classes is provided in table 2 , including sample size, student profile student (first or repeating students) and gender distribution. participation refers to the level of student involvement, showed as percentage of student taking part in course activities. this reflects the response of students during the covid-19 crisis to the alternative learning and assessment system. repeating students, enrolled in the course for the second time after failing the course the year before, often have scheduling problems in attending and completing the activities part of the continuous assessment. these students are therefore able to be assessed only based on their marks in a final exam. however, despite all the educational challenges posed by the covid-19 crisis, remote teaching was able to encourage these students to participate in these activities, improving their learning experience. the ufv quality management department conducted a mid-term survey of the students to learn their opinions on the quality of teaching and the difficulty of the course. the student´s perception of the difficulty of chemical engineering is shown in table 3 . looking to adapt teaching and assessment to the lockdown situation, an innovative approach was designed for the second mid-term exam on mass balance. this methodology was based on cooperative learning, with students taking an active role and taking into account the diversity of learning styles. a sequence of learning activities was designed to assess not only student´s knowledge, but also to improve their transversal competences and higher-order thinking skills. the complete sequence of learning activities, assessment and weighted grades are shown in table 4 . step the mass balances unit was taught over 5 weeks of which 3 were remote learning. to facilitate independent learning adapted to student's particular circumstances during lockdown, each lecture was recorded. videos of problem-solving were also recorded. step 1a creating a problem in cooperative learning groups step 1a consisted of building a problem for the mass balances unit. students worked in teams of 4 and were allowed to choose their own team. the proposed problem had to meet the specifications of the teacher and also be related the field of biotechnology. these j o u r n a l p r e -p r o o f specifications are summarised in table 5 . after week 1, each group submitted their assignment, including a description and solution to the problem. the teacher was available to clarify student´s questions during the assignment, addressing misconceptions and providing additional information. the task focused on developing hands-on learning, with students working in small-groups toward the same goal. creating a problem using cooperative learning groups was an exploratory task. students were encouraged to contribute examples and discover connections between chemical engineering and their specific experience in biotechnology. the proposed methodology invited students to build their own knowledge collectively. as noted above, the aim was to apply a practical, hands-on and collaborative approach (torras, 2015; stephenson & sangrà, 2013) . thus, students were asked to produce the content to be taught to their classmates. it is also an opportunity for students to internalise concepts, understanding and helping others to learn. reactor size and rate must be provided so that generation can be established. the process does not work under steady state the assessment system consisted in a rubric, providing a specific guide about the assessment criteria and expectations and criteria. the rubric for the evaluation of the activity is shown is j o u r n a l p r e -p r o o f step 1b using the created problem for learning the specifications were designed to provide a wide-ranging set that addressed the main types of problems studied previously in the unit. the set was available in the virtual classroom immediately after the submission deadline for the assignment in step 1a. biotechnology students tend to find the mass balances unit particularly abstract and confusing. for this reason, some of them opt to memorise solutions rather than truly understand them for future application. the objective of step 1b is to provide students a useful tool to facilitate learning by presenting chemical engineering through familiar examples. a forum was also included in the virtual classroom to provide students with a tool to chat and discuss the proposed problems. step 2 concept quiz step 2 was an individual test consisting of 12 questions on the various topics in the unit. during the 2019/2020 academic year, the virtual platform at ufv was moodle tm . a questionnaire module was employed to create a bank of questions with similar content for each category. the concepts, types of question and weighting is shown in table 7 . multiple choice questions allowed a single answer (only one answer could be chosen). correct answer got full marks (100 %) and wrong answers were penalised with negative mark ( 25%). numerical questions accepted numerical answers with an error of 5 %. units also needed to be j o u r n a l p r e -p r o o f specified. numerical questions were used to grade the number (90 %) and the unit (10 %). to avoid cheating or copying between students, each student was given a different quiz. tests were designed so that questions in different categories were randomised. moreover, questions were arranged on separate pages and browser's back button was deactivated. the aim of the concept quiz was to evaluate student comprehension and the test was at lowintermediate level. however, the short duration, 40 minutes, raised the difficulty to score high marks. therefore, students needed to have competent calculation skills and applicable knowledge to pass the test. a points-based assessment system was employed, with each question scoring 1 point. the final mark was the sum total of points divided by the total points (12). step 3a finding the solution to a given problem finally, the teams of 4 students were required to solve a given problem in 90 minutes. each team received a different problem but all at the upper intermediate level of difficulty. the teacher prepared a blackboard ultracollaborate tm session per group. students joined their session to discuss how to solve the problem, draw a process flow diagram, apply and resolve mass balances while answering the questions provided. during the execution period, the teacher was available to clarify any questions. the aim of this activity was to create a forum for debate about the problem and to find a solution through collaborative learning. step 3b recording the solution to a given problem immediately after step 3a, students were assigned activity 3b: recording the solution to the given problem. each student had previously checked the proper functioning of their recording device (mobile, tablet or webcam). students were given 30 minutes to record their videos and an additional 30 minutes to prepare and submit the assignment. for student identification, a short introduction was required at the beginning of the video. the recommended length of the video was 10-15 minutes. videos that were too short or too long were penalised. the assessment system was outlined in a rubric previously provided to students, shown in table 8 . understanding processes, applying mass balances equations and solving the proposed problem accounted for 60% of the mark. digital content production and oral communication were also assessed (20 % each item). transversal competences can be considered generic and applicable skills that students acquire throughout their degrees. the development of transversal competencies complements technical-scientific skills and ensures graduates have the profile to act as competent professionals in the future (sa & serpa, 2018) . however, course programs at the ufv do not specifically include these competences. given their importance to student learning, the chart of learning activities provided below indicates the transversal competences and higher-order thinking skills which may be difficult to impart through remote learning. table 9 summarises the main competences developed at each step. applying are lower-order thinking skills. these proposed activities encourage students to think "out-of-the-box", to be creative and innovative. to achieve this, activities involve projects which assess the synthesis of knowledge and creation of new understanding. step 2 is a more conventional teaching approach, assessing primarily lower-order levels of thinking. an online survey was conducted to evaluate the experience of the second mid-term exam and student's view of the teaching approach, their development of transversal skills and other relevant aspects. the survey was constructed using google forms®. students were given 9 j o u r n a l p r e -p r o o f statements, indicating their level of agreement using a likert scale (1 strongly disagree; 2 disagree; 3 neutral; 4 agree; 5 strongly agree). the survey structure is shown in table 10 . all enrolled students were invited to take the survey at the end of the term. 3.i the created problem´s set was a useful tool in the study of the subject. recorded lectures and problem-solving videos were available to students during the semester. as there is a record of the views and reviews of these videos, the availability of these recordings was considered to have a positive effect on students learning. videos were a useful tool allowing students to view them at them at any time. in fact, few students requested teacher support to resolve their questions dealing with mass balances, whereas, in previous years, most students asked teachers help several times throughout the semester. student results on the second mid-term exam was more successful and higher than expected. the majority of repeating students took part in the activity, indicating that the aim of encouraging their participation had been achieved. regarding problem-creation and cooperative learning, all the students completed the task properly. some teams formulated highly creative problems while others found excellent examples within the field of biotechnology. problem-solving was also required in the assignment, but the format was not fixed. some teams decided to record a video explaining the solution, often producing engaging, high-quality work. with regards to the forum provided step 1.b, this was not widely used; only by 2 students from class 2. the forum was created as a support tool, but students did participate, perhaps because this activity did not count towards their final mark. after the concept quizzes, students reported they were not given enough time to finish the task. they also complained about calculation questions because only the final results were marked. however, most students passed the concept quizzes and a few scored outstanding marks. calculation questions were new to the students and they reported a lack of confidence in this area. the final exams also included this type of questions relating to other units and there were no significant complaints about the marking system. all the students joined the blackboard ultra collaborate tm session to work on finding the solution to a given problem in groups. they worked on this task for 90 minutes without incident. however, several students experienced some difficulties in completing the exercise. submitting video recordings was a difficult assignment for some students, as they had several technical problems (sounds failures, not enough available memory, mobile/tablet connection failures, etc). file uploading took longer than expected. in addition, 3 students in class 1 did not submit their recording although they had done previous learning activities. lockdown was a stressful experience in terms of isolation and deprivation and students needed to adapt using the resources they had available. in the 2019/2020 academic year, the virtual platform did not offer an embedded video recording application. fortunately, this will be available from the current academic year and will certainly be a useful tool to develop innovative learning activities. the overall instructor perception was that video recordings are a good formative activity. students were asked to organise and communicate their knowledge, developing a deeper level of thinking. in spite of the stressful situation, student's final results were good. a series of frames the problem-solving videos of different students are provided in figure 3 . the results of students in class 1 were quite similar to those of students in class 2 in terms of average marks and standard deviation. therefore, the outcomes of learning activities have been addressed and discussed jointly as a single class. the results of problem-solving through cooperative learning (step 1.a), based on the assessment rubric (table 6) , are provided in figure 4 . the sample size was 83 participants. all students passed step 1.a, obtaining a mark higher than 5. around 75% of students scored higher than 7.5 points. the average mark for this learning activity was 8.2 ± 1.5. more specifically, the results obtained in the wording of the problem were between 5-10 points, mostly between 7.5 and 10 (63 out of 83 students), and only 8 students scored below 5. the results for the specifications followed the same trend. however, the results for problem resolution were different. all but 4 students scored higher than 5, and 63% scored between 7.5 and 10 points. these results suggest that students were able not only to correctly formulate their proposed systems, but also to successfully identify and apply the instructions in order to create the problem to be solved, drawing on working in cooperative learning groups (as shown in table 5 ). higher-order thinking skills were put into practice; most student properly formulated and explained their proposal, extracted information and solved the problems. however, difficulties in systematic organisation and applying mathematics were detected. this may be related to the need for students to master not only concept regarding mass balances, but mathematical tools and calculus skills as well. the overall results and the results in each category of the quiz (step 2) are summarised in figure 5 . approximately 75% of students passed the test. specifically, 43 students, out of 83, scored between 5 and 7.5, while 17 students obtained a mark higher than 7.5. a total of 24 students failed the test. the average grade of this activity was 5.9 ± 1.8. the results in each category clearly reveal the most difficult issues for participants. broadly speaking, systems involving deeper mathematical skills proved most difficult for students. particularly, changes in system composition due to transformation (chemical, enzymatic, biological, etc.) increase complexity. likewise, if the problem also involves a batch system, the difficulty is higher, requiring the integration of kinetic expressions. this is consistent not only with a lack of a mathematical foundation within the degree syllabus, but also with a general prejudice or fear of engineering subjects in general, as previously mentioned. the results of step 3b (recording the solution to a given problem) were evaluated using the rubric shown in table 8 . grades obtained are provided in figure 6 . the marks obtained in two of the three items of the rubric (digital content and oral skills) followed the same trend. some 88% of the students scored higher than 5 in all the items indicated. additionally, 71% of students scored between 7.5 and 10. however, 23% of students did not pass the resolution item, which slightly deviated the observed trend. consequently, for the final grades of students, shown below, 93% attained positive results, higher than 5 (as shown on the right side of figure 6 ). therefore, only 7% of students failed the step which consisted in recording the solution to a given problem. the average mark for this learning activity was 7.1 ± 1.5. students have been acquired oral and digital skills as well as the ability to synthesise and abstract information. however, the application of theoretical knowledge and performing the right calculations are the main difficulty for students. in order to determine the effectiveness of innovative methodologies, the final results of the second mid-term exam were compared to the first mid-term exam (figure 7) . firstly, overall participation in mid-term exam 2 increased compared to participation in mid-exam 1. the number of students who passed increased noticeably when the proposed methodology was applied. the percentage of students being exempted from the respective units in the final exam increased from 33% (first mid-term exam conducted with a conventional methodology) to 83% (second mid-term exam using the innovative methodology). in conclusion, the results show that the innovative learning methodology employed led to a slight increase in the student participation. moreover, there was a significant increase in the number of participants passing the mid-term exams, being exempted from the respective units in the final exam (with marks higher than 65% of the scale). this improvement can be explained by a number of factors, including greater motivation among students with the use of new methodologies, a significant improvement in student accompaniment, monitoring and feedback, a better continuation of learning processes, an improvement in student autonomy, the reduction in the perceived difficulty of the subject, variability in learning tools and assessment. j o u r n a l p r e -p r o o f academic year. however, the improvement of results obtained by repeating students was less noticeable due to the lack of connection of this group of students to the subject. comparison between final marks and perceived difficulty by students is shown in figure 8 . around 90% of the students indicated that biochemical engineering is a difficult or very difficult subject. only 3% of the students perceived the subject as easy or very easy. however, final marks show that around 70% passed the course. in addition, 41% of the students obtained a final mark ranging from 7 to 8.9. thus, there is no relationship between both trends. this confirms that biotechnology students are often conditioned negatively towards the engineering field. to change student perceptions of the subject´s difficulty, figure 8 was shown at the end of the term. thus, reaching open-minded students about bioreactors will also lead to improved outcomes and motivating students to change their opinions may result in better student performance. the online survey created to assess the learning experience was divided into the following evaluable parts: i) teamworking activities, ii) scientific video recording, and iii) problem-solving through cooperative learning. some 80% of students (68) completed the online survey. the results are show in figure 10 . in general, it can be seen that students have a positive perception of the three items surveyed. specifically, a positive trend can be seen relating to teamworking activities. however, in questions dealing with video recording, this positive trend is lower when students were asked about any improvement in their digital skills. some 50% of the students perceived an improvement in digital skills due to the activity, 25% of the students were neutral and the remaining 25% concluded that their digital skills had not improved. a positive trend can also be j o u r n a l p r e -p r o o f seen in the question regarding problem-solving through cooperative learning. this positive perception increases when students were asked about the effect of these activities on integrating mass balances concepts and finding a link between chemical engineering and biotechnology. however, some students (around 35%) did not consider this a useful resource to study the subject. the present work describes the experience of using new learning methodologies for biochemical engineering in response to the disruptions caused by the covid-19 crisis. the student perceptions and learning outcomes of these methodologies were evaluated. however, designing the survey is a complex process to accurately measure the opinions, it should be noted that the methodology was only applied to biotechnology students at a single university. the results may differ based on institutional context and participants' academic background. finally, the data collected related to student impressions and academic results were handled anonymously and separate from any personal data. as a consequence, only global conclusions can be extracted; obtaining individual results inferring personal motivations and situations was neither possible nor the aim of this work. this study offers a comparative analysis of results over the last two years. although a longer study would provide more robust results, the substantial differences observed allow conclusions to be drawn regarding the hypothesis of this work. an alternative learning methodology, taking advantage of information and communications technologies succeeded in overcoming the direct consequences of the covid-19 lockdown, motivating students and improving participation for successful learning outcomes both in student performance and perception of biochemical engineering. an innovative methodology based on cooperative learning, focused on the active role of students and considering different learning styles, was proposed in response to the covid-19 j o u r n a l p r e -p r o o f crisis. students were provided, not only with remote lectures and problem-solving assignments, but other activities as well. for instance, the creation of a problem, working collaboratively, building a set of significant problems covering the course contents, answering a quiz on relevant concepts in biochemical engineering, and, finally, finding the solution to practical problems and recording the solution, fostering collaborative learning using created videos. although there was a dramatic change in learning due to covid-19, the application of these innovative methodologies succeeded in aligning course competences and skills, while responding rapidly to student needs, maintaining class dynamics and motivating learning. collaborative learning, including working as part of a team, discussing information, extracting relevant data, and building knowledge together within a common repository, are a great help in building autonomy, effective knowledge assimilation and the acquisition of key competences. this connected and cooperative learning enhanced student participation significantly, despite the challenges of the covid-19 lockdown, the unfamiliar use of remote learning and often difficult personal circumstances. student participation also increased significantly among repeating students. as noted above, a collaborative approach helped students develop creative solutions and high-quality work in problem-solving and also connecting biochemical engineering to the world of biotechnology. in particular, quizzes about relevant concepts in chemical engineering produced very good results although the limited time allowed to complete this task, and questions on calculations and units, were the main difficulties students encountered. the following learning outcomes were achieved through the use of new learning activities:  abstracting to extract relevant information from a practical case using the proper mathematical methods.  being able to obtain information from a practical case for the application of the principles of biochemical engineering.  discussion among classmates about the results and repercussions of a practical case. finding a correct solution to a practical case. considering these learning outcomes, no significant differences can be observed between the studied groups. thus, it appears that the learning sequence was correctly designed to meet the needs of different learning styles, regardless of student background and their area of study. students reported a generally positive perception of the learning experience. cooperative learning was found useful for developing skills during the course while teamworking was rated very positively among participants. globally speaking, in comparison with previous years, before this new methodology was introduced, there was a significant improvement in student performance in the biochemical engineering course. this was particularly the case in student motivation, greater student accompaniment, monitoring and feedback, and developing student autonomy. furthermore, the new methodology is fully transferable to in-class learning. each activity enhances student learning from different perspectives and styles. transversal competences that are often difficult to teach and assess even in face-to-face lessons have been achieved by means of the proposed activities. therefore, applying this methodology in traditional courses may also have a positive impact on the acquisition of these competences. in the future, the technique of peer-review of created problems and recorded solutions will also be proposed to enhance the acquisition of higher-level skills and competences. there are no conflicts to declare. j o u r n a l p r e -p r o o f online and remote learning in higher education institutes: a necessity in light of covid-19 pandemic emergency remote teaching in a time of global crisis due to coronavirus pandemic boundary crossings: cooperative learning, collaborative learning, and problem-based learning the intellectual development of science and engineering students. part 1: models and challenges the intellectual development of science and engineering students. part 2: teaching to promote growth understanding student differences applications, reliability, and validity of the index of learning styles reflections on interdisciplinarity and teaching chemical engineering on an interdisciplinary degree programme in biotechnology teorías del aprendizaje y la instrucción en el diseño de materiales didácticos informáticos the bloom's taxonomy revisited in the context of online tools transversal competences: their importance and learning processes by higher education students connectivism: a learning theory for the digital age. e-learn space fundamentos del diseño técnico-pedagógico en e-learning aproximación conceptual a la enseñanza y aprendizaje en línea. barcelona: universital oberta de cataluya. unesco, 2020. covid-19 education response guía docente de la asignatura de ingeniería bioquímica coronavirus disease (covid-19) pandemic. world health organization key: cord-322168-bi6rw2df authors: taylor, ruth; thomas-gregory, annette; hofmeyer, anne title: teaching empathy and resilience to undergraduate nursing students: a call to action in the context of covid-19 date: 2020-07-12 journal: nurse educ today doi: 10.1016/j.nedt.2020.104524 sha: doc_id: 322168 cord_uid: bi6rw2df nan there is a compelling narrative in research, practice, and education that resilience is an important component in an individual's ability to cope with clinical nursing practice (for example, taylor, 2019) . the need for compassion in practice is emphasised (nhs england, 2019) with (in our view) the implication that nursing often lacks this attribute. patients justifiably expect to be cared for with competence, kindness, and compassion, and the opportunity to demonstrate these qualities is what attracts many to nursing. but when nursing students and nurses are exhausted and burnt-out, it is unlikely that they can care compassionately for others. in addition to the challenges of healthcare practice that student nurses may experience in 'usual' times, the advent of the covid-19 pandemic brings with it a new reality. students nurses are at the frontline of healthcare practice with, in some countries, changes in regulatory practices that enable students to appropriately use their knowledge and skills in ways that go beyond what was previously the norm (nmc, 2020). in this paper we explain how empathic healthcare cultures and constructs such as empathy, emotion regulation, compassion and self-care are relevant to sustaining wellbeing, resilience and effectiveness. we suggest that student nurses need to be supported to learn to better regulate their emotions through self-care practices to prevent distress and burnout, particularly within the context of this global pandemic. this article seeks to address the dissonance that we believe exists: between the ideal scenario that negates the need for individual resilience, and the reality of clinical practice where resilience is required, with some practical strategies that nurse educators can implement to address the challenges. we assert that nurse education needs to place an emphasis on the development of emotion regulation which will go some way towards supporting the development of resilience in student nurses. in these unprecedented times, it is more urgent than ever to support students to develop emotion regulation. j o u r n a l p r e -p r o o f additional unpaid hours, management taking an interest in staff health and wellbeing. the evidence is clear that organisational empathy and direct managerial involvement in the wellbeing of staff correlates with "a positive effect on patient satisfaction and health outcomes" (west et al., 2017 p.3) . even though as we write we are in the midst of the covid-19 crisis, we strongly assert that we should debate these issues as part of the learning from the crisis that will, we hope, impact positively on the culture of healthcare practice. this article underscores the crucial role of nurse educators in the education and development of nursing students to enable them to work effectively and to thrive in volatile contemporary healthcare environments. to meet this call to action, curricula must be evidence-based and address a range of current topics including those identified in box 1. there is a crisis in the nhs workforce supply in the uk (bungeroth and fennell, 2018; buchan et al., 2019) . the nhs 10 year plan confirms that workforce growth has not kept up with workforce requirements, and that staff are "feeling the strain" (p.78) in part due to the large numbers of vacancies. whilst the plan identifies the perceived need to "strengthen and support good, compassionate and diverse leadership" (p.79), it does not clearly acknowledge the level of challenge that staff are dealing with every day and nor does it put forward, in our view, innovative policy solutions that have the capacity to directly address these challenges. what we have seen as a response to the covid-19 pandemic is rapid recruitment to healthcare posts and increasing numbers of hospital beds: not planned as part of a sustainable response to growing demand pre-covid-19, but a reaction to the urgency of the situation. what we also see, in the context of covid-19 are student nurses being asked to work in clinical j o u r n a l p r e -p r o o f there is little in the literature about organisational empathy, although it is a term that in business sometimes refers to the psychology of working with people. when we talk about 'organisational empathy', we are conceptualising this as the ability of the organisation to understand what, in this case, can make the working lives of the staff better (nichols and ojala, 2009 ). whilst of course the organisation itself is not a sentient being, it is made up of people and our argument is that we should work towards a greater understanding of the lived experience of the workforce, in other words the empathic organisation. in order for empathy to be enacted well, the individual must be capable of emotion regulation which is the ability to tolerate difficult emotions when confronted with someone else's suffering without becoming overwhelmed by those emotions. if we over-identify with another person's suffering we may feel the need to get away from the situation, avoid the person, or reduce our awareness of their distress preventing a compassionate response, and potentially leading to empathic distress fatigue (singer and klimecki, 2014) . the term 'compassion fatigue' has been used in the nursing literature to refer to this condition. using functional magnetic resonance imaging (fmri), neuroscientists have shown that inadequate emotion regulation (i.e. blurring of the 'self-other' distinction) means clinicians can experience the distress of others and typically withdraw to emotionally protect themselves. subsequently this can lead to what is commonly termed 'burnout'. burnout has been described as "the loss of the ability to care" (schwenk, 2018 (schwenk, , p.1543 ). the world health organisation (who, 2019) defined burnout as: "a syndrome resulting from chronic work stress that has not been successfully managed. burnout is characterised by three dimensions:  feelings of energy depletion or exhaustion;  increased mental distance from one's job, or feelings of negativism or cynicism related to one's job; and  reduced professional efficacy." we argue that the development of resilience strategies enhances the ability to be empathic, in part therefore preventing empathic distress and burnout. going back to our earlier discussion on organisational empathy, in order to lead to sustainable change within the healthcare system, the empathic organisation can make systemic changes that will address the causes of burnout in the culture and will increase individual resilience. to achieve the latter, awareness and understanding of the relevant theory is required, alongside leadership that is willing to make those changes. this is where education comes in. whilst in the uk and elsewhere there is an educational emphasis on the demonstration of resilience, effective communication, and managing relationships, we believe that there is possibly a gap in that emphasis: the need for education that supports the development of emotion regulation incorporating a highly-developed sense of self and other (personal and professional boundaries). the aim here is to, in part, develop resilience for the realities of practice alongside a desire to change the status quo (leadership) and to be part of a new commonly held approach to developing organisational empathy. nurse educators have a critical role to educate and develop nursing students to be primed to work in contemporary healthcare environments, and in circumstances beyond what we might have understood to be possible in light of this current pandemic. this means that curricula need to address all the areas we defined above. box 1 provides an overview of some of the areas that we believe should be covered within curricula. we acknowledge that, in the context of the current pandemic, it might feel more challenging to consider how education can address empathy. however, we assert that it is more important than ever and will continue to be so when the pandemic is over. j o u r n a l p r e -p r o o f 7 the central point here is the importance of good communication, so we need to teach within a framework of affective and cognitive empathy. teaching and using the evidence-base (e.g. the neuroscience) is vital so that the perception of the importance of this work is articulated through that evidence-base. do we really expect students to 'walk in another's shoes'? to really understand what it is like for any individual in any challenging situation? we don't need to do this to be useful as practitioners especially when we think about the situations that students will be working in as they try to manage the many emotional and practical challenges of covid-19. and this is our over-riding point: that it is so important to create ways of being that enable empathy but ensure othering so that distress and burnout do not occur. awareness of the current evidence that demonstrates that the symptoms of burnout build up over time is crucial for students and qualified nurses to understand so that it is possible that this 'secret issue' can be identified early perhaps by others around those affected. this is why staff need to be educated about the current evidence and the strategies to promote 'routine self-care' and 'urgent self-care' when a major emotional disruption occurs that can trigger distress, withdrawal, exhaustion, and reduced efficacy (burnout symptoms). we need students to feel the impact of empathic actions and of the empathic organisation. we are saying that if educators facilitate learning that models these ways of being, students will see and feel the differences to their own experiences. we are also asserting that, whilst each person does have a responsibility for self-care, it is the organisation that, as a collective, can make transformative differences through organisational empowerment and associated actions. again, developing students who have this understanding and the leadership skills will lead eventually to a critical mass within organisations who actually can make that difference. it is important to acknowledge that additional interventions may be needed depending on other issues in an individual student's life and, whilst we are promoting an embedded curriculum approach that incorporates self-awareness and self-care strategies, there will be times when professional therapy or support is required. journal pre-proof j o u r n a l p r e -p r o o f 8 we have made the case that education plays a vital role in developing students' ability to work well and remain resilient in challenging healthcare environments, including in the context of a global pandemic. the evidence-based approach that we propose is extensive but, we believe, can be embedded within existing curricula. yes, there is new knowledge but much of what we have described relates to how this knowledge is conveyed. resilient nurses will not only remain in the nursing workforce but will also provide higher-quality patient care (west et al., 2017) . we make the case for change and hope to open up debate about how best to do this. a critical moment: nhs staffing trends, retention and attrition. london: the health foundation left to chance: the health and care nursing workforce supply in england contesting the term compassion fatigue: integrating personal resilience as a strategy for surviving and thriving in the face of workplace adversity: a literature review the theory-practice gap: the impact of professionalbureaucratic work conflict on newly-qualified nurses the nhs long term plan understanding the management of sports events volunteers through psychological contract theory. voluntas: international journal of voluntary and non-profit physician wellbeing and the regenerative power of caring contemporary issues: resilience training alone is an incomplete intervention caring to change: how compassionate leadership can stimulate innovation in health care. london: king's fund world health organisation the organisation, and the individual's role in that organisation  strategies for resilience  neuroscience -empathy, empathic distress, emotion regulation, self regulation techniques, compassion, burnout  professional (and emotional) boundaries  empathic communications, increasing emotional awareness, self-awareness, self-reflection, how to perceive others' emotions, and other empathy skills  self-care practices and self compassion to maintain wellbeing including self-care care plans to improve physical, psychological, or spiritual wellbeing  planning for self-care  curious conversations i.e. those conversations to which we go openly and with curiosity so as to understand another's perspective or experience  'straightforward' language for honest interactions one example of a range of impressive resources that can help some of this learning is the 'virtual empathy museum' website with its evidencebased resources: https://theempathyinitiative.org/virtualempathy-museum  the context of healthcare practice including the challenges  organisational empathy and leadership (compassionate, collective)  approaches to listening to the organisation  how to be part of the change  ways of being together in an empathic 'zone'  strategies to address workplace issues identified through listening and empathic / curious conversations e.g. buddy systems key: cord-304678-fcdhnabw authors: beasley, samuel t.; vandiver, beverly j.; dillard, ronald; malone, walter; ott, randy title: the development of an academic engagement intervention for academically dismissed students date: 2020-05-25 journal: innov high educ doi: 10.1007/s10755-020-09510-0 sha: doc_id: 304678 cord_uid: fcdhnabw “reclaim the w” is an academic recovery program at a medium-sized midwestern university that offers undergraduates who have been academically dismissed a chance to reenroll at the institution. in this article we describe the reclaim the w program and its target population of academically at-risk undergraduates. we then outline the first two phases of the process we used to develop an intervention for students in the reclaim the w program, which we call engage to excel (e(2)). in the first phase we collected data from focus groups with academically at-risk students and professional staff who serve these students. this qualitative data informed the creation of the e(2) intervention for students in the reclaim the w program. in the second phase we identified key components of the quasi-experimental e(2) intervention and outlined the student outcomes that will be evaluated during the intervention. finally, we highlight research and practice implications of the e(2) intervention. the reclaim the w program. in the second phase we identified key components of the quasi-experimental e 2 intervention and outlined the student outcomes that will be evaluated during the intervention. finally, we highlight research and practice implications of the e 2 intervention. keywords academic recovery . student success courses . at-risk college students . engagement . academic intervention given the individual and social benefits associated with earning a postsecondary degree (baum, ma, & payea, 2010) , scholars have explored ways to increase the academic success of undergraduates identified as being "at risk" for poor academic outcomes. students who have been placed on academic probation represent one of the target populations among at-risk college students. while there are many types of students who fit into the at-risk student category (e.g., first generation college students, low income students), in using the label of at-risk students throughout this article we are explicitly focusing on students who have been placed on academic probation and those who have been academically dismissed. some scholars have identified the time when students are on academic probation as a "dangerous opportunity" because students and institutional stakeholders only have a small window to institute changes that will keep the student enrolled in school (tovar & simon, 2006) . while the specific guidelines will differ based on the institution, most postsecondary institutions offer students a limited timeframe to improve their academic performance after they have been placed on academic probation. if students fail to attain a minimally acceptable gpa, they are academically dismissed. most institutions cut ties with students who have been academically dismissed, and typically they only allow students to return after they have taken classes at another institution and improved their gpa or after having not been enrolled for at least a semester or two and then reapplying to reenroll at the institution. a dearth of research exists on students who have been readmitted to the institution after they have been academically dismissed and on the academic recovery programs that work with them. this unique group of undergraduates thus represents a population that may require a different approach to facilitate their academic and social success. in this article we focus on reclaim the w (rtw), a unique academic recovery program at a midwestern university. rtw is an innovative program that invites students who have been academically dismissed to return to the university, improve their academic performance, and continue pursuing their degree. we first describe the rtw academic recovery program and the students it has served for the past five years. next, we provide a brief literature review focused on academic interventions for at-risk students. we specifically highlight student success courses as an academic intervention given that these courses have received the most attention in the literature and that they allowed us to use an existing infrastructure to deliver our new intervention. considering the limited literature on academic interventions for students on probation, this article outlines the process our research team, which includes the authors of this manuscript, used to develop a theory-driven intervention for students in the rtw program. it is important to note that this article primarily focuses on the two-phase process we used before delivering the new intervention for rtw students. later studies will focus on evaluating the outcomes of this longitudinal intervention. the first step of this process included four focus groups to capture the experiences of multiple stakeholders, including students, professional support staff, and administrators, that informed the intervention we designed for the rtw students. the research team then incorporated this qualitative focus group data into the creation of an academic intervention called engage to excel (e 2 ). we decided to deliver the e 2 intervention using existing student success courses in the rtw program since this course-based approach utilized existing resources at the institution. the courses provided a contained environment to address a key component of our approach which was to assess the effects and benefits of the e 2 intervention on students in the rtw program. we conclude the article by highlighting the research and practice implications of the e 2 intervention for academically at-risk college students. established in 2014, rtw is an academic recovery program at a medium-sized, public university in the midwest. the program was initiated under the guidance of randy ott, director of the center for academic success programs. rtw was designed to target first and second-year students who were dismissed from the university in the spring semester. at this university students are placed on probation after one semester with a cumulative gpa below 2.0. if they do not obtain a cumulative gpa of 2.0 after being placed on probation for one semester, they will be academically dismissed from the university at the end of their second semester. however, if students earn a semester gpa of 2.0 or above but their cumulative gpa remains below 2.0, they are placed on extended probation and can remain enrolled in the university. students remain on extended probation until their cumulative gpa is 2.0 or higher or they are academically dismissed from the university for earning a semester gpa below 2.0. students have only one path into the rtw program. at the end of each spring semester, rtw administrators receive a list of all students who were academically dismissed from the university. they invite students to apply for the rtw program based on their cumulative gpa being between 1.0 and 1.9. students must submit a written essay detailing the issues that led to their dismissal from the university and engage in a conversation with rtw administrators as a condition of being readmitted. a team composed of rtw administrators and staff evaluates all applications for the program. after being readmitted to the university, students are required to attend an orientation meeting that outlines the policies and performance expectations for the rtw program. all students in the rtw program reenter the university in the fall semester as a cohort. students are required to enroll in one section of a student success course reserved just for them. typically, there have been three to four rtw sections each fall semester with an average of 17 students in each class. since its inception in 2014, 266 students have participated in the rtw program. of these students, 16 (6%) have completed their degree requirements and received their bachelor's degree, while another 17 (6%) have applied to graduate this academic year. of the remaining students, 93 (34%) of rtw students are currently enrolled at the university. the final 140 students have either been academically dismissed or left the university for other reasons (e.g., finances, health issues, transfer to another institution). throughout the initial process of creating the e 2 intervention for rtw students, our research team's goal was to document the critical elements of the intervention. this approach is consistent with calls for higher education researchers to align their practices with student development theory and research as well as to incorporate students' and practitioners' perspectives (kuh, kinzie, buckley, bridges, & hayek, 2006) . prior to the development of the e 2 intervention in 2018, rtw instructors were given freedom to design their specific section of the rtw course. a review of syllabi from previous courses taught within the rtw program reveals that common topics covered in the class included time management, study strategies, test-taking skills, and learning strategies. our team wanted to build upon the academic instruction provided to students in the rtw program over the previous four years. however, our goal was to link the new e 2 intervention to established higher education theory and research as well as contextual information from the university's at-risk students. furthermore, as a multidisciplinary research team, we sought to capitalize on our experiences as faculty members in counseling psychology while collaborating with professionals and administrators in the rtw program and other entities at the university. collaborative academic recovery interventions between academic and student affairs units deliver the best results for students, particularly when working with marginalized students and those who are at-risk for poor postsecondary outcomes (lepeau, 2015) . lepeau argued that these efforts to work with atrisk students should be grounded in available research. we now offer a brief review of the literature on student success courses since our intervention used this course-based approach. while academic recovery interventions can be offered in multiple formats (e.g., individualized advising sessions, workshops, classes), student success courses represent one of the more popular modes of delivery for academic interventions at postsecondary institutions (culver & bowman, 2020) . thus, the literature review focuses on this area. although all students can enroll in and benefit from student success courses, they have been identified as a particularly useful intervention for at-risk students at both two-year and four-year institutions (connolly, flynn, jemmott, & oestreicher, 2017; kimbark, peters, & richardson, 2017 ). generally, student success courses are designed to provide structure and support for students, identify early concerns they may have, and connect them to appropriate on-campus resources (hoops, yu, burridge, & wolters, 2015) . students glean information about goal setting, study habits and skills, learning styles, time management, and test-taking skills to facilitate their success in college. addressing deficits in these areas is critical for academically vulnerable students. much of the available research on student success courses on college campuses has focused on first-year student success courses or seminars, and a mixed picture regarding their effectiveness has emerged (culver & bowman, 2020) . in their meta-analysis of 284 independent samples focused on the effectiveness of first-year seminars, permzadian and credé (2016) found that student success courses had a small positive effect on students' first year gpa and retention. the authors noted that, while this difference was statistically small, it has important implications for interventions within an educational context. likewise, in their synthesis of over 1,800 empirical studies of undergraduate students' experiences, mayhew, rockenbach, bowman, seifert, and wolniak (2016) found that first-year student success courses had a stronger effect on academic achievement and retention for academically underprepared students compared to their peers who reported stronger academic preparation for college. mayhew et al. revealed that academic interventions are more effective if they pair these courses with tutoring or support services outside of the classroom. other scholars have contended that, while student success courses appear to positively affect some student outcomes, they may not exert a significant long-term effect since students may stop using the information they have learned and return to their old patterns of behavior after completing the course (permzadian & credé, 2016) . additionally, these researchers have argued that few studies explore how participation in student success courses influences other academically relevant variables (e.g., students' engagement). examining the effects of student success courses on outcomes other than gpa and retention, particularly for academically atrisk students, will help to assess the value of these courses. despite the extensive literature on student success courses (engberg & mayhew, 2007) , there is limited research on these courses and college students on academic probation (mcgrath & burd, 2012) . while sparse, extant studies present limited but promising results. in their quasi-experimental study with students on probation enrolled in a student success program at a large, public university, mcgrath and burd (2012) found that students who participated in the student success course had higher rates of getting off academic probation and evidenced higher persistence and graduation rates compared to their peers who had not taken the course. lizzio and wilson (2013) reported that providing structured feedback to firstyear students who had failed coursework contributed to their achieving both higher short-term success (i.e., improved grades on in-class assignments) and long-term success (i.e., passing the course). faculty members who provide structured feedback to academically at-risk students can help them identify specific strategies to improve their performance. other research shows that student success courses at 2-year institutions were more effective than those offered at 4-year institutions because they target the higher number of academically underprepared students at these institutions who most need academic assistance (permzadian & credé, 2016) . these studies indicate that courses for students on probation may be effective but that more work is needed to identify the critical elements of these courses that assist at-risk learners. while studies on using student success courses to deliver academic recovery interventions for students on probation is sparse, even less work is available that explores academic recovery interventions for academically dismissed students after they have been readmitted to the institution. most institutions cut ties with students after they have experienced academic problems for at least two semesters and have not adequately improved their academic performance. as such, readmitted students represent a unique population that has not received attention within the extant literature even though they may require a different approach to facilitate their success. this study outlines our effort to address this gap in the literature. purpose this brief literature review highlighted the limited information on student success courses for students on probation (hensley, wolters, won, & brady, 2018; lizzio & wilson, 2013; mcgrath & burd, 2012) as well as the paucity of studies on students who have been readmitted to their institutions after being academically dismissed. to address this gap in the literature, we outline the first two phases of the process our research team used to create an academic intervention for students in the rtw program. after receiving irb approval, our primary goal for the first phase was to glean more insights about the experiences and needs of academically at-risk students who attend the university and identify gaps highlighted by students and professional staff who are involved with academic support programs for at-risk students. this approach was designed to incorporate valuable contextual information from the perspectives of multiple stakeholders (i.e., at-risk students enrolled in academic support programs at the institution, professional support staff, and administrators) about their experiences at the university. it also allowed us to build content into the intervention that would address the unique challenges students in the rtw program faced while pursuing their undergraduate degrees. prior to collecting focus group data, the study had received approval from the institution's human subjects review board. participants for this study were derived from three key groups of stakeholders (at-risk students attending the university, student support professionals, and administrators who served these students). students who could provide insights about their experiences participating in academic recovery or student success programs were recruited from three academic support programs at the institution (collier, parnther, fitzpatrick, brehm, & beach, 2019) . these participants for the focus groups were recruited from these programs since their demographic profile (i.e., first generation status, ses background, race/ethnicity, level of academic preparation) mirrored the backgrounds of students who enter the rtw program. we invited 120 student participants from these three programs to participate in the focus groups, and 26 students responded for a 21% response rate. the size of the focus groups ranged from 4 to 13 participants. fifteen persons identified as women, and 10 identified as men, with one individual not reporting this information. in terms of racial make-up, the sample included 16 african american/blacks, 3 afro-latinos, 2 latinx, 2 white, 2 multi/biracial, and 1 middle eastern student. the second author facilitated the student focus groups, which lasted an average of 78 min. we also recruited student support professionals at the university and administrators for the rtw program to participate in a separate focus group, which included four women and three men. we had invited all 15 professionals who served students in four academic support programs to participate, which resulted in a 47% response rate. these individuals reported having a minimum of two years of experience in providing services to students at the institution. the first author facilitated this focus group, which lasted 84 min. we recruited participants for the student focus groups by sending a recruitment email to administrators from their respective academic support programs. these administrators then forwarded the recruitment email to all of their students. we recruited the administrators and professional staff by directly inviting them to participate in the study via email. we offered separate time slots for the three student focus groups and the professional staff focus group which allowed participants to select an interview time that fit their schedule. we developed two semi-structured interview protocols for the focus groups. the student focus group protocol contained eight questions, and the support protocol consisted of seven questions. both protocols were developed using available literature on academic support programs and supporting at-risk college students. the first author used thematic analysis to analyze the focus group data (braun & clarke, 2006) . using the process outlined by braun and clarke (2006) , he analyzed the data by grouping similar ideas or meaning units to develop preliminary themes. after these initial codes were developed, he recoded the data to eliminate redundant themes. in a follow-up round of coding, the first author reanalyzed the data to ensure the final thematic structure accurately reflected the data from the focus groups (braun & clarke, 2006) . two doctoral students then reviewed the data to ensure clarity and thoroughness of the thematic structure. following this review process, the first author made minor changes based on their feedback prior to writing up the data. students' perceptions of their inadequate preparation for college represented the first theme in the data. twenty-two of the 26 students reported that they did not feel prepared for college. one participant stated that he realized he was not ready for college on "the first day of orientation." the students acknowledged that they struggled to adjust to the academic demands of college compared to the academic standards of their high schools. for example, one participant explained that she had been informed by teachers in her high school that she was being prepared for the expectations of college professors; but she later realized that their preparation had set the bar too low: "they would claim this is a college-going culture, and that was just them saying 'stay off of your phone' or 'come to class on time'." students argued that the focus on behavior management at their high school had failed to provide them with the requisite knowledge and skills needed for postsecondary work, which made the transition to doing college-level classes more challenging and partially explained their early academic struggles on campus. given that students in the rtw program had been academically dismissed, these initial academic struggles clearly persisted across their time at the university. in light of this perceived and apparent poor academic preparation, students in the focus groups reported that access and use of university resources was important for their academic success. these responses accounted for the second theme from the student focus groups. one student stated: i've been to the [masked name], and i love it because i have a peer coach. i'll meet with her twice a week. i would not have passed math first semester if it wasn't for her. she's my same peer mentor this semester. it's that one-on-one; i need that personally. i'm selfmotivated, and i'll do it myself, but just that extra pushing, that extra help really helps me. students explained that they had been directed to use various campus support services (e.g., writing center), but they noted that they did not consistently access these resources. sometimes when students did engage with these support services, they acknowledged that the services they received were not as helpful for them. one participant reported that "the tutors, they try their best. sometimes they know more than what they can express in layman's terms." similar to the tutors, faculty members were perceived as a potential on-campus resource, but students were not always able to receive the help they needed from them. one participant explained, "it's a mixed response. i have some teachers who are willing to help, but then i have some of the other teachers where [they say], 'oh yeah, you can figure it out.'" when academically at-risk students like those in the rtw program encounter faculty or staff members who are perceived as not being helpful to them when they seek academic support, they may be more likely to get discouraged and not actively work to locate alternative sources of support. considering that students found on-campus resources were generally helpful to them if they used them consistently, encouraging students to persist in using these resources may be one strategy that could be stressed for students in the rtw program. the impact of social identities on academic experiences represented the third theme identified from the student focus groups. in all three focus groups, participants discussed how their personal identities, including their race/ethnicity, gender, first-generation status, and age, shaped their experiences on campus. one participant noted that racial group membership served as a motivational factor: i'm hispanic and black, and the way it's influenced me so far in college is just staying focused on school because both of my parents didn't have the opportunity. and most of my hispanic family didn't go to college or finish high school, so that's really kind of motivating me to focus on school. other students shared how their social identities affected their interactions with faculty and peers. for example, students detailed how their status as visible minorities at the predominately white institution influenced their peer relationships and their sense of belonging in the classroom and on campus. for example, one student noticed that his white peers "feel more comfortable around each other. they talk to each other about assignments and whatnot. i have to literally step outside of myself and deal with all these extra barriers and boundaries that they don't have to deal with." other students reported that the limited diversity on campus contributed to negative experiences with peers, which they connected to several of their identities. one black woman's story exemplified how race and gender influenced her in-class experiences: so i was in class a few weeks ago, and i was having a discussion with two caucasian males in class. and they were [saying] , 'i'm going to be a corporate, corporate white guy, ceo, stuff like this and, you know, you're going to be the woman being the assistant.' and when he said that you're going to be the assistant, so basically not making me eligible to be the ceo. and why is that? because i'm a woman? because i'm black? because we're in the same class, getting the same education, but why can't i? these stories highlight ways in which students' social identities can serve to motivate them, but these same identities can also expose them to additional stressors that affect their integration and satisfaction at the university. taken together, students in the focus groups felt that they lacked the necessary academic skills when they entered college and had to seek oncampus resources to deal with gaps in their academic preparation. accessing these resources did not automatically ensure their success given the differences in the quality of the resources utilized, and students have to be able to persist in their efforts to find helpful resources. additionally, students recognized that their different social identities affected their experiences on campus. while their identities were not explicitly linked to the academic problems they reported, these students noted that they shaped their on-campus interactions. these experiences add another barrier that academically at-risk students in the rtw program must navigate to achieve academic success. professional support staff identified three themes that underscored their experiences working with academically at-risk students at the institution. they recognized students' inadequate preparation for college, which represented the first theme. they reported that students were often not ready for the expectations of higher education and struggled to manage the freedom and responsibilities of college life. one staff member explained as follows: i hear a lot of 'oh, i didn't have to study in high school, and now i have to study. i don't know how to study. i don't know how to read a textbook, understand it. i don't know what i'm reading.' so i see a lot of that. one administrator concurred with their colleagues' assessment of students' inadequate preparation for college and their early academic struggles at the university: you'll have a kid that might have a 4.0 who never studied, and then they come here. and since they're not in front of their teachers as much, they're not in class as much, there's so much down time. you pretty much come to campus, do your classes, and then you can go home and do whatever. the participants noted that students' lack of prior preparation contributed to their failure to structure their downtime to minimize procrastination and distractions that negatively affected their performance. students would consequently get stuck in a cycle of academic procrastination and failure that eventually led to their dismissal from the university. while student support staff and administrators acknowledged that academic problems, such as poor academic preparation, were contributors to students' subpar grades, they noted that focusing exclusively on academics failed to explain students' struggles. professional staff reported in the second theme that broader issues affect students' performance. they noted that students were often dealing with multiple issues that directly and indirectly influenced their academic performance. one administrator reported that some students' efforts to balance their family's expectations contributed to their academic struggles: [parents] pushing students into majors that they may not be passionate about because it makes more money. i see a lot of that from our students so they get stuck in this cycle of doing poorly and struggling, and that adds to their stress. they also discussed how family issues, mental and physical health problems, and financial concerns represented other non-academic concerns that affected at-risk students they served. one staff member reported that their students were juggling multiple responsibilities that inhibited their engagement on campus: they are the responsible one in their family, so they have a hard time leaving that behind and being the college student on-campus because they may still be driving back and forth to take brothers and sisters to school in the morning or whatever they have to do. i think that's hard for them too. in the final theme, professionals focused on the impact of students' social identities on their engagement. they highlighted how students' social identities affected their perceptions of cultural fit and sense of belonging at the university. one staff member explained that students struggled to develop a sense of belonging during their transition to college. one of my students described coming here was a culture shock for her. there wasn't [sic] enough [black] people. she's never been in a place where she was literally the minority and being in the classroom with 200 and something people and you're the only african american person sitting in there. that scared her, and she felt like she didn't belong here. she felt like a lot of the clubs and organizations that were geared for her were very few, and she just felt like it wasn't the place for her to be, and she was thinking about going to an hbcu for those reasons. professional staff noted that they worked to get students more engaged by helping them to navigate experiences of being either first generation, low income, or a student of color in higher education by addressing students' "exposure gaps." one staff member commented that many students "don't get to know people in positions of power on campus. they don't go out. they stay in their bubbles and don't branch out to expand their networks beyond their friends." these professionals noted that students' pre-existing beliefs may lead to lower engagement on campus. one administrator reported that he worked with students to address their worries: [some underrepresented students deal with the] fear of doing something different from what you've already done or what you've been exposed to, especially when it comes to study abroad or going to an organization that you wouldn't have thought to go to because of the benefits that it can add to your skillset. by addressing knowledge gaps created by students' first generation or low income status as well as their experiences as racial minorities at the institution, these professionals worked to build relationships with students and help them connect with culturally relevant and supportive resources on campus. these bridging efforts can enhance students' academic and social engagement on campus, which can contribute to their satisfaction and academic performance in school. the purpose of the first phase of this project, the focus groups, was to catalogue the experiences of academically at-risk students and the professional stakeholders who work directly with them as we created an intervention for students in the rtw program. prior research has encouraged researchers to include the voices of multiple stakeholders when crafting student-centered interventions (lepeau, 2015) . the inclusion of data from the four focus groups ensured that our intervention incorporated contextual factors that may affect students in the rtw program. by gathering the perspectives of both at-risk students as well as the professionals tasked with facilitating their academic success prior to creating the intervention, we were able to tailor the intervention specifically to the at-risk students in the rtw program. given that both groups identified similar factors contributing to students' academic challenges, we were able to hone in on these areas and increase buy-in from students and professional stakeholders. other programs may benefit from taking a similar approach as they create new academic interventions. this early buy-in can help to generate a shared problem statement and lead to a more cohesive plan that does not rely exclusively on certain stakeholders' perspectives of the problem. stories from the four focus groups revealed that students felt unprepared for the rigors of college and needed assistance developing the academic skills (e.g., time management, effective study strategies) that would improve their academic performance. while all students must develop these vital skills, our findings indicate that students like those in the rtw program need additional support. compared to other at-risk students on campus, students in the rtw program are unique since they are being readmitted to the institution after they failed to make adequate academic progress for multiple semesters on campus and were academically dismissed. the stressors associated with these chronic academic struggles not only affect students' academic self-concept and motivation; they can also affect them psychologically and emotionally. our findings suggest that this subset of students requires supplemental academic skills development and support beyond what is typically offered during most students' first semester or year on campus. this support entails a more hands-on and holistic approach that may less necessary for other at-risk students at the institution after this timeframe. in the focus groups both students and professional staff agreed that students were aware of resources on campus but did not always engage with them to deal with gaps in their academic competence. students' inconsistent use of services is exacerbated by their inadequate academic preparation entering college. given that some of students did not always have the perception that on-campus resources (i.e., faculty or tutoring services) met their needs or expectations, some atrisk students at the university may be discouraged after seeking out the resources identified by professional staff. this sense of disillusionment with the quality of academic services may be exacerbated for at-risk students like those in the rtw program due to procrastination and the internalized stigma associated with having multiple semesters of academic failure. this stigma likely contributes to feelings of hopelessness and learned helplessness as well as limited oncampus engagement that must be addressed in subsequent interventions. psychological barriers to engagement, such as help-seeking stigma and learned helplessness, may also account for the differing explanations that students and professional stakeholders provided for at-risk students' lack of engagement at the university. both groups spoke to the value of engagement with multiple parties on campus to learn new academic skills, address knowledge and exposure gaps, and achieve optimal results. however, students who did not engage on campus may focus on internal traits to explain their behavior. professional staff, on the other hand, attributed students' lack of engagement to non-academic factors and recognized that these factors called for a holistic perspective to account for students' academic struggles. thus, interventions for academically dismissed students must not only focus on academic skills deficits; they must also address students' psychological, emotional, and cognitive beliefs given that these factors may affect their academic performance, particularly for students who have experienced sustained academic problems at the university. furthermore, our findings also revealed that some students, particularly students of color, encounter unique identity-related challenges on campus. both students and professionals detailed the impact of the university's cultural climate on students and emphasized the need to take this context into account when working to engage this at-risk population. negative experiences on campus can have an adverse effect on these students as they are navigating the everyday challenges associated with being a college student (quaye & harper, 2015) . our findings suggest the need for a multidimensional understanding since students' lack of academic preparation, social identities, and non-academic concerns may contribute to their ability to get off academic probation and progress in their academic programs. combining the insights we gleaned from the focus groups with students as well as professional staff in the first phase of this project, we transitioned to the second phase of the research process. during this phase we created an interdisciplinary and theory-driven intervention for students in the rtw program, which we labeled engage to excel (e 2 ). the e 2 intervention for rtw students has two distinctive and innovative features. these two features are its focus on the integration and application of a targeted model of student engagement alongside a group counseling approach in the classroom, which we now explain. the foundational component of the e 2 intervention for rtw students is its explicit emphasis on student engagement (astin, 1999; tinto, 2001) . higher education research has clearly documented that students who are more engaged on campus have more positive educational outcomes than do their less engaged peers (kuh et al., 2006; quaye & harper, 2015) . although prior research acknowledges the relationship between student engagement and positive educational outcomes (kuh et al., 2006) , less is known about specific interventions that are designed to increase at-risk students' engagement. based on our focus group findings, we recognized that students in the rtw program had already learned some basic academic skills in their first year seminar courses; but they needed more time to apply and practice these skills due to the gaps in their academic preparation. thus, the primary objective in the e 2 intervention was to help students increase their level of engagement across multiple domains and evaluate its effect on their subsequent academic performance. the e 2 intervention introduces students to the abcs (i.e., affective, behavioral, cognitive, and social) of engagement and assesses them on their application of these principles (parsons, nuland, & parsons, 2014) . students are taught the abcs of engagement during the first part of the intervention. these four areas of engagement are then explicitly integrated and assessed through in-class and outof-class activities throughout the rest of the intervention. additional information about the abcs of engagement that we are in the process of evaluating within the e 2 intervention is outlined below. postsecondary scholars acknowledge that engagement is multidimensional and must incorporate students' affect or feelings about school when evaluating their engagement (parsons et al., 2014) . affective engagement incorporates students' positive (e.g., enjoyment or interest) and negative (e.g., anxiety or discouragement) emotional responses to their academic experiences (wilson et al., 2015) . affective engagement includes students' sense of belonging within the classroom or at college. in the e 2 intervention, affective engagement is being evaluated using measures of affect, students' selfreports of affective engagement as well as multiple in-class evaluations by trained observers. behavioral engagement denotes actions and practices that students use to stay involved in learning and academic tasks (wang & eccles, 2013) . it can involve effort and participation in academic activities (wilson et al., 2015) . drawing on the work of behavioral psychologists, the focus of behavioral engagement within the e 2 intervention is ensuring that students engage in specific behaviors on a daily and weekly basis. prior research has shown the value of behavioral activation and mastery activities for changing unhelpful behaviors, particularly when combined with other types of engagement (kanter, busch, & rusch, 2009 ). examples of behavioral engagement within the e 2 intervention include but are not limited to the following: (a) completion of weekly behavioral tracking sheets, (b) attending structured study times (alone or with others), and (c) use of oncampus and off-campus resources (e.g., university's writing center, meetings with faculty). cognitive engagement involves students' use of metacognitive and self-regulated strategies (parsons et al., 2014) . it also addresses the psychological and attitudinal factors that can contribute to or hinder students' academic performance based on their cognition or thoughts. cognitive engagement can incorporate evaluation of students' internalized self-beliefs that subsequently contribute to their outcomes (kanter et al., 2009 ) as well as positive and negative patterns of cognitive engagement. positive cognitive engagement involves cognitive strategies that students use to overcome barriers to their academic success, including academic selfconcept, hope, and resilience. negative cognitive engagement involves overreliance on cognitions that fuel self-handicapping, academic avoidance, procrastination, and imposter feelings. both of these forms of cognitive engagement are being assessed in the intervention. social engagement refers to students' involvement with their peers in both in-class and out-ofclass activities (quaye & harper, 2015) . these out-of-class activities can be both formal and informal. formal out-of-class activities include involvement with student organizations, volunteer opportunities, and extracurricular activities. informal out-of-class activities involve any activities that are not formally organized by the institution but that allow students to engage with each other (quaye & harper, 2015) . social engagement incorporates student-faculty interactions, such as visiting professors in their offices and talking with professors before or after class. also, we assess the impact of cultural variables on social climate and engagement for diverse students in the intervention (quaye & harper, 2015) . social engagement is being evaluated using weekly behavioral trackers as well as using measures on social engagement attitudes. the second innovative feature of the e 2 intervention is our team's use of a classroom-based group counseling approach. while the e 2 intervention will be offered in the classroom like other student success courses, it differs from these courses since it integrates aspects more commonly found in group counseling offered by psychologists in the university counseling center. this group counseling approach in the classroom combines the traditional focus within student success courses on developing academic skills with some of the interventions psychologists use to address holistically more of the broader concerns faced by academically atrisk students as highlighted in the focus groups. it is important to note that student affairs professionals often offer this type of support to students individually. however, our team believed that we could use the classroom setting to deliver a similar intervention in a group setting using our team's expertise as counseling psychologists. while student affairs professionals and some faculty receive limited training to address the social and emotional aspects of students' academic experiences, psychologists are equipped to delve deeper into these issues and can use the power of the group setting to facilitate this process. the group counseling approach is embedded into the structure and content of the e 2 intervention for rtw students. for example, similar to group counseling process, students "check in" at the beginning of each class period. during their check-ins, they share about successes and challenges from the previous week or for the upcoming week. the instructor and classmates can ask clarifying questions and give encouraging words. students can also request additional class time to discuss and receive feedback about any pressing concerns affecting their academic performance. this open sharing at the start of every class period helps to normalize the act of seeking help and allows students to recognize their common academic challenges, which can reduce the stigma for doing so. it also allows students to support each other's victories from the previous week and hold each other accountable if they are not reaching their established engagement goals. additionally, students can learn to recognize maladaptive academic behaviors in themselves and others and hear about resources that helped their peers address their academic problems. this group counseling approach differs from available academic recovery interventions since it draws on the power of the group to normalize seeking help and to challenge psychological barriers to engagement. using the classroom as an academic group counseling session, the instructor of the course uses their psychological training to begin helping students to address the multifaceted problems they are facing without having them visit the university's counseling center immediately. students' non-academic concerns, such as mental health, family stressors, financial barriers, and adjustment concerns, can be discussed in the checkins as well as in the assigned readings for the course (beattie, laliberté, michaud-leclerc, & oreopoulos, 2019) . this counseling group approach reduces the separation of students' academic challenges from the broader issues that they face and allows students' problems to be diagnosed and treated earlier. using the group counseling format in the classroom, students continue to receive traditional instruction on academic skills, such as time management, procrastination, and study skills, given that our focus groups revealed that these deficits were a concern for at-risk students at our university. however, it also allows this subset of academically at-risk students to receive assistance in identifying and removing barriers to their full academic and social engagement on campus. we delivered the e 2 intervention using the existing course sections within the rtw program. this intervention utilizes a mixed methods, pre/post-test experimental design that is occurring in two phases and started with the fall 2018 cohort of students in the rtw program. in fall 2018 students were randomly assigned to one of the three student success course sections reserved for rtw students. we gathered baseline data from all students in the rtw program at the start of the fall semester. our research team was assigned one section of the rtw students, and the first author delivered the experimental e 2 intervention throughout the fall 2018 semester to this one section. using a longitudinal design, we have been gathering data on the effects of the e 2 intervention over the past two academic years to document the critical elements of this approach and to assess its effectiveness empirically. all students in the fall 2018 cohort were re-assessed in spring 2019, and additional data will be collected from this first cohort in spring 2020. 1 in fall 2019, one section of rtw students received the e 2 intervention. this second cohort of rtw students from fall 2019 will be tracked and assessed over a 2-year time span to gather enough data in order to evaluate the statistical significance of the e 2 intervention for rtw students. we will assess outcomes including whether the e 2 intervention helps students' increase their gpa, transition off academic probation, and achieve junior level status at the university. additionally, we will analyze the engagement patterns of students in the fall 2018 and 2019 sections of the e 2 classes of rtw students with their peers in the two other fall 2018 and 2019 sections of students in the rtw program. students in the other two sections of the rtw classes will serve as a treatment-as-usual control group. we have collected background information for all students in the fall 2018 and 2019 cohorts so that their academic profiles prior to their entry into the university can be compared. finally, we will supplement these quantitative data with qualitative interviews from students in the fall 2018 and fall 2019 cohorts. it is important to acknowledge the challenges of evaluating the e 2 intervention. the use of control versus treatment groups represents one ethical concern. however, considering that students in the control group will continue to receive services that were previously available via the rtw program, they will not be harmed by remaining in the control group. additionally, the use of a control group allows us to account for the volunteer effect for the students who elect to apply for the rtw program. the control group allows us to assess whether the intervention provides benefits above and beyond what is already offered to students in the academic recovery program. embedded within the rtw program, we believe that the e 2 intervention represents an innovative academic recovery approach. the effort to operationalize engagement principles within a classroom setting represents one strategy for translating theory and research on student engagement into practice. additionally, another strength of the e 2 intervention is the use of a group counseling approach in the classroom, which integrates multi-dimensional factors for improving the academic outcomes of at-risk students. its integration of the strengths of counseling psychologists, student affairs personnel, and other higher education professionals allows it to more flexibly address the problems academically dismissed students encounter. this approach can shift how academic recovery courses are designed and delivered. while it is clear that a focus on basic academic skills needs to remain a focus, moving from a didactic approach to one that centers upon student engagement may prove more efficacious for students who are academically struggling. additionally, encouraging faculty and staff who teach these courses to integrate a focus on the personal aspects of students' experiences may help to destigmatize these issues and highlight resources that students can use. future researchers and practitioners may seek to replicate and extend the principles embedded in the e 2 intervention as well as to explore other interventions that center student engagement. researchers can continue exploring proactive engagement interventions with at-risk students, with a particular emphasis on students who are placed on academic probation and at-risk of academic dismissal. for example, it may be useful to evaluate the benefits of introducing an engagement intervention for students placed on academic probation after the first semester. given that engagement is shaped by contextual factors, it is important to gather quantitative and qualitative data that capture how students' social experiences and the social climate affects their engagement (quaye & harper, 2015) . while the research is clear that engagement matters for students' success, more work can be done to identify barriers to engagement for marginalized populations and incorporate practices that allow all students to receive the benefits of academic and social engagement. although we believe that the rtw program is an innovative program for academically at-risk college students, the creation of a targeted engagement intervention represents the next step in serving these students. the e 2 intervention introduces practitioners and researchers to one approach for setting up and assessing a student engagement intervention. postsecondary institutions must strive to create learning environments where the most academically vulnerable students are able to engage and fully realize their potential. this effort can be achieved by placing more emphasis on interdisciplinary partnerships that combine the expertise of all institutional stakeholders to the benefit of these students. student involvement: a developmental theory for higher education education pays 2010: the benefits of higher education for individuals and society (trends in higher education series) what sets college thrivers and divers apart? a contrast in study habits, attitudes, and mental health using thematic analysis in psychology helping students keep the promise: exploring how kalamazoo promise scholars' basic needs, motivation, and engagement correlate to performance and persistence in a 4-year institution. innovative higher education first year experience for at risk college students is what glitters really gold? a quasi-experimental study of first-year seminars and college student success the influence of first-year "success" courses on student learning and democratic outcomes academic probation, time management, and time use in a college success course impact of a student success course on undergraduate academic outcomes behavioral activation: distinctive features effectiveness of the student success course on persistence, retention, academic achievement, and student engagement what matters to student success: a review of the literature a grounded theory of academic affairs and student affairs partnerships for diversity and inclusion aims. the review of higher education early intervention to support the academic recovery of first-year students at risk of non-continuation. innovations in education and teaching international how college affects students: 21st century evidence that higher education works a success course for freshmen on academic probation: persistence and graduation outcomes the abcs of student engagement do first-year seminars improve college grades and retention? a quantitative review of their overall effectiveness and an examination of moderators of effectiveness student engagement in higher education: theoretical perspectives and practical approaches for diverse populations rethinking the first year of college academic probation as a dangerous opportunity: factors influencing diverse college students' success school context, achievement motivation, and academic engagement: a longitudinal study of school engagement using a multidimensional perspective. learning and instruction belonging and academic engagement among undergraduate stem students: a multi-institutional study publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. key: cord-337076-fslggn9b authors: de tantillo, lila; christopher, roberta title: transforming graduate nursing education during an era of social distancing: tools from the field() date: 2020-05-29 journal: nurse educ today doi: 10.1016/j.nedt.2020.104472 sha: doc_id: 337076 cord_uid: fslggn9b nan in the era of covid-19 pandemic infection, furthering the education of the graduate prepared nurse is more important than ever. our advanced practice nurses, doctorally prepared nurse leaders, and phd nurse scientists are critical elements of our health care system. at this time our nursing schools must ensure graduate education is continued with the least disruption possible. to this end, many graduate nursing educational institutions have responded by making sudden, drastic transformations to their curricula. while some nursing schools may already have a strong online presence, others may be scrambling to transfer material to a web-based format. wherever a nursing school begins on this spectrum, it is possible to implement measures that will enhance the student experience and promote learning throughout masters and doctoral level programs. implementing capacity for these changes is essential for of graduate level nurses to complete their education and mobilize their potential in response to disasters (veenema et al., 2016) . the purpose of this article is to provide best practices for adaptation of graduate nursing education in response to the present challenge posed by the covid-19 pandemic and expectations of social distancing. although the essence of this process is to adapt educational content to an online format, it is essential for nursing schools to do so thoughtfully and deliberately, prioritizing the interests of the students. lectures. in the era of social distancing, graduate nursing programs have scrambled to translate their curriculums to online formats as expediently as possible. learning management platforms such as blackboard (https://www.blackboard.com/) have been essential in this process. when starting this process, a key objective is to ensure students receive all the knowledge they were slated to receive from the course as originally conceptualized. however, it is crucial to be mindful that students may not be able to manage listening to lectures online in the same manner as they would in person. one option is to record lectures for later asynchronous review, so that students have options when to listen, pause, and resume the content. consideration should also be given to supplementing the original course with additional online material. youtube and other video platform can be powerful resources that provide supplemental educational content scaffolded into smaller chunks. nevertheless, faculty must be selective and invest the time necessary to identify high quality videos, preferably from reputable schools of nursing or professional organizations. all videos from outside sources should be reviewed for appropriateness before sharing with students. short videos of approximately five minutes in length and focused on one concept are ideal. learning modules. once content has been located, faculty may organize the material into learning modules. learning modules may be developed using a scaffold design, which builds from foundational knowledge into more advanced reflective activities (webb 1997; 1999) . videos may then be integrated with key professional practice resources via web links. students will also benefit from links integrating contemporary peer-reviewed journal articles. textbooks. course textbooks frequently include interactive presentations or videos with enhanced content provided through purchase of a code that may be uploaded to the course and j o u r n a l p r e -p r o o f learning modules via a learning management system course cartridge. such enhanced content, such as course-specific videos, are frequently created by the textbook author and leading experts. a key limitation, however, is that not all students may have the current textbook edition or access to the code, especially if the book was previously owned or loaned to the student from a recent graduate. students and faculty may contact their respective publisher representative for assistance with purchasing codes. publishers such as elsevier (https://www.elsevier.com/connect/coronavirus-information-center) are offering these enhanced textbook resources at a greatly reduced cost or for free during key pandemic time periods. there are several websites that produce quality educational material relevant for nursing graduate education that may be accessed and posted free of charge for students. among them are khan academy (khanacademy.org) and next gen u (nextgenu.org). additional engaged and active learning tools that may be used in conjunction with live lectures and as part of learning modules include:  kahoot (https://kahoot.com/) -gamified quizzes for assessment that are great as live lecture warm-ups  quizizz (https://quizizz.com/) -self-paced gamified quizzes  brainpop (https://www.brainpop.com/) -short animated video lessons on specific topics that are great to use for teaching courses such as anatomy, physiology, pathophysiology, leadership topics (e.g. conflict resolution), and caring behaviors (e.g. empathy and respect)  gimkit (https://www.gimkit.com/) -gamified quizzes which add a collaborative component  piktochart (https://piktochart.com/formats/infographics/) -tool to create infographics  edpuzzle (https://edpuzzle.com/) -tool for creating interactive videos the flipped classroom. the flipped classroom has been recently emphasized in nursing education and may be an especially appropriate model to consider during a transition to the online learning environment. this setting differs from the traditional classroom in which faculty introduces the course content, which students subsequently reinforce through reading and assignments. in a flipped classroom, there is increased emphasis on student-based learning as the initial exposure to course material (betihavas, bridgman, kornhaber, & cross, 2016) . the faculty member will provide additional opportunities for knowledge building, application or skill development. bergmann and sams (2012; emphasized several benefits of a flipped classroom that may be extended to remote learning. one key advantage is students may progress through the material at their own pace and then virtual class time may be focused on content the student found challenging and on application and higher order cognitive processes. another benefit specific to the pandemic environment is augmentation of missed instruction. practicing nurses and advanced practice nurses may not be able to participate in virtual live lectures due to work, family, or their own health needs. additionally, faculty may also not be able to host live lectures for the same reasons. thus, the flipped classroom supports student-centered pedagogy. group work. one of the more difficult elements of a syllabus to manage online may be assignments that are expected to be completed as group work. these challenges can be mitigated by providing clear guidelines and communication tools (monsivais & robbins, 2017) . for example, many online platforms (including blackboard collaborate) have opportunities for break-out sessions. fostering group assignments in online learning may be especially appropriate j o u r n a l p r e -p r o o f at this time, as they have the potential to foster camaraderie and increase engagement during periods of social distancing. one such example is the use of peer buddies. having a student peer buddy for written assignment peer review and social support can be instrumental in student success and overall satisfaction. faculty may allow students to select their own peer buddies or may be assign collaborations, which helps to reduce isolation that may occur with social distancing. discussion boards. when possible, it is recommended to structure online coursework to engage students with multiple opportunities for interaction. previous research with online msn students has found positive correlations between number of submissions and course gpas. for example, the hart model for discussion boards (2016) recommends students to create an initial post, two responses to peers, and two replies to responses during the course of one week. video discussion posts. asynchronous discussions using a video response format provide the opportunity for faculty and students to remain both socially and cognitive connected while physically off campus. presence is further enhanced through the ability to connect beyond written communication. the ability to visually see and hear each other allows for visualization of nonverbal communication as well as verbal communication nuisances that far surpass written only communication. learning management systems, such as canvas (www.canvas.net), have built in functionality allowing both students and faculty to post video responses directly into the discussion board forum. direct posting of video responses reduces the need to upload mp4 files that tend to be large in size and can take up to an hour to upload. the internet will be taxed with increased usage, so it is not recommended to use manual video uploads to the discussion board. canvas has smartphone and tablet applications which allow for increased flexibility in posting from anywhere. flipgrid (https://info.flipgrid.com/) is a cloud-based asynchronous platform that allows for video discussions that are behind a password protected firewall. students may download the smartphone or tablet app and post video responses from anywhere. the mobile flexibility allows practicing nurse to post from anywhere. flipgrid also works on computers which have a webcam and microphone. in the faculty dashboard, faculty can sort by student, download an excel file to ensure all posts are included in the grading process, and post video responses to students. the flipgrid platform works well for socratic discussions, research presentations, and project-based learning. flipgrid has directions posted on their website specific to remote learning and how to be operational within five minutes. simulation. one of the top concerns during the covid-19 crisis has been ensuring students receive appropriate clinical hours, as many health care facilities have curtailed programs. with limited access to health care facilities, nursing programs may consider simulation instruction as an alternative. however, use of traditional mannequin-based simulators may not be an option during a period of social distancing. in this case, programs may consider web-based simulation for advanced practice programs (leflor & thomas, 2016) . several cloudbased simulation products may be used to provide develop and enhance clinical reasoning. two examples are ehr go and novex. ehr go (https://ehrgo.com/ ) has more than 600 simulated case studies. many include clinical documentation in the ehr go simulated electronic health record and are well suited for interprofessional education. faculty may set up cases easily and adapt to specific course and learning outcomes. novex (http://novicetoexpert.org/a-homepagesection/novex-experience/ ) is based on the work of dr. patricia benner, dr. tom ahrens, and j o u r n a l p r e -p r o o f dr. patricia hooper kyriakidis. the sophisticated clinical reasoning and judgment simulations allow faculty to visualize the student thought process as well as observe clicks and actions taken throughout the case. novex also has the capacity to identify sentinel events related to patient safety and flag students who may require remediation. across all virtual platforms, it is vital to ensure the concept of caringfidelity tm is imparted to students along with technical competencies and skills, so that graduate nursing students are able to implement into practice the real-life processes of reflection, contemplation, and action (christopher, in press) . flexibility. a literature review describing best practices on graduate nursing education (gazza & hunker, 2014 ) grouped its recommendations for enhancing student retention into three categories: social presence of the course, academic quality, and responsiveness to individual student characteristics. it is also recommended that those faculty with more experience provide guidance to those who may have less. nevertheless, it is important for all faculty to understand that this is a new and evolving situation for all. if a student poses a question that faculty is not prepared to answer, it is perfectly acceptable to defer reply until the information is available. of course, faculty should adhere to the principle of fidelity and ensure follow-up is provided when available. academic integrity. as increasing amounts of course content is transferred online, it should be noted that this format may increase the opportunity and perhaps the temptation for academic misconduct. to curtail opportunities for such a breach, it is recommended that faculty utilize a variety of resources affiliated with most academic learning platforms. these resources may include tools that are integrated directly into the learning management system or indirectly from the provided academic link provided by the vendor. such tools include plagiarism detection software such as turnitin (turnitin.com) and secure test-taking via lockdown browser ® and respondus monitor ® (https://web.respondus.com/he/lockdownbrowser/). another option is top hat (tophat.com), which has announced it is free of charge through spring 2020. during these turbulent times, it is especially vital for faculty to maintain open lines of communication with all students. students may be experiencing increased demands from work or other pressures influencing a potential lapse in judgment. maintaining faculty accessibility and clear expectations of student responsibilities are essential in promoting an ethical environment. academic leaders, departments, and program chairs should promote psychological safety for faculty as they may be new to many of these advanced technologies and learning management systems. partnerships with university departments, such as information technology and centers of academic excellence, are vital in supporting the transformation of graduate nursing education to a remote learning model. use of faculty who are considered super users may further augment university and departmental resources to support faculty as they learn to use and implement remote learning pedagogy and practices while ensuring accessibility to all students. furthering such partnerships will need to support students as they transition from traditional face to face classroom settings to remote learning formats. the common thread present throughout these elements is that nursing schools must continue to support the education and safety of all students. despite the logistical challenges, graduate nursing students will continue to thrive, driven by a desire and commitment to serve others. as educators, it is our duty to continue the preparation of a graduate nursing workforce capable of responding to our nation's health care needs. flip your classroom: reach every student in every class every day the flipped classroom the evidence for 'flipping out': a systematic review of the flipped classroom in nursing education chapter 17: simulation, narrative pedagogy, & caring fidelity tm -the new reality facilitating student retention in online graduate nursing education programs: a review of the literature designing a graduate discussion board rubric to facilitate higher-order learning nursing student anxiety as a context for teaching/learning educational changes to support advanced practice nursing education using evidence-based education to guide quality improvement in a graduate online nursing program: group projects as an exemplar interventional strategies to decrease nursing student anxiety in the clinical learning environment key: cord-325732-hva5dasd authors: krawiec, conrad; myers, abigail title: remote assessment of video-recorded oral presentations centered on a virtual case-based module: a covid-19 feasibility study date: 2020-06-20 journal: cureus doi: 10.7759/cureus.8726 sha: doc_id: 325732 cord_uid: hva5dasd introduction the coronavirus disease 2019 (covid-19) pandemic has resulted in the suspension of our pediatric clerkship, which may result in medical student skill erosion due to lack of patient contact. our clerkship has developed and assessed the feasibility of implementing a video-recorded oral presentation assignment and formative assessment centered on virtual case-based modules. methods this retrospective study examined the feasibility of providing a remote formative assessment of third-year medical student video-recorded oral presentation submissions centered on virtual case-based modules over a one-week time period after pediatric clerkship suspension (march 16th to 20th, 2020). descriptive statistics were used to assess the video length and assessment scores of the oral presentations. results twelve subjects were included in this study. overall median assessment score [median score, (25th, 75th percentile)] was 5 (4,6), described as “mostly on target” per the patient presentation rating tool. conclusion patient-related activities during the pediatric clerkship were halted during the covid-19 pandemic. this study demonstrated the possibility of remotely assessing oral presentation skills centered on virtual case-based modules using a patient presentation tool intended for non-virtual patients. this may prepare students for their clinical experiences when covid-19 restrictions are lifted. future studies are needed to determine if suspended clerkships should consider this approach. in 2020, the coronavirus disease 2019 (covid-19) pandemic resulted in the unprecedented prolonged closure of educational institutions worldwide to curb the spread of the virus [1, 2] . medical students were included in this group of learners per the guidance of the association of american medical colleges (aamc) [3] . thus, our institution temporarily suspended the clinical portion of the pediatric clerkship. electronic resources exist to supplement the pediatric clerkship curriculum, thus key aspects can be taught remotely [4] [5] [6] [7] . one aspect that electronic sources lack, however, is patient contact. lack of patient contact results in the inability to practice clinical skills, including interviewing or orally presenting patients recently seen. these clinical skills are often assessed during the pediatric clerkship and students will often specifically receive feedback on these skills [8] . they are also prioritized by some clerkships for the summative evaluation as students must develop these skills to demonstrate they can assess a patient and synthesize their medical knowledge [8] . at our institution, we have instituted a remote learning curriculum for our third-year medical students starting at the end of april 2020. when covid-19 restrictions are lifted, our students will undergo two weeks of patient contact time. because our students will not have been in a clinical environment for a prolonged time period, they may have difficulty transitioning [9] . to minimize the impact this transition will have on our students, our pediatric clerkship has developed a video-recorded oral presentation assignment centered on a virtual case-based module with remote formative assessment. our goal was to enhance the development of this clinical skill remotely thereby allowing students to focus on clinical skill development in areas that cannot be achieved without patient contact (i.e., patient interviewing) when restrictions are lifted. the objective of this study is to demonstrate the feasibility of student video-recording an oral presentation centered on a virtual case-based module and having our attending faculty members provide a formative assessment. the study hypothesis is that it is feasible to assess and provide formative feedback on video-recorded oral presentations by pediatric attending faculty members using a patient presentation rating tool intended for non-virtual patients. this is a feasibility study requesting students to video-record an oral presentation centered on a virtual case-based module for formative assessment during a time period (march 16th, 2020 until march 19th, 2020) when pennsylvania state college of medicine third-year medical students were abruptly restricted from providing direct patient care during the pediatric clerkship. a retrospective review of faculty submitted formative assessments of the videorecorded oral presentations centered on virtual case-based modules was completed. this study was reviewed by our institution's review board and determined to be non-human research. third-year medical students -(1) part of our institution's traditional curriculum, (2) rotated at the pediatric clerkship's primary site or off-campus affiliate sites during the first month of the academic year (2020-2021), (3) were abruptly restricted from direct patient care due to the covid-19 pandemic, and (4) completed a video-recorded oral presentation centered on a virtual case-based module -were included in this study. students who were part of the longitudinal integrated curriculum were excluded. the pediatric clerkship at our institution is a four-week rotation with the following clinical requirements: outpatient clinic, nursery, and inpatient service. on march 16th, third-year students were restricted from direct patient care, thus only three weeks of the clerkship was completed. the video-recorded oral presentation assignment was developed by a pediatric clerkship director experienced in inpatient medicine and an outpatient pediatrician. a patient presentation tool developed by lewin et al. was utilized for this assessment [10] . using behavioral and verbal anchors, the patient presentation tool assesses various oral presentation sections including patient history, physical exam and diagnostic study results, summary statement, assessment and plan, clinical reasoning/synthesis of information, and general aspects (organization, speaking style) based on a 5-point scale (5 being the highest) [10] . overall assessment of presentation is based on a 9-point scale (9 being the highest and described as "well above expectations"). eight faculty members were recruited to use this tool as they were assessing the video recordings. starting on march 16th, 2020, the subjects were provided a remote learning curriculum and were notified of the video-recorded oral presentation assignment. they were informed that the pediatric clerkship will be graded pass/fail, that submission of a video-recorded oral presentation for formative assessment will be required, and was due on march 19th, 2020. the subjects were instructed to (1) video-record an oral presentation of either a patient they have seen during the course of the clerkship or after completing a virtual online case-based module through aquifer © (lebanon, new hampshire, usa) and (2) upload the assignment via the instructure canvas (salt lake city, utah, usa) learning management system. students were given specific directions including the use of professional attire, limiting the video-recording to 10 minutes, and requesting students to review the video prior to submission for clarity and organization. after receiving the video-recordings, the files were securely distributed through the canvas © learning management system among eight pediatric attending faculty volunteers who reviewed and provided formative assessment scores of the oral presentation. all completed assignments were collected using the instructure canvas learning management system. using the canvas learning management system, we extracted the following data: overall video-recorded oral presentation rating scores and video-recorded oral presentation scores divided by section as outlined by the patient presentation tool [10] . if students elected to give an oral presentation based on a virtual case-based module, we asked students to complete the pediatric aquifer © case-based module 3, a 3-year-old male seen for a well-child visit [4] . this case was chosen as it provides a robust history and physical examination, tasks the student to identify and prioritize problems uncovered during this visit, allows the student to apply a differential diagnosis when appropriate, formulate a management plan, and practice their organization skills during the oral presentation. we used descriptive statistics to assess the study population in terms of length of presentation, type of patient presented, and assessment scores based on the patient presentation tool [10] . formative assessment of each oral presentation was reported in the median and interquartile range. twelve individual oral presentation videos centered on the virtual case-based module were included in this study. median video length [median time (mm:ss), (25th, 75th percentile)] was 06:20 (05:04,07:21). overall, median overall formative assessment score [median score, 25th, 75th percentile] of video-recorded oral presentation centered on virtual case-based modules was 5 (4,6), described as "mostly on target" per the patient presentation tool [10] . the lowest items scored were pertinent positives and negatives of the differential diagnosis [2 (1,3)] ( table 1) . 2020 note: sections scored on a 1 to 5 scale, 5 being the highest score patient presentation rating tool for oral case presentations [10] . oral presentations are an essential clinical skill that facilitates physician to physician communication, improves efficiency on rounds, and enables individual as well as group learning [8] . it also can be complex and time-consuming as students must use their medical knowledge and clinical reasoning skills to select the pertinent details to present from a patient's history, physical, diagnostic, and laboratory tests [8, 10] . in this study, we hypothesized that video-recorded oral presentations centered on a virtual case-based module can undergo a formative assessment. this study successfully demonstrated that a formative assessment can be remotely provided for video-recorded presentations based on virtual casebased modules. these results imply that this form of assessment is possible, may prepare students for the eventual live clinical experience (with patient contact), and potentially optimize the transition period from covid-19 remote learning to a post-covid-19 clinical patient experience. to our knowledge, a pediatric clerkship has never been halted in this manner for a prolonged period due to a nationwide health emergency. because of this, our pediatric clerkship, like others across the united states was placed in an unprecedented situation, potentially placing our students at risk of achieving suboptimal competency in various clinical areas [11] . novel approaches are necessary to ensure that our students, who were hastily restricted during their pediatric clerkship and future students that have yet to complete their pediatric clerkship, are adequately trained [11] . our institution's current plans are for each clerkship to institute a remote learning curriculum and complete a two-week immersive clinical experience in each of the core clerkships. the remote learning curriculum will allow students to learn the basic concepts relevant for pediatrics and the two-week patient contact experience will allow students to apply their knowledge. when the two-week patient contact experience begins, however, the transition period may be difficult. students will not have seen a patient (possibly for months) and similar to transitioning from the pre-clerkship to clerkship years, students may be overwhelmed by clerkship logistics, expectations, and adjusting to the clinical culture [12] . in all, students may be overwhelmed by this and the number of tasks they must complete in a short time period post-clerkship suspension, potentially limiting their clinical experience. thus, it is the clerkship's responsibility to ensure that students in a remote curriculum continue to be comparably trained and are provided as many similar clinical experiences as possible to ease the transition that will occur on clerkship reinstatement. while the pediatric clerkship is currently limited in allowing students to see patients during the remote learning experience, there are other ways that students can be robustly prepared for the clinical environment. the area that our clerkship elected to focus on is the oral presentation. if students are rigorously prepared to practice oral presentation skills using pediatric faculty members (that they will eventually present to), students may start to apply their communication, medical knowledge, and clinical reasoning skills earlier and potentially focus their clinical skills on other areas that they cannot easily achieve remotely (i.e., history taking and physical examination and providing live patient care) when they return to clerkship. students may also have a better understanding of their expectations, roles, and responsibilities of this skill for our clerkship and thus are better prepared to provide meaningful patient care and be effective team members sooner. in our study, we found that it is feasible for students to submit a video-recorded oral presentation centered on a virtual case-based module and recruit pediatric attending faculty members to assess and provide formative feedback. we also found that the overall median scores were "mostly on target" according to the patient presentation tool. the students who completed these assessments were the first students for the academic year, thus these results may indicate that these students developmentally require more practice. alternatively, these results may indicate that because these assessments are formative and the clerkship is now pass/fail, these students were given the feedback necessary to improve their skills. finally, students may not have received enough individual educational attention during the normal clinical workflow and thus were not given enough instruction. more studies are necessary to determine if these assessments are consistent. there were several limitations to this study. this includes its small sample size, the short intervention period, and the lack of randomization. the patient presentation rating tool intended for live patients was used without the opportunity to validate it for use in virtual casebased modules due to the haste in its implementation. future studies will be required to validate the tool for this purpose. student perception is also unknown regarding the effectiveness of this assessment technique, thus future qualitative studies are planned to determine this. our pediatric clerkship was suddenly curtailed during the covid-19 pandemic. the students were provided a remote learning curriculum to emphasize pediatric concepts but may not be able to demonstrate their clinical skills in communication, data synthesis, and patient assessment. our study demonstrated that it is possible to assess oral presentation skills centered on virtual case-based modules using a patient presentation rating tool intended for non-virtual patients and may potentially prepare students for their clinical experiences when covid-19 restrictions are lifted. future studies are needed to determine if suspended clerkships should consider this approach. human subjects: consent was obtained by all participants in this study. penn state college of medicine institutional review board issued approval study00014941. the human subjects protection office determined that the proposed activity, as described in the above-referenced submission, does not meet the definition of human subject research as defined in 45 cfr 46.102(e) and/or (l). institutional review board (irb) review and approval is not required. please note: while irb review and approval is not required, you remain responsible for ensuring compliance with ferpa. if you have additional questions regarding ferpa regulations, please contact the office of the university registrar. the irb requires notification and review if there are any proposed changes to the activities described in the irb submission that may affect this determination. if changes are being considered and there are questions about whether irb review is needed, please contact the human subjects protection office. animal subjects: all authors have confirmed that this study did not involve animal subjects or tissue. conflicts of interest: in compliance with the icmje uniform disclosure form, all authors declare the following: payment/services info: all authors have declared that no financial support was received from any organization for the submitted work. financial relationships: all authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. other relationships: all authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work. considering inequalities in the school closure response to covid-19 mitigate the effects of home confinement on children during the covid-19 outbreak guidance on medical students' participation in direct patient contact activities aquifer pediatrics (formerly clipp): a case-based virtual course blended learning educational format for third-year pediatrics clinical rotation 360 degrees: planning a new pediatric clerkship curriculum competencies and objectives developing and implementing universal guidelines for oral patient presentation skills ready or not? expectations of faculty and medical students for clinical skills preparation for clerkships the patient presentation rating tool for oral case presentations covid-19 and medical education transition to clerkship courses: preparing students to enter the workplace the authors are grateful to our pediatric faculty, who took the time to assess our students during this stressful time. key: cord-281745-jkscwdjh authors: adarkwah, michael agyemang title: “i’m not against online teaching, but what about us?”: ict in ghana post covid-19 date: 2020-09-16 journal: educ inf technol (dordr) doi: 10.1007/s10639-020-10331-z sha: doc_id: 281745 cord_uid: jkscwdjh globally, information and communication technology (ict) is regarded as a dependable vehicle for facilitating educational reform and development, a platform for communication, and as a means to achieve the sustainable development goal four (sdg 4). since the enactment of the no child left behind act (nclb) and declaration of the sdg 4, many countries have opted to embrace the lifelong education for all by integrating ict in teaching and learning at all school levels. the ghanaian government’s initiative to ensure “education anytime anywhere for everyone” by revolutionizing teaching and learning through ict has faced a lot of challenges and criticisms. the main mission of the ghana ict for accelerated development (ict4ad) in 2003 was to transform ghana into an information and technology-driven high-income economy through education, but this goal is yet to be realized. in the wake of the covid-19 which has forced many countries and educational sectors to adopt online learning, there is a need to discuss the effectiveness of online learning and barriers to online learning in the developing contexts, and how to successfully integrate ict in schools for online learning, especially rural schools where students’ educational careers are in jeopardy because they benefit less from online learning. the paper identifies critical factors that affect online learning, recommends post covid-19 strategies to promote e-learning for policymakers in education and the government, and concludes with a conceptual model for emergency transition to e-learning. online learning is hailed as an essential force in democratizing education (jones 1997) . it is a way of opening education to populations who had restricted access because of geography, status or physical handicap (carr-chellman 2005) . khan (1997) defines it as an innovative method to instruct students in remote areas, which involves all types of learning that is done using the computer or other ict resources. since the term was first introduced in 1995, it is used as an overlapping concept with blended learning, elearning, online courses, and online or distance education (singh and thurman 2019) . the advancement of ict has called for online learning as a feasible and economically appropriate means of extending quality higher education (asunka 2008) . the role of ict in education, specifically higher education, cannot be downplayed, it is beneficial for teachers and students (aljaraideh and bataineh 2019) . however, the challenge associated with online learning is the access to ict resources because online learning thrives on the availability of ict facilities (arthur-nyarko and kariuki, 2019, b) . there is an uneven spread of access to ict among different populations, households, and spaces because network is not the same everywhere (lembani, gunter, breines, tapiwa, & dalu, 2019) . the significant role ict plays in the development of any nation has called for countries to produce ict literate citizens (crisolo, 2018) . ict has gained a solid reputation in the education systems of both developed and developing countries (law, pelgrum, & plomp, 2008) . ict in education can provide the twenty-firstcentury skills needed to adapt and compete in this knowledge and information society (haji, moluayonge, & park, 2017) . according to the authors, ict in education enhances student learning, provides education to students with no or limited access to education, aid in facilitating the training of teachers, and enhances the skilled workforce and promote social mobility. ict is also considered as essential for quality higher education (liebenberg, chetty, & prinsloo, 2012) . ict has the ability to enhance learning, make a subject appealing, facilitate problem-solving, communication, research skills, and decision-making process (hong, 2016) . waluyo (2019) adds that ict is a positive predictor of the academic achievement of students in mathematics, science, and reading. ict also facilitates teaching and e-learning, which ultimately have a positive effect on teaching, learning, and research activities. a general assumption is that for ghana to meet economic, social, and political global demands, ict should be prioritized. in light of this, the ministry of education of ghana introduced the "one laptop, one student" policy where 1,000 laptops were dispensed to 30 schools (three schools in each region) across the country (education sector performance report, 2010). another 60,000 laptops were also purchased to be distributed to 2500 junior high schools (education sector performance report, 2012) . according to amanortsu, dzandu, & asabere (2013) , most of the government initiatives to ensure quality and accessible ict education for all in ghana has failed to achieve its goals. for example, agyemang & dadzie (2010) found in their study that a policy statement for supplying ict based-support for distance education (de) learners were available but were yet to be implemented. the integration of ict in teaching and learning is still at the early stage in the education systems of most developing countries (antwi, bansah, & franklin, 2018) . over a decade ago, ghana introduced de as a means to provide learners with access to quality education and promote human resource development (addah, kpebu, & kwapong, 2012) . one of the major challenges in the ghanaian education system is the inequality of educational resources which includes usage of computers and other ict materials (buabeng-andoh, 2012) . antwi et al. (2018) mentioned that the provision of ict in secondary schools for learning is skewed towards schools categorized as premier ("a") schools and located in urban areas. also, at the primary level, provision of ict is also skewed towards private schools (ayebi-arthur, aidoo, & wilson, 2009) . governments should fund ict projects that emphasizes on the needs, capacities, perspectives, and aspirations of the greater number of people living in rural areas (david, 2009) . the main objective of the study is to provide possible solutions for successful transitioning, implementation and sustenance of e-learning programs. the paper first identifies the e-learning challenges in the ghanaian context based on respondent's views and then provide strategies for smooth delivery of e-learning derived from extant literature the research questions leading this investigation are; what are the perceived effectiveness of the online learning, what are the perceived barriers to online learning and what are the perceived strategies for a successful ict integration in education for students who are unable to partake in e-learning process because of environmental and contextual factors related to ict in lieu of the covid-19 crisis. there are limited studies that thoroughly examines the effectiveness of and barriers to the online learning, and identify critical contextualized factors to enhance online learning in developing countries like ghana since the inception of the covid-19. the government of ghana has invested several resources and initiated many promulgated policies to ensure ict is accessible to every student in ghana because of its pivotal role in education. the first governmental policy on ict was the ict for accelerated development (ict4ad) in 2003 which outlined a framework that sought to transform ghana into an information and knowledge-driven ict literate nation (ministry of education, 2015) . this policy was reviewed twice (in 2006 and 2008) until a revised policy document was made available in 2009. the main goal of the revised version was to facilitate the integration, utilization, and modernization of ict in schools in ghana. it was proposed that ict should be integrated as a core subject, an elective subject, and as a teaching tool for all other subject areas. the policy highlighted that ict access and literacy is low in ghana. one of the main goals of the 2007 educational reforms is to ensure that all students in ghana in the pre-tertiary institutions acquire basic ict literary skills (which also include the use of internet), and apply the skills in their studies and in their daily activities (mereku, yidana, hordzi, tete-mensah, & williams, 2009) . a survey of 20 lecturers and 105 students in accra polytechnic revealed that access to ict facilities was inadequate, time to access ict was inadequate, and there was little use of ict software (amanortsu, dzandu, & asabere, 2013) . another study in ghana found that 60% of the respondents teaching ict in the early years of school had knowledge about ict, and 67% of the teachers did not integrate ict in their teaching (asante, 2014) . boni (2018) in his study concluded that both teachers and students in ghana lacked the efficacy and creativity in using ict for teaching and learning. mubashir-ahmed (2009) attributes the the challenges associated with ict integration in ghanaian schools to lack of internet access, lack of quality teachers, insufficient number of computers, high cost of ict gadgets, and lack of electric power in some communities. another study in ghana revealed that access to electricity is a major factor affecting online learning in the country (arthur-nyarko and kariuki, 2019, b) . despite the unprecedented efforts by successive governments to ensure ict is integrated in teaching and learning, the process have been fraught with a lot of challenges in ghana, especially in the rural areas. ghana ranked 112th among 175 countries in the global ict development index in 2016 after being ranked 123rd in 2014 (international telecommunication union, 2014; international telecommunication union, 2016) . in lieu of the covid-19 crisis, the government of ghana announced the suspension of all school operations both pre-tertiary and tertiary on march 15, 2020, as parts of the efforts to stop the spread of the virus (cromwell, 2020) . the closure of universities and schools have disrupted the learning of students and have deprived students opportunities for growth and development (unesco, 2020) . digital learning emerged as an ultimate response to the disruption in education due to the covid-19 lockdown of schools. since the lockdown, the government of ghana has considered the possibility of initiating online courses for students. instead of been proactive in ensuring rigorous implementation of online teaching and learning, the education system was stagnant until it was prompted by the covid-19 crisis. the minister of education, matthew opoku prempah announced that the center for national distance learning and open schooling (cndlos) has rolled out an online platform that will make core content accessible to all students in the senior high school (shs) level. he further declared that the ministry had plans in place to broadcast more contents to the junior high school (jhs) and upper primary students. however, the national union of ghana students (nugs) has petitioned the government to halt all online academic activities launched in the universities across the country (anyorigya, 2020) . the nugs referred to the online learning as "challenge-ridden online learning". the association cited inadequate bundle incentives for schools (lecturers and students), lack of properly laid framework for the implementation of online learning, and the plight of needy students who have been left out of the online learning platforms because of their inability to settle school bills. another concern raised by the nugs is the possibility of compulsory exams and assignments on e-learning platforms that will disadvantage students who are unable to participate in the online learning because of factors beyond their control. online learning in sub-saharan africa is a great challenge because it relies on the availability of ict facilities (asunka, 2008) . according to the unesco (2012), there is a gap between the availability of ict infrastructure and the capability of agrarian communities to integrate ict to boost the economy. also, the report mentioned that african universities and teacher education institutions do not have the adequate ability to help integrate ict in schools. with the limited access to ict resources and institutional challenges in its implementation in sub-saharan countries such as ghana, there is a need for the government and education sectors across the country to address contextual and environmental difficulties faced by needy and rural school students who are excluded from the e-learning platforms which are no fault of theirs. e-learning is beneficial (arthur-nyarko and kariuki, 2019, b), but its effectiveness is contextualized (lembani, gunter, breines, tapiwa, & dalu, 2019) . students from developing countries score lower in online learning and are likely to withdraw from the online courses compared to their colleagues in developed countries (kizilcec & halawa, 2015) . in the us, a meta-analysis of existing data revealed that students engaged in online learning performed better than those in face-to-face sessions while students who blended online and traditional learning performed the best of all (the council of independent colleges, 2016). a student may be part of the online learning but may not actively use the service or follow the tutor (bean, et al., 2019) . barriers identified in literature include; high cost, inadequate infrastructure, lack of ict skills, rejection of e-learning by faculty members, and lack of accessibility to quality internet connection and electricity. start-up cost for online learning may be expensive (queiros & de villiers, 2016) . the high cost of purchasing ict equipment affect the adoption of online learning (sinha & bagarukayo, 2019) . technology and gadgets needed to make learning effective may not be cheap, affecting the online process negatively (srichanyachon, 2014) . limited funding can affect institutions from hosting online learning (bean, et al., 2019) . online education may cost more to develop and deliver than face-to-face courses (turk & cherney, 2016) . in tanzania, 68% of teachers declared they lacked access to computer, while 73% revealed they experienced low internet bandwidth (mtebe & raisamo, 2014) . teaching and learning can take place at anywhere and at anytime with the help of ict tools (cradler & bridgforth, 2002) . inadequate access to technology, studying materials and computers can leave students marginalized and anxious, which affect the online learning process (queiros & de villiers, 2016) . asunka (2008) , he concluded that only 5 out of 22 students had access to computer and internet connectivity at home. lack of technology skills and inadequate background experience with online learning are constraints to online education (olesova, yang, & richardson, 2011) . tutors ought to have required advanced technological skills which sometimes require schools to hire tutors from outside companies who already have the skills (bean, et al., 2019) . lack of experience with online teaching is a barrier to organizing online education (luongo, 2018) . students who do not have prior knowledge and experience in online learning sometimes also lack technical assistance and support systems (srichanyachon, 2014) . according to mtebe & raisamo (2014) , 63% of teachers engaged in online learning in tanzania lacked the skills needed to create or use online educational resources. most faculty members reject online learning as alien to them and are skeptical about it (bacow, bowen, guthrie, lack, & long, 2012) . some faculty members also consider developing online courses as time consuming in comparison to traditional methods of teaching (the council of independent colleges, 2016). hesitance by faculty members to teach online courses and their lack of acceptance of online instruction is a perceived barrier to online learning (turk & cherney, 2016) . slow and unreliable network connections affect the quality of online learning (bean, et al., 2019) . instructions can be delayed as a result of poor internet connections (srichanyachon, 2014) . in the study. limited internet access which includes poor internet connection and low speed demotivate institutions for pursuing online education (sinha & bagarukayo, 2019) . rural communities involved in online learning find it difficult to fund and attract qualified teachers to instruct advanced courses (de la varre, keane, & irvin, 2010). lack of internet access and computers in homes in rural areas affect the progress of online learning (depaul, 2020). lack of constant supply of electricity and internet access in rural communities also makes it difficult to assimilate the online education process (ivala, 2013) . in education, ict integration involves many interrelated factors such as curriculum, teacher characteristics, training and development, infrastructure, organisational factors like school leadership, school culture and supportive framework (judge, 2013) . ict integration concerns with the application of technology to aid student learn traditional academic subjects (grabe & grabe, 2007) . according to the authors, successful ict integration has transformed communities and shaped their lives and thoughts. in 2005, the british columbia ministry of education announced the integration of ict in kindergarten to grade12 throughout the country to ensure that the education system remains important and actively engage students (birch & irvine, 2009) . successful integration of ict in schools still face myriads of obstacles (hew & brush, 2007) . integration of ict in developing countries is still a challenge (aksal & gazi, 2015) . one of the first steps to ensure successful ict integration is to consider the structural (availability of resources and classroom space, availability of ict support and maintenance) and cultural (mission and vision of the school for ict integration) elements of the school where ict is being integrated (tondeur, devos, van houtte, van braak, & valcke, 2009 ). bingimlas (2009) outline some necessary steps to ensure successful ict integration; provision of ict resources (both hardware and software), training new pedagogical approaches to faculty members resistance to change, provision of training courses on how to deal with new gadgets and modern technologies, providing sufficient time for daily lessons, and provision of continual reliable technological support. sarkar (2012) lists the critical elements for to be considered during the implementation of ict in learning; leadership issues, equitable distribution of ict resources and sustainability, and financial issues. the study adopted a qualitative method using a narrative inquiry approach to explore the perceptions of students on online learning in ghana and how to successfully integrate ict in education to improve online learning for students, especially those in urban poor and rural areas in ghana. in a qualitative study, a central phenomenon is the process, key concept or idea that is studied, and a researcher learns more from participants through exploration (creswell, 2018) . data was collected through interviews. a list of post-covid-19 strategies was formulated from extant literature based on respondents' views. the researcher used a semi-structured interview guide comprising of a list of prepared questions related to the research questions to conduct the interview. the instrument was prepared based on recurring themes in extant literature and was screened for accuracy and validity by a researcher in ghana. this allowed the researchers to ask open-ended questions to ensure respondents give their broad perspective about the topic of the study. interviews allow a researcher to probe and get an in-depth meaning of the feelings, perceptions and attitudes of participants (gaffas, 2019) . all tertiary students from urban poor and rural areas who took part in the online learning in ghana formed the target population. fifteen (15) of the students were randomly selected to be part of the study. five (5) students were taken each from the 3 most cosmopolitan regions in ghana (greater accra, ashanti region, and central region) to make up the total sample of the study. the student came from the universities, teacher and nursing training schools in the country (table 1) . students from the tertiary institutions in ghana who were taking part in the online learning in ghana were recruited to find answers to the research questions. an introductory letter was sent to the randomly selected students about the purpose of the study and to seek their consent to be part of the study. after the consent of the study respondents were gained, the researcher scheduled an appointment with each of the participants to conduct the interview. all interviews were conducted in english using the "whatsapp" application. the researcher transcribed the recorded interviews verbatim and the data was analyzed into themes and sub-themes using the nvivo 11.0 software. the interviews were replayed severally to ensure the accuracy of the transcription. the researcher used pseudonyms for each of the participants to ensure confidentiality. almost all the barriers associated with e-learning in literature reviewed earlier were prevalent in the ghanaian context (see table 2 ). the findings of the interviews conducted suggest that the online learning in ghana is not effective enough and is also fraught with a lot of challenges. the section concludes with innovative and some novel solutions found in extant literature to the problems identified in the study and a conceptual framework for transitioning to e-learning. most of the 10 students interviewed indicated that the online learning is the best alternative approach to teaching and learning during this pandemic. however, because of its spontaneous nature, and not an approach to education carefully thought of my school leaders and the government, they felt the online learning was not effective as they hoped for. social interaction an ample amount of the students felt the lack of student-student interaction and teacher-student interaction negatively affected the effectiveness of the course. "oh ok, personally, i think this online learning is not that effective. let me take campus for example, on campus you go for lectures, you meet the lecturer, you will have this interaction, like face-to-face, so that makes you really comprehend what the lecturer is putting across, and aside that you even have the teaching assistants who are willing to help you understand or if you have any difficulty they are able address that issue." [lucy-university] student outcomes only few of the students believed students' outcomes in the online learning would be better than the traditional approach. many believed that the difficulty with internet access and network challenges will result in a negative effect on the outcomes "the student outcome of online learning as compared to the traditional approach is very sad and heartbreaking. let me take the quiz for example, you will take a quiz online, and the system gets jammed, you've really learnt, not that you didn't learn, you've really learnt, and you go, and you're able to answer the questions, let me say if it is out of 20, you are able to get 15 or 10 correct, and then you get an f, it just saddens my heart" [ellen-university] communication some of the students were of the view that sometimes communication between teacher and students is not possible because the e-learning system can go off for a while before it starts functioning. "the intercommunication between lecturers and students is very poor. communication is not good. you will be having a class, and then the network starts misbehaving, meanwhile, the teacher is talking! how do you retrieve the words he has already said? what if the words he has said is the stepping stone to understand the next sentences?" [sandra-nursing training] traditional versus online approach more than half of the participants interviewed believed that the traditional approach to teaching and learning is more suitable for them as compared to the online learning they were experiencing. "ok, what i can say is my university, for example, am reading chinese and political science, so with chinese the lecturer is supposed to write down the characters on the whiteboard for us to really eerrm know the stroke orders. also, there are instances that you need you need to even, eerrm for chinese oral you need to listen to the lecturer, like look at the lecturer's mouth, the expression that she makes for you to be able to understand whatever she is saying. so i think i prefer the marker-board because taking my course, for example, it really helps, yes." [ellen-university] cost one of the major challenge mentioned by almost all of the participants was the financial commitments they have to make to ensure they actively take part of the course. according to the students, the online method of learning was expensive than the traditional approach to learning they experienced. "online learning is quite expensive. so far the has given us a data sim which gives us 1.5 gigabytes every month. but the problem is, this bundle is just something small. it doesn't get us anywhere. i remember my first zoom class was not more than 30 minutes, and 400megabytes was already gone." [ted-university] online platform persistently, the students lamented about the breakdown of the online platform. "with the accessibility to the online platform, i think with that is okay. sometimes you log in, and the site just jams, so that really hurt and when you have quiz to take, and you log in and the system jams and just log you out of the system." [paul-university] study materials the students believed that study materials were readily available to download at any time. "oh okay, every course has its learning materials on the site provided they have not given us already" [ted-university] ict tools a considerable amount of the respondents asserted that ict tools were not given to them prior learning. every student had to get an ict tool for himself/herself to engage in the online mode of delivery. "we use our own ict tools, like phones and laptops. the school does not provide us with ict tools. if you don't have these ict tools, you have to share that of a friend." [joe-university] prior knowledge most of the participants had little exposure to online mode of learning prior to their current online method of learning. "mmmm as for background knowledge i don't have much experience but per our experience of sending messages through e-mail and using other platforms to send documents that is what helped us in using the online learning platform, since we couldn't have an orientation on how to engage in e-learning." [bright-teacher training] internet access accessing the internet was one of the major challenges students mentioned. "good internet access is a huge challenge. so most of the students we adopted that midnight is when we will log into the online learning site to download and upload materials. it is effective as compared to the normal day hours. however, this has affected our sleep" [paul-university] electricity there were mixed views on the availability of electricity for the online learning. some believed lack of electric power affected their learning while others were okay with it. "hmmm electricity is another impediment. recently our transformer got spoilt, and it took almost two weeks for the local government to fix it. meanwhile, tests and studies were ongoing." [mercy-university] education many students indicated orientation of teachers and students on ict for teaching and learning is integral for successful integration of online learning in schools. "for me, i think it is not only students who have to be educated on e-learning. my brother is not ict teacher and has no special skills in ict, but he is also supposed to use online learning platforms. he likes to use social media than to use the online platform for sending documents." [ted-university] provision of ict tools some students called for the provision of ict tools for schools and students who can't afford the tools for themselves. "the government should provide laptops to students in the middle or high school so that by that the time the student comes to the tertiary level, he or she has a laptop and can have access to many things online." [joe-university] motivation the students believed the motivation of both teachers and students would improve the online learning. "personally, i think the government should motivate teachers and students to engage in the e-learning. e-learning can be effective when there is motivation. the government can give allowances to teachers and reduce our school fees". [julius-university] "we students in the rural areas have no joy in this online learning. the challenges are too much. electricity and internet access are a major problem. if the government wants ict to be effective, all these problems should be solved." [prince-teacher training] school leadership students mentioned that school leaders should liaise with the government to improve the online learning. they believed school leaders could act as a channel for their voices to be heard. "school leaders like principals and vc have to eerrm, we have open forum where students bring their grievances on board, so as the src also take eerrm an active role in presenting the pleas and plights of students the school authorities, they also have to make the students' grievances known to the government so that the government can put the necessary measures in place to provide the tools that are needed to help the teachers and the students as well." [ellen-university] 6.4 post covid-19 strategies to promote e-learning providing electricity solar power gadgets and human-powered electricity have emerged as a promising solution to the increasing digital divide as a result of lack of electric supply (wyche & murphy, 2013) . the authors states that although solar photovoltaic-powered devices can charge mobile phones and some ict gadgets for e-learning, human-powered electricity are more preferable because human power is abundant worldwide. human-powered electricity is when human activities such as cranking (by hand) and pedalling (using the legs) are captured to turn a dynamo to generate electrical charges for electronic applications. in kenya, the researchers found the two models effective for providing micro-electricity for electronic devices. jimba & ogundele (2015) opined that solar power supply, biomass, standby power generators and wind power supply are promising solutions to generating electricity for virtual classrooms in nigeria. according to the authors, this makes teaching and learning effective without any disruption in power. hamajoda (2018) is of the same view that solar power supply and standby generators can facilitate e-learning by dealing with the challenges associated with electricity in rural areas. fostering acceptance of e-learning academics are often slow to embrace e-learning but are required to adapt positively to changes resulting from technology (flavell, harris, price, logan, & peterson, 2019) , and one of the best ways to foster adaptive learning of technology is to improve instructor and student self-efficacy (solangi, shahrani, & pandhiani, 2018) . flavell et al. (2019) state that one of the ideal ways to empower academics to be adaptive with e-learning is to engage them in recreational activities such as reading on technological devices for pleasure, playing games, or using social media. this makes users comfortable with the e-learning and also increases their self-efficacy. media literacy (such as the use of social media) can foster acceptance of e-learning and increase digital literacy (nowell, 2014) . collaborative e-learning also promotes exchange of experiences and ideas among students, encourage students to work together, develop their academic competence, increase their self-efficacy, develop their social skills and also fosters both their social and cognitive qualities for e-learning (ngai, lee, ng, & wu, 2018) . a wide range of e-teaching and e-learning methodologies can be introduced to encourage both teachers and students to embrace e-learning such as podcast and vodcast (uren & uren, 2009) . also, to ensure rapid acceptance of e-learning, a clear pedagogical rationale for online teaching which is rooted in instructor's personal philosophy for teaching and learning should be communicated to instructors (donnelly and o'rourke, 2007) . user-need analysis for adaptive e-learning adaptive e-learning has to do with the personalization of e-learning in accordance with individual user's knowledge and behavior, and one way to ensure that is through user-need analysis (agustini, 2017) . developers of e-learning platforms need to perform a user-need analysis; they have to take into account the learning styles of individual learners when creating, selecting, and evaluating e-learning platforms (muhammad, albejaidi, & akhtar, 2017) . according to the authors, user needs can be solicited by employing mix techniques including direct observation, expert review, administering web-based and paper questionnaires, and interviewing teachers and students. online learning should be planned to meet the learner's expectations (hachey & lachapelle, 2018) . alhabeeb & rowley (2018) adds that identifying both student and instructor characteristics for e-learning is a critical success factor for e-learning, "success" for one group does not necessarily connote "success" for another group. learners on e-learning platforms should be thought of as customers by making it easy to use, making effective use of learner's time, and it has to be attractive and comfortable (morrison, 2003) . leaner preferences and experiences are integral for a flexible mode of e-learning delivery (arthur-nyarko and kariuki, 2019, b). teacher's presentations and course materials for e-learning should also be prepared based on students' needs and learning styles (osubor & chiemeke, 2015) . digital literacy of users the training of digital users is instrumental to ensure the successful transition and integration of e-learning platforms (muhammad, albejaidi, & akhtar, 2017) . training has been recognized as having a positive influence on the usage of e-learning systems (solangi, shahrani, & pandhiani, 2018) . training is a critical success factor for e-learning and should be provided for both learners and instructors (alhabeeb & rowley, 2018) . professional development training programs provide teachers with "hands-on" activities, it is essential for them to develop ict competence and technological pedagogical content knowledge for twenty-first-century learning (alt, 2018) . course facilitators should be trained on computer literacy and application for effective e-learning (jimba & ogundele, 2015) . one way to ensure digital literacy for students is to embed digital literacy skills in the curriculum (johnston, 2020) . school leaders can ensure students develop their own skills by supporting them during orientation activities on technology, peer learning, and encouraging them to use personal technology (sharpe & benfield, 2012) . workshops and training on ict for e-learning are crucial for users especially teachers, for a flexibility in online delivery (forsyth, pizzica, laxton, & mahony, 2010) . infrastructure support infrastructure and technical support that provides an opportunity for teachers and students to adopt e-learning should be made available (solangi, shahrani, & pandhiani, 2018) . availability of telecenters which are equipped with computers and internet connectivity in a poor rural community can help in the integration of ict in developing countries (avgerou, 2008) . research findings on ict in ghana revealed that institutions engaged in online learning need technology infrastructural support from the government, such as ict devices (arthur-nyarko and kariuki, 2019, b). the "one student one laptop" policy by the ghanaian government failed to realize its goal and was an expensive investment. heeks (2008 heeks ( , 2012 argues that mobile devices can now be used to surf the internet and create new content, it could be used as an alternative for poor households in accessing the internet for learning purposes to defray the cost of expensive laptops, mobile technologies can be an alternative. school leaders can lassie with the government to also establish e-learning centers in the various tertiary institutions (piña, lowell, & harris, 2018) . the e-learning centers may be staffed with professionals such as technology specialists, media, administrators, and faculty support specialists who provide training support and training for effective engagement with e-learning tools. funding aside fees from students and government which are made available to schools, institutions which are new to online learning can establish e-learning centers and charge some type of fee (convenience fee) or online tuition rate (piña, lowell, & harris, 2018) . funds from the general student population and the government are often claimed by other areas. according to the authors, students who prefer the flexibility of taking online courses are willing to pay this extra fee. the institution can decide to access an online learning fee or a higher online tuition rate, which could serve as direct source of funds to e-learning centers on campus. it is near to impossible to reassign such funds to other areas. the institution receiving such funds would have the capacity to run e-platforms effectively. motivation incentives and reward systems can motivate teachers and students to engage smoothly in e-learning (o'doherty, et al., 2018) . when teachers are rewarded on the time spent instructing students on e-platforms, they are motivated to spend extra time and dedicate more efforts in instructing students. in a study conducted in russia, students decision to engage in online courses was due to available incentives (markova, glazkova, & zaborova, 2017) . incentives for teachers can be in the form of allowances and compensation. students can be provided with free wifi or subsidized cost of internet charge. this would encourage them to spend more time surfing for information on the internet, downloading course materials and uploading assignments. the government of ghana can work with internet service providers in the country such as mtn and vodaphone to provide internet access with high bandwidth to prevent them from withdrawing from e-platforms. supervision and evaluation an uninterrupted technical support is required from elearning users (teachers, students, and administrators) in addressing hardware and software problems associated with the computer application (muhammad, albejaidi, & akhtar, 2017) . rigorous supervision will help address glitches associated with eplatforms to ensure its continual functional. it personnel and e-platform administrators can be assigned to monitor the progress of the online learning to identify and solve all hardware and software challenges. academics involved in e-learning need assistance on shifting from the conventional mode of teaching and learning to a virtual teaching and learning environment (donnelly and o'rourke, 2007) . this assistance can be provided by "e-champions" and ict skilled personnel hired by the school. the authors mentioned that feedback of users on e-platforms are also essential for evaluating the success of e-learning, and this can inform school leaders on what to include in subsequent workshops for e-learning users. also, using data derived from users of eplatforms will help the administrators and designers of the e-platform on how to structure the platform to meet students' needs and the unique culture of the institution. blended learning picciano (2009) define blended learning as a combination of both online and face-to-face interaction. response from the participants suggests that a blended learning approach could help address the differences among students. picciano also mentioned that a blended learning approach could mitigate the challenge associated with geographical distance. students who are far away from their institutions and prefer an online mode of delivery can learn from home through their e-learning platforms while those who prefer the traditional face-to-face interaction can learn at the classroom. onguko (2014) believes that a blended mode of delivery is more ideal for universities who are not yet ready for only online learning. the author indicated that some universities in kenya incorporated a face-to-face session in their virtual classes to address some difficulties connected with online learning. the model serves as a framework for universities and academic institutions for an emergency transition to e-learning. this is adapted from texas tech university health science center el paso (ttuhsc ep) in their response to rapid transition to e-learning during the novel coronavirus pandemic (mulla et al., 2020) . quality e-learning programs require time and unprecedented efforts to ensure successful implementation. the model suggests that for successful implementation and sustenance of e-learning programs, school leaders must first inform faculty leaders about available resources and the best method to ensure the transition; a simulation course should be created, faculty staffs need to be trained and assisted, as well as soliciting their feedback and suggestions; the course can then be submitted; an e-learning compliance committee (elcc) should be set up for peer evaluation and for acquiring copyright permissions to keep the e-learning platform functioning (fig. 1) . the paper adds value to prior literature on ict and online learning by first identifying the challenges associated with online learning in the developing context like ghana during the covid-19 pandemic. the study concluded by highlighting on some practical strategies for the adoption and usage of online learning in institutions in developing countries new to online learning. findings from the study suggest that the mulla et al. (2020) online learning in ghana is fraught with a lot of challenges apart from high cost and infrastructure (asunka, 2008; bean, et al., 2019; cradler & bridgforth, 2002; turk & cherney, 2016) . to address the problem of cost, school leaders are encouraged to establish e-learning centers which can generate funds solely for the purpose of online learning. also, the limited amount of electric supply for e-learning activities can be mitigated by using affordable solutions like solar power gadgets, biomass, and humanpowered electricity which are readily accessible in the country. again, school leaders are to employ recreational approaches like the use of social media and games to increase the self-efficacy of students for online learning. collaborative e-learning for teachers and recreational approaches are essential for fostering acceptance of e-learning by both faculty staffs and students. in addition, to effectively develop an e-platform tailored to the needs users, a user-need analysis has to be performed to know individual preferences and need. the university administration also has to set up professional training programs for teachers to improve their digital literacy. courses needed to improve students' it skills can be mounted during the first year at school or during the few weeks of orientation. moreover, because of geographical distance and individual learning styles and behavior, a blended mode of delivery is preferable such that students can choose either a face-to-face session or an online mode of delivery. it was also identified that the motivation of teachers and students is integral for smooth online learning. school leaders should motivate teachers and students in the form of incentives, allowances, free wifi and subsidized cost of data bundle. the school administrators can laisse together with the government to employ mobile technologies to replace the high cost of laptops. furthermore, because e-learning often experience hardware and software issues, school administrators should put in place a team of experts and technical resources for routine supervision and the evaluation of the elearning program. overall, findings of the study provide novel insights for educators and policymakers for the transition and usage of e-learning in the and post covid-19 pandemic. despite the significant investment of most low-income countries such as ghana in ict, challenges such as access to ict, reluctance by both teachers and students to use ict, network and electricity issues still persists in ghana. in lieu of the online learning activity taking place in most institutions in ghana, the study revisits how ict is fundamental to online learning and education in general. the principal objective of the study was identify possible solutions to some perceived barriers in e-learning and ict integration in ghana. findings from the study suggests that access to ict resources in ghanaian schools is still a challenge. as a result, students perceived the online learning is not effective. according to the students, although the online learning is a good initiative, they prefer the traditional approach as opposed to the online learning, which is fraught with a lot of challenges. the lack of social interactions, poor communication, and poor students' outcomes were associated with the perceived ineffectiveness of the e-learning. the students cited a lack of ict tools, internet, electricity as some of the barriers to online learning. most of the students interviewed did not have prior exposure to online learning or using ict tools to study. the students believed that orientation of both teachers and students on ict for online learning, motivation, and school leadership practices affect the integration of ict in education. the usage of solar power gadgets, the establishment of e-learning centers, acquiring funds from e-learning centers, adoption of blended learning, and providing infrastructural support such as using mobile technologies to replace the cost of procuring laptops are some of the listed strategies for educators to consider. ict plays an integral role in education and the economy of any country, help in resolving the barriers associated with online learning, and facilitates the successful integration of ict in education to enhance online learning. dessalegn & dagmawi (2018) opined that developed countries benefited from the integration of ict in education, unlike developing countries. the fact that higher education plays a vital role in the success of students and countries but there is limited access in africa, online learning can help bridge this gap (lembani, gunter, breines, & dalu, 2019) . it is therefore vital for stakeholders in education to integrate ict in education in ghana, and implement realistic and rigorous ict policies to ensure effective online learning where the needs of both urban, urban poor, and rural students are taken into consideration. the author recommend that future research examine the effectiveness, challenges of e-learning and ict integration in other contexts support their findings in literature with empirical evidence using a quantitative approach with a large sample. the sample of the study is very small, which makes it difficult for generalization. the study only focused on students' perspectives of the online learning in ghana without including the voices of teachers instructing students. also, the participants of the study were only from tertiary institutions in ghana without including pre-tertiary students who were also taking the part of the e-learning. promoting e-learning in distance education programs in an african country the adaptive elearning system design: student learning style trend analysis providing information communication technology support to distance education students: a case of the university of ghana examination on ict integration into special education schools for developing countries e-learning critical success factors: comparing perspectives from academic staff and students jordanian students' barriers of utilizing online learning: a survey study science teachers' conceptions of teaching and learning, ict efficacy, ict professional development and ict practices enacted in their classrooms. teaching and teacher education towards the access to and usage of information and communication technology (ict) in polytechnic education the information technology challenge in teaching senior high school geography in ghana covid-19: halt challenge-ridden online learning in universities -nugs to government learner access to resources for elearning and preference for elearning delivery mode in distance education programmes in ghana the state of ict integration in the early years in ghana schools online learning in higher education in sub-saharan africa: ghanaian university students' experiences and perceptions information systems in developing countries: a critical research review a study on the use of the internet in senior high schools in the cape coast metropolis of ghana barriers to adoption of online learning systems in u.s. higher education effective practices for online tutoring. sacramento: academic senate for california community colleges barriers to the successful integraton of ict in teaching and learning environments: a review of the literature preservice teachers' acceptance of ict integration in the classroom: applying the utaut model the use of ict for teaching and learning in senior high schools in ghana: a study of nungua and presbyterian factors influencing teachers' adoption and integration of information and communication technology into teaching: a review of the literature global perspectives on e-learning: rhetoric and reality recent research on the effects of technology on teaching and learning research design: qualitative, quantitative, and mixed methods approaches sharpening education through the use of information technology and communication technology. sdo quezon: sta education ministry set to release online learning platform for senior high schools the impact of ict in rural education: case study-enugu state enhancing online distance education in small rural us schools: a hybrid, learner-centred model can online learning mitigate rural schools' biggest challenges? the promises and challenges of ict-pedagogy integration in developing countries: the case of ethiopia what now? evaluating elearning cpd practice in irish third-level education empowering academics to be adaptive with elearning technologies: an exploratory case study distance education in an era of elearning: challenges and opportunities for a campus-focused institution. higher education research and development students' perceptions of the impact of egp and esp courses on their english language development: voices from saudi arabia integrating technology for meaningful learning trousse des superhéros téluq : un outil pour les professeurs teachers' use of information and communications technology in education: cameroon secondary schools perspectives towards a topology for ict & e-learning in nigerian rural public primary schools priorities in mobiles-for-development research deriving an ict4d research agenda integrating technology into k-12 teaching and learning: current knowledge gaps and recommendations for future research social studies teachers' views of ict integration measuring the information society-the ict development index. geneva: international telecommunication union. international telecommunication union icel2013-proceedings of the 8th international conference on e-learning: icel 2013. academic conferences limited power and energy generation as a threat for developing virtual classroom in nigerian distance education. international centre for science the shift towards digital literacy in australian university libraries: developing a digital literacy framework cyberschools. englewood: jones digital century mapping out the ict integration terrain in the school context: identifying the challenges in an innovative project web-based instruction: what is it and why is it attrition and achievement gaps in online learning pedagogy and ict use the same course, different access: the digital divide between urban and rural distance education students in south africa student access to and skills in using technology in an open and distance learning context. the international review of research in open distance learning an examination of distance learning faculty satisfaction levels and self-perceived barriers quality issues of online distance learning executive summary: ghana-pan african report on technology integration e-learning strategies: how to get implementation and delivery right first time investigating perceived barriers to the use of open educational resources in higher education in tanzania. the international review of research in open and distance learning an assessment of the ict situation in senior high schools in the upper west region of ghana challenges in development of elearning systems in higher education of the developing countries novel coronavirus, novel faculty development programs: rapid transition to elearning during the pandemic innovating an integrated approach to collaborative elearning practices in higher education: the case study of a corporate communication e-platform using disruptive technologies to make digital connections: stories of media use and digital literacy in secondary classrooms barriers and solutions to online learning in medical education -an integrative review cross-cultural differences in undergraduate students' perceptions of online barriers jifunzeni: a blended learning approach for sustainable teachers' professional development e-learning functional model: a technology-based teaching method for providing access to sustainable quality education blending with purpose: the multimodal model leading and managing e-learning: what the e-learning leader needs to know online learning in a south african higher education institution: determining the right connections for the student. the international review of research in open and distance learning the role of information and communication technology (ict) in higher education for the 21stcentury. the science probe institutional strategies for supporting learners in a digital age how many ways can we define online learning? a systematic literature review of definitions of online learning online education in emerging knowledge economies: exploring factors of motivation, de-motivation and potential facilitators; and studying the effects of demographic variables factors affecting successful implementation of elearning: study of colleges and institutes sector rcj saudi arabia the barriers and needs of online learners high-tech or high-touch? online learning and independent higher education understanding structural and cultural school characteristics in relation to educational change: the case of ict integration perceived online education barriers of administrators and faculty at a u.s. university in lebanon ict-enhanced teacher standards for africa (ictetsa) covid-19 educational disruption and response eteaching and elearning to enhance learning for a diverse cohort in engineering education the effects of ict on achievement: criticizing the exclusion of ict from world bank's education sector strategy 2020 powering the cellphone revolution: findings from mobile phone charging trials in off-grid kenya publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations acknowledgements the author sincerely expresses his special thanks to professor yu zeyuan (southwest university, china), robert kyei (nsawam presby basic school), and edna agyemang (university of ghana). key: cord-011188-59hn9wsv authors: bruce, lori title: a pot ignored boils on: sustained calls for explicit consent of intimate medical exams date: 2020-03-09 journal: hec forum doi: 10.1007/s10730-020-09399-4 sha: doc_id: 11188 cord_uid: 59hn9wsv unconsented intimate exams (uies) on men and women are known to occur for training purposes and diagnostic reasons, mostly during gynecological surgeries but also during prostate examinations and abdominal surgeries. uies most often occur on anesthetized patients but have also been reported on conscious patients. over the last 30 years, several parties—both within and external to medicine—have increasingly voiced opposition to these exams. arguments from medical associations, legal scholars, ethicists, nurses, and some physicians have not compelled meaningful institutional change. opposition is escalating in the form of legislative bans and whistleblower reports. aspiring to professional and scientific detachment, institutional consent policies make no distinction between intimate exams and exams on any other body part, but patients do not think of their intimate regions in a detached or neutral way and believe intimate exams call for special protections. uies are found to contribute to moral erosion and moral distress of medical students and compromise the sacred trust between the medical community and the general public. this paper refutes the main arguments in favor of the status quo, identifies a series of harms related to continuing the current practice, and proposes an explicit consent policy for intimate exams along with specific changes to medical school curriculum and institutional culture. because patients are the rights-holders of their bodies, consent practices should reflect and uphold patient values which call for explicit consent for intimate exams. from hippocrates to percival, classic medical scholars demonstrated "virtually no appreciation of a patient's right to consent" (beauchamp 2011, p. 515) . they avoided disclosures perceived to "harm or upset patients" and viewed patients' bodies as vessels for research and physician education; a view not to be superseded by a patient's preferences (beauchamp 2011) . prioritizing research and education over a patient's preferences implies utilitarianism, when the goodness of an action is rooted in creating the most positive outcome for the greatest number of people. with the advent of bioethics, clinical medicine moved to a deontological frame in which the goodness of an action is based on whether that action in itself is right or wrong. bioethics also phased out the idea that physicians' perceptions of a patient's best interests should supersede the patient's own values. informed consent allows the patient to decide what happens to their body based on an understanding of the proposed procedure, and this concept was "imposed on medicine through nonmedical forms of authority such as judges in courts and government officials in regulatory agencies" (beauchamp 2011, p. 515) , underscoring the dichotomy between physicians' and nonphysicians' perceptions of rights. consent is now a well-established tenet of medical practice. physicians must request a patient's permission before conducting research or performing procedures on them. consent "must be rooted in autonomous choice by patients" (beauchamp 2011, p. 518) and requires an explanation of the procedure and disclosure of the risks and benefits. medicine is comprised of committed, talented individuals who dedicate their lives to healing the wounded and comforting the dying; however, certain practices still demonstrate concerning perceptions. recent surveys and reports reveal that some physicians and medical students still conduct unconsented intimate exams (uies) on male and female patients. unconsented rectal exams on men and women have been reported for training purposes (reese and monrouxe 2011; coldicott et al. 2003a, b) and diagnostic reasons (chan 2017) . unconsented vaginal exams have also been reported, mostly during gynecological surgeries (schniederjan and donovan 2005; barnes 2012; picard 2018) but also during abdominal surgeries (graham 2019; kelly 2019) and in efforts to rule out sexually transmitted disease in a patient with uncontrolled vomiting (laird 2019) . the referenced studies surveyed hundreds of medical students in the u.s., england, wales, and australia. in one study, nearly three-quarters of students believed that patients did not consent to their training exams (schniederjan and donovan 2005) . in another study, patient consent for intimate exams was not acquired by half of the third-year students or one-third of patients examined by second-year students (coldicott et al. 2003a, b) . medical students are eager to participate in procedures and quickly realize the delicate balance of gaining new skills while respecting patient autonomy. students are also careful to navigate complex organizational dynamics inherent to the academic culture of their medical schools. a student refusal to participate in an unconsented intimate exam (uie) may be viewed as an act of personal embarrassment or unwarranted disobedience. because students worry about jeopardizing their budding careers, they sometimes participate in clinical experiences that conflict with their personal moral compasses. fraught dynamics between student and supervisor are central to the controversy of intimate exams on patients. a pelvic exam is a standard precursor to gynecologic surgery to ascertain the position and mobility of the reproductive organs. the exam requires the insertion of one or two fingers into the vagina to palpate the ovaries and uterus. rectal exams similarly require digital penetration with the aim of detecting abnormalities in the rectum, prostate, and abdomen. medical schools consider these digital exams as standard training procedures and typically require all students to practice them during their third-year clerkships. some institutions have policies in place requiring explicit consent for intimate exams, but they are the exception; most institutions have no such policies. physicians often assume consent for these training exams, believing it is covered under general admissions clauses-or vague language on consent forms-relating to student participation in teaching hospitals. consent forms often require the attending physician to hand-write the major risks of a specific procedure but otherwise follow a standard template. patients and members of the general public are often shocked to learn that physicians and medical students may perform intimate exams on them without explicit consent and believe such exams meet the criteria for battery, malpractice, or sexual assault. whether these exams meet any such criteria is contested, but this tension underscores the dichotomy between societal and medical views of the intimate regions of the body. while the medical professional may aspire to a professional, scientific detachment, patients simply do not think of their intimate regions in a detached or neutral way and tend to have an inherent conviction that consent practices should mark this distinction. for several decades, ethicists and some number of physicians have expressed concern about uies. in 1989, cohen et al. emphasized the uniqueness of the pelvic exam and how patients have a right to refuse participation in these training exams. since then, uies continue while opposition increases. singer considers uies "definitely among the most egregious of… ethical issues" in medical schools (boyles 2003) ; caplan (2018b) recently called for "an abrupt and immediate halt" to these exams; and friesen (2018a) refers to uies as "immoral and indefensible" (p. 298). adashi, a former dean at the warren alpert medical school of brown university, states, "viewed in hindsight, it is difficult to see how the conduct of unapproved pelvic examinations by medical students could have been rationalized, let alone condoned" (2019). medical students are increasingly raising concerns to their ethics advisors (friesen 2018b; caplan 2018a) , researchers (reese and monrouxe 2011), the media (tsai 2019), and state legislators (barnes 2019) . when medical students feel they must warn the public about uies in their medical institution, their actions meet the criteria for whistleblowing which: results from a malfunction of the ethical environment of the organization to focus on its accountability for the safety and welfare of the patients. individuals [who engage in whistleblowing] believe they must take a stand for the wrongdoing in the organization. (lachman 2008, p. 265) some of these whistleblower reports may have contributed to changes in association guidelines. medical associations have developed consensus statements recommending bans on unconsented pelvic exams (association of american medical colleges 2003; american college of obstetricians and gynecologists 2009); however, these statements are advisory and incomplete. associations simply do not have the capacity to compel systemic change, as evidenced by institutions' inaction. in response, nine states recently passed legislation related to unconsented intimate exams, and other states are considering similar measures. this paper describes current laws and association guidelines related to uies then explores the main arguments in favor of continuing the status quo. next, the paper examines the implications of continuing the status quo and closes with a series of practical recommendations to curtail the occurrence of uies. the association of american medical colleges (aamc), american college of obstetricians and gynecologists (acog), and the american medical association (ama) have called for explicit, mandatory consent before pelvic exams. the aamc's statement (2003) declared, "medical students and residents not only must acquire clinical skills, they must also learn to always treat their patients with respect and dignity. recent reports have suggested that medical students are performing pelvic examinations on women under anesthesia, without their knowledge and approval. aamc believes that such practice is unethical and unacceptable" (2003). acog's 2011 opinion includes, "pelvic examinations on an anesthetized woman that offer her no personal benefit and are performed solely for teaching purposes should be performed only with her specific informed consent obtained before her surgery" (p. 358). the ama's code of medical ethics opinions on patient-physician relationships states that patients' "refusal of care by a trainee should be respected in keeping with ethics guidance" (2001, p. 87) , and a statement by the ama council on ethical and judicial affairs calls for explicit disclosure of student involvement in exams on anesthetized patients (2001) . nurses have also expressed concern: fourteen nursing organizations banded together to urge the american academy of medical colleges to require explicit consent (wilson 2005) . the nine states who passed laws banning some form of intimate exams are iowa, illinois, utah, oregon, maryland, virginia, new york, california, and hawaii. other states are also considering or have proposed legislative action. utah and maryland's laws are gender-neutral but most other states explicitly specify unconsented pelvic exams on women. uies are also under scrutiny in other countries. new zealand has a national consensus statement requiring written consent (bagg et al. 2015) , and france has taken steps to ban both unconsented rectal and pelvic exams (the local 2015) . associations in other countries, including canada (liu et al. 2010 ) and the united kingdom (royal college of obstetricians and gynaecologists 2015), have explicit consent guidelines but they are limited to women's intimate exams. as mentioned, medical associations, legislators, ethicists, and nurses-along with some physicians and medical students-have argued in favor of explicit consent of intimate exams. arguments in favor of the status quo (which we define as failing to explicitly consent for intimate exams) are limited to those made by some physicians, institutions, and medical students. this section defines and discusses arguments in favor of the status quo. some medical students voice disinterest in the importance of consent (cohen et al. 1988 ). one medical student called explicit consent an "attempt to justify the obsession with political correctness," opining that explicit consent is an inappropriate overreaction to a necessary training procedure (bhangu 2003 (bhangu , p. 1326 . some physicians also find that explicit consent is simply not a priority. citing his inclination as a "policy minimalist," george washington university's ob-gyn chairman, john larsen, expressed disinterest in codifying explicit consent (goldstein 2003) . interestingly, ubel, a physician who examined uie practices in teaching hospitals in a 2003 survey of 401 medical students, found that completion of the obstetrics/gynecology clerkship is associated with decreased interest in informed consent (ubel 2003) . after this clerkship, students were also "less likely to think consent for rectal examinations was important" (p. 578). in a follow-up interview, ubel stated that the experience of participating in "exams without detailed consent on anesthetized patients desensitized doctors about the need for patients to grant consent" (goldstein 2003 ). this view is in contrast with women who "almost universally [feel] that pelvic examinations that are conducted by medical students while the woman is anesthetized should occur only after the woman has given her permission to the students' supervisors" (ubel 2003, p. 575) . to claim that explicit consent is unnecessary is in defiance of the fundamental principles of autonomy and self-determination. adult patients have the right to self-determination: "the right to make the ultimate decision concerning what will or will not be done to their bodies" (annas et al. 1977) . along with this right comes the right to refuse procedures. "courts have… declared that both the common law and the united states constitution protect an individual's right to refuse medical treatment" (annas et al. 1983, p. 918) . similarly, the ama's ethical and policy guidelines voice respect for the patient's refusal of an exam by a trainee (2016). the idea that explicit consent for intimate exams is optional therefore fails to meet the most basic tenets outlined in law, bioethics principles, and association guidelines. another argument against explicit consent relates to discomfort felt by novice medical students when performing intimate exams. such exams are "anxiety-or fear-provoking for the novice learner, [and they] often feel embarrassed or even uncomfortable" (nelson 2016, p. 2) . some therefore say there is a benefit to uies because it is easier on the medical student to learn the exam while they do not have to consider student-patient dynamics. if the medical student makes a mistake or does something that would be physically or emotionally uncomfortable for the patient, the student will not be embarrassed by a conscious patient's reaction. however, there are several points against this argument. first, teaching institutions have a duty to train their medical students, but training is secondary to their primary responsibility of the patient's health and well-being. second, part of medical school is learning to exhibit professionalism with patients even when the student may be uncomfortable. third, there are many ways of learning intimate exams to help students overcome their nerves before direct patient contact. pressure-sensitive pelvic mannequins, training videos, and gynecologic teaching assistants (gtas) contribute to comprehensive training. gtas are women "trained on how to instruct the pelvic exam and allow students to perform the exam on them" (nelson 2016, p. 3) . gtas are used in the u.s. as well as the netherlands, sweden, belgium, australia, and the uk (walsh 2013) . women with a history of sexual violence are at risk for pronounced fears, anxiety, pain, and discomfort during intimate exams (qaseem et al. 2014) , and it is reasonable to assume men with similar histories would have similar reactions. other populations who experience pronounced levels of distress during intimate exams include people with disabilities, those who meet the criteria for obesity, and members of sexual and gender minorities (potter 2015) . in the u.s., the prevalence for sexual violence is 33% for women and 17% for men (national sexual violence resource center 2018); the prevalence for disabilities is approximately 20% (center for disease control and prevention 2013); the prevalence of obesity is nearly 40% (centers for disease control and prevention 2016); and those who identify as lgbt + in the u.s. account for approximately 5% of the population (family equality council 2017). medical students ought to develop sensitivity and understanding for these vulnerable populations which comprise a notable portion of the patient population. gtas, sexual assault counselors, and experts in lgbt + ethics, and other experts can and should be consulted to help students learn to communicate sensitively with vulnerable patients during intimate exams. for instance, gtas could expand their teaching to role-play as anxious patients so that students can practice their communication skills in a forgiving environment. sexual assault counselors uniquely understand the trauma felt by survivors of sexual assault and can offer guidance in terms of vocabulary and alternative positioning of the patient body to more fully empower these patients. experts in lgbt + ethics can similarly be consulted to help medical students more fully understand how current medical practice may marginalize these communities and how to more fully provide empathic care. medical schools can carefully plan their teaching modules so that students can participate in these lower-stress learning experiences-and learn basic patient interaction skills and consent procedures-before they perform the exam on a patient. in short, there are obvious, uncomplicated means to help students overcome their nerves and increase competency, and student nervousness should not take priority over the right of a patient to decide what will happen to their body during a medical procedure. some argue that students must perform extensive numbers of intimate exams because the exams are a necessary component of regular patient care. however, in their final recommendation statement: screening for prostate cancer, the u.s. preventive services task force (uspstf) recommends that men can either forgo a prostate exam or "make an individual decision about whether to be screened after a conversation with their clinician about the potential benefits and harms" (2018). similarly, physicians are now questioning the "very utility of the traditional pelvic examination" (adashi 2019). a recent study reviewed data from 2.6 million women aged 15-20 years old and found that 54.4% of their pelvic exams "were potentially unnecessary… and may cause harms such as false-positive test results, overdiagnosis, anxiety, and unnecessary costs" (qin et al. 2020, p. e2) . the american college of obstetricians and gynecologists advocates "against performing screening pelvic examinations in asymptomatic, nonpregnant, adult women" (2018, p. e174). similarly, the uspstf's final recommendation statement for gynecological conditions states, "it is unclear whether performing screening pelvic examinations in asymptomatic women reduces morbidity and mortality" (2017). as medicine has evolved, the need for frequent, across-the-board digital intimate exams has lessened. this development especially decreases the need for frequent training exams by students in their clerkships. some say that uies should continue because an explicit consent policy would substantially reduce the number of training exams, resulting in poorly trained physicians. in other words, if patients are asked, the belief is that they would likely refuse, so physicians may avoid the consent conversation so as not to jeopardize training opportunities. however, a recent survey of over 100 women indicated that 62% of women would consent if asked, and only 14% were sure they would refuse (wainberg et al. 2010) . other surveys have similar positive feedback from patients related to their willingness to participate (friesen 2018a) , and students also have additional training opportunities in clinics and during training sessions with gtas. in sum, explicit consent is an unlikely barrier to physician training. some physicians argue that explicit consent for intimate exams is unnecessary because intimate regions of the body are no different than any other body part. in response to coldicott's landmark study on intimate exams (2003a, b), kaushik, an ophthalmic surgeon, argues in favor of the status quo, believing it is "dangerous" to consider intimate exams differently, and that "examination of the fundus of the uterus is just as intimate for a gynaecologist [emphasis added] as examining the fundus of the eye is to an ophthalmologist" (2003a, b, p. 1327) . this thinking contributes to physician preference for a standard, genericized surgical consent form for all surgeries. yet other physicians understand that the frame of reference should not be about whether the physician finds the exam to be intimate. rather, the decades-long conflict relating to uies relates to how these exams are perceived by the patients themselves. women's pelvic exams are "particularly threatening" to the patient (cohen et al. 1988 ) and "in the area of gynecologic care [patients' right to refuse student exams] takes on heightened sensitivity" (cohen et al. 1988) . and unlike eye exams, some women may find the standard gynecologic exam to be "disempowering, abusive, and humiliating" (bates et al. 2011, p. 652) . the american college of physician's clinical practice guideline also acknowledges the uniqueness of pelvic exams, noting how women may experience "fear, anxiety, embarrassment, pain, and discomfort" (qaseem et al. 2014, p. 68 ) during a pelvic exam. men may also express embarrassment (myers et al. 1996) , shame (consedine et al. 2007) , and mistrust (robinson 1996) during rectal exams, and there is no literature relating to women's or men's shame or embarrassment during eye exams. these responses to intimate exams contribute to lower rates of doctor visits which may predispose patients to poorer health in the long term. as discussed, women with a history of sexual violence (along with patients with sexual and gender minority status and other groups described earlier) are even more likely to experience distress during intimate exams. ubel stated, "we don't see a pelvic exam as having any sexual content at all, but that's not how other people perceive it" (goldstein 2003) . "there's no way a physician would ever equate a pelvic exam with rape-there is no rape content to it. but the fact that someone else perceives it that way makes it important" (goldstein 2003 ). an unconsented intimate exam may feel like a sexual violation to patients. when medical school administrations consider the values they wish to promote in the next generation of physicians, they must consider patient perception. even if some physicians and institutions have ambivalence over explicit consent for intimate procedures, patients, the general public, and others have repeatedly issued calls for explicit consent. surveys indicate that women are unanimous in their stance against unconsented intimate exams by students when anesthetized (bibby et al. 1988) , and this is another signal that the medical community cannot gloss over this issue by, as an example, comparing the uterus to the eye. even though a patient's body may be "naked on a brightly lit table for all to see" (friesen 2018a, p. 299 ) during a procedure, the patient should still have rights to decide what happens to their body. in a recent discussion forum about uies, a student wrote: as a female student not yet in the medical field, i am disturbed to hear that by consenting to surgery, i risk having someone literally in my vagina without consent for purposes that benefit only [the students], and not me. are patients really viewed as a teaching tool rather than a human being? that i will be splayed and sliced during the procedure doesn't mean that additional indignities are acceptable. it's still my vagina, even if i am naked and unconscious. i didn't lend it to anyone to practice techniques (friesen 2018a, p. 300) . when people repeatedly express a fervent and heartfelt preference for how their bodies are treated, and this preference differs from physicians' perceptions, physicians should carefully examine why their perceptions should prevail. clearly, standard consent forms fail to give patients the granularity they need to properly consent. some argue that explicit consent is unnecessary because uies rarely occur. however, greger, a physician who has researched unconsented exams, stated, "if they have five medical students on an ob-gyn rotation, they aren't going to let one do it and not the other four… i never heard of anyone out of some kind of respect for the patient just limiting it to a few" (goldstein 2003) . and, following recent discussions with concerned medical students, caplan agreed: "sometimes, more than one student [on their gynecology rotation] will practice the exam, with many sets of gloved fingers in the patient's vagina without their knowledge" (caplan 2018a) . even if physicians rarely practice or condone uies, some do, and when institutions fail to enact explicit consent on the grounds that uies are infrequent, they send a message to the public that the affected patients' experiences are unimportant. institutions should not fear that explicit consent will look badly on them. instead they should view them as necessary protections for patients and know that patients are then more likely to relax in a clinical setting because of the policy's existence. in sum, the most common arguments in favor of the status quo generally lack substance and should not pose barriers to explicit consent. critics of explicit consent point to perceived harms of intimate exams but fail to mention harms of continuing the status quo. critics may argue that patients are not harmed if they are anesthetized (and therefore unable to remember the procedure), and perhaps that is true. however, some patients have recently woken during an uie under anesthesia. ashley weitz, a young woman in utah, sought emergency care for uncontrolled vomiting and when the physician suggested a pelvic exam to rule out a sexually transmitted disease, she explicitly declined (laird 2019) . weitz passed out from the sedative effects of phenergan then "woke up screaming" (laird 2019) because the physician was performing a pelvic exam. a survivor of childhood sexual assault, weitz was so traumatized by the unconsented exam that she became an advocate for a law banning the procedure in her home state. weitz is not the only patient to wake during an intimate exam (mcdermott and johnson 2019), and the idea that patients are unaware under anesthesia is also increasingly refuted: a large u.s. multicenter study found that intra-anesthetic awareness occurs in nearly 2000 cases every year (sebel et al. 2004) . patients with awareness during surgeries most commonly report "hearing conversations… but there are many cases where they report pain, paralysis and anxiety because of the fact that they feel helpless [and some] patients describe this situation as the worst experience they ever had in a hospital" (kotsovolis and komninos 2009). one compelling report is from an anesthesiologist who, as a patient, was aware of aspects of his urogenital surgical experience and found his experience "terrifying" (peduto et al. 1994) . therefore, the assumption that patients are unaware of their unconsented exams may be questionable. other patients have been informed of an uie after they woke from anesthesia. one such patient, a nurse, woke from her laparoscopic stomach surgery in 2017 to be informed by a medical student that her cervix seemed abnormal (graham 2019) . she, too, is a survivor of sexual assault and was so distraught by the procedure that she tried to bring charges but was told by local attorneys that state laws would not allow her to prosecute the hospital. definitions of medical malpractice vary by state, but medical malpractice occurs when a physician fails to disclose the risks of a surgery, provides substandard medical care, and subsequently injures the patient (bal 2009 ). this nurse-patient's sense of violation and trauma could be considered injurious to her and may therefore meet the definition of medical malpractice. unconsented intimate exams typically occur on unconscious patients. after all, few would ever perform an unconsented intimate exam on a conscious patient because such an act is far outside of ethical standards. and yet there are recent cases of unconsented intimate exams on conscious patients. in some of these cases, the patients denied consent before the procedure but felt too overwhelmed to advocate for themselves during the exam. one patient, a physician from maryland, was at his oncologist's office for a manual rectal exam related to his recovery from prostate cancer. '…the doctor turned to a med student and said, 'why don't you go ahead.' the student dug right in, caving to the intense pressure med students are placed under during training. stern, himself a doctor, was flabbergasted-not only because the doctor hadn't asked permission, but because stern had expressly denied the student's own request to perform the exam not 10 min prior. "it was terrible," says stern now. "i was awake. i'd said no. but the trainee went ahead anyway, and neither of them spent any time telling me why they thought it was useful. "a patient shouldn't have to meet the needs of the provider," stern goes on. "if he decides it's appropriate to help, great. but he needs to be asked" (redfearn and cao 2004) . while some may say that the patient could have spoken up, consent "must be given freely [and] validity is easily threatened if the patient… is asked for consent at an inappropriate time or is coerced" (uk department of health 2009). in stern's case, the consent process was poorly executed and failed to protect the patient. it is interesting to note that the patient was a physician, someone who would seemingly feel most empowered to refuse a student exam, and yet even he felt constrained by hierarchical dynamics. other cases of uies on conscious patients may occur when the patient is unaware of the exams. in one such case (rees and monrouxe 2011), a male patient was conscious but unaware of receiving sequential digital rectal exams by several students since a sheet divided him from the students. two of the students discussed this case as part of a recent multicenter study on intimate exams. one of the students stated: …they just had a queue of medical students doing a rectal examination he wasn't consented but… you don't have the confidence to say 'no' you just do it…i think there was [sic] about three or four medical students… [the patient] had no idea that we were there and… none of the theatre staff spoke to him about what was happening (rees and monrouxe 2011, p. 268) . these cases find their way into public dialogue. as cases of conscious and unconscious uies become more widely known, patients may suspect their physicians are failing to fully disclose what will happen to their bodies during an exam. the fear that physicians may not share a sense of primary responsibility for their patients' health and well-being can damage the relationship between the general public and the medical community and lead to legal protections. medical students can also experience harm when the status quo goes unchallenged. as discussed, ubel's study found that medical students were less supportive of explicit consent after their obstetrics and gynecology clerkship. no other rotations were reported to have this effect on medical students. …this decline [in valuing the importance of informed consent] is not gradual and inexorable, but instead… it is associated with specific experiences. students who have completed obstetrics/gynecology clerkships place significantly less importance on seeking permission from women… (ubel et al., 2003, p. 578 ). this shift in perceptions of consent is an example of ethical erosion, an attrition of values, leading to decreased respect for the patient's autonomy and emotional detachment. this shift may be attributable to the hidden curriculum, described by mahood (2011) as: a socialization process [by which] norms and values transmitted to future physicians often undermine the formal messages of the declared curriculum… [it] consists of what is implicitly taught by example day to day, not the explicit teaching of lectures, grand rounds, and seminars. students move from being open-minded to being closed-minded; from being intellectually curious to narrowly focusing on facts; from empathy to emotional detachment; from idealism to cynicism; and often from civility and caring to arrogance and irritability. this erosion of empathy and "vanquishing of virtue" is repeatedly documented in studies of physicians in training (p. 983). feudtner et al. (1994) have examined ethical erosion of medical students during their clerkships. nearly all of the affected students in their study overheard physicians disparage patients, and over half observed members of the clinical team engage in unethical behavior and felt that their own ethical principles suffered during their clerkship. when students are faced with ethical dilemmas, feudtner et al. find that "for at least a sizable minority, exposures to… [ethical] dilemmas coincide with deterioration of the students' ethical self-identities" (1994, p. 677) , and those who saw others participate in unethical behaviors were increasingly likely to do so themselves. ethical erosion is not the only ill effect on medical students when they perform uies on patients. the experience of some students may be more accurately described in terms of moral distress, a "response to a perceived conflict between what one is expected to do and what morality requires" (weber 2016, p. 249) . moral distress may arise "when one knows the right thing to do, but institutional constraints make it nearly impossible to pursue the right course of action" (jameton 1984, p. 6 ). moral distress is a "uniquely painful phenomenon… frequently associated with feelings of being powerless" (tigard 2019, p. 605) , and it correlates with compassion fatigue and high turnover rates. when shawn barnes (2012) was still a medical student, he felt coerced by institutional constraints to perform unconsented pelvic exams. he stated: for 3 weeks, four to five times a day, i was asked to, and did, perform pelvic examinations on anesthetized women, without specific consent, solely for the purpose of my education. [when] the patient was asleep, the attending or resident would ask me to perform a pelvic examination on the patient for educational purposes. to my shame, i obeyed… my medical education experience has reinforced the notion that the medical student should not question the practices of those above him or her. i was very conflicted about performing an act that i felt was unethical, but owing to both the culture of medicine and my own lack of courage, i did not immediately speak out against what i was asked to do by residents and attendings" (barnes 2012, p. 941 ). barnes felt powerless at the time but later chose to take action and helped to pass hawaii's law banning unconsented pelvic exams. symptoms of moral distress can ensue after direct or indirect participation in ethically questionable behavior, so nurses and other members of the clinical team may also experience moral distress when they witness or facilitate uies. it is not unreasonable to assume that indirect participants of uies may also experience ethical erosion. thus, a move to an explicit consent process is not only good for the patient; it is also beneficial for the health and well-being of students and practitioners because explicit consent safeguards them from ethical erosion and symptoms of moral distress. when institutions decline to make consent explicit, they are a complicit party to ethical erosion and moral distress. and when institutions embrace an explicit consent process, they signal to their students and practitioners that they care about their well-being and are willing to make a small policy change to protect students' and practitioners' mental health and well-being. in sum, the continuation of the status quo results in a series of harms. harms may occur to patients who are aware of having received an uie, to medical students who are directed to conduct these exams, and to other members of the clinical team who watch or facilitate the procedure. harms may also occur to the fragile relationship between the public and the medical community. while the process of explicit consent may seem to be a trivial, burdensome, and perhaps uncomfortable administrative task, failure to acquire explicit consent inevitably harms some number of students, clinicians, and patients-and tarnishes the medical community's relationship with the general public. when medical schools enact an explicit consent process, they demonstrate the institution's respect for the patient as the gatekeeper to their body, recognize medical students' worth, and bolster their relationship with the general public. as discussed, most unconsented intimate exams are for training purposes on anesthetized patients during urological and obstetrical/ gynecological clerkships, and some uies have occurred for diagnostic purposes, on conscious patients, or during abdominal procedures. a sufficient resolution therefore requires explicit, institution-wide policies and also requires time, a precious commodity in medicine. mercurio (2018) reflects on the physician's daily challenges, stating: we know what we ought to do. the right to informed consent or permission, for example, has important ethical underpinnings… most often, we fall short of our ethical ideals not from lack of deliberation, poor analysis, or inadequate understanding of ethical principles. we fall short because of time (p. 3). these words should be taken to heart. even when time is constrained (and it always seems to be), physicians should not overlook or gloss over the consent process with the patient. indeed, the aspect of time is a fundamental determinant of resolving the decades-long conflict of uies. we spend our time on what we value. spending time on explicit consent sends a signal to all parties that a patient's permission is sacred and valued. the resolution to uies requires a multi-tiered response. first, institutions should develop an explicit consent policy for all intimate exams. second, medical school curriculum should support the explicit consent policy. third, institutions should consider top-down changes to bolster ethics across the organization. institutions should create an explicit consent policy for all intimate exams regardless of the patient's gender, whether the patient shall be conscious or anesthetized during the procedure, and whether the exam is diagnostic or for training purposes. if the patient is unconscious and in a medical crisis, physicians can presume consent if and only if the patient's condition is so dire that the intimate exam cannot wait until after the patient regains consciousness. finally, a patient should not receive an intimate exam unless it is indicated for the patient's underlying medical condition; a patient should never fear having an unconsented intimate exam when they are, for example, undergoing a cardiac procedure. additionally, some consent forms may include a vague reference to student participation but may not specify whether a student intends to conduct an extra exam explicitly for training purposes. the presence of students-and the intent of their exams-should be made explicit to the patient. the design of the consent form matters. as discussed, some hospitals use a standard surgical consent form. these forms should be modified to document explicit consent for intimate exams. consent forms are also sometimes in electronic form. while some patients may prefer this method (winter et al. 2016) , electronic consent should not replace the physician-patient conversation. policy logistics also matter. the explicit consent policy should be publicly available and easily accessible so that all interested parties, including legislators and members of the general public, can locate it and have confidence in its existence. also, when institutions care about the intent of a policy, they invest in training and prioritize periodic evaluations. periodic reviews which thoughtfully examine the successes and failures of the policy could be employed to gauge progress. communications between physician and medical student also matter. students rarely witness the consent conversation between the attending and the patient, so they do not always know whether explicit consent was acquired. efforts should be made to allow students to observe (and learn) the consent process when possible. when students are unaware of the consent conversation, and are subsequently told to perform a student exam, they worry they are on shaky ethical ground. students may not feel empowered to directly ask their supervisor whether explicit consent was obtained; they do not wish to challenge their superior's actions. to clear the air, once the physician has completed the explicit consent process with the patient, the physician should unambiguously inform students that explicit consent was obtained. this brief conversation between physician and medical student seems like a banal administrative task but it will ameliorate students' misplaced guilt, shame, and distress and provide enormous relief to them. spending time on these conversationsand including students in the conversation whenever possible-underscores the importance of consent to students and the rest of the clinical team. enhancements to the medical school curriculum will better prepare students to perform intimate exams. as discussed, some medical schools already employ complementary methods to learn intimate exams. these methods should become standard in all medical schools to help students learn intimate exams before patient contact, and the gta position could be expanded to include role-playing. and as mentioned earlier, experts such as sexual assault counselors should be consulted to learn how to best support the sizeable patient population with risk factors predisposing them to anxiety or discomfort during intimate exams. the inclusion of these expert voices honors the psychosocial complexities inherent to the patient population. as discussed, ubel's survey (2003) found that the completion of the obstetrics/ gynecology clerkship is associated with students' decreased interest in informed consent-a trend not found after any other clerkship. medical schools should examine the dynamics of this rotation to determine root causes and take action to reverse this trend. medical training should also teach students how to have a consent discussion with patients. informed consent is a learned skill that can improve with practice and (as discussed) should be incorporated in students' training experiences with gtas. mercurio (2009) wrote a thoughtful reflection to help physicians request access to the bodies of newly deceased children for training exams. his reflection demonstrated how the use of sensitive, respectful language can build rapport with patients and families and provided guidelines which give structure and confidence to physicians also wishing to have these conversations with families. similarly, guidelines could be developed to help physicians request consent for intimate exams and give them confidence during these sensitive discussions. medical school curriculum should provide bioethics education to students before and during clerkship rotations. discussing cases in advance-and paths to resolution-will bolster students' understanding of ethical principles, help them understand how to navigate ethical conflicts before they arise, and may build their confidence. medical schools should also tell students where to turn when they experience an ethical conflict. medical schools should offer multiple channels for students to seek counsel; these delicate conversations require rapport between student and mentor. during their clerkships, students should have avenues to report an ethically challenging encounter. some medical schools have anonymous upward feedback mechanisms. medical schools could also consider allowing a student to opt out of a procedure on ethical grounds. physicians may elect a conscientious objection when they face an ethical challenge, and medical students should also have the means to refrain from an ethically fraught procedure without repercussions. student access to conscientious objection is not widely available but is offered in some u.s. institutions (card 2012 ). even with enhanced medical school education, students may not feel at liberty to refrain from conducting an unconsented intimate exam for training purposes. in coldicott's study (2003a, b) , many students felt unable to refuse their supervisor's call to perform an uie. similar concerns were voiced by medical students in rees and monrouxe's open-ended survey about professionalism in medicine (2011). this study collected reflections from 200 medical students and when asked generally about ethically charged circumstances, many students elected to discuss their experience with unconsented prostate and vaginal exams. one student who performed an uie remembers feeling "almost star struck that i was in theatre for the first time" (p. 266). another student felt bowled over by their supervisor, stating, "he was quite an aggressive surgeon anyway so i just went and did [the unconsented intimate exam]" (p. 266). these examples highlight medical students' inability to voice their concerns within the institution. it makes sense then, that some students approach outsiders to talk about their conflicts. it is insufficient to make medical students solely responsible for halting their participation in uies. therefore, directors of clerkships, the dean of students, attending physicians, and residents should all make clear that they support explicit consent, that explicit consent is mandatory, and that they support a student's right to exercise and voice an ethical concern. institutions could also declare that violations to explicit consent will be investigated and offenders will be held accountable. physicians must act impeccably to preserve their trust with patients; when they do not, they damage the good name of their profession. physicians should remember that their actions impact how patients view other physicians, and even small incidences of mistrust create fissures that are hard-if not impossible-to heal. thus, physicians and medical schools should be supportive of policies that protect students and patients from indiscriminate physician behavior and help restore trust between patients and physicians. ethics can also be integrated across all levels of the medical institution. a notable example of integrated ethics includes the model developed for the veterans health administration (fox et al. 2010 ). fox's model consists of three functions which target ethics at "the level of decisions and actions;… the level of systems and processes; and…the level of environment and culture" (p. 8). work by callahan et al. (2002) can also be considered: they promote "three trust-building tactics: 'accountability, reliance, and aspiration'… these principles provide people and mechanisms to reach the outcome of an ethical organizational culture" (p. 201). for instance, they suggest an ombudsperson to build employees' trust in the institution's policies and standards and recommend principles to increase empathy and trust and thereby foster organizational integrity. similarly, lachman (2008) suggests an implementation of committees, forums, and procedures to amplify integrity, and greene and latting (2004) recommend the formation of an organizational ethics committee. while these scholars propose robust approaches to institutional ethics, even small organizational changes can have lasting and far-reaching effects. to bolster a sound ethical culture, institutions should review these studies to identify methods most suitable to their institution. in summary, recommendations include an explicit consent policy and enhancements to both medical school curriculum and institutional culture. conflicts between physicians and families are a part of the public conversation. the case of ms. jahi mcmath, an example of a family's refusal to accept a brain death diagnosis, caught national attention (truog et al. 2018) . the mcmath family did not feel heard by their practitioners when their repeated calls to curb jahi's postsurgical loss of blood were disregarded-neglect believed to have caused her brain death-and the case is emblematic of widespread racial bias in medicine (goodwin 2018) . other families may also experience treatment denial, such as vaccine-hesitant families who are denied pediatric care (flanagan-klygis et al. 2005) . these fissures reflect the challenge of conveying complex medical information to families who may already mistrust the medical system and the fissures seem to be so far-reaching that some fear there is little hope for resolution. other recent news stories have covered larry nassar's rampant abuse of his pediatric patients (mountjoy 2019 ) and the atlanta journal-constitution's investigation uncovering "more than 3500 cases of sexual misconduct by physicians across the nation since 1999" (teegardin and norder 2019, p. 1) . given these recent headlines, it comes as no surprise that news coverage on unconsented intimate exams could cause further discord between physicians and the general public. ethicists, legal scholars, nurses, and professional associations-along with some physicians and medical students-have voiced persistent and prolonged calls for change, and the dichotomy between societal and traditional medical views of intimate regions shows no signs of abatement. uies continue to receive attention in the media, and legislators continue to propose statutes-underscoring the sustainability of these concerns. however, most institutions still decline to implement an explicit consent policy. we are reminded that the bodies in question belong to the patients, and patients are the rights-holders of their bodies. asking for permission is a fundamental tenet of modern medicine. without permission, touching intimate areas can feel like an assault, and may even meet the criteria for battery or malpractice. this paper demonstrates how the status quo leads to ethical erosion, moral distress, patient distress, and distrust by the general public. these ill effects are surely not intended by physicians whose goals are to heal and comfort patients. when rhodes urges physicians to "seek trust and deserve it as their moral law, as their creed," she notes that physicians should "pay serious attention to the patient's view of what is good" (2001, p. 497) . formalizing explicit consent can therefore be an act of trust-building between patients and physicians and call attention to physicians' honorable commitment to patients' safety and well-being. we encourage medical institutions to reexamine their stance towards intimate exams to restore trust, protect clinicians, medical students, and patients, and underscore the sacred right of a patient to decide what happens to their body. finally, while institutional change can minimize future harms, consideration should be paid to healing the wounds of medical students, physicians, and patients who have already suffered from past harms related to their roles in unconsented intimate exams. jama forum: teaching pelvic examination under anesthesia without patient consent medical student involvement in patient care professional responsibilities in obstetric-gynecologic education the utility of and indications for routine pelvic examination opinions on patient-physician relationships & the legal advisors committee, concern for dying informed consent to human experimentation: the subject's dilemma medical students and informed consent: a consensus statement prepared by the faculties of medical and health science of the universities of auckland and otago, chief medical officers of district health boards, new zealand medical students' association and the medical council of new zealand an introduction to medical malpractice in the united states practicing pelvic examinations by medical students on women under anesthesia: why not ask first? in support of sb 2578 the challenging pelvic examination informed consent: its history, meaning, and present challenges please don't touch me there: the ethics of intimate examinations: consent is crucial-but don't go too far, for students' and patients' sakes consent for vaginal examination by students on anaesthetised patients intimate exams without patient consent integrating trends in whistleblowing and corporate governance: promoting organizational effectiveness, societal responsibility, and employee empowerment pelvic exams done on anesthetized women without consent: still happening med students still do pelvic exams on women under anesthesia is there no alternative? conscientious objection by medical students prevalence of disability in the us | cdc forced rectal exam stirs ethics questions pelvic examinations by medical students a national survey concerning the ethical aspects of informed consent and role of medical students the ethics of intimate examinations-teaching tomorrow's doctors. commentary: respecting the patient's integrity is the key. commentary: teaching pelvic examination-putting the patient first the ethics of intimate examinations-teaching tomorrow's doctors (education and debate) fear, knowledge, and efficacy beliefs differentially predict the frequency of digital rectal examination versus prostate specific antigen screening in ethnically diverse samples of older men reference guide to consent for examination or treatment lgbtq family fact sheet do clinical clerks suffer ethical erosion? students' perceptions of their ethical environment and personal development dismissing the family who refuses vaccines: a study of pediatrician attitudes inte-gratedethics: an innovative program to improve ethics quality in health care educational pelvic exams on anesthetized women: why consent matters why are pelvic exams on unconscious, unconsenting women still part of medical training? practice vs. privacy on pelvic exams revisiting death: implicit bias and the case of jahi mcmath the ethics of pelvic exams performed on anesthetized women without their knowledge whistle-blowing as a form of advocacy: guidelines for the practitioner and the organization nursing practice: the ethical issues pelvic exams are being performed on unconscious women without consent awareness during anesthesia: how sure can we be that the patient is sleeping indeed? whistleblowing: role of organizational culture in prevention and management pelvic exams performed without patients' permission spur new legislation pelvic examinations by medical students medical education: beware the hidden curriculum states seek explicit patient consent for pelvic exams teaching intubation with cadavers: generosity at a time of loss time is ethics only by speaking out can we create lasting change': what can we learn from the dr larry nassar tragedy? factors associated with intention to undergo annual prostate cancer screening among african american men in philadelphia get statistics teaching the female breast and pelvic exam: a student-and patient-centered approach an anesthetized anesthesiologist tells his experience of waking up accidentally during the operation time to end pelvic exams done without consent challenging pelvic exam screening pelvic examination in adult women: a clinical practice guideline from the american college of physicians prevalence of potentially unnecessary bimanual pelvic examinations and papanicolaou tests among adolescent girls and young women aged 15-20 years in the united states are med students practicing on you? medical students learning intimate examinations without valid consent: a multicentre study understanding the trusted doctor and constructing a theory of bioethics attitudes of african americans regarding screening for prostate cancer obtaining valid consent ethics versus education: pelvic exams on anesthetized women the incidence of awareness during anesthesia: a multicenter united states study abusive doctors: how the atlanta newspaper exposed a system that tolerates sexual misconduct by physicians france to ban vaginal exams without consent the positive value of moral distress brain death at fifty: exploring consensus, controversy, and contexts medical students regularly practice pelvic exams on unconscious patients don't ask, don't tell: a change in medical student attitudes after obstetrics/gynecology clerkships toward seeking consent for pelvic examinations on an anesthetized patient final recommendation statement. gynecological conditions: periodic screening with the pelvic examination teaching pelvic examinations under anaesthesia: what do women think? oxford textbook of medical education moral distress, workplace health, and intrinsic harm autonomy suspended: using female patients to teach intimate exams without their knowledge or consent the use of portable video media vs standard verbal communication in the urological consent process: a multicentre, randomised controlled, crossover trial publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations the author wishes to thank jason eiseman and jordan jefferson at the lillian goldman law library at yale law school, and patty martin at the bioethics research library, kennedy institute of ethics, georgetown university for their valuable support during the research process. key: cord-265984-fyafd386 authors: cheslock, john j.; shamekhi, yahya title: decomposing financial inequality across u.s. higher education institutions date: 2020-10-31 journal: economics of education review doi: 10.1016/j.econedurev.2020.102035 sha: doc_id: 265984 cord_uid: fyafd386 abstract the level of financial inequality among u.s. higher education institutions has important implications for students and society, yet few scholars have examined this topic using established methods for measuring inequality. this paper updates and extends previous work while introducing decompositions that shed light into key trends that we observed for the 2004–2017 period: increasing inequality in total expenditures and decreasing inequality in per-student expenditures. the results of our decomposition highlight how these opposing trends related to rising differences in enrollments and an increasingly positive correlation between an institution's enrollment level and its expenditures per student. our decomposition results also show that both between-group differences and within-group differences contributed to the observed trends. further examination of within-group differences reveals that inequality patterns differed meaningfully by institutional type, with doctoral universities and private baccalaureate colleges possessing higher levels of inequality and a more positive correlation between per-student expenditures and enrollments than master's institutions and public associate's colleges. the disparities in financial resources across u.s. colleges and universities are vast and important. for example, the higher education institutions with the 25 largest endowments hold over half of all endowment assets (milton & ehrenberg, 2013) . recent books with striking titles such as unequal colleges in the age of disparity and unequal higher education compellingly describe the implications of organizational financial inequality for students and for society (clotfelter, 2017; taylor & cantwell, 2019) . furthermore, many worry that this form of inequality is growing, with prominent institutions like the delta cost project suggesting that " [r] ich institutions are getting richer, and poor institutions are getting poorer" (desrochers and wellman, 2011) . despite the importance of this topic, only a few academic studies have examined it using established methods for measuring inequality (davies & zarifa, 2012; lau & rosen, 2016) . our collective understanding of how to study organizational financial inequality in the higher education industry is consequently underdeveloped, as is our empirical understanding of the levels and trends that exist for this form of inequality. this study advances the literature by describing levels and trends in overall inequality for the period spanning the 2004-2017 fiscal years and then decomposing those levels and trends. our description updates and extends previous estimates; our decomposition makes a more fundamental contribution by introducing to the higher education literature a decomposition method that illuminates key questions pertaining to organizational inequality. our decomposition method helps us to better understand differences in inequality trends that depend upon whether dollars are measured in total or on a per full-time-equivalent (fte) student 1 basis. lau and rosen (2016) noted that because colleges and universities provide students with a mix of rival and non-rival goods and services, the optimal measure of finances is unknown but would fall somewhere between the extremes of total dollars and dollars per student. they consequently suggested examining trends for both measures and noted that clear conclusions can be reached when trends in inequality are consistent across these two measures, which it was for the time period they examined. they left open the question of how to interpret findings when trends differ across these measures, a question of importance because our period of study experiences opposing trends. to understand the source of these differential trends, we employed a technique that is used in labor economics to decompose total earnings inequality into wage inequality, hours of work inequality, and a term measuring the correlation of wages and hours of work (checchi, garcía-peñalosa, & vivian, 2016) . for our study, total expenditure inequality was decomposed into per-student expenditure inequality, enrollment inequality, and a term measuring the correlation between per-student expenditures and enrollments. we extended this decomposition so that each term is further divided into shares associated with between-group and within-group inequality, which allowed us to examine important variation relating to an institution's control (public and private non-profit) and carnegie type (doctoral, master's, baccalaureate, and associate's). our description and decomposition of organizational financial inequality reveal a number of important findings. for a data set drawn from the integrated postsecondary education data system for the fullpopulation of non-profit title iv eligible u.s. higher education institutions, 2 we report a range of inequality indices. these indices indicate that total expenditure inequality increased over the 2004-2017 period while per-student expenditure inequality declined. our decomposition highlights how these differing trends related to rising differences in enrollments and an increasingly positive correlation between an institution's enrollment level and its expenditures per student. our extended decomposition shows that both between-group differences and within-group differences contributed to these patterns. our results are partially driven by the presence of high research spending at large universities but remain present when more narrow measures of expenditures, that exclude research and service, are used. our work has important implications for understanding inequality in organizational finances. our findings highlight how measures of organizational expenditures and revenues relate to enrollment levels, a relationship that will become increasingly important given demographic trends stemming from the "birth dearth" since the great recession that could lead to enrollment declines at many institutions (grawe, 2018) . the covid-19 pandemic may create a further set of demographic challenges. because some revenue sources (e.g. endowments, state appropriations) and expenditure types (e.g. campus infrastructure) do not decline commensurately with enrollments, per-student expenditures could remain steady or even rise at institutions with worsening financial situations and declining enrollments. yet scholars regularly use per-student revenue or expenditure measures to capture the state of a higher education institution's finances. although no perfect measure of expenditures or revenues exists, our paper highlights key factors shaping differences between alternative measures. only two studies have used established methods for measuring inequality to examine whether the distributions of revenues and expenditures across higher education institutions have become more or less unequal over time. 3 both davies and zarifa (2012) and lau and rosen (2016) examined gini coefficients for large numbers of u.s. higher education institutions, but they differed in a number of regards. the former study used data for a longer period but only examined four-year institutions and a smaller set of measures (perstudent measures of revenues and expenditures). the latter used data for a shorter period (2002) (2003) (2004) (2005) (2006) (2007) (2008) (2009) (2010) but also included two-year institutions and measures of total revenues and expenditures. the studies also differed in how they measured revenues, as davies and zarifa (2012) did not include realized or unrealized capital gains on endowment portfolios for most of their sample period, while lau and rosen (2016) included this form of income. the studies both found that the gini coefficient for expenditures per student increased over time, but the studies interpreted the increases differently. for example, the analysis in davies and zarifa (2012) revealed an increase in the gini coefficient from 0.427 to 0.467 for the 1971-1996 period, while the analysis in lau and rosen (2016) showed an increase from 0.341 to 0.370 between 2002 and 2010. the latter increase was more substantial given that it occurred over an 8-year period rather than a 25-year period. yet lau and rosen concluded that their observed change was not a significant increase in expenditure inequality during their period of study, while davies and zarifa generally interpreted their observed increases as consequential. 4 the findings from these two studies also differed by the level of inequality and the differences by sector. for the 2006 fiscal year, the one year held in common for both studies, davies and zarifa (2012) reported gini coefficients for total expenditures per student of 0.579 for the public sector and 0.460 for the private sector. in contrast, lau and rosen (2016) reported gini coefficients of 0.264 for publics and 0.366 for privates. these varying results likely relate to one additional difference between the studies: davies & zarifa included institutions with a special focus classification within the carnegie classification system while lau and rosen omitted these institutions. special focus institutions often produce extreme outliers for per-student measures due to high revenues and low enrollments. boxplots reported by davies and zarifa confirm that their data set contains a number of extreme outliers. in the analysis that follows, we will report gini coefficients for expenditures per student that are much closer to the results from lau and rosen (2016) than the results from davies and zarifa (2012) , which is not surprising because we also exclude special focus institutions. similar to previous studies, we used data from the integrated postsecondary education data system (ipeds), which is an annual census of postsecondary institutions in the united states. ipeds is the only publicly-available data set that contains detailed revenue and expenditure figures for both public and private higher education institutions. our study period spans the 2004 through 2017 fiscal years. the 2004 fiscal year is the first one in which all public institutions reported data using the new governmental accounting standards boards (gasb) reporting standards. 5 2 our population does not include institutions with a special focus carnegie classification. special focus institutions offer degrees in a single field or a set of related fields, which sometimes leads to very atypical combinations of enrollments and expenditures. examples of special focus institutions include seminaries, medical centers, law schools, and art schools. we do not include these institutions in order to eliminate outliers and focus attention on institutions with more broad-based offerings. 3 other studies examined related topics. desrochers & wellman (2011) examined trends in expenditures per student between 1999 and 2009 for different carnegie classification and institutional control groupings. they reported growing between-group inequality as average expenditures per student increased more rapidly at private institutions than at public institutions and more rapidly at four-year public institutions than at two-year public institutions. winston (2004) examined institutional subsidies per student for the 1986/87-1995/96 period. the subsidy equals the difference between the cost associated with producing the education and the amount the student paid for it. winston found growing between-sector inequality in subsidies as the gap between public and private institutions grew over the period. he also found varying trends in within-sector inequality: the gini coefficient for public institutions fell from 0.189 to 0.176, and the gini coefficient for private institutions increased from (footnote continued) 0.299 to 0.338. 4 these studies also differed in that lau and rosen (2016) focused on linear time trends based upon data for every year of their sample, while davies and zarifa (2012) focused on differences between the beginning year and ending year of the period. when using linear time trends, lau and rosen (2016) estimated a smaller difference of 0.020 rather than 0.029 for their time period. the use of linear time trends, however, does not explain the differing conclusions as an increase of 0.020 over eight years is still more impressive than an increase of 0.040 over 25 years. 5 although consistent reporting standards were generally used over our selected period, reporting standards did change slightly over the period. the treatment of maintenance, depreciation, and interest (mdi) expenses changed for those institutions following the gasb standards. during fy2007 and earlier j.j. cheslock and y. shamekhi economics of education review 78 (2020) 102035 unlike previous studies of organizational financial inequality, we prepared ipeds data using the recommendations provided by jaquette and parra (2016) . 6 consequently, we examine title iv institutions. title iv institutions provide postsecondary education and are eligible to enroll students receiving title iv federal financial aid (congressional research service, 2007) . in order to obtain title iv status, an institution must obtain accreditation by an approved accrediting agency and satisfy requirements, including annual completion of the ipeds survey, stipulated in the program participation agreement (ppa) contract with the u.s. department of education. a title iv institution may be a single-campus institution or a multi-campus institution. because multiple campuses from the same title iv institution sometimes report data separately 7 , we aggregate data from separate campuses in order create a title iv institution-level data set. our aggregation uses digits 2-6 of the office of postsecondary education identification (opeid) as recommended by jaquette and parra (2016) . during the aggregation, we used the values for the ppa holder (i.e. main campus), which is identified by a value of "00" for digits 7-8 of the opeid, when calculating categorical variables. 8 for continuous variables that count items like dollars and students, we summed across all unitid-level observations that share the same ppa. we constructed our data set with aggregation procedures performed separately for each academic year. one limitation of this approach is that the composition of ppas changed over time so identical groupings were not used across years. to determine whether or not changes to the composition of ppas affected our results, we repeated the analysis for a data set that employed identical groupings across years. we achieved identical groupings by further aggregating observations to account for the movement of unitid-level observations across ppas. 9 our results did not meaningfully differ when this alternative data set was used. we examined institutions that meet the following criteria: (a) notfor-profit; (b) title iv eligible; (c) two-year or four-year institutions; (d) residing within the 50 u.s. states or washington d.c.; (e) degree granting; (f) not system offices unaffiliated with other ipeds observations; (g) not missing finance or enrollment data; and (h) possessing enrollment levels of at least 100 students. we then added another criterion via the carnegie classification by only keeping institutions classified as doctoral, master's, baccalaureate, or associate's institutions. this decision primarily results in the omission of institutions classified in the special focus classifications, which contain types of institutions (e.g. seminaries, medical centers) that produce extreme outliers for perstudent measures due to high revenues and low enrollments. another reason for this omission lies with the motivation for the paper, which primarily centers around policy questions pertaining to more general types of higher education institutions. a range of potential measures of total expenditures and total revenues exist, and each measure offers drawbacks and advantages. although our tables only include findings for one revenue measure and one expenditure measure, we examined findings for a number of alternatives to ensure that our findings are robust. our primary measure of total revenues includes tuition, government appropriations, grants and contracts, donations, and investment return. this measure excludes revenues generated by auxiliary enterprises, hospitals, sales of educational services, independent operations, and those revenues categorized as "other" in ipeds finance tables. to maintain consistency with our revenue measure, we used education and general (e&g) expenditures as our primary measure of total expenditures. e&g expenditures include instruction, academic support, student services, institutional support, research, public service, and scholarship expenses, but exclude costs of auxiliary enterprises, hospitals, independent operations, and expenses categorized as "other" in ipeds. to check for sensitivity of our results, we repeated our analysis using alternative measures of revenues and expenditures. first, we examined a more restrictive measure of expenditures−education and related (e&r) expenditures−which differs from e&g expenditures in that direct and indirect expenditures associated with research and public service are not included. 10 our inequality estimates, especially those examining between-sector differences, were lower when e&r expenditures were used. second, we examined measures that incorporated all sources of revenues and expenditures including auxiliary, hospital, independent operations, and other categories. we observed a slight increase in the magnitude of inequality when we employed these more expansive measures. however, our primary findings pertaining to changes over time or disparities between total and per-student measures of inequality were not substantially altered by the use of more restrictive or more expansive measures of organizational finances. to measure enrollment, we used 12-month enrollment rather than fall enrollment so that we can account for enrollment over an entire year and provide a more complete picture of the number of students that benefit from the financial resources of the institution. we replaced 12-month enrollment data with an adjusted figure based on fall enrollment when 12-month enrollment data was missing or problematic. 11 our enrollment measure is weighted with graduate (footnote continued) years, gasb institutions reported mdi expenses as a separate expense category. during fy2010 and later years, these institutions allocated mdi expenses to other expense categories and reported them as subcategories. in fy2008 and fy2009, gasb institutions varied in their treatment of mdi expenses. to ensure comparability, for all years prior to fy2010, we allocated the mdis across all the other expense categories proportional to the size of each expense category. to ensure similarities across schools for the fy2008 and fy2009 years, we first rolled back the mdi expenses to the old system for those schools that adopted the new reporting before performing our proportionate allocation. 6 lau and rosen (2016) used a version of ipeds data produced by the delta cost project, which contains limitations that can affect estimates, especially for public institutions (jaquette & parra, 2016) . davies and zarifa (2012) created their own ipeds data set, but organized their data at the unitid-level, which creates a different set of challenges that are described in jaquette and parra (2014; . their choice was motivated by restrictions created by limitations in the pre-2000 data, as the authors were combining data from the ipeds survey (which was created in the mid-1980s) and the higher education general information system survey (which preceded the ipeds survey). the methods proposed by jaquette and parra (2016) are preferred to these approaches, because they produce a data set that contains a unit of analysis (title iv institutions) that is constant across observations and has a helpful interpretation. furthermore, this unit of analysis is "the most disaggregated unit of analysis that contains data from all ipeds survey components" (jaquette & parra, 2016, p. 648 separate title iv institutions in later years. for the alternative data set containing identical groupings over time, uc-merced and uc-davis data are combined into one observation for all years in our study period. 10 the measure of education and related (e&r) expenditures was introduced by the delta cost project. they described it as "instruction + student services + (education share *(academic support + institutional support + operations and maintenance)), where education share = (instruction + student services)/(instruction + student services + research + public service)" (dcp, 2011, p. 17). 11 the median difference between 12-month enrollment and fall enrollment enrollments receiving a weight of two, because graduate education is more expensive to provide. 12 although our measure captures weighted fte enrollments, we use the term "expenditures per student" rather than "expenditures per weighted fte student" for brevity's sake when describing results for expenditures measures that were normalized by our enrollment measure. to check for robustness, we incorporated alternative enrollment measures based on fall enrollments or unweighted calculations. we observed a slight decline in the magnitude of inequality measures when we used fall enrollment rather than 12-month enrollment. we detected a slight increase in inequality indices when we used unweighted, rather than weighted, enrollment. as was the case with revenue and expenditure measures, the use of alternative enrollment measures did not substantially alter our primary findings pertaining to changes over time or disparities between total and per-student measures of inequality. we sought to measure the level of financial inequality across u.s. higher education institutions and decompose the observed inequality into factor components. researchers have identified a substantial number of inequality indices, each containing both drawbacks and merits (cowell, 2011; hao & naiman, 2010) . for this study, we primarily used the mean log deviation (mld) to measure inequality. we also examined overall trends using the gini, thiel index, and atkinson index (ε=1) to demonstrate that the observed trends are not specific to the mld. 13 the mld contains a number of desirable properties. it belongs to the generalized entropy family of inequality indices, and like other members of this family, the mld is independent of scale and population size and satisfies the strong principle of transfer (cowell, 2011 ). an important feature of the mld for the purposes of this study is its decomposability. like many other measures, the mld can be used to decompose total inequality into a portion due to inequality between constituent subgroups and a portion due to inequality within the subgroups. the mld can also be used to decompose two terms of interest that have a multiplicative relationship, a trait that is rare among inequality indices (checchi et al., 2016) . this last consideration is important for the purposes of our paper, because we are interested in the relationship between total expenditures and expenditures per student, which can be summarized as where t i represents total expenditures, s i represents expenditures per student, and e i represents enrollment. a similar equation can be stated for revenues and revenues per student. this relationship between total expenditures, expenditures per student, and enrollments is identical in structure to the relationship between total earnings, hourly wages, and the number of hours worked. checchi et al. (2016) decomposed the mld for this latter set of variables, and we employed a similar approach for our measures of organizational finances. the decomposition highlights how inequality in total revenues can be expressed as the sum of three components: inequality in expenditures per student, inequality in enrollment, and a component capturing the correlation between expenditures per student and enrollments. this relationship can be described more formally as: where i t , i s , and i e represent the mld of total expenditures, the mld of expenditures per student, and the mld of enrollments, respectively, and ρ is a term relating to the correlation between per-student expenditures and enrollments. the mld is defined as the difference between the log of the average of a variable and the average of its log, so the mld of total expenditures is = both i s and i e are defined similarly. checchi et al. (2016) note that the term representing the correlation between expenditures per student and enrollments is defined as the covariance between expenditures per student and enrollments can be shown to be given by the numerator of the second term in the parenthesis of eq. (4). the sign of the covariance determines the sign of the correlation term. we can use equations (2) through (4) to better understand factors that relate to the difference between inequality in total expenditures and inequality in expenditures per student. if the covariance between expenditures per student and enrollments is positive or equal to zero, then inequality in total expenditures will always be higher than inequality in expenditures per student. if the covariance is negative, then the relationship between total expenditure inequality and expenditure per student inequality will depend upon the magnitude of that correlation term relative to enrollment inequality. for example, when a negative value for the correlation term perfectly offsets the level of enrollment inequality, total expenditure inequality will equal expenditure per student inequality. checchi et al. (2016) note that eq. (2) can be further decomposed into terms capturing between-group inequality and terms capturing within-group inequality. for example, when observations can be divided into j exhaustive groups, inequality in total expenditures, i t , can be decomposed into between-group and within-group components eq. (6) can be further extended to provide details about each component (footnote continued) was around 7% of fall enrollment during our period of study, so fall enrollment figures were multiplied by 1.07 to calculate 12-month enrollment when necessary. this replacement procedure was rarely used as less than 0.1% of observations possessed missing or problematic 12-month enrollment data. we identified problematic data using an algorithm that detected large one-year blips in 12-month enrollment data for an individual institution that were not matched by blips in fall enrollment data. 12 in a study of higher education expenditures in four states, sheeo (2010) found that the cost of producing a graduate student credit hour is 2-4 times as expensive as the cost of producing an undergraduate student credit hour. these estimates are similar to earlier studies by james (1978) and to (1987) . available estimates are very imprecise because cost researchers must allocate expenditures across levels of study using a number of questionable assumptions due to data limitations. to be conservative, we use the lower end of the estimated range for our adjustment. another factor driving our choice of weights is that full-time graduate students may enroll in fewer credit hours than full-time undergraduate students. 13 see cowell (2011) and hao and naiman (2011) for a description of available inequality indices, their relative strengths and weaknesses, and their conceptual underpinnings. hao and naiman (2011) organize inequality indices into different categories, and we include indicators from each of the three categories with the most desirable properties. the gini coefficient belongs to the category of indicators that are based on quantile functions and lorenz curves. the atkinson index is derived from social welfare functions, and the mld and the thiel index are developed from information theory. j.j. cheslock and y. shamekhi economics of education review 78 (2020) 102035 where p j equals the proportion of observations belonging to group j, t j equals the mean expenditures of group j, and i tj equals the level of inequality in total expenditures among members of group j. similar decompositions can be performed for i s and i e so that eq. (2) can be extended to the terms b ρ and w ρ are not calculated directly but are instead calculated indirectly from the other components of eqs. (6) and (8), so that (checchi et al., 2016) . in this section, we present and discuss the results of our empirical analyses. we start by focusing on the decomposition of the difference between the mld of total expenditures and the mld of expenditures per student. the decomposition is then extended so that differences between and within organizational types can be examined. we conclude our results section by further examining two items highlighted by our decomposition: rising enrollment inequality, and the positive correlation between enrollments and expenditures per student. table 1 reports a range of inequality indices for total revenues, total expenditures, revenues per student, and expenditures per student for the 2004 and 2017 fiscal years. all of the indices reveal major differences between the first two measures and the latter two measures. the level of inequality was much greater for total revenues and total expenditures than for revenues per student and expenditures per student. the trends over time also differed, with inequality increasing for measures of total dollars and falling for measures of dollars per student. these differences between total dollars and dollars per student were robust. 14 they were present for both revenues and expenditures. they held for all of our general inequality indices (mld, thiel, atkinson, and gini). with the exception of the trend in per-student revenues for private institutions 15 , they persisted when analysis was conducted separately for public and private institutions. table 2 reports results solely for the mld but adds findings for all years during the 2004-17 period and for two additional measures: enrollment inequality and the term measuring the correlation between per-student dollars and enrollments. the yearly figures indicate that changes over time in revenue measures were highly erratic while the trends for expenditures were relatively steady. 16 for example, the mld for total expenditures per student for the full set of institutions only differed by 0.029 (between 0.134 and 0.163) during the entire 14-year period. in contrast, the absolute magnitude of year-to-year changes in the mld for total revenues per student exceeded 0.029 eleven times, with the largest being a reduction of 0.132 between 2011 and 2012. the fluctuations in revenue were primarily present in the private sector and related to the revenue categories related to investment income and private gifts, grants, and contracts. when these categories were removed from revenue calculations, the year-to-year changes in revenues per student became similar in nature to the corresponding changes in expenditures per student and only differed by 0.037 (between 0.075 and 0.111) during the entire 14-year period. because revenue measures are volatile, we focus on expenditure trends for the remainder of this paper. as described in our methods section (see eq. (5)), one can decompose the difference between inequality in total expenditures and inequality in expenditures per student using the results in table 2 . for the full set of institutions between 2004 and 2017, the average mld for total expenditures was 0.922 and the average mld for expenditures per student was 0.148, so the average difference between these two measures was 0.774. this difference between measures was related to both inequality in enrollments and a positive correlation between enrollments and expenditures per student. the average magnitude of the former equaled 0.615 while the average magnitude of the latter equaled . when all sources of expenditures were used, we again found an increase in the former (from 0.985 to 1.094) and a decrease in the latter (from 0.202 to 0.175) was again found. 15 as we will note in our upcoming discussion of table 2 , revenue trends were affected heavily by the presence of revenues associated with private gifts and investment income, which fluctuated greatly from year-to-year. when these types of revenues removed, the trends for per-student revenues for private institutions turned negative and no longer differed from the trends observed for other financial measures. 16 lau and rosen (2016) also noted these patterns and that they indicate that higher education institutions smooth their spending in a manner that is consistent with the permanent income hypothesis. 0.159. consequently, 79% of the difference between the average mld for total expenditures and the average mld for expenditures per student was associated with inequality in enrollments. 17 the role of inequality in enrollments relates to a straightforward point: institutions that enrolled larger numbers of students experienced, on average, much higher levels of total expenditures than institutions that enrolled smaller numbers of students. the positive correlation term communicates a less obvious relationship: institutions that enrolled larger numbers of students spent more per student, on average, than institutions that enrolled smaller numbers of students. the trends for enrollment inequality and the correlation term indicate that both terms contributed to the growing divergence between total expenditure inequality and expenditure per student inequality. fig. 1 illustrates these trends. between 2004 and 2017 for the full set of institutions, the mld for total expenditures increased by 0.073 while the mld for expenditures per student fell by 0.026, leading the gap between these two measures to increase by 0.099. both enrollment inequality and the correlation term increased in magnitude over the period by similar amounts, with the 0.054 increase in the former slightly exceeding the 0.045 increase in the latter. the increase in enrollment inequality was much higher than the increase in the correlation term when estimates were calculated separately for the private and public sectors. 18 table 3 extends the analysis by reporting the portion of each term that is due to differences between institutional types and the portion due to differences within institutional types. this extended decomposition employs eight institutional types, because we examined the intersection of institutional control (public, private) and carnegie classification (doctoral, master's, baccalaureate, associate's). the results indicate that a majority of inequality in total expenditures was due to differences between institutional types, while the majority of inequality in expenditures per student was due to differences within institutional type. for example, the between-group mld for total expenditures was 0.617 in 2017 while the within-group mld for total expenditures was 0.361, so 63% of total expenditure inequality was due to betweengroup differences. in contrast, the corresponding figures for expenditures per-student were 0.058 and 0.079, so only 42% of per-student expenditure inequality was due to between-group differences. these patterns are expected because organizational scale, which has a bigger impact on total expenditures than per-student expenditures, varies dramatically across different carnegie classifications. one implication of these patterns is that the gap between total expenditure inequality and expenditure per student inequality was greater for between-group measures than for within-group measures. in notes: the row titles signify the following: "total" contains the mld of total expenditures; "total/enr" contains the mld of per-student expenditures; "enrollment" contains the mld of enrollment; "correlation" contains a term measuring the correlation between per-student expenditures and enrollment. see the methods section for a description of how the first term can be decomposed into the latter three terms. see table 1 notes for further information. 17 the share of the difference that was associated with enrollment inequality dropped from 79% to 70-71% when analyses were conducted separately for the public and private sectors. sector-specific enrollment inequality was lower than overall enrollment inequality because private institutions had substantially lower enrollments than public institutions. in contrast, the sector-specific correlation terms were larger than the overall correlation term. the between-sector correlation between enrollments and expenditures per student was negative because, in addition to having lower enrollment levels, private institutions had higher levels of expenditures per student. 18 between-sector enrollment inequality fell over the period because enrollments at private institutions grew at a faster rate than enrollments at public institutions. in addition, the trend in expenditures per student was less favorable for private institutions than for public institutions. as a result, the between-sector correlation between enrollments and expenditures per student became less negative over time. 2017, this gap equaled 0.559 (0.617-0.058) for between-group measures and 0.282 (0.361-0.079) for within-group measures. the difference between these two gaps related to variation in the term measuring the correlation between expenditures per student and enrollments. the between-group correlation term was positive (0.227), while the withingroup correlation term was slightly negative (-0.043). the differential trends over time for total expenditures and perstudent expenditures that we observed for the overall measures in tables 1 and 2 were mostly present for both the between-group and within-group measures in table 3 . for between-group, total expenditure inequality increased by 0.053 between 2004 and 2017 while per-student expenditure inequality essentially stayed the same, falling by only 0.007. for within-group, total expenditure inequality grew by 0.020 and per-student expenditure inequality fell by 0.020. due to these differential trends, the gap between total expenditure inequality and per-student expenditure inequality increased over time notes: for each of the four terms, "mld between" represents the portion of the mld that is attributable to differences between institutional types, as categorized by the intersection of institutional control and carnegie classification. "mld within" represents the portion of the mld that is attributable to differences within institutional types. see the notes from tables 1 and 2 for further information. for both the between-group and within-group figures. for betweengroup figures, the growing gap was associated with a slight increase in enrollment inequality of 0.018 and an increase in the correlation term of 0.042. 19 in contrast, enrollment inequality increased by 0.036 while the correlation term barely changed for the within-group figures. the specific findings for table 3 discussed to this point relate to analyses conducted for the full set of institutions. table 3 also contains results from separate analyses of the public and private sectors, and in general, the patterns observed for the full set of institutions were mostly present in each sector. to examine whether results vary by the intersection of institutional control and carnegie classification, we turn now to table 4 , which provides inequality figures for the six carnegie-control categories with large numbers of institutions. private doctoral universities possessed the highest levels of inequality. for these schools, the mld in 2017 was 0.853 for total expenditures and 0.322 for expenditures per student. both of these figures were more than twice as large as the corresponding figures for each of the other institutional types. private doctoral universities were also atypical in regards to the correlation term between enrollments and expenditures per student, which equaled 0.210 in 2017. public doctoral universities comprised the only other set of institutions that contained a positive correlation term, but the magnitude of the correlation term (0.081) was much smaller. despite the positive correlation term and the third highest level of per-student expenditure inequality 20 , the mld for total expenditures for public doctoral universities was similar to others due to the low level of enrollment inequality. private baccalaureate colleges were akin to doctoral universities in that they possessed relatively high levels of per-student expenditure inequality and a correlation term that was not meaningfully negative. 21 in contrast, public associate's, public master's, and private master's institutions had negative correlation terms and the smallest levels of per-student expenditure inequality. public associate's colleges still possessed fairly high levels of total expenditure inequality, because these schools experienced high levels of enrollment inequality. 22 in terms of changes over time, the results in table 4 reveal some variation across institutional types. the within-group increase in total expenditure inequality was primarily driven by increased inequality among private doctoral universities and private master's institutions, while the within-group decrease in per-student expenditure inequality was primarily present among public doctoral universities and public associate's colleges. each institutional type experienced a small or moderate increase in enrollment inequality. in contrast, the trends associated with the correlation term were much more variable, with large increases for public and private doctoral universities and small declines for public master's institutions and private baccalaureate colleges. for our period of study, the trends for total expenditure inequality and expenditure per student inequality differed. our decomposition results highlight how these differing trends are related to rising enrollment inequality. in general, we should expect enrollment shifts to have differing effects on total expenditures and expenditures per student. enrollment gains would generally lead to increased total expenditures, because additional students bring additional net tuition revenue and additional funding from entities, such as state governments, that relate funding levels to enrollment levels. enrollment declines would generally lead to decreased total expenditures for similar reasons. the effect of enrollment shifts on expenditures per student would likely be different. although institutional revenues increase with enrollment, they are unlikely to increase proportionally. some revenue sources, such as investment income, have little relationship to enrollment levels. other revenue sources, such as state appropriations, will generally increase with enrollments but not by commensurate amounts. so, revenues per student, and by extension expenditures per student, will typically fall for individual institutions as their enrollments rise. similarly, enrollment declines will generally lead to increases in expenditures per student. these relationships are present in our data set. an institution's change in enrollment between 2004 and 2017 has a strong positive relationship with its corresponding change in total expenditures and a strong negative relationship with its corresponding change in expenditures per student. 23 to illustrate these relationships in a way that notes: results were estimated separately for each of the six listed institutional types. see the notes from tables 1 and 2 for further information. 19 public and private doctoral universities, which had the highest mean expenditures per student, experienced strong enrollment growth over the 2004-2017 period. in contrast, public associate's colleges, which had the lowest mean expenditures per student, experienced the weakest enrollment growth over the period. these trends contributed to the growth in the between-group correlation term and the rising between-group inequality in total expenditures. 20 the differences between doctoral universities and other institutional types were partially related to the financial impact of research activity. the distribution of research expenditures within both categories of doctoral universities was highly unequal, as the mld of research expenditures per student was 1.332 for private doctoral universities and 0.525 for public doctoral universities in 2017. when table 4 was re-estimated using e&r expenditures, which don't include expenditures relating to research and service, the mld of expenditures per student decreased by around one-third for doctoral (footnote continued) universities. section 5.4 of this paper will discuss how research expenditures relate to differences in the correlation term across carnegie classifications. 21 the correlation term for private baccalaureate colleges was heavily shaped by one outlier, brigham young university-idaho, that educates large numbers of students via online education. in 2017, this institution enrolled 43,000 students and had per-student education and general expenditures of $5,976. the corresponding averages for private baccalaureate colleges was 1,647 and $27,026. when we removed this one outlier, the correlation term in figure 4 moves from -0.017 to 0.034. 22 the high levels of enrollment inequality among associate's colleges likely relates to heterogeneity in the size of nearby student populations and breadth of mission. 23 for our population of institutions, the pearson correlation between the change in log enrollments and the change in log total expenditures was 0.643 j.j. cheslock and y. shamekhi economics of education review 78 (2020) 102035 highlights state-level policy implications, fig. 2 describes 2010-17 changes in expenditures, expenditures per student, and enrollments for universities in the pennsylvania state system of higher education (passhe). the enrollment trends for passhe universities since 2010 were generally negative but varied meaningfully across institutions. between 2010 and 2017, two schools (west chester and slippery rock) experienced enrollment increases, two schools (cheyney and mansfield) saw their enrollments drop by almost half, and the remaining schools faced meaningful but much less drastic enrollment reductions. the enrollment shifts were positively correlated with changes in total expenditures but negatively correlated with changes in per-student expenditures. contexts like the passhe raise a series of noteworthy questions. labor economists will be interested in the effect of institutional enrollments and expenditures per student on the experiences and outcomes of college students. bound and turner (2007) examined this question at the state level and found that cohort size (i.e. the number of state citizens that are 18) is negatively related to college attainment rates. furthermore, they provided evidence that this negative relationship is due to limited adjustment among colleges and universities on the supply-side of the higher education market. they found that nontuition revenues in a state do not increase or decrease commensurately with changes in cohort size, so students in smaller cohorts experience higher per-student expenditures and higher college attainment, on average. the link between per-student expenditures and college attainment was examined more directly by deming and walters (2017) , who found that per-student expenditure reductions led to lower levels of attainment. the available evidence suggests that this link may be driven by informal capacity constraints that cause students to experience reduced course offerings, limited advising, and larger class sizes when per-student funding is lower (bound et al, 2012; deming & walters, 2017) from this perspective, one might conclude that student experiences improved between 2010 and 2017 at lincoln, mansfield, and other passhe universities experiencing enrollment declines, due to increases in expenditures per student. 24 students at these schools may increasingly experience smaller class sizes and greater opportunities to engage faculty and staff members directly. yet, enrollment declines can also have negative impacts on the student experience, as they may harm campus morale and limit the ability of the institution to provide items that carry high fixed costs. the net effect of these considerations on student outcomes could vary by the institution's initial level of enrollment and the size of the enrollment decline. public finance economists and policymakers will be interested in the state funding questions raised by enrollment divergence. the president of west chester, the passhe university experiencing the largest enrollment gains, recently highlighted conflicting goals of state policymakers that shape how they respond to varying enrollment trends (fiorentino, 2020) . state policymakers wish to expand college attainment and success, which would call for increasing (decreasing) funding for institutions with growing (declining) enrollments. but state policymakers also wish to ensure that no existing public college or university fails, as they fear the employment and economic impact of such an outcome. this latter concern might limit the extent to which state appropriations are redistributed in response to enrollment changes. these tensions will likely increase in the years ahead as states and higher education institutions navigate the lingering financial challenges associated with the covid-19 pandemic and the expected decrease in the number of traditionally-aged college students stemming from low birth rates during the great recession (grawe, 2018) . although an institution's expenditures per-student will typically rise as its enrollment declines, this does not mean that a similar relationship will be present when between-institution comparisons are made. instead, institutions with above-average enrollments may also generally have above-average expenditures per students. our decomposition indicated a positive between-institution correlation between enrollments (footnote continued) and the pearson correlation between the change in log enrollments and the change in log expenditures per students was -0.499. we observed the same pattern when changes in levels or percentage changes were examined rather than changes in logs, and we obtained similar results when analysis was confined to particular institutional types or when analysis was just run for the 2004-2010 and 2010-2017 periods. 24 the analyses by bound and turner (2007) do not provide direct insights into the institution-level relationship between enrollments and expenditures per student. they examined the effect of cohort size rather than actual enrollments, so part of their observed effect could be due to students in a large cohort who never enroll. the enrollment shifts that are related to cohort size have impacts that move beyond within-institution changes in enrollments and expenditures per student. bound and turner (2007) found that rising cohort size leads to a shift in enrollments across institutional types, with students increasingly attending institutional types (e.g. two-year institutions) that spend less per student. and expenditures per student. in this section, we examine how research activity and variation in institutional wealth and prestige both contribute to this relationship. research activity is worthy of consideration because table 3 showed that differences across institutional types were driving the overall positive correlation between enrollments and expenditures per student. furthermore, table 4 revealed that the correlation between enrollments and expenditures per student is strongly positive for public and private research universities but not for other institutional types. research activity could be partially or fully driving these results because our measure of expenditures-education and general expenditures-includes research expenditures. in 2017, the pearson correlation between research expenditures per student and enrollment was 0.343, which is very different than the 0.012 correlation between instructional expenditures per student and enrollment. the positive correlation for research expenditures was driven by differences between institutional types as research universities spent more on research and had larger enrollments. the positive correlation was also driven by differences within institutional types, as the pearson correlation in 2017 was 0.357 for public doctoral universities and 0.233 for private doctoral universities. to illustrate the degree to which research expenditures were driving the differences across institutional types highlighted in table 4 , we also calculated figures using education and related (e&r) expenditures, which do not include expenditures relating to research and public service. when this narrower measure of expenditures was used, the differences between doctoral universities and other types of higher education institutions were diminished. for 2017, the total expenditure mld declined from 0.360 to 0.290 for public doctoral universities and from 0.853 to 0.679 for private doctoral universities. per-student expenditure mld declined from 0.089 to 0.060 for public doctoral universities and from 0.322 to 0.195 for private doctoral universities. the correlation term similarly declined from 0.081 to 0.040 for public doctoral universities and from 0.210 to 0.164 for private doctoral universities. in contrast, the figures for other institutional types were not meaningfully changed when e&r expenditures were used. thus, the positive correlation terms for doctoral universities observed in table 4 are partially due to a positive relationship between enrollments and research expenditures per student. this relationship could relate to the joint production of instruction and research within u.s. universities. perhaps u.s. research universities can only sustain large research capacity when they also enroll large numbers of students. high enrollments help universities maintain large numbers of faculty in a wide variety of fields, and this breadth and depth of faculty expertise may help a university attract substantial sums of research funding. an empirical test of this possibility would be a complicated and challenging task that would constitute a separate paper. the results for doctoral universities remained very different from those observed for other institutional types even after the removal of research and service expenditures, so other factors were also driving differences by institutional type. winston (1999) points to one potential explanation: institutional prestige and wealth are closely linked and lead to large differences in expenditures per student across higher education institutions. the top 10-20% of higher education institutions in the u.s. spend much more per student than other institutions, and the spending inequality even among the institutions located in the top two deciles is great (winston, 1999) . the most prestigious and wealthy institutions are almost exclusively public doctoral universities, private doctoral universities, or private baccalaureate colleges. 25 the distribution of endowments and private donations across colleges and universities is highly skewed, and these elite institutions capture a large share of endowments and gifts (cheslock & giannesschi, 2008; milton & ehrenberg, 2013) . during our period of study (2004) (2005) (2006) (2007) (2008) (2009) (2010) (2011) (2012) (2013) (2014) (2015) (2016) (2017) , schools that are ranked among the top 25 national universities or the top 25 national colleges received 47% of the total private income 26 generated by our population of institutions despite comprising only 2% of institutions and capturing 4% of enrollments. for those institutional types that contained these schools, the presence of elite institutions led to substantial inequalities in private income. the mld of per-student private income in 2017 was 0.540 for public doctoral universities, 1.084 for private doctoral universities, and 0.765 for private baccalaureate colleges. the large sums of private income at elite institutions allowed them to simultaneously spend more per student and maintain larger enrollments than their counterparts of the same institutional type. 27 even though they spread their resources more thinly due to higher enrollments, elite institutions were still able to spend more per student due to their wealth advantage. for both public and private doctoral universities, the relationship between enrollments and expenditures per student at these institutional types is no longer positive when controls for institutional wealth or prestige are added. table 5 contains log-log regressions (with no control variables) that reveal similar relationships as the correlation terms from table 4 . the first and third columns of regression results again demonstrate that expenditures per student and enrollments are positively correlated for public and private doctoral universities and close to zero for private baccalaureate colleges. the second column of results reveals that that this relationship becomes very close to zero once per-student private income is included as a control variable. the final column of results shows that a similar pattern is present when the u.s. news peer assessment measure is instead included as a control variable. 28 the results for the last regression are a bit more striking in that the relationship between expenditures per student and enrollments becomes negative. as a whole, the results in table 5 indicate that variation in institutional wealth and prestige is closely intertwined with the positive correlation between per-student expenditures and enrollment. we examined financial inequality across the full-population of nonprofit title iv eligible u.s. higher education institutions with a carnegie classification of doctoral, master's, baccalaureate, or associate's. we 25 the u.s. news rankings of best national universities and best national colleges are a commonly used proxy for institutional prestige. for our set of institutions in 2017, all of the institutions that were in the top 25 of these two rankings were classified as private doctoral universities, public doctoral news rankings. these scores are based on ratings for each higher education institution provided by presidents, provosts, and deans of admissions. ratings are based on a scale of 1 (marginal) to 5 (distinguished). we use u.s. news peer assessment scores rather than u.s. news rankings because these scores provide a direct measure of prestige and the formula for rankings includes some measures of expenditures. the regressions in the last two columns of table 5 are based on a slightly smaller data set because u.s. news did not report scores for some institutions. conclude here by highlighting four key findings from our work and discussing their implications for future research. our first finding of note is that inequality levels for total expenditures increased between 2004 and 2017 but fell for expenditures per student during the same period. these opposing trends prompt questions about the proper measure to use when examining financial inequality across higher education institutions. lau and rosen (2016) noted that neither total expenditures nor perstudent expenditures are ideal measures of the resources available at a specific college or university. if higher education institutions are providing a pure public good (that is non-rival in consumption), then a measure based on total expenditures is ideal. in contrast, a measure based on expenditures per student is ideal if these institutions are providing a pure private good (that is rival in consumption). because the mix of goods and services provided by higher education institutions presumably falls somewhere between these two extremes, measures based on total expenditures and measures based on per-student expenditures will bracket the ideal measure (lau & rosen, 2016) . when studying organizational finances, however, researchers often only examine one measure, which is typically the per-student measure. our second finding of note highlights how the differences in inequality patterns between the total expenditure measure and the perstudent expenditure measure related to enrollment levels. our decomposition results indicate that these differences were driven by growing inequality in enrollments across higher education institutions as well as an increasingly positive correlation between enrollments and expenditures per student. the logic underlying the decomposition implies that the future trends for these two measures will be affected by future changes to the distribution of enrollments across higher education institutions. recent analyses by grawe (2018) suggest that the distribution of enrollments will substantially change in the upcoming decades. the college-aged populations in most northeast and midwestern states will substantially decline, while the same populations in mountain, western north central, and south atlantic states will increase. grawe's analyses also suggest that enrollment trends will vary by institutions' rankings in the u.s. news and world report. as lower-ranked institutions in the midwest and northeast experience enrollment declines during the upcoming decades, their total expenditures will likely decrease dramatically but their per-student expenditures may not fall given that some revenue sources do not decline commensurately with enrollments. so, the use of multiple measures of organizational revenues and expenditures will become even more vital in the future. our third finding relates to our decomposition of inequality figures into the portions associated with between-group variation and the portions associated with within-group variation. both components contributed to most of our general findings, especially those pertaining to changes over time. for example, the growth in total expenditure inequality, the decline in per-student expenditure inequality, and the growth in enrollment inequality were present in the results for both between-group and within-group variation. findings relating to a point in time were more likely to vary across these two sources of variation. for example, the between-group correlation term for enrollments and per-student expenditures was positive while the corresponding withingroup correlation term was negative. previous research did not separately examine within-group inequality as it either examined inequality in general (davies & zarifa, 2012; lau & rosen, 2016) or examined between-group inequality by comparing means for different organizational groupings (desrochers & wellman, 2011) . further analyses of within-group differences produced our fourth finding of note: inequality patterns vary across institutional control and carnegie classification categories. we observed the greatest inequality in expenditures among institutional types that rely more upon private sources of funding. the private sector had greater levels of inequality than the public sector, and within-sector, inequality was higher for those carnegie classifications that contain institutions with high levels of private income. future research could further investigate how organizational financial inequality is shaped by the larger funding environment, which includes governmental policies that distribute public sources of funding to higher education institutions. notes: the dependent variable is log per-student e&g expenditures. each regression includes an intercept. * represents p < 0.05 and ** represents p < 0.01. increasing time to baccalaureate degree in the united states cohort crowding: how resources affect college attainment decomposing earnings inequality: the role of working hours replacing state appropriations with alternative revenues sources: the case of voluntary support institutional eligibility for participation in title iv student aid programs under the higher education act: background and reauthorization issues measuring inequality the stratification of universities: structural inequality in canada and the united states delta cost project documentation of ipeds database and related products table of contents the impact of price caps and spending cuts on us postsecondary attainment trends in college spending 1999-2009. where does the money come from? where does it go? what does it buy? delta project on postsecondary education costs, productivity and accountability preserving at-risk public universities as economic engines. insidehighered.com demographics and the demand for higher education assessing inequality product mix and cost disaggregation: a reinterpretation of the economics of higher education the problem with the delta cost project database using ipeds for panel analyses: core concepts, data challenges, and empirical applications are universities becoming more unequal? the review of higher education decomposing the dispersion of higher education endowments state higher education executive officers (sheeo). (2010). four-state cost study unequal higher education: wealth, status, and student opportunity estimating the cost of a bachelor's degree: an institutional cost analysis subsidies, hierarchy and peers: the awkward economics of higher education we found especially interesting variation in the correlation term between enrollments and expenditures per student, as this relationship was positive for private and public doctoral universities and negative for private and public master's institutions and public associate's colleges. in our results section, we discussed how these differences relate to research activity as well as institutional wealth and prestige. future research could examine these relationships in greater depth. in this and other areas, this paper is not the last word on organizational financial inequality. we hope that the insights into this topic that we have provided can be built upon in the future. key: cord-318817-95tw84z8 authors: hubner, c. v. k.; bruscatto, m. l.; lima, r. d. title: distress among brazilian university students due to the covid-19 pandemic: survey results and reflections date: 2020-06-20 journal: nan doi: 10.1101/2020.06.19.20135251 sha: doc_id: 318817 cord_uid: 95tw84z8 the first case of infection with the new coronavirus was identified in december 2019 in wuhan, china. in march, the world health organization (who) defined the disease epidemic as a pandemic. thus, a quarantine was imposed by many governments. as a consequence, and given that epidemiological outbreaks of infectious diseases, such as covid-19, are associated with psychological disorders and symptoms of mental illness, researchers at the shanghai mental health center have created the covid-19 peritraumatic distress index (cpdi), in which the results are obtained: normal, mild/moderate distress and severe distress. the main objective of the study was based on the application of cpdi, in order to identify the health and well-being of brazilian students from different undergraduate courses at the pontifical catholic university of sao paulo (puc/sp) during the covid-19 pandemic and to test the hypothesis that medical students suffer more than students from other courses. the research is based on a cross-sectional observational study, in which we applied, using google formsr, the questions contained in cpdi, among with demographic data: age, sex, educational institution, undergraduate course and school year. the index was applied online for seven days in which a total of 654 valid responses were obtained: 501 (76.6%) female and 149 (22.8%) male. regarding age, 333 students (50.91%) were 17-20 years old, 279 (42.66%) between 21-25, 30 (4.59%) between 26-30 and 12 (1.84%) between 31-50. the results indicate that the participants reported significant psychological distress, according to the cpdi score. practically 90% (87.92%) of the students experienced suffering, while only 12.08% did not suffer. the study provides the first empirical evidence on the level of psychological distress in brazilian university students during the covid-19 pandemic. also, it suggests support and monitoring of university students during and after the pandemic, with effective and efficient intervention in their mental health. the first case of infection by the new coronavirus was identified in december 2019, in wuhan, china 1 . since then, cases have spread rapidly across the world. in february, brazil's first case was identified in são paulo. at april 24, 2020, in the country, there were 52,995 confirmed cases and 3670 deaths, while in the world the number of cases totals 2.79 million and a total of 196 thousand deaths 2 . in march, the world health organization (who) defined the disease epidemic as a progressive pandemic. 3 covid-19, the name given to the disease caused by the new coronavirus, sars-cov2 4 , despite not having its pathophysiology completely elucidated, presents with a clinical picture of pneumonia, fever, difficulty breathing and pulmonary infection 5, 6 , associated with a disseminated intravascular coagulation (dic) 7, 8 . as it is a disease whose transmissibility occurs through one sick person to another or through close contact 9 , the best form of prevention is social isolation 10 . therefore, quarantine was established by many governments. as a consequence, and given that epidemiological outbreaks of infectious diseases, such as covid-19, are associated with psychological distress and symptoms of mental illness [11] [12] [13] , many people have their lives impacted and this should have an important impact on their health, since in 1947 the who developed the concept of health as "a state of complete physical, mental and social well-being and not just the absence of disease or infirmity" 14 . for instance, it is worth mentioning that medical students in china, during the pandemic, experienced more stress and anxiety. 15 researchers at the shanghai mental health center, during the epidemic, developed a questionnaire to research and assess the level of peritraumatic distress specific to covid-19. the word distress represents an act or effect of suffering, physical pain, anguish, affliction, bitterness, patience and resignation 16 the main objective of the study was based on the application of the covid-19 peritraumatic distress index, in order to identify the health and well-being of brazilian students from different undergraduate courses at the pontifical catholic university of são paulo (puc/sp) during the covid-19 pandemic and to test the hypothesis that medical students suffer more than students from other courses. besides that, quantify the prevalence and severity of psychological distress in the sample studied, and therefore provide data to adapt and implement relevant mental health intervention policies to deal with this challenge in an efficient and effective manner, prioritizing assistance to psychologically affected people. university of são paulo, was voluntary, and we assured the participants confidentiality and anonymity of their responses, evidenced by the free and informed consent term (can be seen in the appendix), in which the participants had to read and accept it in order to answer it. the inclusion criteria used were undergraduate students, specifically from the pontifical catholic university of são paulo, using the electronic form. those who were not undergraduate students enrolled at the pontifical catholic university of são paulo were excluded, as well as those who, perhaps, were not actively enrolled. the study uses a cross-sectional observational analytical primary study design, evaluating medical students and other undergraduate courses at puc/sp. to them it was sent a semi-structured and self-administered questionnaire, answered electronically, using google forms r , with the questions contained in cpdi along with demographic data: age, gender, educational institution, undergraduate course and the respective academic year. the questionnaire contains 24 questions, each of which has 5 answers, whose score is 0 to 4 points for each question. at the end of the questionnaire, the result is obtained from the sum of the answers in addition to the sum of 4 more points. finally, the points of the responses of each participant, individually, will be added to reach a score. thus, the compiled data will be analyzed according to the following statistics: 1. description of the distribution of absolute and relative frequencies of the variable degrees of suffering, age, gender, course and grade; . cc-by-nc 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june 20, 2020. . https://doi.org/10.1101/2020.06.19.20135251 doi: medrxiv preprint 2. description of the mean and standard deviation of the suffering score variable. 3. analysis of differences in suffering scores according to age, gender, course and grade, with special attention to the medical school and its boarding school, using linear regression. for all tests, a significance level of 0.05, bilateral, were adopted. the english and portuguese versions of cpdi, as well as its scoring key can be found in the appendix. the covid-19 peritraumatic distress index (cpdi) was applied online for seven days, from 12 may 2020 to 18 may 2020, in which a total of 764 responses were obtained. of these, 110 were excluded, as these participants did not belong to (table 1) . . cc-by-nc 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june 20, 2020. languages 0 (0%) 6 (0,918%) 0 (0%) 0 (0%) 0 (0%) 0 (0%) 6 (0,918%) social service 13 (1,988%) 0 (0%) 0 (0%) 0 (0%) 1 (0,153%) 0 (0%) 14 (2,141%) theology 0 (0%) 0 (0%) 0 (0%) 2 (0,306%) 0 (0%) 0 (0%) 2 (0,306%) it is important to point out that when the research was started the state of são paulo, where the university is located, in brazil, was already in its 50th day of . cc-by-nc 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june 20, 2020. 18 based on these data, it is possible to speculate a correlation between the increase in psychological distress and these stressors. through statistical analysis of multiple linear regression, a high rate of peri traumatic distress was associated with the female gender, with undergraduate courses that not medicine and with non-medical interns. the women who responded showed higher psychological distress when compared to men (mean (sd) = 47.14 age did not appear to be a significant predictor of psychological distress (p=0.738), as occurred in a chinese study sample. 20 the study has certain limitations. first, the research is based on a crosssectional observational study. therefore, there was no follow-up of individuals. second, our sample does not reflect the national college students, since the objective was to evaluate the students from puc/sp. third, although there were responses from several undergraduate courses at the university, some of them presented a very small sample number, such as communication and multimedia (0.153%), data science and artificial intelligence (0.153%), theology (0.306%), philosophy (0.306%), when compared to other courses, such as medicine (18.196%) . the study provides the first empirical evidence on the level of psychological distress in brazilian university students during the covid-19 pandemic. the results demonstrate a significant psychological disorder in the participants. therefore, the study suggests support and monitoring of university students during and after the pandemic, with effective and efficient intervention in their mental health, such as (1) greater attention to vulnerable groups, such as women, non-medical interns and students of others courses that not medicine; (2) university strategy, with planning . cc-by-nc 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted june 20, 2020. . cc-by-nc 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. your anonymity will be guaranteed. the online system, chosen for the application of the questionnaire, does not allow us to identify the respondents to the questionnaire and any material that identifies their participation will be released without their authorization. if you still have doubts, you can be informed about any aspect of the study you want, before or after answering this questionnaire, through the researchers' contacts. we clarify that there will be no costs and no payment for those who participate in the study. we consider that there are no risks in participating in this research. at the end of the study, the researchers commit to communicating the results obtained with the research. the students will also participate in the project: . cc-by-nc 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted june 20, 2020. understood what was explained to me in this term, i agree to participate in the research project. your acceptance to participate will be considered as your electronic signature of this informed consent form and the questionnaire link will only be available after your acceptance. . cc-by-nc 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted june 20, 2020. . https://doi.org/10.1101/2020.06.19.20135251 doi: medrxiv preprint a novel coronavirus from patients with pneumonia in china who. coronavirus disease (covid-2019) situation reports coronavirus confirmed as pandemic who. novel coronavirus-china epidemiology, causes, clinical manifestation and diagnosis, prevention and control of coronavirus disease (covid-19) during the early outbreak period: a scoping review municipal health and health commission's briefing on the current pneumonia epidemic situation in our city 2019-ncov epidemic: address mental health care to empower society timely mental health care for the 2019 novel coronavirus outbreak is urgently needed. the lancet psychiatry mental health problems and social media exposure during covid-19 outbreak the psychological impact of the covid-19 epidemic on college students in china dicionário houaiss da língua portuguesa a nationwide survey of psychological distress among chinese people in the covid-19 epidemic: implications and policy recommendations painel coronavírus brasil unprecedented disruption of lives and work: health, distress and life satisfaction of working adults in china one month into the covid-19 outbreak key: cord-272667-vh3uvvha authors: aggarwal, rashi; coverdale, john; balon, richard; beresin, eugene v.; guerrero, anthony p. s.; louie, alan k.; morreale, mary k.; brenner, adam m. title: to disclose or not: residency application and psychiatric illness date: 2020-08-28 journal: acad psychiatry doi: 10.1007/s40596-020-01296-9 sha: doc_id: 272667 cord_uid: vh3uvvha nan # academic psychiatry 2020 the process of applying for and being accepted into a residency program is the culmination of successfully negotiating all of the emotional, behavioral, and educational challenges accompanying medical education. medical students usually perceive this process as very stressful because so much is at stake. students worry about matching into a program/location of their choice in a specialty of their choice. in recent years, this process has intensified, with students applying to a much larger number of programs and accepting more interviews to increase their odds of matching successfully [1] [2] [3] . what makes matters worse is that this process comes at the tail end of a highly challenging medical education. this stress can be compounded when deciding whether or not to disclose a personal experience with psychiatric illness [4] . in the october 2020 issue of academic psychiatry, three articles examine different aspects of psychiatric illness in medical students and the residency application process. alfayez and alshehri [5] examine the prevalence of anxiety and depressive symptoms in medical students in saudi arabia and associated perceived stigma. pheister et al. [6] examine the impact of disclosure of psychiatric illness on the residency selection process. goldenberg and wilkins [7] provide a glimpse of how educators think through the process of protecting the privacy of a medical student receiving inpatient treatment from other medical students and their take on the stigma of psychiatric illness. these issues are all the more relevant given the findings that a substantial number of medical students and residents experience depression, anxiety, or burnout. in one large systematically conducted review [8] , for example, rates of depression and depressive symptoms in medical students were about 27%, and the rate of suicidal ideation about 11%. rates increased on entry into medical school, and only a minority with depressive symptoms reportedly sought treatment. some medical students also suffer from burnout, although prevalence varies across countries [9] . studies indicate that only a minority of medical students experiencing high levels of burnout seek help [10, 11] . with such high rates of psychological stress, why do so many medical students decide not to seek treatment? the article by alfayez and alshehri [5] in this issue suggests that despite having low personal stigma, a majority of the students indicated that if they received such treatment, they would hide it from others. around half of the students thought that program directors, supervisors, and other students would react negatively to knowledge that a student had or was receiving treatment for psychiatric illness [5] . their findings have a striking similarity to studies examining stigma among medical students in the us [11] [12] [13] [14] . in one such study [11] , 50.3% of students felt that a residency program director would pass over their application if the program director were aware that the student had emotional or mental health problems. in one personal account, a student was advised to avoid disclosing psychiatric illness during the residency application process [15] . mentors and educators created a dichotomy by indicating on the one hand that having a psychiatric illness and undergoing treatment is not stigmatized and on the other hand that disclosure might impact a student's residency process negatively. medical students tend to worry about the possibility of stigma and discrimination, which can factor in a decision to not seek help [5, [11] [12] [13] [14] . this situation is troubling, because lack of disclosure is linked to lack of receiving help [16] . medical student attitudes toward getting treatment for for example, in a study of female physicians [17] , almost 50% reported that they believed they met the criteria for a psychiatric illness at some point in their life but had not sought treatment for reasons including that they could manage it themselves, that the diagnosis was embarrassing or shameful, fear of resulting licensing issues, and impediments to available time. in a large study of surgeons [18] , around 30% reported depressive symptoms but only about one quarter of those with recent suicidal ideation sought psychiatric help, with the majority being concerned about licensure requirements. a third example concerned a survey of practicing physicians in michigan and their views on the impact of depression on their approach to seeking mental health care [19] . in this study, roughly one quarter of respondents reported knowing a physician whose professional standing had been compromised by being depressed. in addition, physician concerns about confidentiality were associated with a higher likelihood of their avoiding care. are medical students correct that disclosure of a psychiatric illness puts them at a disadvantage in the application process? pheister et al. [6] attempt to answer this question by examining the effect of disclosure of major depressive disorder (mdd) during the residency application process on the chances of being invited for an interview and the eventual ranking. the 380 program director respondents, which constituted an 11% response rate, were given vignettes with some applicants disclosing mdd, others disclosing diabetes mellitus type i, and some without either. the vignettes were further categorized into "perfect," "good," and "average." the authors found that applicants with mdd were at a disadvantage compared with other applicants, including those with diabetes. however, candidates with mdd and a "perfect" application did better than the "average" or "good" applicant without any illness. simply put, if all things are equal in an application, disclosure of depression at this stage may put an applicant at a disadvantage compared with those who disclose diabetes. the impact was not viewed as severe enough, though, to discount for other strengths in the application when compared with "weaker" applicants. as brenner et al. [4] point out, while bias against applicants with disclosed illness can exist, there may also be concerns about the wellbeing of the applicant as a future resident and of the possible impact of their illness on the program. a program director's decision to not rank an applicant with a disclosed illness-psychiatric or not-might stem in part from a concern about the resident's future performance during residency. pheister et al. [6] found that some comments by residency directors indicated a concern about an illness causing unexpected leaves of absence and the resulting impact on the program. the authors speculated whether this concern was true only for programs with limited resources in that should training need to be extended, there may not be adequate call coverage and monetary resources. while program directors may consider an applicant's fitness for future practice, the question remains as to whether all psychiatric and nonpsychiatric illnesses are judged by the same standards or whether psychiatric illnesses are unduly stigmatized. although pheister et al. [6] did not find evidence for such parity, their study cannot answer whether program directors prefer applicants with non-psychiatric illnesses over psychiatric illnesses. moreover, in one, now dated, multi-institutional study [20] , students appeared more concerned about the potential impact of illnesses such as depression, substance use, cancer, anxiety, and hiv than diabetes or arthritis, so diabetes may not be an optimal comparison. are the attitudes of psychiatry program directors toward applicants with mental illness different from the attitudes of non-psychiatry program directors? studies have found that psychiatrists are not immune to stigmatizing attitudes toward those with psychiatric illnesses [21] [22] [23] [24] . these studies measured bias in psychiatrists toward patients with psychiatric illnesses, however, not of program directors who evaluate residency candidates with mental health issues. nevertheless, there may be some common elements in how program directors and the general population of psychiatrists perceive mental illness. looking ahead, educators should seek to understand how the nature and course of an illness, possible impairment, and potential to relapse might affect residency application processes and decision-making. how the size and availability of resources in a program impact decision-making also needs to be understood. given the studies mentioned, do we have an answer to the difficult question of whether a student should disclose a personal experience of psychiatric illness? unfortunately, the answer is not a straight yes or no but more nuanced. when there are gaps or weaknesses in the application that are explained by an illness, then disclosure is likely not a choice. otherwise, students might consider whether a narrative exists around disclosure. would the applicant decide to disclose if the applicant had another type of illness? did the illness result in a positive experience for the applicant? for example, did it lead to selfgrowth? did the illness inspire the applicant to choose a particular medical specialty? or did it lead to a change in the applicant's attitude or behavior toward patients? did the illness lead toward greater understanding and appreciation of the experience of a psychiatric disorder-perhaps increasing empathy and compassion for patients? did it lead toward helping others who have similar illnesses? personal experiences with illnesses can help foster empathy for patients [25, 26] . for example, one study found that newly qualified doctors with personal illness experiences reported more academic difficulties and higher anxiety but also reported that reflecting on these experiences improved their empathy and compassion for patients [26] . answers to the above questions might serve as a guidepost for the eventual decision on whether to disclose or not. medical students' narratives of their struggle with and successful treatment of a psychiatric condition might prove valuable in an appreciation of courage, fortitude, and dedication to helping others. in addition, it may shed light on how a particular individual navigates adversity and indicate a level of personal resilience. residency directors might very well be disposed to applicants who self-disclose for this reason. medical student and resident distress are, after all, commonplace, and residency programs are invested in providing support and mental health care for all residents. in addition, residents have reported that they want their program director to inquire about wellness and they may be more likely to seek help if recommended and facilitated to do so by their program director [27] . knowing that an applicant has a medical or psychiatric condition may actually be a benefit for a program as support, information, resources, and help may be offered. not knowing and finding out later could potentially be more devastating to a program. proactively and universally addressing issues of mental health, lessons learned through inevitable adversities experienced, and coping strategies that enhance resilience becomes particularly important when one considers the covid-19 pandemic and its impact on the mental health of students, residents, and physicians. in addition, while this editorial's discussion has focused on applicants' disclosure during the application process, program directors must remember that it is not only unethical but also a legal violation to seek this information during the interview process. decisions that are prejudiced against applicants with specific illnesses could violate the americans with disability act. as psychiatry program directors and role models, academic psychiatrists need to be responsible not to let bias or information about an applicant's medical history negatively affect their decision making, particularly if the applicant has no history of impaired performance. separate potential concern about students' medical conditions from their talent, achievements, and goodness of fitting into one's program. most importantly, let us all aim to recognize and correct any potential biases that may operate to the detriment of applicants in residency application processes. disclosure on behalf of all authors, the corresponding author states that there is no conflict of interest. viewpoint from 2 undergraduate medical education deans the residency application process: working well, needs fixing, or broken beyond repair? residency placement fever: is it time for a reevaluation residency interviews in the 21st century training as a psychiatrist when having a psychiatric illness perceived stigma towards psychological illness in relation to psychological distress among medical students in riyadh, saudi arabia. acad psychiatry the impact of mental illness disclosure in applying for residency a medical student is psychiatrically hospitalized. acad psychiatry prevalence of depression, depressive symptoms, and suicidal ideation among medical students: a systematic review and meta-analysis burnout in medical students before residency: a systematic review and meta-analysis utilization of counseling services at one medical school the impact of stigma and personal experiences on the help-seeking behaviors of medical students with burnout depression, stigma, and suicidal ideation in medical students factors associated with undertreatment of medical student depression predictors of depression stigma in medical students: potential targets for prevention and education to those "out there global pattern of experienced and anticipated discrimination reported by people with major depressive disorder: a cross-sectional survey i would never want to have a mental health diagnosis on my record": a survey of female physicians on mental health diagnosis, treatment, and reporting special report: suicidal ideation among american surgeons a survey on the impact of being depressed on the professional status and mental health care of physicians perceptions of academic vulnerability associated with personal illness: a study of 1,027 students at nine medical schools what about psychiatrists' attitude to mentally ill people? attitudes of mental health professionals toward people with schizophrenia and major depression stigma and mental health professionals: a review of the evidence on an intricate relationship attitudes of mental health professionals about mental illness: a review of the recent literature are doctors who have been ill more compassionate? attitudes of resident physicians regarding personal health issues and the expression of compassion in clinical care the relationship between medical students' and doctors' personal illness experiences and their performance: a qualitative and quantitative study resident preferences for program director role in wellness management publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-303182-k7j4iwhp authors: chan, maggie mee kie; yu, doris sf; lam, veronica sf; wong, janet yh title: online clinical training in the covid‐19 pandemic date: 2020-07-09 journal: clin teach doi: 10.1111/tct.13218 sha: doc_id: 303182 cord_uid: k7j4iwhp nan t he spread of the coronavirus disease 2019 (covid-19) around the world has had a major impact on nursing education. to minimise the transmission of the infection, the hong kong government has recommended that everyone must strictly follow preventive measures, such as staying at home and social distancing, and the food and health bureau has restricted the maximum size of groups to four people. the university of hong kong is in lockdown and clinical practicals have been suspended. this has interrupted student learning and has created uncertainty for teaching. all classes have moved online. what can teachers do to enhance nursing students' clinical experience without clinical practicals during the covid-19 pandemic? our nursing school has implemented different strategies based on the concept of experience-based learning. 1 as nursing educators, we also formed a team to work out strategic plans to ensure that our students continue learning and are well prepared for clinical practice. experience-based learning helps a student to become capable through reflection on the experiences of participating within the triad of student, patient and clinical teacher. clinical teachers should engage, encourage and empower students to extend their current capabilities. 1 how should instructors teach clinical experiences when nursing students are learning at home? to increase nursing students' capabilities, curiosity and perceived clinical relevance in learning in a caring context, remote learning tools such as zoom™ (zoom video communications inc., san jose, ca, usa) and google meet (google llc, mountain view, ca, usa) play key roles when relocating teaching and learning activities to a virtual classroom. other virtual simulation programmes can also be used to enhance students' critical thinking skills and clinical judgement in case management. chinese students are often depicted as passive and quiet in classroom learning. 2 students seldom ask questions during classes, or even in laboratory sessions, until they fully understand what is being taught. this may decrease the level of interaction in online classes. we also identified this behaviour when we implemented online classes. we have to take an active role in engaging students to create a learning environment. as such, online teaching can add further challenges when engaging students in the active cognitive process. we have to take an active role in engaging students to create a learning environment for the online classes, first we used flipped classrooms to engage students in pre-reading, with online quizzes to increase their self-awareness of learning needs. during the online class, clinical scenarios were created to illustrate the clinical relevance of the subject matter being taught. by requiring them to identify clinical problems and plan nursing care through mini quizzes, students are engaged in an active cognitive process of knowledge integration and application. constructive feedback on their performance is given immediately to correct misconceptions and to optimise learning. an atmosphere of encouragement is created to invite questions and dialogue from the students. second, as students perceive the nursing change-of-shift handovers as the most challenging in the actual clinical setting, a pragmatic approach using two teachers to demonstrate nursing change-ofshift handover in a simulated clinical environment was used to arouse the students' interest. to strengthen the cognitive process in knowledge integration and application, a clinical case was designed to illustrate disease manifestation, and nursing assessment and observation, thereby further strengthening the cognitive process in learning. finally, students are responsive to teachers who care for their needs. because they were anxious, stressed and overwhelmed about the global pandemic, a 10-minute 'relaxed chat' was integrated into each class to address concerns about issues outside academia. simulation teaching has been used in our school since 2010. we developed a simulated training ward in 2018 to enhance nursing students' ability to provide care to patients competently and safely. 3 we believe that a simulated training ward can provide a constructive educational environment similar to the clinical setting to help nursing students learn and build their knowledge and skills. with the university in lockdown, however, nursing students cannot access simulation training. therefore, we introduced clinical virtual simulation in our pedagogy. clinical virtual simulation is an effective innovative technology for clinical training, in which students can interact with virtual patients in a re-creation of a real environment. clinical virtual simulation can enhance nursing students' decision making and critical thinking, as well as their competence in performance and psychomotor skills. 4 we use briefing and debriefing in the virtual simulation to facilitate student learning. we have found that innovative technology can improve our teaching and student learning; however, there are still challenges that we must solve. how can we ensure the quality assurance of online teaching? how can we provide an environment in which students feel safe without cyberbullying? could we develop an online teaching model to guide and structure our online teaching? the covid-19 pandemic has been a wake-up call propelling us to a new era in nursing pedagogy. the impact of the covid-19 pandemic is not short term, and long-term issues will affect our nursing pedagogy. experience based learning (exbl): clinical teaching for the twenty-first century chinese nursing students' culture-related learning styles and behaviours: a discussion paper nursing to a new level clinical virtual simulation in nursing education: randomized controlled trial key: cord-320542-ihsr7bhp authors: spanemberg, juliana cassol; simões, cinthia coelho; cardoso, juliana andrade title: the impacts of the covid‐19 pandemic on the teaching of dentistry in brazil date: 2020-08-19 journal: j dent educ doi: 10.1002/jdd.12364 sha: doc_id: 320542 cord_uid: ihsr7bhp a new disease called coronavirus disease 2019 (covid‐19), caused by the severe acute respiratory syndrome coronavirus‐2 (sars‐cov‐2) virus, was discovered in the city of wuhan in china in december 2019, and has reached, quickly and progressively, several countries on different continents. even before the world health organization recognized the covid‐19 epidemic as a pandemic, the brazilian ministry of health had already declared covid‐19 a national public health emergency due to the confirmation of cases in brazil. in this scenario, the educational sector was one of the first to suffer the effects of the pandemic soon after the announcement of social distancing as a way to prevent the collapse of the unified health system. the aim of this paper is to report how brazilians dental schools are leading with the new coronavirus pandemic. a new disease called coronavirus disease 2019 , caused by the severe acute respiratory syndrome coronavirus-2 (sars-cov-2) virus, was discovered in the city of wuhan in china in december 2019, and has reached, quickly and progressively, several countries on different continents. in early march, the director general of the world health organization (who) recognized the covid-19 epidemic as a pandemic because the disease had already caused 4291 deaths and infected >118,000 people in 114 countries. 1 in february, the brazilian ministry of health had already declared covid-19 a national public health emergency due to the confirmation of cases in brazil. in this scenario, the educational sector was one of the first to suffer the effects of the pandemic soon after the announcement of social distancing. the brazilian supreme court of justice decided to grant to states and municipalities the decision on the implementation of necessary measures to fight the pandemic using epidemiological data from each location as a thermometer. however, following the who guidelines, most mayors and governors suspended face-to-face classes at education institutions, among other measures, aiming to prioritize social isolation in an attempt to reduce the spread of the virus and prevent the collapse of the unified health system. then, the ministry of education authorized the replacement of on-going classroom subjects for remote classes in undergraduate courses using information and communication technology platforms. the objective was to maintain the student's study routine and create a virtual teacher-student contact, so that there would be the least possible damage to the subjects' teaching plan and the academic calendar. 2 higher education institutions, on an emergency basis, provided digital platforms for remote classes. however, there was no prior training, which requires a gradual adaptation of the use of such tools. thus, professors sought alternatives and teaching strategies to engage students who had opted for face-to-face classes upon entering undergraduate courses. with such a challenge, undergraduate and graduate courses in dentistry and students who opted for face-toface classes had their theoretical classes delivered live in remote classes. the practice (laboratory, preclinical, and clinical), which is important for the development of specific skills of dentist training, was suspended. there was also a search for alternative methods to apply tests using tools of information and communication technologies (icts) aiming a safe application of virtual tests. however, the adoption of these technologies in brazil is still limited due to low accessibility by the low-income population, which does not always have compatible electronic equipment and access to a quality internet network. 3 new multimedia tools have driven significant social and cultural changes in modern society. these resources have allowed the maintenance of the student-teacher relationship in real time, differentiating it from distance learning. icts potentiated the flow of information in remote educational environments, allowing the student, an interacting individual, to break hierarchical knowledge. a group from italy, a country severely affected by the pandemic, reported in an article that self-learning among students improved their ability to use online resources and somehow encouraged learning independence. 4 however, it is responsibility of dentistry faculties to promote, by structuring pedagogical projects, the development of students' skills in diagnosis, planning, and treatment of orofacial pathologies through the clinical care they offer to the population. adaptation of techniques and refinement of manual dexterity are part of the daily training of dentistry students. however, how can these qualities be developed within a pandemic scenario? could these activities be carried out in a simulated way using specific software or other techniques? one of the challenges of higher education is to create new pedagogical models that promote students' creativity in order to avoid exhausting the traditional model of teaching centered on the figure of the teacher. in view of the reality of the brazilian population in the covid-19 epidemic, the work of teachers has become challenging. the search for alternatives for the development of students' skills and competences, as provided for in the pedagogical plan of courses, has been incessant, highlighting remote classes. for basic courses, the use of virtual blades and other software allows manual practice through virtual means. the challenge is greater in specific courses. thus, a suggestion is to work on the discussion of interdisciplinary clinical cases in small groups of students aiming to plan, analyze, interpret, and make logical decisions, in addition to working on communication and interpersonal relationships. other possibilities are the adoption of software to perform interactive activities, such as kahoot!, game design by students, quiz etc. health promotion can be worked on by disseminating educational material, such as educational videos and folders, on social networks. another possibility is to make calls and send messages to patients at the school clinic using the institution's database. such contact with the patient aims individual educational guidance by updating signs and symptoms of disease progression, always following guidelines and the legislation of the respective agencies in brazil. for the development of manual skills in different dental specialties, it is possible to propose the acquisition or adaptation of training materials, such as the use of dish-washing sponges, ox tongue, pig jaw, wax/soap sculptures, assembly training in a semi-adjustable articulator, molding, making removable orthodontic appliances, etc. all these activities are carried out with remote supervision by the teacher who assists students, guiding and correcting them whenever necessary. in view of these uncertainties, the organization of commissions of teachers, dental class councils, and the brazilian dental education association (abeno), thought of a safe way for students and professors to return to undergraduate and graduate courses in dentistry. this proposition was based on evidence and scientific publications. the group published manuals on biosafety standards containing guidelines for adapting physical spaces, necessary equipment, and behavior in school clinics to this new reality until the development of a vaccine. the american dental association and the brazilian federal council of dentistry (cfo) have recommended the use of extraoral imaging tests (panoramic radiography and cone beam computed tomography) preferably to intraoral examinations due to the tactile stimulus to salivary flow during execution of these techniques. 5, 6 in addition, the brazilian association of dental radiology (abro), together with the cfo, carried out a campaign to increase the digital flow of imaging exams performed in dental radiology clinics, eliminating the stage of transport and handling of printed exams aiming to prevent the virus from spreading. however, it is known that there is a need to adapt practice physical spaces of undergraduate courses in dentistry in brazil for the interpretation of these digital images. on the other hand, the low purchasing power of most patients seen in clinical schools, who cannot afford ct scans, is well known. in this context, it is important to emphasize that intraoral radiographs have certain indications and are valuable in providing complementary data essential to the diagnosis of some pathologies. they also help in planning. therefore, they must be performed whenever necessary. 6, 7 in addition, one of the safest ways to acquire practical clinical skills in healthcare, including dentistry, is simulation exercises without the need for the student's physical presence in the clinical environment and, obviously, without a direct contact with patients. the advance in virtual reality (vr) has allowed simulation technologies to create a series of opportunities for education systems in health centers. such systems provide the tutor and students with continuous and integrated feedback. 8 vr simulators have the capability of tactile feedback, which allows students to touch and feel the dental tissue virtually. studies have shown that the use of vr has improved the acquisition of manual dexterity in dentistry courses in the operative area. 9 however, a high investment from colleges is required to offer this type of teaching methodology. thus, as professionals of dental teaching institutions, we must be aware of new education models and new vr simulation technologies and consider them as a useful and complementary tool for our students, given the current world pandemic situation and future illnesses that may arise. perhaps this modality, already practiced in some developed countries, will become fundamental in universities and a conventional dental training approach. its effective and safe use for both students and patients is possibly one of the many changes that will allow remote learning during the covid-19 pandemic and in the years that follow this event. finally, a greater attention is needed to the mental health of the student and the teaching staff. despite existing knowledge, the scale of this pandemic is different, as are its effects on the population. a recent study has shown that this pandemic has deleterious effects on the mental health of university students. this fact reinforces the need to continue investigating this issue to understand the mechanisms of psychological reactions underlying such an atypical and challenging period of life. 10 sadness, anxiety, and confusion generated by the information transmitted by the media have been detected and are constantly associated with isolation. the results of a study have shown a significant increase in psychological distress in university students during a pandemic period compared to that of normal periods. 10 thus, it is likely that students may need counseling and psychological support services during and after the covid-19 pandemic to min-imize the negative impacts on teaching and the development of their skills. there are reports of students taken by the fear of being infected by the virus, the lack of contact with colleagues, and the insecurity of real learning during this period. institutions must have the feeling of knowing how to deal with this situation in the best possible way and with less impacts for the entire community. o r c i d juliana andrade cardoso msc https://orcid.org/0000-0002-6502-8667 who director-general's opening remarks at the media briefing on covid-19 -11 use of dentistry education web resources during pandemic covid-19 covid-19: its impact on dental schools in italy, clinical problems in endodontic therapy and general considerations american dental association. ada interim guidance for minimizing risk of covid-19 transmission conselho federal de odontologia. manual de boas práticas em biossegurança para ambientes odontológicos computer assisted learning: a new paradigm in dental education use of simulation technology in dental education anxiety, depression and stress in university students: the impact of covid-19 key: cord-332290-tyr9n64g authors: gillespie, hannah; findlay white, florence; kennedy, neil; dornan, tim title: enhancing workplace learning at the transition into practice. lessons from a pandemic date: 2020-05-16 journal: med educ doi: 10.1111/medu.14240 sha: doc_id: 332290 cord_uid: tyr9n64g taking responsibility for prescribing is one of newly qualified doctors’ greatest stressors.(1) despite being a routine task, prescribing insulin is particularly stress‐inducing. the global pandemic has made it more important to minimise transitioning students’ stress; yet there are fewer clinicians to support their accelerated transitions. we had planned an intervention during 9‐week ‘clinical assistantships’ immediately before qualifying. students would write insulin ‘pre‐prescriptions’, which supervisors would endorse as prescriptions that were appropriate to dispense. a trained healthcare professional or person with diabetes (‘debriefer’) would conduct one‐to‐one case based discussions (cbds) to help students learn reflectively from experience. taking responsibility for prescribing is one of newly qualified doctors' greatest stressors. 1 despite being a routine task, prescribing insulin is particularly stress-inducing. the global pandemic has made it more important to minimise transitioning students' stress; yet there are fewer clinicians to support their accelerated transitions. we had planned an intervention during 9-week 'clinical assistantships' immediately before qualifying. students would write insulin 'pre-prescriptions', which supervisors would endorse as prescriptions that were appropriate to dispense. a trained healthcare professional or person with diabetes ('debriefer') would conduct one-to-one case based discussions (cbds) to help students learn reflectively from experience. we could not carry this out as intended: clinical placements were cut short, and students who had been exposed to covid-19 posed a risk to our debriefers with diabetes. despite that, we set out to enhance, rather than abandon, our novel form of reflective, experiential education. we encouraged students to gain experience of pre-prescribing insulin for hospitalised patients, as intended, but under pandemic conditions. since face-to-face cbds were impossible, yet one-to-one reflective discussions play an essential part in students' learning, we made a rapid decision to migrate these sometimes very personal discussions onto an on-line videoconferencing platform. a curriculum administrator took over responsibility for booking cbds from eleven teaching hospitals. central organisation of technology-supported reflective conversations made optimum use of available resources, aligned the availabilities of debriefers and students, minimised missed appointments, and made a cbd available to every interested student. we revised our operating procedures so that, as intended, deep reflective discussions took place, allowing students to reflect on experiences and make commitments to safe and appropriate future prescribing behaviour. records of these cbds provided rich, anonymised descriptions of students' reflective learning; qualitative analysis of these allowed us to evaluate the intervention. from the 77 students (approx. 25% of the cohort) who participated, detailed records are available for 25 students who both wrote an insulin pre-prescription and completed a voluntary cbd, yielding >7000 words of evaluation data for qualitative analysis. this article is protected by copyright. all rights reserved the pandemic helped students learn three key lessons: 1) participants who had navigated indeterminate situations, where factors like covid-19 created ambiguity and made prescribing decisions complex, were able to verbalise how they would manage experiences like these in future; 2) reflecting on an instance of prescribing helped students situate individual prescribing decisions in the wider context of patients' trajectories of care, where insulin prescribing had to be reconciled with other medications and comorbidities; 3) participants learned they were not alone and could develop collaborative expertise by seeking help from fellow professionals, patients, and senior colleagues. we learned lessons too: 4) it is tempting to abandon workplace education when students are numerous and clinicians are busy, yet pandemic conditions encouraged us to use good learning opportunities creatively; 5) transitioning to an online communication platform made supporting students' reflection on workplace learning so much easier that conducting cbds online is our new normal; 6) helping students seek out educational opportunities can "engage their brains" and encourage them to "ask why things are being done the way they are". this applies to medical education as much as it does clinical medicine. perceptions of uk medical graduates' preparedness for practice: a multi-centre qualitative study reflecting the importance of learning on the job accepted article key: cord-292976-qctfl6r8 authors: monday, lea m; gaynier, anthony; berschback, madeline; gelovani, david; kwon, henry y; ilyas, sahrish; shaik, asra n; levine, diane l title: outcomes of an online virtual boot camp to prepare fourth-year medical students for a successful transition to internship date: 2020-06-11 journal: cureus doi: 10.7759/cureus.8558 sha: doc_id: 292976 cord_uid: qctfl6r8 introduction changes in medical education and health care delivery have limited the ability of fourth-year medical students to perform the role of an intern prior to graduating from medical school. to address this issue, many schools have instituted residency preparation courses (sometimes referred to as boot camps) particularly for students entering surgical fields. courses for students entering nonprocedural fields are less common and most assess increases in self-reported confidence without providing objective evidence of a gain in knowledge or skills improvement. materials and methods we used a plan, do, study, act (pdsa) model to develop and pilot cycle 1 of a nonprocedural internship preparation elective in 2019. feedback was used to refine the course and map sessions to core competencies outlined by the accreditation council of graduate medical education (acgme) for pdsa cycle 2. the curriculum was adapted for remote synchronous delivery due to the coronavirus pandemic in spring 2020 using a combination of didactic lectures containing embedded polls and case-based role play responses using a chat box. students completed anonymous surveys assessing self-perceived levels of confidence, as well as an objective comprehensive assessment after course completion. results a total of 89 students participated in the course. pre-session confidence was lowest for transfusion medicine, handling pages from nursing while on call, and knowledge of the role of a chief resident. a statistically significant increase in median scores for self-reported knowledge or confidence was seen in all sessions. the percentage of students reporting that they were either confident or extremely confident also increased significantly after each session (p<0.001 for all). all sessions analyzed were rated as useful or extremely useful by more than half of the students, and 94% of the students scored 70% or higher on the comprehensive course assessment. conclusions an online virtual synchronous boot camp increased students’ confidence in handling common topics encountered during residency and demonstrated an appropriate gain in knowledge using a comprehensive assessment. we were able to adapt our curriculum to a remote model and will likely keep several sessions in an online format in the future. the goal of medical schools is to prepare students to be doctors ready for postgraduate training in their field of choice. since the publication of flexner's 2 x 2 model in 1910, undergraduate medical education (ume) curriculum has been divided into two phases: the preclinical phase, which is focused on pathophysiology and medical theory, and the clinical phase, which is focused on training in surgical and non-surgical fields. in the united states, the fourth year of medical school varies in content and structure to maximize student freedom to choose elective courses and non-clinical pursuits [1] [2] [3] . ideally, students will have opportunities during clerkship to care for patients from admission until discharge and immersed in patient care as if they are already interns. however, the degree of student immersion and experience deviates from this ideal due to several factors. patient length of stay has shortened, procedures are increasingly outsourced to interventional subspecialists, patient handoffs are frequent, and care is generally more fragmented than decades ago [4] . in addition, changes in medical education such as duty hour regulation and limited privileges within the electronic medical record (emr) reduce student autonomy. the degree of student experience with the emr varies. in one recent survey, fewer than 10% of students on their obstetrics clerkship had entered orders and less than half (47%) had written a history and physical note in the patient chart [5] . in another large survey of around 16,600 graduating medical students, 84% had entered information of some type into the emr on their internal medicine clerkship or subinternship; however, 43% had never entered admission orders [6] . yet students are expected to step into residency on july 1 and care for patients, including night shifts and with varying degrees of supervision and support. these challenges result in increased stress for interns, educational difficulties for program directors, and, most importantly, potentially unsafe care for patients. in order to address the difficulties in the transition to internship, some medical schools have created capstone courses focused on communication, common problems, procedural skills, or some combination of topics for students entering internship. elective courses for students entering dentistry [7] , surgery [8] [9] [10] [11] [12] , and obstetrics and gynecology [13] [14] [15] have been shown to increase confidence in procedural skills and communication. similar courses in non-surgical specialties are not as common; however, their numbers are increasing. boot camp experiences have been replicated in family medicine [16] , pediatrics [16, 17] , emergency medicine [18] , and with students entering a mix of fields including internal medicine [19] [20] [21] . a common result is an increase in the level of confidence or preparedness reported among students. in a 2014 meta-analysis, boot camps were shown to be an effective educational strategy to improve learners' clinical skills, knowledge, and confidence [22] . medical students graduating in 2020 were removed from all direct patient care and in-person educational activities due to the coronavirus disease 2019 (covid-19) pandemic. unfortunately, this drastically altered the content and delivery of courses at medical schools throughout the united states, including internship preparation courses. remote delivery of one boot camp course has been briefly reported for students entering pediatric residency [23] . as of may 2020, there is no published work describing the logistics and results of executing a nonsurgical internship preparation course open to all medical students regardless of match specialty. we describe a one-month intensive course entitled, "internship boot camp", which was designed after a previous pilot and adapted to be delivered remotely during the covid-19 pandemic. this online virtual synchronous interactive elective was taught by faculty, residents, and near peers, with the goal of assisting fourth-year medical students with the transition to internship. the goals of this study were to determine the effectiveness of the internship boot camp on three measures: (1) level of confidence about entering their internship, (2) ability to develop a framework for responding to common challenges, and (3) ability to demonstrate an appropriate gain in clinical knowledge using a comprehensive post-course assessment. here, we report the outcomes of a non-surgical intern preparatory course. faculty from the department of surgery at our institution had been offering a procedural boot camp focused on laparoscopic skills and knot tying since 2005; however, no such course existed for students entering nonprocedural fields. starting in spring 2019, we developed and piloted a nonprocedural preparation elective utilizing pdsa (plan, do, study, act) methodology ( figure 1 ). fifty-three students from the wayne state university school of medicine who matched into a variety of non-surgical residencies were offered an ungraded elective entitled, "intern boot camp." before developing the boot camp curriculum, a needs assessment (a systematic process for determining and addressing needs or "gaps" existing in the knowledge and skills of medical students) was conducted through a voluntary online survey. input was provided by clerkship directors, residency program directors, residents, recent graduates, and our graduating medical students. next, a targeted needs assessment was performed to determine specific deficiencies, and course sessions were created with specific goals and objectives to meet the deficiencies identified. for example, if program directors answered that new interns did not know how to write admission orders, the specific deficiencies were asked on the targeted needs assessment (such as not understanding order sets, or diet and activity orders). this course was approved by the curriculum management committee and added to the course catalog for april and may of 2019. faculty, chief medical residents, and residents taught each session. oversight was provided by the course director. course content was delivered using a variety of modalities including a combination of didactic powerpoint lectures, procedures on lightly preserved cadavers, a session on how to teach and give effective feedback, and case-based role play for handling difficult conversations, medical emergencies, and simulated pages from nursing. lastly, students completed 12 symptom-based modules (which covered common clinical scenarios encountered on call) through the wise-on-call program distributed by aquifer® on behalf of new york university (nyu) school of medicine [24] . after the pilot, a post-course survey was conducted to evaluate students' self-assessed learning achieved through the course and to learn where improvement was needed. these data were taken into consideration while adjusting and creating the schedule for april 2020. institutional review board approval was not obtained as students participated voluntarily, topics covered constituted routine education, and the assessment of the learners was part of the normal course evaluation process. in early 2020, planning began to scale this activity up to approximately one half of the class for april 1, 2020 ( figure 1 ), in preparation for delivery to all 300 students by spring 2021. student feedback was used to adjust session topics, and the curriculum was redeveloped so that content was divided into six interrelated domains based on the accreditation council for graduate medical education (acgme) core competencies: patient care, medical knowledge, practicebased learning and improvement, interpersonal communication skills, professionalism, and systems-based practice ( table 1 ) [25] . residents must be able to provide patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health. residents must be able to demonstrate knowledge about established and evolving biomedical, clinical, and cognate (e.g. epidemiological and social-behavioral) sciences, and the application of this knowledge to patient care. practice-based learning and improvement residents must be able to investigate and evaluate their patient care practices, appraise and assimilate scientific evidence, and improve their patient care practices. interpersonal communication skills residents must be able to demonstrate interpersonal and communication skills that result in effective information exchange and teaming with patients, patients' families, and professional associates. residents must be able to demonstrate a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population. systems-based practice residents must be able to demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value. the curriculum was also designed to reinforce the american academy of medical colleges (aamc) 13 core entrustable professional activity's (epas) whenever possible [26] . content was delivered in 26 sessions (22 mandatory and 4 optional). detailed mapping of each boot camp session topic with a description and the corresponding acgme core competency is provided in table 2 . with the onset of the covid-19 pandemic, all student face-to-face activities were furloughed on march 17, 2020. the course was rapidly transformed in 14 days to an entirely online curriculum. we adapted content for synchronous remote delivery using the canvas online learning management system (instructure®, salt lake city, ut, usa). synchronous sessions used a combination of didactic powerpoint presentations that were made interactive by adding facilitator polling, multiple-choice questions, and small group or individual writing exercises that were shared with the group through incorporation into the presentation or chat board. for example, students were given a mock patient sign out electronically and had to use this to respond to virtual pages from a nurse for a simulated night-on-call. students were asked to write a note about some of the overnight events, which were later de-identified and shared with the group for feedback during the "sign out" lecture. procedural training on cadavers was adapted to include an independent review of online procedure videos by students. training in pronouncing death on a recently deceased cadaver and calling his/her family was adapted to include an interactive session on pronouncing patient death and a reflective reading assignment. an "ask the interns" panel discussion was adapted by requesting students to email questions for review by three high functioning interns who then took turns answering them through video chat and the live message board. pertinent hot topics were added, including a session on telehealth and a covid-19 patient case discussion that included a rich discussion of end-of-life care and the ethics surrounding decision-making with limited resources. in addition to the synchronous remote sessions, students were required to submit written exercises with a focus on compassionate care, wellness, and self-reflection. students wrote a letter to members of the incoming class of medical students. additional independent study requirements included 12 aquifer cases and online naloxone training. sessions were grouped so that students had a monday or friday free each week to facilitate their end-of-year personal tasks such as finalizing housing plans or pre-employment requirements for their upcoming residencies, which may be in different cities or states. four additional optional sessions were held, one about preparing a professional powerpoint and three focused on topics pertinent to the care of pediatric patients. for 14 out of 26 sessions, the facilitators created a voluntary pre-session survey through a secure online data collection tool. medical students were asked to rate their confidence and understanding of requisite knowledge points on the topic before these sessions. items were graded on a 5-point likert scale (1 meaning confidence or knowledge was very poor, 3 meaning neutral, and 5 meaning very high). after delivery of each session, students were given a postsurvey with one additional question about whether they found the activity useful. a competency-based exam was administered at the conclusion of the course to assess the knowledge gained. two to three multiple-choice questions were written for each lecture session assessing student competency of objectives identified by the educator. the questions were each mapped to acgme core competencies and written with varying degrees of difficulty addressing knowledge, application, and problem-solving [25, 27] . the resulting 53-question comprehensive assessment was completed by all students except for one who contracted covid-19. after completing the comprehensive assessment, students were given a link to the correct responses. pre-/post-survey median scores were analyzed using the wilcoxon rank-sum test (mann-whitney u test) for unpaired non-parametric data. to categorically compare the percentages of students who answered positively about each session, responses in the likert scale were combined into a 3 x 2 contingency table (positive, neutral, or negative responses), and data were analyzed using the chi-square test. for example, the likert scale response categories of extremely confident and confident were combined into one, "positive" category. usefulness of sessions and post-test grades were analyzed with descriptive statistics. all tests were twotailed, with p < 0.05 as the significance level. statistical analysis was performed using ibm spss version 25 (ibm corp., armonk, ny, usa). a total of 89 students and 15 instructors participated remotely. student learners had matched into a mix of non-surgical residency programs, with internal medicine, emergency medicine, family medicine, being most common. instructors included faculty, chief residents, and current interns. pre-surveys were completed by 72-83 students (76-87%), and post-surveys were submitted by 57-78 students (60-82%). pre-test confidence scores were lowest for the sessions on transfusion medicine, the nursing pager on call activity, and chief medical resident expectations, with less than 10% of students feeling confident or extremely confident in those topics. pre-session confidence scores were highest in teaching medical students and providing feedback (55% and 69% of students felt at least confident and comfortable, respectively). the percentages of students who answered that they were confident or extremely confident in the subject matter before and after each session are summarized in table 3 . all synchronous sessions resulted in a measurable increase in the portion of students reporting confidence or extreme confidence in the subject matter. the largest increases in confidence (over 75% increase) were seen in the session about when go to your program director, chief medical resident expectations, and what do after a patient dies, followed by the session on using blood products (66% increase). all values are rounded to the nearest 1%. to compare the percentages of students who answered positively for each question, the response categories in the likert scale were combined into a 3 x 2 contingency table (positive, neutral, and negative), and data were analyzed using the chi-square test. median confidence scores before and after the sessions are presented in table 4 . before the session, median scores were lowest regarding knowledge about the role of a chief resident, confidence writing admission orders independently, using blood products, and how to fill out a death certificate. baseline median confidence scores were highest for teaching medical students and knowledge about how to give students feedback. a measurable increase in numerical confidence or knowledge scores was demonstrated for all sessions, even if the median score did not change. figure 2 . overall, all sessions analyzed were rated as useful or extremely useful by over half of the students (52-97%). the top five most useful sessions were the sessions on clinical documentation, common clinical scenarios, the night-on-call nursing paging activity, and clinical reasoning, which were rated as useful or extremely useful by 97%, 91%, 91%, and 89%, respectively. only 4 of the 13 sessions received any feedback of "not useful at all", and in all cases it was fewer than 3% of students. the cumulative post-test was taken by 88 of 89 students. of the students, 83 (94%) achieved a score of 70% or higher, 4 (4.5%) scored in the 60-70% range, and 1 scored 55%. students were asked to provide feedback comments and state whether they believed certain sessions could remain virtual; comments are summarized in table 5 . positive comments i enjoyed that this course was online. this is a big transition time for us, and it was very convenient to be able to have this online. i am glad that this was online because it allowed me to go home and be with my family for the month. the interactive nature of being able to type comments at any point and have your comments be addressed and interacted with by other students was helpful. i felt much more engaged than i normally do during in-person lectures. mandatory attendance and the chat box made it very interactive. virtual boot camp made it easier and less tiring to attend lecture since i could do it from home, and there was still great participation from the students. chat allowed other students to clarify things while the instructor was speaking, so the instructor didn't need to answer more questions. excellent resource that i plan on reviewing again the days prior to starting residency. constructive comments i think it would have been helpful to have more clinically oriented courses in-person. anything that's super participation heavy should be done in person if possible, but lectures that are just lectures (with perhaps a poll here and there) are great to have online. if you kept these lectures online, i would suggest shortening/separating them if possible. as a student, it is hard to pay full attention online with multiple hour-long lectures. summarizing handouts would be helpful to carry forward into residency. positive feedback was related to the flexibility of an online course and utility of the chat box, which, as some students reported, allowed them to be more engaged than they had been in live lectures. constructive comments included the suggestion that sessions with a focus on patient care may be better achieved by a live session in coming years. positive feedback was related to the flexibility of an online course and utility of the chat box, which, as some students reported, allowed them to be more engaged than they had been in live lectures. constructive comments included the suggestion that sessions with a focus on patient care may be better achieved by a live session in coming years. student preferences for which sessions could continue remotely versus which would be better facilitated in person are shown in figure 3 . for a majority of sessions, the percentage of students preferring a virtual format outnumbered those preferring an in-person session. the sessions most heavily focused on medical knowledge and patient care such as the clinical scenarios, sign-out activity, and night-on-call had the highest percentage of students preferring an in-person session. this study demonstrates that students who participated in this elective non-surgical boot camp felt more confident and knowledgeable about handling situations they may encounter as interns and demonstrated an objective gain in knowledge beyond self-reported levels. most of the teaching sessions were adapted to an online virtual synchronous format without difficulty, and feedback from students was overwhelmingly positive despite the change in format. had there been more time to prepare, pagers could have been given to students to make the night-on-call activity more realistic; however, despite the absence of a pager, this was one of the highestrated sessions. the course was well received with significant increases in student's perceptions of readiness and comfort in multiple domains, which were linked to acgme core competencies. facilitators volunteered their time to construct and deliver the course content, and students used their personal computers; therefore, no high-fidelity simulation manikins or costly equipment was required. although several studies on internship preparation courses have been published demonstrating increases in self-reported confidence, very few have included objective assessments such as the cumulative post-test included in our study. all students except for one achieved a score of above 60%, which would correlate with the score required to pass a national shelf examination. limitations of this remote activity include the fact it was a single-school study and that student's level of interest and engagement may have been related to not having any other residency preparation options given the covid-19 pandemic. the number of students who signed up for the course increased significantly in the seven days prior to the start of the course, which may reflect that. other limitations include those of analyzing participation in pre-and post-surveys when some of the students did not complete both for each session. lastly, it is a possibility that students who filled out their surveys were more engaged, and thus results were more likely to be positive. covid-19 has presented many challenges to students and educators in undergraduate medicine. now more than ever students need all the help they can and to feel more confident and prepared when they start internship on july 1. despite the challenges we encountered, synchronous remote delivery of this boot camp was successful, and we will likely continue to hold certain sessions remotely in the future. this may be more useful for students traveling to arrange for new living arrangements during residency. future work includes include scaling up to all 300 students and obtaining more volunteer facilitators that would enable small group breakout sessions which may increase opportunities to interact. we plan to re-survey students after their first month of internship to re-evaluate their perceived usefulness of this course. overall, an online synchronous boot camp was rated as useful by students, helped them feel more confident about common topics encountered during residency, and demonstrated an appropriate gain in knowledge using a comprehensive post-course assessment. we were able to adapt our curriculum to a remote model and will likely keep several sessions in an online format in coming years based on positive feedback from students. human subjects: all authors have confirmed that this study did not involve human participants or tissue. animal subjects: all authors have confirmed that this study did not involve animal subjects or tissue. conflicts of interest: in compliance with the icmje uniform disclosure form, all authors declare the following: payment/services info: all authors have declared that no financial support was received from any organization for the submitted work. financial relationships: all authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. other relationships: all authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work. course offerings in the fourth year of medical school: how u.s. medical schools are preparing students for internship what training is needed in the fourth year of medical school? views of residency program directors the fourth year of medical education: a literature review the impact of fragmentation of hospitalist care on length of stay medical student experiences with accessing and entering patient information in electronic health records during the obstetrics-gynecology clerkship electronic health record use in internal medicine clerkships and sub-internships for medical students graduating from 2012 to 2016 an innovative clinical skills "boot camp" for dental medicine residents a senior elective designed to prepare medical students for surgical residency senior medical student "boot camp" can result in increased self-confidence before starting surgery internships the effect and durability of a pregraduation boot camp on the confidence of senior medical student entering surgical residencies a surgical skills elective can improve student confidence prior to internship do preclinical background and clerkship experiences impact skills performance in an accelerated internship preparation course for senior medical students re-boot: simulation elective for medical students as preparation boot camp for obstetrics and gynecology residency improving the medical school-residency transition ob/gyn boot camp using high-fidelity human simulators: enhancing residents' perceived competency, confidence in taking a leadership role, and stress hardiness a brief boot camp for 4th-year medical students entering into pediatric and family medicine residencies a pediatric preintern boot camp: program development and evaluation informed by a conceptual framework a suggested emergency medicine boot camp curriculum for medical students based on the mapping of core entrustable professional activities to emergency medicine level 1 milestones a 1-week simulated internship course helps prepare medical students for transition to residency improving the transition from medical school to internship -evaluation of a preparation for internship course the key role of a transition course in preparing medical students for internship effects of postgraduate medical education "boot camps" on clinical skills, knowledge, and confidence: a meta-analysis a remotely conducted pediatric boot camp for 4th year medical students the milestones guidebook. acgme core entrustable professional activities for entering residency: toolkits for the 13 core epas education techniques for lifelong learning: writing multiple-choice questions for continuing medical education activities and self-assessment modules drs. monday and levine had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. key: cord-027057-g7onopr1 authors: george, marcus l. title: effective teaching and examination strategies for undergraduate learning during covid-19 school restrictions date: 2020-06-17 journal: nan doi: 10.1177/0047239520934017 sha: doc_id: 27057 cord_uid: g7onopr1 on friday, march 13, 2020, all school teaching in the republic of trinidad and tobago, west indies was suspended until further notice because of the novel coronavirus covid-19 pandemic. this immediately jeopardized the completion of course content at the university of the west indies, st. augustine campus. this article presents effective teaching and examination strategies that can be utilized in teaching undergraduates during covid-19 school restrictions. the introductory digital electronics course of the department of electrical and computer engineering at the university of the west indies will be utilized to demonstrate the merits of these strategies. the research will focus on demonstrating that the teaching methodologies utilized avoided the student performance from degrading below what has been experienced in the past 5 academic years. student feedback on the methodology utilized is also incorporated in this article to highlight key benefits gained by students. tertiary-level education serves to provide both strong theoretical foundation as well as the capability to solve practical issues faced in the industry. every academic year introductory courses are presented to level 1 undergraduates at the university of the west indies (uwi), st. augustine campus. one such course is the introductory digital electronics course presented to students of the department of electrical and computer engineering. all students begin this course without previous knowledge and practical capabilities of digital logic design. traditionally, this introductory digital electronics course was delivered in the classroom via contact teaching methods; all labs, exams, and quizzes were administered in a classroom setting. in 2020, during the covid-19, this was the methodology of teaching up until the thursday, march 12, 2020 after which significant restrictions were placed on school teaching, and in-class delivery of course content was suspended indefinitely. before a solution to this issue was found, it was crucial to review literature of existing teaching approaches that could have some merit in the way forward. nickels (2000) presented a strategy for the teaching digital electronics to undergraduates with the use of computer-aided design (cad) tools and hardware description languages. a hierarchical strategy was utilized, and this involved having students beginning with the study of simple digital electronic circuits after which they progressed to problems involving more complex digital electronic. nickels (2000) , however, did not conclude whether the methodology implored led to any benefit in the area of student learning or performance. the merits of this teaching methodology are that it provides a step-by-step graduation of students from less difficult digital electronics problems to more difficult ones. like digital electronics, other disciplines can benefit from the use of this teaching methodology because of the step-by-step progression of students from less difficult problems in the discipline to more difficult ones. one consideration not discussed by nickels (2000) is that of student support, and it is expected that no matter the discipline, student support will be an invaluable resource as the difficulty of problems they attempt increases. weng et al. (2009) conducted a study to determine if the use of programmable logic tool kits could assist computer science students in the learning of digital electronics. students were introduced to the use of programmable logic boards via laboratory exercises while course demonstrators monitored students' reactions to the material administered. a comprehensive survey that captured opinions of students for the use of programmable logic boards in their learning of digital electronics was conducted by weng et al. (2009) . according to weng et al. (2009) , there was no disagreement among students that the use of programmable logic boards assisted them in learning digital electronics. eighty-six percent of the students found the use of programmable logic board was a pleasant experience. weng et al. (2009) also indicated that there was an improvement in student performance when programmable logic boards were used when compared with previous years when programmable logic boards were not used. this methodology provides a practical approach to teaching, and it presents opportunities for other disciplines. programmable logic tool kits were utilized for digital electronics. for other disciplines such as mathematics, physics, biology, or even finance, tool kits appropriate for practical teaching of those subject areas can be incorporated into the curriculum to enhance student learning. for online learning, students must have access to these resources at home. zhao and okamoto (2009) presented an adaptive framework aimed at engaging students in collaborative discussion of course material in ubiquitous environments. the methodology promoted student use of mobile technology to conduct email-based discussion of course material at any time during the teaching period. zhao and okamoto (2009) claimed that the proposed adaptive framework improved the learning experience of the students and increased student performance in the area of study. this methodology presents opportunities for lecturing of most disciplines, even during pandemics. use of mobile technology by college students is more popular today than in the past; hence, there is merit in the use of ubiquitous environments to support teaching. joseph et al. (2013) presented the utilization of case method and role-play in teaching the topic of finite state machines (fsms) to undergraduates. joseph et al. (2013) indicated that the case method is normally used as a very important pedagogical tool in academia, and its use is intended for the enhancement and development of the general conclusions of the research being done. joseph et al. (2013) indicated that role-play is less technologically elaborate and is utilized for the learning interpersonal skills. joseph et al. (2013) claimed students displayed greater interest in the use of the case method and role-play techniques for learning of course topics and that there was an increase in the level of collaboration and active participation by students in the learning process. finally, joseph et al. (2013) claimed that student performance was better when case method and roleplay techniques were utilized compared with when they were not utilized. this methodology can benefit the teaching of most disciplines simply because it increases interaction between students in the learning process without a demand on the use of discipline-specific resources. this is especially beneficial to courses containing group projects. prasad et al. (2014) conducted a study of the impact of software simulators in the teaching of digital logic design. the tools logic gate simulator, digisim, tinycad, and logisim were utilized in this study. prasad et al. (2014) claimed that the use of logic simulators such as logic gate simulator, digisim, tinycad, and logisim resulted in the number of students scoring above 45% in the course increasing from 82% in the previous year to 96% in the present year. this methodology provides opportunities for expanding on the teaching of topics in any discipline such as engineering, mathematics, finance, or even geography, as long as the appropriate simulators are made available. the use of simulators can also support self-study by students at home as long as the software is stand-alone. montãnanaa et al. (2015) discussed the use of participative learning in teaching of very high speed integrated circuit hardware description language (vhdl). this methodology gave the students the freedom to think and explore and hence allowing them the opportunity to discover better methods of learning the topic. at the end of the study, students indicated that the appropriate planning, distribution, and clear definition of the specifications of project tasks can eliminate the possibility of conflicts in the design and implementation stages of projects. montãnanaa et al. (2015) also indicated that students appreciated the need for teamwork in managing large projects when this methodology was utilized. the strength of the methodology presented by montãnanaa et al. (2015) is the freedom to participate in learning without restrictions. students are introduced to the topics, given objectives, and allowed the freedom to explore the most appropriate methods of learning the topics. this can benefit the teaching of most disciplines especially because the method utilizes no discipline-specific resources. roy et al. (2015) presented the use of web-based virtual laboratory called coldvl in the teaching of the topic of computer organization and logic design. coldvl contains a hierarchical module-level logic tool that contains a logic simulator component and also a large number of technical features. the system also consists a graphical user interface that can be used in the construction and simulation of logic circuits. roy et al. (2015) was unable to indicate whether the use of coldvl enhanced student learning of the topic. like prasad et al. (2014) , this methodology provides opportunities for expanding on the teaching of topics in any discipline as long as the appropriate software is made available to students. this methodology will promote self-study by students, and consultation opportunities must be available to guarantee student progress. george (2018) presented the exploration of a classroom-based methodology for teaching of digital logic to engineering undergraduates at the uwi, st. augustine campus. in the classroom-based methodology, all lab work was conducted inside the classroom rather than the laboratory. to assess the merits of students under the new approach, the performance of students when this classroom-based teaching methodology was utilized was compared with that of the previous lab-based teaching methodology. george (2018) indicated that students' performance was better in all quizzes and design project when this classroom-based teaching methodology was utilized when compared with the previous teaching methodology. students also endorsed the use of this classroom-based methodology in teaching digital logic design. the merit of this teaching methodology is that all laboratory work was conducted in a classroom setting and not a laboratory, hence allowing the lecturer the opportunity to incorporate practical work while presenting course content. if a course is to be conducted online, this teaching methodology can be utilized as long as a video conference platform is utilized and students themselves have access to development resources such as the field programmable gate array (fpga) tool kits.the methodology of george (2018) however only benefits courses with practical components and as such other courses such as english language and history which normally would not require the use of a laboratory may not receive any additional benefits from this methodology. george (2020) presented a study on the effect of using three consultation types-office, email, tutorial-in the teaching of fsms to electrical and computer engineering undergraduates at the uwi, st. augustine campus. george (2020) made the conclusion that students who participated in consultation activities offered by the course lecturer performed better in fsm-related quizzes and projects than students who did not participate in such in consultation activities. george (2020) also concluded failures in the quizzes occurred only in the population of students who attended class lectures alone but never participated in consultation exercises. this methodology provides many opportunities for lecturers of any discipline to provide different levels of support to meet the varying needs of students. in a pandemic, however, the tutorial-based and office-based consultations will only be possible via video conferencing. the email-based consultations will not be affected regardless of the existence of pandemic and regardless of the discipline being taught. with the consideration of the fact that covid-19 school restrictions were in place in trinidad and tobago for the remainder of second semester year 2019/ 2020, it was important to review the traditional approach to delivering the introductory digital electronics course and determine what was possible and not possible under the circumstances. then, the merits and drawbacks of the literature reviewed earlier can be used as a guide to developing a modified strategy to completing the delivery and assessment of the course despite covid-19 teaching restrictions. student performance using these strategies will be evaluated to determine if they were effective in avoiding degradation below what has been experienced in the past five academic years, despite the covid-19 school restrictions. it is expected that the study will provide a basis in which other disciplines can benefit from for teaching of undergraduates during covid-19 school restrictions. the introductory digital electronics course provided students with a firm foundation in the concepts of digital logic analysis and design. the course covers topics such as number systems, boolean algebra, minimization using karnaugh maps, combinational logic circuits, and integrated circuit technology. although this is an introductory course in digital logic analysis and design, the course also served to expose students to practical tools and devices used in the development of digital circuits such as the use of vhdl and xilinx ise. on completion of the course, students must be capable of constructing, analyzing, verifying, and troubleshooting digital circuits using appropriate techniques and test equipment. the previous teaching methodology for the undergraduate introductory digital electronics (ecng1014) was utilized over a 12-year period from academic year 2007/2008 to year 2019/2020. because this was an introductory course in digital electronics, students always entered the course with little or no experience in digital logic design. the learning outcomes of the course are given in table 1 . the course was intended for delivery by 3 contact hours of lectures per week for 12 weeks (total of 36 contact hours). the course was assessed via two quizzes, one midterm exam and one final exam as shown in table 2 . the 36 hours of contact with students included 27 hours of class lectures, 5.5 hours allocated to delivering lab #3 using of classroom-based lab delivery using the approach entailed in (george, 2018) , one 90-minute slot allocated to midterm exam, and two 60-minute slots allocated to quiz #1 and quiz #2. students were required to attempt labs #1 and #2 by themselves using a detailed laboratory manual, a nexys3 tool kit available from the laboratory facility and digilent (2016). students were expected to progress in these two labs without issue, but if any issues were faced, the course lecturer was available to assist using the approach of george (2020). the labs 1 to 3 did not contribute to the course mark but instead served the purpose of informing and preparing students for the quizzes that carried marks, and most important, these labs contributed to the students' practical awareness of topics in the course. the class enrollment for academic year 2019/2020 was 76 students. the introductory digital electronics course presented in year 2019/2020 consisted of the same learning outcomes of previous offering in years 2015/2016 to 2018/2019 (see table 1 ). unlike previous years, the offering of 2019/2020 consisted of two different delivery phases (table 3) . at the beginning of the semester, the course lecturer made available for purchase of a concise workbook specifically designed for the course (george, 2019) . it was expected that this textbook serve as an assistant teacher for students of the course, allowing the opportunity to obtain detailed solutions and explanations of digital electronic problems. of the 76 students, 67 enrolled in the course secured copies of the textbook. this will be expanded on later on in this section of the article. also at the beginning of the semester, the traditional teaching strategy was executed including an investment in effective in-class teaching, labs conducted via the method presented in george (2018) , and a variety of consultation activities as presented in george (2020) . however, on friday, march 13, 2020, when covid-19 teaching restrictions were imposed in trinidad and tobago, in-class teaching was stopped, and the modified teaching strategy was enforced. the following are the elements of the modified teaching strategy: a. use of myelearning as the online teaching platform the myelearning online platform was utilized over the years as an avenue for students to access all course materials including lectures, lab manuals, and so forth. the advent of covid-19 teaching restrictions made myelearning a crucial resource for continued support of students. even examination activities were conducted on this platform after the covid-19 restrictions. more information on these items will be presented in the upcoming sections. a very useful resource in times of teaching restrictions is that of a very detailed workbook-type text. many textbooks do not adequately meet the needs of an introductory digital electronics course. as such, the lecturer of the course has a responsibility to provide a concise text that meets the needs of his/her course. george (2019) has written a text to comprehensively cover the contents of the introductory digital electronics courses for most electrical and computer engineering undergraduate degrees. the text presented material for all topics of the course with the aid of step-by-step solution of problems and diagrams. practice questions along with answers to them were presented on conclusion of each chapter. as indicated previously, this textbook served as an assistant teacher for students. during covid-19 teaching restrictions, the course lecturer was able to easily refer students to aspects of george (2019) for detailed explanation on problems encountered. the digital electronics visual tutor ( figure 1 ) was a fantastic learning resource provided to students after the covid-19 restrictions were placed. this visual tutor was the result of a final year project supervised by the course lecturer and was developed for this introductory digital electronics course. the course lecturer did not intend to make use of this resource for teaching the cohort of 2019/ 2020, neither was the resource utilized in the teaching of the last eight cohorts of level 1 students in the department. because of the covid-19 restrictions, the course lecturer made this resource available for download from the myelearning course page. this visual tutor provided an interactive learning experience of all topics of the introductory digital electronics course for all students who utilized it. this was especially useful in students learning the last two topics of the course that were not delivered before the covid-19 restrictions: topic #5-introduction to vhdl and topic #6-integrated circuit technology. like the digital electronics visual tutor of (c), this port-mapping tool was a result of a final year project supervised by the course lecturer and has been identified in george and bissoon (2019)-image seen in figure 2 . students of the introductory digital electronics course were supposed to be introduced to vhdl, of which port mapping (component instantiation) was the most important aspect of the material to be presented. to further support student practical learning of the topic of port mapping, this port-mapping tool was made available for student download from the myelearning course page. with this resource, students could have taken any datapath block diagram and use the step-by-step teaching feature to arrive at the complete vhdl code for the port mapping of any system. immediately after the covid-19 school restrictions were enforced, the majority of the population of lecturers who continued teaching moved teaching online and attempted to use the obvious online methods to deliver course material to students. some invested time in creating online videos for upload, while some invested their time in the use of software such as zoom and blackboard collaborate. although these approaches provided a convenient alternative to the in-class lectures, they introduced several drawbacks not experienced in the in-class lecture approach as indicated in kebritchi et al. (2017) and nbc news(2020). because of the variety of support resources offered by the lecturer of this introductory digital electronics course, a decision was made to locate the best available youtube videos to support students learning of the last two topics that were not completed prior to the covid-19 restrictions: topic #5-introduction to vhdl and topic #6-integrated circuit technology. the url of these videos were posted on the myelearning course page for students to click and view. this activity avoided the course lecturer wasting crucial time in creating videos of holding online class sessions which may be plagued with issues of online teaching indicated in kebritchi et al. (2017) and nbc news (2020). the lecturer was hence able to maximize time invested in email-based consultations with students. as indicated in george (2020) , this consultation type involved the course lecturer presenting students with a tutorial or supplementary with topics associated with the course topics. students were required to attempt questions on their own time in the comfort of their home, scan their attempts, and email them to the course lecturer for review and correction. the course lecturer then presented the students with a summary of corrections to their attempts via an email reply. students who had started working on questions but unable to finish were also required to scan their attempts and email the lecturer for his review. the lecturer would then email the students on mistakes they made and the required corrections, after which they were given another opportunity to attempt the questions again and return for a second email consultation with the lecturer. students were presented with supplementary worksheets at the beginning of the semester; however, the need for this was not apparent until there were no longer any in-class lectures because of covid-19 teaching restrictions. when there were no restrictions, students could have completed the supplementary sheet for each topic, meet with course lecturer, and have office consultation as indicated in george (2020) . however, because face-to-face meetings were no longer possible, students were able to use email-based consultation (george, 2020) as indicated in (f) described earlier. students were required in this case to attempt supplementary sheets, scan their solutions, and submit via email for the lecturer's review. the lecturer then reviewed the solutions and then set up an email-based consultation with the students to discuss any mistake made and entertain any questions. online mock quizzes were issued to students to provide students with an opportunity to attempt structured exam-type questions for the topics of the course and to give students experience in attempting exams online in the event that an online approach was to be utilized for examination of students for the remainder of the course semester. as a matter of fact, it was the intention of the course lecturer to host all remaining exams online because it was expected that the school restrictions may have remained in place for the rest of the semester, and hence in-class exams would not have been possible. these quizzes were arranged as structured essay-type questions on the myelearning course page, and students were required to respond to questions by placing solutions in fields provided. if students were required to provide illustrations, they were allowed to use computer programmes such as microsoft paint, microsoft visio, and then upload images to fields provided in the exam. students were allowed multiple attempts of the quizzes, and when completed, they were manually graded. a very important part of this resource is that the quizzes allowed for the lecturer to insert comments on the student attempts so students can learn from mistakes made (see figure 3 ). the provision of lecturer comments (feedback) was expected to add value to the student learning experience during the mock exam. to ensure that all students were allowed the opportunity to learn from each other's mistakes and lecturer feedback, what is called a mock quiz feedback document was developed for each mock quiz provided. at least 10 students' responses for each question administered in the quizzes were anonymously compiled into a mock quiz feedback document that most times exceeded 30 pages. these documents were uploaded to the myelearning course page and made available to all students. these documents had the following for each question: students were required to take advantage of the use of all these learning resources (a) to (i) which were made available by the course lecturer. it was expected that the combined use of these resources will benefit students more than just a simple movement of class to online lectures using zoom or blackboard collaborate. it is expected that the abundance of resources would increase the students' chances of obtaining high marks in the course. at the time in which covid-19 school restrictions were enforced, only the midterm examination for the introductory digital electronics course was administered. the quizzes and final exam were still to be administered. because of the covid-19 restrictions and the uncertainty of resumption of classroom-based teaching at the university, a decision was made by the course lecturer to administer these quizzes and exams online using quizzes on the myelearning course page. because the midterm exam was unaffected by the covid-19 restrictions, it was not necessary to include the results of the midterm in this article because the article focusses on those aspects affected by the covid-19 teaching restrictions. just for the purpose of completeness, it is to be noted that the midterm exam covered the first two learning outcomes based on number systems (lo #1) and boolean algebra (lo #2). the first assessment affected by the covid-19 teaching restriction was quiz #1 which was based on minimization with karnaugh maps (lo #3) and combinational logic circuits (lo #4) and which is normally administered as a 60-minute open-book multiple-choice quiz containing 20 questions. the multiple-choice questions were not simple as they normally required students to utilize the combinational logic procedure to arrive at the correct answers. students cannot simply guess the correct answer. to ensure that this quiz could have been accommodated after covid-19 teaching restrictions, the course lecturer moved the quiz from the classroom to the myelearning course page. a total of 100 different multiple-choice questions were created and placed in a question bank. on the day of the quiz, 20 of these questions were randomly selected and assigned by the system to each student. students were given 60 minutes to attempt all 20 questions. the fact that students were assigned 20 randomly selected questions and each student had a different exam of similar difficulty minimized the possibility of collusion between students. to further minimize the possibility of collusion, the available answers for each question were shuffled. the questions in the quiz were also shuffled. to ensure that issues related to availability of internet and reliability of internet source did not affect students' progress in the quiz the quiz was run for 24 hours, students were allowed 60 minutes to attempt the quiz. after the quiz period ended, all quiz attempts were automatically graded by the myelearning facility, and students were issued their grades. figure 4 shows one of the multiple-choice questions administered to students in this quiz. the second assessment affected by the covid-19 teaching restriction was quiz #2 which was based on an introduction to vhdl (lo #7) and which is normally administered as a 60-minute structured essay-type open-book quiz administered in the classroom and containing 4 questions covering the areas: to ensure that this quiz could have been accommodated after covid-19 teaching restrictions, the course lecturer moved the quiz from the classroom to the myelearning course page. this quiz was conducted similar to the online mock quizzes discussed in the previous section of this article. twenty different structured essay-type questions (5 for each area of study) were created and placed in a question bank. on the day of the quiz, four of these questions were randomly selected (one for each area of study) and assigned to each student. students were given 60 minutes to attempt all four questions. the fact that students were assigned four randomly selected questions served to minimizing the possibility of collusion between students. to further minimize the possibility of collusion, the questions in the quiz were also shuffled. to ensure that issues related to availability of internet and reliability of internet source did not affect students' progress in the quiz, the quiz was run for 24 hours, and students were allowed 60 minutes to attempt the quiz. because the questions for this quiz were structured essay-type questions, the quiz attempts unfortunately could only be manually graded on the myelearning platform. figures 5 and 6 show some of the structured essaytype questions administered to students in this quiz. the final exam was the last assessment affected by the covid-19 teaching restriction and was normally administered as a 3-hour written examination with full supervision by paid invigilators and containing structured essay-type questions. the following are the topics that are normally examined: at the time of production of this article, the university had not as yet made a decision on how the final exam would be administered for all courses. the progress of this article could not be delayed indefinitely for the outcome of such decision so the lecturer decided to host a mock final exam of similar difficulty of the traditional final exams for the course, and it was mandatory all students attempt the quiz as a means of preparing for the final exam. the mock final exam was hosted on the myelearning course page. this quiz was conducted similar to the online mock quizzes discussed in the previous section of this article. the final exam normally contained four structured questions essay-type questions containing several subquestions. because the mock final exam was to be conducted online and to maximize student readability of exam questions while at the same time minimizing the complication of marking such exam responses, the online questions had to be arranged as stand-alone essay-type questions with no subquestions. to minimize the possibility of students colluding and also most important to allow the online exam to be conducted as similar as possible to the traditional final exam, the students had to be administered the online exam at the same time. additional guidelines applied to the administration of this examination were as follows: • all students attempt exam same time, and this time line must be announced at least 2 weeks in advance. • shuffle questions to minimize possibility of collusion by students. • students must have no opportunity to return to previously attempted questions, hence minimizing possibility of collusion by students. • fair time budget in light of online delivery, but not excessive. • a backup exam should also be prepared in the event students could not make it to the first on because of unforeseen circumstances. students were required to attempt a mock final exam quiz containing seven stand-alone questions with no subquestions, and they were required to attempt this entire quiz in 2 hours, which was 1 hour less than previous years. only one attempt was allowed. because the questions for this quiz were essay-type questions, the quiz attempts unfortunately could only be manually graded on the myelearning platform. figures 7 and 8 show some of the structured essay-type questions administered to students in this mock final exam quiz. tables 4 to 6 present the student performance for quiz #1, quiz #2, and the final exam over a 5-year period. it is important to note that the results of the mock final exam of academic year 2019/2020 were used in place of that of the eventual final exam because the university had not made a decision on how the final exams would have been conducted, and it was the author's belief that this mock final exam gave an excellent indication of how a final exam for the course would be had it been conducted online using the strategies outlined in this article. according to the data given in tables 4 to 6, it was realized that student performance in quiz #1 and quiz #2 were consistent with (and in some instances better than) that of their traditional counterparts of the previous academic years, despite the presence of covid-19 teaching restriction in academic year 2019/2020 resulting in alternative teaching strategies having to be applied to prepare students for these examinations. in the case if the final exam, table 6 indicated that student performance in the mock final exam of year 2019/2020 was better than that of written final exams of the previous four academic years. this may have been because the mock final exam was an online exam that students were attempting without supervision, and hence students had access to reading materials to assist attempt of the exam questions. on the other end, there is no evidence to suggest that students would have performed worst if there was no possibility of accessing reading material during the online quiz. after 1 week of submission of the original version of this article, the uwi had made a decision on how the final exam would have been administered to students; however, the exam was not to be administered until late june 2020. traditionally, the final exam was administered as a 3-hour written examination with full supervision by paid invigilators and containing structured essay-type questions. as a result of the covid-19 school restrictions, the university made a decision to administer the final exams as take-home final exams without supervision, where students would be allowed 48 hours to prepare handwritten or typed solutions to the questions of the exam paper, scan the solutions, and upload them on in appropriate sections of the myelearning online platform. the student responses would then be marked by the course examiner. the author of this article however believed that the use of the mock final exam discussed earlier provided a more appropriate avenue for verifying the effectiveness of the modified teaching methodology presented in this article because it better emulated the traditional method of administering the final exam, and students were least likely to benefit from collaboration with other classmates, social media, or even internet search engines for the duration of the mock final exam. in each academic year 2015/2016 to 2019/2020, students were required to anonymously complete course feedback questionnaires. in the academic year 2019/ 2020, students were fortunate to experience both traditional and modified teaching approaches so an additional questionnaire was provided to them so they can contrast both approaches. according to the feedback questionnaires, students in all five academic years indicated that the traditional teaching strategies benefitted them enormously in understanding of course material, and they always welcomed the opportunity to have face-to-face correspondence with the lecturer without use of devices. to have an idea of students' opinions of the modified teaching methodology that was issued with consideration of covid-19 school restrictions in trinidad and tobago, the students of academic year 2019/2020 were asked to contrast their experience under this new (modified) methodology in comparison with the traditional teaching methodology. students indicated that the new strategies allowed students to do self-study under the guidance of the course lecturer and that this enabled them a level of convenience not allowed under the traditional teaching methodology. students also praised the abundance of learning resources under the new teaching methodology including visual tutors and the new course textbook prepared by the lecturer specifically for the course. students also praised the use of mock quizzes in the new methodology that allowed them the opportunity to trial run the examination of material taught under online-exam conditions. the immediate feedback given for mistakes made were very valuable for their learning experience. students finally indicated that the support received via the new teaching methodology eliminated their fears of failing the course because of the interruption of teaching by the covid-19 pandemic. at the end of the review, students rated the digital logic theory for engineers classic workbook, digital electronics visual tutor, port-mapping tool for digital logic design, and mock quiz feedback documents as the most helpful elements of the new/modified teaching methodology. this article presented effective teaching and examination strategies that can be utilized for undergraduate learning of courses during covid-19 school restrictions. to demonstrate the use of these strategies the teaching and examination of the introductory digital electronics course of the department of electrical and computer engineering, uwi, st. augustine campus was utilized. the article also served to demonstrate that the application of such teaching methodologies to the introductory digital electronics course avoided the student performance from degrading below what has been experienced in the past five academic years, despite the presence of covid-19 school restrictions. student performance in quiz #1, quiz #2, and the final examination quiz were consistent with that of their traditional counterparts of the previous academic years, despite the presence of covid-19 teaching restrictions, resulting in alternative teaching strategies having to be applied to prepare students for these examinations. students also endorsed the use of the elements of the new teaching methodology utilized. the success of the teaching and examination strategies of this article bring to light the possibility of moving the entire introductory digital electronics course online and facilitating a distance learning version of the course for a large market. the research serves to indicate that there is great merit in the use of online resources to support teaching of the introductory course in digital electronics at the university. feedback from students indicated that students may have collaborated in the learning of topics. although the results of students under the teaching strategies outlined in this article appear to be consistent with that of previous years under the traditional techniques, there however is no evidence to indicate that students did not collaborate at all while attempting online quizzes. although the mock final exam was conducted under stricter conditions, there is still the possibility that students could have collaborated. future work should involve expanding the study to conduct the quizzes and mock final exam using online exam proctoring methods where the candidate is monitored during the exam, hence allowing the validation (without opportunity for doubt) of the effectiveness of the teaching strategies identified in this article in student learning of the material. one must not discount the importance of adequate internet access for the online learning. it is important to note that the teaching and examination strategies utilized during covid-19 teaching restrictions faced no interruptions. both lecturer and students had adequate access to internet resources and access to the myelearning course page. however, the methodology is not immune to interruptions. inadequate access to internet by lecturer or students could adversely affect the use of these strategies. the cohort of academic year 2019/2020 benefitted from both traditional and new teaching strategies during the undertaking of the introductory digital electronics course. future works should include administering each method to independent groups of students and assessing them both via a monitored quizzes and final exams to determine which methodology resulted in better student performance. the research of this article served to indicate that the new teaching strategy does not result in the degradation of student performance below what normally is obtained. however, it will be good to know which methodology results in the better student performance and that experiment requires more control of variables. several lessons were learned from this experience of teaching and examination during covid-19 teaching restrictions. the first lesson learned is the importance of presenting students with a variety of learning resources to facilitate their study of the subject area. students learn the same subject area in a variety of ways and some methods may facilitate learning more than others, depending on the individual. in covid-19 restrictions, students could no longer benefit from in-class lectures or face-to-face consultations with lecturers, so switching to synchronous methods of lecturing was not guaranteed to offer an effective alternative. as such, providing an abundance of learning resources and allowing students freedom of selecting the ones that best facilitate their learning was a viable decision. the second lesson learned from this experience is that students appreciate the availability of visual tutors for their learning of the course material. students' use of all learning resources were tracked via the feedback questionnaire and according to this the most preferred learning resources were the digital electronics visual tutor and the port-mapping tool for digital logic design. the popular reason given for this preference was because of the ability to interact with these resources and obtain feedback for queries made. lecturers from all disciplines should make an effort to incorporate visual tutors in the teaching strategy because students appear to rely on these resources outside of the classroom. the third lesson learned is the importance of consultations with the course lecturer to student education. whether or not there is a pandemic, students need the opportunity to approach the course lecturer for consultations, and according to george (2020) , email-based consultations are the most utilized type of consultation when compared with tutorial (in class) and office-based consultation types. what was recognized during this study is students mainly used emailbased consultations for clarification on matters after other avenues had been exhausted. if students found the answer using other resources, they were least likely to use email consultations, but for students who could not succeed with other resources, they eventually requested assistance via email. the fourth lesson learned was the importance of a concise and reliable workbook for the course delivered. a workbook that concisely presents students with the most appropriate approach to attempting questions is recommended for students while outside of the classroom. some students indicated that the textbooks utilized for other courses many times presented ambiguous explanations to questions; however, the workbooks such as george (2019) left little room for confusion. based on this study, it can be recommended that even if a complete textbook exist for a discipline, the lecturer should still make an effort to prepare a concise workbook that serves to expand on the methodology to attempting questions that may be asked at the examination level. these books should be very clear on linking the theoretical and practical material with the decisions taken in progressing with the questions to be attempted. the fifth lesson learned was the benefits of mock exams to both lecturers and students. mocks exams utilized in this study served to both promote student learning of the topics being examined as well as get students acclimatized with the attempting of exams on the online platform. mock exams give lecturers the opportunity to discover the drawbacks of examining students online. some drawbacks realized were the difficulties of marking large volumes of student responses online compared with on paper. as such, lecturers must be more innovative in administering exams online to maximize the convenience in the marking of student response. if this introductory digital electronics course was to be taught again online for an entire semester, the first addition would be the upgrading of the existing lecture notes with audio so that students can be guided through the notes with the voice of the lecturer. this will replace the use of online youtube videos as presented in this study, hence eliminating the reliance on materials not developed by the lecturer. the second addition to the course must be the administering of the final exam online similar to how the mock final exam of this study was conducted. it is unknown at this time how the covid-19 pandemic will eventually impact college education; however, the teaching and examination strategies of this study are feasible and can be adopted for the lecturing of all disciplines even if not in a pandemic. nexys-3 toolkit reference manual an effective classroom-based approach for teaching digital logic design to engineering undergraduates digital logic theory for engineers -a classic workbook study on the effect of tutorial, email and office consultations on undergraduate performance in the topic of finite state machines port mapping tool for digital logic development. i-manager's teaching finite state machines with case method and role play issues and challenges for teaching successful online courses in higher education: a literature review teaching strategy on vhdl course based on participative learning students at 25 colleges sue for refunds after campuses close because of coronavirus pros and cons of replacing discrete logic with programmable logic in introductory digital logic courses incorporating simulation tools in the teaching of digital logic design december 10-12). a virtual laboratory package to support teaching of logic design and computer organization enhancing learning effectiveness in digital design course through the use of programmable logic boards an email-based discussion learning system with ubiquitous device support author biography he is the author of several academic books. his research interest includes the business administration, strategic planning and management, engineering education, formal specification, modeling and verification the author would like to thank the students of the introductory digital electronics course of the department of electrical and computer engineering at the university of the west indies, st. augustine for their excellent cooperation in this research work. the authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. the authors received no financial support for the research, authorship, and/or publication of this article. marcus l. george https://orcid.org/0000-0002-7038-2089 key: cord-332573-pltz5viz authors: lieberman, joshua a; nester, theresa; emrich, brooke; staley, elizabeth m; bourassa, lori a; tsang, hamilton c title: coping with covid-19: emerging medical student clinical pathology education in the pacific northwest in the face of a global pandemic date: 2020-09-02 journal: am j clin pathol doi: 10.1093/ajcp/aqaa152 sha: doc_id: 332573 cord_uid: pltz5viz objectives: the first coronavirus disease 2019 (covid-19) case in the united states was reported in washington state. the pandemic caused drastic disruptions to medical institutions, including medical education. the department of laboratory medicine at the university of washington responded by rapidly implementing substantial changes to medical student clerkships. methods: in real time, we converted one ongoing caseand didactic-based course, labm 685, to remote learning. results: fifteen of 17 scheduled sessions proceeded as planned, including two sessions for student presentations. two didactics were canceled as the functions of the teleconferencing platform were not sufficient to proceed. one grand rounds speaker canceled due to covid-19 precautions. elements of an immersive clinical laboratory clerkship, labm 680, were repurposed to accommodate 40 medical students per class via remote learning, highlighting clinical laboratory activities that continue throughout the outbreak. a new remote clerkship, medsci 585c, was developed incorporating distance learning and guided small-group sessions. this coincided with parallel efforts to make resident and fellow service work, conferences, and didactics available remotely to comply with social distancing. conclusions: the changes in medical education described reflect the dynamic interplay of current events affecting the world of clinical pathology. throughout this, technology—while with some limitations—has provided the platform for innovative learning. on maintaining optimal patient care while protecting the well-being of the workforce and trainees. 6 in march 2020, as cases of covid-19 increased throughout the state, the som halted all clinical clerkships and advised students to return to their home states. a parallel gubernatorial proclamation required all classes and meetings at the university to be held remotely, prohibiting in-person classroom instruction. 7 simultaneously, the us federal government declared the pandemic a national emergency. 8 these developments required the rapid implementation of significant changes to medical education in clinical pathology at our institution. the department of laboratory medicine routinely offers two 2-week clerkships for medical students: laboratory medicine 685: "laboratory case studies for clinical diagnosis" (labm 685) and laboratory medicine 680: "clinical laboratory testing-methods and interpretation" (labm 680). labm 685 is a casebased course offered once per academic year (ay) using lectures and guided discussion to teach senior medical students the efficient selection and rational interpretation of laboratory tests. labm 685 pedagogy relies on classroom discussions, laboratory shadowing, and student presentations. enrollment is an average of 11 students per ay. two days prior to the start of labm 685, the viral genome analysis from two regional patients diagnosed with covid-19 suggested 6 weeks of cryptic transmission in the local community, a finding broadcast through social media 9 and that set the tone for the session. labm 680 is a 2-week, modular clinical laboratory experience offered throughout the ay in which one to three students participate at an introductory level in division-specific and interdisciplinary conferences, didactics, and a weekly review of calls handled by the laboratory medicine residents ("call rounds") and tour clinical laboratories at the university-based medical center and the general trauma hospital. division-specific activities include sign-out of coagulation and hemoglobinopathy testing, identification of alloantibodies, review of transfusion reactions, and other clinical consultation encountered in resident service work. in addition, three interdisciplinary conferences occur weekly between clinical microbiology faculty and infectious diseases clinical services ("plate rounds"). both courses are graded with a pass or fail format. the clerkships are not available simultaneously because each clerkship draws from the same pool of instructors. medical student instruction in laboratory medicine is distinct from other specialties in part because while critical patient care activities are performed, relatively little direct patient contact takes place. this feature greatly facilitates the transition to remote learning. however, being provider-facing means clinical laboratory activities are often less recognizable to nonpathologists as patient care. nonpathologists are often unaware of the regulatory environment in which clinical laboratories operate, a topic that has been a source of increased discussion recently in the context of developing diagnostic testing for sars-cov-2. 10 these features make medical education in laboratory medicine a valuable resource, especially during the covid-19 pandemic, which serves as a unique realtime case study highlighting multiple central aspects of laboratory medicine for learners at all levels. sars-cov-2 remained a key topic of discussion throughout the course. specific questions from or interests expressed by students prompted additional discussions and supplemental reading/journal articles related to both the new pathogen and vaccine development in other outbreak situations (ie, ebola, severe acute respiratory syndrome virus, zika virus). impromptu and extemporaneous discussions or modifications were made to existing scheduled lectures to incorporate sars-cov-2 biology, epidemiology, and laboratory responses. the rapidly changing som policies for medical students and employee health guidelines were also discussed with students. the zoom teleconferencing platform (zoom video communications) was used to conduct all remote learning. the professional version of the teleconferencing application was made available for university students, faculty, and staff on march 4, 2020, including a health insurance portability and accountability act (hipaa)compliant version for protected health information (phi). support for the application was available through administrative staff and course directors. the new online-only course, medical science 585c: "clinical laboratory testing distance learning-methods and interpretation" (medsci 585c), relied on the cloud-based canvas learning management system (canvas gfx) for distribution of course materials and submission of assignments. the zoom "breakout room" feature was used for small-group discussions in medsci 585c. changes to course structures made due to covid-19 are summarized in ❚table 1❚. briefly, the second week of labm 685 was conducted online. scheduled sessions of labm 680 (march to june 2020) were canceled by the uw som along with all in-person medical student rotations. labm 680 was replaced with medsci 585c, which provided most regularly scheduled labm 680 curricula in an online format. in addition, we identified previously recorded and upcoming lectures that could be recorded, as well as the clinical services with remote-access capabilities for trainees, and created supplemental activities not typically part of labm 680, such as guided discussions, self-study resources, and reflective writing assignments. after assessing capacity, we planned to accommodate up to 40 students. to maintain low trainee/faculty ratios, we also planned twice-weekly small-group discussions facilitated by at least one faculty plus a second faculty or senior fellow. no changes were made to an additional clerkship with rare enrollment, labm 600, which covered individualized independent study or research projects with a faculty mentor. no students were enrolled at the time, and this course likely would have been canceled under som policy. no changes were made to previously planned labm 685 feedback collection through an anonymous survey and discussion between course directors and students. for medsci 585c, student feedback was gathered through (1) a final wrap-up session with the whole class, (2) an assigned reflection piece, and (3) a course survey. the final wrap-up session invited students to provide verbal feedback and closing remarks. the reflection piece, due at the end of the course, was a one-to two-page reply to the following prompt: "describe three key aspects of lab medicine you've learned and how they apply to the covid-19 pandemic." the anonymous course survey contained quantitative and freetext elements for students to give feedback on small groups, lectures, and the overall course (supplemental table 1 ; all supplemental materials can be found at american journal of clinical pathology online). survey questions were designed to assess aspects of the course such as interactivity, clinical casework integration, achievement of course objectives, and overall impression. students graded the course on a scale of 1 to 5, with 5 indicating the strongest level of agreement or most positive assessment. when the centers for disease control and prevention began issuing guidance to curb the spread of covid-19, 11 students were enrolled in course-based learning via labm 685 ❚table 2❚. adjustments to labm 685 (table 1) were implemented in both week 1 and week 2, with significant structural changes in week 2 ❚table 3❚. the most significant alteration implemented in week 1 was the real-time incorporation of the covid-19 pandemic as an overarching theme or case study to highlight the role of laboratory medicine. course directors led either impromptu or rapidly prepared discussions focused on multiple aspects of the growing pandemic. these covered a diverse array of topics, including clinical features of covid-19, understanding outbreak dynamics through representative graphs, specific infection control precautions in clinical laboratories, molecular epidemiology from viral genome sequence data, and sharing emerging data and reports. other discussions covered sars-cov-2 test development and regulatory requirements for clinical laboratories. these discussions were facilitated by the start of clinical testing for sars-cov-2 by the clinical virology laboratory 11 on the first day of labm 685. one clinicalhematopathology conference was canceled. during week 1, the faculty and administrative staff responsible for the course anticipated and planned to convert to remote-only instruction using the university teleconferencing platform. in-person instruction was suspended at the midpoint of the course in concordance with university official policy. 12 week 2 of labm 685 was conducted entirely online. two of the scheduled presentations were canceled as they were formatted to be given as "chalk talks." the "whiteboard" feature of the teleconferencing platform did not meet the needs of the two presenters, and the lectures could not be rewritten as slide-based presentations in the time available. in addition, one grand rounds presentation was canceled by the invited speaker, an expert in tropical medicine, who declined to travel given the risks of sars-cov-2 transmission. all other pedagogic activities proceeded, including the student presentations, which were conducted in two sessions at the end of the course. the success of these changes informed modifications to the upcoming labm 680 course. before the next section of labm 680 began, the som canceled all upcoming clerkships to allow a window of opportunity for the development and implementation of alternate educational plans. the course directors and clerkship administrator for labm 680 collaborated to develop a new course, medsci 585c, based on labm 680 activities but conducted as remote learning. many activities in which labm 680 students participated were concurrently being transitioned online to allow residents and fellows to work remotely. these included clinical microbiology-infectious diseases plate rounds, resident didactics, call rounds, and coagulation/hemoglobinopathy testing sign-out. the course directors effectively leveraged these parallel changes in clinical activity and resident education to provide medical students remote access to these activities, which had been components of labm 680 ❚table 4❚. the online format allowed a greater capacity of up to 40 students with the first course scheduled to begin late april. small-group sessions and flipped classroom teaching sessions were incorporated. the new course, medsci 585c, provided third-and fourth-year medical students an opportunity to develop their diagnostic and patient management skills through directed distance learning in the hospital-based clinical laboratory. fourteen students completed each session of the new course, medsci 585c. in the first session, five of 14 thirty-four students were registered for labm 680, but nine were dropped from the rotation due to covid-19. students were located in states outside of washington (one in alaska, two in idaho, one in iowa, and one in wyoming) and four of 14 students in the second session (one in alaska, one in idaho, two in wyoming). three additional students in the second session were more than a 3-hour drive from seattle. as a required assignment, all students completed the reflection piece. students most frequently commented on phases of testing (n = 10), test utilization (n = 10), regulatory agencies (n = 8), and the experience of teaching reverse lectures (n = 3). all students completed the course evaluation survey in the first session, and 11 of 14 students completed the course evaluation in the second session. average rating of the course overall for each session was 4.45 and 4.57 out of 5 ❚table 5❚, corresponding to good to excellent (supplemental table 1 ). the response to "this course helped me appreciate the significance of the subject matter" was consistently in the top two highest scores (4.7 of 5). for both sessions, the lowest average score to a targeted question was in response to whether there was adequate protected study time (table 5 ). in response to whether the class length was appropriate for the subject matter, 16 respondents answered it was "just right," five "too short," and four "too long." critical remarks about the course most frequently included teleconferencing fatigue (n = 4) and the density of complex subject matter in sign-out and conferences (n = 7), with the majority of these comments directed toward the first session. only free text or verbal feedback on the adaptation to labm 685 was available. during the covid-19 pandemic, the clinical laboratory response has been dynamic. as new developments have arisen on a global and national scale, these disruptions to the community at large have greatly affected academic practice. 6, 13, 14 other groups in dermatology, 15 surgery, 16, 17 nursing, 18 and anatomic pathology 19, 20 have reported on rapid and drastic changes in medical education and ways to proceed. in laboratory medicine, most divisions-except for therapeutic apheresis-conduct clinical service work without direct patient contact. in compliance with covid-19 distancing measures, laboratory medicine service work, sign-out, and didactics have shifted to a predominately remote-access paradigm for trainees and faculty alike. this represents a unique opportunity for medical students to participate in similar activities with additional guidance to interpret the clinical experience. therefore, the course structure involved elements of direct sign-out observation, in addition to both guided and independent study. additional activities involve small-group sessions, journal article reading, and live or prerecorded didactics. the lack of a vaccine against sars-cov-2 has placed diagnostic tests for the virus at the leading edge of the pandemic response. the result has been enormous medical and popular interest in laboratory diagnostics with the recognition that issues pertaining to the laboratory have vital repercussions for society. this was apparent from the student response during labm 685. we have leveraged this interest by creating a remote course, medsci 585c, so future clinicians have a better understanding of the laboratory's role. as in the pre-covid rotation, the goal was to provide exposure to multiple laboratories and the breadth of testing performed at our institution. special focus included how the clinical laboratories have mobilized in response to an urgent pandemic and whether this response is generalizable or unique to this institution. more quotidian goals included introducing students to common activities of the laboratory; exploring concepts of preclinical variables, including labeling/patient identification and patient safety in the clinical laboratory; and encouraging consultation with laboratorian/pathologist colleagues in future clinical practice. the change of the course from a two-to four-student rotation conducted in person to a potentially 40-student rotation conducted entirely online necessitated changes in structure to preserve interactivity, engagement, and participation in sign-out. to facilitate discussions, we structured portions of the clerkship as small groups of up to 10 students co-led by laboratory medicine faculty and fellows who meet twice a week for an hour. the first two sessions consisted of guided discussion and question-and-answer sessions, while the last two sessions consist of flipped classroom teaching sessions moderated by small-group instructors ( table 4 ). the benefits of small-group and reverse teaching methods in the classroom in promoting interaction and collaboration have been reported on previously, [21] [22] [23] and students commented positively on their experience in evaluations. for the first spring quarter that this clerkship was offered, 28 students rotated through the clerkship. this brings the total students rotating through uw laboratory medicine clerkships in ay2019 to 64 students-roughly double the number of students who normally would have been exposed to laboratory medicine ( table 2 ). together with the distance learning clerkship offered by our anatomic pathology colleagues, pathology accounted for two of 13 courses offered to third-and fourth-year medical students at this institution during covid-19. the changes in the clerkship increased the visibility of and access to laboratory medicine education, including nine (32%) medical students who were out of state. however, serving students in different regions revealed two unanticipated challenges: accommodating a 3-hour span of time zones and coping with limited or disrupted internet access, particularly in rural areas. to meet these needs, we minimized activities at the extremes of the workday and encouraged early submission of student presentations when weather threatened to disrupt infrastructure, with the option to have students narrate their presentations by phone. additional trade-offs between traditional and online format are presented in table 1 . technology such as teleconferencing, didactic recording, and learning management software was already partially used in the som, but the global pandemic quickly forced universal adoption of these tools to facilitate education in this period of social distancing. 6, 18, 24 an unprecedented amount of online resources, media, and readings have been made available through both online commercial modules purchased by the university and free online learning materials. 19, 20, 25, 26 stored content has the main advantage of flexibility-students and trainees can access high-quality content at any time-but lacks the immersion of a real case and an experienced mentor to navigate the clinical thinking around each case. hence, a curriculum that would bring students into the environment of clinical pathology, combined with small-group activities to navigate cases and increase learning engagement, was used. we identified two additional challenges unique to the remote format: teleconferencing fatigue and hipaa compliance. students in the first session reported teleconferencing fatigue on days with back-to-back conferences. this was partially mitigated in the second session by increasing the number of breaks between live sessions and reliance on several previously recorded lectures that were viewable at any time. concerns over the security of teleconferencing applications are not unique to this setting, 27 but because teleconferencing was used extensively for didactics and sign-out of clinical cases that could involve sensitive phi, hipaa compliance was critical. this required that teleconferencing applications could (1) ensure the confidentiality of electronic phi, (2) protect against reasonably anticipated threats to such information, (3) protect against reasonably anticipated nonpermitted uses or disclosures of such information, and (4) ensure compliance by its workforce. 28, 29 we relied on the hipaa-compliant accounts made available by our institution, required students include their role in their screen name (eg, "medical student," "ms-4"), and encouraged teleconference hosts of phi-sensitive meetings to require a passcode for access as recommended. 28 our efforts reflect one of several ways in which medical educators have adapted to covid-19. on the basis of this experience and the positive student feedback (table 5 ), we will continue to offer a remote learning course for up to 24 medical students per session-including students in quarantine, those at wwami/rural sites, and visiting students-for the rest of calendar year 2020. even though our som is returning to in-person clerkships with some limitations in the summer and fall of 2020, this remote learning approach has the benefits of providing in-depth instruction in laboratory medicine, exposing students to the critical role of the clinical laboratory in response to emerging infections, and affords flexibility for educators to respond to the continuing pandemic. we note that other solutions have been employed for medical and nursing students. these have ranged from graduating senior medical students early 30, 31 to training medical students to perform critical public health activities, particularly contact tracing, conducting telehealth visits, and staffing covid-19 call centers. [32] [33] [34] we encourage such innovative solutions and believe that our approach to rapidly adapt to distance learning dovetails with engaging medical students in a variety of alternative learning activities. the changes in medical education afford a new opportunity for medical student exposure to pathology and laboratory medicine. as the united states faces a prolonged first wave of covid-19 infections, such changes in medical education appear to reflect a new normal and may require further evolution. corresponding author: hamilton c. tsang, md; hamiltn@ uw.edu. first case of 2019 novel coronavirus in the united states novel coronavirus outbreak (covid-19) covid-19 united states cases by county prepare to adapt: blood supply and transfusion support during the first 2 weeks of the 2019 novel coronavirus (covid-19) pandemic affecting washington state three uw students in seattle screened for novel coronavirus medical student education in the time of covid-19 proclamation by the governor amending proclamations proclamation on declaring a national emergency concerning the novel coronavirus disease (covid-19) outbreak. white house proclamations the team at the @seattleflustudy have sequenced the genome the #covid19 community case reported yesterday from snohomish county sars-cov-2 testing comparative performance of sars-cov-2 detection assays using seven different primer/probe sets and one assay kit moving classes online and concluding the winter quarter covid-19 and medical education strange days medical education and care in dermatology during the sars-cov2 pandemia: challenges and chances the effects of covid-19 on academic activities and surgical education in italy residency and fellowship program accreditation: effects of the novel coronavirus (covid-19) pandemic [published blended learning via distance in pre-registration nursing education: a scoping review implementing a structured digitalbased online pathology curriculum for trainees at the time of covid-19 leveraging technology for remote learning in the era of covid-19 and social distancing: tips and resources for pathology educators and trainees self-regulated learning: the effect on medical student learning outcomes in a flipped classroom environment analysis of satisfaction and academic achievement of medical students in a flipped class informatics training for pathology practice and research in the digital era daily medical education for confined students during covid-19 pandemic: a simple videoconference solution blood bank guy transfusion medicine questions the coronavirus pandemic made zoom the most popular app, and that came with consequences voip for telerehabilitation: a risk analysis for privacy, security, and hipaa compliance covid-19: states call on early medical school grads to bolster workforce itching to get back in": medical students graduate early to join the fight. association of american medical colleges medical students: how to keep learning as covid-19 volunteers alberta medical students help with covid-19 contact tracing. global news uw nursing students join frontline efforts to battle covid-19, meet public health needs key: cord-295240-76ee00i0 authors: kruchten, anne e. title: a curricular bioinformatics approach to teaching undergraduates to analyze metagenomic datasets using r date: 2020-09-10 journal: front microbiol doi: 10.3389/fmicb.2020.578600 sha: doc_id: 295240 cord_uid: 76ee00i0 biologists with bioinformatic skills will be better prepared for the job market, but relatively few biology programs require bioinformatics courses. inclusion in the curriculum may be hindered by several barriers, including lack of faculty expertise, student resistance to computational work, and few examples in the pedagogical literature. an 8-week wet-lab and in silico research experience for undergraduates was implemented. students performed dna purification and metagenomics analysis to compare the diversity and abundance of microbes in two samples. students sampled snow from sites in northern minnesota and purified genomic dna from the microbes, followed by metagenomic analysis. students used an existing metagenomic dataset to practice analysis skills, including comparing the use of excel versus r for analysis and visualization of a large dataset. upon receipt of the snow data, students applied their recently acquired skills to their new dataset and reported their results via a poster. several outcomes were achieved as a result of this module. first, youtube videos demonstrating hands-on metagenomics and r techniques were used as professional development for faculty, leading to broadened research capabilities and comfort with bioinformatics. second, students were introduced to computational skills in a manner that was intentional, with time for both introduction and reinforcement of skills. finally, the module was effectively included in a biology curriculum because it could function as either a stand-alone course or a module within another course such as microbiology. this module, developed with course-based undergraduate research experience guidelines in mind, introduces students and faculty to bioinformatics in biology research. in 1920, botanist hans winkler coined the term "genome" as a fusion of the words gene and chromosome (winkler, 1920) . since that time, the "omics" fields have exploded, creating such terms as "pseudome" (the population of pseudogenes), "translatome" (the population of proteins in the cell, weighted by their abundance level), and many others that are increasingly becoming a normal part of the lexicon for biologists 1 . the term "bioinformatics" was defined by luscombe et al. (2001) , as "conceptualizing biology in terms of molecules (in the sense of physical chemistry) and applying "informatics techniques" (derived from disciplines such as applied maths, computer science and statistics) to understand and organize the information associated with these molecules, on a large scale. in short, bioinformatics is a management information system for molecular biology and has many practical applications." undergraduates in biology should be trained in this field to successfully compete in the job market and make vital contributions to the biological sciences as their careers mature. the vision and change: a call to action report of 2011 (brewer and smith, 2011) emphasized that undergraduate biology students should have competence in computational and systems level approaches and the ability to use large databases. only a small fraction of institutions offer a full undergraduate bioinformatics program (mellon, 2020) , but several offer courses on bioinformatics. in the state of minnesota, 50% of public and private school biology departments offer a bioinformatics course in their curriculum but none appear to require it for the degree. this may reflect a lack of expertise among faculty to teach the course. in 116 ,759 bachelor's degrees in biology were conferred to graduates in the united states (us-doe, 2020). among bachelor's degree holders 25-29 years old, biology graduates' annual salaries were not significantly different than the median annual income of all degree holders of $50,600, but computer and information science degree holders had an annual income of $70,100, well above the median income (nces, 2020) . a slight increase in biology-related computer information jobs is predicted, suggesting that biology majors would be well-served to develop computer information skills to complement their biology degrees (araneo et al., 2017) . bioinformatics is a broad field that encompasses gene alignment tools, crowdsourcing approaches, metagenomics, and many others. rather than lecturing about bioinformatics, many groups have chosen to incorporate bioinformatics tools into cures (course-based undergraduate research experiences). in cures, students are working in classes on research projects of interest to the broader scientific community (auchincloss et al., 2014) . on the curenet website 2 , several bioinformatics cures have been shared for faculty adoption and participation, including a crispr-cas9 project 3 , a study of iron uptake in insects 4 , genome solver: microbial comparative genomics 5 , and the genomics education partnership (gep) 6 . these programs, and many others across the country, teach students a variety of gene-based bioinformatics approaches including using blast, multiple gene alignment, primer design, and many others. students develop strong gene analysis skills while also contributing to active scientific research projects in the process. while (wang et al., 2015) and crowdsourcing datasets of antibiotic resistance in microbes (freeman et al., 2016; small-world, 2020) . students in these courses develop research skills such as bacterial culturing, sterile technique, pcr, and hypothesis building. few projects, however, teach undergraduates the computational skills required to statistically analyze "big data" in biological fields. computational skills are required to analyze and find patterns in big data, which includes the four vs: volume of data, velocity of processing the data, variability of data sources, and veracity of the data quality. applications of big data analysis can be found everywhere, but for biologists especially important applications include genome sequencing, ecological studies (such as of microbiomes), and health care information (li and chen, 2014) . graduates of biology programs have opportunities for employment in any of these fields but may not have the important computational skills in parallel with wet lab or field biology skills to be successful in big data fields. there seem to be few cures or similar programs published in the literature that provide instructions for how faculty can implement curricular modules to help students develop these big data skills. several groups have outlined a series of bioinformatics competencies for life scientists, including coursesource (the bioinformatics learning framework) (rosenwald et al., 2016) , the curriculum task force of the international society of computational biology (iscb) education committee (mulder et al., 2018) , and the network for integrating bioinformatics into life sciences education (niblse) (wilson sayres et al., 2018) . building on previous work from both coursesource and iscb, niblse surveyed instructors at us institutions and used the data to develop a list of core competencies for undergraduate life scientists. while many of the core competencies focus on genomics-based bioinformatics skills, several of the competencies are addressed by the work in this project. the competencies are listed below (wilson sayres et al., 2018) , and the bolded ones are addressed by the approach in this project: • c1. explain the role of computation and data mining in addressing hypothesis-driven and hypothesis-generating questions within the life sciences. • c2. summarize key computational concepts, such as algorithms and relational databases, and their applications in the life sciences. • c3. apply statistical concepts used in bioinformatics. • c4. use bioinformatics tools to examine complex biological problems in evolution, information flow, and other important areas of biology. • c5. find, retrieve, and organize various types of biological data. • c6. explore and/or model biological interactions, networks, and data integration using bioinformatics. • c7. use command-line bioinformatics tools and write simple computer scripts. • c8. describe and manage biological data types, structure, and reproducibility. • c9. interpret the ethical, legal, medical, and social implications of biological data. importantly, both the coursesource bioinformatics learning framework and the iscb curriculum task force recognize that there are different levels of users of bioinformatics curriculum, including bioinformatics engineers, bioinformatics scientists, and bioinformatics users. the approach described here is geared toward bioinformatics users, including both faculty who are interested in learning about these tools and students who will be moving forward into a variety of careers in research, medicine, education, and others. this course module is a starting point for introducing students to low level bloom's taxonomy areas such as knowledge and comprehension of bioinformatics. it is hoped that this introduction will spark an interest in students to learn more about the field and become bioinformatics scientists. this approach is also intended to provide an entry point for faculty to begin developing new courses in bioinformatics within their undergraduate biology programs and collaborate with colleagues in computer science fields to pool interests and resources. in response to the need for a big data cure, i have developed an 8 week course that meets for two 2-hour sessions weekly in which students gain hands-on experience using r and excel to analyze large datasets. to mimic an authentic research experience as closely as possible, the 10 students work as a research group as they discuss the literature, develop hypotheses, and plan experiments. individuals or pairs are responsible for collecting samples and performing the actual sample preparation and experiments. data analysis is completed individually and then discussed and improved in the full research group. while this course was developed as a stand-alone experience, it could easily be incorporated as a module in a broader full length course. the primary student learning outcome for this course was to develop students' data science skills using excel and r. the premise of this research course was to perform a metagenomic analysis of the microbiota in two different snow samples. to accomplish this research project, students perform a literature review, develop hypotheses, collect and prepare samples, perform metagenomic sequencing (through a third party vendor), learn data analysis skills, and present their research findings via a poster presentation. secondary student learning outcomes for this course include those described in the cure network: making discoveries of interest to the broader scientific community, an iterative work experience, communication of their findings, and development of scientific research skills (curenet, 2020). weeks 1 and 2: literature review, hypothesis development, and sampling table 1 highlights the main activities completed in the course, beginning with a literature review. because the primary learning outcome for this course is the development of r and excel skills, the instructor can assist in the literature review process by developing the initial research question and providing some preliminary resources to begin the discussion. in this project, i developed the initial research question of "how does the bacterial population vary between two snow samples from different locations on campus?" and provided several primary and secondary articles about microbiomes, microorganisms often found in snow, and bacterial abundance and diversity. students used these resources as jumping off points to find more sources (usually pdfs, websites, and videos) which were collected in a class google folder. students visually mapped these sources into three broad categories on the whiteboard: "snow, " "microbiomes, " and "microbial diversity." after a group discussion, each student was responsible for developing an individual literature review from these and other sources they found. this fast-paced literature review process leads to the development of a research question, hypothesis, and sampling procedure. metagenomic analysis with our vendor takes 3-4 weeks, so it was essential to collect and prepare samples right away to allow time for the primary student learning outcome of developing skills in excel and r. to this end, after discussion, most of the class agreed upon the same research question and hypothesis, with slight variations that could be accommodated within the sampling and sample preparation processes. our research question asked if the microbiota of snow samples would differ between an area heavily trafficked by both foot and automobile traffic compared to campus trails primarily traveled by snowshoe. most students hypothesized that the area with both foot and automobile traffic would have more bacteria overall and more diversity of bacteria. students demonstrated their understanding of the field and our research question development by submitting a draft of an introduction for their final poster project (see supplementary material section 5 for teaching materials). sampling and sample purification were relatively simple and inexpensive. students used 50 ml plastic conical tubes (vwr 89039-656) to collect three samples spaced at one meter intervals along a line at each of the two sites for 3 days in a row. to purify microbial dna from the samples, the snow was melted and filtered through a 0.2 micron polyester membrane using an aeropress coffee press 7 . the membranes containing the filtered microorganisms were then processed using the qiagen dneasy powerwater kit (qiagen 14900-50-nf). after confirming the presence of bacterial dna via pcr with a 16s primer set (idtdna.com; 16s rrna for #51-01-19-06, 16s rrna rev 51-01-19-07), the samples were sent for metagenomic sequencing off campus. weeks 3 and 4: introducing metagenomics, big data, and r the first step in teaching students about bioinformatics was to guide them through an understanding of how metagenomic sequencing works and how the dataset was generated. a prerequisite for this course was a one semester foundations in biology course covering the essentials of molecular biology, including central dogma concepts such as dna, rna, base pairing, replication, and transcription. the literature review is initiated by the instructor to save time and is further developed by students. students use their literature review to develop their hypothesis, identify sampling methods, and prepare dna samples, allowing a pairing of wet lab skills with in silico activities. when wet lab resources are unavailable, this step can be completed by the instructor or replaced with an existing publicly available dataset. students build on foundational knowledge of dna from prerequisite courses by viewing video material on pcr and sequencing. instruction in statistics, excel, and r using a combination of video material and in class discussions builds a foundation of data analysis skills. practice data analysis using an existing dataset students use their developing data analysis skills to mimic the instructors' actions using excel and r to analyze an existing dataset. 6 reinforcing data analysis skills with snow dataset students apply the data analysis skills they have learned and practiced to a new dataset from samples collected on campus. students showcase all the skills practiced in the course in a poster containing a research question, background material, a hypothesis, methods, results, and discussion. students complete the semester by recording a video presentation of themselves presenting their poster. if available, students also present their posters at a campus-wide research symposium. the supplementary material contain a list of resources used for reviewing foundational dna and pcr knowledge (supplementary material section 1) . with this background in mind, students work to understand the polymerase chain reaction, or pcr. this foundational knowledge is essential, in part because it strips away the complexities of how we typically teach replication with emphases on all of the different enzymes (polymerases i and iii, primase, ligase, helicase, etc.) and focuses on the simple concept of creating a complement sequence of dna to the template. after mastering pcr, students then move on to understanding dna sequencing, beginning with sanger sequencing. to do this, students watch a series of youtube videos on sanger sequencing 8 , the evolution of next-generation sequencing 9 , and finally illumina sequencing 10 used by our vendor (see supplementary material section 1 for more details). after watching the video on illumina sequencing, students usually express a combination of fascination and confusion. to provide further practice in understanding this extremely important process, we break into student pairs and have each pair illustrate the processes of cluster formation on whiteboards using color coding. after performing a similar exercise to better understand base calling, we complete this section of the instruction by discussing how multiple overlapping dna segments from one organism can be used to generate the sequence for the entire 16s ribosomal rna gene. it is common for biology students in our program to have a fear or aversion to mathematical and other quantitative or computational approaches. 65% of traditional undergraduate 8 https://www.youtube.com/watch?v=jnk_4maf5fk 9 https://www.youtube.com/watch?v=jfcd8q6qstm&t=176s 10 https://www.youtube.com/watch?v=fcd6b5hraz8 students enrolled in our college identify as female, 31% identify as first generation college students, 35% have family incomes less than $50,000, and 70% come from rural communities and small cities. many students have taken the minimum mathematics courses required by the state graduation guidelines. in a study of life sciences majors conducted by andrews and aikens (andrews and aikens, 2018) , both females and first generation students exhibited a lower interest in mathematics topics in biology than their counterparts, and females perceived a higher cost associated with doing math in biology than their male counterparts. they also found that students' likelihood of taking a biostatistics class was positively related to their interest and perceived utility of the course. a goal for this course module is to spark future interest in bioinformatics training, so it was important to demonstrate to students the utility of statistical analysis both for the project and their future careers. in recognition of these factors, i began the bioinformatics instruction with a review (or novel instruction) of basic statistical analysis. to accomplish this, students first reviewed major statistical functions such as mean, median, standard deviation, standard error, p-values, and student's t-test using a freely available resource compiled by mit 11 . these concepts were practiced using a very simple assignment completed in pairs during class time examining the statistical significance of simple drug treatment data (see supplementary material section 2 for details). in class discussion helped to sort out problems in understanding before moving on to larger dataset analysis. next, students are introduced to fundamental concepts in data analysis, including data clean up and developing the research question. to facilitate this process, i provided the students with a dataset previously collected in the boundary waters canoe area 11 https://web.mit.edu/$\sim$csvoss/public/usabo/stats_handout.pdf frontiers in microbiology | www.frontiersin.org wilderness (bwcaw). this dataset included triplicate sampling of four different sample sites resulting in 12 columns of data on a spreadsheet. after metagenomic sequencing, 15,000 unique bacterial species or otus (operational taxonomic units) were identified in the spreadsheet rows, resulting in 180,000 unique cells of data. given that most students' experience of using excel to this point had been in traditional lab courses, this was by far the largest excel file any of them had ever opened. to make the experience less overwhelming for the students, i provide them with a version of the dataset that condensed otus into phyla, resulting in a dataset with 12 sampling columns and 23 rows of identified phyla. my goal was for them to be able to use excel to average the triplicate results from each sample site and make comparisons across the data, either between the four individual sample sites or between phyla. to do this, i created a video of myself using excel to average the sites, perform a t-test comparing the data between sites, and then sort the data by increasing p-value, thus reordering the data so that the most significant p-values were at the top of the list. students then were required to repeat the actions of this video on both the phyla dataset and the otu dataset. in doing this, students gained experience cleaning up and renaming columns, writing formulas, accessing the formula bank, sorting, and visualizing data. after establishing comfort with analyzing data in excel, we moved on to r. r is a freely available statistical computing program (the r-foundation, 2020) used across many fields for the analysis and visualization of data. for the purposes of this course module, i wanted to introduce students to the pros and cons of using the programming language r versus using excel both for data analysis and for data visualization, particularly for its ability to generate a heat map of large datasets. this includes establishing student knowledge, but not necessarily application, of using a command line and understanding the function of packages, bundles of shareable code created by experts in the field and freely available for use. when students learned coding was involved, there was an immediate sense of anxiety in the room. to alleviate this stress, i returned to an approach with which the students were familiar: learning by watching videos. just as they had learned to use excel functions by watching me perform tasks via video, the basics of r were laid out by watching a series of publicly available youtube videos. many videos are available, but i chose the "r programming for beginners" playlist from the r programming 101 youtube channel 12 (see the supplementary material section 3 for a complete list of videos). in this series, the host, public health specialist greg martin, guides viewers through the whole process of using r, including downloading r and r studio onto their computers, learning basic commands such as identifying variables and manipulating a preloaded dataset of health characteristics of star wars characters, and installing and using r packages. this playlist resonated with the students, both 12 https://www.youtube.com/channel/ucfjyq3p2k_suqfxvdqieqnw because of the clear instructions and because of the link to public health, a field with which many of the biology students could identify. students watched this series of videos on their own and their sole assignment was to replicate exactly what the host did and turn in a screenshot of their final r studio product. once the students achieved some initial comfort with r, i gave them a fully composed sheet of code to copy and paste into the script window of r. the code was created by modifying freely available code (albert and yoder, 2013) , including the packages gplots, vegan, and rcolorbrewer to plot data, create the heatmap, and apply a color scheme. i used this approach for three reasons. first, students did not yet have the capability to compose their own code because they didn't have enough knowledge of syntax to do what was needed. second, because r is an open access community, students and instructors can find existing code for many functions on the internet and modify it to fit their needs. third, by providing code that was annotated (with # lines explaining each line of code), students were able to walk through each line of code, understand the function, and run the code to achieve a final product of a heat map demonstrating the diversity and abundance of microbial samples across sampling sites in the bwcaw (figure 1 ; full code in the supplementary material section 4). because the purpose of this course module was to introduce bioinformatics users to command line coding, the ability to generate a finished product was important both to increase their level of confidence in using r and in order to demonstrate the analysis capabilities available in r that were not available in excel. at this point in the course, students had participated in a strong introduction to data analysis using both excel and r. they had manipulated a dataset larger than any of them had seen before and reflected on the pros and cons of each tool in analyzing the datasets. each student had observed excel and r being used via video and followed up with practice completing the work themselves. this iterative approach follows best practices in pedagogy where students are offered multiple opportunities to observe, practice, and learn a skill. when the data from the metagenomic analysis of the snow samples was returned to us in week six, students were ready to analyze it. the final project was a standard scientific poster presentation of their background, research question, hypothesis, methods, results, and discussion. to accomplish this task, students had to return to the notes they took for the analysis of the bwcaw dataset and apply these approaches for the snow dataset. this task involved cleaning up the data, and properly labeling sample columns, and changing existing lines of code in r to import the proper.csv file, identifying columns correctly and creating an appropriate visualization. by using this iterative approach of first observing, then practicing, and finally applying, all the students were able to successfully assign the right syntax to the code and create a successful project. as presented, this process allows students to experience both wet bench and in silico research. however, it is important to note that the project could be modified to include only the in silico figure 1 | example heat map and r code. students used r to generate two heat maps in the course, first with a practice set of data from the boundary waters canoe area wilderness which was followed by a heat map of snow sample data to reinforce skills. (a) representative student-generated heat map of the bwcaw data. on the right axis, triplicate samples are boxed with corresponding colors; bacterial species' names are on the bottom axis. r-generated dendrograms are on the left and top axes. (b) a snapshot of the script window of r studio showing the code students used to generate the heat maps. a full copy of the code is available in supplementary material. frontiers in microbiology | www.frontiersin.org 6 september 2020 | volume 11 | article 578600 experience for students, as was the case in the second iteration of this course in spring 2020 due to the covid-19 pandemic and the closure of college facilities. it would be possible to provide this experience with the many publicly available datasets, but during the college closure i chose to perform the wet bench portion myself prior to the beginning of the course so that students felt they had a more "personal" sample rather than a dataset to which they had no personal attachment. this approach resonated with students as evidenced in their comments in the course evaluations. during the covid-19 pandemic in spring 2020, the course was delivered using both asynchronous and synchronous (zoom) methods. the course meeting schedule was altered to limit zoom fatigue by meeting synchronously on tuesdays and working asynchronously on course materials during the remainder of the week. thursday meeting sessions were reserved for open office hours, an approach that well was received by students and widely used. tuesday synchronous meetings were initially used for discussions of the overall project, research design, and sequencing videos. breakout rooms in zoom were used extensively to facilitate small group discussions of research questions and to build comprehension of the sequencing videos. beginning in week 3, the course took on essentially a "flipped" format. students viewed and practiced skills introduced in the videos and synchronous class time was used for troubleshooting, comprehension checks, and setting up the "next steps." students in this virtual course were still able to successfully use r for statistical analysis and visualization of their data. one of the most important take home messages of this work is that we should take advantage of technology both to continue skill development as faculty and to teach resourcefulness to students. many faculty who teach undergraduate students completed their dissertations before the age of bioinformatics or in an area that did not focus on quantitative skills. these faculty may not currently possess the skills to incorporate a bioinformatics module into a course. youtube affords faculty the opportunity to learn new skills in a step-bystep manner when the technology and approaches may be wholly new to them. this is a very inexpensive and efficient way to acquire professional development that can serve to enhance both classroom teaching and potential new areas of research. as part of the course evaluation, students were asked to answer a series of confidence questions about skills developed in the course ( table 2) . on a scale of 1-10, with 10 being high confidence, all students rated themselves as a ten when asked about confidence in pipetting a variety of liquids with micropipettors, reflecting the skills developed in the wet lab portion of the course. when asked about explaining sanger sequencing and next generation sequencing to another scientist, the class averages were 6.8 and 6.7 out of 10, respectively, for these new skills learned in the course. the course successfully introduced students to basic knowledge about r, as reflected in an 8.9 average score to "i can copy, perform, and run a simple code in r." as expected from an 8 week introductory module, the students did not feel confident enough to create and run their own r code (average score 5.2). students were also asked, "after completing this course, how has your interest in biological research changed?" all of their free response answers are below: • i am still interested in it, and now realize the importance of being able to effectively use r and excel to convey my data. • my interest in research has stayed quite high after taking this course. i am planning on working in the more biochemical side, but this was still very interesting and helped me make sure that a career in research is where i belong. • i feel like i have a better understanding of how questions are being asked in the biological community. • my interest in biological research has grown even stronger. i knew before that i love research, but every time i continue to do it, my passions grow stronger. • it greatly raised my interest in biological research. it was cool to see how the experiments we performed gave us numbers, that we could find relationships between. • i was always curious about how scientists made the figures they did. after using r, examining larger datasets is a lot less frightening. • i have a greater understanding of the importance of microbiomes and am interested in my own microbiome! • i was very hesitant about research before this course because i had a few bad experiences, but this class changed my outlook on it. i am definitely more interested and would like to do more. • my interest has greatly increased in biological research, specifically, on human microbiomes like the gut microbiota. also, conducting my own biological research and experiencing the challenges of creating a poster has made me appreciate all the hard work scientist do to give us informative papers. • i am once again excited now about the medical applications of molecular biology and studies! i'm excited to skim new articles and have a better toolbox to understand them after learning about r and how microbiome data can be represented. while this course had a very small sample size (n = 10), these responses suggest that this approach to using r was positively received by students. moreover, the students saw a utility in learning r, which research shows may lead to continued interest in participating in mathematical biology experiences (andrews and aikens, 2018) . in a short time frame, the course introduced students to bioinformatics and provided an opportunity for further practice. because of the students' ability to effectively visualize the dataset with r, they were able to think critically about the data and consider future research questions. from the r-generated heat map, the students realized that their initial hypothesis was incorrect. the heavy foot and automobile traffic sample site did have a higher abundance of bacteria but the diversity of bacteria was much lower than the sample site with light traffic. several students continued their analysis of the data even after the course ended and proposed a new research question for the next offering of the course. several outcomes were achieved as a result of this module. first, faculty expertise was enhanced in a time efficient manner using youtube training videos, leading to broadened research capabilities and comfort. second, students were introduced to computational skills in a manner that was effective and intentional, with time for both introduction and reinforcement of skills. finally, the module was effectively included in a biology curriculum because it could function as either a stand-alone course or a module within another course such as microbiology, leading to flexibility in the curriculum. this module, developed with cure guidelines in mind, is an effective and easily implementable way to introduce a broad group of students to bioinformatics in biology research, and also serves as a springboard for interested students to pursue further training and research in bioinformatics. the original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author. the author confirms being the sole contributor of this work and has approved it for publication. making heatmaps with r for microbiome analysis. the molecular ecologist life science majors' math-biology task values relate to student characteristics and predict the likelihood of taking quantitative biology courses advising biology majors about career choices: resources & information for biology instructors assessment of course-based undergraduate research experiences: a meeting report vision and change in undergraduate biology education: a call to action what is a cure crowdsourced data indicate widespread multidrug resistance in skin flora of healthy young adults † big biological data: challenges and opportunities what is bioinformatics? a proposed definition and overview of the field list of educational programs in computational biology the development and application of bioinformatics core competencies to improve bioinformatics training and education chapter 3 employment outcomes of bachelor's degree holders the coursesource bioinformatics learning framework small world initiative: crowdsourcing antibiotic discovery the r project for statistical computing. vienna: the r foundation digest of education statistics do you kiss your mother with that mouth? an authentic large-scale undergraduate research experience in mapping the human oral microbiome † bioinformatics core competencies for undergraduate life sciences education verbreitung und ursache der parthenogenesis im pflanzen-und tierreiche special thanks to the students in biol 4160 for enthusiastically and successfully attempting a task outside their comfort zone. the supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmicb. 2020.578600/full#supplementary-material conflict of interest: the author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.copyright © 2020 kruchten. this is an open-access article distributed under the terms of the creative commons attribution license (cc by). the use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. no use, distribution or reproduction is permitted which does not comply with these terms. key: cord-280449-7tfvmwyi authors: hoplock, lisa b.; lobchuk, michelle m.; lemoine, jocelyne title: perceptions of an evidence-based empathy mobile app in post-secondary education date: 2020-08-25 journal: educ inf technol (dordr) doi: 10.1007/s10639-020-10311-3 sha: doc_id: 280449 cord_uid: 7tfvmwyi cognitive empathy (also known as perspective-taking) is an important, teachable, skill. as part of a knowledge translation project, we identified a) interest in an evidence-based cognitive empathy mobile app and b) which faculties believe that cognitive empathy is important for their profession. students (n = 638) and instructors/professors (n = 38) completed a university-wide survey. participants in education, social work, and the health sciences were among those most interested in the app. the majority of participants said that they would prefer for the app to be free or less than $3 for students. most participants preferred a one-time payment option. across 17 faculties, all but one had 60% or more of its sampled members say that cognitive empathy is important for their profession. results illuminate perceptions of cognitive empathy instruction and technology. results also provide insight into issues to consider when developing and implementing an educational communication app. self-reflecting; hoplock and lobchuk 2019; vorauer 2013) , it has been linked to positive outcomes such as increased patient compliance and satisfaction within healthcare (kim et al. 2004) , engaging in prosocial behavior (davis 2015) , team effectiveness, and engaging in high quality communication (parker et al. 2008) . people vary in their ability to engage in cognitive empathy, but fortunately, it is a skill that can be taught (e.g., brunero et al. 2010; richardson et al. 2015; teding van berkhout and malouff 2016) . one way that cognitive empathy can be taught is by using technology (e.g., lobchuk et al. 2018) . the present research examines people's perceptions of a cognitive empathy-training mobile app as well as how perceptions of cognitive empathy's importance vary by profession. cognitive empathy is studied in a variety of fields, such as healthcare (e.g., blanch-hartigan and ruben 2013; lobchuk et al. 2016 lobchuk et al. , 2018 , psychology (e.g., marangoni et al. 1995; vorauer and sasaki 2014) , business (e.g., ku et al. 2015) , and law (e.g., bandes 2009 ). one frequently-used paradigm involves filming an interaction between two people, having the interaction members report what they were thinking and feeling throughout the interaction, having the interaction members guess what the other person was thinking and feeling throughout the interaction, and then providing an accuracy score (ickes 2001) . this accuracy score indicates how accurate a person is at inferring the thoughts and feelings of another. accuracy is associated with relationship satisfaction (sened et al. 2017; thomas and fletcher 2003) , skillfully providing social support (verhofstadt et al. 2016) , and accommodating during conflict (kilpatrick et al. 2003) . thus, accuracy is helpful for facilitating successful communication and relationships across contexts. people's perspective-taking ability improves when they are self-aware of personal values/emotions that can thwart empathy (lobchuk et al. 2012) ; receive instruction (lelorain et al. 2012 ) and feedback (noordman et al. 2012) , and self-evaluate with video-feedback (fukkink et al. 2011 ). the authors have been conducting iterative research on an intervention that takes these findings into consideration, incorporating ickes' (2001) paradigm and including perspective-taking instruction. however, currently, the intervention involves coming into a lab for the filming. to increase accessibility and cost-effectiveness, the intervention could be adapted as a mobile app for use within any setting, not just the classroom. most students world-wide own a smartphone (e.g., farley et al. 2015; nason et al. 2015; o'connor and andrews 2018; williamson and muckle 2018) . while mobile devices such as smartphones and tablets (crompton and burke 2018) are sometimes seen as a distraction in the classroom, they are increasingly being leveraged to facilitate learning (langmia and glass 2014; nguyen et al. 2015) . indeed, with classes being forced online due to covid-19, it is expected that instructors will incorporate technology into their courses more than ever before (bates 2020) . it is important that devices are used in a way that facilitates taking an active role in learning (norris et al. 2011) . when they are used in that way, mobile devices may improve student motivation and productivity (cotter et al. 2015) , confidence (koohestani et al. 2018) , and performance (dunleavy et al. 2019; hsueh et al. 2018) . for example, a recent metaanalysis of healthcare research on using mobile technology within education (mlearning) found that participants who experienced mlearning tended to have superior knowledge and skills than those who received traditional education (dunleavy et al. 2019) . thus, growing evidence supports the use of mlearning. we are interested in people's perceptions of using an app for teaching and practicing cognitive empathy. research findings do not always get translated into practice in fields like education, and it can take a long time for them to be implemented if they do (burkhardt and schoenfeld 2003) . we follow the canadian institutes of health research knowledge to action process (kta; government of canada 2016) framework to expedite the integration of our evidence-based intervention into students' learning activities. according to this framework, knowledge is created (e.g., through research) and then translated into application through an iterative cycle. knowledge creation includes inquiry, synthesis, and product-creation (government of canada 2016). we aim to move our research towards the product-creation phase and ensure that it becomes incorporated more quickly into practice. we currently conduct the intervention with students from a variety of healthcare disciplines (e.g., nursing, occupational therapy, kinesiology; (e.g., lobchuk et al. 2016 lobchuk et al. , 2018 . one version of the intervention has also included videoconferencing (hoplock and lobchuk 2019b ). yet, this intervention could be made more accessible if it were converted to mobile technology. many students and instructors/professors own mobile devices and use apps, making them the ideal population to assess the potential use of mobile learning for educational purposes in an academic setting. it is important to conduct market research with the population of interest (university students and instructors/tenure-track and tenured professors), so that we understand potential app users and the merit of creating the app before we start creating it. we want to ensure that what we create is meaningful to the people who will use it and that it will be used within communication skills curricula. to date, we have conducted intervention research with health professionals. however, it is possible that the potential intervention user-base is larger than just those in the health profession (e.g., law, education, or business). thus, we circulated a university-wide survey to students and instructors/professors to better understand who might be interested in the intervention and cognitive empathy. the purpose of the present research is to identify the target market, demand, and price point as well as to solicit student and instructor/professor perceptions of the empathy-training mobile app. we had the following research questions: this work may appeal to people who study empathy, education technology, marketing, and business. this research may also help people who want to create related apps. cognitive empathy is a valuable skill for successful interpersonal relationships (e.g., batson and ahmad 2009; davis 2009; davis 2015; galinsky et al. 2005 ) and so converting a successful intervention to be able to teach empathy accessibly is a worthy goal. after obtaining ethics approval, we conducted a survey to answer our research questions. in accordance with simmons et al. (2011 simmons et al. ( , 2012 , we report how we determined our sample size, all data exclusions, all experimental manipulations (there were none), and all study measures. our aim was to give all students and instructor/professors the opportunity to describe their opinions or attitudes toward our empathy application. a census sampling frame was determined to be most appropriate. this sampling frame aims to collect information from every eligible member of the population. our decision was not to exclude any student or instructor/professor at the university so as to boost our success in accruing a representative sample of students and faculty by taking a census sampling approach. because this work was exploratory, we did not conduct a formal power analysis. instead, we aimed to recruit at least 300 students (hao et al. 2017 ) and 100 instructors/professors (vrana 2018) or as many participants as we could before our stopping rule: complete data collection by the end of december, 2018 (approximately 1 month after data collection started). we chose this stopping rule due to time constraints. participation was restricted to students and instructors/professors over the age of 18. approximately 30,000 students and 1100 instructors/professors were emailed a survey link; 1081 people accessed the survey. data from 676 participants were retained (638 students; 38 instructors/professors) after exclusions (n = 13 did not provide consent; n = 391 did not fully complete the survey; one person provided nonsensical responses). most participants identified as white (66% students, 86% instructors/professors) and as women (68% students, 75% instructors/professors). the average ages of students and instructors/professors were 24.64 years (sd = 8.02) and 48.82 years (sd = 9.19) respectively. participants volunteered for an online study on "perceptions of an evidence-based empathy mobile app in post-secondary education." at the start of december 2018, they were emailed a study description and link. they were told that they would have until the end of the month to complete the study. a reminder was sent mid-way through the month. participants did not receive compensation. materials can be found on the open science framework: https://osf.io/bh9 su/?view_only=0ff44fe143914e0088c1f5d54ac7f8c6. the majority of questionnaire items were author-created and inspired by the literature (e.g., alwraikat and tokhaim 2014; sevillano-garcia and vazquez-cano 2015; vrana 2018) . we obtained suitability and coverage feedback on our survey from experts in technology development at the university's technology transfer office. the overall structure of the questionnaire was the same for students and instructors/professors, however the content of the questions varied. students completed questions regarding using the app as part of their education. instructors/professors completed questions regarding using the app as a teaching resource as well as for personal use. after providing informed consent, participants were first presented with a description of the app's purpose and how it would work. they saw two wireframes to give them a sense of the app idea and what the app might look like. next, they were asked their perceptions of the app. students rated their agreement on four statements (1 = strongly disagree, 7 = strongly agree), providing their perceptions of the app as being applicable to their profession, their perceptions of the app helping them to meet their needs, their liking using mobile learning as part of their educational curriculum, and their confidence in using mobile technology to achieve their learning goals. instructors/professors were asked to rate their agreement with six similar statements using the same scale. four of the statements related to using the app for teaching or professional work, one related to their perceptions of using mobile technology as an education tool, and one related to their confidence in using mobile technology to achieve their teaching goals. participants then indicated their interest in the app, with faculty indicating both their interest in the app as a teaching tool (1 item) and their personal interest in the app (1 item; 1 = very disinterested, 7 = very interested). using an open-ended question format, participants provided their reaction to the app (adapted to an open-ended question format from a 5-pt likert scale from surveymonkey n.d.). they then indicated how likely they would be to consider buying the app (1 = extremely unlikely, 7 = extremely likely; adapted from a 5-pt scale; surveymonkey n.d.). those who selected 1 or 2 on the scale were asked an open-ended question regarding why they were unlikely to consider buying the app. faculty were also asked how likely they would be to recommend that student buy the app (1 = extremely unlikely, 7 = extremely likely; adapted from a 10-pt scale and to this context; surveymonkey n.d.). those who selected 1 or 2 on the scale were asked an open-ended question regarding why they were unlikely to consider recommending buying the app. to better understand price point and pricing strategies, we asked participants to check all that apply when considering what one-time price they would feel comfortable paying for the app ($0.00; $0.01-$0.99; $1.00-$2.99; $3.00-$5.99; over $5.99), what payment type they would prefer (subscription (i.e., several smaller payments); one-time payment (i.e., one larger payment)), and what payment version they would prefer (consumers may choose between a basic version of the in your shoes mobile app that is free and a version that has extra features and a cost; consumers use a free trial of the in your shoes mobile app with extra features and payment is required later). students responded while imagining that the app was required in a course and then responded imagining that the app was not required in a course. instructors/professors were asked what they would feel comfortable asking students to pay if the app was required in a course, and then if it was not required. instructors/ professors were also asked what they themselves would pay if using the app for personal use. participants next completed demographics questions asking about gender, age, ethnicity, income (statistics canada 2020), smartphone or tablet ownership and use (chen and denoyelles 2013), device brand (adapted from chen and denoyelles 2013 to ask about what brand they primarily use instead of what device they own), hours spent on their phone or tablet for things related to work/school (chen and denoyelles 2013), whether they use an app for coursework (adapted from chen and denoyelles 2013 to ask whether they have used an app instead of how often), their faculty, college, department, and class format (in-person/in-class; online/distance; mix of in-class and online courses). students were asked about their program year, degree, and student status. instructors/ professors were asked their academic rank and teaching experience (the latter was adapted to include more options for those with fewer years of experience; alwraikat and tokhaim 2014). we also wanted to know if participants' respective programs had at least one course that focuses on interpersonal communication (yes, no, unsure) . those who answered "no" or "unsure" were asked if interpersonal communication was incorporated into their curriculum to some degree. finally, participants rated whether cognitive empathy was important for their profession (1 = not at all, 7 = extremely). we employed a descriptive, cross-sectional, online survey with participants. with all students and instructor/professors at the university having been invited to participate, we had the opportunity to 'drill down' and conduct exploratory analyses of linkages between student and instructor/professor characteristics and their opinions or attitudes toward the empathy application. descriptive statistics (medians, means, standard deviations, frequency counts, and percentages) were used to describe the sample of students and instructor/professors, as well as address research questions 1 to 3. we ran the responses to the open-ended questions through a sentiment analyzer and a word cloud generator located on danielsoper.com to get an objective sense of participants' sentiment towards the product. sentiment analyzers use "computational linguistics and text mining to automatically" determine the overall degree of negativity (â��100), neutrality (0), or positivity (100) in the text (soper n.d.) . word cloud generators analyze text to determine whether certain words are used more frequently. a strength of these two approaches is that it analyzes the results impartially and, thus, will help validate results found using our other methods. a limitation of these approaches is that they examine the overall text, without nuance. we also analyzed the responses to the open-ended questions using content analysis (see online supplemental materials for additional details; lincoln and guba 1985; patton 2002) . credibility was established by recruiting participants from the target population (elo et al. 2014) . dependability and confirmability were met with an audit trail documenting coding decisions and template development (e.g., saldana 2009 ). confirmability also occurred through independent coding and analysis and by an iterative feedback process until consensus was reached (graneheim and lundman 2004) . we also used participants' own words for codes and themes when possible to ensure that we stayed close to the data (levitt et al. 2018 ). there are 17 faculties at the university. at least one student from every faculty participated. instructors/professors from 10 faculties participated. therefore, we obtained diversity in the faculties that participated. over 90% of participants owned and used a smartphone or tablet, and most participants (61.3% students; 83.3% instructors/ professors) used apple products. for students, 44% have used a mobile app required or suggested by their instructor for course work. for instructors/professors, 57% have required or suggested to students to use a mobile app for course work. most participants preferred the app to be free (41% of students; 34% of instructors/professors) or less than $3 for students (56% of students; 34.2% of instructors/professors). participants also preferred a one-time payment option (85% of students; 66% of instructors/professors) as well as the ability to choose between a basic version of the app that is free and a version that has extra features and cost (74% of students; 63% of instructors/professors; see online supplemental materials for other participant details). addressing research question 1 (who would want to use this app), 44% of students and 53% of instructors/professors said that their program features at least one course that focuses on interpersonal communication. of those who said that their program did not feature an interpersonal communication course or that they were not sure if it did, 33% of students and 53% of instructors/professors said that interpersonal communication is incorporated into the curriculum to some degree. faculties from where most professors and instructors indicated that they had an interpersonal communication course in their program included business, education, health sciences, law, and social work. instructors/professors from the arts, education, health sciences, and law were among those most interested in using the app as a teaching tool (table 1) . instructor/professors from business; education; environment, earth, and resources; and law were interested in using the app for personal use (table 2 ). students in agriculture; art (e.g., fine art); arts (e.g., sociology); education; environment, earth, and resources; health sciences; music; science; social work; extended education; and university 1 (a faculty for those just starting at the university) were interested in the app (table 2) . thus, these faculties may be a good target for uptake of the app. when analyzing participants' reactions to the app, one theme that emerged, target or use case, involved commenting on the target people or use case (i.e., when or in what contexts someone might use it) for the app. this theme's categories included that the participant thought the app was not relevant to them; the participant commented on who the target audience might be; and the participant commented on potential use cases for the app (table 3) . for example, one participant (woman, instructor/professor, faculty of agricultural and food sciences) wrote, "looks great for areas where you need to communicate with a patient/client on more than one level. not so much in my area." another (man, student, faculty of health sciences) wrote, "the app sounds fun and with right pricing could be helpful to new international students." these responses help narrow the target market. addressing research question 2 (what people from the target population think of the app idea), sentiment analysis and word cloud generator results of the open-ended questions indicated that responses were generally neutral to negative. students' responses to "what is your reaction to the in your shoes mobile app" were neutral (0.3; possible range = â��100 to 100). the word cloud indicated that responses were focused ratings were made on a 7-pt scale with higher numbers indicating greater interest in the app around empathy, apps, and the perception that the idea is interesting (see online supplemental materials for the word clouds). instructor/professor responses were somewhat negative (â��42.4). their word cloud was somewhat similar to that of the students, but was also focused on students, teaching, and learning. diving deeper, when analyzing participants' reactions to the app, two additional themes emerged: participants described practical issues with the app, which might affect uptake and participants commented on the concept idea (table 3 ). the first theme's categories included barriers to empathy and barriers relating to the intervention procedure. for example, one participant (man, student, university 1 faculty) wrote, "seems like a great idea but would be a little weird to find a partner to record conversations and get them to tag their thoughts and feelings." these responses help identify potential concerns and factors to watch out for when creating the app; for example, finding the right dialogue partner and drawing on a relevant context to engage in a meaningful dialogue. the second theme had four categories: 1) positive reactions to the app idea (e.g., favourable evaluation of the app, willingness to try the app, and visualizing positive outcomes as a result of using the app); negative reactions to the app idea (e.g., unfavourable evaluation of the app; doubts in teaching empathy with an app; negative comments about empathy training in general; and comments about the app not practical issues with the app: participant describes barriers to using the app, which might affect uptake barriers to empathy: participant describes barriers that relate to empathy "it seems impractical -a lot of people would probably be very uncomfortable with the exercise described above and would not want to put in the time to tag a recording of their conversation." (woman, student, faculty of arts) procedure: participant describes barriers that relate to the intervention protocol "the instructions are a little complicated, which i believe will prevent some students from using it unless they really want to put in effort. it would be better if scenerios (sic) were supplied in addition to this feature so it didn't require two people to use." (man, student, university 1) these responses indicate varying support for the app idea and help clarify the sentiment analysis and word cloud results. participants who indicated that they would be unlikely to buy the app (i.e., they selected 1 or 2 on the question "how likely are you to consider buying the app") were asked about their response. students were understandably somewhat negative given the nature of the question (â��18.3). their word cloud indicated that responses focused on money, apps, and the word "don't." (e.g., "i don't buy apps"). instructors/professors' responses were unexpectedly positive (99.1). 2 the associated word cloud focused on empathy and technology needing to add value to the course. three themes emerged from qualitative analyses that help us to understand the disinterest in buying the app: 1) responses indicating that participants did not think they fit the target market; 2) responses about money or purchasing apps in general; and 3) responses about the app itself (table 4 ). like the target or use case theme for who would use the app, the first theme's categories included that the app is not relevant or that they lack interest in it. for example, one participant (woman, instructor/professor, faculty of social work) wrote, "i have taught interpersonal communication skills for approximately 25 yearsnot necessary" the second theme's categories included rarely or never spending money on apps; preferring a free app or a free alternative to the app; and that the app would not be a priority purchase (not worth the money). for example, one participant (agender, student, faculty of arts) wrote, "i don't buy apps." another (woman, student, school of business) wrote, "i would only get the app if it had a free trial period to try it out." the third theme's categories related to perceptions of the app's effectiveness and the app's protocol (e.g., the app is impractical, privacy concerns, and confusion about the app's use case or procedure). for example, one participant (man, student, faculty of engineering) wrote, "low chance of repeated use, requires time commitment and analysis." together, these responses provide additional insight into potential concerns, factors to consider when creating the app, and the target market (e.g., offer a free trial, ensure reliable security, promote broad applicability). 2 it is possible that negation within the responses affects scoring as all responses except one included the word "not." thus, responses like "not necessary" may have been interpreted positively. according to the site, "this tool produces an overall sentiment score. although various passages within a sample of text may be particularly positive or negative, the sentiment score produced by this tool considers all of the text in the sampleâ�¦ research shows that in about 20% of all cases human beings will disagree about the sentiment of written text." (https://www.danielsoper.com/sentimentanalysis/default.aspx) faculty who indicated that they would be unlikely to recommend buying the app (i.e., they selected 1 or 2 on the question "how likely are you to consider recommending buying the app") were asked about their response and (as would be expected given the nature of the question) responses were quite negative (â��69.1). the word cloud highlighted a focus on the classroom, skill development, empathy, and being unconvinced. three themes emerged from qualitative analyses: 1) responses indicating that the app is not relevant to the participant or their field (mirroring responses to being unlikely to buy the app); 2) responses indicating skepticism that empathy can be taught with an app (i.e., they are not convinced); and 3) responses about money or purchasing apps (table 5) . for example, one participant (did not report gender, instructor/ not relevant/no need: participant states that the app is not relevant to them or their field or states disbelief regarding the need for an app to teach empathy (e.g., they say that they are already empathetic) "not appropriate to my area of teaching" (man, instructor/professor, faculty of environment, earth, and resources) lack of interest: participant lacks interest "there would never be a time when i would want to use this. the only way i would ever use it is if were part of a leadership training thing." (man, student, faculty of science) about money: participant makes a comment related to money or purchasing apps rarely or never spend money on apps: participant says that they do not like purchasing apps or that they do so rarely "i never pay for apps" (woman, student, faculty of agriculture and food sciences) free app or free alternative: participant indicates that they would download the app if it were free, otherwise they would prefer a free alternative professor, faculty of arts) wrote, "i don't think they would use it and i'm not sure that i feel that an app is the best approach to learning empathy, so i'm not convinced on its utility." while another (did not report gender, instructor/professor, did not report faculty) wrote, "students have better things to spend their money on, however small the amount." together, these responses provide additional insight into the target market and instructor perceptions of using technology to teach empathy. addressing research question 3 (which professions believe that cognitive empathy is important to their profession), 81% of students and 83% of instructors/professors said that cognitive empathy is important for their profession. most participants (60% or higher) from all 17 faculties except one (school of agriculture) thought that cognitive empathy was important for their profession (table 6 ). the present research helps us progress towards the product-creation phase within the kta framework to facilitate uptake of an evidence-based intervention more quickly into the classroom. like other research (farley et al. 2015; nason et al. 2015 ; o'connor and andrews 2018; williamson and muckle 2018) over 90% of participants owned and used a smartphone or tablet. some, but not the majority, of instructors/professors were asking students to use mobile apps for their course work (see also ariel and elishar-malka 2019) . this amount is likely to increase spurred by the covid-19 pandemic. we also found that when considering the empathy-based app for class use, qualitative results echoed those found in previous research: participants mentioned usefulness, money, the ability to use a trial version, and enjoyment (kim et al. 2016 ). these variables are important because they predict intention to purchase an app (kim et al. 2016 ). together, results indicate a potentially large market for apps within higher education and that apps might be successful if perceived as useful, enjoyable, and triable. these findings are timely because there is a worldwide movement in education toward remote learning and using technology to deliver content (e.g., bates 2020). our research adds to the literature by illuminating perceptions of app cost within education. the majority of participants said that they would prefer for the app to be free or less than $3 for students. most participants preferred a one-time payment option. they also preferred the ability to choose between a basic version of the app that is free and a version that has extra features and a cost over a free trial and later payment. qualitative responses indicated that some participants just do not purchase apps. additionally, student participants described being stretched for resources and unable to spend money on apps. apps tend to be free and people often prefer free alternatives to having to pay (hsu and lin 2015) . thus, making the app free may increase uptake. app developers and marketers should consider making apps free for students and passing the cost on to institutions, when possible (see table 7 for all recommendations). doing so would allow a greater number and diversity of people to learn about concepts such as empathy. apps provide opportunities to advance changes in how content is being taught in and outside of the classroom. currently, when instructors and researchers want to use paradigms like ickes' (2001) , they often require students to come into a lab (e.g., lobchuk et al. 2016) . this makes participating unfeasible and costly for many. indeed, the present research indicates that cost is top of mind for students. the ubiquity of (lobchuk et al. 2012) ; receive instruction (lelorain et al. 2012 ) and feedback (noordman et al. 2012) ; and self-evaluate with video-feedback (fukkink et al. 2011 ) so that their perspective-taking ability improves. the present research indicates that people recognize the value that a cognitive empathy app can hold and how it can be used. the present research also highlights concerns that people have about using apps for empathy instruction. we found that some people are wary of using their personal devices for recording conversations and being vulnerable. these people might benefit from coming into the lab and participating in a safe experience before using the app, as well as from receiving tips on finding a dialogue partner and drawing on relevant context to engage in a meaningful dialogue. testing to ensure app security and communicating this security to users will also be important as will be ensuring a seamless experience and reducing procedural barriers via usability testing. additionally, our research finds that some people might need to be convinced of the benefits of using technology to teach empathy skills. developing a compelling marketing strategy that demonstrates how mobile devices can improve confidence (koohestani et al. 2018) and performance (dunleavy et al. 2019) , and cites evidence of the intervention's effectiveness might be beneficial here. the present research indicates that many people recognize the importance of cognitive empathy for interpersonal relationships and for their profession. convincing them that it can be taught with technology would ensure that more people get the instruction that they need. consistent with previous research (keyworth et al. 2013) , students want communication skills training. while previous research has studied cognitive empathy within a variety of fields (e.g., teding van berkhout and malouff 2016), the present research indicates that more fields are interested in it than likely previously thought. participants were interested in the app idea, with those in education, social work, and health sciences among the most interested. even participants in the physical sciences were interested and believed that cognitive empathy is important to their profession despite other research finding that being low in cognitive empathy predicts enrollment in the table 7 key recommendations to develop a mobile empathy app recommendations 1. make the app free or for a minimal cost to students and pass the cost to institutions, when possible 2. make the app a one-time payment for students 3. prioritize targeting people in the fields of business, education, health sciences, and law 4. empathize the benefits of dialoguing, self-reflection, and actively taking another person's perspective in marketing communications 5. provide tips on how to find a dialogue partner and draw on relevant context to engage in meaningful dialogue 6. engage in testing to ensure app security and communicate this security to users 7. conduct usability testing to ensure a seamless user experience and reduce procedural barriers 8. develop a compelling marketing strategy that highlights benefits gained across a variety of disciplines as indicated from previous research. developing concrete examples of when the app could be used and including testimonials would be helpful physical sciences (thomson et al. 2015) . perhaps they are aware of being low and want to improve their perspective-taking skills because they believe it is important for their profession. future research should investigate this further. the confusion and questions about the app protocol as well as the varying sentiment about the app indicates that more research is needed. future research will iterate app descriptions and wireframes to increase clarity and value of the app. scenarios where the app might be used will be guided by research and used in marketing so that the value of the app (e.g., trust-building; blatt et al. 2010 ) is evident to a wider range of people. one reason why increasing perceived value will be important is because it predicts app purchase intentions (hsu and lin 2015) . while the present research was a necessary first step into assessing perceptions of the concept idea, future research will also indicate how perceptions of the app and the idea of using technology to teach cognitive empathy changes as people start interacting with app prototypes. one limitation to this study is participant self-selection bias: the people who opted to participate in a study about empathy may have been more interested in this topic than others. this is evident, for example, in the number of instructors/professors who indicated that they taught courses having to do with interpersonal communication (53%), as well as the greater participation from people who participated in certain fields (i.e., health sciences, education). while we received student participation from every faculty area, this participation was not equal and there was less representation among instructors/professors. it is possible that we would have received a wider variety of responses had participants not known the app's topic (empathy) prior to starting the study. stating that the study focused on an instructional app or an app on communication skills (without mentioning empathy) might have increased the variability. generalizability is also limited because the sample was restricted to one canadian university. future pre-registered research examining perceptions of empathy's importance across disciplines should include a greater number of participants from diverse areas and could examine interest in an empathy app with a non-student population. for example, corporations may have more resources and be more interested in furthering the empathy of their employees than those within educational institutions. additionally, the survey was accessible during a busy month (december), thus people may have been more likely to participate had it occurred in a different month. moreover, the survey contained some adapted and investigator-developed questions that would benefit from additional validation. future research could conduct cognitive interviews to ensure that questions are being interpreted as they were written. despite these limitations, this research has a number of strengths, including the use of multiple methods and analysis techniques (morse 2015) . our methods allowed us to identify potential barriers to uptake, which is a step in the kta framework (graham et al. 2006) . potential user feedback spotlighted critical features to incorporate in an empathy app that is designed to foster targeted, meaningful, and efficient learning which can be differentiated based on discipline or profession. additionally, to our knowledge, this is the first study that has examined perceptions of the importance of cognitive empathy across a broad range of fields/professions. our finding that participants in a variety of fields/professions find cognitive empathy important will be of interest to people studying empathy, marketing, and education. taking the perspective of another and understanding where they are coming from is an essential skill for many professions. thus, being able to teach cognitive empathy effectively and accessibly is an important goal for instructors. as impacted by the covid-19 pandemic, efforts in restructuring how we teach in basic and continuing education are reflective of increasing reliance on technology-based learning. not everyone is enthusiastic about using technology in empathy education. some people are concerned that technology impedes empathy and learning by, for example, acting as a barrier between people. however, we believe that, done correctly, technology can instead act as a bridge. availability of data and material consent was not provided to share non-aggregated data. materials and word clouds are available on the open science framework: https://osf.io/bh9su/?view_only=0ff44fe143914 e0088c1f5d54ac7f8c6 funding information this research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors, nor has the app development received any funding to date. conflicts of interest/competing interests the results of the present research are being used to guide development of an evidence-based empathy app that the first two authors are creating. code availability not applicable. exploring the potential of mobile learning use among faculty members learning in the smartphone era: viewpoints and perceptions on both sides of the lectern. education and information technologies empathetic judging and the rule of law online enrolments after covid-19: some predictions for canada training clinicians to accurately perceive their patients: current state and future directions does perspective-taking increase patient satisfaction in medical encounters? a review of empathy education in nursing improving educational research: toward a more useful, more influential, and better-funded enterprise exploring students' mobile learning practices in higher education perceived benefits of mobile learning devices for doctoral students in a school of allied health professionals the use of mobile learning in higher education: a systematic review developing empathy in nursing students: a cohort longitudinal study a perspective on cultivating clinical empathy oxford library of psychology. the oxford handbook of prosocial behavior mobile digital education for health professions: systematic review and meta-analysis by the digital health education collaboration qualitative content analysis: a focus on trustworthiness how do students use their mobile devices to support learning? a case study from an australian regional university video feedback in education and training: putting learning in the picture perspective-taking and self-other overlap: fostering social bonds and facilitating social coordination lost in knowledge translation: time for a map? the journal of continuing education in the health professions qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness influential factors for mobile learning acceptance among chinese users are perspective-taking outcomes always positive? challenges and mitigation strategies qualitative insights into adapting an in-lab empathy intervention for use via videoconference with groups of nursing students what drives purchase intention for paid mobile apps? an expectation confirmation model with perceived value an app to enhance resident education in otolaryngology measuring empathic accuracy nursing students' perceptions of obesity and behaviour change: implications for undergraduate nurse education empathic accuracy and accommodative behavior among newly married couples the effects of physician empathy on patient satisfaction and compliance. evaluation & the health professions investigating decision factors in mobile application purchase: a mixed-methods approach. information and management the educational effects of mobile learning on students of medical sciences: a systematic review in experimental studies the promise and perversity of perspective-taking in organizations coping with smart phone 'distractions' in a college classroom a systematic review of the associations between empathy measures and patient outcomes in cancer care journal article reporting standards for qualitative primary, qualitative, meta-analytic, and mixed methods research in psychology: the apa publications and communications board task force report naturalistic inquiry impact of patient smoking behaviour on empathic helping by family caregivers in lung cancer development of a novel empathy-related video-feedback intervention to improve empathic accuracy of nursing students: a pilot study heart health whispering: a randomized, controlled pilot study to promote nursing student perspective-taking on carers' unhealthy lifestyle choices empathic accuracy in a clinically relevant setting critical analysis of strategies for determining rigor in qualitative inquiry the use of smartphone applications by urology trainees. the surgeon ipads in higher education -hype and hope communication-related behaviour change techniques used in face-to-face lifestyle interventions in primary care: a systematic review of the literature using smartphones as essential tools for learning smartphones and mobile applications (apps) in clinical nursing education: a student perspective building better workplaces through individual perspective taking: a fresh look at a fundamental human process thousand oaks nursing therapeutics: teaching student nurses care, compassion and empathy the coding manual for qualitative researchers empathic accuracy and relationship satisfaction: a meta-analytic review the impact of digital mobile devices in higher education false-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant a 21 word solution tax filers and dependants with income by total income, sex and age product testing survey template the efficacy of empathy training: a meta-analysis of randomized controlled trials mind-reading accuracy in intimate relationships: assessing the roles of the relationship, the target, and the judge empathy or science? empathy explains physical science enrollment for men and women the role of cognitive and affective empathy in spouses' support interactions: an observational study the case for and against perspective-taking distinct effects of imagine-other versus imagine-self perspective-taking on prejudice reduction acceptance of mobile technologies and m-learning in higher education learning: an explorative study at the faculty of humanities and social science at the university of zagreb students' perception of technology use in nursing education publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-330318-2v2exya7 authors: chua, amelia ze; lo, daryl yk; ho, wilbert hh; koh, yun qing; lim, daniel sy; tam, john kc; liaw, sok ying; koh, gerald ch title: the effectiveness of a shared conference experience in improving undergraduate medical and nursing students’ attitudes towards inter-professional education in an asian country: a before and after study date: 2015-12-23 journal: bmc med educ doi: 10.1186/s12909-015-0509-9 sha: doc_id: 330318 cord_uid: 2v2exya7 background: in recent years, increasing emphasis has been placed on the importance of collaboration within multi-disciplinary healthcare teams, so as to facilitate holistic patient care and thus allow improved treatment outcomes. there is hence an urgent need to educate healthcare undergraduates early in their professional careers on the importance of and complexities involved in cooperating with counterparts from other allied healthcare professions. in conjunction with this, a milestone student-led conference for undergraduate students, the 9th student medical-nursing education conference (smec), was organised in 2013 to provide a unique opportunity for shared learning among the entire cohort of undergraduate medical and nursing students in singapore matriculating in that year. methods: this study evaluated the effectiveness of the 9th smec 2013 as a shared conference experience in improving the attitudes of undergraduate medical and nursing students in singapore towards inter-professional education (ipe). a 19-point readiness for inter-professional learning scale (ripls) questionnaire comprising three subscales was administered to participants both before and after the conference. 352 responses were collected, giving a response rate of 75.1 %. results were analysed using paired-samples t-tests with statistical significance set at p = 0.05. results: improvements in overall scores for both medical and nursing students were reported for all three ripls subscales. examining the ripls items individually, significant improvement in scores for both medical and nursing students was obtained in all 19 items. prior exposure to ipe activities was not a predictor of improvement in ipe attitudes. conclusion: the authors propose that student-led jointly-organised conference experiences are effective in improving healthcare students’ attitudes towards ipe. this study provides valuable insights to facilitate the development of further ipe programs to allow for the rapid and effective promotion of cooperation and collaboration between students across various healthcare disciplines. traditionally, doctors have been trained to be self-reliant and independent, with the profession relying more on expertise, autonomy and responsibility rather than interdependence, deliberation and dialogue [1] . in recent years however, increasing focus has been placed on the importance of team-based care and collaboration between various healthcare professionals. critical to this shift is the advent of inter-professional education (ipe). ipe can be defined as an "educational process through which students and practitioners are provided with structured opportunities for 'shared learning'" [2] , allowing healthcare students to understand the intricacies of working together with members of other healthcare professions. "working together" involves "acknowledging that all participants bring equally valid knowledge and expertise from their professional and personal experiences", and can result in novel methods of problem solving [1] , improving the effectiveness of patient care in the process while also allowing for superior treatment outcomes. the literature suggests that ipe at the level of undergraduate learning could translate to improved working relations and understanding between the different healthcare professions. it is recommended that ipe be introduced early in the commencement of undergraduate healthcare courses, as this may help amend negative attitudes and avoid the formation of stereotypical views [2] [3] [4] . medical and nursing students in singapore have in fact responded positively towards the concept of incorporating ipe into their professional education [3] . it was hence decided that the 9 th student medical education conference (smec), the only student-led healthcare-focused conference for undergraduate medical students in singapore, would be expanded to encompass both medical and nursing disciplines. the resultant 9 th student medical-nursing education conference 2013 (9 th smec 2013) was aptly accorded the theme of "under one roof", providing a milestone joint ipe event for first-year undergraduate medical and nursing students. in this paper, the authors present an assessment of the effectiveness of the 9 th smec 2013 in improving attitudes of conference participants towards ipe. the 9 th smec 2013 was held for first-year medical and nursing students at the very start of the 2013-2014 academic year in august. notably, with the opening of singapore's second undergraduate medical school that same year, the conference was able to transcend institutional boundaries as well, ultimately reaching out to all matriculating undergraduate medical and nursing students across the nation. participants encompassed undergraduate medical students from the yong loo lin school of medicine, national university of singapore and the lee kong chian school of medicine, nanyang technological university, as well as undergraduate nursing students from the alice lee center for nursing studies, yong loo lin school of medicine, national university of singapore. with this, the programme of the conference was especially tailored to ensure that all participants were able to gain insight from qualified professionals and educators in both the medical and nursing fields. table 1 provides details on the ipe-focused plenary session and workshops that comprised the bulk of the conference. the conference was assessed via the administration of the readiness for inter-professional learning scale (ripls) to all conference participants. the ripls was originally formulated in 1999 by parsell and bligh [5] as a 19-item questionnaire consisting of 3 subscales (teamwork and collaboration; professional identity; and roles and responsibilities). as the first instrument designed to evaluate the "readiness" of healthcare students for shared activities, the ripls allows educators to quantify the impact of interventions on healthcare students [6, 7] . ripls has subsequently been proven to be a valid and useful tool for measuring student attitudes towards ipe in the undergraduate context [8] . for each item, participants were asked to provide their response using a likert scale with 1 representing "strongly disagree" and 5 representing "strongly agree". the questionnaire was administered twice to all participants -before and after the ipe components of the conference (prior to the plenary session and after the small-group workshops) to determine the effectiveness of the conference in improving students' attitudes towards ipe. participation in this study was voluntary, with consent taken after provision of a participation information sheet containing details of the study. the study was approved by the national university of singapore institutional review board. six of the 19 items in the ripls were negatively worded in the survey form; however for the sake of presentation, the scores recorded in this paper are such that a higher score is always indicative of a more positive attitude towards ipe. cronbach alpha values were calculated to determine the internal consistency of the ripls instrument in our study population. paired-samples t-tests were employed for each of the 19 items, as well as the 3 subscale scores and overall total score in order to evaluate changes in the conference participants' attitudes towards ipe between before and after the conference. chi-squared (χ 2 ) tests were used to compare ripls scores between those who had prior exposure to ipe versus those who did not, so as to evaluate the effect of prior exposure to ipe on changes in ripls scores before and after the conference. statistical significance was set at the conventional p < 0.05. a total of 352 responses were collected out of a total possible 469. 81.6 % of medical students from the yong loo lin school of medicine, 68.5 % of medical students from the lee kong chian school of medicine and 61.2 % of nursing students from the alice lee center for nursing studies responded, giving an overall response rate of 75.1 %. the demographics of the students who responded to the survey are illustrated in table 2 . the internal reliability of the pre and post-conference questionnaires was assessed separately. cronbach's α coefficients of 0.809 and 0.861 respectively were obtained, indicating a high internal consistency of the ripls questionnaire used. the results obtained from the participants' responses are shown in table 3 with respondents stratified according to their course of study (medicine or nursing). improvements in overall scores for both medical and nursing students were observed for all three ripls subscales. the scores for both medical and nursing students also improved significantly for all 19 individual ripls items. prior exposure to ipe activities was not a predictor of improvement in ipe attitudes. thirty-seven conference participants were found to have had previous exposure to ipe activities. healthcare students who had undergone previous ipe experiences had a significantly higher baseline score (pre-conference) as compared to those without such experiences. however, there was no significant difference in the improvement in scores between those who had prior exposure to ipe and those who did not. results obtained for all 3 ripls subscales showed overall significant improvements in scores, indicating that the 9 th smec 2013 was effective in improving the attitudes of singaporean healthcare students towards ipe. notably, these improvements were obtained for both medical students as well as nursing students, implying that the 9th smec was not only able to improve attitudes towards ipe in both groups, but was also able to promote the importance of teamwork specifically between these two healthcare professions. looking individually at the 19 ripls items; with the exception of a minority of questions, the baseline scores for both medical and nursing students were already high prior to the conference and additionally showed statistically significant improvements post-conference. this overall positive result is extremely encouraging as it not only small-group workshops participants were given the opportunity to select the workshop that interested them the most out of a choice of 15. each workshop was co-facilitated by at least one doctor and one nurse; with some facilitators inviting additional colleagues to share at the session. facilitators were given the freedom to conduct each workshop in any preferred format, but with the guideline that focus should be on the teamwork and cooperation between various healthcare professionals in daily practice. activities chosen by the facilitators ranged widely from video presentations to role-playing and group discussions. many also chose to use examples from key events in the history of healthcare in singapore, such as the 2003 epidemic of severe acute respiratory syndrome (sars). it was encouraging that all facilitators displayed a great keenness to share their thoughts on ipe with the conference participants. the 15 workshop topics were as follows: (additional non-ipe events held during the conference included a symposium on "surviving medical and nursing school", as well as a scientific poster competition and a symposium on hospital residency for senior medical students.) indicates that ipe activities are highly effective in improving students' attitudes, but also suggests that singaporean healthcare students display a high readiness to participate in and learn from such activities. this reflects the findings of studies on ipe activities in other countries [7] . with regards to question 3 of subscale 1 (see table 3 ); the baseline scores for both medical and nursing students were high. however, while the cohort of medical students who participated showed improved attitudes towards ipe as a whole, it is noted that for this particular question, only the score for nursing students showed a significant improvement post-conference, whereas that for medical students showed only a slight increase that was not statistically significant. we hypothesize that this result is due to medical students' perception of doctors as being preeminent members of the healthcare team who work independently, such that medical students accordingly have a tendency to view teamwork and collaboration with less importance as compared to other healthcare students. this appears to be a global phenomenon, with studies from new zealand, the united arab emirates and sweden reporting similar results [6, 9, 10] . notably, the literature indicates that such perceptions extend past graduation into working life as well, with doctors valuing teamwork less as compared to other healthcare professionals such as nurses and pharmacists [8] . it is thus particularly important to correct this perception when attempting to improve medical students' attitudes towards ipe. the authors hence suggest that additional studies are necessary to further elicit the detailed reasons for the prevalence among medical students of these specific perceptions as well as to identify the best ways of correcting such beliefs within the context of ipe. healthcare students who had undergone previous ipe experiences had a significantly higher baseline score (pre-conference) as compared to those without such experiences. however, there was no significant difference in improvement in ripls scores between those who had prior exposure to ipe compared to those who did not. this suggests that the 9 th smec 2013 was able to prove insightful by the same extent even to those who had prior exposures to ipe. it is hence worth considering whether participation in multiple ipe events would allow one to have a linear or even an exponential increase in the extent to which students are able to appreciate and work with members of other healthcare professions. a notable aspect of this study is that almost the entire cohort of singapore undergraduate medical and nursing students matriculating in 2013 was surveyed. with participants comprising the majority of first-year students from both undergraduate medical and nursing schools in singapore, and coupled with the high response rate of 75.1 %, this study provides a comprehensive indication of both the attitudes of local singaporean medical and nursing students towards ipe, as well as the potential effectiveness of ipe initiatives in the singaporean context. as the 9 th smec 2013 was one of the few healthcare conferences that are organised for students, by students, the results of this study suggest that student-run initiatives can be highly effective in improving attitudes towards ipe. given that ultimately, healthcare students make up the target audience of these initiatives, it is sensible that such initiatives be organised by fellow students who are most equipped to identify the needs of their current generation; being the most able to organise the fulfilling, informative and enjoyable ipe experiences for their peers. this is supported by literature which has concluded that student-run initiatives should not just be considered to be nonessential electives, but as cherished events that will fuel the growth of ipe [11] . perhaps another factor contributing to the effectiveness of the 9 th smec 2013 was the timing at which it was conducted; specifically being approximately one week into the beginning of university education for the undergraduate student participants. holding ipe events at such an early stage of education makes it easier to avoid the phenomenon of students pigeonholing members of other healthcare professions, a phenomenon extremely common in post-graduate education should students not have previously participated in ipe [9] . studies have in fact shown that holding ipe events in post-graduate education is much less effective, due to healthcare professionals having already developed their professional identity and hence holding more rigid views on inter-professional collaboration [5, 6, 12] . within the period of undergraduate education itself, the literature in fact indicates that students who have just joined health profession courses are more receptive to ipe as compared to students who are nearing the end of their course [13] [14] [15] . it is acknowledged that there are limitations to the study. it is noted that this paper is unable to evaluate the long-term impact of the conference on participants' ipe attitudes post-graduation. at the time of writing, it was not possible to elicit this as the cohort surveyed is yet to graduate for several more years. however, the authors suggest that given the long timeframe, any follow-up studies will be limited in significance, as the scope of further ipe and clinical exposures that will affect participants' attitudes are likely to vary significantly among participants by the time of graduation. hence it would very likely be inappropriate for any such studies to draw direct links between the effects of the conference in first year and ipe attitudes post-graduation. nevertheless, the inability to extrapolate the results presented here to predict changes in the long-run does not preclude the fact that the conference was able to have a significant impact on participants. rather, the authors suggest that the possibility of such impacts having an effect only in the shortterm means that further ipe exposures throughout the course of undergraduate education may be necessary to supplement the shared conference experience described in this study. additionally, the vast majority of conference participants were of the same age; hence it may not be appropriate to generalize our results to graduate medical and nursing students, or students from non-asian countries. the use of open-ended questions in addition to multiple-choice questions in the questionnaire could possibly be useful in allowing for a more comprehensive approach when studying this aspect in the future [16, 17] . a further extension to this study could be to consider if a similar conference targeted at other combinations of healthcare professions such as dentistry, pharmacy and occupational therapy for example would prove to be as effective. the authors also suggest that while ipe is known to have an important impact on improving patient care, it would be interesting to further explore the ways in which an improvement in ipe attitudes affects healthcare professionals' specific working practices, possibly allowing for more targeted interventions to be implemented. our study found that participation in a student-led jointly-organised conference event was effective in improving medical and nursing students' improve attitudes towards ipe. for students barely into the first year of healthcare education, such an ipe event is extremely important and vital to developing a readiness to work with other members of the healthcare profession, a very important quality to possess especially in light of today's scientific advancements that herald interdependent healthcare cooperation. ipe is a relatively new pedagogical tool in medical and nursing education, thus further research should be undertaken to elicit more ways in which ipe can be incorporated into the curriculum. it is hoped that this paper will assist others in conveying the concepts of fostering teamwork, collaboration, an awareness of one's roles and responsibilities as well as uniqueness of one's discipline through their own ipe experiences. getting health professionals to work together interprofessional learning are first-year healthcare undergraduates at an asian university ready for interprofessional education? the need for multiprofessional health education in undergraduate studies the development of a questionnaire to assess the readiness of health care students for interprofessional learning (ripls) multiprofessional learning: the attitudes of medical, nursing and pharmacy students to shared learning a comparison of student attitudes and perceptions before and after an introductory interprofessional education experience validating the readiness for interprofessional learning scale (ripls) in the postgraduate context: are health care professionals ready for ipl? are senior uae medical and nursing students ready for interprofessional learning? validating the ripl scale in a middle eastern context are female students in general and nursing students more ready for teamwork and interprofessional collaboration in healthcare student leadership in interprofessional education: benefits, challenges and implications for educators, researchers and policymakers exploring the value of interprofessional shared learning the test-retest reliability of a revised version of the readiness for collaborative learning for collaborative working? initial findings from a longitudinal study of health and social care students from students to professionals: results of a longitudinal study of attitudes to pre-qualifying collaborative learning and working in health and social care in the united kingdom research design: qualitative, quantitative, and mixed methods approaches qualitative inquiry and research design: choosing among five approaches the authors would like to thank the following members of the smec 2013 organising committee who contributed in data collection and entry: douglas wc chee 4 , eunice jh goh 4 , lee hanyi 4 , joshua sh lim 4 , ooi tong li 4 , chloe pawa 4 , nur khairunnizzafiqah 5 , tee shi yun 5 . 4 yong loo lin school of medicine, national university of singapore, national university health system, singapore 5 alice lee center for nursing studies, yong loo lin school of medicine, national university of singapore, national university health system, singapore the authors would also like to thank the medical education unit, yong loo lin school of medicine, national university of singapore, national university health system, singapore, for helping to facilitate the running of the 9 th smec 2013. the authors declare that they have no competing interest. the authors alone are responsible for the content and writing of the article.authors' contributions gchk, jkct and lsy contributed to the concept and design of the study. azec, dykl, whhh, kyq and dsyl acquired the data. gchk, azec, dykl, whhh, kyq and dsyl analysed the data and all authors interpreted the data. azec, dykl, whhh, kyq and dsyl drafted the manuscript and gchk, jkct and lsy revised it critically for important intellectual content. all authors approved the final manuscript and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.• we accept pre-submission inquiries • our selector tool helps you to find the most relevant journal submit your next manuscript to biomed central and we will help you at every step: key: cord-302305-xr067v2n authors: van aalst, jan title: distinguishing knowledge-sharing, knowledge-construction, and knowledge-creation discourses date: 2009-06-20 journal: int j comput support collab learn doi: 10.1007/s11412-009-9069-5 sha: doc_id: 302305 cord_uid: xr067v2n the study reported here sought to obtain the clear articulation of asynchronous computer-mediated discourse needed for carl bereiter and marlene scardamalia’s knowledge-creation model. distinctions were set up between three modes of discourse: knowledge sharing, knowledge construction, and knowledge creation. these were applied to the asynchronous online discourses of four groups of secondary school students (40 students in total) who studied aspects of an outbreak of severe acute respiratory syndrome (sars) and related topics. the participants completed a pretest of relevant knowledge and a collaborative summary note in knowledge forum, in which they self-assessed their collective knowledge advances. a coding scheme was then developed and applied to the group discourses to obtain a possible explanation of the between-group differences in the performance of the summary notes and examine the discourses as examples of the three modes. the findings indicate that the group with the best summary note was involved in a threshold knowledge-creation discourse. of the other groups, one engaged in a knowledge-sharing discourse and the discourses of other two groups were hybrids of all three modes. several strategies for cultivating knowledge-creation discourse are proposed. for two decades, carl bereiter and marlene scardamalia have been developing an educational model intended to make the processes experts use to advance the state of knowledge in their fields more common in education. the model was initially called "intentional learning" to emphasize that learning needs to be an intended goal rather than the by-product of activities (bereiter and scardamalia 1989) and then "knowledge building," suggesting that knowledge is the product of a constructive process (bereiter and scardamalia 1993) . but as constructivism has gained wide receiver if they comprehend its content and significance. examples are providing factual information to answer a query or uploading various kinds of information to an intranet. one thing that makes such interactions effective is that the receiver has already identified a need for the information. for example, someone new to editing digital video may need to be shown how to add music to the video, which will address an already meaningful goal. as a social practice, knowledge sharing is an accomplishment, especially in competitive environments; people are not naturally inclined to share what they know unless doing so is likely to enhance their own social position. the management literature indicates that knowledge-sharing practices can make organizations more effective, but they need to be cultivated (lencioni 2002) . in a community engaged in collaborative inquiry, knowledgesharing practices involve the introduction of information and ideas without paying extensive attention to their interpretation, evaluation, and development. the perceived lack of a need for interpretation and evaluation can be related to naïve realism, an epistemic position according to which data speak for themselves (science council of canada 1984) . a related epistemic belief is "quick learning," which has been linked to overconfidence in knowledge (schommer 1990 ). the ideas shared are not modified by the sharing interaction (bereiter and scardamalia 1987; pea 1994) , and knowledge sharing is not reflective. knowledge construction refers to the processes by which students solve problems and construct understanding of concepts, phenomena, and situations, considered within cognitive psychology. it is effortful, situated, and reflective, and can be individual or social (sullivan palincsar 1998) . the basic assumption of constructivism is that the student must make ideas meaningful in relation to his or her prior knowledge and to the situation in which the need for ideas arises (von glasersfeld 1995) . the cognitive processes are "situated" because they are mediated (enabled) by social interactions within the particular group that is working together and by the particular technologies used (brown et al. 1989; hutchins 1995) . knowledge construction is often associated with deep learning, which involves "qualitative changes in the complexity of students' thinking about and conceptualization of context-specific subject matter" (moore 2002, p. 27 ; also see biggs 1987) . dole and sinatra (1998) conceptualize the effort students invest in information processing as "engagement," ranging from simple processing that leads to assimilation (low), to deeper processing and some reflection that leads to knowledge restructuring (moderate), and on to substantially metacognitive processing (high). at moderate to high levels of engagement, knowledge construction can lead to the substantial restructuring of knowledge, which may include the invention of new concepts and enhanced meta-conceptual knowledge (e.g., knowledge about the hierarchical nature of networks of concepts). for example, students may initially consider the motion of an apple that falls from a tree to be unrelated to the motion of the earth in its orbit around the sun, but then come to realize that both can be described using the universal law of gravitation. this change would imply deeper insight into the nature of gravity and would lead to a restructuring of knowledge; the resulting knowledge structure would explain a greater range of observations and require fewer assumptions. more generally, synthesis that results in understanding phenomena on a higher plane and the creation of new concepts is an important form of knowledge advancement. for example, mendeleev's introduction of the periodic table of the elements accelerated progress in chemistry by predicting the existence of unobserved elements and the creation of new concepts to explain the partially observed patterns. scardamalia (2002) conceptualizes such advances as "rise-above," which she described as "working toward more inclusive principles and higher-level formulations of problems. it means learning to work with diversity, complexity and messiness, and out of that achieve new syntheses. by moving to higher planes of understanding knowledge [creators] transcend trivialities and oversimplifications and move beyond current best practices" (p. 79). although scardamalia proposes rise-above as a knowledge-creation principle, i regard it as a cognitive act whereby students articulate higher levels of understanding and not merely reorganize knowledge (gil-perez et al. 2002) ; nevertheless, the need for rise-above is greater when the need for synthesis is greater. knowledge construction involves a range of cognitive processes, including the use of explanation-seeking questions and problems, interpreting and evaluating new information, sharing, critiquing, and testing ideas at different levels (e.g., conjectures versus explanations that refer to concepts and/or causal mechanisms), and efforts to rise above current levels of explanation, including summarization, synthesis, and the creation of new concepts. however, educational approaches vary considerably in the extent to which they make it possible for students to engage in these processes. although most emphasize working with information and ideas (e.g. goldberg and bendall 1995; hunt and minstrell 1996; linn et al. 2003) , there may be limited opportunities for students to pursue problems they have identified themselves or to synthesize ideas and formulate new concepts. for example, in problem-based learning (hmelo-silver and barrows 2008), students are provided problems, although these are ill-structured and need considerable articulation. in other approaches, students may collaborate in small groups on relatively simple tasks that require little synthesis and reflection on progress. in the vast majority of approaches, knowledgeconstruction processes are directed at acquiring the reliable knowledge of a field (edelson et al. 1999; kolodner et al. 2003; krajcik et al. 2008) . knowledge construction, with its emphasis on building on students' prior ideas, concepts and explanations, and their metacognition, produces deeper knowledge in complex domains than does knowledge sharing (bransford et al. 1999; hmelo-silver et al. 2007 ). the term "knowledge creation" is used in the literature on expertise and innovation to describe how companies, organizations, and academic fields develop the ideas needed to sustain innovation (e.g. engeström 2001; gundling 2000; nonaka and takeuchi 1995) . knowledge creation depends on conditions in which creative work on ideas is valued and there are mechanisms for choosing the most promising ideas for further development, and rewarding creativity. these elements need to work together to create what gundling (2000) has called an "ecology of innovation" that produces "a dazzling variety of new products each year" (p. 14). at one level, knowledge-creation discourse involves the design and improvement of intellectual artifacts such as theories, explanations, and proofs (bereiter 2002) . drawing from popper's theory of objective knowledge, bereiter considers ideas to be real objects similar to bicycles or telephones. we may ask how a bicycle can be improved, and we can ask the same of an idea. this aspect of the discourse is known as "design-mode" (bereiter and scardamalia 2003) , with an emphasis on explanations, casual mechanisms, and the coordination of claims and evidence. however, knowledge creation is not just a rational effort. for example, the community periodically needs discourse to identify priorities and long-term goals, decide how to mentor newcomers, and evaluate knowledge advances. as studies of scientific practice have shown, the associated discourse tends to be more argumentative (feyerabend 1975; kuhn 1970; lakatos 1970; latour 1987) . to mention just a few examples, in science, good problems may not be investigated because they are not currently considered important (latour 1987) . researchers promote their own work and that of close colleagues by alerting the community to recent findings, and may ignore important new findings that they do not find appealing (reeves 2008) . other researchers may not make their insights public, for fear of attracting criticism (e.g., madame curie's reluctance to make public the health hazards associated with radium, see quinn 1995) . in other words, belief-mode discourse also plays an important role in knowledge creation. despite individual idiosyncrasies, members of a scientific field share a goal of innovation and the advancement of knowledge. commitment to shared goals within a team is also important in a variety of other innovative contexts (gundling 2000; lencioni 2002; nonaka and takeuchi 1995) . in bereiter and scardamalia's knowledge-creation model (bereiter 2002; bereiter and scardamalia 1996; scardamalia 2002; scardamalia and bereiter 2006) , a class of students is considered a community that shares a commitment to creative work on ideas and advancement of the state of knowledge in that community. ideas are considered intellectual artifacts of the community; they reside in the community's discourse rather than in people's minds. the community needs to be able to identify gaps in its collective knowledge, map out ways to fill those gaps, design and manage inquiries, manage social processes, and evaluate progress. thus, the community's goals are emergent. students are expected to make "constructive use of authoritative sources" (scardamalia 2002) such as books, websites, and experiments, treating them as potentially useful for informing their work. they are also expected to engage in progressive problem solving, reinvesting cognitive resources to deepen their understanding of problems and taking on more difficult problems over time (bereiter and scardamalia 1993) . one of the most important roles of the teacher in this process is to facilitate the development of an innovation ecology. important progress has been made in this direction by the development of a system of principles that describe the socio-cognitive and socio-technological dynamics of knowledge creation, including collective cognitive responsibility for knowledge advancement, real ideas/authentic problems, epistemic agency, improvable ideas, rise-above, and constructive use of authoritative sources (scardamalia 2002) . these principles provide a technical vocabulary that students, teachers, and researchers can use to reflect on the extent to which there is evidence of a knowledge-creation discourse. initial studies show that elementary and secondary school students are capable of engaging in the dynamics described by these principles (niu and van aalst in press; zhang et al. 2007 zhang et al. , 2009 ). however, more work is needed to characterize the innovation ecology, such as by determining the social practices that make collaboration possible, the overall school culture, and the community's experience at knowledge creation and its long-term goals (bielaczyc 2006; truong 2008) . knowledge creation requires discourse for maintaining social relations, setting goals, deepening inquiry, and lending support to ideas that are already understood by some in the community. for example, van aalst (2006) discusses how a grade 6 student referred to the scientist francis bacon to support an explanation he had proposed earlier that had not been accepted by the community. this move was directed less at improving understanding than at improving the impact of the student's own ideas. similarly, students who wish to further a line of inquiry need the ability to argue the case for doing so. these types of moves cannot be understood by examining short-term goals such as the problem students are currently attempting to understand, but require the consideration of higher level and longer term goals such as the diffusion of new insight throughout the community and progressive problem solving (hmelo-silver 2003) . in groups that work together for short periods, there is less need for such moves. there are important theoretical differences between knowledge construction and knowledge creation, although they involve similar processes such as posing questions, formulating conjectures and explanations, summarizing progress, and proposing rise-above ideas. these processes are interpreted within different psychological perspectives. knowledge construction corresponds to cognitive psychology, in which improved understanding is regarded as the emergence of more complex cognitive structures and schemata (novak and gowin 1984) . such views have been criticized for their cartesian split between the knower and what is known, and for treating knowledge as residing in the mind. proponents of sociocultural theories posit knowing as the ability to participate in cultural practices (lave and wenger 1991; roth and tobin 2002) . for example, roth and tobin argue that "knowing physics … means to participate in talking about relevant objects and events in the ways physicists do, using acknowledged words, sentences, gestures, inscriptions, and so forth …" (p. 152). these developments have given rise to a division between learning as the acquisition of mental representations and learning as participation; sfard (1998) argues that both views are needed for a complete understanding of learning. brownell and sims propose a pragmatic and relational view of understanding implied by the ability to "act, feel, or think intelligently with respect to a situation" (1946 ( , quoted in bereiter 2002 , which bereiter uses to argue that understanding is always mediated by the object to be understood. accordingly, understanding has an "out-in-the-world" character. drawing from bereiter's analysis and work on expansive learning and knowledge-creating companies (engeström 2001; nonaka and takeuchi 1995) , paavola et al. (2004) propose a "knowledge creation metaphor" that further articulates this view. thus, understanding and knowing are mediated by the objects that a community creates and shares, and the cartesian split appears to be avoided. rather than residing inside individual minds, ideas are regarded as cultural objects (or artifacts) that mediate knowing and understanding. in summary, knowledge sharing, knowledge construction, and knowledge creation correspond to different theoretical perspectives. however, this does not mean that a community will use a single mode of discourse. for example, we would expect students to use a knowledge-sharing discourse when it meets their needs, and for there to be individual differences in epistemic beliefs and conceptions of learning that make the identification of a single discourse mode difficult. nevertheless, we can examine which discourse mode, in the balance, is most consistent with the observed discourse. the three modes of discourse can be supported by a wide variety of educational tools and activity structures, including online discussion forums (synchronous and asynchronous), mobile devices, face-to-face conversations, and lessons. this paper focuses on the use of an online discourse environment, knowledge forum. from a cognitive perspective, knowledge forum is designed to support knowledge construction through the use of scaffolds, which are sentence starters such as "my theory" that keep the writer and reader focused on cognitive processes. knowledge forum also has a variety of features that support working with ideas after they have been posted including: (a) the ability to revise notes; (b) the ability to add a note as a reference to another note; (c) the ability to reuse a note introduced in one workspace in a later workspace created for a different purpose (a workspace in knowledge forum is called a view for "point of view"); and (d) the ability to create rise-above notes, which have a special icon and are used to take the discourse to a higher conceptual plane. the ability to link notes is useful for making visually evident the connections between ideas. knowledge forum also makes it possible to objectify ideasto share them and then allow the community to work on them. the above-mentioned features then support the work of improving such objects, reviewing progress, and synthesis. the remainder of this paper reports a case study of asynchronous online discourse in knowledge forum using a coding scheme based on the distinctions between the three discourse modes. the data are drawn from a design experiment (brown 1992; collins et al. 2004) in which the researcher and teacher collaborated to achieve two goals: to achieve a fuller implementation of the knowledge-creation model than in previous iterations, and to test a new assessment strategy (van aalst et al. 2005) . the assessment task was designed to extend our previous work on portfolio notes, in which students had used concepts describing collective aspects of knowledge creation individually (lee et al. 2006 ; van aalst and chan 2007) . the new task was intended to underscore that knowledge advancements are collective achievements in a knowledge-creation community; it asked students to collaborate to review whether knowledge advances had been made on the problems they investigated and, if so, to coauthor a collaborative summary note with all who had contributed to the collective advance. while the work students did together throughout the project involved both the division of labor (cooperation) and joint activity to understand the same problems and ideas (collaboration), the word "collaborative" in the name of the task signified that students were to work together to review and create these notes (for details see van aalst et al. 2005) . the study evaluated performance on the collaborative summary notes and related that to what students were doing in knowledge forum. the unit of analysis was a group of students that worked together in the same workspaces (views) in knowledge forum; there were four such groups in the study (groups a-d). the analysis proceeded in five parts: (1) several relevant independent variables were examined to check whether the groups could be considered to be equivalent. (2) two dependent measures, knowledge quality and significance of findings, based on the collaborative summary notes, were measured to assess advances in collective knowledge made by the groups. (3) to identify mechanisms that could explain observed between-group differences in the dependent variables, the group discourses (all the notes written by each group) were coded and analyzed using a new coding scheme with 7 main codes and 33 subcodes. statistical analysis was then performed on the main code frequencies to determine which main codes provided the greatest group separation. (4) the results were used to select several main codes for qualitative analysis to further elucidate what the groups were doing differently. (5) the observed patterns in the subcode frequencies were used to examine the fit of the four group discourses to the knowledge-sharing, knowledge-construction, and knowledge-creation discourse modes. 2 the participants were two classes of secondary school students, from a grade 10 course on career preparation and inquiry (n=21) and a grade 11 course focusing on computers and their impact on "global society" (n=19). the courses were taught concurrently by the same teacher at an inner city school in western canada. approximately 40% of the students had some experience with knowledge forum in previous grades, such as in discussing "problems of the week" in mathematics. however, these experiences did not last more than one or 2 weeks and were not integrated into a pedagogical approach based on knowledge-creation principles. the teacher had 10 years of experience teaching secondary school mathematics. he had recently completed a master's degree focusing on cognitive strategy instruction and was in his third year of using knowledge forum. the researcher and teacher met several times at the beginning of the school year to plan the project, deciding that the then recent outbreaks of severe acute respiratory syndrome (sars) and avian flu in 2003 and 2004 could provide a suitable area of inquiry for secondary school students. for example, students could build on their knowledge of science to study what was known about these phenomena, critique media attention, examine the economic impact, or form a position on how governments should have responded to the outbreaks. the grade 10 course provided a promising context for integrating a focus on such questions into the curriculum, as one of its main goals was learning how to conduct research. the grade 11 course also provided a good opportunity to engage in knowledge creation, as one of its main goals was for students to learn how information and communication technology could be utilized for learning in global societies. the second main topic on the grade 11 course syllabus was "computer viruses," which was added to sars and avian flu as a third main topic for inquiry with the aim of having the students examine the nature of viruses in both biological and non-biological systems and identify patterns across them. (however, the topic only accounted for 11.5% of the coded notes.) the two classes shared a knowledge forum database and worked on the same topics. to limit the number of notes they would encounter, the students were divided into four groups. each group had students from both classes, with an equal number of students from each class; the students could choose their own groups but the teacher made some minor changes. each group had its own views on knowledge forum and the groups were not expected to interact with each other during the inquiry. in the week before the project commenced, all students responded to an icebreaker topic. the researcher then introduced both classes to knowledge-creation principles, and students were reminded of these by means of posters in their classrooms. both classes had daily access to a computer lab (70-minute periods), but students had a number of other assignments to complete. during typical periods, the teacher would spend 10 to 20 min interacting with the whole class, and the students would then work on one of their assignments. most of the students worked on knowledge forum during class a few times per week, and after school hours. the teacher discussed the students' work in knowledge forum with them from time to time, but he only read 23% of their notes and posted 7 of his own. the researcher visited the classes four times, and occasionally the teacher emailed the researcher to ask for advice on issues that arose during conversations with the students. because the teacher and students had little experience with extended and collaborative inquiry, a three-phase inquiry model was employed. phase 1 developed a focus, phase 2 was the main inquiry phase, and phase 3 involved the students evaluating what they had learned. the researcher provided extensive instructions for the three phases as outlined below. phase 1: orientation (2 weeks) the goal of the first phase was to enable the students to identify problems and select the most promising inquiry foci. research into inquiry-based learning has shown that the nature of students' own questions constrains student-led inquiry (krajcik et al. 1998; lipponen 2000; polman 2000) . the students were thus asked to read widely and post notes in their group's view, summarizing the main points and raising questions and ideas. toward the end of phase 1, they were asked to propose problems of understanding, using a research question note format stating the question, its background (relation to earlier notes), and ideas for studying the question. finally, they were asked to select a few of the most promising problems for further research, considering: (a) the extent to which a question might lend itself to inquiry worth several weeks of effort, (b) whether they had ideas or resources for researching the question, and (c) the coherence among the questions that were under consideration. the researcher explained the rationale for these processes and related it to knowledge-creation principles. phase 2: research (4 weeks) the students were asked to create a view in knowledge forum for each research question. they were then expected to work within their groups to research their problems by reading additional information on the internet and from other sources. the students were encouraged to evaluate the credibility of the sources (e.g., the world health organization website would be a more trustworthy source than writing by a person who did not declare his or her credentials), and to examine the evidence used to support the claims made in the sources. they were encouraged to extend their inquiries after they developed preliminary answers to deepen their understanding. the researcher and teacher were less involved in scaffolding the inquiry than in phase 1. phase 3: evaluation of learning (2 weeks) as knowledge advancement is an important outcome of knowledge creation, each group was asked to create a collaborative summary note for the problems on which progress had been made by the end of phase 2. the students began their review face-to-face within their own group and class, and then created coauthored notes in knowledge forum; in the best examples, the coauthors then edited the notes to gradually improve them. the note format was similar to a brief scientific research report, with the groups asked to (a) state the problem on which they were reporting, (b) explain the problem's background, with links to their work in phase 1, (c) describe what they did to investigate the problem, (d) report the main findings, and (e) explain the significance of the findings and outline opportunities for further inquiry. the instructions also indicated that a student could be coauthor of several summary notes. the notes were designed as self-assessments of group accomplishment but were not used by the teacher for formal assessment. to guide their work, the students were provided a rubric showing several dimensions of the task (writing quality, identification of collaborators, organization, findings, and implications) with levels of performance for each (van aalst et al. 2005) . the following baseline data were collected to examine the extent to which the groups could be considered equivalent in terms of their opportunities to create knowledge: prior knowledge relevant to the inquiry topics, general indexes of participation in knowledge forum, and the research questions proposed. a short test with eight questions was administered at the beginning of the project to assess existing knowledge of sars and avian flu. the questions asked students to describe their knowledge of sars, the corona virus, and what measures had been taken to control it; one asked whether a nurse should enter a hospital ward with sars patients, and another asked what students knew about avian flu. each question was scored on a 0-3 scale, ranging from "no domain knowledge evident" to "at least two relevant points." for example, in a response that received a score of "3" for knowledge of avian flu, a student stated that "it was the same thing as bird flu," which she further explained as follows: "the birds get the flu because they have to live in small spaces where bacteria grow and become more dangerous." the scores were added to create a scale with a range from 0 to 24 points. the papers were scored by the researcher; 50% of the papers were also scored independently by a research assistant resulting in an inter-rater reliability of .88 (pearson correlation). general indexes of participation in knowledge forum-notes created, percentage of notes read, and percentage of notes linked-were obtained using the analytic toolkit (atk) for knowledge forum; these kinds of measures have been used in many studies of online discourse (guzdial and turns 2000; hsi and hoadley 1997; lee et al. 2006 ; van aalst and chan 2007; zhang et al. 2007) . while high values of all three measures are not necessarily indicative of knowledge construction or knowledge creation, the measures can be informative. for example, a low percentage of notes read would suggest a low level of awareness of ideas in the database. conversely, a high percentage of linked notes could indicate attempts to synthesize and integrate contributions. these measures are correlated with both performance on self-assessment tasks and knowledge advancement, although such effects are contingent on the discourse being explanation-driven (niu and van aalst in press). the potential for knowledge advancement is also influenced by the nature of the research questions posed. do they require explanations or will descriptive information suffice? do the students have relevant knowledge that they can apply? the research question notes posed in phase 1 were thus checked to determine whether all groups posed some explanation-seeking questions and questions that related to prior learning. two dependent variables were derived from the collaborative summary notes. the knowledge quality scale measured: (a) an epistemic position ranging from knowledge as a single factual claim to a fully integrated explanation in which several concepts and/or causal mechanisms were invoked (hakkarainen et al. 2002) ; and (b) the extent and correctness of knowledge from a single finding, possibly with evidence of misconceptions, to at least three findings without evidence of misconceptions. the significance of findings scale was intended to measure the students' ability to identify the significance of what they had learned, ranging from a brief restatement of their findings to a clear explanation of the significance, limitations, and potential for further inquiry. self-assessment of the significance of learning is a metacognitive ability needed for knowledge construction and knowledge creation, especially for setting new learning goals. the descriptors for each point on these two scales are shown in table 1 . all summary notes were scored independently by the researcher and a research assistant who had completed a course on knowledge creation but was not familiar with the database. the inter-rater reliability was .85 for knowledge quality and .82 for implications of findings (pearson correlation coefficients). a coding scheme was developed for analyzing the group discourses during phase 1 and phase 2. the goal of the analysis was to identify mechanisms that could explain betweengroup differences in the dependent variables. the scheme was intended to be general enough for use in analyzing discourse from a variety of perspectives within the computersupported collaborative learning field, particularly knowledge sharing, knowledge construction, and knowledge creation. it includes seven main codes: community, ideas, questions, information, links, agency, and meta-discourse. the community code describes the extent to which the social interactions within a group suggest a "sense of community," in which "people feel they will be treated sympathetically by their fellows, seems to be a first necessary step for collaborative learning" (wegeriff 1998, as quoted in kirschner and kreijns 2005, p. 176 ). indicators of a sense of community include commitment to shared goals, appreciation for the work of group members, identification with the group, and ways of getting things done that are specific to the group (wenger 1998) . discourse that involves risk-taking requires a stronger sense of community than other types of discourse (e.g., improving ideas versus only sharing them). although the knowledge-creation model refers to communities, the discussion in its literature has been limited to the socio-cognitive features of those communities. the next five main codes-ideas, questions, information, links, and agency-are based on research into a wide variety of cognitively oriented inquiry approaches (chan 2001; hakkarainen 2003; hakkarainen et al. 2002; hmelo-silver 2004; kolodner et al. 2003; linn et al. 2003) . this body of work has shown that a focus on explanation is more likely to lead to knowledge advancement than answering fact-seeking questions (hakkarainen 2003) . the idea code captures the ways in which students contribute to and work on ideas (e.g., opinions, conjectures, and explanations), with its focus on the nature of those ideas. in contrast, the information code focuses on the extent to which students interpret or evaluate the information they introduce. the agency code is intended to describe the ways in which students selfregulate their inquiries; the subcodes emphasize planning and reflection relating to logistics and the epistemic features of their inquiries. in terms of these codes, we would expect informationsharing discourse to be characterized by fact-seeking questions and limited evidence of ideation, interpretation of information, synthesis, and planning and reflection. in contrast, both knowledge construction and knowledge creation would be characterized by stronger evidence in these areas, with minor differences between the two modes of discourse. for example, although rise-above should occur in knowledge construction, it should occur more often in knowledge creation, which takes place over a longer period and has greater need for synthesis. the final main code, meta-discourse, describes a level of discourse beyond maintaining social relations and building understanding, and relates to the existence of long-range goals in a knowledge-creation community. scardamalia and bereiter (2006) suggest that this feature is lacking in most online discussions. examples of meta-discourse would be reviews of the state of knowledge in the community, work aimed at helping new insights diffuse through the community, making arguments for a new phase of inquiry, and establishing more difficult goals over time. although evidence of meta-discourse may not be strong in an inquiry of 8 weeks, there should be some examples. to capture the different ways the seven codes could be exemplified, 33 subcodes were identified and their relevance to each of the discourse modes estimated (see table 4 in the "results" section). for these estimates, a three-point rating scale was used (low, medium, high). for example, the subcode fact (under ideas) was rated high for knowledge sharing and low for both knowledge construction and knowledge creation. in this example, knowledge construction and knowledge creation are called degenerate to indicate that the scale for this code does not differentiate between them. major review (under meta-discourse) was rated low for knowledge sharing, medium for knowledge construction, and high for knowledge creation on the assumption that knowledge creation is generally more complex and requires more time than knowledge construction, so the need for major review is greater. all ratings were completed independently by the researcher and an independent second rater, leading to an inter-rater reliability of .82 (cohen kappa). the computer notes were entered into atlas-ti® qualitative data analysis (qda) software for coding; 399 notes were coded (approximately 60,000 words). each view in knowledge forum was entered separately, beginning with the first view of group a and ending with the last view of group d. most of the development of the coding scheme was done using the data from groups a and b. the researcher started with a small set of codes based on knowledgecreation principles and prior research into asynchronous discourse, and gradually expanded the set. he started by focusing on the text, and applied each code that seemed relevant to a given text segment; the amount of text varied from a sentence to a few notes depending on the code (hmelo-silver 2003). the process was then repeated focusing on the codes and working through the corpus checking for potential examples for small groups of codes. the researcher began by coding data from groups a and b, and reflexively improved both the code definitions and coding procedures. it soon became clear that coding was needed for both the nature of the idea (e.g., conjecture or explanation) and the extent to which the students processed new information. after three rounds of improving the code definitions and procedures in groups a and b, the codes were organized into main codes and subcodes and the remaining data were coded. as employing a second coder was not possible, to further ensure the accuracy of the coding the researcher returned to it after an absence of approximately 3 months. the qda software was then used to check the consistency of subcode allocations, with 12% of the quotes needing to be recoded. most changes were between subcodes of the same main code (e.g., switching from "opinion" to "conjecture"). the coding results were analyzed in three ways. first, a frequency analysis was conducted to examine the extent to which each main code could be used to separate the four groups. the goal of this analysis was to identify potential mechanisms that could explain betweengroup differences for knowledge quality and implications of findings. next, several of the main codes were selected for qualitative analysis to further elucidate what students were doing in knowledge forum. main codes were selected for this analysis based on the amount of group separation. finally, the alignment of the subcodes with the three discourse modes allowed the mapping of the four group discourses onto those modes. the goal of the first analysis was to determine whether the four groups could be considered equivalent in subsequent analyses. table 2 shows the results for the knowledge pretest and atk indexes. the pretest results show that prior content knowledge was not extensive and varied very little between the four groups; the group means varied from 45.4% (group c) to 50.0% (group b). the majority of students (55%) stated at least two substantive points about sars, but 75% stated they knew nothing about the corona virus, and 60% stated they knew nothing about avian flu. between-group differences were also relatively minor for the atk indexes, the most noticeable being that the students in groups c and d read fewer notes. overall, the amounts of note writing and reading were consistent with those in other studies of online discourse (guzdial and turns 2000; hsi and hoadley 1997) . in contrast, the amount of linking (40.9% to 50.4%) was less than in other studies using knowledge forum, in which it reached 80% (lee et al. 2006; yoon 2008) . each group posted approximately 10 research question notes, although group c required 8 days longer than the others to reach this point. each group's output included some explanation-seeking questions, such as "why is it children are less likely to develop sars?" there were, however, important differences in the extent to which the questions allowed the students to build on prior knowledge. for example, while discussing the question about sars and children, the students used their knowledge of viruses and infection, but in discussing "is killing chickens the only way to end avian flu" they resorted to exchanging opinions. in sum, these data suggest that the four groups were similar in terms of prior knowledge about the main inquiry topics, the extent to which they used knowledge forum, and their ability to formulate research questions. however, group c had fallen behind the other groups by the time it had generated its research questions, and the research questions varied in their potential for knowledge creation. the goal of the second analysis was to evaluate the advances in collective knowledge reported by those students who collaborated on summary notes. the students collectively submitted 32 summary notes; 81.0% of grade 10 students and 84.2% of grade 11 students were coauthors of at least one note. all of the summary notes were assessed for knowledge quality and implications of findings with the scales shown in table 1 . table 3 shows the group means and standard errors for knowledge quality and significance of findings for the 32 summary notes. some students did not realize that a group was required to write only one note on a given research question, resulting in duplicate notes for some questions; in such cases, only the best note from the group was considered in the calculation of group means. group a had a higher mean score than the other groups for knowledge quality (effect sizes ≥ 0.7, cohen's d); for most groups. the knowledge gained was factual and did not reach the level needed for a 3 or 4 on the scale. group c had the lowest mean knowledge quality score; its small number of notes is understandable because it needed more time to articulate its focus. the goal of the third analysis was to identify possible mechanisms for the between-group differences in the dependent variables by coding the group discourses leading up to the creation of the summary notes. the code and subcode frequencies are shown in table 4 . the total frequencies for all subcodes associated with a main code are shown in the first row of each section. before examining intergroup variation, it will be useful to consider the total frequencies over all groups (last column). in descending order of total frequency, the following patterns can be observed. first, although there were many linkages (f=206), there were few examples (8) in which features of knowledge forum such as adding a note as a reference to another note were used; the majority of links were to web pages (106), although some groups did link their ideas verbally to earlier contributions in knowledge forum (66). this finding suggests that the level of competence with features of knowledge forum designed to support linking ideas was low, and may explain the lower than expected atk index for linking (table 2) . second, although there were many instances of working with ideas (171) and information (124), the subcodes suggest that information sharing was a significant aspect of all group discourses. third, there were few instances of two codes: questions and meta-discourse (both 65). group a had substantially more code instances than the other groups (329, compared with 181, 165, and 165) reflecting that it invested more effort into the processes measured by the coding scheme, which may explain the better performance on the summary notes to some extent. however, a more interesting question is what group a did differently, controlling for the difference in overall effort. thus, a two-way analysis of the main code frequencies (code × group) was conducted. the results are shown in table 5 ; cohen's guidelines regarding effect sizes were used to arrange the codes in three groups from large to small effect size. accordingly, community and questions were very effective in separating the groups-especially group a from the other groups. the next two codes (ideas, information) provided statistically significant but more moderate separation. the last three codes (linking, agency, meta-discourse) provided limited or non-significant group separation. the relatively low frequencies for these codes indicate that these aspects of the discourse were generally not well developed. to gain additional understanding of the nature of the group discourses, three sets of main codes were selected for qualitative analysis based on the group separations shown in table 5 : community (large), ideas/information (moderate), and agency/argument (non-significant). in the knowledge-creation model students contribute ideas, on which the community works and which thus become its "intellectual artifacts" (bereiter 2002) . at the same time, students continue to own their ideas, and whether their ideas are appreciated and taken up by the community is important to the formation of students' identities as community members (wenger 1998) . the coding revealed between-group differences relating to aspects of this issue. for example, group a encouraged its members more often than the other groups (table 4): i think your ideas for groups are good … it would mean that we could get a start on all the topics right away. good job of actually getting things going! i really like [s's] idea of setting ourselves little mini-deadlines so that everybody will stay on task and finish the job more efficiently. there were also examples in which additional views were sought from students who had not yet contributed ideas. some students also felt a responsibility toward the group and apologized for failing to contribute to the discourse: sorry i've been away at a tournament for quite a while, so i'm just trying to get caught up. i don't know how much work you've all got done already, but the groups etc. sound pretty good ... i'll get onto researching as soon as i'm sure what's going on. another way in which group a promoted a sense that students belonged to a community was by instituting a democratic voting scheme for prioritizing research questions (11 of 60 code instances). while taking a vote can be a superficial process, it was accompanied in this group by considerable ideation, information processing, and linking. the voting process was also present in group c but was absent from groups b and d. group b appeared to harbor some tension between the grade 10 and 11 students arising from miscommunication. one student in the grade 11 class wrote: as of now, we have less than 1 week left and because your class have not been very active in this final phase, we've decided to go with these two questions above because we've a student from the grade 10 class responded as follows: yeah, alright. if the rest of our group wants to do it then i guess that's what's being done since "we have not been very active." i thought we were only supposed to research our own questions first. are those the only questions that we are doing then? we are sorry that you are not satisfied with the level of our commitment on kf. we weren't aware that we needed to pick from your questions as well as ours. sorry for the inconvenience. in summary, group a had a shared commitment to the task, a sense of belonging to the group, and an appreciation for all group members' contributions, all of which are indicators of communities (wenger 1998) . these social processes were also present to some extent in group c, but they may have had less effect. group a's discourse had most of the kinds of idea units needed for knowledge construction, particularly concepts and explanations; only 22.2% were coded as facts or opinions. however, only 2.4% of its idea units were classified as rise-above, suggesting that the discourse was not yet a well-developed example of knowledge creation. all of the other groups' discourses were more fact oriented, with percentages of idea units coded as facts or opinions ranging from 39.1% (group b) to 62.5% (group d). nevertheless, in group b, there were some examples of concepts and explanations, and group c had 10 explanations and a few rise-above units. this mix of conceptual and factual contributions is the main reason for the idea code providing only moderate group separation ( table 5 ). the information code revealed a tendency toward knowledge sharing in all groups: the many instances where information was presented without interpretation or evaluation (group c, 66.7%; group d, 61.2%; group a, 50.0%, group b, 35.4%). this prevalence of information sharing may be related to an epistemological understanding of inquiry as asking questions, finding answers, and reporting them, with information assumed to be self-explanatory (a realist position). instead of describing information, students need to be developing explanations and using information to support them. the difference is illustrated below: i found this information on: [web link]. "thailand, the world's seventh largest poultry producer, will suffer only 'modest losses' to its economy due to the h5n1 strain of the avian virus, it says. a 'complete decimation' of thailand's poultry industry would this note describes information but does not interpret or evaluate it. in contrast, in the following note, the student is providing an explanation and uses the web link at the end of the first paragraph as a reference for further information; in the second paragraph, the description of the second web link is integrated into the explanation. first off, let's look at the very definition of 'quarantine.' we're not talking maybe putting up some red tape around affected farms here, we're talking the slaughter and elimination of literally millions of chickens in the us, canada, and 10 asian nations including vietnam and china. the most recent outbreak, in hong kong, resulted in the slaughter of about 40,000 poultry; the birds were killed, then bagged for dumping in landfills. this outbreak alone cost the poultry industry the equivalent of us $10.26 million. [web link] the short-term costs incurred may seem rather drastic, but if the flu can be contained now, by eliminating all possibly infected birds, it will cost far less than trying to contain it later on if there are more outbreaks. i found the following website had some really valuable information concerning avian flu outbreaks in the past-there have been 21 large-scale epidemics all over the world, ranging from australia to pakistan, and this site talks about the economic impact of each-for example, "the 1983 pennsylvania (usa) outbreak took 2 years to control. some 17 million birds were destroyed at a direct cost of us$62 million. indirect costs have been estimated at more than us$250 million." [web link] neither agency nor meta-discourse provided statistically significant group separation. the agency code looked separately at planning and reflection relating to epistemic aspects of the inquiry and the completion of the project. there were not many instances of agency relating to the inquiry (33). reflection on learning is an important aspect of knowledge construction and knowledge creation, and the lack of reflection in the discourses provides separate evidence that most groups treated information as unproblematic. another important finding from the agency code is the many instances of project planning from group c, suggesting that it had considerable difficulty in self-directing its inquiry. project planning is an important aspect of knowledge-creation discourse, but it should not dominate the cognitive features. the meta-discourse code is conspicuous because it occurred infrequently, but there were attempts by all groups. for example, a group b student attempted to advance the inquiry to a new stage by suggesting a new question: … i guess the question now is how can we make the chickens less likely to develop serious symptoms, and to become more like the wild poultry. and maybe an effective method of keeping the chickens from getting sick and to stop the spread of the avian flu is by doing something to the wild fowl to make them unable to carry the virus. it raises some interesting questions that can probably be analyzed further! perhaps the suggestion came too late, but it was not taken up by the group. relating the group discourses to the discourse modes the goal of the fourth analysis was to map the group discourses onto the three modes of discourse. first, the subcode frequencies were classified as small (0 to 5 instances), moderate (6 to 10), and large (greater than 10). the results were then compared to the relevance ratings of the subcodes (table 4) to predict the discourse modes. for example, group a had 20 explanation-seeking questions (large), which corresponds to knowledgecreation discourse. group c had two instances of fact-seeking questions (small), which is consistent with knowledge-construction and knowledge-creation discourses (a degenerate prediction). group b had nine instances of opinion (moderate), which did not correctly predict any discourse mode. to sample the main codes evenly, the two subcodes that predicted the most complex discourse mode were selected for creating profiles. figure 1 shows the number of correct predictions of each discourse mode for the four groups. perfect agreement with a discourse mode would include 14 predictions of that mode; however, because there are many degenerate predictions, these would be accompanied by some predictions of the other modes. the profile of group a is most consistent with knowledge creation: it includes nine predictions of that mode, of which only one is degenerate. it also includes two predictions of knowledge sharing. the overall fit of the predictions to the discourse mode is best for this group. the profile of group d is almost the reverse: it has nine predictions of knowledge sharing (seven nondegenerate) but includes more predictions of the other modes than the group a profile. the profiles of groups b and c are more difficult to interpret because they include nearly equal numbers of predictions of all three of the discourse types. this could be caused by a variety of factors including the existence of smaller units of social organization that approach the discourse differently and contextual dependencies that cause the discourse on one problem to be qualitatively different from that on another. this possibility was explored for group b using inquiry thread analysis. (group b was chosen for this because we already know that group c fell behind in phase 1 and had less time for its inquiry in phase 2.) fig. 1 number of correct predictions of discourse types from sub-code frequencies. two sub-codes from each main code were used, leading to at most 14 correct predictions per group. however, because some subcodes did not uniquely predict a single discourse type and some did not correctly predict any type the number of predictions per group is generally different from 14 an inquiry thread is a temporally ordered sequence of notes on the same problem or topic. the notes need not be hyperlinked to be part of the same inquiry thread, and it also is possible that notes that are hyperlinked are not part of the same inquiry thread (for details on the method see zhang et al. 2007 ). nine inquiry threads were identified, of which six were active for more than a month. the longest thread (13 notes, 7 authors, and 12 readers) was active throughout the entire project and examined scientific mechanisms by which avian flu infection occurs; it included relatively many of the instances of concept, explanation, and deepening inquiry identified in the coding. a similar inquiry thread, but of shorter duration, began in the second half of phase 2, and focused on a causal explanation of why children may be less susceptible to sars (10 notes, 6 authors, 10 readers). threads that were more descriptive were general explorations of sars and avian flu in the first 3 weeks of the project, and somewhat argumentative discussions of how the media had handled the sars outbreak, the disposal of chickens infected by avian flu, and the prevention of avian flu. although deeper analysis would be useful, these results generally support the context-dependence hypothesis. group b engaged in more explanation-oriented discourse when relevant concepts were available, and less when exploring sars and avian flu in general and when concepts were not available. this paper seeks a clearer articulation of the nature of computer-mediated discourse needed for bereiter and scardmalia's knowledge-creation model. its main contributions are the conceptual framework for distinguishing between knowledge-sharing, knowledgeconstruction, and knowledge-creation discourses, an accompanying coding scheme, and the application of both to an evaluation of discourse in knowledge forum. this section reviews what has been accomplished, suggests several strategies for improving the alignment of online discourse to the knowledge-creation model, and outlines further development of the coding scheme. conceptual framework i have argued for a conceptual framework that contrasts three modes of discourse, which can be associated with different theoretical perspectives (transmission/naïve realism, cognitive psychology, and interactive learning mediated by shared objects). knowledge sharing is included because it remains a common discourse mode and is useful in some situations; knowledge construction is included because it is what knowledge creation needs to be distinguished from most. knowledge creation is not a new example of constructivism (in the cognitive paradigm), but an example that reifies a new theory of mind that does not depend on a notion of the mind as a container (bereiter 2002) . however, due to the incommensurability of the underlying theories, i do not regard the discourse modes as stages in the development of a community's discourse, as gunawardena et al. (1997) have suggested for knowledge sharing and knowledge construction. the framework extends the distinction between learning and knowledge creation (bereiter and scardamalia 1996) by differentiating between learning by knowledge sharing and learning by knowledge construction. this differentiation makes it possible to bring into focus both differences and similarities between knowledge construction and knowledge creation. the treatment of the knowledge-creation model in the framework marks a departure from the extant literature. bereiter and scardamalia focus on ideas as improvable objects and the socio-cognitive and epistemic dynamics of improving them, as though that could happen without regard for the social context (bereiter and scardamalia 2003; scardamalia 2002; scardamalia and bereiter 2006 ). in the framework described here, the recommended use of design-mode discourse over belief-mode discourse, a distinction valid for the epistemic work of improving ideas but not for the additional work needed to prioritize goals, ensures that new ideas diffuse throughout the community and possible advances in knowledge are evaluated. one of the most apparent differences between examples of knowledge construction and knowledge creation is the discourse by which this additional work is achieved. paying more attention to the social context in which knowledge creation occurs is not only important for an adequate portrayal of knowledge creation for students, but it also reflects the conditions in classrooms. recently, interest in these aspects of knowledge creation has been mounting. bielaczyc (2006) develops a social infrastructure framework that emphasizes culture and practices, and hakkarainen (2009) introduces the notion of "knowledge practices" to combine epistemic and social practice elements of knowledge creation. it is hoped that the framework described here will stimulate further research to clarify the relationship between explanationoriented discourse, argumentation, and the advancement of collective knowledge. group a's discourse was identified as knowledge creation, providing the strongest evidence of a sense of community, explanation-seeking inquiry, interpreting and evaluating information, knowledge advancement, and insight into these processes. from the analysis of frequencies in table 5 , we know that the leading factor differentiating group a from the others was its sense of community, but it is likely that all of the observed effects are necessary. the relatively clear identification of knowledge-creation discourse and better knowledge advances are encouraging because they suggest that knowledge creation is feasible for secondary school settings. however, there is a need for caution because there was little evidence of rising above, meta-discourse, and use of the advanced features of knowledge forum, and there was still too much evidence of knowledge sharing. the relatively clear identification of group d's discourse as knowledge sharing is more disconcerting in a classroom generally oriented toward student centered and constructivist learning. nevertheless, my work with many teachers in the last decade suggests it is a common occurrence. perhaps in this case context dependence mattered less, and the results may point to deeply held beliefs such as quick learning (schommer 1990 ) and achievement motivation. indeed, group d's results on the summary notes were second to group a's. group c was also problematic. like group a, it expended much effort on maintaining its sense of community, but was relatively inactive in posing questions and working with information (see table 4 ), and created fewer summary notes. we also know the group had less time for its inquiry than the others because coming up with research questions took longer than planned. although motivation could have been a factor, this was probably an example of an inadequate level of guidance (hmelo-silver et al. 2007 ). nevertheless, cases like this, in which students are unable to manage inquiries very well, are also common in project-based science (krajcik et al. 1998; polman 2000) . cultivating an innovation ecology as the study described here was a case study, its claims pertain to the observed groups only. further studies investigating the phenomena using different methods and in different settings would lead to a fuller understanding of the generality of the claims (yin 2003) . nevertheless, some cautious recommendations for encouraging an innovation ecology can be made based on the findings. this subsection examines several conditions that constrain or enable knowledge creation-the nature of the task, the sense of community, idea-centered discourse, the use of technology, and meta-discourse-and discusses how they can be optimized. a common reason for the failure of efforts at knowledge creation in school is their lack of authentic problems (bereiter and scardamalia 2003) . asking students to investigate what interested them about sars and avian flu held some attraction from this point of view because there was a lack of knowledge, the topics were discussed in society, and students could pursue their own questions. these are considered good things by proponents of socio-scientific issues in teaching (walker and zeidler 2007 ). yet, the students' interests frequently took them into areas where they could not build on their initial knowledge. it is possible that the extensive reliance on knowledge sharing resulted in part from a general lack of knowledge that led the students to explore opinions and "chart the territory," and from a lack of concepts and perspectives that could be used to question sources. some students seemed to suggest that what they were doing lacked authenticity as schoolwork because the depth of knowledge was inferior to what was normally expected of them: it is important to understand that there is not one answer to this question. i am limited in my understanding because i get my information from news sources that may be biased. i can also not understand how sars spreads scientifically because i am not a scientist or a doctor. i am like the rest of the public that gets information from news sources. (group d summary note) these considerations have important implications for developing an innovation ecology. in school, it is important that students develop academic knowledge: for example, concepts, explanations, explanatory principles, inquiry methods, and meta-conceptual knowledge. social norms are needed in the classroom to keep these things in focus as students engage in knowledge creation. in other words, in getting a knowledge-creation experience started, the curriculum, students' prior academic knowledge, and their interests should be explored together to forge a closer connection to the curriculum and assess the potential of ideas for inquiry. in the study, students did explore the potential of their ideas but did so independently in their own groups, and the social norms were not developed. a closer connection to the curriculum would also be needed for scaling up knowledge creation in schools. many other researchers link inquiry to the attainment of national educational standards (krajcik et al. 2008) but do this in ways that undermine key goals of knowledge creation (e.g., epistemic agency, adding to the intellectual heritage of a community). further research into how knowledge-creation experiences can be integrated into the curriculum is much needed. the study identified social processes that constitute a sense of community (such as encouragement, giving credit, drawing in participants, and apologizing) as the distinction between group discourses. these processes have been studied extensively in education and social psychology (e.g. johnson and johnson 1989; slavin 1990 ) but have received little attention in computer-supported collaborative learning research, where social interactions tend to be studied in the context of problem solving from the perspectives of cognitive theory (de laat et al. 2007; roschelle 1992; suthers et al. 2007 ), intersubjective meaningmaking (koschmann and lebaron 2002; stahl 2006) , or are dismissed as off-task behavior (meier et al. 2007 ). the social interactions identified in the study were neither part of problem-solving sessions nor irrelevant, but were directed at maintaining and improving how the groups worked. cultivating the social dynamics identified in the study would be important for creating a safe environment for knowledge creation, and thus an important aspect of an innovation ecology. the considerable differences in how the groups functioned socially provide reasons for reexamining the social organization of the class for its knowledge creation. in the assessment literature, the inequities of group work have been noted (webb et al. 1998 ). if students work together for several months, inequities arising from individual differences in motivation, effort, and ability could lead to substantial disadvantages for some students. using flexible and opportunistic groups, in which students join a group for a short time to accomplish specific goals, would make students less susceptible to the potential inequities and would help them learn and work with many different students. recently, zhang et al. (2009) used social network analysis to compare three social configurations-fixed groups, interacting groups, and opportunistic groups-and found that opportunistic groups best diffused new knowledge. in the present study, the decision to form fixed groups was intended to limit the number of notes students would need to deal with, but this problem could also be addressed by encouraging more reflective discourse with greater attention paid to synthesis and rise-above ( van aalst 2006) . one thing that this cognitively demanding work does is slow down the growth of the database. the importance of idea-centered discourse is so well established in the literature that it does not require further amplification (e.g. bereiter 2002; hakkarainen 2003; scardamalia 2002) . nevertheless, in the study students frequently introduced information without generating ideas or questions. if this kind of discourse is widespread and consistent within a community, it may suggest naïve epistemic beliefs (schommer 1990 ). deeper reflection on what makes a valuable contribution to knowledge forum may lead students to interpret and evaluate information, and to elaborate by providing examples and counterexamples. students could do this even when exploring a new content area, provided that they have concepts that can provide a lens for interpretation. social norms about the quality of knowledge to be created could also help students to focus on developing explanations. to facilitate developing a set of coherent explanations, it seems important to cultivate rise-above as a prominent dynamic of the discourse. although it can be used late in an inquiry to articulate what has been learned, it may also be useful for scaffolding the discourse, suggesting how students can contribute next. in studies of portfolio notes in knowledge forum, the reflections needed to prepare the portfolio notes have also had such a scaffolding function (lee et al. 2006 ; van aalst and chan 2007) . rise-above can produce incomplete explanatory frameworks, which may lead to predictions and new inquiry goals. thus, rise-above needs to be a social norm that is in focus throughout the inquiry. this would allow an approach whereby students contribute new information and ideas and regularly look for opportunities to review progress and identify more new ideas and lines of inquiry. the approach taken to the use of technology was typical of what i have seen in work with many teachers. the basic features (creating and responding to notes) were demonstrated at the beginning of the project, and instructions for creating links between notes were given with the instructions for phase 3. that was not sufficient. we made an implicit assumption that learning about knowledge forum would not occupy very much instructional time-it was just "a tool." to the contrary, the use of knowledge forum needs to mediate the social and cognitive work of creation (cole and engeström 1993; hakkarainen 2009 ); students need to learn to coordinate use of its features with use of the concepts of the knowledge-creation model. for example, rise-above involves important social skills because the ideas contributed by different students are combined and the authors may disagree with how their ideas are used. technical skills such as the ability to create a private view (accessible to a subgroup) and annotations can be helpful for temporarily storing copies of notes that are being considered, as well as draft ideas and notes before the final result is made public. thus, rise-above is a social practice that can exist only because the technology makes it possible, and it is a practice that needs to be developed. it is also worth considering whether knowledge forum provides the best medium for creating knowledge. at least one group had difficulties using it to reach a consensus about priorities and goal setting in phase 1, and talking face-to-face may have been more effective. while asynchronous writing can support reflective thought, reading and writing notes is time consuming and should only be used when it provides advantages over more social ways of interacting. some researchers and teachers have developed practices such as poster presentations, gallery walks, and whole-class talk, whereby students report and discuss the ideas, questions, and challenges they are considering within their groups (kolodner et al. 2003; zhang et al. 2009 ). according to kolodner and colleagues, such practices become routines and rituals within the community, and students come to see why they are necessary. zhang et al. (2009) found that students requested "kb talks" to discuss the database with each other. these practices provide opportunities for students to be aware of their progress, suggest ways of addressing problems, and identify learning needs not otherwise recognized. in their absence, work on knowledge forum is disconnected from the educational culture of the class and feels like a special project. however, research into the role of the social infrastructure that supports knowledge creation is still in an early phase (bielaczyc 2006; truong 2008; zhang et al. 2009 ). the evidence of meta-discourse in the groups was limited, partly due to the short duration of the project but also because it was an intervention. had we not intended to study their work, the students would have been completing individual inquiries. when knowledge creation pervades the general approach students take to their schoolwork, long-range effects may become more evident. for example, students may discuss how to improve on previous efforts or evaluate the evolution of ideas over a substantial period such as an entire school year. before this can happen, their inquiries need to be connected more deeply to the curriculum, and the use of technology woven into a set of coherent practices aimed at knowledge creation. the study provides a coding scheme for analyzing asynchronous discourse, extending earlier schemes that emphasized the socio-cognitive aspects of online discourse and drawing from previous work on rating scales on levels of questioning and explanations (chan 2001; hakkarainen et al. 2002) . the main codes can serve as a general framework for coding to facilitate comparisons across studies of computer-supported collaborative learning. however, if the subcodes are considered as indicators of the phenomena intended with the main codes, further research is needed to improve and expand the current set. for example, it would be useful to add further subcodes for questioning, agency, and meta-discourse to provide more balance among the main codes and improve the usability of the coding scheme for a wider range of research questions. the currently limited set of subcodes for some main codes reflects the overall limited evidence of the underlying phenomena (i.e., additional subcodes could have been induced from the data had the evidence of these main codes been stronger). while it is not my intention to fully map the codes onto the knowledge-creation principles, the coding scheme may provide a complementary framework useful for elaborating several principles. for example, information is intended to describe different levels of information processing, ranging from uncritical sharing to evaluation of a collection of sources in the context of the problems under investigation. if one correlates frequencies for this code with those relating to working with ideas, a fuller understanding of the principle of the constructive use of authoritative sources could be achieved. this paper has elaborated distinctions between three modes of online discourse-knowledge sharing, knowledge construction, and knowledge creation-which correspond to theories of transmission/naïve realism, cognitive psychology, and interactive learning using shared mediating objects. the framework was applied to a case study of four groups of students who used knowledge forum as part of an attempt to create knowledge about sars and avian flu. through the use of a new coding scheme, one group discourse was indentified as a threshold case of knowledge creation, one as knowledge sharing, and two as hybrids of all three modes. the study revealed the importance of the social interactions needed for a sense of community as one of the leading factors separating the group discourses. education and mind in the knowledge age the psychology of written composition knowing, learning and instruction: essays in honour of robert glaser surpassing ourselves: an inquiry into the nature and implications of expertise the handbook of education and human development: new models of learning, teaching and schooling learning to work creatively with knowledge designing social infrastructure: critical issues in creating learningenvironments with technology student approaches to learning and studying how people learn: brain, mind, experience and school design experiments: theoretical and methodological challenges for creating complex interventions in classroom settings situated cognition and the culture of learning peer collaboration and discourse patterns in processing incompatible information distributed cognitions: psychological and educational considerations design research: theoretical and methodological issues investigating patterns of interaction in networked learning and computer-supported collaborative learning: a role for social network analysis reconceptualizing change in the cognitive construction of knowledge addressing the challenges of inquiry-based learning through technology and curriculum design expansive learning at work: toward activity-theoretical reconceptualization against method: outline of an anarchistic theory of knowledge: humanities defending constructivismin science education making the invisible visible: a teaching/learning environment that builds on a new view of the physics learner interaction analysis of a global on-line debate and the development of a constructivist interaction analysis model for computer conferencing the 3m way to innovation: balancing people and profit effective discussion through a computer-mediated anchored forum emergence of progressive-inquiry culture in computer-supported collaborative learning a knowledge-practice perspective on technology-mediated learning epistemology of inquiry and computer-supported collaborative learning analyzing collaborative knowledge construction: multiple methods for integrated understanding problem-based learning: what and how do students learn? facilitating collaborative knowledge building scaffolding and achievement in problem-based and inquiry learning: a response to kirschner productive discussion in science: gender equity through electronic discourse a cognitive approach to the teaching of physics cognition in the wild cooperation and competition: theory and research enhancing sociability of computer-supported collaborative learning environments problem-based learning meets case-based reasoning in the middle-school science classroom: putting learning by design into practice learner articulation as interactional achievement: studying the conversation of gesture inquiry in projectbased science classrooms: initial attempts by middle school students learning-goals-driven design model: developing curriculum materials that align with national standards and incorporate project-based pedagogy the structure of scientific revolutions falsification and the methodology of scientific research programmes science in action: how to follow scientists and engineers through society situated learning: legitimate peripheral participation students assessing their own collaborative knowlegde building the five dysfunctions of a team wise design for knowledge integration towards knowledge building: from facts to explanations in primary students' computer mediated discourse a rating scheme for assessing the quality of computersupported collaboration processes understanding learning in a postmodern world: reconsidering the perry scheme of intellectual and ethical development participation in knowledge-building discourse: an analysis of online discussions in mainstream and honours social studies courses the knowledge creating company: how japanese companies create the dynamics of innovation learning how to learn models of innovative knowledge communities and three metaphors of learning seeing what we build together: distributed multimedia learning environments for transformative communities designing project-based science: connecting learners through guided inquiry marie curie, a life in science a force of nature: the frontier nature of ernest rutherford learning by collaborating: convergent conceptual change college physics teaching: from boundary work to border crossing and community building collective cognitive responsibility for the advancement of knowledge knowledge building environments: extending the limits of the possible in education and knowledge work knowledge building: theory, pedagogy, and technology effects of beliefs about the nature of knowledge on comprehension epistemology and the teaching of science. ottawa, canada: science council of canada on two metaphors for learning and the dangers of choosing one cooperative learning: theory, research and practice group cognition: computer support for building collaborative knowledge social constructivist perspectives on teaching and learning beyond threaded discussion: representational guidance in asynchronous collaborative learning environments exploring social practices that support knowledge building in a primary school student-directed assessment of knowledge building using electronic portfolios exploring collective aspects of knowledge building through assessment radical constructivism a way of knowing and learning promoting discourse about socioscientific issues through scaffolded inquiry equity issues in collaborative group assessment: group composition and performance communities of practice, meaning, and identity case study research: design and methods an evolutionary approach to harnessing complex systems thinking in the science and technology classroom socio-cognitive dynamics of knowledge building in the work of 9-and 10-year-olds designs for collective cogniitve responsibility in knowledge-building communities acknowledgments the classroom work of this study was completed while i was at simon fraser university; it was supported by a discovery parks grant from that university and the social sciences and humanities research council of canada. i thank the students and teacher for their work. preliminary findings from the summary note task were reported at the 2005 cscl conference held in taipei, taiwan. i thank cindy hmelo-silver, jianwei zhang, carol chan, and three anonymous reviewers for their comments on a draft of the paper.open access this article is distributed under the terms of the creative commons attribution noncommercial license which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited. key: cord-311910-2m67822c authors: durfee, sara m.; goldenson, robin p.; gill, ritu r.; rincon, sandra p.; flower, elisa; avery, laura l. title: medical student education roadblock due to covid-19: virtual radiology core clerkship to the rescue date: 2020-07-24 journal: acad radiol doi: 10.1016/j.acra.2020.07.020 sha: doc_id: 311910 cord_uid: 2m67822c rationale and objectives: medical schools were upended by the covid-19 pandemic, resulting in suspension of all in-person educational activities, and leaving clinical clerkships on hold indefinitely. a virtual curriculum and novel teaching methods were needed to fulfill curricular requirements. we developed a comprehensive virtual radiology clerkship and evaluated the efficacy of this novel method of teaching. materials and methods: a four-week virtual radiology clerkship was designed to accommodate medical students who had not yet completed the required clerkship. the design included online flipped classroom modules, large group didactic lectures, and small group homeroom activities. student performance was assessed via a standardized online final exam. feedback from students was collected using online surveys. student performance was compared with the in-person radiology clerkship. results: one hundred and eleven medical students were enrolled in the virtual radiology clerkship. final exam scores were similar to the in-person clerkship. students indicated that small group homeroom activities had the highest overall satisfaction. students recognized enthusiastic teachers regardless of class format. exceptional course content and organization were also noted. course weaknesses included didactic lecture content which was repetitive or too advanced, the limited opportunity to build personal connections with faculty, and scheduling conflicts with other competing school activities. conclusion: a completely virtual radiology core clerkship can be a successful educational experience for medical students during a time when remote learning is required. a small group learning environment is most successful for student engagement. personal connections between faculty and students can be challenging in a virtual course. the covid-19 pandemic has created a worldwide health care crisis with a range of unprecedented challenges, including a lasting and profound impact on the delivery of medical education. 1 the covid-19 pandemic swept through hospital-based medical education like a tsunami, with little time for advanced planning and no clear endpoint. in early winter, coronavirus was a distant threat, affecting mainly international travel and conferences, but by mid-march the virus had invaded our daily lives with dramatic changes to our home, work and educational environments, with no clear end in sight. 2 fears of disease exposure, lack of personal protective equipment and the need for social distancing abruptly exiled medical students from their hospital-based clerkships. on march 17, 2020 , the association of american medical colleges provided guidelines suggesting that medical schools pause clinical rotations for medical students. 3 students were quarantined and dispersed to remote locations, and an immediate and dramatic pivot to an alternative educational model was required. at our institution, the radiology clerkship committee was called upon to provide a comprehensive 4week core radiology clerkship for all students enrolled in the year-long principal clinical experience course who had not yet completed the required course. in two weeks, a unique virtual radiology core clerkship was designed, utilizing a variety of learning experiences including online flipped classroom modules, large group didactic lectures, and small group homeroom activities. in this article we describe the design and the logistical challenges involved in structuring a virtual radiology core clerkship for medical students and assess the efficacy of such a novel course. harvard medical school utilizes three major teaching hospitals to provide clinical educational experiences for its students. a required four-week radiology clerkship is a component of the core clerkship year. during the embedded month of dedicated radiology training, students participate in small group lectures, simulation sessions, clinical observations, and assessments. typically, 4-6 students are enrolled in the radiology clerkship each month at each hospital site. our radiology clerkship curriculum is well-established, and historically, feedback from faculty and students has been excellent. at the onset of the covid-19 pandemic, approximately one half of the students in the clinical year had yet to complete the radiology clerkship. with an immediate need to transition to online learning, the radiology core clerkship became the disaster plan for students enrolled in their required clinical year. with two weeks of preparation, a virtual radiology core clerkship was designed to educate all clerkship level medical students who had not yet completed their radiology clerkship. one hundred and eleven students were enrolled in the virtual radiology core clerkship, which was taught over a four-week block in april 2020. the group of students was heterogeneous, including students midway through their clinical year, students for whom this was their first clerkship, and md/phd students whose laboratory work was put on immediate hold. the challenge to educate a diverse student population via online teaching methods with very short preparation time is presented. the construct of the virtual radiology core clerkship was designed by the clerkship directors from the three hospitals. the learning objectives of the alliance of medical student educators in radiology (amser) were the basis for the 4-week clerkship. nineteen online "aquifer modules" served as a framework for the clinical material (aquifer, inc.). these modules emphasize a patient-centered approach to clinical scenarios, imaging, proper imaging utilization and patient safety. these resources had been the foundation of our pre-existing in-person core radiology clerkship, which was critical in allowing a quick pivot to virtual learning. each night, aquifer modules were assigned as pre-reading for the large group didactic lecture on the topic of the day. these lectures, lasting approximately 90 minutes, were given to the entire class via zoom. a diverse group of lecturers was chosen from the three hospitals to highlight the wealth of expertise at our medical school. internationally renowned experts in their field gave presentations alongside established educational leaders, retired faculty, and talented junior attendings. anticipating that the large group didactic format would pose challenges to learner engagement, a moderator was engaged in most lectures (usually one of the course directors). to promote clarification of material, the "chat" function on zoom was enabled during the lectures, and students were encouraged to submit questions as they arose. the moderator monitored the chat function during the lectures, answering questions directly via the chat function or collecting several questions to be reviewed in real-time with the speaker or during a pause in the presentation. intermittent question and answer breaks during the lecture helped maintain learner engagement in an otherwise lengthy presentation and allowed the speaker to interact with the audience. some of the lecturers employed "polleverywhere" for additional means of learner engagement. one of the highlights of our traditional in-person radiology clerkship course has been the opportunity to work and learn in a small group setting. representative past student experiences have included interactive case-based tutorials, unknown case panels, reading room observations, and gamified presentations. modifying these types of experiences for a large group of students via an exclusively virtual platform posed a particular challenge. in order to provide a comparable experience to our traditional clerkship, the concept of a "virtual homeroom" model was created, allowing for small group interactive learning to occur. a total of twelve homerooms were formed, across the three hospitals, with 8-10 students assigned to each homeroom. two homeroom teaching sessions were planned for each day, for a total of 24 homeroom sessions taught per day across all hospital sites, and a total of 432 homeroom sessions over the entire course. the first homeroom session each day was an interactive workshop on the topic of the day, with the teaching material provided by aquifer, and presented via zoom. because the students had prepared the material the night before, the workshop employed the "flipped classroom" model. the second homeroom session each day was an unknown case conference, related to the topic of the day and also presented via zoom. in this session, students experienced a more typical radiology readout session. the students took turns describing imaging findings using appropriate radiologic terminology and generating differential diagnoses. our hope with these sessions was to emulate the learning which typically occurs during reading room observations while reinforcing the content presented the same day in the didactic lecture and the workshop. both daily homeroom sessions included prepared speaker notes so that the material was uniformly covered across homerooms. the readily accessible teaching material was helpful for faculty recruitment. little preparation was necessary to teach, and even junior residents could run a homeroom session, gaining experience and comfort in teaching. to provide continuity and mentorship in each homeroom, one or two leaders or "captains" were assigned to each homeroom for the duration of the course. these instructors were radiology residents who were handpicked by the clerkship directors because of their enthusiasm and demonstrated effectiveness as medical student teachers. the homeroom captains were tasked with providing a supportive learning environment and sense of community for the students throughout the month, teaching unknown case sessions, serving as the contact person for concerns, and providing assessments on student participation and engagement. in total, seventeen homeroom "captains" were recruited across the three hospital sites to lead the twelve homerooms. students were assessed during the course in a similar fashion to our traditional in-person clerkship. an ungraded online multiple choice amser exam was completed at the midpoint of the course, in order for students to self-identify areas of strengths and weaknesses which could guide students' study plans for the remainder of the course. the online multiple choice standardized amser final exam was administered on the last day of the course. this multiple choice exam is used in medical schools across the country as a means of assessing competency in radiology clerkships and has been used as the final exam in our in-person clerkship for several years. the students were required to earn a passing score on this exam in order to pass the clerkship. in addition, attendance at didactic lectures and completion of all of the online aquifer modules were required to pass. these requirements were made to formally monitor student participation in a completely virtual course and also allowed course directors to assess student professionalism. standard medical school course evaluation forms were sent to students at the completion of the virtual radiology clerkship. the course survey included thirty-two questions asking the students to assess facets of the course using a rating system of excellent/good/average/fair/poor (1=excellent, 5= poor). course evaluations were anonymized and were submitted by students before they received their final grade. in order to comply with lcme standards, the course learning objectives and requirement checklists from our traditional core clerkship were reviewed, and no modifications were needed as we transitioned to a virtual platform. self-reported student duty hours were tracked in accordance with lcme requirements. students received online feedback at both the midpoint of the clerkship and after its completion. optional one-on-one meetings were also available at the end of the course for a student to meet virtually with one of the course directors, to obtain additional feedback or address any concerns the student might have had. because students routinely complete course evaluations as part of an ongoing quality assurance program, this study was considered to be a quality improvement study, and irb approval was waived. the online, closed book amser standardized final exam was administered to all students simultaneously in real-time, with a three-hour time limit. all students passed the exam, with a mean test score of 85% (range 64-95%). these scores were similar to final exam scores achieved by students enrolled in our in-person clerkship over the past five years, where the mean score was 78% (range 62-95%). the midterm multiple choice exam was completed online as an independent formative exercise, without time limitations. this format is identical to what has been used in our in-person clerkship. as the test served as a self-assessment, results are not reported. fifty-six students completed the course evaluation survey (50% of enrolled students). survey results are reported in table 1 . eighty four percent of responding students rated the course as excellent, while 16% rated the course as good. no students rated the course as average, fair or poor. more than 95% of responding students rated the overall structural organization of the clerkship (including delineation of course objectives and expectations) as either excellent or good. the quality of teaching in the course was rated as excellent or good by 98% of responding students. ninety-five percent of students felt that the content in the clerkship enabled them to master core knowledge related to radiology, and all students reported that the clerkship promoted an integrated view of the material. the clerkship fostered self-directed learning in 91% of students and enhanced an interest in radiology in 87% of students. despite an exclusively virtual platform, 98% of students reported that faculty were very accessible. direction and constructive feedback were "always" or "very often" given to 86% of students. on average, students received feedback approximately 3 times during the month from a variety of teachers, including clerkship directors, tutors, house staff, preceptors and others. students were also asked to rate the learning environment of our virtual radiology course ( table 2) . 100% of students reported that throughout the course faculty and staff "always" or "almost always" resolved conflicts in ways that respected the dignity of all involved, respected diversity, were respectful of other health professions, showed respectful interactions with students and showed empathy and compassion. no students experienced mistreatment during the clerkship. ninety-one percent of students reported that professional language was "always" or "almost always" used. as part of the formal course evaluation, students were asked to list three major strengths of the virtual radiology clerkship. the strength most frequently noted by the students was the small group homeroom sessions. students felt that the homeroom format was interactive and fostered an engaging learning environment. students identified the enthusiastic and dedicated teachers as a course strength, particularly those who led the homeroom sessions. the students felt that the course content was a major strength, noting a cohesive curriculum, the clinical relevance of the material, and targeting the material to the right audience level. in addition, the students identified the course organization as another strength, appreciating clear communication, an organized schedule and readily available resources for class and homework. as part of the formal course evaluation, students were also asked to identify three major weaknesses. the most frequently cited weakness was the repetition of material in the didactic lectures. at times, didactic content overlapped. students also noted that some of the didactic lectures were too advanced for a medical student audience or were too long in duration. some students felt the virtual format limited the opportunity to build personal connections with faculty and to obtain feedback on image interpretation. scheduling conflicts were also experienced by a small number of students while juggling course meetings with other medical student events which were not suspended during the pandemic. students were asked to provide suggestions for improvements to the virtual radiology course. the most frequent responses included limiting the didactic lectures to one hour and adding audience response questions to every didactic lecture. students suggested informing didactic lecturers of content previously covered to avoid repetition. a few students requested more imaging interpretation activities and others suggested that more emphasis be placed on developing longitudinal relationships with faculty. at the onset of the covid-19 pandemic, medical schools across the united states were tasked with balancing medical student education with safety and diversion of resources to clinical care. 4 to address this unprecedented challenge, most medical schools suspended student participation in direct patient care. medical students in the midst of clinical clerkships were sent home on short notice with little understanding of how they could continue their education during the pandemic. medical schools had to think quickly and creatively in order to provide virtual educational experiences for students which were of high educational quality and still met educational program objectives and graduation requirements. previous studies have validated the effective use of online teaching in medical education. 5-10 these established teaching tools would be needed as medical schools quickly transitioned to a completely online format. because of the unique nature of the field of radiology, including its reliance on electronic transmission of imaging and information, radiology educators were in a unique position to quickly respond to the need for virtual education for a large number of students as the covid-19 pandemic emerged. at our institution, using a combination of readily available radiologic teaching resources (i.e. aquifer modules) and original material (didactic lectures and unknown case conferences), we were able to modify the traditional in-person core radiology clerkship to an online platform in a very short period of time. the online platform enabled us to enroll a much larger number of students (111 students/class vs. 4-6 students/class in the in-person clerkship). despite this, small group learning was preserved through the use of virtual homerooms, allowing for learner engagement, clarification of material, and student assessment, similar to opportunities afforded in an in-person clerkship. in order to meet lcme standards, learning objectives, requirement checklists, and assessment activities remained the same for the virtual radiology clerkship. overall, the virtual radiology core clerkship was successful. a large number of students were able to complete a clinical course required for graduation, exclusively via an online platform. overall, learning objectives were met, and student performance on the amser standardized final exam was excellent. the majority of students rated the overall course as excellent. highlights of the course included the virtual homeroom experiences, dedicated faculty, a cohesive course curriculum and course organization. one reason for the success of the virtual clerkship is consistent with the adult learning theory of "blending learning" in which presenting material in a variety of formats maximizes educational benefits for a variety of learners. 11 presenting the course content in a variety of learning formats including large group didactic lectures, small group learning and flipped classroom models created an effective blended learning environment and were well received. given the rapid move to virtual learning, with little time allotted for planning, one would expect that didactic lectures would have played the most prominent role in such our course. theoretically, faculty could be recruited on short notice to give previously prepared lectures to the class, and course material could be covered in an efficient manner. most of the didactic lectures presented during the course were rated as excellent or outstanding. students rated lectures with appropriately targeted content and level and with lively, engaged speakers the highest. praise was also given to lectures with some form of audience response system as this was felt by the students to enhance engagement with the speaker and the material. these results are similar to results from a recent study that reported on patterns of narrative student feedback that are associated with relatively higher and lower evaluation scores. 12 the zoom chat function was also well received and created an ongoing dialogue during the lectures to enhance student engagement and understanding. the disadvantage of the didactic sessions of the course was the potential lack of continuity throughout the course due to the multiple speakers. students expressed frustration when content was repeated in more than one lecture or if content was too advanced for the expectations of the course. students also noted that attention waned after one hour. the traditional didactic approach is passive and does not foster critical thinking and complex reasoning skills that are at the core of medical education. 13 providing students with interactive activities such as problem solving, discussions and debates during class time has been found to improve learning outcomes, motivation and attitudes. 14 creating an interactive learning environment in our virtual radiology course, however, was particularly challenging given the large class size. in addition, we wanted to provide a "reading room observation" type experience for students enrolled in the virtual radiology clerkship, similar to the reading room observation experience students have in an in-person clerkship. to meet these goals, virtual homerooms were created. overall, the homeroom experience was rated as the highest and most rewarding component of the virtual course. this is not surprising as the homerooms were designed as an active learning environment to enhance the students' learning experience. in the homerooms, case-based learning was emphasized, and students had an opportunity to work through cases independently, ask questions, and clarify concepts. active learning in the homeroom facilitated higher order cognitive skills as students could connect new knowledge and skills to existing knowledge. [15] [16] faculty were able to monitor student progress through the course and identify as well as address any potential knowledge gaps. resident and fellow teachers had an opportunity to teach and provide mentorship. while the homeroom experience was an integral and essential part of the course, the disadvantage of a virtual homeroom was the demand it made on departmental resources and faculty and radiology trainee time. the extensive resources needed to operate a homeroom-based curriculum are similar to those required in a fully problem-based in-person medical school curriculum. [17] [18] for such a large course, twelve homerooms were required in order to achieve the small group size desired. because two homeroom activities were scheduled per day, four hundred and thirty-two unique homeroom sessions met over the four-week course. this required a large number of interested faculty and resident trainees to run these sessions. because our in-person radiology clerkship has historically been a core rotation at our medical school, the radiology departments at our teaching hospitals have traditionally been very supportive of medical student education and are committed to providing teaching resources and personnel to the clerkship. faculty and trainees routinely teach in the clerkship and are very familiar with the student' level of education and the curriculum. these institution-based factors made recruitment of faculty and trainees easier. teacher recruitment was also made easier because many faculty and trainees had reduced or reconfigured clinical responsibilities during the covid-19 pandemic and were available to teach. schedule flexibility was also required by both the medical students and the teaching faculty in order to support both groups' non-clerkship responsibilities, although students had few additional academic commitments during his time period. daily homeroom sessions could be scheduled around a resident teacher's clinical responsibilities or around educational resident conferences. administrative support was handled by a dedicated team at the medical school as well as by individual clerkship coordinators on site. with these factors in mind, if a repetition of this completely virtual course was desired in non-pandemic times, when clinical volume are robust and clinical responsibilities are more taxing, we anticipate it would be difficult to recruit the necessary number of faculty and trainees to lead the small group sessions. given how important and valued these homeroom sessions were in the overall success of the program, it is likely that the virtual clerkship experience would be diminished if these sessions were limited. while we were able to meet all of our learning objectives for the radiology core clerkship through a virtual platform, there were some areas of concern. in our course feedback, students missed the inperson interactions that would have occurred in the reading rooms in the traditional curriculum. students had less of a chance to build long-term relationships with faculty and staff. because the students were not in the reading room, they did not observe a radiologist in action -what a typical day is like, how a radiologist "reads" a case, or how a radiologist consults with referring clinicians. the remote nature of this course left no opportunity to see how "live" cases unfold in the reading room. these reading room experiences have been shown to be extremely impactful on students and could potentially influence specialty choice. 19 in order to recruit enthusiastic medical students into radiology, either virtual or in-person reading room observations should be included in a radiology clerkship. while the large number of students enrolled in the course made it nearly impossible to provide these types of personalized experiences to students, it would be important to find ways in the future to incorporate real-time reading room experiences into a virtual radiology course. video conferencing has recently been applied to virtual read-outs to improve the educational experience of radiology trainees in the setting of distance learning. 20 as virtual read-outs with trainees become more routine in the post covid-19 workflow, medical students could be incorporated into these virtual read-outs. this is an area for further investigation. the virtual radiology core clerkship was a successful educational experience for medical students despite the background of chaos and anxiety created by the covid-19 pandemic. closure of universities due to coronavirus disease 2019 (covid-19): impact on education and mental health of students and academic staff the socio-economic implications of the coronavirus and covid-19 pandemic: a review guidance on medical students' clinical participation: effective immediately updated interim guidance for medical students' participation in patient care during the coronavirus (covid-19) outbreak effect of virtual versus traditional education on theoretical knowledge and reporting skills of dental students in radiographic interpretation of bony lesions of the jaw case-based asynchronous interactive modules in undergraduate medical education learning clinical reasoning; how virtual patient case format and prior knowledge interact introduction of a virtual workstation into radiology medical student education adaptive tutorials versus web-based resources in radiology: a mixed methods analysis of efficacy and engagement in senior medical students medical student performance after a vertically integrated radiology clerkship blended learning in medical education: use of an integrated approach with web-based small group modules and didactic instruction for teaching radiologic anatomy lecture evaluations by medical students: concepts that correlate with scores calls for reform of medical education by the carnegie foundation for the advancement of teaching: 1910 and flipping radiology education right side up the integration of active learning teaching strategies into a radiology rotation for medical students improves radiological interpretation sills and attitudes toward radiology express team-based learning (etbl): a time-efficient tbl approach in neuroradiology effectiveness of problem-based learning curricula: research and theory undergraduate medical education: comparison of problembased learning and conventional teaching factors influencing choice of radiology and relationship to resident job satisfaction virtual read-out: radiology education for the 21st century during the covid-19 pandemic students were able learn the fundamentals of radiology while respecting social distancing requirements. students enjoyed the sense of community and engagement afforded by small group homerooms. personal connections between faculty and students, however, were challenging in this virtual course. key: cord-331135-4u99yxw2 authors: arsandaux, j.; montagni, i.; macalli, m.; texier, n.; pouriel, m.; germain, r.; mebarki, a.; kinouani, s.; tournier, m.; schuck, s.; tzourio, c. title: higher risk of mental health deterioration during the covid-19 lockdown among students rather than non-students. the french confins study date: 2020-11-05 journal: nan doi: 10.1101/2020.11.04.20225706 sha: doc_id: 331135 cord_uid: 4u99yxw2 background: covid-19 pandemic and its consequences have raised fears of its psychological impact. the objective of this study was to estimate the effect of student status on mental health conditions during covid-19 general lockdown among adults in france. methods: using cross-sectional data of the confins cohort, we estimated the effect of student status on depressive and anxiety symptoms, suicidal thoughts and perceived stress using multivariate logistic regression analyses. stratified models for college students and non-students were performed to identify associated population-specific factors. results: among the 2260 included participants, students represented 59% (n=1335 vs 925 non-students) and 78% of the total sample were female. student status was more frequently associated with depressive symptoms (adjusted or(aor)=1.58; 95%ci 1.17;2.14), anxiety symptoms (aor=1.51; 95%ci 1.10;2.07), perceived stress (n=1919, aor=1.70, 95%ci 1.26;2.29) and frequent suicidal thoughts (n=1919, aor=1.57, 95%ci 0.97;2.53). lockdown conditions that could be potentially aggravating on mental health like isolation had a higher impact on students than non-students. limitations: participants were volunteers, which could limit generalisation of the findings. the cross-sectional design did not allow determining if lockdown impacted directly mental health or if there is another cause. however, we adjusted analyses with the history of psychiatric disorders, and factors related to lockdown conditions were associated with mental health disturbances. conclusions: college student's mental health is of great importance in the context of the general lockdown set up during the pandemic. follow-up and interventions should be implemented especially for those at high-risk (younger people and those with history of psychiatric disorders). young adults are particularly exposed to psychiatric disorders which often start in young adulthood. college students have been identified as a vulnerable population and have higher prevalence of psychiatric symptoms (e.g. depressive symptoms, anxiety symptoms and suicidal risk) than other adults (kessler et al., 2007; kovess-masfety et al., 2016; verger et al., 2010 verger et al., , 2009 . in this population, and particularly in the first years at university, a high level of stress related to academic achievement, a low self-esteem, and depressive symptoms are frequently reported (augesti et al., 2015; beiter et al., 2015; cheung et al., 2020) . the unprecedented context of 2020 with the major sars-cov-2 pandemic and its consequences have raised fears of its psychological impact in the population and more particularly in the fragile population of college students (galea et al., 2020) . in february 2020, the sars-cov-2 epidemic, causing the covid-19 disease, hit europe and france, one of the most affected countries in the world in number of cases and deaths ("who coronavirus disease dashboard | who coronavirus " n.d.) . to contain the spread of the epidemic, france established a general lockdown from march 17 to may 11, 2020. it has been shown in previous lockdown situations (e.g. sars epidemic in 2013) that quarantine can have an impact on mental health (depression, post-traumatic stress symptoms, confusion, anger as well as suicide) by itself (barbisch et al., 2015; brooks et al., 2020) . little information is available about the psychological impact of lockdown on college students and its risk of exacerbating their isolation and their psychological vulnerability (beck et al., n.d.; husky et al., 2020) . a few studies have reported high prevalence of depressive symptoms, anxiety symptoms and stress during the covid-19 lockdown among college students (cao et al., 2020; husky et al., 2020; odriozola-gonzález et al., 2020; tang et al., 2020) but it is unkown whether the impact was different in this population compared to non-students adults. the objectives of this study were to estimate the effect of lockdown on mental health conditions (depressive symptoms, anxiety symptoms, suicidal thoughts and perceived stress) in college students and to compare their frequency and associated factors to a sample of non-students recruited in the same study. all rights reserved. no reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in the copyright holder for this this version posted november 5, 2020. ; https://doi.org/10.1101 https://doi.org/10. /2020 this study is based on the confins e-cohort (www.confins.org), a prospective online population-based cohort study of adults in france set up since april 2020 during covid-19 national lockdown and still ongoing. the objectives of the confins cohort were: (1) to investigate the impact of the covid-19 pandemic and the general lockdown, established in france between march and may 2020, on the well-being and mental health of the population; and (2) to explore opinions and beliefs of the population about the pandemic, its treatment and vaccines. a large communication campaign was deployed in france on social media and press, with a focus on specific populations (i.e. college students, health workers) reached also by emailing. the eligibility criteria were to be more than 18 and locked down in the french territory until the end of the general lockdown in france (may, 11, 2020) . enrolled participants signed in a secured web-site and complete questionnaires online. the baseline questionnaire collected socio-demographic information, medical history, lockdown conditions, mental health parameters, as well as opinions and beliefs. thanks to the confins cohort, direct comparison between students and non-students is possible (same recruitment and data collection for both populations). this study is based on baseline data collected during the general lockdown in france (until may, 11, 2020) . depressive symptoms. depressive symptoms were measured using the patient health questionnaire-9 (phq-9) (kroenke et al., 2001; pfizer, n.d.) modified to assess symptoms within the last seven days (instead of the last 14 days) for close monitoring purpose. items are rated from 0 to 3 according to increased frequency of experiencing difficulties in each area covered. scores are summed and can range from 0 to 27. higher scores represent higher depressive symptoms. we used the validated french version of the phq-9 (arthurs et al., 2012) . since, distribution of the score was not normal, we used a validated cut-off of 10 to define the presence of depressive symptoms (kroenke et al., 2001; manea et al., 2012) . anxiety symptoms. anxiety symptoms were measured using the generalized anxiety disorder-7 (gad-7) (spitzer et al., 2006) modified to assessed symptoms within the last seven days (instead of the last 14 days). items are rated from 0 to 3 according to increased frequency of experiencing difficulties in each area covered. scores are summed and can range from 0 to 21. higher scores represent higher anxiety symptoms. we used the validated french version of the all rights reserved. no reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in the copyright holder for this this version posted november 5, 2020. ; https://doi.org/10.1101/2020.11.04.20225706 doi: medrxiv preprint gad-7 (micoulaud-franchi et al., 2016) . since, distribution of the score was not normal, we used a validated cut-off of 10 to define the presence of anxiety symptoms (kroenke et al., 2001) . suicidal thoughts. participants reported if they experienced suicidal thoughts during the last seven days ("during the past 7 days, have you ever thought about killing yourself (suicidal ideas)": with the responses "no, never" and "yes sometimes" or "yes, on multiple occasions" considering together for analysis purpose). perceived stress. participants rated their current stress on a 10 points-scale ("how worried or stressed are you right now on a scale of 0-10"), with 0 representing the lowest level of stress and 10 the highest. since, distribution of the score was not normal, we used a cut-off of 7 to defined high perceived stress (corresponding approximately to the third quartile of the distribution). participants declared if they were currently college students or not and were asked about specific information according to their academic situation (e.g. cursus, university year) if they were students and their professional situation otherwise (having a high professional position and having a stable professional situation like time-undetermined work contract). sociodemographic information included sex and age (in years), familial situation (in a relationship or not) and education level collected differently for students and non-students and transformed for analysis into three categories: minor than a second-year university level or currently in their 1 st or 2 nd year for students ; holding a second-year university level or currently in their 3 rd year for students ; holding more than a second-year university level or currently in their 4 th or more year for students. other variables possibly influencing on mental health were also recorded like working or studying in the medical field, having an history of psychiatric disorder (among depression, bipolar disorders, generalized anxiety), history of another disease at risk for severe forms of covid-19 (among cardiovascular, respiratory, chronic digestive disease, cancer and diabetes). other covariates/exposures were collected according to the student status or not. for students: having a paid activity (student job) and self-rated financial situation during childhood (correct, difficult or very difficult vs comfortable or very all rights reserved. no reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in the copyright holder for this this version posted november 5, 2020. ; https://doi.org/10.1101/2020.11.04.20225706 doi: medrxiv preprint comfortable). for non-students: having a high professional position and having a stable professional situation (time-undetermined work contract). information related to the covid-19 pandemic or lockdown were also recorded: week of inclusion (w14-15 corresponding to march 30 to april 12, w16-17 corresponding to april 13 to 26 and w18-19 corresponding to april 27 to may 11), being in lockdown in a high-risk region (defined by region that registered more than 10 death during the week 14), having acquaintance or family with covid-19 or covid-19 suspicion, being alone in lockdown accommodation, being in lockdown at home (in the same place than before lockdown), lockdown accommodation with an outdoor space (like a balcony or a garden), lockdown accommodation surface (in m 2 ), having a pet, doing physical exercise during lockdown. nonstudents declared if they were working remotely during lockdown or not. first, we described the study sample overall and in both student and non-student categories. second, we estimated the effect of student status on each mental condition (i.e. depressive symptoms, anxiety symptoms, suicidal thoughts and perceived stress) using separated logistic regression models. three multivariate models were built using a different adjustment: model 1 adjusted for age and sex, model 2 adjusted for age, sex and variables not related to the covid-19 pandemic or lockdown (i.e. being in a relationship, education level, working or studying for the medical sector, history of psychiatric disease, history of other disease at risk for and model 3 adjusted for age, sex, pre-cited variables not related to the covid-19 pandemic or lockdown and variables related to the covid-19 pandemic or lockdown (i.e. week of inclusion in the cohort, being in lockdown in a high-risk region, acquaintance or family with or covid-19 suspicion, being in lockdown alone, lockdown accommodation with an outdoor space, having a pet, lockdown accommodation surface, physical activity during lockdown). we estimated odds ratio (or), their 95% confidence interval (95% ci) and the p-value of the wald test (and p-value of the type 3 test for categorical variables) related to the effect of student status. we performed secondary analysis restricting the sample to young adults (≤ 30 years old) to better take into account potential confusion by age. third, we computed a general model for each mental health conditions among students and non-students separately to investigate associated factors of these two populations. we performed the same model than primary analysis extending the list of exposure with all factors all rights reserved. no reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in the copyright holder for this this version posted november 5, 2020. ; https://doi.org/10.1101/2020.11.04.20225706 doi: medrxiv preprint (model 3) and entering specific variables for students (i.e. paid activity and financial situation during childhood) and for non-students (i.e. high professional position, stable professional position, teleworking). to test the robustness of the findings, we conducted sensitivity analyses using alternative cutoff for depressive, anxiety symptoms and perceived stress: (a) cut-off that represents severe depressive symptoms (phq-9 ≥ 15), severe anxiety symptoms (gad-7 ≥ 15) and mild perceived stress (≥ 5), (b) cut-off that represents the full spectrum of depressive (minimal: 0-9; mild: 10-14; moderate: 15-19; severe: 20-27) and anxiety symptoms (minimal: 0-4; mild: 5-9; moderate: 10-14; severe: 15-21). we also performed sensitivity analyses among the complete case population to test the robustness of results regarding the imputation process. our missing data analysis procedures used missing at random assumptions. we used the multivariate imputation by chained equations (mice) method of multiple multivariate imputation in sas software (proc mi and mianalyze) (janssen et al., 2010; rubin and schenker, 1991; schafer, 1997) . we independently analysed 10 copies of the data, each with suitably imputed missing values, in the multivariate linear or logistic regression analyses. we averaged estimates of the variables to give a single mean estimate and adjusted standard errors according to rubin's rules. we imputed only data from covariates using covariate data and completed the imputation process with other data collected in the confins cohort (having a child, housing type before and during the lockdown, self-rated health and self-rated quality of life before the lockdown). we performed additional imputations for the student subsample since paid activity and financial conditions during childhood had missing data. we performed all analyses using the sas statistical software (sas v9.3). of the 2344 participants enrolled in the confins cohort, 2309 were eligible for the study and 2260 were ultimately included in the primary analysis for depressive and anxiety symptoms and 1919 for suicidal thoughts and perceived stress (fig1). table 1 and table s1 provided in supplementary material describe the study sample. students represented 59% of the total sample (n=1335 vs 925 non-students). mean age in the student sample was about 16 years lower (mean=23.3 vs 40.1) whereas sex ratio was similar (3/4 female) in both samples. students had less frequently a partner (48.1% vs 76.3%) and were more frequently at risk for severe forms all rights reserved. no reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in the copyright holder for this this version posted november 5, 2020. ; https://doi.org/10.1101/2020.11.04.20225706 doi: medrxiv preprint for covid-19 (35.4% vs 24.0%). however, both populations were similar regarding history of psychiatric disorders (about 23%), education level (with a majority of more than a second-year university level) and proportion of individual working or studying in health domains (about 40%). although non-students spent the lockdown more frequently in their usual place than students (88.2% vs 63.4%), the quality of lockdown accommodation (e.g. surface, outdoor space) was similar for both. students were fewer than non-students to be in a high-risk region (11.3% vs 31.2%) and they were enrolled in the cohort later (mostly during weeks 16-17). during lockdown period, mental health conditions were different between the two populations. students presented more frequently depressive symptoms (32.5% vs 16.2%), anxiety symptoms (24.0% vs 14.7%), suicidal thoughts (11.7% vs 7.6%) and perceived stress (33.1% vs 22.1%) than non-students (table 2 ). in multivariate models, student status was associated with an increased probability to report depressive symptoms (or fully adjusted=1.58; 95%ci=1.17;2.14), anxiety symptoms (or fully adjusted=1.51; 95%ci=1.10;2.07), and perceived stress (or fully adjusted=1.70, 95%ci=1.26;2.29), independently from covariates related or not to the covid-19 pandemic or lockdown (table 3) . for suicidal thoughts, the oddratios were in the same range (or fully adjusted=1.57; 95%ci=0.97;2.53) but did not reach significance. sensitivity analyses showed consistent results when analyses were performed among the complete case population and with different cut-offs (results presented in table s2 in supplementary material). when both samples were restricted to young adults (≤ 30), students still had a higher risk of depressive symptoms (or fully adjusted=1.54; 95%ci=1.03;2.29), anxiety symptoms (or fully adjusted=1.60; 95%ci=1.04;2.45) and perceived stress (or fully adjusted=1.59; 95%ci=1.07;2.34) (results presented in table s3 supplementary material). students reported consistently more frequently depressive symptoms whatever the strata: female, male, with a history of psychiatric disorders, or according to various lockdown conditions (e.g. being in lockdown alone) (fig2). in explicative models for mental health conditions, we found shared factors between students and non-students: past history of psychiatric disorders (e.g. for depressive symptoms: or=2.80, 95%ci=2.06;3.82 among students and or=3.10, 95%ci=1.99;4.81 among non-students) and physical exercise during lockdown which was associated with less frequent mental health disturbances (e.g. for all rights reserved. no reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in the copyright holder for this this version posted november 5, 2020. in this large sample of adults observed during the general lockdown, high level of mental health disturbances was observed. students had a much higher prevalence of mental health problems (more depressive symptoms, anxiety symptoms and perceived stress) than non-students. in multivariate models adjusting for a large variety of potential confounding factors, students had a more than 50% increased risk of mental health problems compared to non-students. this increased risk was also observed in participants with or without a past history of psychiatric diseases, in the various strata of lockdown situations, or when both samples were restricted to young adults. finally, lockdown conditions that could be potentially aggravating on mental health like isolation had a higher impact on students. high frequency of mental health disturbances found in our study corroborates previous research conducted separately among students and other adults. among the general non-student population, several studies reported a prevalence between 17% and 28% for depressive symptoms, between 14% and 45% for anxiety symptoms and between 8% and 27% for stress (bäuerle et al., n.d.; choi et al., 2020; gonzález-sanguino et al., 2020; gualano et al., 2020; mazza et al., 2020; ozamiz-etxebarria et al., 2020; wang et al., 2020) . among college students, several studies reported a prevalence during the lockdown ranging between 9% and 21% for depressive symptoms, between 25% and 34% for anxiety symptoms and 28% for stress (cao et al., 2020; odriozola-gonzález et al., 2020; tang et al., 2020) . our study directly compared students and non-students in the same cohort and brings strong evidence for a higher risk of mental health disturbances among students rather than non-students during general lockdown. all rights reserved. no reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in the copyright holder for this this version posted november 5, 2020. ; https://doi.org/10. 1101 /2020 mental health during lockdown was associated with lockdown conditions, especially among students. indeed, estimated frequency of mental health disturbances are higher during the general lockdown (in our study and in above-mentioned studies) compared to previous research conducted before covid-19 pandemic and lockdown among students (between 11% and 20% of depressive symptoms, 15% of anxiety symptoms, and 11% of stress) (beiter et al., 2015; ibrahim et al., 2013; rotenstein et al., 2016) . furthermore, lockdown conditions were associated with mental health among students in our study, showing that lockdown represents a high-risk period for students. lockdown conditions were yet partially explored in the literature with an association found between mental health and having relative or acquaintance who got covid-19 (cao et al., 2020; tang et al., 2020) . to explain this higher mental health risk for individuals during general lockdown, several mechanisms could be discussed: disturbance in mood homeostasis during lockdown (i.e. failure to stabilize mood via mood-modifying activities) (taquet et al., 2020) , lack of social and familial support (gariépy et al., 2016; kmietowicz, 2020; pössel et al., 2018) or specific vulnerability of young adults and students already explored in the literature that could be exacerbated by pandemic and lockdown (e.g. worries about the future and employment) (hunt and eisenberg, 2010; tang et al., 2020; twenge et al., 2017) . the strengths of our study include the large sample, the standardised assessment tools used for mental health conditions and broad adjustment for other factors (related or not the covid-19 pandemic or lockdown). this study profits recruitment of these two populations in the same cohort as well as similar data collection that allow direct comparison. some limitations should however be taken into consideration. first, a sampling bias could have arisen since participants were volunteers, which could limit generalisation of the findings. regarding mental health disturbances, the consequences appear however limited as frequencies were consistent with existing literature. second, the cross-sectional design did not allow determining if lockdown impacted directly mental health or if there is another cause. however, we considered the history of psychiatric disorders that is an important confounder, and factors related to lockdown conditions were associated with mental health disturbances, suggesting that lockdown should have an impact on mental health especially for students. longitudinal studies have been set up in the general population and have showed a deterioration of mental health during lockdown all rights reserved. no reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in the copyright holder for this this version posted november 5, 2020. ; https://doi.org/10.1101/2020.11.04.20225706 doi: medrxiv preprint (pierce et al., 2020) . similar studies are needed among students to establish clearly if lockdown can impact mental health in the short and in the long term. from a public health perspective, this study confirms that students were a vulnerable population for mental health disturbances during pandemic period and lockdown, suggesting that screening might be useful to offer adapted support. besides, for preventive purpose in case of a new lockdown perspective, support and interventions adapted to students should be created and evaluated. especially psychological support could be provided by university health student services both online and through crisis hotlines. peers could also organize self-help groups and detect students with particular distress. from a research perspective, we recommend to explore two priority areas. first, research is warranted in order to clarify causality of the relationship between lockdown and mental health disturbances among students. to this end, longitudinal mental health data covering a period before, during and after lockdown would be necessary. second, it appears relevant to better understand the mechanisms underlying the specific vulnerability of students, i.e. identifying intermediate factors and especially those that can be modified. besides, qualitative studies or mixed design (i.e. mix of quantitative and qualitative data) may be useful to better understand both psychologic disorders students experienced during this period and solutions some of them put in place to better cope with depression, anxiety, stress and suicidal thoughts. the i-share and confins team are currently supported by an unrestricted grant of the nouvelle-aquitaine regional council (conseil régional nouvelle-aquitaine) (grant n° 4370420) and by the bordeaux 'initiatives d'excellence' (idex) program of the university of bordeaux (anr-10-idex-03-02). the team has also received grants from public health france (santé publique france, contract n° 19dppp023-0) and the nouvelle-aquitaine regional health agency (agence régionale de santé nouvelle-aquitaine). m. macalli was supported by a phd grant of the nouvelle-aquitaine regional council (grant n° 17-eure-0019). the funding bodies were neither involved in the study design, or in the collection, analysis, or interpretation of the data. all rights reserved. no reuse allowed without permission. perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in the copyright holder for this this version posted november 5, 2020. ; https://doi.org/10.1101/2020.11.04.20225706 doi: medrxiv preprint perpetuity. preprint (which was not certified by peer review) is the author/funder, who has granted medrxiv a license to display the preprint in the copyright holder for this this version posted november 5, 2020. ; https://doi.org/10. 1101 /2020 are scores on english and french versions of the phq-9 comparable? an assessment of differential item functioning differences in stress level between first year and last year medical students in medical faculty of lampung university increased generalized anxiety, depression and distress during the covid-19 pandemic: a cross-sectional study in germany covid-19 health crisis and lockdown associated with high level of sleep complaints and hypnotic uptake at the population level the prevalence and correlates of depression, anxiety, and stress in a sample of college students the psychological impact of quarantine and how to reduce it: rapid review of the evidence. the lancet the psychological impact of the covid-19 epidemic on college students in china depression, anxiety and stress in different subgroups of first-year university students from 4-year cohort data depression and anxiety in hong kong during covid-19 the mental health consequences of covid-19 and physical distancing: the need for prevention and early intervention social support and protection from depression: systematic review of current findings in western countries covid-19) in spain effects of covid-19 lockdown on mental health and sleep disturbances in italy mental health problems and help-seeking behavior among college students stress and anxiety among university students in france during covid-19 mandatory confinement a systematic review of studies of depression prevalence in university students missing covariate data in medical research: to impute is better than to ignore age of onset of mental disorders: a review of recent literature rules on isolation rooms for suspected covid-19 cases in gp surgeries to be relaxed mental health of college students and their non-college-attending peers: results from a large french cross-sectional survey the phq-9 optimal cut-off score for diagnosing depression with the patient health questionnaire (phq-9): a meta-analysis a nationwide survey of psychological distress among italian people during the covid-19 pandemic: immediate psychological responses and associated factors rapid detection of generalized anxiety disorder and major depression in epilepsy: validation of the gad-7 as a complementary tool to the nddi-e in a french sample psychological effects of the covid-19 outbreak and lockdown among students and workers of a spanish university psychological symptoms during the two stages of lockdown in response to the covid-19 outbreak: an investigation in a sample of citizens in northern spain patient health questionnaire (phq) screeners [www document mental health before and during the covid-19 pandemic: a longitudinal probability sample survey of the uk population associations between social support from family, friends, and teachers and depressive symptoms in adolescents prevalence of depression, depressive symptoms, and suicidal ideation among medical students: a systematic review and meta-analysis multiple imputation in health-are databases: an overview and some applications analysis of incomplete multivariate data a brief measure for assessing generalized anxiety disorder: the gad-7 prevalence and correlates of ptsd and depressive symptoms one month after the outbreak of the covid-19 epidemic in a sample of home-quarantined chinese university students mood homeostasis before and during the coronavirus disease 2019 (covid-19) lockdown among students in the netherlands increases in depressive symptoms, suicide-related outcomes, and suicide rates among u.s. adolescents after 2010 and links to increased new media screen time psychological distress in first year university students: socioeconomic and academic stressors, mastery and social support in young men and women covid-19) epidemic among the general population in china having a pet, n (%) 1002 health conditions during lockdown key: cord-272218-6vpu59xq authors: wu, anette; leask, betty; choi, edward; unangst, lisa; de wit, hans title: internationalization of medical education—a scoping review of the current status in the united states date: 2020-08-05 journal: med sci educ doi: 10.1007/s40670-020-01034-8 sha: doc_id: 272218 cord_uid: 6vpu59xq this article explores the current status of internationalization of medical education in the united states. dominant themes of articles published from 2000 to 2018 indicate that common formats are institutional partnerships, international learning at home, and student mobility programs. critical analysis on the basis of internationalization of higher education, recommendations, and future perspective is given. in the last 25 years, the internationalization of higher education (iohe), which includes internationalization of the curriculum, has emerged as a dynamic field of study supported by a rich body of scholarly literature and research. the consensus is that internationalization of the curriculum, referring to the incorporation of international, intercultural, and global dimensions into the curriculum in ways that are relevant to graduates' professional practice [1] , is important for all students. while motivations for internationalization of individual higher education institutions are varied, including quality improvement, provision of access, competitiveness, growth, and financial profits [2] , the provision of a professionally relevant education that prepares all students to be interculturally proficient professionals and citizens is the responsibility of the disciplines [3] [4] [5] . in medical education, this might mean, for example, developing students' ability to access and evaluate new ideas and practices from diverse national and cultural sources, using advanced intercultural skills in their professional practice, and practicing medicine in a global context. recent global health events, namely the ebola virus, sars, or covid-19 pandemic, have demonstrated the need for efficient international collaboration and communication in biomedical research, education, and patient care. such global health emergencies require efficiency in international communication, culturally competent and expert healthcare leadership (locally, nationally, and internationally), rapid international public health action, and collaborative international biotechnology and medical science research. today, more than ever before, these elements are not optional choices but rather represent essential components that should be included in medical education curricula in all parts of the world. in this context, it is important to take stock of where we are and where we might want to go next in the internationalization of medical education (iome), using research that has been undertaken into iohe curriculum, teaching, and learning, tailored to the needs of the medical profession. the authors argue that this offers a useful way forward because it focuses attention on preparing all medical graduates to meaningfully contribute to society as professionals who practice locally but are also global citizens. to date, iome is a broad term with no formally agreed upon definition. for the purpose of this paper, we utilize a definition commonly used in research on iohe and apply it to medical education. thus, we define iome as the process of purposefully integrating international, intercultural, or global dimensions into medical education in order to enhance its quality and prepare graduates for professional practice in a globalized world. this definition is consistent with research-based and frequently cited definitions of higher education internationalization [6] , internationalization of the curriculum, and internationalization at home frequently cited in the literature on higher education [6] [7] [8] . it constructs iome as a concept that includes intentional, systematic, and evidence-based activities designed to ensure students achieve specific learning outcomes through engaging in high-quality learning experiences. it positions iome as a means to a specific international educational goal, not a goal in and of itself. educational goals and competencies for iome overlap with other subject matter taught in medical school (i.e., cultural competency). in recent years, public health (ph) and global health (gh) have become an important part of medical curricula, incorporating elements of social equity, diversity, inclusivity, and cultural competence into medical education. in order to differentiate these different terms, short definitions are provided below. ph is regarded as "…the science and art of preventing disease, prolonging life, and promoting health through the organized efforts and informed choices of society, organizations, public and private communities, and individuals…" [9] . gh as its global counterpart historically evolved from international health-an area that addresses local, national, and international health concerns on all levels [10] . while both terms still exist, gh was defined by koplan in 2009 as "…an area for study, research, and practice that places a priority on improving health and achieving health equity for all people worldwide…" [11] . based on koplan's initial definition, others have proposed a shorter definition-"collaborative trans-national research and action for promoting health for all" [12] . there is still ongoing debate on a commonly agreed upon definition [13] and there is inconsistency in the way medical educators use the term gh. gh is frequently conceived as a multidisciplinary field of service, research, and education undertaken as part of an altruistic "global north to global south" collaboration, and/or as part of a national/local health equity program. its primary goals-as it oftentimes appears in published work-are to improve the health of underserved individuals and populations, and to achieve health equity for all people worldwide, with a primary focus on the resource-poor abroad [14] . for medical education, gh education programs in the in low-and middle-income countries (lmics) are frequently used to report on the internationalization of the medical curriculum-mostly to describe extracurricular activities or projects in lmics or the underserved populations. thus, the enactment of gh in medical education programs appears to be narrower and more limiting than the definition of gh itself suggests. while there is overlap with the broader definition of gh, iome should not be equated with gh. gh-as the broader term suggests-includes the vision of improvement of health for all people worldwide. in contrast, iome is an area of educational science, an educational concept, a framework, and a means to develop international and intercultural learning outcomes in all students. despite their value as a means of internationalizing the learning outcomes of some students, gh programs-particularly its narrower term-should not be a proxy for the modern concept of iome. goals and outcomes set by gh (i.e., the improvement of health for all people worldwide) are considered the ultimate goal for iome. thus, iome is a means to achieving this ultimate goal, and includes the acquisition of cultural competency and knowledge about health issues worldwide. iome encompasses the international collaborative goals and dimensions between nations and does not solely focus on cultural differences. iome can play an important role by preparing future physicians to practice medicine in a global context and laying the framework for international collaboration and understanding of differences among nations. iome includes, but is not limited to, learning about and understanding differences in international healthcare education, healthcare delivery systems, health economics, health ethics, and health laws; building international peer networks; and providing future physicians with skills in intercultural competencies, collaboration, and international leadership. iome can therefore improve the practice of future physicians by purposefully developing students' understanding of international social, cultural, economic, and ethical differences in healthcare, and by supporting better communication, fostering international leadership, collaboration, and understanding. iome creates awareness building of the importance of these differences regarding patients and healthcare, and creates physician feelings of being part of a global medical world. one can argue that iome highlights meaningful differences between individual nations with the goal of international understanding and improvement in healthcare, whereas gh issues transcend individual nations. iome is a means to improve gh, leading to a globalized and better healthcare world. hanson [15] describes three different models of iome drawing on literature from 1992 to 2010. the "market model" focuses on competition for students internationally and positioning of academic institutions in the global market. hanson argues that this approach, focused on revenue growth, market consolidation, and fiscal returns rather than educative outcomes, typifies the us approach to internationalization. in contrast, the "liberal model" focuses on the promotion of international and intercultural understanding and cooperationintroduced for international higher education in the post-wwii era. curricula in this model might prepare students to work in international or diverse cultural settings, or to be future global collaborators through exchange. the third "social transformation model" emphasizes cross-cultural understanding operationalized "in a spirit of mutuality and reciprocity, through networks or partnerships." hanson argues that this model is the one most closely aligned with approaches to current international practices in medical education. iome necessarily involves multiple players-students, faculty, medical professionals, medical educators, administrators, institutions, and governments [16] -and includes curricular components such as content, pedagogy, assessment, and learning outcomes. the key players include students, for whom iome may involve didactic lectures in the classroom ("iome at home"), peer-to-peer connections and networking, intercultural experiences in clinical settings, outbound mobility activities, and inbound mobility including international student recruitment. other key players are faculty, for whom iome involves research, teaching, and service. and finally, the academic institutions as major stakeholders, who play a significant role in initiating and orchestrating these international endeavors. stutz et al. [17] highlighted the paucity of contextualized national studies on the topic of internationalizing the curriculum in medicine. this is concerning, given the myriad varieties of professional practice, and the impact of our globalized world on local medical practice. the overall goal of this study was to evaluate the current status of iome in the united states (us) by capturing scholarly work on iome in the published literature. the aims were to distill what is being reported and how, and to assess whether more educational research in this area is needed. identifying gaps and limitations in reporting can help iome moving forward as an area of investigational research for the medical educator community. the authors addressed the following question: "what is the current status of scholarly work on the internationalization of medical education in the united states?" intentionally, the authors avoided searching non-scholarly work such as websites or surveyed institutions for international educational curriculum elements or programs. in order to find a way forward, this article summarizes usbased reports on programs, including gh programs at medical schools, discussed over the last 18 years in peer-reviewed journals. the authors focus their discussion on didactics and formats of programs and reasons for gaps in the literature. it provides a picture of the evolving conceptualization and practice of the internationalization of medical education as it is commonly understood and compares this with contemporary approaches to iohe curriculum that have evolved over a similar period of time. the comparison informs suggestions for future approaches to iome. a scoping review was undertaken to identify what has been published on iome in us medical schools. the purpose of this scoping review was to provide an overview of current approaches. the review was conducted in several steps [18] : identify the research question, identify relevant studies, study selection, chart the data, and summary of results. the research was undertaken to answer the question "what is the current status of scholarly work on the internationalization of medical education in the united states?" a scoping review was conducted using a literature search in medline, pubmed, google scholar, and ebscohost. searches targeted the date range from january 1, 2000, to august 1, 2018. this task was performed by an interdisciplinary team consisting of a medical educator, researchers in internationalization, and research assistants in international higher education, from diverse linguistic and cultural backgrounds, working at two major us institutions. the search included keywords, titles, and abstracts of studies from the databases listed. the search terms/keywords or phrases used for this review were as follows: "international," "medical," "education," "united states," the boolean operators "or" and "and" were used to expand and narrow the searches to include all the pertinent publications within the period indicated above. the following limitation was encountered when using a combination of these keywords. for example, a search in pubmed exceeded 180,000 articles. to narrow down the results, an advanced search included the term "curriculum," resulting in over 170,000. since this did not represent a significant reduction of results, the keywords "medical school" and "training" were added. albeit not a term the authors were intending to include initially, "global" was added, resulting in about 46,000 results. adding "usa" or "united states" resulted in about 29,000 hits. combining with "internationalization" and "internationalisation," the search resulted in 77 or 29 articles respectively. none of the articles was deemed appropriate for the intended search as they included other health professions or were not relevant for medical education. given the paucity of the initial search pairing, the authors decided to focus on scanning titles and abstracts. overall, searches using the keyword combinations below were effective in returning a moderate quantity of articles with topic relevance. the authors avoided the term "global health" in the search (1) to not limit the searches to global health education, and (2) to not mix key terms that have different meanings. when hand-searching individual articles, crossreferencing articles that were not found in the initial search were added to supplement the list. pubmed-the keywords "international and medical and education and united states" resulted in a total of 6563 articles using a filter for journal articles inclusive of reviews and systematic reviews. a title and abstract search was then conducted. similar to the pubmed searches, the queries in google scholar, ebscohost, medline, and web of science combining the keywords returned too many results. this limitation was addressed by narrowing the search parameters to scanning article titles, which provided published works with a primary focus on iome. the purpose of our scoping review was to provide an overview of current approaches to iome. as iome as a research field itself is not formally defined and still evolving, modification and adjustment of keywords used in the searches were necessary in order to capture all relevant articles. based on a title and abstract scan, the relevant studies were compiled into a master spreadsheet of 219 non-duplicate articles. this spreadsheet including title and abstracts was reviewed independently by the authors (a.w., e.c., l.u.) and as a group using a set of inclusion and exclusion criteria. criteria for inclusion comprised any articles that reported on international activities and/or programmatic efforts for us medical students. those included, but were not limited to, journal articles of (1) original research, (2) case reports, (3) reviews and systematic reviews, (4) descriptive articles, and (5) program overviews. exclusion criteria were as follows: (1) articles written in non-english language, (2) articles about programs in non-us institutions (unless collaborating with us institution on iome, (3) articles describing international postgraduate (residency) training, (4) reports on international students at us medical schools or foreign medical graduates, (5) international comparisons of medical curricula, (4) opinions/oped on iome, (5) work reporting on offshore us academic institutions or us institutions supporting other nations in building medical curricula/schools, (6) studies from other allied health professions (nursing, dentistry, public health, pharmacy, social work, emergency medicine technician, biomedical sciences, or veterinary medicine), and (7) reflective pieces, book reviews, or commentaries. this narrow screening was deemed important to identify relevant articles on iome and to answer the research question on the current status of, and approaches to, iome in the us. cross-references in identified review articles were only included when deemed appropriate for the purpose of the study. based on this screening, the authors indexed each article according to the extent to which it met the inclusion/exclusion criteria, resulting in 49 articles selected for intensive study. a total of 49 articles over a span of 18 years were identifiedsorted by author, publication year, location, sample, method, major findings, discussion points, and limitations ( table 1) . despite this relatively low count, we found a diversity of internationalization efforts described, including narratives of individual international experiences [42] , research on student initiatives [61] , description of single school programs [24, 25, 37, 43] , institutional partnerships [24, 25, 67] , cultural learning at home [21, 30] , reviews on school websites [54] , and international gh programs [32, 51] . as the goal of the study was to determine the current status of iome in the us via viewing of published literature, the authors identified three overarching themes that summarized the articles, to help in identifying current practices in iome-institutional partnerships, international learning at home, and student mobility programs. definition of each category below is based on descriptions found in iohe [68] : institutional partnerships articles that described programs in which us academic institutions and medical schools partnered with one or more schools or programs outside of the country to provide elements of iome to medical students were listed in this category. in the majority of the articles, these partnerships were formed with low-and middleincome countries (lmics). internationalization at home studies that contained information on international learning and exposure of medical students to either global content or international courses or formats within their school curriculum (at home) fell into this category. studies that included students' pre-departure training for site visits were included. student mobility programs programs describing study abroad, international medical service missions, clinical electives, or short-term rotations were included in this category. many, if not the majority, of the programs described student outbound mobility. while often these studies were part of activities organized via institutional partnerships, some that report trips that were self-organized by students and individual faculty members are included. with few exceptions, most described reports focused on partnerships between us schools and institutions in the lmics, the majority in south america and africa [25-27, 37, 41, 56, 60, 63, 67] , and some in south east asia [34, 49] . rybak described a partnership with the ukraine [56] , and there were two reports of the same partnership with israel [26, 27] . the described partnerships with lmics originated from only nine us medical schools, with some schools reporting on multiple occasions, such as brown university or suny [24, 25, 53, 57, 58, 60] . the report by finkel et al. in 2006 was the only article found that reported on partnerships solely with european schools [29] . ip et al. do not mention formal partnerships with japan and germany, but student exchanges at brown university were administered via an official exchange program; so there is an assumption that a partnership was in place [60] . international institutional partnerships appear to be a common activity associated with the iome, although partnership strength and impact varied greatly. at one end of the scale; formal institutional partnerships were reported "for show" [37, 56, 63, 66] , while at the other end of the scale, they are mentioned as part of other reports on iome activities without further details [39, 55, 62] . overall, there was a full range of number of partners reported, including schools partnering with multiple partners [24, 25, 29, 46] , whereas others report on one specific partnership only [37, 56, 62, 63, 66] . while many more international partnerships may exist, they appear not to 2017 description of a global health program for medical education be utilized or recognized as vehicles for the iome or are simply not reported in the literature. several articles discussed international learning at home including descriptions of on-campus didactics. the articles were wide-ranging, including descriptions of full gh programs, individual courses, local student involvement with refugees, or preparation for travel [19-21, 24, 25, 28, 30-33, 38, 41, 43-45, 47, 56, 61] . two articles were reviews on international gh programs and one an overview of international school activities. drain [32] suggested including gh education (e.g., learning about tropical disease, different cultures) in medical school curricula and/or creating a gh track for interested students. in seven articles, pre-departure training and activities were mentioned [19, 20, 24, 25, 28, 31, 38] . in one article by vora [38] , the international student travels were part of a requirement of an at home international elective. furthermore, we found nine articles addressing or mentioning cultural competencies and language education as part of their international efforts at home [21-23, 30, 33, 36, 51, 55, 61] . of these, two are review/summary articles [36, 69] . griswold described in two reports the involvement of medical students working at home with international refugees [21, 30] -obtaining cultural competency through this experience. we found a much narrower interpretation of the iome at home than that found in the broader scholarship. most articles focused on preparatory training for students prior to travel to a lmic as part of a gh program and failed to report on international or intercultural learning objectives or outcomes of these programs, which medical educators agree should be specific and measurable [51] . thus, while we found evidence for internationalization at home in the peer-reviewed articles we reviewed, it was clearly connected to gh-related student mobility programs and did not engage the majority of students. student mobility, with travel duration ranging from 1 week to several months, involved in almost all cases a low-and middle-income country (lmic). although it is often advocated from work in other countries [70] , we found limited reporting on pre-departure training in the us [19, 20, 24, 25, 28, 31, 38] . ramsey describes pre-departure training but only as a cross-reference for a program description [28] . furthermore, we identified case reports [71] , summaries [54] , reviews [23, 36, 40, 64] , and follow-up studies or reviews that study long-term outcomes of these international experiences [20, 24, 25, 28, 31, 40, 50, 62, 65] . the followup studies focused on effect on career choices and postgraduate international work of the graduates. at least two articles summarized reviews of gh programs [51, 54] . one article reported on the iome not specific to gh programs [36] . finkel [29] described the only article that addressed a focused comparative component of international healthcare systems as an educational goal for students to study while aboard. with very few exceptions, our research demonstrated that us-focused literature regarding international experience during medical school appears limited to work contexts in the lmics. we found limited evidence in the literature of us medical programs engaging with high-income countries (hics) for the purpose of internationalization of the curriculum, teaching, and learning. although historically outbound student exchanges in medical education involved the european countries [24] , only five reports involved student mobility to european or japanese hics [25, 29, 46, 60, 67] according to our search. vora [38] includes european countries such as armenia and the czech republic. in one case, where international placements included australia, students worked specifically with indigenous populations in remote areas [25] . in general terms and overall, our review found a range of literature on the theme of student mobility, from individual small program reports to comprehensive reviews of large gh programs. this suggests attention to both micro and macro detail in considering the impact of gh programs, and a strong emphasis on mobility programs for a minority of students. the articles were published in a wide range of medical journals of relevance to various health professionals, suggesting broad interest. some groups and institutions were represented multiple times in our pool of literature, perhaps reflecting uneven interest in these themes across us institutions. there was a general consensus that these activities have a positive impact for student learning [36, 50] . furthermore, most articles addressing student mobility did not go beyond students' personal perceptions of the impact of their experience on their learning, which is necessarily subjective. this may be a result of no scientific metric of international/intercultural competency being recognized by medical educators [51] . the purpose of this review was to assess the current state of iome in the us by looking at its presence in published scholarly work. in today's world, and for the foreseeable future, graduates of medical programs will require international and intercultural perspectives to perform both their professional and civic duties. while scholarly research on the development of all students' international understanding and intercultural skills in professional programs of higher education has increased dramatically in the last 25 years, the present review of the literature on iome in the us did not find a commensurate rise in published peer-reviewed articles or research on this topic in medical education. a number of themes were identified characterizing the current state of scholarship regarding iome in the us. some of them deserve more attention going forward. overall, we found a focus on program descriptions in the majority of articles with few discussing research into the learning outcomes and goals achieved. the reports were classified into three main categories found in iohe (i.e., institutional partnerships, the internationalization of medical education at home, and student mobility). the significance of the findings of the studies reported was limited because they considered outputs from only a few medical schools and were lacking in comparative data from related professions, and/or for not having a focus on standardized goal settings and outcome research. there was little evidence of structured learning regarding differences in healthcare delivery systems between nations. a number of missing elements and gaps in the three areas of reported activity were identified. these and some possible improvements that could be made are discussed below. first, in relation to institutional partnerships, the authors found no descriptions of how institutional partnerships were used to facilitate peer connections between students and between faculty respectively, or how technology was used to facilitate students at home interacting with students, researchers, or professionals abroad. only one paper reported that students connected via videoconferencing [45] . this is consistent with approaches to international partnership engagement in professional programs outside of medicine, and is recognized in the literature on internationalization of the curriculum as a lost opportunity [1, 72] . reports on how to build a larger global medical community using modern technology and social media led by institutional partnerships will be of utmost importance in the time to come-particularly given recent global health events. as with other areas of online education, existing technology needs to further be embedded and expanded in medical school curricula given the limited reports the study shows. second, the commonly described partnerships with lmics limit students' ability to appreciate the full spectrum of healthcare systems and collaborative opportunities worldwide, and restricts them from acquainting themselves with other hics-thus preventing them from including the "best of each world" in their future practice. a more balanced combination of partners from different areas of the world is suggested. however, the authors also advise against connecting with too many partners, given experiences shared from iohe [8] . third, according to the published reports reviewed, internationalization at home has not been enacted to reach its full potential in medical schools in the us. by comparison, in other disciplines and professional programs (particularly, in the social sciences), there is a stronger focus on identifying international and intercultural learning outcomes for all students, and on how to teach and assess these outcomes at home [4, 73, 74] . given that iome is an emerging concept in medical education, it may be that there is more evidence of internationalization at home in conference papers and institutional reports. however, scholarly work on this topic is of high importance so that educators can standardize goals, research outcomes, agree on best practices, and share ideas and work. the authors suggest that medical educational societies (e.g., the american association of medical colleges, aamc) take the lead and set the framework to encourage sharing this work. finally, student outbound mobility in us medical schools-representing the main theme for reports on iome-appears to be reported when it involves the lmics. in contrast, other industrialized countries, mainly european, more commonly report outbound student mobility and related activities in other hics [75] . this may be due to entrenched european medical student exchange programs which have been in existence for several decades, stimulated and supported by the erasmus program of the european commission [76] and others. the authors advocate increasing bilateral exchange between the us and other hic to be more inclusive for building a global medical world. while mobility programs are an important aspect of internationalization, they are on their own insufficient. global estimates reflect that at most 10% of students will engage in mobility programs. furthermore, the international literature recognizes that students from marginalized backgrounds are less likely to participate in these programs. approaches to internationalization that focus primarily on mobility are likely to exacerbate rather than address recognized inequalities in medical education and the practice of medicine [14] . another area that is not well researched is medical students (often from the aforementioned marginalized backgrounds) who obtain part or all of their medical training in the caribbean and move to the us for medical practice. these students and their experiences in the lmics provide diversity, an important component for iome at home [77] [78] [79] [80] . however, these students are not captured within reports about iome. likewise, in this report, studies on international students were not included, to focus solely on reports about offerings from us medical schools. the authors argue that the reported approaches for student mobility as an element for iome are insufficient and call for an internationally informed approach to iome that focuses on all students and medical educators. furthermore, during a time of discussion about climate change, increasing ethical sensitivity, and sustainability of global resources, it is important to take a step back and rethink current existing formats for student mobility as a means for iome. particularly in the wake of the 2020 pandemic, medical educators need to evaluate if medical students' outbound short-term missions to the lmics is ethical, safe, and justified, and whether goals and outcomes achieved from these missions can be acquired via other alternative routes. the authors argue that optimizing internationalization at home may be a suitable alternative and should be the focus of further research. the very limited number of published articles on comparative international health systems requires further investigation. while us schools of public health offer these fields of study, most medical students apparently have no access to acquiring knowledge about this topic. for future collaborative international work, this information is deemed important by the authors. considering that medical school curricula generally leave no space for the inclusion of additional coursework, medical educators need to come to a decision regarding what is deemed necessary in the future. furthermore, in the us, medical educators are mostly practicing clinician-researchers with little or no international professional experience and little time for curriculum review and innovation. unless medical educators, as a community, identify and articulate a strong need for the internationalization of medical education for all students, it is likely that it will continue to be regarded as an extracurricular, optional activity for an elite minority. we found little connection between the iome in the us and the broader literature on iohe and the curriculum. educational research is an acknowledged field of study in its own right, as is internationalization. although 87% of us medical schools have established internationalization programs, we identified a relatively small number of peerreviewed articles on the topic over a span of nearly 20 years [36, 54] . lack of research and publication has a significant impact on exchange of experience, programming efforts, and quality improvement. increased communication among us and international peers via publications and conferences can have a positive impact on medical education and is important for the future of the iome. the limited publication of peer-reviewed articles on the internationalization of medical education may not be a reflection of the iome in the us occupying a low priority within the medical educator community, but the consequence of unreported programs and experiences. while clearly international options are available at most us medical schools, the lack of reporting found in this study warrants further investigation and research [36] . considering that international programs and activities involve a significant amount of resources, an analysis of financial impact or studies on cost efficiency would appear to be important but were not easily found [24] . at a time of growing globalization of healthcare, the internationalization of medical education can help in improving the global healthcare world. despite a high proportion of international offerings in us medical schools, this study demonstrates that very limited sharing of experiences via published work in this field exists. the reported literature details three areas of medical educational research consistent with research in international higher education: institutional partnerships, internationalization at home education, and student mobility. all three areas will benefit from further investigation and expansion to promote internationalization of medical education for all students. these improvements should be investigated in an interdisciplinary manner in collaboration with the longstanding field of international higher education, thus leading to the betterment of global health. internationalizing the curriculum: taylor & francis global perspectives on higher education engaging the disciplines in internationalising the curriculum in the disciplines comparing internationalisation of the curriculum in action across disciplines: theoretical and practical perspectives redefining internationalization at home. the european higher education area: amsterdam university of applied sciences using formal and informal curricula to improve interactions between home and international students redefining internationalization at home internationalization of higher education -nine misconceptions. international higher education introduction to public health: center for disease control the world health organization and the transition from "international" to "global" public health towards a common definition of global health what is global health? glob health action global health': meaning what? viewpoint -internationalisation of medical education: hierarchies, inequalities and future directions https:// ipedjournal.com/2015/03/30/viewpoint-internationalisation-ofmedical-education-hierarchies-inequalities-and-future-directions critical perspectives on internationalising the curriculum in disciplines global health competencies and approaches in medical education: a literature review preparing medical graduates for an interconnected world: current practices and future possibilities for internationalizing the medical curriculum in different contexts scoping studies: towards a methodological framework new world views: preparing physicians in training for global health work an international health/tropical medicine elective refugee health and medical student training the effect of medical students' international experiences on attitudes toward serving underserved multicultural populations educational effects of international health electives on u.s. and canadian medical students and residents: a literature review a third world international health elective for u.s. medical students: the 25-year experience of the state university of new york, downstate medical center a global health elective for us medical students: the 35 year experience of the state university evaluation of student attitudes and knowledge in a new program in international health and medicine a medical school for international health run by international partners career influence of an international health experience during medical school teaching medical students about different health care systems: an international exchange program refugees and medical student training: results of a programme in primary care capturing medical students' idealism global health in medical education: a call for more training and opportunities three domains of competency in global health education: recommendations for all medical students going home: lessons for a brown medical student in cambodia. medicine and health, rhode island a gain in cultural competence through an international acupuncture elective international exchange programs and u.s. medical schools sustainable international partnership building for academic medical centers: experiences with the botswana-upenn partnership. the virtual mentor a studentinitiated and student-facilitated international health elective for preclinical medical students medical student surgery elective in rural haiti: a novel approach to satisfying clerkship requirements while providing surgical care to an underserved population effects of international health electives on medical student learning and career choice: results of a systematic literature review new academic partnerships in global health: innovations at mount sinai school of medicine. the mount sinai journal of medicine the impact of global health experiences the global health curriculum of weill cornell medical college: how one school developed a global health program how we developed a locally focused global health clinical preceptorship at weill cornell medical college global health education for medical students: new learning opportunities and strategies student objectives and learning experiences in a global health elective bridging the global health training gap: design and evaluation of a new clinical global health course at harvard medical school partnering on education for health: muhimbili university of health and allied sciences and the university of california san francisco student perspectives on international/rural experiences in medical education educational objectives for international medical electives: a literature review global health education in u.s. medical schools international surgical clerkship rotation: perceptions and academic performance practical aspects of the kenya medical exchange program structured global health programs in u.s. medical schools: a web-based review of certificates, tracks, and concentrations global health educational engagement -a tale of two models building international collaborations from the ground up: brown university partnerships in haiti and ukraine the brown kenya medical exchange program: an overview the medical exchange: brown, moi students/ residents share their experiences designing an ethics curriculum to support global health experiences in surgery part ii -brown university's program in liberal medical education (plme): medical exchange programs; student research development of global health education at johns hopkins university school of medicine: a student-driven initiative career choices and global health engagement: 24-year follow-up of the guatemala-penn partners: an innovative inter-institutional model for scientific capacity-building, healthcare education, and public health bidirectional exchanges of medical students between institutional partners in global health clinical education programs: putting ethical principles into practice on becoming a global citizen: transformative learning through global health experiences. annals of global health reframing undergraduate medical education in global health: rationale and key principles from the bellagio global health education initiative ensuring the health, safety and preparedness of u.s. medical students participating in global health electives overseas comprehensive internationalization-from concept to action global health education: international collaboration at icddr. b j health pop nutr pre-departure training and the social accountability of international medical electives. education for health international study in premedical education mapping internationalization on u critical perspectives on internationalising the curriculum in disciplines: reflective narrative accounts from business, education and health (global perspectives on higher education) whitsed c, editor: sensepublishers internationalisation of the curriculum in an undergraduate nursing degree the profession of medicine: a joint us-german collaborative project in medical education erasmus cultural exchanges in europe medical education in the caribbean: a longitudinal study of united states medical licensing examination performance medical education in the caribbean: quantifying the contribution of caribbean-educated physicians to the primary care workforce in the united states the medical education of united states citizens who train abroad -citizen international medical graduates-a boon for the workforce? publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations acknowledgements the authors would like to thank laura rumbley, phd, at the european association for international education (eaie) for her helpful discussions. many thanks to michael fortgang, md, for helpful review of the manuscript. conflict of interest the authors declare that they have no conflict of interest. key: cord-332497-f1lbnpwi authors: santos, justin j.; chang, donald d.; robbins, katherine k.; cam, elise le; garbuzov, anna; miyakawa-liu, monica; goodwin, bailey; malik, iman; tholen, carl; green, mark; archie, samantha; tucker, tiffany; ebberman, monique; denton, gerald dodd title: answering the call: medical students reinforce health system frontlines through ochsner covid-19 hotline date: 2020 journal: ochsner j doi: 10.31486/toj.20.0065 sha: doc_id: 332497 cord_uid: f1lbnpwi nan on saturday evening, march 14, 2020, third-and fourthyear medical students at the ochsner clinical school (ocs) received an urgent email from their school administration: due to the growing covid-19 pandemic, there have been over 1,500 calls to the ochsner call line with over a 3-hour wait time …we are looking for volunteers to assist in answering the ochsner call line and help triage patients. the danger and scope of the pandemic were underscored the following day when the american association of medical colleges ordered clinical rotations to pause. 1 the idea for a student-run call center was born when the ocs administration, led by leonardo seoane, md, facp, senior vice president and chief academic officer; g. dodd denton, md, assistant dean and associate professor; and carl tholen, assistant vice president of education operations, were exploring options for how best to support patients, families, employees, and students amid the changes and safety restrictions implemented as covid-19 emerged and began to spread in louisiana. anticipating the mandated pause in clinical rotations, the administrators recognized the value of engaging students in a support role. clinical triage would provide both an educational opportunity for the medical students and an important resource for the community. within 24 hours after the initial call for volunteers was sent, ocs administration received responses from more than 100 students interested in staffing the call line. on monday, march 16, 2 days after the call for volunteers, phones were installed at multiple stations in the academic testing center, and a crash course orientation was held in the afternoon. the following day, students began taking the first calls. from that point on, ochsner medical students supported the nurse triage call line 12 hours per day (8:00 am to 8:00 pm), 7 days per week, from march 17 through april 26. as with any project born out of a crisis, the call center required intense effort in the beginning. during the first week, 3 ochsner student leads worked closely with call center staff each morning to ensure that the information provided to patients was up to date and to develop training materials for onboarding incoming volunteers. samantha archie from the ochsner access to care division was present in the call center monday through friday to provide technical support, ranging from call station functions to electronic medical record system (emr) navigation. monique ebberman and tiffany tucker established medical student access to the emr and managed emr training and troubleshooting. mark green of the patient engagement and outreach division coordinated data aggregation for the call center and acted as liaison between the student leadership and ochsner call center management. these members of the ochsner call center team provided the necessary experience and expertise as the medical students familiarized themselves with the call center workflow. one of the greatest challenges during the initial phase of operation was ensuring the students had the latest updates regarding available covid-19 testing sites and specific screening criteria for each site. with the influx of information from multiple sources, including the ochsner covid-19 sharepoint website, nola ready text updates, social media platforms, and the louisiana children's medical center (lcmc) covid-19 call center, student leads and call center staff spent hours each day cross-referencing updates among sources and calling sites to verify testing criteria and availability status. the combined workload of continuously updating information and making sure volunteers of varying experience levels were trained and informed quickly overwhelmed the student leads. during the first week, the 3 leads worked both on shift, often in overlapping shifts, and at home as they designed training modules and workflow guides for volunteers. training modules consisted of topics as diverse as how to work a phone to how to navigate the emr. workflow guides included a library of frequently asked questions; detailed scripts for fielding calls; and algorithms for determining screening recommendations for patients, as well as which testing site patients should be referred to based on patient location and site-specific criteria such as the presence and severity of symptoms, age and living conditions, and immunocompromised status. at the end of the first week, the student leads created a shift coordinator system to distribute the workload. the coordinator roles were filled by 4 volunteers proficient with the call center workflow. coordinators were tasked with santos, jj training and briefing volunteers, assisting in updating information, and helping maintain continuity from shift to shift by performing handovers that included a review of any updates or challenging calls received during the prior shift. any call that presented a unique problem was worth discussing, such as patients with atypical symptoms, calls from lesser-known areas of the state, or callers with special circumstances such as healthcare employees. the student coordinators developed a schedule to ensure that all 3 shifts were covered each day. the impact of the student call center was quickly seen. by late march, 2 weeks after the first call was taken, the call line wait time had dropped from more than 3 hours to less than 10 minutes. during the first 3 weeks, more than 100 medical students staffed the call center, with 7 to 15 students taking calls every shift, volunteering nearly 2,000 hours, and assisting more than 5,200 callers. the most calls fielded in a single day was 409. assigning medical students to the ochsner covid-19 symptom tracking program was a natural extension of their call center role. the symptom tracking program was developed for patients who tested positive for covid-19, had symptoms but did not meet criteria for testing, were awaiting test results, had received positive results and were still experiencing symptoms, and anyone else who may have had the virus and wanted to continue to be checked on. beginning on march 29, the symptom tracking program provided an additional layer of support for patients to monitor their symptoms on a daily basis for 14 to 30 days and ensured they had a resource for their questions and assistance if they began to feel worse. patients received daily text messages inquiring about their symptoms. patients who responded with worsening symptoms or questions would be contacted by a nurse or medical student for triage. the goal was to make sure any patient who needed escalation of care was referred to the appropriate provider and to provide reassurance to patients during a time of uncertainty. another goal was to ensure that patients understood their responsibility to prevent the virus from spreading by following the recommendations from the centers for disease control and prevention. in early may, the medical students' role continued to expand as they began to assist with the ochsner covid-19 prevalence study that involved testing more than 2,500 community members in orleans and jefferson parishes. the aims of this study were to gain an understanding of the spread of the virus, help healthcare providers and local leaders determine how many residents were infected, and to help inform how the city could safely reopen. medical students answered community members' questions related to the study, assisted people with study enrollment through the test nola website, booked ride services for people without transportation, and directed patients to available testing sites. overall, the impact of the ochsner medical students' volunteerism extended beyond patient triage, symptom tracking, and study enrollment. serving as a volunteer for the call center helped many medical students cope with the challenges of social distancing and the pause in their clinical training. instead of staying home, the students served the community and safely engaged in meaningful human connection. they gained an understanding of and appreciation for healthcare roles that many had not previously encountered. close collaborations with nurses, information technology staff, and administrators allowed the medical students to learn about their positions and create meaningful partnerships throughout the health system. as dr denton pointed out, "we are living in a historical moment … and while medical students traditionally are restricted in what they can do, here is an opportunity where they can directly contribute and help stem the tide of the growing covid-19 pandemic." important guidance for medical students on clinical rotations during the coronavirus (covid-19) outbreak association of american medical colleges ©2020 by the author(s) this article is an open access article distributed under the terms and conditions of the creative commons attribution (cc by) license (creativecommons.org/licenses/by/4.0/legalcode) that permits unrestricted use, distribution, and reproduction in any medium the authors thank maggie thompson for her revising and editing contributions. key: cord-284985-q3eiyp7e authors: o’connell, mary beth; samman, leah; bailey, teresa; king, larissa; wellman, gregory s. title: attitudes of michigan female college students about pharmacists prescribing birth control in a community pharmacy date: 2020-06-09 journal: pharmacy (basel) doi: 10.3390/pharmacy8020099 sha: doc_id: 284985 cord_uid: q3eiyp7e in the united states, the overall unintended pregnancy rate is about 45%. women between 20–24 years old account for 59% of the unintended pregnancies. continuous birth control use is related to decreasing unintended pregnancies. therefore, we assessed female college students’ opinions about pharmacists prescribing birth control in a community pharmacy using an intersectionality framework. a survey with 49 items about provider attributes, pharmacy services use and evaluation, advantages and barriers of pharmacists prescribing birth control, sexual and reproductive history, and demographics was distributed by survey link and qr code. recruitment was done by investigators and students (snowballing technique) via emails, social media posts, and direct student contact. respondents (n = 859) were 23.0 ± 4.9 years old, 83% white, 64% healthcare students, 32% student pharmacists, 69% sexually active, 68% with at least one episode of unprotected intercourse within a year, and 29% never using condoms. forty-six percent of students were extremely likely and 26% moderately likely to get birth control from a pharmacist because it would be easier to adhere to birth control, could prevent unintended pregnancies, would be more convenient, and require less time. concerns included the lack of pap screenings and prescriptions written for the wrong birth control. within most student characteristics or attitudes assessed, at least 70% of the students would use this service. based on student opinions, female college students would use pharmacists prescribing birth control services. unintended pregnancies are recognized as individual, family, and societal concerns. forty-five percent of pregnancies are unintended in the united states with higher prevalence in those 20-24 years old, with low income, and/or from minority groups [1] . the cost of unintended pregnancies was estimated to be $11.1 billion dollars per year [2] . of the unintended pregnancies, 49% of the mothers give birth, 36% have an elective abortion, and 15% have a miscarriage [3] . prior to an abortion, 54% of the women were not using contraception and 29% had wanted to use birth control but experienced barriers to obtaining it [4] . about 19 million women of childbearing age live in areas without federally funded facilities that prescribe birth control [5] ; however, many of these areas would have a community pharmacy. other barriers include cost and insurance coverage of contraception, payment of aged 20-24 are particularly important since they have the highest prevalence of unintended pregnancy in the united states [2] . therefore, the purpose of this study was to determine female college students' opinions about and willingness to use pharmacists for obtaining hormonal contraception in a community pharmacy across a wide range of student characteristics, attitudes, and health service opinions. investigators developed a survey that was distributed to a convenience sample. we chose an intersectionality approach by querying about characteristics, attitudes, sexual behaviors, and health services use and opinions ( figure 1 ) that overlap to influence a woman's health decision making. the research project was deemed exempt via category 2 criteria (because identity of the students would not be ascertained from the data and no criminal or civil liability or student damage could be determined from results) by two university institutional review boards. an assent information form was included as the first question of the survey. viewpoints of women aged 20-24 are particularly important since they have the highest prevalence of unintended pregnancy in the united states [2] . therefore, the purpose of this study was to determine female college students' opinions about and willingness to use pharmacists for obtaining hormonal contraception in a community pharmacy across a wide range of student characteristics, attitudes, and health service opinions. investigators developed a survey that was distributed to a convenience sample. we chose an intersectionality approach by querying about characteristics, attitudes, sexual behaviors, and health services use and opinions ( figure 1 ) that overlap to influence a woman's health decision making. the research project was deemed exempt via category 2 criteria (because identity of the students would not be ascertained from the data and no criminal or civil liability or student damage could be determined from results) by two university institutional review boards. an assent information form was included as the first question of the survey. female college students of any age from any college in any curriculum were eligible to participate. students selecting intersex as their gender were included but male students were excluded. surveys needed to have at least one other item besides gender answered to be included. surveys from students residing or attending college outside of michigan were excluded. recruitment was primarily conducted at two universities and two community colleges. these educational institutes were chosen to gather student opinion throughout the whole state and provide metro, urban, and rural geographical variability. one university was in detroit, one community college in a detroit suburb, one university on the west side of the state, and one community college in the upper peninsula. a validated survey related to our project was not available. literature was reviewed related to over-the-counter birth control, emergency contraception, pharmacists prescribing contraception, female college students of any age from any college in any curriculum were eligible to participate. students selecting intersex as their gender were included but male students were excluded. surveys needed to have at least one other item besides gender answered to be included. surveys from students residing or attending college outside of michigan were excluded. recruitment was primarily conducted at two universities and two community colleges. these educational institutes were chosen to gather student opinion throughout the whole state and provide metro, urban, and rural geographical variability. one university was in detroit, one community college in a detroit suburb, one university on the west side of the state, and one community college in the upper peninsula. a validated survey related to our project was not available. literature was reviewed related to over-the-counter birth control, emergency contraception, pharmacists prescribing contraception, obstetrics and gynecology practice/provider patient preferences, pharmacy clinical services in community pharmacies, and position statements related to these topics to create our survey. survey items were developed based on published non-validated survey items or newly created to assess this study's purposes. a 46-item survey (appendix a) was initially developed with two additional items (i.e., college attending and current city of residence) added after the study started. the survey topics (in survey order) were assent (1 item), health care experience attributes of gynecologists or clinic providers and future pharmacists prescribing birth control (2 items with 9 sub-items each), pharmacy services and evaluation (3 items), pharmacist prescribing birth control advantages and barriers (6 items), demographics (21 items), and sexual and reproductive history (15 items) . for the health care experience attributes section, students were presented with two scenarios where she obtained a birth control prescription from a gynecologist or primary care provider and one from a community pharmacist with the ability to prescribe birth control. students were then asked to rate their perception of nine provider attributes for both scenario providers on a scale of one (lowest) to ten (highest). students could skip answering any of the items. no personal identification information was collected. a second survey was created for the raffle of two $250 gift certificates. the survey platform was qualtrics. the survey was distributed to ten students from the two universities for review of item clarity and understanding, and survey completion time. suggestions were incorporated into the final survey. the qualtrics survey url link was used and later the survey qr code was added to advertisements. one university posted an advertisement two times a week in a daily newsletter sent out to every student over a span of six weeks. they also sent a mass email containing the survey link to all college of pharmacy students at this university. at the other university, a study notice or flyer was posted in student news feeds, research recruitment webpage, campus housing, facebook pages and other social media. both universities sent emails to female student organizations asking for them to email or post the advertisements. some in class pharmacy and other program announcements were made. students were also asked to participate while dining in the university commons area. one community college sent out broadcast emails to the whole student population. the other community college sent advertisement to one curriculum, i.e., nursing, for which a student mass email list existed. students participated in snowballing techniques by sending emails to their female college friends or posting on their own social media sites. recruitment occurred from april 2019 to february 2020. target goal of 1500 completed surveys was not achieved. descriptive statistics with inferential statistics, chi square, and ordered logistic regression were used to analyze the data. for chi square analyses, five-point likert scales were collapsed to three-point likert scales. for likelihood of obtaining birth control from a community pharmacist, those stating no birth control use were excluded from crosstab analysis. extremely and moderately likely responses were collapsed to likely, not sure responses remained, and not very likely and not likely at all responses were collapsed to not likely. these analyses were done with ibm spss statistics version 26.0 (ibm corp., armonk, ny, usa). for the ordered logistic regression, the full five-point likert scale and the students not using birth control were included. the pharmacist attributes were on a scale of one being low to ten being high. this analysis was run on 816 of the respondents using listwise deletion. the ologit module in stata 16.0 was used for this analysis. a p-value of less than or equal to 0.05 was considered significant. because of the survey distribution techniques, total sampling cannot be quantitated and therefore a response rate cannot be calculated. the survey was opened by 1110 students with 859 (77.4%) surveys included in the analysis. the reasons for survey deletions were 8 surveys had no responses, 73 were started by male students, 153 had no answers except for gender (1 intersex, 152 women), 11 were completed by female students attending non-michigan colleges, and 6 were completed by female students living outside of michigan. the average age and standard deviation of the participants was 23.0 ± 4.9 years (range 17-56 years old). the student participant characteristics and associated likelihood to obtain birth control from a pharmacist are in table 1 . overall, female college students stated they would obtain birth control from a pharmacist with most students stating they were extremely likely (46.3%) to do so, followed by 26.3% moderately likely, 12.7% unsure, 8.7% not very likely, and 3.3% not likely at all. the mean response and standard deviation was 2.1 ± 1.3 [six-point scale with one (extremely likely) to six (never)]. students had positive comments about pharmacists prescribing birth control but also some concerns. survey advantages and concerns prompted responses (survey items) are listed in table 2 . the mean number of prompted and other stated advantages was 4.8 ± 1.6 with most students listing five (27.4%) or six (31.7%) advantages. additional advantages included a pap or pregnancy test not needed (3), insurance coverage checked (3), birth control switching easier (3), less stress (2), blood pressure checked (1), fewer appointments (1), pharmacists more knowledgeable (1), pharmacists more available for questions (1), easier to get in an emergency (1), easier to get refill (1), and easier to get answers (1). five students (0.6%) listed no advantages. the mean number of prompted and other stated concerns were 2.1 ± 1.3 with most students listing one (34.3%) or two (24.9%) concerns (table 2) . additional concerns included no medical chart access (16) , pharmacists are overworked (12) , relationship with physician needed (10), lack of privacy (8), less education than other providers (7), pharmacists need more birth control education (6), insufficient patient education provided (5), lack of insurance coverage of visit (2), should be pharmacist and doctor team (1), limited birth control options to be prescribed (1), pharmacist not able to resolve side effects (1), pharmacist refusal to dispense (1), non-hormonal options needed (1), less professional environment (1), lack of documentation (1), and uncomfortable with pharmacists prescribing (1). forty-eight students (5.7%) listed no concerns. in table 1 , the percent of students stating they were extremely or moderately likely to get their birth control from a pharmacist in a community pharmacy is displayed. only citizenship, degree program, health professional student status, and type of health professional student had significant differences within the category. sixty-seven to 77% of the various religious/spirituality groups would obtain birth control from a pharmacist. one third of the students would use birth control even if their religion opposed this medication. one percent stated because their religion did not support birth control, they would not use this medication. for 4.7% of the students, they stated their religion only approved birth control if married, and 1.7% stated permission to use birth control only after marriage and with partner approval. almost a third of the students (31.1%) had no beliefs that would impact birth control use. some students (27.7%) had religions that did not oppose birth control use. sexual practices of the students and likelihood of obtaining birth control from a pharmacist results are presented in table 3 . many students (68.8%) were in a sexually active relationship with a man and 3% with a woman. about 14% of the students stated they did not need birth control because they were not sexually active (9.5%), or were incapable of getting pregnant (1.6%), pregnant (0.3%), or postmenopausal (0.3%). nineteen students (2.2%) practiced abstinence. prescription or provider administered birth control used by the students were 49% pills, 9.1% iuds, 3.4% implants, 2.6% shots, 2.1% emergency contraception, and 2.0% rings. of those using hormonal birth control, most students had only one indication (42.8%), with 41.3% having two indications, and 15.9% having three indications. the most common indication for birth control was contraception (53.6%), followed by menstrual disorders (38.8%), acne (18.5%), and other (3.7%). for the specific sexual practice variables, 69% to 89% of the female college students stated they were extremely or moderately likely to get their birth control from a pharmacist (table 3) . only sexually active with a man (p = 0.027) and times of unprotected intercourse in the last year (p = 0.044) had significant differences within the variable. an ordinal regression on likelihood of obtaining birth control from a pharmacist did not yield a significant model for the variables age, age at first intercourse, number of sexual partners, or number of times intercourse per week (model significance p = 0.7413). previous use and quality assessment of pharmacy services and their impact on opinions about obtaining birth control from a pharmacy are depicted in figure 2 . having received a vaccination from a pharmacist (p = 0.01), having greater confidence in pharmacist prescription dispensing (p = 0.01) and counseling (p = 0.000), and believing pharmacists have more knowledge than their doctor, nurse practitioner, or physician assistant (p = 0.000) were associated with greater likelihood of getting their birth control from a pharmacist. most female college students thought that receiving birth control from their pharmacist would definitely (59.5%) and probably (30.5%) increase their birth control adherence, and would definitely (63.0%) and probably (27.0%) decrease unintended pregnancies. the greater the belief of an impact of pharmacists prescribing birth control on increasing birth control adherence (p = 0.000) and decreasing unintended pregnancies (p = 0.000), the greater the likelihood of receiving their birth control from a community pharmacist. students ranked personal attributes of a pharmacist in the 8.4 to 8.9 (out of 10) range with some aspects of the pharmacy and getting birth control in that environment lower (7.4 to 8.7). the results are shown in table 4 . using the nine attributes in table 4 , an ordered logistic regression analysis was done with the variable female college students' likelihood of getting their birth control filled at a community pharmacy. the scale for likelihood of getting their birth control filled at a community pharmacy was reversed to aid in the interpretation of the odds ratios. the model was significant in describing pharmacist prescribed birth control (p < 0.0001). the attributes trustworthiness (or 1.267, ci 1.130-1.420, p < 0.001), approachability (or 1.127, ci 1.005-1.265, p = 0.041) and visit expense (or 0.890, ci 0.824-0.960, p = 0.003) were statistically significant in predicting likelihood of using a pharmacist as a primary provider for contraceptives. the greater the perception of trustworthiness and approachability the students had for the pharmacist, the greater their likelihood to use the pharmacist as a primary provider for birth control. the lower the students perceived the pharmacy visit expense to be, the more likely they would use the pharmacist as a primary provider of birth control. this study provided evidence that women are interested in pharmacists prescribing birth control. seventy-three percent of michigan female college students felt they were likely to obtain their birth control from a community pharmacist. they felt the community pharmacist visit for birth control would be more convenient, save time, and make obtaining birth control easier. they also felt this pharmacy service would help them continue and be more adherent to their birth control, and prevent unintended pregnancies. the predominant concerns the students had were not getting regular pap smears and screenings and potentially being prescribed the wrong birth control. these advantages and concerns are similar to other studies [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] . many women do not know that a pap smear and breast exam are not required for birth control use [2] . in terms of targeting this service to select groups of college students, we found across all our student characteristics to have high support for obtaining birth control in a community pharmacy, making the service generalizable across diverse students. overall, at least 70% of the female college students across the various student characteristics stated they would use pharmacist birth control services, a measurement of acceptability. no statistically significant differences were seen between race and ethnic groups, religion/spirituality groups, residence while going to college, size of city living in during childhood, political party, type of college, student enrollment status, employment, and medical or medication insurance coverage. students from a foreign country (i.e., student visa or immigrant status) were more likely to use this service than american born and naturalized citizen students, which might be related to lack of a primary care provider, health insurance, and or transportation, or non-prescription access to contraception in their birth country. healthcare students were more likely than non-healthcare students to use a pharmacist for birth control, which might reflect greater understanding of pharmacist education and training and subsequent medication expertise. variability within the healthcare students existed with pharmacy students having the highest, medical students in the middle, and nursing students the lowest likelihood. these differences could be related to territoriality and profession-centrism regarding writing prescriptions for patient management [25] . no differences existed within the various non-healthcare student disciplines, which could mean similar awareness of pharmacist skills and training. graduate/professional degree students were more likely to use their pharmacist for birth control than undergraduate and associate degree students, which could represent healthcare professional awareness, maturity, and independence in health decisions. we thought more risky sexuality practices might increase willingness to access birth control from a pharmacist, but variability in willingness to use this service did not vary by type of relationship, number of unprotected intercourse events per year, condom use, or worry about getting pregnant. our study is the first study with a diverse female college participant cohort to assess opinions about pharmacists prescribing birth control, a group which could use this service. we chose this cohort since college students are in the age groups with the highest unintended pregnancy rates [1] . another college student study surveyed california student pharmacists and almost all the student pharmacists (96%) wanted to provide this service [23] . they were not asked if they would use this service. these student pharmacists identified similar advantages as our college students about this pharmacy service. they had more concerns than our students such as the adequacy of pharmacist training, lack of a medical chart with concerns about patient data adequacy, and no incentive for pharmacists providing this service. by knowing more about pharmacy operations, student pharmacists could conceptualize this service differently than non-student pharmacists. teenagers and young adults also liked the privacy and convenience of the community pharmacist prescribed birth control [16, [19] [20] [21] . they also had some similar concerns to the college students such as pharmacists' abilities, minimal patient specific data available, and limited birth control products. similar to our student cohort, analysis of public commentary to news articles from women and men in oregon supported pharmacists prescribing birth control, citing the service would be an asset and allow women to make reproductive health decisions [26] . within the comments were similar concerns about incorporating this service into pharmacy operations, importance of physician patient relationship, safety issues, and insurance coverage. unintended pregnancy rates are higher in minority women and those living in poverty [2] . in our study, many minority women (68-83%) would get birth control from a pharmacist. also, in our study, 69% of the female college students with a public insurance plan would get their birth control at a pharmacy. how many of these students would use this service is unknown. in oregon, a state in which medicaid offers and pays for this service, 10% of women with this insurance benefit chose a pharmacy to obtain their new birth control prescriptions [26] . in california, oregon, and new mexico, pharmacists offered birth control prescribing in rural and urban community pharmacies [27, 28] , which would align with our female college students in rural, small, and urban residences willing to get birth control from a community pharmacist. past experiences with pharmacists influenced the female college students' likelihood of having a pharmacist prescribe birth control to them in a community pharmacy. using pharmacist birth control services in the future varied based on past pharmacy services used and confidence in pharmacists. having received a vaccination from a pharmacist and being more confident with pharmacists filling prescriptions and providing counseling were associated with a greater likelihood of obtaining birth control from a community pharmacist. about one third of the students (37.3%) thought pharmacists knew more about birth control than primary care providers, which could be influenced by the high percentage of pharmacy students in the database. students' rankings of pharmacist attributes were very positive, at least 8.4 or higher on a 10-point scale. pharmacy attributes such as privacy, time, and cost were lower but still above 7.4 on a 10-point scale. pharmacist trustworthiness, approachability, and visit expenses significantly influenced their likelihood of using birth control pharmacists. in the opinion-based studies about pharmacist prescribing, women of all ages would use or support this service, but they do express concerns as well [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] . our research adds that prior use of pharmacy services and perceptions of pharmacist attributes influence these opinions. our study had a few limitations. the results represent one midwest state and only college student opinions, which might not represent other states or women of different ages. within all demographics, likelihood to use pharmacist birth control prescribing was high. the chance to win one of two $250 gift cards was most likely not an incentive for survey completion. since the survey incentive was insignificant, respondent bias could exist with those more positive or opposed more likely to complete the survey. recruitment was challenging with only 857 completed surveys after mass emailing, social media posts, and face to face recruiting at two major universities and two community colleges. difficulty recruiting for another public perception of pharmacists prescribing birth control also was noted in a study using social media to recruit women of any age [19] . only 2.2% of 44,994 social media advertisements on facebook were clicked on by viewers, and only 28.9% of those clicks resulted in opening the survey. although the survey had good diversity with most demographic variables, race and ethnicity diversity were limited with 83% of the students being white and 4.4% being hispanic. in michigan, 66% of the undergraduate college students are white, 12.1% black, 5.3% hispanic, 4.6% asian, 0.6% american indian or alaska native, 3.5% of two or more races [29] . we attempted targeting minority female college organizations and courses, but these strategies were not successful. our study represented the group with highest number of unintended pregnancies (60% were 20-24 years old) but other groups such as teenagers (19%) and those 30 years and older (8%) were less represented. the survey was long because we had 4 areas to explore. survey burden might explain the number of students that opened the survey without answering anything besides their gender or did not finish or skipped certain sections. our pre-study student sample stated the survey was doable. some but not all students might have been aware of pharmacists prescribing birth control prior to the study so different degrees of previous opinion development. many student pharmacists could have learned about this service in class or at national and state pharmacy meetings. learning about this service might have occurred in other curriculums. medical students might have discussed the american college of obstetrics and gynecology preference for over-the-counter birth control, which could have influenced their opinions on pharmacists prescribing contraception. little is known about the coverage of this new pharmacy service in other healthcare and non-healthcare (e.g., law, women's studies) student curriculums. at the initiation of this survey, this pharmacy service was not available in michigan. now one large pharmacy chain in michigan is offering this service through a collaborative practice agreement. the pharmacy chain started its service the month after our recruitment ended, which was the month covid-19 hit michigan. prior to covid-19, minimal marketing had occurred and during covid-19, pharmacists prescribing birth control was not the focus of pharmacy patient health care, so most likely this new pharmacy provider of birth control had little impact on students' opinions. many female college students in michigan are receptive to pharmacists prescribing birth control in a community pharmacy. they feel this service will make getting and adhering to birth control easier and decrease their worries about getting pregnant. their positive attitude results combined with reported prevention of unintended pregnancies and decreased health care costs outcomes in oregon [17] support all states offering pharmacists prescribed birth control services. further research is needed to confirm women's and societal benefits and use of pharmacists prescribing birth control. the authors would like to thank travis blume, bay de noc community college, and mary miles, e.d, oakland community college health professions for recruiting at their colleges, and all students that helped with snowball recruitment strategies. the authors declare no conflict of interest. q4 recall a previous experience with, or imagine visiting, a gynecologist or clinic provider for a prescription for birth control. how would you rate each of the below topics related to that visit on a scale of 1 being the lowest to 10 being the highest? 1 2 3 4 5 6 7 8 9 10 provider trustworthiness provider approachability provider respecting me provider knowledge of birth control birth control education received privacy ease of making an appointment time commitment-all time related to getting the birth control prescription, e.g., calling to get an appointment, waiting time to see provider, provider interaction, getting the prescription filled expense-all costs related to getting the birth control prescription, e.g., clinic visit, prescription cost, travel q5 imagine you want to begin taking birth control while away at college. you opt to go see a pharmacist at your community pharmacy, who now has the legal right to prescribe birth control. the pharmacist will ask you a series of questions and then determine which birth control would be best for you. the pharmacist would write your birth control prescription that could be filled that day in the pharmacy. how would you rate each of the below topics related to that visit with the pharmacist on a scale of 1 being the lowest to 10 being the highest? q50 thank you for participating in this study. we appreciate your time and responses. your responses will be very helpful to understand the needs of women in michigan and your opinions about a potential new women's health service-pharmacists prescribing birth control in a community pharmacy. would you like to enter the raffle for a chance of receiving one of the two $250 gift cards? your name and email will not be connected with your survey responses. yes enter me in the draw do not enter me in the draw declines in unintended pregnancy in the united states american college of obstetricians and gynecologists committee on gynecologic practice. committee opinion: over-the-counter access to oral contraceptives contraceptive needs and services contraceptive practices, preferences, and barriers among abortion clients in north carolina lack of access = lack of power to decide american college of obstetricians and gynecologists committee on health care for underserved women healthy people 2020. family planning. available online evidence summary: prevent unintended pregnancy cdc's 6/18 initiative impacts of four title v, section 510 abstinence education programs: final report new research findings on programs to reduce teen pregnancy; the national campaign to prevent teen and unplanned pregnancy adolescent pregnancy and contraception american college of obstetricians and gynecologists committee on gynecologic practice: acog. committee opinion summary: over-the-counter access to oral contraceptives unprotected intercourse among women wanting to avoid pregnancy: attitudes, behaviors, and beliefs opening new doors to birth control state efforts to expand access to contraception in community pharmacies expanding adolescent access to hormonal contraception: an update on over-the-counter, pharmacist prescribing, and web-based telehealth approaches hormonal contraception prescribing by pharmacists: 2019 update association of pharmacist prescription of hormonal contraception with unintended pregnancies and medicaid costs youth perspectives on pharmacists' provision of birth control: findings from a focus group study older teen attitudes toward birth control access in pharmacies: a qualitative study adolescents' perceptions of contraception access through pharmacies public perception of pharmacist-prescribed self-administered non-emergency hormonal contraception: an analysis of online social discourse pharmacist outlooks on prescribing hormonal contraception following statewide score of practice expansion student pharmacist perspectives on providing pharmacy-access hormonal contraception services provider opinions regarding expanding access to hormonal contraception in pharmacies interprofessional education: a summary of reports and barriers t 2016; o recommendations pharmacist provision of hormonal contraception in the oregon medicaid population an evaluation of the implementation of pharmacist-prescribed hormonal contraceptives in california availability of pharmacist prescription of contraception in rural areas of oregon and new mexico college undergraduate enrollment snapshot this article is an open access article distributed under the terms and conditions of the creative commons attribution (cc by) license key: cord-252795-x66zqmgv authors: islam, md. akhtarul; barna, sutapa dey; raihan, hasin; khan, md. nafiul alam; hossain, md. tanvir title: depression and anxiety among university students during the covid-19 pandemic in bangladesh: a web-based cross-sectional survey date: 2020-08-26 journal: plos one doi: 10.1371/journal.pone.0238162 sha: doc_id: 252795 cord_uid: x66zqmgv the purpose of this study was to investigate the prevalence of depression and anxiety among bangladeshi university students during the covid-19 pandemic. it also aimed at identifying the determinants of depression and anxiety. a total of 476 university students living in bangladesh participated in this cross-sectional web-based survey. a standardized e-questionnaire was generated using the google form, and the link was shared through social media—facebook. the information was analyzed in three consecutive levels, such as univariate, bivariate, and multivariate analysis. students were experiencing heightened depression and anxiety. around 15% of the students reportedly had moderately severe depression, whereas 18.1% were severely suffering from anxiety. the binary logistic regression suggests that older students have greater depression (or = 2.886, 95% ci = 0.961–8.669). it is also evident that students who provided private tuition in the pre-pandemic period had depression (or = 1.199, 95% ci = 0.736–1.952). it is expected that both the government and universities could work together to fix the academic delays and financial problems to reduce depression and anxiety among university students. the outbreak of coronavirus diseases (covid -19) has been substantially influencing the life and living of people across the world, especially after the declaration of a global pandemic by the world health organization in the second week of march 2020 [1] . as of june 7, 2020, around 6.91 million people were infected with the covid-19, with a confirmed fatality of another 0.4 million worldwide [2] . hence, many countries implemented a range of anti-epidemic measures, such as restricting travel for foreign nationals [3] , closing down public spaces, and shutting down the entire transit system [4, 5] , to contain the transmission of the highly contagious infections from human-to-human. following the detection of first covid-19 case on march 8, 2020 [6] , bangladesh like many other countries put the lockdown strategy into effect on march 26, 2020 , to ensure 'social distance' through 'home quarantine' to curb the 'spread' among its population [7] [8] [9] , since a precise treatment or vaccine for the infected and people at risk are yet to achieved by the global health community [10, 11] . however, all education institutions were closed initially from march 18 to march 31, 2020 across the country and later extended to the mid of june 2020 in phases [12, 13] . this unprecedented experience of 'home quarantine' under lockdown with the uncertainty of academic and professional career has multifaceted impacts on the mental health of students. for example, a canadian study focusing on the effects of quarantine after the severe acute respiratory syndrome (sars) epidemic found an association between longer duration of quarantine with a high prevalence of anxiety and depression among people [14] . the ongoing covid-19 pandemic is creating a psycho-emotional chaotic situation as countries have been reporting a sharp rise of mental health problems, including anxiety, depression, stress, sleep disorder as well as fear, among its citizens [15] [16] [17] [18] [19] , that eventually increased the substance use [15] and sometimes suicidal behavior [20] [21] [22] . researchers in china observed that the greater exposure to 'misinformation' through social media are more likely contributing to the development of anxiety, depression, and other mental health problems among its population of different socioeconomic background [23] [24] [25] [26] . studies before the covid-19 pandemic also suggested an inverse relationship between media exposure and mental health [27, 28] . on the contrary, a study in south korea during the middle east respiratory syndrome (mers) reported a positive relationship between risk perception and media exposure [29] . given the unexpected circumstances, it is crucial to explore the psycho-social experience of university students in bangladesh, especially during the covid-19 pandemic. such a study is expected to measure the psychological impacts of an unforeseen emergency on students, as well as to formulate and execute effective interventions and strategies to mitigate the mental health of people at large. this study was designed to address the psychological problems experienced by university students in bangladesh. the survey was conducted in the second week of may, from may 6 to may 12, 2020. students enrolled in different universities across bangladesh were the target population. an easy to understand questionnaire was used to collect 'basic information,' 'depression,' and 'anxiety' related information. an online-based platform was used to distribute the e-questionnaire, developed by using the google form, to the students. university students from all the divisions in bangladesh were contacted through different social networks and interviewed (see fig 1) . the snowball sampling technique was used for collecting information from students. an informed consent form was attached to the e-questionnaire, and each participant consented to participate in the survey after reading the consent form. the participants were asked to share the e-questionnaire with their friends using their personal and institutional facebook and messenger. this study was formally approved by the ethical clearance committee of khulna university, bangladesh. the participants responded anonymously to the online survey by filling up an informed consent letter in the first section of the e-questionnaire. in the consent form, all the participants were provided with information concerning the research purpose, confidentiality of information, and right to revoke the participation without prior justification. basic information. 'basic information' contained the personal information of the respondents. current 'age' of students ('17-20', '21-24', '>24'), whether the student is 'lagging behind study' ('yes' and 'no'), doing any sorts of 'exercise during lockdown' ('yes' and 'no'), students who did 'tuition' before lockdown ('yes' and 'no'), the gender of the student ('male' and 'female'), 'place of residence' of students ('rural' and 'urban'), is he/she 'living with family' during lockdown ('yes' and 'no'). depression. depression was determined by using the patient health questionnaire (phq-9). phq-9 is an easy way to use in a questionnaire for screening depression of the responses that are used to predict depression of an individual and what state he/she is in during the survey. the scores in phq-9 range from '0 = not at all' to '3 = nearly every day' [30] . the reason for choosing phq-9 was that it proved to be a useful tool for detecting depression [31] . the levels of depression for the study were categorized as 'mild = 5-9', 'moderate = 10-14,' 'moderately severe = 15-19,' 'severe = � 20.' anxiety. anxiety was evaluated by using the generalized anxiety disorder (gad-7). in the questionnaire, the questions were used for screening anxiety state of an individual on a scale ranging from '0 = not at all sure' to '3 = nearly every day' [32] . gad-7 has been found successful in identifying anxiety among different populations and thus used for its reliability [33] . the levels of anxiety for the study were categorized as 'none-minimal = <5,' 'mild = 5-9,' 'moderate = 10-14 and 'severe = � 15.'. frequency tabulation was used to summarize basic information of respondents, as well as their response to depression and anxiety. binary logistic regression [34] was used to identify variables influencing depression and anxiety among students by categorizing the outcome variable into two categories, i.e., depressed = 'yes' and 'no' and anxious = 'yes' and 'no,' which would provide a clearer idea about how intensely different factors are influencing the outcomes. logistic regression generates the coefficients (and its standard errors and significance levels) of a formula to predict a logit transformation of the probability of the presence of the characteristic of interest: where p is the probability of the presence of the characteristic of interest. the logit transformation was defined as the logged odds: and, rather than choosing parameters that minimize the sum of squared errors (like in ordinary regression), estimation in logistic regression accepts parameters that maximize the likelihood of observing the sample values. table 1 shows the descriptive information of different selected variables of the university student in bangladesh. results show that 392 (82.4%) students were found to have mild to severe depressive symptoms, and 389 (87.7%) students were found to have mild to severe anxiety symptoms. more than 60% of the students were male (67.2%), and the rest were female. one in three students lived in rural areas (35.1%). less than a quarter percent of students (24.8%) believed that they were not academically lagging, and just over 30% reportedly have exercise regularly during the lockdown at home. table 2 shows the prevalence of depression and anxiety among bangladeshi university students. out of the total 476 valid participants, 392 (82.4%) were found to have mild to severe depressive symptoms. male (67.35%) had higher depressive symptoms than the female (32.65%) counterparts, whereas students in the early twenties (66.07%) showed higher depressive symptoms than other age groups. depression was also prevalent among students with no physical exercise (62.24%) and those who consider themselves lagging behind others in terms of academic activities (76.78%). besides, students living with families (96.93%) and in urban areas (65.05%) showed higher depressive symptoms. in the case of anxiety, 389 (87.7%) students exhibited mild to severe anxiety symptoms. out of the total students suffering from an anxiety disorder, females (33.67%) had lower anxiety symptoms than males (66.33%), whereas students in the early twenties (66.58%) showed higher anxiety. like depression, anxiety was also prevalent mostly among students with no physical exercise (61.95%), troubled with the thought of lagging behind others academically (76.60%). moreover, students living in urban areas (62.21%) with families (96.40%) also showed symptoms of anxiety. table 3 reveals that students who thought that s/he was lagging behind others in academic activities were 1.8 times (95% ci: 1.098, 2.935) more likely to be depressed than the student with no such worries. students living with families were 2.6 times (95% ci: 1.418, 4.751), more likely to be depressed than the students living apart from families. on the other hand, students providing supplementary classes before lockdown were 1.4 times (95% ci: 0.856, 2.227), more likely to show mild to severe anxiety symptoms than their counterparts with no such involvement. students who were worried about their academic activities were 1.8 times (95% ci: 1.099, 2.883) more likely to exhibit mild to severe anxiety symptoms than students with no such worries. students living with families were 1.8 times (95% ci: 1.021, 3.308), more likely to have mild to severe anxiety symptoms than students staying away from families during the lockdown. covid-19 pandemic came out as the most devastating and challenging crisis for public health in the contemporary world. apart from the soaring mortality rate, nations across the globe have also been suffering from a spike of the excruciating psychological outcomes, i.e., anxiety and depression among people of all ages. university students are no exception, as all the educational institutions are unprecedentedly closed for more than usual, and for bangladesh, it is more than two months in a row. such closure, in general, triggers a sense of uncertainty about academic and professional career among the educands and intensifies persistent mental health challenges among university students [33, 35, 36] . given such circumstances, the main goal of this study was to investigate the prevalence of depression and anxiety among the bangladeshi university students during the covid-19 pandemic and to explore the factors influencing the presence of depression and anxiety disorder. the findings of the web-based cross-sectional survey indicate that more than two-thirds of the students were experiencing mild to severe depression (82.4%) and anxiety (87.7%). earlier studies in bangladesh observed the presence of both depression and anxiety among students in higher academia. for example, a survey of medical students in 2015 suggested that more than 50% of students in medical colleges are suffering from depression (54.3%) and anxiety (64.8%) [37] . another study, on university students excluding the freshmen, complemented the previous work and found that the prevalence rate of depression and anxiety was 52.2% and 58.1%, respectively [38] . compared to the earlier studies, our study suggests that university students in bangladesh are experiencing an unparalleled growth of depression and anxiety under the current global pandemic situation. the results also suggest that the university students' involvement in private tuition is a critical factor in understanding the increased prevalence of depression and anxiety among them. in bangladesh, a significant number of students are involved in part-time jobs, such as private tuition, to finance their educational expenses, and sometimes to support their families, and their reliance on private tutoring as a part-time job is increasing gradually [39] . however, being unable to provide tuition under the lockdown situation means disruption of regular income and joblessness. the prolonged unemployment, together with financial insecurity, is the most significant stressors contributing to the increased rates of depression and anxiety among university students in bangladesh. a study suggests that unemployment is significantly associated with mental and somatic disorders, which could limit the individuals' chances for feelings of achievement, accomplishment, and satisfaction, and eventually lead to the impairment of psychological functioning [40] . self-esteem could also be affected by the loss of work as studies found that lack of family support during unemployment adversely affects the mental well-being of individuals [41, 42] . apparently, the sudden joblessness and financial insecurity are putting the university students in an unpleasant situation, affecting their socioeconomic and mental well-being [43] . it has been well accepted that living with families strongly generate reassurance among the individuals, therefore, reduce depression and anxiety. because positive family environments often benefit the mental health of the vulnerable youth experiencing depression or anxiety [44] . however, this pandemic has brought extreme financial pressure on families. most of the families have been suffering from unmanageable debts and a decline in income, thus, leaving the family members in a traumatized situation [45, 46] . university students, who used to earn and contribute to their families before lockdown, can hardly assist their parents in this crisis moment. the results of this study suggest that despite living with family, anxiety and depressive symptoms have been increasing among university students in bangladesh mainly due to financial insecurity. universities in developed countries put strict health protocols into action, such as washing hand, using face-mask, advising 'stay-home' strategy when sick, to facilitate continuation of education in higher academia and later switched to campus-wide online learning [47, 48] . in bangladesh, the protective interventions, such as wearing mask or using the personal protective equipment, are yet to be enforced largely due to limited supplies [49, 50] , hence, the government opted to implement the country-wide lockdown. approximately two-thirds of the students are getting depressed thinking they might be falling academically behind their contemporaries in other parts of the world during the prolonged closure of universities. they, however, reiterated that the online classes could not fulfill their requirements [51] and a significant percentage of the students are still out of the reach of the online class. in addition, their research projects and internships had to be ceased since they were instructed to leave the halls (dormitories for students) of their respective universities [3] . not only that, the covid-19 crisis also created a severe challenge of the global reversion for the graduates to accomplish their future academic and working goals [52] . although university closures were intended to keep students safe, for many, these notions came out with different sets of mental health issues. meanwhile, a study reported that graduate students generally experience significant amounts of stress and anxiety, which also affects their usual behavior [53] . the results in this study stressed on the fact that the nation-wide lockdown in bangladesh is going to cause a significant disruption in the academic programs and create a gap in both teaching and learning. the academic delays could have long-term impacts on the psychology of students as they are more likely to be graduated later than they have expected. in this regard, faculties, as well as university authorities, should stay connected with the students using social media platforms and motivate them to move forward together during this difficult time. apart from the issues mentioned above, this study found no significant differences between male and female students with relation to depression or anxiety, thus complement previous studies [36, 37, 54] . however, egyptian research remarked that female university students are more likely to suffer anxiety and less prone to depression than male students [55] . the current study did not find any statistically significant association between the socio-demographic variables (including place of residence and exercise) with depression and anxiety. a few studies, on the contrary, reported a significant association between socio-demographic variables [37] and exercise [56] with depression and anxiety. a malaysian study reported substantial differences concerning age and permanent residence with depression or anxiety, however, observed no significant association between some socio-demographic variables (including gender, ethnicity, study major, monthly family income) and the psychological problems [36] . the strengths and limitations of the current study are determined by several issues. the equestionnaire allows to assess the prevalence of anxiety and depression among university students while maintaining the who recommended "social distance" during the covid-19 pandemic, which otherwise would be impossible. moreover, the data for the e-survey were collected by globally validated standardized tools for quantitative analysis. on the contrary, given the limited resources available and the time-sensitivity of the covid-19 outbreak, the snowball sampling strategy was chosen instead of random samples. in this cross-sectional study, the identified factors are regarded as associated factors, which could be either be the causes or the results of depression or anxiety. furthermore, due to ethical requirements on anonymity and confidentiality, the contact details of the respondents was not collected. however, the use of validated screening e-questionnaire was considered as a cost-effective approach to explore the situation in general, therefore, used in this study. since the research methodology could not reach people with medically examined depression and anxiety symptoms, the provision of the results may not fully reflect the severity of depressive and anxiety symptoms among students. another limitation of this study is not using the tools designed specifically for the covid-19 pandemic, such as the coronavirus anxiety scale (cas). meanwhile, it would be ideal for conducting a prospective study on the same group of participants with tools developed especially for the covid-19 pandemic after a period to provide a concrete finding and to facilitate the demand for a focused public health initiative. despite some limitations, this study gives the first empirical evidence that a large percentage of bangladeshi university students have been suffering from depression and anxiety symptoms during the ongoing pandemic. in addition to academic and professional uncertainty, financial insecurity is contributing to the rise of depression and anxiety among university students. to minimize the growing mental health problems, the government, along with the universities, should work together to deliver promptly and accurately economy-oriented psychological support to the university students. to ensure the continuous involvement of students in educational processes, the universities should initiate all-inclusive online-based educational programs to reach out the students living in remote areas with or without devices in association with internet-service providers by providing scholarship or student loan. furthermore, parents should be encouraged, by providing pandemic response and recovery support from the government, to create a friendly and positive family environment for university students without imposing pressure on the future academic and working career. world health organization. who director-general's opening remarks at the media briefing on covid-19-11 coronavirus disease (covid-19): situation report-138 world health organization mental health care for international chinese students affected by the covid-19 outbreak epidemic of covid-19 in china and associated psychological problems china's ongoing battle against the coronavirus: why did the lockdown strategy work well? socio-ecological practice research covid-19-related suicides in bangladesh due to lockdown and economic factors: case study evidence from media reports covid-19 in bangladesh: public awareness and insufficient health facility remain key challenges. public health coronavirus pandemic: 45 districts now under complete lockdown. the daily star discovering drugs to treat coronavirus disease 2019 (covid-19) presumed asymptomatic carrier transmission of covid-19 education institutions to remain closed till may 30. dhaka tribune educational institutions to remain shut till sars control and psychological effects of quarantine covid 19 fear, stress, anxiety, and substance use among russian and belarusian university students. international journal of mental health and addiction associations between fear of covid-19, mental health, and preventive behaviours across pregnant women and husbands: an actorpartner interdependence modelling anxiety and coping strategies among nursing students during the covid-19 pandemic study of knowledge, attitude, anxiety & perceived mental healthcare need in indian population during covid-19 pandemic. asian journal of psychiatry social capital and sleep quality in individuals who self-isolated for 14 days during the coronavirus disease 2019 (covid-19) outbreak in january 2020 in china first covid-19 suicide case in bangladesh due to fear of covid-19 and xenophobia: possible suicide prevention strategies fear of covid 2019: first suicidal case in india! asian journal of psychiatry reims football club doctor commits suicide in quarantine, mentions covid-19 in note 2019-ncov epidemic: address mental health care to empower society. the lancet the psychological impact of the covid-19 epidemic on college students in china mental health problems and social media exposure during covid-19 outbreak internet-related behaviors and psychological distress among schoolchildren during covid-19 school suspension. j am acad child adolesc psychiatry covid-19, coronavirus and mental health rehabilitation at times of crisis the impact of epidemic, war, prohibition and media on suicide: united states, 1910-1920. suicide and life-threatening behavior the impact of social media on risk perceptions during the mers outbreak in south korea the phq-9: validity of a brief depression severity measure validity of the brief patient health questionnaire mood scale (phq-9) in the general population a brief measure for assessing generalized anxiety disorder: the gad-7 impacts of socio-cultural environment and lifestyle factors on the psychological health of university students in bangladesh: a longitudinal study applied logistic regression psychological investigation of university students in a university in jilin province correlates of depression, anxiety and stress among malaysian university students assessment of depression, anxiety and stress among first year mbbs students of a public medical college mental health problems and associated predictors among bangladeshi students patterns and determinants of private tutoring: the case of bangladesh households effects of unemployment on mental and physical health the experience of losing a job: reported changes in health, symptoms and illness behavior socioeconomic status and the career aspirations of australian school students: testing enduring assumptions. the australian educational researcher the importance of work in an age of uncertainty: the eroding work experience in america friendships and family support reduce subsequent depressive symptoms in at-risk adolescents challenges and burden of the coronavirus 2019 (covid-19) pandemic for child and adolescent mental health: a narrative review to highlight clinical and research needs in the acute phase and the long return to normality. child and adolescent psychiatry and mental health covid-19 suicides in pakistan, dying off not covid-19 fear but poverty?-the forthcoming economic challenges for a developing country from offline to online: challenges and opportunities for entrepreneurship education following the covid-19 pandemic. entrepreneurship education and pedagogy the impact of covid-19 on the sports medicine fellowship class of 2020 easing covid-19 lockdown: will wearing face masks be safe enough? dhaka tribune covid-19: use of face mask. the financial express online learning: a panacea in the time of covid-19 crisis closure of universities due to coronavirus disease 2019 (covid-19): impact on education and mental health of students and academic staff mental health and suicidal behavior among graduate students prevalence and socio-demographic correlates of mental health problems among iranian health sciences students. academic psychiatry inadequate sleep and exercise associated with burnout and depression among medical students depression, anxiety and stress among first year medical students in an egyptian public university we are grateful to the participants, as well as thankful to the editors and anonymous reviewers. key: cord-310553-qsk42zf7 authors: alkhowailed, mohammad s.; rasheed, zafar; shariq, ali; elzainy, ahmed; el sadik, abir; alkhamiss, abdullah; alsolai, ahmed m.; alduraibi, sharifa k.; alduraibi, alaa; alamro, ahmad; alhomaidan, homaidan t.; al abdulmonem, waleed title: digitalization plan in medical education during covid-19 lockdown date: 2020-09-17 journal: inform med unlocked doi: 10.1016/j.imu.2020.100432 sha: doc_id: 310553 cord_uid: qsk42zf7 background: the covid-19 pandemic has enhanced the adoption of virtual learning after the urgent suspension of traditional teaching. different online learning strategies were established to face this learning crisis. the present descriptive cross-sectional study was conducted to reveal the different digital procedures implemented by the college of medicine at qassim university for better student performance and achievement. methods: the switch into distance-based learning was managed by the digitalization committee. multiple online workshops were conducted to the staff and students about the value and procedures of such a shift. new procedures for online problem-based learning (pbl) sessions were designed. students’ satisfaction was recorded regarding the efficiency of live streaming educational activities and online assessment. results: the students were satisfied with the overall shift into this collaborative e-learning environment and the new successful procedures of virtual pbl sessions. the digital learning tools facilitated the performance of the students and their peer sharing of knowledge. the role of informatics computer technologies was evident in promoting the students, research skills, and technical competencies. conclusions: the present work elaborated on the procedures and privileges of the transformation into digitalized learning, particularly the pbl sessions, which were appreciated by the students and staff. it recommended the adoption of future online theoretical courses as well as the development of informatics computer technologies. scientific technology and digitalization have a tremendous impact on escalating the efficiency and productivity of work in nearly all the fields in the modern era, from farming to health services and innovations and has proved itself as an efficient tool to make human life better and easier [1, 2] . online learning is considered a feasible and compliant method for training and scientific meetings and the sustainability of learning [3] . the expansion in global virtual learning is dependent on the availability of technology-enhanced active learning tools, and the options of online learning and their role in the field of medical education cannot be disregarded [3, 4] . the use of digital technology in medical education is now believed to have a crucial aspect of learning resources [3] [4] [5] . it does not only augment the understanding of the subject, but it also prepares the students to deal with the real-life scenario in a more practical way [6] . the pandemic of coronavirus disease 2019 (covid-19) demands for virtual classrooms to foster creative thinking and solving problems capabilities of the students. the existing digital platform enabled communication with learners with a lower barrier and the online teaching was found to be a method that challenged our traditional approach [3] [4] [5] [6] . a previously conducted study on twenty-two thousand students in the united kingdom using digital education organization revealed that the technology as a learning tool is yet to be realized for effective pedagogy and learning [7] . another study conducted in australia, including more than one thousand students, revealed that digital learning was the most valuable technology for studying [8] . this study was conducted to disclose the digital resources used in teaching of the medical students at the college of medicine at qassim university. currently, there are two sections for male and female students in j o u r n a l p r e -p r o o f digital processes used by the college of medicine in qassim university for better performance and achievement in the field of medical education. this is a cross-sectional study performed on undergraduate medical students (n=674) from the 9 th of march till the end of april 2020 in the college of medicine at qassim university in ksa. ethical approval was obtained from the research ethics committee at qassim university and the confidentiality of the information obtained was kept in consideration. a committee was formed, which included faculty members of the college of medicine and the task was given to digitalize the teaching activities through utilization of the available learning recourses required to provide effectual medical education. the digitalization process was subdivided into two main functional parts. the first one was to digitalize the different learning activities, which included pbl, lectures, seminars, and assignments. the second task was to construct digital evaluation and feedback sheets. students of the basic years were subdivided into groups for virtual pbl on the blackboard with one faculty member assigned for each group for facilitating and evaluating the pbl sessions. after each session, anonymous electronic feedback from the staff about the pbl materials was sent to the pbl reviewing committee. the complete steps of the pbl digitalization are summarized in figure 2 . qassim college of medicine used blackboard learning management system as the primary tool for virtual teaching. in case of its interruption, the zoom cloud meetings platform was utilized as an alternative tool. live webinars for the detailed instructions on the proper digitalization of educational activities were implemented to increase the competencies of the students and staff during online teaching. for any obstacle facing the students or staff during the e-learning process, different communication tools were suggested, such as using the social media application whatsapp to respond promptly to any inquiry. an exam committee was formed, comprised of senior faculty members. the task for the committee was to design digital exams composed of clinical scenario-based multiplechoice questions, which were prepared through the contribution of the relevant department of the medical college. the students' performance was evaluated after online j o u r n a l p r e -p r o o f exams based upon the marks they obtained. in order to assess the clarity, difficulty level, and quality of the digital evaluation activity, students were provided with online feedback. after the suspension of study due to the covid-19 pandemic, all college meetings were conducted using blackboard collaborate ultra for web conferencing. in case of any technical issue facing the live streaming via blackboard, zoom cloud meetings was purchased by qassim university, and the login details using university domain identification were provided to each faculty member was used as an alternative for conducting the meetings. an online satisfaction survey was performed to collect the student's perception of the whole online learning process using a 5-points likert scale. analysed data were revised and analysed by an independent investigator. qassim university, such as dentistry and nursing colleges ( table 1 ). the feedback of the students on their perception about the whole digitalization process was recorded through an online survey (fig. 3) . 67.30% of the students were satisfied with online teaching for all theoretical courses. about 64% agreed that the live this study described the procedures of the transformation from the traditional medical learning into the digital during the complete lockdown of covid-19 pandemic. as the world is converting into a global village, technology has a tremendous contribution, especially in health sciences and medical care [13] . the covid-19 pandemic has enhanced the adoption of virtual learning being the only safe and satisfactory educational option to ensure students' engagement, fulfilling their knowledge gaps as well as competencies of the health professional educators [14] . according to the current needs, it is a vital step to prepare the medical students and staff to cope with the modern technological innovations. medical education is speedily upgraded under the influence of various factors that included the diversity in societal prospects and the health care system [15, 16] . in concretion to this vision, the college of medicine at qassim university adapted the pathway of virtual classrooms under the supervision of the digitalization committee. the success of this committee in the organization of logistics and procedures of the distant learning and online evaluation was conveyed from the positive reflection of the students regarding the live-streaming sessions, online assessment, and overall satisfaction. in this scope, the accomplishment of virtual learning began with the obligatory social distancing due to the covid-19. most importantly the financial support provided by qassim university to all students and high standard living status of saudi society through which they utilized high-speed internet services played a vital role for the successful implantation of the digitization plan during this pandemic. digitalization of educational materials has a considerable impact on the environment in which medical students learn [16] . the present study elaborated that blackboard was the main software j o u r n a l p r e -p r o o f utilized for conducting most of the teaching activities. the rest of the live streaming session was conducted via zoom cloud meetings as an alternative online platform utilized if the access to the official blackboard was non-feasible. this was made possible due to our staff and students as they were accustomed previously by handling the various online activities such as they were officially using blackboard, an official learning management system for the last three years. in addition, the faculty development unit organized multiple training sessions every year for the effective utilization of the elearning, and switch to cope with the advance in the informatics and distant learning. moreover, the expert staff was also selected in dealing with the online teaching, particularly in the first week of practicing the e-learning. in spite of these, the staff experiences were unsatisfactory for about 56% of the students. this could be due to the overloaded burden of both the staff and students to prepare didactic materials and to adopt the online learning process in a very short period of time. different medical colleges around the globe utilized a variety of digital software for distance-based learning. the university of sarajevo, located in bosnia and herzegovina, had implemented the virtual classrooms as a learning strategy on their e-learning website [4] . our newly designed steps of the online pbl sessions were effective and enhanced the performance of students and tutors, which was evident from the students' perception. this successful application of distant learning and assessment motivated the college to construct efficient online procedures for other educational activities such as team-based learning (tbl) and the digital learning tools facilitated the performance of the students and their peer sharing of knowledge. the evaluation of students is an integral step in assessing the knowledge they have gained during the learning process. putten et al., in their study, found that conducting the online assessments by digital means is a beneficial strategy [17] . in view of these, the college of medicine at qassim university conducted all exams via digital means on blackboard software. during digital learning, the students were evaluated not only on the basis of their performance in the summative exams built on multiple choice questions, but their evaluation was also conducted on the basis of various other modalities such as pbl. the students were evaluated in the pbl sessions according to their peer interaction, commitment, team spirit, problem solving and critical thinking. in addition, the evaluation of the presentation skills and cognitive learning domains were performed through their seminars. enthusiasm among the students who are attending the distancebased learning courses is also an important aspect, since if the students are not mounting interest, it will lead to a high dropout rate [18] . a few students were inconvenienced with the online assessment to test their knowledge precisely. this could be explained by the fact that the students were not adapted yet to online exams, being the first time they were assessed by such a method. according to a recent perception survey performed by the imperial college london, some students mentioned that online exams were imperfect as they did not have a quiet surrounding environment at their homes while giving a timed online exam [19] . routine online surveys and feedback were taken from the students in order to become aware and overcome the deficiencies in virtual teaching. the motivation of most of the students to implement online theoretical courses in the future was observed. however, the psychomotor skills performed in the practical and clinical settings required direct hands-on training as in real life. the students have to deal with j o u r n a l p r e -p r o o f practical laboratory procedures and real patients that require direct human interaction. according to a study conducted by warnecke et al., most medical students looked on elearning as a pleasurable and efficient mode of learning; however, they do not perceive it as a replacement of traditional educational methods [20] . another study, conducted on medical students, had shown that learning was comparatively superior in a blended learning environment in which both e-learning and traditional teaching was merged [21] . conversely, this contentment is not correlated with exam scores in which there was no significant distinction between the two pedagogic methods [22] . the level of satisfaction of the students about the interaction and open discussion during the live streaming sessions was average as it depended upon the type of educational activity as well as the role of faculty member moderating this activity [16] . this also could be attributed to the urgent switch into this new teaching modality. during this digitalization based learning, we also encountered several limitations including insufficient learning resources, such as the overload on the official learning management system that necessitated sustained upgrade and maintenance. in addition, there was a lack of information technologists as well as a lack of optimal online learning experience of staff and students. these could be overcome by developing and integrating informatics computer technologies in the field of medical education, collaboration with other universities having a good practice of distance-based teaching methods, and enhancement of digital literacy among students as well as faculty. j o u r n a l p r e -p r o o f the current work elaborated on the procedures, privileges, and challenges of the shift into distance-based learning, particularly online pbl. this switch was appreciated by both students and staff. the study recommended the development of informatics computer technologies to promote technologically-enhanced learning and the implementation of online courses in subsequent years. the value of sts in medical and health humanities pedagogy witnessing the advance of science and technology in life sciences in the new era barriers and solutions to online learning in medical education -an integrative review e-learning as new method of medical education evaluation of e-learning for medical education in low-and middle-income countries: a systematic review influences of digital classrooms on education student digital experience tracker 2017: the voice of 22,000 uk learners what works and why? student perceptions of "useful" digital technology in university teaching and learning china novel coronavirus investigating and research team. a novel coronavirus from patients with pneumoniain china covid-19: a global public health disaster outbreak of coronavirus disease 2019. the lancet saudi arabia announces the first case of coronavirus. saudi research and marketing group systematic review: impact of health information technology on quality, efficiency, and costs of medical care coronavirus disease (covid-19) prevention: virtual classroom education for hand hygiene the impact of e-learning in medical education the impact of e-learning in medical education use of the internet for educational applications in prosthodontics factors associated with dropout in medical education: a literature review medical students' perceptions of using e-learning to enhance the acquisition of consulting skills comparison of the effect of lecture and blended teaching methods on students' learning and satisfaction blended learning versus traditional lecture in introductory nursing pathophysiology courses researchers would like to thank the deanship of scientific research, qassim university for funding publication of this project. the authors declare no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. mohammad s. alkhowailed: data collection and interpretation and manuscript drafting. key: cord-026889-hx6xvn0k authors: arity, viktor; vesty, gillian title: designing authentic assessments: engaging business students in flow experience with digital technologies date: 2020-05-11 journal: tertiary education in a time of change doi: 10.1007/978-981-15-5883-2_3 sha: doc_id: 26889 cord_uid: hx6xvn0k embracing technology in higher education has become a necessity, rather than a desired value-added addition to educational delivery. embracing technology in higher education has become a necessity, rather than a desired value-added addition to educational delivery. previously, innovative technological adoption was the result of academic dissatisfaction of the current status quo (ely, 1990) , and/or the push from institutional leaders and the financial benefits that technology can bring through increased student volumes, enhanced reputation and competitive advantage (price & kirkwood, 2013) . more recently, particularly with the covid-19 pandemic, academics are now being forced to engage with digital technologies and consider the transformational benefits that technology can potentially bring to the online classroom. more than ever, academics are faced with addressing the long-standing calls for innovative student-centric and project-based teaching practices rather than using technology to merely sustain, 'replicate or supplement traditional activities' (price & kirkwood, 2013, p. 8) . to date, there is still a void in the uptake of digital innovations in education, which is arguably confounded by performance pressures, competing priorities, financial and time constraints (brimble, 2016; harper et al., 2019) . as a result, issues relating to poor curriculum design and academic integrity continue to exist. the contention for this chapter is that innovative authentic digital assessment designs can contribute to relieving key pressure points such as last-minute assessment preparation, poor-quality assessment designs and submitted work, minimal opportunities for regular academic engagement and feedback and minimising the growing impact of plagiarism and contract cheating in higher education (bretag et al., 2016; harper et al., 2019) . it is argued that contract cheating can be minimised by providing students with a learning environment that motivates them not to cheat, by providing them with personalised and sequential assessment designs that encourage them to realise the intrinsic (skills) and extrinsic (work ready) motivation that engaging in the assessment will provide (bretag et al., 2016; harper et al., 2019; walker & townley, 2012) . most importantly, digital innovations allow for more 'individualisation of learning' and enhances the development of twenty-first century skills of 'independent learning, initiative, communication, teamwork, adaptability, collaboration, networking, and thinking skills within a particular professional or subject domain' (bates & sangrà, 2011, p. xxi) . individual learning can be better managed through coaching and scaffolding with assessment appearing to be seamlessly integrated with the learning activity (herrington & standen, 1999; lameras et al., 2017) . if the innovative environment is engaging enough, success can be measured by the extent to which the immersive experience is described by students in terms of their flow experience, that is, the 'holistic sensation that people feel when they act with total involvement' and lose a sense of time (csikszentmihalyi, 1990, p. 477) . the flow experience occurs when learners experience cognitive efficiency, are intrinsically motivated and happy (csikszentmihalyi, 1975) . the broad research question investigated in this chapter is the extent to which digital assessment design features can reduce the cognitive load burden of students and accomplish flow experience. in the sections that follow, contributions to the authentic assessment literature, underpinned by cognitive load and flow theories, are provided. this is followed by the design and pilot testing of a digital, scaffolded assessment tool intended to provide an immersive learning environment and reduce the cognitive burden of higher education business students. survey data is used to determine the topics students find most challenging. the survey results guide the direction taken in the reportwriting journey, whereby students are required to propose a 'big idea' which is linked to improved performance and includes the achievement of united nations sustainable development goals (un, 2018) . in the design sections, we explain the educator role in the technologically based management world with the ability to provide individualised, real-time feedback to students. the platform designs ensure that academics and students engage in regular conversations through a system that is dynamic and adaptable to new queries, topics and assessment formats (including written, graphics and numerical). the system can be used to motivate students across different educational settings and time zones. in the findings section, we discuss the analysis of the secondary data from undergraduate business students undertaking a large core business course and use this data to evaluate the impact of the digital platform on student flow experience. we conclude the chapter with a discussion of limitations and insights for further research in this area. according to cognitive load theory, a scaffolded approach to learning results in cognitive efficiency and reduces the cognitive load burden (sweller, 1988) . these factors are particularly important for students facing challenging assessment topics when they could easily give up. as such, the digital environment provides a perfect setting for scaffolding learning and examining flow experience in education (annetta, 2010; giasiranis & sofos, 2017; shin, 2006) , including distance education (liao, 2006; pearce, ainley & howard, 2005) . the ability for students to comprehend the individual schema in scaffolded designs is an important part of pedagogical designs (sweller, van merriënboer & paas, 1998) . these important design features provide educators with more informed learning analytics. they also contribute to the observable flow experience associated with cognitive efficiency (annetta, 2010) . cognitive load theory comprises three parts: intrinsic, extraneous and germane cognitive loads. intrinsic cognitive load is the inherent difficulty level of the specific topic or the complexity that emerges from dealing with a number of elements that must be processed at the same time in a learner's working memory (gerjets & scheiter, 2003) . importantly, the inherent difficulty level of a specific topic cannot be changed (e.g., higher education maths compared with primary school mathematics). as such, the ability of the learner to break down the components into manageable schema depends on the topic and the learner's expertise (sweller's et al., 1998) . educational designs can support learning by breaking down the complex topic into schema or subtopics before combining them back together for final, holistic understanding (sweller et al., 1998) . extraneous cognitive load relates to the knowledge seeking of individual learners due to ineffective instructional techniques. learners are required to tap into their cognitive resources or working memory for additional information to support learning (sweller, 1994) . extraneous cognitive load is brought into play when learners are required to expend their cognitive resources by searching for information (i.e., internet, other resources, guides and instructions) that is needed to complete a learning task (paas, renk & sweller, 2003) . this may fail if learners have limited cognitive resources to utilise. it also means that the more the extraneous cognitive resources are utilised, the less cognitive resources are available for schema construction or automation. hence, learning the topic becomes more difficult, and intrinsic load schema construction less possible. nevertheless, if learning materials are suitably designed, extraneous cognitive load is reduced, and more resources can subsequently be allocated to process the intrinsic cognitive load. students may feel baffled if the schema has not been constructed and/or available for them to access (sweller, 1994) . however, when sufficient working memory resources remain after the intrinsic and extraneous cognitive load processing, learners may expend additional efforts in value-added processes which are related to learning, such as schema construction (knowledge formation). this is referred to as the germane cognitive load. germane cognitive load is the desired or effective cognitive load, the result of beneficial cognitive processes such as abstractions and elaborations that are promoted by instructional presentation (gerjets & scheiter, 2003) . when sufficient working memory resources remain, after the intrinsic and extraneous cognitive load processing, learners may expend additional efforts in value-added processes which are related to learning, such as schema construction. these processes also increase cognitive load, but it is only germane cognitive load that will contribute to, instead of interfering with, learning (sweller et al., 1998) . germane load is described as the mental resource learners use to learn and conceptualise ideas-schemata. promoting germane load may enhance learning performance. cognitive feedback is facilitated by digital technologies and plays an important role in capturing learner's attention and focusing it on the essential schema (ketamo & kiili, 2010) . instructional designers can support the working memory of learners by reducing extraneous cognitive load and wasteful effort. this leaves learners with the capacity to invest in their own resources, such as constructing mental maps, or other advanced cognitive processing techniques associated with germane cognitive load (gerjets & scheiter, 2003) . debue and leemput (2014) confirm when the extraneous load is reduced, such as through animation and pictures, the germane load increases, and learner performance improves. early feedback has also been found to enable reflexive development and validation of mental models, along with the effective formation of new pedagogically informed strategies (ketamo & kiili, 2010; bellotti et al., 2011) . as learners become immersed in the project, their active participation transfers to passive participation as they become completely absorbed in the task at hand. the flow experience concept has been measured (csikszentmihalyi, 1975 (csikszentmihalyi, , 1990 ) and used to test immersive learning experiences (annetta, 2010; bitrián, buil & catalan, 2020; giasiranis & sofos, 2017; jackson & marsh, 1996; liao, 2006; pearce et al., 2005; shin, 2006; van schaik, martin & vallance, 2012) . drawing on the earlier flow theories (csikszentmihalyi, 1975; jackson & march, 1996) and adapting to digital immersive learning designs, annetta (2010, p. 107) defines flow as 1. feeling the activity can be successfully completed, 2. the player can concentrate fully on the activity, 3. the activity has clear goals, 4. the activity provides fast feedback, 5. the player is deeply involved in the activity, 6. a sense of control over the actions is necessary to perform the activity, 7. self-awareness disappears during flow, and 8. there is an altered sense of time. flow experience in digital designs has been studied, for example, in immersive technologies such as virtual reality (giasiranis & sofos, 2017) and distance learning (liao, 2006; shin, 2006) . liao (2006) found positive relationships between the learner and instructor and digital interface in flow experience. shin (2006) similarly examined the flow effect in an online virtual course and found that student perceptions of levels of skills and challenges are critical to determining the level of flow. these authors confirm that teacher role and designed learning pedagogy are important in impacting the cognitive burden of students, thus contributing to improved flow effects. shin (2006) also found a relationship between flow and student satisfaction scores. together these findings provide the impetus for this study to examine whether our individual authentic assessment design can similarly achieve flow experience for our business accounting students. furthermore, pearce et al. (2005) flow process helps to understand that an examination of the alternative paths students take when dealing with challenging concepts requires specific skills. we address their calls for examining the interactions required so students can navigate the 'challenge-skill space' on the way towards the flow experience. the theoretical framework that underpins the design of the artefact and data collection approach is outlined in fig. 3 .1. it is argued that the scaffolded approach to learning design will lead to a reduction in the cognitive load burden for students. this approach reduces the intrinsic load supported by extraneous design features. we contend that because students feel more comfortable with the task at hand, and engage more readily with design features, this will contribute to their flow experience. that is, there will be greater immersive engagement with the task and communication through the built platform with their teacher. in addition, we expect fewer requests for extensions, fewer queries about the specificities of each of the tasks and better-quality assignment submissions because they are not left to the last minute. the built 'assessment' artefact is designed according to the cognitive load and flow theories with attention to immersion, interactivity, increasing complexity, informed teaching and instructional design. these elements are achieved by attention to the assessment storyline, developed as a result of a short survey of students about the course topics they found cognitively challenging. python coding contributed to scaffolding the assessment design. a survey was conducted with final year management accounting students across two campuses (singapore and melbourne) over three semesters, asking them about topics they found most cognitively challenging. the question asked students to list their top three (3) challenging topics from the following 10 items. from 858 students, we received 283 responses, representing a 33% response rate. results indicated that they found a qualitative balanced scorecard and non-financial performance evaluation (9) and risk management (10), along with a quantitatively challenging topic transfer pricing (3). we used this final year student data to redesign the first-year course, to ensure our immersive digital designs would contribute to an overall scaffolded approach to learning. by breaking down the complexity of the balanced scorecard to a practical 'immersive' experience, we were hoping that by the time our students faced third year, they would find this topic less cognitively challenging. we also added quantitative items in the digital design that would later build to the more complex transfer pricing topics experienced in their third-year courses. the tailored storyline is designed to promote authentic assessment and foster individual creativity by engaging business students in developing their own business plan for an idea that would not only improve business performance but also address the broader impacts on the united nations sustainable development goals (sdgs). students were required to select their own avatar (or perspective) that they take when they pitch their business case to generate interest and potential funding to help realise the idea in practice. they might be an owner of a company, marketing manager, supply chain manager, ceo, cfo, etc., giving them more autonomy in the project itself and contributing to the flow experience and level of 'immersion' in the topic. by encouraging them to be innovative with their idea, this also contributes to achieving flow experience as measured by annetta (2010) . the extent of interactivity was limited to the interactions between the student and their teacher at each designated stage of the project. the system administrator has the capacity to intervene, provide more instructions, change questions, set deadlines as well as hide and lock cells. in this example, the idea section was 'locked' after a designated deadline to ensure the rest of the business report remains unique to the idea, and students cannot deviate from their original plan. this was decided as we wanted students to engage with and 'own' their own idea from the beginning to end. locking the template ensured students could not adjust their idea to make it 'easier' for themselves or provide an easier avenue for plagiarism or contract cheating. we considered the major project as an exam equivalent, hence posed these stricter conditions. increasing complexity of the project was an important part of the design with the complexity increasing as the students learned the topics in class. they were required to begin to pitch their idea in terms of explaining how it would contribute to society (economically, environmentally and socially). then they were required to use the sdgs to determine how performance would be measured and evaluated. next, they had to consider the costs associated with bringing the idea to fruition and the template was coded to use randomly generated numbers for individual students to calculate cost-volume-profit (cvp) and financial performance in terms of ratio analysis. because of the student-entered qualitative data and the randomly assigned input information, requiring different calculations, every assessment piece was unique, requiring individualised written interpretations. students are unable to anticipate questions, as these are blocked from view until the designated time. we did not allow students to change their previous inputs without staff feedback and intervention. the entire course content is matched with topic content, so learning and assessment can be managed in staged developments. informed teaching and regular feedback are designed to contribute to the immersive learning experience. the digital artefact was coded in python and situated on amazon web server (aws). through microsoft api, students and teachers use their rmit login credentials to enter their learning or teacher interface, respectively. the digital platform can be accessed on computers, tablets or mobile phones. students advised us that they used their mobiles to enter thoughts and ideas, even while commuting and would continue to develop until the deadlines. the current server structure and flow diagram of the digital artefact design is indicated in fig. 3 .2, with plans to improve this structure further with the autoscaling technique. this greater flexibility was designed to support work-life balance and other challenges students face getting their work done in a timely manner. the dynamic interface embeds instructions for students. as soon as similar queries emerge through email queries, this can be answered by the administrator (course coordinator) entering direct edits in the interface to ensure all students are clear about the instructions and goals to be achieved. this dynamic feedback and adjustment are essential in large course delivery and also contribute to the well-being of the teaching staff, who do not have to repeat instructions over and over. even if students are provided updated details on the learning management system (lms), our experience is that students tend not to search for clarification updates on the lms but go directly to their instructors. by updating the system, the students avoid the cognitive extraneous effort or 'search' for additional information to support and confirm understanding. is required to access the system. once logged in, all the active assignments will be displayed to students. students can select which project they would like to work on by clicking the blue button 'business plan' (or another identifying label). this screenshot indicates how we use writepal across different courses and jurisdictions. in fig. 3 .5, note that the due date is clearly identifiable for students and the tasks that followed appear blurred, until the designated release time. students cannot see the questions but know how many they must complete to finish the assignment. the instruction for each question is clearly displayed. input can be written in numerical or picture format. in one question, students are asked to present an organisational diagram and draw a value-chain activity diagram. students can hand draw and upload a photo, or they can graphically design in a word document to upload to the platform. the teacher platform enables student tracking of performance and time when each task is completed. further graphical design features can be included, such as badges or tokens on recognising each task completed. figure 3 .6 shows how the feedback is displayed to students. in the writepal system, the student will be able to review instant feedback for each question and the score for the question. teachers have the option to set a designated feedback release date or result release date. the multiple deadlines are clearly shown on the righthand side with upcoming deadlines shown in red as a reminder. recall that students cannot change their previous inputs without staff feedback and intervention, unlike the traditional assessment which could be easily sent to a contract writer to complete. figure 3 .7 provides an overview of the teacher's assignment management portal. in this portal, teachers can add courses, semesters and seminars; set up questions; add users; give extensions; and review student's responses. figure 3 .8 displays how a teacher reviews, provides feedback and checks for marking memo. on the left side of the figure is the student response. the system is set for teachers to select a question type. for example, refer to q9 (fig. 3.7) . this question requires students to upload an image of their company value chain. in the middle of the figure are teacher comments and scores. in the right of the figure is a marking guideline memo providing instructions to staff. figure 3 .9 displays the interface for markers to review how many papers/questions are unfinished. given the assignment is staged, it is important to show how many questions have been answered and how many questions remained ungraded so teaching staff have clear instructions. as indicated in fig. 3 .8, the interface clearly displays student information and the marked and unmarked questions. the system also offers both teachers and students opportunities to generate and download the full report by clicking the 'report' button when necessary. for students, this button is made visible once the report is completed. data was collected in accordance with rmit ethics guidelines. survey data was collected from the final year on topics they found challenging. this initial data was used to inform the digital artefact design. we then pilot tested the digital artefact in our first-year accounting course, which comprises students undertaking business degrees across a number of major programs including accounting, finance, economics, law, management marketing, supply chain and logistics. this is a large course with enrolments ranging 1200-1900 per semester. secondary data from the course experience survey and comments from teachers are used to evaluate the flow experience of our students undertaking assessment through the digital artefact. we used annetta's (2010) eight elements of flow for immersive digital designs to analyse the data. the standard questions on the course experience survey ask students to comment on what is best about the course as well as what they think should be improved. because the digital artefact was being used as a formal assessment piece, we did not want the potential for biased feedback; hence the decision to evaluate what emerged 'unsolicited' from the students at the end of the semester. we did not ask any specific or additional questions about this assessment piece. we also used data in relation to the number of student assignment extension requests, along with direct evidence from the teacher interface, to determine whether students believe the task can be successfully completed. the other items are gathered from themed analysis of the qualitative responses and evidence presented through the unsolicited student feedback. we received positive feedback from both students and staff. we had 979 students and 13 educators teaching in the course. throughout the semester and staged use of the digital artefact, both staff and students agreed that the interface is easy to use. students engaged with the template and enjoyed the continuous feedback and ability to adjust their responses and build on them based on weekly topic content. of the 979 students, we received qualitative comments from 194 students (20% response rate) for the question 'what is the best part of this course'. of the responses, 16% explicitly mentioned the digital assessment was what they enjoyed the most and 28% of the responses indicated that they really liked the staged assignment. the course received the highest overall satisfaction rating and good teaching scores on record for a common core course. we believe the following discussion provides evidence of the flow experience of our large student cohort. while we cannot give a definitive measure of flow experience for every student, the following qualitative evidence helps to support that flow was achieved. evidence of findings for flow element, (1) feeling the activity can be successfully completed, we used data for the number of student extension requests which indicated that our number of extension requests dropped by 50%. we also generated a graph of the system data ( fig. 3 .10) which demonstrates that students remain active before and after the due date for each of the subsequent stages. the graph is also useful in demonstrating the extent to which (2) the player can concentrate fully on the activity. because there is evidence of ongoing activity during the semester, we can claim there must be ongoing concentration and the ability to go in and out of the activity and pick up when required. we also found several qualitative responses from the course experience survey (ces) indicating that students were able to engage with the assessment as an ongoing activity: in further analysing, the entire qualitative dataset data visualisation techniques were utilised. figure 3 .11 provides an overview of the keywords that emerged from the data. figure 3 .11 also helped to indicate that the students felt that they were able to accomplish the task as the data visualisation helped to indirectly demonstrate that (3) the activity has clear goals. we did not find glaring evidence that students were not sure of the assessment requirements and conducted a more detailed examination of the use of some of the apparent negative words. we found that the word 'deadlines' was a positive response to the staged approach and that the tasks were 'broken' into manageable parts. furthermore, the ces results also indicated that 92% of student respondents agreed that they met assessment deadlines. most importantly the word 'stressful' designated that students felt that the digital tool made the experience less stressful. this was evidenced in one of the students' comments: i love the assessment layout for the report. easy to use, easy to meet deadlines, not too much to stress over and it encourages me to get it all done early rather than procrastinating and leaving doing the whole report till the last minute and stressing out the day before it's due. i wish every course could adopt this style. unlike all my other courses, i've never had a panic attack when working on this assignment, especially after procrastinating, thank you so much. (student ces response) another student liked our experimentation and expressed the following sentiment: ......courage to try new ideas in order to help and improve student engagement. the new system used to incrementally complete our individual business report is carefully designed for students in mind, and i believe it is much better than a standard assignment of completing everything by a due date. it is also fairly user friendly. (student ces response) in evidencing that (4) the activity provides fast feedback, at this stage of the project design-we can only provide evidence of teacher interaction. if the digital elements are further enhanced to provide automated feedback to students (for example, through algorithms, bots and ai), we can address this area further. at this stage, we are working with a simplified digital design. during one stage of the question release, we received two (2) emails simultaneously from students asking the same question. we immediately responded to the students directly and updated the instructions in the digital template and this stopped all further questions in relation to clarifying the goals of that specific activity. in terms of flow element (5) the player is deeply involved in the activity, student engagement was evidenced in their novel business case ideas. many came and explained their innovative approaches to us, largely because they were extremely proud of their big ideas. the entrepreneurial spirit was evident in the teacher feedback too. while we cannot determine the level of involvement in the activity for every individual student, we can only surmise from the ces comments and the interactive feedback provided to use during the semester, that a large percentage of the student cohort were willing to engage. the feedback acknowledged the creative freedom as well as the pragmatic gains associated with this activity: it is kind of interesting. i like assignment 2 because it allows for freedom and creativity (student ces response) and, the assignment where students are able to use their interests in making a business plan, which helps with motivation and connection to the content (student ces response). in determining the flow experience of this digital artefact and the response to the element that (6) a sense of control over the actions is necessary to perform the activity, the qualitative evidence from students helped demonstrate their comfort with the task at hand, in which they would not lose control in meeting deadlines, or of conceptual knowledge development throughout the 12-week semester: i really liked how the report was a staged submission. it allowed me to focus on a part at a time and do my best work. this also ensured i didn't leave things to the last minute. (student ces response) …, the individual assignment was a good idea to have going through the semester, with learning content so you can practise the stuff you learn while moving through. (student ces response) it's great that the business plan assignment was done in stages as it made it less stressful to complete. (student ces response) the business report was the best aspect because it allowed us to sequentially submit parts of our assignment. this meant that we weren't stressed about completing the whole assignment by the due date, but rather focus on certain aspects and spend time on each part. i was really motivated to do this business report and it's the first time i enjoyed doing a report. (student ces response) …digital learning and assessments particularly the online system that was developed for the business report is making students in-control as they feel self-confident and independent… (academic peer evaluation response) the final elements of flow-(7) self -awareness disappears during flow and (8) there is an altered sense of time-are difficult to claim evidence without directly asking or observing the students. the data visualisations from all positive and negative ces responses indicate that the most dominant word is 'engaging', meaning that overall the students were satisfied with their course experience. while the other dominant words 'structured' and 'deadlines' that were clearly aligned with the digital artefact also could be viewed as negative, a more detailed analysis of the comments around these words indicates that the students enjoyed the formal way they were navigated through the system. nevertheless, this also indicates that the scaffolded, layered deadlines evident in the course design potentially are a trade-off to the latter two flow elements. nevertheless, in handing control back to the students, we explained that the important 'print' button at the end of the task ensured that the template they used would print to a formatted document that they could be proud to take to job interviews. we also demonstrated that the sdg topic is relatively new to businesses and having expertise and understanding in this area is valuable for future employers. part of the sense of involvement in the activity is related to the direct link to the goals of authentic assessment for students to develop twenty-first century skills and be work ready. the game aspect was really engaging. the teaching team was extremely helpful and friendly. the course content itself was not something i thought i would enjoy however it was super interesting and taught me skills and concepts that seem useful for my future as a possible employee or employer. (student ces response) the findings contributed to confirming that the implementation of a scaffolded, staged approach not only contributed to the flow experience of students but also contributed to their cognitive load. the digital artefact was a relatively successful pilot experiment built on cognitive load and flow experience literature. we consider success in terms of meeting the flow experience criteria designated by the psychology literature dealing with immersive technologies (annetta, 2010; pearce et al., 2005) and contribute to the emerging but minimal literature in this area (bitrián et al., 2020) . while we were able to describe many of the eight (8) flow criteria, some of the elements were harder to directly evidence, thus requiring further exploration in future research initiatives. we also relied on secondary data, and unprompted qualitative responses to measure student flow experience. in recognising this as a potential limitation to the study, further evidence, through targeted surveys and interviews, is recommended. to date, the system appears to minimise plagiarism and contract cheating problems identified by bretag et al. (2019) . this is arguably due to the personalised, unique and progressively released questions that build on the previous inputs. while we cannot be definitive, the ability for contract cheating is harder when assessments are staged and not all questions are made available at the outset. the most important aspect of the system is that it can continually be adapted to new queries, topics and new assessment formats (written, graphics, numerical, etc.) . this means the system is transferable across semesters, courses and educational disciplines. the flow experience, an important part of understanding the degree of engagement and immersion with the digital artefact with the benefit of this dynamic digital artefact, is that other aspects, such as identity and interactivity, can be developed further and tested in new iterations of this digital assessment design. the elements of increasing complexity are important in addressing the ability of the digital design to address the cognitive loads of students when being introduced to challenging topics (sweller, 1988) . likewise, the pedagogical design is in accordance with informed teaching whereby the teacher can play an active role in engaging in the student journey. the findings indicated that this was made possible. we focused more on the student experience, and further research would provide more insights, particularly from the teacher perspective. the instructional design was evident in the student responses, which directly links to the formality of the scaffolded approach identified as important in the cognitive load theory literature. however, this was not exploited as much as the emerging flow literature in digital pedagogy would expect. further iterations of the digital artefact could also engage with more gamified elements such as rewards, badges and leader boards. we also recommend further longitudinal research that follows the first-year students through to their third year, to determine whether this first-year assessment experience contributed to long-term germane, cognitive benefit. we could also give the same digital assignment to different student groups, test different design features, such as staged deadlines, as well as address the cognitive load benefits between the different experimental groups. in conclusion, we consider that the digital artefact has contributed to relieving key pressure points for both academics and students including last-minute assessment preparation, poor-quality assessment designs and submitted work, plagiarism and contract cheating, minimal opportunities for regular academic engagement, and feedback and overall well-being concerns. the "i's" have it: a framework for serious educational game design managing technology in higher education: strategies for transforming teaching and learning designing serious games for education: from pedagogical principles to game mechanisms flow and business simulation games: a typology of students core elements of exemplar academic integrity policy in australian higher education contract cheating and assessment design: exploring the relationship why students cheat. in an exploration of the motivators of student academic dishonesty in higher education beyond boredom and anxiety flow: the psychology of optimal experience what does germane load mean? an empirical contribution to the cognitive load theory conditions that facilitate the implementation of educational technology innovations goal configurations and processing strategies as moderators between instructional design and cognitive load: evidence from hypertext-based instruction flow experience and educational effectiveness of teaching informatics using ar contract cheating: a survey of australian university staff moving from an instructivist to a constructivist multimedia learning environment development and validation of a scale to measure optimal experience: the flow state scale conceptual change takes time: game based learning cannot be only supplementary amusement essential features of serious game design in higher education: linking learning attributes to game mechanics a flow theory perspective on learner motivation and behavior in distance education cognitive load theory and instructional design: recent developments the ebb and flow of online learning using technology for teaching and learning in higher education: a critical review of the role of evidence in informing practice online learner's "flow" experience: an empirical study cognitive load during problem solving: effects on learning cognitive load theory, learning difficulty and instructional design cognitive architecture and instructional design sustainable development goals, united nations department of public information measuring flow experience in and immersive virtual environment for collaborative learning contract cheating: a new challenge for academic honesty key: cord-316222-cm4k04wv authors: cozzolino, mauro; vivo, deborah r.; girelli, laura; limone, pierpaolo; celia, giovanna title: the evaluation of a mind-body intervention (mbt-t) for stress reduction in academic settings: a pilot study date: 2020-07-30 journal: behav sci (basel) doi: 10.3390/bs10080124 sha: doc_id: 316222 cord_uid: cm4k04wv this study is aimed at evaluating the outcomes of mind-body transformation therapy (mbt-t), previously known as the creative psychosocial genomic healing experience© (cpghe). the intervention was aimed at reducing the perceived level of stress in two non-clinical groups of students with different educational levels and different expertise in the domain of well-being. whereas participants from the first group were first-year university students, participants from the second group were students attending a post-graduate program in psychotherapy. all participants (n = 159) were exposed to a single session of mbt-t, each group in a separate session. the results of two paired-samples t-tests, conducted separately on the two samples, showed that there was a statistically significant reduction in the participants’ perceived level of stress between preand post-intervention states in both samples (t(88) = 5.39, p < 0.001; t(53) = 4.56, p < 0.001 respectively). the results, therefore, showed that a single session of mbt-t was beneficial in reducing the perceived level of stress in both first-year university students and students attending a post-graduate program in psychotherapy, regardless of educational level and expertise in the domain of well-being. rates of distress are increasing in schools and universities [1] . current research [2, 3] indicates that the recent covid-19 pandemic, in particular, has significantly worsened mental health issues in students, and there is growing concern regarding the long-term psychological consequences of this outbreak in higher education settings [3, 4] . under normal conditions, undergraduate students witness several important changes related to personal life and education. university students might feel homesick and overwhelmed by academic demands, which might become a major cause of stress for undergraduate students [5, 6] . during this transition, students might also experience minor psychological problems related to sleep, eating habits, and concentration [5] . likewise, more severe psychological problems, such as mental disorders, also share onset within this age range [4] , which might explain the high rates of depression, anxiety, and suicidality [2] [3] [4] . though a few years older, graduate students, too, are often exposed to significant stress because of the pressure and the challenges posed by post-graduate programs. moreover, graduate students are more likely to be married and have children and, thus, they might be more exposed to personal and family problems that cause stress in their everyday life. greater stress may lead to increased anxiety, depression, and suicidality [1] . among graduate students, we believe that post-graduate medical students and students attending a post-graduate program in psychotherapy can be regarded as a special class because stress-related issues are particularly relevant to their work. recent studies on post-graduate medical students [2, 3] demonstrate that they are under severe stress and that a high level of stress leads to academic underperformance, absenteeism, and a poor quality of life. moreover, graduate students attending a post-graduate program in psychotherapy, who generally hold a master's degree in psychology, are expected to be able to command state-of-the art information and other key skills in their field of study, including stress management and well-being. yet, further pressures arising from this expectation might lead to increased distress and feelings of failure in these students [7, 8] . despite experiencing significant stress, most students do not reach out for professional help [2] . research indicates that the most important barriers to seeking mental health therapy among young adults include perceived stigma, self-denial of a mental health problem, negative attitudes about treatment, and practical barriers, such as not knowing where to seek help, cost issues, etc. [4, 5] . these findings prompt universities and institutions providing undergraduate and post-graduate education and training to ensure that stress management programs are always available and, importantly, that these services are structured in such a way as to engage young people. mind-body interventions (mbis) are generally well suited to this purpose because they are often practiced by non-clinical individuals as well, and are often endorsed by celebrities, which steers clear of the stigma usually attached to mental health treatments. mind-body interventions are designed to enhance the mind's positive impact on the body [9, 10] . they are based on various practices that range from ancient techniques for self-care and well-being (e.g., meditation, yoga, tai chi, etc.) to more modern western practices (e.g., mindfulness, hypnotherapy, psychological therapies, etc.) [11] . several studies support the effectiveness of a number of mind-body interventions in reducing stress in university students [12] [13] [14] [15] [16] . moreover, research indicates that the cause of many disease conditions, including inflammatory and neurodegenerative diseases, is a complex interaction between stressful life experiences, the genome, the mind, and behavioral factors [9, [17] [18] [19] . a recent line of research [10, 17, [20] [21] [22] [23] [24] has provided new insights into the pathophysiology of stress-related disorders and it has identified the gene sets involved in a number of biological pathways, including stress response, inflammation, and physical health. these studies describe the genomic and epigenetic pathways of stress, focusing on gene expression changes brought about by mind-body therapies. despite a number of studies supporting the effectiveness of these interventions in lowering the level of stress, [13, [25] [26] [27] [28] , improving well-being [29] [30] [31] [32] , and academic attainment in student populations [33] , they may also present adverse effects and contraindications [34] [35] [36] [37] [38] , especially for those individuals who are advised against mild to moderate physical exertion. in addition, mind-body therapies often require training that may be challenging to learn, and they are generally time-consuming to perform. in order to avoid some of the issues posed by these interventions, we carried out a study on stress reduction using a novel mind-body technique known as mind-body transformation therapy (mbt-t). mbt-t-previously known as the creative psychosocial genomic healing experience© (cpghe) [39] [40] [41] -is a therapeutic protocol that has been shown to improve therapeutic results without the need for long traditional therapies. from both a theoretical and an empirical perspective, it is derived from the studies of m. h. erickson and e. rossi's mind-body therapy [10, 42, 43] . the protocol is based on the so-called four-stage creative process [10] , which is a very easy to learn procedure, allowing individuals to obtain stress reduction without the need for traditional, complex, and intricate methods. although there is extensive literature evaluating stress reduction among university students [44] , relatively few studies have evaluated stress reduction techniques in graduate students [25] and we are not aware of many studies on graduate students attending a post-graduate program in psychotherapy. because we believe that graduate students likely face the same (if not a greater) amount of stress as undergraduates, even if their expertise in stress management is different, we chose to include both groups in our study. our goal was to evaluate whether the positive effects of the mbt-t intervention on stress in undergraduates would be noted in graduate students as well. such findings would suggest that mbt-t is a suitable stress reduction intervention in more than one academic setting, with implications for decision-makers regarding the psychological support programs made available for students in such settings. we used an uncontrolled quasi-experimental design to evaluate the effects of a single mbt-t session on two groups of students sampled by cluster. the first group (group 1) included n = 58 first-year university students, 50% males, mean age = 24.45 (±8.70) years old. the second group (group 2) consisted of n = 101 graduate students attending a post-graduate program in psychotherapy, 22.8% males; mean age = 35.56 (±10.14) years old. women were overrepresented and the age range was broad, reflecting the typical demographics of the university and post-graduate courses our participants attended. overall, 159 students were selected for the study. participants were all exposed to a single session of mbt-t, each group in a separate session. stress was measured using the distress thermometer (dt) [45] [46] [47] [48] [49] [50] [51] [52] [53] . the dt is a single-item screening tool that is well validated to be sensitive and specific to the construct of stress [45] . in order to assess change in self-perceived stress, at the beginning of the session, the researcher asked participants to indicate their perceived "initial stress" on an 11-point scale ranging from 0 (no stress) to 10 (maximum stress) [54] . once the session was terminated, participants were asked again to indicate their "final stress" on the same scale. moreover, participants' feedback about the benefits of cpghe was also collected. as for the measures, although longer measures for the screening of stress might have been used, we opted for a single-item scale for its brevity and ease of administration. however, we based our choice on studies that validated the dt against other robust measures [45] , and confirmed that the single-item dt can be compared with other measures [46] . besides, a number of validation studies and reviews [45, [47] [48] [49] [50] show that the dt has good psychometric properties across countries and cultures. its sensitivity and specificity, as well as its positive and negative predictive value, are in the range of good overall accuracy (donovan et al., 2014; snowden et al., 2014) . moreover, the median scores for these properties are consistent with existing studies written in the english language, whereas different language versions of the dt may have different cut-off scores for clinically significant problems, which is likely due to cultural differences [46, 48] . the cut-off score has also been found to change depending on subject characteristics and setting [46, 49] . nonetheless, a cut-off score of four is widely agreed to indicate clinically significant distress [45, 46, 48] . thanks to its good psychometric properties, as well as its brevity, the dt was an ideal screening tool to include in a study on stress management with university students. the stress reduction method we used in this study was mbt-t, previously known as the creative psychosocial genomic healing experience© (cpghe). the cpghe is based on the four-stage creative process [10, 17] , which facilitates positive psychosocial transformations. it presents certain advantages over traditional mind-body methods (i.e., it is very easy to learn, it can be performed in minutes, it can be administered to both single individuals and large groups, it does not require specific premises or tools, and it only demands one researcher). the cpghe protocol is a four-stage protocol. the first stage is "focusing consciousness", which implies self-awareness of thoughts and feelings. the second stage is "problem review", during which participants assessed the thoughts and feelings from the first stage by focusing on the problem at hand. the third stage is "problem solving", during which participants learned how to cope with the problem arising from the third stage. the final stage is "self-care". within this stage, participants applied what they had learnt from the third stage to their present situation. participants were allowed to share their experiences with each other [39] . the cpghe protocol starts a therapeutic dialogue that may generate new consciousness for a positive exploration of all emotions related to experience, thus reducing acute and/or chronic stress [10] (see appendix a). in order to evaluate changes in the level of participants' stress over time 0 (pre-intervention) and time 1 (post intervention), a paired-samples t-test was conducted separately for each sample (undergraduates and post-graduates). the analyses were conducted with ibm spss statistics for windows, version 23 (ibm corp., armonk, new york, ny, usa). all participants gave their informed consent for inclusion before they participated in the study. tests were anonymous to ensure the confidentiality and reliability of the data. all procedures in this study were performed in accordance with the ethical standards of the italian association of psychology (aip) research committee and with the 1964 helsinki declaration and its later amendments. no further approval was required. we computed two paired-samples t-tests to make a pre-test-post-test comparison, one for each sample. the results of the paired-samples t-test conducted in undergraduate students showed that the level of post-treatment stress was statistically significantly different and lower than the level of pre-treatment stress (t (53) = 4.56, p < 0.001), as displayed in figure 1 . furthermore, the results of the paired-samples t-test conducted in graduate students showed that the level of post-treatment stress was statistically significantly different and lower than the level of pre-treatment stress (t (88) = 5.39, p < 0.001), as is also displayed in figure 1 . in order to test for regression toward the mean, we computed a bivariate correlation between stress scores at t0 and the change scores (calculated by subtracting the time 0 stress scores from the time 1 stress scores) separately for each group of participants. the results of these analyses showed that the correlation was statistically significantly different and negative for both groups (r = −0.56, p < 0.001; r = −0.60, p < 0.001 for undergraduate and graduate students, respectively). behav. sci. 2020, 10, x for peer review 4 of 11 (undergraduates and post-graduates). the analyses were conducted with ibm spss statistics for windows, version 23 (ibm corp., armonk, new york, ny, usa). all participants gave their informed consent for inclusion before they participated in the study. tests were anonymous to ensure the confidentiality and reliability of the data. all procedures in this study were performed in accordance with the ethical standards of the italian association of psychology (aip) research committee and with the 1964 helsinki declaration and its later amendments. no further approval was required. we computed two paired-samples t-tests to make a pre-test-post-test comparison, one for each sample. the results of the paired-samples t-test conducted in undergraduate students showed that the level of post-treatment stress was statistically significantly different and lower than the level of pre-treatment stress (t (53) = 4.56, p < 0.001), as displayed in figure 1 . furthermore, the results of the paired-samples t-test conducted in graduate students showed that the level of post-treatment stress was statistically significantly different and lower than the level of pre-treatment stress (t (88) = 5.39, p < 0.001), as is also displayed in figure 1 . in order to test for regression toward the mean, we computed a bivariate correlation between stress scores at t0 and the change scores (calculated by subtracting the time 0 stress scores from the time 1 stress scores) separately for each group of participants. the results of these analyses showed that the correlation was statistically significantly different and negative for both groups (r = −0.56, p < 0.001; r = −0.60, p < 0.001 for undergraduate and graduate students, respectively). the present study described the implementation of mbt-t on stress management to reduce the perceived level of stress in a non-clinical group of university students. we compared the levels of stress before and after a single intervention of mbt-t in a population of both undergraduate students and graduate students attending a post-graduate program in psychotherapy. the present study described the implementation of mbt-t on stress management to reduce the perceived level of stress in a non-clinical group of university students. we compared the levels of stress before and after a single intervention of mbt-t in a population of both undergraduate students and graduate students attending a post-graduate program in psychotherapy. the results suggested that a single session of mbt-t could reduce the perceived level of stress among our participants. in particular, our results indicated that cpghe/mbt-t showed a significant reduction in final stress as measured at the end of the interventions regardless of educational level and previous expertise in the domain of well-being. our preliminary investigation seems to confirm that mbt-t might be a suitable stress reduction intervention for university students, as well as graduate students attending a post-graduate program in psychotherapy. in being an easy to implement, sustainable, and reproducible intervention, mbt-t might be a suitable approach that students may be willing to use as compared to other types of intervention. therefore, we believe that if mbt-t were incorporated into undergraduate and post-graduate stress reduction programs, students would probably benefit from it and improve their well-being. the mbt-t presents certain advantages over traditional mind-body methods (i.e., it is very easy to learn, it can be performed in minutes, it can be administered to both individuals and large groups, it does not require specific premises or tools, and it only demands one researcher). despite these encouraging findings, we are aware that the uncontrolled quasi-experimental design was a major limitation to our study. furthermore, the absence of a control group did not allow us to draw definitive conclusions regarding the effects of the intervention and caution must be made when interpreting the presented results. the results of the correlations between stress scores at t0 and the change scores (calculated by subtracting the time 0 stress scores from the time 1 stress scores), showed no regression to the mean. however, future research should address this potential critical threat to internal validity by using a clinical trial with a pre-test-post-test assessment in at least two conditions: experimental and control. other limitations to our study could have included circadian rhythms, time effects, history effects, local history effects, and test-retest sensitization. yet, with regard to time effects, the study included a single intervention, which was carried out within the same time slot (from 10 a.m. to 12 p.m.) in both groups. therefore, neither circadian rhythm nor ultradian cycle effects were relevant. furthermore, the intervention was carried out before the covid-19 pandemic, which made history effects and local history effects relatively non-salient. moreover, we acknowledge that participants may have become sensitized to the measure. yet, the mechanism of the so-called pre-test effect or pre-test sensitization is typically not investigated or even known [51] , therefore, it was difficult for us to reduce this effect in our uncontrolled quasi-experimental study. finally, we did not include follow-ups because, at the time of this study, our interest was in investigating the technique's effect after just a single intervention. we acknowledge that the positive results presented may require discounting due to the absence of a control group. besides, this was a pilot study for a preliminary investigation, and we are planning to carry out a randomized controlled trial to evaluate the technique with more statistical power. despite these issues, the present study has a major strength, which is the replication. although reproducibility is the foundation of science [54, 55] , direct replication is seldom pursued [54, 56] . furthermore, as one study shows, replication rates are much higher when the original researchers are involved [57] . our study included replication because it was conducted with two different groups; importantly, it involved the same researchers, so the methods and procedures were carefully mirrored. the results we achieved held true in both settings and populations. hence, the replication gave greater validity to our findings. in addition, our study could be a springboard for further studies on the effects of mbt-t/cpghe. this method could be integrated into more structured programs including more sessions (four to eight) over a longer period of time, which would help us understand how the benefits obtained with a single mbt-t intervention can be maintained and/or enhanced over time. because stress among university students is a widespread and growing problem [44, 52, 53] , we believe that universities should develop more effective and sustainable stress management programs for students that are based on mind-body therapies. in particular, future research should examine innovative methods like mbt-t. in our view, future studies should investigate the effects of this type of intervention to identify and examine methods to maintain the benefits of stress reduction interventions, as well as their effects on different individuals. for this reason, we hope that researchers will further investigate this field of study, taking into consideration the possible use of mbt-t. the therapist begins with the following: "it is wonderful to know how our best thoughts and positive feelings can improve health and well-being. here are a few exercises that will inspire you to explore some interesting questions that can help you solve your problems creatively in your own way." (optional facilitation of stage 1: if needed, the therapist may add a few empathetic statements to clarify whatever concerns and questions some people and the group may have regarding this brief protocol). the therapist now says: "an important aspect of creative problem solving is to realize how you can explore new life possibilities by looking at things from many points of view at the same time. these creativity exercises will ask you to carefully observe yourself and carefully remember what you are experiencing". so, the therapist says: "you can begin by filling out the distress thermometer (pause for a moment). this instrument asks you to record what level of stress you are experiencing right now on a scale of 0 to 10, where 0 is no stress or discomfort and 10 would be the worst stress or discomfort you have ever experienced in your whole life. go ahead and circle or write in your initial stress level right now on the dotted line." (pause for a moment). the therapist models the first stage of the creative process, with the palms of the hands about 20 cm apart facing each other at about chest level, and says: "you can begin by looking at your hands like this". the therapist asks, "which hand feels a bit warmer or cooler?" subjects may sometimes seem puzzled about what this question means. the therapist simply continues, saying "most people don't realize how their hands or other parts of their body usually feel slightly warmer or cooler when they really pay attention to it. this is a good exercise to help you become more aware of yourself. it helps to focus your attention and positive feeling about your natural abilities." after a minute, the therapist adds support and states emphatically, "notice and remember how warm or cool your hands seem to be." (allow another minute for the subject's inner focus). the therapist now asks, "now notice which hand feels stronger or weaker." after 1 min, the therapist adds support by stating emphatically, "notice and remember how strong or weak your hand seems to be!" (allow another minute for the subject's inner focus). the therapist asks, "now notice which hand feels lighter or heavier." after 1 min, the therapist adds support by stating emphatically, "notice and remember how light or heavy your hand seems to be!" (allow another minute for the subject's inner focus). the therapist asks, "now let's explore your imagination. which hand seems to be you today-and which hand feels more like you as a child?" after 1 min, the therapist adds support by stating emphatically, "notice and remember which hand seems to be you at your present age and which hand seems to be more like you as a child!" (allow another minute for the subject's inner focus). the therapist now asks, "which hand represents some problem you would like to solve right now-today in this exercise?" (therapist pauses for 1 min). "and which hand seems to be the opposite, perhaps holds an answer to your problem?" after 1 min, the therapist adds support by stating emphatically, "remember which hand represents your problem and which hand seems to hold the opposite-perhaps an answer, even if you do not know what it is yet!" (allow another minute for the subject's inner focus). the therapist states emphatically, "now let the hand that represents your problem begin to drift down very slowly as you privately review the history, memories, and feelings of your problem from the beginning to the present moment." the therapist offers motivational support with these remarks, administered 1 min apart: • "that's right! do you have the courage to allow that hand and arm to drift down a bit . . . with each memory you find yourself reviewing?" • "allow yourself feel only as much of that as you need to and then move on to the next memory that comes up more or less by itself." • "that's right! let yourself have the courage to continue only as long as you need to to feel everything as fully as you need to privately." • "that's right! while another part of you observes wisely, you learn how to take care of yourself to imagine and create the best possible outcome for yourself." when the problem hand finally touches down in the person's lap, the therapist adds support and offers empathetically, "that's right! allow your problem hand drift down to your lap and come to a comfortable rest, wonderful, appreciate your job well done! remember as much of this stage 2 of your creative process as you need to build a better future! and now, get ready to move on to the solution of your problem with your other hand. let your other hand, holding the solutions to your problem, remain up for a moment so you can now turn your full attention to it!" stage 3: illumination and insight. problem solving the therapist continues saying: "now allow your other hand to drift down slowly as you explore new possibilities about how to solve your problem today. allow that hand to begin drifting down slowly as you begin to explore something new. explore your best hopes and imagination for today and the future, that could be some interesting and wonderful possibilities for problem solving, healing and well-being. speculate about exciting and fascinating turning points in your life. create the best of all possible world for yourself. enjoy your best dreams about yourself!" the therapist observes the shifts from negativity, stress, sadness, and conflict (of stage 2) to the more searching expressions of positive expectation in stage 3 of the creative process that are often punctuated with a slight smile and even a short laugh. he supports these positive shifts with a few warm following remarks, such as these, administered at 1-min intervals: • "something pleasantly surprising you can look forward to? what do you really need that is most interesting and important to you?" • "simply receive and continue to explore the sources of your strength for dealing successfully with that issue." • "yes, appreciate the value of that as fully as you need to while taking good care of yourself as that hand finally comes to rest in your lap." • when the hand finally touches down in the subject's lap, the therapist states in a supportive manner: "remember how real and strong these new positive possibilities and feelings for changing your life for the better are!" • "wonderful, really appreciate yourself for a job well done! and now, get ready to move on to the resolution of this issue (concern, problem, or symptom)!" stage 4: reality testing and self-care the therapist concludes this fourth part with the following remarks, administered at 30-s intervals: • "when (brief pause for emphasis) a part of you knows it can continue this creative work entirely on your own at appropriate times throughout the day." (30-s pause)". • "and when (brief pause for emphasis) your conscious mind knows it can simply cooperate in helping you recognize when is the right time to tune in and continue this creative work privately on your own." (30-s pause). • "learn how you can explore and practice your new ideas in the real world and give yourself positive prescriptions for taking good care of yourself." (30-s pause). "you will bring this creative exercise to an end for now so you can stretch and become fully alert." the therapist can provide support by stating: "some of you may wish to share a few of your insights with the group," or, "all this creative work can remain private within you." optional facilitation of stage 5: if needed, the therapist may encourage the group to share with supportive remarks such as these: • "is there something interesting some of you would like to share about your creative inner work?" • "what is surprising and unexpected about this that is new to you?" • "what interesting possibilities are opening up for you to now?" the therapist asks: "please complete the distress thermometer again by filling in what your stress level is now at the end of your creative exercise." assessment of perceived stress in postgraduate medical students during training programme perceived stress and barriers to seeking help from mental health professionals among trainee doctors at a tertiary care centre in north india prevalence of stress and its determinants among residents enrolled in china standardized training program for resident doctor (c-strd) program: a cross-sectional study young adults face major barriers to seeking help from mental health services the role of personality traits and barriers to mental health treatment seeking among college students coping with the first exams: motivation, autonomy support and perceived control predict the performance of first-year university students the cardiac rehabilitation psychodynamic group intervention (cr-pgi): an explorative study beyond verbal behavior: an empirical analysis of speech rates in psychotherapy sessions the bioinformatics of integrative medical insights: proposals for an international psycho-social and cultural bioinformatics project a brief protocol for the creative psychosocial genomic healing experience: the 4-stage creative process in therapeutic hypnosis and brief psychotherapy a mind-body intervention for stress reduction as an adjunct to an information session on stress management in university students mindfulness and emotion regulation: promoting well-being during the transition to college efecto de intervenciones mente/cuerpo sobre los niveles de ansiedad, estrés y depresión en futuros docentes de edu walking versus biofeedback: a comparison of acute interventions for stressed students a pilot study: the effect of bio-energy on stress for university faculty, staff, and students during finals week effect of a single session of a yogic meditation technique on cognitive performance in medical students: a randomized crossover trial the mind-body healing experience (mhe) is associated with gene expression in human leukocytes epigenetics, hippocampal neurogenesis, and neuropsychiatric disorders: unraveling the genome to understand the mind ultradian rhythms from molecules to mind: a new vision of life what is the molecular signature of mind-body interventions? a systematic review of gene expression changes induced by meditation and related practices epigenetic mechanisms of integrative medicine. evidence-based complement an epigenome-wide dna methylation study of ptsd and depression in world trade center responders crp polymorphisms and dna methylation of the aim2 gene influence associations between trauma exposure, ptsd, and c-reactive protein functional genomics in the study of mind-body therapies interventions to reduce perceived stress among graduate students: a systematic review. worldviews evidence-based nurs use of mind-body therapies among young adults aged 18-24 years: findings from the 2012 national health interview survey the benefits of yoga in children perceived benefits of yoga among urban school students: a qualitative analysis. evidence-based complement a randomized controlled trial of mindfulness-based tai chi chuan for subthreshold depression adolescents building spiritual capital: the effects of kundalini yoga on adolescent stress, emotional affect, and resilience stress and wellbeing of junior doctors in australia: a comparison with american doctors and population norms an exploration of formal and informal mindfulness practice and associations with wellbeing the impact of a mindfulness-based stress reduction course (mbsr) on well-being and academic attainment of sixth-form students tba: terapia bionomico-autogena: fondamenti, principi for whom may participation in a mindfulness-based stress reduction program be contraindicated? mindfulness the effect of imaginary working qigong on the psychological well-being of college students mindfulness meditation and psychopathology the creative psychosocial and cultural genomic healing experience: a new top-down epigenomic psychotherapeutic protocol pilot study on epigenetic response to a mind-body treatment les styles narratifs du groupe comme indicateurs de changement mind-body transformations therapy (mbt-t). a single case study of trauma and rehabilitation the creative psychosocial genomic healing experience©: an open invitation to mind-body psychotherapy interventions to reduce stress in university students: a review and meta-analysis the clinical utility of the distress thermometer: a review screening for psychologic distress in ambulatory cancer patients: a multicenter evaluation of the distress thermometer the distress thermometer: cutoff points and clinical use validation of the distress thermometer worldwide: state of the science measurement of psychological distress in patients with intracranial tumours: the nccn distress thermometer the psychometric properties of the icelandic version of the distress thermometer and problem list the prevalence and correlates of depression, anxiety, and stress in a sample of college students an open, large-scale, collaborative effort to estimate the reproducibility of psychological science the value of direct replication a new etiquette for replication replications in psychology research: how often do they really occur? this article is an open access article distributed under the terms and conditions of the creative commons attribution (cc by) license we would like to thank the university of foggia, department of humanistic study and key: cord-297842-hkr1wm3k authors: tilley, kimberly; ayvazyan, vladimir; martinez, lauren; nanda, neha; kawaguchi, eric s.; o’gorman, maurice; conti, david; gauderman, w. james; van orman, sarah title: a cross-sectional study examining the seroprevalence of severe acute respiratory syndrome coronavirus 2 antibodies in a university student population date: 2020-10-15 journal: j adolesc health doi: 10.1016/j.jadohealth.2020.09.001 sha: doc_id: 297842 cord_uid: hkr1wm3k purpose: the aim of the study was to determine the prevalence of severe acute respiratory syndrome coronavirus 2 (sars-cov-2) antibodies in a university student population. methods: this was a cross-sectional survey study based on the world health organization population-based seroepidemiological investigational protocol for sars-cov-2 conducted between april 29, 2020, and may 8, 2020, examining sars-cov-2 antibody prevalence among 790 university students in los angeles, ca. participants completed a questionnaire on potential risk factors before blood sampling. samples were analyzed using the euroimmun anti-sars-cov-2 elisa (igg) for the qualitative detection of igg class antibodies to sars-cov-2 in human serum or plasma. results: the estimated prevalence of sars-cov-2 antibody was 4.0% (3.0%, 5.1%). factors associated with having a positive test included history of anosmia and/or loss of taste (95% ci: 1.4–9.6). a history of respiratory symptoms, with or without fever, was not associated with a positive antibody test. conclusions: prevalence of sars-cov-2 antibodies in the undergraduate and graduate student university population was similar to community prevalence. this study demonstrates that the seroprevalence of sars-cov-2 antibodies in a representative sample of a large urban university population is similar to that of the surrounding community. symptoms of prior sars-cov-2 infection in the college-aged student include loss of taste or smell but not a history of respiratory symptoms. at the end of december 2019, a novel coronavirus was identified in patients with pneumonia of unclear etiology in wuhan, china [1, 2] . the disease was termed covid-19 by the world health organization, and the underlying viral agent was subsequently termed severe acute respiratory syndrome coronavirus 2 (sars-cov-2) [3] . over the ensuing months, sars-cov-2 spread across six continents, infecting millions of persons. in attempts to control the rapid spread of the virus, governments around the world have taken unprecedented steps to contain the virusdissuing shelter-in-place orders and closing of all nonessential businesses, including institutes of higher education (ihe). because of widespread testing shortages and the varied range of clinical presentations for covid-19, including asymptomatic infection, the true disease incidence in the u.s. since january 2020 is unknown. given public health concerns with reopening ihe, serologic evaluation to determine the student population prevalence of antibodies to the virus is of utmost importance. in the fall of 2019, approximately 19.9 million students were attending ihe in the u.s., representing more than 5% of the u.s. population [4, 5] . there is limited data on the number of infections with sars-cov-2 associated with ihe. surveys done in the spring of 2020 indicate that <1% of college students report having a confirmed case of covid-19 [6, 7] . early cases of covid-19 infection were associated with international travel, yet by mid-march 2020, community spread was evident in many locations across the u.s. for many ihe, with diverse, interconnected, often residential populations, it was unclear what proportion of the population had been infected with sars-cov-2 before the closure of many campuses in march 2020. a national survey of college students conducted between march and may 2020 reported that 14% of college students indicated that they may be or probably had covid-19 based on health care provider assessment or symptoms but not confirmed by a test [6] . given that the severity of the symptoms increases with increasing age and existence of medical comorbidities, many infections among undergraduate and graduate university students are expected to be asymptomatic or minimally symptomatic [6, 8] . thus, the presence of antibodies to sars-cov-2 is important in assessing the current prevalence of infection on college campuses. the assessment of risk factors in students both with and without the presence of antibodies to sars-cov-2 will help guide ihe as they plan for reopening their physical campuses. given the close proximity in which many university students live in dormitories or other off-campus high-density housing [9] , the potential for rapid spread of sars-cov-2 is a relevant concern. university students are also expected to have different social connectedness compared with the general population, such as participation in athletics or social clubs [9, 10] . in addition, the academic course structure of both undergraduate and graduate student education has been shown to have a high degree of connected networks, thus fostering the social conditions for the spread of an infectious disease such as sars-cov-2 [11] . a recent survey on the behaviors of college students who experienced symptoms consistent with covid-19 found that 30.1% continued to attend classes [7] . previously published seroprevalence studies have focused on community-level spread in cities or countries [12e14]. this study aimed to provide an estimate of infection in students attending a los angeles university with a diverse population, including a large number of international students. it also aimed to explore the risk factors for infection in this population. this information will offer evidence-based strategies for control measures, as ihe plan to reopen campuses. the study was a cross-sectional study examining the prevalence of covid-19 antibodies from blood samples obtained from college students. the study design was based on the world health organization population-based seroepidemiological investigational protocol for covid-19 virus infection [15] . data were collected from april 29, 2020, to may 8, 2020, and prevalence estimates reflect this specific snapshot in time. it is not known how long antibodies persist after infection. there is also potential for some lag in ability to detect antibodies after infection, so prevalence estimates could reflect cumulative infection up to approximately late april 2020. the study was conducted at a large urban university in los angeles, ca. approval was obtained from the institutional review board at this institution, and all participants provided electronic informed consent and health insurance portability and accountability act authorization. on april 29, 2020, the county of los angeles, ca, reported 27,641 cumulative cases of sars-cov-2 infection and had 1,156 recorded deaths related to covid-19 [16] . participants were invited to participate in this study via an email invitation, which was sent to 9,135 students (25% of eligible students) enrolled in the spring 2020 semester. inclusion criteria were (1) participants were eligible to access services at the student health center (i.e., students must be enrolled in six or more credit hours for on-campus programs, not in online degree programs) and (2) students' primary campus was the main campus for this institution (this was a practical consideration). exclusion criteria included (1) students aged <18 years and (2) students who were not in the randomly selected pool of potential participants (i.e., students who heard of the study via word of mouth or other similar means were excluded from participating). although not a criterion for receiving a study invitation, only students still living in the region were truly eligible to participate in this study, given that they had to come to the student health center for a blood draw. a stratified random sampling approach was used with the following subgroups: female undergraduates, male undergraduates, female graduate students, and male graduate students. with the goal of having the sample distributions match distributions in the population, these strata were selected based on internal university data, indicating that females are more likely to participate in health-related research projects compared with males, and fewer undergraduates (36.4%) spent the end of the spring 2020 semester in los angeles relative to graduate students (76.5%). within each stratum, a random selection of students was invited to participate. sign-ups were monitored, and demographic distributions of invitations were adjusted as needed. during recruitment, it was also determined that white domestic students were overrepresented in sign-ups, and students in this group were downweighted for random selection in later waves of invitations so that sufficient data could be collected from nonwhite domestic and international students. five waves of invitations were sent. the invitation email was sent to the student's university email address through the student health electronic health record. each email had a unique study code to ensure that only invited students participated in the study. students accepted the invitation by scheduling an appointment at the student health center. the number of appointments available for students to participate in this study was capped at 800. the euroimmun anti-sars-cov-2 elisa (igg) is an enzymelinked immunosorbent assay intended for the qualitative detection of igg class antibodies to sars-cov-2 in human serum or plasma. euroimmun is licensed for use under the food and drug administration's emergency use authorization. testing requires .5 ml of serum or 2 ml of whole blood. specimens were collected at the student health center by the laboratory staff and sent via courier within 24 hours of collection for testing. the validation program at the frederick national laboratory for cancer research determined that sensitivity of this test is 90% (27/30; 95% confidence interval [ci]: 74.4%e96.5%) and specificity is 100% (80/80; 95% ci: 95.4%e100%) [17] . this test classifies individuals into negative (ratio <.8 is considered negativedthis is a normal result for patients who have no or early sars-cov-2 exposure), borderline (ratio .8 to <1.1), and positive (ratio 1.1) for anti-sars-cov-2. in the independent clinical agreement validation study from the manufacturer borderline, results were counted as negative [17] . individuals who were classified as borderline were asked to return for a follow-up test 14 days later. all participants completed a study questionnaire with questions on the following items: prior exposure to a person with suspected or confirmed covid-19 infection; prior covid-19 diagnosis; prior illness history and symptoms since january 1, 2020; prior loss of smell and/or taste since january 1, 2020; receipt of the influenza vaccine during this academic year; travel history since december 1, 2019, including international and domestic; living situation in early march 2020; and current living situation. those who reported coughing, wheezing, shortness of breath or "other" respiratory symptoms were classified as having experienced respiratory symptoms. demographic data on students were obtained from electronic health records, and demographic data in these records are provided by the university. available data included sex (two categories available only: male and female), race/ethnicity/ international status (the university combines these into one variable with levels including asian/pacific islander, black/african american, hispanic/latino, international from any non-u.s. country, multiracial/multiethnic, white/caucasian, and other/ unknown; at this school, the majority of international students are from china), and degree program category (collapsed into two categories: undergraduate and graduate). to reduce potential bias, weights were created for each participant using iterative proportional fitting to match demographics available on the overall university population. the following variables were included: sex by degree program category and race/ethnicity/international status by degree program category. we used a bayesian approach to estimate the antibody prevalence of covid-19 in our student population by incorporating the sensitivity and specificity of the diagnostic test into the prevalence estimate [18] . to be consistent with the euroimmun anti-sars-cov-2 elisa (igg) manual [17] , we classified a borderline test result as negative. prior values for the sensitivity and specificity were taken from the validation program at the frederick national laboratory for cancer research [17] . the prior on the true prevalence was taken from a recent study on antibody prevalence in la county [13] and reflects the community from which our sample was derived from. of the 863 adults in la county that consented to the study and were tested, 35 individuals tested positive. more information on the prior and model specifications are found in the appendix. odds ratios (ors; unadjusted for covariates in the model) were used to explore potential factors associated with a positive antibody test. for these analyses, positive cases were compared with borderline and negative cases combined. given the exploratory nature of this work, adjustments were not made for multiple comparisons, so results should be interpreted within this context. reference groups for demographic factors were female for sex, white/caucasian for race/ethnicity/international status, and undergraduate for degree program category. for all other variables, the reference was not having the given characteristic. there were 790 students who participated in this study. the sample was 48.2% female, 30% international, and 52.2% undergraduate (table 1 ). at the time of data collection, most students lived in off-campus housing, alone or with roommates/friends (61.3%); in their family home (26.5%); and in university-owned housing (9.1%). initially, 23 individuals tested as borderline. all but four of these people returned for follow-up testing, and five people had a borderline test the second time, with one person having a positive test. estimates for the prevalence and its 95% credible intervals are reported in table 2 . the estimated prevalence of covid-19 antibody positivity at the time of our study was 4.0% (3.0%, 5.1%). posterior estimates for the sensitivity and specificity can be found in the appendix. furthermore, when stratified by academic status, the estimated prevalence was 4.0% (3.0%, 5.2%) and 4.0% (2.9%, 5.2%) for undergraduate and graduate students, respectively, suggesting that the two student groups may not have differing lifestyles that will make them more (or less) susceptible to infection. our analysis treated borderline subjects (n ¼ 10) as test negative. to investigate the influence of potentially misclassifying borderline subjects, we provide a supplementary analysis where we treat borderline subjects as test positive. this increased the prevalence slightly to 4.3% (3.2%, 5.7%). these results can be found in the appendix. to assess the influence of prior specifications to the prevalence of covid-19 antibody positivity, we ran the model, classifying borderline results as test negative, by placing a noninformative prior to the prevalence. with a noninformative prior, the estimated prevalence of antibody positivity was 3.6% (.7%, 6.0%). there were three participants (reflecting 2.8 weighting observations) who reported that they had a previous covid-19 diagnosis via a nasal or throat swab; all these participants also had positive antibody tests (reflects 8.8% of positive antibody cases). more than half (61.4%, weighted n ¼ 19.4) of the positive cases did not report any prior illness with respiratory symptoms, and more than one third (37.0%) did not report any prior illness at all (others reported symptoms such as fatigue, headache, sore throat, and runny nose). we report the unadjusted ors and their 95% cis in table 3 . in this sample of university students, the following factors were associated with having a positive test for sars-cov-2 antibodies: students with a history of loss of taste and/or smell were 4.0 times as likely to have a positive test, compared with those without this history (95% ci: 1.4e9.6); students with confirmed or suspected exposure to a positive covid-19 case were 3.3 times as likely to have a positive test, relative to those without confirmed or suspected exposure (95% ci: 1.4e7.0). we observed increased test positivity for those reporting international travel (or ¼ 1.6, 95% ci: .8e3.3) and domestic travel (or ¼ 2.1, 95% ci: .9e5.2) since december 1, 2019, although neither was statistically significant. of the 26 positive cases who reported domestic travel, only 10 reported regional travel within southern california. sex, race/ethnicity, academic level, current and prior living in university-owned housing, and history of flu shot during the 2019e2020 academic year were not significantly associated with having a positive antibody test (data not shown for all variables). a history of respiratory symptoms, with or without fever, was also not associated with a positive antibody test. we perform a supplementary analysis by creating weights for each participant using iterative proportional fitting to match demographics available on the overall university population to reduce potential bias. the following variables were included in calculating these: sex by degree program category and race/ethnicity/international status by degree program category. we present the weighted unadjusted or in the appendix and note that the overall conclusions are consistent with what we previously observed. seroprevalence of sars-cov-2 antibodies in a los angeles university student population as of may 8, 2020, was estimated to be 4.0%. this does not substantially differ from what has been reported in prior population studies from santa clara county and los angeles county. in the los angeles county study, done in april 2020, prevalence was estimated at 4.65% (95% ci: 2.8%e 5.6%) [13] . likewise, an analysis of the blood samples from approximately 3,300 people living in santa clara county in early april estimated prevalence at 2.8% (95% ci: 1.3%e4.7%) [14] . this study demonstrates that the prevalence of infection at this institute did not differ from the larger community. the low prevalence indicates that sars-cov-2 was not widely circulating in the student population before the closure of the physical campus in mid-march 2020. this institute started the transition to virtual learning on march 11, 2020, with spring break the following week. for the remainder of the spring semester, the students completed all coursework virtually, and the physical campus was closed except for a small percentage of students who remained in universityowned housing. on march 19, 2020, governor gavin newsom issued a stay-at-home order for the state of california. it is likely that these events played a part in the low seroprevalence of sars-cov-2 in this study. with the low seroprevalence to sars-cov-2 in our population at the time of this study, there is not a strong argument for widespread antibody testing to inform decisions on reopening the college campuses. the reopening of ihe will rest on the ability to mitigate spread through continued physical distancing measures, environmental measures, promotion of behaviors that reduce spread, contact tracing, and access to testing. the centers for disease control is collaborating with public health departments and private laboratories to use seroprevalence surveys in different locations and populations to help estimate the number of persons who may have been infected with sars-cov-2 and not included in official case counts [19] . although widespread antibody testing at this time has limited usefulness, additional seroprevalence surveys from ihe will be important to estimate the number of infections associated with ihe and to further our understanding of risk factors for infection as physical college campuses reopen. approximately 1% of the samples in our study fell in the borderline or indeterminate range. this may represent early infection with a rising antibody titer, prior infection with waning antibody production, or cross-reactivity with another virus. loss of smell and/or taste could be a relevant indicator in this population, which has also been reported by others [20] . this may be a screening tool relevant to ihes, although more information on duration and intensity of lost smell/taste would be beneficial. notably, 39.6% of negative cases reported respiratory symptoms, suggesting that their answers to this question likely reflected respiratory effects from seasonal conditions unrelated to sars-cov-2. the presence of sars-cov-2 antibodies indicates prior infection, but it is still unclear whether this indicates immunity. we did not perform additional neutralization antibody assays to determine further characteristics of the antibodies. it is also unknown what percentage of students with asymptomatic or mild infections develop detectable antibody response. prior studies with the middle east respiratory syndrome (mers-cov) demonstrated that the severity of the disease correlated with the antibody response [21] . this consideration is important for ihe; if large numbers of the population have asymptomatic or mildly symptomatic infections, seroprevalence studies may underrepresent the prior disease incidence. further longitudinal serological studies on the college population are needed to determine ongoing disease incidence as well as the extent and duration of immunity to sars-cov-2. there are several other potential limitations of this study. there is limited data available on the validity of the assay used in this study, although this uncertainty was accounted for in statistical analyses. limitations in our understanding of borderline findings also make it challenging to draw conclusions. in analyses of potential risk factors, adjustments were not made for multiple comparisons, and estimated associations of each factor with prevalence were not adjusted for other risk factors, and so these findings should be interpreted with caution. only students currently residing in the region were eligible to participate in this study, so true prevalence in this university population is unknown. demographic data are limited in their availability, so there may be other biases present because of unmeasured or insufficiently measured variables. it is also possible that there are factors that drove interest in participating in this study, such as prior symptoms, which may affect prevalence estimates. furthermore, it is not known how generalizable these findings may be to other university populations. although it is still unknown the frequency in which positive sars-cov-2 antibodies develop or are sustained in asymptomatic or subclinical infection or if antibodies confer protective immunity and how long that immunity will last, antibody testing remains a powerful tool to examine prevalence of a disease in a population after the initial infection. over time, this information can be used to assess risk factors, monitor spread, and help administrators and public health officials plan for easing current mitigation and future health care needs. our results reflect antibody positivity in this university population because of infection with sars-cov-2 in the early days of the pandemic and during the statewide shutdown. in addition to showing comparability with general population prevalence, our estimates serve as a useful comparison for future studies that may be conducted in this population as this university campus returns to normal operations. a novel coronavirus from patients with pneumonia in china a familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster the species severe acute respiratory syndrome-related coronavirus: classifying 2019-ncov and naming it sars-cov-2 digest of education statistics united states census bureau. us and world population clock the impact of covid-19 on college student well-being. american college health association a descriptive study of coronavirus disease 2019 e related experiences and perspective of a national sample of college students in spring 2020 evidence supporting transmission of severe acute respiratory syndrome coronavirus 2 while presymptomatic or asymptomatic american college health association-national college health assessment ii: reference group executive summary spring 2019. silver spring, md: american college health association club and intermural sports participation and college student academic success the small-world network of college classes: implications for epidemic spread on a university campus spread of sars-cov-2 in the icelandic population seroprevalence of sars-cov-2especific antibodies among adults in covid-19 antibody seroprevalence population-based age-stratified seroepidemiological investigation protocol for covid-19 virus infection county of los angeles public health. covid-19 data dashboard anti-sars-cov-2 elisa (igg) instruction for use a tutorial in estimating the prevalence of disease in humans and animals in the absence of a gold standard diagnostic large-scale geographic seroprevalence survey real-time tracking of self-reported symptoms to predict potential covid-19 mers-cov antibody responses 1 year after symptom onset, south korea the authors thank the students who participated in this study for completing the risk factor questionnaire and providing a sample for antibody testing in a timely manner. supplementary data related to this article can be found at https://doi.org/10.1016/j.jadohealth.2020.09.001. key: cord-326372-pjmk4eru authors: oliver, david title: medical education and covid-19: a personal view date: 2020-06-17 journal: croat med j doi: 10.3325/cmj.2020.61.213 sha: doc_id: 326372 cord_uid: pjmk4eru nan the covid-19 pandemic challenges us all -personally, academically, and within society. the way we interact day to day has been altered, certainly in the short term over the coming months, and maybe in the longer term over the next years. this has affected all areas of life, but education has been particularly affected, as in the past courses have usually been delivered face to face with close interaction between teacher and student. the greatest change that is developing is the increasing use of online resources. these have already been developing for many years, with the recording of lectures becoming a routine in many universities, so that students could revisit them and use the resources as part of their overall study. however, online lectures have become the main form of teaching and may continue to be needed over the coming months. this does bring some benefits but there are also risks. care needs to be taken to ensure that true education is provided, with stimulation and encouragement of the students and true learning. the benefits of online resources include the following: • the lecture can be optimized so that it is of the best quality possible -teachers can use the opportunity to record the lecture at their convenience and when all the necessary resources are present. • the possibility of the lecture being watched at a time convenient to the student, and on a repeated basis if this is necessary for full understanding • lectures can be given in real time -with the possibility of interaction, although this is more difficult at a distance. • new teachers can be involved, even from other areas or countries. at a recent conference in february, just before lockdown, one speaker, from another country, was unable to attend and presented online, with no problems and full interaction, including answering questions. • students are often used to online resources and cope very well with the system, minimizing the technical issues. • time is used efficiently as there is no travel to and from venues. the future of large international congresses is in doubt as everyone realizes that if the meeting is online, travel and accommodation costs will be eliminated and time away from the workplace reduced. a recent virtual congress of the european academy of neurology had over 42 000 participants from all over the world. • patients could be included within sessions. they can be recorded, giving their history and views on care as well as having signs demonstrated. this recording can be paced and at their convenience, so that the risks of them becoming overtired is reduced. in this way patients are more likely to agree to being involved. their input can enhance teaching and be very helpful in engaging students in the session. • students could be involved with some areas that they may have difficulty seeing normally -recordings of patients who are usually unable to come to clinics or lectures, recordings of operations, continual recordings over time of a patient showing the development of a disease process, and rural or isolated areas can be involved. • online sessions can be shared with other areas -even with other countries, where the availability of educational resources may be more limited (1) . there are also risks and challenges of developing online systems: • the personal contact and interaction are more complex and difficult to organize. discussion and cross-fertilization of students, which may happen in an interactive session or seminar, is not as easy. we may all have to consider how this can be facilitated. • the teaching of practical skills may be more difficult, although there is increasing evidence that basic skills, including communication skills, can be developed virtually (2). • there will be a need for the development of skills for both teachers and students in the use of online resources. it may not be as simple as recording "the usual lecture, " but a new approach may need to be developed. these skills will be necessary for teachers, but students may also need to develop skills in the best use of virtual teaching. • lectures may need to be carefully planned and extra techniques used, such as video clips, to ensure that students do not become bored and lose attention. • students may become isolated. student-to-student interactions are important, and need to be facilitated. • it is essential that resources are kept up to date -the same lecture cannot necessarily be used every time, with new student groups. there are areas that may be particularly challenging. within the teaching of palliative care, the aim is often to help students develop communication skills, consider and develop their attitudes and feelings in care, and to look at how they do interact with patients, families, and other members of the multidisciplinary team. teaching these issues may not be insurmountable, but care is needed to ensure students are able to discuss some issues in greater depth, and a blended approach, of online teaching with small face-toface seminar groups, may be necessary. these are great challenges, but with openness to change, and a desire to move forward, online resources may become firmly developed with the curriculum. however, we must never lose the opportunity for face-to-face discussion, interaction and sharing of experiences, as these are crucial in the development of a student into a medical doctor. moodle and online learning in pakistani medical universities: an opportunity worth exploring in higher education and research effectiveness of digital education on communication skills among medical students: systematic review and meta-analysis by the digital health education collaboration key: cord-330860-zxms4nel authors: patel, shalizeh a.; halpin, richard m.; olson, gregory w.; franklin, amy title: global pandemic and the rise of teledentistry date: 2020-08-08 journal: j dent educ doi: 10.1002/jdd.12355 sha: doc_id: 330860 cord_uid: zxms4nel nan before the pandemic, teledentistry was in its infancy, officially permitted in only a few states. covid-19 arrived and changed the landscape, halting in-person dental care except for emergencies. dentists and patients were instantly disconnected, thrusting teledentistry into the spotlight. additionally, with clinics closed, dental education live-patient experiences paused. educators were forced to implement remote learning and find meaningful clinical experiences for students to bridge the gaps in the curriculum. could high-fidelity simulation exercises teach telehealth foundational skills to support dental students as they become comfortable with this mode of care? could this be created and implemented in a timely fashion as part of a summer rotation? to answer these questions, the simulation team, in cooperation with the innovation team, developed a series of synchronous teledentistry encounters using live actors as patients. three cases were generated as new patient encounters and addressed patients experiencing (1) postoperatory sensitivity, (2) stress-triggered tmd, and (3) primary herpetic gingivostomatis. these cases were designed to have features amenable to a teledentistry visit. students were provided with a suggested checklist of key points to address during a teledentistry encounter, a brief patient note containing pertinent medical and dental history, and access to the patients via a video-conferencing platform. 1 third-year dental students were tasked with 2 time-limited (8 minutes) patient encounters in which they were expected to evaluate the patient, provide a differential diagnosis, and introduce the next step in care. prior to the simulation experience, students were asked to complete a survey covering their impressions of teledentistry and comfort level leading a teledentistry encounter. a faculty member concurrently evaluated the students using a rubric designed to assess teledentistry encounters. following each visit, immediate feedback was provided by the faculty observer using the plus/delta model. all encounters were recorded. in addition, 3 weeks after the exercise, students reviewed their videos and rated their performance. during a full class debrief, students were once again asked to complete the short survey of impressions and comfort level. 2,3 preliminary analysis of data focused on the level of comfort the students reported, pre-and post-simulation. ninetythree (n = 93) students completed both surveys with 5% attrition. for the pre-survey, 63 students (64%) reported some level of discomfort ("slightly"/"very" uncomfortable). following the exercise, 21 students (23%) reported ta b l e 1 numbers (percentages) of students declaring comfort level of "slightly/very" uncomfortable" pre-simulation and post-simulation pre-simulation survey (n = 93) "slightly"/"very" uncomfortable 63 (64%) 21 (23%) an increase of self-reported comfort level is observed post-simulation. continued discomfort with the idea of leading a teledentistry visit ( table 1 ). the students were asked to provide a unique tracking code. this code allowed the pre-and postresponses to be analyzed. overall, matched pairs (n = 76) reported a level of comfort one step higher on the agreement scale on the post-survey than on the pre-survey (average = 1.3 on a 5-point scale). from pre-experience surveys, students anticipated their discomfort with teledentistry. post-experience surveys revealed that their comfort level improved. this rapidly created simulation exercise provided opportunities to teach and test telehealth practices as our profession considers expanding remote delivery of care in the time of covid-19. furthermore, these simulated encounters enabled the innovation team to study the feasibility of teaching as well as implementing telehealth into its preclinical and clinical educational models. american health information management association (ahima) the mediated debrief of problem flights impact of simulated patients on students' self-assessment of competency in practice of geriatric dentistry key: cord-318528-yc0jw3s1 authors: romero-blanco, cristina; rodríguez-almagro, julián; onieva-zafra, maría dolores; parra-fernández, maría laura; prado-laguna, maría del carmen; hernández-martínez, antonio title: physical activity and sedentary lifestyle in university students: changes during confinement due to the covid-19 pandemic date: 2020-09-09 journal: int j environ res public health doi: 10.3390/ijerph17186567 sha: doc_id: 318528 cord_uid: yc0jw3s1 regular physical activity is related to many factors in a university student’s environment. the coronavirus pandemic and the resulting lockdown have restricted many elements of our environment. the aim of this study was to evaluate students’ physical activity and sedentary behaviour at two points in time: before and during the coronavirus lockdown. as a secondary aim, we also wanted to look at changes resulting from other factors (alcohol, tobacco, diet, stages of change, symptoms of anxiety/depression and sociodemographic characteristics). we conducted an observational, cross-sectional, pre-post study with two cut-off points. two hundred and thirteen students took part in the study. the main dependent variables were physical activity and sitting time, measured using the international physical activity questionnaire—short form (ipaq-sf). parametric and non-parametric tests were used for paired and unpaired data, as well as group-stratified analysis. during lockdown, both weekly physical activity (md: −159.87; ci: −100.44, −219.31) and weekly sitting time increased (md: −106.76; ci: −71.85, −141.67). in the group analysis, differences were observed in relation to gender, year of study, bmi, alcohol consumption, tobacco use, symptoms of anxiety/depression, mediterranean diet, living situation and stage of change. the results showed an increase in both physical activity and sitting time globally and by group. a healthy lifestyle should be promoted among all ages, but the earlier a habit is formed, the more likely it is to become rooted [1] . regular physical activity is one of the most effective ways of preventing premature death [2, 3] . the world health organization (who) recommends at least 150 min of moderate physical activity, 75 min of vigorous activity, or a combination of the two, per week [4] . independently of the physical activity carried out, it is important to assess sedentary behaviour (sb) as this is related to increased morbidity and cardiovascular risk factors [5] . by 2030, the who aims to reduce the prevalence of physical inactivity by 15% worldwide [6] . in spain, the amount of physical activity carried out by university students is low [7] and is in many cases linked to other healthy habits such as eating fruit and not smoking [8] . meanwhile, sedentary behaviour is a health problem in the child and youth population, which is aggravated with age [9] . in university students, sitting time can exceed 9 h a day [10] . it is known that individual factors such as age, sex and health status affect the physical activity that individuals do [11] . other factors associated with physical activity are motivation, lack of time and aspects related to body image or physical appearance [12] ; some of the beneficial effects of physical activity are reduced anxiety and depression [13, 14] . however, there are several factors that come into play throughout an individual's lifetime that can either facilitate or impede a behaviour, with the transition from secondary education to university being a decisive moment [15] . it is at this time that young adults form their behavioural habits, so the role of healthy universities and the healthy habits they acquire at this stage are fundamental in maintaining this behaviour in the years to come [16] . when it comes to making physical activity a regular habit, the elements that may be related have been studied in depth [17] . ecological models are considered one of the most significant theoretical approaches when it comes to analysing habit formation [18] . these models establish that in addition to individual factors, social and environmental factors are determinant in forming and maintaining physical activity habits [19] . the covid-19 pandemic led to the population being confined to their homes [20] . in spain, from march to april 2020, there was a prohibition on going outside to engage in sporting or social activities. during this period, elements of the built environment and other factors related to individuals' environments were restricted due to the state of alarm. this created a valuable opportunity to assess physical activity without taking these factors into account. experts' recommendations to prevent sedentary behaviour during lockdown included taking active breaks, getting up and walking around the house, and doing online workouts [21] . however, during the pandemic, an overall negative effect on physical activity intensity was observed, as well as a rise in the consumption of less healthy food and a 28.6% increase in sedentary behaviour [22] . a reduction in physical activity was also observed in university students [23] , along with increased levels of anxiety among 18-to 34-year-olds [24] . spanish university students had to continue attending classes online, and their social lives were limited due to the prohibition on going outside. during lockdown, physical activity could have been an opportunity to pass the time, or, conversely, sedentary behaviour could have increased. the other characteristics of each individual (gender, motivation, eating habits, mental state etc.) could have either facilitated or interfered with the decision to exercise. the hypothesis put forward was that students' sedentary behaviour would have increased during lockdown since they were confined to their homes, and that their physical activity would have decreased since they could not go outside to exercise. in this study, we aimed to analyse the physical activity university students did before and during lockdown. to broaden our approach, as a secondary aim, we also wanted to look at changes in physical activity and sedentary behaviour resulting from other factors such as alcohol and tobacco consumption, adherence to a mediterranean diet, motivation, symptoms of anxiety/depression and sociodemographic characteristics. we aimed to evaluate whether there were any differences when certain factors affecting individuals' environments were restricted. this was an observational, cross-sectional, pre-post study on health sciences students, with two cut-off points. the first cut-off point was between 15 and 30 january 2020, prior to the state of alarm being put in place, and the second sample point was between 1 and 15 april 2020. this study received the approval of the ethics and clinical research committee of ciudad real, in spain, with protocol number (c-291, 11/2019). this study was carried out within the context of another study that we conducted on healthy habits and lifestyles, with an estimated follow-up period of 9 months. due to the state of alarm and lockdown, recruitment of subjects was temporarily suspended and a decision was made to study the impact of lockdown on the population already participating. there were no exclusion criteria, other than failure to fully complete the questionnaire. to estimate the sample considering a bilateral hypothesis, the following criteria were used: variance in the pre-lockdown control group of 33,929.60, obtained using the total minutes of physical activity [25] , a beta risk of 20% (power = 80%), a confidence level of 95% and a clinically important difference of 60 min with respect to the control group. it was therefore estimated that a minimum of 148 study subjects would be needed. considering a missing values ratio of 20%, the resulting sample size would be 185 subjects. the students invited to take part were first-to fourth-year students who agreed to respond to the questionnaire at both time points. the questionnaires were administered during the second university semester. the first data collection point was two weeks after the end of the exam period, while the second data collection point was four weeks into lockdown. at the second data collection point, students could not leave their homes except for essential purposes such as buying food or going to hospital. outdoor exercise was prohibited across spain; anyone breaching the rules faced a 600 euro fine. during lockdown, university classes continued online with the same schedule as usual. the university provided internet access or technological devices to any students who requested them so that they could continue attending classes. online classes did not contain any recommendations for students to carry out physical activity. an ad hoc self-administered questionnaire was used, collecting sociodemographic information such as sex, age, weight, height, place of residence during the academic year, smoking habits (yes/no and number of cigarettes per day) and alcohol consumption (yes/no and number of drinks per week). for perceived health status and the existence of problems with anxiety/depression, the euroqol 5d (eq-5d) questionnaire was used [26] . to assess adherence to the mediterranean diet, the predimed questionnaire [27] was used, which uses 14 questions to assess the frequency of food consumption and eating habits. each question has a possible score of 0 or 1. the result allows classification into low adherence or high adherence. stages of change (soc) in physical activity were assessed using prochaska and diclemente's transtheoretical model (ttm) [28] . five stages of motivation for change were evaluated: pre-contemplation (i don't exercise and i don't intend to), contemplation (i don't exercise, but i'd like to), preparation (i exercise sometimes), action (i have been regularly exercising for less than 6 months) and maintenance (i have been regularly exercising for more than 6 months). physical activity was measured using the international physical activity questionnaire-short form (ipaq-sf), which contains 7 questions [29] . the questionnaire was used to obtain the total minutes of physical activity per week and sitting time per day. first, descriptive statistical analysis was performed using absolute and relative frequencies for categorical variables and mean with standard deviation (sd) for the quantitative variables. next, bivariate analysis was performed on the whole sample for paired data between weekly minutes of physical activity for the two sample points (pre-lockdown and lockdown). we used the kolmogorov-smirnov test to verify the normality of the quantitative variables. since there were variables that were not normally distributed, we then used the non-parametric wilcoxon signed-rank test. we also used the parametric student-fisher t-test to evaluate whether there were statistical differences in some comparisons and to obtain an approximation of the differences found. finally, the same analyses were performed again, but this time stratified for different sub-groups. mean differences (md) were obtained with a confidence interval of 95% (ci). all calculations were done using the program spss v24.0 (ibm corp, new york, ny, usa). two hundred and thirteen health sciences students participated in this study. the mean age was 20.5 years (sd = 4.56). of the participants, 80.8% (172) were women, 76.5% (163) were normal weight and 9.9% (21) were smokers. the rest of the demographic characteristics and health parameters are shown in table 1 . then, the results of the ipaq questionnaire were analysed: days and minutes of physical activity per week, as well as time spent sitting per week at both time points studied ( table 2) . we observed a significant increase in the number of days on which students engaged in physical activity, both vigorous we then analysed physical activity by group (table 3 ). when we looked at the differences in average minutes of physical activity, all groups analysed spent more time doing physical activity during lockdown (although not all of them significantly). groups that showed significant differences were women; first, second and third year of study; normal or low bmi; and those who did not eat a mediterranean diet. average physical activity time reduced during lockdown for participants in the pre-contemplation (md: 37.50; 95% ci: −115. 33, 190.33) and contemplation (md: 31.08; 95%ci: −15.87, 78.03) stages. in other words, they spent less time on physical activity, although this difference was not significant. conversely, for those in the preparation (md: −75.59; 95%ci: −0.92, −150.25) and action (md: 322.69; 95%ci: −214.84, −430.55) stages, significant differences (p < 0.05) were observed. in the rest of the groups analysed, statistically significant differences were observed between the two time points, except for men, final-year students, those that were overweight or obese and those that ate a mediterranean diet. finally, the analysis by group (table 4 ) showed significant differences (p < 0.05) in sitting time before and during lockdown in all groups except first-year students, those that were overweight or obese, smokers and those in the pre-contemplation stage. sitting time increased in all groups of the variables gender, alcohol, symptoms of anxiety/depression and mediterranean diet. it also increased in the following groups: second, third and fourth year of study; normal and underweight bmi; non-smokers; those living in a university residence, shared apartment or with family; and those in the contemplation, preparation, action and maintenance stages. this study aimed to evaluate physical activity and sedentary behaviour in health sciences students before and during the lockdown. at the first time point, students were in their normal study environment, while at the second, their social and environmental setting was limited due to lockdown. the results showed changes in physical activity and sedentary behaviour patterns both globally and by group. overall, students spent more time doing physical activity and spent more time sitting when their usual environment was limited. in the analysis by group, minutes of physical activity increased significantly during lockdown among the following groups: women; all years of study except final year; normal or low bmi; those who did not eat a mediterranean diet; and those in the preparation or action stage of change. sitting time increased in all groups of the variables gender, alcohol, symptoms of anxiety/depression and mediterranean diet. the groups that did not experience differences were: first year of study, overweight or obese, smokers and those in the pre-contemplation stage. these four groups spent the most time sitting at the first data collection point when compared with the rest of their cohort; in other words, sedentary behaviour was already high before lockdown and there were no significant differences at the second data collection point. some researchers believed that lockdown would cause inactivity and an increase in sedentary behaviour and that measures would need to be taken to prevent these effects [30] . in fact, during lockdown, people modified their lifestyles, with an increase in sitting time due to people spending more time at home, and there was also a reduction in the amount of time spent on physical activity [22] . in our study, the initial hypothesis was partially confirmed: there was an increase in sitting time, but unexpectedly, there was also an increase in both the amount of time spent doing physical activity and the number of days on which participants were active. we expected to find an increase in sitting time due to the restrictions on movement; however, we also thought that the increase in screen time would reduce physical activity time, since in previous studies conducted in the spanish university population, more screen time was associated with higher inactivity levels [31] . we do not know the exact reasons why physical activity increased, and we do not know if the effects on physical activity habits would have been maintained if the lockdown had gone on for longer. the environment in which students live affects their sedentary behaviour patterns [32] , and it seems that the characteristics of health sciences students' environments do not facilitate physical activity. rather than being an obstacle, restricted social relations and not having access to the built environment in their community increased the number of days and minutes students spent doing physical activity. in the case of health sciences students, another factor to consider is that their training in promoting healthy habits may have influenced their decision to exercise at home. no changes in physical activity were found in men. perhaps men and women had different motivations and the environment influences one gender more strongly. in previous studies on motives for physical activity by gender [33] , some variables that motivated men but not women were elements related to the environment, such as competition or social recognition, while weight control was the main motivation for women. in our study, women accounted for more than 80% of the sample, so the lack of results may also be due to the fact that there were fewer male participants. the effect of the built environment is yet to be determined for those with a high bmi [34] . the data in this study show that in overweight or obese students, there were no changes in time spent doing physical activity or sitting time. as we have seen, healthy habits that are ingrained in the population are not affected by the lockdown: this is the case of the mediterranean diet [35] . in this study, we observed that students that ate a mediterranean diet spent more time doing physical activity and that their physical activity patterns did not change significantly. this suggests that those that lead a healthy lifestyle pay attention to both diet and exercise and persist with their habits regardless of the environment. conversely, those with unhealthy habits stick to them and experience no changes during lockdown. this is the case for smoking and sedentary behaviour. grouping of healthy and non-healthy factors is habitual in university students [8, 25] : those that are more sedentary are also more likely to smoke or spend a lot of time watching screens, while those that exercise regularly tend to eat more fruit and vegetables and drink less alcohol. contrary to what we expected, smokers did spend more time doing physical activity during lockdown. it would be interesting to investigate the reasons for this. in our sample of the population, the percentage of smokers was very low, and the number of cigarettes smoked per day was also low, so we believe more research is needed in a sample with more smokers. in our results, we also found differences based on year of study. among final-year students, physical activity did not vary significantly. this group also spent the least time doing physical activity at both time points analysed. in their meta-analysis, keating et al. indicate that with regard to year of study, the majority of studies find no differences in physical activity, but that some studies suggest that higher years of study are less active [36] . as for sedentary behaviour, it was observed that first-year students spent more time sitting and that lockdown did not bring about any significant changes. some studies, contrary to the findings of our study, observed that students in higher years of study were more sedentary due to a higher workload [10] . in health sciences students, most of the theoretical workload is in the first year, while in their final year students spend most of their time on placement. another possible factor could be that first-year students might have practiced sport in secondary school and kept up the habit. it would have been interesting to ask students about their sports histories. in this study, we evaluated stages of change, one of the central concepts of the transtheoretical model of change. this model was initially used to treat tobacco and alcohol problems, but it was later adapted to other aspects of health such as physical activity and sedentary behaviour [37] [38] [39] . the analysis of the stages of change and how they affected the participants was very interesting. participants in the first two stages did not experience any changes, and neither did those in the last stage. the behaviour of participants that exercised as part of their routine remained practically the same, as did the behavior of those that did not do any exercise. however, for those that were motivated but had not yet made exercise a regular habit, lockdown was a good opportunity to increase their dedication. in line with these findings, di renzo et al. [35] observed in a recent study that lockdown increased activity among people that did sport occasionally because they had more time at home, but those that did not do any exercise did not use the situation as an opportunity to start. overall, the results show that minutes of physical activity increased, as did minutes of sitting time. although the results during lockdown are positive in terms of physical activity, it is necessary to recognise that this population might suffer from health issues in the future due to an increase in sedentary behaviour. it would be interesting to find out what the reasons were for students having this behaviour. perhaps they realised that their sitting time increased (they were not walking to class, walking to their car, going shopping, standing up, going to their jobs etc.) and compensated for this with some high-intensity exercise. another aspect that could have affected the results is that the students were involved in the health sciences field, so they may have been more prone to exercising during the pandemic than students in other majors such as engineering or literature. this is why we cannot exclusively consider the limitation of the environment during lockdown to be the cause of the changes in physical activity and sedentary behaviour. it would be interesting to continue studying the elements related to university students' physical activity/sedentary behaviour and their surroundings in order to plan strategies that promote an increase in physical activity levels in this group. our study has various limitations that should be considered. firstly, it is an observational study and all study subjects volunteered to participate in the questionnaire, so there may be a selection bias. secondly, we did not measure whether there was any risk of exposure to covid-19 infection, a factor that could have influenced our assessment of physical activity and sedentary behaviour. another limitation is the use of a self-administered questionnaire to evaluate physical activity and sedentary behaviour. it would have been more interesting to perform a real assessment of physical activity using accelerometry and also investigate their sports history. this could be a future line of research. finally, the lack of significance in some of the strata analysed could be due to a lack of statistical power because of the low number of subjects in some groups. furthermore, we do not know if these changes in physical activity would have been maintained if lockdown had gone on longer. as for the strengths, this is the first study to look at physical activity and sedentary behaviour in university students studying health sciences both before and during lockdown. in this study, we observed the behaviour of health sciences students when deprived of their usual social and community environment. participants spent more time doing physical activity and also spent more time sitting. university students' social environment may be a barrier to building an exercise habit, especially among women, and motivation seems to have a significant bearing on whether university students engage in physical activity. more efforts should be made to create strategies that motivate students to lead a healthy lifestyle in all aspects (diet, avoiding harmful substances, mental health etc.), with a particular emphasis on engaging in physical activity and reducing sitting time. programs and policies that promote positive youth development and prevent risky behaviors: an international perspective health benefits of physical activity: the evidence health benefits of physical activity: a systematic review of current systematic reviews world health organization. global recommendations on physical activity for health sedentary behavior and cardiovascular morbidity and mortality: a science advisory from the american heart association world health organization. global action plan on physical activity 2018-2030: more active people for a healthier world levels and patterns of objectively assessed physical activity and compliance with different public health guidelines in university students determinants and patterns of physical activity practice among spanish university students sedentary behavior among spanish children and adolescents: findings from the anibes study how sedentary are university students? a systematic review and meta-analysis correlates of physical activity: why are some people physically active and others not? cultural factors associated with physical activity among u.s. adults: an integrative review physical activity and incident depression: a meta-analysis of prospective cohort studies physical activity protects from incident anxiety: a meta-analysis of prospective cohort studies determinants of health-related lifestyles among university students nouri-aria, k. changes in student physical health behaviour: an opportunity to turn the concept of a healthy university into a reality toward a better understanding of the influences on physical activity: the role of determinants, correlates, causal variables, mediators, moderators, and confounders an ecological perspective on health promotion programs an ecological approach to creating active living communities the resilience of the spanish health system against the covid-19 pandemic recommendations for physical inactivity and sedentary behavior during the coronavirus disease (covid-19) pandemic. front. public health effects of covid-19 home confinement on eating behaviour and physical activity: results of the eclb-covid19 international online survey the impact of isolation measures due to covid-19 on energy intake and physical activity levels in australian university students exploring lifestyle habits, physical activity, anxiety and basic psychological needs in a sample of portuguese adults during covid-19 aguilo-pons, a. clustering of lifestyle factors in spanish university students: the relationship between smoking, alcohol consumption, physical activity and diet quality the spanish version of euroqol: a description and its applications a 14-item mediterranean diet assessment tool and obesity indexes among high-risk subjects: the predimed trial stages and processes of selfchange of smoking:toward an integrative model of change validation of three short physical activity questionnaires with accelerometers among university students in spain how to deal with covid-19 epidemic-related lockdown physical inactivity and sedentary increase in youth? adaptation of anses' benchmarks cluster analysis of health-related lifestyles in university students neighborhood built environment and socioeconomic status are associated with active commuting and sedentary behavior, but not with leisure-time physical activity, in university students college students' motivation for physical activity: differentiating men's and women's motives for sport participation and exercise role of built environments in physical activity, obesity, and cardiovascular disease eating habits and lifestyle changes during covid-19 lockdown: an italian survey a meta-analysis of college students' physical activity behaviors role of counseling to promote adherence in healthy lifestyle medicine: strategies to improve exercise adherence and enhance physical activity application of the transtheoretical model to sedentary behaviors and its association with physical activity status levels of physical activity, motivation and barriers to participation in university students funding: this research received no external funding. the authors declare no conflict of interest. key: cord-276204-ibmnuj5u authors: ratten, vanessa; jones, paul title: covid-19 and entrepreneurship education: implications for advancing research and practice date: 2020-10-20 journal: nan doi: 10.1016/j.ijme.2020.100432 sha: doc_id: 276204 cord_uid: ibmnuj5u this article aims at critically examining the linkage between entrepreneurship education and covid-19 in order to help understand future research and practice paths. due to the large global impact covid-19 has had on society, new entrepreneurial education management practices are required to deal with the change. to do this, this article discusses why covid-19 can be a transformational opportunity for entrepreneurship education research due to the new thought processes raised by the pandemic. the article suggests several assumptions that have changed as a result of covid-19 and how entrepreneurship education is required in order to help solve the pandemic. by doing this, the article suggests that more entrepreneurship education research embedding a covid-19 context is required to breakthrough new frontiers and reset the research agenda. by taking an entrepreneurial stakeholder perspective that looks at entrepreneurship education as a holistic process, an enhanced analysis of how response mechanisms including recovery and change are conducted can be made. this enables a way to view the covid-19 crisis as an opportunity for more attention placed on the importance of entrepreneurship education for society. covid-19 is posing a significant challenge to management education especially for international students and courses that have an experiential nature (brammer and clark, 2020; marshall and wolanskyi-spinner, 2020) . restrictions on mass gatherings and social distancing requirements have limited in class teaching, which has resulted in a massive quick shift to online teaching methods as a result of the covid-19 pandemic (ratten, 2020) . this has meant an increase in courses taught through digital communication methods (krishnamurthy, 2020) . border closures and cuts to international travel have further restricted international student mobility and business activities (donthu and gustafsson, 2020) . to respond to these changes there has been a rapid uptake in remote and digital learning entrepreneurship education methods (bacq, geoghegan, josefy, stevenson and williams, 2020) . entrepreneurship education is considered as an important way to influence the competitiveness of any country or industry, so it provides opportunities in the pandemic to progress to a more competitive educational environment (liguori and winkler, 2020) . whilst there are many debates about whether entrepreneurship can be taught, the general consensus is that any form of education regardless of an individual's personality can have beneficial outcomes (glaveli, 2008) . this means that the design and structure of an entrepreneurship course can be moulded to suit an individual's learning preference (suseno and ratten, 2007) . these preferences are embedded in more tailored courses that incorporate cultural dimensions to entrepreneurship education. the goal of these courses is to inculculate a specific orientation whether it be an industry or experience within the teaching philosophy (miragaia, da costa and ratten, 2018) . entrepreneurship education generally tends to be defined as study that teaches skills to start and manage a business for growth (mentoor and friedrich, 2007) . however, more recently this has changed due to more students interested in acquiring knowledge about j o u r n a l p r e -p r o o f entrepreneurial behaviour that does not necessarily equate to starting a business . for this reason, studying entrepreneurship is encouraged as a way of getting students to think about future career directions. this means that students learn about different possible careers that range from a start-up, to include small business management and corporate venturing (peterman and kennedy, 2003) . thus, entrepreneurship education is recognised not just for its ability to teach practical skills but to also obtain knowledge about how to help communities and promote a improved quality of life (ratten, 2017 ). an entrepreneurial attitude includes an emphasis on a personal control over a situation that incorporates some degree of innovation, which is important during the covid-19 crisis (brown and rocha, 2020) . this means emphasizing an individual's ability to change a course of action because of their self-esteem and need for achievement (rauchand hulsink, 2015) . the growth of entrepreneurship education during the past decade has been phenomenal and is now a common course in most business schools (santos, neumeyer and morris, 2019) . this importance of entrepreneurship education in changing student's perceptions is reflected in numerous studies (souitaris, zerbinati and al-laham, 2007) . for example, zhang and cain (2017) found that more than 50% of entrepreneurship education students intended to become entrepreneurs after finishing their courses. this is supported by kubberod and pettersen (2017) who found that students who took entrepreneurship training were more positive towards entrepreneurship. entrepreneurship education has become more complex due to the need to teach a range of topics related to innovation and futuristic thinking (oosterbeek, van praag and ijsselstein, 2010) . consequently, pedagogical approaches now focus on managing expectations by teaching students how to be resilient (ahmed, chandran, klobas, linan and kokkalis, 2020) . this means embedding a real world context that exposes students to service learning (santos et al, 2019) . addressing world problems is part of entrepreneurship curricula as it provides a j o u r n a l p r e -p r o o f way for students to help solve societal issues through focusing on their intention to become entrepreneurial (beynon, jones and pickernell, 2016 ). an entrepreneurial intent is defined as "a clear and conscious decision to start a new venture" (elliott et al, 2020) . this means students can apply entrepreneurship principles that empower them to take strategic action. a recent review article on entrepreneurship research by ferreira, fernandes and kraus (2020 found that entrepreneurial principles can be analysed in many different ways depending on the environmental context including health, technology and social-related concerns. in this commentary, we explore the way to encourage entrepreneurial intention in students by focusing on the effects of covid-19 on entrepreneurship education. to do this we draw on existing and emerging management education practices surrounding covid-19 (beech and anseel, 2020) . these reflections will enable a better understanding about the current state of play in management education regarding entrepreneurship (duval-couetil, ladisch and yi, 2020) . crisis are complex and their effects are felt immediately but also over a long period of time (ansell and boin, 2019) . the covid-19 crisis was a low-probability event that was unpredictable. doern et al (2019:401) state that crises can be classified as "extreme unexpected unpredictable events or as more mundane everyday disturbances, sudden or gradual, crisis have also been categorizes as 'major' or 'minor', 'internal' or 'external' and as 'technical/economic' in nature or 'people/social/organizational centric'". this dichotomy in approaches reflects the varying nature of crisis from natural ecological events to economic disasters (buchanan and denyer, 2013) . whilst the nature of a crisis can be ranked on a scale of severe to inconsequential, all crisis influence human life in some way or another (eggers, j o u r n a l p r e -p r o o f 2020). this results in some businesses perform better than others due to their level of resilience, which means they can cope due to creating solutions based on available resources (faulkner, 2001) . the covid-19 pandemic was unpredictable and a surprise (world health organisation, 2020). as clark, davila, regis and kraus (2020) state, there are many different ways countries have responded to the covid-19 crisis depending on their regulatory policies. as a result, the covid-19 pandemic has resulted in significant societal upheaval and it has been a career shock for many students and educators (akkermans, richardson and kraimer, 2020) . most management educators had not considered a need to move rapidly to working and studying from home and in a digital environment. the good news is that for most students and educators the change meant they could continue their studies albeit in a different environment. this continuity in education was needed particularly in terms of ensuring students course progression (jones, 2019) . unlike other industries, the education industry could move online in a short time period, which enabled students to continue their studies but also enabled others to learn through online courses. this has been helpful in ensuring society continuity and also providing a way for individuals to learn during times of crisis (doern, 2016) . since the world health organization on march 11, 2020 declared the covid-19 crisis a pandemic, there has been an enormous impact on management education. this impact can be analysed through a geography approach using the space dimensions of territory, place, scale and network (brinks and ibert, 2020) . as different countries and regions have varying policies regarding in-class versus digital class participation there are conflicting views about the best way to change educational practices as a result of the crisis (donthu and gustafsson, 2020) . this means the scale of the impact can be high or low depending on the amount of educational courses that are moved to a fully online environment. as a result, education and j o u r n a l p r e -p r o o f learning networks make a difference in finding the right approach to take. thus, as its effects continue to be felt in the education sector, policy makers have issued and implemented improvements. crisis are not new to the education sector but the impact of covid-19 has had the biggest impact on education practices compared to other crisis. as a result, the crisis has brought forth new questions about the role of education in society and how the education industry can respond to the crisis. over the past decade, management education as a scientific area of inquiry has blossomed with the continued growth in enrolments (ferreira, fayolle, ratten and raposo, 2018) . this means there is an already existing substantial body of knowledge that can help decipher ways to deal with crises (williams, gruber, sutcliffe, shepherd and zhao, 2017) . however, due to the impact of covid-19 being high, a complex solution that takes into account multiple stakeholders is required (bailey and breslin, 2020) . there is some confusion about pandemics due to pre-pandemic stages being communicated that might not lead into a serious situation (alon, farrell and li, 2020) . this means the public health response to a pandemic can change depending on the severity of a disease and how it progresses (watson, 2011). there was a delay in declaring the covid-19 a pandemic despite the seriousness of the situation, which meant there has been some controversy over whether it was managed by global health authorities (cortez and johnston, 2020) . in addition, as there are political repercussions from having a pandemic declared, this created accusations of politicization (hall et al, 2020) . whilst a coordinated international approach to dealing with a pandemic is required, the current closure of country borders means this is hard to do (hall, scott and gossling, 2020) . in addition, each country is competing for medical personnel and other health experts (he and harris, 2020) . this has created a talent war but also increased competition for required medical devices. the covid-19 pandemic is global whereas previous pandemics have been largely focused on specific areas (he and harris, 2020) . this has meant it is difficult or almost impossible for health professionals to travel to other geographic regions. this has resulted in a sense of national interest instead of international solidarity (hall et al, 2020) . in addition, many developed countries have had significant outbreaks of the disease and required help from developing countries. this reversal in health policy has been unprecedented and is surprising given the increased levels of internationalisation we have seen in the past. therefore, the covid-19 pandemic represents a unique opportunity for entrepreneurs to transform existing practices (kirk and rifkin, 2020) . linnenluecke (2017) suggests three main ways to build resilience: develop adaptable business models, alter global supply chains and strengthen employees. by adapting business models, organisations can move their production facilities to focusing on relevant market needs. this enables rapid innovation by shifting the market focus into new business practices. in the current covid-19 crisis contactless and online services have been emphasized so organisations have had to change their business models to reflect this (kirk and rifkin, 2020) . as supply chains have become more global, it is useful to have alternative countries for sources of production. this helps to decrease the reliance on one country and means that in times of a crisis there are multiple source of supply. as a result, employees in the education sector need to be strengthened in terms of the autonomy and independence they have in making decisions (hahn, minola, bosio and cassia, 2020) . this enables teaching and mentoring employees about business ideas. in addition, businesses that have a higher level of reliance are better able to cope with change, which can enable them to act swiftly by preparing new business ideas (hills, 1998) . this means that in times of a crisis some degree of improvisation with educational methods that incorporate entrepreneurial thinking is needed (krishnamurthy, 2020) . entrepreneurship education is defined as "any pedagogical program or process of education for entrepreneurial attitudes and skills" (fayolle et al, 2006:702) . this definition reflects the notion that the field of entrepreneurship education involves the application, design and implementation of innovative, futuristic and proactive strategies to an educational environment. this involves analysing new market potential and technological solutions. fayolle and klandt (2006: 1) also took a pedagogical approach in defining entrepreneurship education as "any pedagogical programme or process of education for entrepreneurial attitudes and skills, which involves developing certain personal qualities". therefore, most definitions of entrepreneurship recognise the need for concurrent study of multiple topics that lead to change. part of entrepreneurship education involves taking an inter-disciplinary perspective that incorporates different study areas for a practical solution. jones (2019:243) mor recently describes entrepreneurship education as a "collective of initiatives operating in universities, community colleges, vocational (or trade) schools, high schools and elementary (or primary) schools, that are held together by a common desire to develop in students a greater capacity for entrepreneurial agency". this reflects the emphasis on entrepreneurship education on its real life suitability that reflects changing societal conditions. therefore, entrepreneurship education unlike other scientific areas that are more theory based offers a promising way to teach others how to deal with the covid-19 crisis. in higher education there is a tendency to evaluate employability in the form of hard skills rather than soft skills (clinkard, 2018) . entrepreneurship education tries to include both hard and soft skills in order to contribute to lifelong learning. this means hard skills that are technical or role-specific are combined with soft skills such as personability. employability is a key issue facing graduates as they embark on their career. increasingly students after they finish their courses will have multiple jobs due to a reliance on project-based work (marginson, 2011) . this means students will need to continually upgrade their skill repertoire depending on market conditions. in addition, the increased casualisation of the workforce means some individuals will need to have a number of jobs. helyer and lee (2014) discuss how work experience is crucial to the future employability of higher education graduates. therefore, internships have been advocated as a way to mix workplace learning with study needs (guile and lahiff, 2013) . internships enable a more economically efficient way of learning on the job whilst acquiring knowledge. there are different streams within entrepreneurship education that reflects its intricate nature (ferreira, fayolle, ratten and raposo, 2018) . entrepreneurs are change agents that pursue opportunities in the marketplace so entrepreneurship education has a positive impact on student's capabilities and ability to adjust to emerging technologies (beynon, jones and pickernell, 2016) there is a multitude of existing studies on entrepreneurship education at the undergraduate (eg von graevenitz et al, 2010) , masters (eg raunch and hulsink, 2015) and doctoral level (eg munoz et al, 2020). these different levels reflect how entrepreneurship is taught from a number of different perspectives including at the micro, macro and meso level. at the micro level, the emphasis is on small businesses and teaching individuals to become entrepreneurs whilst at the macro level it focuses more on global entrepreneurship issues evident in large scale companies. at the meso level social entrepreneurship and communitybased entrepreneurship topics are at the heart of entrepreneurship courses (ferreira, j o u r n a l p r e -p r o o f fernandes and ratten, 2017) . thus, most research on entrepreneurship education tends to associate positively the impact of entrepreneurship education on entrepreneurial intention and attitude (ahmed et al, 2020) . for example, kolvereid and moen (1997) found that participants in entrepreneurship education classes were more likely to start their own businesses than non-participants. peterman and kennedy (2003) found that participation in entrepreneurship education classes increases the desirability of pursuing an entrepreneurial career. whilst most research on the link between entrepreneurship education and behaviour is positive, some studies find no positive effect. for example, in a study on undergraduate students, oosterbeek, van praag and ijsselstenn (2010) found that intentions to start a business decreased after finishing an entrepreneurship class. similarly, mentoor and friedrich (2007) found that participation in entrepreneurship education did not affect intention to be an entrepreneur. entrepreneurship education involves a variety of activities that include both in-class and outof-class learning (jones, penaluna and pittaway, 2014) . this means a holistic and more longterm approach to evaluating the effect of entrepreneurship education on behaviour is needed. increasingly entrepreneurial education is viewed as a programme consisting of a number of components that can alter based on contextual situations. ahmed et al (2020:4) describe an entrepreneurship education programme as consisting of "several components including course content (eg lecture material, guest speakers, online resources, modes of delivery etc) and course goals (eg learning introductory concepts and theory) compared to learning specific skills". this means in the covid-19 environment course content can change to reflect new environmental contexts whilst the course goals can remain the same depending on the entrepreneurial intent (iivari, sharma and venta-olkkonen, 2020) . to understand the link between entrepreneurship education and intentions different theories can be applied (secundo, mele, sansone and paolucci, 2020) . human capital theory proposes that individuals study entrepreneurship as a way to build their business skills (ahmed et al, 2020) . this means individuals acquire certain skills and knowledge by studying certain subjects. unger et al (2011) suggest that human capital is linked to entrepreneurial success as individuals acquire more practical training. entrepreneurial self-efficacy theory has also been used as a way of understanding how individuals develop skills through entrepreneurship training. this is evident in chen et al (1998) finding that entrepreneurial self-efficacy distinguishes entrepreneurs from managers. this means entrepreneurship education can embed a more independent and risk-taking spirit in students. this helps them build confidence about their ability to start a new business. jena (2020) this results in some inequity in terms of attitudes towards entrepreneurship education. the social environment in terms of family, school and work interaction will also play a part in influencing feelings towards education (jones, klapper, ratten and fayolle, 2018) . the affective component concerns feelings and emotions an individual has towards education. this means some individuals will react differently to learning stimuli. in order to increase the perceived desirability of entrepreneurship education it is important that consideration regarding intuition is acknowledged. some individuals will express themselves j o u r n a l p r e -p r o o f in different ways that impact on the entrepreneurship experience (jones, jones, williams-burnett and ratten, 2017) . in addition, courses will be evaluated differently depending on the perception of value co-creation. this is due to students having different motivations and reasons for studying entrepreneurship. the behavioural component involves the responses an individual has towards education, which means that individuals will act differently depending on how they learn . for some students, the desire to study entrepreneurship is motivated by a financial incentive whilst for others non-financial reasons such as contributing to society are paramount. therefore, the expected responses to entrepreneurship education are based on the aspirations of students (ratten, 2017) . students will have different attitudes towards entrepreneurship education depending on whether they are taking the subject as an elective or compulsory part of their course. this means there will tend to be more positive feelings towards the experience if the student chooses to study the topic. the covid-19 pandemic emerged at the start of the academic year for students in australia and asia that begin studies early in the year. for most european and north american universities, the crisis happened in the middle of the teaching year. in australia, the crisis initially meant many international students predominately from china could not enter the country so the focus was moving to online teaching to accommodate these students. this students particularly those in management courses requiring experiential learning have been highly affected. whilst classes have altered new pedagogy and assessment to suit the new conditions, student have had to quickly adapt to new learning methods. in addition, students have been physically and socially isolated from their peers that has caused mental hardship. this has been mitigated in some way through online social activities that try to recreate physical environments. for many students the campus environment plays a big role in their intellectual development. with many campuses closed this has affected the way students learn and their ability to learn through peer networks. extra-curricula activities including sport and drama societies have closed impacting the social life that students experience in conjunction with their academic activities. in addition, internships, international study tours and site visits have stopped. this has limited the interaction students have with communities. however, there has been a positive increase in online activities that re-create normal environments. this has had a favourable impact in terms of students thinking and acting in creative way and helped to build resilience and entrepreneurial thinking in students that is needed in the competitive global marketplace. entrepreneurship students can be considered as nascent entrepreneurs as they are involved in business activities whilst studying (souitaris et al, 2007) . this means sometimes entrepreneurship students are already entrepreneurs and are taking the class as a way to learn j o u r n a l p r e -p r o o f more about entrepreneurship. university students near the end of their courses are more likely to want to start a business (ahmed et al, 2020) . thus, in times of crisis entrepreneurship plays a key role in retraining individuals for new industries that are emerging in society. traditional employment channels are changing as there is more reliance on the gig economy. this means students are likely to have a portfolio career in which different employment paths become available. this has meant that there is less emphasis by students on finding employment in large firms and the government sector. as a result, more students have become interested in startups and starting their own business. young people are being encouraged to apply for jobs in new industries some of which were not in existence in the previous year. this means the emphasis in young people is to think creativity and a way to learn this is through entrepreneurship education. there has been much uncertainty associated with covid-19 that has resulted in misinformation (krauss, clauss, breier, gast, zardini and tiberius, 2020). in addition, rumours about its origin continue to cause political effects which has had an impact on students. as there is scarce work integrating a medical and entrepreneurship approach more collaboration is needed in terms of student learning. the link between medical science and entrepreneurship is intuitive and natural due to both being about innovation. this means taking a big-picture perspective to covid-19 makes sense due to the complementary strengths of both science and entrepreneurship (kuckertz, brandle, gaudig, hinderer, reyes, prochotta and berger, 2020) . this will help to unveil digestible knowledge that can be used in different segments of society. another way to look at the interdisciplinary nature is to focus on the hard or soft sciences. typically the hard sciences such as medicine and engineering are rule and results orientated whilst the soft sciences such as entrepreneurship focus more on creativity. this means it is worth noting that due to the current covid-19 crisis causing much panic in society j o u r n a l p r e -p r o o f interdisciplinary research using hard science but with a soft science mentality can help. this will enable strategies to be developed to protect public health whilst reducing negative effects on students. by leveraging the abilities of both the hard and soft sciences it can showcase innovative developments for societal good. therefore, raising awareness of an interdisciplinary approach for enabling medical and social scientists to work together. entrepreneurship education normally involves some form of interactivity in which students are immersed in an environment that involves them learning about how to perform a task. the environment whilst normally physical can be virtual in terms of augmenting reality to enable students to learn about a behaviour. examples of learning platforms include writing a business plan, pitching an idea or conducting a market analysis (ahmed et al, 2020) . within an entrepreneurship course there are normally some tasks involving how to develop creativity that can lead to a business venture. another key learning benefit of entrepreneurship education is the exposure students have to real entrepreneurs. this can involve role models or mentors that inspire students to be entrepreneurs. management education at all levels from undergraduate to graduate has been dramatically affected by the covid-19 pandemic. practical placements in most entrepreneurship courses have been removed or significantly reduced. this means that developing innovative methods of entrepreneurship education is challenging without traditional classroom environments. therefore, it is essential that entrepreneurship educators sustain high-quality teaching methods as the success of future entrepreneurs depends on it. an entrepreneurs role in society can be culturally acquired due to societies expectations about individual behaviour based on economic conditions. this means in times of crisis, an entrepreneur will experientially j o u r n a l p r e -p r o o f acquire certain skills. guest speakers are often used in classes as a way of describing their experiences. although online guest speakers can also be done when social distancing limits physical contact. in addition, it can be beneficial to have guest speakers from international locations via online platforms that overcome geographical distance limitations. in addition, incubators and accelerator programs are increasingly being used in entrepreneurship programs. this is due to initial ideas needing help in terms of making them commercially viable. most universities have programs designed to nurture an idea and enable participants to access required resources. this can include competitions and internships designed to expose students to outside ideas that provide helpful feedback. in addition, science and technology parks are often located on university grounds enabling students to have more interaction with business. krishnamurthy (2020) suggest that there are five main trends impacting business education as a result of covid-19: algorithms, service, assessment, personalisation and problem solving. algorithms mean that students can learn via artificial intelligence rather than through direct human interaction. this means that increasing amounts of theory will be taught via online activities rather than through human contact. service means that relevant and contextual information will be taught. this means instead of students studying a set number of subjects, there will be more timely courses taught that depend on current events. thus, the emphasis will be on continuing learning rather than a discrete number of subjects. assessment means that objective learning outcomes will be emphasised so that learning is the result of a multitude of factors that are interwoven to create a specialised learning experience. personalization refers to making changes to an educational offering to suit a specific individual. this helps to cater for specific learning needs that can enable better performance. problem solving means finding answers to issues facing society. increasingly this skill is viewed in a positive way in light of increased inequalities amongst members of society. given that the global management education environment has significantly changed as a result of the covid-19 pandemic the content of entrepreneurship education programs varies with some valuing a practical and immersive experience more than others. this means there is a debate about the right approach to take in entrepreneurship education from a practical approach to more of a hybrid model including both practice and theory. this is due to the importance played on learning concepts and theory related to entrepreneurship whilst engaging in the act of being an entrepreneur. therefore, the emphasis is placed on learning about business planning and how to take calculated risks. more recently skills such as creativity and decision making have been emphasised. the most commonly referred to objectives of entrepreneurship education programs is to acquire knowledge about action plans related to business and to stimulate an entrepreneurial mindset towards change (glaveli, 2008) . this means skills related to management including accounting and marketing are embedded in the teaching of attitudes towards entrepreneurship. due to the wide ranging number of objectives in entrepreneurship courses it can be hard to align the design of the course with the outcomes. hence, the content of entrepreneurship education courses needs to be treated in an analytical manner. as the covid-19 crisis has deepened around the world, so does the need to practice and think in an entrepreneurial manner (parnell, widdop, bond and wilson, 2020) . the effects of the covid-19 crisis are felt in diverse parts of society and has resulted in increased inequality gaps between rich and poor (pantano, pizzi, scarpi and dennis, 2020) . the interest in entrepreneurship education has grown due to changing economic conditions emphasising ecological sustainability and social equity. this has resulted in more entrepreneurship education courses that incorporate both economic activities and social objectives. the covid-19 crisis has the potential to transform management education but to realize this potential management education needs to critically evaluate its epistemological foundation in order to challenge existing paradigms (nicola, alsafi, sohrabi, kerwan, al-jabir, iosifidis, agha and agha, 2020) . finding solutions to how education can evolve as a socio-economic activity is crucial. technology plays a key role in this due to the need for digital learning environments. since the introduction of the internet and online forums for education, information technology has been an important enabler and disruptor for management education. this is especially true in today's education environment in which most aspects of teaching have a technology component. technology is now interwoven into education and has become even more important during the covid crisis. technology has been adapted and developed to solve some of the issues faced by changing environmental conditions. to build resilience in education, technology is used as an effective way to manage change (liu, shankar and yun, 2017) . whilst there are many benefits of technology it cannot fully replace physical contact in which non-verbal communication is important. the arrival of covid-19 has been a watershed moment as it totally changed how life is conducted (higgins-desbiolles, 2020). previously there was a large trend towards international entrepreneurship and transnational entrepreneurship so the introduction of border closures as a result of covid-19 has significantly altered existing business models. social activities have been reduced and people told to communicate via digital technology. these changes have been made due to an effort by governments to control the pandemic. this has meant trying to strike a balance between public health requirements and economic needs (perrow, 1984) . as a result, difficult circumstances have emerged in which governments have kept essential services whilst trying to reduce the impact on public health (milliken, 1987) . the unprecedented size of government interventions have helped reduce the impact of j o u r n a l p r e -p r o o f covid-19 on society so the covid-19 crisis offers a way to grow better and prioritize entrepreneurship education. there is little clarity in the entrepreneurship education literature about the inputs and intended outputs (pittaway and cope, 2007) . this means there is significant diversity about what the right teaching and learning methods are in entrepreneurship education. this is due to a variety of attitudes and skills needed for entrepreneurship. entrepreneurship education research has experienced an extraordinary increase recently due to its practical nature. this means it is challenging to find a specific answer to deal with the problems. this article offers at least two contributions to the study of entrepreneurship education in the covid-19 crisis can be analysed in terms of entrepreneurship from before, during and after the event (sharma, leung, kingshott, davcik and cardinali, 2020) . this enables different time frames to be compared in order to understand the changes occurring in the global business environment. by taking a multi-time period approach it will enable an overview of how the covid-19 crisis has impacted stakeholders in terms of demand, supply and policy issues (toresdahl and asif, 2020) . by doing so entrepreneurship education research can be transformative as it can embed a covid-19 perspective into new research studies. this enables a significant unravelling of the way entrepreneurship educators have responded to the crisis and how relationships have changed. based on the discussion included in this article, there are a number of policy implications that have emerged. this involves the need for government investment on entrepreneurship education programs in times of crisis (shrivastava, 1993) the shutting down of many industries most notably the sport and tourism sectors provides an opportunity to rethink current strategies (veil, 2011) . to do this, entrepreneurship is required to reset business approaches that focus on a community-centred initiative. this will enable a reorganisation of entrepreneurial practices to build on local communities and people (weick and sutcliffe, 2011) . this helps entrepreneurship to be used as a way to focus on social good. in addition, the spread of covid-19 around the world indicates how global and networked our society has become. covid-19 is more contagious than the seasonal flu and has a higher death rate so the impact of the virus will have lasting effects particularly in terms of human interaction (wen, wang, kozak, liy and hou, 2020) . governments have responded to the virus by implementing more hygiene practices and social distancing procedures. this is in conjunction with restricting individual movement through border closures and city lockdowns. in addition, other activities such as sport and socialising have been restricted. the reduction in close contact activities is a radical change to previous behaviour that emphasises close interaction. covid-19 likely came from an initial zoonotic (animal to human) transmission event. entrepreneurship research needs to assess the impact of the covid-19 using financial and non-financial metrics. financial implications might be more readily available and easier to understand. global economic data shows there has been a decrease in share markets resulting from the coivd-19 crisis. non-financial implications are harder to understand due to the perceived effects often being subjective. more research is needed on how economies can bounce back from the covid-19 crisis and how policy makers can help. the answers to this is not simple and require care in terms of managing the associated costs. resolving the covid-19 crisis immediately is not possible as it will take time to handle. this means restoring social and economic stability is a priority. policy makers need to balance short term health and social objectives with long term economic considerations. in the short term, governments need to support entrepreneurship that can bring increases in economic growth. to do this requires entities in the entrepreneurial ecosystem working together. this will enable more positive flow on effects in the economy to emerge. in the long term, entrepreneurship education can enable better societal conditions and help prevent or minimise future crisis. this means entrepreneurship educators need to work together with other stakeholders in the ecosystem in order to lessen the effect of the crisis. as there is no magic billet to solve the covid-19 crisis as it has been an unexpected and highly impactful event, complex trade-offs need to be considered that incorporate creative thinking endemic in entrepreneurship education. entrepreneurs have experienced the uncertainty caused by the covid-19 crisis through personal or shared experiences of others. increasingly user-generated content about the covid-19 crisis has provided exposure to other's experiences that is magnified in social media settings. moreover, the emotional contagion of the crisis has meant entrepreneurs intentions and behaviours have been affected. business environments have become unstable as a result of the covid-19 crisis and an entrepreneurial approach is the best way to go for the sustainability of the global economy. the past reliance of businesses on government subsidies does not guarantee survival as the government is having to spend money on health and social needs. in the context of the covid-19 crisis, entrepreneurship education remains very relevant. an ability to navigate the business environment in uncertain times is intrinsic to having a successful business. inherent in conducting business in the crisis is the need to reconcile the urgency of delivering services with the need to move forward. the availability of government support is time limited so business need to be proactive about finding alternative support. in addition, countries (eg australia) that have recently suffered devastating natural disasters (eg bushfires) have had to struggle on multiple fronts. educators and practitioners can derive valuable suggestions from this paper. now more than ever entrepreneurship education is needed. this means making entrepreneurship education programs accessible to more people that can help alleviate some current societal problems. due to there being many different ways to teach entrepreneurship, the role of crisis management needs to be embedded in all courses. this means being aware of the underlying reasons for the course and the resulting impact on entrepreneurial intention. the covid-19 crisis will make firms rethink their current business models in order to make them more agile and flexible in the future. due to the closing of state and country borders, locations will become more nationalistic. this is in contrast to the internationalisation trends of the past. most businesses are struggling due to the impacts of covid-19 although some businesses particularly online ones are flourishing so encouraging entrepreneurship training is needed. this conceptual article has discussed the impact of covid-19 on entrepreneurship education. due to the recent and ongoing nature of the covid-19 crisis at the time of writing this article, longitudinal data could not be collected. thus, more information about how entrepreneurship educators have adjusted to the new environmental conditions are required. this includes focusing more on how technological innovation has been utilised both by educators and students. to do this it would be useful to conduct international cross-country comparisons to see the differences and similarities with responses to covid-19 related entrepreneurship education changes. the current thinking in the field of entrepreneurship education needs to include a covid-19 lens in order to understand the change. this is due to context-wise, the covid-19 pandemic has had a lot of influence on entrepreneurship education so this needs to be acknowledged. in addition, the role educators play in crisis j o u r n a l p r e -p r o o f settings needs to be stressed and this can be conducted in real time or via proxy in order to help uncover the contributions. most existing studies are concerned with crisis planning and response without considering the intricacies of the experience (runyan, 2006) . therefore, research is required on how entrepreneurship educators learn from crisis and their knowledge management techniques. studies on covid-19 and entrepreneurship need in turn to incorporate an education perspective in order to be more definitive about their contribution. in addition, more research on different time periods during the covid-19 crisis from the initial world health organisation declaring it a pandemic in march 2020 to the second wave of infections occurring in melbourne, australia in august 2020. comparing how students and educators adjusted to these different time periods would be useful in terms of analysing the effect of the crisis on learning levels. it would be interesting to see if because of the stay at home restrictions students study more or alternatively if they are too stressed by the pandemic to learn in an efficient manner. table 1 below states potential research issues in each of the time periods of the covid-19 crisis. to what extend did entrepreneurship education move back to where it was prior to the pandemic? how different is existing entrepreneurship education compared to prior to the crisis? in conclusion, this article has discussed the changing nature of entrepreneurship education as a result of the covid-19 pandemic. the way crises affect business and the resulting affects on management education were discussed. this highlights the need to think more strategically given the continued nature of covid-19 and its ongoing affects on entrepreneurs in society. as a result there is a greater need now more than ever to embed an entrepreneurial spirit in management education courses and standalone entrepreneurship subjects in order to help focus on the positive benefits of finding solutions for the crisis. the article has focused on the different affects of the crisis from a student learning, technological environment and course content perspective. this will help to build better learning and teaching methods in entrepreneurship education courses that can use crisis analogy as a way of finding productive solutions. j o u r n a l p r e -p r o o f entrepreneurship education programmes: how learning, inspiration and resources affect intentions for new venture creation in a developing economy the covid-19 crisis as a career shock: implications for careers and vocational behaviour regime type and covid-19 response taming deep uncertainty: the potential of pragmatist principles for understanding and improving strategic crisis management the covid-19virtual idea blitz: marshaling social entrepreneurship to rapidly respond to urgent grand challenges the covid-19 pandemic: what can we learn from past research in organisations and management? covid 19 and its impact on management research and education: threats, opportunities and a manifesto country-based comparison analysis using fsqca investigating entrepreneurial attitudes and activity covid-19 and management education: reflections on challenges, opportunities, and potential futures from coronavirus to coronacrisis: the value of an analytical and geographical understanding of crisis entrepreneurial uncertainty during the covid-19 crisis: mapping the temporal dynamics of entrepreneurial finance researching tomorrow's crisis: methodological innovations and wider implications does entrepreneurial self-efficacy distinguish entrepreneurs from managers predictors of covid-19 voluntary compliance behaviors: an international investigation are employability and entrepreneurial measures for higher education relevant? introducing agile reflection. industry and higher education the coronavirus crisis in b2b settings: crisis uniqueness and managerial implications based on social exchange theory. industrial marketing management entrepreneurship and crisis management: the experiences of small businesses during the london effects of covid-19 on business and research addressing academic researcher priorities through science and technology entrepreneurship education masters of disasters? challenges and opportunities for smes in times of crisis an entrepreneurship education and peer mentoring program for women in stem: mentors' experiences and perceptions of entrepreneurial self-efficacy and intent towards a framework for tourism disaster management international entrepreneurship education: issues and newness assessing the impact of entrepreneurship education programmes: a new methodology entrepreneurial universities entrepreneurship research: mapping intellectual structures and research trends the influence of entrepreneurship education on entrepreneurial intentions rural women entrepreneurship within co-operatives: training support. gender in management internship: conventional wisdom, models & recommendations. london the impact of entrepreneurship education on university students' entrepreneurial skills: a family embeddedness perspective pandemics, transformations and tourism: be careful what you wish for. tourism geographies the impact of covid-19 pandemic on corporate social responsibility and marketing philosophy the role of work experience in the future employability of higher education graduates socialising tourism for social and ecological justice after covid-19 seduced by recovery: the consequences of misunderstanding disaster digital transformation of everyday life-how covid-19 pandemic transformed the basic education of the young generation and why information management research should care? measuring the impact of business management student's attitudes towards entrepreneurship education on entrepreneurial intention: a case study. computers in human behaviour a signature pedagogy for entrepreneurship education let's get physical: stories of entrepreneurial activity from sports coaches/instructors emerging themes in entrepreneurial behaviours, identities and contexts entrepreneurship education: a recipe for change entrepreneurial identity and context: current trends and an agenda for future research on the about/for through framework of entrepreneurship education: a critical analysis i'll trade you diamonds for toilet paper: consumer reacting, coping and adapting behaviors in the covid-19 pandemic entrepreneurship among business graduates: does a major in entrepreneurship make a difference the economics of covid-19: initial empirical evidence on how family firms in five european countries cope with the corona crisis the future of business education: a commentary in the shadow of the covid-19 pandemic exploring situated ambiguity in students' entrepreneurial learning startups in times of crisis-a rapid response to the covid-19 pandemic from offline to online: challenges and opportunities for entrepreneurship education following the covid-19 pandemic. entrepreneurship education and pedagogy resilience in business and management research: a review of influential publications and a research agenda crisis management strategies and the long-term effects of product recalls on firm value higher education and public good covid-19: challenges and opportunities for educators and generation z learners is entrepreneurial education at south african universities successful? an empirical example. industry and higher education three types of perceived uncertainty about the environment: state, effect, and response uncertainty sport events at the community level: a pedagogical tool to improve skills for students and teachers the potential impact of entrepreneurship education on doctoral students within the noncommercial research environment in chile the socio-economic implications of the coronavirus pandemic (covid-19): a review competing during a pandemic? retailers' ups and downs during the covid-19 outbreak covid-19, networks and sport. managing sport and leisure enterprise education: influencing students' perceptions of entrepreneurship entrepreneurship education: a systematic review of the evidence entrepreneurial universities: the role of communities, people and places coronavirus disease (covid-19) and sport entrepreneurship future research directions for sport education: toward an entrepreneurial learning approach putting entrepreneurship education where the intention to act lies. an investigation into the impact of entrepreneurship education on entrepreneurial behaviour small business in the face of crisis: identifying barriers to recovery from a natural disaster entrepreneurship education in a poverty context: an empowerment perspective entrepreneurship education centres in universities: evidence and insights from italian "contamination lab" cases managing uncertainty during a global pandemic: an international business perspective crisis theory/practice: towards a sustainable future entrepreneurship educationparadoxes in school-company interaction do entrepreneurship programmes raise entrepreneurial intention of science and engineering students? the effect of learning, inspiration and resources a theoretical framework of alliance performance: the role of trust, social capital and knowledge development coronavirus disease 2019 (covid-19): considerations for the competitive athlete human capital and entrepreneurial success: a meta-analytical review the impact of entrepreneurship education on entrepreneurship skills and motivation mindful learning in crisis management the effects of entrepreneurship education managing the unexpected: resilient performance in an many brains are better than one: the importance of interdisciplinary studies on covid-19 in and beyond tourism. tourism recreation research organizational response to adversity: fusing crisis management and resilience research streams who, coronavirus disease (covid-19) outbreak reassessing the link between risk aversion and entrepreneurial intention: the mediating role of the determinants of planned behavior key: cord-347645-xj4p68ba authors: savage, nicholas; jain, anshini; ng, suat chin title: the impacts of the covid‐19 pandemic on medical students in australia date: 2020-06-14 journal: anz j surg doi: 10.1111/ans.16103 sha: doc_id: 347645 cord_uid: xj4p68ba nan covid-19 has been formally declared a pandemic by who on the 11 th of march 2020(1). the effects of this on the health systems cannot be underrated. in this unprecedented situation, perhaps one of the more underestimated repercussions will be on medical students. there are several challenges common to all medical students such as the uncertainty in course requirements, the fear of exposure to covid-19, the lack of guidance and the loss of learning opportunities. nonetheless, it has also affected medical students of different levels indistinctly. preclinical medical students are accessing e-learning solely while socially isolating themselves. e-learning certainly has its merits but lacks the engaging qualities of in-person learning. the study of medicine is complex, requiring a multimodal approach with specialists providing clinical experience and guiding students out of the maze of endless archaic textbooks. adjuncts to classroom teaching such as anatomy dissections, microbiology and histopathology labarotory work as well as the development of interpersonal skills in clinical interactions are paramount to these years. furthermore, e-learning may present new challenges including technical issues, adaptability-struggle and computer literacy. it may be unaccessible by some, without access to on-campus wifi. to overcome this, certain universities have provided usb modems to students to increase accessibility to internet facilities (2) . other universities have altered term outlines and exam dates(3). these amendments in addition to technical issues that arise during exams may cause unneccesary added stress. there have also been concerns that students may be disadvantaged when exams are uninvigilated, with co-students cheating. during such times, we have little choice but to trust that students will maintain the highest standards of academic integrity. students in clinical years are experiencing different challenges. disruption to normal teaching and examination has left many students feeling uncertain and disoriented. the structure and length of clinical placement blocks are modified, along with semester breaks. traditional examinations en masse are not possible. practical assessments such as osce's require alteration given the transmission risk associated with close contact of simulated patients and examiners. most students in this group have been removed from clinical placement, leaving them without any practical teaching to rely upon. final year medical students face a more diverse spectrum of outcomes, albeit from a much stronger vantage point to both comprehend pandemic management and contribute to it directly. their core study comprises almost exclusively of clinical placement with practical teaching in preparation for internship. for these students, changes to staffing and services are challenging. in victoria, where a merit-based system is normally used for internship positions, the pmcv has now announced that references will no longer be considered due to the already significant workload on consultants(4). some final year students have been allowed to continue clinical placement but are left to determine what risk they are willing to accept in order to graduate. most clinical specialties have reduced clinic attendance in favour of telehealth, resulting in less face-to-face time with patients. attendence of students are often discouraged or limited, e.g. during ward rounds. the reduction in elective surgery has also affected learning opportunities. with the presence of students in hospital, consideration must be given to the increased risk of cross-contamination among medical teams. education on the correct use of personal protective equipement is also vital. several countries have accelerated final year medical students to the level of intern during the pandemic (5, 6) . whether this occurs in australia remains to be seen. they may have the option to work in covid-19 or non-covid-19 facing roles, be it voluntary or via formal employment. this needs to be very carefully assessed at all levels by the state government in consultation with medical schools, taking into account the students' competency and scope of practice, before being executed. there has been the suggestion that students can volunteer as clinical assistants, receiving telehealth accepted article calls, performing sample collection and packing test kits in fever clinics (7) . while this relieves workforce strain, it also places students in higher risk situations without compensation. students are in general younger, have less comorbidities and at a lower risk of becoming critically ill than the senior clinicians. it may therefore be logical to place them on the frontlines. whether students feel compelled to act in such a selfless manner for the public good is a personal decision that should be made voluntarily, with the provision of avenues for open disclosure of any safety concerns. at a global perspective, there were 2870 international students enrolled in australian medical schools in 2019 (8) . covid-19 has far reaching effects on this cohort, who have invested significant financial and personal sacrifice to undertake their degree. several are met with disappointment at the abridged version of schooling provided, despite ongoing full fee payment requirements (9) . some rely on part-time employment to fund their international living, which may also be affected with social distancing measures in place (10) . some have their education interrupted by international travel restrictions (9) . compounding these changes is the fear that if less students graduate in 2020, future years may enrol a larger cohort and as such, increase competition for desired internship positions, for which they are already ranked inferior to their domestic colleagues. the covid-19 pandemic presents a dynamic situation, making it difficult for faculty, health services and the government to plan and deliver new strategies for education and support of medical students. communication with students is now more vital than ever for wellbeing and continuing learning. additionally, the consideration and recognition that students often contribute to the health system without any direct compensation is important during these times of increased risk. undoubtedly, there are more important issues for the public good than medical students in this time of crisis. however, medical students are also being significantly affected and many are willing to contribute to the effort against covid-19. overall, the widespread ramifications of covid-19 are becoming more evident over time. the authors hope to highlight the effect on and potential contribution that the medical student cohort can offer in these unprecedented times. this article is protected by copyright. all rights reserved. get online with a usb modem 2020 covid-19 faqs for uq doctor of medicine students 2020 postgraduate medical council of victoria inc2020 umms, three boston medical schools to accelerate graduation of class of fresh out of medical school, young italian doctors are being fasttracked to the coronavirus frontline: cnn ready for the rmh recruitment campaign: the rmh student statistics tables medical deans australia and new zealand2020 australia's international students are anxious about experiencing covid-19 away from home sbs news2020 how the coronavirus pandemic could shift the multicultural make-up of our society abc news2020 disclosure statement: nicholas savage, anshini jain and suat chin ng declare that they have no conflict of interest. this article is protected by copyright. all rights reserved. this article is protected by copyright. all rights reserved. key: cord-285244-2suwmika authors: blau, ina; goldberg, shira; friedman, adi; eshet-alkalai, yoram title: violation of digital and analog academic integrity through the eyes of faculty members and students: do institutional role and technology change ethical perspectives? date: 2020-07-22 journal: j comput high educ doi: 10.1007/s12528-020-09260-0 sha: doc_id: 285244 cord_uid: 2suwmika this study aimed to address the gap in the literature through a comprehensive comparison of different types of violations of academic integrity (vai), cheating, plagiarism, fabrication and facilitation (pavela in j college univ law 24(1):1–22, 1997), conducted in analog versus digital settings, as well as students’ and faculty members’ perceptions regarding their severity. the study explored differences in perceptions regarding students’ vai and penalties for vai among 1482 students and 42 faculty members. furthermore, we explored the impact of socio-demographic characteristics (ethnic majority vs. minority students), gender, and academic degree on the perceived severity of vai. presented with a battery of scenarios, participants assessed the severity of penalties imposed by a university disciplinary committee. furthermore, participants selected the penalties they deemed appropriate for violations engaged in by students, including: reprimanding, financial, academic, and accessibility penalties. all participants tended to suggest more severe penalties for vai conducted in traditional analog environments than for the same offenses in digital settings. students perceived all four types of penalties imposed by the disciplinary committee to be significantly more severe than faculty members. moreover, findings demonstrated a significant difference between faculty and students in both perceptions of the severity of vai and in relation to suggested punishments. consistent with the self-concept maintenance model (mazar et al. in j mark res 45(6):633–644, 2008) and neutralizing effect (brimble, in: bretag (ed) handbook of academic integrity, springernature, singapore, pp 365–382, 2016), ethnic minority students estimated cheating, plagiarism, and facilitation violations as more severe than majority students. the theoretical and practical implications of the findings are discussed. violations of academic integrity (vai) have become increasingly prevalent among students in education systems across the world. vai involve one of the following four general categories of behavior (which will be discussed in the next section): cheating, plagiarism, fabrication of information, or facilitation of such misconduct (pavela 1997) . for instance, in a study of more than a thousand university students in romania (ives et al. 2017) , about 95% reported having engaged in one or more vai. this phenomenon continues to preoccupy researchers and educators in the hopes of reaching a greater understanding of its determinants and of finding effective methods of reducing its scope. findings reported by peled et al. (2012) indicate that 3/4 of 1500 faculty members from israel, germany and the us perceived vai as a problem at their colleges and universities. a vast majority of faculty members reported having considerable leeway in handling instances of vai involving students within their institutions. in addition, 3/4 of this very large sample of respondents perceived vai as a problem within their department or school. these findings are consistent with a shift in research on vai around the globe-from looking at student behavior and characteristics alone to focusing on the roles played by instructors, instruction style, academic practices (fishman 2016) , and evaluation methods (thomas and scott 2016) . although vai is an extensively researched phenomenon, there is a gap in the literature regarding the systematic comparison between vai conducted in traditional, analog settings (i.e., non-digital face-to-face or handwritten paper-based contexts) and vai conducted in digital settings. the recent covid-19 pandemic has forced faculty members to experiment with online assessment by integrating proctoring tools or/and changing the nature of the exams and requiring students to apply the study material. this highlights the importance of reaching a deeper understanding of digital vai and of raising faculty awareness in relation to these offenses. this study aims to address this gap by comparing analog versus digital vai in order to understand the role that digital environments play in this phenomenon. moreover, previous studies mostly focus on either the perspective of instructors or of students, rather than mapping the gap between the perspectives held by different stakeholders regarding the same phenomenon. this study explores the perceptions of both university students and faculty members from a large israeli university regarding the severity of different types of vai conducted by students and the penalties for these types of misconduct. the following sections first present the conceptual framework for the different vai explored in this study and the prevalence of vai. following that, we discuss how these misconducts may differ in analog and digital environments. we conclude the literature review by addressing strategies for preventing and coping with vai. this study adopted a comprehensive conceptual framework of vai, which was proposed by pavela (1997) . the framework distinguishes between four types of vai: (1) cheating-the intentional use of study materials, information or any kind of aid, the use of which is not allowed, including consulting others; (2) plagiarism-the use of text, images, figures, tables, and other types of content created by other people, presented without crediting the source, as if it were one's own; (3) fabrication-the intentional fabrication of information and data that do not actually exist; and (4) facilitation-intentional assistance in vai of any type conducted by others. this moadel was chosen since it is widely used and encompasses a wide range of vai in comparison with other models (for review of the categorizations of vai types see: . pavela's framework of different vai types was introduced before the massive integration of digital technologies for communication and information-sharing in teaching and learning practices. consequently, the model does not distinguish between offenses conducted in analog versus digital settings. findings among school students provide contemporary empirical support for the validity of pavela's conceptual framework in exploring the phenomenon of vai in both analog and digital learning environments. this study reported that all four types of vai, described by pavela, exist in the classroom and are conducted in both digital and non-digital settings. furthermore, no additional types of vai were revealed. similar results were obtained in a separate qualitative study in which teachers and students' parents were interviewed rotem et al. 2016) . in contrast to studies conducted in the education system, additional studies which examined vai in higher education through an analysis of the disciplinary committee's protocols at a large university revealed only three out of four types of vai (etgar et al. 2019; friedman et al. 2016a, b) . these protocols, which include all of the offenses examined by the disciplinary committee over either 1.5 or 4 years, contained no cases of fabrication in either analog or digital settings. such inconsistencies in previous findings call for further examination of vai types in higher education based on pavela's comprehensive conceptual framework. the researchers hypothesized that these surprising results can be explained by the chosen methodology-analysis of cases that were caught and punished by the disciplinary committee. namely, in contrast with other vai types (e.g., plagiarism), it can be difficult to identify and prove students' fabrication of data or arguments in their university work. some factorial analytic studies of vai have used different terminology but have provided support for pavela's categorization of vai. for instance, a study on vai conducted among turkish undergraduates (akbulut et al. 2008 ) revealed components termed fraudulence, plagiarism, falsification, delinquency, and unauthorized help. apart from delinquency, which is not necessary associated with vai, fraudulence is equivalent to cheating, falsification to fabrication, and unauthorized help to facilitation in pavela's terms. vai have been found to be prevalent in different countries, among students from different academic fields, and at different educational levels. for example, a study of 1500 undergraduates from a variety of american universities and colleges (carpenter et al. 2010) indicated that 80% of the participants had conducted at least one of the vai cases that were presented to them. an additional study focused on the academic misconduct of plagiarism (teixeira and rocha 2008) and found that among approximately seven thousand economics and business students from a variety of universities in 21 different countries across the globe, the majority admitted having copied at least once. while this study examined vai among business students, a study conducted in croatia examined vai among almost 700 medical students and found that 97% admitted having engaged in some type of vai (kukolja taradi et al. 2012) . furthermore, while the studies described above found that vai was prevalent in higher education, research has indicated that it is present in high school as well. for example, a study conducted on cheating in high schools in advantaged areas of the west coast in the united states, found that more than 90% of the students had engaged in vai at least once (galloway 2012) . studies on vai in the education system in israel reveal a similar trend to that reported worldwide. for example, cheshin (2006) found that approximately 95% of higher education students in israel admitted having been involved in some type of vai, the majority of whom had been involved in copying papers and/or cheating on exams. regarding the reasons for engaging in vai, a study conducted at a large israeli university indicated (friedman et al. 2016a ) that almost 60% of students who were charged with vai claimed that they had acted innocently, in the belief that their behavior was acceptable. moreover, a study among israeli school students revealed that the pervasiveness of vai was significantly higher than the perception of its legitimacy. this suggests that even when academic offenses are perceived as unacceptable, it does not necessarily prevent students from actually engaging in such offenses. if this issue is not addressed, students are likely to suffer from "ethical dissonance" (etgar et al. 2019; sidi et al. 2019) , i.e., the gap between ethical standards and actual behavior, while continuing to behave unethically. by the end of the twentieth century, the internet had increased access to information, and enabled its dissemination to an unlimited number of learners, who could edit and share it using a variety of tools (nilsson 2016) . however, in addition to its benefits, the availability of online information has contributed to students copying from the internet or peer-to-peer sharing as an easy way of completing their academic studies successfully (rogerson and basanta 2016; sutherland-smith 2016) . moreover, digital plagiarism is further encouraged by the availability of websites offering help in preparing study assignments, as well as the option of "contract cheating" and buying academic papers online (newton and lang 2016; o'connor 2003) . thus, alongside the great advantages of digital technologies in promoting learning processes, they also increase the prevalence of digital vai (etgar et al. 2019 ) because of the great accessibility of information and the almost unlimited capacity to edit and disseminate it (stephens et al. 2007) . the characteristics of the digital technologies described above, as well as the anonymity of authors' identities (e.g., in wikipedia) in many cases, often lead to blurred ethical boundaries in students' perceptions of acceptable academic behavior. blurred ethical boundaries may lead students to engage in vai in order to attain high achievements with minimal effort (alroi-stein 2008; newton and lang 2016). indeed, a high percentage of teachers and students have been found to perceive copying information from the internet as an acceptable behavior, rather than a punishable offense (baratz and reinhold 2008) . furthermore, findings suggest that digital vai may be perceived as more acceptable than analog vai. for example, eshet-alkalai (2014, 2017) found that plagiarism and facilitation are perceived by middle school students as more acceptable behaviors when the vai are engaged in using digital tools, compared to analog tools. thus, the use of technology may pose ethical challenges that students have difficulty coping with. indeed, lathrop and foss (2000) found that the more advanced the technology is in terms of the facility to copy, edit and disseminate material, the lower the ethical standards of students. other studies distinguish between the main effect of technology on vai on the one hand, which they usually fail to demonstrate (e.g., friedman et al. 2016a) , and the interaction effect between the technological factor and vai type on the other hand (sidi et al. 2019) . for example, eshet-alkalai (2014, 2017 ) expanded pavela's (1997) conceptual model described above to examine vai among school students with and without the use of digital tools. they found a significant interaction effect between the technology factor and violation type. namely, digital plagiarism was more prevalent than analog plagiarism, whereas analog cheating and fabrication were more prevalent than digital cheating and fabrication. a study by cheshin (2006) found that digital plagiarism was the most common form of vai among students in higher education settings. in a study that compared integrity in analog versus digital academic environments according to the protocols of a university's disciplinary committee (friedman et al. 2016a) , cheating was found to be more prevalently conducted in analog settings, whereas plagiarism was more prevalently conducted digitally. nonetheless, the vast majority of the offenses, both analog and digital, which were sentenced by the disciplinary committee, involved cheating (78%), whereas only 17.5% involved plagiarism. thus, findings from the study conducted by friedman and colleagues indicated that digital cheating was more prevalent than digital plagiarism. inconsistent findings reported in previous studies call for use of an appropriate methodology that would enable researchers to explore the interaction effect between the use of technology and the different types of vai, based on pavela's framework. in order to prevent and successfully cope with vai, research emphasizes the importance of the following characteristics: (1) clear institutional policy and punishment, (2) improvement of teaching and evaluation practices, and (3) strengthening of ethical standards among students and faculty (gilmore et al. 2016) . enforcement and punishment is obtained by formulating strict rules relating to what is permitted and what is forbidden, enforcing these rules, and punishing those who break the rules (for a review, see: murdock and anderman 2006) . according to this approach, when the chance of being caught for vai is high and when students are aware of the seriousness of the punishment for vai, they are less likely to choose to engage in vai (graham et al. 1994) . thus, students' perceptions of the severity of penalties for vai may influence their likelihood of engaging in vai (brimble 2016) . other studies highlight the importance of strengthening students' ethical beliefs and of teaching ethics in higher education (e.g., christensen hughes and bertram gallant 2016) . consistent with this claim, blau and eshet-alkalai (2016) found that school students' ethical beliefs explained an additional 13% of the variance in digital vai, after controlling for a variety of sociodemographic variables. moreover, ethical beliefs mediated the effect of intrinsic motivation and ethnicity on vai. namely, adding students' ethical beliefs to the analysis neutralized the advantage of students with high intrinsic motivation, who otherwise engaged in less vai than students with low levels of motivation. similarly, adding ethical beliefs to the regression analysis mediated the disadvantage of ethnic minority students who otherwise engaged in more academic offenses compared to ethnic majority students. other authors (e.g., harkins and kubik 2010) suggest that it may be important to revise standards of what is considered ethical and unethical. namely, harkins and kubik argued that in the context of learning in modern digital environments, current notions of vai are outdated, and introduced the term "ethical cheating". ethical cheating acknowledges learning practices related to the exchange of information and ideas, as well as practices of helping peers in the context of collaborative learning, participation in online learning communities, and the use of open source information to construct knowledge as acceptable behaviors and an integral part of the learning process. acknowledging acceptable learning practices and differentiating them from vai is especially important in light of the literature review suggesting that some vai are perceived by students as "not cheating" and therefore, when students engage in these offenses, they do not think that they are doing anything 'wrong' (de lambert et al. 2016) . the authors emphasize that students should not be left with room to make legitimacy assumptions in novel areas such as the use of new technologies. the pedagogical approach for dealing with vai focuses on the important role that instructors, teaching style, and evaluation methods play, not only in treating, but also in preventing academic offenses from occurring (for review see: brimble 2016; davies and howard 2016; de lambert et al. 2016 ). this may include teaching students referencing skills and discussing expected academic standards (e.g., summarizing in one's own words; davies and howard 2016; riedman et al. 2016a) . equally important are formative or/and summative assessment strategies, such as creating ongoing tasks and assignments that are meaningful and relevant for students , and even designing the criteria for grading assignments together with students (thomas and scott 2016) . in addition to these methods of preventing academic offenses, some researchers (bertram gallant 2017) also suggest utilizing the moment the offense is caught as a teaching opportunity (e.g., for explaining appropriate citation rules). similarly important in the prevention of vai is the degree of "fairness" perceived by students in terms of the quality of teaching, availability of the instructor, content difficulty level, and the amount of time required for reading and preparing learning assignments (brimble and stevenson-clarke 2006) . respondents of several studies (for review see: de lambert et al. 2016) indicated that their main reasons for engaging in vai were fear of failing and the course workload being too high. studies revealed a range of variables, which may be associated with vai, including neutralizing effects, institutional identification, and sociodemographic variables (for review see brimble 2016) . in relation to perceptions of fairness, brimble discusses a neutralizing effect in which, although students believe that their behavior was wrong, they deny it or/and blame external factors or people for it. such external factors may reflect a cost-benefit analysis of financial investment in education, "fair and equal opportunity to succeed" (owunwanne et al. 2010) , or being an international student and thus, not native in the language in which teaching, learning and evaluation are conducted (brimble 2016) . interestingly, from an institutional perspective, regardless of academic success, students who did not have a strong sense of identification with their institution had a stronger tendency towards vai than those students who had a strong affiliation (de lambert et al. 2016 ). thus, low levels of identification with the academic institution may have a neutralizing effect on students' integrity. the neutralizing effect is consistent with the self-concept maintenance model (mazar et al. 2008 ). this model, which was suggested within the approach of behavioral ethics and was not aimed at explaining violations of academic integrity specifically (but rather violations of integrity in general), can also successfully explain vai in educational settings (e.g., friedman et al. 2016a) . the model argues that the key psychological factor which enables individuals to engage in vai is the ability to maintain one's self-image as an honest person, despite vai. although the neutralizing effect does not impact the decision to engage in academic offenses, students use external factors after engaging in the offenses for the same reason suggested by mazar and colleagues-to protect and maintain the self-image of an honest person. various sociodemographic variables may produce neutralizing effects in relation to the vai phenomenon, including gender, ethnic origin, and seniority in studies. regarding gender, previous research has shown that regardless of age, vai was more prevalent and perceived as more acceptable by male school students compared to female students. on the other hand, friedman et al. (2016a) reported the disciplinary committee imposing significantly more severe penalties on women than men and this finding was recently replicated by etgar et al. (2019) . this gender gap could not be explained by the different vai behavior of female students, the reason for engaging in unethical behavior as explained by students, or their unwillingness to cooperate after being caught. since the disciplinary committee protocols were publicly available, female students could potentially become aware of such punishing inequity and it might trigger the neutralizing effect in further occurrences of vai. other research (e.g., ives et al. 2017) failed to demonstrate gender differences in vai. regarding the students' origin, a number of studies have supported the neutralizing effect. for example, in a study conducted at the university of minnesota, 85% of all reported cases of vai were conducted by non-native english speakers (marshall and garry 2006) . similarly, ethnic minority students studying in arabic-speaking schools reported significantly higher levels of vai in classrooms compared to ethnic majority students in hebrew-speaking schools . moreover, kremmer et al. (2007) found that compared to domestic students, international students were more likely to cheat on exams, but were less likely to self-report cheating. non-native speakers or ethnic minority students may feel under greater pressure to compete, and consequently be tempted to a greater extent than others to "make the grade" (gilmore et al. 2016) . regarding students' age, in contrast with pupils who engage in more vai in highschool compared to elementary and middle-school (sidi et al. 2019) , in higher education the variables such as academic degree and year in college were not associated with excuses related to vai behaviors, such as lacking academic experience and enculturation into one's discipline. namely, a previous study (sheard et al. 2003) failed to detect differences in the academic offenses of undergraduate versus graduate students and of junior versus senior college students. as mentioned above, there is a gap in the literature in relation to the comprehensive comparison of different vai types conducted in analog versus digital settings, in order to distinguish between the main effect of technology in this phenomenon and its interaction with offense type sidi et al. 2019 ). in addition, previous studies mostly focus on either the perspective of instructors or of students, and there is a shortage of comparisons between the perspectives of different stakeholders, such as students and faculty members, in relation to the same phenomenon-regarding different types of vai and the penalties for these types of misconduct (e.g., . moreover, previous studies (gilmore et al. 2016; kremmer et al. 2007; sheard et al. 2003 ) did not systematically explore whether sociodemographic characteristics of students, such as gender, ethnic origin, and seniority in studies, may produce neutralizing effects (mazar et al. 2008 ) in relation to the vai phenomenon. this study compares perceptions of students and faculty members regarding different types of vai engaged in by students. this was done whilst comparing digital and analog vai, as well as the perceived severity of penalties imposed for engaging in different types of analog and digital vai as defined by pavela's (1997) comprehensive conceptual framework. furthermore, this study examines whether such perceptions vary according to students' sociodemographic characteristics such as ethnicity, gender and academic degree, in order to reveal additional variables, which may influence perceptions of vai. the between-subjects factor was the study group (faculty members, hebrew-speaking students, and arabic-speaking students) and the within-subjects factor was digital versus analog vai. the study explored the following research questions: 1. do students and faculty members have different perceptions regarding the severity of penalties imposed by the university disciplinary committee? 2. among students, are there differences in perceptions regarding the severity of penalties imposed by the university disciplinary committee according to sociodemographic variables (ethnicity, gender, and degree)? 3. do students and faculty members have different perceptions regarding the severity of different analog and digital vai engaged in by students (cheating, plagiarism, fabrication, and facilitation) and suggested penalties for these types of violations? 4. among students, are there differences in perceptions regarding the severity of different analog and digital vai engaged in by students (cheating, plagiarism, fabrication, and facilitation) and suggested penalties for these types of violations according to sociodemographic variables (ethnicity, gender, and degree)? to answer the research questions, this study employed an online survey method. in addition to the advantages of survey in general, such as high capability in representing a large population and no observer subjectivity, online data gathering makes it possible to ensure the participants' anonymity, which is important for such a sensitive topic as vai. to deal with disadvantages of surveys in general and online surveys in particular, which are not ideal for controversial issues and eliminate the inappropriateness of questions, we piloted the instruments in face-to-face, nonself-selected sample and compared the patterns with the main online self-selected sample. the main sample consisted of 1482 students studying in a large israeli university, 1300 (87.7%) of whom were a hebrew-speaking ethnic majority and 182 (12.3%) of whom were an arabic-speaking ethnic minority. the sample was gender-balanced: 704 (49.3%) of the participants were male and 724 (50.7%) were female. the participants' ages ranged between 13 and 90, which represents the age range of the student population at the university, with a mean age of 32.6 (sd 10.8). in terms of students' field of study, 215 (14.5%) of the students were studying humanities, 722 (48.7%) social sciences, 158 (10.7%) life sciences, and 308 (20.8%) other subjects. since we could not find either reports in the literature or systematic differences in our data regarding the dependent variables as a function of students' field of study, we do not address this factor as an additional independent variable. in relation to the students' degree, 1147 (77.4%) of the participants were undergraduate students and 159 (10.7%) were graduate students. although the university in which the study was conducted is a distance education institution, most of the courses are blended, rather than completely online and the vast majority of the learning materials are still printed. while most of the assessment and seminar work are digitally prepared and submitted, almost all of the final exams are conducted in an analog environment. that is to say, while much of the learning and course work are conducted online, exams are attended at university centers across the country and hand-written. in addition, the sample included a comparison group of 42 faculty members from the same university. among them 26 (61.9%) were male and 16 (38.1%) were female. the faculty members' age ranged between 35 and 68, with an average of 50.69 (sd 8.68). faculty members taught in the fields of humanities, social sciences, life and natural sciences, and exact sciences (including computer science). 1. to answer research questions 1 and 2, the following measure was used to assess the perceived severity of penalties imposed by the university disciplinary committee. students and lecturers were asked to rank the severity of actual penalties taken from the committee regulations and imposed by the university disciplinary committee among students who conduct vai. the instrument (see appendix) was piloted in a previous study . each punishment was rated on a scale from 1 'a very lenient punishment' to 6 'a very severe punishment'. penalties (see appendix) were categorized into four types: (1) reprimanding for behavior-1 item (q1); (2) financial penalties-4 items, α = 0.67 (q2-q5); (3) academic penalties-7 items, α = 0.81 (q6-q11, q16); and (4) accessibility penalties-4 items, α = 0.84 (q12-q15). the categories were based on the face validity of the items revised by four experts in the field. mean severity scores were calculated for each category. 2. to answer research questions 3 and 4, the following measure was used to assess the perceived severity of violations of academic integrity by students-students and lecturers were presented with a series of scenarios of students' vai (adapted from blau and eshet-alkalai 2017). the scenarios were based on descriptions of actual incidents of vai from the disciplinary committee protocols and corresponded with the 4 types of vai depicted in pavela's conceptual model (1997) . a total of 24 incidents of vai were presented, representing each of the 4 types of vai, in analog and digital settings, for each of the 3 types of assessment in the university (exam, assignment and paper). an example of an analog violation is copying a seminar paper from an example paper found in the library. an example of a digital violation is copying a seminar paper from an example paper found on the course website. the students and lecturers were asked to rank the severity of the violation in each scenario on a scale from 1 'a very minor violation' to 6 'a very severe violation'. mean severity scores were calculated for scenarios representing each of the following types of vai: analog cheating, digital cheating, analog plagiarism, digital plagiarism, analog fabrication, digital fabrication, analog facilitation, and digital facilitation. 3. to answer research questions 3 and 4, the following measure was used to assess suggested penalties for vai. for each of the scenarios described above, students and lecturers were asked to choose an appropriate punishment taken from the committee regulations and described above, ranging from 0 'acquittal' to 16 'permanent expulsion', the most severe punishment (friedman et al. 2016a ). the suggested penalties, ranging in score from 0 to 16, in accordance with their increasing severity, can be found in "appendix". mean punishment scores were calculated for scenarios representing each of the following types of vai: analog cheating, digital cheating, analog plagiarism, digital plagiarism, analog fabrication, digital fabrication, analog facilitation, and digital facilitation. 4. to answer research questions 2 & 3, participants were asked to report on a number of sociodemographic variables, including ethnicity (hebrew-speaking/arabicspeaking/other), gender (male/female), and degree (undergraduate/graduate). the institutional ethics committee approved the study. the questionnaire was piloted among a sample of 70 undergraduates in psychology and education, who anonymously participated in several research laboratory experiments as a requirement for their studies. students' and faculty members' participation in the main study was anonymous and voluntary. the organizational email, which contained a link to the online questionnaire, was sent by the researchers to all faculty members and students at the university. the academic institution is the largest university in israel and includes approximately 100 senior faculty members, 330 teaching faculty members, and 50,000 students. the data was collected through the institutional survey system. the survey was closed after 1 week, with almost 1500 responses collected. statistical analyses were conducted using spss 22. in order to address research questions 1 and 2, multiple analyses of variance (manova) were conducted in order to examine differences between students of different ethnicity (arabic-speaking compared to hebrew-speaking students) and faculty members in terms of perceptions regarding the severity of penalties imposed by the university disciplinary committee (table 1) . lsd post hoc tests were performed in order to identify the source of significant differences between the study groups. as table 1 shows, findings related to research question 1 (comparing students and faculty members) indicated that students perceived all four types of penalties imposed by the university disciplinary committee to be significantly more severe than faculty members. furthermore, in relation to research question 2 concerning ethnicity differences between students, arabic-speaking minority students were found to perceive being reprimanded as significantly more severe a punishment than hebrew-speaking students. no additional significant differences were found between hebrew-speaking and arabic-speaking students. in accordance with research question 2 (comparing students on sociodemographic variables), additional manovas were conducted comparing male and female students and comparing undergraduate and graduate students in terms of perceptions regarding the severity of penalties imposed by the university disciplinary committee ( table 2) . the findings presented in table 2 indicated that female students perceived being reprimanded, financial penalties, and academic penalties to be significantly more severe compared to male students. no significant difference was found between the groups in relation to the perceived severity of accessibility penalties. in terms of academic degree, findings indicated that the only significant difference between undergraduate and graduate students was in the perceived severity of financial penalties. namely, undergraduate students perceived financial penalties to be more severe compared to graduate students. after reporting the severity of penalties imposed by the disciplinary committee, students of different ethnicity (arabic-speaking vs. hebrew-speaking) and faculty members were asked to judge the severity of different types of vai and to suggest appropriate punishment for such behaviors. to examine research questions 3 and 4 exploring differences in perceptions between the study groups (faculty and students, and students of different ethnic groups) regarding the severity of each type of vai (cheating, plagiarism, fabrication, and facilitation) and suggested penalties for these behaviors, repeated measures anovas were conducted. the between-subjects factor was the study group (faculty members, hebrew-speaking students, and arabicspeaking students) and the within-subjects factor was digital versus analog vai. lsd post hoc tests were performed in order to identify the source of significant differences between the study groups. table 3 presents the findings of these analyses. in relation to research question 3 (comparing students and faculty members), a significant main effect was found for the study group. namely, lsd post hoc tests indicated that faculty members perceived cheating, plagiarism, and fabrication to be significantly more severe compared to students. similarly, in relation to research question 4 regarding students' ethnicity, faculty members perceived facilitation to be more severe than hebrew-speaking students. however, no significant difference was found in perceptions regarding the severity of engaging in facilitation between arabic-speaking students and faculty members. furthermore, as presented above table 2 comparison between students' perceptions regarding the severity of penalties imposed by the disciplinary committee according to gender, and degree in table 3 , significant differences were found between hebrew-and arabic-speaking students' perceptions of the severity of cheating, plagiarism, and facilitation. namely, arabic-speaking students perceived cheating, plagiarism and facilitation to be more severe than hebrew-speaking students. the use of technology was found to have a significant within-subjects effect on the perception of the severity of fabrication and facilitation. namely, while digital fabrication was perceived to be significantly more severe compared to analog fabrication, digital facilitation was perceived to be significantly less severe compared to analog facilitation. no significant interaction was found between the study group and the use of technology in vai in perceptions regarding the severity of vai. similarly, repeated measures anovas in table 4 show that the between-subjects factor, study group, had a significant main effect on suggested penalties for vai. in relation to research question 3 (comparing students and faculty members), and in accordance with the previously presented finding regarding more severe perceptions of vai among faculty members compared to students, findings in table 4 indicate that faculty members suggested significantly more severe penalties for cheating, plagiarism, fabrication, and facilitation compared to students. in relation to research question 4 regarding ethnicity, no significant differences were found between the suggested penalties for the different types of vai suggested by arabic-and hebrewspeaking students, with the exception of plagiarism. namely, arabic speaking students suggested more severe penalties for plagiarism in comparison with hebrewspeaking students. in addition, table 4 shows that the within-subjects factor, use of technology in vai, was found to have a significant main effect on the suggested punishment for vai. interestingly, the penalties suggested for analog cheating, fabrication, and facilitation were significantly more severe compared to those suggested for digital cheating, fabrication, and facilitation. no significant interaction was found between the study group and use of technology in vai in relation to suggested penalties for such behaviors. in relation to research question 4 regarding gender differences among students and differences between undergraduate and graduate students, additional mano-vas were conducted comparing male and female students and comparing undergraduate and graduate students in terms of their perceptions regarding the severity and suggested penalties for digital and analog vai (cheating, plagiarism, fabrication, and facilitation). as can be seen in table 5 , significant gender differences were found regarding the severity of plagiarism and fabrication. namely, female students perceived plagiarism and fabrication, both analog and digital, to be significantly more severe violations than male students. furthermore, significant differences were found between undergraduate and graduate students' perceptions of vai in relation to digital fabrication and analog and digital facilitation. whereas digital fabrication was considered to be more severe by graduate students, both analog and digital facilitation were considered to be more severe by undergraduate students. although the findings above showed no significant gender differences in students' perceptions of cheating and facilitation, table 6 shows that male students suggested table 4 between and within-subjects effects of comparing students of different ethnicity and faculty members' suggested penalties more severe penalties for analog cheating, whereas female students suggested more severe penalties for analog fabrication. in addition, significant difference was found between the penalties suggested by undergraduate and graduates for analog and digital facilitation. namely, undergraduate students suggested more severe penalties for both analog and digital facilitation compared to graduate students. this chapter first discusses the severity of penalties imposed by a disciplinary committee and addresses differences between students and faculty, as well as differences between students as a function of their socio-demographic variables. following that, the chapter discusses severity of academic offenses and penalties in analog versus digital settings, by comparing the perspectives of students and faculty in relation to this phenomenon and by exploring differences between students based on their socio-demographics. the first research question explored differences between students and faculty members concerning their perceptions of the severity of penalties imposed by a university disciplinary committee. findings show a clear difference between students and faculty members concerning their perceptions of the severity of penalties for vai. namely, students perceived all four types of penalties (reprimanding, financial, academic, and accessibility) to be significantly more severe than faculty members. the severe attitude towards vai by faculty members in this study is consistent with findings reported by peled et al. (2012) from a comparative large-scale study conducted on faculty members in the us, israel, and germany. in that study, approximately 75% of the faculty members perceived students' vai as a problem in their department, and were personally involved in handling such violations in their institution. among students, previous findings (brimble 2016; murdock and anderman 2006) indicate that students' perception of the severity of penalties for vai may influence their likelihood of engaging in such offenses. this seems to indicate the importance of continuing to impose penalties that are perceived to be appropriately severe by faculty members, despite students' perceptions of these penalties as being "too severe" in our study. the second research question explored differences between students regarding the perceived severity of penalties, according to socio-demographic variables (ethnicity, gender, and academic degree). apart from being reprimanded, which was perceived as significantly more severe by ethnic minority students, no additional differences were found between ethnic minority and majority students (table 1) . the difference in perceptions of the severity of being reprimanded, which was found between ethnic majority and minority students in higher education, is consistent with blau and eshet-alkalai's (2016) findings among school students from elementary and secondary schools, and could possibly reflect cultural diversity concerning reprimanding. interestingly, adding the variable "ethical beliefs" to the analysis conducted in a previous study mediated the disadvantage of ethnic minority students, who were otherwise found to engage in more vai than ethnic majority students. nonetheless, the findings regarding the particular ethnic minority examined in this study may not be generalizable to ethnic minorities in other countries. unlike the hispanic population in the us, the arabic-speaking minority students in this study are not immigrants who study the country's official language as a foreign language, but rather an israelborn ethnic minority, speaking the country's second official language (arabic), similar to the french-speaking population of canadian quebec. however, unlike the french-speaking population of canadian quebec, arabic-speakers in israel do not live in a discrete territory, but rather, are spread in geographically dispersed villages and towns. in contrast to arabic-speaking minorities in europe, the minority students in this study finished arabic-speaking schools, but have chosen to continue studying in a hebrew-teaching university. thus, their language-related issues are similar to immigrants and international students around the world and can provoke vai. concerning gender differences (table 2) , findings revealed that female students perceived being reprimanded, financial penalties, and academic penalties to be significantly more severe compared to male students. in contrast, no significant gender difference was found for the perceived severity of accessibility penalties. it seems that although male students have been found to engage in vai to a greater extent in higher education (for review see : brimble 2016; de lambert et al. 2016 ) and in schools , female students tend to feel more shame than males for behaving unethically (tibbetts 1999) . our finding apparently contradict recent reports on the actual penalties imposed by disciplinary committees (etgar et al. 2019) , suggesting that female students are punished more severely than males. this tendency was found to be stable over time and could not be explained by other variables, such as the kind of offense, the reason for engaging in it, or the willingness to take responsibility for such behavior. however, the explanation based on the shifting standard model (biernat 2012) suggested by the authors is actually consistent with our findings. according to this model, people from stereotyped groups (e.g., women) are being judged in comparison to their own group's standards, and not in relation to general standards. accordingly, women are expected to hold higher moral standards (reichel et al. 2010 ) and be more honest and loyal (kahn 2017) than men. consequently, their immoral behavior is perceived as more problematicby themselves in our research and by the disciplinary committee members in a previous study (etgar et al. 2019) . concerning the students' academic degree, the only significant difference between undergraduate and graduate students was found in relation to the perceived severity of financial penalties. as expected, undergraduates, who are often financially-dependent, perceived financial penalties as more severe than graduate students, who are on average more financially independent. the absence of differences in other comparisons related to academic degree is consistent with a previous study (sheard et al. 2003) , which did not detect differences in academic offenses between undergraduates and graduate students. the third research question explored differences between students' and faculty members' perceptions regarding the severity of different analog and digital vai (cheating, plagiarism, fabrication, and facilitation) engaged in by students, and the suggested penalties for these types of misconduct. the findings (table 3) indicated that faculty members perceived the offenses of cheating, plagiarism, and fabrication to be significantly more severe compared to students. in addition, facilitation was perceived to be more severe by majority students, but not by minority students. similarly, compared to students, faculty members suggested significantly more severe penalties for all types of vai (table 4 ). these findings indicate a significant gap in the perception of the same offenses' severity and penalty as a function of position in the academic institution-faculty versus students. these findings are consistent with an analysis of vai among students in a large israeli university as reflected in the protocols of the disciplinary committee (friedman et al. 2016a ). the analysis indicated that the majority of students who were tried for engaging in vai claimed that they had acted innocently, believing that what they had done was acceptable. regarding the effect of the use of technology, digital fabrication was found to be perceived significantly more severely compared to analog fabrication, while digital facilitation was perceived significantly less severely compared to analog facilitation. interestingly, the penalties suggested for almost all analog offenses-cheating, fabrication, and facilitation-were significantly more severe compared to those suggested for similar digital offenses. however, no significant interaction effect was found between the study groups and use of technology in relation to perceptions regarding the severity of vai or penalties for such behaviors. findings suggest that, regardless of their position in the university, both faculty members and students tend to penalize analog offenses more severely than digital ones, even when they perceived the digital offense itself as being more severe, as in the case of digital facilitation. a possible explanation for these findings is that faculty members might be not be sufficiently aware of the potential of digital technologies in promoting practices such as cut-and-paste and unauthorized peer-to-peer sharing (rogerson and basanta 2016; sutherland-smith 2016) or the availability of websites that sell academic papers and offer "contract cheating" (newton and lang 2016; o'connor 2003) . the university students' perceptions in our study are consistent with findings from school students, indicating that digital plagiarism and facilitation are perceived as being more acceptable than analog plagiarism and facilitation eshet-alkalai 2014, 2017) . in addition, there are indications (lathrop and foss 2000) that the more advanced the technology is in its facilitation of copying, editing and information dissemination, the lower the integrity standards of students and their ability to cope with temptation to act unethically. thus, it seems that the use of technology poses dual ethical challenges to academic institutions. on the one hand, academic institutions should raise faculty members' awareness regarding the role of technology in engagement in vai. on the other hand, academic institutions are faced with raising students' understanding that digital vai are equally unacceptable and will be penalized as severely as analog offenses. from a pedagogical perspective regarding the phenomenon of digital vai, consistent with the concept of "ethical cheating" (harkins and kubik 2010) , it is important to teach students acceptable ways of incorporating digital sources in their writing (de lambert et al. 2016) , as well as proper dissemination of information in collaborative digital learning environments and social networks. the last research question explored the differences in students' perception regarding the severity of different analog and digital vai (cheating, plagiarism, fabrication, and facilitation) engaged in by students, as well as suggested penalties for these types of misconduct according to sociodemographic variables (ethnicity, gender, and degree). regarding the role of ethnicity (table 3) , the findings revealed that minority students perceived cheating, plagiarism and facilitation offenses to be more severe compared to majority students. this finding is consistent with a previous comparison of majority and minority students' perceptions regarding different types of vai . however, no significant differences related to ethnicity were found between the penalties (table 4) for the different types of vai suggested by students. the only exception was plagiarism, for which the minority students suggested more severe penalties than the majority students. on the one hand, feeling more pressure to compete might lead ethnic minority students to be more tempted to conduct vai (gilmore et al. 2016) . indeed, those who are non-native speakers may tend to use more cut-and paste techniques instead of re-phrasing and original writing (brimble 2016; rotem et al. 2016 ). on the other hand, their perceptions of the vai phenomenon might reflect a neutralizing effect, in which students believe that a certain behavior is wrong and yet deny it or blame external factors for it. such external factors may represent the cost-benefit analyses of disadvantaged students for not having "fair and equal opportunity to succeed" (owunwanne et al. 2010) , or not being native in the language in which testing or learning are conducted (brimble 2016) . this explanation according to the "neutralizing effect" in the educational literature is consistent with the self-concept maintenance model (mazar et al. 2008) in the behavioral ethics literature. according to this model, people behave unethically up to the point at which they are able to perceive themselves as honest people. for instance, feeling disadvantaged may make unethical behavior seem more acceptable to minority students; at the same time, they may report more severe perceptions of vai in order to maintain positive self-image. it is important to examine these issues among ethnic minorities in other countries to understand whether our findings are generalizable to other contexts. concerning the effect of gender, our findings show that female students perceived cases of plagiarism and fabrication in both analog and digital settings to be significantly more severe than male students (table 5) . it seems that, similarly to the explanation of gender differences in vai reported in the literature (e.g., brimble 2016), our finding can be explained by tibbett's finding (1999) that females tend to feel more shame than males for unethical behavior. moreover, although no significant gender differences were found in students' perception of cheating and facilitation, male students suggested more severe penalties for analog cheating, whereas female students suggested more severe penalties for analog fabrication (table 6) . a possible explanation for this finding might be the neutralizing effect (brimble, 2016) , indicating that each gender tends to suggest more severe penalties for the offense in which s/he tends to be more involved. more research is needed to explore this possible explanation. regarding the impact of students' academic degree, digital fabrication was considered to be a more severe offense by graduate students compared to undergraduates. in contrast, both analog and digital facilitation were considered to be more severe by undergraduates compared to graduate students (table 5 ). consistent with their perception of the severity of these offenses, undergraduates suggested more severe penalties for both analog and digital facilitation compared to graduate students (table 6 ). this finding is different from a previous study (sheard et al. 2003 ), which did not detect differences in academic offenses between either undergraduates versus graduate students, or between junior versus senior college students. one possible explanation is a different demographic of the samples. while the current study examined students from different fields of study, sheard et al. (2003) focused specifically on it students. another possible explanation is that since almost a decade and a half has passed between both studies, the difference between the findings might reflect changes in the learning culture. our findings regarding the higher tolerance of graduate students in relation to facilitation seem to be consistent with the spreading culture of collaborative learning. this culture is particularly prevalent in smaller graduate courses, which nurture teamwork and discussions in learning communities shamir-inbal 2017, 2018) . as mentioned earlier, some authors even use the concept of "ethical cheating" (harkins and kubik 2010) to convey the message of acceptable peer help in digital learning communities. regarding fabrication, graduate students conduct more advanced research projects as a part of seminar courses, final projects or theses and thus, they are more aware and less tolerant towards fabrication of data and references in their writing. this study compared the perceptions of a large sample of faculty members, majority and minority students regarding the actual penalties imposed by the institutional disciplinary committee for a variety of types of vai. the offenses included correspond with the vai framework suggested by pavela (1997) , which was expanded to include digital learning environments. the study aimed to address the gap in the literature in the comprehensive comparison of different vai types conducted in analog versus digital settings. surprisingly, all participants suggested more severe penalties for vai conducted in traditional analog environments than for the same offenses conducted in digital settings. in addition, we compared the perspectives held by different stakeholders-students and faculty-on different types of vai and the penalties for these types of misconduct. students perceived all four types of penalties to be significantly more severe than faculty members and suggested more severe punishments for such vai. lastly, we aimed to close the existing gap in the literature by exploring whether the sociodemographic characteristics of students may produce neutralizing effects in relation to the vai phenomenon. consistent with the self-concept maintenance model (mazar et al. 2008) and neutralizing effect (brimble 2016) , ethnic minority students estimated cheating, plagiarism and facilitation violations as more severe compared to majority students. it should be taken into consideration that the study was conducted at one large distance university and might reflect a particular educational culture. future studies may include replications of our findings in different types of institutions, comparisons between campus and distance universities, or comparisons between the same institution types in countries with a different educational culture, e.g., more individualistic versus more collectivistic culture. moreover, replicating our results in a more generalizable group (e.g., immigrants, international students) would be an important research direction for future studies. finally, the current study's methodology did not enable a comparison between intentional and unintentional vai according to students' perspectives, which could be an important factor. future studies may analyze actual protocols of the disciplinary committee in order to separate and compare intentional and unintentional vai. moreover, although the differences found in this study were statistically significant, the effect sizes were low. these limitations emphasize the importance of conducting further studies to continue exploring the topics raised in this paper in additional academic institutions, using different methodologies. based on the findings, we recommend acknowledging acceptable learning practices related to collaboration, exchange of ideas and information, and clearly differentiating them from vai in order to change students' perception of vai as "not cheating" and acceptable behavior. moreover, we recommend that faculty members promote "student voice" in their courses, i.e., actively involving students in designing their learning practices and assessment methods (blau and shamir-inbal 2018) . this can serve as pedagogical prevention of vai. in addition, our findings showed a significant difference between faculty and students in both perceptions of the severity of vai and in relation to suggested penalties. we recommend conducting open discussions between students and faculty members in order to reach a consolidated perception of the phenomenon of vai. at the institutional level, it may be effective to have a clear policy regarding vai punishment and the faculty's role in this process, strengthening the ethical standards of students, and improving teaching and assessment practices in order to prevent vai from occurring (gilmore et al. 2016) . moreover, all participants tended to suggest more severe penalties for vai conducted in traditional analog environments than for the same offenses in digital settings. neither students nor faculty members should be left with room to make legitimacy assumptions in relatively novel areas, such as technology-enhanced learning and assessment. it is therefore important to design interventions for both students and faculty in order to change their approach to digital vai. we suggest conducting a workshop for faculty members in order to analyze scenarios of analog and digital academic offenses and explicitly clarify the expectation to deal with these situations with similar levels of severity. as for students, we suggest designing online tutorials with simulations of appropriate conduct in similar scenarios. lastly, it is important to cope with the neutralizing effect of minority students, in which although understanding that their behavior was wrong, they deny it or/and blame external factors (e.g., learning in a second language) or people/circumstances for it (e.g., questioning having a fair and equal opportunity to succeed). we recommend strengthening identification with the academic institution in order to prevent a neutralizing effect on students' integrity. to build a sense of allegiance and strengthen students' identification with an institution, it might be beneficial to promote student voice and involve students, especially minorities, in revising an institutional honor code to address issues related to technological advancements (e.g., using smart watches in exams, disseminating information via social networks). 13. limited use of student card 14. confiscation of student card 15. cancelling confirmation of student status 16. permanent expulsion below are a series of scenarios of students' violations of academic integrity that were tried by the academic disciplinary committee. please rate the severity of each violation of academic integrity engaged in by the student (on a scale from 1 'a very minor violation' to 6 'a very severe violation'), and choose an appropriate penalty, in your opinion, for each violation (ranging from 0 'acquittal' to 16 'permanent expulsion'-detailed above). [note: the scenarios of students behaviors below were randomized by the form platform. the penalties above, which were presented after each scenario to be chosen by the participants, were in the fixed order.] 1. the student fabricated statements and data on an assignment which were supposedly taken from a book s/he had read. 2. the student sent answers to another student through a text message (sms) during an exam. 3. the student fabricated statements and data on a seminar paper, which were supposedly taken from an academic article which s/he said s/he had found on the internet. 4. on a course assignment, the student included content from the internet without citing the source or the fact that the content had been taken from a different source. 5. when answering a question on an exam with open material, the student fabricated statements and data supposedly taken from articles on the course syllabus. 6. the student asked another student in the course to help him/her solve an assignment which student were clearly expected to be prepared independently. 7. during an exam, the student passed a note with an answer to another student. 8. on an assignment, the student fabricated and included data supposedly taken from an academic article which s/he said s/he found on the internet. 9. the student gave the assignment s/he prepared to another student in the course. 10. the student sent a request to an internet forum to ask for help solving an assignment which s/he was expected to prepare on his/her own. 11. the student sent his/her assignment by email to another student in the course. 12. the student asked another student for help writing a seminar paper which s/he expected to prepare independently. 13. the student fabricated statements and data on a seminar paper, which were supposedly taken from an article s/he read in the library. 14. the student glued course notes, which are not allowed to be used in an exam, into the course textbook which was allowed to be brought to the exam. 15. on a course assignment, the student included content from a book s/he borrowed from the library without citing the source it was taken from or mentioning the fact that the content had been taken from another source. 16. during an exam without open material, the student used his/her smartphone to search for answers on the internet. 17. during an exam with open material, the student included content from an article on the course syllabus without citing the source it was taken from or mentioning the fact that the content had been taken from another source. 18. during an exam with open material (in which use of supplementary material and searching for sources on the internet was permitted), the student fabricated statements and data which were supposedly taken from the central bureau of statistics website. 19. the student sent an email with the file of his/her seminar paper to another student writing a seminar paper on a similar topic. 20. on a seminar paper, the student included content taken from another student's academic work, which was accessible in the library, without citing the source it was taken from or the fact that the content had been taken from another source. 21. during an exam with open material (in which use of supplementary material and sources on the internet is permitted), the student included content taken from a digital textbook, without citing the source it was taken from or the fact that the content had been taken from another source. 22. the student sent a request to an internet forum to ask for help writing a seminar paper which s/he was supposed to prepare on his/her own. 23. the student gave his/her printed seminar paper to another student writing a seminar paper on a similar topic. 24. the student included content from an internet website on a seminar paper without citing the source it was taken from or mentioning the fact that the content had been taken from another source. a fine of up to 7000 nis repealing the grade achieved on the relevant assignment and requiring submission of another assignment in its place exploring the types and reasons of internet-triggered academic dishonesty among turkish undergraduate students: development of internet-triggered academic dishonesty scale (itads) contempt for academic values in the age of the conscious and media revolution. the academic college of management follow the straight path copy is not unnecessary academic integrity as a teaching and learning issue: from theory to practice stereotypes and shifting standards: forming, communicating, and translating person impressions school students' perception of academic dishonesty with and without digital tools: not acceptable how digital technologies affect academic dishonesty behavior of school students? the ethical dissonance in digital and non-digital learning environments: does technology promotes cheating among middle school students? close educational gap by cheating? comparing faculty, majority and minority students' perceptions of academic dishonesty cheating and e-cheating: a qualitative investigation of expanding an academic dishonesty framework into digital learning settings. paper presented at the 8th international technology, education and development conference-inted2014 re-designed flipped learning model in an academic course: the role of co-creation and co-regulation digital technologies for promoting "student voice" and co-creating learning experience in an academic course why students cheat. an exploration of the motivators of student academic dishonesty in higher education managing academic dishonesty in australian universities: implications for teaching, learning and scholarship. accounting, accountability and performance using research to identify academic dishonesty deterrents among engineering undergraduates academic integrity in institutions of higher education in israel. al hagova, academic track at the college of management infusing ethics and ethical decision making into the curriculum plagiarism and the internet. fears, facts, and pedagogies academic dishonesty among students in tertiary institutions: a literature review white-collar crime in academia: trends in digital academic dishonesty over time and their effect on penalty severity academic integrity as an educational concept, concern, and movement in us institutions of higher learning cheating and feeling honest: committing and punishing analog versus digital academic dishonesty behaviors in higher education how and why students cheat? the role of technology in academic dishonesty and the penalties for conducting academic dishonesty cheating in advantaged high schools: prevalence, justifications, and possibilities for change prevalence, prevention, and pedagogical techniques: academic integrity and ethical professional practice among stem students cheating at small colleges: an examination of student and faculty attitudes and behaviors ethical" cheating in formal education patterns and predictors of academic dishonesty in romanian university students. higher education how gender stereotypes influence the impact of state supreme court advertisements. doctoral dissertation, arizona state university investigating the probability of student cheating: the relevance of student characteristics, assessment items, perceptions of prevalence and history of engagement croatian medical students see academic dishonesty as an acceptable behaviour: a cross-sectional multicampus study student cheating and plagiarism in the internet era: a wake-up call nesb and esb students' attitudes and perceptions of plagiarism the dishonesty of honest people: a theory of self-concept maintenance motivational perspectives on student cheating: toward an integrated model of academic dishonesty custom essay writers, freelancers, and other paid third parties technology as a double edged sword: a promise yet to be fulfilled or a vehicle for cheating cheating and electronic plagiarism-scope, consequences and detection students' perceptions of cheating and plagiarism in higher institutions applying the power of association on campus: a model code of academic integrity institutional policies and faculty perceptions of student academic dishonesty. educational practice and theory the strongest link: legitimacy of top management diversity, sex stereotypes and the rise of women in human resource management peer-to-peer file sharing and academic integrity in the internet age academic dishonesty in the digital era: perceptions and coping strategies of school teachers and parents investigating differences in cheating behaviours of it undergraduate and graduate students: the maturity and motivation factors how is the ethical dissonance index affected by technology, academic dishonesty type and individual differences? does moral judgment go offline when students are online? a comparative analysis of undergraduates' beliefs and behaviors related to conventional and digital cheating authorship, ownership, and plagiarism in the digital age academic cheating in spain and portugal: an empirical explanation uk perspectives of academic integrity difference between women and men regarding decision to commit test cheating below are penalties for students' violations of academic integrity (taken from the university disciplinary committee regulations). please rank the severity of each of the penalties according to your opinion (from 1 'a very lenient punishment' to 6 'a very severe punishment'):publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. ina blau, ph.d., is an associate professor of educational technology and cyber-psychology. she is the head of the research center for innovation in learning technologies and the head of the graduate program in educational technologies and learning systems at the open university of israel. her research interests include integration of innovative technologies in k-12 and academia; digital literacy competencies; computational thinking, visual programming and educational robotics; social aspects of e-communication; and psychological ownership in e-collaboration.shira goldberg, ph.d., is a post-doctoral fellow at the department of social work at the hebrew university of jerusalem, israel and a faculty member at the department of education & psychology at the open university of israel. she has a ba in linguistics and school counseling, an ma in school counseling, and a phd in education from tel aviv university. her major research interests are processes of parental and educational socialization, psychological adjustment and maladjustment, and engagement in risky and problematic behaviors.adi friedman, m.a., holds a master degree in educational technologies and works as an ict specialist at the open university of israel. her research focuses on implications of integrating innovative technologies in academia as well as on coping with digital and analog academic dishonesty in higher education.yoram eshet-alkalai, ph.d., is a professor at the open university of israel, department of education & psychology. he is the founder and former head of the research center for innovation in learning technologies. he holds a ba in archeology (hebrew university), msc in geology (hebrew university) and phd in earth & environmental sciences (city university of new-york). he has 20 years' experience in developing technology-based instructional solutions for educational systems in israel and the usa. in this capacity, he was involved in the design of hundreds of simulations, data-bases, tutorials and large-scale curriculum integration projects. his major research and publications interests focus on cognitive aspects of working with digital technologies, print vs. digital reading, digital literacy and academic dishonesty. key: cord-283477-6clp5nz7 authors: gabrielson, andrew t.; kohn, jaden r.; sparks, hayley t.; clifton, marisa m.; kohn, taylor p. title: proposed changes to the 2021 residency application process in the wake of covid-19 date: 2020-06-01 journal: acad med doi: 10.1097/acm.0000000000003520 sha: doc_id: 283477 cord_uid: 6clp5nz7 the covid-19 pandemic has brought unique challenges to the delivery of undergraduate medical education, particularly for current third-year medical students who are preparing to apply to residency. in mid-march, medical schools suspended all clinical rotations for the remainder of the 2019–2020 academic year. as such, third-year medical students may not be able to complete sufficient clinical experiences to make important career choices before they have to submit their residency applications. while the decision to suspend clinical rotations was necessary to protect students, specialty organizations and residency programs must mitigate the deficits in students’ clinical education caused by the covid-19 pandemic. in this perspective, the authors identify potential challenges for third-year medical students and advocate for solutions to improve the residency application process for students and programs. first, they propose delaying the date that programs can access applicant data through the electronic residency application service, thereby affording students more time to complete clinical experiences, solidify their specialty decision, and strengthen their residency application. second, the authors recommend a restriction on the number of visiting rotations that students are expected to complete to allow for a more equitable distribution of these important experiences. third, they suggest that program directors from each specialty agree on a maximum number of applications per applicant (based on historical data) to curb an upsurge in applications that may stem from the unique circumstances created by covid-19 without causing applicants undue stress. lastly, the authors advocate that residency programs develop infrastructure to conduct video-based interviews and engage students through virtual networking events. amidst the unique environment created by covid-19, the authors urge governing bodies, specialty organizations, and residency programs to consider these recommendations to improve the efficiency and reduce the stress surrounding the 2021 match. as the u.s. health care system grapples with the covid-19 pandemic, unprecedented changes in medical education have emerged that will significantly affect the upcoming 2021 residency application process. this spring, several medical schools graduated their fourth-year students early so these new physicians could alleviate workforce shortages in regions severely affected by covid-19; simultaneously, the educational trajectory of third-year medical students was brought to a halt in mid-march when schools suspended clinical rotations. estimates suggest that the covid-19 pandemic might extend into the late summer in some regions of the united states, with a second surge in cases in the fall. 1 as such, most medical schools have suspended clinical rotations for the remainder of the 2019-2020 academic year. while this change is needed to protect learners, it creates a difficult and uncertain situation for both students and residency programs. governing bodies in medicine need to take concrete actions to mitigate the deficits in students' clinical education caused by the covid-19 pandemic. this crisis is not the first time that medical students applying to residency have been unable to complete core and elective rotations at their home institutions because of a disaster scenario. recent examples include medical students at the university of puerto rico school of medicine following hurricane maria 2 and students at medical schools in gulf states like louisiana following hurricane katrina. however, the situation created by covid-19 is unprecedented in its scope, affecting medical students across the united states. in the following sections, we identify potential challenges for third-year medical students navigating the upcoming 2021 match and advocate for solutions to improve the residency application process for them and residency programs. medical students typically complete their core clerkships, solidify the foundation of their medical education, and make career decisions in the spring of their third year. historically, one-third of students do not decide on a specialty until they have completed their core third-year rotations. 3 they often rely on noncore elective rotations, sub-or actinginternships at their home institution, and visiting rotations to confirm these decisions. for students who struggled with the united states medical licensing examination (usmle) step 1, these elective and visiting rotations offer a crucial opportunity to demonstrate their clinical acumen and make their residency applications more competitive. in particular, underrepresented minority students may be disproportionately the covid-19 pandemic has brought unique challenges to the delivery of undergraduate medical education, particularly for current third-year medical students who are preparing to apply to residency. in mid-march, medical schools suspended all clinical rotations for the remainder of the 2019-2020 academic year. as such, third-year medical students may not be able to complete sufficient clinical experiences to make important career choices before they have to submit their residency applications. while the decision to suspend clinical rotations was necessary to protect students, specialty organizations and residency programs must mitigate the deficits in students' clinical education caused by the covid-19 pandemic. in this perspective, the authors identify potential challenges for third-year medical students and advocate for solutions to improve the residency application process for students and programs. first, they propose delaying the date that programs can access applicant data through the electronic residency application service, thereby affording students more time to complete clinical experiences, solidify their specialty decision, and strengthen their residency application. second, the authors recommend a restriction on the number of visiting rotations that students are expected to complete to allow for a more equitable distribution of these important experiences. third, they suggest that program directors from each specialty agree on a maximum number of applications per applicant (based on historical data) to curb an upsurge in applications that may stem from the unique circumstances created by covid-19 without causing applicants undue stress. lastly, the authors advocate that residency programs develop infrastructure to conduct videobased interviews and engage students through virtual networking events. amidst the unique environment created by covid-19, the authors urge governing bodies, specialty organizations, and residency programs to consider these recommendations to improve the efficiency and reduce the stress surrounding the 2021 match. proposed changes to the 2021 residency application process in the wake of covid-19 affected by a reduction in clinical exposure given that they historically do not perform as well on step 1 compared with their peers. 4 before covid-19, several articles have proposed eliminating using a minimum step 1 score to determine residency interviews as a strategy to reduce racial bias in the residency application process. 4, 5 additionally, the usmle has announced that it will change step 1 scoring to pass/fail in part to address this overemphasis on standardized testing scores. 6, 7 given that students will have fewer opportunities for clinical experiences because of covid-19, the upcoming 2021 residency application process could further rely on standardized testing scores, exacerbating existing disparities. likely, this will most severely affect those applicants applying to more competitive fields, which place greater emphasis on standardized testing performance, as well as those applicants with underrepresented identities including race, gender, and older age. 8 these applicants would benefit from residency programs recognizing the inequalities that step 1 score minimums promote, particularly with the dearth of clinical opportunities during this particular match cycle. in addition to the loss of total available months of clinical training and specialty exposure, there could be a shortage of clinical rotation spots once medical schools reopen, creating a more competitive environment among students who wish to complete these rotations. to meet graduation requirements, students will need to prioritize their core clerkships, which will limit their exposure to noncore specialties (urology, orthopedic surgery, neurosurgery, otolaryngology, etc.), potentially resulting in a lower volume of students applying to these specialties. in addition, current third-year medical students may struggle to obtain sufficient experiences to make important career decisions. given the established association between specialty regret and burnout, an accelerated timeline for making specialty decisions could exacerbate burnout in these future physicians. 9 the shortened timeline for completing clinical rotations also may affect students' ability to explore novel educational environments and to assess mutual "fit" between themselves and a residency program, which happens during visiting rotations. often, the letters of recommendation obtained during these rotations are among the most important factors considered by program directors when making interview decisions. 10, 11 in less competitive specialties, more than 50% of applicants report completing at least one visiting rotation. 11 for many competitive specialties, these rotations are viewed as a de facto requirement for applying to residency, with applicants performing 2-to 3-month-long visiting rotations on average, in addition to a rotation at their home institution in their chosen specialty. 10 programs already had a limited number of students (home or visiting) they could accommodate. now, these clinical rotations will become a scarce commodity if medical schools reintroduce students to the clinical environment. finally, there is wide variation in the amount of clinical experience that medical students obtained before clinical rotations being suspended because of covid-19. while all students will have fewer clinical experiences, those enrolled in medical schools with a traditional 2-year preclinical curriculum will have 6 months less clinical time than students enrolled in programs with an accelerated one-and-a-half-year preclinical curriculum. furthermore, students enrolled in 3-year programs may be particularly disadvantaged, as their clinical curriculum may already be truncated. 12 residency programs will need to critically assess the total volume of clinical experience for each applicant, given the diversity of curricular structures and the differing impacts of covid-19 for each type of medical school curriculum. depending on the structure of their medical school, foreign medical graduates also may experience a loss of essential clinical time. to address these anticipated challenges, we propose the following 4 solutions (see figure 1 ).* first, we suggest delaying when residency programs can access applicant data through eras. currently, eras allows programs to download applications from their online servers starting in mid-september. however, the actual date of first download and the frequency with which applications are updated in the subsequent months vary between specialties. delaying the date of first download by a few weeks to months would allow medical students to maximize their limited time to complete core clerkships as well as visiting or elective rotations. each specialty could collectively determine (and should publicize) its own agreed-on timeline for initial data extraction from eras, based on the anticipated volume of applicants, requirements for visiting student rotations, and the typical interview schedule. for example, internal medicine and dermatology would need very different timelines. a delay in the initial download date by even a month would allow students the opportunity to have additional clinical experiences, which would aid both their specialty decision-making process and the competitiveness of their application. this change could be accomplished without significantly interrupting interview dates or the date of the national resident matching program (nrmp) match. however, delaying the date on which programs first access applicant data may require residency programs to review applications on an accelerated timetable, which could prevent them from performing a holistic review and force them to rely on test score minimums, a strategy with potential negative consequences as discussed above. second, we recommend a formal restriction on the number of visiting rotations medical students are expected to complete, an approach already supported by program directors. 6 each specialty should come to a consensus on the number of visiting rotations that are expected of their applicants. likely, this year that number will be lower than that in previous years. this change will allow for a more equitable distribution of what will undoubtedly be a scarce resource. visiting rotations are also costly; placing *this perspective was originally submitted to academic medicine on april 2, 2020, before many of the changes suggested and implemented by the association of american medical colleges and other organizations. the authors are pleased that many of their proposals have been adopted, including pushing back the date residency programs can access applicant data in the electronic residency application service to october 21, 2020; canceling all visiting rotations for the summer and fall of 2020; and urging all residency programs to offer only virtual interviews. a restriction on them may alleviate some financial pressures on students to choose one specialty over another. 11 in addition, the educational value of these rotations has been questioned, and students often view them more as an audition than an opportunity to garner clinical skills. 13 while visiting rotations are typically 4 weeks long, programs also might consider shortening their duration, which would still allow students to assess the "fit" between themselves and the program, while providing more students with the opportunity to participate in these essential experiences. conversely, restricting visiting rotations will reduce students' opportunities to obtain clinical experiences in diverse learning environments and to demonstrate interest in multiple geographic regions, which could negatively impact their specialty choice and their competitiveness to match at specific programs. we are not aware of any literature demonstrating the efficacy of rationing away rotations as we suggest, given that in prior disaster scenarios, only individual schools or regions were affected and large-scale policies were not required. for example, students at the university of puerto rico school of medicine were able to complete their rotations at programs in the continental united states following hurricane maria, and students at tulane university school of medicine were able to complete their rotations at baylor college of medicine following hurricane katrina. third, we propose a specialty-specific cap on the number of applications that each medical student can submit through eras. over the last 2 decades, the number of applications per applicant has increased substantially. 14 annual data from the nrmp suggest that each specialty has a unique threshold for the number of applications and interviews necessary for a student to have a high probability of matching. 15 no evidence suggests that additional applications confer improved outcomes, while research has shown that superfluous applications lead to increased costs and inefficiencies in the process. 16 we anticipate that the uncertainty associated with the covid-19 pandemic and the widespread disruption of medical education could exacerbate this tendency for students to apply to an excessive number of residency programs. additionally, if there is an intentional delay in the first download date for applications, residency programs may be overwhelmed with the number of applications that need to be reviewed in a truncated time period. to address these concerns, we recommend that program directors from each specialty agree on a maximum number of applications allowed per applicant. research has already demonstrated an existing consensus among program directors that application numbers should be caped. 10 this approach could curb an upsurge in applications stemming from the unique environment created by covid-19, without causing undue panic among students, which could occur if a cap is imposed that they perceive to be so low as to be disadvantageous. we recognize that a cap on the number of applications may disproportionately affect applicants who are perceived as less competitive, including international and osteopathic medical students and students with relatively low standardized testing scores. however, a permissive specialty-specific cap improves the efficiency of the application process without adversely affecting the vast majority of applicants. for example, a reasonable strategy could be to limit applicants to the average number of applications submitted in the 2020 cycle (e.g., for pediatrics and orthopedic surgery, that was 36 and 82, respectively). 17 lastly, if the heavy restrictions on domestic travel remain in place nationally or in certain regions of the country as covid-19 outbreaks linger, residency programs may need to develop the infrastructure for video-based interviews as well as for virtual networking opportunities. many fellowships that typically interview in the spring have already transitioned to synchronous video interviewing (live video and audio between the applicant and interviewer), a method supported by existing evidence. 18 asynchronous video interviews (recorded applicant responses to specific questions that are later viewed by interviewers) could be considered as well. these have been in use by all emergency medicine programs and one obstetrics and gynecology program for the last 3 years. 19, 20 to ensure applicants' exposure to the unique characteristics of a residency program, virtual networking events should be considered, during which applicants can interact with residents, faculty, and other program members. additionally, programs should consider expanding their virtual and online presence (department websites, social media pages, video content, etc.). we strongly recommend that residency programs restructure their traditional interviewing methods to accommodate the rare situation we are in and that they begin these preparations early. amidst the unique environment created by the covid-19 pandemic, we urge the association of american medical colleges, the nrmp, medical schools, residency program director associations, and specialty organizations to consider the solutions we have proposed to improve the efficiency and reduce the stress surrounding the 2021 match. finally, we acknowledge the adaptability and perseverance that third-year medical students have shown during this frustrating and uncertain time, and we urge them to embrace these qualities, which will serve them well in the match as well as in life. funding/support: none reported. other disclosures: none reported. ihme covid-19 health service utilization forecasting team, murray cjl. forecasting covid-19 impact on hospital bed-days, icu-days, ventilator days and deaths by us state in the next 4 months san juan bautista school of medicine's experience in the aftermath of hurricane maria changes in u.s. medical students' specialty interests over the course of medical school racial bias in using usmle step 1 scores to grant internal medicine residency interviews underrepresented minorities are underrepresented among general surgery applicants selected to interview the usmle step 1 pass/fail reporting proposal: another view change to pass/fail score reporting for step 1 overemphasis of step 1 scores may affect application pool diversity in radiation oncology association of clinical specialty with symptoms of burnout and career choice regret among us resident physicians orthopaedic surgery residency: perspectives of applicants and program directors on medical student away rotations time to reevaluate the away rotation: improving return on investment for students and schools three-year md programs: perspectives from the consortium of accelerated medical pathway programs (campp) the prevalence and cost of medical student visiting rotations relationship between the number of residency applications and the yearly match rate: time to start thinking about an application limit? results and data: 2020 main residency match interview from anywhere: feasibility and utility of web-based videoconference interviews in the gastroenterology fellowship selection process use of asynchronous video interviews for selecting obstetrics and gynecology residents assessing residency applicants' communication and professionalism: standardized video interview scores compared to faculty gestalt ethical approval: reported as not applicable. key: cord-327207-bwuag9ek authors: matloubieh, jubin e.; eghbali, manizheh; abraham, nitya title: strategies to encourage medical student interest in urology date: 2020-08-07 journal: curr urol rep doi: 10.1007/s11934-020-00984-1 sha: doc_id: 327207 cord_uid: bwuag9ek purpose of review: urology residency positions have steadily increased but applications have remained stagnant. this is an alarming trend given the aging general population and thus increased need for urologists. the purpose of this review is to describe barriers and suggest strategies to encourage medical students to pursue urology. recent findings: barriers to interest in urology include educational factors, such as timing of exposure to urology in medical school, usmle scores, research experience, and deciding in time for an early match, as well as socioeconomic barriers, such as cost, being underrepresented in medicine, and gender. steps the urological community can take include increasing involvement in medical school curricula, increasing faculty mentor availability, and broadening students’ range of urological experiences. summary: strategies to encourage interest in urology fall into three categories: creating interest, supporting interest, and removing barriers for students considering urology. ultimately, the goal is to garner excellent residents in a field that must expand to meet the needs of a growing and aging population. in 2020, the american urological association (aua) match had a record 354 urology residency positions, an increase from 285 in 2014. however, the number of applicants over the same period has remained about the same, with 441 applications and an overall match rate of 83% in 2020 (table 1 ) [1] . given the increased number of residency positions, the growing need for urological care due to the aging us population, and the relatively high happiness outside of work reported by urologists relative to their peers in other specialties, it is surprising that the number of applicants has not increased in tandem with the increase in residency positions [2] . to further compound the issue of recruiting medical students into urology, urologists as a group are aging and approaching retirement age, which is projected to lead to a larger shortage of urologists by 2035 [3•] . a survey of the literature reveals barriers that deter students from urology include academic factors, the application process, and certain socioeconomic factors. on the other hand, factors that motivate students to enter urology include clinical exposure, educational tools, and mentorship. by better understanding barriers that medical students face, as well as factors that motivate medical students to enter urology, the urological community can devise new or revise current strategies to encourage medical students' interest in urology and fill the increasing need for urological care (table 2 ). barriers medical students face in pursuing urology include educational factors, such as audition rotations and research, as well as socioeconomic factors, such as cost, underrepresentation in medicine, and gender. another element worthy of mention is the burnout rate among urologists, despite their high career satisfaction. without addressing these often intertwined barriers, medical students often face a high wall to entering urology that may deter even highly qualified students from applying in the first place. educational barriers that medical students face in choosing urology include medical school curricular obstacles, a common feeling among applicants that a research background is necessary, and the perceived difficulty of applying and matching to urology. curricular barriers result from wellintentioned medical school curriculum committees that seek diversify medical student experiences as well as the inherant structure of medical education in the usa. some schools, for example, only allow students to take a certain number of rotations within a given field. in other schools, students may take their surgical clerkship later in their third year, which compounds the already shortened deadline imposed by the aua's relatively early match. subsequently, many students have little to no exposure to urology as a field until their third or even fourth year, resulting in less time to decide on urology as a specialty to participate in multiple audition rotations, which are a strong predictor for matching into a given program [4] [5] [6] . the problem of limited time during medical school also arises in research. anecdotally, many medical students are told that research is important for applying to surgical specialties. a large survey of us medical students found that one quarter had no research interest, and that a majority of medical students found research experiences to not be meaningful. barriers to involvement in research included the time necessary to complete research, lack of research mentors availability, and the perception that the student may not be appropriately acknowledged for their contributions [7] . both curriculum design and the research conundrum factor into a larger barrier for medical students applying to urology: the perceived difficulty of matching into urology. while urology remains one of the more competitive specialties, this perceived difficulty likely further deters many qualified applicants. the first barrier that medical students must overcome in this sense is the united states medical licensing examination (usmle) step 1 exam, where various "score cutoffs" are circulated among medical students. a study looking at the association of step 1 score and specialty at their institution from 2011 to 2015 found that the mean usmle step 1 score was 243 for those matching in urology [8] . a large survey of program directors found that a lower step 1 score was one of the most deleterious criteria for selection for interviews [9] . while this has historically been an important factor in applying to urology, the sponsors of the usmle recently announced a change in step 1 score reporting to make it purely a pass-fail examination [10•] . for students considering applying to competitive specialties, this is a noteworthy change in one of the most significant selection criteria, and it raises the question of what will fill the role that the step 1 score had as an "objective" measure for applicants. it remains to be seen whether usmle step 2 will become a requirement so that programs will have a new "objective" measure, adding further pressure to applicants who are attempting to balance multiple audition rotations and applying to urology, or whether programs devise more holistic ways of evaluating applicants. gender and underrepresentation in urology are oft-discussed topics, as urology has remained a white male-dominated field even as other medical and surgical fields have diversified. although the percentage of female applicants in urology has been increasing, male applicants still outnumber their female counterparts by double, while the match rate for male and female applicants is about equal [1] . the gender disparity has been studied from a number of angles, including mentorship and from trainees' perspectives. for example, the disproportionate underrepresentation of female urology faculty manifests as a lack of female mentors, impeding female students from considering urology-a finding corroborated in a study that found having more female faculty predicted having a higher number of female applicants [11••] . further studies have found that having pediatric, trauma, reconstructive, and/or female pelvic medicine programs predicted a higher number of female applicants as they may be more interested in those subfields [12•] . urology departments throughout the country have attempted to address the gender gap by hiring more female faculty and diversifying their clinical offerings, but the disparity remains [13] . another unique barrier for females is misogyny within a male-centered field. a study of female urology residents found that over half of those surveyed had experienced negative behaviors from both male patients and colleagues [14] . students from communities underrepresented in medicine also face similar barriers from lack of mentorship and faculty they can identify with. one study of residents in urology, surgery, and in general found that urology as a field had a lower proportion of student underrepresented in medicine compared to both surgerical residents and residents in all fields, similar to the representation of women in urology [15] . steps the urological community can take to mitigate these issues and improve female and underrepresented medical student engagement in urology include hiring diverse faculty, involving diverse faculty in direct medical education, providing sensitivity and implicit bias training for all attendings and residents, and fostering healthy work cultures where people feel comfortable discussing issues involving sexism or other hostile elements. another large barrier to pursuing a career in urology is the enormous time and money cost seemingly required by the application process. for urology applicants, the basic residency process is stretched out and inflated: students start considering where to do sub-internships midway through their third year, go through a competitive sub-internship application process with a financial and time cost, travel and pay to participate in month-long audition rotations in hopes of obtaining a letter of recommendation where they must secure stable housing remotely, and then apply to residency programs. in 2020, the average number of programs applied to was 74, an increase from prior years, yielding a further cost since the application service has financial penalties for a larger number of applications [1, 16] . once interview invitations arrive, students then arrange travel and lodging, often at the last minute and with little flexibility. studies of urology applicants have found that the financial cost of applying to residency for subspecialties can be upward of $7000 since almost 100% of applicants undertake away rotations [17•, 18•] . given the rising number of applications students submit, it follows that the financial barrier to applying for urology will continue rising. because of this alarming picture, institutions and larger governing bodies have taken some steps to attempt to overcome the enormous financial burden that may deter students from considering urology. multiple institutions offer scholarships for students from disadvantaged or underrepresented backgrounds that can be applied toward room, board, and transportation for audition rotations. other institutions have low or no charge for visiting student rotators. in perhaps the most dramatic attempt to address this issue, all of the canadian programs joined together to create the canadian urology fair starting in 1994. after surveying students and programs in the pilot year of this program, the overseeing committee found that the cost to students was significantly less than traveling to interviews, the costs to programs were not significantly different than from hosting interviews, and that students missed fewer days of school compared to their counterparts in other surgical specialties who had to travel to interviews [19] . the main drawbacks to this process are the lack of the social element, for applicants are also interviewing the programs, and the fact that there are ten times as many urology residency programs in the usa than in canada. one way to reconcile the canadian model in the sprawling usa and thus reduce the cost barrier could be to adopt a similar model to music and dance school audition committees that host auditions at sites around the country, perhaps at regional aua sectional meetings. in addition, the changes to the 2021 application process in response to the covid-19 pandemic have resulted in residency programs creating new ways to connect with interested medical students. anectdotally, these changes include virtual open houses, sub-internships, and interviews, as well as increased use of social media. it remains to be seen whether these changes will remain permanent if the pandemic subsides. urology ranked highest (54%) in a 2020 medscape national survey of 15,181 physicians on burnout and suicide with the leading cause of burnout being too many bureaucratic tasks (55%), like charting and paperwork. critics of these findings point to the limited sample size of urologists surveyed in the past (n = 119) [2] . the 2016 aua census data surveying a matrix sample of 1126 practicing urologists found a burnout rate of 38.8%, which is comparable to other specialties [20• ]. nevertheless, a systematic review of factors influencing medical students' choice of subspecialty found that the third most important factor was controllable lifestyles or flexible work schedules (53%), following academic interests (75)% and competencies (55) [21] . high reported burnout rates in urology could deter student interest, especially in those concerned about work-life balance. anectodally, urologists report a high work-life balance satisfaction, so perhaps increasing medical student mentorship could combat the perception that urologists have a higher burnout rate. early clinical exposure while medical students may have some exposure to urology prior to medical school, the initial exposure to most topics in medicine, including urology, occurs during the preclinical years of medical schooling. a number of studies have found that there is a paucity of exposure to urology in these formative years of medical school. unfortunately, these studies have also found a decline in urology exposure. a 2007 study found that 32% of medical schools had no exposure to urology in the preclinical years [5] . a 2014 follow-up to this study found that this number had increased to 50% of medical schools not having urology exposure in the preclinical years [6] . this decline in urology exposure can be to the detriment of future patients, for as one study noted, knowledge of basic urology was low for both medical students and primary care faculty taking care of the bulk of communities [22] . as exposure is critical to fostering medical student interest in urology, it is imperative that the urological community devise strategies to increase and maintain exposure to the field of urology. one such strategy was studied in england where early sustained exposure to urology was correlated with considering a career in urology [23] . despite this encouraging finding, the state of medical education in the usa seems to slant students away from pursuing urology, for a 2014 study found a decline in medical schools requiring urology rotations from 100% in the 1950s to 5% in 2014 [6] . further compounding this is another discouraging report that 65% of american medical schools surveyed reported that it was possible to graduate from their school without any clinical exposure to urology [5] . fortunately, a number of methods have been studied that increase students' knowledge of and exposure to urology, and thus hopefully increase their interest. while the number of studies and data obtained is limited, the variety of approaches examined is encouraging in the way of urological educational innovation. educational strategies that have specifically been studied in the urological literature include dedicated clinical rotations in urology, the use of simulators, the use of technology in education, the presence of a urology interest group, and mentorship. multiple studies have found that having a clinical rotation in urology increases students' comfort in managing urological problems and potentially increases consideration of a career in urology. clinical exposure to medical problems and treatments is crucial in cultivating, expanding, and reinforcing knowledge learned in the preclinical years. a study of outpatient versus inpatient settings for learning urology found that students in the outpatient clinical setting had a larger exposure to urological problems, better knowledge of urological physical examination, and were more likely to perceive that they learned more. further, students in the outpatient setting may have greater exposure to urology faculty in the clinic compared to the floors, encouraging the mentorship needed to motivate students into a career in urology [24] . a study conducted 13 years later found similar results and expanded into acquiring skills such as foley catheter insertion [25] . by including students more in clinical management, such as in placement of foley catheters, a sense of inclusion in the team is fostered, further reinforcing the social bonds that encourage students to pursue urology. the social aspect to medical education and garnering student interest was reported in an earlier study where third year medical students felt that clinical learning followed by resident teaching to be the most helpful, whereas watching surgery, especially endoscopic surgery, to be less helpful [26] . this calls into question surgical clerkship structure where maximal operative exposure is encouraged, even in endoscopic or robotic cases, perhaps at the expense of the potentially more educational and social clinical setting. the use of various simulations in urological education has had mixed results in increasing interest in urology. a study of a clinical clerkship in urologic laparoscopy had a high level of interest and satisfaction among medical students, but did not find a change in the desire to pursue a career in urology [27] . a different study found that students' confidence in performing catheterization and digital rectal examination, especially female students' confidence, was improved postsimulation; while this study did not comment on the change in interest in pursuing urology, it did address an issue urology faces in recruiting female trainees [26] . the conclusion to be drawn from these experiences may be that simulations have an adjunctive role in urological education, but may be less helpful in increasing interest in urology when used alone. similarly, the use of media and technology in urological education to allow for broader exposure to urology is inconclusive in the ability to attract students to urology. two methods described in the literature include the use of a massive open online course (mooc) and the use of a "learning package for medical students in a busy urology department." the former method was innovative in that it was the first mooc that expressly taught urology; while over 500 participants completed this mooc and 82% of countries were represented, it neither commented on its utility in attracting students to urology nor was it specifically geared toward medical students [28] . in another study, the use of online "learning package for medical students" was helpful for learning core clinical problems. again, this study did not comment on the utility of the online material in helping attract students to urology, and even its title emphasized important barriers for students considering a career in urology, namely lack of access to faculty mentors in a "busy urology department" [29] . the educational strategy that has been most uniformly effective in recruiting students to a career in urology has been mentorship. a study of why certain medical schools have more students pursuing urology found that mentorship was the single most important factor between schools [30] . this broader finding was confirmed in a study within a single school where having positive role models in urology led to a positive perception of urology and thus increased interest in urology [31] . mentorship may be a helpful factor in helping bridge the gender gap in urology, as having a higher percentage of female urology faculty is correlated with a higher percentage of female students pursuing urology [11••] . a similar conclusion can be extrapolated to students underrepresented in medicine. one specific method that has been studied to connect medical students to mentors is the presence of a urology interest group, which has also been found to have a positive impact on medical student interest in urology [11••] . other opportunities through which urology faculty can serve as mentors include offering research projects in urology during the summer between first and second year of medical school, leading small group or one-on-one sessions in first and second year clinical courses, and serving as teachers in preclinical classes like anatomy or male reproductive pathophysiology. creating a more formal mentorship program is a further step that the urological community can take to encourage students to enter urology. this idea was tested in germany, where a national mentoring program for urology was started in 2006 and has continued since then due to an overwhelmingly positive response [32] . while this may seemingly be more difficult in the more populous usa, the urological community remains small and well-connected, so a national, or at least regional, mentorship program may not be difficult to implement. the early match requires that applicants decide before the end of third year if they are interested in urology. switching to the national residency matching program (nrmp) match, which occurs in march, may afford more time to students exploring urology late in third year and may increase the number of applicants. in 2006 otolaryngology, another competitive surgical subspecialty switched from an early match to the regular nrmp match. however, a comparison of the applicants and match rate before and after the change found no difference in the number of applicants or competitiveness of the specialty, suggesting a switch to the nrmp match may not necessarily lead to increased interest in urology [33] . it remains to be seen whether changes in the 2021 application cycle in response to the covid-19 pandemic will have an impact on students applying to urology. a recent viewpoint piece in jama suggests another alternative to the current early match and regular match: an optional early application and acceptance program. this strategy is akin to the early acceptance process used by colleges and universities and may suit several groups of students including highly competitive students with clear program choice or the seemingly less competitive students who may be hesitant in applying to a competitive field such as urology. another advantage of this program would be a substantial reduction in time and cost expenditure as each applicant pursues fewer programs [34••] . broadly speaking, strategies to encourage medical student interest in urology fall into three categories: creating student interest, supporting student interest, and removing barriers to students considering urology. to address the issue of encouraging medical student interest in urology requires a threepronged approach and increased effort on the part of urology residency programs. future steps that the urological community can take include increasing involvement in the medical school curriculum during the preclinical years, increasing faculty availability as mentors, broadening the range of urological experiences that students have, lowering academic and socioeconomic barriers, and finding ways to continue vetting quality applicants despite changes in board exam scoring and possible restrictions in the number of programs students can apply to. ultimately, the goal is to continue garnering excellent residents in a field that needs to expand to meet the demands of a growing and aging population. conflict of interest the authors declare that they have no conflict of interest. human and animal rights and informed consent this article does not contain any studies with human or animal subjects performed by any of the authors. medscape physician lifestyle & happiness report 2020: the generational divide projects an increased deficit in urologists by 2035 due to both the aging us population as well as the aging urologist population who will be retiring the afmc electives diversification policy: potential drawbacks and benefits for medical students applying to urology the current status of medical student urological education in the united states an update on the current status of medical student urology education in the united states a survey on the attitudes towards research in medical school the association of usmle step 1 and step 2 ck scores with residency match specialty and location program directors criteria for selection into urology residency usmle step 1 scoring will be changing from a numerical score to purely pass/fail, eliminating one of the filters that urology residencies use for residency applications 028 having more female urology faculty is positively correlated with more female medical students applying to urology. having a urology residency program and having a urology 065 having a broad array of urological subspecialties is correlated with more female medical students applying to urology gender and subspecialty of urology faculty in department-based leadership roles a survey of women urology residents regarding career choice and practice challenges examining trends in underrepresented minorities in urology residency fees for eras residency applications survey of applicant experience and cost in the urology match: opportunities for reform corroborates the value of medical students undertaking audition rotations with respect to matching into a residency program the canadian urology fair: a model for minimizing the financial and academic costs of the residency selection process burnout statistics match those of other specialties, yet there is still a growing deficit in the number of students applying factors influencing subspecialty choice among medical students: a systematic review and meta-analysis deficits in urological knowledge among medical students and primary care providers: potential for impact on urological care factors affecting uk medical students' decision to train in urology: a national survey does the outpatient setting provide the best environment for medical student learning of urology? the value of a core clinical rotation in urology for medical students simulators help improve student confidence to acquire skills in urology an innovative medical student clinical clerkship in advanced urologic laparoscopy: a preliminary experience kiurologyx: urology as you like it-a massive open online course for medical students, professionals, patients, and laypeople alike a learning package for medical students in a busy urology department: design, implementation, and evaluation the gatekeeper disparity-why do some medical schools send more medical students into urology? survey on the perception of urology as a specialty by medical students bringing excellence into urology: how to improve the future training of residents? state of otolaryngology match: has competition increased since the "early" match? otolaryngol head neck surg improving the residency application and selection process: an optional early result acceptance program describes a novel method for decreasing the cost of applying into urology and lowering potential barriers for students applying into urology publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-010639-6d0u79f1 authors: jackson, jennifer m.; skelton, joseph a.; peters, timothy r. title: medical students’ clinical reasoning during a simulated viral pandemic: evidence of cognitive integration and insights on novices’ approach to diagnostic reasoning date: 2020-05-06 journal: med sci educ doi: 10.1007/s40670-020-00946-9 sha: doc_id: 10639 cord_uid: 6d0u79f1 introduction: cognitive integration from multiple disciplines is essential to clinical problem-solving. because it is not directly observable, demonstrating evidence of learners’ cognitive integration remains a challenge. in addition, little is known about preclinical medical students’ approach to diagnostic reasoning despite widespread implementation of clinical reasoning curricula for these early learners. the objectives of this study were to characterize how first-year medical students integrated knowledge to problem-solve during a simulated viral pandemic and to characterize students’ diagnostic reasoning approach to this clinical scenario. materials and methods: student teams analyzed clinical data to formulate hypotheses for the pandemic’s source and submitted reports justifying their hypotheses and treatment recommendations. a content analysis on students’ reports identified codes and themes characterizing the learning content integrated and students’ approaches to diagnostic reasoning tasks. results: sixteen problem-solving codes were identified, demonstrating integration of new and previously encountered content from multiple disciplines. a compare-contrast analytical approach was the most commonly employed diagnostic reasoning approach (100%), with a smaller subset of teams also using a causal approach (20%). discussion: content analysis of preclinical students’ diagnostic justification tasks provided insights into their approach to diagnostic reasoning, which was most consistent with the search-inference framework rather than a causal approach, likely due to limited pathophysiological knowledge at that point in training. conclusions: evidence of cognitive integration can be made explicit through learners’ narrative justification of diagnostic reasoning tasks. preclinical students’ diagnostic reasoning development has implications for curricular design and implementation for this learner group. to make appropriate diagnostic and management decisions, physicians must cognitively integrate and apply knowledge and skills from multiple disciplines. cognitive integration is defined as "conceptual, cognitive connections between different types of knowledge," occurring within the mind of the learner [1] . previous studies suggest novices' cognitive integration of basic science and clinical concepts leads to improved diagnostic performance in this early stage of training [2] [3] [4] [5] [6] [7] [8] [9] [10] . because cognitive integration is not directly observable, demonstrating evidence of successful cognitive integration remains a challenge. the diagnostic reasoning processes of experts versus novices have been extensively studied, and characteristic patterns of diagnostic task performance have been described that correlate with level of expertise. previous studies have suggested that novices primarily utilize a pathophysiologic analytical (i.e., causal) approach to diagnostic reasoning and rely less on the compare-contrast analytical approach or on pattern recognition, both of which are thought to develop only after trainees begin building robust illness scripts based on clinical experience [11] [12] [13] [14] [15] . however, the novices in published studies examining diagnostic reasoning approaches have been limited to intermediate (third year or fourth year) medical students, resident physicians, or nonmedical health professions students (e.g., massage therapy students, undergraduate kinesiology students, physical therapy students) [5, 8, [16] [17] [18] [19] [20] ; little is known about the diagnostic reasoning approaches of early, preclinical medical students. published studies involving preclinical medical students consist of intervention studies reporting results of knowledge-based testing or diagnostic accuracy rather than characterization of how these students formulate their differential or working diagnoses [9, [21] [22] [23] [24] . since clinical reasoning curricula now extend into the preclinical years for many medical schools, understanding these learners' approach to diagnostic reasoning from the beginning of their development is important for both curriculum development and learner assessment in these cognitive skills. simulation is an instructional delivery method thought to promote cognitive integration and diagnostic reasoning by providing learners with opportunities to apply basic science knowledge in relevant clinical contexts, including opportunities to practice applying their causal knowledge-that is, using basic science concepts to explain clinical manifestations of disease [12, [25] [26] [27] [28] . with this in mind, we developed a collaborative, simulation-based learning activity for our firstyear medical students requiring them to integrate knowledge from multiple disciplines in order to problem-solve during a simulated viral pandemic. in this study, we analyzed students' reports from this learning activity in order to (1) characterize how early preclinical students integrate new and existing knowledge as reflected in their written diagnostic justification tasks and (2) characterize early preclinical students' diagnostic reasoning approaches to this case. a secondary objective of this study was to identify the resources these students used to approach their self-directed learning during this activity. the subjects in this study were 140 first-year medical students who participated in a 2-hour simulated pandemic activity in january 2018. the event was part of students' virology course and occurred following core virology instruction. the virology course occurred approximately 6 months into students' first year of medical school and followed the anatomy, biochemistry, and bacteriology courses. prior to this event, the students had had pathophysiology instruction on metabolic, bacterial, genetic, fungal, and immunological diseases as well as sepsis. concurrently throughout the first year, these students also participated in longitudinal clinical skills, epidemiology, and bioethics courses. prior to this simulation event, these students had learned how to collect a full history in their longitudinal clinical skills course; this included history-taking skills practice in numerous real and simulated patient encounters in each of the 11 clinical skills class sessions leading up to this event. these students had also participated in 24 sessions of a longitudinal, problem-based learning (pbl) course prior to this simulation event. the pbl course is case based and focused on differential diagnosis formulation using a causal approach, guided by a mnemonic ("vindicatem-p") to systemically consider diseases in each pathophysiological category. during these sessions, students engaged in self-directed learning as needed when their baseline knowledge was insufficient to generate a relevant list of diseases from each category. for the simulated viral pandemic activity, students were assembled into 20 teams of 6 to 7 students, each of which was tasked with collecting and analyzing data to formulate hypotheses for the source of the pandemic. each team rotated through a series of 3 stations: a 10-minute viewing of a simulated inpatient room, during which student teams gathered data from bedside clinicians about their observations of patients' physical exam findings and clinical course, and two 10minute encounters with actors portraying eyewitnesses, during which students gathered data about patients' initial symptoms (of note, students could not directly interview the patients themselves, as the simulated illness was characterized by severe aggression and disorientation). during portions of the activity when they were not rotating in one of the above stations, students worked with their teammates to review and analyze other data provided on a simulated "emergency operations center" website which housed simulated epidemiological data and video clips of frontline clinician reports. the epidemiological data included the case definition of the illness; 2 distribution maps illustrating the number of cases of the disease across the world and among states within the usa, respectively; epidemic curves for 6 us cities and one other country where the illness was first identified; and tables of risk ratios for 5 us states, which included risk ratios and 95% confidence intervals for various age groups, males versus females, different racial/ethnic groups, and rural versus urban locations. the video clips of frontline clinician reports provided clinical observations about infected patients from the perspectives of an emergency medicine physician, a neurologist, a pathologist, and a critical care physician (videos were 3 to 7 min, each). links to ethics resources were also provided, including brief descriptions of a list of relevant ethical considerations (e.g., duty to care, fairness, transparency, etc.) as well as a brief video of an ethicist highlighting several ethical issues that have arisen during the simulated pandemic, including acting in patients' best interest, struggles with resource allocation, challenges in protecting frontline healthcare providers who are at high risk of infection, and mandatory quarantine (though specific strategies for addressing these issues were not provided). periodically throughout the activity, students were also emailed a series of "updates from the field," the first of which was a simulated chart review summarizing the clinical characteristics noted at presentation of patients seen in the local emergency department, including the mean and range of patient ages, frequencies of specific presenting signs and symptoms, laboratory and radiological findings, and clinical outcome (discharged to home vs. ongoing hospitalization vs. death). the second email provided students with electron micrograph images of the mystery pathogen, including captions providing clues to the viral structure. the third email, sent in the latter half of the activity, presented students with risk ratios of adult and pediatric patients treated with fresh frozen plasma and/or cryoprecipitate, indicating lower risk of mortality among patients treated with the latter; this email presented students with the additional task of developing an ethically sound strategy for allocating this limited resource (which was in short supply) to infected patients. students were instructed to electronically submit a team report using a prepared template on microsoft word (microsoft corporation, redmond, wa); the template included prompts directing students to (1) analyze clinical, epidemiologic, and molecular data to determine the most likely pathogen causing this outbreak; (2) develop recommendations for treatments or treatment strategies, based on the leading hypothesis, and (3) develop recommendations for public health measures in the local community to prevent further spread of the infection. students were required to explicitly justify each of these items, using supporting evidence from the data they collected during the activity and/or relevant information collected during self-directed learning. students were also instructed to cite the sources they used during their selfdirected learning. following the conclusion of the 2-hour attendance-required activities, student teams were allowed an additional 2 hours to work on their team reports, if desired, before final submission. students' reports were analyzed using an exploratory sequential mixed method design. first, a content analysis approach was used to characterize students' diagnostic reasoning and patient management tasks [29] . we systematically reviewed all reports and formulated descriptive codes for the data through an iterative data analysis process and then organized these codes into related themes. we then repeatedly reviewed the reports to clarify and refine the codes and themes using constant comparison; we then calculated the frequency of each code among students' reports to provide insight into which problem-solving tasks students selected most frequently to address this case scenario. these codes were initially organized by themes according to the course or discipline representing the source of learning content described in each code (e.g., virology, epidemiology), to characterize the disciplines integrated by students in their reports for this problemsolving exercise. next, we identified a subset of these task codes as diagnostic reasoning tasks by identifying the codes indicating students' use of data to identify potential pathogens for students' differential diagnosis and/or students' use of data to justify their working diagnosis. these diagnostic reasoning task codes were then categorized by diagnostic reasoning approach type: a compare-contrast approach (i.e., illness script elements of patients' disease presentation are compared with those of known diseases) vs. a causal approach (i.e., a postulated pathophysiologic mechanism explaining patients' clinical findings is used to select diagnoses for the differential) [11, 12] . compare-contrast approach codes were characterized as such if a task involved a direct comparison of the simulated patients' features to the features of known diseases. next, we analyzed the learning content of students' reports using the same exploratory sequential mixed method approach described above. once coded, we identified which of these topics had been covered previously in the curriculum prior to this simulation event by comparing this list of topics with students' learning materials (e.g., lecture slides, notes, assigned readings) from all courses leading up to this simulation. we also met with applicable course directors to confirm which content was new versus previously encountered. topics not identified in students' prior learning materials, or those that were only briefly mentioned but not covered in detail, were considered new learning content. finally, we analyzed the resources students cited from their self-directed learning in a similar fashion, organized them into categories of source type (specific websites, uptodate®, journal articles, textbooks, or other online sources), and calculated the citation frequency of these sources, among all sources cited in students' reports. twenty student team reports were collected and analyzed (139 total students). we identified 16 codes for problemsolving tasks, organized into themes by the source course or discipline; these findings are summarized in table 1 . these task codes demonstrated that students integrated knowledge and skills from a variety of disciplines in order to address their assigned problem-solving tasks, including virology, epidemiology, clinical skills, and bioethics. eleven of the above 16 problem-solving task codes were identified as diagnostic reasoning task codes. when these codes were categorized according to the diagnostic reasoning approach used to formulate the differential diagnosis, all (100%) student team reports used a compare-contrast approach to identify candidate diseases for their differential diagnoses, and a subset (4, 20%) of team reports also included a causal approach to diagnostic reasoning. among the virology topics cited in students' reports (table 2) , most had been encountered previously in the microbiology course prior to this simulation event, though several viruses and viral disease treatments mentioned in the reports represented new learning content. aspects of disease states (e.g., disseminated intravascular coagulation, shock) mentioned in students' reports had been introduced previously, but management of these conditions represented new content. as for epidemiology topics, the biostatistics topics and the disease surveillance, transmission, and prevention measures concepts explicitly mentioned in students' reports had all been encountered previously, though management of large-scale disease outbreaks at the community and healthcare system levels represented new content. among the sources selected for self-directed learning cited by students' reports, we identified 6 codes for source types, listed in table 3 . with the exception of 1 hard-copy textbook cited in 1 team report, all sources cited were externally derived, online resources. analysis of students' hypotheses and treatment recommendations for this simulated viral pandemic activity provided data demonstrating evidence of cognitive integration of both previously encountered learning content from multiple • identifies similarities between unknown pathogen's cellular/molecular structure and diseases with similar features a 95% (19) • rationale for recommended treatments linked to diseases in the differential diagnosis for this case 95% (19) • identifies similarities between patients' physical exam findings and diseases with similar presenting exam findings a 90% (18) • identifies similarities between patients' geographic location and those of the diseases in the differential diagnosis a 70% (14) • rationale for recommended treatments linked to suspected pathophysiologic mechanism occurring in patients in this case 70% (14) • identifies similarities between patients' lab findings and diseases with similar presenting lab findings a 60% (12) • identifies direct exposure risks among patients related to risk factors for the diseases in the differential diagnosis for this case a 40% (8) • explains abnormal clinical findings based on pathophysiologic mechanisms of diseases in the differential diagnosis b 20% (4) tasks applying clinical skills content (data collection, communication skills) % reports (n) • identifies similarities between patients' symptoms and diseases with similar presenting symptoms a 100% (20) • identifies similarities between patients' illness narrative (tempo, timing, and sequence of symptoms) and diseases with similar presenting symptoms a 80% (16) tasks applying epidemiology content % reports (n) • rationale for recommended public health measures linked to transmission route(s) of the diseases in the differential diagnosis for this case 75% (15) • identifies similarities between likely transmission route and that of diseases in the differential diagnosis, based on interpretation of epidemiologic data a 45% (9) • identifies groups with higher vs. lower risk of mortality, based on interpretation of epidemiologic data 45% (9) • identifies groups with higher vs. lower risk of acquiring disease, based on interpretation of epidemiologic data a 15% (3) tasks applying ethics content % reports (n) • rationale for recommended resource allocation explicitly linked to relevant, sound ethical principles 95% (19) a diagnostic reasoning task codes using a compare-contrast approach b diagnostic reasoning task codes using a causal approach disciplines and newly acquired learning content. these reports also provided insights into first-year medical students' diagnostic reasoning approach, which consisted primarily of a compare-contrast process between the clinical features of patients in the case scenario versus illness scripts of known diseases, though a subset of students also applied a causal approach to identify candidate diseases. our findings were somewhat surprising given that, over the preceding initial 6 months of medical school, the medical students in our study had been participating in a longitudinal case-centered, pbl course that had routinely tasked them with using a causal (i.e., pathophysiologic analytical) approach to differential diagnosis formulation. in addition, previous studies on diagnostic reasoning among medical students indicate these novice learners tend to utilize a causal approach [11] [12] [13] [14] [15] . however, in our study, only a small proportion of student reports alluded to using a causal approach. these contrasting findings are likely in part due to the fact that most prior studies were conducted on students at more advanced stages of training than that of our learner group. the use of a compare-contrast approach we observed among students' reports following the pandemic simulation is likely explained by the fact that their biomedical knowledge base was not yet sufficient at that point in their training to allow them to analyze the clinical data of the pandemic case with a causal approach. indeed, these students had not yet been exposed to much of the pathophysiology implicated by the signs and symptoms of the patients in the simulated case scenario (excessive bleeding and aggression). our findings may be explained by the search-inference framework proposed by aberegg et al., which is not unlike what patients often do to investigate their own symptoms [30] . in this framework, novice problemsolvers-motivated by a specific goal-develop a list of possibilities by seeking their memory or external sources and then use inference to compare and evaluate these possibilities by seeking evidence for and against each one, again using their memory or external sources. the problem-solver then selects the solution mostly likely to satisfy the goal, based on his/her evaluation of the evidence. however, as aberegg and colleagues point out, such an approach-though practical for the novice-is susceptible to many types of cognitive biases that, left unchecked, could easily lead the clinician astray. without an adequate understanding of the underlying pathophysiological mechanisms of disease, simply using a compare-contrast approach is not likely to be a successful long-term diagnostic strategy for novices. in fact, understanding and connecting the underlying causal mechanisms to their corresponding clinical manifestations of disease has been shown to be beneficial to the diagnostic reasoning performance of novice learners, with improved long-term memory retention of diagnostic categories and superior diagnostic accuracy when solving difficult cases [5] [6] [7] . as novices' biomedical knowledge base continues to expand during their preclinical training, practicing diagnostic reasoning through challenging cases, such as the one presented in our simulated pandemic activity, has benefits for further developing their mental representations of disease. chamberland and colleagues found that challenging cases appear to activate medical students' biomedical knowledge by stimulating pathophysiological inferences during self-explanation [18] . other studies have demonstrated the benefit of self-explanation during diagnostic reasoning, resulting in improved diagnostic accuracy of complex cases, even without receiving feedback on one's clinical reasoning [17, [31] [32] [33] . self-explanation is thought to improve learners' understanding of the learning material by activating cognitive processes that ultimately strengthen the cognitive connections between elements of the information learned [32] . some investigators have found that the use of specific prompts can enhance the effectiveness of learners' selfexplanations on learning [19, 34, 35] : justification prompts (prompts to justify one's reasoning with underlying principles and concepts) are thought to trigger learners' biomedical knowledge by focusing them on the underlying pathophysiological mechanisms; by contrast, mental model revision prompts (prompts to compare and contrast one's existing knowledge to new knowledge) are thought to help students revise and make corrections to their existing knowledge. despite the fact that justification prompts were included in the report template and student instruction documents for the simulated pandemic activity, our students' written diagnostic justifications did not include much discussion of pathophysiological mechanisms. again, this discrepancy may be due to our more novice learner group and their limited biomedical knowledge at the time of the simulation. this study provided insights into how preclinical medical students select sources of information during their self-directed learning, which is consistent with the findings of several prior studies. graber and colleagues found that third-year medical students use electronic resources extensively during problemsolving, particularly google's search engine; online journal articles and other clinical decision tools such as uptodate® were used much less frequently. focus group findings from their study suggested potential reasons for these source usage patterns related to students' familiarity with google and their inability to assess the relative efficacy of online decision-support resources [36] . more recent studies have shown similar findings, with google, wikipedia, and social media being the information-seeking resources most frequently used by contemporary medical students, whereas online journals, scholarly databases, and medical texts are accessed relatively infrequently [37] [38] [39] . features such as ease of use and efficiency of access appear to heavily influence today's medical students' choices of such resources [39] [40] [41] . limitations to this study include that it was a single institution study, and the generalizability of our findings may be limited by the specific case scenario and learning activity used in this study. student team reports were submitted as a group and therefore may not fully reflect every individual student member's diagnostic justification performance. the 4-hour time limitation for the simulated pandemic activity may have affected students' performance of the assigned tasks, particularly with respect to their selected information-seeking behaviors. in addition, students' actual diagnostic reasoning approaches while working in their small groups may have included other strategies beyond those documented in their reports; their use of causal inferences, for example, could have been more prevalent than what was reflected in their written reports. similarly, students' information sources during self-directed learning may have included other sources beyond those cited in their reports. this study provides insight into early medical students' diagnostic reasoning approach and information-seeking behaviors during self-directed learning tasks. future study is needed to determine if the same behaviors we observed in this study would occur in this learner group when presented with other clinical problem-solving scenarios. such data would add further support to the search-inference framework hypothesis and could have important implications for clinical reasoning curricula for learners at this stage in medical school training. collaborative, task-based problem-solving during a simulated pandemic can provide preclinical medical students opportunities to integrate learning content from multiple disciplines. evidence of learners' cognitive integration can be made explicit through their narrative justification of problem-solving tasks. patterns of learners' task performance can provide important insights into their problem-solving, clinical reasoning, and self-directed learning approaches. our study suggests that early medical students tend to rely on a compare-contrast approach to diagnostic reasoning similar to the search-inference framework described by previous authors. students' sources of information for self-directed learning were consistent with findings of other studies of this generation of digital native learners. these findings could have implications for clinical reasoning curricular design for these early learners. author's contribution all authors contributed to the study conceptualization and design. material preparation and data collection were performed by jennifer jackson and timothy peters. data analysis was performed by jennifer jackson, timothy peters, and joseph skelton. the first draft of the manuscript was written by jennifer jackson, and all authors commented on previous versions of the manuscript. all authors read and approved the final manuscript. ethical approval all procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee (wake forest school of medicine institutional review board, irb00050949) and with the 1964 helsinki declaration and its later amendments or comparable ethical standards. cognition before curriculum: rethinking the integration of basic science and clinical learning integration of basic sciences and clinical sciences in oral radiology education for dental students the role of basic sciences in diagnostic oral radiology cause and effect: testing a mechanism and method for the cognitive integration of basic science exploring cognitive integration of basic science and its effect on diagnostic reasoning in novices science is fundamental: the role of biomedical knowledge in clinical reasoning it all make sense: biomedical knowledge, causal connections and memory in the novice diagnostician the value of basic science in clinical diagnosis: creating coherence among signs and symptoms the value of basic science in clinical diagnosis expanding the basic science debate: the role of physics knowledge in interpreting clinical findings a universal model of diagnostic reasoning how expertise develops in medicine: knowledge encapsulation and illness script formation a cognitive perspective on medical expertise: theory and implication the role of basic science knowledge and clinical knowledge in diagnostic reasoning: a structural equation modeling approach development of clinical reasoning in an undergraduate medical program at a brazilian university adjusting our lens: can developmental differences in diagnostic reasoning be harnessed to improve health professional and trainee assessment? examining the effect of selfexplanation on cognitive integration of basic and clinical sciences in novices students' self-explanations while solving unfamiliar cases: the role of biomedical knowledge self-explanation in learning clinical reasoning: the added value of examples and prompts examining the diagnostic justification abilities of fourth-year medical students illness script development in pre-clinical education through case-based clinical reasoning training to think is good: querying an initial hypothesis reduces diagnostic error in medical students concept mapping enhances learning of biochemistry assessing the effectiveness of using mechanistic concept maps in case-based collaborative learning the use of simulation in teaching the basic sciences early bedside care during preclinical medical education: can technology-enhanced patient simulation advance the flexnerian ideal the role of causality in judgment under uncertainty causal learning and inference as a rational process: the new synthesis analyzing qualitative data: systematic approaches the searchinference framework: a proposed strategy for novice clinical problem solving the influence of medical students' selfexplanations on diagnostic performance self-explaining expository texts: the dual processes of generating inferences and repairing mental models the effects of self-explanation training on students' problem solving in high-school mathematics learning argumentation skills through the use of prompts for self-explaining examples testing the instructional fit hypothesis: the case of self-explanation prompts resources medical students use to derive a differential diagnosis selection and use of online learning resources by first-year medical students: cross-sectional study informationseeking behaviors of medical students: a cross-sectional web-based survey expediency-based practice? medical students' reliance on google and wikipedia for biomedical inquiries qualified doctor and medical students' use of resources for accessing information: what is used and why? project information literacy: what can be learned about the information-seeking behavior of today's college students publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations the authors declare that they have no conflicts of interest. key: cord-289451-yjrh5l4u authors: xiao, huidi; shu, wen; li, menglong; li, ziang; tao, fangbiao; wu, xiaoyan; yu, yizhen; meng, heng; vermund, sten h.; hu, yifei title: social distancing among medical students during the 2019 coronavirus disease pandemic in china: disease awareness, anxiety disorder, depression, and behavioral activities date: 2020-07-14 journal: int j environ res public health doi: 10.3390/ijerph17145047 sha: doc_id: 289451 cord_uid: yjrh5l4u background: during the coronavirus disease (covid-19) pandemic, harsh social distancing measures were taken in china to contain viral spread. we examined their impact on the lives of medical students. methods: a nation-wide cross-sectional survey of college students was conducted from 4–12 february 2020. we enrolled medical students studying public health in beijing and wuhan to assess their covid-19 awareness and to evaluate their mental health status/behaviors using a self-administered questionnaire. we used the patient generalized anxiety disorder-7 and health questionnaire-9 to measure anxiety disorders and depression. we used multivariable logistic regression and path analysis to assess the associations between covariates and anxiety disorder/depression. results: of 933 students, 898 (96.2%) reported wearing masks frequently when going out, 723 (77.5%) reported daily handwashing with soap, 676 (72.5%) washed hands immediately after arriving home, and 914 (98.0%) reported staying home as much as possible. prevalence of anxiety disorder was 17.1% and depression was 25.3%. multivariable logistic regression showed anxiety to be associated with graduate student status (odds ratio (aor) = 2.0; 95% confidence interval (ci): 1.2–3.5), negative thoughts or actions (aor = 1.6; 95% ci: 1.4–1.7), and feeling depressed (aor = 6.8; 95% ci: 4.0–11.7). beijing students were significantly less likely to have anxiety than those in the wuhan epicenter (aor = 0.9; 95% ci: 0.8–1.0), but depression did not differ. depression was associated with female students (aor = 2.0; 95% ci: 1.2–3.3), negative thoughts or actions (aor = 1.7; 95% ci: 1.5–1.9), and anxiety disorder (aor = 5.8; 95% ci: 3.4–9.9). path analysis validated these same predictors. conclusions: despite medical students’ knowledge of disease control and prevention, their lives were greatly affected by social distancing, especially in the wuhan epicenter. even well-informed students needed psychological support during these extraordinarily stressful times. coronavirus disease (covid-19) is a public health emergency of worldwide concern [1] . up through june 2020, covid-19 had affected 213 countries, territories or areas, and two international conveyances (cruise ships) [2] since its initial december 2019 report as an "unknown pneumonia" in wuhan, hubei province, china [3] . infections and deaths increased rapidly with global travel fueling transmission spread worldwide, first to other parts of china, south korea, and iran, and soon thereafter to europe and the united states. other nations such as russia and brazil then experienced expanding case numbers while infections in africa have also been mounting. to interrupt further transmission, many chinese provinces and cities suspended public transportation [4] and even locked down cities altogether. these were unprecedented and harsh social distancing measures and they affected every aspect of daily life in china, as elsewhere. "social distancing" is done to reduce close physical interactions (<2 m) between people to avoid viral transmission. chinese measures included closure of schools, office buildings, and public markets; events were cancelled, and gatherings discouraged [5] . the united nations educational, scientific and cultural organization (unesco) estimates that nationwide closures in more than 160 countries have affected over 87% of students in the world, with wider closures affecting even more [6] . a 2009 influenza a (h1n1) pandemic survey in hong kong demonstrated marked anxiety associated with distancing [7] . a 2020 covid-19 survey in china found that many respondents (including students) reported symptoms of moderate to severe anxiety and depression [8] . medical students are a reservoir for the future health labor force. in the early response in the epicenter in china, as well as other countries, the sudden outbreak overwhelmed health professional's preparedness in terms of personal protective equipment (ppe) shortages and psychological readiness [9, 10] . it is critical to understand how much the pandemic affects this future health labor force. we sought to understand the psychological effects of distancing measures and possible effects on medical student wellness. even without pandemic stresses, medical students may have some anxiety and depression due to their high study-related or job-seeking burdens [11] [12] [13] . we studied the associations between covid-19 distancing and the lives of public health students on anxiety [14] , depression, and other behaviors. we also examined the differences among public health students in two universities, capital medical university (ccmu) in beijing, and huazhong university of science and technology (hust) in wuhan, the early epicenter of the covid-19 outbreak in china. we conducted a cross-sectional survey among the medical students in two schools of public health at two universities: ccmu in beijing and hust in wuhan. the present study is part of a nation-wide cross-sectional survey of college students conducted from 4-12 february 2020. we developed a self-administrated, 84-item questionnaire focused on: (1) covid-19 awareness; (2) personal protective measures; (3) mental health status; and (4) behavioral changes in the past 14 days. our instrument was based on social-cognitive theory and the theory of planned behavior and we measured knowledge, attitude, belief, and practice (kabp) [15] , enhanced with psychological assessments. we used the validated generalized anxiety disorder-7 (gad-7) and patient health questionnaire-9 (phq-9) to assess anxiety disorder and depression, respectively [16, 17] . most questions were one-choice, brief, and easy to answer, such that it took only 10 min on average to complete the survey. we used wenjuanxing ® (changsha haoxing information technology co., ltd., changsha, china) software for the informed consent process and the online survey questionnaire. participants obtained and completed the self-administrated questionnaire on the mobile phone by scanning the qr code using wechat ® software. after consent was obtained, we detailed the workflow of the survey with one page of text. the investigators received the completed questionnaires via the software platform. facilitated by their teachers, we approached all medical students with public health majors in the ccmu and hust schools of public health. the study was approved by ethics review board of capital medical university (2020sy004) and anhui medical university (20200319). informed consent and a response to the questionnaire were obtained from 933 of 1061 (87.9%) students who were approached. hence, we could infer the different preventive knowledge, behaviors, and psychological status modified by distance to the epicenter amid social distancing in the pandemic. covariates included socio-demographic characteristics, knowledge of covid-19, personal protective measure, behaviors, and degree of worry about the virus. variables included sex, age, year of study, university, living quarters, knowledge about the covid-19 incubation period, mortality, susceptibility (e.g., chronic illness), drugs taken, mask wearing, face and hand hygiene, times and reasons for going out, room cleanliness and ventilation via windows, and concerns about the covid-19 epidemic and contracting the virus. we also asked about negative thoughts or actions ("always feel dirty", "feel uneasy in a crowded place", "often suspect being infected", "worse appetite than before", "feel less energic than before", "hold unhappy intentions in my heart" and "angry with others when in a bad mood"), positive thoughts or actions ("accept the truth when facing an obstacle" and "relieve pain in a positive way"), healthy lifestyles ("work and rest regularly", "arise regularly", "sleep regularly" and "have meals regularly"), video screen time per day, and the number of anger episodes or quarrels in the past week. principal outcomes were anxiety disorder and depression measured by scale scores of gad-7 and phq-9, respectively. the gad-7 scale score was divided into four categories: normal (0-4), mild (5-9), moderate (10) (11) (12) (13) (14) , and severe (15) (16) (17) (18) (19) (20) (21) . the phq-9 scale score was divided into five categories: normal (0-4), mild (5-9), moderate (10) (11) (12) (13) (14) , moderate to severe (15) (16) (17) (18) (19) , and severe (20) (21) (22) (23) (24) (25) (26) (27) . if a participant's score was ≥5 points (i.e., mild or above), we considered the student to have evidence of anxiety disorder. we used descriptive statistics including the chi-squared test for the associations of covid-19 knowledge with sociodemographic characteristics and use of personal protective measures. we used multivariable logistic regression to examine the association between independent variables and covariates with anxiety disorder and/or depression. we deployed the hosmer-lemeshow test to determine the goodness-of-fit of the logistic regression model. only variables with two-sided p ≤ 0.05 were deemed significant in the final model. we used path analysis to determine the interplay of covariates with anxiety disorder and depression via a structural equation model. we used maximum likelihood estimation and assessed the goodness-of-fit by absolute fit indices that determine how well the a priori model predicts the actual data, including the root mean square error of approximation (rmsea), goodness of fit index (gfi), and adjusted goodness of fit index (agfi). we also used incremental or relative fit indices, specifically the incremental fit indices (ifi), comparative fit indices (cfi), normed fit index (nfi), and the non-normed fit tucker-lewis index (tli). rmsea < 0.10 and gfi and agfi > 0.90 indicate the model fits well. the incremental fit measures-cfi, nfi, ifi, and tli-are >0.90 when the model fits well. we used spss statistic ® 21.0 and spss amos ® 26.0 graphics software (ibm spss statistics, new york, ny, usa). of 933 participated students, 558 students attended ccmu (94.4% agreed to participate) in beijing, and 375 students attended hust (79.6% agreed) in wuhan. graduate students doing masters of public health degrees did better than undergraduates in the knowledge questions that were answered correctly (p = 0.037). it is important to note that the chinese system has undergraduate medical students and before graduation, medical students can pursue specialties such as public health, pediatrics, etc.; then at the graduate level they may choose to major in epidemiology and statistics, environmental health, etc., if they chose the public health specialty. the prevalence of anxiety disorder (p = 0.015) and depression (p < 0.001) in women was significantly higher than in men. anxiety disorder was higher in wuhan students than ccmu students (p = 0.001; table 1 ). the correct answers to the four knowledge questions were given by 97.2%, 67.1%, 69.3%, and 89.4% of respondents (table s1 ). the awareness of mortality risk was higher in hust students than ccmu students (p = 0.04). as to preventive measures and behaviors during social distancing, 898 (96.2%) reported wearing masks frequently when going out, 869 (93.1%) washed their hands with water regularly, 723 (77.5%) washed their hands with soap every day, 676 (72.5%) washed their hands immediately after arriving home, 239 (25.6%) considered it difficult to wash their hands for at least 20 s, 295 (31.6%) washed their hands for more than 20 s frequently, 914 (98.0%) avoided unnecessary outings (i.e., they tried to stay at home as much as possible), and 878 (94.1%) kept clean, well-ventilated rooms. beijing-based ccmu students were significantly more likely to report wearing masks (p = 0.037), avoiding touching their mouths, noses, and eyes with their hands (p < 0.001), washing their hands immediately after arriving home (p < 0.001), and handwashing for at least 20 s (p < 0.001). wuhan-based students at hust were more likely to report washing their hands with soap (p < 0.001), staying at home (p = 0.028), and keeping their rooms clean and well-ventilated (p = 0.044; table 2 ). assessing anxiety disorder, 773 (82.9%) were classified as normal, 117 (12.5%) had mild anxiety, 30 (3.2%) had moderate anxiety, and 13 (1.4%) had severe anxiety disorder. assessing depression, 697 (74.7%) students were classified as normal, 165 (17.7%) had mild depression, 43 (4.6%) had moderate depression, 18 (1.9%) had moderate to severe depression, and 10 (1.1%) had severe depression. the prevalence of anxiety disorder differed between the two universities, and was significantly higher in wuhan (p = 0.001) which was far more severely affected by covid-19. the prevalence of depression between the two universities was also higher in wuhan, but this may have been due to chance (p = 0.12; table 1 and figure 1 ). bar charts present the distribution of different degrees of anxiety disorder and depression, comparing students at capital medical university (ccmu) in beijing with students at huazhong university of science and technology (hust) in wuhan. the x-axis represents the different degrees of anxiety disorder and depression, and the y-axis represents the proportion of students. bar charts present the distribution of different degrees of anxiety disorder and depression, comparing students at capital medical university (ccmu) in beijing with students at huazhong university of science and technology (hust) in wuhan. the x-axis represents the different degrees of anxiety disorder and depression, and the y-axis represents the proportion of students. multivariable logistic regression shows that being a graduate student (adjusted odds ratio (aor) = 2.03; 95% confidence interval (ci): 1.18-3.49; p = 0.011), having negative thoughts or actions (aor = 1.55; 95% ci: 1.38-1.73; p < 0.001), and feeling depressed (aor = 6.84; 95% ci: 4.00-11.71; p < 0.001) were associated with a higher likelihood of anxiety. students at ccmu, far from the wuhan epicenter, were less likely to experience anxiety (aor = 0.90; 95% ci: 0.82-1.00; p = 0.049; table 3 ). women students (aor = 1.98; 95% ci: 1.19-3.29; p = 0.009), persons having negative thoughts or actions (aor 1.68; 95% ci: 1.50-1.88; p < 0.001), and persons with anxiety (aor = 5.81; 95% ci: 3.43-9.86; p < 0.001) had higher odds of having some depression. having a healthy lifestyle was associated with less depression (aor = 0.88; 95% ci: 0.79-0.97; p = 0.013). a includes: "always feel dirty", "feel uneasy in a crowded place", "often suspect being infected", "worse appetite than before", "feel less energic than before", "hold unhappy intentions in my heart", and "angry with others when in a bad mood". b includes: "accept the truth when facing obstacles", and "relieve pain in a positive way". c includes: "work and rest regularly", "arise regularly", "sleep regularly", and "have meals regularly". across the two sites, 426 (48.7%) students reported using computers or other electronic devices over 4 h daily, 484 (51.9%) used their cellphones over 4 h daily, 635 (68.1%) woke up later than usual, 234 (24.0%) went to bed later than usual, 234 (35.1%) worked and rested irregularly, 157 (16.8%) had meals irregularly, and 221 (23.7%) ate different volumes of food from usual (either more or less). within one week before the survey, 306 (32.8%) reported having gotten angry at others, 201 (21.5%) reported one or more quarrels, 585 (62.7%) felt terrible because of the epidemic, and 112 (12.0%) quarreled with others online. figure 2 shows the factors relevant to anxiety disorder and depression, and table s2 presents standardized estimation of coefficient values. students in their senior grade year (β = 0.074) suffered more from anxiety disorders. negative thoughts or actions were associated with depression (β = 0.86) while healthy lifestyles (β = −0.077) were negatively associated with depression. longer video screen time (β = −0.24) negatively affected a healthy lifestyle. concerns about the covid-19 epidemic were associated with more negative thoughts or actions (β = 0.23) and anger and quarreling behaviors (β = 0.20); negativity and anger/quarrels were correlated (β = 0.34). overall, negative thoughts or actions predicted a higher impact of both anxiety disorders (β = 0.87) and depression (β = 0.86). goodness-of-fit indices for the model were good and key parameters were rmsea = 0.051, gfi = 0.93, agfi = 0.91, cfi = 0.93, nfi = 0.91, ifi = 0.93, and tli = 0.92. standardized estimation of coefficient values. students in their senior grade year (β = 0.074) suffered more from anxiety disorders. negative thoughts or actions were associated with depression (β = 0.86) while healthy lifestyles (β = −0.077) were negatively associated with depression. longer video screen time (β = −0.24) negatively affected a healthy lifestyle. concerns about the covid-19 epidemic were associated with more negative thoughts or actions (β = 0.23) and anger and quarreling behaviors (β = 0.20); negativity and anger/quarrels were correlated (β = 0.34). overall, negative thoughts or actions predicted a higher impact of both anxiety disorders (β = 0.87) and depression (β = 0.86). goodnessof-fit indices for the model were good and key parameters were rmsea = 0.051, gfi = 0.93, agfi = 0.91, cfi = 0.93, nfi = 0.91, ifi = 0.93, and tli = 0.92. our survey findings suggest that the awareness of medical students of covid-19 fundamentals was very high, as might be expected given the profile in chinese society by february 2020. anxiety and depression were common among medical students in whom social distancing was reported with longer video screen time and less healthy lifestyles. concern about the epidemic was associated with negative actions and thoughts, which were, in turn, associated with an increased likelihood of anxiety disorders. the hust medical students in wuhan, the epicenter, presented with higher anxiety than our survey findings suggest that the awareness of medical students of covid-19 fundamentals was very high, as might be expected given the profile in chinese society by february 2020. anxiety and depression were common among medical students in whom social distancing was reported with longer video screen time and less healthy lifestyles. concern about the epidemic was associated with negative actions and thoughts, which were, in turn, associated with an increased likelihood of anxiety disorders. the hust medical students in wuhan, the epicenter, presented with higher anxiety than did ccmu medical students in beijing. the findings underscore our study's significance that the health reservoir suffer from psychological stress and need attention. less than half (43.7%) of the medical students had fully correct (four out of four questions) knowledge of the covid-19 epidemic situation. students, all of whom were studying public health, were less knowledgeable about mortality and susceptible groups, compared to incubation period and available drugs (table s1 ). over 90% of students complied with social distancing and effective preventive measures, like wearing masks frequently when going out, washing hands with water regularly, avoiding unnecessary outings (i.e., staying at home), and keeping their rooms clean and ventilated [18] . according to our findings, students preferred washing their hands with only water to also using soap; about half (54%) of students tried to avoid touching their mouth, nose, and eyes with their hands, though both behaviors can reduce the risk of exposure [19] . a quarter (25.6%) of the students considered it hard to wash their hands for at least 20 s and less than a third (31.6%) washed their hands over 20 s frequently. given that good hand hygiene can effectively prevent virus transmission [20] and washing hands for at least 20 s is a basic component of hand hygiene as per the world health organization (who, geneva, switzerland), it was disappointing to learn from medical students that they found this challenging [21] . it is likely that, comparing the two participating universities, notable differences were fueled by the differing epidemiologic context of the two urban venues. among the 44.0% of students from hust who lived in the epicenter of hubei province or adjacent provinces, they were more likely to obey harsher social distancing rules, like staying at home and deploying hand hygiene. while most students (88.2%) from ccmu lived far away from hubei province, they reported being more fastidious in wearing masks. during social distancing, our study showed 17.1% of students had anxiety disorder symptoms (mainly mild), and 25.3% of students had depression symptoms (mainly mild). an interesting survey in 190 chinese cities surveyed the general population twice: during the initial outbreak and during the epidemic's peak four weeks later [22] . they surveyed demographics, symptoms, knowledge, concerns, and precautionary measures against covid-19. among the 333 persons who took both surveys, post-traumatic stress disorder (ptsd by the mean impact of event scale-revised (ies-r) scale scores) declined with time, but at both surveys, the mean ies-r scores of the first-and second-survey respondents were above the cut-off scores (>24) for ptsd symptoms, suggesting that the reduction in scores was not of clinical significance [22] . previous studies report that student populations can be more vulnerable towards stress-related anxiety and depression [22, 23] . our findings underscore the importance of providing essential psychological support to students, even when they are as well-informed as are medical students in a public health track. both multivariable logistic regression and path analysis reinforced the findings of principal factors related to anxiety disorder and depression. compared to undergraduates, graduate students had a higher risk of anxiety [13, 24] . we think that this may be due to increased pressure for job-seeking or completion of thesis required for graduation. females were more likely to be depressed compared to men, consistent with prior studies [24, 25] . healthy lifestyles were negatively associated with depression, compatible with other findings that healthy lifestyles can improve mental health [26] . some students adopted an unhealthy lifestyle during social distancing, reflecting difficulties in adjusting to domestic life for a prolonged time; as expected, many students reported bad moods and/or behaviors [27] . nearly a quarter of students (23.7%) changed their diets by eating more or less than usual. either excessive or poor appetites can be symptoms of depression [28] . longer video screen time had an indirect impact on depression, as other studies reported [29, 30] . half of our participants spent over four hours on electronic devices; aside from increasing risk for depression, this can negatively affect vision, or spawn weight gain and cardiovascular risk from adoption of a sedentary life [31] . we found that the prevalence of anxiety disorder was higher in wuhan than in beijing (p = 0.001), while depression was also somewhat higher (p = 0.12). research on college students from changzhi medical college (shanxi province, about midway between wuhan and beijing) in the same time period reported that 24.9% of students had anxiety disorder using screening criteria similar to ours [32] . it is unclear whether differences in reported prevalence are related to school location, though it is plausible that proximity to the pandemic's epicenter would be more anxiety-provoking [33] . that negative thoughts or actions, anger and quarreling behaviors, and concerns about covid-19 were all positively related to either depression or anxiety disorder has been seen in other studies [32, 34, 35] . many of our participants reported that they had been angry and quarreling within one week before the survey; 46.7% of them felt terrible because of covid-19. it is plausible that irritable behaviors suggest that mental status or moods may have been affected by social distancing due to lack of normal social activities during the pandemic [36] . as expected, we found a strong positive interaction effect between anxiety disorder and depression. reduced physical activity may increase anxiety or depression, but we did not find a clear association in our survey. strengths of the study include its uniqueness, a survey in two sites (including wuhan) conducted just two months after the report of the pulmonary syndrome and one month after recognition of sars-cov-2 circulation in china. limitations include the cross-sectional survey design. while we could compare sites and assess predictors of anxiety and depression, we could not assess mental health circumstances before the emergence of the virus and therefore cannot infer temporality, the vital element to assess causality. the participants are medical students studying public health from two universities; therefore, results and conclusions cannot be generalized to other populations. moreover, because we used self-rating scales, the frequency of anxiety disorder and depression symptoms self-reported by the students is less reliable than thorough clinical diagnoses. as an emerging disease, our understanding of covid-19 keeps evolving and we selected "knowledge questions" based on what was known in late january 2020, based on several rounds of expert consultations and consistent with who and us centers for disease control and prevention updates. the covid-19 pandemic has posed an unprecedented impact on the lives of medical students. necessary psychological support was not available to them during the time of social distancing beyond an awareness campaign regarding preventive measures. it is likely that more economically vulnerable persons would have stress levels exceeding that of medical students, but we learned that this well-informed population was nonetheless in substantial distress in the context of epidemic concerns. mental health pressures in medical students are likely compounded among practicing clinicians; these professionals need psychosocial assistance at the time of pandemic mental stress. we recommend incorporating pandemic preparedness education within health education, including mental health elements, especially within the healthcare labor force. supplementary materials: the following are available online at http://www.mdpi.com/1660-4601/17/14/5047/s1, table s1 : preventive knowledge of covid-19, comparing correct answer frequencies between the two universities (n = 933), table s2 the sponsors had no role in the study design; in the collection, analyses, and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication. china coronavirus: who declares international emergency as death toll exceeds 200 who. coronavirus disease (covid-19) situation report the continuing 2019-ncov epidemic threat of novel coronaviruses to global health-the latest 2019 novel coronavirus outbreak in wuhan offline: 2019-ncov-"a desperate plea isolation, quarantine, social distancing and community containment: pivotal role for old-style public health measures in the novel coronavirus (2019-ncov) outbreak impact of sars-cov-2 and its reverberation in global higher education and mental health community psychological and behavioral responses through the first wave of the 2009 influenza a(h1n1) pandemic in hong kong immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (covid-19) epidemic among the general population in china personal protective equipment (ppe) for surgeons during covid-19 pandemic: a systematic review of availability, usage, and rationing covid-19: present and future challenges for dental practice depression, stress and anxiety in medical students: a cross-sectional comparison between students from different semesters prevalence of mental health problems among medical students in china: a meta-analysis a systematic review of depression and anxiety in medical students in china patient education in sudden sensorineural hearing loss: knowledge, attitude/belief, and practice findings among otolaryngologists and otologists in china validation and standardization of the generalized anxiety disorder screener (gad-7) in the general population the patient health questionnaire somatic, anxiety, and depressive symptom scales: a systematic review the epidemiological and clinical features of covid-19 and lessons from this global infectious public health event epidemiology, causes, clinical manifestation and diagnosis, prevention and control of coronavirus disease (covid-19) during the early outbreak period: a scoping review masks and coronavirus disease 2019 (covid-19) hand hygiene teaching strategies among nursing staff: a systematic review a longitudinal study on the mental health of general population during the covid-19 epidemic in china covid-19 and mental health: a review of the existing literature study on the public psychological states and its related factors during the outbreak of coronavirus disease 2019 (covid-19) in some regions of china a nationwide survey of psychological distress among chinese people in the covid-19 epidemic: implications and policy recommendations lifestyle choices and mental health: a longitudinal survey with german and chinese students nutrition and lifestyle intervention on mood and neurological disorders appetite changes in depression is screen time associated with anxiety or depression in young people? results from a uk birth cohort mental health problems and social media exposure during covid-19 outbreak sedentary behaviour and risk of all-cause, cardiovascular and cancer mortality, and incident type 2 diabetes: a systematic review and dose response meta-analysis the psychological impact of the covid-19 epidemic on college students in china factors associated to depression and anxiety in medical students: a multicenter study associations between negative life events and anxiety, depressive, and stress symptoms: a cross-sectional study among chinese male senior college students relationship between intrinsic motivation and undergraduate students' depression and stress: the moderating effect of interpersonal conflict social integration, perceived stress, locus of control, and psychological wellbeing among chinese emerging adult migrants: a conditional process analysis acknowledgments: we thank qi fu and niu piye and dean rui chen for survey facilitation and all the students participating in the survey for their time. the authors declare no conflicts of interest. key: cord-349014-zkm2mph9 authors: wang, jessie j.; deng, aaron; tsui, ban c.h. title: covid-19: novel pandemic, novel generation of medical students date: 2020-06-02 journal: br j anaesth doi: 10.1016/j.bja.2020.05.025 sha: doc_id: 349014 cord_uid: zkm2mph9 nan united kingdom (uk) and worldwide, and while healthcare professionals fight frontline battles and systems leaders negotiate public health measures, the needs of medical students at various stages of training cannot be forgotten. preclinical students, removed from in-person sessions, have transitioned to rapidly adapted online curricula and assessments. final-year students' experiences differ widely by jurisdiction, ranging from early graduation and provisional registration to entering residency despite postponed board examinations. clinical students not in their final year face perhaps the greatest uncertainty, with limitations to and/or suspension of inpatient and outpatient rotations. as the pandemic evolves, so too must strategies to address the issue it raises for medical education. internationally, displacing medical trainees from clinical settings amidst an infectious disease outbreak has precedent in the severe acute respiratory syndrome (sars) outbreak of 2003. medical programmes had variable restrictions depending on the local extent of the pandemic, including interruptions to licensing examinations, human subject research, and clinical rotations. 1 in canada, clerkship placements saw a significant reduction in quantity and scope, and adaptations to the canadian residency application process were required even following containment of the crisis. 2 nevertheless, even in impacted areas around the world, medical school cohorts were able to receive accreditation as usual. with mounting caseloads and increasing strains on healthcare capacities, some jurisdictions have released final-year medical students into the workforce. the general medical council (gmc) is provisionally registering early graduates as foundation year 1 doctors. 3 association of american medical colleges (aamc) allows student involvement in patient care in locales with critical healthcare workforce shortage, 4 and medical schools in several states have offered students the option to graduate early and join the frontlines. harnessing medical students in the fight against covid-19 has the advantage, while addressing public health needs, of providing a learning environment with authentic patient experiences for students. many medical students deem it an obligation to contribute to the healthcare response during a pandemic and are eager to do so. considering that today's students will be tomorrow's doctors, it is arguably counterproductive to artificially shield them from the realities of medicine in crisis times. indeed, even off the wards, students are far from immune to the reverberations of the recently evolving medical landscape, as a profession so soughtafter for reasons of security and prestige is suddenly cast as a dangerous occupation. by being proactively incorporated into the healthcare team and allowed to step into the altruistic role that inspired them to pursue medicine, students are able to experience firsthand the role their specialty of interest plays in a medical crisis and make more informed decisions for their future. however, bringing students into hospitals at this time does have significant implications. although final-year medical students are mere months from the change in status to residents and interns, they are not yet bound by contractual obligation as residents are, thus requiring official policies from regulatory bodies. the uk has organised a nationwide, unified approach to deploying early graduates on an "opt in" basis, 3 whereas the aamc has provided guidance on students' participation in direct patient contact activities but left decisions to individual institutions. 4 in the early weeks of the pandemic, personal protective equipment (ppe) shortage, imperfect infection control curricula and adherence, and limited availability of covid-19 testing posed major safety concerns. students on the wards also increase supervisory demands on physicians, adding strain to already overwhelmed staff capacities. 5 students themselves face unprecedented pressures of high-stakes rapid training, high-acuity hospital environments, and potentially emotionally taxing patient encounters. there is no easy replacement for real-life patient interactions, however. clinical students are understandably concerned about their ability to achieve the competencies tested in clinical exit examinations and, more critically, their future finesse in the postgraduate environment. one solution for keeping students involved with patients is to engage them in the telemedicine systems already in place for physician providers. having been shown to improve equitable access to care, reduce costs, and increase patient satisfaction, telemedicine is sure to become a growing part of the futures of current medical students. while 60% of us medical schools included telemedicine as a component of required or elective courses in 2018, formal telemedicine training has yet to become mainstream. 8, 9 there is no better time than now to bridge the gap between medical education and the demands for telemedicine services, both to support virtual patient care during the pandemic, as one us institution is already doing, 10 and looking beyond to an increasingly technology-driven healthcare landscape. as the global medical community addresses and mitigates the immediate effects of the covid-19 pandemic, indirect impacts on medical education cannot be overlooked. whether or not jurisdictions harness students in the healthcare workforce to care for patients directly, their clinical training should not simply be put on hold. telemedicine can maintain a clinical learning environment through virtual interactions with patients and should be explored fully both for the present crisis and for meeting mounting demands in years ahead. now is the time to instate novel educational experiences to equip this generation of medical students for their future as physicians, beyond the pandemic. collateral damage: the effect of sars on medical education joint statement from the uk health departments, the general medical council, health education england, nhs education for scotland, health education and improvement wales, the northern ireland medical and dental training agency, and the medical schools cou guidance on medical students' participation in direct patient contact activities medical student education in the time of covid-19 statement of expectation: medical student volunteers in the nhs simulation-based learning: no longer a novelty in undergraduate education lack of telemedicine training in academic medicine: are we preparing the next generation from bedside to webside: future doctors learn how to practice remotely medical students train to support telehealth visits with patients the authors have no conflicts to declare. key: cord-319221-x834qscq authors: abuhegazy, mohamed; talaat, khaled; anderoglu, osman; poroseva, svetlana v. title: numerical investigation of aerosol transport in a classroom with relevance to covid-19 date: 2020-10-01 journal: phys fluids (1994) doi: 10.1063/5.0029118 sha: doc_id: 319221 cord_uid: x834qscq the present study investigates aerosol transport and surface deposition in a realistic classroom environment using computational fluid-particle dynamics simulations. effects of particle size, aerosol source location, glass barriers, and windows are explored. while aerosol transport in air exhibits some stochasticity, it is found that a significant fraction (24%–50%) of particles smaller than 15 µm exit the system within 15 min through the air conditioning system. particles larger than 20 µm almost entirely deposit on the ground, desks, and nearby surfaces in the room. source location strongly influences the trajectory and deposition distribution of the exhaled aerosol particles and affects the effectiveness of mitigation measures such as glass barriers. glass barriers are found to reduce the aerosol transmission of 1 µm particles from the source individual to others separated by at least 2.4 m by ∼92%. by opening windows, the particle exit fraction can be increased by ∼38% compared to the case with closed windows and reduces aerosol deposition on people in the room. on average, ∼69% of 1 µm particles exit the system when the windows are open. transmission of covid-19 occurs primarily through sars-cov2-laden droplets and aerosol particles inhaled directly or transmitted from contaminated surfaces. 1 effective mitigation measures necessitate clear understanding of droplet and aerosol transport, surface retention, and evaporation kinetics in different environments and conditions. 2 aerosols are generated during exhalation, talking, coughing, sneezing, and other activities. 3, 4 in indoor environments, some of the generated particles exit the system through ventilation, some deposit on surfaces in the room and may settle or re-enter the air, and others may be directly inhaled. of primary interest to mitigation measures is maximizing the fraction of particles that exit the system and minimizing aerosol deposition on people to reduce disease transmission. 5, 6 aerosol transport within a control volume is primarily affected by inertial forces due to airflow and drag on the particle, and gravitational sedimentation. 7 the forces acting on a particle primarily depend on particle size and its position in the flow field. for smaller particles (<0.5 μm), brownian force can play a significant role in aerosol transport but becomes less important with increased particle size. 7, 8 the velocity field of the fluid (air) under known boundary conditions can in principle be estimated by numerically solving navier-stokes equations through direct numerical simulations (dns), or more practically by numerically solving reynolds-averaged navier-stokes (rans) equations with approximate turbulence closures such as k-ϵ and k-ω closures. 9, 10 as particle properties significantly affect aerosol and droplet transport within a system, it is necessary to consider accurate particle shape, size, and evaporation kinetics. the distinction between aerosols and droplets is rather arbitrary with no general agreement on a particle size threshold or suspension time threshold. 3 however, droplets are typically considered to be larger particles where evaporation kinetics is rapid leading to the production of smaller aerosols with slow evaporation kinetics. 3 aerosol particles and droplets released from activities such as exhalation, talking, or coughing are polydisperse in nature. exhalation and talking release particles mostly <1 μm, 11 and coughing releases larger particles typically <10 μm, 12 while sneezing was found in one study to release particles characterized by a bimodal size distribution with peaks edge, no studies have investigated aerosol transport in a classroom environment although classroom sizes, the air conditioning layout, and aerosol source distribution are characteristically different than hospital care units and other indoor spaces discussed in the literature. while a typical 900 sq. ft classroom can fit 18 students and an instructor, guidelines for re-opening schools have restricted the number of students to less than 10 students with 6 ft minimum spacing between the students. the effectiveness of these measures is dependent in part on aerosol transport within the classroom's air conditioned environment, which remains under-characterized. other strategies for covid-19 mitigation may include the use of glass screens as barriers to reduce aerosol transport between people in the room, opening windows, and redistributing students in classrooms, but the ability of these measures to reduce aerosol transmission from one person to another needs to be carefully evaluated. the objective of the present work is to investigate aerosol transport and surface deposition in a model classroom environment using computational fluid-particle dynamics (cfpd) simulations. particularly, it is of interest to estimate the fraction of particles that exit the system, deposit on students, and deposit on surfaces such as desks, ground, walls, and ceiling. the effects of particle size, aerosol source location, glass barriers, and windows are investigated. aerosol deposition on different students from different sources is compared to qualitatively explore the risk posed to individuals in the room due to their position with respect to an infected student. a three-dimensional model of a classroom consisting of nine students and an instructor was developed. the model uses realistic classroom dimensions and air conditioning. the classroom shown in fig. 1 is 9 × 9 m 2 in area and 3 m in height. the distance between each student is 2.4 m (7 ′ 10 ′′ ), which is greater than the recommended 6 ft separation distance for covid-19 mitigation. the model includes desks (with glass screens and without them) and windows. all students are represented similarly and have the same dimensions. each student consists of a cuboid body (0.5 × 0.25 × 1 m 3 ) and a cuboid head (0.16 × 0.15 × 0.2 m 3 ) with a rectangular mouth surface (0.06 × 0.03 m 2 ) through which particles and air are injected into the system. the simplified human model is inspired by models used in a numerical investigation of cross-transmission in hospitals. 6 no chairs are considered in the model due to the extensive variability in chair sizes and shapes. students are assumed to be exposed to aerosols in order not to underestimate deposition on students. an instructor is defined in the front, as shown in fig. 1(a) , and is assumed to be 1.7 m in height. independent surfaces are defined in the model for each object for tracking the aerosol deposition on objects and students, respectively. air conditioning of the classroom follows ashrae 62.1 ventilation standards for acceptable indoor air quality. 26 the air conditioning system consists of five supply diffusers and four return air diffusers distributed as shown in fig. 1(a) . the cubic feet per minute (cfm) required for adequate ventilation was found to be ∼1230 cfm. the supply diffusers (1, 3, 5, 7, and 9 ) supply air at a 37 ○ angle from the horizontal surface with an inlet flow area of 0.294 m 2 and a diffuser inlet vertical air velocity of 0.395 m/s based on ashrae recommendations. 27 in the present work, the effect of opening windows while the air conditioning system is running on particle removal is explored. for this purpose, the model includes 3 windows (2.2 × 1.3 m 2 ) that can be opened up to 50% in 10% increments. an unstructured, tetrahedral mesh is used, as shown in fig. 1 . the mesh was generated using ansys icem 19.1. the mesh consists of 3.3 × 10 6 mesh cells with a minimum cell size of 0.5 cm and maximum cell size of 10 cm with gradual transition, maximum skewness of 0.823 (a mean value of 0.593), and maximum aspect ratio of 3.21 (a mean value of 1.43). the grid is refined near surfaces to maintain a wall y + <10 during the simulations. each case of the 20 cases simulated in this work consumed ∼9 h running on four computer cores. the present study uses the commercial cfd code, ansys fluent 19.2, to simulate the airflow and particle transport. the continuity and momentum equations of the continuum phase (air) are solved in the beginning independent of the discrete phase using the steady state reynolds averaged navier-stokes (rans) incompressible solver. the present simulations use the re-normalization group (rng) k-ε model. 28 the choice of the rng k-ε model is motivated by the work of ramponi and blocken who investigated the influence of turbulence models on cross-ventilation for a generic isolated building, and it was found that the rng k-ε model was suitable for their application and operation conditions, which, in part, resemble the current application. 29 the simple algorithm implemented in ansys fluent is applied for pressure velocity coupling with pressure interpolation of first order. the convection and viscous terms of the governing equations were discretized utilizing the second-order discretization scheme. the solution is assumed to be converged when all the scaled residuals stabilize and approach a minimum of 10 −5 for k, ε, x, y, and z momentum equations as well as 10 −4 for the continuity equation. once the continuum phase solution converges, the flow field is then frozen and is used to transport the discrete phase (aerosol particles). the effect of the particles on the flow of air is negligible. one-way coupling between the continuum phase and the discrete phase is used given the low concentration of the aerosol particles in air. the particle trajectory is determined by solving the equation of motion for the particle in a lagrangian framework. the where v i is the velocity of the particle, m is the mass of the particle, → f drag is the drag force between the air and the particle, → fg is the gravity force, and → fa represents the other additional forces including the pressure force, virtual mass force, basset force, brownian force, and saffman's lift force. the particles used in the present work are sufficiently small to neglect pressure and virtual mass forces and sufficiently large to neglect brownian force. 7, 8, 30 as the particles are much smaller than the mesh elements, it is necessary to use drag models. the present work uses the stokes-cunningham drag model. therefore, the equation of motion of the particles could be written more explicitly as follows [eq. (2)]: where ui is the velocity of the flow, f is the drag factor, 31 τp is the particle reaction time, and cc is the cunningham correction factor. 32 the present simulations use 96 000 particles, which is a reasonable number of particles for sound statistics and is greater than those used in another study of aerosol removal in hospital care units. 15 the turbulent dispersion of particles and the random effects of turbulence on particle dispersion were taken into account using the discrete random walk method implemented in ansys fluent. since the particles are small enough to stick to surfaces, the trap boundary condition is used for the particles over all solid surfaces. in reality, some of the particles will be reflected and others may re-enter the air after deposition. however, re-entry and reflection are difficult to account for as they are affected by particle properties, surface properties, and flow conditions. 33 an escape boundary condition is employed for the diffusers and mouths. air flow from mouths is assumed to be exhaled at 20 l/min specified as a velocity inlet boundary condition (0.185 m/s) for a mouth inlet area of 0.0018 m 2 . the particles are released with the same velocity normal to the mouth surface. the base case uses 1 μm particles, student 5 as the source, no glass barriers, and windows closed. the choice of 1 μm particles is in the range of the particle size of aerosol particles released in exhalation and talking. 11 student 5 is used as the source for the base case due to their location far away from vortices at the edges of the room. the present study investigates the effects of particle size, source position, glass barriers, and windows on the fate of the exhaled aerosol particles. parameters of the base case are varied to investigate these effects. particle sizes studied are 1 μm, 4 μm, 10 μm, 15 μm, 20 μm, and 50 μm. aerosol sources considered are students 1, 2, 5, 8, and 9 and are studied with and without 70 cm high glass barriers/screens placed on top of the desks. the effect of windows is explored by comparing aerosol deposition and transfer in the classroom with 0%, 10%, 20%, 30%, 40%, and 50% open windows. as three windows are available in the room, 10% open windows implies that 10% of each of the three windows is open. the effect of windows is explored while the air conditioning system is running. while this can increase the cooling/heating load and decrease the energy efficiency of the air conditioning system, the present work is concerned with the effect on particle removal. table i summarizes the parameter combinations investigated in the present work. the velocity field of the continuum phase and the distribution of turbulent kinetic energy and vorticity are of fundamental importance to aerosol transport. figure 2 shows the turbulent kinetic energy distribution, velocity magnitude distribution, and velocity vectors of air across a two-dimensional slice going through students 2, 5, and 8. in this slice, air is injected into the system through the supply diffuser in the middle (inlet 5) at a 37 ○ angle with the ceiling. return diffusers 2 and 8 are shown at the sides. the turbulent kinetic energy is more significant at the edges of the room (especially at the outlets) and close to student 8 by the virtue of their location with respect to air conditioning [ fig. 2(a) ]. the velocity magnitude is strongest at the inlets and outlets, but the air is not stagnant in the rest of the room due to air conditioning [ fig. 2(b) ]. the velocity vectors [ fig. 2(c) ] demonstrate the recirculation near the edges of the room and near student 8's head. vortices can partially trap aerosol particles that are transported to those regions and increase deposition on neighboring surfaces. particle transport in the classroom environment due to an impulse aerosol source is a transient process. for the purposes of characterizing the dynamics and the fate of exhaled aerosol particles, a single-release impulse source is used. figure 3 shows the distribution of 1 μm aerosol particles in the classroom at different points in time since particle release. figure 3 (a) illustrates the transport of particles released from student 5. after 1 s of release, the aerosol particles exhibit a parabolic distribution at the front of the particle swarm. the particles slowly disperse and rise up during the first 50 s. once the particles reach the downstream of the air conditioner, the particles are rapidly transported to different parts of the room. as air flows from the supply diffusers to the return diffusers, the particles that reach the downstream of the air tend to follow the flow and exit the system. overall, there are significantly more 1 μm particles in the upper half of the room than the bottom half due to the flow of air to the return diffusers that are located in the ceiling in the present model. figure 3 (a) highlights the significance of the flow velocity distribution on aerosol transport in the room. therefore, the results of the present work are applicable to classrooms with comparable air conditioning. figure 3 (b) illustrates the transport of 1 μm particles released from student 8. aerosols released from student 8 [ fig. 3(b) ] exhibit a substantially different distribution than aerosols released from student 5 [ fig. 3(a) ]. at 1 s, the particle swarm curves downward and much of the particles deposit on the source student (student 8). this is a result of the position of student 8 with respect to the air conditioning system. as shown in fig. 2 , the velocity magnitude near student 8 due to air conditioning is strong compared to that near student 5. student 8 is also present near a region with recirculation and strong vortices compared to the rest of the room. the particles disperse slowly, and even after 5 min, most of the particles are present in the back half of the room. particle size is of fundamental importance to aerosol transport. 7 the present work considers spherical aerosol particles in the 1 μm-50 μm size range. figure 4 shows the effect of particle size on the fraction of aerosol particles released from student 5's mouth that deposit on different surfaces in the room, such as ground, ceiling and walls, desks, and students, or escape from the outlet of the air conditioning system. no significant difference is observed between 1 μm and 4 μm particles [ figs. 4(a) and 4(b) ]. nearly 50% of 1 μm and 4 μm aerosol particles exit the room through the air conditioning system after 15 min. roughly 15% of the particles deposit on the ceiling, and ∼10% deposit on the walls of the classroom, which is comparable to 14%-15% deposition on the ground [figs. 4(a) and 4(b) ]. this suggests that gravity does not play a significant role in the transport of 1 μm and 4 μm particles in the timescale of 15 min. in the case of 10 μm particles [ fig. 4(c) ], deposition on the ground increases to 27% compared to 14%-15% in 1 μm and 4 μm desks, and students. the rest of the particles deposit on the ceiling and walls, exit the room through the air conditioning system, or remain in the air for longer than 15 min. figure 4 also shows that ∼15 min is adequate for 1 μm-50 μm particles to have at least one interaction with a surface or exit the room. in the case of 50 μm particles, the particles deposit rapidly in less than a minute and mostly on the source student. the extensive deposition of 50 μm particles on the source student is due to gravitational settling and the simplified, rectangular geometry of the student modeled (figs. 1 and 2) . much of these 50 μm particles would deposit on the ground if not for the simplified student geometry. the position of the initial aerosol source in the fluid flow field affects the trajectory of the released particles [eqs. (1) and (2)]. the location of the student with respect to air conditioning influences the local flow field and particle dynamics (figs. 2 and 3) . it is, therefore, of interest to understand the extent of the effect of source location on the fate of the exhaled particles. figure 5 compares the aerosol deposition on various surfaces originating from different sources (students 1, 2, 8, and 9) using 1 μm particles. the aerosol deposition for the student 5 source was shown earlier in fig. 4(a) . the results in fig. 5 show that the effect of source location on aerosol transport can be substantial as in the case for student 8. the deposition distribution in the case of student 1 [ fig. 5(a) ] is similar to that of student 5 [ fig. 4(a) ] except for very low aerosol deposition on the ground compared to student 5 (3.9% vs 13.7%) and the increased aerosol deposition on the walls and ceiling (∼38% vs 26%). the deposition results for student 9, who is positioned in the back corner, also suggest increased deposition on the wall and ceiling to ∼44% of exhaled aerosol particles. in the case of student 1 and student 9, the increased deposition on walls and ceiling can be explained in part by proximity to walls and in part due to the vortex structures present near the edges of the room (fig. 2) . student 2 who is positioned in the front-middle, far from walls, experiences increased deposition on the walls and ceiling compared to student 5 [ figs. 5(b) and 4(a) ]. this increase in deposition on the wall may be explained by the vortices present in the flow in front of student 2 (fig. 2) . the deposition on the ground appears somewhat stochastic due to the vortices, but in general, it is <20% for 1 μm particles. the fraction of particles that exit through the air conditioning system is consistently >30% except for student 8 [ fig. 5(c) ]. the case of student 8 is special due to their unique position with respect to the air conditioning system (fig. 2) , which directs the particles downward and onto themselves (fig. 3) . less than 10% of the particles exhaled by student 8 exit the room through the air conditioning system. one of the commonly used measures to reduce covid-19 transmission is the use of sneeze guards in the form of glass or plastic barriers. the efficiency of barriers is not independent of the flow field where they are employed, which depends on air conditioning and the geometry of the surroundings. therefore, it is necessary to evaluate its effectiveness in the classroom environment especially for small particles such as 1 μm particles which can diffuse for long distances in the room. figure 6 shows the deposition distribution of 1 μm particles released from different student sources in the presence of 70 cm tall glass barriers on top of the desks [ fig. 6(f) ]. the deposition of the particles on the screens varies significantly from one source to another. the fraction of 1 μm particles deposited on the screens is very small (<1.5%) for students 5, 8, and 9 [figs. 6(c)-6(e)]. more significant deposition on the screens is observed in the case of student 1 (9%) and student 2 (∼51%), as shown in figs. 6(a) and 6(b). differences in the aerosol deposition compared to the case with no barriers (fig. 5) are also observed. the differences can be attributed to the modulation of the local flow field as a result of the barriers, which further depends on the position of the barrier in the flow field. notably, the inclusion of barriers decreases the total fraction of particles deposited on the students by ∼63% on average compared to the case with no barriers. however, barriers appear to slow down aerosol removal and deposition. for instance, ∼20% of the particles remain in the air after 15 min in the case of student 9 when barriers are used, while only ∼3% of particles remain in the case with no barriers. it is difficult to assess the effectiveness of glass barriers in reducing aerosol transmission based on figs. 5 and 6, which do not discriminate between the source student and receivers. for a clearer comparison, fig. 7 shows source-receiver maps for 1 μm particles in the absence and presence of screens. the sources considered are student 1, student 2, student 5, student 8, and student 9. self-deposition is indicated in a box next to each student, and the fraction of aerosol deposited on other students is marked by arrows from the source to the receiver. a threshold of 0.01% (∼10 particles) is applied to the maps. the use of a threshold is to ensure that only statistically meaningful numbers are reported. on average, the total fraction of aerosols transmitted from a source student to others in the classroom decreases by ∼92% in the case with screens. in the presence of screens, very few aerosol particles (<0.01%) are transmitted from student 1 to the others in the room, and self-deposition is significantly reduced from 1.5% to 0.3%. in the cases of student 2 and student 5, aerosol transmission to others and self is consistently reduced with the exception of increased transmission from student 5 to student 9 (from 0.04% to 0.08%). in the case of student 8, self-deposition increases from ∼47% to ∼60% and deposition on students 6 and 9 increases, but deposition on others decreases significantly. in the case of student 9, total aerosol transmitted to others is reduced by ∼74%. however, transmission from student 9 to student 5 increases from 0.02% to 0.3% and that from student 9 to student 7 increases from 0.05% to 0.18%. overall, the addition of screens substantially reduces aerosol transmission from one student to another, but it does not eliminate particle transmission between students. the effect of opening windows while the air conditioning system is running is investigated in order to understand its impact on particle removal compared to the case with windows closed. a typical sliding window can be opened up to 50% of its total width. the present work considered cases with 0%, 10%, 20%, 30%, 40%, and 50% open windows using 1 μm particles. the source student is assumed to be student 5. figure 8 shows the effect of opening windows on aerosol deposition and removal. the total fraction of particles that exit the system through the windows and air conditioning outlet is increased on average by ∼38% [ figs. 8(a)-8(f) ]. the fraction of particles that exit the system through air conditioning is reduced by ∼60%. this is advantageous as fewer particles may be able to transfer to other rooms bypassing the air conditioning filters. the fraction of particles that exit through the windows appears to be affected by the extent to which the windows are open. the results shown in figs. 8(a)-8(f) suggest that there may be an optimal configuration such that the fraction of particles that exit the system is maximized although no systematic trend is observed. the fraction of particles that exit the system for 0%, 10%, 20%, 30%, 40%, and 50% open windows is 50%, 87%, 51%, 92%, 36%, and 78%, respectively. on average, ∼69% of particles exit the system when windows are open at all, compared to 50% with windows closed. with the exception of 40% open windows, opening windows increases the fraction of particles that exit the system. the results demonstrate that a large fraction (24%-50%) of smaller particles (<15 μm) exit the room without interacting with any surfaces in the room. this finding highlights the need for efficient filtering in the air conditioning systems. the aerosol released from students disperses in the room, and its concentration decreases. the concentration of the aerosol particles increases again as they enter the air conditioning system. the transfer of a larger fraction of exhaled particles to the air conditioning return diffuser, although beneficial to individuals in the room, may pose greater risk to individuals in other rooms as air conditioning systems often use recycled air. it is also found that a 2.4 m separation distance between students is inadequate to eliminate particle transmission between students with the exception of 50 μm particles. the fraction of particles that exit the system without interacting with any surfaces depends on the source location. interestingly, students closer to the supply diffusers such as student 1, student 5, and student 9 are associated with greater particle exit fractions than students closer to outlets such as student 2 and student 8. the position of the student in the flow field significantly affects particle transport. significant aerosol deposition (∼47%) on student 8 is observed due to the aerosol they released. this is due to their unique position in the flow field near a vortex region close to the edge of the room and close to an outlet. an important implication of this increased aerosol deposition on student 8 is that it suggests the presence of mixing hotspots in the room where aerosol deposition can increase by as much as tenfold. in such a hotspot, if two students are present, the chances of aerosol transmission between the two will be significantly higher than elsewhere in the room. this highlights the need for thorough characterization of aerosol transport in different environments to identify and avoid hotspot areas. sneeze guards/glass barriers were found to effectively reduce the transmission of 1 μm aerosol between students by ∼92% on average. while the fraction of particles deposited on the screens directly is small in most cases studied, the screens appear to modulate the local flow field resulting in less aerosol transmission between students. screens, however, do not completely eliminate transmission of 1 μm particles between students and their effectiveness depends on source location within the classroom with respect to the air conditioning system. nevertheless, the 92% reduction in aerosol transmission is highly beneficial. opening windows was found to increase the fraction of particles that exit the system by ∼38% compared to the case with closed windows. the fraction of aerosol particles that deposit on students (including the source) decreased from 2.3% to an average of 0.45% when windows are open at all suggesting that opening windows reduces aerosol deposition on students by ∼80%. the present study only investigated one source (student 5) for cases with open windows. however, the results suggest that opening windows while the air conditioning system is running reduces aerosol transmission between students and increases the fraction of particles that exit the system. the present work is subject to many limitations. first, deposition of aerosol particles on contact with solid surfaces is assumed. reflection and re-entry are not considered. this is, however, justified as most of the simulations conducted in this study are of 1 μm particles. particles <50 μm in diameter can stick to surfaces through van der waals forces. 34 adhesion forces acting on 1 μm particles can exceed gravitational force acting on the particle by factors greater than 1 × 10 6 . 34 adhesion forces, however, depend on particle properties, surface properties, and environmental factors. 33 second, the present work does not investigate the synergy between the different factors considered. for instance, the effect of opening windows on aerosol removal and deposition is not necessarily independent of particle size. nevertheless, investigating the synergy between the different variables would necessitate extensive computational resources not available to the current project. the current study is rather focused on identifying what factors are important for aerosol transport in a classroom in order to inform other studies that may further investigate the interactions between the different factors. third, the deposition fraction is assumed to be a single deterministic value. statistical characterization of the deposition fraction would be of interest especially because of the existence of recirculation and vortices near the edges of the classroom. fourth, classrooms are subject to extensive variability in sizes, air conditioning, student distribution, and student age/, which would affect aerosol deposition and removal. effective mitigation strategies should consider multi-layer approaches including using masks, redistributing students, using glass barriers, opening windows, optimizing the air conditioning system for maximum particle removal, and improving air conditioning filters. understanding aerosol transport in different environments is of critical importance to covid-19 mitigation measures. the present study investigated aerosol removal and surface deposition in a realistic classroom environment using computational fluidparticle dynamics (cfpd) simulations. a model classroom that included nine students and a teacher was constructed. air conditioning of the classroom followed ashrae 62.1 ventilation standards for acceptable indoor air quality. four different factors were considered: particle size (1 μm-50 μm), source location (students 1, 2, 5, 8, and 9), presence of barriers/sneeze guards, and opening windows (10%-50% of window width). the following points highlight the main findings of this work and the implications of these findings: (a) aerosol distribution in the room is not uniform and is strongly influenced by the air conditioning layout. (b) even with only 9 students in the room and 2.4 m distance between students, the aerosol (1 μm-20 μm) is transmitted in significant quantities between students and from one student to other students' desks with aerosol transmission between two neighboring students reaching 0.9% of exhaled particles in some 1 μm particle cases. studies have estimated that ∼20 000 particles in the 0.8 μm-5.5 μm range are released and that over 100 000 virions are emitted per minute of physics of fluids article scitation.org/journal/phf speaking. 35, 36 therefore, particles transmitted between neighboring students separated by a 2.4 m distance in a classroom may exceed 180 particles per minute. the transmission of particles from one student to other students' desks highlights the need for hand sanitization even without contact with other students' belongings. (c) the effect of source location on aerosol transport is significant. student 1 in the front corner transmitted ∼0.55% of exhaled 1 μm aerosol particles to other students, while student 5 in the middle transmitted ∼2.1% of exhaled particles to others. removing the middle student seat (student 5) may help reduce the risk of infection to others. furthermore, student position appears to affect the likelihood of receiving aerosol particles from others. students 7 and 9 in the back corners received 2 to 3 times less particles on average than most other students in the room. therefore, students at risk of covid-19 complications may be placed in positions with a lower chance of receiving particles. (d) opening windows while the air conditioning system is running, while not recommended from an hvac point of view, significantly increases particle exit fraction by ∼38% and reduces transmission between students by ∼80%. (e) glass screens reduce aerosol transmission from one student to another and should be used. the extent of their effectiveness depends on the source location with respect to the air conditioning system. (f) particles disperse in the room and re-concentrate at the return ducts of the air conditioning system. a large fraction of exhaled particles end up in the air conditioning system, which highlights the need for effective filtration and sterilization systems within air conditioners. finally, the results of this work should be interpreted under the context of the air conditioning layout and student distribution used. other classrooms may employ different air conditioning standards and might necessitate aerosol transport investigations tailored to the specific classroom. each case of the 20 cases simulated in this work consumed ∼9 h running on four computer cores. notably, this runtime was enabled by freezing the continuum solver upon convergence of the residuals. only the discrete phase transport is solved for a simulation time of 15 min. mr. ibrahim el-hagali at pennsylvania state university is gratefully acknowledged for providing computational resources used in this work. the data that support the findings of this study are available from the corresponding author upon reasonable request. transmission of covid-19 virus by droplets and aerosols: a critical review on the unresolved dichotomy the flow physics of covid-19 review of indoor aerosol generation, transport, and control in the context of covid-19 visualizing speech-generated oral fluid droplets with laser light scattering how can airborne transmission of covid-19 indoors be minimised? an air distribution optimization of hospital wards for minimizing cross-infection effectiveness of direct lagrangian tracking models for simulating nanoparticle deposition in the upper airways the numerical computation of turbulent flows two-equation eddy-viscosity turbulence models for engineering applications characteristics of exhaled particle production in healthy volunteers: possible implications for infectious disease transmission quantity and size distribution of cough-generated aerosol particles produced by influenza patients during and after illness characterizations of particle size distribution of the droplets exhaled by sneeze a cfd modeling study in an urban street canyon for ultrafine particles and population exposure: the intake fraction approach removal of exhaled particles by ventilation and deposition in a multibed airborne infection isolation room radiation dosimetry of inhaled radioactive aerosols: cfpd and mcnp transport simulations of radionuclides in the lung visualization of local deposition of nebulized aerosols in a human upper respiratory tract model sneezing and asymptomatic virus transmission on respiratory droplets and face masks virus transmission from urinals can a toilet promote virus transmission? from a fluid dynamics perspective modelling aerosol transport and virus exposure with numerical simulations in relation to sars-cov-2 transmission by inhalation indoors transport and trajectory of cough-induced bimodal aerosol in an air-conditioned space american society of heating, refrigerating, and air-conditioning engineers american society of heating, refrigerating, and air-conditioning engineers development of turbulence models for shear flows by a double expansion technique cfd simulation of cross-ventilation for a generic isolated building: impact of computational parameters comparison of indoor aerosol particle concentration and deposition in different ventilated rooms by numerical method an investigation of particle trajectories in twophase flow systems slip correction measurements of spherical solid aerosol particles in an improved millikan apparatus deposition of micron-sized particles on flat surfaces: effects of hydrodynamic and physicochemical conditions on particle attachment efficiency the airborne lifetime of small speech droplets and their potential importance in sars-cov-2 transmission size distribution and sites of origin of droplets expelled from the human respiratory tract during expiratory activities key: cord-332305-8qhytz72 authors: lolla, aruna title: impact of bhagavad gita course on college students: a study based on students feedback date: 2020-08-14 journal: j relig health doi: 10.1007/s10943-020-01073-w sha: doc_id: 332305 cord_uid: 8qhytz72 srimad bhagavad gita, one of the most authentic scriptures of india, is being taught as an elective or regular course in various universities around the world. i have been teaching gita in birla institute of technology and science, hyderabad campus (bphc) since 2012. this paper presents some feedback given by the students revealing the impact of the course on them and possible improvements in its pedagogy. twelve batches of students (a little over 2000) have studied this course between january 2012 and may 2019. from these, 300 students volunteered in giving individual written feedback. in this elaborate write-up, they shared their experience of the course and possible improvements that they would suggest in its teaching. this includes 28 alumni, who came back to meet the author. an analysis of the student input reveals that the impact is at various levels which are broadly placed under three categories here: (1) an immediate effect of sanctity and strengthening of faith, (2) improved clarity of the mind, better focus, calm and content disposition in general and (3) long-term effect on personality traits like development of leadership and problem-solving abilities. they have also suggested a combination of traditional and modern methods of teaching for better comprehension and participation. srimad bhagavad gita, one of the most authentic scriptures of india, is being taught as an elective or regular course in various universities around the world. many schools, colleges and universities have included the gita in their curriculum. top ranking professional institutes in india like iits (gitasupersite 2019), iims (verma 2018 ) and bits-pilani (bits-pilani), kurukshetra university (adarsha 2012 ) and universities abroad like oxford (ochs 2019), harvard (dnaindia.com 2012) and seton hall (rediff.com 2008) among many others have been teaching gita besides other courses in humanities and social sciences. they mostly believe that the course helps in personality development and professional competence of their students. the present generation of students is well aware of the distinction between religion and spiritual science. they are highly enthusiastic and receptive towards the application-oriented science of spirituality. these youngsters are poised to sit down and ponder over the law of karma, understand the powers of the mantras and even practise yoga and meditation in their daily lives. this paper presents some feedback given by the students who have taken the course, revealing its impact on them and possible improvements in its pedagogy. it may be also of interest to university teachers and academicians. let us take a broad overview of srimad bhagavad gita to begin with. the gita basically is a dialogue between sri krishna, who is an incarnation of the divine and arjuna, who is the principal warrior of the righteous side in the war of mahabharata. the essential teachings of the scripture promulgate an inner renunciation of personal desires and results of our actions or work. such renunciation leads to equanimity, accomplishes our total surrender to the divine and supports a liberation from the dividing ego which brings us oneness. one of the prominent yogis of twentieth century, sri aurobindo puts it thus: the gita promises us freedom for the spirit even in the midst of works and the full energies of nature, if we accept subjection of our whole being to that which is higher than the separating and limiting ego. it proposes an integral dynamic activity found on a still passivity; the largest possible action irrevocably based on an immobile calm is its secret-free expression out of a supreme inward silence. (sri aurobindo, the synthesis of yoga, part one: the yoga of divine works, iii self surrender in works the way of the gita) (aurobindo 1999) . the syllabus of this course includes all the 18 chapters of the gita and a brief account of the background, which is explained in two to three classes. i have been following the book mentioned below as the main or prescribed text in my classes: srimad bhagavad gita (with sadhana vyakhya). trans. s. subba rao hyderabad: sri gita grandhalaya, 1969. around the age of 25 years, sri s. subba rao, the author of the prescribed gita text book for the students, came into close association with saints and sages of high order such as ramana maharshi and upasni baba. he writes in the preface of the book that, as a result of his service to the sages, the teachings of the gita came flashing upon him and very soon he took them to his wife first and then to his friends. later on, he became a gita pracharak (a teacher of gita) and taught the scripture to a cross section of population for 30 years. he further acknowledges the active presence and work of the divine teacher, sri krishna, in composing this work has helped him in completing it in a short period of mere 8 months in spite of his physical handicap of blindness. vedanta kesari, a reputed journal in the subject comments on the book saying, "the translation is perfectly clear and intelligible. examples drawn from everyday life and the sayings and incidents in the lives of sages add worth of the commentaries." (1961) . another textbook which is prescribed for the students is: the to return to our focus area of "impact of the course", let us first take a look at the work done by other researchers on the topic. dabas and singh (2018) conducted a study to assess the effect of bhagavad gita teachings in bringing about positive psychological effects on semi-urban school students in india's national capital region. the results showed that the bhagavad gita intervention group of students showed statistically significant increased scores for the three positive variables, hope, optimism and resilience, compared to control group. the authors attributed the outcomes to the fact that the indian interventions consisted of profound thoughts and "mantras" that the students could relate to and easily incorporate into most life situations, whereas the western positive psychology interventions required more time and expertise for effective implementation. srimad bhagavad gita is being taught as an elective in birla institute of technology and science, pilani campus (which started in 1964) for several decades now. however, in hyderabad campus (bphc) of the university, which came into existence much later in 2008, the author of this paper has started teaching it in january 2012 soon after she joined. it is a one semester course and the duration of each semester is around 4 months. forty-seven students had registered for the course in the first batch which started in january 2012. the second batch had 98 students and third batch (january to may 2013) had an unprecedented number of 324 students which is highest in any elective course in humanities in the university. later on, the authorities of bphc restricted the number of registrations in any course to 200 per batch. it was further reduced to 150 and then to 90 students as the number of teaching faculty increased. looking at the number of registrations for gita course which was mostly filled to the upper limit allowed and also the degree of attendance (which is not mandatory in bits) as well as active student participation in the class, i felt encouraged to conduct a survey and take their feedback on the course. i was curious to know their main interest and motivation in opting for the course. engineering undergraduate students anywhere between their second and fourth (final) years can opt for this course. twelve batches of students (around 2000) have studied this course between january 2012 and may 2019. these inputs were collected from the students who have completed the course or are about to complete. not all had participated in the feedback survey which was conducted along with the regular end-semester feedback taken by the university for all the courses. a total of 300 students volunteered in giving individual written feedback. in this elaborate write-up, they shared their experiences of the course and suggestions for possible improvements in its pedagogy. the students expressed this in their own style or kind of language. based on the main idea or intention, similar responses were classified into similar heads. the responses of the students regarding the impact of the bhagavad gita course on them are summarized in table 1 . out of the 300 students, 242 said in that the course has given them better perception of the scheme of human life, human problem solving and satisfaction with themselves 90 other effects: sense of well-being, physical fitness 27 nature, clarity in thought and in decision-making, and above all an attitude of optimism and faith in the divine. let us now take a closer look at specific inputs by the students. some 64 students wrote that participation in the course bestows an inner ambience of peace and sanctity within them which helps in relieving stress and aid in generating "good thoughts" which paves way for staying calm in various life situations and also remain focused on studies. around 90 of them reported an improvement in problem-solving ability and a feeling of satisfaction with their lives. twenty-seven students who are well versed in sanskrit language, who are few in number, shared their experience of chanting the verses of gita in their solitary hours in hostel rooms and said that this has significantly improved their sense of well-being and even physical fitness. this effect is similar to playing of outdoor sports or chanting of mantras like lalitha and vishnu sahasranamam (lolla 2018) . more than half of the respondents have written that the course is easily understandable and motivational. they felt that simple day-to-day examples in life were used in the classes which made it interesting and helped them apply certain principles of the gita in their daily life. for example, while explaining secondary duties (optional duties), the present author once gave the example of sharing of their notes with those students who missed some class. it is not compulsory that they help the absentees, but it would be very apt and good for both parties if they did it. coming to the suggestions part, 58 students wrote that student participation in the course could be increased in the form of group discussions and presentations on related aspects of studying bhagavad gita, and also include these in the evaluation components. twenty-four students felt that concepts should be explained through more of case studies and relevant anecdotes. eleven students from the early batches suggested that the videos of mythological serials and excerpts of talks given by spiritual stalwarts can be shown to enable better understanding and appreciation of the text. this suggestion was followed for the later batches, especially after 2015. others (8-10 students) gave suggestions in the area of evaluation pattern, like increasing short answer type of questions, allotting marks for class attendance, etc. the present method of teaching the course has been about 80%, interactive lecturing. as and when there is a need or feasibility like smaller class strength on a given day or inclination of the students or the teacher, group discussions and presentations are conducted in the class. for some batches, presentations were included as evaluation components. videos of mythological serials (ramayan by ramanand sagar and mahabharat by b. r. chopra) and excerpts of talks given by spiritual stalwarts and informative documentaries on topics like yoga and meditation were shown to the interested students as and when required and when permitted by time. the bhagavad gita is a universal scripture. it is relevant to human lives at all times and places. its message is meant to guide humanity in all the aspects of its existence. sri krishna's message guides not only our individual and social life but also leads to our spiritual upliftment and the ultimate goal of our existence, union with the divine. teaching this course to undergraduate students is especially useful since these youngsters have their whole life ahead of them. they can imbibe the wisdom of the gita and lead their lives enriched by its spiritual force and light. the study conducted by dabas and singh (2018) (refer to 2.2) with its substantial sample size and systematic methodology establishes a lay observation of some of my students that a "participation in the course bestows an inner ambience of peace and sanctity within them which helps in relieving stress and aid in generating "good thoughts" which paves way for staying calm in various life situations and also remain focused on studies". dr. anand kulkarni in his recent article bhagavad gita-a panacea for covid 19 published in business world (kulkarni 2020) highlights the necessity of the scripture thus: as the world continues to deal with the outbreak of coronavirus, there are increasing anxiety, dilemmas, nervousness, faintheartedness among the doctors, nurses and health care workers. dealing with coronavirus is the biggest challenge to face the health sector as it's going to ask a lot of them all. i am a doctor myself and none of us have experience of a pandemic like this… it is also important at this stage we consider medical staff well-being and resilience and this is where bhagavad gita plays a big role. the religious leaders should provide this as a medicine to health care workers at this crucial time. we can further research on the impact of studying the gita by foreign students who may not be aware of or exposed to the religious-cultural background of the scripture. it would also be interesting to observe the students taking the course through standardized research tools over longer durations of time. development of personality traits and general competence happen over a long period of time and may not be directly measurable. an eminent scholar of the gita, prof. mahadevan of the indian institute of management, bangalore (mahadevan 2008) , while addressing a seminar titled "towards a new paradigm of business management-alternative perspectives from ancient indian wisdom", observes thus: the bhagavad gita is the most popular of the ancient texts not only among the indians but also among the westerners. bhagavad gita has inspired many of our national leaders and provided them strength, moral courage and clarity of thought with which they have led the country in its struggle. arguably, these are important elements of making a good manager or a leader today. ancient education, in india as well as the world, was mostly focused on spiritual wisdom and applied fields like law, economics and arithmetic. contemporary humanity and its intelligentsia would also benefit and flourish if its present courses of study are complemented on large scale by an imparting of spiritual wisdom drawn from highly reputed scriptures from all streams of established religions. we can then look forward to an enlightened work force and truly secular citizens of a golden era in the foreseeable future. funding none. conflict of interest the author declares that there is no conflict of interest in this work. this article does not contain any studies with animals performed by any of the authors. informed consent was obtained from all individual participants included in the study. a bhagavad gita course by kurukshetra university essays on the gita the synthesis of yoga. pondicherry: sri aurobindo ashram publication department bhagavad gita teachings and positive psychology: efficacy for semiurban indian students of ncr harvard students studying the gita, ramayana: univ president srimad bhagavad gita bhagavad gita-a panacea for covid 19. business world mantras help the general psychological well-being of college students: a pilot study management lesson from the bhagavad gita the bhagavad gita. trans. new delhi: harpercollins gita study to be mandatory at seton hall university srimad bhagavad gita: with sadhana vyakhya trans the oxford centre for hindu studies the vedanta kesari the latest to take the plunge is iim ahmedabad, which is drawing lessons from the bhagavad gita to train future corporate leaders. the economic times publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-297129-vpg1sfsk authors: pather, nalini; blyth, phil; chapman, jamie a.; dayal, manisha r.; flack, natasha a.m.s.; fogg, quentin a.; green, rodney a.; hulme, anneliese k.; johnson, ian p.; meyer, amanda j.; morley, john w.; shortland, peter j.; štrkalj, goran; štrkalj, mirjana; valter, krisztina; webb, alexandra l.; woodley, stephanie j.; lazarus, michelle d. title: forced disruption of anatomy education in australia and new zealand: an acute response to the covid‐19 pandemic date: 2020-05-10 journal: anat sci educ doi: 10.1002/ase.1968 sha: doc_id: 297129 cord_uid: vpg1sfsk australian and new zealand universities commenced a new academic year in february/march 2020 largely with “business as usual.” the subsequent covid‐19 pandemic imposed unexpected disruptions to anatomical educational practice. rapid change occurred due to government‐imposed physical distancing regulations from march 2020 that increasingly restricted anatomy laboratory teaching practices. anatomy educators in both these countries were mobilized to adjust their teaching approaches. this study on anatomy education disruption at pandemic onset within australia and new zealand adopts a social constructivist lens. the research question was “what are the perceived disruptions and changes made to anatomy education in australia and new zealand during the initial period of the covid‐19 pandemic, as reflected on by anatomy educators?.” thematic analysis to elucidate “the what and why” of anatomy education was applied to these reflections. about 18 anatomy academics from ten institutions participated in this exercise. the analysis revealed loss of integrated “hands‐on” experiences, and impacts on workload, traditional roles, students, pedagogy, and anatomists' personal educational philosophies. the key opportunities recognized for anatomy education included: enabling synchronous teaching across remote sites, expanding offerings into the remote learning space, and embracing new pedagogies. in managing anatomy education's transition in response to the pandemic, six critical elements were identified: community care, clear communications, clarified expectations, constructive alignment, community of practice, ability to compromise, and adapt and continuity planning. there is no doubt that anatomy education has stepped into a yet unknown future in the island countries of australia and new zealand. australian and new zealand universities commenced a new academic year in february/ march 2020 largely with "business as usual." the subsequent covid-19 pandemic imposed unexpected disruptions to anatomical educational practice. rapid change occurred due to government-imposed physical distancing regulations from march 2020 that increasingly restricted anatomy laboratory teaching practices. anatomy educators in both these countries were mobilized to adjust their teaching approaches. this study on anatomy education disruption at pandemic onset within australia and new zealand adopts a social constructivist lens. the research question was "what are the perceived disruptions and changes made to anatomy education in australia and new zealand during the initial period of the covid-19 pandemic, as reflected on by anatomy educators?." thematic analysis to elucidate "the what and why" of anatomy education was applied to these reflections. about 18 anatomy academics from ten institutions participated in this exercise. the analysis revealed loss of integrated "hands-on" experiences, and impacts on workload, traditional roles, students, pedagogy, and anatomists' personal educational philosophies. the key opportunities recognized for anatomy education included: enabling synchronous teaching across remote universities across australia and new zealand commenced the new academic year in february and march 2020. for most institutions, it was largely business as usual. few realized the extent of disruption and the pace of change that the unfolding covid-19 global pandemic, caused by the severe acute respiratory syndrome coronavirus 2 (sars-cov-2), would impose upon all higher education (evans et al., 2020) . the first covid-19 cases were reported in australia and new zealand at the end of january and february 2020 (who, 2020), respectively, and were related to people travelling for tourism, work or study. even though most universities at first adopted a "watch and wait" approach, it became quickly evident that a higher education sector interruption was imminent (truu, 2020) . the extent of this disruption was evidenced by an almost immediate restriction on business travel for many university staff, initially to china (perpitch, 2020) ; and a resulting large number of australian and new zealand anatomy academics cancelled plans to attend international meetings and conferences. as the governments acted to prevent the pandemic spread, some university leaders drew on the experiences and expertise of their faculty, and acted quickly to source equipment, manage logistics, and provide in-time assistance to support the university's continued operations. in the background, core university business appeared to continue as usual. managements' "watch and wait" approach was perceived to be passive, causing some staff and student distress with the apparent lack of decisive action. university leaders, however, were carefully monitoring the situation to delay an inevitable transition to remote learning for as long as possible. at the beginning of march, the signs of higher educations' imminent disruption presented as australia and new zealand entered the "protect" phase of pandemic management learnt from past corona virus experiences (collignon, 2011; de wit et al., 2016) . by mid-march, the responsibility fell on academic staff to actively repurpose and redeploy resources, upskill their digital competencies, and develop new material to transition traditionally face-to-face (f2f) and blended programs to a remote learning and/or online education delivery mode (johnston, 2020) . like in many countries, australia and new zealand included, technology-integrated learning has been embedded in medicine and allied health programs for some time (harden, 2006) . anatomists in this region have been actively involved in the development of digital resources and in curriculum reform (green and whitburn, 2016; colibaba et al., 2017; otton et al., 2017; green et al., 2018; birbara et al., 2019; ho et al., 2019; birbara et al., 2020; o'rourke et al., 2020) . anatomy education, however, has never before been delivered solely online or remotely for medicine and allied health programs in these countries. the tertiary education quality and standards agency (teqsa), which is the mandatory independent national quality assurance and regulatory body for higher education in australia (supporting information table 1 ), acted quickly and decisively, enabling rapid educational transformation by announcing to the sector a broad reduction of the regulatory burden for universities managing covid-19 pandemic challenges (saunders, 2020) . this deregulation provided universities with approval to effect changes to delivery modes, as well as to provide remote learning to students who were offshore. this last aspect is critical to higher education in australia which serves a large international student body (davies and karp, 2020) . in 2018, 398,593 students enrolled with australian higher education providers were on student visas and comprised a third of the student population (ferguson and sherrell, 2019) . during march 2020, an overwhelming number of international students were forced to return to their homeland as the pandemic spread. the teqsa caveat, however, was that: "providers should assure themselves that such arrangements [changes to program delivery] maintain assessment and quality standards and are appropriately documented" (saunders, 2020) . most institutions were, thus able to rapidly initiate a business continuity plan enabling students to continue the academic year, if they chose. in quick succession over the last 2 weeks of march 2020, medicine and science faculties of most universities in australia and new zealand converted to online and/or remote delivery as university staff moved into a new paradigm of "working from home" (wfh) arrangements. defined by simonson et al. (2011) , remote/distance learning refers to institution-based, formal learning, where the learner and instructor are physically separated, and where interactive telecommunications systems are used to connect learners, resources, and instructors. in part, the business continuity plan (especially for medicine faculties) in australia was driven by pragmatism to ensure a functioning health care system in 2021, which relies heavily on graduating medical students entering the system as junior doctors (interns) each year. for the discipline of anatomy, this decision has potentially fundamental consequences. the governments mandated covid-19 response resulted in restricted movement and physical distancing requirements. the regulations initially constrained anatomy laboratory access, but quickly progressed to a prohibition of students' laboratory access entirely as wfh and remote learning arrangements became the new normal. the initial stage of this transition may be particularly challenging to those in the anatomy discipline as many conventional anatomy education pedagogies rely on "hands-on" practical experience; it is through these small groups that many anatomy educators identify and respond to students' learning needs especially those at-risk of underperforming (kumar ghosh and kumar, 2019) . furthermore, delivering anatomy education solely online or at a distance, if only for a short period, challenges the long-held educational philosophy of many anatomy educators (pather, 2015) . additionally, for those who strive to frame their anatomy programs through a lens of ethics and humanism (štrkalj and pather, 2017; evans et al., 2018; hildebrandt, 2019) , there is a need to work harder to connect students to these paradigms when educating solely within the digital realm, and in the absence of a once-living person (stephens et al., 2019; kumar ghosh and kumar, 2019) . notwithstanding these issues, the remarkable mobilization, commitment, speed, and agility of the australian and new zealand anatomy community, including academic and support/technical staff, presents the beginning of a historic bridge into an unknown future. the continuity of anatomy education during the covid-19 pandemic in australia and new zealand is not "business as usual" and is not likely to return to the pre-pandemic approaches again, given the extent of the present changes. this study, therefore, aimed at documenting the shared experiences of anatomy educators in changing their teaching practice during the early period of the covid-19 pandemic within australian and new zealand higher education. participants were voluntarily recruited via an email invitation to anatomists serving in leadership roles in the australia and new zealand association of clinical anatomists (anzaca), with snowballing recruitment following. the resultant participant group, and co-authors, included 18 academic anatomists representing ten institutions across these two island countries. in australia, this group represented four of the australian states and territories, and five of the "group of eight" universities (leading eight australian research-intensive universities) (table 1) . in new zealand, the participants represented one of the (only) two new zealand universities that teach anatomy. all participants taught anatomy (including neuroanatomy), histology and/or embryology to a variety of student cohorts including science, biomedical science, allied health, and undergraduate and graduate medicine, as well as junior (i.e., resident) and postgraduate (attendings) doctors using multiple approaches before the pandemic (supporting information table 2 ). the research question was "what are the perceived disruptions and changes made to anatomy education in australia and new zealand during the initial period of the covid-19 pandemic, as reflected on by anatomy educators?." all participants were asked to reflect on the experience of delivering anatomy education from the start of the academic year through to the end of march 2020. this descriptive qualitative account of anatomy education disruption focusses on the early stages of the pandemic within australia and new zealand, with the methodology of adopting a social constructivist lens, wherein the truth is negotiated by those experiencing it (rees et al., 2020) . thematic analysis to elucidate "the what and why" of anatomy education was applied to these reflections, as outlined by nowell et al. (2017) . the names of institutions were de-identified during thematic analysis. six phases of coding were undertaken with phase 1 ("data familiarization"), phase 2 ("initial coding"), and phase 3 ("theme searching"), carried out by m.d.l. phases 4 and 5 ("theme review" and "naming themes") were carried out separately by two authors (m.d.l., and n.p.). phase 6 ("findings report production") was initially drafted by m.d.l., with n.p. providing feedback, debating, and discussing interpretation discrepancies. while these stages appear linear, each progressive stage was iterative, and thus previous stages were addressed where needed during phase 6. all analyses were performed in microsoft word (microsoft corp., redmond, wa) as higher order complex coding was not undertaken due to the research question. this approach to thematic analysis included a team-based reflexive exercise (barry et al., 1999) by those undertaking the analysis (m.d.l., and n.p.). both authors have extensive experience in qualitative research and value grounded theory in answering the research question, but they differ slightly in world views. while one allows the research question to dictate the methods, the other struggles with the idea of quantitative analysis ever being objective, due to inherent complexities of the natural world. both thought the opportunity to reflect, and learn from others' reflections, was a valuable opportunity. the fears expressed in undertaking this study included concern for peer wellness and rapidity of data analysis-but overall, both authors thought the study merits outweighed the risks. this study was approved by the monash human ethics review board research ethics committee (muhrec #24292). in australia and new zealand, f2f laboratory practical sessions are central to anatomy teaching (table 1) . participating anatomy educators' reflections outlined the progressively increasing restrictions placed on f2f teaching (e.g., increased physical distancing, cancellation of practical sessions, or complete campus lockdown). these restrictions were coupled with an almost consistent university message: the universities in australia and new zealand are "open for business" with a status of "business as usual." this disconnect between university messaging and imposed social constraints appeared to contribute to at least some of the shared academic anatomists' challenges outlined below. one new zealand academic summed up the higher education pandemic setting concisely in stating: "there is no doubt from our perspective that the world is suddenly a different place." in continuing anatomy education during the initial pandemic period, nearly all universities appeared to employ a mix of synchronous and asynchronous (chen et al., 2005; watts, 2016) online activities (table 1) , with some institutions having used evidence-based online activities to deliver the curriculum prior to the pandemic. notably, nearly all chose at least some synchronism to mitigate the loss of the practical "hands-on" synchronous live streamed small group sessions a member of the "group of eight" leading australian research-intensive universities. almost all donor programs were suspended in the face of the pandemic. lectures typically relied on prerecorded material, a mix of asynchronous and synchronous teaching was used for practical modification. active learning engaged large group webinars or discussion boards to facilitate class discussion; f2f, face-to-face. component of their delivery. of the 10 institutions included, six supported a teaching "pause" (table 2) . this provided students a short time to relocate to home, if appropriate. it also, even if notionally, acknowledged the significant work required from academic staff to change the mode of delivery within a limited timeframe, and gave credence to the ubiquitous messaging that the health and well-being of the university community is of primary importance, especially during the covid-19 pandemic. the urgency of the situation was evident in all narratives. each anatomists' reflection tended to focus on identifying a . many australian and new zealand anatomy educators used digital resources to overcome the legal and ethical limitations related to human donor programs which often prohibit the sharing, photographing or recording of donor material. some institutions were offered a brief (1 week) pause to convert a semester of teaching into online approaches, while many others were not. despite typically engaging demonstrators only for f2f teaching, many institutions engaged sessional demonstrators in this transition -though their roles changed. while some still led online delivery of material, many were engaged to help develop resources for this pandemic anatomy teaching. 3dvr, three-dimensional virtual reality. temporary solution, with many also incorporating and planning for future practical laboratory-based intensives ( table 2) . the need for a balanced teaching approach, staff adaptability, and well-being support were discussed in nearly all reflections. as expected, some academics found the experience of accelerated change exciting and fruitful, while others found this experience daunting and worrisome. universally, there were shared concerns about the extraordinary workloads this transition required. some academics reported that even when commercially available material was used in teaching, the time spent curating and annotating this to ensure learning outcomes were met, was exorbitant. a reflection from one australian colleague highlights the increased workload: "online teaching extends the working day and the working week (to now include weekend work); the volume of email has increased exponentially." almost all academic anatomists reported that anatomy education is likely to change permanently given the scale of change during the pandemic, with some concerned that this change will call into question traditional laboratory-based approaches, in favor of modern (now trialed) online and remote learning approaches (table 1) . this concern was compounded as many academics reflected that these curricular changes, despite attention to quality, were a rapid response to an unprecedented situation, allowing little time for reflection and evidence-based refinement. there were a variety of personnel-and infrastructural-related challenges shared across the australian and new zealand academics' reflections. these challenges fell into four categories: staff (faculty/academic staff and technical/professional staff), students, infrastructure and resources, and curriculum changes to accommodate online and/or remote delivery of anatomy education. with respect to personnel, there were multiple staff categories that were perceived to be affected by the pandemic teaching context; for all staff (regardless of previous role), there was a drastic change in the type of work that they were expected to do. the academic staff appeared to have the least professional role change, as many reported that they were, themselves, still focused on pedagogy; the expectations and skills for executing this new pedagogy, however, was different. a common challenge reflected on was that of information technology (it) skills, and the lack of prior training or knowledge for effective online education delivery practices. technical staff and anatomy sessional staff/demonstrators (defined in supplementary table 1) were perceived as having some of the largest role transitions. some anatomy educators noted that demonstrator roles changed from leading f2f small group discussions to that of digital media development and/ or to small group instruction via streaming platforms such as zoom ® (zoom video communications, inc., san jose, ca and microsoft teams ® (microsoft corp., redmond, wa) ( table 2 ). in contrast, other anatomists commented on the sudden elimination of the demonstrator role due to the pandemic's impact on university finances. colleagues from one australian university described their experience: "one of our first thoughts went to our casual demonstra-tors…this sudden change would remove their opportunity to extend their teaching experience, develop their knowledge and skills and reduce their income. to rectify this, we immediately instigated mechanisms to involve our demonstrators in the process of developing our remote learning opportunities … for one anatomy practical that required transformation into an online format in less than 24 hours, they [the demonstrators] came into the laboratory after hours to create demonstration videos." technical staff appeared to similarly be learning new skills and pivoting their roles, as illustrated by another australian anatomists' reflection: "our tech [nical] team have been enlisted to create videos of rotating models so that students have at least a visual impression of these resources that are typically available to them in a resource area on each campus." professional and technical staff faced the additional stresses related to job security. initially this stress was related to the seemingly seamless way in which academic staff mobilized to deliver a program of online anatomy education, and was exacerbated when some departments took the strategic decision to temporarily suspend the body donor program due to uncertainty regarding the risk to covid-19 exposure (table 1) . as the pandemic further unfolded, and with increased communications of university budget deficits, the future for professional staff remains tenuous. interestingly, almost universally the academic anatomists' reflections included perceptions that both demonstrators and technical staff appeared to value these new roles and demonstrated skills of adaptability and resilience. the reflective narratives about the academics' own role (and role of their academic team) often also illustrated gratefulness for the flexibility of their peers. a major challenge for all participating academic anatomists was the time investment to support the pandemic's curricular change. there were also many who reflected that the increased workload was not accompanied by the usual rewards of student interaction, a shared sentiment of the anatomy educators expressed in one anatomist's reflection: "we go into these roles because we truly love watching students learn, not being able to see that in a traditional sense does make this workload harder" and this was further explained by an anatomist from another institution: "for some academics the anticipation of contact with students [in the usual f2f delivery] is something to look forward to, but this is difficult [during the pandemic] due to the city-wide physical distancing laws." as mentioned, some universities did "pause" teaching to accommodate the transition to online delivery (table 2) . this "pause" was for a relatively brief period (<1 or up to 2 weeks) considering the length of a semester (~10-13 weeks typically) and the number of learning activities that needed modification for online/remote learning. the challenge was compounded as universities moved to wfh, and with primary and secondary schools increasingly transitioning to home-based learning. this was an unprecedented experience and has meant that academics already working to difficult timelines with a challenging workload were also adjusting to wfh, and for some, with the additional tasks associated with concurrent childcare and home-schooling. the academic reflections included a focus on perceived student-centered challenges. academics expressed awareness of students concerns that centered around future degree and career progression, across the participating universities. according to the academics, students appeared to face financial pressures due to lost income, resulting in concerns about university fees, and for many, a forced intermittence. some of the perceived student challenges included managing student perceptions of "missing out" or "getting what they [the students] paid for." there was an observation that these student concerns were at least, in part, due to the universities' messaging on expectations versus what was achievable by staff in the timeframe available, and within the pandemic education context. also impacting these perceptions may be the pre-pandemic emphasis of anatomy education on, and the now lack of exposure to, human donor material, which is often perceived as a signature experience of the discipline. a colleague summarizes some of the student concerns: "students were concerned about not having access to learning from human donors, which they perceived as the best opportunity to integrate concepts with the 'real' experience. the anatomy lab was where deep understanding of anatomy was achieved. students further reported that they struggled without the f2f interaction with academics and tutors in the lectures and labs. in general, the student body was struggling with a lack of motivation without direct teacher interactions.... however, they [students] also highlighted the significant shift that they are making in a short time to adjust from f2f to online learning, and without the benefit of physical interactions with peers." academics themselves also had concerns about student learning, specifically around equity and access. this concern was compounded when considering the implementation of synchronous learning activities, and students' repatriating. this enforced geographical distancing meant that students were located across multiple time zones, with some returning to countries with restricted or poor internet access. australia and new zealand also have internet network accessibility and reliability limitations, particularly in rural and remote areas. together, these potential learner challenges were considered in the redevelopment of australian and new zealand anatomy education, and (at least in part) accounts for the suite of synchronous and asynchronous activities (table 2 ). regarding resources and infrastructure, the challenges expressed by anatomy educators are likely globally universal in university programs, including access to software and support for online teaching, technical failures, and network sustainability. however, unique to anatomy education are some of the ethical and medical considerations related to working with human donor resources. multiple state/regional legislations pertain to body donor programs and consent for donation. specifically, the ethical and legal constraints of sharing and displaying digital images of prosected human donor bodies/body parts, increased the challenges to anatomy online education in australia and new zealand; the net result was a reliance on proprietary digital anatomy resources (table 2 ). in addition to the impacts on teaching, reflections communicated concerns about body donor programs sustainability and continued maintenance in the face of the present acute global health crisis. anatomy lectures appeared to be the least complicated of all activities to translate in the pandemic, while anatomy practical experiences and assessments were consistently the most challenging to redesign across these two countries (table 1) . lectures. while some anatomy educators chose to livestream lectures on campus and/or from home, most chose prerecorded asynchronous approaches by re-using lecture recordings from previous years or developing new recordings. this decision reportedly allowed academics to focus effort on more complicated curricular translations (i.e., practical sessions and active learning), as noted by one reflection: "in the covid-19 context, most course conveners released the pre-recorded lectures from the year before. this provided the opportunity for academics to have cognitive capacity to concentrate on upskilling their comfort with other technologies, and to reorganize material for the online space." these prerecorded lectures were reportedly often supplemented with additional lecture material, which included short concept videos and formative assessments ( table 1 ). the sentiments expressed in many reflections was summed up by one colleague: "one strategic decision was to move away from big topics, by encouraging the breakdown of long lecture content into digestible chunks. the organization, search and review for existing digital material on [the] world wide web were additional tasks that were both time consuming and time 'robbing'." some academics incorporated interactive lectures, mirroring some of the active learning principles. active learning, by definition, depends on student interaction to foster learning, and the opportunities for learner reflection (haidet et al., 2012) . much effort was placed into translating these same principles online using interactive applications like kahoot! © (kahoot! a.s., oslo, norway) and slido © (slido s.r.o., bratislava, slovakia), and incorporating formative quizzes, live streaming, and applied anatomy tasks such as clinical case activities. one academic describes the active learning translation for a medical anatomy curriculum: "our active learning sessions… allow for students' application of learned knowledge in the context of clinical uncertainty. to develop our online learning activities, we first asked ourselves what were core aspects we wanted to reproduce online? for us this included: clinical reasoning, taking chances, and learning from peers. from this, it was decided to develop a series of discussion forums… we gave clear instructions on what was an appropriate approach to responding.... students progression through [clinical] cases required them to work through the previous case, as would happen during the faceto-face active learning session… peers and facilitators were the primary source of feedback and re-direction if a student went off track, again emulating the teamwork needed for successful active learning. once a learner had gone through each case, they were then able to access post-class videos which touched upon key anatomy knowledge." practical experiences. many institutions reflected that student-teacher interaction was a desired attribute of practical sessions and replicating this online required supporting synchronous delivery experiences (table 2) . many anatomists reflected a plan for intensive practical experiences postpandemic to make up for the lack of exposure to authentic human donor resources during the remote learning phase ( table 1) . some of the shared university practical experiences is represented in one colleague's peer reflection: "for our practical experiences, we decided to focus on two key attributes, the short-term knowledge gains, and opportunities for students to be exposed to experts. we engaged the zoom platform, because of its capacity to allow for user interaction. the medical imaging stayed much the same; herein demonstrators shared curated imaging, allowing students to use zoom's "annotation" function to answer imaging identification questions…tutorials were almost exactly the same with students engaging in presentations related to assigned learning objectives. the dissection component was the area that entirely changed." many institutions integrated practical videos, both prerecorded (pre-pandemic) and novel recordings were developed for supplementing the new synchronous online activities ( table 2 ). the purpose of the videos varied and were highlighted in a colleague's reflection: "explanatory video to describe how and why we were making these changes, and how to use the four (three commercial and one in-house) online anatomy resources… [and] staff prepared additional short videos to emphasize key concepts for each weekly prac[tical] class using these online resources." many used widely available videos from youtube (youtube llc., san bruno, ca) or proprietary anatomy software to minimize "reinventing the wheel," but even this approach resulted in significant workload to both "source appropriate videos" and annotate them, with suggestions ranging from ~10 hours for a 2-hour active learning session to ~16 hours for a single practical conversion. for those who had already begun transitioning to technology-enhanced learning before the pandemic, the rapid online transition was perceived as a step backwards in their planned implementation as it was done with limited time for reflection and testing. many also highlighted concerns that this rapid online shift may result in their institutions questioning the role of human donor material altogether. assessments. at the time of writing this report, assessments remained the most complicated and yet to be determined component of australian and new zealand anatomy education. reflections from each university indicated that assessments were to be online. while online practical assessments have previously been effectively trialed (inuwa et al., 2012) , both the assessment approach and the platform for administration still needed to be determined within a short timeframe. the challenges were summarized in this statement: "we want to ensure that our assessment is a true reflection of learning, remains robust, maintains integrity and is secure." some reflections also outlined a plan to prepare the student body for the anticipated changes to the assessment style: "any assessments that we deliver will be preceded by at least one formative examination in order to familiarize ourselves and the students with the online assessment process." many academics shared awareness of student collusion risks and a difficulty with assessment invigilation; some translated this to developing assessments where collusion became irrelevant using variations of formative assessments or relative grading approaches. other universities reported simply accepting collusion as inevitable in their solutions, as exemplified by one strategy in which assessment incorporated higher order questions: "the questions involve interpretation and analysis rather than recall of information and so whilst we cannot control students cheating, the nature of the questions reduces the likelihood of them locating the answer. if students collaborate then at least they have engaged together in reaching a solution." others modified their assessment approaches to minimize the chances of collusion using, for example, "flag race tests [aka practical exams] … conducted [online with] randomized flags." they report that: "this now effectively becomes an open-book test, so to minimize collusion and maintain some academic rigor, the test will be run at a single time for all students and flags are randomized so that they [students] are not all getting the same question (even if they are sitting side by side.)". one institution had further, in partnership with students in decision-making during the pandemic, implemented pass-fail grading reporting: "students were concerned that others in the student body would resort to extreme measures of colluding or compromise their academic integrity, which would be unfair on those who did not." still others were adjusting assessment weighting to minimize the drive for collusion, and were considering sensitivities around what is being examined: "we are creating our own assessments deploying them [online] as timed low-stakes summative tests. care must be taken to make sure non-sensitive images are used as students may screenshot and distribute questions." even with these mitigating measures, assessment remains challenging-how do we build assessment with constructive alignment for pandemic anatomy education? constructive alignment is based on constructivist theory wherein the learner constructs knowledge through learning activities which are pre-aligned with outcomes and assessment (biggs, 2014) . adjusting assessments for students with learning disabilities and those requiring extra time for completing assessments further extends the complexity of ensuring a robust assessment practice delivered remotely, within a limited timeframe. this is further compounded when assessing a student body with heightened anxiety due to the constant media reporting and ever-changing circumstances presented by the pandemic. there were also some recognized opportunities that the pandemic presented including as previously noted, the rapid skill acquisition in online pedagogy (schmidt et al., 2016) and skills in digital media production. with this in mind, and considering the changes already initiated in a short timeframe, the pandemic provided some with an impetus for curriculum review and strategic development of educational resources that would be useful after the pandemic, especially if these further enhance access and equity: "we are developing resources that can be utilized even when we are back to functioning normally. if the resources are aligned with the curriculum content and approach, then they can definitely supplement the practical classes ..." many reflections universally shared benefits such as gratefulness for both peers (staff) and students, and reported enhanced teamwork and skill acquisition of all involved (both staff and students). of note, most participants reported a positive student response, even if this was initially negative. as the pandemic progressed, students increasingly responded to the anatomy learning experience with unsolicited positive feedback, and in some cases also with concern for staff. the timing of the pandemic is of note for the island countries of australia and new zealand. the pandemic's "arrival" coincided with the beginning of the 2020 academic year, a very different time to that of their northern hemisphere counterparts (who were nearing the end of the academic year). the implication of this, is that changes to the anatomy education delivery, even if only limited to the first half of 2020, potentially will have knock-on impacts across the entire academic year with respect to both curriculum delivery and assessment. as the covid-19 pandemic unfolds, and in the face of the significant disruption, the combined experiences and reflections of anatomy academics identify two main contributions to the practice of anatomical education in australia and new zealand. first, there are both opportunities and challenges presented with the rapid change in anatomy education, and there are related significant impacts on workload and professional roles. these were exemplified in the rapid upskilling of human resources, flexibility in redefining roles, and creativeness to enable and support continued learning at a distance. second, there are the perceptions of the impact on pedagogy, technology, higher education institutions, staff, students, and, of course, on personal educational philosophies in both the short and long term. in this domain, concerns on the loss of an integrated "hands-on" laboratory experience, and the lack of being physically present, were raised. the insights into these two areas will have important implications for anyone contemplating anatomy education in the future. while it is an unchartered time in the history of higher education, as always, there are lessons that can be gleaned from disruption, and anatomy education in australia and new zealand to accommodate the acute covid-19 crisis presents several insights (fig. 1) . of primary importance is the care of the whole community. care should be taken to ensure that there is a safe environment. change often creates anxiety for some, and rapid change in response to the covid-19 pandemic is disruptive for both students and staff. for staff that deal with body donations, the pandemic raised concerns about both job security and risk of exposure if a body donor was a virus carrier. it is imperative that department leaders are aware of these concerns and find means to be "present" to engage in conversations and develop plans to mitigate risk to all constituents of the anatomy community. these concerns whether real or perceived, impact well-being. it is, therefore, imperative for department leaders to act quickly in circumstances where exposure to potential harm (e.g., sars-cov-2 virus) is unknown and to mitigate any threat to the health of the community, even if it requires temporary suspension of body donor programs. equally, student mental wellness is of concern to the academic anatomist, both prior to, and during the pandemic. while academics grapple with how to deliver anatomy education online, there needs to be an awareness that students who had previously experienced anatomy as a hands-on subject, or had preferred f2f learning, require support in the transition to learning remotely (blackley and sheffield, 2015) . in this remote learning environment, the lack of physically present peers and instructors can lead some to have negative learning experiences. this raises concerns about the cognitive load placed on students through the change of learning environments, and whether the pace of change during the pandemic leaves too little mental capacity to construct knowledge (schwonke, 2015) . ideally student learning in this new environment requires explicit monitoring of cognitive processes to minimize cognitive overload, as advocated by valcke (2002) . broadbent and poon (2015) highlighted, through systematic review, four strategies students require for online learning: time management, metacognition, effort regulation, and critical thinking. in the short time to transition to remote learning during the pandemic, providing structure was a key element of effective support for managing stress and anxiety, and anatomy educators used a variety of strategies to provide structure like scaffolding learning through structured weekly activities, packaging topics into digestible chunks to reduce cognitive load, the use of short concept videos and quizzes to regulate effort, and interactive synchronous learning to develop critical thinking and social interactions. from the authors experience, students have also been reassured by knowing that hands-on laboratory sessions would be provided post-pandemic to provide opportunities to review cadaveric prosected specimens and surface and living anatomy, and possibly undertake dissection. one of the challenges of the current landscape is that messaging understandably appears changing, unclear and often conflicting. clearly communicating, even when the answer is still being discussed, is essential and will ensure reciprocity and cooperation among students (chickering and gamson, 1989) . as academics, who were also managing change for students, modelling effective communication strategies and approaches is useful to alleviate any change-induced stress. communication of expectations, as well as simple guidance on managing the online space is of great importance and can make a positive impact on learning (broadbent and poon, 2015) . some examples of this include clarifying the expectations around the expected daily/weekly progress and assessment criteria, and providing formative assessment opportunities as a preamble to high-stakes assessment tasks, as highlighted by the academics' reflections. in the light of increased student anxiety during the pandemic, communications should be carefully framed within positive messages of support and in a context of open dialogue. useful strategies employed during the pandemic included using anonymous dialogue platforms like slido © to allow students to ask questions in the comfort of anonymity, discussion boards to enable asynchronous student-teacher interactions (green and hughes, 2013) , and including students in the decision-making process affecting aspects of anatomy delivery, for example, assessments. of note, during this pandemic education delivery, both the academic staff and students described a loss of in person interaction and its impact on their individual motivation. while this has previously been connected to student disengagement in the online learning environment (gillett-swan, 2017) , the impact of this loss on staff has only recently been reported in the context of embedded-reward systems in higher education (cuseo, 2018) . as workloads and roles morph, effective change management strategies can be helpful, including clarifying stakeholder expectations. this is especially important given the increasing concern over the last two decades about mental wellness among infographic of shared australian and new zealand anatomy education teaching approaches during the covid-19 pandemic. scoping implications for this teaching crossed multiple teaching modalities summarized as the six cs, and affected all stakeholders (students and staff) including: change and flexibility, clarify expectations, clear communication, constructive alignment of new material, a focus on community care, and continuity planning in the face of this global health emergency. lectures and active learning tended to engage asynchronous online approaches, using prerecorded videos and discussion boards, while practical activities tended to be synchronous streaming approaches combined with a reliance on future intensive experiences. assessment was the most widely varied aspect across these two countries. all approaches tended to accept collusion as a risk to online assessments, and educators attempted to mitigate this risk by delivering questions with randomized sequencing for each student, decreasing weighting of assessments, making assessments pass or fail, or embracing teamwork as part of the assessment strategy. university academics even in the absence of a crisis (watts and robertson, 2011; gulliver et al., 2017; kinman, 2019) . existing evidence suggests that academics are already overcommitted to their work due to implicit system rewards (hamilton, 2019) . in the absence of a pandemic, the increase in educators' workloads in developing online resources (green and whitburn, 2016) and in transitioning to remote and online delivery (kimball, 1998) is well recognized. managing and supporting students has been identified as the most demanding aspect of digital learning (canninzzo et al., 2019) . this correlates with the reflections of a large number of participants regarding the increased workload, the extended online work day/week, and the increased student correspondence. regarding the latter, kimball (1998) explains that in remote learning, learners have a perception of a "rolling presence" which can make educators feel pressured to spend large amounts of time interacting with students who perceive online education as "always open" for business. the social and personal impact of the rapid response by anatomy academics to transform to a highly demanding form of delivery (i.e., distance/online) is yet to be determined. it is reasonable to expect that discussions and plans to reset stakeholder expectations will be required, especially if this pandemic continues for an extended period. as workloads explode, there is a need for leaders to provide immediate assurance, acknowledgement of the situation and empathy for the challenges faced with rapid change. those in leadership roles, have the added responsibility to advocate for staff, especially considering high expectations and the complexities of wfh arrangements. clarifying expectations of academics is imperative. for academic colleagues, it is impossible to quantify the time required for repurposing and sourcing resources, but acknowledging this reality is useful, as is acknowledging all who contribute to delivering the learning experience. in good education practice, constructive alignment is paramount. in a pandemic, alignment, if clearly articulated, will potentially aid students in making meaning of the change and will be reassuring and decrease anxiety. in delivering anatomy education remotely and online within a changing pandemic landscape, constructive alignment remains a challenge; a simple effective process to ensure alignment with learning outcomes, activities, and assessment is, however, required. as saunders (2020) posits, distance teaching requires teaching practices to change, and "to change the way in which you teach is a significant undertaking." there are several fundamental elements of effective online pedagogy design (chen et al., 2005; martin and bolliger, 2018) employed in delivering anatomy teaching during the pandemic, these elements were variably highlighted and included providing feedback through quizzes, establishing social connections, active facilitation and using a range of technologies. one approach highlighted selecting a pedagogical element to emphasize through the delivery (e.g., active learning) rather than focusing on issues such as infrastructure, policies and budget. even with this foresight, the aspect of remotely delivering assessment requires more deliberation, and institutional resources (kirkwood and price, 2014) . it is expected that this is an area that australian and new zealand universities will need further investment in. at some stage, once the crisis is over, activities undertaken now will have to be reviewed. for this reason, clear documentation of expectations and modifications made (what, why, and how) will be useful. there is, however, little opportunity with the present pace of change, for an adequate change documentation process. uncertainties also remain around how documentation will be reviewed by professional accreditation bodies, and for australia, teqsa. no doubt there will need to be a targeted program of evaluating competencies and providing in-time education to address any perceived learner knowledge gaps. it is hoped that this compensatory process will also heed the evidence on the benefits of cadaveric-based learning for medical students (estai and bunt, 2016; flack and nicholson, 2018) . it will, however, be important to determine what perceived gap, if any, is being addressed before a compensatory activity is planned (wilson et al., 2019) . at that stage, and with time, there will also be opportunity to evaluate whether other digital resources and immersive learning experiences can be used to address knowledge gaps (doubleday et al., 2011; losco et al., 2017; birbara et al., 2020) . as we cope with this pandemic, being able to compromise, and adapt in practice and curriculum design and teaching delivery is important. anatomy academics' reflections on persevering and adapting to deliver anatomy education amidst constant and daunting change is possibly an indication of a commitment of anatomy educators to the education sector, student body, and discipline. given the limited time available for transition to remote/online delivery, the authors experiences demonstrated an almost consistent strategy of repurposing existing material. this enabled academics to concentrate on redesigning activities related to higher learning outcomes usually attained through the practical components of learning in the anatomy laboratory. some anatomy educators planned later opportunities after the pandemic to address perceived gaps in knowledge. evidence suggests that the quality of remote learning materials needs to be higher than campus-based materials (kimball, 1998) and this has been corroborated more recently in the practice of developing online content for a blended anatomy course (green and whitburn, 2016) . however, an important aspect of pandemic change for anatomy academics themselves is to adapt self-imposed standards. it will take confidence to heed calls to compromise self-imposed excellence standards. as saunders (2020) explains, academics incur additional stress in online teaching with the increased pressure that prerecorded videos, case studies, and discussion boards are a permanent record of their knowledge and expertise. given the context of the pandemic, and the speed of change, it befits entire academic community to model self-care and moderation to those around, and to accept that current makeshift resources may not be a true reflection of the educators' abilities, but it will be fit-forpurpose in response to acute transitional change. one of the challenges of change in the paradigm of physical distancing regulations, is that it almost instantly cauterized the existing community of practice academics access in their home departments and institutions. anatomy academics, however, appear to be resourceful in effectively engaging with a wider community of practice to share not just experiences and capabilities but also resources. this was evidenced in the almost immediate role change of technical staff and demonstrators to develop digital resources, and through the sharing of resources and of experiences using social media communities (e.g., twitter inc., san francisco, ca) and by software developers and societies of anatomy (evans et al, 2020) . smaller communities like local departments of anatomy, have also utilized technologies such as whatsapp © (facebook inc., mountain view, ca), to troubleshoot through challenges, and motivate and aid staff well-being. as in past times of crisis, a positive outcome of the shared challenge is that it unifies communities around common goals. while this is certainly a time of justified anxiety, it is also a time when the positive and resilient aspects of connectedness and community come together to encourage and build each other up-even with physical distance. notwithstanding this, the rapid onset of the covid-19 pandemic, highlights the need for the discipline and for departments to be proactive in keeping updated with new pedagogies and to be embedded in wider communities of practice outside home institutions. as negative as it sounds, it is essential at this time (as at any) to develop and communicate a continuity plan. in a global health emergency, circumstances are likely to arise that a team member is unable to continue their responsibilities. inevitably, the pandemic highlights to those in leadership roles, the need for contingency planning and investment in staff upskilling. the disruption of anatomy education in response to the pandemic may, however, also be perceived as an opportunity to expand anatomy education more broadly beyond the traditional f2f delivery model into the remote learning space. fully online models of anatomy education may have been impeded over the last decade by the lack of acceptance among anatomy educators. the quick response of the anatomy community in australia and new zealand demonstrates their flexibility and potentially enables the discipline to personalize education (gillett-swan, 2017) and to expand its reach to untapped learner pools, an increasingly important need as institutions enter a post-pandemic financial recovery phase. no doubt that whatever the context, the main inhibitor of remote anatomy education delivery will be the quality of the offering especially if deficient of a hands-on practical experience, time, support, and people capacity. some of the benefits of the pandemic-related disruption are already being recognized and include, for example, using live streaming to deliver lectures to cohorts of students distributed across multiple (sometimes even remote) campuses during clinical placements, thus addressing a challenge experienced by most australian and new zealand anatomy departments. this model could also assist students on extended periods of absence, like for example indigenous students attending community and family ceremonies and funerals. some anatomy academics have, indeed, perceived this rapid change as a novel opportunity to apply newly learnt pedagogies and delve into a freshly acquired space for modern education. there remain several limitations of this current profile of anatomy education during the early stages of the pandemic. while it uniquely captures a cross-section of experiences of delivering anatomy education in australia and new zealand, the reflections were not triangulated, and it does not include all anatomists or all anatomy departments in australia and new zealand. importantly, the participants narratives are a snapshot of their perceptions and views based on a one-off reflection during a period of crisis, and not a longitudinal account of their experiences. the constructivist approach to the study is an appropriate tool to examine anatomy education in the rapidly changing context of the pandemic, as it allows interpretation of experiences at different time points. a follow-up study with the participants post-pandemic would, therefore, provide deeper insights on how academic attitudes in this period of acute change have changed and matured, and how the experience has impacted their personal educational philosophies and their institutional practice of anatomy education. as the covid-19 pandemic continues to unfold in australia and new zealand, there is no doubt that university leaders and academics deserve recognition for their management and support to all the communities that they serve. their focused efforts in the face of global change, disruption, uncertainty, and potential volatility appears to ensure an effective and safe learning environment, and where possible, serve their nations and their institutions. for anatomy education, these decisions were particularly impactful in that they disrupted access to human donor body resources (a widely utilized discipline pedagogical tool) and forced change to ensure the continued delivery of anatomy teaching. several challenges were recognized by anatomy academics in this period including time, resources, and technical capability. professional and technical staff demonstrated role adaptability and assisted their academic colleagues in meeting teaching obligations. for anatomy academics, flexibility and adaptability enabled the continuity of anatomy education programs which were effectively fit-for-purpose in the pandemic context. interestingly, in person interactions between anatomy academic staff and students, an often-overlooked motivator, was uniquely highlighted as an integral part of the anatomy learning equation. of note, and in a time of crisis and uncertainty, anatomists appear to act and think like leaders-health and safety first, followed by academic standards, and developing flexible solutions for business continuity. there is no doubt that there has been a significant change in the teaching of anatomy in australia and new zealand over the last month. it remains, however, to be seen how the rapid change effected at this time correlates with student performance and satisfaction, and how the bridges that have been built impact on the future of anatomy education. the figure was designed in collaboration with kat orgallo, teaching resource support unit, faculty of medicine, nursing and health sciences at monash university. nalini pather, m.med.sci. (clin. anat.), ph.d., is a professor and the chair of anatomy at the university of new south wales faculty of medicine, sydney, nsw, australia. she teaches anatomy across multiple programs including medicine, allied health, medical science, and biomedical engineering students. her research group focuses on the applications of anatomy, visualization, and immersive technologies to clinical, and imaging specialties, and on medical education. using reflexivity to optimize teamwork in qualitative research constructive alignment in university teaching 3d modelling and printing technology to produce patient-specific 3d models virtual reality in anatomy: a pilot study evaluating different delivery modalities digital andragogy: a richer blend of initial teacher education in the 21st century self-regulated learning strategies and academic achievement in online higher education learning environments: a systematic review moral barriers between work/life balance policy and practice in academia a model for synchronous learning using the internet seven principles for good practice in undergraduate education exploring deep space -uncovering the anatomy of periventricular structures to reveal the lateral ventricles of the human brain swine flu: lessons we need to learn from our global experience student-faculty engagement a downward spiral: coronavirus spins australian universities into economic crisis. the guardian sars and mers: recent insights into emerging coronaviruses the virtual anatomy laboratory: usability testing to improve an online learning resource for anatomy education best teaching practices in anatomy education: a critical view going virtual to support anatomy education: a stop gap in the midst of the covid-19 pandemic human skills for human[istic] anatomy: an emphasis on nontraditional discipline-independent skills overseas students in australian higher education: a quick guide. parliamentary library research papers series what do medical students learn from dissection? the challenges of online learning: supporting and engaging the isolated learner student outcomes associated with use of asynchronous online discussion forums in gross anatomy teaching impact of introduction of blended learning in gross anatomy on student outcomes the relationship between student engagement with online content and achievement in a blended learning anatomy course university staff mental health literacy, stigma and their experience of students with mental health problems perspective: guidelines for reporting teambased learning activities in the medical and health sciences education literature cash or kudos: addressing the effort-reward imbalance for academic employees international medical education and future directions: a global perspective the role of history and ethics of anatomy in medical education knowledge maps: an online tool for knowledge mapping with automated feedback anatomy practical examinations: how does student performance on computerized evaluation compare with the traditional format? online mass exodus: how australian unis are coping with covid-19. itnews managing remote learning: new challenges for faculty effort-reward imbalance in academic employees: examining different reward systems technology-enhanced learning and teaching in higher education: what is 'enhanced' and how do we know? a critical literature review building professionalism in human dissection room as a component of hidden curriculum delivery: a systematic review of good practices effective methods of teaching and learning in anatomy as a basic science: a beme systematic review: beme guide 44 engagement matters: student perceptions on the importance of engagement strategies in the online environment thematic analysis: striving to meet the trustworthiness criteria how can we show you, if you can't see it? trialing the use of an interactive three-dimensional micro-ct model in medical education 3d printing from cardiovascular ct: a practical guide and review teaching anatomy: prosection and dissection australian broadcasting corporation, government of australia re-visioning academic medicine through a constructionist lens reducing regulatory burden during covid-19 -important update from teqsa. australian government tertiary education quality and standards agency (teqsa) how faculty learn to teach online: what administrators need to know metacognitive load -useful, or extraneous concept? metacognitive and self-regulatory demands in computer-based learning distance education research: a review of the literature how does donor dissection influence medical students' perceptions of ethics? a cross-sectional and longitudinal qualitative study commemorations and memorials: exploring the human face of anatomy australian universities defer exams, increase response to coronavirus outbreak cognitive load: updating the theory? synchronous and asynchronous communication in distance learning: a review of the literature who. 2020. world health organization. coronavirus disease 2019 (covid019) situation report -39 breaking with tradition: a scoping meta-analysis analyzing the effects of student-centered learning and computer-aided instruction on student performance in anatomy phil blyth, b.h.b., m.b.ch.b., ph.d. (bioeng.) , is a senior lecturer in elearning in medicine in the department of anatomy, school of biomedical sciences, university of otago, dunedin, new zealand. he coordinates the elearning team within the mbchb program and teaches anatomy within the medical and science programs. his research interests include medical education and he has developed orthopedic surgical simulation apps. he practices within the emergency department. jamie a. chapman, ph.d., is a senior lecturer in human anatomy at the tasmanian school of medicine, college of health and medicine, university of tasmania, hobart, tasmania, australia. he coordinates and teaches histology in the first 3 years of the mbbs course and coordinates a first-year, cross-campus human anatomy, and physiology unit. his research is focused on technology-enhanced learning and teaching.manisha r. dayal, ph.d., is a senior lecturer in human anatomy in the school of science at western sydney university, australia. she teaches gross anatomy, embryology, and forensic anthropology to medical and forensic science students including forensic mortuary practice students. her main area of research focuses on retention of anatomical knowledge through kinesthetic learning. , is an associate professor in human anatomy in the department of pharmacy and biomedical sciences, college of science, health, and engineering at la trobe university, bendigo, vic, australia. he is a discipline leader across the university campuses and teaches anatomy to a variety of students including physiotherapy and biomedical sciences. anneliese k. hulme, m.chiro., m.res., is a lecturer in the department of anatomy at the university of new south wales in sydney, australia. she teaches anatomy to medical and science students. her main research interest is in medical education.ian p. johnson, ph.d., is a professor of anatomy in the department of biomedical sciences, faculty of medicine, health and human sciences at macquarie university in sydney, nsw, australia. he is involved in undergraduate science and postgraduate medical education and his research is in neuroscience and medical education.amanda j. meyer, ph.d., is an anatomy lecturer in the school of human sciences at the university of western australia. she teaches neuroanatomy, histology, and gross anatomy to science, biomedical and medical students. her research focuses on the student experience (particularly anxiety and motivation) of anatomy learning. john w. morley, ph.d., is a professor of anatomy and cell biology and the deputy dean in the school of medicine, western sydney university, australia. his main area of teaching is topographical anatomy and neuroanatomy. his research is in sensory neuroscience (vision and touch), cns involvement in duchenne muscular dystrophy, and the development of visual prostheses. peter j. shortland, ph.d., is an associate professor of human anatomy in the school of science at western sydney university, australia. his main area of teaching is neuroanatomy but he also contributes to anatomy courses in musculoskeletal anatomy. his research is on the neuroanatomical plasticity that occurs in the spinal cord following nerve injuries, chemotherapy treatment or ingestion of toxic compounds that contribute to neuropathic pain.goran štrkalj, ph.d., is an associate professor in the department of anatomy education, school of medical sciences at the faculty of medicine, university of new south wales, sydney, new south wales, australia. he teaches anatomy and biological anthropology to science and medical students. his main research interests are in human variation and evolution, history of science, and medical/science education.mirjana , is an associate professor in the medical school, college of health and medicine at australian national university, canberra, act, australia. she leads the medical school technology-enhanced learning and teaching team. her research encompasses anatomy and its clinical application in a variety of contexts and she teaches anatomy to medical and science students. stephanie j. woodley, b.phty., ph.d., is an associate professor in the department of anatomy, school of biomedical sciences, university of otago, dunedin, new zealand. she teaches anatomy across a variety of undergraduate and postgraduate science and health science programs. her research is focused on musculoskeletal anatomy and imaging, and its application to clinical practice. michelle d. lazarus, ph.d., is an associate professor and deputy head (education) of the department of anatomy and developmental biology at monash university in melbourne, victoria, australia. she is also director of the centre for human anatomy education at monash university. her research focuses on exploring anatomy education curricular impacts on medical student professional identity and role formation. key: cord-316879-nbkvd0le authors: ashcroft, james; byrne, matthew h v; brennan, peter a; davies, richard justin title: preparing medical students for a pandemic: a systematic review of student disaster training programmes date: 2020-06-09 journal: postgrad med j doi: 10.1136/postgradmedj-2020-137906 sha: doc_id: 316879 cord_uid: nbkvd0le objective: to identify pandemic and disaster medicine-themed training programmes aimed at medical students and to assess whether these interventions had an effect on objective measures of disaster preparedness and clinical outcomes. to suggest a training approach that can be used to train medical students for the current covid-19 pandemic. results: 23 studies met inclusion criteria assessing knowledge (n=18, 78.3%), attitude (n=14, 60.9%) or skill (n=10, 43.5%) following medical student disaster training. no studies assessed clinical improvement. the length of studies ranged from 1 day to 28 days, and the median length of training was 2 days (iqr=1–14). overall, medical student disaster training programmes improved student disaster and pandemic preparedness and resulted in improved attitude, knowledge and skills. 18 studies used pretest and post-test measures which demonstrated an improvement in all outcomes from all studies. conclusions: implementing disaster training programmes for medical students improves preparedness, knowledge and skills that are important for medical students during times of pandemic. if medical students are recruited to assist in the covid-19 pandemic, there needs to be a specific training programme for them. this review demonstrates that medical students undergoing appropriate training could play an essential role in pandemic management and suggests a course and assessment structure for medical student covid-19 training. registration: the search strategy was not registered on prospero—the international prospective register of systematic reviews—to prevent unnecessary delay. global disasters, such as a pandemics or warfare, are events that cause a major disruption to health and social care, industry and economy, and community and education. 1 disasters on this scale result in substantial loss of life, and an immeasurable burden is placed on healthcare services to deliver core medical care. 1 disaster healthcare provision requires a collaborative approach that uses the expertise and skills of as many people as possible. much of what is formally taught in medical school is around the knowledge, skills and behaviours required of a physician for patients at the bedside. 2 however, the broad training medical students receive could be applied to disaster scenarios especially if supported with adjunct specialist training. the current medical student curriculum already covers a wide range of specialties, and some may argue it is stretched. however, the rising incidence of worldwide disasters and the impact of the current coronavirus (covid-19) pandemic has justified the need for disaster preparation training in medical students. 1 in some respects, students with disaster training may be better suited to assist in both clinical and non-clinical roles in disaster scenarios than redeployment of senior physicians with super-specialist skills and knowledge. curricula using multidisciplinary methods of simulation and human factors training have been proposed for implementation by the usa (association of american medical colleges 3 ) and europe (government of the federal republic of germany 4 computer science applied to medical practice, italy 5 ). however, at present, it is recognised that there is a brief or non-existent exposure to disaster training within current medical training curricula across the world, which may leave students unprepared for an intimidating and unfamiliar setting if assisting in the healthcare workforce. 1 the current covid-19 pandemic is rapidly driving the need for healthcare workers in the uk. 6 on 24 march 2020, the uk health secretary, matt hancock announced plans to introduce medical students as volunteers to the nhs in order to assist in this pandemic. 7 in response, the british medical association and medical schools council issued clear advice regarding medical students joining the uk healthcare workforce including ensuring correct induction, training and supervision. 8 the aim of this study was to systematically review disaster training courses for medical students. we describe the educational structure and methodology employed, and evaluate both preparedness for disaster medicine and learning outcomes to inform the development of covid-19-specific training programmes. we adhered to prisma (preferred reporting items for systematic reviews and meta-analyses) guidelines and recommendations for systematic reviews of observational studies. 9 we searched embase, medline and cochrane central from 1996 for all articles published until 19 march 2020 evaluating training that medical students receive to prepare them for pandemics and disasters, with no language restriction. we identified 1289 articles, which we then screened for inclusion. the search was conducted using the following medical search headings: 'coronavirus', 'covid-19', 'sars virus', 'disasters', 'natural disaster', 'major catastrophe', 'mass casualties', 'crisis event', 'extreme weather', 'disease outbreaks', 'infectious disease transmission', 'epidemics', 'pandemics', 'mass drug administration', 'warfare', 'biohazard release', 'chemical hazard release', 'radioactive hazard release', 'radiation exposure', 'radiation injuries', 'hazardous', 'waste', 'chemical water pollution', 'radioactive water pollution', 'medical students', 'medical schools', 'education' with terms exploded as appropriate. the search strategy was not registered on prospero-the international prospective register of systematic reviews-to prevent unnecessary delay. we selected randomised controlled trials, case-control studies and cohort studies that measured medical student training outcomes in the context of pandemics and disasters. studies were selected only if they contained a detailed report of the training implementation and used objective precourse and/or postcourse assessments related to medical student knowledge, attitude, skills or clinical care outcomes. importantly, if medical student outcomes were grouped with other healthcare students or professionals and not reported separately, the study was excluded. we excluded non-english language articles in order to ensure data quality, logistical training process evaluations, literature reviews, case reports, clinical trial proposals, conference abstracts, editorials, letters and articles evaluating non-medical student populations only. pandemic infections that may be secondary issues to a disaster, but were not the primary cause, were also excluded, for example, hiv, dengue, malaria; situations where medical students were unlikely to be required to volunteer en masse, for example, active shooter situations; and interventions that were not in a disaster setting, for example, basic life support and routine clinical infection control procedures were excluded. duplicates were removed and two reviewers (ja and mhvb) independently screened titles and abstracts using rayyan, an online software to aid blinded abstract screening. 10 any discrepancies were resolved by consensus. of the 1289 citations screened, we identified 65 articles for full text and reference review, of which 23 final studies met the inclusion criteria for data synthesis (figure 1). two reviewers (ja and mhvb) independently extracted relevant information from each training report using standardised data extraction proforma in keeping with best evidence medical education recommendations and one author with medical training expertise (rjd) reviewed all extracted data. 11 we recorded administrative information including authorship, institution and year of publication; training-related data including details and duration of intervention, participants and teaching methods; and quantitative and qualitative outcome measures. 11 the quality of training intervention and risk of bias in reporting of results was assessed using the robins-1 for non-randomised controlled trials. 12 meta-analysis was not performed on the training outcomes assessed due to the wide heterogeneity in training interventions and reporting of results. descriptive analysis was performed instead. interventions were assessed against kirkpatrick criteria and kirkpatrick's levels were assigned: impact on learners' satisfaction (level 1), changes in learners' attitudes (level 2a), measures of learners' knowledge and skills (level 2b), change in learners' behaviour (level 3), changes to clinical processes/organisational practice (level 4a) and benefits to patients (level 4b). 11 twenty-three studies met the inclusion criteria, and their characteristics are displayed in table 1. the majority of studies (n=18, 78.3%) were from the usa, and other countries were germany (n=1), israel (n=1), italy (n=1), saudi arabia (n=1) and south korea (n=1). five studies (21.7%) involved a multidisciplinary cohort and reported outcomes for other healthcare students and professionals, as well as outcomes for medical students individually without pooling of results. the course structures and learning objectives were grouped into three categories: broad concepts in disaster medicine (n=12, 52.2%), trauma or haemorrhage mass casualty management (n=8, 34.8%), or influenza pandemic management, airborne viral management or personal protection (n=3, 13%). the length of studies ranged from single day teaching to 4-week boot camps, and the median length of training was 2 days (iqr=1-14). the majority of training interventions used traditional didactic lectures with simulative or experiential teaching methods, with 12 courses (52.2%) containing lectures and simulation. of the simulation experiences, four courses (17.4%) contained outdoor actor-based mass casualty simulation. multimedia approaches were used in eight courses (34.8%) as an adjunct to training, often precourse, in order to efficiently provide material to attendees. problem-based learning or casebased learning was used as a predominant feature in five courses (21.7%) in a classroom setting with or without other teaching methods. no courses involved only didactic teaching methods, all 23 included studies were prospective cohort studies measuring the impact of their training intervention postcourse evaluation, with 18 using precourse evaluation for comparison as displayed in table 2. the majority used subjective assessments of knowledge or preparedness in disaster medicine (n=20, 87.0%) with nine studies (39.1%) using objective measures. there was a wide range in number of medical students attending the courses, with the median number of participants being 61 (iqr 24-123). however, five studies did not clearly describe their medical student population, either omitting total number of participants or seniority of students. only two studies 13 14 reported data that assessed longitudinal learning beyond the year of course implementation. common limitations of study design included training being limited to a single institution, studies which were excluded due to being randomised controlled trials evaluating the efficacy of different teaching methods or simulation technology, or studies excluded due to reporting medical student outcomes pooled with outcomes of other healthcare student or professionals. of the studies included in this review, 18 studies in total measured precourse and postcourse outcomes. of these 18 studies, knowledge was measured in 16 (88.9%), of which 10 undertook objective knowledge measurements, making this the most measured outcome. however, none of the subjective or objective measures of knowledge were previously described or undertaken with validated measures. a total of four studies measured attitude-either preparedness for disaster (n=2) or confidence in approaching a disaster (n=2) by precourse and postcourse assessment. disaster medicine skills were subjectively measured in four studies and objectively measured in two, one being pass rates in personal protective equipment fitting and the second being accuracy of disaster triage. in these studies, validated measures were used to create scores or pass rates. of the five studies inviting a multi-disciplinary participant group, none trainee satisfaction was assessed in 13 (56.5%) studies and most commonly assessed using likert type scales. medical students were asked to rate the overall quality of the courses in addition to whether they would recommend courses to colleagues for disaster preparedness. course satisfaction was generally very high and appeared to be enhanced by multimodal approaches to curriculum design including the incorporation of simulation and technology. 15 however, one group discussion and interactive activity-based study did report mixed reviews with postcourse overall ratings of positive (33%), undecided (13%) and negative (54%). this is reflective of overburdening medical students with work, with one student stating, "this was way more work than it should have been. i would rather have an hour lecture on the flu than do all that group stuff. this was just frustrating to have at the end of the year when finals are right around the corner". 16 attitudes and perceptions of knowledge in medical students were assessed in 10 studies (43.5%). attitudes were broadly assessed as level 2a following courses measuring either a simple change such as interest in disaster medicine, a measurement of a medical student's willingness to volunteer in or preparedness to practice disaster medicine, or by mapping trainee responses to learning objectives. level 2b was measured by a total of 20 studies (87.0%) which assessed medical student knowledge or skill acquisition, with 18 assessing knowledge and 11 assessing skills, either alone or in combination. in courses training students in mass casualty scenarios, discrete and measurable skills were easily assessed including tourniquet application and triage skills. 15 17 as behavioural change is a difficult area to measure in nonpracticing medical students, only one study was deemed to adequately assess behavioural change. this study assessed confidence and perceived stress handling emergencies once the medical students had graduated 1.5 years following the course. 13 no studies investigated the impact of disaster training on clinical performance (level 4a) or organisational delivery of care (level 4b). all included studies were cohort studies and risk of bias was assessed using robins-1 (figure 2). 12 risk of bias was low to goolsby, 15 2014, university of the health sciences, usa postcourse questionnaire assessing perceived confidence at assessment and procedures in a combat casualty situation and perceived preparedness at managing combat casualties. the majority of students feel more confident and better prepared to assess and perform procedures in a combat casualty situation after course. the majority of students preferred the high-fidelity simulation to their normal learning environment. attitude, skill 1 to 2a ingrassia, 29 2014 crimedim, italy precourse and postcourse examination assessing disaster medicine knowledge. triage accuracy was measured in a disaster simulation assessed by an examiner. there was a significant improvement in knowledge of disaster medicine after course from 40% to 83% (p<0.01), and a significant improvement in triage accuracy in the disaster medicine simulation after course from 45% to 78% (p<0.01). the majority of students felt that disaster medicine should be part of their curriculum and evaluated the course highly. knowledge and assess learning of the didactic material immediately before the 90 min case-based lecture, and the posttest immediately after the lectures. the 2009 class could take their pretest via an e-learning tool up to several days before the class, and the post-test was available online for 3 weeks after completion of the course. over a 2-year assessment period the first year cohort's post-test knowledge scored improved from 3.8/10 (below average to average) compared with 7.6/10 (average to above average) and the second year's post-test scores improved from 2.5/5 (average) before and 3.8/5 (above average). in the first year cohort he average overall rating for the experience was 4.9/5, and 100% of the respondents recommended the class for next year's students allowing it to continue. knowledge, attitude 1 to 2b scott, 41 2012, medical university of south carolina, usa participants undertook precourse and postcourse assessment developed to meet learning objectives of the course. selfassessment of personal capability and comfort to handle a disaster and multiple choice questions of knowledge and subjective skill were undertaken. most (70%) of the trainees considered their emergency preparedness knowledge and skill as average or below average before the training experience. after the curriculum, 100% of trainees considered their emergency preparedness knowledge and skill above average, and 90% would recommend the course to other healthcare workers. knowledge, skill 1 to 2b scott, 14 2013, medical university of south carolina, usa participants undertook an online precourse and postcourse assessment developed to meet the learning objectives and competencies of the course in addition to giving post-test feedback on the implementation of the course. in discrete knowledge, subjective knowledge and skills all participants demonstrated significant improvements in their postcourse test results when compared with pre-test. course evaluation was performed, and it was found that students would recommend this course (median 92.5%), whether the course was feasible (median 82.5%) and overall evaluation (94.5%). knowledge, skill 1 to 2b silenas, 16 2008, the texas a&m college of medicine, usa students answered likert type scales to assess the extent to which the objectives and understanding of key concepts had been accomplished. written and verbal comments from the students and facilitators about their experience were gathered. sixty-six medical students completed the knowledge test before and again 4 days after the avian influenza exercise. the lowest scores for knowledge were best and all tested knowledge areas except one (endemic influenza as a public health issue) decreased postcourse. the course received mixed ratings which overall were positive (33%), undecided (13%) and negative (54%). knowledge. knowledge, skill 2b table 3 continued figure 2 individual risk of bias for non-randomised control trials determined by robins-1. overall risk of bias for non-randomised control trials determined by robins-1. critical ( figure 3 ). confounding bias was serious overall as many studies did not present pretest control data. there was critical overall bias in the selection of participants as courses were often not open to all students, for example, self-selected recruitment from student emergency medicine interest groups. classification bias and bias due to deviation from intended interventions was low. a single study had a classification bias because they did not adequately describe their educational intervention-as all other studies had low bias, this domain was classified as having low overall bias. overall missing data bias was moderate as three studies had some form of missing data and were not able to adjust for this in their analysis. measurement of outcomes had serious bias overall, as many questionnaire evaluations were subjective without any objective measures. selection and report of results had serious bias overall. while some studies did note ethical approval there was no priori registration of the results and some studies had limited reporting of results. this systematic review identified approaches used to train medical students in disaster medicine in order to suggest training approaches for medical students in the current covid-19 pandemic. we identified 23 studies published between 1996 and march 2020. overall, medical student disaster training programmes improved student disaster and pandemic preparedness and resulted in improved attitude, knowledge and skills. there was an improvement in all studies that measured precourse and postcourse outcomes. we found that all interventions ranging from simple classroombased interactive discussion to complex multimodal simulative experiences resulted in improved knowledge, skill and attitudes towards participation in disaster medicine. the main outcomes of the courses reviewed were subjective; however, there was evidence to suggest that disaster medicine training does improve objective knowledge and can teach skills which can be used by medical students, relevant to a pandemic. the majority of courses were just 1 day in duration, indicating that short courses can still be impactful. the courses identified in this review required expert faculty or high-fidelity equipment and were implemented alongside an already busy medical school curriculum. these barriers prevented the majority of courses in this review from reaching longitudinal integration into medical school training. however, this may be overcome in the current covid-19 pandemic by collaboration and coordination, particularly when many medical students have had their studies either postponed or converted to telemedicine/online teaching. the main limitations of this review are related to study design, as the majority of studies were single centre and often focused on very specific aspects of disaster medicine. the overall reporting of both participant factors and outcome factors was generally poor, and the educational methodology was very heterogeneous-this was represented by critical risk of bias in selection of participants, and serious risk of bias in measurement of outcomes. this bias inevitably weakens the strength of the conclusions drawn, but given that all studies demonstrated a positive benefit, it can still be concluded that there will be benefit to students who undertake disaster preparedness courses. another limitation was the kirkpatrick levels that were evaluated. only one study evaluated change in behaviour (level 3) and no studies evaluated change in clinical performance (level 4a) or organisational patient benefit (level 4b). furthermore, only three studies focused solely on pandemic influenza, airborne viral management or personal protective equipment (n=3, 13%), and only a single study assessed resuscitation in a disaster setting. this is of particular importance for the covid-19 pandemic, where respiratory personal protective equipment is a necessity and there are specific resuscitation guidelines. 18 clinical impact and clinical utility must be taken into account when making suggestions for training during the covid-19 pandemic. although medical students working during the covid-19 pandemic will likely be deployed to non-infectious areas of work, there is no guarantee that medical students will not be exposed to the virus. 19 furthermore, a strain will be placed on healthcare services and contingency care may need to be provided in place of a traditional care service. 20 here, students may be essential in ► medical students could play a crucial role in the sars-cov-2 healthcare response. ► disaster medicine programmes using multimodal techniques improve knowledge, skills and attitudes which are imperative for medical practice in a pandemic. ► training programmes incorporating previously successful techniques could ensure the successful integration of disaster training into global medical school curricula. ► do disaster training programmes aimed at medical students demonstrate direct patient benefit? ► can disaster training programmes improve the integration of medical students into the healthcare workforce during the sars-cov-2 pandemic? ► how can disaster training programmes be adapted to manage future pandemics? preserving the resilience of hospitals and community healthcare systems. 20 there will ultimately be more pressure on medical students to work than previous cohorts and this review suggests that disaster medicine training as a part of medical school's curriculum is not common practice. therefore, medical students may require a very different set of competencies than those acquired during medical school. unsurprisingly, some final year medical students do not feel ready to start as a newly qualified doctor, due to worries they are not well prepared for clinical placements, or feeling under prepared for covid-19. 21 moreover, the medical schools council have advised that medical students from any year should not take on roles that will impact on their studies. 8 this review suggests that early mobilisation of medical students into the workforce could be accompanied by disaster medicine training. all courses reviewed in this study were positively evaluated by medical students, and if a similar programme was offered to current medical students, it would likely be well received improving willingness and preparedness to work in the healthcare service. this is of particular importance as medical students are already being asked to join the workforce as volunteers, or to graduate early in order to join healthcare systems as physicians. there is great concern that students who give assistance during a disaster without training are at an increased risk of both harm to themselves and psychological consequences. 22 there is therefore a need to create novel courses to teach medical students pandemic skills in these unprecedented circumstances. this review suggests that the most beneficial medical student disaster medicine courses should consist of mixed modalities of didactic sessions, case-studies, practical hands on training and simulation experiences. these training methods could be used to train medical students in covid-19 specific knowledge and skills and prepare them for clinical practice. table 4 shows a proposed covid-19 course and assessment based on the findings of this systematic review. the course structure includes the variety of elements found in other studies. didactic lectures on covid-19 could be delivered in a lecture hall with social distancing measures in place, or perhaps more appropriately as a distance learning component consisting of video, podcast and computer activities. practical activities could include fitting of respiratory personal equipment as well as donning and doffing. the simulated element could consist of a patient with covid-19 who requires cardiopulmonary resuscitation. in resource-limited scenarios, this could be undertaken using computer-based tutorials or video tutorials. as new doctors and medical students may have a substantial volume of information to learn in addition to this course, handouts and online refresher courses should be offered. the proposed assessment aims to cover all kirkpatrick levels and criteria. it is also important to teach and train human factors awareness, particularly in relation to team dynamics, lowering authority gradients and empowering anyone to speak up if concerned. 23 maintaining both individual and team situational awareness is also important during any clinical duty, and even more so during a crisis setting. 23 it may be useful to incorporate a credentialing process for medical students undergoing disaster training, thereby allowing students to demonstrate a background of competency and separating this cohort from unskilled volunteers when aiding a disaster medicine response. 24 the successful implementation of these suggested disaster training techniques will require the encouragement of people-centred training, the development of peer-learning, coordination and funding of training systems, and regular disaster preparedness exercises of multimodality format. 25 the covid-19 pandemic has caused unprecedented disruption to healthcare services in peacetime. medical students may play a crucial role in the healthcare response. there is an imminent demand for educational interventions to train medical students to better assist in this response. the disaster medicine courses reviewed in this article improved knowledge, skills and attitudes through multimodal techniques and were well received original research by learners. although no studies in this review demonstrated direct patient benefit, the courses increased student preparedness and similar courses should be implemented prior to medical students joining the healthcare workforce during the covid-19 pandemic. future courses should note the methodological and longitudinal flaws demonstrated in previous studies so that direct patient benefit can be demonstrated in the covid-19 pandemic. future work should be undertaken to ensure the successful integration of disaster training into global medical school curricula. twitter james ashcroft @jamesashcroftmd, matthew h v byrne @mhvbyrne, peter a brennan @brennansurgeon and richard justin davies @jdcamcolorectal contributors ja and rjd undertook conceptualisation of this article. ja and mhvb undertook data collection, analysed the data, and drafted and revised the manuscript. rjd and pb undertook critical review of all data collection and analysis and guided critical revisions. rjd approved final manuscript for submission. funding the authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors. the importance of evidence-based disaster planning achieving good medical practice training future physicians about weapons of mass destruction: report of the expert panel on bioterrorism education for medical students code of federal regulation 8. nouvelle of german medical licensure act. fed law gaz i nationwide program of education for undergraduates in the field of disaster medicine: development of a core curriculum centered on blended learning and simulation tools clinical features of patients infected with 2019 novel coronavirus in wuhan, china the guardian. matt hancock calls for 25000 volunteers to help the nhs. guard medical schools council. medical student volunteers in the nhs preferred reporting items for systematic reviews and meta-analyses: the prisma statement rayyan-a web and mobile app for systematic reviews conducting a best evidence systematic review. part 1: from idea to data coding. beme guide no robins-i: a tool for assessing risk of bias in non-randomised studies of interventions evaluation of a new community-based curriculum in disaster medicine for undergraduates competency in chaos: lifesaving performance of care providers utilizing a competency-based, multi-actor emergency preparedness training curriculum new wide area virtual environment (wave) medical education developing disaster preparedness competence: an experiential learning exercise for multiprofessional education stop the bleed training empowers learners to act to prevent unnecessary hemorrhagic death resuscitation council uk. adult advanced life support for covid 19 patients, 2020. 19 uk foundation programme. allocations to interim f1 posts refining surge capacity: conventional, contingency, and crisis capacity covid-19-medical students face disruption and uncertainty why medical students should be trained in disaster management: our experience of the kashmir earthquake improving patient safety: we need to reduce hierarchy and empower junior doctors to speak up willingness of health care personnel to work in a disaster: an integrative review of the literature united nations office for disaster risk reduction. sendai framework for disaster risk reduction deployment and disaster medicine in an undergraduate teaching module planning and incorporating public health preparedness into the medical curriculum integrating the bleeding control basic course into medical school curriculum nationwide program of education for undergraduates in the field of disaster medicine: development of a core curriculum centered on blended learning and simulation tools a disaster medicine curriculum for medical students stop the bleed training empowers learners to act to prevent unnecessary hemorrhagic death training medical students in bag-valve-mask technique as an alternative to mechanical ventilation in a disaster surge setting herd immunity" in a civilian community through incorporation of "just-in-time using problem-based learning for pandemic preparedness the education and practice program for medical students with quantitative and qualitative fit test for respiratory protective equipment simulation training for in-flight medical emergencies improves provider knowledge and confidence a short medical school course on responding to bioterrorism and other disasters disaster preparedness medical school elective combating terror: a new paradigm in student trauma education disaster 101: a novel approach to disaster medicine training for health professionals high-fidelity multiactor emergency preparedness training for patient care providers teaching mass casualty triage skills using immersive three-dimensional virtual reality teaching mass casualty triage skills using iterative multimanikin simulations disaster training in 24 hours: evaluation of a novel medical student curriculum in disaster medicine competing interests none declared. provenance and peer review not commissioned; externally peer reviewed. this article is made freely available for use in accordance with bmj's website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by bmj. you may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained. james ashcroft http:// orcid. org/ 0000-0003-2964-5032 matthew h v byrne http:// orcid. org/ 0000-0002-2414-352x key: cord-338298-83htflfj authors: franchi, thomas title: the impact of the covid‐19 pandemic on current anatomy education and future careers: a student’s perspective date: 2020-05-05 journal: anat sci educ doi: 10.1002/ase.1966 sha: doc_id: 338298 cord_uid: 83htflfj as both an anatomy student and anatomy demonstrator, anatomical sciences education has become my go-to journal to further my knowledge of the latest developments in anatomy education, innovations in pedagogical practice, and indeed perspectives on current affairs in the field. to the editor, anatomical sciences education: as both an anatomy student and anatomy demonstrator, anatomical sciences education has become my go-to journal to further my knowledge of the latest developments in anatomy education, innovations in pedagogical practice, and indeed perspectives on current affairs in the field. the world is experiencing an ongoing and serious pandemic, and i felt it necessary to comment, from the perspective of a student, on the impact this pandemic has had and is having on students' anatomical education, and my concerns about the implications it might have on the future of current anatomy students. in the remainder of this letter, i wish to present some of my reflections on this matter. readers will be aware that the outbreak of the novel coronavirus began in wuhan, china, in late december 2019 (zhu et al., 2020) , and spread exponentially in our age where increasing urbanization and frequent international travel allow for the uninterrupted transmission of infectious diseases (alirol et al., 2011) . the first reported case in the united states was on 20th january 2020 (holshue et al., 2020) , and the first cases in the united kingdom was detected on 31st january (moss et al., 2020) . the world health organization named the disease caused by this severe acute respiratory syndrome coronavirus 2 (sars-cov-2) as covid-19 on 11th february (who, 2020a) , and subsequently labeled it a pandemic on 11th march (who, 2020b). on 2nd april, the number of cases reported worldwide crossed 1 million, with 205 countries and territories affected (worldometer, 2020). as governments struggle to contain the vicious spread of covid-19, and with over a third of the world's population currently under some form of lockdown (kaplan et al., 2020) , the effects the virus has had on people's daily lives is clearly like nothing most people have experienced before. one of the many affected sectors is education (unesco, 2020). in the united kingdom, the majority of universities and medical schools had suspended face-to-face teaching by 17th march 2020 (staton and jack, 2020) , forcing students to move to online distance learning for the indefinite future. pandemics are not a new occurrence. indeed, galen of pergamon documented a 20-year smallpox pandemic starting in 166 ad (mattern, 2011) , and more recently the world experienced the 2003 severe acute respiratory syndrome (leduc and barry, 2004) and 2009 h1n1 swine flu pandemics (collignon, 2011) . although the covid-19 pandemic is likely to be the first which current anatomy students are affected by, lessons and experiences can be drawn from the previous crises to help us adapt and continue education. for example, web-based learning was already successfully used in the 2003 pandemic to minimize the reduction in education that students received (patil et al., 2003; lim et al., 2009) , and is once again being used to great effect (lim, 2020) . it is of course entirely possible for students to learn anatomy without a cadaver and solely from textbooks and online resources (mcmenamin et al., 2018) , and indeed a number of medical schools no longer use cadaver dissection (patel et al., 2015) . however, the disappeared practical teaching at the hands of covid-19, regardless of whether students normally receive cadaveric teaching or not, will most likely have many lasting impacts on students. in light of the learning environment now being far less than optimal, the loss of face-to-face contact and direct interactions with both peers and teachers may potentially stunt students' development as anatomists. despite the advances in technology that allow for online distance learning, acquiring anatomical knowledge in the laboratory, ideally through cadaver dissection, is often still regarded as not only a rite of passage but also the most effective method (ghosh, 2017) . when students lost access to dissection rooms, they lost access not only to cadavers, but also to a range of other optimal learning modalities: prosections, models, pathology specimens, skeletons, and others (sugand et al., 2010) . previous authors have highlighted that the modern medical curriculum already restricts students' exposure to anatomy (warner and rizzolo, 2006) , and indeed this pandemic has further shortened the contact time current students have received. as a result, current anatomy students are being taught anatomy without access to practical-based learning materials, be that cadavers, prosections, or models. anatomy learning without cadavers is a practice which is generally seen as less favorable, but one which has arguable merits and has been used as standard in many institutions (mclachlan et al., 2004) , but when prosections, models, and other learning materials are also removed, learning becomes difficult. adaptation to online distance learning is no easy task for students or teachers, and simply providing an online atlas is unlikely to provide students with an "appreciation for the fabric of the human body" (gregory and cole, 2002) . despite there being a large number of online anatomy software programs available for students to use, they can often be costly. institutions that can afford to do so should endeavor to give their students access to these during the current situation. however, to account for equality of opportunity between institutions with varying financial freedom, i would implore software companies to consider providing all anatomy students with temporary free access to their programs during the pandemic. having said this, previous studies have demonstrated that, despite being useful, online programs provide significantly lower rates of self-perceived learning and satisfaction compared to dissection (mathiowetz et al., 2016) . there is also a steep learning curve associated with using these programs for both teachers and students (doubleday et al., 2011) , with many students finding it difficult to manipulate models and focus on structures of interest (attardi et al., 2016) , thus further bringing into question their usefulness in times as challenging as a pandemic. however, studies investigating the efficacy of purposely designed, solely online programs have not yet been done. indeed, the covid-19 crisis may serve to inform us on whether such approaches are able to deliver appropriate learning gain. further, if these online programs were suitably integrated into curriculum design and used to carefully guide students through a learning journey, rather than simply made available as another resource, then perhaps these tools could prove very beneficial. mixed methods of teaching and learning anatomy in the current crisis are clearly needed. the addition of instructional dissection videos goes part of the way toward normality, where students can essentially watch a prerecorded dissection taking place (langfield et al., 2018) . indeed, even direction toward appropriate youtube videos can help students to understand anatomical concepts (jaffar, 2012) . in conjunction with online digital photographs of cadavers, interactive anatomy images, and the provision of self-testing tools (o'byrne et al., 2008) , students may start to feel supported in their online distance learning. the importance of personal online interactions cannot be overlooked however, and attempting to reduce the distance between learners through provision of chat rooms or real-time tutorials is a key element to successful online learning (stone and barry, 2019) . it would seem that a purposely designed online course which integrates a number of elements into a learning journey would potentially provide a solution to the current pause in face-to-face teaching. further, a modern ideal for home learning of anatomy would be in the form of virtual reality (vr) resources (erolin et al., 2019) , and although we are not quite yet in the age where this technology is a household staple, there are elements of it which could be adapted to a remote learning environment. indeed, most smartphones are compatible with google cardboard allowing students to experience vr from their own phones, if provided with suitable software (izard et al., 2017) . the implication the pandemic has on summative assessment is a further worry for students. among the plethora of modalities used to assess anatomy are the written spotter examination (smith and mcmanus, 2015) and oral viva (evans et al., 2014) . however, these methods can obviously not be used in the current situation. with a move to online examinations seemingly inevitable for this year's global cohort of anatomy students, i question their preparedness for this form of assessment and wonder whether students will perform to the standard that they might have if their examinations were in the modality they were planned to be. this being said, studies have demonstrated that students tend to score similarly regardless of whether the examination is practical or online (inuwa et al., 2012) . although many students already use anatomy flash cards and digitized spotter-like tests in their learning, i would call on institutions to provide students with clear guidance on the adjusted format of their examinations and to provide ample opportunity for no-stakes practice of these new modalities. indeed, it has been raised by previous authors that traditional spotter examinations are arguably not an effective assessment technique, as they focus almost entirely on testing a students' ability to recall information (choudhury et al., 2016) . therefore, perhaps the covid-19 pandemic presents institutions with an opportunity for innovation in assessment approaches that allow for accurate representation of both a student's knowledge and understanding of anatomical sciences. aside from the challenges relating to continuing students' anatomical education online, the covid-19 pandemic also raises issues relating to current anatomy students' futures. students consider working with cadaveric material a crucial part of their development toward becoming a professional in the field (smith et al., 2014) , be that as a doctor, dentist, or biomedical scientist. clinically meaningful learning of anatomy is crucial to students' understanding of the relevance of their knowledge to future practice (collins, 2008) . as such, students' clinical understanding and appreciation for the relevance of anatomy might well suffer due to the current lack in practical teaching, at the detriment to their future, and so adaptive institutions must ensure that online learning resources do not lose this important clinical relevance (turney, 2007) . as an aspiring surgeon, i am particularly concerned about the reduction in dissection experience that current anatomy students received (drake et al., 2014) . cadaver dissection is an invaluable opportunity for the development of fine motor skills in a stress-free environment (krähenbühl et al., 2017) , and so i wonder what implications this might have on the future of students in similar positions to myself. indeed, this worry extends further to the future of surgery in a much wider sense. poor anatomy teaching at medical school is often cited by students as a reason for not considering a surgical career (cooper and gray, 2014) , and although the current situation of anatomy teaching is not intentional, it is possible that the quality of teaching that students are now able to receive may be of lower than prior to the pandemic. whether the pandemic causes a drop in applications to surgical training posts for this year's students will not be known for many years, but abandoning dissection has proven detrimental to the competency of future surgeons (memon, 2018) . a further compounding factor on this issue is the fact that many students discover their love or natural talent for surgery as an anatomy student-"gross anatomy […] may also be an unrecognized fork in the road in [students'] pursuit of choosing a medical specialty" (archibald and carlson, 2009 ). perhaps through this disappeared practical teaching, we are losing the opportunity to discover the next top surgeons of our time, or creating students that had a distinct gap in their anatomical understanding. for those students who wish to become future anatomy teachers, the lack of complete and ongoing exposure to a variety of teaching and learning techniques may well impact on the styles and methods they will later employ as teachers. without giving them the opportunity to consider best teaching practices from their perspectives as students (estai and bunt, 2016) , i question whether they will feel suitably prepared to enter the employment market with the confidence that they possess not only the anatomical understanding but also the pedagogical experience to become effective educators of the future. on the other hand, students in this situation might wish to seize the opportunity to expose themselves to a broader range of teaching techniques than they might have otherwise encountered. many institutions worldwide have now made prerecorded lectures freely available to the general public, and so students could experience other teachers' styles from around the world. alternatively, students could experiment with online peer teaching groups, and take turns delivering a short anatomy tutorial to their peers through a video conference with the opportunity to receive feedback on the effectiveness of their teaching style. for those more inclined toward practical work, students could experiment with more artistic techniques, such as body painting, anatomical drawing, clay work, or the use of pipe cleaners in order to see what techniques work for them and which they therefore may wish to trial when teaching (mcmenamin, 2008; lefroy et al., 2011; kooloos et al., 2014) . there are also a number of practical issues to consider as a result of the pandemic. with students no longer allowed to attend face-to-face teaching, the cadavers they were working on may now not be fully utilized, depending on individual institutional set-up. aside from the many implications this has on students' learning, as outlined above, it is also a sad situation as it is not what the donors wanted. however, perhaps dissection laboratory staff could utilize these cadavers for making prosection materials, creating image libraries, or using them for specialist short courses, in order to ensure that the donors are used for the highest education benefit which the circumstances allow. an argument could of course be made for allowing students to complete their dissection in the new academic year, and indeed most embalming techniques would allow for the cadavers to still be usable by that time (brenner, 2014) , however this is clearly logistically difficult from both storage space and time commitment perspectives, and so would vary in viability between institutions. indeed, even if this provision were possible, graduating students would still miss out. it is important to note that missed practical experiences are a disappointment to both students and their teachers, not just the students. what is clear from the contributions regularly published in anatomical sciences education is that the anatomical community is a very creative and adaptive one. anatomy educators will undoubtedly do their upmost to accommodate students who wish to regain at least part of the cadaveric experience which covid-19 has caused them to lose, and this is something which students can take comfort in. the immediate future of dissection is called into question also, with the indefinite suspension of the vast majority of body donation schemes to universities and hospitals (hta, 2020) . it is obvious that this will result in a severe shortage of donor bodies for the incoming academic cohorts, which in turn will have significant influence on the modality and quality of teaching which they will receive. with the potential risks associated from coming into contact with people who died from covid-19 (finegan et al., 2020) , it is unclear when and how body donation schemes will restart. nevertheless, even when the human tissue authority was introduced in the united kingdom in 2004 following a national scandal of organ retention without consent (sheach leith, 2007) , the drop in number of body donors still recovered, so we can only assume that they will do so again after this crisis. this issue does however lead me to question whether this pandemic may leave us with lasting change on how anatomy education and indeed wider university education is carried out (jones, 2020) , just as it is likely to leave us with a realization that many of our social norms are obsolete, like traveling to work at an office (hern, 2020) . potential educational disruption and uncertainty about students' futures are no doubt two of the unavoidable by-products of the pandemic we currently live through, but there is also a more fundamental emotional experience which many anatomy students may now be facing. it is not just anatomy which students learn from the body donors. indeed, students develop personal and professional competencies through interactions with their donor (weeks et al., 1995) , and build a certain special emotional relationship with them over the course of their program. the fact that students learn a whole range of nontraditional discipline-independent skills (ntdis) through their study of and interaction with anatomy should not be disregarded (evans and pawlina, 2015) . for students who learn on cadavers, the donor is their first patient, and for those who learn anatomy through modalities, the use of human representations symbolizes the future patient . students' internal and external development of emotional intelligence, situational awareness, and professional behaviors, as well as personal feelings of love and empathy are all catalyzed in the anatomy laboratory through interactions with peers, teachers, technicians, academics, and the donors. it is clear that anatomy curricula teach students much more than just anatomy, and ntdis are a crucial element of this (evans and pawlina, 2015) . in these challenging times, it is therefore especially important for students to be aware of their ntdis sets, and to be adaptable and resilient to their circumstance . as a result of prematurely leaving the laboratory, i fear many students will be left feeling guilty that they have not yet had the opportunity to say thank you and goodbye to their donor bodies-"a necessary ritual for students" (boeckers and boeckers, 2016) . although there will be opportunities for students to pay their respects, these will undoubtedly not occur in the usual manner and so may not serve their full purpose for some students. i therefore wish to end this letter by asking students to spend some time reflecting on their experiences in the anatomy laboratory prior to the covid-19 pandemic, and to look forward to the time when they will be allowed to return to it. urbanisation and infectious diseases in a globalised world the impact of gross anatomy on the future head and neck surgeon mixed methods student evaluation of an online systemic human anatomy course with laboratory the dissection course-a psychological burden or an opportunity to teach core medical competencies: a narrative review of the literature human body preservation-old and new techniques the intelligent anatomy spotter: a new approach to incorporate higher levels of bloom's taxonomy swine flu: lessons we need to learn from our global experience modern approaches to teaching and learning anatomy lack of anatomy training could lead to shortage of surgeons. the independent the virtual anatomy laboratory: usability testing to improve an online learning resource for anatomy education an update on the status of anatomical sciences education in united states medical schools using virtual reality to complement and enhance anatomy education best teaching practices in anatomy education: a critical review motivating student learning using a formative assessment journey the role of the anatomist in teaching of nontraditional discipline-independent skills human skills for human[istic] anatomy: an emphasis on nontraditional discipline-independent skills icrc advisory group on the management of covid-19 related fatalities. 2020. international committee of the red cross (icrc): general guidance for the management of the dead related to covid-19 cadaveric dissection as an educational tool for anatomical sciences in the 21st century the changing role of dissection in medical education covid-19 could cause permanent shift towards home working. the guardian washington state 2019-ncov case investigation team. 2020. first case of 2019 novel coronavirus in the united states hta. 2020. human tissue authority. body donations: frequently asked questions. human tissue authority anatomy practical examinations: how does student performance on computerized evaluation compare with the traditional format? virtual reality educational tool for human anatomy youtube: an emerging tool in anatomy education covid-19 is our best chance to change universities for good. the guardian business insider. a third of the global population is on coronavirus lockdown-here's our constantly updated list of countries and restrictions anatomical knowledge gain through a clay-modeling exercise compared to live and video observations return of the cadaver: key role of anatomic dissection for plastic surgery resident training online instructional anatomy videos: student usage, self-efficacy, and performance in upper limb regional anatomy assessment sars, the first pandemic of the 21st century a novel technique for teaching the brachial plexus educating despite the covid-19 outbreak: lessons from singapore. times higher education the challenges of "continuing medical education" in a pandemic era comparison of a gross anatomy laboratory to online anatomy software for teaching anatomy galen and his patients teaching anatomy without cadavers body painting as a tool in clinical anatomy teaching do we really need cadavers anymore to learn anatomy in undergraduate medicine? cadaver dissection is obsolete in medical training! a misinterpreted notion lessons for managing high-consequence infections from first covid-19 cases in the uk is dissection the only way to learn anatomy? thoughts from students at a non-dissecting based medical school sars and its effect on medical education in hong kong consent and nothing but consent? the organ retention scandal the context of learning anatomy: does it make a difference? the integrated anatomy practical paper: a robust assessment method for anatomy education today uk universities suspend face-to-face teaching improving virtual learning interactions: reducing the transactional distance of online anatomy modules the anatomy of anatomy: a review for its modernization anatomy in a modern medical curriculum united nations educational, scientific and cultural organization anatomical instruction and training for professionalism from the 19th to the 21st centuries who. 2020a. world health organization. who director-general's remarks at the media briefing on 2019-ncov on 11 world health organization. who director-general's opening remarks at the media briefing on covid-19 -11 worldometer. 2020. worldometers.info. covid-19 coronavirus pandemic china novel coronavirus investigating and research team. 2020. a novel coronavirus from patients with pneumonia in china key: cord-011812-x8bzz83d authors: ayala, alexander; ukeje, chideraa title: there is no place like home: rethinking away rotations date: 2020-07-06 journal: acad med doi: 10.1097/acm.0000000000003564 sha: doc_id: 11812 cord_uid: x8bzz83d nan to the editor: on april 24, 2020, the visiting student learning opportunities (vlso) program extended its closure due to the covid-19 pandemic, and uncertainty continued to plague medical students who had planned on completing rotations at other institutions. for some, the potential cancelation of away rotations or "aways" was bittersweet. on one hand, the opportunity to impress future colleagues and program directors was lost. on the other hand, not being able to travel for these rotations potentially saved students thousands of dollars in double rent, application fees, and travel expenses. 1 students who are unable to rely on financial help from family members may have released the largest sigh of relief. similar to our society at large, a socioeconomic gap exists among medical students. more than 50% come from the top income quintile, a trend that has minimally changed in the past 13 years. 2 while this gap is largely hidden by the availability of loans for tuition and rent, it reemerges when extra funds are needed, including for aways. some students, specifically those who are financially stable, likely feel more comfortable spending the required amount of money to complete these rotations. in turn, these students may have greater access to competitive specialties, which traditionally require, or highly encourage, students to complete at least one away rotation. 3 since the average cost of an away rotation is $2,000, those who complete upwards of 4 could easily spend far more than they are comfortable with while auditioning at other institutions. 1 many students simply cannot afford this additional cost. scholarships do exist for disadvantaged students, but their availability varies by institution and sometimes by department within the same institution. we do not propose the elimination of all aways, as they provide the benefit of career exploration, especially for students whose home institution does not offer certain rotations. however, continuing to promote away rotations without consideration of how to financially support students will only continue to hinder financially disadvantaged students. relieving students' financial burden or limiting the number of aways students are pressured to complete are possible avenues of reform. the covid-19 pandemic has caused great tragedy and disrupted medical education. nonetheless, it has presented an opportunity to reevaluate how we approach residency applications by allowing students to focus on other academic pursuits that do not require extra monetary support. the prevalence and cost of medical student visiting rotations an updated look at the economic diversity of u.s. medical students do audition electives impact match success? key: cord-334498-w34wwfhi authors: brassett, cecilia; cosker, thomas; davies, d. ceri; dockery, peter; gillingwater, thomas h.; lee, t. clive; milz, stefan; parson, simon h.; quondamatteo, fabio; wilkinson, tracey title: covid‐19 and anatomy: stimulus and initial response date: 2020-07-06 journal: j anat doi: 10.1111/joa.13274 sha: doc_id: 334498 cord_uid: w34wwfhi the outbreak of covid‐19, resulting from widespread transmission of the sars‐cov‐2 virus, represents one of the foremost current challenges to societies across the globe, with few areas of life remaining untouched. here, we detail the immediate impact that covid‐19 has had on the teaching and practice of anatomy, providing specific examples of the varied responses from several uk, irish and german universities and medical schools. alongside significant issues for, and suspension of, body donation programmes, the widespread closure of university campuses has led to challenges in delivering anatomy education via online methods, a particular problem for a practical, experience‐based subject such as anatomy. we discuss the short‐term consequences of covid‐19 for body donation programmes and anatomical education, and highlight issues and challenges that will need to be addressed in the medium to long term in order to restore anatomy education and practice throughout the world.[image: see text] the emergence of infectious diseases with the potential to spread rapidly among the human population, such as severe acute respiratory syndrome (sars) and zika virus, presents a major threat to global public health (wolfe et al., 2007; ventura et al., 2016) . however, the recent arrival of a severe respiratory disease, first reported in wuhan city (hubei province) china in december 2019 , zhu et al., 2020 , has generated an unprecedented global response. covid-19 is caused by a novel and highly pathogenic coronavirus (sars-cov-2) that is easily and rapidly transmitted through respiratory droplets , zhu et al., 2020 . the highly contagious nature of covid-19, and the potentially life-threatening nature of symptoms for an affected individual, has led to widespread, global attempts to control person-to-person contact and disease spread within and between populations (wilder-this respect at least, they might be. the challenges that have arisen, and/or are likely to arise, over the coming months and years are outlined, and the immediate decisions taken, the rationale for them and their effects are described on an institution-by-institution basis. in the final section, possible longer-term effects and challenges are also discussed. the principal of the university announced university-wide lockdown on 13 march, to be in place by 20 march. this affected teaching sessions for c.800 undergraduate students on medical, dental, physician associate and science programmes. fortunately, the majority of practical teaching was able to be completed prior to lockdown, except for year 3 medicine students who would have returned after the easter vacation. the practical element of two, ongoing blended anatomy postgraduate courses was also suspended. at the same time, several surgical skills courses catering for c.100 delegates had to be cancelled. postgraduate (phd) student work on cadaveric material also ceased. all anatomy staff commenced working from home by 20 march, and the anatomy building closed to all but essential staff, which comprised two licensed teachers and two technicians. the extensive collection of prosected cadaveric material and freezer content was monitored weekly. as the building is shared by the university and the national health service (nhs), it was immediately opened for essential training which was coordinated by staff in the clinical skills centre. to date, no requests have been forthcoming for assistance with mortuary facilities. the anatomy team donated all basic personal protective equipment (ppe) to the nhs. currently, the university senate has agreed a fortnight's delay to the start of the first semester in september. an announcement was made by the vice-chancellor on 18 march that by the end of 20 march, the university's normal operations would cease. the teaching term for all three years of preclinical medical students had already ended on 13 march, with completion of an anatomy practical exam for over 300 second year medical students on that day. most students returned home on or soon after 13 march, although students who could not do so, including international students and those with immunosuppressed family members, were supported in university/college accommodation where some remain in residence. clinical teaching for years 4 & 5 students ceased after 13 march, and final year (year 6) clinical examinations were cancelled. several external courses in surgery and ultrasound in regional anaesthesia were also cancelled, affecting c.100 delegates. the department was not asked to provide mortuary facilities. a donation of ppe was made to the local hospital following a university-wide appeal. it was announced on 15 march by the interim principal that all faceto-face teaching would be suspended from 16 march, with an immediate move to online teaching. the university then closed down for all but essential activities on 18 march. three weeks of anatomy teaching remained for most students, who comprised c.900 undergraduate and postgraduate students in anatomical sciences, forensic anthropology, medicine, dentistry, oral health sciences, and medical and forensic art. medical students are taught for a longer period, with classes for year 3 students continuing well into may. however, there was no more dissection in that period, with only some anatomy revision sessions remaining. msc students carrying out practical projects on thiel embalmed bodies either completed data collection swiftly before university closure, or moved to literature projects instead. phd students using thiel specimens had to suspend data collection for the foreseeable future. all external activities, including surgical skills courses, research projects, service work with medical device companies, and any other events utilising thiel bodies were suspended. while most staff members commenced working from home on 19 march, technical staff continued to enter the building for essential maintenance, and both technical and administrative staff were permitted to go into work to prepare bodies for cremation. the local dundee covid-19 response teams requested provision of the following: (a) access to dissecting rooms for body storage if the local mortuaries were stretched beyond capacity; (b) assistance from technical staff to ease anticipated pressure on pathologists and technicians at the local police mortuary, where anatomy staff had already undergone induction, as well as the mortuary at the local teaching hospital; (c) loan of the anatomy van, which has the capacity to transport six bodies at a time; and (d) provision of any excess ppe as part of a dundee-wide initiative. all on-campus teaching at the university of edinburgh was formally suspended following an announcement from the principal on 13 march with a move to online teaching and examinations. the university of glasgow announced on 14 march that all face-toface teaching would be suspended from 16 march, and that arrangements had to be put in place to deliver online teaching. in advance of this, considerations had already been made regarding cancellations of continuing professional development (cpd) courses. this coincided with the announcement on 13 march that the educational activities of the royal college of physicians and surgeons of glasgow (rcpsg) would be suspended until the summer. these included courses hosted in the clinical anatomy skill centre, a joint initiative between the college of medical veterinary and life sciences of the university of glasgow and the rcpsg. only a week of teaching remained for the life sciences courses, which comprised c.380 students in year 2 and c.80 students in years 3 & 4. a longer period of teaching was outstanding for the following groups: over 600 medical students in years 1 & 2; a smaller number in the pre-medical glasgow access programme; c.160 dental students in years 1 & 2 who receive anatomy teaching in our facility; and c.120 nursing students in years 1 & 2. however, the bulk of gross anatomy teaching and dissection activities had already been completed for these students. in addition, an international undergraduate group of 12 students taking a functional anatomy course had two weeks of teaching remaining, and a postgraduate student cohort, comprising 14 master's students and 2 undertaking a postgraduate certificate that incorporates anatomy teaching, still required a substantial amount of teaching. these two groups were most affected by dissection room closure. from the week beginning 23 march, all staff commenced working from home, as the building in which the anatomy facility is housed was closed and only accessible for essential maintenance and key workers in case of emergency. imperial college london closed for face-to-face formal teaching on 20 march and all staff, apart from key workers, were instructed to work from home. this coincided with the last day of the spring term all other postgraduate anatomy courses have also been cancelled. anatomy facilities were offered to the nhs but were deemed unsuitable for its current needs. the human anatomy unit donated ppe for use by the nhs. the state of bavaria closed all state-run university buildings to students and the general public on 17 march. employees who had returned within 14 days prior to that date from a region of the world recognised as a high-risk area by the robert-koch-institute berlin were immediately home quarantined for 14 days. all employees were asked to work from home if possible. on 20 march, the bavarian government issued a general public contact restriction for all citizens, resulting in further reduction of employee personal contacts within the university sector. from 24 march, all pregnant lmu employees were sent home for the duration of the crisis. clinical departments prepared for treatment of covid-19 patients and other emergency cases. staff members on the preclinical medical faculty continued working and were tasked with preparing for a summer term of online teaching. all university employees, especially medically trained personnel, were registered for emergency service in the public healthcare sector, potentially depleting the number of anatomy staff able to continue to undertake anatomy education activities. on 12 march, the irish government announced a range of stringent measures to help combat the spread of covid-19, which included the closure of all schools and colleges in the country. consistent with public health protocols and priorities, campuses were only accessible by those doing work related to, or supporting the public health service in covid-19-specific work. only key technical staff were permitted access to maintain facilities. there were only 3 weeks left in the nui galway term, ending on 4 april in 2020. the lockdown mainly affected medical students taking gastrointestinal system and renal system modules, normally delivered as an integrated systems-based module with lectures from anatomy, physiology, biochemistry and clinical disciplines. only 60% of the gastrointestinal system practical sessions were able to be completed, and the renal system practical did not run. there were similar cancellations of practical classes for undergraduate science students taking gastrointestinal system and head and neck modules. teaching for biomedical device companies and the msc/pg dip in multidisciplinary radiology was also suspended. in accordance with the uk government directive, face-to-face teaching was suspended on 13 march, which primarily affected the university of aberdeen suspended its body donation programme from the evening of 13 march. this decision was taken in conjunction with other anatomy units in scotland and her majesty's inspector of anatomy for scotland (hmias). the rationale for this was two-fold: an inadequate knowledge of the potential risks from covid-19 donors to staff, and the increased risk from a continued need to reenter the anatomy facility on the aberdeen royal infirmary campus. planned cremations and burials for march and april were also put on hold. subsequently, the scottish government relaxed the requirement to dispose of donated cadavers within 3 years, extending this by 6 months in the first instance. the annual memorial service for families of donors, which was scheduled for late march, had already been cancelled ahead of the university-wide closure, as many attendees would be from the vulnerable population. notification of two bodies was received on 13 march, and these were accepted into the facility on 16 march. while both were elderly and stated to be covid-19 negative, they had not been tested for confirmation. the bequeathal secretary continued to accept completed bequeathal forms and deal with enquiries while working from home. import of frozen anatomical material from science care usa was suspended, with an offer to hold material for 6 months in the first instance. acceptance of body donations ceased on 18 march, but a donor who had already been processed was accepted on 19 march. teaching of first year medical students, with hands-on cadaveric dissection, had already been completed in the first two terms (from october 2019 to mid-march 2020) of the academic year. therefore, the decision was made to manage the current donors by performing retention of parts with the appropriate consent for future teaching and research, completing cremation paperwork, purchasing coffins and booking cremation slots as soon as possible. this would enable respectful disposal of donors, as well as fulfilling their wishes for the use of their bodies for teaching and research. as these donors had already been dissected by students, their remaining in the dissection room (dr) for a prolonged lockdown would constitute a health and safety hazard. in addition, existing holdings of anatomical prosections needed to be secured, as many were stored in cabinets and required weekly spraying. the decision was made to submerge these prosections in preserving fluid within sealed containers to obviate the need for staff to make regular journeys to the facility. anatomist on selected days for retention of tissue, encoffining of donors and supervising transport to the crematorium. cremations were completed by the end of april, at least six weeks earlier than in previous years. the committal service for students and staff was delivered online via our secure virtual learning environment (vle) platform. students provided a donor tribute from each table group, with two representatives giving general tributes on behalf of the whole cohort. tributes and biographical material from donors' relatives were also included. (finegan et al., 2020) . in view of the vulnerability of the attendees at our thanksgiving memorial event, due to be held in may, the decision was made to postpone this until the autumn, which would also allow students to attend. the response from both potential donors and relatives of current donors has been one of complete understanding in all respects. some phoned to ask whether a covid-19 patient would be accepted, others to enquire whether the donor programme had been suspended and whether bequeathal forms were still available, and others got in touch with regards to collection of their loved one's ashes. in order to free up the dissecting room and other secure spaces which may be required for body storage due to the current crisis, donor cremations continued. following local discussions with other scottish anatomy departments, the licensed teachers took the collective decision to suspend the anatomy@edinburgh body donation programme from the evening of the 12 march, with immediate effect. this decision was communicated to hmias and senior colleagues at the university of edinburgh, and was swiftly enacted due to the significant amount of fresh frozen material handled by the facility that does not undergo fixation. formal notices of the suspension were placed on the anatomy@edinburgh website, social media channels, and as an answerphone message on the bequest telephone line. at this stage, to ensure that all donor material coming close to the 3-year retention rule enforced by the human tissue [scotland] act 2006 would not be held beyond this period, cremation was prioritised for the longest held donors. at the same time, the decision was taken to postpone the annual anatomy memorial service, due to take place in late april, until a later date. there was sufficient embalmed donor material to meet teaching requirements for the 2020-21 academic year, although the impact on fresh frozen material, largely used for postgraduate and professional courses, remained unclear. on the evening of the 12 march, as the crisis began to escalate in the uk, the decision was taken to suspend the acceptance of donors until further notice from the following day. a notice was placed online and on the bequest coordinator's telephone answering machine as well as in an out-of-office reply to emails. the suspension was based on the body donation programme in munich was not suspended. however, sars-cov-2-infected potential donors were not accepted and incoming donors were subject to a virus test. all cadavers designated for use in anatomical teaching were fixed with a formalin and alcohol solution, and then stored for several weeks in embalming solution prior to dissection. cadavers for clinical courses in surgical procedures were usually embalmed with an alcohol glycerine solution. fresh frozen human material is currently only used for research purposes and not for teaching. the intake of donations had already been suspended in november 2019 until january 2021 due to storage capacity issues. sufficient pre-covid-19 donations exist for the next 2 years, based upon current usage. the annual memorial service planned for 26 march had already been cancelled on 6 march due to concerns over the potential risk to vulnerable elderly attendees. monitoring and spraying of prosected specimens contained in cabinets in the department were continued by key workers on the anatomy staff team. planned burial of donors was suspended until further notice. acceptance of all new donations was suspended from 13 march with the escalation of the covid-19 crisis across the uk. this decision was taken by the director of anatomy with the head of department. a number of donor families, both current and prospective, were informed prior to the suspension and all understood the reasons for it. the message on the bequeathal answerphone was changed to provide an explanation for the suspension of donations, with information on the procedure to be followed in the event of a donor's death. an emergency contingency planning meeting was held to ensure that all technical staff were conversant with the new procedures during the period of lockdown and a rota was established to ensure that current donors would receive regular and dignified care. an emergency contact system was in place at all times, ultimately leading to the director of anatomy, in order that any environmental changes in the dissection room could be managed expeditiously. it was planned to suspend acceptance of body donations due to covid-19, but the decision had already been taken two days ahead of the enforced closure, as full capacity for body storage had already been reached. the decision was taken in consultation with college management. prior to the suspension, information was provided for both current and prospective donor families, who understood the immediate reason and the covid-19 situation. a memorial service for donors is held every second year and as the next event will be held in 2021, it is unlikely to be affected. the immediate priorities were the summer spotter examinations for science and medical students, which had originally been scheduled for march and april. the medical school had clear plans at all levels at an early stage, but a centralised message from the university did not fit well with healthcare programmes, which have a different tempo and timescale to other university courses. therefore, considerable work had to be spent on designing online alternatives for the summer examinations. immediately prior to lockdown, some science course spotters had already been conducted in an electronic format in lecture theatres as usual. a final decision was made to convert all year 1-3 medical summative spotters into formative assessments. while these were open for 2 weeks to allow for circumstances such as illness, having to care for relatives and differing time zones, each student would be required to select a shorter time frame within this period to take the paper. these results were principally used to identify students who needed additional support. it was accepted that more time was needed to enable students to reach the necessary level of knowledge, and that conventional end of year examinations covering a wider range of material may still be required, with extensive validation from internal quality audit. a combination of mscaa (medicine), practique™ (physician associate) and examsoft™ (dentistry) platforms were used across the school of medicine, medical sciences and nutrition. as each platform has a different set of operating parameters, examinations had to be modified in each case to suit the platform. for example, using the mscaa platform, only a mark of 0 or 1 can be awarded, while a mark out of 2 would usually be given, with a half-mark of 1 given as an option. being a condensed 2-year course, the physician associate programme was most severely affected. the national final examination was cancelled and students offered nhs bank hours until examinations resume. cadaveric images from anatomy tv ((™) primal pictures) were used under licence. this had been put in place in advance of the lockdown. existing, self-directed, supported workbooks (findlater et al., 2012) , which support all anatomy practical classes, were converted into e-workbooks. these have cadaveric online content, form acland's video atlas of anatomy and again, anatomy-tv linked, where there would previously have been a cadaver or a labelled specimen available for examination. radiological images were relatively easy to put online, and online versions of current face-to-face teaching with expert clinical input were developed. these were augmented with linked existing bespoke 3d photogrammetry material, which can be delivered through the medical school's bespoke vle (mymbchb). online panapto™ lectures were delivered at the usual times. in addition, blackboard collaborate tm was used to host small group, interactive sessions during timetabled practical class slots, to enable face-to-face discussion of areas of difficulty and misunderstanding, and to provide elements of essential discourse. regular, formative quizzes were offered through the moodle tm platform. all content was linked from the bespoke mymbchb platform. the musculoskeletal system, which was taught after the easter vacation, was an important test bed for online anatomy teaching. in the longer term, it will be desirable for students to catch-up on the experience of handling and dissecting cadaveric material, but the timing and practicalities remain to be determined. for surgical skills courses, blended courses continued to be held online, with completion of the practical, workshop components deferred to whenever possible in the future. equality of internet access, presumed to be a key issue for students, has also been a significant issue for many staff residing in rural aberdeenshire. for the first year medical course, hands-on cadaveric dissection was completed in the first two terms of the academic year, with the third term being reserved mainly for revision sessions. these would learning produced a specific guide to moving to online teaching, as well as regular webinars for teaching staff. the move to online teaching provision was definitely a steep learning curve for everyone. concerns had also been raised regarding the difficulties some students faced in accessing online educational resources due to factors such as domestic circumstances and internet availability, as well as the possibility of compromised mental health due to prolonged isolation. all anatomy teaching moved online immediately after suspension of face-to-face teaching. while anatomy provision is heavily dissection-based, only relatively few dissections remained uncompleted, as closure came shortly before the end of the semester, when revision sessions are often scheduled. although several practical classes had to be cancelled, some were histology classes that were easily delivered via the existing 'virtual microscope' system, normally accessible to students online. a variety of communication tools were swiftly brought into play, including audio powerpoint presentations to replace lectures, and the use of vle (blackboard) to hold classes in collaborate. all modules were set up in microsoft teams, which allows live events with presentations and/or q&a, meetings for specific groups or the whole class, and group or personal 'chat' channels. students also had access to a wealth of in-house online 3d anatomy learning models and resources, created by in-house medical art and other students. the whole university worked on an alternative assessment plan at module level to cater for the fact that written and practical examinations were no longer possible; this plan was completed, approved and communicated to all students on 26 march. in anatomy, several staff took a 'conveyor belt' approach to photographing all existing prosections on the final day at work to ensure a good variety of digital images for use in online spotters, which replaced the end of year practical tests. most written examinations were conducted online, with some in modified formats. for medical students, the school of medicine made the decision to cancel all examinations and replace them with 'enhanced formative assessments' to be held in the next academic year. final year medical students were permitted to graduate early in order to join the nhs workforce. all anatomy teaching at the university of edinburgh moved online immediately following closure of the campus. the university made available significant resources and support for academic staff to obtain remote access training using both commercial and bespoke/ in-house platforms, including the secure, password-protected vle (learn), microsoft teams and kaltura capture. it was decided to take different approaches for postgraduate and undergraduate students. for the former, given the smaller cohort size, lectures were live-streamed at a mutually convenient time using skype, with screen sharing by the academic to show their powerpoint presentation, with slides made available to students in advance. this allowed for real-time discussion with students, which was felt to be beneficial. as this approach would not be feasible for the larger undergraduate cohorts, learn was used to upload pre-recorded lectures and associated materials, including access to online textbooks and videos. to ensure that a human/personal touch was retained within remote delivery, a short 'to camera' introductory video for each session was given by an academic. the transition to online teaching was aided greatly by having an established online anatomical sciences postgraduate programme (kelsey et al., 2020) , from which it was possible to repurpose materials, and to obtain mentoring and support from academics with experience of delivering online anatomy teaching. none of the online materials contains images of edinburgh donor material, but comprised only images from published resources. to date, the main issue that was identified was a low level of student engagement with online resources, with the vast majority of lectures and other material having been accessed by only c.25% of the student cohort, even after numerous email and learn reminders. as all practical spotter examinations were cancelled, anatomy questions were embedded into online mbchb and biomedical sciences papers in the form of both multiple choice and short essay questions. a decision was taken not to attempt replication of practical, laboratory-based examinations online, or online replacement for hands-on dissection classes for the msc student cohort. similarly, attempts were not made to reconfigure anatomy-based cpd or commercial courses and activities using online approaches. early, anecdotal feedback from students that were moved to an online environment, excluding those who are already enrolled on online courses, suggested that they were missing the face-to-face contact with anatomy staff and the ability to physically interact with specimens (gillingwater, 2008) in the anatomy teaching laboratory. the crisis started at a time when the bulk of cadaveric gross anatomy teaching in glasgow had already been delivered and dissection had been completed for the majority of students. had the crisis started 4 to 6 weeks earlier, disruption of cadaveric practical teaching would have been much more severe. once the crisis is over, dissection experience will be offered to the latter group. in general, practical cadaveric anatomy sessions were replaced by online resources, and students given access to lecture material, with additional explanations and links to existing material available in the public domain. for spotter examinations, images of specimens were used. where practical cadaveric anatomy sessions were replaced by online resources, students were given access to further electronic reading material and resources, with additional explanations, and links to existing educational material already available in the public domain. this was complemented by interactive live sessions delivered online, as a 'virtual' replacement for dissection classes, and also interactive discussions in a virtual learning environment platform (moodle and canvas). the bulk of preclinical anatomy teaching for the current academic year had been completed before closure of the college. therefore, the impact of the pandemic on undergraduate anatomy teaching will depend largely on when the college reopens. if 'normal service' is resumed for the beginning of the academic year 2020-2021, the effect will not be significant. while there will be problems such as the lack of newly prepared prosections and the backlog of donors to be released for funerals, as well as the many 'housekeeping' tasks that would have been carried out in anatomy facilities over the summer months, these should be manageable. however, if the college does not reopen fully, considerable reorganisation of the curriculum will be required. this will inevitably involve more online learning and less direct staff-student contact. the college invested heavily in hosting external online resources and developing in-house ones. subjects that are practical in nature are being delayed, at least in part, until the pandemic is over. this delay is likely to involve 'catch-up' sum on 30 march, the issue of a german ministry of health directive regarding medical education provided the legal framework for preparations for a virtual summer term at the medical faculty of the lmu. the measures are expected to be in place until the clause of national importance relating to the covid-19 crisis that was issued by the german parliament on 28 march is withdrawn. the summer term at the lmu started on 20 april with online teaching. the university acquired a one-year campus licence from the commercial provider zoom for all academic staff and this was used for preclinical teaching. due to reported security problems, zoom was only used when no personalised data (i.e. patient data) were transmitted during teaching. for sensitive meetings, a different system, dfnconf, was utilised. all medical students, especially those who have passed their first major examination (m1 or physikum), were asked to apply for a part-time contract of 19 hours a week at the university hospitals, which are expected to see high numbers of covid-19 patients. these students were provided with an adapted curriculum adjusted to their clinical duties. much of the preclinical anatomy teaching for the current academic year had been completed before the lockdown. however, the final academic term was significantly affected. the longer-term impact of the pandemic on anatomy teaching will depend largely on when the university reopens. there is optimism for a return to the 'new normal' by the start of the academic year in october 2020, but if the situation extends beyond this, the level of disruption will be considerable. if the university does not reopen fully by then, considerable reorganisation of the medical curriculum will be required. significant resources have already been devoted to developing online teaching, which migrated from a 'weblearn' system to a new 'canvas' application. the way in which online teaching might best be developed is currently being investigated within a broad framework offering a variety of learning resources, including instant anatomy and acland's anatomy as well as an extensive set of in-house notes, diagrams and videos to cover the curriculum. intensive pre-professional examination preparation courses held in the evening for regional surgical trainees were suspended. lectures with slides and commentaries were recorded on powerpoint show and additionally as mp4 movies, so that the laser dot was visible on mac computers. a research lecturer, an engineer, was the department 'superuser' and liaised with staff and information technology. bespoke online guides were produced for dissection and histology 'in-house'. a bespoke surface anatomy guide was made available as freeware on youtube (https://bit.ly/rcsis urfac eanatomy) to all anatomy students . staff communicated with students via email and conducted online q&a sessions. some students preferred recorded lectures, as they could pause and rewind at will. a number of students commented that they missed the personal tuition and 3d aspects of anatomy room teaching. anatomy examinations, first-sitting and supplemental, took place online in mcq format, with only pass/fail grades being awarded. examination dates were postponed and an exceptional third sitting was offered to mitigate against the disruption to students in view of home circumstances and adaptation to distance learning. given the widespread disruption to anatomy teaching and practice detailed above, the final section of this review discusses potentially important issues that are likely to need addressing as the anatomical community emerges from the covid-19 pandemic. it is perhaps fortuitous that the anatomical society has recently completed the process of design, revision and publication of core anatomy syllabi for a range of student populations (e.g. smith et al., 2016; connolly et al., 2018; finn et al., 2018; holland et al., 2019; matthan et al., 2020) . these syllabi provide a useful standardised framework for anatomy educators to design and assess the content of courses, whether delivered face-to-face or online, albeit with efforts to validate their potential application and usefulness currently ongoing . what remains unclear is the extent to which online replacement of anatomy teaching, or the uptake of blended learning models (combining on campus with online approaches), will leave unavoidable gaps in core content, knowledge and practical application. this issue will need to be investigated in significant detail over the coming months and years, together with the longer-term impact on student knowledge and professional capabilities. whilst the move to online teaching is going to affect all subjects to a greater or lesser degree, the loss of hands-on practical teaching using cadaveric material is of particular importance and relevance for the study of anatomy. early, largely anecdotal, experience suggests that the online resources and opportunities being made available at short notice are not capable of replacing the face-to-face, practical-based experience of an anatomy teaching laboratory. it will, therefore, be important to address how students that have been affected by the covid-19 pandemic can replace or substitute these activities in the future, taking into consideration their own concerns and views. such factors may also influence the decision-making process for students considering applying for enrolment on courses with an anatomical component over the coming years. as a group, the authors hold the view that hands-on examination of cadaveric specimens, and where possible dissection, remains the gold standard for anatomical education. such activities, with associated benefits concerning student engagement, cannot be replaced or substituted for by virtual/online methods alone. moreover, considerations regarding the value of working with cadaveric specimens in terms of developing necessary professionalism and manual dexterity need to be considered, when students are missing the opportunity to have a platform for developing and practising empathy, hand and teamwork skills, as well as an appropriate professional attitude. the cancellation of practical-based examinations is also a concern. when such methods of assessment have been largely removed, with no detriment to student progression in many cases, it is important to try to assess students' practical skills and knowledge using other robust methodologies. in the long term, therefore, reinstatement of practical-based anatomy examinations will be one of the most important elements of resuming 'normality' once the pandemic is over. this situation is likely to be particularly pertinent with regards to surgical trainees, since royal college membership is required for entry into specialist training, but at present, the royal colleges have suspended their mrcs part b examinations (which by definition contain a substantial anatomy component) until further notice. this raises significant concerns in terms of the progression of trainees to surgical training with inherent workforce planning implications for the delivery of front-line medical care. taken together, it is clear that the short-and medium-term consequences of covid-19 disruption for the assessment of anatomical knowledge and skills will need to be addressed by targeted, quantitative research studies over the coming months and years. given the importance of cadaveric donor material for all levels of anatomy teaching (undergraduate, postgraduate and cpd), the longterm supply of donors is of concern. there will need to be national/ international guidance on the requirements to add coronavirus to other existing risk factors (e.g. prion/bse, hiv and tb) as potential post-mortem risks for both staff and students. given that a number of institutions rely on imported anatomical material to meet demand, the development of an internationally recognised framework and reporting procedure (as well as best practice guidelines) will be important. moreover, the resumption of bequeathal programmes will be important in order to meet demand for anatomy training over the coming years. many universities and medical schools continue to receive several enquiries a day from donors and/or their relatives, despite the closure of body donation programmes. fortunately, most potential donors do appear to understand that the cessation is temporary. nevertheless, there is likely to be a decrease in donor acceptance rates due to covid-19 as a cause of death and the increased mortality during this period, which may result in a decreased death rate among the donor base in future years. therefore, when the pandemic is over, raising public awareness of the continuing need for donors for anatomical examination may be necessary. it should be noted that some elderly potential donors have been distressed at the thought they will not be able to complete their lifelong intention to donate their body for anatomical examination. anatomists, as a community, must not forget what an important decision this is, both for them and their families. several options exist to deal with the issue of donor availability. it is possible, where facilities and skills are present, to introduce or reintroduce longer-term preservation techniques, such as plastination. this serves to increase the 'shelf-life' of specimens and may also permit future sharing of resources between anatomy facilities, as long as traceability and secure transport facilities can be ensured. such activities may be supplemented and supported by the use of emerging 3d printing technologies to generate anatomical 'specimens' from tomographic radiological data from donor or patient material. it may also be necessary to prioritise embalming of donors that would previously have been used for fresh frozen work. although the response of the sars-cov-2 virus to embalming and fixation is currently unknown, the wide range of embalming techniques available offers a good chance of finding at least one that can render covid-19 donor material safe for anatomical examination. of course, the process of embalming will still remain a highrisk activity for anatomy staff. information and guidance papers are being published rapidly as the international community gains more experience and knowledge of the virus (e.g. finegan et al., 2020; kampf et al., 2020; royal college of pathologists, 2020) , and the anatomy community will also be informed by these. regardless, covid-19 testing facilities may be required for anatomy mortuaries, as well as the provision of full ppe for all staff undertaking embalming activities. the united nations has described covid-19 as the most significant event since the second world war. things will never be the same again. however, it affords both challenges and opportunities. one opportunity moving forward is for the anatomical community to cooperate more effectively and share resources (both physical and intellectual) more widely. cooperation will be required to define best practice guidelines for embalming to deal with this new infectious agent. we also need to develop a protocol for dealing with future pandemics that will enable us to respond faster and better than at present. the current situation similarly presents an opportunity to test rigorously the strengths and weaknesses of online anatomical teaching in practice. anatomy has been at the heart of medical and scientific teaching and research for several centuries. there is no reason for this not to continue into the future. the strength and willingness of anatomists (including support staff as well as academics) to rise to the challenges that have presented has been a source of great pride within and outside the anatomical community. this is something that we all need to celebrate and recognise. the anatomical society's core anatomy syllabus for undergraduate nursing development of a supported self-directed learning approach for anatomy education icrc advisory group on the management of covid-19 related fatalities. international committee of the red cross (icrc): general guidance for the management of the dead related to covid-19 the anatomical society core anatomy syllabus for pharmacists: outcomes to create a foundation for practice the importance of exposure to human material in anatomical education: a philosophical perspective the anatomical society core embryology syllabus for undergraduate medicine 2020) persistence of coronaviruses on inanimate surfaces and their inactivation with biocidal agents anatomical sciences at the university of edinburgh: initial experiences of teaching anatomy online the anatomical society's core anatomy syllabus for dental undergraduates anatomy from the outside. in: a new on-line surface anatomy guide briefing on covid-19: autopsy practice relating to possible cases of covid-19 (2019-ncov online surface anatomy resource the anatomical society core regional anatomy syllabus for undergraduate medicine the initial impact of the anatomical society gross anatomy core syllabus for medicine in the united kingdom: student and teacher perspectives zika virus in brazil and macular atrophy in a child with microcephaly can we contain the covid-19 outbreak with the same measures as for sars? origins of major human infectious diseases a new coronavirus associated with human respiratory disease in china a novel coronavirus from patients with pneumonia in china how to cite this article covid-19 and anatomy: stimulus and initial response key: cord-339440-qu913a8q authors: fonseca, david; garcía-peñalvo, francisco josé; camba, jorge d. title: new methods and technologies for enhancing usability and accessibility of educational data date: 2020-10-26 journal: univers access inf soc doi: 10.1007/s10209-020-00765-0 sha: doc_id: 339440 cord_uid: qu913a8q nan recent advances in information and communication technologies (icts) have fostered the development of new methods and tools for exploring the increasingly large amounts of data that come from pedagogical domains [1] [2] [3] [4] [5] . these data have the potential to transform education into a personalized experience [6, 7] that meets the needs of each individual student [8] . educational data research is becoming highly relevant in massive online courses [9] , especially moocs (massive open online courses) [10] [11] [12] [13] and spocs (small private online courses) [14] [15] [16] . educational data are also the basis for learning analytics [17] [18] [19] , with an increasing focus on the way educational data are presented [20] [21] [22] , how users interact with the data [23] [24] [25] [26] , and data privacy and security [27] [28] [29] [30] . there are many types of data that can support student's learning [31] , but the type and nature of the data, how they can be accessed, and who can access them, vary significantly. whether educational data are collected from collaborative learning environments [32] [33] [34] , course management systems [35, 36] , gamified training applications [37, 38] , or administrative systems from schools and universities [39] [40] [41] , valuable properties, patterns, and insights often emerge. when combined with other factors such as timing and context, these factors play an important role in understanding how students learn [42] , the settings in which they learn [43] , and the effectiveness of the educational approaches [44] . extracting information from data to ultimately turn it into knowledge [45, 46] can contribute to draw a more comprehensive picture of student's learning, which can empower students, parents, and educators as well as education stakeholders and policymakers [47] . educational data usability and accessibility is even more relevant in the context of the global pandemic due to the sars-cov-2 virus, which causes covid-19 disease. this situation is having an unprecedented impact on education. according to unesco [48] , in the first months of 2020, the pandemic has affected 91.3% of the total number of students enrolled worldwide: over 1.5 billion people have been unable to receive face-to-face instruction because of the closure of schools and universities [49] . the consequences are more severe in emerging countries [50, 51] and to families affected by poverty and risk exclusion [52] , presenting digital inequalities [53] , and causing exclusion and inequality situations in vulnerable groups, ethnic minorities, and people with disabilities [54] . significant challenges have been reported in the online transformation of educational activities [55, 56] , particularly assessment processes [57] . consequently, it is vital to improve access to educational technologies and reduce gaps in use and literacy [58] . a multi-disciplinary approach is required to deploy technological ecosystems [59, 60] that favor blended or online training, teacher and student training for the efficient use of educational technologies [61] , and policies for both government and academic leaders to define strategies and manage uncertain scenarios [62] . in the context of educational data access, it is critical to ensure transparency, ethics, and individuals' rights. this uais special issue builds on the work started in a number of previous special issues [63] [64] [65] [66] and two international events: • the invited session entitled "emerging interactive systems for education", in the thematic area "learning and this special issue focuses on how to improve universal access to educational data, with emphasis on (a) new technologies and associated data in educational contexts: artificial intelligence systems [70] , robotics [71] [72] [73] , augmented [74] [75] [76] and virtual reality (vr) [77] [78] [79] [80] [81] , and educational data integration and management [82] ; (b) the role of data in the digital transformation and future of higher education: personal learning environments (ple) [83, 84] , mobile ple [85, 86] , stealth assessment [87] , technology-supported collaboration and teamwork in educational environments [88] , and student's engagement and interactions [89, 90] ; (c) user and case studies on icts in education [91, 92] ; (d) educational data in serious games and gamification: gamification design [93] [94] [95] [96] , serious game mechanics for education [97, 98] , ubiquitous/pervasive gaming [99] , and game-based learning and teaching programming [100, 101] ; and (e) educational data visualization and data mining [102] : learning analytics [103] , knowledge discovery [104] , user experience [105, 106] , social impact [107] , good practices [108] , and accessibility [109, 110] . the special issue comprises the following 11 accepted papers. collaborative learning systems are a niche for analyzing educational data. for example, virtual reality and 3d modeling applications can leverage the integration of collaborative approaches in medicine [111] , architecture [112] , or urbanism [113] . huang et al. developed a study devoted to construct a 3d modeling practice field based on virtual reality technology, in which students can learn 3d modeling through a new vr design collaboration framework and complete design goals. the proposed design collaboration model includes the concept of a learning community. the results of this study indicate that the system usability of the vr modeling practice field is superior to that of the traditional modeling learning field and learners are more creative and motivated. the authors emphasize that through the new design collaboration model, students can effectively learn 3d modeling in vr. conde et al. explore the assessment of instant messaging tools for the acquisition of teamwork competence throughout a case study about the use of the instant messaging app whatsapp. from the results, the authors conclude that students prefer instant messaging tools in teamwork activities over other interaction tools such as forums; and that the use of those tools has a positive impact on students' grades. in an effort to demonstrate the potential of virtual worlds in education [114] , especially in distance education [115] , krassmann et al. introduce a framework to prepare the implementation of virtual worlds. their approach emphasizes requirements that distance education students need to meet in order to have a successful learning experience. the authors present an exploratory study and propose eight guidelines to harness the potential of the technology of virtual worlds for distance education. pervasive games [116] enhance the gaming experience and level of engagement by including real world aspects into the game space. arango-lópez et al. propose geopgd, a methodology that integrates the design of geolocated narrative as the core of the game experience. this methodology guides designers and developers through the different stages of building a pervasive game by providing tools for defining the narrative components, places, and interactions between the user and the game. gallego-durán et al. tackle the challenges of learning programming as a universal ability [117, 118] . the authors propose a radically different perspective to this issue, teaching students with a bottom-up approach, starting from machine code and assembly programming. their results suggest that such a small intervention could have a limited positive influence on the students' programming skills. pazmiño et al. did a systematic literature review [119] to answer the question: what is the baseline of scientific documents on learning analytics in ecuador? the selected documents were analyzed using statistical implicative analysis after removing duplicates and applying inclusion, exclusion, and quality criteria. the outcome of this research has allowed building up a baseline of scientific knowledge about learning analytics in ecuador. user experience analysis in the educational realm is directly linked to the levels of user acceptance and satisfaction [120] of the new wave of educational technological ecosystems and the personalization of learning [121] . barneche-naya and hernández-ibañez describe the results of a case study intended to compare three different user movement paradigms (metaphoric, symbolic and natural) designed to control the visit to virtual environments for a nui-based museum installation. the study evaluates the performance of each movement scheme with respect to the navigation of the environment, the degree of intuitiveness perceived by the users, and the overall user experience. the results show that the natural movement scheme stands out as the most adequate for the contemplation of the virtual environment and the most balanced at a general level for the three variables considered. the symbolic scheme proved to be the most efficient. the natural movement and symbolic schemes appear to be the most appropriate to navigate digital environments such as museum installations. in another paper related to user experience, zardari et al. introduce an e-learning portal for higher education that was assessed from a user experience standpoint using an eye-tracking system. the results emphasize students' satisfaction with the learning portal. finally, toborda et al. analyze metrics to measure effectiveness and engagement levels in pervasive gaming experiences. regarding the analytics of accessibility, martins et al. present a study that assesses accessibility in mobile applications, which may be applicable to education and tourism [122] . fourteen mobile applications were analyzed using a manual and automatic methodology through an evaluation model based on quantitative and qualitative requirements, as well as the use of features such as voiceover and talkback. the results show a high number of errors in most quantitative requirements as well as non-compliance with most qualitative requirements. also, in the context of accessibility, romero yesa et al. share a good practice in designing accessible educational resources [123] . the authors developed a new virtual teaching unit for supporting classroom teaching based on usability and accessibility criteria. the goal is to help increase teaching quality by improving syllabus design. a large-scale dataset for educational question generation time-dependent performance prediction system for early insight in learning trends exploration of youth's digital competencies: a dataset in the educational context of vietnam presenting the regression tree method and its application in a large-scale educational dataset dashboard for large educational dataset understanding the implementation of personalized learning: a research synthesis appraising research on personalized learning: definitions, theoretical alignment, advancements, and future directions towards elearning 2 0 university la sociedad del conocimiento y sus implicaciones en la formación universitaria docente an adaptive hybrid mooc model: disrupting the mooc concept in higher education participantes heterogéneos en moocs y sus necesidades de aprendizaje adaptativo innovation in the instructional design of open mass courses (moocs) to develop entrepreneurship competencies in energy sustainability massive open online courses in the initial training of social science teachers: experiences, methodological conceptions, and technological use for sustainable development a recommender system for videos suggestion in a spoc: a proposed personalized learning method the construction of teaching interaction platform and teaching practice based on spoc mode exploration and practice of spoc mixed teaching mode in data structure course learning analytics as a breakthrough in educational improvement sein-echaluce ml can we apply learning analytics tools in challenge based learning contexts? learning analytics the emergence of a discipline learning analytics dashboard applications representing data visualization goals and tasks through meta-modeling to tailor information dashboards connecting domain-specific features to source code: towards the automatization of dashboard generation tap into visual analysis of customization of grouping of activities in elearning mejora de los procesos de evaluación mediante analítica visual del aprendizaje visual learning analytics of educational data: a systematic literature review and research agenda multimodal data to design visual learning analytics for understanding regulation of learning protected users: a moodle plugin to improve confidentiality and privacy support through user aliases privacidad, seguridad y legalidad en soluciones educativas basadas en blockchain: una revisión sistemática de la literatura exploring the relationship of ethics and privacy in learning analytics and design: implications for the field of educational technology whose data? which rights? whose power? a policy discourse analysis of student privacy policy documents educational data mining: a review of the state of the art beyond grades: improving college students' social-cognitive outcomes in stem through a collaborative learning environment learning analytics in collaborative learning supported by slack: from the perspective of engagement eclectic as a learning ecosystem for higher education disruption the current state of analytics: implications for learning management system (lms) use in writing pedagogy valoración y evaluación de los aprendizajes basados en juegos (gbl) en contextos e-learning engagement in the course of programming in higher education through the use of gamification educational data mining and learning analytics: an updated survey management process of big data in high education as sociotechnical system automatic tutoring system to support cross-disciplinary training in big data how students learn: history, mathematics, and science in the classroom improving the information society skills: is knowledge accessible for all? smart teachers, successful students? a systematic review of the literature on teachers' cognitive abilities and teacher effectiveness knowledge spirals in higher education teaching innovation epistemological and ontological spirals: from individual experience in educational innovation to the organisational knowledge in the university sector systematic review of how engineering schools around the world are deploying the 2030 agenda unesco covid-19 impact on education school closure and management practices during coronavirus outbreaks including covid-19: a rapid systematic review estudio exploratorio en iberoamérica sobre procesos de enseñanza-aprendizaje y propuesta de evaluación en tiempos de pandemia impact of the pandemic on higher education in emerging countries: emerging opportunities. ssrn, challenges and research agenda covid-19, school closures, and child poverty: a social crisis in the making covid-19 and digital inequalities: reciprocal impacts and mitigation strategies. comput simulating the potential impacts of covid-19 school closures on schooling and learning outcomes: a set of global estimates the difference between emergency remote teaching and online learning education and the covid-19 pandemic la evaluación online en la educación superior en tiempos de la covid-19 la covid-19: ¿enzima de la transformación digital de la docencia o reflejo de una crisis metodológica y competencial en la educación superior? a metamodel proposal for developing learning ecosystems validation of the learning ecosystem metamodel using transformation rules digital competence of early childhood education teachers: attitude, knowledge and use of ict modelo de referencia para la enseñanza no presencial en universidades presenciales user experience and access using augmented and multimedia technologies: special issue of uxelate (2012) workshop and hci international conference (2013) special sessions information society skills: is knowledge accessible for all? part i information society skills: is knowledge accessible for all? part ii interactive and collaborative technological ecosystems for improving academic motivation and engagement learning and collaboration technologies. designing learning experiences. 6th international conference, lct 2019, held as part of the 21st hci international conference, hcii 2019 learning and collaboration technologies. designing learning experiences. 6th international conference, lct 2019, held as part of the 21st hci international conference, hcii 2019 teem'19 proceedings of the seventh international conference on technological ecosystems for enhancing multiculturality visualizing artificial intelligence used in education over two decades garcía-peñalvo fj robosteam project systematic mapping: challenge based learning and robotics robotics from stem areas in primary school: a systematic review social steam learning at an early age with robotic platforms: a case study in four schools in spain augmented reality and pedestrian navigation through its implementation in m-learning and e-learning: evaluation of an educational program in chile from reality to augmented reality: rapid strategies for developing marker-based ar content using image capturing and authoring tools relationship between student profile, tool use, participation, and academic performance with the use of augmented reality technology for visualized architecture models nextmed: automatic imaging segmentation, 3d reconstruction, and 3d model visualization platform using augmented and virtual reality virtual reality simulation-based learning technology in education. 4th international conference, lct 2017. held as part of hci international methodologies of learning served by virtual reality: a case study in urban interventions virtual interactive innovations applied for digital urban transformations. mixed approach future gener improvement of an online education model with the integration of machine learning and data analysis in an modeling the personal adult learner: the concept of ple re-interpreted personal learning environments and online classrooms: an experience with university students entornos personales de aprendizaje móvil: una revisión sistemática de la literatura use of mobile technologies in personal learning environments of intercultural contexts: individual and group tasks deepstealth: game-based learning stealth assessment with deep neural networks teamwork assessment in the educational web of data: a learning analytics approach towards iso 10018. telemat mapping research in student engagement and educational technology in higher education: a systematic evidence map facilitating student engagement through the flipped learning approach in k-12: a systematic review technological ecosystems in citizen science: a framework to involve children and young people web-based system for adaptable rubrics: case study on cad assessment review of gamification design frameworks opening the black box of gameful experience: implications for gamification process design a immersive visualization technologies to facilitate multidisciplinary design education methodology i'm in applied to workshop: successful educational practice for consultants in user experience with gamification fields serious gaming for climate adaptation assessing the potential and challenges of a digital serious game for urban climate adaptation designing productively negative experiences with serious game mechanics: qualitative analysis of game-play and game design in a randomized trial exploring features of the pervasive game pokémon go that enable behavior change: qualitative study state of the art in the teaching of computational thinking and programming in childhood education influence of problem-based learning games on effective computer programming learning in higher education urban data and urban design: a data mining approach to architecture education implementing learning analytics for learning impact: taking tools to task a knowledge discovery education framework targeting the effective budget use and opinion explorations in designing specific high cost product analysis of user satisfaction with online education platforms in china during the covid-19 pandemic analyzing the usability of the wyred platform with undergraduate students to improve its features peer social acceptance and academic achievement: a meta-analytic study semantic spiral timelines used as support for e-learning analysis of accessibility in computing education research a social-mediabased approach to assessing the effectiveness of equitable housing policy in mitigating education accessibility induced social inequalities in shanghai. china virtual reality educational tool for human anatomy motivation and academic improvement using augmented reality for 3d architectural visualization evaluation of an interactive educational system in urban knowledge acquisition and representation based on students' profiles retrieving objective indicators from student logs in virtual worlds educación virtual para todos: una revisión sistemática pervasive games: theory and design exploring the computational thinking effects in pre-university education computational thinking unplugged guidelines for performing systematic literature reviews in software engineering. version 2.3 school of arkaevision vr game: user experience research between real and virtual paestum measuring user experience on personalized online training system to support online learning towards a social and context-aware mobile recommendation system for tourism bridging the accessibility gap in open educational resources the guest editors would like to thank the universal access in the information society journal editors-on-chief, dr. constantine stephanidis and dr. margherita antona, for their confidence in our responsibility to lead this special issue. we want to express our gratitude to all the researchers that have made this special issue a reality. this work was partially funded by the spanish government ministry of economy and competitiveness throughout the defines project (ref. tin2016-80172-r). key: cord-026500-6k3b55lt authors: ma, xiang; fan, xuhui; li, wei; li, jiangtao; li, qiong title: the exploration and research of blended teaching mode based on “internet+” big data cloud platform date: 2020-06-10 journal: innovative mobile and internet services in ubiquitous computing doi: 10.1007/978-3-030-50399-4_32 sha: doc_id: 26500 cord_uid: 6k3b55lt in the light of problems in online teaching of new coronavirus (2019-ncov) epidemic period in advanced mathematics curriculum, this paper analyzes the teaching effect of short-term one-way online education mode in the teaching practice of concrete courses by using the real-time data of “rain classroom” teaching management platform. above that this paper explore and practice the hybrid teaching mode based on “internet+” big data cloud platform, aiming at promoting the “internet+education” information technology and curriculum teaching depth integration, improving the teaching effect, and promoting education and teaching reform. with the development of educational technology, the updating of educational mode and the change of talent demand, the current teaching mode of college [1, 2] mathematics has been challenged severely. in particular, with the new coronavirus (2019-ncov) outbreak in large areas of the world, forcing offline education to temporarily halt. in the "epidemic" impact, the current teaching situation will push back the teaching philosophy must be further updated. at the same time, the teaching situation must require the integration of specific curriculum teaching and online teaching, drive the coordinated development of educational resources in the whole society, promote the reform of information teaching to be concrete, deep and universal, and then promote a new round of teaching mode [3] [4] [5] reform to the deep water area. at this time, a large number of scholars, educators are conducting in-depth thinking and exploration. this paper is aimed at the exploration and practice of online teaching of advanced mathematics, and analyzes the new problems in the process of implementing "rain classroom" platform in curriculum teaching. a hybrid teaching model based on "internet+" big data is put forward in order to improve the teaching quality of new-type compound talents. since the 1970s, china has established radio and television universities in various parts of the country [6] . the use of satellite technology for distance learning to meet the needs of social workers to learn, to obtain diploma. with the development of science and technology, distance education is challenged severely. in 2012, the national open university was officially established, and the original radio and television university has been suspended or transformed into a local open university. as a result, an open university system with chinese characteristics has been initially formed. based on this, 2012 is also known as china's "first year of mocs". with the further development of science and technology, there are many forms of online courses, such as micro-course [7] , mooc [8, 9] and spoc [10] . in order to better regulate the ecology of online education, in april 2018, the ministry of education issued the "action plan for education informatization 2.0", which promotes the rapid development of online education based on the "internet+education" platform. however, the exploration of online teaching mode for mathematics curriculum has not obtained the universal law, and its teaching effect has not been verified by the teaching practice on a large scale. china first began the "suspension of classes non-stop learning" teaching model, with a new coronavirus outbreak in the world by 2020. as a result, online education has been carried out in an all-round way in china, which provides the possibility for online teaching practice verification. this paper based on the teaching management platform of "rain classroom", the course of higher mathematics uses the data obtained by online teaching, and utilizes the method of comparative analysis to get the following results, as shown in fig. 1 . through the data analysis [11] , we can conclude that the students' satisfaction with the online learning platform is high, which fully shows that the online teaching platform has played a supporting role in the teaching of higher mathematics courses [2] . at the same time, the analysis results also reflect some problems in online teaching. "live video" is the main form of online teaching used by teachers, however, students rarely use watching "mooc" for self-study, which shows that personalized learning experience is the focus of students. the online interactive seminar did not achieve the expected results, which shows that it is very important to answer questions and provide tutoring to ensure the quality of teaching. teachers provide materials students selfstudy popularization rate is low, indicating that the preparation stage of online teaching still needs to be improved. the assignment of homework is more frequent, which shows that online teaching does not really reduce the teaching burden of teachers and students. the development of online education is based on the theory of constructivism. it is a teaching organization activity in which teachers rely on the network platform to complete the teaching task and achieve the teaching goal according to the teaching plan. this model can make up for the shortcomings of traditional teaching, and to a large extent can satisfy the public education, especially the vocational education. especially in the "2019-ncov epidemic" period, online education shows its strong advantages. in addition, during the special period of 2019-ncov epidemic, online education shows strong advantages. at the same time, online education also has many disadvantages. the advantages and disadvantages of online teaching mode are analyzed and compared as shown in table 1 . advanced mathematics belongs to the basic course of culture in colleges and universities. the course has a high degree of abstraction, rigorous logic and wide application. for course contents involving calculation, proof and logical reasoning, students prefer the classroom teaching mode of gradual deduction and calculus display by the blackboard. but for some courses that are more closely related to life, students prefer teachers to use experimental demonstration or dynamic display with the help of network resources. thus, the two different teaching modes cannot fully adapt to different teaching contents. online teaching can help students to expand their knowledge with the help of online high-quality resources, let the students acquire learning knowledge anytime, anywhere, conveniently and quickly, and students adjust their learning progress in real time according to their learning situation. traditional classroom teaching is helpful for students to acquire learning knowledge systematically, comprehensively and efficiently in a short time, and it can cultivate students' systematic, logical and creative thinking mode, at the same time, cultivate students' emotional values and mold complete independent personality. the teaching effect is poor because there is no blackboard effective educational models for special times (e.g., global outbreak of 2019-ncov in 2020) the teaching contents are fragmentary and lack of systematization, which is cannot cultivate students' systematic thinking, logical thinking and associative creative thinking therefore, if we can integrate the advantages of online teaching and offline teaching organically, and explore a hybrid teaching mode in line with the characteristics of higher mathematics curriculum, and then apply it to teaching practice to improve the teaching effect, it is beneficial for the realization of teaching objectives and the improvement of teaching quality. based on the teaching practice and research of "advanced mathematics", this paper proposes a blended teaching mode based on "internet+" big data cloud platform for classroom teaching and online teaching [12] (as shown in fig. 2 ). as shown in fig. 3 , in the process of teaching, there are three stages: pre-class, inclass and after-class. before class, teachers send class announcements, push preview courseware through the online platform, and improve pre-class preparation work. in this way, students can be familiar with the key and difficult points in this course in advance, and can find materials before class, preview in advance, and change passive acceptance learning into active exploration learning, which is beneficial for students with different levels of cultural foundation to prepare for different situations before class, so as to enhance students' learning enthusiasm and improve the teaching quality and effectiveness of the course in the teaching process. in the class, it mainly uses the form of classroom intensive lecture or classroom discussion to teach (the teaching content of special period or special chapter can also use the online teaching mode), so as to realize the new teaching mode with students as the main body and teachers as the leading role, so as to promote students to become the protagonists of learning and teachers to become the guiders of students' growth. finally, in view of the deficiencies in the learning process of students, through the after-school q & a and after-school test and comment link to check the deficiencies and make up the omissions, improve the teaching process. according to the different actual situation, the blended teaching mode has many different methods of organization and implementation. advanced mathematics course is organized on the platform of "rain classroom", in this paper, the researchers deeply think and analysis, and put forward the exploration and research of the mixed teaching mode from the following dimensions. in the course, based on the classroom teaching, the rain classroom screen recording mode is adopted to integrate the online and offline teaching process. first of all, when confronted with a difficult problem, the students expressed their confusion by sending shrapnel. according to the cue of bullet screen combined with the students' expressions, the teacher adjusted the teaching status in time to answer the students' doubts. secondly, with the help of network platform, it is convenient to record the data of students' learning in class. finally, the recording software can record the whole teaching process, which is convenient for students to review after class and listen to the course content repeatedly. through the combination of online and offline models, teachers can count the subjects with high error rate and elaborate in the classroom. we explain these topics precisely in class. for a small number of topics with low error rate, teachers use the way of recording and explaining videos to push to everyone online, students with a weak foundation can check the explanation of the corresponding topic according to their actual situation. this paper selects "spatial curve and its equation" as the teaching content, combines with the analysis of learning situation, adopts the mixed teaching mode to design the teaching content, and carries out the teaching evaluation based on the background data of "rain class", in order to test the teaching effect of the above mixed teaching mode. "spatial curves and its equations" are the important geometric basis for supporting the calculus module of multivariate functions. it is located in the sixth section of the eighth chapter of seventh of the advanced mathematics curriculum edition, tongji university (volume two). before that, students have been familiar with plane curves and have learned special curves -straight lines and their equations, which paves the way for this lesson. civil engineering students selected as teaching objects in this class. the students are in good learning condition, active in thinking and strong in practical operation, but their basic knowledge is weak, and lack the ability of independent analysis and independent inquiry learning. before learning, students have known about space straight line and plane curve, but they don't know about space curve. in view of these characteristics of students, using the form of online exercise test, teachers can detect the student's spatial analytic geometry of the basic knowledge grasp. according to the talent training plan and the implementation plan of advanced mathematics course, combined with students' cognitive psychological characteristics, the teaching objectives of this course are determined as follows ( table 2) : see table 3 . table 2 . three-dimensional learning goals the general equation of space curve and parameter equation should be mastered by students they should be mastered by the students with knowledge of the projection of intersecting lines of a space surface on a coordinate plane capability objectives students were able to establish the relationship between spatial curves and equations, in their minds, geometry is presented. their spatial imagination is enhanced with the help of the combination of numbers and shapes, students have improved their ability to use advanced mathematics theory, knowledge and skills to solve practical problems emotional goals the interest of the students is stimulated by the study of mathematic. students' desire to explore independently is satisfied. they are eager to study the quantitative relationship in the natural environment they were able to appreciate the connection between mathematics and reality. they experienced the process of mathematical knowledge being discovered and created dialectical materialism is accepted by the students and patriotic sentiment is aroused according to the results of the exercises in the preview courseware, we can evaluate [13] the students' preview effect. teaching module implementation steps implementation process in the classroom 1. according to the preparation before class, teachers comment on the effect of preview 2. the teacher explained the concept of general equations of spatial curves parametric equation of space curve in the class 3. collaborative group discussions: ( express what kind of curve? (2) find the projection of the space curve c on the xoy coordinate plane 7. the teacher summarized the projection methods and explained their application in engineering drawing 8. for some engineering objects, the students explored their projections on different coordinates 9. according to the teaching contents of this class, teachers and students summed up the methods to deal with practical problems after-school evaluation release the homework after class through the teaching platform of "rain classroom" through the "rain classroom" teaching platform to promote the development of resources: such as the application of "mu he fang gai" (for example cctv standard, mechanical engineering components) according to the correct rate of the time-limited questions sent in class, we can evaluate the students' mastery of knowledge. according to the time-limited examination papers sent online after class, we can test the students' learning effect. based on the teaching practice of higher mathematics, relying on the "internet+" big data cloud platform and using the "rain classroom" information technology, this paper innovates the teaching mode, to promote students to become the initiator of learning, the organizer of seminars and even the innovator of knowledge application. this is bound to break the original knowledge system and framework structure, so as to promote the education reform to a deeper level so that the teaching quality, the teaching effect as well as the docking social demand will make the breakthrough progress. predictive big data analytics and cyber physical systems for tes systems a transactional model of college teaching research on project-based teaching model based on flipped classroom the flipped classroom inverting the classroom: a gateway to creating an inclusive learning environment research on mixed teaching of ideological and political course in open university exploration and research on teaching mode of computer micro course knowledge, attitudes, and practices of plagiarism as reported by participants completing the authoraid mooc on research writing an overview of massive open online course platforms: personalization and semantic web technologies and standards study on the feasibility of spoc mixed teaching model in higher mathematics classroom of military academy investigation report on online learning of college students during epidemic research on personalized teaching mode of "internet+" integration of online and offline research on teaching mode of microbial pharmaceutical technology key: cord-011859-6ksss988 authors: harendza, sigrid title: improvisation – a new strategy in medical education? date: 2020-06-15 journal: gms j med educ doi: 10.3205/zma001337 sha: doc_id: 11859 cord_uid: 6ksss988 nan suddenly everything is different. a virus keeps the world in suspense and medical education as well. yet the older teachers among us know quite well what viral diseases are and how they can change undergraduate and postgraduate medical training. during my own undergraduate training the hiv epidemic began, which even after well over 30 years still requires further learning [1] , i.e. learning about contents that could not be taught at all the during the time of my studies. the situation is similar with the viral disease hepatitis c. while it was still called hepatitis non-a non-b during my studies [2] , it has since been given its current name and everyone, including the teachers, had to learn to understand over the next decades that no vaccination was made possible, but that in the meantime drugs had been developed which can eliminate the hepatitis c virus [3] . such influences of pathogens on the content of medical studies could be relatively easily dealt with by changing the learning objectives adapted to medical progress. in case of structural changes, which may become necessary in teaching during epidemics, adaptation processes are not quite so easily implemented. during the ehec epidemic in 2011, our nephrological-internal ward and several other wards of the university medical center hamburg-eppendorf accommodated almost exclusively patients suffering from ehec-induced haemolytic uremic syndrome (hus) [4] . this is a rare disease of which students only had to have heard of at that time according to the hamburg catalogue of learning objectives [5] . within two weeks it was almost impossible for final-year students in internal medicine to see patients with other diseases. hence, we, as teachers, hat to improvise teaching -in addition to caring for the patients -and maybe the final-year students at that time learned a little less about different diseases but a lot instead about medical behaviour in unknown clinical situations. in the national competence based catalogue of learning objectives (nklm) of 2015, hus is still listed as a rare disease under point 21.1.58 [http://www.nklm.de, accessed: 04.05.2020], but the labelling with competence level a requires in any case more extensive knowledge than just knowing the name of the disease. so now, in 2020, it is again a pathogen that influences medical education. but this time the change affects all teachers and all students and almost all teaching and examination structures in medical education and in all other healthcare professions studies as well as in all other courses of study in general -and this worldwide [university world news: https://www.universityworldnews.com/post.php?story=20200324065639773, accessed: 02.05.2020]. for the study of medicine, dentistry, and veterinary medicine, but also for the study of other healthcare professions, we are currently confronted with a rather small-scale structuring of teaching instruction, which prescribes content and form of teaching at most universities right down to the individual lesson. however, in the current situation, which will probably keep us busy for the next few months or even years, the ability to improvise and flexibility are required while keeping the main educational goal in mind. for postgraduate medical education, van loon and scheele recently demanded to renounce from detailed regulations and to move towards enabling teachers to engage in curricular innovation that is "only" oriented towards the educational goal [6]. confidence in the creative design of the curriculum by teachers and their empowerment for free design opportunities [6] should also enable universities to act quickly in times of a pandemic-related lockdown. this kind of action, i.e. the development of one's own strategies within the frame of one's own current possibilities paying attention to the global goal, but without prescription of all individual steps in detail, is called mission tactics or command and control with mission in the military. this leadership tactic has proven to be particularly effective during confusing situations to achieve a global goal [7] . the acquisition of improvisation skills is, for example, explicitly required for students of teacher training and is already being practiced in class in some cases [8] . for medical educators and medical students, such techniques of improvisational theatre seem to be useful as wellboth for medical activities and for teaching medical students or for designing lessons, respectively [9] . medical work is unpredictable by its very nature. medical students must learn to deal adequately with uncertainty inherent in medical problems. this is already being implemented didactically in problem-based learning and leads to a better handling of uncertainty in everyday medical practice [10] . furthermore, there are frameworks that use techniques of improvisational theatre to enable medical students to learn how to deal with unknown medical situations [11] . these techniques of improvisation could also be appropriate to enable teachers to teach adequately in uncertain times [12] . they appear to be particularly useful for learning communication skills and professional behaviour [13] . but other teaching techniques also help to improvise appropriate medical lessons in times of a lockdown, especially e-learning, of course [14] , because it is particularly easy to keep one's distance. this issue also contains some interesting approaches in scientific work and projects which encourage creative thinking for medical teaching and testing in the current situation, although at the time of their creation there was no mention of sars-cov-2 at all. rauch et al. report on the development of an instructional video for dental students to examine patients with suspected craniomandibular dysfunction [15] . perhaps a way can be found to allow dental students to practice clinical examination techniques with people in their own homes during a lockdown, guided by such videos. the project could perhaps also be adapted to a 4-step video format based on the so-called peyton method, as is already used at the ludwigs-maximilians-university (lmu) in munich [https://www.med.moodle. elearning.lmu.de/mod/book/view.php?id=58629& chapterid=1638, accessed: 04.05.2020]. möltner et al. were able to show that the assessments of student raters in a formative osce in general medicine correlated highly with the assessments of medical experts [16] . this scientific finding may also lead to the development of further training and assessment options for medical students as peers for practical and communicative skills. findyartini et al. were able to show in their study that the motivation profile of medical students is associated with the empathy they express [17] . thus, this project also offers interesting starting points for teaching and learning empathy. zimmermann and kadmon used standardized examinees who had received training for different levels of proficiency in osce stations that were filmed and can be used both for quality assurance of osce stations and for rater training [18] . this concept can probably be easily and contactlessly used for rater training at other universities. these examples show the essential contribution that projects in medical education research make to enabling teachers to draw on evidence even in times when they have to improvise. so let's stay tuned to improvised teaching -scientifically sound and creative. 4 [7] . der erwerb von kenntnissen im improvisieren wird beispielsweise für lehramtsstudierende explizit gefordert und zum teil auch im unterricht schon praktiziert [8] . für lehrende der medizin und medizinstudierende scheinen solche techniken des improvisationstheaters ebenfalls nützlich zu sein -und zwar sowohl für die ärztliche tätigkeit als auch für das unterrichten von medizinstudierenden bzw. für die gestaltung des unterrichts [9] . das medizinische arbeiten ist von der natur der sache her unvorhersehbar. medizinstudierende müssen lernen, der medizinischen problemen innewohnenden unsicherheit adäquat zu begegnen. dies wird bereits didaktisch im problemorientierten lernen umgesetzt und führt zu besserem umgang mit unsicherheit im ärztlichen alltag [10] . außerdem gibt es rahmenwerke, die techniken des improvisationstheaters aufgreifen, um medizinstudierende zu befähigen, mit unbekannten medizinischen situationen umgehen zu lernen [11] . diese techniken der improvisation könnten ebenfalls geeignet sein, um lehrenden in unsicheren zeiten adäquates lehren zu ermöglichen [12] . besonders nützlich scheinen sie zum erlernen von kommunikationsfähigkeiten und professionellem verhalten zu sein [13] . [18] . dieses konzept lässt sich vermutlich leicht und kontaktarm an anderen hochschulstandorten zur prüferschulung einsetzen. diese beispiele zeigen, welchen wesentlichen beitrag projekte der medizinischen ausbildungsforschung leisten, damit lehrende auch in zeiten, in denen sie improvisieren müssen, auf evidenz zurückgreifen können. bleiben wir also dran an improvisierter lehre -wissenschaftlich fundiert und kreativ. die autorin erklärt, dass sie keine interessenkonflikte im zusammenhang mit diesem artikel hat. the fourth hiv epidemic non-a, non-b viral hepatitis from non-a, non-b hepatitis to hepatitis c cure the fourth hiv epidemic non-a, non-b viral hepatitis from non-a, non-b hepatitis to hepatitis c cure hus diary" of a german nephrologist during the current ehec outbreak in europe improving graduate medical education through faculty empowerment instead of detailed guidelines auftragstaktik -geschichte und gegenwart einer führungskonzeption. frankfurt/m: report-verlag improvisation in teaching and education -roots and applications medical education empowered by theater (meet) the effects of problem-based learning during medical school on physican competency: a systematic review common ground: frameworks for teaching improvisational ability in medical education twelve tips for using applied improvisation in medical education medial improv: a novel approach to teaching communication and professionalism skills learning under lockdown: navigating the best way to study online development of an instruction movie illustrating a standardized clinical examination of patients with tmd symptoms formative assessment of practical skills with peerassessors: quality features of an osce in general medicine at the medical faculty of heidelberg relationship between empathy and motivation in undergraduate medical students standardized examinees: development of a new tool to evaluate factors influencing osce scores and to train examiners improvisation -a new strategy in medical education? the author declares that she has no competing interests. key: cord-305736-g0jgfngc authors: mian, areeb; khan, shujhat title: medical education during pandemics: a uk perspective date: 2020-04-09 journal: bmc med doi: 10.1186/s12916-020-01577-y sha: doc_id: 305736 cord_uid: g0jgfngc nan as the coronavirus (covid-19) pandemic becomes widespread, its impact on society is becoming more pervasive and is now threatening medical education. numerous medical schools have suspended all clinical placements and classes with the hopes of mitigating viral transmission. the timing of this disruption is set to have profound consequences as universities, particularly in the uk, are now conducting assessments remotely, and some are considering deferring students due to the inability to carry out teaching and clinical placements. here, we discuss the different modes of teaching that may be offered during this time. over the last several years, some medical schools have shifted from traditional forms of 'in-person' lecturebased teaching to other modes, employing online, distance or electronic learning [1] . whilst not ideal, teleteaching or the delivery of live teaching via online platforms may prove to be an apt solution to the cancellations that are currently taking place. rather than leaving students to their own devices, online teaching guides student learning and places content within the overall context of their curriculum. currently, universities utilise lecture capture technology. however, this is limited in its interactivity and ability for students to ask questions. additionally, outside of the current crisis many are worried that it may lead to empty lecture halls and reduced participation, and often, the missed lectures are not caught up. perhaps it is now time for universities to consider utilising other modes of facilitating learning such as live teleteaching video conference platforms whereby student engagement and interactivity can be preserved, whilst observing appropriate covid-19 social distancing measures. whilst online platforms may be sufficient for students in their pre-clinical years, senior medical students who are placed in clinical environments require patient contact. indeed, communication with and examination of patients is necessary for learning and building a diagnostic clinical thought process, for as william osler proclaimed, 'he who studies medicine without books sails an uncharted sea, but he who studies medicine without patients does not go to sea at all'. as an alternative to clinical placements, students at imperial college london are being given access to an online repository of patient interview recordings and cases. many universities have released their clinical academics to work in the national health service (nhs), and the acute timescale for this event has meant that drastic reorganisation has needed to be done with little time for actual teaching. however, imperial clinicians are still delivering teleteaching through computers on hospital sites, which have seen excellent student attendance and interaction. nonetheless, student-patient engagement is still necessary, and teleteaching does not substitute actual patient contact. patients have a multitude of pathologies and present with varying signs and symptoms. they come with differing educational backgrounds, each presenting with a unique challenge. by not being able to engage with patients, developing key clinical skills will be more difficult. whilst this disruption may not affect senior students' skills, younger years are more likely to be adversely affected, as it is at this stage that their clinical foundation is set. rather than restricting student access to patients, telemedicine technologies may be utilised. one such approach uses tablet computers which can be cleaned between patients following appropriate infection control protocols. they can be used at sites with a high risk of covid-19 transmission; patients can be given a tablet and isolated in an exam room. in turn, both students and physicians can communicate with these patients without risking exposure to the pathogen and wasting personal protective equipment [2] . not only would this help clinical students to maintain and refine their diagnostic thought process but could also allow them to aid healthcare systems by reducing the burden of covid-19 through the triage of patients. healthcare provision through telemedicine will become the mainstream in the coming years. indeed, studies have found that interaction with telemedicine technologies during undergraduate medical training contributes to improved core competencies, medical knowledge, overall learning and higher quality patient care [3] . furthermore, examinations have suffered from cancellations. students are examined regularly throughout the course, and performances in different exams often have a good correlation. in turn, examination disruptions in a single instance would not alter the predictive ability of previous exams in assessing the competency of students. as a compromise, some medical schools are turning to utilise tele-technologies in order to conduct remote assessments in an effort to ensure that final year medical students have met the required competencies before they begin to practise [4] . a key educational dilemma involves looking at the length of the epidemic. if indeed, as seems to be the case, it was set to last several months; this would lead to a substantial loss of learning time for students and probable depreciation in confidence, although the slight loss of clinical skills would likely be quickly rekindled once students are back in a clinical environment. importantly, throughout this crisis, what will be ever-present is the use of textbooks. with the emergence of teleteaching platforms, both can be combined to fill in the gaps that would otherwise normally be learned from lecturers or clinicians on the wards. as a side note, learning should always be placed in the modern context, and great novels of the plague, such as daniel defoe's journal of the plague year, manzoni's the betrothed, or camus's la peste, can provide students with highly relevant perspectives to the current predicament we find ourselves in. this will not only illustrate why microbiologists have worried about 'the big one' for so long but may even motivate students to pursue a career in infectious disease and help in the prevention of futre outbreaks. as healthcare systems are set to be further stretched with the increasing burden of covid-19, disruptions in medical education are inevitable across the world. arrangements need to be made whereby students can retain clinical skills and knowledge. though not without its problems, teleteaching technologies have the potential to substitute in-person lecture and clinical-based teaching, particularly during this pandemic. such approaches may not only be necessary for effectively tackling the medical education dilemma during this current crisis but will also serve to lay the foundation for teaching during future disasters and beyond. abbreviations nhs: national health service current technology in advancing medical education: perspectives for learning and providing care virtually perfect? telemedicine for covid-19 telemedicine training in undergraduate medical education: mixed-methods review covid-19 response: edtech expertise strengthening remote learning transition. imperial college london authors' contributions all authors read and approved the final manuscript. availability of data and materials not applicable ethics approval and consent to participate not applicable the authors declare no competing interests.received: 30 march 2020 accepted: 30 march 2020 key: cord-355657-r7exrtlm authors: pamulapati, lauren g.; hess., danielle title: student pharmacists’ role in enhancing ambulatory care pharmacy practice date: 2020-09-07 journal: pharm pract (granada) doi: 10.18549/pharmpract.2020.3.2150 sha: doc_id: 355657 cord_uid: r7exrtlm with a primary care physician shortage, utilization of pharmacists in the ambulatory care setting has proven to have positive economic and clinical outcomes for the practice and for patients. to extend the reach of the pharmacists, students may assist with patient care activities, such as medication reconciliation, point-of-care testing, and counseling. evidence has shown that students benefit in building confidence, as well as improved perceptions of interprofessional care, while positive patient outcomes are maintained. there are many methods for schools to integrate these experiences early into their curriculum, as well as for students to explore opportunities on their own. a growing and aging population is contributing to an insufficient supply of physicians to meet the primary care provider demand. 1, 2 a report from the association of american medical colleges found that by 2033, there will be a primary care shortage of between 21,400 and 55,200 physicians. this sixth-annual report highlighted that population health goals can be achieved through collaboration with non-physician clinicians. 2 the increase in the number of advanced practice registered nurses and physician assistants has helped address some of the gaps in care by making more chronic and acute visits available for patients; however, pharmacists with an in-depth knowledge of pharmacotherapy can also play an important role in improving medication use to optimize chronic disease state management, thus further increasing access to care. [2] [3] [4] the provision of pharmacy services in the ambulatory setting results in improved clinical and economic outcomes. 5, 6 for the purpose of this article, ambulatory care addresses the delivery of care to patients who are able to ambulate, whether they are in a clinic, transitioning from a hospital to home, or in their home receiving services, through means such as telehealth. 7 pharmacy services are often provided under collaborative practice agreements (cpas), which allow pharmacists to initiate, modify, and discontinue therapy based on an agreement between a provider and a pharmacist. [8] [9] [10] the combination of pharmacist value recognized in the ambulatory setting, along with a changing health care environment, has resulted in the need for more ambulatory care pharmacists. 6 to account for this demand, postgraduate year 2 (pgy-2) specialty residency offerings in ambulatory care have grown by 85% since 2015, and as of 2019, there were 4,342 board certified ambulatory care pharmacists. 11, 12 while the need for and impact of pharmacists on the health care team have been established, the staffing ratio of physician to pharmacist has not been well defined. estimates vary from 0.20 pharmacist full-time equivalent (fte) to 1.0 pharmacist fte matched to a wide range and complexity of physician panels. 13 budget constraints often restrict optimal staffing ratios, thus the utilization of student pharmacists in the ambulatory care setting may offset these needs. the purposeful integration of students into the ambulatory setting may also positively impact clinical care and student education, as has been suggested with medical students. 14 in the ambulatory setting, there are many technical tasks, such as patient scheduling, obtaining vitals, calling pharmacies to obtain up-to-date medication lists, and investigating patient prescription benefits that take away time from more complex aspects of disease-state management, such as implementing medication adjustments, ordering laboratory tests, and documenting clinical services provided. 15 to better utilize clinical pharmacists' time, the university of wisconsin used primary care pharmacy technicians to fulfill technical tasks, which enabled pharmacists to focus on clinical tasks. 15 however, a limitation of this model is the need to hire more staff (e.g., pharmacy technicians) to fulfill this role. an alternative solution would be to utilize student pharmacists to perform technical and low-level clinical tasks, which would also have a minimal budget impact. table 1 provides examples of such tasks, which align with the american association of colleges of pharmacy entrustable professional activities (essential activities and tasks that new pharmacy graduates should be able to perform without direct supervision upon entering practice). 16 student pharmacists' role in enhancing ambulatory care pharmacy practice several studies have demonstrated the positive contribution of student pharmacists collecting medication histories, completing medication reconciliation, conducting patient interviews, performing point-of-care testing, and providing immunizations. [17] [18] [19] [20] [21] [22] [23] in an interprofessional setting, the use of fourth-year student pharmacists to obtain medication histories has been employed in a variety of clinics (e.g., internal medicine, cardiology, and pediatrics). 17 in this model, students spent an average of 10 minutes (range 2-30 minutes) collecting medication histories before reporting their findings back to the attending physician. 17 based on the time spent to obtain a full medication history, this model suggested that medical providers saved time by collaborating with student pharmacists, and students benefited by working in an interprofessional, real-word setting. 17 dalal et al. demonstrated that student pharmacists independently performing subjective interviews and obtaining fingerstick international normalized ratios before making therapeutic recommendations under pharmacist oversight provided comparable clinical care to pharmacist-only visits in an outpatient anticoagulation setting. 18 furthermore, there was no difference between third-year and fourth-year precepted students. 18 this study suggests that the utilization of student pharmacists to reach more patients does not compromise patient care. complementary findings regarding students' abilities to practice in an ambulatory setting have been illustrated when students perform medication reconciliation. 19, 20 in a depression clinic, students identified at least one medication list discrepancy in more than 50% of patients and an insufficient antidepressant trial in approximately 32% of patients through a pilot telephonic medication reconciliation and antidepressant treatment history program. 19 considering each call completed by the students required, on average, 18 minutes, students were able to optimize the pharmacists' time. 19 a retrospective review of this service found there were no differences among student educational level (e.g., all four professional years) for identifying medication discrepancies, suggesting that incorporation of students into ambulatory care practices can occur regardless of the professional year. 20 provision of immunizations is another task that student pharmacists can assist with, as many would have been trained at their institution. [21] [22] [23] training can be costly, due to the supplies needed to ensure hands-on experience; however, woelfel and colleagues compensated for this cost by utilizing immunization-trained student pharmacists to provide vaccinations to medicare beneficiaries in mobile clinics. 21 between 2015 and 2016, the net income generated by student pharmacists administering 1,777 vaccinations was usd 19,937, with the greatest return on investment from the influenza vaccine, followed by tdap and pneumococcal vaccine. 21 this program enabled students to practice their skills and gain experience with immunization coverage and billing while generating revenue for the school. 21 regarding humanistic outcomes, hannings et al. showed that 98% of the individuals who received a vaccine through a student-led immunization clinic were satisfied. furthermore, this clinic was able to capture approximately 68 individuals who would not have been vaccinated otherwise. 22 the availability of pharmacists and student pharmacists to provide immunizations has increased vaccine awareness and access, especially among low-income patients, thus resulting in a positive public health impact. to ensure that student pharmacists have the appropriate skills to practice in an ambulatory care setting, experiences should begin early and be sustained throughout their education. one way for student pharmacists to build skills prior to rotations is through purposeful and innovative didactic education. sando et al. demonstrated that use of a gamelike educational tool to teach medication history skills increased students' confidence levels and skills. 25 however, ambulatory care is more than just the clinical skill set, as it also entails practice management skills. for student pharmacists to obtain more from their experiential opportunities, education embedded into the pharmacy curriculum regarding the business model of ambulatory care is important. 26 students should be able to complete a market analysis, perform a needs assessment, accurately describe the service to be provided, and define how the service will be operated and sustained. 27 colleges and schools of pharmacy should promote classes from both the clinical and the business standpoints so that students stay abreast of the changing health care landscape and are prepared to perform the next generation of pharmacist roles. 28 to build upon the didactic content, students can work on a business plan for implementation of a new service or evaluation of an existing one as part of their fourth-year experiential ambulatory care rotations. since the onset of the coronavirus disease 2019 (covid-19) pandemic, some institutions have found unique ways to engage students in ambulatory care practice by providing patient care via telehealth visits, answering drug information inquiries, and providing educational opportunities for staff. 29, 30 the incorporation of student pharmacists into telehealth ensures their experiential training continues during the pandemic, while introducing -assist a patient with behavior change through motivational interviewing -interpret laboratory data and recommend therapeutic changes or additional monitoring -interpret patient provided health information to develop an evidencebased therapeutic plan them to a new approach to care that could remain postpandemic. evidence is limited regarding patient outcomes when students are involved in telehealth; however, a previous study suggested that students who were involved in telehealth medication therapy management (mtm) programs were more confident in the provision of comprehensive medication reviews compared to students who provided mtm in the community and hospital settings. 31 as telehealth continues to expand, the utilization of students and their impact on patient outcomes will be an area to explore. population health initiatives can also be completed via telephone and introduced into experiential education or offered for volunteer students. kaiser permanente colorado incorporated population health management activities for rotation students that entailed students reviewing patient records and identifying interventions that could be made by the student or a clinical pharmacist. 32 of the 1,406 actionable interventions documented by approximately 46 students, 52% pertained to patient education (e.g., student contacts the patient via telephone or letter for sole purpose of education), 23% to verification (e.g., calling a pharmacy to verify a patient refill history), and 10.5% to medication therapy adjustment (e.g., any dose or medication change within the same medication class). 32 the chart reviews, and subsequent completed interventions, when applicable, conducted by pharmacy students accounted for approximately 765 hours of clinical pharmacist time that would have been required over the four-year study period. 32 these innovative ways for engaging students allow student pharmacists to be exposed to ambulatory care practice, while offsetting pharmacist time. student pharmacists interested in ambulatory care should take it upon themselves to seek out experiences to engage in real-world experience. some opportunities may be facilitated through their school or college, but some may require the student to apply on their own. within the commonwealth of virginia, there are many opportunities for students to gain early, hands-on, exposure to ambulatory care. for example, in partnership with vcu school of pharmacy, free clinics in richmond that provide primary care and chronic disease management for the uninsured population offer ambulatory care electives for third-year student pharmacists, along with numerous volunteer hours. 4, 33 throughout the elective or volunteer hours, student pharmacists work as part of an interprofessional team, conducting patient interviews, addressing medication-related concerns, counseling patients on lifestyle changes and, if needed, providing smoking cessation services under the supervision of the attending clinical pharmacist or physician. in this practice model, student pharmacists not only gain experience with direct patient care and enhance their critical thinking and cultural competence and problem-solving skills, but the clinics benefit from increasing the number of patients that they care for by utilizing student pharmacists as pharmacist extenders. 4 not all students will have the opportunity to partake in a formal internship or have an ambulatory care practicebased elective offered. therefore, students could engage in domestic or international medical outreach events that focus on chronic disease state management (e.g., diabetes and hypertension), or preventive medicine (e.g., cardiovascular disease prevention). 34, 35 on these outreach trips, students engage in direct patient care through physical assessments, medication and disease state counseling, and interprofessional collaboration regarding medication choices or therapeutic interchanges. 34, 35 while these opportunities do not offer longitudinal ambulatory care experiences, they enable student pharmacists to have a glimpse of what ambulatory care practice entails. at the national level, externships are available with the indian health service (ihs) and the junior commissioned officer student training and extern program during the summer months. 36, 37 with a large emphasis on ambulatory care at many ihs facilities, students are immersed in ambulatory care practices with pharmacists to manage diabetes, anticoagulation, asthma, smoking cessation, hiv, and other chronic diseases. 37 beyond participating in patient visits, student pharmacists conduct research projects, create educational handouts for patients, and complete projects that advance clinical services. these opportunities are also available during ihs ambulatory care rotations for schools that have agreements with ihs facilities. 37 further opportunities for internships can be found with local ambulatory clinics or federally qualified health centers, as well as veterans affairs medical centers via the va learning opportunities residency. on an international level, there are opportunities for students to experience pharmacy in other countries through the student exchange program of the international pharmaceutical students' federation. 38 additionally, the vcu school of pharmacy offers an interprofessional international outreach opportunity, which enables students from any professional year to travel to south american countries and work with other health professions for an intensive two-week direct patient care experience. 33 furthermore, some schools of pharmacy may offer international fourth-year rotations. these experiences increase student pharmacists' cultural awareness, selfawareness, and knowledge of public health, while providing an environment for them to gain confidence in providing clinical care and making recommendations. [39] [40] [41] for students interested in a career with a public health emphasis, engaging in domestic and international outreach events is a way to gain a deeper appreciation for disparities in health. despite the many benefits to incorporating student pharmacists into the ambulatory setting, it does not come without barriers. the biggest barrier identified in studies is time. 20, 42, 43 this can include the time spent training students or planning events for students, such as medical missions or diabetes self-management courses. 42, 43 a suggestion to minimize this burden includes planning far enough in advance to overcome any legal barriers or site restrictions for student-led initiatives (e.g., immunization clinic). 42 involving more individuals in the planning is recommended, especially if engaging in an interprofessional activity. 43 lastly, ensuring there is enough time to train students, especially depending on the level of tasks the student will be performing, is a necessary consideration. 20 preceptor time may need to be invested up front to ensure students are properly trained; however, as described throughout, when the experiences are wellplanned, there is a potential for a large impact on student growth, patient care, revenue generation, and time-savings later on. [17] [18] [19] [20] [21] [22] [23] 32, 42, 43 with numerous opportunities for engagement early on in their careers, student pharmacists can actively contribute to the advancement of ambulatory care practice by functioning as pharmacist extenders. with the opportunity to utilize their clinical and business skills, student pharmacists may enhance patient care services, leading to improvements in quality of care and patient access. the population 65 years and older in the united states: 2106. the united states census bureau association of american medical colleges. the complexities of physician supply and demand: projections from the pharmacist as physician extender in family medicine office practice addressing access to care for the uninsured: clinical pharmacists as physician extenders defining and advancing ambulatory care pharmacy practice: it is time to lengthen our stride pharmacists' contributions to primary care in the united states collaborating to address unmet patient care needs: the emerging role for pharmacists to address the shortage of primary care providers ashp long-range vision for the pharmacy workforce in hospitals and health systems advancing team-based care through collaborative practice agreements: a resource and implementation guide for adding pharmacists to the care team the role of pharmacy through collaborative practice in an ambulatory care clinic the team approach to health care: optimizing team-based care with the use of collaborative practice agreements press release: ashp 2020 residency match phase i signals steady growth in positions primary care teams and pharmacist staffing ratios: is there a magic number how can medical students add value? identifying roles, barriers, and strategies to advance the value of undergraduate medical education to patient care and the health system primary care pharmacy technicians: effect on pharmacist workload and patient access to clinical pharmacy services development of a student-led ambulatory medication reconciliation program at an academic institution pharmacy students provide care comparable to pharmacists in an outpatient anticoagulation setting impact of a student pharmacist driven medication reconciliation and antidepressant treatment history project at a depression clinic: a pilot study discrepancies identified through a telephone-based, student-led initiative for medication reconciliation in ambulatory psychiatry administration, billing, and payment for pharmacy student-based immunizations to medicare beneficiaries at patient perceptions of student pharmacist-run mobile influenza vaccination clinics implementation of a pharmacist-driven immunization program designed to improve overall vaccination rates in indigent and uninsured patients impact of pharmacist immunization authority on seasonal influenza immunization rates across states an educational tool for teaching medication history taking to pharmacy students perceptions on a business plan assignment for an ambulatory care pharmacy elective developing a business-practice model for pharmacy services in ambulatory settings preparing for the next generation pharmacists maintaining comprehensive pharmacy services during a pandemic: recommendations from a designated covid-19 facility pharmacist-led chronic care management for medically underserved rural populations in florida during the covid-19 pandemic confidence in skills applied to patient care among pharmd students in telehealth medication management programs versus other settings value of pharmacy students performing population management activity interventions as an advanced pharmacy practice experience pharmacy student learning through community service the value of pharmacy services on a short-term medical mission trip: description of services and assessment of team satisfaction student opportunities: usphs student externship program and senior year rotations global health learning outcomes in pharmacy students completing international advanced pharmacy practice experiences a medical mission to guatemala as an advanced pharmacy practice experience describing pharmacy student participation in an international, interprofessional medical mission trip as part of an advanced pharmacy practice experience (appe) planning and implementation of a student-led immunization clinic a diabetes self-management education class taught by pharmacy students none to declare. the authors have no financial or conflicts of interest to disclose. key: cord-025858-xm0ih4ei authors: mikułowski, dariusz; brzostek-pawłowska, jolanta title: multi-sensual augmented reality in interactive accessible math tutoring system for flipped classroom date: 2020-06-03 journal: intelligent tutoring systems doi: 10.1007/978-3-030-49663-0_1 sha: doc_id: 25858 cord_uid: xm0ih4ei evermore widespread “flipped classroom” learning model is associated with increased independence of learning. the problem is the independence of learning math by students with visual impairments, especially the blind. mathematical content includes spatial objects such as formulas and graphics, inaccessible to blind students and hardly accessible to low vision students. they prevent independent learning. the article presents a method that increases students’ independence in recognising mathematical content in textbooks and worksheets. the method consists in introducing into the document elements of augmented reality (ar), that is texts and sounds extending information about the mathematical objects encountered in the content, beyond the information provided by wcag guidelines and recommendations of the wai-aria standard under development by the w3c consortium. access to ar elements is gained through multi-sensual user interface hearing, the touch of a braille display, touch screen and touch gestures. the method was developed in cooperation with students with visual impairment and math teachers. it is currently undergoing valorisation in poland, the netherlands and ireland. the electronic educational materials are indispensable for the increasingly widespread trend of the "flipped classroom" teaching model. they must be both attractive to keep the student's attention and accessible to disabled students. some groups of students, such as the blind, have limited access to electronic materials containing visualised spatial objects because the blind's perception of the environment mainly focuses on sound and tactile senses. these problems are more apparent in subjects such as mathematics, physics, chemistry, that are rich in math formulas and drawings. the accessibility barrier is reduced or aligned by creating materials according to the wcag (web content accessibility guidelines) and wai-aria (web reach internet application) standard [9, 10] , under development by w3c consortium. the wcag and wai-aria specify additional elements of web pages that increase accessibility as well as a user interface based on keyboard shortcuts for exploring them. so, we can say that by interacting with pages created in this way and using the interface according to the wcag and aria recommendations, the blind users are in virtual reality, which for them becomes an audible and tangible reality. however, the wcag and aria recommendations do not include support for finding, recognising, and exploring specific objects such as mathematical formulas, function graphs, drawings of geometric figures, mathematical quizzes, and tasks of the type "join into pairs". the problem of access to mathematical content by visually impaired users, especially blind users is well known. numerous scientific publications report further new solutions that help in recognising the mathematical content. the research on the latest ar devices has demonstrated their extreme usefulness for increasing the student motivation in the learning process [5] [6] [7] . in [3] an in-depth analysis of published research on ar and statistics on the use of ar in education and goals of their use is presented. the two most frequently indicated goals were additional explanations of the subject and extension of information. the latest book on the subject published in 2020 [2] , a multi-author review of the state of ar applications in education, confirmed the effective impact of multi-sensory ar in early education, similar conclusions are in [8] . in early mathematics, as well as advanced mathematics, for the needs of sighted students, ar is used to visualise 3d mathematical expressions [1] . according to the authors' knowledge, there are no reports on research on the multi-sensual ar in math education of blind and low vision students. especially for this group of students using alternative interfaces, who instead of sight rely on the senses of hearing and touch, there can be effective solutions providing additional information about each mathematical object using the multi-sensual user interface. the article describes information superstructure that is received multi-sensually by the student, beyond the information recommended by wcag and aria. it allows for interactive recognition of mathematical content. in conjunction with the developed technique, accessible for editing formulas, quizzes with math content, and creating graphics, it gives the blind student the possibility of creative, interactive, independent math learning. in order to learn mathematics effectively, blind students should be as independent as possible, especially in the "flipped classroom" learning model. this means that they should have access to all elements of a mathematical multimedia document, e.g. a manual or worksheet. visually impaired students must be able to analyse thoroughly, imagine, and understand each mathematical object. otherwise, they are threatened with cognitive impairment. in their daily work with the computer, blind users use only the keyboard and special software called a screen reader. that is why the blind persons' exploration of the document is different from that of a sighted. for example, instead of clicking a mouse pointer, they will use the tab key to highlight the button and then will press the space bar. the accessibility of different mathematical objects can be achieved by enriching pages with additional information elements forming the augmented reality (ar) layers in opposition to the cognitive reality resulting from the use of the wcag and aria elements. we propose a method of enriching a mathematical document with multi-sensory ar elements, applied by the user in the subsequent stages of more and more explicit recognition of mathematical objects. there is one particular ar information layer associated with each recognition stage. there are three ar information layers supporting the user in three subsequent stages of exploring a mathematical document. there is also the fourth optional layer used depending on the teacher's decision and/or the student's needs. additional ar information elements in the form of texts and sounds of various types are accessible through the multi-sensual user interface such as synthetic speech, touch gestures on the touch screen or by the touch of the haptic braille display also called as a braille line (see table 1 ). it is a device connected to the computer, presenting the characters in the form of protruding pins forming the letters of the braille alphabet. the first general information layer (additional to wcag and aria requirements), contains text elements that are conveyed by synthetic speech, and it informs about encountering object such a graphics, mathematical formulas, quizzes, questions, answers, pairing fields and links to comments recorded by a teacher (see table 1 ). the first layer user interface (ui) is a set of keyboard shortcuts that allows for simple finding locations of initially recognised objects in a document. it is also enriched with the haptic interface for the braille display. for quickly locating mathematical objects of a particular type in a document (formulas, graphics, quizzes and others), there is a set of shortcut keys, the so-called hotkeys for quick access. when the student, while navigating a document, using arrow keys on his keyboard, puts the cursor on a mathematical formula, the screen reader, through a speech synthesiser, will read to them that it is a formula. similarly, he/she will get information on other types of elements such as graphics, pairing fields, questions and answers in quizzes. moreover, each of this message is sent simultaneously to the braille display in a shortened form to accelerate the speed of haptic reading. after encountering a mathematical object and recognising its type, the student can examine it more thoroughly. it is possible, thanks to the ar elements placed in the second layer (see table 1 ). this layer contains information elements such as semantically readable formula text in polish or english via speech synthesiser, graphic titles and descriptions, and content of pairing fields, questions or answers that can be a text, a formula or a graphic. the student can simultaneously read this information using the braille display. ui of the second layer is a set of keyboard shortcuts used to obtain information about each math object in the document. additionally, for purposes of haptic reading of formulas, this layer is equipped with converters of mathematical mathml web notation [11] to ueb -english mathematical braille notation [12] and bnm -polish mathematical braille notation [13] . 3. x equals fraction numerator minus b plus minus square root of b minus four a c end of the square root denominator two an end of fraction braille display selected formula the selected formula in ueb notation on the braille display: an example of using the second ar layer when students encounter an object in a document and are informed that it is a mathematical formula, they can become familiar with its content. the text of semantic reading of the formula in polish or english is automatically created and then read with synthetic speech. for example, for the formula 1/3 + 2/3 = 1, the text of the readout is: "one third plus two thirds equal one." in addition to listening, the student can read the same formula haptic through touch using a braille display. it is possible thanks to converters of mathml formula notation in html to braille bnm (polish) or ueb (english) notation. the third ar layer was developed for detailed exploration of formulas, function graphs and geometric drawings. it consists of the following items: texts of automatically generated or/and manually added descriptions of graphic elements; comments of the whole graph and its elements; texts of the formula elements; continuous sounds with constant and variable monotonicity describing graph elements; short technical sounds called audio-icons. ui access to the third layer is provided by four keyboard shortcuts, numeric keyboard keys, several on-screen touch gestures, a haptic touch of a braille display, synthetic speech and bolded lines outstanding graphic elements (see fig. 1 ). the third informative layer enables detailed, interactive exploration of complex formulas and mathematical drawings by the following elements: 1. reading parts of the structures of formulas and descriptions of drawings introduced during their editing (by synthetic speech and a haptic touch of the braille display), 2. listening to the monotonic sound generated while the user is touching lines of geometrical figures and function graphs (through touch gestures), 3. listening to the sound of variable monotonicity describing the function graph (through touch gestures). additionally, the user is informed with short beeps on zero's points of a function and long beep about surpassing the drawing area during the exploration with touch gestures. the students need to analyse the complex formula more thoroughly (see table 2 ). for this purpose, they can use the third ar layer, thanks to which the formula structure becomes fully accessible. the students use the keyboard shortcut shift + ctrl + 1 to enter the formula structure and navigate, like on a tree, through this structure while hearing the reading of the focused parts of the formula. they use the arrow keys or touch gestures to read the formula elements. in this way, the students can build a formula structure in their imagination. the thorough exploration of mathematical drawings is done with the help of sound signals generated when sliding one's finger on the touch screen with displayed graphics on full screen. the student enters the drawing with two keyboard shortcuts, to interactively explore or to passively recognise it. by moving his finger on the screen, the student hears various sounds representing objects contained in the drawing. reality on the screen augmented reality 1. x equals fraction numerator minus b plus minus square root of b minus four a c end of square root denominator two an end of fraction 2. ctrl+shift+1 x equals fraction numerator minus b plus minus square root of b minus four a c end of square root denominator two an end of fraction 3 . or thanks to the feature automatically supporting graphics creation process, parameters given during its construction, such as coordinates of points, length and width of the polygon or radius of a circle are transformed into texts describing the given object. this automatically generated description is read after the tap gesture when the sound generated along the line or at the point is heard. this automated drawing creation mechanism is fully accessible to a blind student, thanks to which he/she can create or modify mathematical graphics on his own. when the student is "immersed" in graphics by touching its elements, then thanks to the 3rd ar layer, he/she can also modify it on his/her own. thanks to this, immersive graphics learning is implemented. next, we present a detailed description of the three methods of recognising graphics in the third ar layer. when the student hears a sound while sliding finger on the screen, then as a result of the finger tapping gesture he/she will hear information with synthetic speech, relating to the encountered element of the drawing: "axis x", "axis y", <coordinate values> of the touched graphics elements, <text describing the figure's elements>, <text of a com-ment>, <the comments and additional descriptions> in addition to those generated automatically are entered by the teacher or by the student while editing the drawing. listening to monotonic sound while sliding the finger over the bolded line of a figure or a function graph. the displayed drawing line is specially thickened to keep the finger moving within its area. listening to automatically generated sound with variable monotonicity, describing the function graph. the pitch informs the student about the y coordinate. students, knowing that the graph is always played from the smallest to the largest values on the x-axis, can build in their minds an image of the function graph they are listening to. additionally, with particular sounds, students are informed about zeros of a function and on finger movement outside the drawing area. the elements and ui of the part of the third ar layer (for exploration of the function graph) are shown in fig. 1 . the fourth layer, optional, contains information recorded by the teacher as hints, tips, or opinions in the form of static video (just sound) embedded in the content as a link to a resource placed on youtube. this layer is supported by standard keyboard shortcuts that allow for opening links in a browser. it is up to the teacher whether to post comments and up to the student whether to use this form of teacher's hints. thanks to this combination of ar reality providing the right tools, the students in their cognitive reality can enrich their mathematical knowledge. in the proposed method, the mathematical content can be independently, interactively, explored by students with visual impairment due to the addition of several layered ar, which is formed by subsequent information layers. the presented multi-sensual ar was implemented in the online environment euromath, in english and polish, for independent learning of mathematics for blind and visually impaired students, also for the creation of mathematical content by teachers and students. the euromath environment consists of a web application for creating and exploring interactive multimedia mathematical content and portal supporting the repository of open educational math resources (oer) in the form .epub files [4] . oer consists of math content accessible to students with visual impairments (blind, with low vision) and methodological materials for teachers. math teachers from poland, the netherlands, and ireland have already created over 300 multimedia mathematical documents using the web application and uploaded them to the oer repository. mathematical educational resources (oer) are described in detail with metadata that classifies resources by mathematical subject, type of school, level of education, visual impairment. students can search for materials helpful in learning mathematics using the search engine available in the repository. moreover, they can also support cooperation in a group of students on a discussion forum available on the euromath portal. euromath is an intelligent system supporting the learning process of students with visual dysfunction as well as teachers in creating educational materials: • in the euromath system, instead of several versions of a given document, for example, a work card, intended for the sighted student, the blind student and the low vision student, the teacher creates one universal version, which is adapted to the needs of each group of students, as a result of the application of conversion algorithms; • the euromath is teacher-friendly because: a) shortens the time needed to create universal materials by automatically generating and supplementing information about mathematical objects contained in the document (types of objects, the content of formulas and their elements, coordinates of graphic elements, zero points, and function graphs), b) enables the teacher to choose tools for editing and reading formulas and graphics, in extreme cases a blind math teacher (there are some) can create formulas and mathematical graphics available for each student according to their needs; • euromath is useful for blind and visually impaired students who can independently read and edit documents with formulas figures drawings and function graphs. the capacity to read is facilitated by the three described ar layers and the special ui. the possibility of interactive immersion in the environment of mathematical objects, which a student with visual impairment can "touch" and cause their change, is made by a set of developed algorithms for converting mathematical notation (such as ueb/mathml/ueb, asciimath/mathml/ueb, asciimath/mathml/asciimath), translation of mathml notation into semantic texts of the readout, translation of vector graphic inscriptions into sounds of constant and variable monotonicity and special ui adapted to the needs of the student; • mathematical resources in the euromath repository allow each student to follow the appropriate educational path. so that all material from this path is tailored to the needs of the student in terms of methodology and technical availability. euromath was developed as part of a project (2017-2020) under the ue erasmus + program. the presented multi-sensual ar was designed together with blind students and math teachers of the final technical classes of secondary schools. while this method has been developed with the participation of blind and visually impaired students and their math teachers, we can conclude that it has been accepted by potential users. it is currently being evaluated by mid-2020 by teachers with students in poland, ireland, and the netherlands. due to the narrow target group, we can only conduct qualitative research. euro-math valorisation was undertaken by mathematics teachers with their students from the only special centre for children with visual impairment in ireland, from two, out of nine existing, special centres in poland and from five schools run in the netherlands by an expert centre for blind and low vision children. since the outbreak of the covid19 epidemic, when schools and universities have been closed, the interest in ict learning supporting tools in the reverse class model has increased as well as the interest in the euromath system as a tool that could be useful in this model. the third special centre in poland and one of the universities of technology in ireland will also partake in the valorisation of euromath. currently, euromath is being tested by 13 mathematics teachers and students: four from the netherlands, four from ireland, including three academic teachers, five teachers from poland, altogether 12 educational units. the effectiveness of the ar + ui method increasing the availability of mathematical content implemented in euromath has been confirmed by teachers from ireland and poland. one centre in poland is already using euromath in its ongoing operations. the abovementioned university of technology has expressed interest in implementing the euromath system in its infrastructure. teachers from the netherlands also sent a preliminary positive opinion about the method of sharing mathematical content described in this paper. in connection with the closing of schools, blind students from special centres began remote education, in which a visually impaired student is forced to become more independent in the learning process. to what extent our method has increased this independence in practice. we will be able to examine after the covid 19 epidemic. it is worth noting that the students' self-reliance is influenced not only by the ar + ui method itself but also by the quality and attractiveness of the materials prepared in the euromath application -work cards, exercises and tests available in the repository. to facilitate teachers' work in this area, help materials in the form of instructional videos have been made available on the you-tube channel dealing with the euromath system. mathematics learning and augmented reality in virtual school augmented reality in educational settings kinshuk: augmented reality trends in education: a systematic review of research and applications enhancement of math content accessibility in epub3 educational publications a psychological perspective on augmented reality in the mathematics classroom development and behavioral pattern analysis of a mobile guide system with augmented reality for painting appreciation instruction in an art museum impact of an augmented reality system on students' motivation for a visual art course multi-sensory math activities that really work web content accessibility guidelines (wcag) 2.1 mathematical markup language (mathml) version 3.0, 2nd edn brajlowska notacja matematyczna fizyczna i chemiczna (bnm) key: cord-337581-3rmrkq1k authors: ramos-morcillo, antonio jesús; leal-costa, césar; moral-garcía, josé enrique; ruzafa-martínez, maría title: experiences of nursing students during the abrupt change from face-to-face to e-learning education during the first month of confinement due to covid-19 in spain date: 2020-07-30 journal: int j environ res public health doi: 10.3390/ijerph17155519 sha: doc_id: 337581 cord_uid: 3rmrkq1k the current state of alarm due to the covid-19 pandemic has led to the urgent change in the education of nursing students from traditional to distance learning. the objective of this study was to discover the learning experiences and the expectations about the changes in education, in light of the abrupt change from face-to-face to e-learning education, of nursing students enrolled in the bachelor’s and master’s degree of two public spanish universities during the first month of confinement due to the covid-19 pandemic. qualitative study was conducted during the first month of the state of alarm in spain (from 25 march–20 april 2020). semi-structured interviews were given to students enrolled in every academic year of the nursing degree, and nurses who were enrolled in the master’s programs at two public universities. a maximum variation sampling was performed, and an inductive thematic analysis was conducted. the study was reported according with coreq checklist. thirty-two students aged from 18 to 50 years old participated in the study. the interviews lasted from 17 to 51 min. six major themes were defined: (1) practicing care; (2) uncertainty; (3) time; (4) teaching methodologies; (5) context of confinement and added difficulties; (6) face-to-face win. the imposition of e-learning sets limitations for older students, those who live in rural areas, with work and family responsibilities and with limited electronic resources. online education goes beyond a continuation of the face-to-face classes. work should be done about this for the next academic year as we face an uncertain future in the short-term control of covid-19. the fast propagation of the severe acute respiratory syndrome coronavirus 2 (sars-cov-2) led to its definition as a pandemic on 13 march 2020 by the who [1] , as it met the epidemiological criteria and had infected more than 100,000 people in 100 countries [2] [3] [4] . the main public health recommendation was to remain at home and stay safe within it [5] . the world, in a globalized manner, is facing an extraordinary public health emergency in which the nurses are, as always, on the front line. challenges are even greater in this period of pandemic [6, 7] , and nurses have the knowledge and aptitudes for providing the care necessary in the different clinical scenarios [5] that are emerging. this research study was conducted in its entirety during the first month of the state of alarm in spain (which began on 14 march 2020). the state of alarm implied the confinement of the entire population, the closing of all the schools and universities, closing of non-essential businesses, closing of borders and ceasing all non-essential activities. the people were only allowed to go out to the street for essential matters: shopping of food, going to pharmacies, banks, and to care for older people who were dependent, etc. in spain, bachelor's degree in nursing has a duration of four years (240 european credit transfer system, ects) and it is common for a master's degree to have a duration of one year (60 ects). the reference population in this study was students from every academic year in the bachelor´s degree in nursing, or nurses who were conducting their master's studies, enrolled in universities in murcia and granada (spain). the participants were selected through the use of a maximum variation sampling strategy [13] to obtain heterogeneous and rich information that represented the main sociodemographic variables: gender, age, academic year, rural/urban, children, bachelor's/master's, university of murcia and granada. the maximum variation strategy is utilized to find the greatest diversity of discourses possible to identify and analyze the largest volume possible of expressions/presentation of the phenomenon studied to explain conditions/contexts where each one of them takes place. if one did not answer the request, the students themselves proposed a replacement with another participant with similar characteristics. none of the students contacted disagreed to participate. the students were invited to participate through the student delegation at the university, utilizing snowball sampling. this technique allowed us to build the sample by asking each interviewee for suggestions of people who had a similar or different perspective. this is an approach for locating information-rich key informants [13] . the saturation criterion was applied to establish the number of informants needed, an accepted method to estimate the sample size [14] . semi-structured interviews were conducted to obtain the information. the semi-structured interview is normally based on a script, where the subject matter and part of the questions have been planned before starting, but it also offers the possibility of changing or adding new questions as the interview and/or the research study moves forward, with new interviews conducted. it is the most common type of interview utilized in qualitative research on health. data were collected from 25 march to 20 april 2020. this was done in the first month as it the period of time with the greatest cognitive and social impact on learning and to obtain results that could be used to support, or not, the education measures that were utilized. all the interviews were individual and were performed online through electronic resources after agreeing on a day and time. the interviews were recorded and notes were made after each interview. all the interviews were conducted by researchers who had sufficient training and experience in semi-structured interviews (a.j.r.-m., m.r.-m.). the interviewers did not have an academic relationship with the informants. the interview followed a script which shifted from general to specific matters, and dealt with general aspects of the confinement, teaching methodologies utilized, learning and expectations (table a1) . a prior pilot study of the script was conducted [15] . the 6 phases proposed for the thematic analysis were followed [16] : (1) familiarizing yourself with your data; (2) generating initial codes; (3) searching for themes; (4) reviewing themes; (5) defining and naming themes; (6) producing the report. the recorded interviews were transcribed verbatim. once transcribed, the interviews were imported to the maxqda 12 program for its posterior analysis. a.j.r.-m., m.r.-m., c.l.-c. and j.e.m.-g. coded the data. the transcriptions, coding and themes-subthemes were discussed by the research team for their verification. finally, participants provided feedback on the findings. the study was reported according to the consolidated criteria for reporting qualitative research (coreq) [15] . this research study was approved by the research ethics commission from the university of murcia (id: 2800/2020). all the participants received an informational electronic document about the purpose and research process, which they later kept. they were advised that their participation was voluntary. they could ask and reflect prior to the interview. each participant was given a code to maintain anonymity. a total of 32 interviews were conducted, and they lasted between 17 and 51 min. the shortest interviews corresponded to the more advanced academic years (3rd and 4th year students). of these participants, 75% were women and 25% men. the age of the participants oscillated between 18-50 years old, with an average age of 25.3, and with a participation rate of 69% for the students from the university of murcia, and 21.8% for the university of granada students. the sample was composed by 18.75% of the students enrolled in their 1st or 2nd academic year or in the master's program, which accounted for about 57% of the sample, and 21.8% from the 3rd and 4th academic years, for a total of about 43%. of those interviewed, 21.8% had children and 21.8% lived in a rural setting. some of the characteristics of the participants are found in table 1 . six major themes were defined: (1) practicing the nursing care; (2) uncertainty; (3) time; (4) teaching methodologies; (5) the context of confinement and the added difficulties; (6) face-to-face education win. a detailed description of the themes and sub-themes can be found in table 2 . the outstandingly practical component of care in nursing education was the most emotional aspect for the students. the experiences found were differentiated according to the group of students, depending if they had or not practice-based subjects during the education period affected by the state of alarm, the proximity to ending their training as nurses, or if they were health professionals who were conducting post-graduate studies. for 1st and 2nd year students, the learning is normally done with courses that are eminently theoretical or theory/practical. the informants indicated that this transitory e-learning will not have a special influence on their training, as long as all the clinical training on health care institutions is present: "in think that it's not something that will affect us excessively for good or bad. in my year [1st] . in other years it will, because they have clinical training" p5 by contrast, 3rd and 4th year students whose coursework is mainly based on clinical training in health care institutions placed value on clinical training. they linked it with the acquisition of competences and referred to it as being an essential part of health sciences degrees: "my education would not be good if clinical training was missing" p15; "without the clinical training, we can't acquire competences" p19; "especially in our degree, the clinical training . . . " p21 clinical training provides them with security in the learning of nursing care in health care services. part of the students in their last year (4th year) indicated that they would rather not graduate in july to do all the clinical training, therefore graduating later: "i don't feel prepared. my erasmus in italy was really bad because i was a nursing student and a foreigner. at the hospital, i don't feel confident" p10; "some of us prefer not to graduate in june and to do the clinical training" p14 the master's students indicated that not being able to do the clinical training implied the loss of job opportunities: "if you cannot do the clinical training, you will lose job opportunities" p22 all the participants expressed their wish to help during the pandemic. they expressed their desire to be nurses to help. at the university of granada, a list of volunteers in their 4th year was even created. the expectation was present that the government could mobilize them in case of need. independently of the academic year, for all the students, this crisis re-enforced their wishes to become nurses: "i wish i already had my degree" p17; "i wish to be a nurse already, too bad i wasn't in 4th year so i could go" p16; "if this happens in the future, i would like to be helping" p25; "i feel like left out, i can't be in the battlefield helping" p21; "now i really feel like being a nurse. it is a shame that we cannot help. in granada there is a list of volunteers. i really feel like helping" p14 master's students who work feel satisfied to be able to help (aside from being satisfied because they can work): "i feel very well with myself because i can help, even though is very difficult . . . " p23; "i really feel like being in the middle of it and help. i've seen that help is really needed, it is very important work, although not very much appreciated." p24 the lack of concretion about the different aspects related with their studies is mentioned by all the interviewees. this uncertainty is accompanied by unpleasant situations due to the possible outcomes. they are mainly related with matters that could not be resolved relatively fast, such as the clinical practice and the adaptation of evaluation processes: "we don't know how they are going to evaluate us. they will for sure evaluate what we have done in the last month of clinical practices" p6; "we don't know what's going to happen. i hope they don't give a general pass. i want to take the exams and the other things. i don't want them to evaluate me with just one work submitted" p5; "not knowing how things will be done. not getting the grades i want to get because of these circumstances" p16 this is especially important for the 4th-year students, who reported a great feeling of wasting time. they cannot go to the clinical practices and they only have, as well, one subject: the final project (tfg). one of the alternatives to not waste time completely and that is being done by the participants is to prepare for the access exam for clinical nurse specialist training (national post-graduate residency program, eir). some of the participants indicated that preparing for the eir exam was a means of escape from a situation of wasting time and total paralysis: "it takes my motivation away, and (finishing my degree) is getting really hard, because i don't see the end of it" p10; "i am not taking advantage of the time" p3; "i'm preparing for the eir exam at the academy as a means of escape. with the only the tfg . . . i need something else. right now all my time is tfg and the subjects from the eir" p4 the 3rd-year students find themselves in the same situation but without any subjects: "the 3rd years clinical training has been abandoned. they don't know if we are going to recover them" p6 the masters' students have a different point of view. the differences are many. the masters' degree can provide job opportunities, the change from traditional education to e-learning practically affects an entire trimester (half of the master's program), and in their discourse, they have fewer demands and less pressure for obtaining the degree. at the same time, they are the only ones who speak about the teaching guidelines, indicating that they are truly being followed. in comparison, only one bachelor's degree student referred to the teaching guidelines: "if the clinical training cannot be done, you miss job opportunities" p22; "i don't know how the teaching guidelines have changed" p16; "the clinical training have been postponed until september, and it bothers me some because it interferes with the summer contract for working as a nurse" p24 time is a determinant transversal aspect. two differentiated phases are observed as the state of alarm moved forward (1st shock and 2nd normalization). besides, participants reflected regarding a necessary time management and the influence in the future. two well-differentiated phases are distinguished in the timeline. on the first days, the shock phase appears (1st phase), within which we find "disorientation". this first phase lasts between 7-10 days. during this first week, it is observed that mental performance decreases, along with the ability to concentrate. this is a subtle expectant phase, where the situations are not well defined: "you think that the first week is for you, for resting, you take care of unfinished business and uncertainty increases" p11; "the first week was not assimilated, i didn't have routines" p19; "during the first week, i had less concentration and studies less" p5; "the timetable is different, it's more chaotic" p21 after the first phase, the students enter a normalization phase (2nd phase) in which they acquire new routines, attend online classes and seminars. the conditions of confinement start to be assimilated and the new everyday life is normalized: "now i do more things than before, i take more notes. it is very different from the first week, now it is easier" p25; "now i have the habits. before i didn't do anything, and now i do everything, it is as if i'm getting used to it" p28 the first phase, as well as the second phase also coincide with the period in which the university ensured that the online tools were fully functioning and instructions were given to the professors about how to continue with their teaching tasks: "only 2 out of 5 teachers give online classes, the rest upload presentations that we have to understand" p13; "the teachers do not agree with each other. one says one thing and another something else" p6 the 1st and 2nd year students, as well as the masters' students, have classes. this forces them to manage their time differently. the 1st and 2nd year students interviewed indicated that time management was necessary. they indicated that this was beneficial for having good "mental health", and that having due dates helped them with managing their time: "having self-discipline and a timetable. not rigid, but saying that the mornings were for university and the afternoons for watching t.v. series or exercising. if you don't organize your time, work accumulates" p20; "my planning is monday to friday mornings for work, and the afternoon for group work or leisure. i rest on the weekends. having due dates has helped me organize" p13; "the homework is good, because they help with following the course" p11 all of the participants, except for the ones who worked, indicated that they had changed their sleep schedule and go to bed much later, between 1:30 and 3 a.m. the main reason mentioned was that the lack of activity did not make them tired, although this argument was ambivalent, as they went to bed later and got up later as well, so they slept the same number of hours: "it takes me longer to fall sleep. i'm not tired because i don't do anything during the day" p11; "i go to bed later and i get up later. i go to bed at 2-2:30 a.m. and i get up at 10" p5; "i fall sleep very late. at 2:00 a.m. the hours have changed, you sleep when you shouldn't" p19 the participants indicate that this situation affects their future plans and expectations related with obtaining their degree and work. they believe that they can be singled out for being the promotion with missing education, their international training is paralyzed, and they are afraid. their professional expectations are also affected: "i'm afraid of having bad training and that the work exchange says that this year's promotion from the university of granada do not have the competences necessary" p6; "the plans for earning money to go to an erasmus program are cancelled . . . " p13; "the practices have been postponed to september, and yes, it bothers me because it interferes with the summer contract for working as a nurse" p24 the participants indicate that as for the teachers, different teaching methodologies are being utilized: real-time videoconferences (including chats), lessons recorded on video and uploaded to the e-learning platform, audio podcast, chat (exclusively), homework and uploading of documents (word, ppt, pdf). they also mention that as time goes on, the teacher's adaptation to the online resources continuously improve. it is without a doubt the best evaluated. this is because they think it is the most similar to a traditional class (face-to-face), and allow interaction with the professor, and provides them with nearness. another aspect they indicate as being valuable is that this methodology helps with the teacher's explanation of the subject that is more comprehensible as compared to other methodologies. the interaction is also valued, as it allows them to say that something has not been understood and that it should be explained in another way. lastly, they would like all the videoconferences to be recorded so they could be watched again whenever needed. this last aspect was pointed out by the students who were also working: "the interaction in the videoconferences is not the same, because the questions are written and it is not the same to write something than when you talk" p26; "the videoconference is where we receive feedback. you can say that you don't understand something and if it could be explained once again" p16; "it is a way to stay in touch. doubts emerge and the teacher can resolve them" p7 the master's students indicate that on some occasions, the duration of the videoconference classes is excessive. it is interesting to highlight that the bachelor's students did not state this at any time: "we've had videoconferences that lasted 5 h. this can be done better. we had one who did a good summary and it lasted 2 h. this is more relaxing, and then you broaden the knowledge with the documents provided" p24 despite the value of the videoconferences, the discourse is ambivalent, as negative aspects are identified, especially related with the quality of interaction with the professor. the traditional classwork contributes fundamental elements in the quality of communication, and this how it is felt by the participants. "it is worse. when the teacher sees you asking about a doubt, she/he knows where you are coming from. this is lost with e-learning. information is lost and the student does not obtain the same information as in the face-to-face class. the teacher doesn't see your face." p24; "i'm much more in favor of traditional classes. i always obtain more information in them and i'm more comfortable." p24 except for the recorded lessons, the rest of the methodologies are catalogued as sub-standard. the chats (exclusively) and the homework are not attractive, although they value them as positive aspects because it lets them stay connected with the subject and the university: "the worse thing is when they only upload class notes, no one forces you to read them" p25; "in the homework, there are questions because they are not easy to understand, with the explanation it is easy, but when you are going to do it, it is more difficult" p25 among the limitations, they point out that in some asynchronous methodologies and with a rigid format, limited learning is obtained, interaction is needed for explanation, and a certain amount of pressure is needed. another limitation is the lack of feedback with the homework: "we are going to learn the minimum, but not all, because they don't explain it to you, they don't explain it in different ways. the text [from the documents] is only written in one way . . . " p7; "the works that don't have feedback give you half the knowledge" p13; "if you only upload notes, no one is forcing you to read them. it is very easy to fall into laziness when they only upload notes" p25 a limitation of e-learning that was pointed out by all the participants was that everything that was practice-related could not be learned. they identified this as a great limitation, and point out that in nursing, practice was vital: "many things are not understood through the computer. for example, the basic care laboratories have to be observed and practiced" p7; "the practical things not, but the theoretical yes. they can make a video, but it's not the same. they can tell us how to give a bath on a bed, but if you don't do it . . . " p5; "it is impossible to learn the practical part. until you are not in that role, it is impossible to learn" p24 the students are not able to propose other methodologies that are distinct from the ones offered. two students pointed out that it could be completed with gamification (kahoot): "gamification would be good, for example when calculating the dose" p19 within the methodologies, it was found that the least complex, for example, providing word, ppt or pdf documents, were related to the older teachers. the videoconferences and recorded classes were given by younger teachers in general. at the same time, they indicated that teachers from other non-nursing departments utilized the least complex methodologies: "it depends on the difficulty of the course. physiology has only uploaded documents" p16; for example, pharmacology is a very dense and complicated subject, and you need someone to explain it to you, and until now, we have not received anything, only notes. i don't think it's enough, they are too schematic and hard to understand". p1; "the younger ones (teachers) feel like doing more things" p19; "it is more difficult for some teacher, especially those who are older" p5 they pointed out that it is good in the videoconferences. an inconvenience is that sometimes the teacher is not aware of the doubts posted on the chat if there are too many messages. in the chat, the interaction is good, but the interruptions, even though they may be short, makes it impossible to follow it. lastly, the students are surprised about how fast the teachers answers the e-mails: "the chat, if you miss 5 min, you get lost" p22; "there is a good reception by the teacher for communicating" p11; "[tutoring} they are good, the answer sooner. they have improved" p21 the context of confinement has created some limitations for following e-learning education. these are related with internet access, access to electronic devices, and work and family responsibilities. in rural environments, situations exist where internet access is lacking, which creates problems with being up to date with the classes. another problem indicated is that not all had internet at home, and situations exist in which a person only has the limited amount of data available from a smartphone: "some people do not have all the means" p25; "i don't have internet at home, i only have data from the smartphone" p27 "i live here in the countryside, and the internet does not always work well, and if my kids are connected, then i can't do anything" p17 the confinement has obliged working from home whenever possible. this implies that it is possible for a family with three children to need an internet connection at the same time and the availability of five electronic devices simultaneously to be able to work and follow the classes. this availability is not very common. another limitation that was pointed out was working in the presence of children/siblings at home: "with the children at home, things cannot be done [mothers]" p25; "studying at home when the entire family is at home, it is very hard to concentrate sometimes, they make noise, i can't print, etc." p25 part of the students pointed out that is inconvenient, as they are used to studying in public libraries and have had to study at home: "i always study at the library, not at home" p19; "i used to go to the library to study or do homework. no one bothers me there. at home, i set the washer, put on my pajamas and go to sleep" p28 another difficulty added by the confinement is that one is not "trained" for shifting to e-learning. one has experience with an education system that has never been 100% online and where the traditional class is the learning stage. with respect to online exams, they do not feel secure either: "we are used to traditional classes. this has been difficult for everyone, and more for the bachelor students than the masters ones" p23; "i supposed they will give multiple-choice exams in a short time. it is the first time it will be online and one could be tense" p13; "if i hear it from the teacher beforehand, i understand it better, and now it's different. you take notes and then you have to understand them . . . " p16 3.6. face-to-face win 3.6.1. face-to-face is better . . . for everything the participants clearly preferred face-to-face to e-learning education. when faced with the possibility that some percentage of online classes will be provided along with traditional classes in future academic years, they do not think it is an option that will contribute much or needed. an exception is provided by students who have family or work responsibilities, who, exclusively for the theoretical classes, prefer them to be online and recorded, in order to be able to watch them at any time. another aspect that was underlined was that the traditional system of education is the one they know and are used to, and changing it is difficult: "face-to-face is better . . . for everything" p17; "the university of murcia is traditional, and we come from the same type of learning. it takes some time to adapt" p22; "face-to-face is better in every aspect. for example, you learn the lesson and the teacher can provide examples, it can go further than the powerpoint presentation. it is better to be face-to-face with the professor than through a screen" p23 the older students seem to be the most vulnerable group, and various problems are observed. on the one hand, they have to tend to their children now that they are all at home, they have more responsibilities at home, plus certain digital competences that they have yet to incorporate. the management of their time is a great problem, which is influenced by the use of time, space and the electronic devices by the rest of the family, to which they grant them priority without being aware: "for me the chat is not good, because i can't write that fast. i see the limitation in me. i miss the traditional classes. face-to-face classes are better . . . for everything" p17; "you have to be very alert with the online classroom, that you do not ignore the messages. yesterday there was a class, and did not know" p9; "some classmates are much older, and this is difficult. they write to the group [whatsapp] sending pictures, and asking "what do i do? where should i click?"" p25; "i'm much more in favor of face-to-face classes. i always obtain more information in them and i'm more comfortable" p24 it is necessary to underline that all the results and discussion are centered on the first month of confinement after the start of the state of alarm, and this brings with it very specific cognitive and social states that are needed for the proper understanding of the discussion of the present research study. although the sample included a greater number of students from the university of murcia as compared to those from the university of granada, and different percentages of men and women of different ages, we believe that the main sociodemographic variables were well represented through the use of maximum variation sampling. the nurses usually become nurses due to their desire to help other people to recover and maintain optimal health, and here we find ourselves in a situation in which not many options are available to help those who are severely sick due to covid-19 [5] . vocation is a determinant factor for those who decided to study nursing, and the main drive is the opportunity to care for others [17] . our results support these two ideas in two ways: (1) they indicate that this attitude towards their professional life is still true in the new generations, with the remarkable fact that all the participants are so committed and wishing to help. (2) the pandemic has positively re-enforced their wishes to become nurses, obtaining similar results as other authors [18] . although, the state of alarm decree includes the possible mobilization of students in their last year at university, their mobilization was principally needed in a small scale in madrid and catalonia, the areas greatly affected by covid-19 [19] . the fast shift to e-learning education has not ceased to be a continuation of teaching and education through online resources, although it has not been clearly planned and adapted for e-learning [10] . our results clearly present various relevant ideas related to this. in first place, and related with the clinical training, the health science degrees and more specifically the nursing degree have an essential need to be developed in clinical context. this element clearly cannot be substituted, and is perceived by all the students as being essential. nevertheless, at present, a discussion exists about how high-fidelity clinical simulation could substitute the clinical training in real-world environments [20, 21] . this methodology, which facilitates an intermediate learning between the theoretical dimension and the practical dimension, is proposed, aspiring to construct a real environment. however, and despite it being a type of learning established and known by the student body at the university of murcia as well as the granada, it is striking that this type of learning has not been described as an alternative. we interpret this finding as the clinical training being indispensable for the students. also, it forces us to reflect if this is the new reality of health care, and if the future nursing professionals should learn how to navigate in these conditions. the debate regarding the return of the nursing students to clinical environments is open and some recommendation has been provided [22] . the question now for the universities and nursing educators is that if as soon as the resources are provided and an adequate organization and adaptation occurs, the students should return to the health care services, what is the balance between the potential risk for the students and the importance of the clinical training? in second place, and related with the teaching of theory, the students prefer face-to-face teaching as opposed to the e-learning. they believe that the interaction is higher in quality and learning is greater. at present, another debate is open, as shown by two systematic reviews that do not provide concluding results on the existence of the greater learning linked to e-learning education of health professionals and students, highlighting the poor quality of existing studies and the importance of contextual factors [23, 24] . perhaps due to these reasons, the videoconference, distance learning, but synchronous and bi-directional, is the best assessed. another critical aspect is that the change to the online methodology was not chosen by the students, and the expectations they have with respect to their studies have been clearly disrupted. their entire academic life has been marked by a specific style of teaching, and they have become organized to continue with it, but the pandemic has imposed a different one with which they do not feel comfortable yet, thereby creating uncertainty and little security. this worries a great part of the health science academics [25] . it therefore absolutely necessary to start to work on the adaptation to e-learning that takes into account the previously-mentioned aspects so that the student's uncertainty decreases, especially in light of the evaluations. academics have already expressed awareness of the students' concerns that are centered on their future degree and career progression [25] . the university counted with a technological infrastructure that has been able to deal with a drastic and fast change to distance teaching. however, the urgency of adapting this type of teaching has highlighted some situations of disadvantage. thus, the older students, as compared to the younger ones, and in great part women and mothers, do not possess the most basic digital competences. this finding is robust, as the older students themselves, as well as the younger ones, are able to point this out in agreement with each other. they also point out that there is a small percentage of students who do not have the electronic resources or a connection to the internet necessary for adequately following the teaching. universities are trying to provide answers to some of these problems. it could be said that the phases of shock and normalization described by the students coincided with the period of reaction and acts of implementation by the institutions. there are activities that allow for fast implementation. for example, the universities of murcia and granada freely loaned laptop computers with software to 100% of the students who requested them [26] , with this number being more than 300 students in murcia alone, as well as mobile internet-access devices [27] . however, the implementation of activities related with the evaluation has required conscientious reflection and consensus that has forced their implementation later in time [11] . in any case, once this first stage has been overcome, and faced with the absence of permanent solutions for this pandemic in the short term, it is necessary to propose distance learning strategies with a robust design, with the time necessary to create study plans that are well thought-out and durable [10, 28] . we should be aware that we are currently undergoing an "emergency" education, a temporary shift of instructional delivery to an alternate delivery mode due to crisis circumstances [29] . the reality is that this transition to e-learning under these circumstances has nothing to do with a design that takes the maximum advantage and possibilities of the online format. we should reflect on the differences in the rhythm, the student-instructor relationship, pedagogy, the role of the instructor, the role of the student, the synchronicity of online communication, the role of online evaluations, and the source of feedback [30] . among the limitations of the study, we find that a thorough discussion and comparison with the opinions of other authors has not been possible, given the novel and exceptional situation we are currently living in. on the other hand, we should be aware that the sample studied cannot be representative of the reference population, and this can evidently affect the generalization of the results. after the first week of adaptation to the conditions of confinement and the establishment of new online teaching systems, the students begin a new normality. the imposition of e-learning brings more limitations to students who are older, with work and family responsibilities, living in a rural environment and with limited electronic resources. online teaching has allowed substituting the teaching of theory, although face-to-face teaching is preferred, at the same time it has shown that clinical practices are indispensable for the training of the nursing students. online education goes beyond the online continuation of the classes. the parties responsible should already be working on this for the next academic year, in light of the uncertain future of a short-term control of covid-19. acknowledgments: our most sincere thanks to the people who agreed to participate in this research. the authors declare no conflict of interest. 4th year students -aa1 what is going to happen? -aa2 what are you afraid of? -aa3 what do you think will happen with the practice? -aa4 what do you think will happen to your degree? -aa5 if you miss a practicum how good is your training? life expectancy -how does the situation caused by the crisis in the professional field affect your desire to become a nurse? world health organization. who director-general's opening remarks at the media briefing on covid-19-11 covid-19 and italy: what next? italian public health response to the covid-19 pandemic: case report from the field, insights and challenges for the department of prevention public health emergency and crisis management: case study of sars-cov-2 outbreak life in the pandemic: some reflections on nursing in the context of covid-19 covid-19: a potential public health problem for homeless populations impact of the burden of covid-19 in italy: results of disability-adjusted life years (dalys) and productivity loss unesco & iesalc covid-19 and higher education: today and tomorrow teaching in times of pandemic nursing education after covid-19: same or different? using thematic analysis in psychology qualitative evaluation and research methods the significance of saturation consolidated criteria for reporting qualitative research (coreq): a 32-item checklist for interviews and focus groups thematic analysis of qualitative data: amee guide no. 131 why did i become a nurse? personality traits and reasons for entering nursing covid-19 and student nurses: a view from england nursing students are already risking their lives against covid-19 with precarious contracts [las estudiantes de enfermería ya se juegan la vida contra el covid-19 con contratos precarios suzie) emerging evidence toward a 2:1 clinical to simulation ratio: a study comparing the traditional clinical and simulation settings simulation in nursing education: current regulations and practices pre-registration undergraduate nurses and the covid-19 pandemic: students or workers? efficacy of adaptive e-learning for health professionals and students: a systematic review and meta-analysis conventional vs. e-learning in nursing education: a systematic review and meta-analysis forced disruption of anatomy education in australia and new zealand: an acute response to the covid-19 pandemic university of granada 1,000 internet connection lines for students who have difficulty connecting líneas de conexión a internet para los estudiantes con dificultades para conectarse what value nursing knowledge in a time of crisis? the difference between emergency remote teaching and online learning what research tells us about whether, when and how key: cord-316047-d9cpe9yl authors: gonzalez, t.; de la rubia, m. a.; hincz, k. p.; comas-lopez, m.; subirats, laia; fort, santi; sacha, g. m. title: influence of covid-19 confinement on students’ performance in higher education date: 2020-10-09 journal: plos one doi: 10.1371/journal.pone.0239490 sha: doc_id: 316047 cord_uid: d9cpe9yl this study analyzes the effects of covid-19 confinement on the autonomous learning performance of students in higher education. using a field experiment with 458 students from three different subjects at universidad autónoma de madrid (spain), we study the differences in assessments by dividing students into two groups. the first group (control) corresponds to academic years 2017/2018 and 2018/2019. the second group (experimental) corresponds to students from 2019/2020, which is the group of students that had their face-to-face activities interrupted because of the confinement. the results show that there is a significant positive effect of the covid-19 confinement on students’ performance. this effect is also significant in activities that did not change their format when performed after the confinement. we find that this effect is significant both in subjects that increased the number of assessment activities and subjects that did not change the student workload. additionally, an analysis of students’ learning strategies before confinement shows that students did not study on a continuous basis. based on these results, we conclude that covid-19 confinement changed students’ learning strategies to a more continuous habit, improving their efficiency. for these reasons, better scores in students’ assessment are expected due to covid-19 confinement that can be explained by an improvement in their learning performance. the coronavirus covid-19 outbreak disrupted life around the globe in 2020. as in any other sector, the covid-19 pandemic affected education in many ways. government actions have followed a common goal of reducing the spread of coronavirus by introducing measures limiting social contact. many countries suspended face-to-face teaching and exams as well as placing restrictions on immigration affecting erasmus students [1] . where possible, traditional classes are being replaced with books and materials taken from school. various e-learning platforms enable interaction between teachers and students, and, in some cases, national television shows or social media platforms are being used for education. some education systems announced exceptional holidays to better prepare for this distance-learning scenario. a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 in terms of the impact of the covid-19 pandemic on different countries' education systems many differences exist. this lack of homogeneity is caused by such factors as the start and end dates of academic years and the timing of school holidays. while some countries suspended in-person classes from march/april until further notice, others were less restrictive, and universities were only advised to reduce face-to-face teaching and replace it with online solutions wherever practicable. in other cases, depending on the academic calendar, it was possible to postpone the start of the summer semester [2] . fortunately, there is a range of modern tools available to face the challenge of distance learning imposed by the covid-19 pandemic [3] . using these tools, the modification of contents that were previously taught face-to-face is easily conceivable. there are however other important tasks in the learning process, such as assessment or autonomous learning, that can still be challenging without the direct supervision of teachers. all these arguments end in a common topic: how to ensure the assessment's adequacy to correctly measure students' progress. thus, how can teachers compare students' results if they differ from previous years? on one hand, if students achieve higher scores than in previous years, this could be linked with cheating in online exams or with changes in the format of the evaluation tools. on the other hand, lower grades could also be caused by the evaluation format change or be attributable to autonomous learning as a less effective teaching method. the objective of this article is to reduce the uncertainty in the assessment process in higher education during the covid-19 pandemic. to achieve this goal, we analyze students' learning strategies before and after confinement. altogether, our data indicates that autonomous learning in this scenario has increased students' performance and higher scores should be expected. we also discuss the reasons underneath this effect. we present a study that involves more than 450 students enrolled in 3 subjects from different degrees from the universidad autónoma de madrid (spain) during three academic years, including data obtained in the 2019/2020 academic year, when the restrictions due to the covid-19 pandemic have been in force. e-learning has experienced significant change due to the exponential growth of the internet and information technology [4] . new e-learning platforms are being developed for tutors to facilitate assessments and for learners to participate in lectures [4, 5] . both assessment processes and self-evaluation have been proven to benefit from technological advancement. even courses that solely offer online contents such as massive open online courses (moocs) [6, 7] have also become popular. the inclusion of e-learning tools in higher education implies that a greater amount of information can be analyzed, improving teaching quality [8] [9] [10] . in recent years, many studies have been performed analyzing the advantages and challenges of massive data analysis in higher education [11] . for example, a study of gasevic et al. [12] indicates that time management tactics had significant correlations with academic performance. jovanovic et al also demonstrated that assisting students in their management of learning resources is critical for a correct management of their learning strategies in terms of regularity [13] . within few days, the covid-19 pandemic enhanced the role of remote working, e-learning, video streaming, etc. on a broad scale [14] . in [15] , we can see that the most popular remote collaboration tools are private chat messages, followed by two-participant-calls, multiperson-meetings, and team chat messages. in addition, several recommendations to help teachers in the process of online instruction have appeared [16] . furthermore, mobile learning has become an alternative suitable for some students with fewer technological resources. regarding the feedback of e-classes given by students, some studies [17] point out that students were satisfied with the teacher's way of delivering the lecture and that the main problem was poor internet connection. related to autonomous learning, many studies have been performed regarding the concept of self-regulated learning (srl), in which students are active and responsible for their own learning process [18, 19] as well as being knowledgeable, self-aware and able to select their own approach to learning [20, 21] . some studies indicated that srl significantly affected students' academic achievement and learning performance [22] [23] [24] . researchers indicated that students with strongly developed srl skills were more likely to be successful both in classrooms [25] and online learning [26] . these studies and the development of adequate tools for evaluation and self-evaluation of learners have become especially necessary in the covid-19 pandemic in order to guarantee good performance in e-learning environments [27] . linear tests, which require all students to take the same assessment in terms of the number and order of items during a test session, are among the most common tools used in computerbased testing. computer adaptive test (cat), based on item response theory, was formally proposed by lord in 1980 [28] [29] [30] , as is the case with linear testing. some platforms couple the advantages of cat-specific feedback with multistage adaptive testing [38] . the use of cat is also increasingly being promoted in clinical practice to improve patient quality of life. over the decades, different systems and approaches based on cat have been used in the educational space to enhance the learning process [39, 40] . considering the usage of cat as a learning tool, establishing the knowledge of the learner is crucial for personalizing subsequent question difficulty. cat does have some negative aspects such as continued test item exposure, which allows learners to memorize the test answers and share them with their peers [41, 42]. as a solution to limit test item exposure, a large question bank has been suggested. this solution is unfeasible in most cases, since most of the cat models already require more items than comparable linear testing [43]. the aim of this study is to identify the effect of covid-19 confinement on students' performance. this main objective leads to the first hypothesis of this study which can be formulated as h1: covid-19 confinement has a significant effect on students' performance. the confirmation of this hypothesis should be done discarding any potential side effects such as students cheating in their assessment process related to remote learning. moreover, a further analysis should be done to investigate which factors of covid-19 confinement are responsible for the change. a second hypothesis is h2: covid-19 confinement has a significant effect on the assessment process. the aim of the project was therefore to investigate the following questions: 1. is there any effect (positive or negative) of the covid-19 confinement on students' performance? 2. is it possible to be sure that the covid-19 confinement is the origin of the different performance (if any)? 3. what are the reasons for the differences (if any) in students' performance? 4. what are the expected effects of the differences in students' performance (if any) in the assessment process? we have used two online platforms. the first one is e-valuam [44] , an online platform that aims to increase the quality of tests by improving the objectivity, robustness, security and relevance of assessment content. e-valuam implements all the cat tests described in the following sections. the second online platform used in this study is the moodle platform provided by the biochemistry department from universidad autónoma de madrid, where all the tests that do not use adaptive questions are implemented. adaptive tests have been used in the subjects "applied computing" and "design of water treatment facilities". traditional tests have been used in the subject "metabolism". 2.1.1 cat theoretical model. let us consider a test composed by n q items. in the most general form, the normalized grade s j obtained by a student in the j-attempt will be a function of the weights of all the questions α and the normalized scores ψ (s j = s j (α, φ)), and can be defined as: where the φ i is defined as where δ is the kronecker delta, a i the correct answer and r i the student's answer to the i-question. by using this definition, we limit φ i to only two possible values: 1 and 0; φ i = 1 when the student's answer is correct and φ i = 0 when the student gives a wrong value. this definition is valid for both open answer and multiple-choice tests. in the case of multiple-choice test with n r possible answers, φ i can be reduced to consider the random effect. in this case: independently of using eqs 2 or 3, to be sure that s j (α, φ) is normalized (i.e. 0< = s j (α, φ)< = 1), we must impose the following additional condition on α: in the context of needing a final grade (fg) between 0 and a certain value m, which typically takes values such as 10 or 100, we just need to rescale the s j (α, φ) value obtained in our model by a factor k, i.e. fg j = k s j (α, φ). we will now include the option of having questions with an additional parameter l, which will be related to the level of relevance of the question. l is a number that we will assign to all the questions included in the repository of the test (i.e. the pool of questions from where the questions of a j-test will be selected). the concept of relevance can take different significances depending on the context and the opinion of the teachers. in our model, the questions with lower l values will be shown initially to the students, when the students answer correctly a certain number of questions with the lower l value, the system starts proposing questions from the next l value. by defining n l as the number of possible l values, the l value that must be obtained in the k-question of the j-test can be defined as: where trunc means the truncation of the value between brackets. it is worth noting that l k is proportional to the sum of the student's answers to all the previous questions in the test. this fact means that, in our model, the l k depends on the full history of answers given by the student. l k is inversely proportional to n q , which means that it takes a higher number of correct answers to increase l k . once l k is defined, a randomly selected question is shown to the student. another important fact that implies the use of eq 5 in the adaptive test is that we will never have l k <l k-1 . in other words, once a student starts answering questions from a certain l value, they will never go back to the previous one. in fig 1 we show a simple example of the multiple possible paths that a student can follow when facing a test that uses our model. in the example shown in fig 1 we have used n q = 6 and n l = 3. in this very straightforward example, the l value changes every time a student gives 2 correct answers (following diagonal arrows on the diagram). wrong answers imply a vertical drop on the diagram. in this case, the l value will not change (l k = l k-1 ). the final student's grade g is shown on the lower part of the diagram. the lowest grade (g = 0) corresponds to a test where the student has failed all the n q questions. in this case, l = 1 for all the possible j values. the highest grade (g = 6) implies that the student has faced questions from all the possible l values (l = {1,2,3} in fig 1) . ma-ts are used for evaluation and self-evaluation of theoretical contents. in this case, we use a standard format where a single correct answer must be chosen from a pool of possible answers shown by the system. since students can sometimes answer correctly by a random selection, eq (3) must be used when evaluating the score of any item. the format of the ma-t in our cat method requires the following elements: in addition, the e-valuam platform, where the method proposed in this article has been implemented, allows optional use of images or sounds for both the statement and the answers. e-valuam shows the statement and the possible answers. all other information relating to the cat method such as the level of the item is hidden from the student. the interface also shows optional feedback information about the performance of the learner in the previous item. finally, it shows the time remaining to finish the test. to train the contents of numerical problems, an oa-t was developed in which the statements include at least one parameter that will change its value with each execution of the application. this kind of question requires the following elements to be created: • statement with explicit indication of the modifiable parameter(s). • minimum and maximum values of each modifiable parameter. • programming code (matlab in e-valuam) that calculates the solution to the problem. • level of the item. as with ma-t, there is also a possibility of including multimedia files in the item. however, in this type of question, the multimedia option is only available in the question as the answer is numerical and included by the user in all cases. in this case the interface has a space for entering the numerical answer. in this figure, we also show the feedback of the application when an incorrect previous answer has been introduced. traditional tests have been used in the subject "metabolism" from the human nutrition and dietetics degree. the course contents are divided into 6 parts corresponding to different aspects of human metabolism. each part is taught with face-to-face lectures followed by different on-line activities that the students perform in the moodle platform and that are later discussed between them and with the professor in face-to-face workshops. all 6 parts of the course have a similar format, being a lecture followed by a selection of online activities in this order: "exercises workshop", "discussion workshop", "self-assessment" and "test". except for the discussion workshops outlined below, all activities are on-line questionnaires (closed, multiple-choice questions) that the students must perform in moodle, achieving a score that is automatically calculated and showed to them as part of their continuous assessment. exercises workshops and self-assessments are completed by the students without tutor supervision, whereas the tests are done on-line but under controlled conditions, just in the same way as a regular examination. each student should perform a total of 15 online activities: 5 exercises workshops, 6 self-assessment activities and 4 tests. discussion workshops are practical cases that must be debated by the students and sent to the tutor on moodle. as these activities do not obtain an automatic score but require a correction by the professor, they have not been considered in this study. the control group of our study is formed by the students of "applied computing" and "metabolism" from academic years 2017/2018 and 2018/2019 and by students of "design of water treatment facilities" from academic year 2017/2018. in the case of "design of water treatment facilities", a longitudinal study has been performed in academic year 2017/2018 to analyse the effect of rewards in the students' learning strategies, especially those related to time management. the experimental group of our study are students of "applied computing" and "metabolism" from academic year 2019/2020. in the longitudinal study of "design of water treatment facilities", experimental group corresponds to the third stage of the study. to answer our research questions, we first set up an experiment that obtains accurate measurements of the autonomous learning activities both in the control and the experimental groups. high accuracy in the measurements of the autonomous learning activities is achieved by using the previously described adaptive tools both in learning and assessment. in our experiment, the evaluation format and the e-learning tools are known to the students from the start, which motivates them to use the tools and perform the tests more consistently. in fig 2 we show the procedure used in our experiments to measure autonomous learning. there are two kinds of contents in the subjects included in the experiment: theory and numerical problems. in the case of theory, students can use the e-valuam platform through ma-t. the final evaluation is performed by oa-t. in the case of numerical problems, both learning and evaluation use oa-t. in this case, student motivation for using the platform is even higher because their final scores will be obtained by the same tool they use for their autonomous learning. this system is applied in all the experiments related to analysing the autonomous learning activity. in the case of "design of water treatment facilities", an additional longitudinal study has been performed and is shown in fig 2. this study has been performed in three stages. in the first stage, the self-evaluation material was presented to the students right before the evaluation test. in the second stage, the self-evaluation material was available 3 weeks before. to manage the theory contents, a ma-t with three possible answers to each question and only one correct has been created. in the first stage, with the test made available only one day before the exam, students had at their disposal a ma-t composed of 16 questions and with a time limit of 15 minutes. in the second stage, the time available was increased to 30 minutes and the question number to 18. these changes are related to the nature of the subject contents and do not have a significant impact on the study. the oa-t tests had 6 and 9 questions in the first and second stage, respectively. in order to maintain consistency in the subsequent study, an additional effort has been made to keep the difficulty of the questions as similar as possible. stage 3 had no new material added, instead, it used all the material created in stages 1 and 2. stage 3 corresponded to the time window between the end of classes and the last exam. in this stage, all the material was available from the beginning. the protocol used in this stage only varied from stage 2 in the inclusion of a reward for students who used the application on 3 or more different days. the reward was related to a bonus in the final grades. once the correct measure of the autonomous learning is ensured, as explained in the previous section, we must develop an experiment to describe the effect of covid-19 confinement. this experiment compares results of control and experimental groups in the whole assessment process. there are two stages that must be considered to determine the effect of the covid-19 confinement. the first one corresponds to the period without confinement in the year 2019/2020 (before march 11), when all the measurable activities are performed in similar conditions for experimental and control groups. the second stage corresponds to the period of covid-19 confinement (after march 11), where some measurable activities were performed in a different format and statistical differences can be found by comparing experimental and control groups. in the case of "applied computing", students use the e-valuam application on a continuous basis as it is available from the beginning of the course. they use the same oa-t for training and for their final exam, ensuring often and consistent use of the platform throughout the course. for this reason, in "applied computing", data analysis is performed over a continuous self-evaluation process over a single test along the whole academic year. the test performed by the students did not change its format or available questions in the whole three years under study. in the case of "metabolism", the confinement started just before the beginning of part iv of the subject. therefore, in this second stage, the scheduled in-person lectures from part iv were recorded on video by the tutor and provided to the students. afterwards, the students had an opportunity to perform the corresponding on-line continuous assessment activities, just in the same way as they would have done under normal conditions (as these activities were always not face-to-face). as this subject has already had an important number of on-line activities programmed prior to the suspension of face-to-face teaching, it is an extremely valuable tool for analysing the effect of the confinement on students' performance. in fig 3, we show the full experiment described in this section. first, the two studies performed in years prior to covid-19 confinement. in these first studies, we focus on the learning strategies of the students and their response to reward scenarios. information from these first studies will be useful for later discussion of the reasons for the changes caused by the covid-19 confinement. the second part of our experiment is related to the effect of covid-19 confinement, where we compare results of similar assessment activities between control and experimental groups. in fig 3 we include information about all control and experimental groups in the different interventions. this work involves data that has been collected from 458 students of the universidad autónoma de madrid. this data has been collected in the following manner: 1. it does not involve minors. 2. it has been collected anonymously. students have been identified by a numerical code, avoiding gathering of any personal information. 3. students have been informed by the lecturers that some information about their activity could be anonymously collected for statistical purposes. authors of this study did not receive any objections. 4. the tasks related to this study were completely voluntary and they did not in any form alter students' activities, classes, or the assessment process. considering these circumstances, we have received confirmation from the research ethics committee of universidad autónoma de madrid that we do not need to apply for ethics approval from our university since no personal data, minors or potentially hazardous activities were involved in the study. teachers involved in the study (coauthors of the present manuscript) who were responsible for the subjects taught also gave consent to carry out the study. we obtained verbal consent from the participants in the study. experiments have been performed in the subjects "applied computing", "metabolism" and "design of water treatment facilities". 2.3.1 applied computing. this study with real students took place during the 2017/2018, 2018/2019 and 2019/2020 academic years in the first-year subject "applied computing". 97, 73 and 91 students were involved each academic year, respectively. this subject was taught through theory lessons and practical classes in the computer laboratory. this course corresponds to 6 ects and belongs to the chemical engineering degree in the faculty of sciences from universidad autónoma de madrid, spain. due to covid-19 pandemic, the face-to-face teaching was cancelled on march 11, having a strong impact on the 2019/2020 course since it started in the first week of february and lasted until mid-may. the data for this study has been obtained from real students enrolled on the "metabolism" course in the human nutrition and dietetics degree from universidad autónoma de madrid, spain. this is a 6 ects compulsory subject taught during the second semester of the first year of the degree. the study comprises data from 2017/2018 and 2018/ 2019 academic years, with 64 and 63 enrolled students, respectively, and from the present 2019/2020 academic year (47 students), which has been strongly affected by the restrictions caused by the covid-19 pandemic. this year, the course started on january 28 and finished on april 30. face-to-face teaching cancellation and beginning of confinement occurred approximately halfway through the semester. the experiment with real students took place during the 2017/2018 academic year. 23 students were involved in the optional fourth-year subject "design of water treatment facilities". this subject was taught through theoretical and practical classes in the classroom and laboratory, respectively. our study focused on the theoretical part of the subject. this course corresponds to 6 ects and belongs to the chemical engineering degree in the faculty of sciences of the universidad autónoma de madrid, spain. 2.4.1 applied computing. in this subject we have expressed student performance as a 0-10 score and analyzed it across all tests performed in self-evaluation. those oa-t format tests were available from the beginning of the academic year and students used them on a continuous basis. data from this subject imply results from an adaptive test. this imply that there are some points in the test where students get stacked and some scores are repeated more often than others. we tested if data followed a gaussian distribution using a d'agostino and pearson omnibus normality test [45] . as the data did not pass the normality test, we used nonparametric statistical tests to compare the data from all 3 academic years. means were compared by a kruskal-wallis test [46] and, when differences were found, by a mann-whitney post hoc test [47] to determine which pairs of means were different. all statistical analysis was performed using graphpad prism 6 (graphpad software, la jolla, california, usa). data is presented as mean±sd and statistical significance was set at p<0.05. here, we have analysed 2 variables, the score (0-10) obtained by the students in the different activities and the proportion of students that passed the activity (score�5), from 2019/2020 and the 2 previous academic years. firstly, we checked if the results from the previous 2 years (2017/2018 and 2018/2019) were similar, thus allowing them to be compared with results from the present year. to perform the analysis, we tested the score data for normality using a d'agostino and pearson omnibus normality test [41], later comparing them with an unpaired 2-tailed t-test, if they were normal, or with a non-parametric mann-whitney test, if they were not normal. afterwards, we proceeded similarly to compare the results from the present year with those from the previous 2 years. when data from each activity followed a normal distribution, we compared the 3 means by 1-way anova followed by an unpaired 2-tailed t-test. when the data did not follow a normal distribution, we compared the 3 means with a kruskal-wallis test followed by a mann-whitney post hoc test. in order to compare the proportion of students that passed the activities (score�5) in different years, we performed a z-test (after confirming that our data met the central limit theorem). all statistical analysis was performed using graphpad prism 6. data is presented as mean±sd and statistical significance was set at p<0.05. in order to substantiate related samples of results in our experiment, we performed a wilcoxon signed rank test which compares two related samples that cannot be assumed to be normally distributed. in the experiments related to this subject, the same students were involved in both samples. data is presented as mean ±sd and statistical significance was set at p<0.05. in this section, we analyze students' self-learning strategies in the subject "applied computing" in the years that were not influenced by covid-19 (2017/2018 and 2018/2019). however, since the data is very similar in both courses, we will focus con 2017/2018 for clarity. all statistical results can be also applied to 2018/2019. in both years, theoretical lessons were taught in the classroom and practical lessons in computer laboratories with the teacher physically present. practical lessons always used the e-valuam platform and the adaptive test described previously. in fig 4a, we show the scores obtained by all 2017/2018 students in all the attempts made with adaptive tests. attempts are ordered chronologically. a score of -1 in the figure implies that the student did not finish the test and did not obtain a numerical score. those tests have been included in the figure as a part of student's autonomous work and must be considered as learning time; however, we will exclude them from score data analysis. there are some patterns in the figure that must be considered. first, certain scores repeat more than others (2.5, 5 and 7.5), which corresponds to a jump in question level (l) (level 1: 0-2.5, level 2: 2.5-5, level 3: 5-7.5 and, level 4: 7.5-10), meaning that students were not yet able to answer more difficult questions. another pattern that can be easily seen in the figure is a region where a huge number of tests had a score of -1 (between 600 and 700 approximately). on those days teachers asked the students to find the most difficult questions in the test. for that reason, students interrupted the tests when they reached those questions. however, the most interesting pattern, which can be observed in fig 4a, is the distribution of the tests in time, with a much bigger number of tests done in may, being the last month of the academic year for this subject. students used the platform 688 times in total in may and only 486 times in the period between february and april. it is worth noting that the academic year for this subject ended with the final exam on may 21. in fig 4b we show the distribution of attempts between february and may. we found 62 in february, which is a low but reasonable number since students did not yet have enough knowledge to face tests properly. they generally started using the platform in the middle of the month. in march, we noted an increase in the number of attempts followed by a slight decrease in april. this difference can be justified by the easter holidays (one week) in april. the number of attempts strongly increased in may. even when counting only the first two weeks of the month, we have more attempts than in previous months (292 from may 1-15). in the period of may 16-21, we have recorded 396 attempts (469 if we count the attempts in the final exam). in the inset of fig 4b we show the attempt distribution in the last period (may 16-21). as we can see, students used the platform 127 times the day before the exam and 173 times the day of the final exam (95 for studying and 78 in the exam itself). more than 33% of tests were performed in the last 6 days before the final exam (and more that 50% of those tests were performed the day before the exam). to study the effects of a rewarded scenario, we have developed an experiment in the academic year 2017/2018 for students from the subject "design of water treatment facilities". we have used 3 stages (fully described in the materials and methods section): stage 1, where the self-evaluation tests have been made available for 1 day; stage 2, where the self-evaluation tests have been made available for 3 weeks; and stage 3, where self-evaluation tests were available for a longer period with a reward related to their use. fig 5 shows the attempts distribution in all the stages. focusing on the ma-t, 13/20 students made more attempts in the first stage than in the second, which is striking considering that in the second stage application was available for longer. three students have not made any attempts in either of the 2 stages. as for the group of students who have used the application for more than one day, we noted only 3 individuals working for 3 days or more. considering that the application was available for 3 weeks in this second stage, these results show the very low willingness of the students to work continuously. focusing on the oa-t, this effect is even more significant, as only two cases in which students used the application for more than one day were recorded. increasing student motivation and analyzing the results of a more persistent use of the tool was the objective of stage 3. in fig 5 we can see the distribution of attempts made by the students in the whole study (3 stages) . there is a fourth stage in the figure that corresponds to an additional exam taken by the students that did not pass the previous ones. since only a few of them were involved in this fourth stage, it was excluded from the analysis. as we can see in the figure, attempts from stage 1 were limited to a couple of days, which corresponds to the short period when the test was available. some attempts can be found in a wider window in stage 2. however, a much higher number of attempts extended in time can be found in stage 3, which corresponds to the rewarded stage. in order to measure the use of the tool over time, we summarized the use of the system by students in stage 2 and 3 by comparing the number of days the students attempted the stage 2 ma-t and oa-t tests during the 3 weeks leading up to the exams given after each stage. it is interesting to note that, during stage 3, six more students utilized the system than in stage 2; however, the total attempts reduced between the 2 stages from 123 to 102. this translates to a 23% increase in the average use of the application over time from 1.9 days in stage 2 to 2.4 days in stage 3 combined with a reduction of attempts per student from 9.5 in stage 2, to 5.4 in stage 3. in the earlier stage, fewer students used the application intensively by making multiple attempts a day. students who logged in at the later stage were more likely to use the application once or twice a day over several days because of the reward related to this behavior. out of a total sample of 19 students who used the tool in stage 3, 5 were eliminated by the test as they accessed the tests the same number of days in both stages. then, for the use of the application by the remaining 14 students the w-value was of 15.5, with the critical value of w = 25 at p.0.05. therefore, the result is significant. we have studied two cases related to two different and common scenarios in covid-19 distance learning. the first one is related to subjects where teachers increased the number of tasks that students should perform autonomously. the second one is related to subjects where all the autonomous tasks were the same as in previous courses. in this case, only face-to-face sessions were replaced by distance learning activities such as online sessions or recorded classes. 3.2.1 additional autonomous activities during confinement. in fig 6 we show the scores obtained by all the students in the 3 courses analysed in all the e-valuam tests performed in the subject "applied computing". in the figure we indicate the period of the covid-19 confinement during the 2019/2020 course (from march 11 to march 30). for clarity, we also indicate the equivalent period in the two previous years. as we can see in the figure, the number of tests performed before confinement was similar (225 and 246) in the two previous years (control group), but there is a remarkable increment in the number of tests (328) performed in the experimental group. these differences can be easily explained normalizing these numbers to the number of students each year (97, 73 and 91 respectively). by doing that, we obtain 2.31, 3.37 and 3.6 test/student, respectively. thus, differences between courses 2018/ 2019 and 2019/2020 are reduced, although there is still some difference with respect students from course 2017/2018, that can be explained by the fact that some of these students were retaking the subject and, therefore, did not use the e-valuam application often until the contents were more difficult. if we count the number of tests finished until the end of the academic year in courses 2017/2018 and 2018/2019, we find 1345 and 1175 tests respectively. normalizing with the number of students, we find 13.87 and 16.1 test/student respectively. this is a difference of 2.23 test/student in a period of 3.5 months, which means a difference of less than 1 test/student each month. the conditions after the covid-19 confinement radically changed in the course 2019/ 2020 because students were required to perform a higher number of tests each week. as we can see in fig 6, there are still small differences between courses 2017/2018 and 2018/2019. however, the number of tests taken in 2019/2020 is almost 5 times higher. in the inset of fig 6 we show a histogram with the data for course 2019/2020. adaptive tests used in this experiment induce the effect of higher number of attempts at scores 2.5, 5 and 7.5, which are the points where the level of the questions increases. for that reason, the curve is not normal. this fact is statistically tested in the following analysis. when comparing the mean scores from the period before confinement (fig 7) , we found that they were statistically different between the 3 academic years. differences between the mean score from the 2019/2020 (experimental group) and both the other 2 years (control group) were . the mean scores are significantly different both in the control and in the experimental groups between both periods (before and after confinement). we have also seen an increase in the mean score differences after the confinement. this study has been performed with students from the subject "metabolism", which is a subject where no additional tasks have been imposed to students because of the covid-19 confinement. we have analysed students' scores and the proportion of students that passed the activity (score�5). as it can be observed in fig 8a, the scores obtained by students in the different activities during the course were consistently similar in both years of the control group, with the only exception in the second activity where there was a slightly decrease in year 2018/2019 (p<0.05). similarly, the proportion of students with a score�5 was very similar in the control group (p>0.05) ( fig 8b) . therefore, students' performance in the previous 2 years (control group) was not significantly different, allowing us to compare these results to the results from 2019/2020. scores obtained by the students in the 2019/2020 academic year before confinement were similar to those obtained by students from the control group, although some differences were found at the beginning of the course (activities number 2, 3 and 4, p<0.05). we think that this could be due to the adaptation of the students to the course and to the new assessment methodology, being afterwards similar again. however, after the end of face-to-face teaching and at the beginning of the confinement, students' scores were significantly higher than in the previous academic years (fig 8a) . the most remarkable differences are evident in activities 10 and 11, that have always been on-line activities. thus, in activity 10, mean score in the present year (2019/2020) was 8.1±0.2 vs. 6.5±0.2 (p<0.0001) and vs. 6.7±0.3 (p = 0.0005) in 2017/2018 and https://doi.org/10.1371/journal.pone.0239490.g008 significant differences when comparing students' performance in confinement with the performance in previous periods where activities were not limited to distant learning. at this point, it is clear that the variable that correlates with the change in students' performance is the beginning of the confinement. however, we cannot yet establish if the difference is due to either: • the new learning methodology, or • the new assessment process. for those reasons, we should now answer research question 2 and establish whether the differences found when answering research question 1 are only due to the covid-19 confinement. the problem with confinement is that not only the learning, and teaching strategies should be modified, but also the assessment process as it cannot be done face-to-face. there are some concerns related to these new assessment methods such as the opportunity for cheating by the students. this is the reason why we have chosen only subjects that include several tests that have not been modified because of the confinement. the results of our study show that students also achieved significant improvements in their scores even in tests that were performed in the on-line format in previous years. moreover, this improvement is only significant when comparing data after the confinement (i.e. there are no significant differences in on-line tests that were performed before the confinement). these findings reveal that the new assessment process cannot be the reason for the improvement in students' performance because the learners also achieved better scores when the format of the assessment did not change. for these reasons, we establish that the new learning methodology is the main reason for the change in students' performance during the confinement. now, we shall discuss research question 3 (what are the reasons of the differences (if any) in students' performance?). we have proven that something has changed in students' learning methodology. the question is: what is the common element in those methodologies that caused the improvement in the learning process? we have analyzed data from two different subjects that used two very different learning strategies during the confinement (section 3.2). in one of them (section 3.2.1), additional elearning tasks were imposed on the students. theoretical lessons were replaced by written documents. in the other one (section 3.2.2), no additional e-learning tasks were given, but theoretical lessons were replaced by multimedia classes. in both cases, we have found a significant increment in students' performance in the evaluation tests after the confinement. it seems that students' performance is increased independently of the learning strategies followed by teachers. since we have established that the assessment process cannot be responsible for the differences, and we cannot find any common elements in the learning methodologies, we must conclude on a general change in the autonomous learning process. we have also analyzed data from the previous years in two subjects that demonstrate that students do not work on a continuous basis (section 3.1). in both subjects, students work hard only in the last days before the final exams. in one subject, we have found that more than 33% of the autonomous work was done in the 5 days immediately preceding the final exam (section 3.1.1). in the other subject, students only used the e-learning material in the last 2 days before the final exam, even when it had been provided three weeks in advance (section 3.1.2). in this study, we have also found that students easily change their learning behavior and study continuously when a reward is offered. this extra motivation dramatically changed their learning strategy and students worked in a much wider time window, increasing their performance. an increase in the students' performance due to adequate time management in the learning process is well-established in the literature [12, 13] . in the present covid-19 confinement, students can find by themselves many different motivations (rewards) to work on a continuous basis. first, the confinement is a new scenario that has never been faced by the students. for this reason, students do not have any previous experience to use as a reference in their learning process. without any previous reference, students need to be confident that they are following the course correctly and therefore, work continuously in order not to miss any important content. another interpretation is that they are afraid of missing the academic year because of the covid-19 confinement and they work harder to overcome any difficulty. finally, students might be motivated by their intrinsic responsibility in a very confused situation and work hard to contribute as much as they can to solve the problems that higher education is facing. most probably, different students will find different motivations in this new scenario (probably a combination of many). we conclude that there is a real and measurable improvement in the students' learning performance that we believe can guarantee good progress this academic year despite the covid-19 confinement. answering question 4, we have demonstrated that students get better grades in activities that did not change their format after the covid-19 confinement. moreover, we have demonstrated that there is an improvement in their learning performance. in conclusion, higher scores are expected due to the covid-19 confinement that can be directly related to a real improvement in students' learning. supporting information s1 data. (xlsx) better performance of students was strongly evident when analysing the proportion of students that passed these 2 activities (fig 8b). thus, 95.6% of students passed activity 10 (p<0.0047 vs tracking e-learning through published papers: a systematic review a comparative analysis of forums and wikis as tools for online collaborative learning designing videogames to improve students' motivation investigating variation in learning processes in a future-learn mooc the mooc model for digital practice. sshrc application, knowledge synthesis for the digital economy teamwork assessment in collaborative projects through process mining techniques assessment of collaborative learning experiences by graphical analysis of wiki contributions. interactive learning environments mining theorybased patterns from big data: identifying self-regulated learning strategies in massive open online courses complexity leadership in learning analytics: drivers, challenges and opportunities analytics of time management strategies in a flipped classroom predictive power of regularity of pre-class activities in a flipped classroom covid-19 epidemic as e-learning boost? chronological development and effects at an austrian university against the background of the concept of "e-learning readiness campus traffic and e-learning during covid-19 pandemic online assessment in higher education in the time of covid-19. education in the knowledge society e-learning during covid 19 pandemic comparing students' self-discipline and self-regulation measures and their prediction of academic achievement investigating self-regulation and motivation: historical background, methodological developments, and future prospects the metacognitive control components of self-regulated learning student differences in self-regulated learning: relating grade, sex, and giftedness to self-efficacy and strategy use using formative assessment to influence self-and coregulated learning: the role of evaluative judgement theorizing and researching levels of processing in self-regulated learning developing web-based assessment strategies for facilitating junior high school students to perform self-regulated learning in an e-learning environment applying a web-based training to foster self-regulated learning-effects of an intervention for large numbers of participants. the internet and higher education comparing online and blended learner's self-regulated learning strategies and academic performance. the internet and higher education the authors want to thank dr. gema perez-chacón for her valuable comments on statistical analysis. conceptualization: santi fort. the aim of this study was to identify the effect of covid-19 confinement on students' performance. therefore, we conducted an experiment among 450 students from three subjects in different degrees of higher education at universidad autónoma de madrid, spain. the results of this study answer our 4 research questions:1. is there any effect (positive or negative) of covid-19 confinement on students' performance?2. is it possible to be sure that covid-19 confinement is the origin of the different performance (if any)?3. what are the reasons of the differences (if any) in students' performance?4. what are the expected effects of the differences in students' performance (if any) in the assessment process?with respect to research question 1, the results analyzed in section 3.2 show that there is a significant positive effect of covid-19 confinement on students' performance. the results indicate that students obtained better scores in all kinds of tests that were performed after the beginning of confinement. different sources of error have been removed from our study by including only subjects that fit the following requirements for the last three academic years:• same teaching methodology and teachers in all the years.• same assessment process in all years, including both distance and on-site activities.• tests performed both before and after the covid-19 confinement in the present academicyear.• tests that only imply an objective assessment process in order to avoid any possible subjective interpretation by the lecturers.we have compared results of students from the present 2019/2020 academic year (experimental group) with results of students from the last two academic years 2018/2019 and 2017/ 2018 (control group). students from the two previous academic years had similar circumstances from the beginning to the end of their course. when compared to students of the 2019/2020 academic year, we have two periods with different conditions. the first period is taught in the same conditions as in previous years (before confinement). in the second period, those conditions dramatically changed, and all the teaching and learning activities were limited to distance learning. the results of our studies (section 3.2) clearly indicate that there are key: cord-011444-6jh3lvm3 authors: loureiro, natália i. v.; viana, henrique v.; rodrigues, carlos r.; cabral, lúcio mendes; silva, thaís d. n.; cardoso, fernanda serpa; santos, dilvani oliveira; castro, helena c. title: solving an ethical issue involved in experimentation with animals in a brazilian teaching laboratory date: 2006-11-03 journal: biochem mol biol educ doi: 10.1002/bmb.2004.494032060404 sha: doc_id: 11444 cord_uid: 6jh3lvm3 changes are occurring within brazilian institutes of higher education; currently several universities are reviewing their course offerings and teaching approaches to determine if they meet the needs of today's undergraduate students. when changes are made to the curriculum of experimental courses, there should be an understood guarantee that all efforts to avoid ethical and biosafety issues have been diligently considered. ethical considerations lead us to create an alternative experimental session to be conducted that eliminated the use of rats, the conventional in vivo model employed for learning metabolism of glycogen in our university. to avoid possible biosafety issues, we prepared an alternative sample to simulate human urine, which we called guarurine. using our new method, it is possible to verify positive results imitating a diabetic and starving people samples for detection of glucose and ketone. the alternative tool described herein is not only particularly suited to bypass the ethics of using animals for teaching, but also permits the discussion of significant aspects of pathological and physiological situations such as diabetics and starvation in a simple, safe, and interesting way. the use of animals for research and teaching is an issue of great concern worldwide. in contrast to the legislative systems in united states, britain, germany, scandinavia, and many other countries [1] [2] [3] [4] [5] [6] , brazilian scientists and teachers still can pursue research projects and teaching class with relative freedom [7] . in brazil, animal research ethical committees were created only in the 1990s. the federal law 6638, which provides for the didactic-scientific practice of animal vivisection, was approved in may 1979 but still waits for regulation. in addition, some drafts that provide regulation for the use of animals for teaching and research purposes are still being analyzed by the brazilian congress [8, 9] . in the brazilian fluminense federal university, the study of biochemistry in the department of molecular and cell biology includes laboratory activities, which mainly allow students to actively explore key concepts in biochemistry in greater depth and acquire skill in scientific reasoning [10 -12] . animals (rats) have been widely used as models for teaching the catabolism of glycogen (glycogenolysis) in this discipline. however, in the last 2 years, the student teaching assistants and some students of several different undergraduate courses that attend this class had considered this experimental session as inappropriate. according to them, there was an "unnecessary sacrifice" of rats to demonstrate the use of glycogen as a fuel by an organism. thus, the refusal to attend this specific session by some of these undergraduate students, including those from the faculty of veterinary school, became common. because ethics is not a matter of adhering to absolute rules, but rather of doing what will have best consequences, given the constraints under which we act, the ethics of using a specific experimentation will depend on if the goal can be reached while causing the animal less suffering, using fewer animals, or without using animals at all. anyway, due to ethical and others concerns, we were facing the challenge of changing this teaching session. in resolving this issue, understanding the reasons why students object to animal laboratories was an important step. in their point of view, this experimental class represented a serious religious/ethical dilemma because the life and death of laboratory animals becomes trivialized in the process of exemplifying only one topic of metabolism. therefore, based on their opinion and using the principle of our self-regulation, rules were set up to conduct the selection of a future practical laboratory experiment. these were i) avoid the use of laboratory animals that would be sacrificed; ii) the inclusion of other topics of metabolism such as glycolysis, citric acid cycle, fatty acid and amino acid synthesis and catabolism, and ketogenesis in the experimental discussion; iii) the experiment should have low cost and be performed after the relevant theory material is studied; and finally iv) it also should be easy and fast, due to the limited time of the practical class. consequently, the whole laboratory experience should not last more than 2 h, according to the scheduled time available for the class. given these limitations, a previous laboratory class was considered to be an ideal candidate for replacement of the class in question. "urinalysis" is a nonanimal laboratory experiment basically consisting of assays to determine glucose and ketone levels in the urine from a diabetic and a healthy people. its experimental work does not last more than 2 h, and all necessary reagents were already part of the stockroom, avoiding any further expenses. this class permits the discussion of several topics of metabolism including those previous established; and, most of all, it allows aligning classroom practices with professional practices. hence "urinalysis" met the main criteria that had been determined. however, despite passing these rules, "urinalysis" raises a biosafety issue, e.g. the risk of using body fluids at the undergraduate level. according to the world health organization, the body fluids and substances of all persons should be considered to contain potentially infectious agents [13] . no distinction may be made between body fluids and substances from individuals with a known disease or infection and those from asymptomatic or undiagnosed individuals [13] . consequently, infection control practices shall be present in the urinalysis class including the use of gloves and masks, frequent hand washing, proper cleaning, and good disinfectant practice. these procedures greatly raise the expense of the exercise, which could be a problem in less economically developed countries such as brazil [14] . but an extreme example of the necessity for these protocols is the sars virus, which can survive in urine for at least 24 h [15] . another virus that can also be transmitted by this body fluid is the cytomegalovirus [16] . in fact, the risk factors due to using body fluids were the reason for the replacement of the "urinalysis" session in this discipline in the first place. however, due to the several advantages of the "urinalysis" session theme, we decided to keep it, creating a harmless substitute for urine, averting the original animal ethical and biosafety issues. additionally, we also planned a different way of class presentation using a problembased learning-like approach, stimulating the student to be more active in the laboratory class [17, 18] . in this article, we will present the protocol and approach used in this practice class, also including the evaluation by student teaching assistants and undergraduate students from nine different courses ("biological science," "pharmacy," "medicine," "veterinary medicine," "nutrition," "nursing," "odontology," "chemistry," and "industrial chemistry"). normal urine is a clear straw-colored liquid, which under normal conditions does not contain sugars, yeast cells, protein, ketones, bacteria, or parasitic organisms. in order to select a substance that imitates perfectly the visual aspect of urine, the famous brazilian guarana (paullinia spp.) was tested. the guarana seeds are used mainly to produce a soft drink that present similar visual appearance of urine [19] . hence the soft drink in its diet version was utilized to prepare samples to be tested on the urine diagnostic assays. these assays include the detection of glucose by the classic benedict's copper reduction reaction and of acetoacetic acid (the physiological ketone) by the reaction with sodium nitroprusside in a strongly basic medium. in order to prepare the guarurine samples, a 150-ml cup of diet guarana was diluted twice with distilled and deionized water and three samples of 100 ml were obtained, which we named guarurines (fig. 1) . on the first sample, glucose (5% w/v) and acetone (10% v/v) were added, representing the diabetic urine (fig. 1) . to the second sample, only acetone (10% v/v) was used to reproduce the urine from a starving person with prolonged diarrhea and vomiting (fig. 1) . the last 100 ml was considered the normal urine, therefore no glucose or acetone were added (fig. 1) . usually the urine specimens should not remain unrefrigerated for longer than 2 h, but in the case of guarurine samples they resisted several weeks out of the fridge, probably due to the presence of preservative in its formula. this aspect permits the storage of guarurine samples for longer periods than urine. classical benedictјs copper reaction (glucose)-in the glass tubes, 3 ml of benedict's reagent was added to 5 ml of the guarurine samples. the test tubes were immediately placed into a beaker of boiling water and left for 3 min. when heated with benedict's reagent, guarurine sample containing glucose (sample 1) formed an orange-brown precipitate similar to the diabetic urine (fig. 1) . sodium nitroprusside reaction in a strongly basic medium (ketone)-two drops of sodium nitroprusside reagent were added in 5 ml of the guarurine samples in a glass tube. the test tube was mixed and tilted to about a 30°angle, and the samples were slowly overlaid with 15% ammonium hydroxide. a pink ring appeared at the junction of the two liquids in the guarurine samples containing acetone (samples 1 and 2) after 15 min, similar to that observed for starving and diabetic people's urine. therefore, guarurine samples showed to be adequate as a harmless substitute for human urine to be used in the "urinalysis" session, because they act similar to the respective human samples (fig. 1) . after the preparation of the protocol and arranging all necessary laboratory material including the guarurine, it was possible to evaluate this new practical class with the group of student teaching assistants from the biochemistry discipline (n ϭ 6). the new problem-based learning-like approach used consisted of the presence of an initial situation where three unlabeled "urine" samples from three different people (a diabetic, a starving, and a healthy person) needed to be identified (fig. 2) . the main purpose was to correctly identify this "biological" material according to the testing results that would be obtained during the experimental class. this approach would lead to the discussion about the initial expected results and those finally found. by providing specific reagents and the three guarurine samples, the students were invited to use the laboratory reagents in order to find out about the origin of these samples. the goal of this practical class was mainly to provide to these students the opportunity to develop their own reasoning and acquire knowledge from the experi-mental results obtained. the support given during this experimental session at this moment consists of aiding in clarifying the objectives of the experiments, organizing the content, and in the "diagnosing" the clinical cases. consequently, these student teaching assistants were also being prepared to coordinate future "guarurinalysis" session for undergraduate students. in the end, all six students were able to complete the whole session on time and also identify correctly the physiological "source" of the given material. an interview at the end of the experimental session revealed that all six student teaching assistants voted unanimously to the inclusion of "guarurinalysis" in the discipline. aspects such as students safety; the nonobligatory use of gloves and mask but keeping other safety procedures such as frequent hand washing, which would permit the discussion about biosafety issues; and most of all, the opportunity of simulating a "body fluid" clinical analysis in a safer way, were pointed as the main advantages in this experimental session during their interview. this experimental protocol was also tested with 206 students from different undergraduate courses ("biological science," "pharmacy," "medicine," "veterinary medicine," "nutrition," "odontology," "chemistry," and "industrial chemistry") using the help of the students teaching assistants. at the beginning, undergraduate students were divided into groups for the organization of the experiments. however, everyone was stimulated to participate of the discussion during the whole teaching session. at the end, most of students could correctly identify the samples, increasing their knowledge about the biochemistry topics discussed. in case of the students, this practical session was evaluated by the application of a questionnaire consisting of simple and objective questions raised about four important issues; the acceptance level of the practical session; its permanence in the discipline for the next years; its direct correlation with biochemical topics discussed on the lecture part; and the clear applicability of this knowledge on their future professional career ( table i) . the survey results indicated that nearly all students find this experimental session interesting because all of them attributed good (51%) or very good (46%) acceptance level for it. in accordance with this data, 99% of the students voted for the permanence of this practical session in the biochemistry discipline for the next years (table i) . the undergraduate students noticed the direct correlations of this experimental simulation with the biochemical theoretical topics (94%) and their future professional life (81%) ( table i) . although these are positive results, the opinion about the professional-related topic radically changes if the "odontology" undergraduate course is analyzed individually. surprisingly, "odontology" students did not agree (38%) or partially agree (31%) that the knowl-edge offered by this practical session will be useful on their professional career, seeing no direct correlation between them (not shown). this result emphasizes the necessity of an improvement of the lectures, including professional themes in the biochemistry discipline to the understanding of the undergraduate student [20] . in case of the "odontology" class, clinical cases involving topics such as the problems with the healing process in teeth extraction procedures on diabetic persons could be a good example on how important is the knowledge about this disease and consequently of this experiment for the dentist. it is important to emphasize the major improvement in the student participation during the learning process using this specific practical approach. in fact, some students asked during the class for testing their own urine (10 -20%), which was not allowed. this specific students behavior indicate a direct interpretation about the applicability of those assays in quotidian. this is probably due to the fact that they were more required to do critical analysis of experiment's result when a problem-based learning-like approach is used. conclusion various pressures are driving changes on brazilian higher education. they are from government, students, professional bodies, and employers; from changes in teaching styles and in the discipline itself; and from ethical/ animal rights considerations. as these demanded changes occur, an alteration in the knowledge, skills, and attitudes of academic professor will be required, which includes the way teaching is delivered and learning is facilitated [21, 22] . herein we described a laboratory class experiment that can be applied in any biochemistry course avoiding ethical and biosafety issues. through the correct application of our established alternative tool and the understanding of the technical and clinical details of this practical session, the students could acquire knowledge not only about biochemistry but also about specific problems related to handling of the samples and reagents and how clinical results are obtained and interpreted. other advantages of this practical class are that it is fast, reasonably safe, and have low cost, requiring fewer resources to simulate clinical cases. this economical advantage is important to a country that struggles to maintain its own public academic institutions such as brazil. historical issues concerning animal experimentation in the united states research with animals: requirement, responsibility, welfare guidelines for animal surgery in research and teaching animal ethical committees the regulations concerning animal experiments in education by the german animal rights law animal experimentation in europe: from its origins to its future aspectos é ticos da pesquisa com animais normas de pesquisa em saú de ethics and animal experiments skills and processes in science education laboratory work in biochemical education: purpose and practice the link between laboratory and learning summit to put education at the heart of brazil's future severe acute respiratory syndrome (sars) multi-country outbreak update 47; studies of sars virus survival, situation in china cytomegalovirus (cmv) infection, available at /www the boyer commission on educating undergraduates in the research university for the carnegie foundation for the advancement of teaching (1998) reinventing undergraduate education: a blueprint for america's research universities approaches to cell biology teaching: learning content in context-problem-based learning species of paullinia with economic potential, available at www the integrative nature of biochemistry: challenges of biochemical education in the u learning to become a teacher: the wheelock way successful teacher education programs in the era of reform (an essay review of studies of excellence in teacher education (3 vols.): preparation in the undergraduate years; preparation in a five-year program; preparation at the graduate level acknowledgments-we thank the fluminense federal university for the honorable mention for practical and teaching approach development in the 2003 coordinators awards. we also thank professor cícero carlos de freitas, mr. hugo rodolfo de oliveira barbosa filho and mrs. dione m. silva for their help and technical assistance. key: cord-342496-4na1e2de authors: savitsky, bella; findling, yifat; ereli, anat; hendel, tova title: anxiety and coping strategies among nursing students during the covid-19 pandemic date: 2020-06-02 journal: nurse educ pract doi: 10.1016/j.nepr.2020.102809 sha: doc_id: 342496 cord_uid: 4na1e2de anxiety is highly prevalent among nursing students even in normal circumstances. in israel during the covid-19 pandemic and mandatory lockdown, nursing students encountered a new reality of economic uncertainty, fear of infection, challenges of distance education, lack of personal protection equipment (ppe) at work etc. the objective of this study was to assess levels of anxiety and ways of coping among nursing students in the ashkelon academic college, southern district, israel. a cross-sectional study was conducted among all 244 students in the nursing department during the third week of a national lockdown. anxiety level was assessed using the generalized anxiety disorder 7-item scale with a cut-off point of 10 for moderate and of 15 for severe anxiety. factor analysis was used to identify coping components. the prevalence of moderate and severe anxiety was 42.8% and 18.1% respectively. gender, lack of ppe, parental status, and fear of infection were significantly associated with a higher anxiety score. stronger self-esteem and usage of humor were associated with significantly lower anxiety levels, while mental disengagement with higher anxiety levels. the nursing department's staff may contribute in lowering student anxiety by staying in contact with students and encouraging and supporting them through this challenging period. at the time of writing (april 06,2020), the small state of israel (population 9,136,000 residents (central bureau of statistics, 2020) is in a state of lockdown with 8,611 verified cases of covid-19, 332 of them moderately or critically ill, and 106 patients on ventilators (weizman institute of science, 2020). israel is a country used to battles and our medical professionals have always been in the forefront, supporting those on the battlefield. however, this battle is different and the medical professionals have become the commanders and the foot soldiers. in the light of this situation, we decided to examine the anxiety of nursing students in these unusual times. anxiety is highly prevalent among college students. the top three concerns among students are academic performance, pressure to succeed, and post-graduation plans (beiter et al., 2015) . nursing education has been consistently associated with anxiety among students. heavy course loads, stringent examinations, continued pressure to attain a high grade point average (chernomas and shapiro, 2013) , complex interpersonal relationships, challenges of the clinical environment (chen et al., 2015) , caring for chronic and terminally ill patients (sancar, yalcin and acikgoz, 2018 ) results in greater anxiety among nursing students than among students from any of the other healthcare disciplines. anxiety has a negative effect on the quality of students' life, their education and clinical practice (sanad, 2019) . during an epidemic/pandemic, state nursing students are exposed to additional stressful factors. a study conducted among nursing students the during the sars outbreak (2003) in hong-kong showed that nursing students perceived themselves to be at higher risk of infection (wong et al., 2004) . similarly, in a study from saudi-arabia during the mers outbreak (2016), healthcare students expressed their reluctance to work in healthcare facilities with inadequate mers infection control isolation policies (elrggal et al., 2018) . in israel during the covid-19 pandemic, a state-wide mandatory closure of all institutions of learning, universities, colleges, schools and kindergartens was imposed from the beginning of march 2020 and an isolation policy was introduced by the government. only 30% of workers in both public and private sectors were allowed to continue to work (others either continued to work from home, received unpaid leave or were fired). in addition, the government mandated that all citizens had to stay within 100 meters of their home if they went out for sport, dog-walking, etc. which made it problematic to perform the most basic chores and religious activities. all stores were closed except for grocery stores, supermarkets and pharmacies. the staff of all academic institutions faced a new reality had to turn to online teaching with the aim of continuing the academic year and trying to carry out end-of-semester exams as usual. staff members started to practice and use remote reaching strategies almost immediately with the college lockdown. meanwhile students working for payment in healthcare facilities (outside of the curriculum framework) continued to work in the new reality of uncertain conditions and controversial information in relation to the new virus and often with lack of sufficient ppe. due to labor shortage in hospitals and in the community, the ministry of health's office of the director of nursing asked all nursing students in the country to voluntarily agree to work in the hospitals and community settings. as a result, more than 1,600 nursing students (100 students of them from our department (41%)) answered the call. the objective of this study was to assess level of anxiety and ways of coping during the period of covid-19 pandemic and identify association of coping strategies with characteristics of the students among nursing students in the ashkelon academic college, southern district, israel. a cross-sectional study was conducted during the third week of the lockdown among all students in the nursing department (244 students), first to fourth year of study. the questionnaire was conducted using google forms and a request to participate in the study was sent to all students. the study received approval from the ethical board of our department of nursing. the questionnaire was anonymous (demographic questions were asked but did not include identification details). filling out the questionnaire reflected consent to participate. the total response rate was 88% (215 out of 244 students -93% among the students of the first year, 95% among the second year students, 82% among the third year students and 80% among the students of the fourth, final year of studies ). the questionnaire included demographic information on students including gender, age, year of the study, family status, ethnic group, country of birth, level of religiosity and employment status (among students who worked for a salary). age was used as a continuous variable and as a dichotomic variable with the median as the cut-off point (25 years). level of religiosity was used as a categorical variable with categories: secular; traditional and religious. occupational status was used as a categorical variable with categories: salaried work in healthcare facilities; work in unrelated to the nursing field; does not work. anxiety level was assessed using the generalized anxiety disorder 7-item scale (gad-7) (löwe et al., 2008) . gad-7 was used with a suggested cut-off point of 10 for defining moderate anxiety and with a cut-off point of 15 for defining severe anxiety. this questionnaire has been widely used and is reported to have high internal consistency and good test -retest reliability among adults (zhong et al., 2015; rutter and brown, 2017), adolescents (spitzer et al., 2006) and college students (bártolo et al., 2017) . in a previous study at a cut point of 10 or greater, sensitivity and specificity exceeded 0.80, and sensitivity was nearly maximized (spitzer et al., 2006) . the internal consistency of the questionnaire was validated in this study using the cronbach's alpha (ca). ca was found 0.93 for the total sample, pointing to high internal consistency. for defining coping strategies we used coping behavior questionnaire (cope) (carver and scheier, 1989) and choose 10 items after adaptation for the current stressors. the differences in anxiety score were assessed with mann-whitney and kruskal-wallis nonparametric tests. the differences in frequency of the moderate and severe anxiety disorder by demographic sub-groups was checked with chi-square test. scores characterizing coping strategies were constructed using factor analysis with varimax rotation and an unrestricted number of factors. the following variables were included in summarizing the information regarding coping strategies of the students: four questions concerning self-esteem and ten items from the coping behavior questionnaire (cope) (carver and scheier, 1989) . variables with factor loadings ≥0.5 were considered contributing variables to a given factor and five factors were created and together explained approximately 60% of the variability ( table 1) . the first factor, referred to as "factor of self-esteem," explained 19.2% of the variance. with an increase in self-esteem of the students that they have "strong" personalities and are capable of dealing with challenges, this factor increased. the second factor, referred to as "factor of seeking information and consultation," explained 13.1% of the variance. this factor increased with escalation of seeking information and using consultations with professionals about the situation by the student. the third factor, referred to as "factor of mental disengagement" explained 9.9% of the variance. with an increase of usage of the coping strategies such as eating, consuming sedative drugs and alcohol, this factor increased. the fourth factor, referred to as "factor of spiritual and not scientific sources of support, " explained 9.8% of the variance. with an increase of belief in god and usage of social networks to get information, this factor increased. the last fifth factor, referred to as "factor of humor" explained 8.1% of the variance. with an increase of usage of humor to deal with the situation, this factor increased. after designating the factors, we used a t-test or anova test to check the association between each factor and demographic variables and a logistic regression model to investigate the associations of each factor simultaneously and after adjustment for gender with moderate and severe anxiety. statistical analyses were performed with spss statistical software version 25.0. for all analyses performed, a value of p<0.05 was considered statistically significant. the demographic characteristics of the study population are presented in table 2 . it should be noted that each year of study sees an increase in the number of married students, students with children and students working for a salary. among students who work with payment in the healthcare facilities, 69% work in a hospital and 31% work in the community setting. among students who work in healthcare facilities 50% reported that they encountered the lack of ppe at work. the percentage of students with gad-7 of 10 and above (moderate anxiety) was 42.8% (30.8% among males and 44.7% among the females, p value of x 2 test=0.21). the percentage of students with gad-7 of 15 and above (severe anxiety) was 18.1% (0% among males and 20.7% among the females, p value of x 2 test=0.006) . differences in the anxiety score was assessed by students' characteristics and presented in table 3 . males had significantly lower anxiety scores (median=7.0, iqr:1.0-11.0) in comparison with females (median=9.0, iqr:5.25-14.0) (p value=0.011). none of the other demographic characteristics were found significantly associated with the anxiety score; academic year of study was not found associated with level of anxiety. lack of ppe among working students was found associated significantly with a higher anxiety score in comparison with those students who did not experience a lack of ppe at work (median=11.0, iqr:8.0-13.5 and median=6.0, iqr:2.5-10.0 respectively) (p value=0.019). students who were more concerned with the future continuation of this academic year had a significantly higher anxiety score (median=9, iqr:6.0-14.0) than those who were concerned at a low or moderate level (median=7, iqr:2.0-12.0) (p value of mann-whitney non-parametric test=0.024) (data is not presented in the table) . the anxiety score increased among those students who are the parents of young children with the increase of the burden following the lack of educational frameworks for children (schools and kindergartens); parents who reported that they do not feel such a burden had a lower mean anxiety score than those who experienced an extreme burden (mean anxiety score was 6.5 [median=6] vs. the anxiety score was found to be positively significantly associated with dose-response relationship with fear of becoming infected (figure 1 ). the anxiety score of students who reported intense fear of infection was found significantly higher. we used logistic regression to identify the association between each of the five factors, representing coping strategies and levels of anxiety (with 10 and 15 cut-of-point respectively) as an outcome. the adjustment for gender was performed in relation to moderate anxiety only as none of males suffered from severe anxiety. table 4 presents the role of each factor. the factor of self-esteem was found significantly and negatively associated with moderate and severe anxiety: among students who perceived themselves as having strong personalities, the odds for moderate and severe anxiety were significantly lower. the factor of mental disengagement was found significantly associated with moderate and severe anxiety. the odds for moderate and severe anxiety were higher among students who reported alcohol usage, sedative drugs or excessive eating. the factor of factor of humor was found associated only with severe anxiety: the more this strategy was in use, the odds for severe anxiety were lower. coping strategies and students' characteristics (data is not presented). seeking information and consultation (factor ii) was significantly associated with the female gender (p value of t-test=0.023) and with occupational status "work as students with payment in the healthcare facilities" (p value of anova test=0.05). mental disengagement (factor iii) was significantly associated with family status "not married" (p value of t-test=0.024 for factor iii), with parental status "no children" (p value of t-test=0.013), and with level of religiosity "secular" (p value of anova test=0.001). using the coping strategy of spiritual and not scientific sources of support (factor iv) was significantly associated with age≤25 (p value of t-test=0.005), female gender (p value of t-test<0.0001), birth county "israel" (p value of t-test=0.001), population group "muslims (arabs and bedouins)" (p value of anova test=0.004), religiosity levels 'traditional" and "religious" p value of anova test<0.0001). using humor (factor v) was significantly associated with the population group "jews" (p value of anova test<0.0001) and with religiosity level "secular" (p value of anova test=0.01). the results of our study reflects anxiety at high levels among nursing students during the continuing covid-19 pandemic. according to previous studies, even in normal circumstances students experience anxiety (among university and college students, prevalence of moderate and severe anxiety was found 12.2% and 5.8% in hong-kong (lun et al., 2018) , 15.6% and 8.3% in portugal (bártolo et al., 2017) and 17.5% in australia (farrer et al., 2016) . among medical students prevalence of moderate anxiety was 25% in uk, 20% in north america, 13.7% in new-zealand and 23% in lebanon (quek et al., 2019) . anxiety level among female students was usually higher than among males in previous studies (mclean and hofmann, 2012; lun et al., 2018; mirón et al., 2019; quek et al., 2019; sanad, 2019) and was also found in our study. given the fact that females comprise the majority of our study population of nursing students, it can explain in part high the prevalence of anxiety in our study. we believe that the reason for this high prevalence of anxiety is explained by the extremely exceptional living situations and conditions during the continuing covid-19 pandemic. these circumstances include social isolation, economic instability, children who need to be taken care of at home, uncertainty about the future, challenges of remote learning, fear of getting infected and more. we did find that the anxiety score was higher as the fear of getting infected was stronger among those who encountered the lack of ppe at work and grew with the increase of concern regarding the continuation of the current academic year. as to the self-esteem factor, we found that stronger self-esteem was associated with the lowest anxiety levels (moderate and severe). our findings are consistent with the theory that people with high self-esteem presumably engage in positive, active attempts to cope with stressors (carver and scheier, 1989) . similar results were found among chinees nursing students who showed that a positive coping style is significantly correlated with a higher level of self-esteem (ni et al., 2010 (ni et al., , 2012 . in our study mental disengagement (usage of alcohol, sedative drugs and excessive eating) was associated with a higher state of anxiety. consistent with what is known about alcohol and drugs to cope with anxiety, this coping strategy is ineffective and may worsen the level of anxiety (carver and scheier, 1989) . maladaptive coping strategies are more likely to lead to eating disorders (zheng et al., 2020) . we still cannot conclude whether excessive eating in our sample represented the destructive coping strategy or whether this was a result of long home quarantine. boredom and frustration related to the lockdown may have resulted in excessive eating. usage of humor was associated in this study with lower levels of anxiety. freud's psychodynamic viewpoint described humor as one of the strongest defense mechanisms that allow individuals to face problems and avoid negative emotions and researchers believe that humor has a stress-moderating effect (penson et al., 2007) . interestingly, searching for information, was not associated with anxiety level in our study. we believe that in the particular case of the covid-19 pandemic, seeking for information might increase anxiety by overflow and multiple sources of information, sometimes broadcasting contradicting information. in our study religiosity was not associated with a lower anxiety level. numerous empirical studies have found that religion assists in an individual's ability to cope with a variety of personal and collective stressors, such as illness, the loss of a child, terrorist threats and war (bryan et al., 2016) . we believe that in particular during this period a religious lifestyle was seriously compromised following mandatory prohibitions against praying in a mosque or synagogue and using a mikveh (jewish ritual bath), the anxiety of religious people could increase. in addition, a religious lifestyle is associated with a higher birth rate (in our study the percentage of parents was twice as high among religious students in comparison with the secular group). being isolated with young children could be challenging, as we found higher anxiety level among those students who are parents and reported a heavy burden following closure of schools and kindergartens. despite the limitations of the present study related to a cross-sectional design with self-reported measures, our findings add new evidence concerning anxiety among nursing students during the 9 pandemic of covid-19. the staff of our nursing department believe that the most important way to help students during this period is to stay in continuous contact with them beyond online teaching. students face severe anxiety related to economic uncertainty, fear for health of their families, fear of infection, the need to support and care for children, and to deal with the challenges of distance education. the main goal of the department faculty is to keep in touch with students, to encourage and support them through this challenging period, which is still far from being finished as we write these words. we are planning to conduct an additional study in the near future to assess whether there has been a change in the state of anxiety of the students and their use of various coping strategies to meet the challenges of the situation. age ≤ 25 years 9.6 (5.5) 9.0 (6.0-13.0) 0.511* >26 years 9.1 (5.7) 9.0 (5.0-14.0) salaried work as a student in healthcare facilities 9.2 (5.3) 9.0 (5.0-13.0) 0.988** work unrelated to nursing field 9.2 (5.9) 9.0 (5.0-14.0) does not work 9.4 (5.4) 8.0 (6.0-12.0) ppe a supplied at work place 6.8 (5.0) 6.0 (2.5-10.0) 0.019 lack of equipment 11.6 (4.5) 11.0 (8.0-13.5) a only among students working as students with payment in the healthcare facilities * p value of mann-whitney non-parametric test ** p value of kruskal-wallis non-parametric test 1 • anxiety is highly prevalent among nursing students even in normal circumstances. • in israel during the covid-19 pandemic and mandatory lockdown, nursing students encountereda challenging reality. • the prevalence of moderate and severe anxiety was 43% and 18% respectively. • female gender, lack of protective equipment at work, and being a parent were significantly associated with higher anxiety scores. • stronger resilience and usage of humor were associated with significantly lower anxiety levels. • mental disengagement (alcohol, sedative drugs usage and excessive eating) was associated with higher anxiety levels. factor structure and construct validity of the generalized anxiety disorder 7-item (gad-7) among portuguese college students the prevalence and correlates of depression, anxiety, and stress in a sample of college students god, can i tell you something? the effect of religious coping on the relationship between anxiety over emotional expression, anxiety, and depressive symptoms assessing coping strategies : a theoretically based approach population of israel on the eve of 2020 the prevalence and related factors of depressive symptoms among junior college nursing students: a cross-sectional study stress, depression, and anxiety among undergraduate nursing students evaluation of preparedness of healthcare student volunteers against middle east respiratory syndrome coronavirus (mers-cov) in makkah, saudi arabia: a crosssectional study demographic and psychosocial predictors of major depression and generalised anxiety disorder in australian university students validation and standardization of the generalized anxiety disorder screener (gad-7) in the general population depression and anxiety among university students in hong kong gender differences in anxiety disorders: prevalence, course of illness, comorbidity and burden of illness perceived stress , anxiety and depression among undergraduate students : an online survey study relationship between coping, self-esteem, individual factors and mental health among chinese nursing students: a matched case-control study chinese female nursing students' coping strategies, self-esteem and related factors in different years of school update: laughter: the best medicine? richard', the the global prevalence of anxiety among medical students : a meta-analysis' psychometric properties of the generalized anxiety disorder scale-7 (gad-7) in outpatients with anxiety and mood disorders stress and anxiety among junior nursing students during the initial clinical training : a descriptive study at college of health sciences an examination of anxiety levels of nursing students caring for patients in terminal period we wish to thank susan holzman for her highly professional editing advices and her valuable and kind support on this project. all authors approve that they do not have any financial and personal relationships with other people, or organizations, that could inappropriately influence (bias) this research and this all authors approve that they do not have any financial and personal relationships with other people, or organizations, that could inappropriately influence (bias) this research and this manuscript. key: cord-331410-mcee0wr4 authors: tempski, p.; arantes-costa, f. m.; kobayasi, r.; siqueira, m. a.; torsani, m. m.; amaro, b. q.; nascimento, m. e. f.; siqueira, s. l.; santos, i. s.; martins, m. a. title: medical students perceptions and motivations in time of covid-19 pandemic date: 2020-05-30 journal: nan doi: 10.1101/2020.05.28.20115956 sha: doc_id: 331410 cord_uid: mcee0wr4 background: there has been a rapid increase in the number of cases of covid19 in latin america, africa and asia, in many countries that have an insufficient number of physicians and other health care personnel, and the need for the inclusion of medical students as part of the health teams is a very important issue. it has been recommended that medical students work as volunteers, have appropriate training, do not undertake any activity beyond their level of competence, have continuous supervision and adequate personal protective equipment. however, motivation of medical students must be evaluated in order to make volunteering a more evidence-based initiative. the aim of our study was to evaluate motivation of medical students to be part of the health team to help in the covid19 pandemic. methods and findings: we developed a questionnaire specifically to evaluate medical students perceptions about participating in care of patients with suspected infection due to coronavirus during the covid19 pandemic. the questionnaire had two parts: a) individual characteristics, year and geographic location of medical school; b) twenty eight statements responded on a 5 point likert scale (totally agree, agree, neither agree nor disagree, disagree and totally disagree). to develop the questionnaire, we performed consensus meetings of a group of faculty and medical students. the questionnaire was sent to student organizations of 257 medical schools in brazil and answered by 10,433 students. we used multinomial logistic regression models to analyse the data. statements with greater odds ratios for participation of medical students in covid19 pandemic were related to sense of purpose or duty (it is the duty of the medical student to put himself at the service of the population in the pandemic), altruism (i am willing to take risks by participating in practical in the context of pandemic), perception of good performance and professional identity (i will be a better health professional for having experienced the pandemic). males had higher odds ratios than females (1.36 [95% ci: 1.24 to 1.49] to 1.68 [95% ci: 1.47 to 1.91]). conclusions: medical students are motivated by sense of purpose or duty, altruism, perception of good performance and values of professionalism more than their interest in learning. these results have implications in the developing of programs of volunteering and in the design of health force policies in the present pandemic and in future health emergencies. the coronavirus covid-19 pandemic is the most important global health crisis of our time and the greatest challenge the health system has faced since world war two. since its emergence in asia in 2019, the virus has spread to every continent, except antarctica. cases are rising daily in europe, north america, and, in the last weeks, also in latin america and africa. [1, 2] covid-19 pandemic resulted in a disruption of undergraduate medical education. in many countries, medical education faculty have quickly transitioned the first-years curriculum to on-line activities in response to the need of social isolation to flatten the curve of new cases of covid-19. [3] in addition, in the final years of medical schools, in many countries, clerkships were severely affected by the rapid changes in hospitals due to the need of care of an increasing number of covid-19 patients and medical students were advised to stay at home, considering the potential risk of medical students spreading covid-19 infection in health care settings and the shortage of personal protective equipment (ppe). [3, 4] however, the role of medical students in the covid-19 pandemic is changing rapidly, due to the shortage of health professionals in many cities, even in developed countries. association of medical colleges (aamc) of the united states have published guidelines for the participation of medical students in the global effort to give the best care to patients with covid-19. [5, 6] both associations recommend that medical students work as volunteers, have appropriate training, do not undertake any activity beyond their level of competence, have continuous supervision and adequate ppe. [5, 6] in the last weeks, there has been a huge increase in the number of cases in countries that have an insufficient number of physicians and other health care personnel, and it is possible to anticipate the need for the inclusion of medical students as part of the health teams. [2, 7] in many countries thousands of medical students have volunteered their services to support the fight against the coronavirus pandemic. [8, 9] motivation is pivotal to volunteering and must be evaluated in order to make volunteering a more evidence-based initiative. the aim of our study was to evaluate motivation of medical students to be part of the heath team to help in the covid-19 pandemic. it was performed in a developing country, brazil, in the first week of the increase in cases of covid-19 in brazil, including 10,433 medical students. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint we developed a questionnaire specifically to evaluate medical student's perceptions about participating in care of patients with suspected infection due to coronavirus during the covid-19 pandemic. the questionnaire had two parts: a) individual characteristics, year and geographic location of medical school; b) twenty-eight statements responded on a 5-point likert scale (totally agree, agree, neither agree nor disagree, disagree and totally disagree). to develop the questionnaire, we performed consensus meetings of a group of faculty and medical students. the questionnaire was developed on a google form survey administration app. with the help of the brazilian section of the international federation of medical students associations (ifmsa), the questionnaire was sent to medical student organizations of medical schools of all regions of brazil, using instagram and whatsapp social media. in the first page of the questionnaire, the purpose of the study was explained, and the student had to agree with the objectives of the study before filling the questionnaire. the survey was performed between 20 th and 22 th of march, since after these three days of survey we had received more than 10,000 answers. in this period, there were only a small number of patients with the diagnosis of covid-19 disease in brazil; 1,546 confirmed cases and 25 deaths due to according to the ministry of health of brazil. [10] . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. [11] the questionnaire was sent to the student organizations of 257 medical schools (75.4% of brazilian medical schools). participants informed their agreement or not with two statements regarding their view about the role medical students should have during the covid-19 pandemic. these questions were (s8) medical internship students must participate in health care assistance during pandemic; and (s9) all students regardless their year during the medical school, must participate in health care assistance during pandemic. participants were classified according to their agreement with statements s8 and s9 as: (1) all students should participate (agree with s9); (2) only students in internship should participate (not agree nor disagree / disagree with s9 and agree with s8); and (3) no students should participate (not agree nor disagree / disagree with s9, not agree nor disagree / disagree with s8). for the analysis, we combined the responses "completely agree" with "agree" for each statement. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted may 30, 2020. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint table 1 shows the number of medical students from the five geographic regions of the country that answered the questionnaire, and the relationship between the number of participants and the number of vacancies in each region of the country. there was a similar proportion of respondents/vacancies, showing that the sample was homogenously distributed across the country. table 2 shows the number (and percentage) of medical students that answered "completely agree" or "agree" for each one of the 28 statements of the questionnaire. students were divided considering year of medical course and sex. supplemental table 1 shows the answers to the questionnaire for the hole group of medical students. of the 10,433 students who completed the survey, 7,267 (69.7%) were females, and 2,445 (23.4%) were in internship years (interns). the mean age was 22.5 ± 3.9 years. all students should participate in the response to the covid-19 pandemic according to . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted may 30, 2020. table 3 shows the participants' characteristics and perceptions according to their view about the role of medical students during the covid-19 pandemic. the statements which most participants agreed to were: s3 "i know how to guide for preventive measures" (95.5%), s5 "i know how to use personal protection equipment (pfe)" (79.1%), and s13 "medical schools must suspend their academic activities during the first to fourth year" (77.9%). . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint table 4 shows the adjusted odds ratios (and 95% confidence intervals), from multinomial models, for the association between students' characteristics and perceptions and their view about the role of medical students during the covid-19 pandemic (crude models results are presented in supplemental table 2 ). compared to the participation of no medical students, odds ratios [or] for the participation of students in internship or all medical students were, respectively, 0.61 (95% confidence interval: 0.55-0.67) and 0.13 (95% ci: 0.10-0.16). men were more prone to support the participation of medical students in the fight against the pandemic (or for the participation of students in internship or all medical students respectively 1.36 [95%ci: 1.24-1.49] and 1.68 [95%ci: beliefs towards a participation of medical students in covid-19 pandemic healthcare with highest odds ratios were: s10 "it is the duty of the medical student to put himself at the service of the population in the pandemic", s28 "i am willing to take risks by participating in practical in the context of pandemic", s6 "i am able to participate in the care of patients who seek health care", s7 "i feel able to communicate the diagnosis for covid-19 infection", and s22 "i will be a better health professional for having experienced the pandemic". on the other hand, beliefs against a participation of medical students in covid-19 pandemic healthcare with lowest odds ratios were statements s13/s14 (suspension of academic activities), s11 "i feel insecure regarding the future", s21 "my emotional state during the pandemic affects my learning", and s27 "the role of the medical students during the pandemics time is irrelevant". . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint analysis with only the students in the last years (interns) was also performed (table 5 ). in this subgroup, male sex was also associated with the support of a participation of medical students. similar to the findings of the entire sample, statements s10, s28, s6, s2, s4 and s22 were the beliefs towards a participation of medical students with highest odds ratios in this subgroup. on the other hand, agreement with the statements s13/s14 (suspension of academic activities), s23 "i feel stressed in the hospital at the moment", s12 "i am afraid of contaminating myself" and s27 "the role of the medical students during the pandemics time is irrelevant" were more relevant beliefs associated with a perception against the participation of medical students in the actions during the pandemic. crude models for this subgroup are presented in supplemental table 3 . . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint beliefs towards a participation of medical students in covid-19 pandemic healthcare s10. it is the duty of the medical student to put himself at the service of the population in the pandemic 1 beliefs not / poorly related with the participation of medical students in covid-19 pandemic healthcare s20. i prefer to study theoretical content using distance learning methods beliefs towards a participation of medical students in covid-19 pandemic healthcare s10. it is the duty of the medical student to put himself at the service of the population in the pandemic 1 beliefs against a participation of medical students in covid-19 pandemic healthcare s13. medical schools must suspend their academic activities during the first to fourth year. in this study we aimed to evaluate the factors that motivate a medical student to participate in the care of patients in the context of covid-19 pandemic. we also aimed to understand the students' desires related to continuing medical training or to have online education during the pandemic. to our knowledge, there is no previous study that evaluated a large sample of medical students with these purposes. we believe that this study can contribute to the planning of medical education and work force organization in the covid-19 pandemic and in other future health emergencies. although our sample was a convenience sample, we evaluated a large group of medical students (10,433) and there was a uniform distribution of subjects across the country (table 1) . when we performed this study, the number of covid-19 cases in brazil was low, and only 7·0% of participants (table 3 ) had a family member or a friend with a diagnosis of covid-19 or had a personal diagnosis of this disease. [10] we were able to study the factors that influence the desire to participate in the care of covid-19 without a strong emotional influence of sick relatives or friends. when we compared the answers to the statements of the questionnaire provided by students in the first two years of medical course to students of the last two years (interns) ( table 2 , univariate analysis) we observed many differences. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint the largest differences in percentages of agreement with the statements were related to feeling of competency to take care of patients (differences from 26.7% to 37.7%, statements s1, s2, s4, s6 and s7). as expected, interns felt more confident concerning identification of a suspected case, identification of signs of severity, guidance of therapeutic measures, participation in the care of patients and communication of diagnosis. in addition, interns were more prone to accept on-line learning and the interruption of both classes and internship activities (differences from 16.0% to 19.0%). in contrast, students from the first years were more insecure about the substitution of classes and practical activities by on-line learning (statements s16 and s17, differences 10.5% and 17.9%). more students preferred to delay their training and fully restore the academic activities. although interns felt more secure concerning competencies to take care of covid-19 patients, many interns probably needed psychological support. in fact, more interns agreed to the statement s23 ("i feel stressed in the hospital at the moment", difference 24.5% compared to first year medical students) and less interns agreed with statement s25 ("i have access to psychological support", difference 17.6%). in addition, many students were afraid of contaminating themselves (statement s12, percentages of agreement from 59.2 to 68.4%) and considered that their emotional state affected learning (statement s21, percentages of agreement from 40.6 to 44.5%). to perform the multivariate analysis, we decided to divide the medical students into three groups, concerning their responses about participation of medical students in the covid-. cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint 19 pandemic (no participation, only interns participate or all students participate, table 3 ), to better understand the factors that were most important in the student's decision to participate. since we observed many differences between the answers of students in the first years of medical program and interns, we performed two analysis, one including all medical students and the second including only interns (tables 4 and 5 ). interestingly, we did not observe important differences when we compared the two analysis, suggesting that individual factors are more important than professional identity developed during medical course. maybe the decision to be a volunteer in a health emergency such as covid-19 pandemic is more linked to an emotional or attitudinal decision than beliefs of self-efficacy or performance. the statements that had greater odds ratios when the groups of "all students should participate" and "only students in internship should participate" were compared to "no students should participate" group (reference, odds 1.0) were, for all students, statements s10, s28, s6, s7 and s22, and for interns only, statements s10, s28, s6, s2, s4 and s22. we observed that sense of purpose or duty (moral values linked to medical profession) (s10 "it is the duty of the medical student to put himself at the service of the population in the pandemic") was the most important factor that influenced the desire to work in the pandemic, followed by the willingness to take risk (altruism) (s28 "i am willing to take risks by participating in practical in the context of pandemic") and the perception of good performance (self-efficacy) (s6 "i am able to participate in the care of patients who seek health care" and s7 "i feel able to communicate the diagnosis for covid-19 infection", . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint for all medical students and s6, s2 "i can identify signs of severity in a patient" and s4 "i know how to guide for therapeutic measures") for interns. in addition to statements that suggest moral values, altruism and confidence in professional competence, in both analysis, statement s22 ("i will be a better health professional for having experienced the pandemic"), that is related to building of professional identity, had odds ratios above 2.0 in all comparisons. in other words, the desire to help was stronger among the students that believed that they should work in the pandemic than their interest in learning during this health emergency. allowing students to participate can reinforce important values, such as altruism, service in times of crisis, and solidarity with the profession, contributing to the building of professional identity. [12] why people act or decide to serve others is important to better organize volunteerism in social and career. [14] they concluded that self-determined motivation results in more volunteer's satisfaction. [14] . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint in our study we observed that the student with higher sense of duty (s10) and altruism (s28) were more prone to engage in health care activities during covid-19, and it could be interpreted as self-determined motivation or values function of motivation. we also observed that the self-perception of competence (s6 and s7) was the third factor influencing motivation and the fourth was the desire of learn with the experience of working in the pandemic. desire to learn demonstrated for student can be interpreted as understanding function by clary et al., and it is also a self-determined motivation defined by guntert et al. [13, 14] the controlled motivation was not important in our data. we also evaluated factors that can influence the decision to not participate in the covid-19 health effort. we observed that predictors of do not be a volunteer were beliefs that all educational activities should be suspended (s13 and s14), fear of contamination (s12 "i am afraid of contaminating myself") and emotional factors (s23 "i feel stressed in the hospital at the moment"). the differences we observed between male and female medical students were smaller than the differences among students of different years of medical program ( table 2, univariate analysis). differences between males and females concerning the percentage of agreement with the statements of the questionnaire were greater than 9% in only 5 statements (females vs males): s11 "i feel insecure regarding the future" (16.0%), s7 "i feel able to communicate the diagnosis for covid-19 infection" (-13.8%), s21 "my emotional state during the pandemic affects my learning" (11.2%), s28 "i am willing to take risks by participating in practical activities in the context of pandemic" (-10.0%) and . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint s12 "i am afraid of contaminating myself" (9.7%). all these statements are related to emotional competencies. results from multinomial regressions performed with all medical students (table 3 ) and with only interns ( table 4 ) also showed that males were more prone to believe that only interns should participate (odds rations 1.36 and 1.29, respectively, for all students and interns) and that all students should participate (odds ratios 1.68 and 1.75, respectively). we argue that this difference is possibly due males' higher propensity to take risks in health/safety domain. [15] also, females are more prone to develop anxiety and stress disorders, be more affected by human suffering and have worse perceptions about their own quality of life, health and skills. [16] [17] [18] these factors can influence their intentions to act in pandemic. nonetheless, we cannot disregard gender bias as a possible limitation of our instrument. [18, 19] covid-19 pandemic had a substantial impact on medical education across the world. [3, 5] there is uncertainty and disagreement about the appropriate roles for medical students during this pandemic, and student participation in clinical care has varied across institutions and countries. [4, 12] many medical schools forbid any patient interaction, others included medical students in patient care and others decided to graduate medical students early in order to make them frontline clinicians. the american association of medical colleges recommended that "unless there is a critical health care workforce need locally, we strongly suggest that medical students not be involved in any direct patient care activities". is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint medical students in a local or global health emergency. some educators chose the position to ensure student security and to avoid exposition, and this perspective can send a message (hidden curriculum) that students have a passive role without a significant social responsibility. on the other hand, medical educators that defend the inclusion of the medical student in health teams send to medical students a message that social responsibility is pivotal to professional identity. interestingly, in our study, 56.9% of medical students agreed with statement s22 ("i will be a better health professional for having experienced the pandemic"). a shortage of health professionals has been reached even in some cities of developed countries. [20,21] this risk will be even greater as the pandemic is reaching developing countries in south america, asia and africa. the participation of medical students in the care of people with suspected or confirmed covid-19 increases personal risk of acquiring this disease. however, the risks of severe disease are probably lower than to retired clinician volunteers, more susceptible to complications of covid-19 owing to their age. [22] since the personal risks cannot be eliminated, there is predominant agreement that involvement of medical students in the care of patients should be voluntary. [5, 6] medical students that work as volunteers, must have appropriate training, do not undertake any activity beyond their level of competence, have continuous supervision and adequate personal protective equipment. [5, 6] one implication of our study for medical education is that allowing students to participate in pandemic efforts reinforces important values, such as altruism, service in times of . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint crisis, and solidarity with the profession and disposition to serve society. [12] it will probably influence the development of professional values and identity. our study showed that medical students who believe that they must participate in the fight against covid-19 pandemic are motivated by sense of purpose or duty, altruism, perception of good performance and values of professionalism more than their interest in learning. these results have implications in the developing of programs of volunteering and in the design of health force policies in the present pandemic and in future health emergencies. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint the authors declare no conflicts of interest related to this study. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 30, 2020. beliefs towards a participation of medical students in covid-19 pandemic healthcare s10. it is the duty of the medical student to put himself at the service of the population in the pandemic beliefs against a participation of medical students in covid-19 pandemic healthcare s15. distance learning must be implemented in the suspension of academic activities . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted may 30, 2020. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted may 30, 2020. . https://doi.org/10.1101/2020.05.28.20115956 doi: medrxiv preprint supplemental table 3 : crude odds ratios (95% confidence intervals) for the association between internship students' characteristics and perceptions and their view about the role of medical students during the covid-19 pandemic. only students in internship should participate all students should participate clinical characteristics of coronavirus disease 2019 in china medical student education in time of covid-19 guidance on medical students' clinical participation: effective immediately. association of american medical colleges medical schools council of united kingdom. statement of expectation. medical student volunteers in the nhs the role of medical students during the covid-19 pandemic understanding and assessing the motivations of volunteers: a functional approach the quality of volunteers' motives: integrating the functional approach and self-determination theory gender differences in risk assessment: why do women take fewer risks than men? in their own words: stressors facing medical students in the millennial generation gender differences in the perception of quality of life during internal medicine training: a qualitative and quantitative analysis assessing gender bias in key: cord-033594-oy70eya0 authors: garcía-montalvo, iván antonio title: covid-19: cambios forzados en la formación de estudiantes de medicina date: 2020-10-09 journal: nan doi: 10.1016/j.edumed.2020.09.011 sha: doc_id: 33594 cord_uid: oy70eya0 nan j o u r n a l p r e -p r o o f with the emergence of the covid-19 pandemic, the implementation of social distancing, the interruption of the educational sector with contagion control policies that forced the closure of schools and universities, thus implementing the partial restructuring of study plans and evaluations, specific case medicine, where the programs are divided into two components: initial preclinical university and clinical in the health field, this pandemic forced the students into a learning environment in which neither they nor the teachers were prepared. online evaluations were implemented, and in some cases cumulative and formative performance was taken to evaluate them. in the case of online evaluations, there is a concern for honesty and fairness as they are not supervised 1 . these changes in andragogy require preclinical students to meet in tutoring groups, employing problembased learning and simulated patient interactions, while clinical students must seek access to patient care centers. lecture-based teaching is not always easily delivered online, small group interactive sessions and clinical exposure are not always feasible, driven by connection issues. exposing senior medical students to accelerated graduation and a frontline placement can exacerbate poor health as well as lead to mental problems. during the academic program, students are introduced to healthcare settings in a controlled manner. throughout their clinical links, they operate passively, taking histories and observation procedures, the integration of medical education in the care process, in the current circumstances, clearly demonstrates their inefficiency. the programs tried to create a stable learning environment, however, it must be ensured that students have the necessary skills to develop adequately 2 . poor educational restructuring due to the spread of covid-19 can be detrimental to the training of medical students, reflecting the need for a change in the paradigms of medical education, modifying current curricula in favor of medical training in the face of new possible pandemics. this work has not received any type of funding. fighting covid-19: enabling graduating students to start internship early at their own medical school medical students and covid-19: the need for pandemic preparedness the authors declare no conflict of interest. key: cord-332583-5enha3g9 authors: bodine, erin n.; panoff, robert m.; voit, eberhard o.; weisstein, anton e. title: agent-based modeling and simulation in mathematics and biology education date: 2020-07-28 journal: bull math biol doi: 10.1007/s11538-020-00778-z sha: doc_id: 332583 cord_uid: 5enha3g9 with advances in computing, agent-based models (abms) have become a feasible and appealing tool to study biological systems. abms are seeing increased incorporation into both the biology and mathematics classrooms as powerful modeling tools to study processes involving substantial amounts of stochasticity, nonlinear interactions, and/or heterogeneous spatial structures. here we present a brief synopsis of the agent-based modeling approach with an emphasis on its use to simulate biological systems, and provide a discussion of its role and limitations in both the biology and mathematics classrooms. agent-based models (abms) are computational structures in which system-level (macro) behavior is generated by the (micro) behavior of individual agents, which may be persons, cells, molecules or any other discrete quantities. typical abms contain three elements: agents, an environment, and rules governing each agent's behavior and its local interactions with other agents and with the environment. decades of advancement in computer power has made agent-based modeling a feasible and appealing tool to study a variety of complex and dynamic systems, especially within the life sciences. as the use of abms in research has grown, so too has the inclusion of abms in life science and mathematical modeling courses as a means of exploring and predicting how individual-level behavior and interactions among individuals lead to system-level observable patterns. abms are now one of the many types of models students studying the life sciences or applied mathematics should encounter in their undergraduate education. prior to the introduction of abms into biological and applied mathematics curricula, the clear model format of choice was the ordinary differential equation (ode), or maybe a pair of them; occasionally, discrete difference equations and/or matrix equations would also be introduced. exponential growth and decay were ready examples, paving the way for extensions of the exponential growth process toward a carrying capacity in the form of the logistic growth process (voit 2020) . this logistic process was easily generalized to two populations, which were at first independent, but then allowed to interact. depending on these interactions, the result was a pair of two populations competing for the same resource or a simple predator-prey model in the format of a two-variable lotka-volterra system. although odes and any other types of "diff-e-qs" are a priori dreaded by almost all but mathematicians and physicists, the concept of an ode, if adequately explained, becomes quite intuitive. for instance, one may ease a novice into the world of odes by considering changes in the water level w of a lake over time (ayalew 2019) . whereas these dynamics are difficult to formulate as an explicit function w (t), newcomers readily understand that changes in the water level depend on influxes from tributaries, rain, and other sources on the supply side, and on effluxes, evaporation and water utilization on the side of reducing the amount of water. just putting these components into an equation leads directly to a differential equation of the system (weisstein 2011 ). on the left side, one finds the change over time as dw /dt, and this change is driven, on the right-hand side, by a sum of augmenting and diminishing processes. there is hardly a limit to what can be achieved with odes in biology, with the very important exception of processes that have genuine spatial features. and while it is not difficult to ease a biology undergraduate into ordinary differential equations, the same is not necessarily true for partial differential equations (pdes). however, spatial phenomena in biology seldom occur in homogeneous conditions. as examples, consider the formation of tumors with angiogenesis and necrosis; the local patterns of cell-to-cell signaling that governs the embryonic development; the spread of the red fire ant (solenopsis invicta) from mobile, al, its alleged port of entry into the usa, all along the gulf and east coasts; or the population size and dynamics of the santa cruz island fox (urocyon littoralis santacruzae) being driven by territory size which in turn depends on local vegetation (scott 2019) . until relatively recently, the conundrum of space was often dealt with in the final chapter of mathematical modeling in biology. a sea change came with the development of abms, which are natural formats for both stochasticity and spatial phenomena. by their nature, these models are computationally expensive, which initially prevented their use in most classrooms. however, this situation has obviously changed. as the bio2010 report (2003) stated: "computer use is a fact of life of all modern life scientists. exposure during the early years of their undergraduate careers will help life science students use current computer methods and learn how to exploit emerging computer technologies as they arise." classroom use of abms has thus become not just logistically feasible, but also very appealing for demonstrating spatial dynamics in a wide range of biological systems (kottonau 2011; triulzi & pyka 2011; shiflet 2013; pinder 2013) . supporting this appeal is a repertoire of software tools, such as simbio and netlogo (see sect. 3), that contain predefined examples and require minimal computer coding skills for model analysis. here, we present a brief synopsis and history of this modeling approach with emphasis on life science applications (sect. 2), describe some of the software tools most frequently used in the classroom (sect. 3), and then focus on some of its roles and limitations in the classroom (sect. 4). the origins of agent-based modeling. the true origins of any method or procedure are seldom identifiable in an unambiguous manner. in the case of agent-based modeling, one could think of craig reynolds' 1987 seminal article on the formation of bird flocks (with the agents denoted as boids, short for "bird-oid object"), which he was able to represent with just three rules of behavior: (1) avoid collisions with nearby birds; (2) attempt to match the velocity of nearby birds; and (3) attempt to stay close to nearby birds in the flock (reynolds 1987; gooding 2019) . the result of simulations with this simple abm was very realistic-looking flocking behavior. particularly intriguing in this study was the fact that there was no leader or a global organizing principle. instead, the virtual birds were truly individual agents that self-organized locally, thereby generating a globally coherent flight pattern. while reynolds' work was a milestone, key concepts leading to modern abms can be found much earlier. one notable contributor of ideas was nobel laureate enrico fermi, who used mechanical addition machines to generate probabilities for stochastic models with which he solved otherwise unwieldy problems (gooding 2019) . this procedure was an early form of the method of a monte carlo simulation, which was later independently developed and published by stanislav ulam, like fermi a member of the manhattan project (metropolis & ulam 1949; metropolis 1987) . another very important contribution to the budding development of abms was the turing machine (turing 1936) , which is a mathematical model of computation that uses a set of rules to manipulate symbols in discrete cells on an infinite tape. much closer to abms were ideas of ulam, who was fascinated by the "automatic" emergence of patterns in two-dimensional games with very simple rules (ulam 1950; metropolis 1987) . together with the new concept of game theory (von neumann & morgenstern 1944) , all these ideas were developed into the concept of cellular automata, which are direct predecessors of abms (ulam 1950; ulametal 1947; von neumann & morgenstern 1944) . a very appealing implementation of a cellular automaton was john conway's famous game of life (gardner 1970) . the social sciences adopted computational methods in the 1960s for microanalytic simulations or, simply, micro-simulations (gilbert & troitzsch 2005; gooding 2019 ). in contrast to today's abms, which use simple rules to recreate observed or unknown patterns, the agents in the original microsimulations acted according to empirical data (bae 2016). the advantage of this strategy is that model analysis can reveal systemic behaviors under different realistic scenarios (gooding 2019) . a seminal paper in this context described generic segregation processes (schelling 1971) . other agent-based modeling work in sociology and economics was gleaned from the biological abm work of smith (1982) , who formulated darwin's ideas of evolution as a computer simulation. this idea inspired nelson & winter to apply similar concepts and implementations to market studies, where firms were modeled like animals that followed specific routines. in particular, the firms were in competition, and the market weeded out bad routines while rewarding the fittest. nelson & winter's influential book an evolutionary theory of economic change (nelson & winter 1982) strongly proposed the use of computer simulations, which today would fall within the scope of agentbased modeling. their ideas led to a school of thought called evolutionary economics (hanappi 2017 ). an early and particularly influential paper in this context tried to shed light on the stock market (palmer 1994) . initially, abms of artificial life simulated simple homogeneous agents that acted like huge colonies of ants that could just move and eat in the pursuit of food. somewhat more sophisticated, economic simulations used as the main agent homo economicus, a consistently rational human pursuing the optimization of some economic goal or utility with exclusive self-interest (persky 1995) . based on these humble beginnings, sophistication in computing soon permitted heterogeneous agents and much more complicated landscapes than before. the successes in economics were so tantalizing that simulation studies eventually reached the most prestigious journals of economics and the social sciences geanakoplos 2012; hanappi 2017) . modern abms in economics are capable of capturing much of the complexity of macroeconomic systems (e.g., caiani (2016) ). following directly the principles of cellular automata, kauffman studied large grids with elements that changed features in a binary fashion (kauffman 1993) . for instance, a white agent could turn black, and this shift occurred according to boolean rules that usually involved some or all neighboring grid points. kauffman was able to demonstrate the emergence of complex patterns, such as oscillations and percolation. starting in the 1980, wolfram performed systematic studies of cellular automata, which led to his influential 2002 book a new kind of science (wolfram 2002 ) that assigns cellular automata a wide range of applications in a variety of fields. biological applications of agent-based modeling. abms have been constructed to study a wide range of biological phenomenon. numerous reviews and research efforts using abms have focused on specific biomedical systems. issues of gene expression were modeled by thomas (2019) . morphogenetic and developmental processes were discussed in grant (2006) , robertson (2007) , , tang (2011), and glen (2019) . models of tissue mechanics and microvasculature are captured in and van liedekerke (2015) . inflammation, wound healing and immune responses were addressed in , ), chavali (2008 , and castiglione & celada (2015) . other authors (wang 2015; segovia 2004; cisse 2013 ) used abms to model cancer growth, tuberculosis, and schistosomiasis, respectively. lardon and colleagues (lardon 2011) used abms to analyze biofilm dynamics. butler and colleagues described the use of abms in pharmacology (butler 2015) . abms studying multicellular systems provide a unique capability to examine interactions and feedback loops across the different hierarchies of the system (hellweger 2016) . reviews of abms in the context of ecology, environmental management, and land use include (bousquet 2004; matthews 2007; grimm & railsback 2005; caplat 2008 ; deangelis & diaz 2019). in some applications, interventions or treatments were addressed and therefore required the adaptation of agents to changing scenarios (berry 2002) . abms have also been used to simulate epidemics with analyses examining the impact of implemented or potential intervention measures (e.g., quarantining/physical distancing, mask wearing, and vaccination) (mniszewski 2013; perez & dragicevic 2009; tracy 2018) . visual representations of epidemiological abms have even been used by news outlets during the covid-19 pandemic to help explain to the public how various intervention methods change the shape of an epidemic (i.e., "flatten the curve) or the basic reproduction number (r 0 ) of an epidemic; see, for example, fox (2020) and stevens (2020). rationale & pitfalls of abms. the two most frequent goals of an abm analysis are (1) the elucidation and explanation of emergent behaviors of a complex system and (2) the inference of rules that govern the actions of the agents and lead to these emerging system behaviors. this type of inference is based on large numbers of simulations, i.e., replicate experiments with the abm using the same assumptions and parameters, and different experiments over which assumptions or parameter values are systematically changed. simulations of abms essentially always yield different outcomes, because movements, actions and interactions of agents with each other or with the environment are stochastic events. the inference of rules from simulation results is an abductive process (voit 2019) that is challenging, because one can easily demonstrate that different rule sets may lead to the emergence of the same systemic behaviors, and because even numerous simulations seldom cover the entire repertoire of a system's possible responses. in fact, hanappi (2017) warned: "assumptions on microagents that play the role of axioms from which the aggregate patterns are derived need not-and indeed never should-be the end of abm research." arguably the greatest appeal of abms, and at the same time a treacherous pitfall, is their enormous flexibility, which is attributatble to the fact that any number of rules can be imposed on the agents, and that the environment may be very simple but can also be exceedingly complicated. for instance, the environment may exhibit gradients or even different individually programmable patches (barth 2012; gooding 2019) including importing geographic information systems (gis) data to define the characteristics of each patch (see scott (2019) for an example with a detailed explanation of how gis data are incorporated into an abm). in particular, the repeated addition of new elements to a model can quickly increase the complexity of the model, thereby possibly distracting from the core drivers of the system's behavior, obscuring the importance of each of the rules the agents must follow, and generally making interpretations of results more difficult. related to this option of adding elements is the critique that abms can be 'tuned' by researchers to create outcomes that support the researcher's narrative (gooding 2019) . to counteract increases in complexity, some authors have begun to develop methods of model reduction that retain the core features of a model, but eliminate unnecessary details (zou 2012). another critique of abms is that simulations are not readily reproducible, because the rules can be complicated and stochasticity rarely repeats the same model trajectories. a strategy to increase the reproducibility of abms was the establishment of protocols for the standardized design and analysis of abms (lorek & sonnenschein 1999; grimm 2006 grimm , 2010 heard 2015) . (2017). in a slight extension of abms, lattilä (2010) and cisse (2013) described hybrid simulations involving abms and dynamic systems, and heard (2015) discussed statistical methods of abm analysis. these introductory tutorials and reviews, however, are typically not designed for undergraduates with limited mathematical or computational modeling experience. the scientific community has also worked hard on facilitating the use of abms by offering software like netlogo, swarm, repast, and mason. summaries and evaluations of some of the currently pertinent software are available in berryman (2008) and abar (2017) , and netlogo is described more fully in sect. 3. a variety of software tools can be used to construct, simulate, and analyze abms. when abms are taught or used in biology or mathematics courses, software should be chosen to align with pedagogical objectives. note that the pedagogy of abms in life science and mathematical modeling courses is discussed in more detail in sect. 4. in this section, we highlight some of the most used software packages in an educational setting. ecobeaker & simbio. ecobeaker was developed by eli meir and first released in 1996 as software that ran simulated experiments designed to explore ecological principles. in 1998, simbio (meir 1998 ) was founded (then called beakerware) for the release of the second version of ecobeaker and has since grown to include simulated experiments in evolution, cell biology, genetics, and neurobiology. many of the simulated experiments in the simbio virtual labs software are agent-based simulations. in a simbio virtual lab, the user interacts with a graphical interface portraying the agents (individuals in the experiment) and sometimes their environment, which can be manipulated in various ways for different experimental designs. the graphical interface also contains relevant graphs which are updated as the simulated experiment runs. ecobeaker and simbio virtual labs are examples of software where the focus is on experimental design and the use of simulation to understand biological concepts. the user never interfaces with the code and does not need to understand the underlying algorithms which produce the simulation. simbio virtual labs are used in many biology classrooms across the united states. the software is priced per student (currently $6/lab/student or $49/student for unlimited labs). netlogo. the agent-based modeling environment netlogo (wilensky 1999) was developed by uri wilensky and first released in 1999. netlogo is free and continues to be improved and updated (the software is currently on version 6). additionally, a simplified version of netlogo can be run through a web browser at http://netlogoweb. org/. netlogo is a programming platform allowing the implementation of any abm a user might design. as such, its user interface includes a tab where abm code is written in the netlogo programming language, and a tab for the user to view a visualization of the abm and user specified outputs as the abm is simulated. textbooks by railsback & grimm (2012) and wilensky & rand (2015) provide introductions to the netlogo prgramming languages as well as providing a thorough overview of the algorithmic structures of abms. since its initial development, netlogo has built up an extensive model library of abms. additionally, over the years, faculty at various institutions have developed abm modules through netlogo that allow students to explore a variety of biological phenomenon. for example, the virtual biology lab (jones 2016) has created 20 different virtual laboratory modules using netlogo through a web browser for exploring topics in ecology, evolution, and cell biology. the virtual biology labs are similar in scope to the ecobeaker and simbio labs. another example is infections on networks (iontw) which provides an abm framework and teaching modules for examining aspects of disease dynamics on various network structures (just 2015a, b, c) . one of the responses to the bio2010 report (2003) has been a push to create biocalculus courses or to insert more biological application examples and projects within traditional calculus courses. indeed, studies have shown that including applications from the life sciences in classic math courses like calculus leads to students gaining equivalent or better conceptual knowledge than from similar courses without life science applications (comar 2008; . however, many mathematics and biology educators have pointed out that the subset of mathematics applicable to biology extends well beyond calculus, and undergraduates (especially those majoring in biology) should be exposed to a variety of mathematical models and methods of analysis across biology and mathematics courses (bressoud 2004; gross 2004; robeva & laubenbacher 2009 ). the only prerequisites for analyzing the simulations of an abm are a basic understanding of the underlying biology and, in some instances, knowledge of how to perform basic statistical calculations or generate graphical representations of results. many pre-built abms in simbio virtual labs and netlogo generate relevant graphs and/or produce spreadsheets or text files of the relevant data. thus, a student can utilize abms in life science courses without having to learn how to implement (by writing code) an abm. the prerequisites for learning how to implement abms are not as extensive as for other forms of mathematical models. the most essential prerequisite is some exposure to the fundamentals of computer programming such as understanding loop structures and conditional statements, implementing stochastic processes, and understanding how the order of executed operations within a program impacts the program's output. agent-based modeling software like netlogo (wilensky 1999 ) keeps model implementation and analysis relatively simple by providing built-in model visualization tools and automatically randomizing the order in which agents execute programmed operations. due to their appealing visual representation, abms can easily be used in the classroom to demonstrate biological processes ranging from chemical reactions to interactions among species, as if the model were simply an animation. however, using abms in this way is much like using a cell phone to hammer nails (q.v., theobald (2004) ): it may work in the desired fashion, but represents an utter waste of the tool's real potential. adding just one more step, the collection of model-generated data, transforms a passive learning experience into an active one. students can be asked to calculate means and variances, graph relationships between variables, discuss the sample size needed for reliable results, and generate quantitative predictions under different hypotheses, all in the context of a specific biological question. this can be done even in a large classroom with only a single, instructor-controlled computer, bridging the gap between lecture and lab. if students have access to individual computers, much more is possible. either individually or in small groups, students can use abms to collect and analyze their own data. free file-sharing resources such as googledocs make it easy to pool data across many groups, thereby crowd-sourcing problems that would be too large for any one group to handle on their own. in smaller classes and lab sections, individuals or groups can be assigned to model different scenarios (e.g., the interaction effects between different parameters), prompting discussions of the most appropriate parameter values and model settings. such models can even be extended into miniature research projects. for example, in a unit on community ecology, students might be assigned a question about two interacting species, then use online resources to find relevant information and parameter estimates, design and conduct a series of model runs, analyze their data using simple statistical techniques, and present their findings to the class. although abms can be used to simulate almost any biological process, meaningful exploration of a model typically requires a substantial commitment of class time and instructor engagement. as a result, except in modeling courses, it is seldom practical to incorporate abms into every lesson plan. in our experience, their educational value is highest for studying processes involving (a) substantial amounts of stochasticity, (b) nonlinear interactions, and/or (c) a defined spatial structure. the inclusion of abms in math courses generally comes in two different modes: (1) abms are taught as one modeling technique in a course covering multiple modeling techniques, and (2) the construction and analysis of abms are taught in a course where abms are the only type of modeling being used. however, due to the minimal prerequisites for learning agent-based modeling, both types of courses can potentially be offered with one or no prerequisite courses. bodine (2018) provides an example of a course where abms are taught as one type of discrete-time modeling technique in an undergraduate course designed for biology, mathematics, biomathematics, and environmental science majors that has no prerequisites beyond high-school algebra. an example of a course where abms are the only type of modeling being used is given by bodine (2019); this course has only a single prerequisite: either a prior math modeling course which introduces basic computer programming or an introduction to computer science course. the use of biological applications in teaching mathematical modeling (including modeling with abms) is often viewed as having a lower entry point with less new vocabulary overhead than other types of applications (e.g., those from physics, chemistry, or economics). in particular, models of population dynamics, even those involving interactions between multiple subpopulations or different species, usually do not require any new vocabulary for most students, which allows for a more immediate focus on mechanisms of population change and impact of interactions between individuals within the populations. video game vs. scientific process. in our experience, students sometimes react to an abm's many controls and visual output by treating the model as a video game, clicking buttons at random to see what entertaining patterns they can create. other students prefer to complete the exercise as quickly as possible by blindly following the prescribed instructions. neither of these approaches substantially engages students in thinking about the question(s) underlying the model, different strategies for collecting and analyzing data, or the model's limitations. to foster these higher-order cognitive skills, use of the abm should be explicitly framed as an example of the scientific process. this approach begins with a set of initial observations and a specific biological question. for example, what management practices would be most effective in controlling the invasive brown tree snake? after familiarizing themselves with the biological system, students propose hypotheses about the factors contributing to the snake's success on guam, suggest management strategies, and set model parameters that reflect their chosen strategy. finally, they run the model multiple times to collect data that allow them to measure their strategy's effectiveness. under this pedagogical approach, abms become a vehicle for design-ing and conducting a miniature research project, enabling experiments that would not otherwise be practical due to cost, logistics, or ethical considerations. the modeling exercise can also reinforce lessons on how scientific knowledge is constructed and tested (e.g., the three p's of science, namely, problem posing, problem solving, and peer persuasion (watkins 1992) ). as part of this exercise, students should engage in the process of deciding how best to collect, analyze, and present their data. for example, as part of the brown tree snake project, students might be asked to explore the practical steps that other pacific islands could take to prevent invasions or eradicate invaders. one group of students decides to focus on two different control measures: cargo checks of all inbound flights and deployment of poison baits around airstrips. following an overview of different statistical approaches, the students determine that a multiple regression analysis would best allow them to address their question. allowed only a limited 'budget' of 30 model runs, students settle on a factorial design using three treatment levels of cargo checks, three levels of baiting, and three replicates of each combination. the students set up a spreadsheet to record the data from each model run, graph their data in a scatter plot, and use software such as microsoft excel's analysis toolpak to conduct their analysis. a model as a caricature of the real world. students at early stages of their academic career often envision science as a collection of factual information and fixed procedures. students with this mindset may dismiss as useless any model that does not incorporate every detail of a particular biological system. by contrast, scientists recognize that models, whether mathematical, physical, or conceptual, are deliberate simplifications that attempt to capture certain properties of a biological system while ignoring others (dahlquist 2017) . for example, the standard epidemiological sir model (diagrammed in fig. 1a ) divides a population in three subpopulations (susceptible, infectious, and removed) while ignoring any potential heterogeneity within each subpopulation (e.g., age, treatment status, groups at higher risk for transmission). students will need to engage in activities that frame the abm as a hypothesis about the organization and function of a specific biological system (weisstein 2011) . after a description (and possible exploration) of the basic model, students can work in groups to suggest additional processes and variables that seem relevant to understanding the system. they can then choose one or two of the factors that they consider most important to addressing the question being asked. finally, they should consider how to modify the model to incorporate the chosen features. for example, a standard epidemiological model divides the host population into susceptible, infectious, and removed subpopulations, and studies the movement of individuals among these subpopulations (fig. 1a) . a group of students decides to modify this model to track a malarial epidemic. after discussing mortality rates, prevention and treatment options, and genetic and age-related variation in host susceptibility, the students decide to focus on incorporating vector transmission into their model. through guided discussion with the instructor, they realize that transmission now occurs in two directions: from infected vectors to susceptible hosts and from infected hosts to uninfected vectors. they therefore develop a schematic model (fig. 1b) that depicts these revised rules for each agent in the abm. even if the students do not actually build the corresponding computational model, this exercise in extending a model to reflect specific biological assumptions helps students understand the iterative process by which models are developed and the utility of even simple models to clarify key features of the system's behavior. algorithms vs. equations. the concept of an equation is introduced fairly early in mathematics education. in the united states, children can encounter simple algebraic equations in elementary school (common core 2019) and then continue to see increasingly complex equations in math classes through college. because of this long exposure to equations, the use of functions and systems of equations to model systems in the natural world feels "natural" or logical to students when they first encounter differential equation models or matrix models. abms, on the other hand, can seem confusing to students because they lack the ability to be expressed as an equation or set of equations. an abm is constructed as an algorithm describing when and how each agent interacts with their local environment (which may include other agents). often these interactions are governed by stochastic processes, and thus "decisions" by agents are made through the generation of random numbers. when first introducing students to abms, it can be helpful to teach students how to read and construct computer program flowcharts and to create a visual representation of what is occurring within an algorithm or portion of an algorithm (see bodine (2019) for example assignments that utilize program flowchart construction). in life science classes where abms are being analyzed but not constructed and implemented, a flow diagram can be a useful tool for conveying the order processes occur in the model. class discussions can question whether the order of processes make biological sense, and whether there are alternatives. in math modeling classes, the construction of flowcharts, even for simple abms, can help students elucidate where decision points are within the code, and what procedures are repeated through loop structures. the construction of flowcharts as students progress to more complicated abms can help students rectify the order of events in their implemented algorithm against the order in which events should be occurring biologically. whether students are working with abms in life science or math modeling classes, it is helpful for them to learn how to read and understand flow diagrams as they are often included in research publications that use agent-based modeling. describing an abm. much to the alarm of many math students beginning to develop abms, the formal description of an abm requires more than just writing the computer code. the standard method for describing abms in scientific publications, referred to as the overview, design concepts, and details (odd) protocol (grimm 2006 (grimm , 2010 railsback & grimm 2012) , often requires more than one page to fully describe even a simple abm. this can make abms seem overwhelming to students as they first begin to explore abms. in courses which teach the implementation and description of abms, instructors should take care not to introduce the computer code implementation simultaneous to the model description via the odd protocol. note that the introductory text on agent-based modeling by railsback & grimm (2012) does not introduce the concept of the odd protocol until chapter 3, which comes after the introduction and implementation (in netlogo) of a simple abm in chapter 2. in the course materials by bodine (2019) , the concept of the odd protocol is introduced prior to project 2, but the students are not required to write their own odd protocol description until the final project, once they have seen odd descriptions for multiple models. model implementation vs. the modeling cycle. courses that aim to teach methods in mathematical modeling often start with a discussion of the modeling cycle, which is typically presented as a flow diagram showing the loop of taking a real world question, representing it as a mathematical model, analyzing the model to address the question, and then using the results to ask the next question or refine the original real world question. figure 2 shows an example of a modeling cycle diagram. in courses where the mathematical models are encapsulated in one or a small handful of equations, the time spent on representing the real world as a mathematical model (the green box in fig. 2) is relatively short. the construction of abms, however, can be a fairly lengthy process, as abms are designed to simulate interactions between individuals and the local environment. when students are in the middle of constructing their first few abms, they often lose sight of where they are in the modeling cycle because the model implementation becomes a cycle of its own; a cycle of writing bits of code, testing the code to see if it runs and produces reasonable results, and repeating this process to slowly add all the components needed for the full algorithm of the abm. as students are first learning agent-based modeling, they need to be reminded often to pull back and view where they are in the modeling cycle; to see the flock for the boids, as it were. model validation. within the modeling cycle, there is a smaller cycle of model validation (see dashed line in fig. 2) . in a course where students are first introduced to the classic lotka-volterra predator-prey model, the students are usually first introduced to a predator-prey data set (like the 200-year data set of canadian lynx and snowshoe hare pelts purchased by the hudson bay company (maclulich 1937; elton & nicholson 1942) ), which shows the oscillating population densities of the predator and prey populations. when the students then simulate the lotka-volterra model for various parameter sets, they find that they are able to produce the same oscillating behavior of the predator and prey populations. this is a form of model validation, and a model which did not display this distinctive trend seen in the data would be considered invalid for that data set. a similar process must occur for validating abms against observed biological patterns. however, in order to engage in this validation process for abms, students must first understand how decisions of individual agents and interactions between neighboring agents can lead to system-level observable patterns, a phenomenon referred to as "emergence" or the "emergent properties" of an abm. the classic abm example for easily identifying an emergent property is a flocking example (a stock example of flocking exists in the netlogo models library, and is explored in chapter 8 of the abm textbook by railsback & grimm (2012) ). the concept of an emergent property can take a little while for students to fully understand. by definition, it is an observable outcome that the system was not specifically programmed to produced. in particular, it is not a summation of individual-level characteristics, and is typically not easily predicted from the behaviors and characteristics of the agents. for example, a variation of the reynolds (1987) flocking model is included in the netlogo library and is explored in railsback & grimm (2012, chapter 8 ). in the model, each agent moves based on three rules: 1. separate: maintain a minimum distance from nearby agents 2. align: move in the same direction as nearby agents 3. cohere: move closer to nearby agents where all agents move at the same speed and different schemes for determining who is a nearby agent can be used. additionally, there are model parameters for the minimum distance to be maintained between agents, and the degree to which an agent can turn left or right in a single time step in order to align, cohere, and separate. it is not immediately evident from this set of rules that individual agents might be able to form flocks (or swarm together), and indeed that system-level behavior does not emerge for all parameter sets. however, certain parameter sets do lead to the agents forming one or more flocks that move together through the model landscape. students observations: algorithms for pattern recognition. one of the most exciting moments for students when they first begin running simulations of an abm is seeing systemlevel patterns emerge before their eyes. one of the challenges for students (and agentbased modelers, in general) is to develop algorithms to identify and/or quantify the patterns we can easily identify by sight. for example, in the flocking abm discussed above, an observer watching the locations of individual agents change at each time step can easily see the formation of flocks. if however, the observer wanted to systematically explore the parameter space of the flocking abm and determine the regions of the parameter space under which flocking occurs (a process which might involve running hundreds or thousands of simulations), it would be a tedious and time-consuming task to physically watch each simulation and record whether the agents formed a flock or not. instead, the observer must choose the criteria that indicate the formation of a flock, and then determine a measure or an algorithm for determining whether a flock (or multiple flocks) have formed. in a course designed to teach the construction and analysis of abms, this is a point where students should be encouraged to be both creative and methodical about developing such measures and algorithms. the development of these observational measures and algorithms also provides a great opportunity for collaboration between students. it is especially helpful if students with a diversity of academic backgrounds can be brought together to brainstorm ideas; for instance, mixing students with various levels of exposure in mathematics, computer science, and biology can be very beneficial. rapid advances in computing power over the past decades have made agent-based modeling a feasible and appealing tool to study biological systems. in undergraduate mathematical biology education, there are multiple modes by which abms are utilized and taught in the classroom. in biology classrooms, abms can be used to engage students in hypothesis testing and in the experimental design and data collection processes of otherwise infeasible experiments, and to enable students to utilize models as a part of the scientific process. all of this can be done without students having to learn a programming language. by contrast, students who have had some exposure to computer programming can learn the construction, implementation, and analysis of agent-based models in a math or computer science modeling class. biological applications are ideal systems for first attempts at agent-based models as they typically do not necessitate learning extensive new vocabulary and theory to understand the basic components that need to be included in the model. throughout this article, we endeavored to articulate the benefits and challenges of including abms in undergraduate life science and math modeling courses. consistent with the bio2010 report (2003), we recommend that undergraduate biology and life science curricula be structured to ensure that all students have some exposure to mathematical modeling. we additionally recommend that this includes agent-based modeling. while not every student necessarily needs to take a course exclusively focused on agent-based modeling, every undergraduate biology student should have the opportunity to utilize an abm to perform experiments and to collect and analyze data. as we educate the next-generation of life scientists, let us empower them with the ability to utilize abms to simulate and better understand our complex and dynamic world. agent based modelling and simulation tools: a review of the state-of-art software in silico experiments of existing and hypotheticalcytokine-directed clinical trials using agentbased modeling agent-based models in translational systems biology aligning simulation models: a case study and results integration of biology, mathematics and computing in the classroom through the creation and repeated use of transdisciplinary modules. primus, in revision bae jw multi-cell agent-based simulation of the microvasculature to study the dynamics of circulating inflammatory cell trafficking typical pitfalls of simulation modeling-lessons learned from armed forces and business adaptive agents, intelligence, and emergent human organization: capturing complexity through agent-based modeling land operations division dsto, defence science and technology organisation, po box 1500 agent-based modeling: methods and techniques for simulating human systems multi-agent simulations and ecosystem management: a review the changing face of calculus: first-and second-semester calculus as college courses mathematics for the life sciences discrete math modeling with biological applications (course materials) agent-based modeling course materials agentbased modeling in systems pharmacology agent based-stock flow consistent macroeconomics: towards a benchmark model symmetric competition causes population oscillations in an individualbased model of forest dynamics immune system modelling and simulation characterizing emergent properties of immunological systems with multicellular rule-based computational modeling assessing the spatial impact on an agent-based modeling of epidemic control: case of schistosomiasis the integration of biology into calculus courses common core state standards initiatives an invitation to modeling: building a community with shared explicit practices decision-making in agent-based modeling: a current review and future prospectus the case for biocalculus: design, retention, and student performance the ten-year cycle in numbers of the lynx in canada a (2020) how epidemics like covid-19 end (and how to end them faster) mathematical games-the fantastic combinations of john conway's new solitaire game getting at systemic risk via an agent-based model of the housing market review: agent-based modeling of morphogenetic systemsadvantages and challenges agent-based model history and development. in: economics for a fairer society-going back to basics using agent-based models simulation of biological cell sorting using a two-dimensional extended potts model simulating properties of in vitro epithelial cell morphogenesis a standard protocol for describing individual-based and agent-based models the odd protocol: a review and update individual-based modeling and ecology interdisciplinarity and the undergraduate biology curriculum: finding a balance a systematic review of agent-based modelling and simulation applications in the higher education domain agent-based modelling. history, essence, future the history, philosophy, and practive of agent-based modeling and the development of the conceptual model for simulation diagram. doctoral dissertation a survey of agent-based modeling practices agent-based models and microsimulation advancing microbial sciences by individual-based modelling cellular potts modeling of complex multicellular behaviors in tissue morphogenesis virtual biology lab: an inquiry-based learning environment transmission of infectious diseases: data, models, and simulations disease transmission dynamics on networks: network structure versus disease dynamics an interactive computer model for improved student understanding of random particle motion and osmosis idynomics: next-generation individual-based modelling of biofilms hybrid simulation models-when, why, how? modelling and simulation software to support individual-oriented ecological modelling tutorial on agent-based modelling and simulation fluctuations in the numbers of the varying hare (lepus americanus) agent-based land-use models: a review of applications the beginning of the monte carlo method the monte carlo method episims: large-scale agent-based modeling of the spread of disease bio2010: transforming undergraduate education for future research biologists. the national academies press an evolutionary theory of economic change agent-based computing from multi-agent systems to agent-based models: a visual survey artificial economic life: a simple model of a stockmarket retrospectives: the ethology of homo economicus an agent-based approach for modeling dynamics of contagious disease spread an active learning exercise for introducing agent-based modeling agent-based and individual-based modeling: a practical introduction flocks, herds and schools: a distributed behaviour model multiscale computational analysis of xenopus laevis morphogenesis reveals key insights of systems-level behavior mathematical biology education: beyond calculus identifying control mechanisms of granuloma formation during m. tuberculosis infection using an agent-based model dynamic models of segregation an agent-based model of the spatial distribution and density of the santa cruz island fox. integrated population biology and modeling part b undergraduate module on computational modeling: introducing modeling the cane toad invasion why outbreaks like coronavirus spread exponentially, and how to "flatten the curve phenotypic transition maps of 3d breast acini obtained by imaging-guided agent-based modeling 29+ evidences for macroevolution intrinsic and extrinsic noise of gene expression in lineage trees agent-based modeling of multicell morphogenic processes during development agent-based modeling in public health: current applications and future directions learning by modeling: insights from an agent-based model of university-industry relationships intelligent machinery a heretical theory statistical methods in neutron diffusion ulam s (1950) random processes and transformations simulating tissue mechanics with agent-based models: concepts, perspectives and some novel results perspective: dimensions of the scientific method press) von neumann j, morgenstern o (1944) theory of games and economic behaviour multi-scale modeling in morphogenesis: a critical analysis of the cellular potts model teaching students to think like scientists: software enables experiments to be conducted that would be impossible in a laboratory simulating cancer growth with multiscale agent-based modeling building mathematical models and biological insights in an introductory biology course an introduction to agent-based modeling: modeling natural, social, and engineers complex systems with netlogo agent-based modeling: an introduction and primer model reduction for agent-based social simulation: coarse-graining a civil violence model publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-102632-yazl9usb authors: lobet, guillaume; descamps, charlotte; leveau, lola; guillet, alain; rees, jean-françois title: quovidi: a open-source web application for the organisation of large scale biological treasure hunts date: 2020-07-01 journal: biorxiv doi: 10.1101/2020.06.30.177006 sha: doc_id: 102632 cord_uid: yazl9usb learning biology, and in particular systematics, requires learning a substantial amount of specific vocabulary, both for botanical and zoological studies. while crucial, the precise identification of structures serving as evolutionary traits and systematic criteria is not per se a highly motivating task for students. teaching this in a traditional teaching setting is quite challenging especially with a large crowd of students to be kept engaged. this is even more difficult if, as during the covid-19 crisis, students are not allowed to access laboratories for hands-on observation on fresh specimens and sometimes restricted to short-range movements outside their home. here we present quovidi, a new open-source web platform for the organisation of large scale treasure hunts. the platform works as follows: students, organised in teams, receive a list of quests that contain morphologic, ecologic or systematic terms. they have to first understand the meaning of the quests, then go and find them in the environment. once they find the organism corresponding to a quest, they upload a geotagged picture of their finding and submit this on the platform. the correctness of each submission is evaluated by the staff. during the covid-19 lockdown, previously validated pictures were also submitted for evaluation to students that were locked in low-biodiversity areas. from a research perspective, the system enables the creation of large image databases by the students, similar to citizen-science projects. beside the enhanced motivation of students to learn the vocabulary and perform observations on self-found specimens, this system allows faculties to remotely follow and assess the work performed by large numbers of students. the interface is freely available, open-source and customizable. it can be used in other disciplines with adapted quests and we expect it to be of interest in many classroom settings. teaching biology to first-year bachelor students is a challenge. as educators, our aim is usually twofold. first, we want the students to learn a new set of knowledge and integrate it. second, and this is for us equally important, we want the students to engage with the topic at hand. we want to transmit our passion and curiosity about the topic that we teach. third, we also want students to learn to observe the world around them. it is one thing to learn a topic from a textbook, it is another to observe it in real life. however, the main issue is that the classroom is, often by design, completely disconnected from the natural world. the challenge is therefore to find a way for students to learn and engage with biology, despite that given disconnection. last but not least, in the spring semester of 2020 (january to june) it was necessary for us to adapt the learning activities to the containment measures related to covid-19. the formal aim of our biology course -given in the bioengineering faculty, uclouvain, belgium -is to discover plant and animal structures, organs and their function at the individual scale. to achieve this, students need to learn specific vocabulary related to these structures. the classic way to present this vocabulary to a student audience is to review a series of slides illustrating these different characteristics. this vocabulary is usually very boring for teachers to describe (imagine the slides showing all the different shapes of leaves) and the content is not very interesting for students to listen to either. yet this vocabulary is an important prerequisite for describing any biological structure and for later systematic identification of taxons using dichotomous keys . its learning is essential. the question is therefore how to make this learning process motivating for the students and give them the opportunity to learn over time instead of memorising a list of words? the additional difficulty is that this learning activity must be able to be set up with more than 300 students and few teaching resources. to create this learning activity, we decided to draw inspiration from all the pedagogical techniques that aim to place the student at the centre of his learning. student-centred learning and active learning emerged as important pedagogical techniques during the last century [ref] . active learning is characterised by (i) involving the student in the construction of his or her learning, (ii) engaging the student in an in-depth treatment of the subject matter, (iii) constructing learning through interaction (with the teacher or other students), (iv) conceiving of learning as the evolution of knowledge and skills [1, 2] . studies have shown that the more cognitively and socially engaged the student is in a learning task, the more perennial the learning task becomes [1, 3] . active learning improves the performance of students and acts to reduce the gap achievement between advantaged and disadvantaged students [4] . in order to stimulate learning through interaction and create a collective emulation around this activity, the idea of creating a campus-wide biological treasure hunt finally emerged from the discussions. beyond simply being active through the manipulation of information, the student has to transform and produce new information that is not provided in the learning material. gamification is another recent technique to better engage the students in a learning activity. gamification is defined by [5] as "game-based mechanics, aesthetics, and game thinking to engage people, motivate action, promote learning, and solve problems". in many studies, students' levels of engagement increased significantly following the introduction of game elements, such as points, challenges, quests or progress bar [6] . the gamified environnement can afford intrinsic motivation and engagement, which are also targeted by active learning. to assemble these different elements -biological vocabulary, observation, active learning and gamification -in a comprehensive learning activity, we created a large scale biological treasure hunt for our students. in short, we provided students with a list of specific biological vocabulary. they had to understand the list and find the different elements outside of the classroom, in the natural world. external resources (books, selected websites, wiki pages) describing this vocabulary were available to them. complexity of understanding (some words are more difficult than others) as well as the difficulty of identification in the field were rewarded with different points. to manage the treasure hunt, we designed a new web-based platform, quovidi (which would loosely translate from latin as "where did you see"), for the organisation of large scale, decentralised, biological treasure hunts. quovidi is an open-source project available at www.quovidi.xyz . the objective of this publication is to describe the project, to show how we were able to adapt this learning activity to the covid-19 crisis, and finally, to show the results and success of the activity with the students. quovidi is a web application for the organisation and management of large scale biological treasure hunts. it was created to teach students to learn new biological terms (both in zoology and botany) and to teach them to observe the natural world surrounding them. first, educators have to prepare a list of quests to find in the natural world. these quests should be tailored and adapted for the target public. for instance, in our experience with first year biology students, the quests revolved around biological structures and families (tab. 1). each quest is given a specific reward (points) depending on its intrinsic difficulty and rareness. quests can be sorted in different groups (for instance "animal" and "plant") and subgroups (for instance "animal species" and "leaf shapes") to help students navigate them. find a siphonaptera animal animal groups 3 find an example of aposematism animal animal physical attributes 1 second, educators have to assign students to groups to perform the activity. students in the same group will be able to share pictures and collaborate on the data collection. when logging into the web interface, students will be able to see the collected pictures and rewards from their own group. they will also be able to see the total number of points of the competing groups. educators also have the possibility to define specific game parameters, such as specific geographic regions in which the game takes place or restriction on the number of submissions in each quest group (adding for instance a point penalty below a certain number of "animal" or "plant" submissions). once the list of quests, users and groups are defined, the activity can start. two main activities are available for the students : an in situ treasure hunt and an ex situ photo quiz activity. the main activity of the platform is the biological treasure hunt. students have to go outside (although some of the creatures may be also found in their home such as food parasites, e.g. lepisma sp. or flies) to find the different quests setup by the educators. once they find a specific quest, they have to take a picture of it with their smartphone. we ask the student to take unambiguous pictures, where the subject of the quest is clearly identified and visible. we also ask them to leave the natural environment intact, without killing any plant or animal in the process. they can then store the picture on the quovidi web interface. when stored, pictures are automatically resized (for efficiency) and added to the activity database. localisation information and date are extracted from the picture exif metadata. any other information is erased at this step. once pictures are stored on the web interface, students can assign them to a specific quest and submit it for evaluation. the web application allows users to follow their progress in detail (which picture was submitted for which quest, what is the evaluation status, etc.) as well as the global progress of the other groups (the total number of collected points). it is worth noting that in belgium -where the web application was first usedthe lockdown due to the covid-19 pandemic still allowed citizens to go outside for some walk and exercise, although at a limited range. as such, the treasure hunt could still be performed by the students, either in their own garden or in neighbouring areas. however, not everyone lives in the countryside or close to a natural environment, or had the opportunity to leave their home during the lockdown. this is why we created a second module in the interface, the photo quiz, which allowed students to learn from photos contributed by other students, without having to submit their own photos. the second module of the interface allows students to evaluate pictures submitted by other students (a modified version of peer evaluation). more precisely, in the photo quiz module, students are presented with pictures submitted by other groups and validated by the educators (see below "expert evaluation). they have to assess whether the picture corresponds to its assigned quests. their assessment is then compared to the assessment of the educators. if it matches, the students gain points that are added to their global group tally. when performing this activity for the first time, it is necessary to have a sufficient amount of submitted (and corrected pictures). without a database large enough, the activity loses some of its interest, as students might all review the same pictures. the third important module of the interface, central to the activity, is the expert evaluation. each submitted picture needs to be manually assessed by the educators. different feedback can be given for each submission, such as "correct", "correct and nice picture", "incorrect" , "not visible" (e.g. the object is not visible in the picture) or "out of rules" (e.g. picture of a houseplant, picture taken outside of the prescribed geographical zones). the interface was designed to easily navigate the different quests and quickly correct the submitted images. the web application was created using the r shiny framework , using the shinydashboard [7] , shiny [8] , shinywidgets [9] , shinybs [10] , miniui [11] packages for the user interface design. the data is stored in a sqlite database, hosted on the server. the database management is done using the dbi [12] and rsqlite [13] packages . pictures are transformed and managed using the magick [14] package. exif information is extracted using the exifr [15] package. data manipulation and visualisation is done using the tidyverse [16] , lubridate [17] , cowplot [18] , formattable [19] , dt [20] , plyr [21] , leaflet [22] packages. the text sentiment analysis was performed using the rfeel package [23] . in our exemple, the web application was deployed on the university server with the following specifications: quovidi is an open source project, released under an apache licence [24] . everyone is free to re-use and modify it, with attribution. the interface was created to be as much user-friendly as possible so that neither students nor staff need technical training . because it is web based, it can be used on any platform, whatever the operating system . it scales on mobile devices as well, allowing users to store and submit pictures directly from the field (if they have an internet connection). figure 1 shows the different panels of the web interface. figure 1a shows the "store" panel, where students can store pictures, before submitting them for evaluation. this allows students from the same group to share and visualise their pictures. at this step, students can already assign a quest to the picture, which can be changed later on. they can also assign a geographic region, if this is required by the educators. a default region will be automatically proposed, based on the metadata of the picture. figure 1b shows the "submit" panel. at this stage, students see all the pictures from their group. they can select a stored picture, assign it to a quest and submit it for evaluation. groups can only submit one picture for each quest. or not as well as their validation status. in the same panel, students can also see the global scores of each group taking part in the activity. this adds a strong gamification aspect to the activity. figure 1d shows the "quests" panel. in that panel students can navigate through the different quests proposed by the educators. they can sort them by groups, subgroups or rewards. in this panel, no explanation is given for the different quests. for instance if the quest is "find an achene", we do not define achene. this is done by design. we want students to look up the different biological terms by themselves. we do provide them with ressources to do so. when an educator logs into the web application, the "quests'' panel becomes the "admin" panel. in this panel, educators can follow the evolution of the activity ( fig. 2a) , change the activity parameters ( fig. 2b ) or correct the student submissions ( fig 2c) . depending on the number of participating students and allowed submissions, the number of corrections can quickly become quite large. therefore we designed the corrections interface to be fast and efficient. the educator first chooses one quest to correct. he·she will in spring 2020, we organised the activity with a rooster of 346 first year bachelor students from the bioengineering faculty of the uclouvain (belgium). students were spread in 346 groups (it was therefore set up as an individual activity). although students had to do the activity individually, we encouraged them to discuss the different quests and collect them together, as long as everyone took their own pictures. each group was allowed to submit a maximum of 50 pictures. 285 quests were created, divided in 175 plant quests and 110 animal quests. specific restrictions were added to the game. a minimal number of animal and plant quests had to be collected by each group. groups were also asked to the activity started on february 11. we had to pause the activity for 20 days at the beginning of the lockdown due to the covid19 crisis. during that pause, we implemented the peer-evaluation in the web interface (it was not part of the interface initially). the activity resumed on the 3d of april and finished on the 15th of may. for the second phase of the activity, during the lockdown, all restrictions (quests groups and zones) were lifted as many students had returned to their home far away from the campus. at the end of the activity, we sent an anonymous feedback form to the students and received 125 answers. a total of 6543 pictures were submitted by students during the 2020 activity. figure 3 shows the repartition of the submitted pictures by the students during the activity. figure 3a & b show the difference before and after the lockdown imposed during the covid-19 crisis. before the lockdown, as we asked students to take pictures around the university, most of them were taken in louvain-la-neuve. during the lockdown, almost no pictures were taken in louvain-la-neuve, as students went back home. the lockdown reduced the number of collected pictures, but did not stop it. this is due to several reasons. at the beginning of the activity, we encouraged students to look for quests in groups, to foster peer-learning between them. this was not possible anymore during the lockdown. the collection of biological data was also influenced by the direct surroundings of the students. students living in an urban area were potentially at a disadvantage compared to students in the countryside. however, because we included the photo quiz module at the beginning of the lockdown, every student could continue the activity. figure 4 shows, for every group, the proportion of points acquired either with the quests collection or the photo quiz. we can see that the dual system allowed students to choose different strategies, to adapt to their individual lockdown conditions. we also observed a strong trend toward the collection of plant-related quests by the students ( fig. 3c & d) . this is probably due to the fact that, in an urban setup, plants are easier to find that animals. for an inexperienced naturalist, it is also probably easier to take pictures of plants than animals that have a tendency to escape. all the pictures can be viewed interactively at the address http://2020.quovidi.xyz overall, we observed a high correctness in the students picture submissions ( fig 5a) . for the treasure hunt and the picture collection, only 10% and 14% of the quests (for the animal and plant, respectively) were assessed as incorrect by ourselves. one reason for such a high accuracy from the students might be the high level of engagement required by the activity. they have to learn the vocabulary and discuss with other students, and go outside often in groups to find what they have identified as appropriate for a quest submission. in the icap framework [3] , we believe this corresponds to the "interactive learning" level, enabling the highest learning capabilities. interestingly, we also observed a much lower accuracy for the photo quiz ( fig. 5b ). for that activity, 37% and 38% of the evaluations by the students (for the animal and plant, respectively) were incorrect. this can be due to several factors. first, contrary to the treasure hunt in itself, the evaluation activity requires a lesser level of engagement by the student. the activity is indeed "reduced" to click on a button in front of a computer screen. second, depending on the quality of the picture to evaluate, said evaluation could be challenging. we tried to keep only good pictures for that activity, but the quality remained nonetheless variable. overall, the activity was very well appreciated by the students. with a few exceptions, students like going outside to observe their surroundings and collect the quests. in a survey performed after the activity ( fig. 6 ), 125 students reported to like the activity and have the feeling to have learned during it. many students spontaneously expressed their enthusiasm for this activity (tab. 3). selected comments from the students "great activity to learn new concepts and look at our environment in a different way. " "i think the game is fun and interactive, it's a great way to learn by seeing things "in real life" and also to decipher the quests. " "very nice way to propose the course, it pushes the students to discover the surrounding nature in a playful way. " the quovidi platform was created for several reasons. we wanted students to learn and know specific plant and animal vocabulary, but we did not want to just give them a list of words to be memorized and repeated. we also wanted them to explore and learn to observe their direct environment. we wanted to show them that you do not need to go to a tropical forest to be able to see a great diversity of plant and animal forms and species. we wanted to spark a strong interest in their surrounding natural world. finally, we were also working with strong practical constraints. we needed to design an activity that was scalable for hundreds of students, without the need to increase the number of educators. this was possible, thanks to the current technologies (camera, mobile network and gps localisation) available in almost every mobile phone. with the creation of the web-platform for quovidi, we have met all those goals. the treasure hunt (and to a lesser extent the photo quiz) strongly motivates students to learn and remember the different technical terms used in the quests. then they have to apply these new terms directly in the field. the gamification process (quests, score points, personal progress panel and scoreboard between all the groups) is also a strong incentive to engage in the activity. the activity is also highly scalable. the number of participants is, from a technical point of view, only limited by the capacity of the server on which the platform is installed. the main limitation remains the expert correction step. as every single picture needs to be validated, the evaluation can quickly require a lot of time from the educator, even though we tried to make the process as efficient as possible. we hope in the future that the platform would benefit from advances in artificial intelligence algorithms to help correct the images (see below). finally, the activity is completely decentralised, which has been a great asset during the covid-19 crisis. students can collect quests at any time and place, making it easy to adapt to every individual situation. if they cannot go outside, or are not in a nature-rich environment, they can still participate in the activity via the peer evaluation module. from the educator point of view, all the management and corrections can be done from anywhere, as long as they have access to a computer and an internet connection. as such, the platform was a real asset during the lockdown period (13 march to 8 of june in belgium), as it enabled us to continue the activity almost seamlessly. similarly to citizen science projects, the use of our platform allows the collection of large numbers of geotagged, dated images of plant and animal structures. by helping create such a database over the years, the students are taking an active role in creating a valuable research ressource. this in itself is viewed by the students as a motivational element of the activity. such databases could be re-used in different ways. from an educational point of view, the images collected could be used to create a quiz to rehearse the vocabulary the following year. the student would therefore create their own teaching and rehearsal material. an example of a quiz created with the students pictures is visible here : http://quiz.quovidi.xyz . from a research point of view, an ever growing database of annotated plant and animal pictures (describing either organ, species or groups), on a limited and well defined area would be a valuable resource. as each record of the database has been validated by an expert (the educators), such a database could be used in research projects. another interesting valuation of the database would be to reuse it to train deep learning recognition algorithms. again, given the size and potential growth of the database, it will be an interesting resource to train machine learning models to recognise plant and animal structures. such models could, in turn, be integrated into the platform to help with the correction. so far, we use the quovidi framework within a single classroom (even if it was a very large one). since the activity is entirely centralised online, we could imagine collaboration between remote classrooms. students from different regions, countries or continents could participate in the same activity, hence increasing the degree of diversity of the observations. here we exemplified the use of our platform with a biological treasure hunt. students were asked to find, in the field, plant and animal structures. however, due to its flexibility, the platform could be used to organise large scale treasure hunts in any context. it could be used in architecture, design or geology classrooms, with quests related to different building structures, street art or rock, respectively. it could be used with children, with simplified quests, or with more advanced students, with more complex ones. in short, we expect the concept could be used in any context to deal with structures present in the "outside" world. we presented in this manuscript a new open-source web platform for the organisation for large tresor hunt, quovidi. during the spring 2020, in the midst of the covid-19 crisis, we successfully used the quovidi platform with more than 300 students, and allowed the collection of more than 6000 geotagged plant and animal pictures. the decentralised nature of the platform enabled us to ensure a continuity in our teaching, despite the nation-wide lockdown. we expect quovidi to be of interest for any teaching activity focused on the identification of real-world structures. quovidi is available at the address http://www.quovidi.xyz active learning increases student performance in science, engineering, and mathematics translating the icap theory of cognitive engagement into practice the icap framework: linking cognitive engagement to active learning outcomes increased structure and active learning reduce the achievement gap in introductory biology kapp bkm. games, gamification, and the quest for learner engagement. in: main [internet the effect of gamification on motivation and engagement create dashboards with shiny: web application framework for r shinywidgets: custom inputs widgets for shiny twitter bootstrap components for shiny shiny ui widgets for small screens dbi: r database interface sqlite" interface for r advanced graphics and image-processing in r exif image data in r easily install and load the "tidyverse dates and times made easy with lubridate streamlined plot theme and plot annotations for "ggplot2 formattable: create "formattable" data structures dt: a wrapper of the javascript library "datatables tools for splitting, applying and combining data leaflet: create interactive web maps with the javascript "leaflet" library feel: a french expanded emotion lexicon. language resources and evaluation apache license, version 2.0 | open source initiative quovidi, then called biogo, was one of the laureates of the "prix wernaers pour la vulgarisation scientifique" in 2020. original draft x x x x x key: cord-031243-p6142nt1 authors: lewis, mae m.; markey, mia k. title: from study-abroad to study-at-home: teaching cross-cultural design thinking during covid-19 date: 2020-09-01 journal: biomed eng education doi: 10.1007/s43683-020-00018-9 sha: doc_id: 31243 cord_uid: p6142nt1 nan the course ''international perspectives on biomedical engineering design'' prepares students for crosscultural design thinking. 4 students learn human-centered design methods to understand the people for whom they are designing and to identify actionable problem statements. 3 the course theme is the potential of artificial intelligence (ai) to transform breast cancer care. as a short-term faculty-led study-abroad in portugal, it facilitates students' exploration of the impact of culture on healthcare delivery and the design of healthcare technologies. reflective exercises are emphasized to help students develop intercultural competence. 13 the primary course number is in biomedical engineering with cross-listings offered through chemical engineering and mechanical engineering. students plan, conduct, and interpret interviews with healthcare professionals who treat breast cancer patients at champalimaud clinical center (lisbon, portugal) and the university of texas md anderson cancer center (houston, us). in study-abroad, at least half of the interviews are conducted via videoconference since half the professionals are in us and half are in portugal. as an online course, all of the interviews are via videoconference. due to scheduling constraints for synchronous online delivery, the number of healthcare professionals interviewed was reduced from 14 in the 2019 study-abroad offering to 10 in the 2020 online offering, with a corresponding decrease from 7 to 5 specialties. study-abroad students' learning from the interviews is assessed in three ways: (1) individual reflective writing; (2) actionable problem statements written in the form of tweets by small groups 9 (find us on twitter at @povpractice.); and (3) nih-style abstracts written, reviewed, and revised by small groups. students are randomly assigned to different groups for different activities. for the 2020 online offering, four groups of four students each were used. given the enrollment and distribution of majors, groups typically included a mix of bme and non-bme students. individual reflective writing required students to summarize the interview experience by discussing three main ideas, answering the prompt: ''how is the culture of the country in which the subject of this interview practices reflected in their answers to the interview questions?'' and proposing a future interview question. actionable problem statements capture needs and insights identified from interviews of healthcare professionals using tweetable memes. after all interviews were completed, students proposed and critiqued solutions to actionable problem statements in the form of nih-style abstracts. for the transition to online, we maintained the first two assessments but dropped the third because the nih-abstract project typically requires students to work together in groups for many hours. we were concerned about students' ability to coordinate extensive groupwork in the online format given the pandemic's disruptions of ''normal'' life, e.g., some students were expected to have increased family responsibilities. it is noteworthy that the online students' interview questions included the impact of the covid-19 pandemic on breast cancer care and health equity topics inspired by the black lives matter movement. study-abroad students visit the breast unit of two very different hospitals: hospital de sa˜o joa˜o participates in the national health service whereas champalimaud clinical center is a private institution. study-abroad students also visit research facilities: centro de investigac¸a˜o em tecnologias e servic¸os de sau´de (cintesis), associac¸a˜o fraunhofer portugal research (fraunhofer aicos), and instituto de investigac¸a˜o e inovac¸a˜o em sau´de (i3s). the online format replaced these site visits with ''virtual field trips'' featuring youtube videos and readings and were assessed by discussion board posts relating them to other course activities. study-abroad students participate in structured cultural activities: short immersive portuguese language lesson; hands-on introduction to cooking traditional portuguese food; and an interactive lesson in traditional portuguese dances. the portuguese language lesson was replaced with a short course through the online language learning platform memrise. in the online dancing lesson, students submit a video of themselves performing a traditional portuguese dance and discuss how it compares to other dance forms. for this and other video assignments, students used flip-grid, which integrates with canvas, our learning management system. flipgrid is a user-friendly platform for instructors to facilitate and moderate student video responses. the convenient privacy settings offer an advantage over self-recorded student videos by allowing instructors to limit influence from past assignments by hiding the responses. the settings can also be adjusted so that students can view and comment on each other's submissions. these video assignments also give the instructor the opportunity to hear from all students, which is an important interaction often lost in online instruction. unlike written reflections, video reflections capture students' feelings in the moment and are a memorable course experience. 12 in lieu of a hands-on cooking lesson, students read portuguese recipes and respond with a reflective flipgrid video submission about which ingredients they had at home and how the recipes compared to what they normally eat. study-abroad students also go on guided tours. we did not try to replace the tours in the online format, but instead picked a few topics to introduce through readings, videos, and interactive websites, and assess by flipgrid: geography of portugal and its connection to wine industry; role of portugal in black history; and religious freedom. changes to students' thinking about culture, design, breast cancer, and ai in medicine were assessed by comparing start-of-course and end-of-course concept maps. 1,7 study-abroad students create concept maps in small groups, whereas the online students created maps individually. in the study aboard format, it was natural to have students do the concept maps in small groups because we had extensive class time together. for the online format, the amount of synchronous class time was very limited in comparison. moreover, we were reluctant to assign group projects because we were concerned about students' ability to coordinate extensive groupwork in the online format given the pandemic's disruptions of ''normal'' life. for this reason, in the transition to online, we chose to change the concept map exercises from group projects to individual assignments. students individually perform start-of-course and end-of-course self-assessments of their global learning and their intercultural knowledge and competence 5,6 and respond to free-response reflection prompts inspired by the set proposed by the university of michigan for international programs in engineering, 10 though substantial modifications were made for the transition to online learning. students write three end-of-course essays. they use rolfe's reflection model 11 to write about the impact of culture on healthcare delivery and the impact of culture on the design of healthcare technologies, especially those based on ai methods; this process was unchanged by the transition to online learning. however, in the study-abroad format, the third essay is a photo essay based on photographs they take during the program, so the online students searched for photos online. six biomedical engineering majors and 10 students in other health-related disciplines enrolled (biochemistry, biology, health & society, international relations, neuroscience, and public health.) the total is similar to prior study-abroad offerings, though the percentage of non-engineering majors was higher (historically~10-30%.) about half of the students identified as male (7) and the others as female (9), which is typical for the course. as expected, they ranged from rising sophomores to graduating seniors. some key ideas typically noted from interviews conducted by study-abroad students were observed in the online format as demonstrated by the students' written reflections and actionable problem statements, e.g., differences between the us and portuguese governments' roles in healthcare. however, the online students' questions emphasized more current events, e.g., how covid-19 is impacting breast cancer care and how ai may perpetuate health inequities. in contrast, study-abroad students' questions emphasized the doctors' daily experiences and their reflective writings stressed the emotional impact on the doctors. we believe that these differences are more likely due to the extraordinary world events that the students are living through rather than the course format. recording interviews was essential because connections were frequently ''dropped'' and students faced pandemic time demands (health concerns, family responsibilities, etc.) video recordings of course sessions can be shared on canvas via zoom or panopto. while either allows the instructor to disable download and restrict access by date, we decided to use panopto because the zoom configuration at our institution does not enable the instructor to restrict access to students enrolled in the course. in particular, students enrolled in a course can share links to course recordings distributed via zoom to any member of our large university, whereas course recordings distributed via panopto cannot be simply shared to people not enrolled in the course. considering the privacy of both our students and professional guests, we prefer greater access control. the study-abroad professional field trips were difficult to mimic online. many portuguese organizations have youtube channels, but often with limited english-language content. youtube's automatic translations from spoken portuguese to english subtitles were not useable. unfortunately, our virtual field trips did not prompt as much critical reflection. for example, the visit to hospital de sa˜o joa˜o is a major culture shock to many of our study-abroad students because it challenges their assumptions about what a ''good'' hospital looks like. in contrast, the students' video reflections about the virtual field trip to hospital de sa˜o joa˜o were not emotional. we believe that this is a substantial loss in the translation from study-abroad to online. online language learning was well-received, and some students exceeded the requirements. in future courses, online or study-abroad, memrise may be beneficial for students to pace their own language learning. students seemingly enjoyed flipgrid. their video reflections were longer and more engaging than their discussion posts. video assignments were also opportunities to involve others in their learning. in the cooking flipgrid, many students reported discussing, sharing, or cooking a meal with family. family members or pets also served as dance partners-some more willingly than others! shared experiences with family can make it easier to reflect after a course is over, which is key to a successful study-abroad. 2 however, these observations may not generalize to post-pandemic online offerings. due to the societal impacts of the pandemic, students are currently spending more time with their families than with fellow students. while the online cultural activities related to geography, history, and governance are limited substitutes for the extensive educational tours of study-abroad, there were complementary learning benefits. for example, some students had not realized the relative size of portugal to its former colonies prior to the interactive map activity. we recommend using more multi-media tools in future offerings, including for study-abroad. a key cultural experience for studyabroad students is the sa˜o joa˜o festival. in the end-ofcourse self-reflections, several online students regretted missing the sa˜o joa˜o festival. online information about festival traditions is a far cry from participation. moreover, in the future we will explore online approximations of the informal interactions with portuguese people experienced in study-abroad. broadly, the learning gains qualitatively assessed using concept maps were similar for the online offering as compared to the prior study-abroad offerings. 4 however, our online students emphasized the risk that ai poses to perpetuate or increase racial and socioeconomic biases. we suspect that this is due to concurrent world events rather than course format. some covid-19 specific themes were also evident, e.g., interest in clinicians' descriptions of breast cancer patients' isolation due to pandemic constraints on visitors. since online submission of concept maps was effective, we will adopt it for future offerings, including study-abroad. for global learning value rubric 5 self-assessments, students' end-of-course scores were significantly improved (wilcoxon signed rank test) relative to start-of-course for global self-awareness (p < 0.01) and perspective taking (p < 0.01), but not for cultural diversity (p = 0.35), understanding global systems (p = 0.22), applying knowledge to contemporary global contexts (p = 0.11), or personal and social responsibility (p = 0.51). for global self-awareness, the most common response on the start-ofcourse self-assessment was ''analyzes ways that human actions influence the natural and human world,'' whereas the most common response on the end-ofcourse self-assessment was ''effectively addresses significant issues in the natural and human world based on articulating one's identity in a global context.'' for perspective taking, the most common response on the start-of-course self-assessment was ''identifies and explains multiple perspectives (such as cultural, disciplinary, and ethical) when exploring subjects within natural and human systems,'' whereas the most common response on the end-of-course self-assessment was ''evaluates and applies diverse perspectives to complex subjects within natural and human systems in the face of multiple and even conflicting positions (i.e., cultural, disciplinary, and ethical.)'' for intercultural knowledge and competence value rubric 6 self-assessments, students' end-of-course scores were significantly improved (wilcoxon signed rank test) relative to start-of-course for empathy (p < 0.01), but not for cultural self-awareness (p = 0.13), knowledge of cultural worldview frameworks (p = 0.12), verbal and nonverbal communication (p = 0.63), curiosity (p = 0.11), and openness (p = 0.77). in contrast, the prior studyabroad cohort self-reported improvements in all areas except personal and social responsibility. 4 for empathy, the most common response on the start-ofcourse self-assessment was ''recognizes intellectual and emotional dimensions of more than one worldview and sometimes uses more than one worldview in interactions,'' whereas the most common response on the end-of-course self-assessment was ''interprets intercultural experience from the perspectives of own and more than one worldview and demonstrates ability to act in a supportive manner that recognizes the feelings of another cultural group.'' the global learning value rubric 5 and intercultural knowledge and competence value rubric 6 selfassessments suggest global cultures topics that are more challenging to address online relative to studyabroad. however, in addition to the limitations of selfassessments, taking a course online during a pandemic is not the same as taking a course online in general, e.g., some students reported that the current situation at home made it harder to think about global issues whereas others said that the ongoing pandemic made global issues more salient. study-abroad students have extensive opportunities for informal feedback because they spend many hours with the instructors. while we endeavored to be readily available to online students via slack, interactions were still limited by comparison. hence, the online distribution of structured rubrics was essential. several online students lamented that they did not get to know their classmates. in contrast, study-abroad students form strong bonds. weaker connections among students are detrimental to learning because each person views the world through the lens of their own experiences, e.g., students' responses to readings about religious freedom varied with their own religious background. in addition, we hypothesize that weaker connections among students decreased their efficiency in using remote collaboration tools for cooperative learning activities, e.g., generating actionable problem statements. in future online offerings, we will promote more student discussion towards the goal of increasing appreciation of the cultural diversity within the us and how one's personal experiences influence one's perceptions of portugal. we propose to build rapport among the students by having them interview each other, through which they would also be practicing skills they need to interview the healthcare professionals. this approach could be easily repeated throughout the course as new interview techniques are introduced. the online version of the class retained some of the major learning objectives in ai, breast cancer, and the role of culture in healthcare delivery and design of healthcare technologies through interviews, discussion board posts, concept maps, and tweetable actionable problem statements. in addition, flipgrid video reflections helped us maintain some of the authenticity of portuguese cultural experiences such as cooking traditional food, dancing to folk music, and learning facts about portuguese culture. students who incorporated members of their families into these assignments may benefit from creating lasting memories at home. however, study abroad experiences relying more heavily on students' interactions with each other and with portuguese people were not captured in this online format. the global value learning rubric and intercultural knowledge and competence value rubric self-assessments show that global cultures topics were more challenging to address, although these findings may also be influenced by the current extraordinary world events rather than solely the shift to online learning. final course surveys also showed that students wished they had taken more initiative to meet their classmates. moving forward, study-abroad courses that must transition to an online format should incorporate more opportunities for student interaction to increase overall student experience and student appreciation for others' diverse perspectives on course material. a systematic review of research on collaborative learning with concept maps impact of a global engineering course on student cultural intelligence and cross-cultural communication design thinking bootleg teaching crosscultural design thinking for healthcare association of american colleges & universities association of american colleges & universities. h ttps://jitp.commons.gc.cuny.edu/reflecting-on-reflections-u sing-video-in-learning-reflection-to-enhance-authenticity concept mapping as a learning tool in higher education: a critical analysis of recent reviews defining global competence for engineering students using twitter to support students' design thinking. in: asee gulf-southwest section annual meeting michigan engineering international programs in engineering critical reflection for nursing and the helping professions a user's guide reflecting on reflections: using video in learning reflection to enhance authenticity student learning abroad: what our students are learning, what they're not, and what we can do about it publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations we thank the international engineering education office at the university of texas at austin for the opportunity to teach international perspectives on biomedical engineering design in the maymester format. we are particularly grateful for the guidance of our program coordinator, ellen aoki. we deeply appreciate the support of our colleagues at champalimaud clinical centre in lisbon, especially maria joa˜o cardoso and sofia braganc¸a. we thank patricia pa-dra˜o at universidade do porto for portuguese recipes. we are indebted to the healthcare professionals who graciously volunteered their time to be interviewed by our students. we also appreciate the technical support and guidance in transitioning to teaching online from our colleagues, especially kathy jackson, jim pollard, and dan puperi. both authors contributed to all aspects of the manuscript. key: cord-340994-m7vazpq9 authors: barello, serena; nania, tiziana; dellafiore, federica; graffigna, guendalina; caruso, rosario title: ‘vaccine hesitancy’ among university students in italy during the covid-19 pandemic date: 2020-08-06 journal: eur j epidemiol doi: 10.1007/s10654-020-00670-z sha: doc_id: 340994 cord_uid: m7vazpq9 the debate around vaccines has been in the spotlight over the last few years in europe, both within the scientific community and the general public debate. in this regard, the case of the italian vaccination debate is particularly worrying given that italy has been one of the european countries with the highest number of measles cases in the recent past. according to this scenario, we conducted a cross-sectional study on a convenience sample of italian university students aimed at: (1) exploring their attitudes towards a future vaccine to prevent covid-19 and; (2) evaluating the impact of the university curricula (healthcare vs. non-healthcare curricula) on the intention to vaccinate. descriptive analysis on the 735 students that answered to the question on the intention to vaccinate showed that 633 (86.1%) students reported that they would choose to have a vaccination for the covid-19 coronavirus; on the other side, 102 (13.9%) students reported that they would not or be not sure to vaccine (low intention to vaccinate). this means that in our sample more than one student out of 10 shows low intention to vaccinate (vaccine hesitancy). furthermore, when running analysis comparing healthcare students versus non-healthcare students we found no significant differences in responses’ percentage distribution (p = .097). understanding the student’s perspective about the future covid-19 vaccine and supporting their health engagement and consciousness may be useful in planning adequate response and multidisciplinary educational strategies—including the psychological perspective on vaccine hesitancy underlying factors in the post-pandemic period. we read with great interest the essay by harrison and wu [1] on vaccine confidence in the time of covid-19. they wondered if the impact of the covid-19 global health emergency will solve the problem of vaccine refusal that has worried the global public health community for the last several decades. indeed, the concept of 'vaccine hesitancy' has been considered by the world health organization (who) as "one of the top-ten threats to global health" [2] . the debate around vaccines has been in the spotlight over the last few years in europe, both within the scientific community and the general public discourse. in this regard, the case of the italian vaccination debate is particularly worrying given that italy has been one of the european countries with the highest number of measles cases in the recent past [3] . this issue led to the introduction of laws prescribing mandatory vaccinations for children to attend school in december 2017, which gave rise to an active debate [3] . in line with harrison and wu [1] , we wonder if the covid-19 experience will fix the problem of vaccine hesitancy in italy. according to this scenario, we conducted a cross-sectional study on a convenience sample of italian university students aimed at: (1) exploring their attitudes towards a future vaccine to prevent covid-19 and; (2) evaluating the impact of the university curricula (healthcare vs. nonhealthcare curricula) on the intention to vaccinate. university students could be considered as an insightful population to investigate their attitudes to accept new vaccination because they are open-minded, educated, and supposed to respond quickly to public health issues. our sample was not designed to be representative of the italian university students, but to provide an initial and insightful description of the investigated phenomena. of the 1152 contacted, 934 students responded (response rate = 81%) to the online questionnaire sent through the participating universities' mailing lists: 551 students (58.9%) attended healthcare curricula (i.e., medicine, nursing) and 383 students (41.1%) attended non-healthcare curricula (i.e. economics, law, engineering, physics, math, human sciences). females were the majority (79.6%); mean age was 23.6 years (standard deviation = 4.9). the majority of the respondents who indicated their provenience (n = 664) lived in one of the most hit italian regions -lombardy -(n = 397; 59.8%), followed by respondents from lazio (n = 63; 9.5%). descriptive analysis on the 735 students that answered to the question on the intention to vaccinate showed that 633 (86.1%) students reported that they would choose to have a vaccination for the covid-19 coronavirus; on the other side, 102 (13.9%) students reported that they would not or be not sure to vaccine (low intention to vaccinate). this means that in our sample more than one student out of 10 shows low intention to vaccinate (vaccine hesitancy), a percentage that reflects the international literature on general influenza vaccine hesitancy [4] [5] [6] [7] . all the comparison analysis showed that responders who chose not to disclose their intention to vaccinate did not significantly differ from the others on demographic and social characteristics. furthermore, when running analysis comparing healthcare students versus non-healthcare students we found no significant differences in responses' percentage distribution (p = .097). these results requires particular attention. first, we expected that the intention to vaccinate would have been higher in students attending healthcare curricula due to higher literacy on health-related issues. second, these findings shed light on the potential risks liked to healthcare students vaccination hesitancy because vaccination of healthcare workers is a key measure in the prevention of healthcare-associated infections due to their close contact with high-risk patients [8] . although preliminary, this finding suggests that vaccination attitude is not only influenced by the students' level of health knowledge, but probably by other motivational and psychological factors, including the sense of individual responsibility for population health and the common sense about the value of civic life and social solidarity, as demonstrated by other studies on the covid-19 pandemic and previous emergencies [1, 9] . not only the general public, but also those studying to become health professionals, seem to struggle to keep up with a growing body of evidence and increasingly complex information and contrasting feelings about vaccines. therefore, public health information campaign should be also supported by other actions aimed at raising students' consciousness regarding the crucial role of individuals' engagement in safeguarding their own and others' health through vaccination. students are a good target for educational campaigns as they are still in their training period and are open to changing their habits. understanding the student's perspective about the future covid-19 vaccine and supporting their health engagement and consciousness may be useful in planning adequate response and management strategies in the post-pandemic period [10] . we think that the strategy to achieve efficient synergy between healthcare professionals and the general public is to better improve medical education of students during university and beyond introducing dedicated multidisciplinary curriculum about vaccinations and preventive behaviours for all university students and in particular to those attending healthcare curricula, an issue that requires increased attention to mitigate and control the covid-19 pandemic. unfortunately, population health and clinical prevention (i.e. vaccine behaviours) have long been relegated as a secondary topics in university curricula of most universities worldwide [11] [12] [13] . moreover, evidence demonstrated how multidisciplinary educational interventions are the preferred strategy to improve students' adherence, attitude and knowledge about vaccinations [14] [15] [16] . that's because a multidisciplinary approach to teaching could contribute to look beyond the traditional approach to health information. teaching complex health-related issues -through appropriate educational program across multiple disciplines, including social, psychological and behavioural sciences to complement the clinical and epidemiological ones -is key to better educate students about the multi-layer drivers and barriers of trust in vaccination, and defining how best to engage patients in preventive behaviours and to understand their behavioural and social reactions to immunization [17] . we think that our findings deserve attention in light of the particularly intense experience that the italian health services had with covid-19. we expected that the population living in this country would have higher levels of intention to vaccinate. the fact that this expectation does not find confirmation in our data makes our results warranted of public health attention and merits follow up studies designed for better understanding of the nature of the hesitancy among university students in italy. public health science should carefully consider the need to implement target and cultural-specific actions for university students, as influencing attitudes at an earlier stage of life can be more advantageous. people involvement around vaccines needs to be broad and multifactorial, with engagement at multiple levels [18] . these include policymaking, sensitizing programs' design, the development of risk communication strategies and psychosocial research aimed to understand people perspectives on vaccination behaviours in order to dialogue with them in an appropriate way. in this arena, educational initiatives-even not standalone interventions-are certainly an important step to ignite appropriate consciousness of individuals and communities vaccine confidence in the time of covid what should we do about vaccine hesitancy? italy's response to vaccine hesitancy: an innovative and cost effective national immunization plan based on scientific evidence quantifying population preferences around vaccination against severe but rare diseases: a conjoint analysis among french university students an observational study of university students of healthcare area: knowledge, attitudes and behaviour towards vaccinations understanding influenza vaccine perspectives and hesitancy in university students to promote increased vaccine uptake uptake of influenza vaccination, awareness and its associated barriers among medical students of a university hospital in central saudi arabia influenza vaccination of healthcare workers and vaccine allocation for healthcare workers during vaccine shortages factors associated with 2009 pandemic influenza a (h1n1) vaccination acceptance among university students from india during the post-pandemic phase using social and behavioural science to support covid-19 pandemic response teaching vaccine safety communication to medical students and health professionals clinical prevention and population health: curriculum framework for health professions prevention for the 21st century: setting the context through undergraduate medical education improvement in vaccination knowledge among health students following an integrated extra curricular intervention, an explorative study in the university of palermo improvement in attitudes toward influenza vaccination in medical students following an integrated curricular intervention mandatory influenza vaccination of health care workers: translating policy to practice missed opportunities? locating health promotion within multidisciplinary public health. public health a multidisciplinary research agenda for understanding vaccine-related decisions conflict of interest the authors declare no conflict of interest. key: cord-030576-ustp3fiw authors: ramo, nicole l.; lin, mei’ai; hald, eric s.; huang-saad, aileen title: synchronous vs. asynchronous vs. blended remote delivery of introduction to biomechanics course date: 2020-08-18 journal: biomed eng education doi: 10.1007/s43683-020-00009-w sha: doc_id: 30576 cord_uid: ustp3fiw nan the faculty of the department of biomedical engineering at shantou university was intentionally developed to include instructors with international teaching or training experience. accordingly, six of the ten full-time faculty are non-chinese nationals with varying international citizenship. when covid-19 necessitated the transition to remote instruction between the fall and spring semesters, these international faculty members left mainland china for their respective home countries while all students relocated to their family homes in mainland china. as a result, students and instructors were faced with time zone disparities of 2-14 h and home learning environments that were often embedded in shared living spaces of an apartment. to address these challenges, instructors needed to establish an appropriate balance between synchronous and asynchronous teaching. this article systematically describes commonly used evidence-based teaching practices and how they were implemented in three versions of remote instruction (each with a different balance of synchronous and asynchronous teaching). the course context was a 2nd-year lecture-based introduction to biomechanics course team-taught by two faculty members, one based in china and one based in the united states (us). the biomechanics course was divided into three, subject-specific sections: statics, dynamics, and mechanics of materials. while all sections leveraged the same instructional technologies and software tools, each section had a different ratio of synchronous to asynchronous components based on instructor preference and availability. first, the specific technologies and tools utilized will be discussed and then a comparison between the delivery approaches of each section will be presented. general tools/pedagogical techniques used for both delivery methods handout/worksheet for each lecture providing guided notes (i.e., ''skeleton notes'') to students can improve attention and academic performance, perhaps by allowing students time to process and synthesize information as opposed to superficially writing every word presented during a lecture. 1,4 accordingly, guided notes for each synchronous and asynchronous lecture were made available as downloadable handouts (examples of two such handouts are given as supplemental material). each handout contained highlighted blanks which the students were expected to complete during the lecture. students were also encouraged to add their own notes and any figures or drawings that would help them understand or remember the content. no answer key was provided for the handouts; instead, students could add any unresolved questions to their weekly discussion posts described below. clear and observable learning objectives for each lecture in general, it is important to make expectations explicit so students can engage in more selfmonitored learning and self-directed review 3 ; this is even more critical in remote learning settings where the typical peer or instructor support structures are less readily available. 10 therefore, every lecture and handout started with 3-5 learning objectives. these were clear statements describing what students should be able to do based on the content presented in that lecture. narrated mp4 lecture videos {microsoft pow-erpoint} the method of presenting new information using powerpoint slides was familiar to both the instructors and students; it also allowed most of the slides used for in-person instruction to be used for remote teaching. most recent versions of powerpoint allow presentations to be exported as mp4 videos including narrations, slide timings and animations, virtual laser pointer, and ''ink'' markings (video from a webcam is also available for microsoft 365 subscribers). 7 the us-based faculty member narrated each slide, using the built-in presentation tools (e.g., highlight, laser pointer, etc.) the same way they would for in-person lectures. as the recordings are saved for individual slides, any slide could be re-recorded if the initial narration was unclear or unfocused. student engagement exercises were incorporated every 10-20 min during playback. these activities asked the students to pause the video to either read the next handout section, complete the previous handout section, or attempt to solve a problem (the solution to which is given as the next part of the video). weekly discussion posts {office 365 (or google slides)} discussion forums are the most common means of whole-class asynchronous communication and interaction; they not only provide flexibility and opportunities for refection, but can also create a sense of community in online courses through sustained interaction and feedback. 9,10,12,13 posting powerpoint slides from every lecture (asynchronous or synchronous) to the course microsoft sharepoint website allowed students to open and collaboratively edit the file online using powerpoint for office 365. before the end of every week, students were required to post two comments on the lecture slides for that week. these comments could be questions, answers to peers' questions, links to additional learning resources, or relevant examples, comments, or reflections ( fig. 1) . at the end of the week, the instructor reviewed all of the posts; addressing most through in-line comments while identifying common themes or points of confusion that were addressed in subsequent asynchronous videos or synchronous lectures. live lectures {zoom: screen share} live powerpointbased lectures were given using the share screen feature in zoom. the advanced sharing feature ''portion of screen'' 15 was used in conjunction with the powerpoint presenter view to display the current slide, speaker notes, and next slide to the instructor while displaying only the current slide to students. student engagement activities were also incorporated during the live lectures where students volunteered or were cold-called to provide their answers. attendance was mandatory for each of the live lectures. zoom-created recordings of the live lectures were posted to the course website such that students could download and review the lecture at a later time. discussion of academic journal article {zoom: breakout rooms and screen share} the instructors felt it was important to introduce students to current research efforts in the field of biomedical engineering to not only increase exposure and interest, but also to demonstrate the applicability and value of course content in addressing on-going questions or problems in the field-a characteristic of authentic learning. 11 therefore, four academic journal articles, each based on a different aspect of biomechanics, were assigned for live discussion. associated discussion questions were posted approximately one week before each scheduled discussion with pre-class answers to these questions graded for completion. during the live zoom discussion, the instructor provided an overview of the article as well as the answers to the easier discussion questions (ones that could be found directly in the article text). then, students were separated into groups of four or five using the breakout room feature of zoom 14 with each group assigned one of the remaining questions (ones that required external research, synthesis of article information, or drawing connections to specific lecture content). the students spent 15-30 min in their small group to develop a consensus answer, creating one or two slides to explain their answer and nominating a ''group captain.'' the breakout rooms were then closed, and all participants returned to the main meeting where each group shared their screen in turn while the ''group captain'' presented their answer. the instructors helped guide the discussion by posing follow-up questions and summarizing important points. after discussion, each student updated their pre-class answers and submitted a post-class assignment which was graded for accuracy. the first four weeks of the course focused on statics and was delivered by the us-based faculty member in a blended fashion with both synchronous and asynchronous components. two lecture videos, each totaling approximately 60-90 min in length (not including pauses for engagement exercises), were posted at the beginning of every week. students were responsible for watching the videos, completing the associated handouts, and submitting at least two discussion posts by the end of the week. on fridays, a 30min zoom lecture was held to address the main points of confusion (as identified in discussion posts). the second section of the course focused on dynamics and was delivered by the china-based faculty member in a largely synchronous way. live zoom lectures, each 100 min long including a 10-min break, were held twice a week. students were still required to submit two comments on the powerpoint files; these submitted comments and questions were addressed during the first lecture of the following week. the final section of the course focused on the mechanics of materials and was delivered by the usbased faculty member in a primarily asynchronous way. the main difference between this section and the initial (blended) approach was the frequency of synchronous class sessions. during the first section of the course, the class met weekly to review the comments and questions submitted as discussion posts. during the final section of the course, the discussion posts were addressed in short, stand-alone review videos; the only synchronous class session of this course section was a journal article discussion. between the dynamics and mechanics of materials sections of the class, students were asked to complete a survey concerning the online implementation (response rate: 33/35). students responded positively to the guided notes. 76% of students reported that they consistently used them; 95% of those students found them somewhat or very helpful to their learning. consistent with the literature, 1,4 students indicated that: ''i can focus on the class more;'' ''i can listen to the lecture [more] efficiently;'' and ''[they] guide me when i am listening to the lecture.'' students were initially resistant to the requirement of discussion posts because it was not clear how it would aid their learning. in a private correspondence to the us-based instructor after three weeks of the course, a class representative stated ''in our high schools, most of the students are not encouraged to ask questions; so now some of us are confused about the significance.'' based on this feedback, the instructor used part of that week's live review session to explicitly discuss the benefit of these posts, both for the instructors and for the students. by the time of the survey, 86% of respondents reported they understood the purpose of the weekly discussion posts assignment, and 89% found the assignment somewhat or very helpful for their learning. the accompanying comments from students underscore the reflective and collaborative benefits of asynchronous discussion threads 9,13 : ''[the posts] allow us to think about the knowledge we have learned, which helps deepen the impact;'' ''helps me test myself and review the content;'' ''asking my questions and solving others' questions helped me to have a better understanding of what i was learning;'' ''i learned a lot from the comments of my classmates.'' at the time of this submission, the shantou university semester is still on-going; therefore, data on student perceptions of the largely asynchronous delivery method have yet to be collected. however, in the survey data currently available, students expressed a preference for the blended approach over the largely synchronous approach (fig. 2) . examining accompanying student survey comments, the accepted benefits of flexibility, convenience, and ability to personalize learning were frequently noted. 2, 12 for example, ''i can choose my own time to study;'' ''the time is flexible;'' ''narrated videos give me more freedom…if i miss some points, i can pause and watch that part of the video again;'' ''i can repeat the points which i didn't get and skip the parts i know.'' however, the minority of students who expressed a preference for synchronous lectures cited the desire for more direct interaction with their instructor and peers, consistent with the literature on feelings of isolation and lack of immediacy that may result from asynchronous instruction. 2, 9, 12 one specific comment stated, ''i like [the live zoom lectures] because i have more interactions with teachers and classmates which makes the classes interesting.'' although we do not have post-course survey results at this time, the instructors feel that the biggest challenge was facilitating student engagement and interaction, especially during live synchronous lessons. the strong student preference for a blended approach over a synchronous approach could reflect not only a preference for that delivery method, but also an indication that we were not leading effective synchronous lectures (indeed, this feeling was one motivation for trying the primarily asynchronous delivery of the final section of the course). therefore, in future offerings of lecture-based online courses, the instructors will adopt a blended approach that follows the ''flipped-class'' model. in a ''flipped-class,'' instructional content is delivered to students asynchronously outside of class (typically through online videos); synchronous in-class time is devoted to discussion, application, and/or collaborative learning. 3, 8 an online flipped-class could look like the following, which describes our plan for future remote offerings of this lecture-based course: at the beginning of every week, a series of shorter (10 min or less), topicspecific, narrated videos will be posted along with a guided note handout. these videos could be edited/ trimmed versions of those created during the spring 2020 semester. the weekly discussion posts assignment will continue, but an auto-graded weekly quiz assignment will be added to the course learning management system. this quiz will help students more directly assess their knowledge based on the week's learning objectives and identify areas of confusion about which to inquire and discuss. the asynchronous narrated videos will include fewer problems for the students to solve independently; instead, those engagement activities will be moved to a weekly synchronous class session. these activities could be done in a ''think-pairshare'' format 3, 8 with the ''pair'' being a small group in a zoom breakout room and the ''share'' involving screen sharing of solutions with the rest of the class. the common points of confusion identified in the discussion posts will also be addressed during these live sessions, but with more emphasis on soliciting input and explanation from students who submitted comments, questions, or answers. journal article discussions will continue to be implemented monthly to demonstrate the applicability of course content. the authors believe the ''flipped-class'' model described above leverages the perceived benefits of both synchronous and asynchronous components. it also aligns with many of the practices supported by pedagogical research and recent student surveys 5 a final advantage of this approach is that it can easily be adapted for in-person classes once both instructors and students are safely back on campus. across future offerings of the course, quantitative comparison of summative assessment results (e.g., mid-term and final exam scores), as well as qualitative comparison of the depth and sophistication of discussion post threads, will help assess the influence of different modes of delivery on student learning outcomes. the online version of this article (https://doi.org/10. 1007/s43683-020-00009-w) contains supplementary material, which is available to authorized users. development and teaching of the course was done by nr and ml. student survey data was collected and reviewed by nr. teaching support and guidance was provided by eh and ah-s. assistance in connecting to engineering education literature was provided by ah-s. the first draft of the manuscript was written by nr and all authors commented on previous versions of the manuscript. all authors read and approved the final manuscript. funding no external funding support. the authors declare that they have no conflict of interest. the data reported in this article are from an anonymous student survey administered in the course of normal education practice. ethics board review is not required in accordance with the ethical standards of our institution. effects of guided notes on university students' responding and recall of information synchronous vs. asynchronous tutorials: factors affecting students' preferences and choices teaching and learning stem: a practical guide a meta-analytic review of guided notes what worked this spring? well-designed and -delivered courses. inside higher ed transform. teach. learn suddenly online: a national survey of undergraduates during the covid-19 pandemic record a slide show with narration and slide timings the use of flipped classrooms in higher education: a scoping review the role of the communication tools in the development of the learning group in an online environment applying best practice online learning, teaching, and support to intensive online environments: an integrative review editorial: the components of authentic learning researchsupported best practices for developing online learning synchronous and asynchronous communication in distance learning: a review of literature managing breakout rooms. settings control sharing your screen in a meeting. audio, video, shar publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. key: cord-354270-445lhfdc authors: chen, rong-ning; liang, shun-wei; peng, yang; li, xue-guo; chen, jian-bin; tang, si-yao; zhao, jing-bo title: mental health status and change in living rhythms among college students in china during the covid-19 pandemic: a large-scale survey date: 2020-08-15 journal: journal of psychosomatic research doi: 10.1016/j.jpsychores.2020.110219 sha: doc_id: 354270 cord_uid: 445lhfdc • 7.7% of college students showed depressive symptoms during the covid-19 pandemic. • college students with depressive symptoms had low regulatory emotional self-efficacy. • browsing covid-19 information over 3 h per day was related to depressive symptoms. • unfavorable living rhythms were associated with depressive symptoms. • pay attention to college students' mental health during the covid-19 pandemic. as a public health emergency of international concern, the 2019 coronavirus disease has gained intense attention globally, posing serious threats to people's lives, as well as to their physical and psychological health [1] . as of 2 april 2020, the number of confirmed cases worldwide has exceeded 1000,000 and the global situation is of the utmost seriousness. the chinese government has implemented strict self-and forced-quarantine measures across the country. under the unified deployment of the education department, the return of students to colleges has been postponed in most provinces [2, 3] . more than one million college students in guangdong province are currently studying online at home. in any biological disaster, the themes of depression, uncertainty, and fear are common [4, 5] . a study included 1210 respondents from 194 cities in china found that 53.8% of respondents rated the psychological impact of the pandemic as moderate or severe; 16 .5% reported moderate to severe depressive symptoms; 28.8% reported moderate to severe anxiety symptoms, and 8.1% reported moderate to severe stress levels [6] . prolonged lockdown had several adverse impacts on mental health. in a survey, respondents aged 12-21.4 years showed a higher psychological impact four weeks later than that on the onset of covid-19. this age group mainly comprised of students who were undergoing prolonged school suspensions, requiring online education support and uncertainty about examinations and enrollment arrangements [7] . furthermore, people with preexisting psychiatric illnesses had difficulty accessing mental health services during the lockdown [8] . strict self-quarantine measures may j o u r n a l p r e -p r o o f also affect college students with mental health problems. we need a larger epidemiological survey evaluating the psychological impact of covid-19 pandemic on college students. during the pandemic, the regular holiday activities of college students are limited as they have lost the freedom to go out and socialize face-to-face as they would usually. this kind of loss can be overwhelming, and students who lack an appropriate coping style may experience depression [9] . furthermore, some college students may experience irregularities in their daily life, such as getting up late, staying up late, spending more time surfing the internet, and so on. all of these may lead to disruption to their biological rhythms, known to be an important clinical feature and pathophysiological mechanism underlying mental illness [10], closely related to the onset, symptoms and social functions of depression [11, 12] . quarantine for covid-19 increases the possibility of psychological and mental health problems [13] , and depression is more likely to occur and worsen as a result [14] . simultaneously, the reduction of activity caused by quarantine can lead to a vicious circle in which ways to alleviate negative emotions are reduced [15] . compared with the sars pandemic in 2003 and the wenchuan earthquake in 2008, internet services are more widely therefore, it is particularly important to evaluate the living rhythms and depressive symptoms of college students during the covid-19 pandemic. as a result of the pandemic, colleges and universities have been prompted to conduct education online, and autonomous learning at home for college students may raise challenges relating to their self-discipline [17] . some students report that it is more difficult to persist with study at home than at school, and that students feel higher levels of self-negation and depression. if effective emotional regulation is absent, the risk of depression is likely to be increased. previous studies have shown that regulatory emotional self-efficacy (rese) could directly predict mental health, such as depression and anxiety [18, 19] . rese refers to the degree of confidence with which individuals can effectively regulate their emotional state, which has two components: perceived self-efficacy in managing negative (neg) and in expressing positive (pos) affect [20] . in a longitudinal study on the relationship between rese and depressive symptoms in a group of early adolescents, having strong rese was not only negatively associated with their current levels of depression, but also predicted their depression after 4 years [21] . people with weak rese were more inclined to choose negative emotional regulation strategies to deal with negative emotions when distressed, leading them to experience worse depression [18] . on the contrary, favorable rese could help individuals use positive approaches to improve their emotion regulation [22] . a study of graduate students found that rese would serve as a buffering factor, decreasing the risk of suicide caused by depression [23] . therefore, it is of great significance to detect the rese of college students during college students during the pandemic by using the 7-item generalized anxiety disorder scale (gad-7), and found that 3.6% of the respondents were experiencing moderate to severe anxiety (gad-7≥10) [24] . the implementation of the depressive symptoms assessment of college students is also an important part of dealing with the pandemic, and to inform the formulation and implementation of relevant mental health intervention policies. therefore, we focused on the period of occurrence and development of covid-19 to comprehensively evaluate the immediate psychological influence on college students by considering both negative symptomatology and positive efficacy. we also sought to determine the relationships between depressive symptoms, rese and living rhythms, and predict the risk factors for mental health difficulties. it was intended that this work would provide a basis for the prevention of mental health difficulties in college students during the covid-19 pandemic. students in guangdong province were able to access the survey using wechat and answer the questionnaire anonymously by scanning the two-dimensional barcodes of the questionnaire address or clicking the relevant link. after participants entering the survey homepage, an online consent form would be displayed before the questionnaire. if the participants had no objection to survey objective in the consent form, they could officially start the survey by clicking "next" button below, or they could have right to cease the survey. only one response per person to the questionnaire was permitted. participation was completely voluntary and non-commercial. in our study, the items of name and personal phone number were optional in the questionnaire. in addition, all investigators had signed confidentiality agreements. the senior investigators performed quality control by checking the collected questionnaires daily. the study was approved by the appropriate institutional research and ethics committee. in total, 361,969 college students completed the questionnaire. before data processing, we applied a series of strict participant exclusion criteria, namely: (1) if the major could not be identified or was filled in indiscriminately, (2) if the name of the school could not be identified or did not belong to guangdong province, (3) if there was an demographic variables included gender (male or female), age, grade, nationality, educational level (junior college or undergraduate college), family residence, and current location. family residence included the following types:(1) country, (2) town, the phq-9 scale was used to measure depressive symptoms. it consists of nine items, each rated on a four-point scale (from 0=not at all to 3=almost every day). the score range is 0-27 and higher scores indicate more severe depressive symptomatology. a cutoff of ≥10 has been recommended for the diagnosis of depressive symptoms, which provides adequate sensitivity and specificity [25] . in our study, a total score of phq-9≥10 was used for categorization (that is, with or without depressive symptoms) j o u r n a l p r e -p r o o f journal pre-proof [26] . the cronbach's alpha for the phq-9 in the current sample was 0.89. the rese scale was developed to assess perceived self-efficacy in managing negative (neg) and in expressing positive (pos) affect [20] . the chinese version of this scale consists of a total of 17 items and has been validated and extensively utilized in the chinese population [27] . neg includes managing the efficacy of anger-irritation (ang), despondency-distress (des), and guilt and shame (com). pos includes the sense of efficacy of expressing happiness (hap) and pride (glo). participants rated their capability to manage their emotional life with responses ranging from 1=not well at all to 5=very well. the higher the score, the higher the sense of rese. in our study, participants with a score less than the median were defined as the low rese group, and those with a score higher than the median was defined as the high rese group. the cronbach's alpha for the rese scale in the current sample was 0.93. first, descriptive analyses were conducted to describe demographic characteristics, living rhythms, depressive symptoms, and rese status of students during the covid-19 pandemic. second, the data for depressive symptoms and rese were checked using the kolmogorov-smirnov test. results showed that they were non-normal continuous variables. the prevalence of depressive symptoms and rese stratified by demographic characteristics and living rhythms characteristics were reported, and mann-whitney u tests or kruskal-wallis tests were used to compare the differences between groups as appropriate. third, binary logistic regression the average score for depressive symptoms overall was 3.47±4.12. according to the cutoff of phq-9≥10, which has been recommended for the diagnosis of depressive symptoms, depressive symptoms was detected in 7.7% of participants. we summarized the mean scores for each of the phq-9 items as responded to by the college students with depressive symptoms, as shown in table 2 . we found that "lack of energy", "lack of pleasure" and "sleep disorder" were the main symptoms with the highest average scores. during the past two weeks, the incidence of suicidal ideation was 7.2%: 0.4% (1,143) of the students had suicidal thoughts almost every day, 0.9% (2,804) had suicidal thoughts for more than a week, and 5.9% (18,986) had suicidal thoughts over a few days. the average score for the rese was 63.95±9.88. the score for pos (4.12±0.60) was higher than for neg (3.56±0.70), especially for ang (3.53±0.77) which was the lowest score overall. we found significant differences in the rese scores between students with and without depressive symptoms: students with depressive symptoms had lower levels of rese (55.5±10.1) than those without depressive symptoms (64.7±9.5) (z=-134.58, p<.001). the living rhythms of students during the covid-19 pandemic are summarized in table 3 . browsing information relating to covid-19 was undertaken by 51.8% of the students for less than 1 h, by 37.8% for 1-2 h, 5.5.% for 3-5 h and 4.9.% for more j o u r n a l p r e -p r o o f than 5 h per day. depressive symptoms of the students differed according to the average time spent on pandemic information per day (χ 2 =99.02, df = 3, p<.001), and those focusing on it for more than 5 h per day had the highest scores of depressive symptoms. in terms of physical exercise, students who had not undertaken exercise in the past two weeks accounted for 21.3% of the total participants whereas 55.8% exercised for 30 minutes, and only 3.6% of the students exercised for more than 1 h. non-parametric tests showed that students who exercised for more than 30 minutes had lower depressive symptoms and higher rese scores compared to the other students (χ 2 =6,151.94, df = 3, p<.001 and χ 2 =1,5035.48, df = 3, p<.001; respectively). as for sleeping rhythms, most students would wake up from 6:00-8:00 and go to bed from 22:00-24:00. many students got up after 10:00 or irregularly, accounting for 33.1% and 13.7% of the participants, respectively. more than half of the students went to bed later than 24:00 or irregularly, accounting for 55.4% in total. there were statistically significant differences in depressive symptoms and rese scores according to the sleeping rhythms of college students (as shown in table 3). in terms of diet, 10.7% of students "seldom" or "never" kept to the usual habit of three meals per day and their depressive symptoms were significantly higher and rese scores were significantly lower than the other students (χ 2 =27,835.34, df = 3, p<.001 and χ 2 significantly associated with depressive symptoms and rese ( in response to the covid-19 pandemic, the national health commission of china released the notification of basic principles for emergency psychological crisis interventions on 26 january 2020 [28] . the chinese ministry of education has issued official documents to support colleges and universities in providing psychological j o u r n a l p r e -p r o o f assistance hotlines on 28 january 2020 [29] . to date, there remains few epidemiological data on the mental health problems and psychiatric morbidity of college students. a recent study on the prevalence of anxiety in chinese college students suggested that the covid-19-related stressors (included effects on daily-life, and academic delays) were positively associated with the level of anxiety symptoms, indicating that the mental health of college students could be affected by public health emergencies [24] . in this study, we supplemented the effect of covid-19 pandemic on the prevalence of depressive symptoms in chinese college students and explored potential influencing factors. evidence indicated that major mental health burdens of the public during the covid-19 outbreak included anxiety symptoms, depressive symptoms and sleep disorder [16] . a study surveyed immediate psychological responses among chinese general population by using the 21-item depression anxiety stress scale (dass-21), showing that 4.3% respondents were considered to suffer from severe and extremely severe depressive symptoms (dass-21≥21) in the initial outbreak of the pandemic (from january 31 to february 2, 2020) [6] . there was no significant longitudinal reduction in depressive level in the pandemic's peak four weeks later (from february 28 to march 1, 2020) [7] . besides, student status was associated with a greater psychological impact of the outbreak, higher levels of stress, and depression [6] . similarly, our study demonstrated that 7.7% of students reported depressive symptoms, relatively higher than the detection rates of depressive symptoms among general population during the pandemic. the incidence of suicidal ideation in our j o u r n a l p r e -p r o o f study was 7.2%. according to a survey conducted on chinese college students in 2014, 9.1% (n=479) of the 5,245 students reported they have at some point thought about committing suicide [30] . one meta-analysis has shown that the overall pooled prevalence of suicidal ideation among chinese college students is 10.72% [31] . furthermore, the low prevalence of suicidal ideation in our study corresponds to the findings of recent epidemiological studies during the covid-19 pandemic. an online survey of 673 healthy adults by tan w et al. found that the prevalence of suicidal ideation was less than 2% [8] . another study of people with and without psychiatric illnesses during the pandemic demonstrated that only 0.9% of healthy control reported suicidal ideation [32] . to some extent, while the current pandemic takes away health and life, college students are reminded to think about the theme of death. although covid-19 has created considerable stress and worry along with having significant impacts on daily life, healthcare professionals are on the front lines, battling diseases and caring for sick and dying patients, even while knowingly putting themselves at risk [33] . the determination of healthcare professionals to save lives as reported in the news is one form of positive life education. this may account for the high prevalence of depressive symptoms but relatively low suicidal ideation among college students during the pandemic as reported here. in terms of demographic factors, female respondents showed significantly higher depressive symptoms than their male counterparts. this finding corresponds to previous study which found that women were at higher risk of depression [34] . another recent study also reported that female had a greater psychological impact as j o u r n a l p r e -p r o o f journal pre-proof well as higher levels of depression during the outbreak of covid-19 [6] . several studies have found that senior students have higher depressive symptoms compared with freshmen [35, 36] , similar to our findings. this may be because the higher grade students face more stressful events, such as graduation and employment [35] . a recent study demonstrated that majority of chinese people stayed at home for 20-24 h per day during the covid-19 pandemic (84.7%) [6] . it is necessary to learn about the relationship between the living rhythms and the mental health when they were quarantined at home. our study noted that college students who went to bed early (before 24:00), rose early (6:00-8:00), undertook physical exercise and kept a regular diet had fewer depressive symptoms. disruptions to circadian rhythms can lead to affective changes and may elicit or exacerbate symptoms in individuals with a predisposition for mental health disorders [37, 38] . the associations between physical exercise, depression, and stress have already been widely discussed. physical exercise was significantly and meaningfully associated with self-reported mental health burden, and might also mitigate the social isolation stress on immune system [39, 40] . furthermore, our research shows that during the covid-19 pandemic, students who browse the pandemic information for less than 2 h a day have the lowest levels of depressive symptoms, and those who browse events for more than 5 h have the highest levels of depressive symptoms. this is consistent with the results by yeen huang and colleagues that young people who spend too much time browsing information of the covid-19 pandemic are at high risk for mental illness [16] . from the perspective of stress theory, excessive attention on information of the current j o u r n a l p r e -p r o o f journal pre-proof pandemic may lead to alternative trauma [41] . although college students quarantined at home have no personal experience of infection and treatment, they show empathy through visceral experience when browsing the information about the death of patients and health care workers. and some college students might study in healthcare subjects and they are empathetic about the high stress levels experienced by healthcare workers [42, 43] . excessive empathy can also cause psychological pain and trauma [44] . we also found that favorable living rhythms are conducive to maintaining good levels of rese, which refers to the degree of confidence with which individuals can effectively regulate their emotional state. this self-confidence can help individuals to use positive emotional regulation methods to adjust their emotions effectively [22] . in our study, the factor score for pos was higher than for neg, and the score for ang was the lowest. these results remind us of the need to pay attention to strengthening college students' skills in regulating negative emotions, in particular in regulating anger. studies have shown that low levels of rese can affect mental health, especially depressive symptoms [21] . rese is a positive psychological quality that can help individuals fight depressive symptoms, anxiety and other negative emotions [19] . people with good rese are less likely to have depressive symptoms [45] , that is, different rese levels have different probabilities for depressive symptoms. therefore, rese is also a protective factor against depressive symptoms in difficult circumstances such as the current pandemic. the findings of this study lead to several recommendations for future interventions. first, we recommend that methods of emotion regulation should be publicized to college students during the pandemic, particularly relating to the relationship between living rhythms and emotion, and skills for improving negative emotion regulation efficacy. this could be achieved through online means. second, we suggest that the home-school alliance should promote the need for students to create positive living rhythms, such as going to bed early, rising on time, and consuming regular meals. pandemic should be provided, and due in course, information about the return to school arrangements and academic tasks should be provided to address students' sense of uncertainty and improve their sense of control over life. fourth, we suggest that college students try to limit their time spent on the current pandemic to no more than 2 h a day, only pay attention to the necessary information (such as clear facts and data) [46] . fifth, colleges and universities should take the following personal and organizational prevention measures before the school reopen, such as keeping hand hygiene and wearing face masks, as well as good ventilation. improvement of school hygiene and increased concerns on students' physical health status could be helpful for reducing psychiatric problems [32] . this study has several limitations. first, the data and relevant analyses presented here were derived from a cross-sectional design, so it is difficult to make causal inferences. second, the study adopts a network survey method because of quarantine. the sampling in our study was voluntary and conducted online but the possibility of bias j o u r n a l p r e -p r o o f should be nonetheless be considered. third, based on the convenience sampling method, the participants in this study were college students in guangdong province. as a result, it cannot be applied to other provinces in china or other countries. in future research, stratified sampling needs to be carried out across the chinese provinces in order to obtain a more comprehensive understanding of the situation of chinese college students. in conclusion, this observational cross-sectional clinical study aimed to determine the living rhythms and mental health of college students during quarantine at home during the covid-19 pandemic. through conducting a large-scale survey, we found that staying up late or irregular bedtimes were prominent problems experienced by college students when quarantined at home, which is worth paying attention to. favorable living rhythms and moderate attention towards the pandemic may be protective factors for depressive symptoms and rese. our findings can be used to formulate psychological interventions aiming to prevent psychological problems in college students during the covid-19 pandemic. the authors have no competing interests to report. the mental health of medical workers in wuhan, china dealing with the 2019 novel coronavirus trend and forecasting of the covid-19 outbreak in china psychological responses, behavioral changes and public perceptions during the early phase of the covid-19 outbreak in china: a population based cross-sectional survey psychosocial and coping responses within the community health care setting towards a national outbreak of an infectious disease timely mental health care for the 2019 novel coronavirus outbreak is urgently needed immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (covid-19) epidemic among the general population in china a longitudinal study on the mental health of general population during the covid-19 epidemic in china do psychiatric patients experience more psychiatric symptoms during covid-19 pandemic and lockdown? a case-control study with service and research implications for immunopsychiatry mental health status and its influencing factors among college students during the epidemic of new coronavirus pneumonia circadian disruption in psychiatric disorders a novel approach of consultation on 2019 novel coronavirus (covid-19)-related psychological and mental problems the psychological impact of quarantine and how to reduce it: rapid review of the evidence prediction of depression scores from aerobic fitness, body fatness, physical activity, and vagal indices in non-exercising generalized anxiety disorder, depressive symptoms and sleep quality during covid-19 epidemic in china: a web-based cross-sectional survey guiding opinions of the office of the leading group of the ministry of education in response to novel coronavirus's pneumonia epidemic on doing a good job in the organization and management of online teaching in colleges and universities during the prevention and control of the epidemic beliefs about emotion: implications for avoidance-based emotion regulation and psychological health emotional self-efficacy moderates anxiety-related impairments in math performance in elementary school-age youth assessing regulatory emotional self-efficacy in three countries counteracting depression and delinquency in late adolescence: the role of regulatory emotional and interpersonal self-efficacy beliefs role of affective self-regulatory efficacy in diverse spheres of psychosocial functioning depressive symptoms, post-traumatic stress symptoms and suicide risk among graduate students: the mediating influence of emotional regulatory self-efficacy the psychological impact of the covid-19 epidemic on college students in china screening and case finding for major depressive disorder using the patient health questionnaire (phq-9): a meta-analysis covid-19 in wuhan: immediate psychological impact on 5062 health workers revision of the scale of regulatory emotional self-efficacy national health commission of the people's republic of china. the guideline of psychological crisis intervention for 2019-ncov pneumonia ministry of education of the people's republic of china. the ministry of education deploys the education system to open psychological support hotlines and network counseling service for the pneumonia epidemic associations between impulsivity, aggression, and suicide in chinese college students prevalence of suicidal ideation in chinese college students: a meta-analysis is returning to work during the covid-19 pandemic stressful? a study on immediate mental health status and psychoneuroimmunity prevention measures of chinese workforce clinician mental health and well-being during global healthcare crises: evidence learned from prior epidemics for covid-19 pandemic risk factors for depression: an autobiographical review depression among chinese university students: prevalence and socio-demographic correlates depressive symptomatology among university students in denizli, turkey: prevalence and j o u r n a l p r e -p r o o f sociodemographic correlates timing of light exposure affects mood and brain circuits eveningness and insomnia: independent risk factors of nonremission in major depressive disorder association between physical exercise and mental health in 1·2 million individuals in the usa between 2011 and 2015: a cross-sectional study the immunological case for staying active during the covid-19 pandemic secondary trauma among trauma researchers: lessons from the field a multinational, multicentre study on the psychological outcomes and associated physical symptoms amongst healthcare workers during covid-19 outbreak psychological impact of the covid-19 pandemic on health care workers in singapore impact of trauma work on social work clinicians: empirical findings core self-evaluation, regulatory emotional self-efficacy, and depressive symptoms: testing two mediation models takaaki ohyama and heymann dl. rumors of disease in the global village: outbreak verification key: cord-343764-xan5g88w authors: butler-henderson, kerryn; crawford, joseph title: a systematic review of online examinations: a pedagogical innovation for scalable authentication and integrity date: 2020-09-22 journal: comput educ doi: 10.1016/j.compedu.2020.104024 sha: doc_id: 343764 cord_uid: xan5g88w digitization and automation across all industries has resulted in improvements in efficiencies and effectiveness to systems and process, and the higher education sector is not immune. online learning, e-learning, electronic teaching tools, and digital assessments are not innovations. however, there has been limited implementation of online invigilated examinations in many countries. this paper provides a brief background on online examinations, followed by the results of a systematic review on the topic to explore the challenges and opportunities. we follow on with an explication of results from thirty-six papers, exploring nine key themes: student perceptions, student performance, anxiety, cheating, staff perceptions, authentication and security, interface design, and technology issues. while the literature on online examinations is growing, there is still a dearth of discussion at the pedagogical and governance levels. learning and teaching is transforming away from the conventional lecture theatre designed to seat 100 to 10,000 passive students towards more active learning environments. in our current climate, this is exacerbated by covid-19 responses (crawford et al., 2020) , where thousands of students are involved in online adaptions of face-to-face examinations (e.g. online zoom rooms with all microphones and videos locked on). this evolution has grown from the need to recognize that students now rarely study exclusively and have commitments that conflict with their university life (e.g. work, family, social obligations). students have more diverse digitally capability (margaryan et al., 2011) and higher age and gender diversity (eagly & sczesny, 2009; schwalb & sedlacek, 1990) . continual change of the demographic and profile of students creates a challenge for scholars seeking to develop a student experience that demonstrates quality and maintains financial and academic viability (gross et al., 2013; hainline et al., 2010) . universities are developing extensive online offerings to grow their international loads and facilitate the massification of higher learning. these protocols, informed by growing policy targets to educate a larger quantity of graduates (e.g. kemp, 1999; reiko, 2001) , have challenged traditional university models of fully on-campus student attendance. the development of online examination software has offered a systematic and technological alternative to the end-of-course summative examination designed for final authentication and testing of student knowledge retention, application, and extension. as a result of the covid-19 pandemic, the initial response in higher education across many countries was to postpone examinations (crawford et al 2020). however, as the pandemic continued, the need to move to either an online examination format or alternative assessment became more urgent. this paper is a timely exploration of the contemporary literature related to online examinations in the university setting, with the hopes to consolidate information on this relatively new pedagogy in higher education. this paper begins with a brief background of traditional examinations, as the assumptions applied in many online examination environments build on the techniques and assumptions of the traditional face-to-face gymnasium-housed invigilated examinations. this is followed by a summary of the systematic review method, including search strategy, procedure, quality review, analysis, and summary of the sample. print-based educational examinations designed to test knowledge have existed for hundreds of years. the new york state education department has "the oldest educational testing service in the united states" and has been delivering entrance examinations since 1865 (johnson, 2009, p. 1; nysed, 2012) . in pre-revolution russia, it was not possible to obtain a diploma to enter university without passing a high-stakes graduation examinations (karp, 2007) . these high school examinations assessed and assured learning of students in rigid and highsecurity conditions. under traditional classroom conditions, these were likely a reasonable practice to validate knowledge. the discussion of authenticating learning was not a consideration at this stage, as students were face to face only. for many high school jurisdictions, these are designed to strengthen the accountability of teachers and assess student performance (mueller & colley, 2015) . in tertiary education, the use of an end-of-course summative examination as a form of validating knowledge has been informed significantly by accreditation bodies and streamlined financially viable assessment options. the american bar association has required a final course examination to remain accredited (sheppard, 1996) . law examinations typically contained brief didactic questions focused on assessing rote memory through to problem-based assessment to j o u r n a l p r e -p r o o f evaluate students' ability to apply knowledge (sheppard, 1996) . in accredited courses, there are significant parallels. alternatives to traditional gymnasium-sized classroom paper-and-pencil invigilated examinations have been developed with educators recognizing the limitations associated with single-point summative examinations (butt, 2018) . the objective structured clinical examinations (osce) incorporate multiple workstations with students performing specific practical tasks from physical examinations on mannequins to short-answer written responses to scenarios (turner & dankoski, 2008) . the osce has parallels with the patient simulation examination used in some medical schools (botezatu et al., 2010) . portfolios assess and demonstrate learning over a whole course and for extracurricular learning (wasley, 2008) . the inclusion of online examinations, e-examinations, and bring-your-own-device models have offered alternatives to the large-scale examination rooms with paper-and-pencil invigilated examinations. each of these offer new opportunities for the inclusion of innovative pedagogies and assessment where examinations are considered necessary. further, some research indicates online examinations are able to discern a true pass from a true fail with a high level of accuracy (ardid et al., 2015) , yet there is no systematic consolidation of the literature. we believe this timely review is critical for the progression of the field in first stepping back and consolidating the existing practices to support dissemination and further innovation. the pursuit of such systems may be to provide formative feedback and to assess learning outcomes, but a dominant rationale for final examinations is to authenticate learning. that is, to ensure the student whose name is on the student register, is the student who is completing the assessed work. the development of digitalized examination pilot studies and case studies are becoming an expected norm with universities developing responses to a growing online curriculum offering j o u r n a l p r e -p r o o f (e.g. al-hakeem & abdulrahman, 2017; alzu'bi, 2015; anderson et al., 2005; fluck et al., 2009; fluck et al., 2017; fluck, 2019; seow & soong, 2014; sindre & vegendla, 2015; steel et al., 2019; wibowo et al., 2016) . as many scholars highlight, cheating is a common component of the contemporary student experience (jordan, 2001; rettinger & kramer, 2009 ) despite that it should not be. some are theorizing responses to the inevitability of cheating from developing student capacity for integrity (crawford, 2015; wright, 2011) to enhancing detection of cheating (dawson & sutherland-smith, 2018 and legislation to ban contract cheating (amigud & dawson, 2020) . we see value in the pursuit of methods that can support integrity in student assessment, including during rapid changes to the curriculum. the objective of this paper is to summarize the current evidence on online examination methods, and scholarly responses to authentication of learning and the mitigation of cheating, within the confines of assessment that enables learning and student wellbeing. we scope out preparation for examinations (e.g. nguyen & henderson, 2020) to enable focus on the online exam setting specifically. to address the objective of this paper, a systematic literature review was undertaken, following the prisma approach for article selection (moher et al., 2009) . the keyword string was developed incorporating the u.s. national library of medicine (2019) mesh (medical subject headings) terms: [("online" or "electronic" or "digital") and ("exam*" or "test") and ("university" or "educat*" or "teach" or "school" or "college")]. the following databases dissertations) are not peer-reviewed, and we aimed to identify rigorous best practice literature, we excluded these. we subsequently conducted a general search in google scholar and found no additional results. all records returned from the search were extracted and imported into the covidence® online software by the first author. the online covidence® software facilitated article selection following the prisma approach. each of the 1,906 titles and abstracts were double-screened by the authors based on the eligibility criteria. we also excluded non-higher education examinations, given the context around student demographics is often considerably different than vocational education, primary and high schools. where there was discordance between the authors on a title or abstract inclusion or exclusion, consensus discussions were undertaken. the screening reduced the volume of papers significantly because numerous papers related to a different education context or involved online or digital forms of medical examinations. next, the full-text for selected abstracts were doublereviewed, with discordance managed through a consensus discussion. the papers selected following the double full-text review were accepted for this review. each accepted paper was reviewed for quality using the mmat system (hong et al., 2018) and the scores were calculated j o u r n a l p r e -p r o o f as high, medium, or low quality based on the matrix (hong et al., 2018) . a summary of this assessment is presented in table 1 . following the process described by braun and clarke (2006) , an inductive thematic approach was undertaken to identify common themes identified in each article. this process involves six stages: data familiarization, data coding, theme searching, theme review, defining themes, and naming themes. familiarization with the literature was achieved during the screening, full-text, and quality review process by triple exposure to works. the named authors then inductively coded half the manuscripts each. the research team consolidated the data together to identify themes. upon final agreement of themes and their definitions, the write-up was split among the team with subsequent review and revision of ideas in themes through independent and collaborative writing and reviewing (creswell & miller, 2000; lincoln & guba, 1985) . this resulted in nine final themes, each discussed in-depth during the discussion. there were thirty-six (36) articles identified that met the eligibility criteria and were selected following the prisma approach, as shown in figure 1 . the selected articles are from a wide range of discipline areas and countries. table 1 summarizes the characteristics of the selected articles. the united states of america held a vast majority (14, j o u r n a l p r e -p r o o f 38.9%) of the publications on online examinations, followed by saudi arabia (4, 11.1%), china (2, 5.6%), and australia (2, 5.6%). when aggregated at the region-level, there was an equality of papers from north america and asia (14, 38.9% each), with europe (6, 16.7%) and oceania (2, 5.6%) least represented in the selection of articles. there has been considerable growth in publications in the past five years, concerning online examinations. publications between the years 2009 and 2015 represented a third (12, 33.3%) of the total number of selected papers. the majority (24, 66.7%) of papers were published in the last three years. papers that described a system but did not include empirical evidence scored a low-quality rank as they did not meet many of the criteria that relate to the evaluation of a system. when examining the types of papers, the majority (30, 83.3%) were empirical research, with the remainder commentary papers (6, 16.7%). of the empirical research papers, three-quarters of the paper reported a quantitative study design (32, 88.9%) compared to two (5.6%) qualitative study designs and two (5.6%) that used a mixed method. for quantitative studies, there was a range between nine and 1,800 student participants (x̄ = 291.62) across 26 studies, and a range between two and 85 staff participants (x̄ = 30.67) in one study. the most common quantitative methods were self-administered surveys and analysis of numerical examination student grades (38% each). qualitative and mixed methods studies only adopted interviews (6%). only one qualitative study reported a sample of students (n = 4), with two qualitative studies reporting a sample of staff (n = 2, n = 5). students perceive the online examinations saves time (75.0% of students surveyed) and is more economical (87.5%) than paper examinations (attia, 2014) . it provides greater flexibility for completing examinations (schmidt, ralph & buskirk, 2009 ) with faster access to remote student papers (87.5%) and students trust the result of online over paper-based examinations (78.1%: attia, 2014) . the majority of students (59.4%: attia, 2014; 55.5%: pagram et al., 2018) perceive that the online examination environment makes it easier to cheat. more than half (56.25%) of students believe that a lack of information communication and technology (ict) skill do not adversely affect performance in online examinations (attia 2014) . nearly a quarter (23%) of students reported (abdel karim & shukur, 2016 ) the most preferred font face (type) was arial, a font also recommended by vision australia (2014) in their guidelines for online and print inclusive design and legibility considerations. nearly all (87%) students preferred black text color on a white background color (87%). with regards to onscreen time counters, a countdown counter was the most preferred option (42%) compared to a traditional analogue j o u r n a l p r e -p r o o f clock (30%) or an ascending counter (22%). many systems allow students to set their preferred remaining time reminder or alert, including 15 minutes remaining (35% students preferred), 5 minutes remaining (26%), mid-examination (15%) or 30 minutes remaining (13%). several studies in the sample referred to a lack of score variation between the results of examination across different administration methods. for example, student performance did not have significant difference in final examination scores across online and traditional examination modalities (gold & mozes-carmel, 2017) . this is reinforced by a test of validity and reliability of computer-based and paper-based assessment that demonstrated no significant difference (oz & ozturan, 2018) , and equality of grades identified across the two modalities (stowell & bennett, 2010) . when considering student perceptions, of the studies documented in our sample, there tended to be favorable ratings of online examinations. in a small sample of 34 postgraduate students, the respondents had positive perceptions towards online learning assessments (67.4%). the students also believed it contributed to improved learning and feedback (67.4%), and 77 percent had favorable attitudes towards online assessment (attia, 2014) . in a pre-examination survey, students indicated they preferred to type than to write, felt more confident about the examination, and had limited issues with software and hardware (pagram, 2018) . with the same sample in a post-examination survey, within the design and technology examination, students felt the software and hardware were simple to use, yet many students did not feel at ease from their use of an e-examination . j o u r n a l p r e -p r o o f rios and liu (2017) compared proctored and non-proctored online examinations across several aspects, including test-taking behavior. their study did not identify any difference in the test-taking behavior of students between the two environments. there was no significant difference between omitted items and not-reached items. furthermore, with regards to rapid guessing, there was no significant difference. a negligible difference existed for students aged older than thirty-five years, yet gender was a nonsignificant factor. scholars have an increasing awareness of the role that test anxiety has in reducing student success in online learning environments (kolski & weible, 2018) . the manuscripts identified by the literature scan, identified inconsistencies of results for the effect that examination modalities have on student test anxiety. a study of 69 psychology undergraduates identified that students who typically experienced high anxiety in traditional test environments had lower anxiety levels when completing an online examination (stowell & bennett, 2010) . in a quasi-experimental study (n = 38 nursing students), when baseline anxiety is controlled, students in computer-based examinations had higher degrees of test anxiety. in 34 postgraduate student interviews, only three opposed online assessment based on perceived lack of technical skill (e.g. typing; attia, 2014) . around two-thirds of participants identified some form of fear-based on internet disconnection, electricity, slow typing, or family disturbances at home. a 37 participant community college study used proximal indicators (e.g. lip licking and biting, furrowed eyebrows, and seat squirming) to assess the rate of test anxiety in webcam-based examination proctoring (kolski & weible, 2018) . teacher strategies to reduce anxiety in their students include enabling students to consider, review, and acknowledge their j o u r n a l p r e -p r o o f anxieties (kolski & weible, 2018) . responses such as students writing of their anxiety, or responding to multiple-choice questionnaire on test anxiety, reduced anxiety. students in the test group and provided anxiety items or expressive writing exercises, performed better (kumar, 2014). cheating was the most prevalent area among all the themes identified. cheating in asynchronous, objective, and online assessments is argued by some to be at unconscionable levels (sullivan, 2016) . in one survey, 73.6 percent of students felt it was easier to cheat on online examinations than regular examinations (aisyah et al., 2018) . this is perhaps because students are monitored in paper and pencil examinations, compared to online examinations where greater control of variables is required to mitigate cheating. some instructors have used randomized examination batteries to minimize cheating potential through peer-to-peer sharing (schmidt et al., 2009 ). scholars identify various methods for mitigating cheating. identifying the test taker, preventing examination theft, unauthorized use of textbook/notes, preparing a set-up for online examination, unauthorized student access to a test bank, preventing the use of devices (e.g. phone, bluetooth, and calculators), limiting access to other people during the examination, equitable access to equipment, identifying computer crashes, inconsistency of method for proctoring (hearn moore et al., 2017) . in another, the issue for solving cheating is social as well as technological. while technology is considered the current norm for reducing cheating, these tools have been mostly ineffective (sullivan, 2016) . access to multiple question banks through effective quiz design and delivery is a mechanism to reduce the propensity to cheat, by reducing the stakes through multiple delivery attempts (sullivan, 2016) . question and answer j o u r n a l p r e -p r o o f randomization, continuous question development, multiple examination versions, open book options, time stamps, and diversity in question formats, sequences, types, and frequency are used to manage the perception and potential for cheating. in the study with mba students, perception of the ability to cheat seemed to be critical for the development of a safe online examination environment (sullivan, 2016) . dawson (2016) in a review of bring-your-own-device examinations including: • copying contents of usb to a hard drive to make a copy of the digital examination available to others, • use of a virtual machine to maintain access to standard applications on their device, • usb keyboard hacks to allow easy access to other documents (e.g. personal notes), • modifying software to maintain complete control of their own device, and • a cold boot attack to maintain a copy of the examination. the research on cheating has focused mainly on technical challenges (e.g. hardware to support cheating), rather than ethical and social issues (e.g. behavioral development to curb future cheating behaviors. the latter has been researched in more depth in traditional assessment methods (e.g. wright, 2015) . in a study on massive open online courses (moocs), motivations for students to engage in optional learning stemmed from knowledge, work, convenience, and personal interest (shapiro et al., 2017) . this provides possible opportunities for future research to consider behavioral elements for responding to cheating, rather than institutional punitive arrangements. j o u r n a l p r e -p r o o f schmidt, ralph and buskirk (2009) also examined the perceptions of academics with regards to online examination. academics reported that their biggest concern with using online examinations is the potential for cheating. there was a perception that students may get assistance during an examination. the reliability of the technology is the second more critical concern of academic staff. this includes concerns about internet connectivity as well as computer or software issues. the third concern is related to ease of use, both for the academic and for students. academics want a system that is easy and quick to create, manage and mark examinations, and students can use with proficient ict skills (schmidt et al., 2009) . furthermore, staff reported in a different study that marking digital work was easier and preferred it over paper examinations because of the reduction in paper (pagram et al., 2018) . they believe preference should be given to using university machines instead of the student using their computer, mainly due to issues around operating system compatibility and data loss. authentication was recognized as a significant issue for examination. some scholars indicate that the primary reason for requiring physical attendance to proctored examinations is to validate and authenticate the student taking the assessment (chao et al., 2012) . importantly, the validity of online proctored examination administration procedures is argued as lower than proctored oncampus examinations (rios & liu, 2017) . most responses to online examinations use bringyour-own-device models where laptops are brought to traditional lecture theatres, use of software on personal devices in any location desired, or use of prescribed devices in a classroom setting. the primary goal of each is to balance the authentication of students and maintain the integrity and value of achieving learning outcomes. in a review of current authentication options (abumansoor, 2017), the use of fingerprint reading, streaming media, and follow-up identifications were used to authenticate small cohorts of students. some learning management systems (lms) have developed subsidiary products (e.g. weaver within moodle) to support authentication processes. some biometric software uses different levels to authenticate keystrokes for motor controls, stylometry for linguistics, application behavior for semantics, capture to physical or behavioral samples, extraction of some scholars recommend online synchronous assessments as an alternative to traditional proctored examinations while maintaining the ability to manually authenticate (chao et al., 2012) . in these assessments: quizzes are designed to test factual knowledge, practice for procedural, essay for conceptual, and oral for metacognitive knowledge. a 'cyber face-to-face' element is required to enable the validation of students. the interface of a system will impact on whether a student perceives the environment to be an enabler or barrier for online examinations. abdel karim and shukur (2016) summarized the potential interface design features that emerged from a systematic review of the literature on this topic, as shown in table 2 . the incorporation of navigation tools has also been identified by students and staff as an essential design feature (rios & liu, 2017) , as is an auto-save functionality (pagram et al., 2018). none of the studies that included technological problems in its design reported any issues (böhmer et al., 2018; matthíasdóttir & arnalds, 2016; schmidt et al., 2009) . one study stated j o u r n a l p r e -p r o o f that 5 percent of students reported some problem ranging from a slow system through to the system not working well with the computer operating system, however, the authors stated no technical problems that resulted in the inability to complete the examination were reported (matthíasdóttir & arnalds, 2016) . in a separate study, students reported that they would prefer to use university technology to complete the examination due to distrust of the system working with their home computer or laptop operating system or the fear of losing data during the examination (pagram et al., 2018) . while the study did not report any problems loading on desktop machines, some student laptops from their workplace had firewalls, and as such had to load the system from a usb. this systematic literature review sought to assess the current state of literature concerning online examinations and its equivalents. for most students, online learning environments created a system more supportive of their wellbeing, personal lives, and learning performance. staff preferred online examinations for their workload implications and ease of completion, and basic evaluation of print-based examination logistics could identify some substantial ongoing cost savings. not all staff and students preferred the idea of online test environments, yet studies that considered age and gender identified only negligible differences (rios & liu, 2017) . while the literature on online examinations is growing, there is still a dearth of discussion at the pedagogical and governance levels. our review and new familiarity with papers led us to point researchers in two principal directions: accreditation and authenticity. we acknowledge that there are many possible pathways to consider, with reference to the consistency of application, the validity and reliability of online examinations, and whether online examinations enable better measurement and greater student success. there are also opportunities to synthesize j o u r n a l p r e -p r o o f online examination literature with other innovative digital pedagogical devices. for example, immersive learning environments (herrington et al., 2007) , mobile technologies (jahnke & liebscher, 2020) ; social media (giannikas, 2020), and web 2.0 technologies (bennett et al., 2012) . the literature examined acknowledges key elements of the underlying needs for online examinations from student, academic, and technical perspectives. this has included the need for online examinations need to accessible, need to be able to distinguish a true pass from a true fail, secure, minimize opportunities for cheating, accurately authenticates the student, reduce marking time, and designed to be agile in software or technological failure. we turn attention now to areas of need in future research, and focus on accreditation and authenticity over these alternates given there is a real need for more research prior to synthesis of knowledge on the latter pathways. the influence of external accreditation bodies was named frequently and ominously among the sample group, but lacked clarity surrounding exact parameters and expectations. rios (2017, p. 231) identified a specific measure was used "for accreditation purposes". hylton et al. (2016, p. 54) specified that the us department of education requires "appropriate procedures or technology are implemented" to authentic distance students. gehringer and peddycord (2013) empirically found that online/open-web examinations provided more significant data for accreditation. underlying university decisions to use face-to-face invigilated examination settings is to enable authentication of learning -a requirement of many governing bodies globally. the continual refinement of rules has enabled a degree of assurance that students are who they say they are. nevertheless, sophisticated networks have been established globally to support direct student cheating from completing quick assessments and calculators with secret search engine capability through to full completion of a course inclusive of attending on-campus invigilated examinations. the authentication process in invigilated examinations does not typically account for distance students who have a forged student identification card to enable a contract service to complete their examinations. under the requirement assure authentication of learning, invigilated examinations will require revision to meet contemporary environments. the inclusion of a broader range of big data from keystroke patterns, linguistics analysis, and whole-of-student analytics over a student lifecycle is necessary to identify areas of risk from the institutional perspective. where a student has a significantly different method of typing or sentence structure, it is necessary to review. an experimental study on the detection of cheating in a psychology unit found teachers could detect cheating 62 percent of the time (dawson & sutherland-smith, 2017) . automated algorithms could be used to support the pre-identification of this process, given lecturers and professors are unlikely to be explicitly coding for cheating propensity when grading multiple hundreds of papers on the same topic. future scholars should be considering the innate differences that exist among test-taking behaviors that could be codified to create pattern recognition software. even in traditional invigilated examinations, the use of linguistics and handwriting evaluations could be used for cheating identification. the literature identified in the sample discussed with limited depth the role of authentic assessment in examinations. the evolution of pedagogy and teaching principles (e.g. j o u r n a l p r e -p r o o f constructive alignment; biggs, 1996) have paved the way for revised approaches to assessment and student learning. in the case of invigilated examinations, universities have been far slower to progress innovative solutions despite growing evidence that students prefer the flexibility and opportunities afforded by digitalizing exams. university commitments to the development of authentic assessment environments will require a radical revision of current examination practice to incorporate real-life learning processes and unstructured problem-solving (williams & wong, 2009 ). while traditional examinations may be influenced by financial efficacy, accreditation, and authentication pressures, there are upward pressures from student demand, student success, and student wellbeing to create more authentic learning opportunities. the online examination setting offers greater connectivity to the kinds of environments graduates will be expected to engage in on a regular basis. the development of time management skills to plan times to complete a fixed time examination is reflected in the business student's need to pitch and present at certain times of the day to corporate stakeholders, or a dentist maintaining a specific time allotment for the extraction of a tooth. the completion of a selfregulated task online with tangible performance outcomes is reflected in many roles from lawyer briefs on time-sensitive court cases to high school teacher completions of student reports at the end of a calendar year. future practitioner implementation and evaluation should be focused on embedding authenticity into the examination setting, and future researchers should seek to understand better the parameters by which online examinations can create authentic learning experiences for students. in some cases, the inclusion of examinations may not be appropriate; and in these cases, they should be progressively extracted from the curriculum. j o u r n a l p r e -p r o o f as institutions begin to provide higher learning flexibility to students with digital and blended offerings, there is scholarly need to consider the efficacy of the examination environment associated with these settings. home computers and high-speed internet are becoming commonplace (rainie & horrigan, 2005) , recognizing that such an assumption has implications for student equity. as warschauer (2007, p. 41) puts it, "the future of learning is digital". our ability as educators will be in seeking to understand how we can create high impact learning opportunities while responding to an era of digitalization. research considering digital fluency in students will be pivotal (crawford & butler-henderson, 2020) . important too, is the scholarly imperative to examine the implementation barriers and successes associated with online examinations in higher education institutions given the lack of clear cross-institutional case studies. there is also a symbiotic question that requires addressing by scholars in our field, beginning with understanding how online examinations can enable higher education, and likewise how higher education can shape and inform the implementation and delivery of online examinations. this study adopted a rigorous prisma method for preliminary identification of papers for inclusion, the mmat protocol for identifying the quality of papers, and an inductive thematic analysis for analyzing papers included. these processes respond directly to limitations of subjectivity and assurance of breadth and depth of literature. however, the systematic literature review method limits the papers included by the search criteria used. while we opted for a broad set of terms, it is possible we missed papers that would typically have been identified in other this systematic literature review considered the contemporary literature on online examinations and their equivalents. we discussed student, staff, and technological research as it was identified in our sample. the dominant focus of the literature is still oriented on preliminary evaluations of implementation. these include what processes changed at a technological level, and how students and staff rated their preferences. there were some early attempts to explore the effect of j o u r n a l p r e -p r o o f online examinations on student wellbeing and student performance, along with how the changes affect the ability for staff to achieve. higher education needs this succinct summary of the literature on online examinations to understand the barriers and how they can be overcome, encouraging greater uptake of online examinations in tertiary education. one of the largest barriers is perceptions of using online examinations. once students have experienced online examinations, there is a preference for this format due to its ease of use. the literature reported student performance did not have significant difference in final examination scores across online and traditional examination modalities. student anxiety decreased once they had used the online examination software. this information needs to be provided to students to change students' perceptions and decrease anxiety when implementing an online examination system. similarly, the information summarized in this paper needs to be provided to staff, such as the data related to cheating, reliability of the technology, ease of use, and reduction in time for establishing and marking examinations. when selecting a system, institutions should seek one that includes biometrics with a high level of precision, such as user authentication, and movement, sound, and keystroke monitoring (reporting deviations so the recording can be reviewed). these features reduce the need for online examinations to be invigilated. other system features should include locking the system or browser, cloud-based technology so local updates are not required, and an interface design that makes using the online examination intuitive. institutions should also consider how it will address technological failures and digital disparities, such as literacy and access to technology. we recognize the need for substantially more evidence surrounding the postimplementation stages of online examinations. the current use of online examinations across disciplines, institutions, and countries needs to be examined to understand the successes and gaps. beyond questions of 'do students prefer online or on-campus exams', serious questions of how student mental wellbeing, employability, and achievement of learning outcomes can be improved as a result of an online examination pedagogy is critical. in conjunction is the need to break down the facets and types of digitally enhanced examinations (e.g. online, e-examination , byod examinations, and similar) and compare each of these for their respective efficacy in enabling student success against institutional implications. while this paper was only able to capture the literature that does exist, we believe the next stage of literature needs to consider broader implications than immediate student perceptions toward the achievement of institutional strategic imperatives that may include student wellbeing, student success, student retention, financial viability, staff enrichment, and student employability. funding: this research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. j o u r n a l p r e -p r o o f proposed features of an online examination interface design and its optimal values proposed bio-authentication system for question bank in learning management systems development of continuous authentication system on android-based online exam application developing a new e-exam platform to enhance the university academic examinations: the case of lebanese french university the effect of using electronic exams on students' achievement and test takers' motivation in an english 101 course the law and the outlaw: is legal prohibition a viable solution to the contract cheating problem online course evaluations: review of literature and a pilot study online exams for blended assessment. study of different application methodologies postgraduate students' perceptions toward online assessment: the case of the faculty of education, umm al-qura university education for a knowledge society in arabian gulf countries implementing web 2.0 technologies in higher education: a collective case study enhancing teaching through constructive alignment e-exams in engineering education-online testing of engineering competencies: experiences and lessons learned a comparison of online vs. proctored final exams in online classes financial aid and attainment among students in a state with changing demographics exploring the influence of facial verification software on student academic performance in online learning environments changing students, faculty, and institutions in the twenty-first century impeding students' efforts to cheat in online classes immersive learning technologies: realism and online authentic learning the mixed methods appraisal tool (mmat) version 2018 for information professionals and researchers. education for information utilizing webcam-based proctoring to deter misconduct in online exams three types of integrated course designs for using mobile technologies to support creativity in higher education history of new york state regents exams college student cheating: the role of motivation, perceived norms, attitudes, and knowledge of institutional policy exams in algebra in russia: toward a history of high stakes testing a decade of adoption: how the internet has woven itself into american life university reform in the post-massification era in japan: analysis of government education policy for the 21st century situational and personal causes of student cheating online proctored versus unproctored low-stakes internet test administration: is there differential test-taking behavior and performance? student verification system for online assessments: bolstering quality and integrity of distance learning utilizing online exams: a case study have college students' attitudes toward older people changed students' perceptions of byod open-book examinations in a large class: a pilot study an informal history of how law schools evaluate students, with a predictable emphasis on law school final exams e-exams and exam process improvement use of e-exams in high stakes law school examinations: student and staff reactions effects of online testing on student exam performance and test anxiety an integrated approach to preempt cheating on asynchronous, objective, online assessments in graduate business classes objective structured clinical exams: a critical review medical subject headings online and print inclusive design and legibility considerations the paradoxical future of digital learning a pilot study of an electronic exam system at an australian university the efficacy of final examinations: a comparative study of closed-book, invigilated exams and open-book, open-web exams distinguished scholar invited essay: reflections on the role of character in business education and student leadership development the research and design of online examination system key: cord-035077-qjrokb17 authors: konstantinov, vsevolod; berdenova, saule; satkangulova, gulnara; reznik, alexander; isralowitz, richard title: covid-19 impact on kazakhstan university student fear, mental health, and substance use date: 2020-11-09 journal: int j ment health addict doi: 10.1007/s11469-020-00412-y sha: doc_id: 35077 cord_uid: qjrokb17 nan the present study builds on covid-19-related research conducted in former ussr countries during the first wave of infection in april and may 2020. such efforts include validation of a survey research tool in russian language for russian-speaking university students reznik et al. 2020) . also, a comparative study was conducted across universities in the former soviet union-particularly russia and belarus that are very close in culture, religion, and language but have different strategies to combat covid-19. unlike these two countries, kazakhstan has a predominately muslim population and there is a dearth of youth and young adult covid-19 information about this important segment of the population linked to the future of the country. based on earlier covid-19 research of university students in russia and belorussia, we hypothesize covid-19 fear, mental health, and substance use among kazakh university students are linked to gender and religious status. specifically, we believe gender predicts fear level, mental health problems, and substance use; and religiosity is not a protective factor among university students in terms of covid-19-related mental health conditions. the methods used for this study of kazakh students are similar to those developed to determine covid-19 impact on russian and belarussian university students . the qualtrics software platform was used for this online survey. the main data collection instrument was the seven-item fear of covid-19 scale (fcv-19s) (ahorsu et al. 2020; reznik et al. 2020) . the levels of agreement with fcv-19s statements were evaluated by a 5-point likert scale from 1 (strongly disagree) to 5 (strongly agree). higher total scores correspond with more covid-19 fear. two questions were added to the scale to determine covid-19 impact on university student studies, social life, and family relations. the influence of covid-19 on student substance use including tobacco, alcohol, cannabis, and prescription drugs was examined. the survey instrument was translated from english to russian, a common language used for education and research purposes, and back translated. the translation method used is consistent with that described by the world health organization for research purposes (who 2020). the survey instrument was used on a cross-sectional sample of university students from baitursynov kostanay regional university (bkru), republic of kazakhstan. students were informed that their survey participation was voluntary and responses were confidential. permission to conduct the survey was received from bkru institutional research committee. all procedures performed in this study involving human participants were in accordance with the ethical standards of the bkru research committee and comparable ethical standards. on average, the online instrument took 10-15 min to complete; and 15.9% of the university student population responded. for the study cohort, the fcv-19s with two additional questions reliability is 0.795 (cronbach's alpha) and 0.813 (mcdonald's omega). additional questions were added to the survey instrument about covid-19 symptoms, testing, psycho-emotional conditions, and compliance with who prevention recommendations. for this study, all statistical analyses were conducted using spss, version 25. this survey included 466 participants-67.0% (n = 312) female and 33.0% (n = 154) male students. the response rate is 15.9%. the mean age of the respondents is 19.0 years (sd = 2.7); and 31.8% (n = 148) reported being not religious and 68.2% (n = 318) religious. student academic disciplines were grouped into 5 areas: health related including biology and veterinary science (13.4%); psychology (29.5%); social and humanitarian including education, philosophy, and journalism (25.6%); and mathematics, economics and technical (31.5%). table 1 provides background characteristics of the survey respondents. for all respondents, mean value of the fcv-19s is 22.1 (sd = 5.8) and a median score is 22.0 with a range of 9 to 41. the total distribution of the fear values among the kazakh students is close to the normal distribution (skewness = 0.087; kurtosis = − 0.086) (see fig. 1 ). based on the distribution of results, 40 respondents missing data, we gradated fear values ranging from 9 of 41 to represent the following levels: low-9 to 19 scores (n = 171); medium-20 to 24 scores (n = 159); and high-25 to 41 scores (n = 137). results evidence no significant difference in fear values based on respondent academic study area. however, higher fear levels on average were reported by females than by males (23.0 vs. 20.1 respectively; t 424 = 4.924; p < .001) and by religious than by secular students (23.1 vs. 20.0 respectively; t 424 = 5.146; p = .008). two-way anova did not evidence a significant influence of gender and religiosity interaction on fear values (f 1,422 = 2.117; n.s.). the last month rate of any substance use (i.e., tobacco, alcohol, cannabis, and/or prescription drugs) before covid-19 was 25.6% and, during covid-19, was 27.7% (n.s.). fear values were not found to be significantly linked to last month substance use (t 367 = 0.454; n.s.). secular, compared to religious, students reported more substance use before (37.9% vs. 20.2%; p < .001) and during (38.7% vs. 23.4%; p < .001) the covid-19 pandemic. in terms of who covid-19 prevention measures, 17.3% of the respondents reported infrequent or no mask use; and 36.1% indicated they do not practice social distancing. females, more than males, were more inclined to adopt such prevention measures (64.0% vs. 51.9%; p = .021). also, students who reported last month substance use, compared to nonusers, were less likely to practice covid-19 prevention (36.8% vs. 21.9; p = .001) and more likely to report lower covid-19 fear (21.3 vs. 22.6; t 366 = 1.993; p = .047). two-way anova did not reveal a significant influence of last month substance use and covid-19 prevention practice (i.e., mask use and distancing) on fear values (f 1,362 = 0.009; n.s.). regarding psychological and emotional conditions, female more than male students reported being depressed (25.4% vs. 6.8%; p < .001), lonely (28.3% vs. 12.3%; p < 0.001), nervous (31.3% vs. 15.0%; p = 0.001), and angry (25.8% vs. 13.6%; p = .006) due to covid-19. no significant difference was found among secular and religious students in terms of covid-19-related psychological and emotional conditions. regardless of gender and religious status, students who reported being more nervous due to covid-19 were more inclined to be last month substance users (36.1% vs. 21.0%; p = .002). students who were checked for covid-19 and/or had symptoms reported being more exhausted than those not tested (21.6% vs. 8.9%; p = .001). also, students who had someone close to them with covid-19 reported more depression (28.6% vs. 16.7%; p = .022) and exhaustion (20.3% vs. 10.0%; p = .018). table 2 shows the links between students' psychological and emotional conditions due to covid-19 fear. studies show pandemic, such as covid-19, increases psychological stress and the consequences of quarantine lead to mental health conditions such as depression, irritability, insomnia, anger, and exhaustion (brooks et al. 2020; prilutskaya and grjibovski 2020; sorokin et al. 2020) . the present fear level of kazakh students tends not to differ from that reported by russian and belarus university students at the peak of first wave infection during april to june 2020. furthermore, based on present findings, the vast majority (94.7%) of kazakh students reported their fear decreased or remained unchanged since the beginning of the pandemic; and, only 5.3% reported an increase. this low rate may be a result of student coping with covid-19 conditions over time with implications that this response may be happening elsewhere (ye et al. 2020) . present findings, consistent with earlier studies in russia and belarus, evidence a higher level of fear among religious female students . this outcome, hypothesized, raises question about the role of religiosity as a protective factor during difficult times (howell et al. 2019; koenig 2015) . the relationship between religiosity and psychological well-being has been documented (leondari and gialamas 2009) but not so in kazakhstan where islam is dominant. however, and important to note, religiosity among kazakh students is like that found among russia and belarus university students. it tends to be of a declarative nature not accompanied by the formation of religious consciousness and regular religious practice including prayers, going to church/mosque, and religious observance of restrictions (alimbekova et al. 2020) . kazakh study results, like those from russia and belarus, evidence that covid-19 does not appear to significantly affect the pattern of student substance use. substance use is higher among secular than religious students who tend to report less covid-19 fear and more disregard for who prevention guidelines (i.e., mask use and social distancing). the link between religion, religiosity, substance use, and problem behavior is well studied (ford and hill 2012; isralowitz and myers 2011) . present study findings evidence that the level of substance use among kazakh university students is lower than that reported in russia and other countries (isralowitz et al. 2018 ). this outcome may be a result of prohibitive government policy toward cannabis and islamic religious beliefs affecting alcohol use. this study, to the best of our knowledge, is the first to examine the impact of covid-19 on kazakh university students. in spite of the cross-sectional nature of the research and limited number of survey respondents that restrict generalizability of the results, present findings tend to confirm that found in other former soviet union block countries. furthermore, and important, present study results evidence a possible trend among university students toward resilience and coping with covid-19 conditions-an important step toward a return to pre-pandemic living conditions. further investigation is needed across kazakh universities and over time to develop a more detailed understanding of covid-19 impact on young adults linked to the development and future of the country. the fear of covid-19 scale: development and initial validation religiosity of the urban community in kazakhstan responses to the covid-19 crisis in russia the psychological impact of quarantine and how to reduce it: rapid review of the evidence the world fact book: kazakhstan religiosity and adolescent substance use: evidence from the national survey on drug use and health. substance use and misuse covid 19 fear, stress, anxiety, and substance use among russian and belarusian university students intolerance of uncertainty moderates the relations among religiosity and motives for religion, depression, and social evaluation fears illicit drugs alcohol use and related problem behavior: a survey among russian federation and israeli female university students coronavirus resource center religion, spirituality, and health: a review and update religiosity and psychological well-being perception of the state of emergency due to covid-19 by medical students and staff in a kazakhstani university covid-19 fear in eastern europe: validation of the fear of covid-19 scale structure of anxiety associated with the covid-19 pandemic in the russian-speaking sample: results from on-line survey process of translation and adaptation of instruments resilience, social support, and coping as mediators between covid-19-related stressful experiences and acute stress disorder among college students in china. applied psychology: health and well-being publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations acknowledgments we acknowledge the contribution of the original authors of "the fear of covid-19 scale: development and initial validation" (daniel kwasi ahorsu, chung-ying lin, vida imani, mohsen saffari, mark d. griffiths, and amir h. pakpour).funding the study was supported by the russian foundation for basic research (grant 20-013-00445 а). conflict of interest the authors declare that they have no conflict of interest.informed consent all procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the helsinki declaration of 1975, as revised in 2000 (5). informed consent was obtained from all students for being included in the study. key: cord-332859-j10n38ah authors: muzumdar, sonal; grant-kels, jane m.; feng, hao title: dear dermatoethicist: medical student dermatology rotations in the context of covid-19 date: 2020-06-24 journal: j am acad dermatol doi: 10.1016/j.jaad.2020.06.070 sha: doc_id: 332859 cord_uid: j10n38ah nan we frequently have medical students rotate through our department but, due to the covid-19 pandemic, we are hesitant to accept rotating students at this time. while we want to optimize the educational opportunities these students receive, we do not want to endanger trainees, staff and our patients. additionally, we are concerned about the impact that this may have on students interested in applying to dermatology. what should we do? dear program director, deciding whether to accept medical students in your department is challenging. beneficence is at play as rotating through dermatology benefits medical students as it is an excellent way for students to learn more about the field and help them determine if dermatology is in fact the specialty they would like to pursue. additionally, rotations help students obtain letters of recommendation from academic dermatologists. since the start of the pandemic, medical students rotating in dermatology have been temporarily sidelined as clinics have closed and are only seeing emergencies. as we reopen, far fewer in-person patient visits are being scheduled and many appointments are now virutal. 1 as a result, students may not benefit significantly from in-person rotations and creative solutions are needed. non-maleficence, or the avoidance of intentional harm, must also be considered. with the covid-19 pandemic, there is a risk that medical students may become infected themselves and subsequently infect their patients and fellow health care workers if allowed to participate in clinical rotations. 2 as medical students are learners, and not considered essential personnel, limiting their exposure to infectious patients is paramount. regarding the ethical principle of justice, limiting student rotations will harm certain groups of students more than others. dermatology is a competitive specialty and rotations and specialty-specific letters of recommendation have been cited as important factors in resident selection. 3 limiting rotations may thus disproportionately affect those with limited or no prior experience in the field. similarly, students without home dermatology departments, who typically rely on away rotations to secure letters of recommendation, are likely to be disadvantaged as well. follow the guidance of your medical school and institution when deciding whether to allow medical students to rotate in-person and ensure they have access to appropriate ppe during in-person encounters if allowed. prioritize safety while optimizing medical student education. creative options include 1) sitting in on virtual lectures and grand rounds sessions, 2) participating in teledermatology care, 3) engaging in case-based learning, and 4) performing scholarly activities. organizations, such as the association of program directors, are devising novel ways to involve students in learning experiences at different institutions. because many students will not have the opportunity to rotate at various dermatology programs or obtain dermatology letters of recommendation prior to applying, this is an opportune time to endeavor to evaluate the applicants based more upon intrinsic values beyond their academic achievements and credentials including their backgrounds and challenges overcome, whether their school has fewer dermatology opportunities, ties to a particular region, unique skills or interests, and aspirations. an assessment of united states dermatology practices during the covid-19 outbreak online ahead of print the role of medical students during the covid-19 pandemic nrmp program director survey. national resident matching program web site key: cord-267491-3ry0gguh authors: huntington-klein, nick; gill, andrew title: semester course load and student performance date: 2020-10-18 journal: res high educ doi: 10.1007/s11162-020-09614-8 sha: doc_id: 267491 cord_uid: 3ry0gguh many college students in the united states take longer than four years to complete their bachelor’s degrees. long time-to-degree can increase higher education costs by billions. time-to-degree can be reduced if students take more credits each term. while academic momentum theory suggests that additional credits may also improve student performance, and there is a strong positive correlation between course load and student performance, high course load may reduce time investment in each course, giving high course load a negative causal effect on performance. concern about the negative impact of course load on performance, especially for struggling students, may lead to pushback against policies to reduce time-to-degree by increasing course load. using longitudinal data from a regional four-year university with a high average time-to-degree, we find no evidence that high course loads have a negative impact on student grades, even for students at the low end of the performance distribution. this result is consistent with a model where students substitute time away from non-education activities when their course loads increase. increased time-to-degree from post-secondary institutions in the united states has taken a prominent position along with low completion rates, access, affordability, and mounting student debt as a major public-policy concern in higher education. among first-time full-time students seeking a bachelor's degree and commencing their studies in 2009, only 39.8% graduated from their first institution attended within 4 years (u.s. department of education 2017). the problem is especially severe at public institutions, where the same figure is 34.8% (58.6% at 6 years). the rate at private not-for-profit private institutions is 53.0% (65.6% at six years). these low four and six-year graduation rates are, in part, a reflection of long time-todegree for successful graduates. the average time enrolled for bachelor's degree completion between july 2014 and june 2015 was 5.2 years in public institutions and 4.8 years in private not-for-profit institutions (shapiro et al. 2016, appendix c: data tables) . among those receiving bachelor's degrees in public institutions, 29.9% were enrolled for 6 years and 18.2% were enrolled for 7 to 8 years. of all graduates, approximately 306,000 bachelor's degree recipients were enrolled for 6 years and 186,000 were enrolled for roughly 7.5 years. the extra time to complete a bachelor's degree can be costly. comparing the wages of current college enrollees aged 20-25 against current ba holders aged 20-25 from the current population survey, each additional year represents on average $23,081 in lost earnings, for a rough total of $29.2 billion in annual foregone wages. in addition, there may be as much as $12.5 billion in additional tuition outlay, given a the full-time average at public institutions of $9,970 (college board 2018) these calculations do not include the expenditures made by the government on education. there can be additional labor market costs if long time to degree is perceived as a negative labor market signal (witteveen and attewell 2019; brugiavini et al. 2020 ). as we discuss below, universities could reduce time-to-degree by encouraging students to take more credits (attewell and monaghan 2016) . many universities throughout the united states have embarked on such a policy (see, for example, venit 2017 ). there are, however, potential unintended negative consequences. taking additional credits could crowd out study time per course, harming student performance. if, on the other hand, there is no discernable effect of credits attempted on student performance, or if, under the concept of academic momentum (attewell, heil, and reisel 2012) , additional credits actually improve performance and persistence by aligning students with their education, advising students to take more credits could be a cost-effective method for improving time-to-degree. the present day covid 19 pandemic has altered the academic landscape and created additional challenges to course taking and timely degree completion. challenges for students include necessary hardware and software requirements for synchronous and asynchronous online instruction and disruptions to family and work life. at the extensive margin, students and their families are making enrollment decisions that will alter their academic trajectories. according to the united states census bureau's household pulse survey, a staggering 80.9% of adults in households where at least one adult was planning on taking post-secondary classes this fall have seen their plans cancelled or changed (united states census bureau, 2020, https ://www.censu s.gov/data-tools /demo/hhp/#/?measu res=csr). at the intensive margin, external pressures related the pandemic including work pressures, housing insecurity, and food insecurity may result in fewer classes attempted per semester. student engagement may suffer when courses are delivered online. the challenges related to covid-19 reinforce the importance of the issues we raise in the current study regarding timely degree completion. at the same time, it is not clear that the results in this study generalize to such an unusual situation, and a setting in which a large majority of students are forced to take classes online. university policy will have to address these and other challenges, with student support services and advising likely taking center stage. in this paper, we investigate the causal effect of course load on grades using a rich set of administrative data from a large four-year public university campus in california. this university has low four-year completion rates, and is currently pursuing a major shift in policy called "graduation initiative 2025" to improve those rates (california state university 2019), with higher course loads-taking 15 credits specifically, rather than 12-as a central part of the policy. similar initiatives are in place at many universities throughout the united states, including the university of hawaii (venit 2017) , the university of south dakota (https ://www.usd.edu/acpc/finis h-in-four), and oklahoma state university (https :// cas.oksta te.edu/advis ing/finis h-in-four) to name only a few. assessing whether course load affects student performance is complicated by the endogeneity of course load. in one empirical strategy, we isolate within-student variation and compare students' grades in semesters when they take 15 + credits against grades in semesters when they take fewer credits. however, our use of student fixed effects is unlikely to be enough to identify a causal effect, and we take four additional approaches to support and refine our causal interpretation. we examine the institutional context and find that a lot of variation in course load is driven by exogenous registration bottlenecks, we control for dynamic academic pressures, we use a time-varying simulated omitted predictor to calculate rosenbaum (2002) -like bounds for our estimate, and we examine coefficient stability using the methods in oster (2019) and cinelli & hazlett (2020) . in addition, to assess whether course load may affect struggling students differently, we examine whether the effect of course load on student performance varies along the gpa distribution using fixedeffects quantile regression methods (machado & santos silva 2019) . our results provide no evidence that taking 15 credits rather than 12 credits harms student performance. controlling for student and class-standing fixed effects, we find that taking one additional course per semester leads to a 0.011 standard deviation increase in course grade (on a four-point scale), rather than a negative effect of any size. our 0.011 estimate may be biased upwards by omitted time-varying pressures like family, work in the labor market, or health problems. so we do not report as our conclusion that a full course load improves grades. however, under reasonable assumptions we can reject that a full course load has a meaningfully large negative effect on grades, where a "meaningfully large effect" is about half the difference between a b and a b + . we include observed time-varying controls and do not find that our results become negative. we bound the sensitivity of our results to unobserved confounders in three different ways: by simulation, and by following the methods of oster (2019) and cinelli & hazlett (2020) . in each case we find that an unobserved confounder could plausibly reduce our positive result to 0 or a small negative value, but that the unobserved confounder would need to be very strong to produce a meaningfully large negative result. because our results reject the presence of a meaningful negative effect, they support a conclusion that policies encouraging 15 credits may effectively decrease time-to-degree without harming student performance. a policy encouraging students to take more credits is in contrast to other measures to improve completion time discussed in the literature, such as improving student preparedness. such policies tend to focus on reducing course failure, where much has been done and the marginal benefit of additional resources is likely to be low. in addition, a policy encouraging students to take more credits has appeal to universities because, relative to measures such as improving student preparedness or reducing student financial need, it can be implemented quickly and at low cost. this section reviews some of the relevant literature, focusing on the theoretical explanations for student performance and course load, as well as empirical evidence on those topics. common explanations for low graduation rates and increased time-to-degree completion typically center on factors such as student preparedness and performance, financial need, and reduced institutional resources (ishitani 2006; bettinger and long 2009; bound, lovenheim, and turner 2010; 2012; shapiro et al. 2016; evans et al. 2017; yue and fu 2017; deming and walters, 2017; sanabria et al. 2020) . regarding financial need, for example, a portion of this literature investigates the effect of merit-based scholarships on completion rates and time to completion with mixed results (cornwell, lee, and mustard 2005; dynarski 2008; scott-clayton 2011) . students themselves focus on slightly different concerns regarding increased time-to-degree completion. moore and tan (2018) found that students see course availability and work and family obligations as primary barriers to timely degree completion. another important predictor of degree completion and time-to-degree is "enrollment intensity," defined as the number of credits taken in a given term (volkwein and lorang 1996; knight 2004; herzog 2006; attewell, heil, and reisel 2012; attewell and monaghan 2016; shapiro et al. 2016; witteveen and attewell 2019) . almost by necessity, a reduction in time-to-degree will require higher enrollment intensity. completing a typical 120-credit degree program in four years on a semester system requires that a non-remedial student complete 15 units per semester. yet, it is not uncommon for universities to allow students to maintain full-time status while taking 12 units per semester (volkwein and lorang 1996; knight 2004) . policies allowing 12 units per semester stem in part from a recognition that students face home and work time pressures (horton 2016 ), but they can increase time-to-degree. advising students to take more credits may be counterproductive if grades suffer and classes have to be repeated, or if many students are unable to handle a full course load and drop out as a result. in what follows, we consider literature relevant to question of students' college performance with particular attention to gpa. we begin with general theoretical considerations followed by a brief review of empirical evidence for those considerations. the workhorse for much of the literature exploring student outcomes, such as gpa, is the so-called education production function (hanushek 1979 (hanushek , 2020 . in this framework, output, which is some measurable student outcome, is related to a set of educational inputs. these inputs traditionally include student characteristics in the form of academic skill, generally measured using high school gpa and standardized test scores (cohn et al. 2004; schmitt et al. 2009; cyrenne and chan 2012; danilowicz-gösele et al. 2017; caviglia-harris and maier 2020) , as well as factors such as complex problem-solving skills (stadler et al. 2018) . traditional inputs also include school characteristics and resources such as teacher skill or school funding. student course load, which will be discussed more in detail in sect. 2.4, fits into this theory as related to student effort and time, both of which are inputs to educational outcomes. recent work in economics, educational psychology, and higher education have expanded and refined the list of student-based inputs affecting educational attainment and academic success to include the importance of personality characteristics (schmitt et al. 2009; heckman and kautz 2012; richardson et al. 2012; kautz et al. 2014; caviglia-harris and maier 2020; sweet et al. 2019; and martínez et al. 2019) , peers (berthelon et al. 2019; pu et al. 2020 ) and student health (larson et al. 2016; and wallis et al. 2019) . each of these categories of determinants represent a wide literature covering specific cases from practical intelligence (schmitt et al. 2009 ), grit (fosnacht et al. 2019) , substance abuse (wallis et al. 2019) , and stress from financial or family difficulties (larson et al. 2016) . these student-based contributors to gpa combine, in educational production function theory, with school-based contributors. the traditional set of education production inputs on the school side, which includes instructor and instruction quality as well as institutional capacity, has expanded to include student support and engagement-enhancement services, which are expected to improve student performance. examples of student-engagement services can include first-year seminars (culver and bowman 2020), international exchange programs (brugiavini et al. (2020) , and living-learning communities (caviglia-harris and maier 2020). traditional funding-based inputs are still expected to apply, and can contribute to institutional support for access to technology (fairlie 2012) . evidence strongly supports the conclusion that prior academic success and intelligencebased measurements reflect positive inputs to later academic performance. there is a consistent positive association between factors such as high school gpa, sat, and act scores, and high-school percentile rank and college gpa (e.g. cohn et al. 2004; schmitt et al. 2009; cyrenne and chan. 2012; danilowicz-gösele et al. 2017; and caviglia-harris and maier 2020) . an interesting feature of the results reported by cyrenne and chan is that the positive association between high school gpa and college gpa is enhanced by high school financial expenditures, neighborhood effects and financial resources (cyrenne and chan 2012, p. 535) . stadler et al. (2018) found that gpa was positively associated with complex problem solving skills. intelligence-based measures are not the only student-based factors correlated with student success. personality characteristics are also shown to be associated with college gpa. richardson et al. (2012) find that conscientiousness is the strongest predictor of gpa among standard personality traits. results on grit are mixed. in a meta-analysis, credé et al. (2017) find a strong role for perseverance in explaining variation in gpa, but almost no incremental role for grit, after controlling for conscientiousness. most recently fosnacht et al. (2019) find that the perseverance subscale of grit was positively related to gpa for college students. caviglia-harris and maier (2020) find that grades are positively associated with conscientiousness in early semesters and with grit up until students' final year. sweet et al. (2019) and martínez et al. (2019) find a positive relationship between psychological capital and college gpa. there has also been attention in the literature to students' health as a determinant of performance. regarding student health and academic performance, larson et al. (2016) and wallis et al. (2019) both find strong relationships between gpa and psychological and physical health. correlates include mental health history and marijuana use. institutions can affect gpa through access to technology and engagement-enhancement programs. fairlie (2012) provides experimental evidence that access to technology in the form of free computers increases grades and course completion among minority students. caviglia-harris and maier (2020) find that living-learning communities are positively associated with retention, but find no association with a student's gpa. culver and bowman (2020) find no effect of first-year seminars on students' grades. the lessons learned from this literature are that variation in student performance is complex and that understanding this variation requires careful attention to both intelligence-based measurement and students' personality and health. attention to students' physical and psychological health is also warranted. there is a role for universities to improve performance through access to technology and support services. a focus on student course load may be one determinant of educational production but it is far from the only one. what might explain the relationship between student achievement and unit taking? the literature identifies two avenues by which a heavier course load could affect a student's performance. the first is a basic time allocation problem. the more courses students take, the less time they have to spend on each course. if performance is any increasing function of effort stinebrickner 2004, 2008; pu et al. 2020 ), this implies a negative effect of course load on performance. one consequence of this relationship between effort and time is that other demands on students' time such as employment and extra curricular activities could affect performance via study time. we review the empirical evidence on the potential negative time effects of additional time demands on learning in sect. 2.5 below. the second theoretical framework, which implies a positive relationship between course load and student performance, focuses on the level of student involvement. students who take more classes may take their heavy load as an opportunity to focus more exclusively on school, responding to additional academic responsibilities by substituting away from other activities. "academic momentum" (adelman 1999; suggests that students who complete more credits in their first year at college are more likely to graduate. attewell, heil, and reisel (2012) outline possible mechanisms. they show that students who begin with heavier course loads display a greater level of commitment to their academic goals and studies, and that positive accomplishments early in one's college career engender future successes by promoting individual "self-efficacy and/or academic self-concept" (attewell, heil, and reisel 2012, 28) . in this section we review the direct and indirect evidence on how course load affects student success. we begin with the theoretical time allocation mechanism. there is consistent evidence that time investment affects performance. stinebrickner and stinebrickner (2004) find a strong positive association between student study time and first-year grades. stinebrickner and stinebrickner (2008) and pu et al. (2020) use exogenous assignment of environmental factors to show that student time and effort, respectively, improve student participation and performance. the positive effects of study time imply an additional cost to other demands on student time while in college, such as employment. however, the evidence on whether the addition of new responsibilities actually reduces study time is mixed. employment has weak crowding-out results in both high school (kalenkoski and pabilonia, 2009 ) and college (babcock and marks, 2011) . weak effects of employment on actual study time are accompanied by inconsistent evidence of the effects of employment on performance. results among high school students are inconsistent across settings, students, and specifications (ruhm 1997; rothstein 2007) . at the college level, the evidence is again mixed. ehrenberg and sherman (1987) find no detrimental effects of work on grades, but do find that persistence and time to completion are adversely affected. stinebrickner and stinebrickner (2003) show that increasing hours of work by one hour per week reduces semester gpa by 0.162, identified by variation in work hours in assigned required jobs at a small liberal arts college. darolia (2014) , in contrast, finds that working reduces credits completed but does not harm grades. sports and other extracurricular activities are another possible avenue where time commitments could crowd out study time and reduce grades. however, athletic or college-gym participation does not appear to have strong direct effects on grades, and in some cases may increase them by increasing campus visits (emerson, brooks, and mckenzie 2009; fricke, lechner, and steinmayr 2018) . thus, while the available evidence supports a positive relationship between study time and student performance, evidence for both high school and college students provide limited evidence that work and other time demands harm academic performance. inconsistent results may be due to whether additional time demands are strong enough to actually reduce study time, or whether they may drive students to visit campus more often, as in fricke, lechner, & steinmayr (2018) . students who take on additional responsibilities, at least at low levels, may be able to substitute from other tasks besides study time. the available evidence on academic momentum, typically identified by matching on observables, broadly supports a positive association between the completion of firstsemester or first-year credits and student success. more intense early enrollment is associated with improved degree completion at two-and four-year colleges, especially among minority students (attewell, heil, and reisel, 2012; attewell and monaghan, 2016) . these effects also apply to grades (brugiavini et al. 2020) , and effects may be stronger if intensity is maintained (belfield et al. 2016 ). volkwein and lorang (1996) , knight (2004) , belfield et al. (2016) , and venit (2017) all find direct or indirect evidence that increase course load reduces completion time, although cornwell, lee, and mustard (2005) find only mixed evidence. several successful large-scale policy implementations, intended to improve completion by a number of simultaneous interventions, include increased course loads among other interventions. scott-clayton (2011) finds that a merit-based scholarship program, which implemented a course load minimum, improved both earned credits by the end of freshman year and ba completion rates, although the effects on completion faded out (scott-clayton and zafar 2016). denning (2018) finds more general evidence that financial aid increases credits taken and reduces time to degree, while also increasing graduation rates. scrivener et al. (2015) report on a randomized controlled trial including a requirement to attend school full time, tuition waivers to cover residual differences in financial aid and tuition and fees, and encouragement to "take developmental classes early and to graduate within three years." (scrivener et al. 2015, iii) . the authors report very large treatment effects, on the order of an 18 percentage-point (82%) increase in completion rates, as well as improvements in completion times. taken together, this literature supports a conclusion that retention and performance improves when students are more focused on school by taking more credits. the bulk of this literature informing the question of whether increased course load harms performance is indirect, looking at other demands on time, on the impact of course load on academic focus, or the impact of policies that increase course load but also change other things such that the impact of course load cannot be separated from other effects. there is a small amount of direct evidence on the effect of course load on retention and grades, although it is generally non-causal. szafran (2001) and jackson et al. (2003) report a associations between first-semester course load, retention, and grades. venit (2017), reporting on the university of hawaii's "15 to finish" program, found no evidence that student performance was harmed when taking a 15 credit course load, and found that taking at least 15 credits per term in the first year, as opposed to fewer, was associated with persistence and gpa, even for the lowest performing high-school students. the evidence on increased course-load appears to tilt in the favor of the academic momentum theory rather than any sort of time allocation explanation in which time use of non-academic tasks is inelastic. however, much of this literature is based on raw correlation or selection on observables, with estimates from matching as the forefront of the causal side of the field when studying course load alone rather than as a part of an rct package. further, the evidence on student grades is scant. we address this part of the literature by using detailed administrative data that allows for a fixed-effects design and the observation of student grades. we use administrative data from a major four-year university, provided by the office of institutional research & analytical studies (iras). the university in question is a regional, moderately selective four-year university and is one of the 23 california state university (csu) campuses. it serves a wide range of students, including a high proportion of hispanic students. it is important to note that the csu system and our own university enacted a policy to recommend higher course load, but that this occurred after our sample window. data are at the student-course level and include information on all courses attempted, including those that the student failed or withdrew from, and grades received for two incoming cohorts of freshmen students, 3,874 students beginning fall 2010 and 4,141 beginning fall 2011. 1 to ensure consistency in estimates, we omitted the small number of students who did not have full demographic information. our particular interest is in analyzing the effect of course load on student performance among students enrolled full-time at the margin between taking a full course load or not, so we drop part-time terms. there are two reasons for this restriction. first, we do not want to pick up potential nonlinearities in the effect of course load below 12 credits. second, our focus on student performance for those students moving from 12 and 15 credits speaks directly to policy prescriptions advocated our own university and by other institutions (e.g. venit 2017) to reduce time to completion. one potential downside of dropping part-time terms is that some of these students may have co-enrolled for additional credits at a community college. our administrative data do not contain information on these additional credits taken. additional restrictions include dropping summer terms, even though summer classes may be one way of graduating on time without taking a full course load, because students generally take only one or two classes at a time in the summer, at triple speed, and so the impact of an additional class would not be comparable. we also drop students who only took classes with extremely low variation in grades (described below). these restrictions result in very few students being dropped from the sample. 2 our final samples sizes for analysis are 3,852 and 4,102 students beginning in fall 2010 and 2011, respectively. all remaining calculations use this subsample. we observe student course-taking and grades through the end of the spring 2017 semester. we observe courses and grades for 18,337 student-terms while the students were freshmen, 13,835 while sophomores, 12,813 while juniors, and 14,199 while seniors. the relationship of sample size to class standing is due to dropout along with students who spend multiple years as seniors. there were 34.4 classes attempted, on average, across all students. 64% of the students from the incoming cohorts had graduated within 6-7 years (by spring 2017). 26.9% of graduates graduated within four years (by spring 2014-15). in addition to course-taking, we observe student background characteristics, including self-reported race/ethnicity and gender, student standing (freshman, sophomore, junior, or senior), financial aid receipt, high school gpa, declared major, and an admissions index based on a combination of high-school gpa and act/sat scores. we also observe admissions indices, or student-terms in which the only classes taken were non-graded or had grade variance beneath the 5 th percentile. footnote 2 (continued) aggregate measures of background and performance by students in other cohorts who were in the same classroom as students in our sample. table 1 provides descriptive statistics. a large fraction of the students in the incoming cohorts analyzed report that they are hispanic (0.405) and a large fraction report receiving some financial aid (0.814). these figures are higher than national averages of hispanic students (0.127) and financial aid receipt (0.740), as shown in the third column. average college course grades are 2.800, with a standard deviation of 0.778. to account for variation in grades driven by between-course differences, as described in the next section, grades are standardized within class section. since the goal is to explain within-class variation in grades, and to avoid inflated standardized-gpa values, we drop courses in which the within-class standard deviation of grades is below the 5th percentile (a standard figure 1 shows the distribution of within-class standardized gpa. on average, students attempted 4.63 classes per semester, where taking 5 classes is a full course load and 4 is the minimum necessary to be considered a full-time student. 46.7% of students take less than a full course load in a modal term. figure 2 shows the full range of course load, without collapsing the data to a full/not-full binary, and shows that across all students and semesters, the modal number of classes attempted was exactly 4, with 5 close behind. for analysis, we consider a term to be a full course load if the student attempts 5 or more courses (15 or more credits), a partial course load if they attempt fewer than 5 courses (12-14 credits), and since we are interested in the margin between partial and full course load for full-time students, the term is dropped if fewer than 12 credits are taken. in the remaining sample, 44.0% of terms are with full course loads. course load is determined on the basis of attempted classes, and so students who later drop or fail a course are still counted as taking it. while the decision to take a full course load has a basis in student-level characteristics (as will be examined later), there is significant within-student variation in the number of courses taken per term, necessary for a fixed effects identification strategy. excluding students who dropped out after one semester, students take a course load other than their modal course load (i.e. a student who usually takes 15 + credits taking 12-14 instead) in 26.1% of semesters. 80.8% of all students have at least some within-student variation in course load. in the context of the university, there is qualitative evidence suggesting that a fair amount of below-full-load taking behavior occurs because students try to take overenrolled classes, do not manage to register from the waiting list, and do not replace the class with another one (moore and tan 2018) . to the extent that this is the driver of withinstudent variation, concerns about selection bias in estimators using within variation are minimized. a clear source of endogeneity in estimating the effect of course load on student performance is ability bias. higher student ability should both lower the effort cost of taking more classes as well as increase the expected grade performance, leading to a positive noncausal observed relationship between classes taken and grades. in this paper we use withinstudent variation to account for ability bias. putting ability bias aside, we take into account the theoretical explanations covered in section ii. the academic momentum theory implies a positive effect of course load on student performance that, if it exists, should be part of the effect identified. if there are "identity" effects to taking many classes, or if additional classes increase time spent on campus, increasing study time per class, then students will increase study time when they take heavy course loads, driving a positive effect of course load on grades. time allocation theory also implies a causal effect: if students substitute away from study time on other courses to some degree when they add a new course, course load should reduce performance. however, in addition to its implied causal effect, the time allocation theory suggests several possible biases in the results, which we outline below. if a student faces a negative shock requiring them to work in the labor market or aid their family more, they may be likely to reduce both the number of classes taken and the effort spent on each class at the same time, driving a positive correlation between classes taken and grades. similarly, if course load is chosen before effort, then a consumption shock occurring between those two choices may require the student to lower effort more sharply in each class if enrolled in more classes, driving a negative bias. the bounded nature of course loads can also drive a positive bias. if a student is incentivized to spend less time on school in a given period, but is already taking few classes, they can only reduce effort rather than taking fewer classes. alternately, if students are planning to take four classes but one is overbooked (moore and tan 2018), they must replace the class to remain adequately enrolled. but for a student taking five classes and failing to enroll in one, replacing the overbooked class is optional, and more motivated students may be more likely do so. finally, there is the endogenous feature of the difficulty of courses. students may choose the number of courses to take on the basis of how difficult they expect their course mix to be (volkwein and lorang 1996; cornwell, lee, and mustard 2005) . if students take more courses when their courses are easier, the impact of course load will be positively biased. identifying the effect of the course load on performance should then focus on withinstudent variation to account for unmeasured ability. further, it must take into account timevarying external factors likely to drive the time allocation decision, including the difficulty of the courses being taken in a given term, which is also endogenous. we describe the relationship between performance gp ijt , measured as the grade points earned in the class by student i taking the mix of classes j in a given term t , and the number of credits taken that term (divided by three to give the number of classes taken). performance is dictated by individual unobserved components-like ability-0i , class-specific components that determine grades-like the generosity of the grading scale-1j , a direct effect of the number of classes taken class it , and individual time-varying determinants that are observed such as class standing or previous grades earned ( x it ) or unobserved such as family events or consumption shocks ( it ): at the university studied, there is not an enforced grade curve, and so 1j varies across classes. we move 1j to the left side of eq. (1), using average grades earned by students in the course as a proxy, and replacing the left-hand side with grades standardized within course section: the number of classes chosen is endogenous, and is determined by some individual fixed factor 0i , features of the course mix the student is facing (also endogenous) 1j , and individual time-varying observed ( x it ) or unobserved components ( it ). we use a fixed-effects estimator that accounts for the influence of both 0i and 0i . still, since class it and 1j are chosen jointly, their influence cannot be separately identified. the estimated coefficient ̂2 should be interpreted as a mix of the effects of classes and of course mix. however, this reduced-form effect is of interest, since any impact of a policy that increases course load on how students choose which courses to take should be a part of (1) gp ijt = 0i + 1j + 2 class it + 3 x it + it . (2) (3) class it = 0i + 1j + 2 x it + it the effect of interest. some analyses will attempt to disentangle the mechanisms at play by controlling for different features of the course mix. there is not an available instrument to directly account for the correlation between it and it , which is likely nonzero because pressures from employment hours are likely to be in both it and it . basic estimates will be biased by this correlation. we estimate partial identification bounds on ̂2 to determine the strength of the relationship necessary to change the substantive results of the paper. additionally, we emphasize that, in the context of the university being studied, a significant driver of variation in course load is the presence of registration bottlenecks (moore and tan 2018), which do not present the same problem for identification as do time-varying pressures in the student's life. this section gives our empirical results. we first report standard fixed effects estimates investigating the effect of course load on student performance, and then follow with examination of potential observed and unobserved sources of bias in these basic findings. table 2 shows our fixed effects estimates explaining student performance. the dependent variable for all regressions in table 2 is a student's grade in each class, standardized within the particular class section. standardization occurs within the section, rather than for the same class across all sections. all standard errors are clustered at the student level. in column 1 we show pooled ols results giving the relationship between attempting a full course load in the term and standardized grade, controlling only for a student's class standing (freshman / sophomore / junior / senior). in column 2 we add student-level characteristics. the within estimates regressing standardized grade on taking a full course load appear in columns 3-5 of the table, where the column 3 results are for all students and columns 4 and 5 results are limited to students who successfully graduated, or did not graduate, respectively. as shown in column 1, taking a full course load is associated with a 0.124 standard deviation increase in grade. the association between full course load and grade is reduced by roughly 28% when we add student-level characteristics in column 2. controlling for high school gpa and admissions index, all race/ethnic groups show lower standardized grades than whites (the reference category), and a one point increase in high school gpa is associated with a 0.533 standard deviation increase in college course grade. column 3 shows a within-student estimate of the effect of classes attempted on grade. the effect is positive, but is so small (0.011) that it is of little practical significance. importantly, we can reject to a reasonable degree of precision that the within-student relationship between a full course load and gpa is negative. additionally, column 3 gives some insight to the likely correlation between fixed unobserved student heterogeneity and classes attempted. the reduction in the estimate from column 1 to column 2 to column 3 implies that it is higher-performing students who do tend to choose higher course loads, and that many of the relevant high-ability characteristics are not measured in our data. 3 columns 4 and 5 of table 2 show that the effect of a full course load is not meaningfully different for graduates or non-graduates. the effect is not large or significant for either subgroup, but is slightly more positive for non-graduates, who are less likely to choose higher course loads in the first place. examining variation in the effect in other ways besides eventual graduation success, we allow for the possibility that the effect of course load on grades differs by key student demographic characteristics. in six separate analyses we interact the full-course-load variable with race/ethnicity (white/black/asian/hispanic/other or mixed), gender, highschool gpa (continuous), parental education (no college, some college, graduate), parental income (seven bins), and lagged cumulative gpa (continuous), respectively. in results available upon request, we find that none of the coefficients for these interactions are significant (the lowest p-value for a joint test of a set of interactions is 0.348). among categorical interactions, there is only one case-the second-lowest parental income bin-where the point estimate for that group is negative (although it is insignificant). finally, we examine whether the effect varies along the gpa distribution. it is reasonable to expect that high-ability students might have a null or positive effect of increased course load, but that students already having difficulties might not be able to handle the additional work. in table 3 we use the method described in machado & santos silva (2019) to perform fixed-effects quantile regression. surprisingly, we find that the effect is most positive at the low end of gpa, and actually turns negative and significant for the 90th percentile. the potential concern that strong students drive the nonnegative result, and that full course loads would still be too much for low-gpa students to handle, is not supported. student ability is one determinant of course load, which can be addressed using fixed effects, but there may be other important factors. table 4 demonstrates the predictors of course load, which may help to determine the extent of the known bias in table 2 and how it might be reduced. all standard errors are clustered at the student level. as shown in column 1, students taking more classes tend to have higher high school gpas. women are more likely to take full course loads. white students are most likely to take a full course load, with black students coming close behind. hispanic students, about 40% of the sample, are 13.4 percentage points less likely to take a full course load than whites. the other columns in table 4 show results giving the influence of time-varying student characteristics on the number of classes taken, and include student fixed effects. as shown in columns 2 and 3, students tend to take more courses when their courses are easier. however, this is based not on the raw grades given out in class (average grade in course), but rather the grades given out relative to what might be expected based on the demographic characteristics and cumulative gpa of students taking the class (population-adjusted average grade). roughly, population-adjusted average grade can be thought of as "easy for the people who choose to take it." an average course grade one gpa unit higher than could be expected given who takes the course is associated with the student being 18.3 percentage points more likely to take a full course load. columns 4 and 5, which omit the student's first term (and first two terms, respectively) examine the student's recent performance in college. columns 4 and 5 show that a student who has recently been doing better than usual is likely to take more courses, which could be interpreted as confidence, or learning one's limits. in all regressions in table 4 , r 2 values are low, indicating that these measured confounders are not especially strong. robustness of the classes attempted and class performance link population-adjusted average grade is the average course grade earned in that course, adjusting for demographics and cumulative gpas of students who take the course. "student standing" is freshmen / sophomore / junior / senior. standard errors clustered within student. sample size drops for columns 3-5 to account for lags dependent variable: standardized class grade despite low r 2 values in table 4 , table 5 checks whether any of these predictors can explain the positive relationship between full course loads and grades. all standard errors are clustered at the student level. columns 1 and 2 display the results when we add average course grades. while the left-hand side variable is already adjusted for average grades in each course, it is still possible that multiple easy classes may allow a student to overperform in all of them at once. however, neither addition eliminates the positive relationship. the effect also is not eliminated by controlling for recent or prior performance in column 3, indicating that the positive relationship is not a result of students, for example, taking more classes as they discover which fields they are good at. finally, in column 5, we include all of the previous controls (using the non-population-adjusted average grade, although it does not make much difference) and also include fixed effects for declared major to account for differing institutional standards for how many courses students should take, especially in stem. the effect remains positive. columns 3 and 4 are especially important. as will be discussed in the next section, a potential source of omitted variable bias is time-varying pressures from work or family, which may reduce course load and, at the same time, harm performance in class. it is reasonable to expect that these time-varying pressures may be serially correlated and would also affect grades. if they are, then lagged gpa acts as a proxy for these time-varying pressures, and so columns 3 and 4 act as a partial test for this source of omitted variable bias, and fail to change the result. the relationship between courses taken and grades remains positive and statistically significant. while the effect is small, the qualitative result of importance is that the relationship is not negative, even accounting for time-constant skills with fixed effects, the tendency to take many easy courses at once, the tendency to take more courses when one is doing particularly well, or institutional differences between majors. there is an important confounder missing from our analysis in sect. 5.2 -consumption shocks or time-varying work pressures, or similar time-varying pressures from family or other sources. at the setting studied, significant portions of students work either part or full time while attending classes. theoretically, we would expect that changes in course load driven by factors that increase work hours would reduce grades and courses taken at the same time, leading to a positive bias. under the assumption that these pressures are serially correlated, lagged gpa is a proxy for these time-varying pressures, and in table 5 a control for lagged gpa does not change the result. still, controlling for lagged gpa is unlikely to eliminate all bias. since we do not observe work hours, we perform a simulated sensitivity test, inspired by rosenbaum (2002) bounds. we randomly generate a binary unobserved confounder z it , which could be considered to represent something like a consumption shock or outside work demands, using the formula where it ∼ n(0, 1) . by construction, z it is positively correlated with both gpa std it and with class it , unless 1 or 2 = 0 . we allow 1 and 2 to each take any value {0, 0.025, 0.05, … , 1} , which generates correlations between z it and gpa std it of between 0 and 0.323, and correlations between z it and class it of between 0 and 0.46. we then add z it as a predictor to the model, and examine how the coefficient on class it changes as a result. we repeat this process 200 times for each combination of 1 and 2 , and for each of two models: the baseline model (table 2 column 3), and including all controls (average grade, prior performance, and current declared major). the goal is to determine the strength of the relationship between z it and the observed variables necessary to generate a significant negative coefficient on class it . figure 3 shows the minimum correlations necessary to generate a statistically significant negative coefficient on class it at the 95% level (190 or more out of 200 simulations are negative). in other words, for this analysis to be heavily biased enough by an omitted predictor such that we should be reporting a negative effect of number-of-classes on grade performance, that omitted predictor would need to have a correlation strength on the drawn boundary or to the top-right of it. in the base model without additional controls, such an omitted predictor would need to have a correlation of about 0.1 or better with gpa std it and a correlation of about 0.2 or better with class it , although one stronger correlation can trade off for another. correlations in the range of .1 − .2 between time-varying work or family pressures and course load or gpa are not particularly high, indicating that an omitted predictor needs to have only moderate strength to mask a statistically significant negative result. however, even in these statistically significant cases, the negative relationship between class it and gpa std it is not large. considering only statistically significant negative coefficients in the simulation, the average coefficient on class it was −0.032 , and ranged from −0.084 to −0.009. to determine the strength of omitted predictor that would be necessary to produce a meaningfully large negative effect, we increase 1 and 2 until we find meaningfully large negative effects. we would consider a 0.2 drop in within-class standard deviations of gpa to be a moderate effect size. in absolute gpa terms this translates to about 0.154 gpa points, about half of the difference between a b and b + . we achieve this moderate negative effect by searching values of 1 and 2 in {2.2, 2.25, … , 2.7} and find moderately large effects of −0.2 with rather large within-student correlations of at least 0.399 with course . lines indicate the correlation between a simulated unobserved variable and treatment/outcome that generates a negative result in 95% or more of simulations. slight positive slopes are due to indirect manipulation of correlation and a discrete search space load and 0.531 with gpa, with a small amount of potential tradeoff between the two. with controls included, correlations need to be 0.430 with course load and 0.531 with gpa. we would consider these to be fairly strong within-student correlations, possibly implausibly high. in addition to the simulation of rosenbaum-like bounds, we also examine coefficient stability using the method in oster (2019) . we make the assumption that observable (using the full model above, average grade, prior performance, and current declared major) and unobservable characteristics have identical levels of relative influence on selection into a full course load on the basis of observables and impact on gpa ( = 1 in the terms oster uses, see cinelli & hazlett (2020) for detail on the interpretation of ). given this assumption, we check every r 2 value in steps of 0.05 to calculate the minimum r 2 of a model including both observables and unobservables that would lead to either a negative effect size ( < 0 ) or a meaningfully large negative effect size ( < −.2 ). if the model including both observable and unobservable controls has a within-student r 2 of 0.1 or greater, then the estimated effect is negative. however, to produce an effect size that is meaningfully large and negative, the full model would need to have a within-student r 2 of 0.6 or greater, which is high. we then use the coefficient stability methods of cinelli & hazlett (2020) , which, like oster (2019), constructs bounds by comparing the influence of observed and unobserved controls. with all controls included, unobserved controls would need to have variation orthogonal to the included controls that explains at least 17.4% of both the residual variation in the treatment and residual variation in the control to produce a point estimate of −0.2 . there is some possibility for tradeoff between explaining residual variation in the treatment and the control. further, cinelli & hazlett (2020) provide a means of bounding the adjusted confidence interval of an unobservable-including estimation using observed covariates. we check all included covariates and find that the adjusted confidence interval never includes −0.2 , and at its lowest is −0.005. we conclude that, even though our main results likely contain positive omitted variable bias, and we would not be surprised to find a small negative effect of course load with the proper controls, our partial identification analysis shows that adjusting for this bias is unlikely to lead to a meaningfully large negative causal relationship between class it and gpa it , which is consistent with our broader conclusion that the effect of course load is meaningfully near zero. for a large negative effect to emerge with proper controls, timevarying outside pressures would need to be extremely strong determinants of course load, which contradicts our earlier discussion that in this context, much of the course load decision has to do with course bottlenecks outside the control of the student. in table 6 we show some supplementary results related to the effects of taking more classes each term. in columns 1 and 2 we predict persistence to the next term. these results address the possibility that additional classes, even if they do not weaken performance, may lead to burnout so that students are less likely to return. consistent with much of the literature in sect. 2, we find no such relationship. columns 3 and 4, which are performed on a one-observation-per-student basis, examine the relationship between taking more classes per term and the rate of graduation as well as the time to graduation. the variable "proportion full course loads" is an average over all enrolled terms. without any within-student variation, the causal identification for these estimates is very weak, and requires that the list of controls is sufficient. so we consider these non-causal. there is a very strong relationship between taking more classes and graduating; students who take one more class per term graduate more than 30 percentage points more often. there is also a negative relationship between classes per term and timeto-degree. students taking an additional class each term take on average 1.1 fewer years to graduate, more than the full year decrease one might expect mechanically. finally, there are one-unit classes and a small number of non-3-unit classes in the data. we ran all results again using a more direct count of classes (excluding one-credit classes) rather than the number of units divided by 3. in results available upon request, we find that the findings are very similar, and while point estimates differ slightly (and some of the positive-and-significant-but-very-small results that we emphasized as being upper bounds became positive-and-insignificant) none of the substantive conclusions about the effect of class taking changed. two results did change: (1) the effect of class-taking for the top decile of students changes from negative and significant to negative and insignificant, and (2) results were different for table 4 , in regards to predicting which students take full course loads. in particular, racial and gender effects become less prominent in column 1, and the influence of average class grades is now negative (columns 2 and 3 ). the negatives of long time-to-degree are clear: earnings penalties for some (witteveen and attewell 2019) , poorer overall performance, perhaps due to skill atrophy (brugiavini et al. 2020) , cohort crowding (kurlaender et at. 2014) , and additional financial cost in terms of tuition outlays and delayed entry into the labor market. one way to improve time to degree would be for institutions to support efforts to increase credits per semester. however, there is a concern that additional credits may harm student performance. we find no evidence that increased course load harms performance, which supports the use of time-to-degree policy that uses course load as a lever. what impact can universities actually have on course load? one avenue is to give students more options in course scheduling, such as evening and on-line courses (witteveen and attewell 2019) . advising can also play a prominent role in increasing credits per semester. advising plans are at the heart of many of the finish-in-four initiatives on college campuses in the united states discussed previously (california state university 2019; university of hawaii (venit 2017) ; the university of south dakota (https ://www.usd.edu/ acpc/finis h-in-four); and oklahoma state university (https ://cas.oksta te.edu/advis ing/finis h-in-four). however, advising on course load may need to be intensive; in a randomized experiment, an informational intervention designed to increase course load had no impact by itself (huntington-klein and gill, 2019). course load policy does not need to be the only tool used for reducing time-to-degree. increasing the number of summer classes is one way to increase the speed at which students accumulate credits. policy on institutional barriers and resources, and student performance, is of course also important. pike and robbins (2020) find that the predominant factors affecting graduation rates are institutional characteristics that are either invariant or cannot be quickly or easily changed. yet, they do find that per pupil expenditures for instruction are associated with better 4-year and 6-year graduation rates. though not stated explicitly by the authors, factors such as peer tutoring and university efforts at remediation typically fall into this category. walvoord and pleitz (2016) provide non-causal evidence that peer tutoring is associated with higher first year grades. additional support for the notion that advising, tutoring, and mentoring matter to degree completion is found in deming and walters (2017). the authors find both positive effects of school spending on degree completion and that spending on categories of academic support services such as advising and tutoring are particularly hard hit when there are state budget shocks. student support services and student-engagement programs have the potential to improve time to degree, particularly in the current covid-19 environment. support for students with disabilities may improve outcomes in light of some evidence that students with disabilities are less likely to complete a bachelor's degree if they enroll in college, controlling for pre-college characteristics (lichtenberger 2010; smalley et al. 2010 ). based on survey results, graham-smith and lafayette (2004) report that students with disabilities themselves highly value a caring staff that provides safety and security. gaddy (2016) outlines strategies campus disability service providers can take to improve student performance, including the use of test-taking and writing strategies. as mentioned, students' mental and physical health may affect performance. larson et al. (2016) conclude that support and assistance with students' health can improve outcomes, and that such assistance requires coordinated efforts across academic and student affairs (larson et al. 2016, p. 600) . evidence on the importance of student-engagement programs on retention and degree receipt is mixed. for example, caviglia-harris and maier (2020) find that living-learning communities are positively associated with retention. in contrast, culver and bowman (2020) find no effect of first-year seminars on retention or four-year graduation. johnson and stage (2018) find little to no impact of so-called high-impact practices on 4-year or 6-year graduation rates at four-year public institutions. course load policy should be thought of as sitting alongside this array of interventions that focus on student performance, and requires similar institutional attention. aforementioned advising efforts will be less effective if students are unable to take their preferred classes at the times needed to finish their course of study in a timely manner. this barrier to timely completion may be particularly relevant at a large public university such as ours. course scarcity could cause students to take fewer classes, take the wrong classes, or change majors, all of which would increase time to degree (kurlaender et al. 2014, pp. 33-35) . reporting on a survey of students and focus groups from three universities in the csu system, moore and tan (2018) indicate that students themselves rate course availability as the number one obstacle to timely completion of their degrees. among students who enrolled as freshmen, roughly half indicated that they, "often or very often have not been able to enroll in a class that they needed" (moore and tan 2018, p. 2) . it is important to note, however, that causal evidence that course scarcity affects time to degree is mixed. kurlaender et al. (2014) , using an iv strategy based on randomly assigned registration times, find no evidence of course scarcity on time to degree using data from the university of california, santa cruz. robles et al. (2019) use a fuzzy regression discontinuity strategy based on waitlist cutoffs at de anza community college find that course scarcity increases the probability of dropout or transfer to another community college. these concerns about course availability reinforce the need for advising and mentoring as a way to assist students in timely degree completion and increase course load without being caught in bottlenecks, and also reinforce that policy pushing additional course load needs to first make sure that it's actually possible for students to take more courses at the institution. in this paper we use administrative observational data in order to assess the causal effect of taking a full course load on student performance. we first focus on within-student variation in course load to avoid bias arising from student ability. we find no evidence of a negative effect of a full course load on student grades, and instead find a small positive effect. our baseline estimates are likely to be positively biased due to a relationship between course difficulty and course load, or the presence of time-varying outside work or family pressures. we find that controlling for course difficulty does not change results. we cannot measure outside pressures, but we provide four pieces of evidence supporting our conclusion. first, we note that, in the context studied, a significant determinant of course load is registration bottlenecks (moore and tan 2018), which would not bias results. second, we control for lagged gpa, which is a proxy for time-varying academic performance as well as time-varying outside pressures under the assumption that those pressures are serially correlated, and find no difference in results. third, we perform a simulation and find that outside pressures would need to have a correlation of around 0.399 with taking a full course load, and also a correlation of around 0.531 with grades, to generate the observed data if the true effect were meaningfully negative. fourth, we show using oster (2019) that obtaining a meaningfully large negative effect of course load on grades requires a fairly large (0.6) within-student r 2 for a model including both observable and unobservable controls. we then show using cinelli & hazlett (2020) that a meaningfully large negative effect requires an unobserved predictor to have variance orthogonal to the included controls that explains 17.4% or more of the within-student residual variance in both treatment and outcome. there are two important takeaways from the evidence presented here. the first is that we find, to a reasonable degree of certainty, that there is no meaningfully large negative causal impact of increasing student course loads. policy directives to improve four-year graduation rates by increasing course load are unlikely to have meaningful negative effects on student performance and learning. it is also unlikely that those average effects mask strong negative effects for weaker students, although it is possible that the 19.2% of students who always take the same number of classes would perform worse if pushed to do otherwise. as mentioned, our university policy encouraging students to take more credits was enacted several years after the first-time freshmen cohorts we study started their coursework. future work looking at the causal effects of full course load as induced by policy would be useful. an analysis comparing the effects of course load for transfer students, as opposed to first-time freshmen, may also be a productive avenue for future research, but transfer students were not available in our data. our work contributes to the broader literature on college student performance. we contribute to a literature that has relatively few studies with plausibly causal estimates of the effect of increasing demands on students' time on their performance in other classes. a small body of associational literature will benefit from the addition of more plausibly causal results. the findings in this paper imply that one obvious solution-higher course loads-to a large and expensive nationwide problem-slow time-to-degree-does not appear to have the feared negative tradeoff for student performance. academic intensity, attendance patterns, and bachelor's degree attainment the toolbox revisited: paths to degree completion from high school through college what is academic momentum? and does it matter how many credits should an undergraduate take? research in higher education the falling time cost of college: evidence from half a century of time use data momentum: the academic and economic value of a 15-credit first-semester course load for college students in tennessee the structure of peers: the impact of peer networks on academic achievement addressing the needs of underprepared students in higher education does college remediation work why have college completion rates declined? an analysis of changing student preparation and collegiate resources increasing time to baccalaureate degree in the united states academic achievements: the effects of excess time to degree on gpa graduation initiative it's not all in their heads: the differing role of cognitive factors and non-cognitive traits in undergraduate success making sense of sensitivity: extending omitted variable bias determinants of undergraduate gpas: sat scores, high-school gpa and high-school rank trends in higher education-the college board student responses to merit scholarship retention rules much ado about grit: a meta-analytic synthesis of the grit literature is what glitters really gold? a quasi-experimental study of first-year seminars and college student success high school grades and university performance: a case study determinants of students' success at university working (and studying) day and night: heterogeneous effects of working on the academic performance of full-time and part-time students the impact of state budget cuts on u.s. postsecondary attainment. working paper 23736 born under a lucky star: financial aid, college completion, labor supply, and credit constraints building the stock of college-educated labor employment while in college, academic achievement, and postcollege outcomes: a summary of results college athletics and student achievement: the evidence at small colleges. new directions for institutional research increasing community college completion rates among low-income students: evidence from a randomized controlled trial evaluation of a case management intervention. working paper 24150 academic achievement, technology and race: experimental evidence how valid is grit in the postsecondary context? a construct and concurrent validity analysis the effects of incentives to exercise on student performance in college consider additional support for your students through collaboration with your academic support area. disability compliance for higher education quality disability support for promoting belonging and academic success within the college community conceptual and empirical issues in the estimation of educational production functions education production functions hard evidence on soft skills estimating student retention and degree-completion time: decision trees and neural networks vis-à-vis regression why students should take at least 15 credits every semester. the california state university system news an informational intervention to increase semester credits in college. series of unsurprising results in economics studying attrition and degree completion behavior among first-generation college students in the united states the impact of hope, procrastination, and social activity on academic performance of midwestern college students academic engagement and student success: do high-impact practices mean higher graduation rates does working while in high school reduce u.s. study time? fostering and measuring skills: improving cognitive and non-cognitive skills to promote lifetime success time to bachelor's degree attainment: an application of descriptive, bivariate, and multiple regression techniques college coursscarcity and time to degree using student health data to understand and promote academic success in higher education settings a longitudinal study of illinois high school graduates with disabilities: a six-year analysis of postsecondary enrollment and completion (ierc 2010-4)ierc 2010-4) quantiles via moments antecedents of academic performance of university students: academic engagement and psychological capital resources get me from point a to point b:' student perspectives on barriers to timely graduation at the california state university unobservable selection and coefficient stability: theory and evidence using panel data to identify the effects of institutional characteristics, cohort characteristics, and institutional actions on graduation rates peers, study effort, and academic performance in college education: evidence from randomly assigned roommates in a flipped classroom psychological correlates of university students'' academic performance: a systematic review and meta-analysis the effect of course shutouts on community college students: evidence from waitlist cutoffs observational studies high school employment and youths' academic achievement is high school employment consumption or investment failing at remediation? college remedial coursetaking, failure and long-term student outcomes prediction of 4-year college student performance using cognitive and noncognitive predictors and the impact on demographic status of admitted students on money and motivation: a quasi-experimental analysis of financial incentives for college achievement financial aid, debt management, and socioeconomic outcomes: postcollege effects of merit-based aid doubling graduation rates: three-year effects of cuny's accelerated study in associate programs (asap) for developmental education students time to degree: a national view of the time enrolled and elapsed for associate and bachelor's degree earners a longitudinal study of the illinois high school class of 2002: a six-year analysis of postsecondary enrollment and completion the logic of success: the relation between complex problem-solving skills and university achievement. higher education time-use and college outcomes the causal effect of studying on academic performance. the b.e working during school and academic performance the relationship between psychological capital and gpa: a study of one freshmen cohort the effect of academic load on success for new college students: is lighter better? research in higher education household pulse survey: change in post-secondary education digest of education statistics why even c students should consider taking 15 credits their first semester. student success insights blog (blog) characteristics of extenders: full-time students who take light credit loads and graduate in more than four years assessing marijuana use, anxiety, and academic performance among college students applying matched sampling to evaluate a university tutoring program for first-year students delayed time-to-degree and post-college earnings rethinking graduation and time to degree: a fresh perspective the authors are grateful for financial support from the california state university, fullerton graduation initiative 2025. conflict of interest the authors have no financial interests to declare. key: cord-346843-z82ikuqc authors: jabbar, abdul; gauci, charles g.; anstead, clare a. title: parasitology education before and after the covid-19 pandemic date: 2020-10-23 journal: trends parasitol doi: 10.1016/j.pt.2020.10.009 sha: doc_id: 346843 cord_uid: z82ikuqc the covid-19 pandemic has disrupted parasitology curricula worldwide, which is expected to lead to the reshaping of parasitology education. here, we share our experiences of remote teaching and learning of veterinary parasitology and discuss opportunities offered by remote teaching during covid-19 lockdowns, enabling the development of interactive online parasitology courses. since the beginning of the 21 st century, several face-to-face and blended (i.e. face-to-face and online) learning practices, including the disciplinary-based and problem-oriented approaches have been used to teach various disciplines of parasitology worldwide [1] . for example, various learning theories for online and/or blended parasitology education, including community of inquiry [2] , connectivism [3] and online collaborative learning [4] have been used. recently, the blending with pedagogical purpose model [5] has most commonly been used for teaching and learning in parasitology [1] and other disciplines [6] , where students are the learning community and multiple technologies (e.g. learning management systems) and multi-media are utilised to (i) scaffold and deliver the content of subjects/courses; (ii) promote reflective blog journals, with collaboration and peer-to-peer learning among students; and (iii) evaluate student learning. however, the social presence in a learning setting is often achieved by face-to-face teaching and interactions among students, which are key components of a learning environment. in the last two decades, these blended, student-centred learning practices [7] using adaptable digital technologies [8, 9] have become useful tools for effective and engaging parasitology teaching and learning. the covid-19 pandemic has challenged the existing teaching and learning practices from primary schools through to tertiary institutions in almost every country of the world [10] . for veterinary parasitology. our goal is to share our experiences in the midst of the covid-19 pandemic which has brought about great uncertainty and challenges in almost every walk of life. the rapid shift seen across the tertiary education system might be the beginning of a new era in course delivery. the adoption and/or modification of teaching and learning practices tested (out of necessity) during the covid-19 lockdown may guide or influence the re-design of parasitology curricula. on tuesday 17 th march 2020, the melbourne veterinary school suspended all face-to-face teaching and we, as educators, were given three days to prepare for the online delivery of lectures and practical parasitology classes scheduled the following week. our parasitology team at the university of melbourne has extensive experience in teaching parasitology to animal science and veterinary students. prior to the covid-19 pandemic, we adopted new approaches to enhance learning outcomes utilising a combination of conventional face-toface teaching and digital technologies [9] . through reflective practice and considering the original intended learning outcomes (ilos), we rapidly adapted the face-to-face lectures and practical classes in parasitology to online modules for doctor of veterinary medicine students (second-and third-year levels within a 4-year degree, 130 students/cohort) and bachelor of agriculture students (third/final year level, 80 students). in support, the university offered various professional development courses through the department of learning environments i to staff and students to aid with the transition to remote/online teaching. based on our experiences of online teaching and learning in the field of veterinary parasitology, we have proposed a toolkit (box 1) for parasitology educators; our teaching j o u r n a l p r e -p r o o f journal pre-proof module received appreciation from students (supplemental table s1 ) and academic peers alike. for timely lecture delivery, we uploaded pre-recorded lectures on a learning management system (e.g. canvas ii ) to allow student self-directed, independent learning and to avoid possible network access issues during live streaming of lectures. subsequently, we held weekly flipped classroom [9] sessions (i.e. live question and answer (q&a)) via zoom iii and used polls iv to provide opportunities for cognitive, social and teaching presence for the online learning experience of students, as per the learning theory of community of inquiry [2] . for practical classes, we used a number of approaches and utilised pre-developed resources available online (supplemental box s1), circumventing the difficulties associated with preparing, at short notice, a semester's worth of high-quality material, especially videos using the loom application v . our typical veterinary parasitology practical class involves the identification of a group of parasites, assorted hands-on techniques to isolate and identify parasites in various organs of a range of hosts, and problem-based clinical scenarios for veterinarians. we developed the 3-hour face-to-face practical classes into online modules using various asynchronous and synchronous teaching approaches to encourage deep learning, while promoting student engagement, the interaction between students and teachers and meta-connective pedagogy [12] . for each online practical class, we developed a powerpoint presentation (supplemental box s1) which outlined all components of the practical and provided links to external online available resources, and these slides were uploaded on canvas one week prior to the practical class. the online practical contained an introduction to the class via a 15-20 min live interactive session using zoom, followed by multiple independent and collaborative sessions for students via breakout rooms in zoom, ending with a 30 min j o u r n a l p r e -p r o o f journal pre-proof summary via a live zoom session using polls. the zoom breakout room feature allowed student collaboration and lecturers to interact within smaller groups of students (10 per group). students were also provided with the opportunity to ask questions directly via zoom unmute, a zoom chat (typed) as well as through an online discussion page in canvas. in addition to recorded lectures, the entire practical class was recorded and made available to students via canvas for later viewing, and for students unable to attend the session, or those that were in another state or country. this method of online delivery of our veterinary parasitology subject received very positive feedback from students (see supplemental table s1 ). additionally, we tried to mimic virtual microscopy sessions by preparing high-quality videos, high-resolution scanning of slides or photography of gross specimens (supplemental box s1) and presenting them via several user-friendly commercially available platforms vi . the best network platform, in particular, is highly interactive as it not only allows a virtual microscopic experience but also permits annotation which is an invaluable tool for both students and teachers. in addition to positive student feedback, the average exam scores for pre-and post-pandemic cohorts was comparable, indicating that the modified methods of content delivery were effective. strategically targeted questions using flipped classroom approach [9] were also posed to students during the online q&a sessions/practicals (using a polling application) which allowed live gauging of the level of student understanding. some use existing 'active' resources and specify the parts that need to be viewed. provide interactive group activities (e.g. online quizzes, polls, discussion boards, subject dedicated facebook page, etc.) which support peer learning and collaboration. communicate clear instructions and expectations about the assessment. emotional openness is a great instructional strategy, so don't hide your feelings and use "icebreakers" during teaching and learning activities. for example, ask at the beginning of a lesson "what is a new skill/interest you have undertaken during isolation?" use virtual office hours using platforms like zoom to interact and provide social support to students. this can be a great way to collect student feedback on your online teaching and identify any issues which need immediate addressing. once you find a teaching style working for you, feel free to repeat it each week until you are back in your classroom! editorial special issue -learning and teaching of veterinary parasitology assessing social presence in asynchronous text-based computer conferencing connectivism: a learning theory for the digital age learning theory and online technologies promoting self-directed learning through the implementation of cooperative learning in a higher education blended learning environment theories and frameworks for online education: seeking an integrated model can new digital technologies support parasitology teaching and learning? covid-19 and rapid adoption and improvisation of online teaching: curating resources for extensive versus intensive online learning experiences e-learning, online learning, and distance learning environments: are they the same? online design education: meta-connective pedagogy asynchronous and synchronous online teaching: perspectives of canadian high school distance education teachers from bricks and mortar to remote teaching: a teacher education programmer's response to covid-19 we are grateful to professor robin b. gasser for his advice and suggestions during the rapid adoption and improvisation of online parasitology teaching as well as providing feedback on the manuscript. key: cord-299261-ew99nraq authors: cipriano, l. e.; haddara, w. m. r.; zaric, g. s.; enns, e. a. title: impact of university re-opening on total community covid-19 burden date: 2020-09-18 journal: nan doi: 10.1101/2020.09.18.20197467 sha: doc_id: 299261 cord_uid: ew99nraq purpose: post-secondary students have higher than average contacts than the general population due to congregate living, use of public transit, high-density academic and social activities, and employment in the services sector. we evaluated the impact of a large student population returning to a mid-sized city currently experiencing a low rate of covid-19 on community health outcomes. we consider whether targeted routine or one-time screening in this population can mitigate community covid-19 impacts. methods: we developed a dynamic transmission model of covid-19 subdivided into three interacting populations: general population, university students, and long-term care residents. we parameterized the model using the medical literature and expert opinion. we calibrated the model to the observed outcomes in a mid-sized canadian city between march 1 and august 15, 2020 prior to the arrival of a relatively large post-secondary student population. we evaluated the impact of the student population (20,000 people arriving on september 1) on cumulative covid-19 infections over the fall semester, the timing of peak infections, the timing and peak level of critical care occupancy, and the timing of re-engaged social and economic restrictions. we consider multiple scenarios with different student and general population covid-19 prevention behaviours as well as different covid-19 screening strategies in students. results: in a city with low levels of covid-19 activity, the return of a relatively large student population substantially increases the total number of covid-19 infections in the community. in a scenario in which students immediately engage in a 24% contact reduction compared to pre-covid levels, the total number of infections in the community increases by 87% (from 3,900 without the students to 7,299 infections with the students), with 71% of the incremental infections occurring in the general population, causing social and economic restrictions to be re-engaged 3 weeks earlier and an incremental 17 covid-19 deaths. scenarios in which students have an initial, short-term increase in contacts with other students before engaging in contact reduction behaviours can increase infections in the community by 150% or more. in such scenarios, screening asymptomatic students every 5 days reduces the number of infections attributable to the introduction of the university student population by 42% and delays the re-engagement of social and economic restrictions by 1 week. compared to screening every 5 days, one-time mass screening of students prevents fewer infections, but is highly efficient in terms of infections prevented per screening test performed. discussion: university students are highly inter-connected with the city communities in which they live and go to school, and they have a higher number of contacts than the general population. high density living environments, enthusiasm for the new school year, and relatively high rates of asymptomatic presentation may decrease their self-protective behaviours and contribute to increased community transmission of covid-19 affecting at-risk members of the city community. screening targeted at this population provides significant public health benefits to the community through averted infections, critical care admissions, and covid-19 deaths. the covid-19 pandemic presents a substantial public health challenge for local, national, and international communities because the virus is highly transmissible, 1-3 including prior to symptom onset, 4, 5 and infections initially present with a wide range of non-specific and sometimes mild symptoms. 6, 7 the relatively high rate of hospitalization and need for critical care among severe cases can quickly overwhelm community health care resources and result in substantial mortality. 8, 9 many communities initially responded to covid-19 with school closures and stay-at-home orders, which included closure of university campuses and conversion of all in-person instruction to online formats. over the summer, universities began announcing plans for the fall term. some universities opted to operate fully online for the fall. 10 others announced plans to partially or fully re-open campus and welcome students back for in-person instruction with covid-19 mitigation strategies in place. these strategies included polices around mask wearing, limiting large gatherings, access to covid-19 testing, reduced dormitory occupancy, and accommodations for isolating and quarantining students. 11 while universities have the autonomy to make decisions about the level of on-campus activities offered to their students, their decisions have implications for the communities in which these campuses are located. university students live, work, and socialize both on and off campus, resulting in significant potential for on-campus covid-19 outbreaks to spill over into the community and vice versa. 12 infectious diseases can spread rapidly through a university campus, as evidenced by outbreaks of serogroup b meningococcal disease, mumps, 13 and the novel h1n1 influenza virus that emerged in 2009. 12, [14] [15] [16] surges of covid-19 cases have already been observed on the first campuses to re-open this fall, 17 prompting many universities to abruptly change their fall semester plans. 18 after 177 students tested positive for the novel coronavirus linked to at least four separate clusters, the university of north carolina abruptly moved all undergraduate classes online after only a week of in-person instruction. 19 high-density housing, including multiple roommates and shared facilities like bathrooms, as well as high levels of social activity puts the university population at particular risk for infectious disease transmission. 15 furthermore, past experience with the 2009 h1n1 pandemic indicates compliance with recommended public health precautions may be sub-optimal; despite public health guidance not to attend classes and other activities while ill during the h1n1 outbreak at a us university, half of students indicated interacting with a symptomatic individual in a classroom and nearly one-quarter indicated interacting with a symptomatic individual at a party or social activity. 15 given the unique features of university communities, several studies have modeled covid-19 transmission dynamics specifically on university campuses and evaluated potential mitigation strategies. 18, [20] [21] [22] [23] [24] these studies used mathematical models of viral transmission dynamics, tailored to reflect a university context, in order to evaluate testing and contact tracing strategies, largely focusing on the question of how frequently to test asymptomatic students. all analyses concluded that frequent testing (sometimes multiple times a week) would be needed to contain covid-19 outbreaks on campus. high frequency testing has been adopted by a number of public and private universities including, for example, the university of illinois (twice per week by saliva testing 25 ), colby college (twice per week by nasopharyngeal swab 26 ), cornell university (twice weekly for students and faculty with student contact by self-collected anterior nares sampling 27 ) , and harvard university (one to three days a week for students, staff, and faculty by saliva testing with frequency depending on types of on-campus activity 28 ). however, most universities do not have the resources or infrastructure to support high-frequency testing. 26 furthermore, while some of these studies did include infections among students arising from offcampus community contact, these studies did not consider the impact of university student infections and university administration prevention and management decisions on the broader community in which that campus is situated. university students themselves may be at lower risk of severe covid-19 disease due to age, but high infection rates on campus may spill over into the broader community, whose members are at higher risk for adverse covid-19 outcomes. it is therefore important to quantify the expected impact of the arrival of a relatively large number of university students on the broader community in terms of incremental infections, hospitalizations, and covid-19 mortality. through the many interactions between the student population and the city in which they reside, covid-19 mitigation policies targeted at university student communities and adopted by university leaders may have substantial public health implications for those in the surrounding community. we developed a dynamic transmission model of covid-19 to estimate the health impacts and health care resource demand in a covid-19 outbreak in a representative mid-sized city with a relatively large destination college campus. we assumed a city experiencing a low level of covid-19 activity going into the semester and explore how the on-campus arrival of the student population impacts covid-19 health outcomes in the community. we consider different assumptions about student behaviours related to physical distancing and mask wearing, as well as the mitigating effects of targeted routine and one-time covid-19 screening in the university population. we developed a dynamic compartmental model to simulate infection dynamics and health resource use of a representative mid-sized city with a population of 500,000 going into fall after experiencing low rates of covid-19 infections in the summer. we divided the population into three sub-populations: long-term care (ltc) residents, university students, and the general population (everyone other than ltc residents and university students). we evaluated covid-19 health outcomes in the city between august 15 and december 31 (4.5 months) with and without the introduction of 20,000 university students on september 1. we explored how the covd-19 risk and prevention behaviours of the general population and the student community affect the incremental covid-19 burden attributable to the arrival of the student population. under different scenarios of community physical distancing effort and routine testing in students, we calculated the number of infected individuals, peak hospital resource demand, and number of deaths over time. institutional ethics review was not required for this modeling study as human subjects were not involved. a schematic of the model is presented in figure 1 . in the model, susceptible individuals may become infected through interaction with infected individuals who may or may not be aware of their infection status. infection has a pre-symptomatic phase in which an infected individual can transmit the infection to others. 4, 5, 29 individuals may become aware of their infection status through symptom-based surveillance, contact tracing, or routine testing of asymptomatic and mildly symptomatic individuals. individuals aware of their infection status with mild or moderate symptoms isolate at home to reduce disease transmission. some patients develop severe symptoms requiring hospitalization or critical symptoms requiring mechanical ventilation (mv) in an intensive care unit or renal replacement therapy (rrt). patients receive medically indicated care, unless resource demand exceeds capacity. when hospital capacity for a medically indicated resource has been reached, patients receive the next-best available care. we estimated model parameters, including the duration of time spent in each health state, the infectiousness of covid-19, demand for hospital resources and disease mortality conditional on disease severity, and the effectiveness of covid-19 prevention strategies using the peer-reviewed literature, pre-published reports, and expert opinion ( table 1) . we calibrated uncertain model inputs to the observed hospitalization and mortality outcomes in london, ontario, canada, a mid-size city with a large university population, between march 1 to august 15. full details about the model structure and input parameter estimates are presented in the supplemental methods. we first establish the epidemic starting conditions in the city on august 15 before the potential arrival of students for fall semester. based on calibration to covid-19 outcomes in london, on, at the start of the simulation, 40 individuals in the general population had active covid-19 infections, 13.8 were exposed but not yet infectious, and 2,476 individuals had already recovered; thus, 493,937 individuals were susceptible at the beginning of the simulation. in sensitivity analysis, we vary the number of active covid-19 infections in the general population at the start of the simulation. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09. 18.20197467 doi: medrxiv preprint in our analysis, 0.7% (3,500 individuals) of the population live in long-term care (ltc). based on the model calibration we estimated that 75 ltc residents were recovered as of august 15. we assumed that there were no active infections in ltc residents on august 15. we assume that 20,000 university students arrive on september 1. in the base case, we conservatively assume that no students are infected with covid-19 upon arrival to campus. we vary this assumption in sensitivity analysis. a schematic of the health states and transition times for infected individuals is presented in appendix figure 1 . we assumed a mean incubation period, the time from exposure to symptom-onset, of 5.6 days (observed median of 5.1 days [95%ci 2.2, 11.5]). 30 in total, we estimated the average duration of infectiousness in asymptomatic and mild or moderate infections to be 10 days, including 2.5 days prior to symptom onset in individuals who do become symptomatic. 4, 31, 32 we assumed that patients with severe and critical symptoms remain infectious until recovered, but that patient isolation protocols prevent transmission for those admitted to hospital. we estimated hospital length of stay and mortality based on a report of over 20,000 hospitalized patients in the uk. 33 we estimated length of stay and mortality for patients receiving critical care using the uk intensive care national audit and research centre report describing the care and outcomes of 10,118 critical care covid-19 patients. 34 for ltc residents who are and are not hospitalized, we estimated the case fatality rate to be 47.4% and 25.5%, respectively, based on the observed outcomes in 680 canadian ltc covid-19 patients. 35 combining the assumptions about disease severity and severityspecific mortality rates, the overall infection fatality rate in our model was 25.1% for ltc residents, 1.3% for the general population, and 0.06% for university students. among hospitalized cases, the fatality rate was 24.2% for the general population, and 5.2% for university students. we did not consider mortality from causes other than covid-19 in the model. general population: based on an extrapolation of the 2008 polymod study in europe to reflect network structure of the canadian population, the average number of contacts per person in canada is 12.6 per day, of which 1.0 contacts is aged 20-24. 36 assuming that the university student population adds 20,000 individuals aged 20-24 to a community with an otherwise typical canadian age distribution, university students would comprise 38% of the population of people aged 20 to 24 in the community. as a result, we assume that a person in the 'general population' has contact with 0.38 university students per day. we calculated the average number of ltc contacts by calculating the number that would balance the staff and visitor contacts estimated for ltc residents, resulting in 0.1 ltc contacts per person in the general population. long-term care residents: we estimated that there are 19.9 resident-resident contacts per day (95% ci: 11.3 -28.5) and 13.7 resident-staff contacts per day (95%ci: 11.4, 15.9) using a canadian study of longterm care residents and staff (personal communication: s. moghadas). 37, 38 this study did not capture . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint resident-visitor contacts; we assumed 0.48 visitors per day based on the distribution of visit frequency in the 2012 ohio nursing home family satisfaction survey. 39 thus, in total, ltc residents have 14.2 contacts with the general population each day. we assumed no direct contact with university students. university students: in our base case analysis, we assume that students have 23.7 contacts per day, based on a study at the university of antwerp, 40 and that 60% of those contacts are with other university students based on the age distribution of student's reported contacts, 40, 41 with the remainder being with members of the 'general population' which includes staff and faculty of the university. because studies evaluating university student contact patterns often occur during late fall and winter months, the contact patterns identified may not be fully representative of student contact rates at other times of the year. in sensitivity analysis we explored higher rates of student-student contacts in the first few weeks of the semester. using exponential regression, we empirically estimated a basic reproduction number, r0, of 3.0 in the general population based on ontario's reported cases between march 7 to march 22. 42 using an average duration of infectiousness of 10 days and an average number of close contacts per person of 12.6, 36 we calculate the probability of transmission between a susceptible and an infected person in close contact, in the absence of any interventions, to be 0.024. interventions such as physical distancing, which reduces the average number of contacts between susceptible and infected people, and mask wearing, which reduces the probability of disease transmission between contacts, can reduce the expected number of infections. we assume the effectiveness of cloth masks in reducing disease transmission is 40% based on a german study evaluating the effectiveness of real-world mask use. 43 for people who are aware of their infection status and in home isolation, we assume a 90% reduction in contacts, which is at the high end of observed adherence to quarantine instructions in past epidemics. 44, 45 we subdivided the general population into two groups based on their intensity of covid-19 prevention behaviours. based on behaviours reported in an angus reid poll of canadians, taken in the first week of august, 46 we estimated that 'high-intensity physical distancers', representing 50% of the general population initially, reduce their average number of contacts by 75% (from 12.6 to 3.2 contacts per day) and that 86% of their remaining contacts are protected by a cloth mask. we assumed that the remainder of the population are not reducing their contacts, but are using a cloth mask to protect 38% of those contacts. 46 in the base case, we assumed that university students initially reduce their contacts by 24% (from 23.7 to 18.0 contacts), which approximates a 75% reduction in contacts for the 32% of 18-24 year-olds who reported substantial covid-19 prevention effort in the angus reid poll (32% x 75% = 24% reduction). in this same survey, 57% of 18-24 year-olds reported wearing a mask indoors with people outside their household and we use this as the base case level of mask wearing in the university student population. 46 responsive behaviour triggers: we assumed that the general population and university students respond to covid-19 outcomes in the community. in practice, this response may be voluntarily adopted due to public concern over reported increases in covid-19 cases, hospitalizations, and/or deaths or . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint imposed through policies that re-establish the social and economic restrictions utilized in the earlier phases of the pandemic. we included two triggers that would cause both the general population and university students to increase their adoption of protective behaviors: a high level of covid-19 patients in critical care and a high number of covid-19 deaths. the critical care trigger was set based on critical care capacity. the overall critical care capacity in ontario is 14.2 beds per 100,000 population. 47 thus, in a city of 500,000, normal critical care capacity would be about 70 beds. while additional capacity can be created by seconding resources and personnel from other hospital services, 48 based on expert opinion, substantial reductions in the provision of other types of health care (such as cancelling elective surgeries) would need to be undertaken if more than 30 critical care beds were used by covid-19 patients. therefore, we set one of the responsive behaviour triggers to activate when there are 15 covid-19 patients in critical care, representing 50% of the capacity available to covid-19 patients without modifying access to other health care services. in the base case, the proportion of the general population who are 'high intensity physical distancers' increases by 0.5% each day if the number of covid-19 patients in critical care exceeds 15 and by an additional 1.0% each day if the number of covid-19 deaths in the past 10 days exceeds 10, up to a maximum of 80% participation in high-intensity physical distancing. similarly, we assume students' reduction in contacts increases at the same rate in response to the same triggers, but up to a maximum of a 50% reduction in contacts (23.7 contacts to 11.9 contacts). for the general population and university students, we assumed the minimum time from symptom onset to diagnosis to be 2.1 days, which is consistent with the minimum time needed to self-assess, seek medical attention, and receive diagnostic results. 7 the observed median time to diagnosis through symptom-based surveillance alone was 4.6 days (95%ci: 4.2, 5.0) and symptom-based surveillance in combination with contact tracing efforts was 2.9 days (95%ci 2.4, 3.4) in shenzhen, china. 49 from this, we estimated that symptom-based surveillance and contact tracing results in a daily probability of diagnosis of 15.8% for symptomatic infections and a daily probability of detection (from contact tracing) of 4.1% for asymptomatic infections. this combination of assumptions resulted in approximately 22% of infected individuals being identified, consistent with the overall rates of diagnosis implied by preliminary serology data in ontario. 50 we considered policy alternatives of routine screening for covid-19 in university students at various screening frequencies, including every 14, 10, 7, 5, and 3 days. we also considered the value of a onetime universal screening three weeks after student arrival. we identified the date of the one-time testing by identifying the date that minimized the number of total infections over the semester. we assumed that testing will be performed with the standard covid-19 nasopharyngeal swab followed by pcr analysis with a test sensitivity of 72.1%. 51, 52 . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint without the introduction of the student population, the base case assumptions for the general population and ltc residents leads to a total of 3,900 infections over 4.5 months (august 15 to december 31). in this scenario, infections, hospitalizations, and daily deaths do not peak until the new year (appendix table 1 ). demand for critical care (mechanical ventilation or renal replacement therapy) peaks at 27 beds early in the new year, and a total of 31 covid-19 deaths occur between august 15 and december 31 (60 deaths by january 31, 2021). the timing and magnitude of the city's covid-19 outbreak, excluding any impacts from students, is determined by the initial number of covid-19 infections in the community, the level of participation in physical distancing, the responsiveness of the community to increasing critical care cases and covid-19 deaths, and the proportion of contacts that are protected with mask wearing (appendix figures 5-7) . in the base case, we conservatively assumed that students would bring no undiagnosed infections of covid-19 to the community and would immediately engage in physical distancing efforts that resulted in a similar average contact reduction to the general population (reduction of 24%, from average of 23.7 contacts to 18.0 contacts). even so, university students continue to have a higher average number of contacts than members of the general population. as a result, in this base case scenario, the introduction of students to the community increases the total number of infections by 3,399 infections, representing an 87% increase (from 3,900 to 7,299) (figure 2) . only 28% (960) of these incremental infections occur in the student population (representing 4.8% of students becoming infected). of the remaining 72% of incremental infections, 2,428 occur in the general population (0.5% of the susceptible general population), and 11 occur in long-term care (0.3% of the susceptible ltc population). the larger absolute increase in infections in the general population occurs due to the connectivity of the university community with the general population and the relative size of the general population. the increase in infections among ltc residents, despite our assumption that there are no direct contacts between university students and ltc residents, occurs due to the increase in infections in the general population. the higher number of infections results in an increase in hospitalizations, demand for critical care, and covid-19 deaths. peak critical care occupancy increases by 48% (from 27 to 40 beds). these outcomes include the mitigation effects of the responsive behaviour of the community to seeing high levels of covid-19 hospitalizations. the introduction of students to the community also moves up the timing of responsive behaviours, with the threshold of 15 covid-19 patients in critical care being reached 3 weeks earlier (appendix table 2 ). if some students arrive exposed or asymptomatically infected, the total number of infections occurring over the course of the semester increases. for example, if 10 students arrive infected, the number of infections increases by 1,047 over the base case. the impact of students arriving already exposed or infected in the community is most substantial on the timing of peak infections, peak hospitalizations, peak critical care utilization, and the timing of responsive behaviour triggers. compared to the scenario without the introduction of the student population, responsive behaviours are triggered 6.5 weeks earlier if 10 students are infected when they arrive (appendix table 3 ). . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint our estimates of average student contacts per day were based on surveys completed at times other than during the first few weeks of the school year, which normally involve a large number of social activities. even if muted, the first few weeks of the new academic year may still result in substantially more contacts than a typical pre-covid era day. therefore, we explored the consequences associated with students having an increased number of student contacts for one or two weeks upon arrival to the community. for example, if students simply delay implementing physical distancing for two weeks, the total number of infections attributable to the introduction of the students into the community increases by 4,713 infections (from 3,900 to 8,613 infections). in this scenario, the arrival of students would result in a 120% increase in the number of infections the community. as in the base case, the majority of these incremental infections occur in the general population (figure 2 ). if students have twice the pre-covid era number of contacts with other students for two weeks (37.9 contacts per day, 28.4 of whom are other students), the total number of infections in the community increases by 6,394 infections representing a 164% increase in the number of infections the community would expect without the students and leading to an additional 72 covid-19 deaths (appendix figure 8 , appendix table 4 ). delays in the implementation of contact reductions, or short-term increases in the number of student-student contacts increases demand for critical care resources and shortens the time until critical care beds dedicated to covid-19 patients exceeds 30 beds (figure 2b) . the impact on total infections is mitigated by the earlier activation of responsive behaviour triggers, which occurs 8.3 weeks after the arrival of the students and 8 weeks earlier than without the arrival of the student population. because young people have a high rate of asymptomatic and mild presentation, routine testing of students has been proposed for university campus settings. testing students every 28 days results in very little reduction in the number of infections but requires a large number of tests (714 students tested per day). testing every 14 days, as is recommended for staff at long-term care facilities, 53 reduces the number of infections in the student population from 960 to 735, a 23% reduction, and reduces the overall number of infections due to the introduction of the university student population by 22% (from 3,399 to 2,653) (figure 3) . more frequent testing reduces infections further. testing students every 5 days reduces the number of infections among the student population by 44% (from 960 to 541) and reduces the total number of infections due to the introduction of the university student population to the community by 41% (from 3,399 to 2,007). routine testing has greater impact in the scenarios when students engage in a shortterm increase in the number of contacts early in the term. in the scenario in which students double their student contacts for the first two weeks of term, testing every 5 days reduces the number of infections in the student population by 47% (from 1,990 to 1,047), reduces the total number of infections due to the introduction of the university student population by 42% (from 6,394 to 3,732), and delays the activation of responsive behaviour triggers by 1 week (appendix table 5 ). . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint routine testing of students also averts critical care admissions and covid-19 mortality in the general population, because approximately two-thirds of averted infections are prevented in the general population ( table 2 ). in the base case analysis, in which students immediately reduce their contacts compared to a pre-covid-19, routine screening every 5 days averts 16.5 critical care admissions and 12.8 deaths. in scenarios in which students have short term increases in their contact behaviours or lower levels of contact reduction, routine testing every 5 days averts more than 29 critical care admissions and greater than 20 covid-19 deaths. sensitivity analysis revealed that routine testing of university students was more valuable when students have a higher rate of asymptomatic infections ( table 2 ) and in scenarios in which the differences in transmission risk between the university students and the general population were greater. for example, in a scenario in which the city had a high level of engagement in physical distancing routine screening of the student population can avert a large number of infections because in these scenarios the city expects very little covid-19 transmission without the introduction of the student population ( figure 3c) . conversely, in scenarios in which the city is engaged in a low level of physical distancing and so expects a large number of infections with or without the student population, the difference in risk profile between the city and the university populations decreases, as does the benefits of targeting prevention efforts at the university population. routine testing to identify and isolate asymptomatic infections for the purposes of reducing community transmission risk requires a large number of tests each day and may strain community testing resources. we also evaluated the benefits of a one-time universal screening event occurring three weeks after the students arrive. compared to routine testing every 5 days, which would require more than 400,000 tests to be performed over the semester, this strategy would only require 20,000 tests. through the isolation of identified cases, one-time testing is able to immediately decrease the daily number of new infections in the student population and, so, indirectly in the general population (appendix figure 9 ). in the case that students double their contacts with other students for a period of two-weeks, this strategy prevents 361 infections (122 infections in students, 238 infections in the general population, and 1 infection in ltc residents), 4.0 critical care admissions, and 3.1 covid-19 deaths ( table 2) . one-time screening does not significantly impact the timing of peak infections, resource utilization, or the time that responsive behaviour triggers are activated (appendix table 5 ). we performed extensive sensitivity analysis exploring the impact of general population and student population covid-19 prevention behaviours on the incremental impact of introducing students into the community. the negative impacts of introducing the student population can be partially mitigated through high uptake of covid-19 preventive behaviours in the student population including high rates of contact reduction or if the rate of mask wearing significantly exceeds the level reported by 18 to 24 year-olds (appendix figure 10 , appendix table 6 ). for example, if students immediately reduce their contacts by, . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint on average, 30% (from 23.7 to 16.6 contacts per day) and wear masks to protect 65% of their remaining contacts, the incremental number of infections attributable to the arrival of the student population can be reduced to 1,698 infections (from 3,900 to 5,598), representing only a 44% increase over the number of infections the community would expect without the students, and delays the activation of responsive behaviour triggers by 1 week. the magnitude of the impact of the introducing the student population is also determined by the covid-19 prevention behaviours of the general population. counter-intuitively, the relative impact of introducing the student population is greatest when the prevention efforts by the general population are high (appendix figure 11) . for example, when 60% of the general population are participating in high-intensity physical distancing, without students the number of new infections per day is nearly constant over time, resulting in a very low level of cumulative infections over the semester (total of 543 infections). introduction of the students results in 939 additional infections, more than doubling the total number of infections expected in the city without the addition of the student population (appendix table 2 ). in such a scenario, because the student population is an important determinant of city outcomes, the impact of routine covid-19 screening in the student population is greater ( figure 3c ). . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint in this analysis, we consider the covid-19 impacts of the re-opening of a destination university in a midsized city with varying epidemiological contexts. without the return of university students, we devised several scenarios involving moderate to severe fall covid-19 waves based on difference levels of preventive behaviours in the general population. the return of a large number of students always worsens these waves, even under the conservative assumptions that arriving students do not introduce any new infections to a community and that they immediately adopt substantial covid-19 prevention behaviours. this is because university students have nearly double the number of contacts as the general population due to residing in shared or congregate living situations, working in the service sector, and higher levels of social activity. in the scenarios we considered, this increase in infections was substantial, potentially doubling of the total number of covid-19 infections in the city over the fall. notably, we found that more than two-thirds of the incremental infections attributable to the arrival of the student population occurred in the general population and, as a result, substantially impacted the city's health care resource needs, covid-19 mortality in the community, and accelerated the need for responsive behaviour which may be in the form of re-engaged social and economic restrictions. our base case finding that the return of university students would increase the number of infections by 87% is likely conservative. in this analysis, we assumed that no students arrive already infected, students do not engage in short term increases in contacts upon arrival, and that students respond to adverse community covid-19 outcomes by increasing the intensity of their prevention behaviours at the same rate as the general population. at the very least, it may take time for students to fully adopt protective measures; moving into dorms, orientation events, and start of semester social events (even if not officially sanctioned by the university) may result in higher-than-normal levels of contact for at least the first few weeks. in the analyses in which we consider short-term increases in the average number of student-student contacts, we show that a higher level of contact for just the first one or two weeks can dramatically increase the total number of infections experienced by the city over the semester. our analysis found that the majority of the increase in infections due to the arrival of students occurred in the general population, not in the student population itself. while university campuses may seem like isolated bubbles, the general community and university students are intertwined, as staff and faculty interact with students on campus and students interact with others off-campus in work, living, and social settings. previous studies modeling university populations did not account for infections in the broader off-campus community. [20] [21] [22] [23] [24] however, we have shown that including the general population when modeling covid-19 transmission on university campuses is critical, since this population bears the brunt of the incremental morbidity and mortality burden of covid-19. as a result, university policies that either discourage student return to the community, 54 such as offering coursework fully online, or mitigate covid-19 risks for students returning to campus, such as screening for covid-19 symptoms and routine covid-19 testing in asymptomatic individuals, can have substantial impacts on the city's covid-19 burden. imposing restrictions on students' off-campus social behavior may be practically difficult, necessitating modified behaviors in the general population in response to university reopening, such as additional reductions in social contacts to balance the increased risk of returning university students. for example, in the base case, the increase in infections due to student arrival could be mitigated if the proportion of the general population engaged in high-intensity physical distancing increased by 4.6% (from 50% to 54.6%). this illustrates the idea of "risk budgets", where increased risk . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09. 18.20197467 doi: medrxiv preprint in one domain of a community necessitates reducing risk in another to keep covid-19 impacts below desired thresholds. 55 our analysis indicates that routine testing of all students every 5 days can avert a substantial number of infections, critical care admissions, and covid-19 deaths. in the base case, we estimate that testing every 5 days prevents 16.5 critical care admissions and 12.8 deaths and, in the scenario in which students double their contacts with other students for two weeks, we estimate that testing every 5 days prevents 29.1 critical care admissions and 22.6 deaths. using the simplifying assumption that all deaths averted will be in 60-year-olds and a willingness to pay of $50,000 per life-year-gained, the economic value of these deaths averted is $13.4 million to $23.6 million, translating to a value of $33 to $59 per test. this calculation underestimates the benefits of testing by not accounting for savings due to averted critical care admissions and the community economic benefits of delaying social and economic restrictions; despite this, because we estimate the lab cost of nasopharyngeal testing for covid-19 at $80 per test at our center, high-frequency routine testing is likely only cost effective using batch testing strategies. alternatively, one-time universal testing of students after an initial burst of social activity among students may be more feasible operationally and economically. we estimated that this strategy can prevent 360 infections, 4.0 critical care admissions, and 3.1 covid-19 deaths corresponding to an approximate economic value of $3.3 million or $164 per test. this strategy is most effective at changing the trajectory of new infections if testing occurs after a short-term period of high social activity and is less effective if students have consistent but lower levels of contact reductions (e.g., immediately reducing their contacts, but by only 15% instead of the 24% in the base case). an important limitation of our analysis is that we assume students with a covid-19 diagnosis will be willing and able to self-isolate effectively. however, it may be challenging for students to isolate from roommates or refrain from using shared facilities, like bathrooms and kitchens, without dedicated university-organized isolation facilities. 56, 57 furthermore, adherence to isolation guidance may be low, especially if the majority of infections in university students are asymptomatic or mild. during the h1n1 pandemic, a survey of symptomatic university students found that only 41% of students followed recommendations to stay home until well. 16 in the base case, we also assume that students are equally responsive as the general population to covid-19 outcomes in the community reducing their contacts in response to high numbers of critical care hospitalizations and deaths. in reality, university students may be less aware of the impacts of covid-19 on hospital resources and less concerned about covid-19 generally given their lower risk of adverse outcomes. the extent and speed with university students respond to hospitalizations and deaths in the local community will impact the number of infections experienced by the community and the benefits of routine testing in the student population. compared to other modeling studies of covid-19 on university campuses, the total number of infections and the number of infections averted by testing we estimate over the semester are modest. this is because we assume that both university students and the general population will increase their self-protective behavior (physical distancing) in response to high numbers of covid-19 hospitalizations and deaths, either through individual decision-making or adaptive community policies. these adaptive behaviors are more realistic than assuming a population will maintain the same behavior no matter the severity of local covid-19 conditions. thus, in our analysis, testing is being layered onto a robust and reactive mitigation response. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint our model includes only three sub-populations and so does not include many other aspects of agestructured mixing or age-dependant health outcomes. the model does not estimate the impact of covid-19 patient utilization on the provision or effectiveness of other health care services and the model does not account for death from causes other than covid-19. 58 the model includes community transmission by stratified random mixing but does not include additional imported index cases from other cities, which may occur into the general population or the student population, nor does the model include the stochastic consequences of super-spreading events. especially early on in an epidemic or when cases have been brought to very low levels, dynamics are sensitive to random outcomes in the number of new infections resulting from each case (e.g., 'patient 31' in south korea 59 and 'patient one' in italy 60 ). we developed a model-based analysis to estimate the impact of a relatively large student population on the covid-19 outcomes of a mid-sized city with relatively few cases of covid-19 prior to the return of students. our analysis is relevant to a number of mid-sized cities in north america with relatively large university and college populations. because university students have substantially more contacts than the general population, due to congregate living environments, high-density social activities, and disproportionate employment in the service sector, the introduction of university students substantially increases the number of covid-19 infections and decreases the time until responsive behaviours are activated. substantial uncertainty exists in the level of contact reduction that students will choose, or is feasible given their living, transit, and work situations. public health interventions, such as routine testing, targeted at this population prevents infections in the entire population, improving community health related and unrelated to covid-19. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint figure 1. model schematics of (a) covid-19 health states and (b) close contact interactions between population subgroups. the number of contacts between groups indicated on the schematic represent the average number of contacts per day in a pre-covid-19 era. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. table 4 . end of the two-week burst in contacts. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . in panel (a), students have an average 24% reduction in contacts compared to normal student social interaction behaviour (average of 23.7 contacts reduced to 18.0 contacts) immediately upon arrival with no shortterm increase in contacts; in panel (b), students double their contacts with other students for the first two weeks and then implement a 24% reduction in their contacts; in panel (c), students double their contacts with other students for the first two weeks and then implement a 24% reduction in contacts and 60% of the general population is participating in high-intensity physical distancing (compared to 50% in the base case and other scenarios presented in this figure). other outcomes for these scenarios are reported in appendix table 5 . . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint 49 from this, we estimated that symptom-based surveillance and contact tracing results in a daily probability of diagnosis of 15.8% and the daily probability of detection from contact tracing of 4.1% in asymptomatic infections. c. among critical care patients, we estimate the ratio of patients requiring renal replacement therapy (rrt) to mechanical ventilation (mv) based on the uk intensive care national audit and research centre (icnarc) report describing the care and outcomes of 10,118 critical care covid-19 patients in the uk. in this report, 7,277 patients required mv and 2,673 required rrt, resulting in a ratio of 0.37 rrt patients per mechanical ventilation patient. 34 d. in canada, based on 63,800 covid cases in people who were not residents of long-term care facilities reported between february 23 and june 21, 20.3% of hospitalized patients received critical care; 35 this is also consistent with rates of critical care observed in the uk (22% overall hospitalized patients go to icu). 33 therefore, we estimate the ratio of 3.92 hospitalized without critical care patients per critical care patient. e. initially estimated using the same process as is described in footnote d. adjusted in calibration process to better fit the observed data (see supplemental methods). f. median and iqr presented in the cited primary work were transformed to mean (95%ci range) assuming a gamma distribution. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint table 2. infections averted in the general population with 5-day testing and one-time testing of students compared to a policy of no routine asymptomatic testing (symptom-based surveillance and contact tracing only). scenarios vary the proportion of infections in the student population that are asymptomatic and timing and level of students contact reductions. we calculate the expected number of critical care admissions averted and covid-19 deaths averted to be 1.7% and 1.32% of general population infections averted which includes hospitalizations and deaths which may occur after december 31 to all individuals infected prior to december 31. one-time testing three weeks after student arrival compared to no routine testing . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint the copyright holder for this this version posted september 18, 2020. . https://doi.org/10.1101/2020.09.18.20197467 doi: medrxiv preprint high contagiousness and rapid spread of severe acute respiratory syndrome coronavirus 2 pattern of early human-to-human transmission of wuhan early transmission dynamics in wuhan, china, of novel coronavirus-infected pneumonia temporal dynamics in viral shedding and transmissibility of covid transmission interval estimates suggest presymptomatic spread of on behalf of the oxford covid-19 evidence service team. in patients of covid-19, what are the symptoms and clinical features of mild and moderate cases? : centre for evidence-based medicine report 8: symptom progression of covid-19. imperial college london covid-19 response team baseline characteristics and outcomes of 1591 patients infected with sars-cov-2 admitted to icus of the lombardy region epidemiology, clinical course, and outcomes of critically ill adults with covid-19 in new york city: a prospective cohort study us centers for disease control and prevention. considerations for institutions of higher education extensive geographical mixing of 2009 human h1n1 influenza a virus in a single university community college campus outbreaks require timely public health response how the h1n1 influenza epidemic spread among university students in japan: experience from shinshu university transmission of 2009 pandemic influenza a (h1n1) at a public university--delaware non-pharmaceutical interventions during an outbreak of 2009 pandemic influenza a (h1n1) virus infection at a large public university schools briefing: university outbreaks and parental angst college professors made models showing how bad covid-19 would be on campus. some administrators ignored them outbreaks drive u.n.c. chapel hill online after a week of classes. the new york times assessment of sars-cov-2 screening strategies to permit the safe reopening of college campuses in the united states covid-19 mathematical modeling for cornell's fall semester high covid-19 transmission potential associated with re-opening universities can be mitigated with layered interventions entry screening and multi-layer mitigation of covid-19 cases for a safe university reopening colleges plan for coronavirus testing, but strategies vary. wcvb news covid-19 and reactivation planning: surveillance testing coronavirus testing & tracing: unobserved self-administered testing risk for transportation of 2019 novel coronavirus disease from wuhan to other cities in china the incubation period of coronavirus disease from publicly reported confirmed cases: estimation and application virological assessment of hospitalized patients with covid-2019 features of 20 133 uk patients in hospital with covid-19 using the isaric who clinical characterisation protocol: prospective observational cohort study statistics canada. detailed preliminary information on confirmed cases of covid-19 (revised), public health agency of canada projecting social contact matrices in 152 countries using contact surveys and demographic data the effect of individual movements and interventions on the spread of influenza in long-term care facilities individual movements and contact patterns in a canadian long-term care facility the predictors of and motivations for increased family involvement in nursing homes social mixing patterns for transmission models of close contact infections: exploring self-evaluation and diary-based data collection through a web-based interface who mixes with whom? a method to determine the contact patterns of adults that may lead to the spread of airborne infections ontario ministry of health and long-term care. covid-19 case data face masks considerably reduce covid-19 cases in germany changes in contact patterns shape the dynamics of the covid-19 outbreak in china how to improve adherence with quarantine: rapid review of the evidence covid-19 compliance: one-in-five canadians making little to no effort to stop coronavirus spread critical care readiness: expanding nursing staff to support covid-19 epidemiology and transmission of covid-19 in 391 cases and 1286 of their close contacts in shenzhen, china: a retrospective cohort study blood tests indicate one per cent of ontario's population had covid-19 variation in false-negative rate of reverse transcriptase polymerase chain reaction-based sars-cov-2 tests by time since exposure comparative accuracy of oropharyngeal and nasopharyngeal swabs for diagnosis of covid-19. centre for evidence-based medicine ontario ministry of health and long-term care. covid-19 testing for long-term care home staff winter term will be online: provost's letter close the bars. reopen the schools. vox college quarantine breakdowns leave some at risk. the new york times a university had a great coronavirus plan, but students partied on. the new york times delayed access or provision of care in italy resulting from fear of covid-19 the korean clusters. reuters graphics updated coronavirus: inquiry opens into hospitals at centre of italy outbreak the guardian outbreak investigation of covid-19 among residents and staff of an independent and assisted living community for older adults in infections in residents of a long-term care skilled nursing facility asymptomatic sars-cov-2 infections: a living systematic review and meta-analysis probability of symptoms and critical disease after sars-cov-2 infection effect of acute renal failure requiring renal replacement therapy on outcome in critically ill patients acute renal failure in intensive care units--causes, outcome, and prognostic factors of hospital mortality; a prospective, multicenter study. french study group on acute renal failure key: cord-341709-nzvon5hc authors: whitley, jess; hollweck, trista title: inclusion and equity in education: current policy reform in nova scotia, canada date: 2020-09-09 journal: prospects (paris) doi: 10.1007/s11125-020-09503-z sha: doc_id: 341709 cord_uid: nzvon5hc this article aims to explore the context of inclusive education policy in canada, and to highlight the particular case of inclusive education policy reform in the province of nova scotia. as with most other provinces and territories, inclusive education policy in nova scotia has broadened to include a lens of equity, with a focus on not only students with special education needs, but all students – particularly those most often marginalized by and within canadian school systems. the article reflects on the first phase of the developmental evaluation process which took place prior to full implementation of the policy. four interconnected key themes emerge: 1) the shifting roles and identities for educators and specialized staff; 2) the changing roles of classroom teachers; 3) the importance of support to ensure effective universal and differentiated classroom practices; and 4) the professional learning of school staff. although situated within the nova scotian and the national canadian context, the discussion and implications can readily be applied to international systems engaged in developing and implementing broad inclusive education policy. across canada, where education is provincially, rather than federally organized, all provinces and territories have demonstrated a commitment to the ideals of inclusion and equity (hutchinson and specht 2019) . several provincial and territorial policies related to inclusive education have been re-imagined over the past decade to include a broader perspective on equity as it relates to providing equal access and opportunity to all students, particularly those who have historically been marginalized or viewed as unsuccessful in canada's education systems (government of alberta 2020; ontario ministry of education 2009; the social planning council of winnipeg 2016). this focus is in contrast to previous and in some provinces, current policies where inclusive education refers primarily to students with special education needs (province of british columbia 2020). the importance of valuing and reflecting the diversity of student identity, including ancestry, ethnicity, gender identity, intellectual ability, and socio-economic status in schools, as well as identifying and dismantling systemic barriers is often highlighted. this type of equity and inclusion policy is reflective of international efforts by organizations like the un with sustainable development goals that include a need to "ensure inclusive and equitable quality education and promote lifelong learning opportunities for all" (united nations 2015, p. 14). the persistence of educational inequities (among others) for certain populations has been noted across canada by those within and beyond the country borders (george et al. 2020 ; gordon and white 2014; united nations 2017) . as an example, black students, those who self-identify as indigenous and those with disabilities are significantly less likely to experience the opportunities and successes of their peers within provincial education systems (canadian human rights commission 2017; james and turner 2017; province of nova scotia 2020a). recognition of continued disparities among student experiences and outcomes has propelled recent efforts in provinces including nova scotia to collect and share achievement and wellbeing-related data that is disaggregated in a number of ways (ontario ministry of education 2017; unicef canada 2019). this approach will in theory allow for greater attention to the gaps that exist and guide policies and programming to improve outcomes for students marginalized by and within education systems, suggesting both an educational and social justification (campbell 2020 ). while many necessary efforts related to equity and inclusion emerge within and across provinces and territories, approaches differ with respect to the positioning of these alongside deeply entrenched special education policies. it has been noted by ainscow (2019) that a focus on including students with disabilities "should be seen as an essential part of an overall strategy for promoting inclusion and equity, not as a separate policy strand" (p. 6). across jurisdictions, separate special education policies, some of which have been developed or updated within the past 5 years, continue to exist in parallel with the equity efforts (bc ministry of education 2016; ontario ministry of education 2017; province of manitoba 2017; province of nova scotia 2008). in terms of class placement, which is how we historically have defined inclusive education, most students with special educational needs in canada are educated in regular classrooms, with varying levels of "pull out" service provision; segregated classes and a small number of segregated schools also exist in most provinces and territories (specht et al. 2016) . certain groups of students, in particular those with intellectual and developmental disabilities, are over-represented in segregated settings in canadian school systems (reid et al. 2018) . within the range of placements available, students with disabilities may be included socially and/or may be supported in engaging with the regular curriculum with same-age peers. however, as in an analysis of inclusive education in ontario by parekh (2018) , "it is critical to actively explore into what are we including students. how do we ensure the environment, curriculum, and climate are ones which students can experience a sense of belonging, a sense of membership and shared power?" (para. 4). physical placement is not synonymous with inclusion. in implementing inclusive education, provincial and territorial ministries of education all include some mention of universal design for learning (udl) and differentiated instruction (di) as pedagogical and assessment practices that can facilitate teaching and learning in diverse, inclusive settings (whitley et al. 2019) . udl principles can align with practices of different levels or "tiers" of intervention and support -typically three (katz 2013) . in canada, multi-tiered systems have been in varying stages of adoption for many years, in the form of response to intervention (rti) (mcintosh et al. 2011) . rti in many provinces is intended to shift from a traditional approach where funding and programming are based on a testing, identification, and placement process to a focus on identifying individual student strengths and needs and to providing interventions where needed. while considering the needs of all learners is possible within a tiered system, the language surrounding udl, mtss and rti in canada tends to reflect a focus on students with special education needs (robinson and hutchinson 2014) . thus, while policies have shifted in some provinces to adopt a more multidimensional inclusive lens, there continues to be evidence of parallel equity and inclusion conversations. our current work in the province of nova scotia is an opportunity to examine the ways in which a new policy of inclusive education is understood and implemented in light of the broader canadian and international discussions of equity and inclusion. since the spring of 2019, our research team based at the university of ottawa (led by professors jess whitley and andy hargreaves) has been engaged in a developmental evaluation of the implementation of a new inclusive education policy in nova scotia. this implementation is in its early phases, with fall 2020 set as the official start date of the policy. nova scotia is the largest maritime province in eastern canada with a population of approximately 940,000; halifax is the capital city with almost half of the residents of the province. the province has a large rural population of over 40 per cent (statistics canada 2019). according to the canadian income survey of 2018, 10.3 per cent of nova scotians live below the poverty line which is the highest share in canada where the average is 8.7 per cent (province of nova scotia 2020b); 24 per cent of nova scotian children live in poverty which is the third highest rate in the country (frank and fisher 2020) . within nova scotia, approximately 5.7 per cent of the population self-identify as indigenous, the majority of whom are from the mi'kmaq nation and are recognized as the founding people of nova scotia (province of nova scotia 2014). approximately 2.5 per cent of nova scotians identify as african nova scotian, most of whom have lived in the province for three generations or more (province of nova scotia 2019a). there is also a strong acadian and francophone presence (province of nova scotia 2020c). with respect to the organization of schools across the province, a major shift occurred in 2018 with the adoption of recommendations arising from a review of the education system (glaze 2018 ). glaze made several controversial recommendations including the elimination of elected school boards and the removal of school administrators from the nova scotia teachers union (nstu). many of the suggested changes were heavily criticized (laroche 2018; ritchie 2019 ) and a few were subsequently rejected in response to imminent job action by the nstu (nova scotia teachers' union 2018). the system was reconfigured to contain seven regional centres of education (rces), with executive directors who report directly to the eecd -the francophone school board (csap) structure remained unchanged. most of the newly formed rces contain broad geographic swaths of the province with many including extensive rural areas. the halifax rce is the most urban region and also the most populated (in all of atlantic canada), with 134 schools and approximately 52,000 students; the tri-county region is among the most rural and has approximately 6,000 students in 22 schools and encompasses over 7,000 square kilometers. in terms of broad achievement and equity, students in english-language schools in nova scotia scored similarly to the canadian average on reading in the 2018 programme for international student assessment (pisa) test; students in french-language schools scored below the canadian average (o'grady et al. 2019) . provincial assessment results in nova scotia are regularly disaggregated by student self-identification as either mi'kmaq or other indigenous ancestry (4 per cent of enrollment), or of african descent (4 per cent of enrollment). the most recent results demonstrate discrepancies in the scores, with indigenous and african nova scotian students scoring lower than non-identified peers in all instances (province of nova scotia 2020a). while we view equity in terms of achievement as central to inclusion, we know it to be integrally related to wellbeing. exclusion, isolation, stigmatization and lack of belonging give rise to problems of emotional, mental and physical health -and most certainly impact achievement. students across nova scotia in grades 4 through 12 were surveyed on topics related to wellbeing and school experiences for the first time in 2019. many positives were noted in the findings, including high rates of reports of having a close friend or adult to talk to, and generally high academic expectations of self and teachers. overall, 78 per cent of students reported feeling like they belong at their school, with 72 per cent of students of african descent and 70 per cent of mi'kmaq/indigenous students in agreement. notably, only 65 per cent of those with disabilities and 54 per cent of students identifying as lgbtq2+ felt like they belonged at school. the same four subgroups of students reported feeling less respected than other students and less safe at school than the provincial average. over several decades, a number of reviews, reports, and position papers have documented the strengths and needs of the nova scotia education system, including the educational disadvantages related to indigenous and african nova scotian students (aylward et al. 2007; glaze 2018; njie et al. 2018; nova scotia teachers' union 2009; nunn 2006 ). in addition, in parallel with other provinces in canada, the concept of inclusive education has expanded beyond students with identified disabilities, to encompass other needs and populations at risk of exclusion and under-achievement such as immigrant and refugee populations, african nova scotians, students growing up in poverty, mi'kmaq communities, and gender-related minorities (ns department of education and early childhood development 2020). a focus on the unique experiences, needs and barriers typifying communities with strong cultural identities and historical presence is not a recent shift in nova scotia. with respect to african nova scotian students, the black learners advisory committee (blac) report on education was released over 25 years ago (1994) and describes a series of inequities and systemic barriers within the education system, and society broadly. following the blac report many policies, reports and frameworks have continued efforts and shed a critical lens in order to guide improved outcomes for african nova scotian students through curriculum, instruction, assessment, leadership and partnerships with families and communities (enidlee consultants inc. 2009; province of nova scotia 2002; sparks 2012). concerns regarding equity were also noted in the report produced by the commission on inclusive education which took place in 2017-2018 (njie et al. 2018 ). this report, entitled "students first", serves as the foundation of the new inclusive education policy and describes obstacles to progress in inclusive education, including inconsistent operationalization and implementation of various policies and procedures. as the commission noted, this lack of shared agreement regarding the basic underpinnings and practices that comprise inclusive education in nova scotia limits the potential of collaborative learning communities that are key to ongoing development of pedagogical practice within a culture of inquiry. the commission and its recommendations were strongly supported by the nstu although concerns regarding a lack of collaboration and transparency continue to be raised in terms of the speed and nature of implementation efforts (nova scotia teachers' union 2019). the current inclusive education policy (province of nova scotia 2019b), furthermore referred to as the policy, emerged from the findings and recommendations within the students first report (njie et al. 2018) , as well as the broader history and context of nova scotia education. the policy was developed by the eecd in consultation with multiple groups and was released to the public in august 2019. implementation of the policy is slated to take effect in september 2020. the policy includes a broad focus on ensuring equity, opening with the statement: "inclusive education is a commitment to ensuring a high-quality, culturally and linguistically responsive and equitable education to support the well-being and achievement of every student. all students should feel that they belong in an inclusive school-accepted, safe, and valued-so they can best learn and succeed" (province of nova scotia 2019b, p. 1). the eight guiding principles of the policy are listed in figure 1 . the policy also includes a number of directives which include classroom teachers taking on responsibility for all students, most often in a common learning environment, with support and collaboration. the development of specific school-based teams to support teachers and students are described, along with the directive that "every school will support student well-being and achievement through a multi-tiered system of supports (mtss)" (province of nova scotia 2019b, p. 4). finally, a series of roles and responsibilities for each group of education stakeholders (e.g., students, families, teachers, administrators) is detailed. following the release and recommendations of the students first report (njie et al. 2018) , the government of nova scotia announced initiatives and funding that were situated within the umbrella of inclusive education implementation. in the fall of 2018, 191 new positions were added to the education system. these new hires included a mix of positions that were new to the system, as well as additional hiring of existing positions; examples include child and youth care practitioners, autism and behaviour specialists, parent navigators, african nova scotian and mi'kmaq student support workers. in 2019, a second wave of hires (173 positions) were added. in late 2018, in order to guide and support the implementation of the policy, and to provide greater accountability as recommended in the students first report, the government sought proposals for a developmental evaluation. our research team bid successfully and, every student can learn with enough time, practice and equitable and responsive teaching. every student, including those with special needs, should receive full-day instruction every day, with flexibility based on the student's individual strengths and challenges. every student should be taught within a common learning environment (e.g., a classroom) with students of similar age within the community school, with flexibility that is based on, and responsive to, the student's individual strengths and challenges. inclusive education values, draws upon, and includes student voices and choices to assist students in achieving their goals. every student deserves to belong (affirmed, validated, and nurtured), be safe, and feel welcomed in all aspects of their daily experience. inclusive education is a commitment to honour and respect each student's cultural and linguistic identities and knowledge systems inclusive education practices use evidence of students' strengths and challenges to determine a system of supports and monitor the effectiveness of those supports. all partners are committed and empowered to work collectively to identify and eliminate barriers that interfere with students' well-being and achievement. nova scotia inclusive education policy: guiding principles in the spring of 2019, as the policy was being finalized, our research team embarked upon a developmental evaluation of its implementation. the next section details our approach and initial findings. the innovation in inclusive education currently underway in nova scotia offers an ideal opportunity to interweave a developmental evaluation. the evaluation is an iterative process, whereby our team works with eecd staff and a broader design team to conceptualize implementation of broad system change for inclusive education. as a collective, we are engaged in collaboratively generating the specific questions to guide the developmental evaluation, establishing priorities, identifying what data is required and how it will be collected, interpreting findings, and tracking development to inform next steps whilst considering changing conditions and new information gathered throughout the process. in our application of developmental evaluation, we situate ourselves within the eight essential principles outlined by patton (2016) which include evaluation rigor, utilizationfocused, systems thinking, co-creation and timely feedback. our first few months engaging with the developmental evaluation have been focused on: a) developing relationships with those knowledgeable about, and likely to be affected by the policy; b) developing a greater understanding of the many moving pieces that comprise the inclusive education policy implementation; c) working closely with a design team to guide the ongoing evaluation planning; d) collecting and analyzing data provided by key informants; and e) providing initial feedback to the eecd based on our findings to date. our first step was to develop an evaluation design team in consultation with eecd which comprises a teacher, principal, regional executive director, eecd executive directors of early learning and student success and the eecd director of research and partnerships. as researchers "from away", we also began the process of developing relationships with key individuals and groups to support and inform our shared work. a project manager, based in nova scotia, was hired to assist with planning and to maintain a consistent local presence in the eecd. beginning in the summer of 2019, we also engaged in early data collection, which will inform the analysis in the next section. we travelled to nova scotia four times in the first eight months of the evaluation, twice to halifax, and twice to other regions. given the pre-implementation phase within which these visits took place, the focus of our evaluation efforts was on: a) developing a deeper sense of the context in which the policy and its implementation was embedded and b) exploring the perceptions of key stakeholders regarding the policy. to the first goal, we engaged in information-gathering discussions with 24 individuals within and outside of the eecd. some of these individuals preferred not to be interviewed and to have their perspective captured on a recording and all provided helpful contextual information that informed our ongoing efforts. our project manager met with key individuals with the eecd on a weekly basis, to stay abreast of ongoing developments and to take part in the everyday conversations of those engaged in the implementation. we also took part in six advisory group or internal leadership meetings, reflecting participation by teachers from across the province, directors of programs and student services from each region and csap, regional executive directors and the eecd leadership team. in addition to our efforts to learn and engage, formal interviews were conducted with 23 key informants, including regional senior staff, school leadership and resource teachers, representatives of unions and professional associations, as well as post-secondary institutions that offer bachelor of education (teacher preparation) and graduate education programmes. interviews focused on the roles and contexts of the participants, their knowledge and perceptions of the policy, and hopes and concerns for the policy implementation. the interviews were audio-recorded and were on average 45 minutes in length. verbatim transcripts of each interview were developed and analyzed thematically. four main themes emerged from our analysis of this preliminary series of interviews. these include: 1) shifting of roles and identities: who am i and what do you do? 2) the changing roles of classroom teachers: islands no longer? 3) inclusion = mtss and success hinges on tier 1, and 4) professional learning: if we teach a teacher to fish. when discussing the inclusion policy, the majority of participants described the state of flux, either present or anticipated, of the roles and responsibilities of a range of school staff. much of the discussion within these themes focused on new positions within the education system that were added prior to the release of the policy. in general, participants were positive about the addition of the positions and the ways in which these individuals and their skill sets could and in some cases already were benefitting staff, students, parents and community partners. however, confusion was regularly expressed by participants, both those in the new positions and working alongside, regarding job descriptions, overlap of tasks with those of existing staff, boundaries of responsibilities and a lack of clarity and communication related to the new hires. efforts were described where regions were mapping the roles and responsibilities of the new positions alongside others with the multi-tiered system structure; these efforts were seen as very beneficial. some of the concerns expressed by participants related to perceived threats to expertise and identity. one school guidance counsellor stated: we need the support, but i'm always going to be like, "but what's our role?" because in some ways i've felt like our role has been chipped away at through all of the new supports that have been put in. i know a more positive way to look at it would be there are supports and we got to join with them, but it's hard when in some of these cases, some of these roles, we don't really understand what the role is. the lack of clarity and alignment was also raised as a barrier to effective supports for students -not knowing who was responsible for what in which type of situation could lead to student needs not being addressed or overlap of services. shifting roles were also discussed by the majority of the participants in response to the perceived emphasis of the inclusion policy on the increased responsibility of classroom teachers to meet student needs. the new inclusive education policy states that "classroom teachers are responsible for teaching all students. teaching and interventions are expected to happen most often in a common learning environment, such as the classroom. classroom teachers cannot do this alone" (province of nova scotia 2019b, p. 3). participants described the significant shift in mindset and practice that this would necessitate for some teachers who were at an earlier point along an inclusive continuum and who were used to what was described by one participant as the "70s pull out model" which was still in evidence in many schools. one regional consultant stated, "…we've got to support those classroom teachers, if we're saying they're the folks that are responsible with support, we got to do more support for them. way more support". as is evident in the policy statement above, the shift towards greater responsibility for classroom teachers is intended to be accompanied by greater collaboration. although many teachers were already working in partnership with other teachers, specialists, families and community, the sense of ownership for planning and programming for students with special education needs in particular was seen in some instances as sitting with a resource or learning centre teacher. the shift in the policy noted by many participants who were providing specialist supports at the school or regional level (e.g., resource teacher, psychologists, student services consultant) was described as changes in their roles from supporting students to supporting teachers. the policy described how a range of support teachers (currently resource or learning centre teachers) "would provide direct, collaborative support to classroom teachers and students" (province of nova scotia 2019b, p. 3). this is in contrast to the current role of some support teachers which strictly involves working directly with students in a pull-out model or of some psychologists who engage primarily in individual student assessment and intervention. one specialist wondered, "can we do more in class consultative types of services with our slps so that teachers are building their skillsets and understanding what they can do at the classroom level before they are asking for direct therapy whether small group or individual?" having greater collaboration within a regular classroom requires an openness that was seen as challenging for some teachers. one regional consultant believed, "the big issue, i think, for teachers is that they're not really used to having people in their classroom… and they feel like they're being judged all the time. and it's not about judging them. it's about supporting them. and with a multi-tiered system of support and inclusive education, you've got to get used to having many people in your room. although described sequentially, this third theme is tightly interwoven with some aspects of the first two, specifically the shifting roles and responsibilities for regular classroom teachers. there was a very strong and shared belief evident among participants of the importance of tier 1 described as the need to bring supports to the classroom and the need to ensure effective universal and differentiated classroom practices. one regional leader described how "there's no question that there's an increased focus on excellence in teaching at tier one, which i am a very strong proponent of" and a school psychologist shared her belief that "the tier 1 has a ton of potential to really improve overall student learning if we are looking at increasing some of the more evidence-based interventions". one guidance counsellor described her desire to work more often alongside classroom teachers rather than spending most of her time supporting individual students but also described the overwhelming mental health needs of students in her school and the existing 1:500 ratio of counsellor to students: "and you want to get into the classrooms and do the proactive piece. you do, because you're hitting 30 kids when you're getting in there and doing cyber stuff, when you're doing positive mental health". in describing ways in which she could meet the needs of a large group of students however, one resource teacher described that supporting individual teachers at the tier 1 level, in the general classroom, was just not feasible. the long-term view adopted by system leaders and supports was less palatable to those working on a daily basis with large caseloads of students: well, i usually group my students according to their needs. so ideally, i get into classes, which i do. but a lot of the time, because of the needs in this building, i find that it turns out being a pull-out model…just for the amount of students that i need to see, like i could have upward of 50 students during first intake and another 50 during second intake. so, the needs are high, and it's much more efficient for me to take them all at the same time. a need for professional learning emerged from the discussions of the first three themes, where the increasing focus on regular classroom practices at the tier 1 level and the shifting roles of educators and staff in a number of different positions generally led to the same recommendations from participants: more professional learning. discussions of professional learning typically centred around capacity-building among classroom teachers and the role of skill development in building confidence and a willingness to take ownership for students experiencing difficulties. one regional consultant described how she noticed in her work, "…a lot of people don't feel that they have the skill set to be able to deal with students that have diverse learning needs, that they really don't feel that they've had the training… because if they've been in the school system a long time, differentiation may be not really a concept that's… or udl, that really is something that they can pull out of their back pocket". the framing of the multi-tiered system was also applied to teacher professional learning as it was recognized by participants that different teachers working in different roles within different regions would have varying professional learning needs. as with the example provided by the resource teacher describing the necessity of a pull-out approach, a shift to a capacity-building versus a child-fixing mindset is a long-term view likely to be challenging for many teachers overwhelmed by day-to-day challenges. one regional consultant described this view: but you really have to look at the staff that you have and the expertise and how can we make this work if we do things in a different fashion. because it's about building capacities, the other thing, and that's what a lot of the time people aren't seeing. they just want the immediate fix or the immediate, like, "come and help me right now", not realizing that if we teach them how to fish it's going to be something that they can use for a long time and the skill set that they develop this year is going to… it is about building their own repertoire as far as i'm concerned, in terms of skill set. a number of challenges were described in considering the professional learning required. a lack of substitute teachers often resulted in in-service opportunities being scheduled after school hours which was not seen as ideal and often precluded the involvement of staff whose workdays were scheduled in different ways (e.g., education assistants). opportunities to collaborate with other schools for professional learning was seen as very valuable but again challenges were noted in terms of the amount of travel required in the often geographically large, rural areas typical of many regions in nova scotia and the costs of releasing teachers should substitutes become available. some participants did share examples of committees, advisory groups or informal networks that they were part of that allowed them to discuss and share plans for implementing the inclusive policy. those most senior in the system mentioned the upcoming opportunities to apply for network school projects, an initiative funded by eecd as part of the implementation of the inclusive education policy. these projects were again seen as opportunities to build collective efficacy among school-based educators and administrators to support students more effectively at the tier 1 level. the majority of the participants had not yet been made aware of this initiative. in contrast to the wide-ranging scope of the policy, the bulk of the discussion with participants, which was prompted by very open questions, centred around student services and special education needs. this is likely in part to be a reflection of the participant pool, which was heavily weighted with those who traditionally held roles in these areas and who were suggested to us as knowledgeable key informants. it is further reflective of many of those charged with leading the work of implementing inclusive education. a few participants spoke specifically about students who were african nova scotian or mi'kmaq and raised issues of equity and of making connections with families and communities. this broader issue was noted by one regional equity consultant, "in order for this to be effective, it cannot sit with student services… and certainly, the optics will be that it is still a policy that is targeting a very specific population. so, there has to be collaboration at the provincial and at the regional level in order for this to be truly realized…if you're going to bring in a team to talk about the implementation of this, it shouldn't be students service coordinators". feedback of our initial findings, which includes data shared here, as well as recommendations specific to the activities of eecd and the regional contexts was shared with the design team, and then the system at large in the fall of 2019. as is typical of a developmental evaluation, this feedback was intended to be accessible and immediately useful. a subsequent visit in the winter of 2020 began to reveal some of the ways in which these results could and were informing implementation planning. the school closures and shift in priorities resulting from the covid-19 pandemic have prompted us to re-imagine our activities and the ways in which the policy may be implemented. ongoing discussions with the eecd have interestingly raised issues of equity that have been revealed by distance and emergency learning efforts -the work of inclusive education remains critical. this paper has described the current context of inclusive education policy in canada, with the specific example of our developmental evaluation of the implementation of inclusive education reform in nova scotia. situated within the national canadian context, the example of nova scotia is both typical and unique. as with most other provinces and territories, inclusive education policy in nova scotia has broadened to include a lens of equity, and to include a focus on not only students with special education needs, but all students -particularly those most often marginalized by and within canadian school systems. the inclusion policy currently sits alongside special education policies and procedures -which is also typical of most provinces and territories. recent inclusion policies reflect both a social and educational justification for inclusive education -a need for a just and equitable society where all students have opportunities to succeed, as well as a belief that in universally designing education, we can ensure the needs of some with the benefits for all. according to ainscow (2020, p. 8) , progress in relation to inclusion and equity…requires new thinking which focuses attention on the barriers experienced by some children that lead them to become marginalized as a result of contextual factors. the implication is that overcoming such barriers is the most important means of development forms of education that are effective for all children. in this way, inclusion becomes a way of achieving the overall improvement of education systems. in assessing and informing equity and inclusion efforts, many systems are beginning to collect and examine achievement and wellbeing data. nova scotia students performed similar to the canadian average on most of the elements of the most recent pisa despite unique challenges associated with poverty. canada's performance is often lauded as reflective of equity alongside excellence. a deeper exploration at a provincial level, however, reveals that academic success and wellbeing are experienced differentially according to self-identification as african nova scotian, mi'kmaq, lgbtq2+ or with a disability. the success for all approach espoused in many of canada's equity and inclusion policies, including that of nova scotia, needs to also reflect that "all" students do not have the same opportunities and possibilities provided to them within schools and societies. the disaggregation of data that has become commonplace within nova scotia education can facilitate ongoing attention on equity issues. most provinces are not yet engaging with this type of data analysis and public sharing of results (campbell 2020) . however, while the availability of data can inform discussions and planning, it is the action -the interventions and subsequent shifts in beliefs and local and systemic practices that can lead to change in student opportunities. a focus on local application is key in exploring, and in our case evaluating, the implementation of policy (datnow et al. 2001) . the mere act of inviting and funding a 3-year developmental evaluation reflects a recognition on the part of the nova scotia government that drawing on evidence and a reflexive process throughout the implementation of this type of policy is a high priority. our presence in the province, including regular discussions with leaders within the eecd and regions, as well as teachers, principals and other school staff, serves as one thread of the implementation. we provoke sharing of understandings, beliefs and practices related to inclusive education which can be personally impactful for participants but can also lead to subsequent shifts in policy implementation as findings are shared with the eecd. in nova scotia, as in many other jurisdictions, regional differences need to be reflected in policy implementation in the same way that student diversity is increasingly being considered in classrooms and schools (smit 2005) . our initial data reveals consistency and disparities with and across regions with regards to understandings of the policy and its implementation; this is in part related to the role of the participant -whether system leader or classroom teacher -and expected due to the emergent nature of the policy at this point. it also reflects the varied readiness of each region with respect to the policy and the sensemaking engaged in by participants (snodgrass rangel et al. 2017; spillane et al. 2002) . the shift in roles and identities of educators and specialized staff as key to the implementation of this policy, and the need for responsive and collaborative supports encircling the tier 1 "common learning environment" emerged as powerful themes in our initial findings. concerns regarding these changes were related to confusion regarding process but also to identity questioning -reflecting the emotional dimension of educational change (datnow 2018; hargreaves 2005) . the professional learning that so often accompanies any discussion of inclusive education reform (cumming et al. 2018; turnbull and turnbull 2020) was also highlighted time and time again by participants -a greater and more varied skill set, and one that is shared with and across educators was seen as essential to policy implementation. existing research is replete with examples of the power of collaborative structures and communities that support educators in shifting in response to required reforms, particularly when leaders champion and model such work (ainscow 2016; butler and schnellert 2012; datnow 2018; hargreaves and o'connor 2018; schnellert et al. 2018 ). the participants we engaged with reflected more senior positions in the system and spoke of individual learning they perceived as required on the part of teachers -the need for collective learning and development was also evident in our analysis (fullan and hargreaves 2016) . examining the potential power of teachers as leaders with a voice in change (hargreaves and shirley 2019) will be important in our ongoing work in nova scotia; facilitated in part by the greater inclusion of the voices of teachers as participants and hopefully those of students and families as well. our work in nova scotia continues over the next two years and the policy is slated to be implemented province-wide in the fall of 2020. however, schools in nova scotia are planning for the school year ahead within similar constraints to those faced by education systems worldwide -school closures and social distancing requirements that may be in place for some time due to the covid-19 pandemic. the developmental evaluation will need to shift in response and be co-constructed to reflect the realities of the system while still maintaining a persistent focus on inclusive education reform. luckily the dynamic and nimble qualities and collaborative principles of developmental evaluation are well-suited to this endeavour. collaboration as a strategy for promoting equity in education: possibilities and barriers the unesco salamanca statement 25 years on: developing inclusive and equitable education systems. discussion paper prepared for the international forum on inclusion and equity in education-every learner matters promoting inclusion and equity in education: lessons from international experiences minister's review of services for students with special needs: review committee report and recommendations special education services: a manual of policies, procedures and guidelines blac report on education: redressing inequity-empowering black learners collaborative inquiry in teacher professional development. teaching and teacher education educational equity in canada: the case of ontario's strategies and actions to advance excellence and equity for students left out: challenges faced by persons with disabilities in canada's schools enhancing inclusive education through teacher education reforms time for change? the emotions of teacher collaboration and reform how context mediates policy: the implementation of single gender public schooling in california reality check: a review of key program areas in the blac report for their effectiveness in enhancing the educational opportunities and achievement of african nova scotian learners 2019 report card on child and family poverty in nova scotia: three decades lost bringing the profession back in: call to action ignoring race: a comparative analysis of education policy in british columbia and ontario raise the bar: a coherent and responsive education administrative system for nova scotia indigenous educational attainment in canada the emotions of teaching and educational change collaborative professionalism: when teaching together means learning for all leading from the middle: its nature, origins and importance inclusion of learners with exceptionalities in canadian schools: a practical handbook for teachers towards race equity in education: the schooling of black students in the greater toronto area resource teachers: a changing role in the three-block model of universal design for learning praise and criticism greet sweeping nova scotia education report. cbc news: nova scotia response to intervention in canada: definitions, the evidence base, and future directions students first: inclusive education that supports teaching, learning, and the success of all nova scotia students nstu position paper: inclusion nstu suspends job action nicat ions/the-media -room/news-relea ses?c=mcnei l-gover nment -needs -to-fully -imple ment-inclu sive-educa tion-refor ms. ns department of education and early childhood development (2020). equity and support for students spiralling out of control: lessons learned from a boy in trouble measuring up: canadian results of the oecd pisa 2018 study-the performance of canadian 15-year-olds in reading, mathematics, and science ontario's equity and inclusive education strategy ontario's education equity action plan exploring inclusion in ontario. national inclusive education month commentary #24 nova scotia assessments and examinations results for students with mi'kmaq or other indigenous ancestry and students of african descent if inclusion means everyone, why not me? recommendations for education accountability nowhere to be found a year after the glaze report tiered approaches to the education of students with learning disabilities networks for learning: effective collaboration for teacher, school and system improvement belonging in an age of exclusion teachers, local knowledge, and policy implementation: a qualitative policy-practice inquiry a descriptive analysis of instructional coaches' data use in science 5 year strategic plan teaching in inclusive classrooms: efficacy and beliefs of canadian preservice teachers policy implementation and cognition: reframing and refocusing implementation research population and dwelling count highlight tables towards equity in education rights, wrongs, and remedies for inclusive education for students with significant support needs: professional development, research, and policy reform. research and practice for persons with severe disabilities the salamanca statement and framework for action on special needs education. world conference on special needs education: access and quality a guide for ensuring inclusion and equity in education transforming our world: the 2030 agenda for sustainable development. resolution adopted by the general assembly on 25 report of the working group of experts on people of african descent on its mission to canada implementing differentiated instruction: a mixed-methods exploration of teacher beliefs and practices publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. whitley is an associate professor of inclusive education at the university of ottawa. her research and teaching are driven by a dedication to improving the school experiences of all students and teachers. her research focuses on mental health literacy, inclusive education policy, teacher preparation for inclusive education and the wellbeing of children and youth with mental health issues. she conducts research in partnership with colleagues at universities across canada and engages collaboratively with community organizations including crossroads children's mental health centre. jess is part of the canadian research centre on inclusive education and an associate of inclusive education canada, as well as the centre for research on educational and community services. in her teaching, she aims to best prepare future teachers of inclusive classrooms and further develop the skills and knowledge of graduate-level learners.trista hollweck is a part-time professor and a project director for the international arc education project, faculty of education, university of ottawa. she is a pracademic who straddles the worlds of research, policy and practice. trista is a former teacher, vice-principal and school district consultant at the western quebec school board. her doctoral dissertation examines a school board teacher induction, mentoring and coaching program and its systemic change implications. trista teaches, researches and publishes about restorative justice, professional learning and development, teacher evaluation, systemic change, mentoring and coaching, teacher induction, as well as pandemic pedagogies and professionalism. trista received her bachelor of arts from mcgill university, her post graduate certificate of education (pgce) from moray house at the university of edinburgh, her master of education from the ontario institute for studies in education (oise) of the university of toronto, and her doctorate from the university of ottawa. key: cord-322135-qbqjh6jh authors: cevikbas, mustafa; kaiser, gabriele title: flipped classroom as a reform-oriented approach to teaching mathematics date: 2020-10-07 journal: zdm doi: 10.1007/s11858-020-01191-5 sha: doc_id: 322135 cord_uid: qbqjh6jh innovative methods can change the paradigm of teaching mathematics and inspire teachers to espouse new ideas and gain new experiences. the flipped classroom (fc) is currently an innovative pedagogical approach that has high potential to transform the teaching of mathematics. in the case study described in this paper, we investigated one mathematics teacher’s transformation of teaching in two mathematics classrooms through implementing interventions based on fc methods; furthermore, we identified several key points of fc design as well as challenges and opportunities afforded by teaching mathematics in fcs. the results of the study showed that the tasks posed by the teacher, the implemented discourse, teacher feedback and scaffolding, and the teaching–learning environment were changed in fcs, although the approaches used by the teacher to analyze the tasks and students’ learning were similar to those used in non-fcs, which points out the strengths of traditional teaching approaches. the study indicates that although teaching mathematics in fcs created some difficulties for teaching, well-designed fcs offered a great opportunity to promote students’ mathematical thinking and understanding. overall, the results highlight that through fc, teachers can develop students’ mathematical potential with fcs. digital technologies have the potential to change the content of school mathematics as well as to foster the development of mathematical knowledge and understanding (heid 2005; olive et al. 2009 ). new technologies present different ways to interpret communication, collaboration, and social interaction (beatty and geiger 2009), and encourage a strong connection between mathematical knowledge and practice (olive et al. 2009 ). educators have attempted to create and familiarize themselves with technological artefacts to enhance mathematics education (lagrange and kynigos 2014) . while these attempts have the power to change classrooms, research into how this change can be accomplished and its actual implementation frequently lag behind the speed of the digital evolution (goos et al. 2020) . the rapid dissemination of technology use in society has not reached education fully, at least before covid-19 technology had still a rather marginal status in mathematics teaching and learning (lavicza 2010) . hence, there is an increasing desire among mathematics teachers to use technology-enriched teaching approaches, consider the use of technology as part of their teaching activities, access resources, and share knowledge and experience with their colleagues (hooks 2015; joubert et al. 2020) . employing technology-supported reform-oriented approaches in mathematics teaching has become a necessity for the twenty-first century, even before the changes forced by the pandemic covid-19 came into the public discussion. we can define the term of reform as "change or changes made to a system or organization in order to improve it" (longman 2020) . the flipped classroom (fc) is one of the reform-oriented approaches that can accelerate digital transformation in teaching mathematics, contribute to technology integration into mathematics education, and engage students in mathematics. fc is described as a teaching reform (he 2020 ) that changes teaching ideas, objectives of teaching, teaching time, and teaching mode with the help of technology (jian 2020) in such a way that within fc instructors teach students in-class, and learners do homework by themselves at home (he 2020) . fc is defined on first access as "school work at home and homework at school" (bergmann and sams 2012) . lecture videos, notes, slides, and articles are shared with students outside of the classroom, and teachers have an opportunity to communicate and interact with their students by means of online learning management systems (lmss), and then support deep learning through faceto-face classroom activities. flipping the teaching provides more time for active learning and problem-solving activities in the classroom (lo and hew 2017) . mcgivney-burelle and xue (2013) describe the main principles of fc as follows: (1) out-of-class time should be well-structured to prepare learners for class hours (2) teachers assess learners' preclass activities (3) class time consists of collaborative and cooperative problem-solving activities and discussions, and (4) well-structured and highly organized implementation of a learning environment is based on scaffolding and feedback by the teacher. in essence, the fc approach has the potential to improve mathematics teaching and learning and use technology to accomplish these goals. teachers should use technology strategically in such a way that all the students can access mathematics (national council of teachers of mathematics 2011). fcs can provide this strategic use of technology and have the potential to improve mathematics teaching. however, it appears that currently only a limited number of studies have examined the teaching experience of (secondary) mathematics teachers and their roles within the fc framework (fredriksen 2020) . the vast majority of studies in the fc context focus on students and their learning rather than teachers and teaching in (secondary) mathematics classrooms. in this study, we analyze the change of mathematics classrooms by fc from the perspective of the teacher. in addition, we identify the pros and cons of teaching mathematics in fcs. overall, the study addresses the following research questions from the teachers' perspective: 1. how does the flipped classroom transform mathematical teaching approaches? 2. what are the challenges and opportunities provided by teaching mathematics in the flipped classrooms? different names for fc pedagogy have been proposed so far, e.g., inverted classroom (lage et al. 2000) , classroom flip (baker 2000) , flipped classroom (bergmann and sams 2012) . furthermore, fc has been defined in different ways, e.g., by lage et al. (2000) as "inverting the classroom means that events that have traditionally taken place inside the classroom now take place outside the classroom and vice versa" (p. 32), although other definitions point out that fc is going beyond this approach. bishop and verleger (2013) offered a definition for fc that is composed of two parts, namely, interactive group work in the classroom, and computer-based instruction out of the classroom, including lecture videos. one of the more current and comprehensive definitions of fc has been developed by the flipped learning network (fln). according to this definition: fc is a pedagogical approach in which direct instruction moves from the group learning space to the individual learning space, and the resulting group space is transformed into a dynamic, interactive learning environment where the educator guides students as they apply concepts and engage creatively in the subject matter. (fln 2014) . a group of educators from the fln suggested that teachers should incorporate four pillars (f-l-i-p) into their teaching practice, creating the widely used acronym flip: (1) f: flexible environment: teachers should rearrange the teaching-learning environment to adapt to each lesson or unit to encourage either independent or group work. these newly created environments empower students to select the desired learning time and place. (2) l: learning culture: in fc, class time should be devoted to inquiring, learning subjects and contentspecific concepts more deeply, and generating learning opportunities. teachers in fc make use of scaffoldings to enable their students to find out specific topics by thorough implementation of student-centered approaches in the zone of their proximal development (vygotsky 1978) . (3) i: intentional content: teachers should constantly consider how they could benefit from the fc approach to support learners in enhancing deeper understanding. teachers need to decide on the content to be taught and materials the students should explore. (4) p: professional educator: although the visibility of the professional educator's role is less obvious in fcs, teachers in this environment are much more important and frequently more challenged than in traditional classrooms. they should consistently observe students, provide support, give comprehensive feedback, and assess students' work. although fc has been defined in different ways, there is a consensus that fc is a student-centered pedagogy, giving teachers more time for implementing active learning activities, enabling social interaction and collaboration, creating technology-rich environments in accordance with differentiated learning, and presenting opportunities for students to move through the zone of proximal development (for details see cevikbas and argün 2017) . in fc, the learning and teaching process is not confined to the classroom; students can progress at their own pace in an interactive way both in and out of the classroom (davies et al. 2013) , and teachers can provide effective guidance for students rather than deliver information directly. in this study, the fc definition offered by the fln (2014) was used, taking into account that lecturing videos are a crucial part of the fcs. the current generation of students has quite different characteristics, expectations, and dispositions compared to students a few decades ago (cevikbas and argün 2017) . nowadays, students prefer to access information quickly, and, especially by using various digital technology channels, they desire to construct their own knowledge by enjoying themselves (cevikbas 2018; engelbrecht et al. 2020) . teaching methods and learning environments have to be adapted to respond to these changes. fcs provide students with a tailored learning environment by inverting traditional teaching approaches with the help of technology. in this way, fcs help to create high-quality mathematics teaching activities (chen and wen 2019) and can develop students' learning opportunities in mathematics. fcs encourage students to enhance their critical thinking abilities, assist in clarifying the goals of learning collaboratively, and think about mathematics problems before participating in classroom activities (mazur et al. 2015; voigt et al. 2020) . teachers in fcs gain additional time to apply inquiry-based activities, problem-solving activities, hands-on activities and comprehensive analysis in their classrooms (schmidt and ralph 2016) . they can spend class hours creatively and strategically and can interpret students' mathematical thinking (fulton 2012) . the fc approach transforms mathematics classrooms into laboratories of inquiry, analytical thinking, and connectedness with other fields of stem (bergmann and sams 2012) . while fcs benefit students by encouraging them to engage in mathematics from behavioral, emotional, and cognitive perspectives (cevikbas and argün 2017), they also benefit teachers by helping them to improve their professional competencies and transforming classroom dynamics. radical changes in the teaching experiences generate persuasive ideas and authentic vision (brown 2018) . due to the nature of fc, even if advanced use of technology is not required, teachers need at least basic competencies in using technology in mathematics teaching for implementing the fc approach. this requirement allows them to learn new technologies and new teaching strategies (brown 2018) . teachers have an active role in fcs and can provide guidance and scaffolding to their students when they need professional support (cevikbas and argün 2017; fln 2014) . they can also follow students' learning progress and offer timely and comprehensive feedback. fcs increase teacher-student interactions (bergmann and sams 2012; brown 2018; cevikbas and argün 2017; lo and hew 2017) and alleviate disciplinary problems in the classroom (cockrum 2014). they also change classroom management and make classrooms more transparent; that is fcs make it possible for parents to follow teachers and students activities (bergmann and sams 2012). despite the numerous advantages of fcs in mathematics teaching, difficulties that teachers may encounter are reported in empirical studies. we can summarize the difficulties of flipped teaching along three basic categories: (1) paradigm shift, (2) content, and (3) technical requirements. the first difficulty refers to changing the pedagogical paradigm of teaching and learning mathematics (cevikbas and argün 2017; lo and hew 2017) . students' and teachers' beliefs and perceptions can create paradigmatic barriers to flipped teaching and learning. there is a potential risk in fcs that students may skip the out-of-class tasks and come to the classroom without watching the lecture videos due to lack of independent learning responsibility. another difficulty is related to creating subject-specific content. teachers need to have well-prepared lecture videos, notes, slides, and teaching materials to implement effective flipped teaching (chen 2016; lo and hew 2017) . in particular, it is difficult to find customized lecture videos or another type of content that effectively meets teachers' and students' needs and expectations. although there are plenty of videos accessible on online platforms such as youtube, teacher tube, khan academy, and so forth, existing videos do not cover or match all topics taught in school mathematics (chen 2016) . in this case, teachers have to create their own lecture videos, which is an extremely time-consuming task. another difficulty for fc implementation is to have the technical requirements to teach and learn mathematics. the technical problems with accessing the internet and mobile devices cannot be underestimated and may destroy the structure of fcs. fc practices do not work well without the internet, and teachers should be able to use technology strategically in the process of teaching mathematics. organizing and operating the resources, tasks, students, and knowledge simultaneously can create complicated problems for teachers who need to improve their technology use competencies (trigueros et al. 2020 ). in addition, while flipping the classes, teachers may have to deal with some additional challenges, such as dissatisfaction, unwillingness, and bias against fc (bagley 2020; chen 2016 ). many educators are still stuck with traditional teaching approaches, which are rooted within the paradigm of transfer of knowledge by lectures, notes, and presentations. these approaches simply concentrate on knowledge memorizing and do not encourage critical thinking, problem solving, collaboration, engagement, and social interaction (marzouki et al. 2017) . contemporary educational approaches promote interactive teaching and learning environments based on technology. fc pedagogy is one of the most adapted innovative approaches for creating interactive teaching and learning environments, as it has the potential to enhance problem solving, collaboration, engagement, social interaction, and communication (bergmann and sams 2012; cevikbas and argün 2017) . fc can improve teachers' differentiated teaching experiences and allow them to guide and support their students in and out of the classroom. overall, fc offers a relatively new teaching approach strongly connected to vygotsky's (1978) ideas based on social constructivist theory (ahmed 2016; jarvis et al 2014). from the social constructivist perspective, knowledge and meaning are socially constructed by interaction and higher-order cognitive teaching-learning activities in fc. learning is described as the creation of an environment in which students are active in constructing their own knowledge (schreiber and valle 2013) . according to the vygotskian approach a good teacher will create an interactive and useful environment by fostering discovery and socialization (kim 2001; schreiber and valle 2013) . vygotsky (1978) introduced the concept of zone of proximal development (zpd) pointing out that learners can proceed to the next zone of their proximal development with the help of more knowledgeable individuals. based on the theoretical framework of social constructivism, teachers are expected to perform the following tasks in fcs (bergmann and sams 2012; cevikbas 2018): (1) design an interactive classroom environment (including virtual classrooms) and help students to prepare for class hours by use of videos and online resources; (2) support students to construct knowledge and meaning by providing scaffolding and sufficient feedback; (3) promote the agency of students to think, inquire, communicate, interact, and discuss; (4) design and implement activities that foster active learning for their students; and (5) use dynamic assessment approaches. when used as an educational tool, technology contributes to the development of social constructivist meaning by encouraging social interaction, communication, discussion, problem solving, engagement and collaboration. therefore, social constructivism is one of the most adopted theories for technologically rich environments (marzouki et al. 2017 ) and aligns with the benefits of fc pedagogy (bishop and verleger 2013; jarvis et al 2014) . accordingly, we have embedded our fc research study within a social constructivist framework, which allows us to examine the changes of the mathematics teaching with fc interventions, determine the opportunities and challenges of flipped mathematics teaching, and construct an effective design of a fc for mathematics teaching. in this study, we employed qualitative research methods to investigate how mathematical teaching was changed by teacher's fc interventions and explore opportunities and challenges inherent in flipped teaching. qualitative research, especially case studies, are appropriate for the investigation of people's knowledge, views, and experiences (merriam and tisdell 2016) . case studies focus on a bounded system by collecting data through multiple sources of information such as interviews, observations, audio-visual media, and documents (creswell 2013) . the most explicit examples of bounded systems are a single person, a group, an institution, or a subject (merriam and tisdell 2016) . this study was designed as a qualitative case study, which is particularly appropriate for exploratory studies aiming for insight into flipped teaching experience and the design of fc, as well as the pros and cons of fcs in teaching mathematics. our bounded system was a single mathematics teacher and we focused on her teaching experiences in two separate secondary classrooms. the voluntary participants of the study were a mathematics teacher and 68 high school students at a public school in turkey. the school was located in a neighborhood where families with medium socio-economic background resided. the participating teacher was selected based on volunteering principles and is named in this study ece (pseudonym). ece had 6 years of professional experience as a mathematics teacher and at the time of study was continuing her doctoral studies in the field of mathematics education in turkey. although she was familiar with student-centered teaching methods from her undergraduate and graduate education, she mostly applied direct instruction methods in her classes and the students were used to learning mathematics through a teacher-centered approach. ece taught mathematics in three separate secondary classes (9th grade, 10th grade, and 11th grade) for a total of 18 h per week. in this study, we focus on her 10th grade and 11th grade classes. using basic digital technologies and having internet-based devices and stable internet connection at home were required for all students as well as the teacher, in order to conduct an effective fc implementation. the main data sources of this research study were classroom observations, video and audio recordings, semi-structured teacher and student interviews, and questionnaires. a significant part of case studies is the use of multiple data sources to prevent systematic faults and to resolve inconsistencies (maxwell 2013) . in this study, ece's teaching experience can be divided into two parts: non-flipped teaching and flipped teaching. in the first part, ece taught mathematics for 4 weeks in the non-fc settings and we (more precisely, the first author of the paper) observed her in two non-fcs (10th grade and 11th grade) and recorded her teaching experiences and classroom activities via audio and video recorders. classroom activities and ece's teaching experiences in non-fcs were recorded with a video camera placed in the back corner of the classroom. in addition, ece's talks were recorded by a voice-recorder placed on the teacher's desk, since ece was generally close to the teacher desk while she was lecturing in non-fcs. in the second part, ece flipped her mathematics classrooms for 4 weeks. she was observed in two fcs and her teaching experiences were recorded with a video camera. in fcs, ece led group work, and walked around the classroom to guide students and follow their work most of the class time. in order to record the conversations between ece and the students, voice-recorders were placed on each group's desks. after the 4-week fc experience, we gave 68 students a questionnaire consisting of 15 open-ended questions asking about their views on the teacher's teaching approach in fcs and non-fcs. then, we performed semistructured interviews, each lasting about an hour, with the teacher and 13 students. for the selection of the students to be interviewed, the teacher's opinions, students' mathematics scores, and results in the questionnaires were used. we gathered the data from interviews about teaching approaches in fcs and non-fcs, opportunities and challenges of the fc approach, and the design of a fc. figure 1 shows the data collection sequence of the study. first, the participating teacher was offered individual professional development concerning the fc approach. this professional development consisted of papers and teaching material to enlarge her knowledge about fc design and implementation. then, with the support of the first author, the teacher developed the fc design and created the related content. lecture videos and notes, worksheets, and classroom activities related to polynomials and logarithms were prepared using online sources and audiovisual or textbased materials. three independent mathematics educators evaluated the appropriateness of the teaching materials that were prepared for the fc implementations. a single virtual classroom for the pilot study and two virtual classrooms for the main study were created using the edmodo lms (https ://new.edmod o.com/). the pilot study was conducted with the same teacher and 10th-grade students for 2 weeks based on two lecture videos shared on the learning platform. we observed the teacher's performance in the pilot study and made some adjustments regarding the videos and the planned classroom activities. additionally, the teacher was able to gain some teaching experience with flipped teaching before the main study. in the pilot study, we realized that the teacher had difficulties in managing the group work due to the crowded classroom and the high number of students in the class. we also realized that lecture videos should not be shared too early (for example, a week before) or too late (for example, a day before) before having the associated meeting in the classroom. based on the teacher's and the students' feedback, the appropriate length of the lecture videos was determined to be between 10 and 20 min and should include examples, problems, solutions, prompt questions, and, of course, lecturing. explanations about the basic structures of fc design adopted in this research are summarized in table 1 . after the pilot study, observations were conducted lasting 4 weeks (24 course-hours with each course-hour lasting 40 min) in traditionally organized classes (10th and 11th grades) where the teacher taught mathematics. the lessons were recorded by video camera in order to evaluate the teacher's usual teaching approach and the usual teaching atmosphere within the traditional teaching environment. immediately afterwards, fc was implemented for 4 weeks (24 course-hours) in both of the classrooms. six lecture videos for the 10th-grade students and four lecture videos for the 11th-grade students were uploaded to the learning platform at least 2 days before the associated lessons. fc activities were applied in accordance with the design described in table 1 . in-class activities were observed and recorded by video cameras and voice-recorders. teacher and students were interviewed and questionnaires were applied in the fourth week of the fc implementations. in this case study, our bounded system consisted of a single mathematics teacher and we focused on her teaching experiences in both fcs and non-fcs. in the data analysis process, firstly we watched the 48 video recordings and listened to the 26 audio recordings (interviews and classroom recordings) one more time and transcribed them verbatim. then, we read the transcribed data from video-audio recordings, the completed questionnaires consisting of open-ended questions about the teaching approach of the teacher, and observation notes several times. then, we encoded the data through the content analysis method (miles and huberman 1994) and identified four categories for the first research question: (1) environment, (2) interaction, (3) feedback and scaffolding and, (4) assessment. we determined differences between teaching in flipped and non-flipped classrooms in terms of these themes that are important in teaching based on social constructivism (marzouki et al. 2017; palincsar 1998; vygotsky 1978) . in social constructivism, knowledge is constructed by means of social interaction, communication and environment, in other words learning should not be considered in isolation from the environment and socialization (kim 2001) . learners can proceed in their zpd with scaffolding and specific supportive types of feedback given by teachers (cevikbas and argün 2017) . thurlings et al. (2013) conducted a review study and identified timing, characteristics and effects of fc as characteristic features based on social constructivism as follows: (a) timing (immediately, when students are reminded of their actions/ when still relevant), (b) characteristics (focused on/related to task, related to goal and student's perceptions of their performance; includes information about progress, targets at a suitable level, gives opportunity to respond, dialogue/ context of collaboration, encourages positive motivational beliefs and builds empathy), and (c) expected effects (supports learner to engage in action to close gap, engages students in thinking) of feedback in social constructivist theory. table 2 indicates some examples of the coding related to the feedback provided. according to the social constructivist approach, scaffolding is a temporary support provided by an expert who helps a learner understand how to perform a similar task that s/he may encounter in the future (wood et al. 1976 ) and assessment is referred to as dynamic assessment that provides a future measure of performance, signifying capacity. this capacity provides information about how the learner will perform individually in the future (palincsar 1998) . we employed content analysis (miles and huberman 1994) to ascertain challenges and opportunities inherent in the flipped mathematics teaching. finally, two themes (1) challenge, and (2) opportunity for the second research question could be identified. since we did not aim to compare the results related to mathematics teaching experience between the 10th grade and the 11th grade students, we analyzed the data collected from both groups together. after coding, code frequencies and quotations backing the results were identified. we used a code-recode strategy in which we recoded 30% of the research data after an interval of 6 weeks and achieved a consistency rate of 99% between these two encodings. furthermore, an additional mathematics educator carried out double coding of 10% of the data. the rate of intercoder reliability was computed using miles and huberman's (1994) formula and was determined to be 92%. we examined our codes and worked on them until we achieved total reliability. we shared our pre-results with the teacher, and she confirmed our results. the member check strategy allowed us to reduce some misinterpretations, errors, and prejudices (maxwell 2013) . for this study, the results of the transformation that fc interventions created in mathematics teaching are presented in four groups; teaching and learning environments, interaction and communication, teacher feedback and scaffolding, and assessment strategies. before the intervention, ece had taught in non-fcs by standing in front of the blackboard or sitting at the teacher desk. she used the direct instruction technique and asked her students to write down what she had explained or written on the blackboard. although there was a smartboard in the classroom, ece preferred to use the blackboard in her lessons. the classroom observations of traditional teaching by ece revealed that she did not draw on technology or concrete materials in order to enhance the understanding of the students. ece lectured based on only one course book and did not tailor her teaching to the knowledge, understanding, and experiences of the students; in contrast, she treated them as passive listeners. ece did not assign her students tasks based on mathematical activities, and she did not conduct problem-solving activities in her traditional teaching. the student desks were located in a way that did not allow for group work, which had not been carried out before in the classes involved. we could observe in the non-fcs that while ece was lecturing, the students from the back rows were engaging in some disruptive behaviors (playing games on smartphones, eating sunflower seeds, talking to peers, listening to music, etc.). it became obvious that the structure of the classes (being crowded, desks arranged in rows) and the teaching approach that ece had adopted did not provide opportunities for active learning. ece explained the change in mathematics classes through fc as follows: ece in fcs, ece created an interactive learning environment for students out of school in virtual classes based on the materials and the communication activities from the learning platform. in addition to audiovisual resources (videos, diagrams, graphics, tables), alternative text-based documents were shared with the students in these virtual classrooms. ece gave her students ambitious pre-class tasks (watching videos, taking notes from the videos, questioning and answering on edmodo, etc.) that should enhance their mathematical understanding. in this way, she could gain insight about the students' levels of knowledge and understanding about the new topics based on their edmodo sharings, and could dwell more on the topics in which students had expressed that they had difficulty, before or within the classroom activities. furthermore, group work as well as individual work was regularly conducted in every lesson in fcs, and the students performed collaborative group work in four-person round-table sessions. it could be observed that ece had difficulties in guiding the group work due to the crowded classes and a chaotic and noisy atmosphere was evident in the classroom from time to time. ece started to use the smartboard and geogebra in fcs. for instance, she asked students to make drawings related to the diagrams of logarithmic functions and helped them to comprehend the relationship between exponential function and logarithmic function (see fig. 3 ). ece created a flexible environment in the class, in which students could put questions to her whenever they wanted; she trained her students through problemsolving activities and encouraged them to produce new ideas and new problem-solving strategies. even if the solutions and ideas that students suggested were mathematically incorrect, she listened to them in the lesson and enabled them to recognize their errors by asking questions. the students, who experienced that the teacher did not judge their ideas negatively, felt in themselves more comfortable, and their belief in their ability to learn mathematics was strengthened. a student described this change as follows: student 1: i did not take the floor in mathematics classes before. i sat in the back row. if i had already wanted to talk, there would not have been such an environment to do it. but in fc, everyone is engaged in mathematical activities. there was a comfortable and interactive learning environment in the class, which pleased me. i did not feel myself alone while learning mathematics. i realized that i could learn mathematics better. [transcript from student interviews translated by first author]. in her traditional teaching, ece used to give a few examples, after explaining the topics at the blackboard and asked the students whether they had comprehended the examples, yet she did not create any opportunities in which students could make mathematical inquiries, and she did not include discussions within group work in her lessons. ece did not allocate time for the students to think aloud on the mathematics problems and ask their questions; she shared the mathematical information directly with the students in class. the students usually did not have the chance to express their own ideas in the course of working on the mathematical problems, and they had difficulties in communicating with the teacher and peers. additionally, there was no communication between ece and the students outside of the lessons and students could not interact with ece or peers about learning mathematics outside of the school. in fcs, ece and her students had a chance to communicate and interact outside of the classroom through edmodo. she primarily encouraged her students to import their lesson-related content into the learning platform edmodo and ask their questions there. she replied to those questions on the platform and communicated with students out of school as well. she noted down the questions that students asked on edmodo and guided classroom discussions around these questions and a few prompt questions (examples: what is the difference between a polynomial and a function? do they mean the same thing? what is the degree of zero polynomial? can all polynomials be factored?). ece wanted the students to develop hypotheses while solving problems and to justify their solutions using both oral and written mathematical expressions. a sample problem included in the problem-solving activities and a dialogue in one of the working groups in fcs is displayed in fig. 2 . the students worked in the groups collaboratively, generated ideas to solve the problems, shared their ideas with their peers, asked questions of each other and developed mathematical argumentations. as can be seen in fig. 2 , some of the ideas shared by the students were accepted (lines 23-24; 29-30) , but some of them were refuted by other group members (lines 14-15; 21-22-23) . as can be seen, students communicated well and solved the problem in an interactive way. ece followed the students closely in the collaborative group work, listened to their ideas carefully, and guided them. ece used technology in mathematics teaching extensively (virtual classrooms, videos, learning platform edmodo, mathematical software geogebra, whiteboard, calculators, etc.) and gave students the chance to embody their ideas by engaging them in mathematical activities. figure 3 shows a section from the activities related to logarithms that ece performed through geogebra (screenshot was taken from video recordings), and in fig. 4 (the photo was taken in group work on polynomials), a section from the students' learning activities related to polynomials in the fcs is displayed. based on the classroom observations and video recordings, it could be reconstructed that ece used direct instruction in non-fcs in the first 25-30 min of the lessons, and in the last 10-15 min, she gave algorithmic tasks to students consisting of 4-5 mathematics questions on the blackboard and then asked students to work on them. these tasks were not suitable for developing students' mathematical thinking, but served as practice of topics and themes covered in the first part of the lesson (example: given that p(x) = x 3 − 7x + 5 and q(x) = x 4 + 15x − 8, find p(x) + q(x). this is an example used by the teacher related to adding polynomials). it could be observed that neither feedback nor scaffolding was given as there was no interaction between ece and her students in the first 25-30 min of the lessons. in the last 10-15 min of the lessons, the practice part, ece gave the students tasks and provided feedback 4-5 times in a lesson according to student answers. since there was not enough time available to examine all students' solutions, she asked a few students to solve the tasks on the blackboard and gave feedback at the end of the lesson. this type of feedback was of corrective and confirmative nature. when students made mistakes in working on the task, ece preferred to solve the task herself instead of providing scaffolding to the students. a dialogue from the interview with ece displays the characteristic aspects of this teaching method. researcher: i have observed that you usually write 4-5 mathematics questions on the blackboard in the last part of your lessons. how do you determine whether students understand the topic with them? ece: i cannot examine all students' solutions because time is limited. i ask him or her to pick one of them and solve the problems. if there are students who have made a mistake, i make them see the correct solution on the blackboard. researcher: do you give feedback or scaffolding to your students? ece: if the solution is correct, i say "correct". if it is wrong, i show why it is wrong. researcher: do you give feedback or provide scaffolding to all your students in the classroom at any time? ece: actually, my feedback is limited. it is not possible for me to determine the needs of the students in the classroom and provide them with scaffolding. from time to time, i give homework to students to make up for their shortcomings. in fcs, ece provided group work in each lesson, gave worksheets to students, and guided their work and problem solving activities closely. students worked on mathematical tasks for 20-25 min in the classroom and attended 5-10 min of question-and-answer sessions. ece gave individualized feedback to her students, including explanations, hints, encouragement, support, inquiry, confirmation, and correction. in each lesson, ece followed the students in their group work and gave immediate feedback to them, whenever it was needed. the difference in the quantity of feedback given by ece in the two kinds of teaching approaches is a strong indicator of the changes in her teaching method: in a non-fc lesson, she gave feedback 4-5 times, compared to 45-55 times in a fc lesson. ece did not provide scaffolding at all in non-fcs, she provided it 7-8 times in fcs. students used a flexible and an interactive learning environment in fcs: they immediately contacted ece when they needed scaffolding or feedback. in this way, ece was able to allocate more time to the students who needed more support. 88% of the students stated in the students' questionnaire that they had received more feedback and scaffolding in fcs and they had understood the topics better due to the teacher feedback and scaffolding. in fcs ece also gave feedback to her students outside of school with the help of edmodo. students who had watched the videos could ask questions of ece in virtual classrooms, and ece provided corrective, confirmative, and explanatory feedback to help her students. table 3 shows the characteristics of scaffolding and feedback given by ece in non-fcs and in fcs. for non-fcs, ece stated that she had graded her students according to two separate mathematics examinations held in one semester; in other words, she applied static assessment methods. examination scores were regarded as the only achievement criteria, and the students who attained an average of 50% and higher in these scores were accepted to receive the passing grade in the course. ece did not take into account the students' mathematical learning processes apart from these scores. the parents were informed of their children's academic standing in parent-teacher meetings generally held once per semester. in fcs, ece observed her students' learning activities more closely, provided them with personal feedback and scaffolding, allowed them to speak in the lesson, listened to their ideas, and gave them two additional quizzes that comprised 20 multiple-choice questions on the topics covered in the lecture videos, in addition to the compulsory examinations. however, she did not change her grading style and determined the students' mathematics achievement solely according to the marks in the compulsory exams as in non-fcs. different from non-fcs, in fcs, parents could communicate with the teacher directly and receive immediate feedback about the students' academic standing as they could log in to the learning platform edmodo with a parent account. the reason why ece did not apply a dynamic assessment approach in fcs was examined within the interview with ece. she expressed her concerns that the changing of her teaching approach was already hazardous for her and that changing the assessment method at the same time might have caused unpredictable strong reactions from students and parents. as she wished to avoid community pressure (from students, parents, and school administration) she did not consider alternative dynamic assessment methods. she added that, while mathematics educators at universities were flexible in their courses, teachers of k-12 levels were under pressure from the triad: parents, students, and administrators. apparently, ece felt intimidated by the probable reactions and indeed could not completely implement the innovations she had planned into action as became clear in the following statement in the interview with her: ece: i underwent some training on active teaching methods, but the reality in practice is not as the one in theory. power at the k-12 level in turkey is in the hand of parents, students, and administrators instead of teachers, unfortunately. they sometimes attempt to interfere in even what and how you are lecturing in the class. therefore, lots of teachers who have begun the job as idealists may become confined to traditional approaches. [transcript from teacher interview, translated by first author]. the results of the study point out that fcs provide challenges and opportunities for teaching in mathematics education. with reference to the challenges of fcs, ece, who underwent a change in her teaching routine, stated that she felt uneasy and incompetent at the beginning of the fc implementation. she thought that if she was not successful in teaching using fcs, the good teacher impression that she had made on the students would be affected negatively, and thus she felt nervous. she also indicated that designing the fcs and producing content took lots of time and was demanding and that she needed the knowledge of how to use certain software in order to create high-quality lecture videos and edit them properly. she found it boring and exhausting to be constantly checking whether the students had watched the lecture videos and complained that the time she allocated for her social life got restricted when she kept in touch with all the students outside of school and needed to follow their development after class-time as well. since the classrooms were crowded, it could be observed that sometimes a chaotic (noisy) atmosphere prevailed in the classes, and ece was challenged in monitoring the students and their learning processes. it became a difficult task for ece to involve two students who refused the change of the teaching approach. ece stressed that she did not consider teaching with fc techniques in the future because of all these difficulties. fcs provided some opportunities as well as challenges in terms of the teacher. in this vein, ece gained experience in fcs and expanded her knowledge on innovative teaching approaches. she reflected that her job satisfaction increased as she made a great effort in teaching mathematics in the fcs. ece had allocated enough time in the fcs for active learning activities that she could not implement in non-fcs. she also accomplished the integration of technology in teaching mathematics in the fcs by use of the internet, edmodo, geogebra, lecture videos, smartboard, calculators, mobile devices, and online resources. ece was able to identify missed learning progress by closely monitoring students' learning activities in the fcs; she could give them feedback and scaffolding in a timely way. because of fc, ece increased the level of communication and interaction with her students and discovered more information about their mathematical learning activities outside the classroom. the results of this case study are limited to the fact that only one teacher's teaching experiences were analyzed, although the study comprised activities in two fcs and two non-fcs covering 8 weeks. the teacher was a well-educated (doctoral candidate) mathematics teacher with 6 years of professional experience; however, her fc experiences were limited to 2 weeks of practice in the pilot study. these limitations have to be taken into account when interpreting the results. the main results of this case study highlighted that fc has the potential to transform mathematics teaching fundamentally, as the teacher changed her teaching in secondary mathematics classrooms quite considerably. the central elements of teaching-teaching and learning environments, interaction mode, feedback and scaffolding-provided by the teacher underwent a radical change through fc in accordance with the perspectives of social constructivism (vygotsky 1978) . however, the teacher did not achieve a shift in her assessment methods, although she applied flipped teaching strategies in her classes. one reason the teacher gave was the fear that a failed fc implementation might produce an unfavorable reaction from the triad of students, parents, and school administrators. these concerns and emphasis point out that-at least in countries with a strong influence of educational administration-teachers are not free in the design of their teaching. overall, this aspect highlights that there are paradigmatic obstacles in the transition from traditional teaching methods to innovative ones such as fc; amongst others, an obstacle is the need for change in the existing values and beliefs of stakeholders of education (melville et al. 2013) . in fcs, the teacher managed to create an interactive and flexible teaching and learning environment, with students being engaged in mathematical activities in an interactive way, starting from pre-class activities. students had plenty of work in fcs, such as watching the lecture videos, making sense of the videos, participating in q&a activities in edmodo, engaging in problem-solving activities and discussion sessions, and working collaboratively as well as independently on mathematical problems. the teacher managed to engage students in discourses on mathematical tasks and learning activities in fcs and encouraged them to communicate effectively. these attempts motivated students to think aloud about mathematical problems and tasks and to discuss their ideas in a relaxed atmosphere. their mathematical communication skills improved, and they interacted with the teacher and their peers while working in groups. the flexible and interactive environment offered by fcs made students more interested in mathematics and contributed to their problem-solving skills. to summarize, these results showed that in line with the constructivist teaching-andlearning approach, the teacher managed to encourage students to be more active and engaged in fcs and enabled the students to foster their own construction of knowledge in social interaction and collaboration. students could develop their own mathematical thinking by interacting with their teachers and by collaboration in classrooms (kim 2001; vygotsky 1978) . the teacher exploited technology (smartboard, calculators, geogebra, edmodo, online sources, videos, internet) as a tool in the physical classroom as well as in the virtual classroom, in order to allow a differentiated teaching approach and an interactive teaching environment. in fcs, the teacher was able to follow the students' work closely, to provide immediate feedback on students' learning performance, and to support students' mathematical understanding with scaffoldings. this approach is in line with the social constructivist theory, where the role of the teacher is guiding students' work and assisting students to develop their own solutions, not prescribing the solution (schreiber and valle 2013) ; good learning occurs in zpd (vygotsky 1978 ) by means of scaffolding. in principle fcs allow teachers to follow students' learning activities and observe their reactions; thus, teachers can better understand and meet students' needs (bergmann and sams 2012; cevikbas and argün 2017; chen and wen 2019). through the online learning platform used, the teacher learned more about her students' out-of-class learning activities as well as in-class activities and customized her scaffolding and feedback strategies. teacher feedback provides students with insight into their learning performance (cevikbas and argün 2016) and scaffolding offers students opportunities for their cognitive development. the teacher invested a high amount of work in order to improve students' mathematical capacity by feedback and scaffolding in fc implementations. these efforts also strengthened her job satisfaction and professional development. although she closely followed the students' learning activities, she assessed students through a static assessment method. it was assumed that using this type of assessment method in fcs might have negative effects on students' engagement in mathematical activities; thus, dynamic assessment approaches were considered to be more appropriate to evaluate fc activities. in a vygotskian perspective, traditional static assessment strategies portray the student's level of actual development, and dynamic assessment strategies uncover the student's potential level of development (palincsar 1998) . from this perspective, it can be stated that the teacher failed to evaluate students' potential development levels. we can consider that in this specific case, changing the assessment strategy was more difficult than changing the teaching method. more emphasis should be placed on this issue in future studies. at a technical level, we identified that the length of the lecture videos in flipped secondary mathematics classrooms can be between 10 and 20 min, based on a pilot study and a teacher's teaching experiences in fcs. the ideal length of lecture videos can vary, depending on different age groups. some researchers recommended that the length of videos be less than 15 min (bergmann and sams 2012) or about 10 min (chen and wen 2019). videos should incorporate prompt questions, examples with solutions, and lecturing, all of which contribute to students' mathematical thinking and development of mathematical meaning. the students preferred that the instructor in the videos be their own teacher, not any other. furthermore, employing videos shared on different online platforms or social media channels that are not reviewed by any mathematics educator can cause problems concerning the validity of the content. additionally, the adequate relation between the topic shared in the lecture videos and in-class activities was the critical point for the basic fc design. in this study, we found that fcs presented challenges as well as opportunities in teaching mathematics. the uncertainty of teaching in fcs for the first time made our teacher feel uncomfortable. she mentioned that it was troublesome and time-consuming to produce the content in the fcs. the study of lo and hew (2017) reported similar results and pointed out that the crucial issues in fcs included teachers' heavy workload in creating content. the teacher also had difficulties in guiding the group work and wasted a lot of energy due to the large class size. as the constant contact with students outside the classroom produced a high amount of work, it is recommended that teachers and students should determine a certain period for out-of-school communication and interactive discussions on learning platforms in order to maintain a work-life balance. although there were several opportunities for teaching mathematics in fcs, our teacher stated that she would not consider teaching mathematics in fcs in the future because of the difficulties mentioned above. overall, fc has the potential to change the paradigm of teaching mathematics and inspire teachers to generate new ideas and gain new educational experiences. however, it is necessary for teachers to consider whether they and their students are ready for fcs and whether they have the families' and administrators' support. although teaching mathematics in fc has some difficulties, well-designed fcs presents a great opportunity to boost students' mathematical thinking and understanding using differentiated teaching in interactive environments. to conclude, teachers who are mentally and technically ready to transform mathematics classrooms into fcs can strongly develop students' mathematical capacity in fcs. the results of the research revealed that fc is not a panacea (lo and hew 2017) , but it can contribute to mathematics education. further research should focus on the impact of teacher characteristics on the success of the introduction of fcs, such as professional experience, teaching experience in fcs, competence in using kinds of required technology, and educational background. teachers who are more experienced with technology in teaching than the teacher in our study, may give their students more effective feedback, may provide stronger scaffolding and more differentiation in their teaching. additionally, future studies may shed light on the role of technology in teaching mathematics in fcs. in this study, our teacher used technology as a tool to make her teaching differentiated and more interactive, but of course technology can provide many more possibilities for enriching the mathematics classrooms. funding open access funding enabled and organized by projekt deal. open access this article is licensed under a creative commons attribution 4.0 international license, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the creative commons licence, and indicate if changes were made. the images or other third party material in this article are included in the article's creative commons licence, unless indicated otherwise in a credit line to the material. if material is not included in the article's creative commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. to view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/. student 2: do we need to use room temperature? 9 student 3: of course, but we need a formula for calculation (they found newton's formula 10 t=-10ln student 1: if we use centigrade? 12 student 4: we should always use the same unit in the formula no, i think we will find the time elapsed after death, then subtract it from the measuring hour student 1: why do we find the temperature difference? 20 student 2: why are we given two temperature values if we are not going to use them? 21 student 1: why should we care about temperature measured at 12? student 3: no, we will see the same result from both of them student 2: it remains stable after a certain period of time student 1: why does it stay stable? 29 student 2: how much can the body temperature decrease? until room temperature student 4: it makes sense, i think just like you cycle, let's take the ln of it while calculating via calculator, we must write this expression in brackets. the order of 35 operation! 36 student 2: approximately let's also calculate according to the temperature measured at 12. then we need to find 8 hours 38 logically student 1: let's find and prove it student 2: yes, i found the result to be 8 student 1: both solutions point to about 4 o'clock clever! i felt like a detective. i wonder how to be a detective, calculations are fun the flipped classroom, lethal mutations, and the didactical contract: a cautionary tale. primus technology, communication, and collaboration: re-thinking communities of inquiry flip your classroom: reach every student in every class every day the flipped classroom: a survey of the research implementing the flipped classroom: challenges and strategies ters-yüz sınıf modeli uygulamalarına dayalı bir matematik sınıfındaki öğrenci katılım sürecinin incelenmesi an innovative learning model in digital age: flipped classroom matematik öğretmenlerinin yanlış cevaplara verdikleri dönütlerin öğrenci özsaygıları üzerindeki rolü impacts of flipped classroom in high school health education research on flipped classroom teaching mode of high school bmathematics under the background of "internet+ flipping your english class to reach all learners: strategies and lesson plans qualitative inquiry and research design choosing among five approaches flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course transformation of the mathematics classroom with the internet. zdm mathematics education definition of flipped learning exploring realistic mathematics education in a flipped classroom context at the tertiary level 10 reasons to flip designing a national blended learning program for "out-of-field" mathematics teacher professional development research and practice of flipped classroom teaching mode based on guidance case. education and information technologies technology in mathematics education: tapping into visions of the future pinterest: a tool for lesson planning a large class engagement( lce) model based on service-dominant logic (sdl) and flipped classrooms reform and innovation of ideological and political course teaching from the perspective of flipped classroom lesson study in a blended approach to support isolated teachers in teaching with technology emerging perspectives on learning, teaching, and technology inverting the classroom: a gateway to creating an inclusive learning environment digital technologies to teach and learn mathematics: context and re-contextualization integrating technology into mathematics teaching at the university level a critical review of flipped classroom challenges in k-12 education: possible solutions and recommendations for future research longman dictionary of contemporary english effects of social constructivist mobile learning environments on knowledge acquisition: a meta-analysis qualitative research design: an interactive approach learning designs using flipped classroom instruction qualitative research: a guide to design and implementation flipping calculus. primus uncertainty and the reform of elementary math education. international scholarly research notices stand ards-and-posit ions/posit ion-state ments /strat egic-use-of-techn ology -in-teach ing-and-learn ing-mathe matic s/last mathematical knowledge and practices resulting from access to digital technologies social constructivist perspectives on teaching and learning the flipped classroom: a twist on teaching social constructivist teaching strategies in the small group classroom understanding feedback: a learning theory perspective ways of acting when using technology in the primary school classroom: contingencies and possibilities for learning leveraging the design heuristics of realistic mathematics education and culturally responsive pedagogy to create a richer flipped classroom calculus curriculum mind in society: the development of higher psychological processes the role of tutoring in problem-solving publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-341038-lomaz6pc authors: odriozola-gonzález, paula; planchuelo-gómez, álvaro; irurtia, maría jesús; de luis-garcía, rodrigo title: psychological effects of the covid-19 outbreak and lockdown among students and workers of a spanish university date: 2020-05-19 journal: psychiatry res doi: 10.1016/j.psychres.2020.113108 sha: doc_id: 341038 cord_uid: lomaz6pc the aim of this study was to analyze the psychological impact of covid-19 in the university community during the first weeks of confinement. a cross-sectional study was conducted. the depression anxiety stress scale (dass-21) was employed to assess symptoms of depression, anxiety and stress. the emotional impact of the situation was analyzed using the impact of event scale. an online survey was fulfilled by 2530 members of the university of valladolid, in spain. moderate to extremely severe scores of anxiety, depression, and stress were reported by 21.34%, 34.19% and 28.14% of the respondents, respectively. a total of 50.43% of respondents presented moderate to severe impact of the outbreak. students from arts & humanities and social sciences & law showed higher scores related to anxiety, depression, stress and impact of event with respect to students from engineering & architecture. university staff presented lower scores in all measures compared to students, who seem to have suffered an important psychological impact during the first weeks of the covid-19 lockdown. in order to provide timely crisis-oriented psychological services and to take preventive measures in future pandemic situations, mental health in university students should be carefully monitored. • college students reflected a possible psychological impact of the lockdown. • symptoms of common mental health disorders were reported by 20-35% of respondents. • around the half of respondents presented moderate to severe impact of the outbreak. • students from the engineering and architecture area showed lower symptomatic scores. • mental health from students should be monitored to mitigate the impact of the crisis. situation was analyzed using the impact of event scale. an online survey was fulfilled by 2530 members of the university of valladolid, in spain. moderate to extremely severe scores of anxiety, depression, and stress were reported by 21. 34%, 34.19% and 28 .14% of the respondents, respectively. a total of 50.43% of respondents presented moderate to severe impact of the outbreak. students from arts & humanities and social sciences & law showed higher scores related to anxiety, depression, stress and impact of event with respect to students from engineering & architecture. university staff presented lower scores in all measures compared to students, who seem to have suffered an important psychological impact during the first weeks of the covid-19 lockdown. in order to provide timely crisis-oriented psychological services and to take preventive measures in future pandemic situations, mental health in university students should be carefully monitored. keywords: university students, anxiety, stress, depression, psychological impact the novel coronavirus disease was firstly detected in wuhan (hubei province, china) in december 2019. from that moment, it began to spread first in china, and soon afterwards throughout the world. confirmed cases and deaths grew rapidly, and on april 5th there were more than 1,200,000 confirmed cases worldwide and more than 68,000 people had died from it (johns hopkins university & medicine, 2020) . this situation has produced a notable emotional impact on medical workers and the general population, with important symptoms of anxiety, stress and depression (kang et al., 2020; wang et al., 2020) . on that same date, 130,759 covid-19 cases had been registered in spain, making it the third country in the world with the most people affected by this pandemic. in response to this situation, between march 9th and 13 th there was a progressive closure of schools and universities. on march 14th, the spanish government declared a nationwide state of alert and population lockdown was imposed as of march 16th. for many spaniards, this is the first experience of an emergency with an imperceptible agent, leading to great uncertainty and significant adverse consequences for mental health (hawryluck et al., 2004; shigemura et al., 2020) . although the overall impact on education and mental health of the university environment is still unknown, it is expected to be very considerable (araújo et al., 2020; sahu, 2020) . considering the usual high incidence of emotional disorders in university students, it can be expected that the current situation may cause a notable impact on this population (auerbach et al., 2016; bruffaerts et al., 2018; hunt and eisenberg, 2010) . for instance, in a recent study focusing on chinese medical college students, higher levels of anxiety were associated with factors strongly related to covid-19, such as acquaintance with a covid-19 diagnosed patient (cao et al., 2020) . likewise, data from international students have shown an increase in concerns not only for their education, but also for the well-being of their families in case of their return home because of suspended in-person classes (zhai and du, 2020) . given the expected impact of the situation on this community, it is crucial to analyze the experience of members of the university community during the covid-19 crisis and confinement in order to develop measures and implement psychological interventions properly adjusted to this situation. that may help mitigate the possible adverse effects on education, work and mental health among the university members. the purpose of this study was to analyze the psychological symptoms on the members of university of valladolid, in spain, during the outbreak of the covid-19. every member of the university of valladolid living in spain at the time of the study was eligible for participation in this study. an anonymous internet survey was announced through the communication channels of the university of valladolid, a regional tv station and social media. ethics approval was obtained from the research ethics board of the university of valladolid. a web-based survey composed of 66 multiple-choice questions was launched on march 28th, 2020, and remained open during 8 days. the survey took approximately 10 minutes to complete, and included questions referred to demographical data, personal situation during the confinement, present and/or past psychological/psychiatric treatment, present intake of psychoactive medication, perceived impact of the confinement on personal and social relationships, self-reported concern about the social and economic situation caused by the crisis and its impact on the health of oneself, partner, parents, children and other family and friends. moreover, questions related specifically to the university of valladolid were included for the university students and workers. they were asked about their role at the university (student, faculty member or academic staff, or administrative staff). students and faculty members or academic staff were asked about their area of study or expertise. the emotional impact and psychological symptoms associated with confinement due to the covid-19 crisis were evaluated using two scales: depression anxiety stress scale (dass-21; antony et al., 1998) and the impact of event scale (ies; horowitz et al., 1997) . the dass-21 is a 4-point likert-type scale (3 = applied to me very much or most of the time; 0 = did not apply to me at all) that measures the negative emotional states experienced during the last week through 21 items. we administered the spanish version of the dass-21 (daza et al., 2002) . it has shown a hierarchical factor structure with three first-order factors (depression, anxiety, and stress) and a second-order factor that is an overall indicator of emotional symptoms (ruiz et al., 2017) . the dass-21 has also shown good internal consistency and convergent and discriminant validity. the ies is a 4-point likert-type scale (0= not at all, 1=rarely, 3=sometimes, 5=often) that assesses subjective distress resulting from a traumatic life event through 15 items. it contains two subscales: intrusion (intrusive thoughts, nightmares, intrusive feelings and imagery, dissociative-like re-experiencing) and avoidance (numbing of responsiveness, avoidance of feelings, situations and ideas). we administered the spanish version by baguena et al. (2001) , which has shown good psychometric properties. slight adaptations were performed from this version (changing verbal tenses where needed) to account for the nature of the event explored. a cut-off of the ies ≥26 was used to reflect moderate to severe impact. generalized linear models (glm) with a gaussian distribution were employed to analyze the association between the dass-21 and ies scores with each of independent variables described at the survey instrument subsection. glm models were also applied to the self-reported concern about the health of different family members and friends, and about the social and economic situation. the independent variables were the same as for the previous glm. the final multivariate glm was selected using the akaike's information criterion (aic) and an automatic stepwise strategy, with forward and backward steps. the model with the lowest aic was automatically selected. diverse multivariate models were obtained to consider the diverse variables specific to each university group. p-values below 0.05 were considered to be statistically significant. the analysis was performed using r statistical software package, version 3.5.2. the survey was completed by a total of 3,707 participants, among which 2,530 belonged to the university of valladolid and are thus the sample considered in this study. there was a 66.1% of female participants, and ages ranged between 18 and 70 years (m = 27.9, sd = 12.4). 76.8% of the participants were students, 9.8% administrative staff and 13.4% faculty members and academic staff (excluding phd students). the qualitative characteristics of the final survey respondents are summarized in table 1 , whereas their characteristics related to their situation in the university of valladolid are described in table 2 . the mean values and standard deviations for each of the dass-21 subscales were 5.52±4.92 for depression, 3.34±3.87 for anxiety and 6.81±4.72 for stress. figure 1 shows the distribution of dass-21 mean scores by area of study or research area (students and faculty members or academic staff). figures 2-4 show the proportion of university students that fall within each of the groups that are usually considered for this instrument, considering their area of study. in the three subscales, students from the e&a area were those with the highest proportion of subclinical scores and lowest of severe and extremely severe scores. on the other hand, students from the a&h and ss&l areas were those with the lowest proportion of subclinical anxiety and depression scores, with a similar proportion to the sciences area for stress. univariate glm coefficients for the specific university characteristics are shown in table 3 . supplementary figures 1-2 show the distribution of dass-21 mean scores by university group and year of study (including master and phd students) excluding the last two years, which in the university of valladolid only include architecture and medicine students. univariate coefficients for the other characteristics are shown in supplementary table 1. significant higher depression, anxiety and stress scores were observed in students with respect to university workers. in addition, scores were significantly higher in undergraduate students compared to master students for depression, anxiety and stress, and also compared to phd students for stress. for the three subscales, significant lower scores were found in e&a students compared to the students from the other areas, except for the depression subscale in students from the hs area. undergraduate students at the sixth year of their studies (which include only medicine students) presented significant lower depression and stress scores compared to first year students, and significant lower depression scores were also found for fourth year students with respect to first year. faculty members and academic staff from the a&h area presented significant higher scores with respect to the e&a area for the three subscales, and workers from the hs area also showed significant higher anxiety scores compared to the e&a area. after adjusting for multiple covariates, administrative staff showed significant lower anxiety scores compared to students, and faculty members and academic staff showed significant lower depression scores compared to students. significant lower stress scores were found in master compared to undergraduate students. significant higher anxiety scores were found in a&h and ss&l with respect to e&a students, and also higher depression scores in a&h compared to e&a students. faculty members and academic staff from the a&h area also showed significant higher depression scores compared to those in e&a. the complete multivariate model can be seen in supplementary table 2 . around 12.5% of the respondents showed scores related to severe symptoms with the ies instrument, and around 75% of the sample presented mild or moderate symptomatic levels, with relatively higher avoidance than intrusion scores. univariate glm coefficients for the specific university characteristics are shown in table 4 . univariate coefficients for the other characteristics are shown in supplementary table 3 . with respect to the students, significant higher intrusion and avoidance scores were observed compared to faculty members and academic staff, and only higher avoidance scores compared to administrative staff. master students showed significant lower intrusion and avoidance scores compared to undergraduate students, and lower avoidance scores in phd compared to undergraduate students. significant lower intrusion and avoidance scores were observed in e&a students in comparison with the other four areas. sixth year undergraduate students (medicine students only) presented significant lower avoidance scores with respect to first year students. e&a faculty members and academic staff showed significant lower intrusion and avoidance scores in comparison with the other areas, except intrusion scores compared to the sciences area. after adjusting for multiple covariates, administrative staff showed significant lower avoidance scores compared to students, and faculty members and academic staff showed significant lower intrusion scores compared to students. significant higher intrusion and avoidance scores were found in a&h and ss&l with respect to e&a students. the same result was also observed in faculty members and academic staff. within this group, workers from the sciences field showed higher avoidance scores in comparison to those in e&a. the complete multivariate model can be seen in supplementary table 4 . the concern scores are summarized in table 5 . univariate glm coefficients for the specific university characteristics are shown in tables 6-7. university workers (both faculty members or academic staff and administrative staff) presented generalized significant higher concern scores with respect to students, except for their concern about relatives' health. generalized significant higher concern scores were found in a&h and ss&l compared to e&a students. some concern scores were significantly lower in undergraduate students compared to studies different to phd and master (labelled as other studies), and in first year students compared to students from higher years. significant higher concern score about social situation was found in a&h in comparison with e&a faculty members and academic staff. univariate models for other characteristics are shown in supplementary tables 5-6. adjusting by multiple covariates, some of the univariate identified differences remained significant, such as the differences between students from e&a with respect to a&h and ss&l field, and the differences between workers and students. the multivariate models are shown in supplementary tables 7-8. the covid-19 outbreak has prompted most countries opt for population confinement and social distancing measures as a way to control the spread of the virus. however, important psychological effects have been pointed out in previous confinement experiences (hawryluck et al., 2004) . the current pandemic has already shown significant psychological symptoms related to anxiety, stress and depression wang et al. (2020) . as these authors point out, this difference in prevalence may be due to the specific evaluation of the impact of the event by ies versus a non-specific evaluation by dass-21. regarding the different groups at the university, significantly higher depression, anxiety and stress scores were observed in students compared to the different groups of employees. the high prevalence of psychological symptoms in university students has been frequently pointed out (auerbach et al., 2016; bayram and bilgel, 2008; bruffaerts et al., 2018) . while there is evidence from several studies in which students from the hs or engineering area were found to present higher symptomatology scores than those in the humanities area (elias et al., 2011; posselt and lipson, 2016) , our results show precisely the opposite. in fact, the lowest scores are shown by e&a students and workers on all three subscales. in line with our results, the study of lipson et al. (2016) showed that a&h students have a greater tendency to develop mental illnesses compared to the other areas, such as the engineering and business students, who also seem to undergo treatment less frequently. the percentage of moderate to severe scores of the subgroups of our sample in ies also showed higher prevalence in the a&h area and lower prevalence in the e&a area, not only in the students group, but also in the group of faculty members and academic staff. specifically, 60.31% of students and 45.07% of faculty members and academic staff of a&h compared to 40.42% of students and 23.86% faculty members and academic staff of e&a showed moderate to severe ies scores. to our knowledge, this study is the first one that analyzes the impact of a major event such as the covid-19 crisis in students from different fields. a study with a large sample size performed in china obtained similar percentage of respondents with moderate or severe ies scores with respect to our study . furthermore, that study also highlights the high levels of symptomatology in students, indicating that the uncertainty and the potential negative impact on academic progress may have motivated the impact on students' mental health. however, results from another study in the general population of china, evaluated using the same instrument (ies), seem to show a much lower percentage of moderate or severe scores on the impact of the event, 7.6% (zhang and ma, 2020) . the reasons for the difference between that study and our results may perhaps be related to the considerable age difference between the respondents of both studies, with younger respondents in our case, and a significantly smaller sample size in the study on the chinese general population. in our study, by contrast, university employees from all groups showed higher concern scores than students, except for concern about relatives' health. once again, the a&h students, along with those in the ss&l areas, showed a higher level of concern than the e&a students. similarly, faculty members and academic staff in a&h had a higher concern score about the social situation in comparison with those in e&a. the presence of a more pragmatic attitude or different world viewpoints in the e&a group might account for the differential results found in this study, but more research would be needed to elucidate this issue. this study has some strengths and limitations. among the strengths, the large sample size (2530 respondents) allowed us to perform a robust analysis and extract solid tendencies and associations. also, this is an early study that offers a unique opportunity to investigate the emotional impact of the covid-19 pandemic in a university environment. it provides valuable information about the current situation useful to gain some insight about the situation in other universities or in possible future global crises. regarding the limitations, firstly, this is a cross-sectional study carried out at a spanish university under an unprecedented situation. longitudinal studies are needed to analyze the long-term impact of this situation on the psychological state of their members and to draw conclusions about the cause and effect relationships between the variables involved. secondly, we adopted a convenient online survey in only one university from spain, which may contribute to some bias in the study results. larger scale surveys in different universities should be conducted in order to extend and generalize the findings. thirdly, our results indicate the need of incorporating additional aspects in future studies. for example, it is important to explore why younger students are suffering a greater psychological impact, which could be related to factors such as their perception of the future, their way of consuming information media, etc. in conclusion, university students have been specially impacted by the covid-19 confinement. students from the ss&l and a&h fields were more affected than students from other areas, particularly with respect to e&a students. this study suggests that mental health from university students and employees should be carefully monitored during this crisis, and that universities should provide psychological services oriented and adapted to these circumstances to mitigate the emotional impact on university members. administrative staff vs. student -7.12**** -2.73**** -1.58**** -2.80**** academic staff vs. student -6.22**** -1.84**** -1.34**** -3.03**** .76** **** p < 0.0001, *** p < 0.001, ** p < 0.01, * p < 0.05 table 5 . concern for health and socio-economic situation. mean ± sd median interquartile range own health (n = 2521) 5.4 ± 2.9 5 5 partner health (n = 1624) 6.6 ± 3.0 7 5 parents health (n = 2419) 8.2 ± 2.3 9 3 children health (n = 613) 5.6 ± 3.8 6 9 other family health (n = 2489) 8.3 ± 2.2 9 3 friends health (n = 2511) 6.6 ± 2.7 7 4 social situation (n = 2503) 8.1 ± 2.2 9 3 economic situation (n = 2490) 8.2 ± 2.2 9 3 sd = standard deviation. table 6 . univariate glm coefficients associating concern with respondents characteristics related to the situation at the university of valladolid (part 1). own health parents health children health group administrative staff vs. student 1.07**** 0.95**** 0.76**** 3.57**** academic staff vs. student 0.76**** 0.62** 0.67**** 3.31**** 27 -0.32 -0.18 **** p < 0.0001, *** p < 0.001, ** p < 0.01, * p < 0.05 table 7 . univariate glm coefficients associating concern with respondents characteristics related to the situation at the university of valladolid (part 2). 0.20 0.18 0.59 **** p < 0.0001, *** p < 0.001, ** p < 0.01, * p < 0.05 psychometric properties of the 42-item and 21-item versions of the depression anxiety stress scales (dass) in clinical groups and a community sample impact of sars-cov-2 and its reverberation in global higher education and mental health mental disorders among college students in the world health organization world mental health surveys psychometric properties of the spanish version of impact of event scale-revised (ies-r) 2019-ncov epidemic: address mental health care to empower society the prevalence and socio-demographic correlations of depression, anxiety and stress among a group of university students mental health problems in college freshmen: prevalence and academic functioning the psychological impact of the covid-19 epidemic on college students in china the depression anxiety stress scale-21: spanish translation and validation with a hispanic sample stress and academic achievement among undergraduate students in universiti putra malaysia sars control and psychological effects of quarantine impact of event scale: a measure of subjective stress mental health problems and help-seeking behavior among college students coronavirus covid-19 global cases by the center for systems science and engineering at johns hopkins [www document the mental health of medical workers in wuhan, china dealing with the 2019 novel coronavirus. the lancet major differences: variations in undergraduate and graduate student mental health and treatment utilization across academic disciplines competition, anxiety, and depression in the college classroom: variations by student identity and field of study the hierarchical factor structure of the depression anxiety and stress scale-21 closure of universities due to coronavirus disease 2019 (covid-19): impact on education and mental health of students and academic staff public responses to the novel 2019 coronavirus (2019-ncov) in japan: mental health consequences and target populations immediate psychological responses and associated factors during the initial stage of the 2019 coronavirus disease (covid-19) epidemic among the general population in china timely mental health care for the 2019 novel coronavirus outbreak is urgently needed. the lancet mental health services for older adults in china during the covid-19 outbreak. the lancet addressing collegiate mental health amid covid-19 pandemic impact of the covid-19 pandemic on mental health and quality of life among local residents in liaoning province, china: a cross-sectional study álvaro planchuelo-gómez was supported by junta de castilla y león (spain) and the european social fund (id: 376062, base de datos nacional de subvenciones). this study received no specific funding. ☒ the authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.☐the authors declare the following financial interests/personal relationships which may be considered as potential competing interests: key: cord-028455-1swijiq3 authors: serban, iulian vlad; gupta, varun; kochmar, ekaterina; vu, dung d.; belfer, robert; pineau, joelle; courville, aaron; charlin, laurent; bengio, yoshua title: a large-scale, open-domain, mixed-interface dialogue-based its for stem date: 2020-06-10 journal: artificial intelligence in education doi: 10.1007/978-3-030-52240-7_70 sha: doc_id: 28455 cord_uid: 1swijiq3 we present korbit, a large-scale, open-domain, mixed-interface, dialogue-based intelligent tutoring system (its). korbit uses machine learning, natural language processing and reinforcement learning to provide interactive, personalized learning online. korbit has been designed to easily scale to thousands of subjects, by automating, standardizing and simplifying the content creation process. unlike other its, a teacher can develop new learning modules for korbit in a matter of hours. to facilitate learning across a wide range of stem subjects, korbit uses a mixed-interface, which includes videos, interactive dialogue-based exercises, question-answering, conceptual diagrams, mathematical exercises and gamification elements. korbit has been built to scale to millions of students, by utilizing a state-of-the-art cloud-based micro-service architecture. korbit launched its first course in 2019 and has over 7, 000 students have enrolled. although korbit was designed to be open-domain and highly scalable, a/b testing experiments with real-world students demonstrate that both student learning outcomes and student motivation are substantially improved compared to typical online courses. intelligent tutoring systems (its) are computer programs powered by artificial intelligence (ai), which deliver real-time, personalized tutoring to students. traditional its implement or imitate the behavior and pedagogy of human tutors. in particular, one type of its are dialogue-based tutors, which use natural language conversations to tutor students [13] . this process is sometimes called "socratic tutoring", because of its similarity to socratic dialogue [17] . newer its have started to interleave their dialogue with interactive media (e.g. interactive videos and web applets) -a so-called "mixed-interface system". it has been shown that its can be twice as effective at promoting learning compared to the previous generation of computer-based instruction and may be as effective as human tutors in general [12] . however, despite the fact that its have been around for decades and are known to be highly effective, their deployment in education and industry has been extremely limited [14, 16] . a major reason for this is the sheer cost of development [5, 14] . as observed by olney [14] : "unfortunately, its are extremely expensive to produce, with some groups estimating that it takes 100 h of authoring time from ai experts, pedagogical experts, and domain experts to produce 1 h of instruction." on the other hand, lower-cost educational approaches, such as massive open online courses (moocs), have flourished and now boast of having millions of learners. it is estimated that today there are over 110 million learners around the world enrolled in moocs [18] . however, the learning outcomes resulting from learning in moocs depend critically on their teaching methodology and quality of content, and remains questionable in general [2, 3, [9] [10] [11] 15] . in particular, recent research indicates that moocs having low levels of active learning, little feedback from instructors and peers, and few peer discussions tend to yield poor learning outcomes [10, 15] . further, it is well-known that student retention in moocs is substantially worse than in traditional classroom learning [8] . by combining low cost and scalability with the personalization and effectiveness of its, we hope korbit may help to effectively teach and motivate millions of students around the world. korbit is a large-scale, open-domain, mixed-interface, dialogue-based its, which uses machine learning, natural language processing (nlp) and reinforcement learning (rl) to provide interactive, personalized learning online. the its has over 7,000 students enrolled from around the world, including students from educational institutions and professionals from industry partners. korbit is capable of teaching topics related to data science, machine learning, and artificial intelligence. the modular platform will soon be expanded with many more topics. students enroll on the korbit website by selecting either a course or a set of skills they would like to study. students may also answer a few questions about their background knowledge. based on these, korbit generates a personalized curriculum for each student. following this, korbit tutors the student by alternating between short lecture videos and interactive problem-solving exercises. the outer-loop system decides on which lecture video or exercise to show next based on the personalized curriculum. work is currently underway to adapt the curriculum during the learning process (fig. 1) . during the exercise sessions, the inner-loop system manages the interaction. first, it shows the student a problem statement (e.g., a question). the student may then attempt to solve the exercise, ask for help, or skip the exercise. if the student attempts to solve the exercise, their solution attempt is compared against the expectation (i.e. reference solution) using an nlp model. if their solution is classified as incorrect, then the inner-loop system will select one of a dozen different pedagogical interventions. the pedagogical interventions include textual hints, mathematical hints, elaborations, explanations, concept tree diagrams, and multiple choice quiz answers. the pedagogical intervention is chosen by an ensemble of machine learning models based on the student's profile and last solution attempt. depending on the pedagogical intervention, the inner-loop system may either ask the student to retry the initial exercise or follow up on the intervention (e.g., with additional questions, confirmations, or prompts). the korbit its is related to the work on dialogue-based its, such as the pioneering autotutor and the newer ibm watson tutor [1, 6, 7, 13, 19] . although korbit is highly constrained compared to existing dialogue-based its, a major innovation of korbit lies in its modular, scalable design. the inner-loop system is implemented as a finite-state machine. each pedagogical intervention is a separate state, with its own logic, data and machine learning models. each state operates independently of the rest of the system, has access to all database content (including exercises and videos) and can autonomously improve as new data becomes available. this ensures that the system gets better and better, that it can adapt to new content and that it can be extended with new pedagogical interventions. the transitions between the states of the finite-state machine is decided by a reinforcement learning model, which itself is agnostic to the underlying implementation of each state and also continues to improve as more and more data becomes available. we have conducted multiple studies to evaluate the korbit its. some of these studies have evaluated the entire system while others have focused on particular aspects or modules of the system. taken together, the studies demonstrate that the korbit its is an effective learning tool and that it overall improves student learning outcomes and motivation compared to alternative online learning approaches. in this paper we limit ourselves and discuss only one of these studies. the study we present compares the entire system (full its) against an xmooclike system [4] . the purpose of this particular study is to evaluate 1) whether students prefer the korbit its or a regular mooc, 2) whether the korbit its increases student motivation, and 3) which aspects of the korbit its students find most useful and least useful. in an ideal world, korbit its would be compared against a regular xmooc teaching students through lecture videos and multiple choice quizzes in a randomized controlled trial (a randomized a/b testing experiment). however, it is not possible to compare against such a system in a randomized controlled trial, because it would create confusion and drastically offset student expectations. therefore, in this study, we compare the full its against a reduced its, which appears identical to the full its and utilizes the same content (video lectures and exercise questions), but defaults to multiple choice quizzes 50% of the time. thus, students assigned to the reduced its spend about half of their interactions in an xmooc-like setting. we refer to this system as the xmooc its. table 1 . a/b testing results comparing the full its against the xmooc its: average time spent by students (in minutes), returning students (in %), students who said they will refer others (in %) and learning gain (in %), with corresponding 95% confidence intervals. the * and * * shows statistical significance at 90% and 95% confidence level. the experiment was conducted in 2019 with n = 612 participants. students who enrolled online were randomly assigned to either the full its (80%) or xmooc its (20%). students came from different countries and were not subject to any selection or filtering process. apart from bug fixes and speed improvements, the system was not modified during the experiment to limit confounding factors. after studying for about 45 min, students were shown a questionnaire to evaluate the system. table 1 shows the experimental results. the average time spent in the full its was 39.86 min compared to 22.98 min in the xmooc its. as such, the full its yields a staggering 73.46% increase in time spent. in addition, the percentage of returning students and the percentage of students who said they would refer others to use the system is substantially higher for the full its compared to the xmooc its. these results were also confirmed by the feedback provided by the students in the questionnaire. thus, we can conclude that students strongly prefer korbit its over xmoocs and that the korbit its increases overall student motivation. table 1 also shows that the average student learning was observed to be 39.14%. the learning gain is measured as the proportion of instances where a student provides a correct exercise solution after having receiving a pedagogical intervention from the korbit its. thus, the pedagogical interventions appear to be effective. finally, in the questionnaire, 85.31% of students reported that they found the chat equally or more fun compared to learning alone and 66.67% of students reported that the chat helped them learn better sometimes, many times or all of the time. for the full its, 54.17% of students reported that they would refer others to use korbit its. in addition, students reported that the korbit its could be improved by more accurately identifying their solutions as being correct or incorrect and, in the case of incorrect solutions, by providing more personalized feedback. adaptive visual dialog for intelligent tutoring systems a large sample comparison of grade based student learning outcomes in online vs. face-to-face courses learning in an introductory physics mooc: all cohorts learn equally including an on-campus class making sense of moocs: musings in a maze of myth, paradox and possibility plan ahead: pricing its learner models autotutor: an intelligent tutoring system with mixed-initiative dialogue intelligent tutoring systems with conversational dialogue exploring the factors affecting mooc retention: a survey study online versus in-class courses: an examination of differences in learning outcomes learning is not a spectator sport: doing is better than watching for learning from a mooc moocs: opportunities for their use in compulsory-age education effectiveness of intelligent tutoring systems: a metaanalytic review autotutor and family: a review of 17 years of natural language tutoring using novices to scale up intelligent tutoring systems it's the learning, stupid! discussing the role of learning outcomes in moocs cognitive tutor: applied research in mathematics education a comparative evaluation of socratic versus didactic tutoring by the numbers: moocs in 2019 preliminary evaluations of a dialogue-based digital tutor key: cord-329224-t45bedot authors: byrnes, yasmeen m.; civantos, alyssa m.; go, beatrice c.; mcwilliams, tara l.; rajasekaran, karthik title: effect of the covid-19 pandemic on medical student career perceptions: a national survey study date: 2020-07-24 journal: medical education online doi: 10.1080/10872981.2020.1798088 sha: doc_id: 329224 cord_uid: t45bedot background & objective: the covid-19 pandemic and resulting cancellation of medical student clinical rotations pose unique challenges to students’ educations, the impact of which has not yet been explored. design: this cross-sectional survey study collected responses from 13 april 2020 until 30 april 2020. students at us allopathic medical schools completed the survey online. results: 1,668 responses were analyzed. a total of 337 (20.2%) respondents thought the pandemic would affect their choice of specialty, with differences across class years: 15.2% (53) of first-years (ms1s), 26.4% (92) of second-years (ms2s), 23.7% (162) of third-years (ms3s), and 9.7% (22) of fourth-years (ms4s) (p < 0.0001). among all classes, the most common reason chosen was inability to explore specialties of interest (244, 72.4%), and the second was inability to bolster their residency application (162, 48.1%). out of the ms3s who chose the latter, the majority were concerned about recommendation letters (68, 81.0%) and away rotations (62, 73.8%). as high as 17.4% (119) of ms3s said they were more likely to take an extra year during medical school as a result of the pandemic. region of the us, number of local covid cases, and number of local covid deaths had no effect on whether respondents thought the pandemic would affect their specialty choice. conclusions: our study found that about one-fifth of surveyed medical students currently believe that the covid-19 pandemic will affect their choice of specialty, with many of these citing concerns that they cannot explore specialties or obtain recommendation letters. with prolonged suspension of clinical rotations, targeted efforts by medical schools to address these concerns through enhanced virtual curriculum development and advising strategies will become increasingly important. further study is needed to explore whether these cross-sectional student perspectives will manifest as changes in upcoming national residency matching program data. the global pandemic caused by sars-cov-2 has caused unprecedented changes to almost every aspect of society, with social distancing orders taking effect across the usa (us) beginning in march 2020 [1, 2] . medical student education has been uniquely impacted by these changes. for the vast majority of medical students across the country, all clinical learning has been indefinitely suspended in accordance with the american association of medical colleges (aamc) recommendations on 17 march 2020 [3], including visiting student learning opportunities (away rotations) [4] . many medical schools have now developed virtual curricula to continue medical education until students are able to return to hospitals and clinics [5] [6] [7] [8] [9] . however, lack of in-person clinical exposure may pose a challenge to medical students, particularly those attempting to choose a specialty or apply to residency programs. published commentaries by medical students have indicated a simultaneous sense of duty to do their part by staying home, as well as concern about how their education will proceed [10, 11] . several previous disruptions to medical school clinical curriculum and their effects have been described in the literature. during part of the 2002-2004 severe acute respiratory syndrome (sars) pandemic, medical students in affected countries were barred from direct in-person patient care [12] [13] [14] . at these institutions, alternative virtual coursework was adopted, such as e-learning modules, video vignettes, and virtual reality simulators [12] . similar measures were adopted in certain regions during the middle eastern respiratory syndrome (mers) pandemic [15] . when hurricane katrina struck new orleans, tulane university medical school relocated medical student education, including clinical rotations, to four medical schools in houston, texas [16] . while to our knowledge there are no studies exploring sars or mers impact on medical student specialty choice, one post-katrina new orleans study found that medical students were significantly more likely to specialize in emergency medicine (em) and less likely to specialize in psychiatry in the 5 years after the hurricane than in the 5 years before [17] . changes in medical education in response to these disasters, while locally disruptive, were not as widespread as the changes brought on by the covid-19 pandemic. the current changes to us medical education are unprecedented. although there are some existing anecdotal commentaries describing effects of the covid-19 pandemic on medical education [5, [18] [19] [20] [21] , studies describing virtual curriculum development [5] [6] [7] [8] , and reports of medical student mobilization in response to the pandemic [22] [23] [24] [25] [26] , to our knowledge no data yet exists describing the current impact on medical students' academic lives. it is not yet known how long the suspension of clinical rotations will continue [27] . as such, early identification of concerns and potential issues may help guide career advising, virtual curriculum development, and general support for medical students during this unique time. this study aimed to provide a crosssectional view into the impact of the covid-19 crisis on us medical students' perception of their education and career development. this cross-sectional online survey study collected response data from medical students across the usa from 13 april 2020 through 30 april 2020. the survey was designed by three investigators (yb, ac, and kr) and contained 9 multiple choice items with up to 6 possible follow-up multiple choice items depending on the participant's answers (see appendix 1) . it contained no identifying information. respondent demographics included us state in which the respondent attended medical school and class year. the survey gave students the option of selecting up to two specialties of interest. a variable for certainty of specialty was created using these responses. those who selected only one specialty were considered certain. those who selected either two specialties or chose the option 'undecided' were considered uncertain. the survey also asked students which electives they had completed if any. using their response to this in conjunction with choice of specialties, a variable was created to indicate whether students had taken an elective in their specialty of interest. those whose specialty was listed as 'undecided', those who had not taken any electives, and those who had taken electives that did not match their specialty of interest were considered as not having taken an elective in their specialty of interest. prior to national distribution, feedback on survey design and content was solicited and incorporated from 7 home-institution medical students with a variety of specialty interests via open-ended verbal discussions (one 3-person group discussion and four 1-on-1 discussions). responses to the survey were collected and managed using redcap electronic data capture tools [28, 29] . the survey was selfadministered and accessed by participants via an electronic link. participation was voluntary, and participants were allowed to terminate the survey at any time. this project was reviewed and determined to qualify as quality improvement by the university of pennsylvania institutional review board. the survey was distributed nationally using a multipronged approach. a list of all 150 american association of medical colleges (aamc) medical schools in the continental usa, hawaii, and alaska was generated [30] . canadian, caribbean, and puerto rican aamc medical schools were excluded from this list. for each school on the list, deans or directors of student affairs were contacted via email. they were given a description of the study and asked to distribute the survey link to their institution's entire medical student body. we did not have a method by which to confirm which deans or directors actually distributed it. in addition to this approach, based on investigators' knowledge of existing communication structures among students, additional medical student-led distribution was also achieved at many schools through email listservs and social media groups. all analyses were conducted using sas version 9.4 (sas institute, cary, nc). cross-tabulated frequencies and percentages were calculated, and all associations were quantified using chi-squared and fisher's exact tests, as applicable. fisher's exact tests were used in cases where a combination of two variables resulted in any cell counts of five or less. a p-value equal to or less than 0.05 was considered to indicate a statistically significant relationship between two variables. state of medical school location was categorized as one of four usa regions (northeast, south, midwest, and west) using guidelines from the us census bureau [31] . publicly available covid-19 data were used as proxy measures for pandemic severity in each us state. covid-19 confirmed cumulative positive case counts and cumulative death counts by state as of 21 april 2020 at 4pm est were acquired from the covid tracking project [32] . this date was chosen because it was halfway through our 18-day study period. the number of positive cases in each state was originally a continuous variable with median 19,881. using this, number of cases was dichotomized into roughly two equal groups, at least 19,000 cases and under 19,000 cases. similarly, the median number of deaths was 584 and was dichotomized into at least 600 deaths and under 600 deaths. 1668 responses were analyzed, from 20 us states [ table 1 ]. the majority of responses were from the northeast (623, 37.4%), south (454, 27.2%), and midwest (530, 31.8%), with a minority (61, 3.7%) from the west. a plurality of respondents were ms3s (684, 41.0%), with 349 (20.9%) ms1s, 349 (20.9%) ms2s, and 226 (13.6%) ms4s. a small minority of respondents (61, 3.7%) were md-phds within the research part of their program, students taking a year out during medical school for research, dual degree, or other reasons, or 'other'. for simplicity of analysis, these respondents were collapsed into one category titled 'other'. respondents' level of clinical exposure (as measured by whether they had completed, not started, or partially completed core rotations) differed across class years (p < 0.0001) [ table 2 ]. a majority of ms2s had not yet begun clerkship year (222, 63.6%), while the majority of ms3s had begun the year but not completed it (439, 64.2%). in total there were 606 (36.3%) students who were part-way through clerkship year when the pandemic started. out of these, roughly half of the ms2s had not completed either surgery or internal medicine (69, 55.2%), while about the same proportion of ms3s had completed both (255, 58.1%). the percentage of respondents who were certain about their specialty choice differed significantly across classes; 64.5% (441) of ms3s were certain, as opposed to 16.1% (56) of ms2s and 14.4% (50) of ms1s (p < 0.0001). when asked to indicate up to two specialties they were 'seriously considering', ms3s were considering a broad range of options, with internal medicine (im), pediatrics, and emergency medicine (em) being the most commonly chosen [ table 3 ]. as high as 20.2% (337) of medical students surveyed thought that the pandemic would affect their choice of specialty. there was a significant difference across class years, with 15.2% (53) ms1s, 26.4% (92) ms2s, 23.7% (162) ms3s, and 9.7% (22) of ms4s saying their specialty choice would be affected (p < 0.0001). across all class years, the most common reason chosen was 'i may not have the opportunity to explore my specialty or specialties of interest' (244, 72.4%). the majority of ms3s had not done an elective in their specialty of choice (601, 87.8%). however, on further analysis of ms3s, there was no significant relationship between completion of an elective of interest and certainty in specialty choice. out of those who thought the pandemic would affect their choice of specialty, the second most common reason given was, across all classes, 'i no longer have the ability to bolster my application' (162, 48.1%). when asked what they meant, ms3s who chose this reason were most commonly concerned about 'letters of recommendation' (68, 81.0%) and 'away rotations' (62, 73.8%), whereas ms2s most often chose 'taking board exams' (27, 71 .1%), and ms1s most often chose 'research' (24, 100.0%). when ms3s were asked whether the covid-19 pandemic has made them more likely to take an extra year in medical school before applying to residency, 119 (17.4%) said yes. out of these, the most common reason given was 'it would make me more likely to match to my satisfaction' (79, 66.4%). three hundred and twenty-seven (47.8%) of ms3s and 55 (24.3%) of ms4s were concerned about fulfilling graduation requirements in time. respondents who thought the pandemic would affect their choice of specialty were asked, as a follow-up, what specialties they were 'more likely' and 'less likely' to apply into, respectively. no clearly favored or disfavored specialties emerged. im was the most common answer for 'more likely', while em was the most commonly chosen as 'less likely'. however, em was also the second most common choice put for 'more likely', and im was the second most common choice put for 'less likely'. an analysis using proxy measures for covid-19 severity level near where respondents attended medical school was done. region of the us, number of statewide covid cases, number of statewide covid deaths all had no correlation with the proportion of respondents who thought the pandemic would affect their specialty choice (p = 0.17, 0.25, and 0.21, respectively). when asked how they were spending their time away from the clinics [ table 4 ], 'research' and 'virtual classes through medical school' were the only academic-related options selected by greater than half of respondents. five hundred and nine (30.5%) and 333 (20.0%) indicated involvement in service work or community engagement/organizing, respectively. in the non-academic category, 'hobbies', 'exercise', and 'self care/relaxing' were each selected by greater than half of respondents. our survey also included an optional comment box. relevant comments will be summarized in the following section as qualitative information to help lend nuance to our discussion of the results. this study serves as an early snapshot into medical students' perspectives on their education and careers at a time when the covid-19 pandemic is in full force and clinical rotations remain cancelled. despite substantial curriculum upheaval, only about one-fifth of our respondents indicated that their specialty choice would be affected. most of these students cited concerns about not having time to explore their specialties of interest. medical student specialty choice has been written about prior to the covid-19 pandemic, and studies have suggested many factors can play a role, such as exposure to role models and core rotation experiences [33] . jones et al. found that, as one might expect, the positive predictive value of a medical student's top specialty choice increases from end of first year (ranging from 17% to 60% depending on specialty) through end of third year (ranging from 79% to 95%), suggesting that more exposure to fields of potential interest throughout medical school may result in increased accuracy of selfpredictions. they also found that students' self-reported level of certainty was not correlated with the positive predictive value of their specialty choice [34] . on the other hand, manuel found that there was a significant relationship between first-semester medical students' predictions of whether they would apply into a technique-oriented or person-oriented specialty and their eventual choice [35] . at baseline (pre-pandemic), though they might have an idea of the broad category of career they prefer, ms1s, ms2s, and even ms3s can be inaccurate at predicting their future specialty choice regardless of whether they feel certain. our results found that 9.7% (22) of ms4s thought the pandemic would affect their specialty choice, an odd finding considering that by april, when our survey was distributed, ms4s have generally already matched or not matched into a single specialty of their choice. there are a couple possible explanations for this finding. some ms4s may have chosen not to apply to residency. also, students taking a year out during medical school between 3 rd and 4 th year may have identified themselves as ms4s rather than year out students. in our survey, students who indicated that they did not think their specialty choice would be affected were not given follow-up questions asking about their specific concerns. however, many of them independently brought up some of the same concepts in the optional comment box at the end of the survey. though these respondents did not think their choice would be affected, many were still concerned about their ability to be as competitive in their chosen specialty as they otherwise would have been, and submitted comments about away rotations, letters of recommendation, completing board exams in time, and inadequate clinical preparedness due to virtual coursework being a poor substitute for rotations. literature on virtual learning in medical students prior to the covid-19 pandemic have found that though students feel it augments their education, they do not view it as an adequate replacement for in-person learning [36, 37] . by now, many medical schools have created virtual coursework so that students can continue learning from home [5] [6] [7] [8] [9] 20] . literature on the precise structure and content of coursework being created at different institutions is limited thus far, but given that course development is not centralized on a national level, one can only imagine it varies greatly from school to school in terms of fidelity to a true clinical rotation, level of interactivity, presence of virtual patient contact, or quality of contact with residents and faculty. as such, these curricula likely also vary in terms of how well they address the concerns described in our results, particularly obtaining letters of recommendation, gaining clinical preparedness, and accurately evaluating specialty fit. it is still unclear when medical students across the us will be able to return to clinical rotations [27] . the longer that rotations remain canceled, the more crucial it will become to carefully examine which types of virtual curricula, if any, are effective at addressing medical students' career-related concerns. our results show that students are currently filling their time with a variety of personal and professional endeavors, most notably, research and the previously mentioned virtual coursework. although medical student volunteering and community-based efforts to combat covid-19 have been frequently featured in the news and described in the literature [4, 22, 24, 38, 39] , our data indicate that only about half of students are involved with these endeavors, which may be less than the common perception. the severity of the covid-19 pandemic in the us state in which medical students attended school did not have an effect on their perception of the pandemic's effect on their careers. this is a logical finding; local climate may certainly affect other aspects of students' lives such as magnitude of student-led volunteer efforts [24, 39] , and concern for safety of self and loved ones. however, if medical students have concerns about their education and career choices, these are most likely to stem from the sudden and drastic disruption to their curriculum [11, 18, 20] . furthermore, our analysis of measures of pandemic severity was at the state level. it is possible that a city-level analysis would produce different results, as it is becoming increasingly clear that covid pandemic severity often varies across different cities within states [40, 41] . strengths of this study include a large sample size, a wide geographical distribution, and timely development and distribution. however, several limitations must be discussed as well, particularly the possibility of response bias. students who are willing to take surveys may be a group with different attitudes toward the pandemic and toward career choices than those who are not willing to take surveys. since our survey was also distributed via medical student social media groups, it is also possible that students who regularly engage with these groups on social media may have different traits and perceptions as well. in addition, to maximize anonymity, our survey did not prompt respondents to include their medical school name (merely the state in which they attended medical school); thus, we were not able to track which institutions out of those we contacted actually distributed the survey. our early cross-sectional study did not identify any specialties that were clearly favored or disfavored as a result of the pandemic. future study will be necessary to determine whether the thought processes and concerns revealed in our results manifest as real changes in national residency matching program (nrmp/the match) data, or whether they are transient issues that resolve as students are allowed to resume rotations. in the aforementioned study on medical student specialty choice before and after hurricane katrina (another public health event that disrupted medical education), there were significant changes in specialty choice before and after the event [17] . however, this change was evident over a 10-year study period. a similar long-term analysis of medical student specialty choices before and after the covid-19 pandemic would certainly be a worthwhile study in the future. the covid-19 pandemic has disrupted clinical education at medical schools across the us. our results indicate that though the majority of medical students do not think this disruption will affect their specialty choice, about one in five do. it is impossible to tell at this early stage whether the careerrelated concerns revealed by our findings will manifest as changes in residency match data. what we can conclude is that the longer the covid-19 pandemic prevents medical students from completing clinical rotations, the more important it will become for medical schools to address these concerns in a targeted manner through enhanced virtual coursework and advising support. ethics approval and consent to participate: this project was reviewed and determined to qualify as quality improvement by the university of pennsylvania's institutional review board. no potential conflict of interest was reported by the authors. beatrice c. go http://orcid.org/0000-0001-7025-0329 trump extends social distancing guidelines through end of april -the new york times social distancing. quarantine, and isolation covid-19) and the vslo program the impact of covid-19 on medical student surgical education: implementing extreme pandemic response measures in a widely distributed surgical clerkship experience leveraging technology for remote learning in the era of covid-19 and social distancing: tips and resources for pathology educators and trainees going virtual to support anatomy education: a stop gap in the midst of the covid-19 pandemic medical students' perceptions and an anatomy teacher's personal experience using an e-learning platform for tutorials during the covid-19 crisis outpatient clinical student rotation without direct patient contact: leveraging care transitions to benefit students and patients. aamc icollaborative medical students are not essential workers: examining institutional responsibility during the covid-19 pandemic our education mx. our concerns: medical student education impact due to covid-19 the challenges of "continuing medical education" in a pandemic era medical school on bypass during the sars outbreak sars and its effect on medical education in hong kong avoiding student infection during a middle east respiratory syndrome (mers) outbreak: a single medical school experience baylor college of medicine's support of tulane university school of medicine following hurricane katrina snap shots: the effect of hurricane katrina on medical student career choice. acad psychiatry medical student education in the time of covid-19 the change to pass/fail scoring for step 1 in the context of covid-19 no classrooms, no clinics: medical education during a pandemic | aamc the impact of covid-19 on medical education bright lights in a dark time: medical students step up to help out | aamc itching to get back in": medical students graduate early to join the fight | aamc medical student mobilization during a crisis: lessons from a covid-19 medical student response team medical student involvement in the covid-19 response let us help'-why senior medical students are the next step in battling the covid-19 pandemic guidance on medical students' participation in direct patient contact activities research electronic data capture (redcap) -a metadata-driven methodology and workflow process for providing translational research informatics support the redcap consortium: building an international community of software partners the impact of clerkships on students' specialty preferences: what do undergraduates learn for their profession? positive predictive value of medical student specialty choices person-oriented versus technique-oriented specialties: early preferences and eventual choice the impact of e-learning in medical education the role of e-learning in medical education senior medical students in the covid-19 response: an opportunity to be proactive fathy r coronavirus is keeping philly medical students out of hospitals, but we are still contributing l opinion. the inquirer pennsylvania department of health key: cord-026595-imn2jxcu authors: qamar, mariam khan; shaikh, babar tasneem; afzal, aamir title: what do the dental students know about infection control? a cross-sectional study in a teaching hospital, rawalpindi, pakistan date: 2020-06-01 journal: biomed res int doi: 10.1155/2020/3413087 sha: doc_id: 26595 cord_uid: imn2jxcu background: dental students are exposed to various infections and infective sources during their training, and on this aspect, their level of knowledge is suboptimal and practices are risky. therefore, improving their knowledge and practices would contribute significantly to infection control. objective: to ascertain the level of understanding of senior dental students regarding the infection control in the dental practice. methods: a cross-sectional study was conducted among dental students (3rd year and 4th year) of the foundation university dental college, pakistan. the sample consisted of 100 third year dental students and 88 fourth year students. a self-administrated questionnaire was used for data collection which consisted of fourteen close-ended items. frequencies of knowledge, attitudes, and practice were calculated separately by using spss 21.0 software. results: almost half of the students would not use any antiseptic for sterilizing their hands, and only two-third would ask their patient to use an oral mouth rinse before starting the treatment. many students did not the optimal temperature of the autoclave for sterilization of the instruments. only one-third would wear the personal protective equipment during a procedure. around one-third of the study participants reported that ineffective sterilization during clinical practice can transmit infection from one patient to another. conclusion: knowledge on infection control among the dental students is though weak, practices are not as per standards but attitudes are positive and encouraging for taking steps and complying with measures on infection control. dental students are one of the dental health care professionals who are at a high risk of exposure to infections because of their direct contact with the patients, infected instruments, and hospital environment. cross infection is a major concern to all the dental health care professionals. it is defined as the transmission of infection between the staff and the patient within the hospital environment [1] . among these, the most serious oral infections are caused by bacteria that colonize in the oral cavity including mycobacterium tuberculosis, influenza virus, and streptococci. students are equally vulnerable to cross infections that are caused by the hepatitis b virus (hbv), hepatitis c virus (hcv), and other viruses [2] . moreover, they are also at a risk of percutaneous occupational injuries and eye exposure while treating the patients [3] . only through strict safety precautions and implementation of guidelines for infection control, we can prevent these mishaps from happening. the center for disease control and prevention (cdc) in the united states has updated its guidelines on infection control in dental settings. the aim of these guidelines is to ensure a safe working environment to prevent the transmission of nosocomial and occupational infections among the dental health professionals [4, 5] . dental colleges are responsible for providing proper training to their students to ensure safety of the patients, implementing infection control measures, and establishing a safe working environment [6] . several studies have been conducted to assess the practices and knowledge of dental students and have demonstrated poor compliance of the students to infection control measures. a study conducted in india to assess the infection control practices among dental students showed that only one-tenth of the respondents adhere to the infection control measures [1] . similar studies have been conducted worldwide to investigate the knowledge and practices of dental students on infection control [6] [7] [8] [9] [10] [11] [12] [13] [14] , and a general consensus is that students need awareness and must be protected in the unsafe environment. fauji foundation hospital in rawalpindi city houses a huge number of medical and dental students for educational and training purposes. this hospital is a busy tertiary health facility catering to mostly ex-army servicemen and their families. it has a fully equipped dental outdoor clinic where the dental undergraduate students come on daily basis for observations, education, hands-on training, or apprenticeship. nevertheless, there is paucity of data with regard to infection control practices among dental students within the hospital environment in pakistan. for this purpose, it is imperative to ascertain the understanding of the senior dental students who have attended the module of infection prevention and control during their study course. this study will help better understand the gaps and deficiencies in the dental college curriculum and will sensitize and educate the future dental surgeons in adopting the necessary infection prevention practices. a cross-sectional study was conducted among dental students (3 rd year and 4 th year) attending the foundation university dental college, pakistan. the duration of study was two months from 14 th july 2019 to 14 th september 2019. the sample size (n = 188) was calculated by using the who sample size calculator with the following parameters: confidence level = 95%, anticipated population proportion = 95:5% [6] , and absolute precision required = 3%. a total of 228 dental students studying in the college included 121 third year and 107 fourth year students. out of these approaches, 82.6% (n = 100) students from the third year and 82.2% (n = 88) students from the fourth year gave a complete response and therefore were included in the study. as per the exclusion criteria, the students of first and second year were excluded from the study because training in infection control is provided in the dentistry curriculum of third year and fourth year. written informed consent was taken from all the participants of the study. the institutional review board of the hospital granted the requisite permission to conduct the study. a total of 188 dental students completed the questionnaire comprising of 14 items. there were 6 questions related to "knowledge," 5 questions related to practice, and 3 questions were based on "attitude." a self-administered questionnaire developed by singh et al. [6] was distributed to the dental students. the consistency of the questionnaire was assessed using cronbach's alpha (α = 381). each student was given fifteen minutes to fill the questionnaire in silence without discussing with each other. they were not asked to put their names on the forms, hence keeping the anonymity intact. data was entered and analyzed on spss v.21. descriptive statistics was done in terms of mean and standard deviation. frequency and percentage were calculated for qualitative variables. independent sample t-test was used to compare the mean score of knowledge, attitude, and practice among both groups. p value ≤ 0.05 was taken as a level of significance. written verbal consent was obtained from all the respondents. study received approval from the ethics and research committee of the foundation university, rawalpindi, dated january 2, 2019. a total of 188 dental students responded to our questionnaire completely and hence included in the study results. hence, the study population comprised of 28 male and 168 female students. table 1 shows the analysis of cross tabs between the independent (gender and class of study) and dependent variables which yielded no significant statistical association (i.e., p > 0:05). table 2 shows that although a majority (94%) reported washing their hands before and after examination of the patient, yet only half of them (49%) would use an antiseptic solution for washing their hands. there were 64% participants who preferred patients to have an oral mouth rinse before commencement of any treatment procedure. the majority (98%) of the students considered isolation as an important measure for the infection control. with regard to vaccination, 85% got vaccinated for hepatitis b, 45% for tetanus, and only 15% of the students were vaccinated for tuberculosis, but surprisingly, 18% have not received any vaccine at all. on the subject of sterilization of instruments, 95% reported use of autoclave for the purpose and only 68% reported using the same for 15 minutes (optimal time) and only 60% thought that 120 degrees is the required temperature on which the autoclave should be used. around 80% study participants considered that hepatitis b has the highest rate of transmission via saliva. in case of direct blood contact with a hiv patient, 62% would opt for anti-hiv immunoglobulins. during the work at a dental surgery, the use of face mask and gloves as an infection control measure was practiced by 38% while 32% would wear an eye protector, and only one-third of them (29%) would wear all of them. after using a pair of gloves on the patient, 57% would dispose them off but 43% think that they can reuse them after washing. only 37% reported that ineffective sterilization during clinical practice can transmit infection from one patient to another, although a large majority (98%) believe that besides instruments, disinfecting the dental chair, clinic, and doctor's office is necessary. in table 3 , it has been observed that there is a significant positive correlation between the mean score of knowledge and attitude r = 0:781, p ≤ 0:05; however, there was no differences hands are the foremost significant reservoir for many pathogens. handwashing is therefore considered as an effective method of prevention of infection [15] . in the current study, the majority of the students described that hands should be washed and the students described plain soap and antiseptic solution before and after the examination as a preferred method. similar results were seen in a study conducted in germany where most of the trainee dentists washed their hands before and after the procedure [8] . nonetheless, there are studies that show poor compliance of the students with handwashing [9] . a study conducted in yemen reported only 43% of the students adhere to handwashing [9] . in the present study, most of the dental students were vaccinated with hepatitis b vaccine which was in concurrence with the other studies [12] [13] [14] . this rate is much higher than the study conducted in india showing that only 38% of the students were vaccinated against hepatitis b [6] . the dental students had good knowledge of autoclaving and sterilization similar to other studies [6, 9] . wearing of face mask and eye protection was observed in only one-third of the students. similar results were recorded in other studies [9] . however, a good majority of the dental students in our study thought that hepatitis b infection and not tuberculosis is transmitted through saliva, which was quite surprising. another surprising finding in our study was that two-thirds of the students considered that ineffective sterilization during clinical practice cannot transmit infection from one patient to another. this highlights the lack of awareness of the concept of cross infection. the majority of the students did believe that disinfection of dental chairs and the office of the dentist is also required. this concurs with the findings of other studies included in our literature review [2, 3] . importance of continuous-based infection control lectures and training could help in raising the level of knowledge regarding the subject [16] . we recommend that every hospital needs to set up infection control measures in accordance with the guidelines. for both the students and faculty of a hospital to improve their practice and knowledge of infection control, hospitals need to address the need for quality assurance and implementation of infection control measures. it is highly recommended that reinforcement training of the students through periodic workshops on infection precaution and control should be planned to highlight the significance of infection control [17] . another recommendation is that of vaccination, especially hepatitis b, which should be mandatory for all medical and dental students prior to their admission. the strength of this study is that it was conducted with a senior group of dental students (3 rd and 4 th year) who have studied the module on infection prevention and control. the use of a self-administered questionnaire itself is one of the limitations of this study; therefore, objectivity in responding to certain questions might have been compromised. results could be generalized with caution because similar group of students in a private hospital setting with better infection control and prevention standard operating procedures might exhibit different behaviors and practices. we concluded that there was lack of correct knowledge on infection control among the dental students, yet they showed a positive attitude towards infection control measures. however, a greater compliance and supervision would be needed. further studies in similar settings and utilizing the mix methods with qualitative enquiry would be beneficial to understand certain careless attitudes and behaviors among dental students. available with first author. the authors declare that they are no conflict of interests. compliance with infection control programs in private dental clinics in jordan attitudes and practices of infection control among senior dental students at college of dentistry, university of sharjah in the united arab emirates infection control: knowledge and compliance among saudi undergraduate dental students updated cdc recommendations for the management of hepatitis b virus-infected health-care providers and students guidelines for infection control in dental health-care settings-2003 knowledge, attitudes, and practice regarding infection control measures among dental students in central india cross-infection control in malaysian dental practice rate of compliance with hand hygiene by dental healthcare personnel (dhcp) within a dentistry healthcare first aid facility knowledge, attitudes, and practice of infection control among dental students at sana'a university knowledge, attitude and practices about hepatitis b and infection control measures among dental students in patiala infection control measures in private dental clinics in lebanon knowledge, attitude, and practice of needle stick and sharps injuries among dental professionals of bangalore, india awareness of droplet and airborne isolation precautions among dental health professionals during the outbreak of corona virus infection in riyadh city, saudi arabia knowledge, attitude and practice of infection control measures among dental practitioners in public setup of karachi, pakistan: cross-sectional survey compliance with infection control practices in a university hospital dental clinic knowledge, attitude and compliance of infection control guidelines among dental faculty members and students in ksu evaluation of final-year turkish dental students' knowledge, attitude, and self-perceived competency towards preventive dentistry appreciation is hereby extended to the students of foundation medical and dental university for participating in our study. mkq, bts, and aa were all responsible for defining the initial research question and developing the protocol and study design. mkq and aa were involved in the collection of data and took lead in writing the manuscript. bts contributed to the completion of the manuscript and has critically revised it. all authors read and approved the final manuscript. key: cord-276976-ybsk1r79 authors: jiang, qinxu; yuen, mantak; horta, hugo title: factors influencing life satisfaction of international students in mainland china date: 2020-08-04 journal: int j adv couns doi: 10.1007/s10447-020-09409-7 sha: doc_id: 276976 cord_uid: ybsk1r79 china is advancing in its bid to internationalize higher education, but little is known to date about the life satisfaction of overseas students in that country. life satisfaction can be understood as the extent to which individuals’ experiences in a host country help them achieve their personal goals and satisfy their expectations regarding acculturation. this paper examines past research, especially international literature, to provide an overview of factors that may influence the life satisfaction of international students in countries other than their own, and looks particularly at those matters that may also affect overseas students in china. comparisons are made between variables identified in general and those in asia, and specifically china. the findings have implications for policy-makers and university practitioners seeking to improve educational quality and counseling support services for international students. in recent years, china has grown in prominence on the world stage. this includes its position as an emergent educational hub for international students (wen and hu 2019) . although many chinese students undertake studies in north america, western europe and oceania, china has also begun attracting increasing numbers of students from overseas. this phenomenon can be traced to the development of higher education in the country, with several chinese universities now placing highly in world rankings, and to the country's growing economic prominence (jiani 2017) . data attract international students, and government initiatives around these matters. following this section, the international literature on matters related to life satisfaction among overseas students in western contexts and in other emerging asian countries is reviewed. similarities and differences are identified between observations in other countries and those in china. factors that might predict the life satisfaction of international students in china are discussed. in this context, a general description of counseling services available in higher education institutions in china is provided, followed by suggestions for developing such services that address the specific needs of international students. in the final section, future possible research directions are proposed. the ease of mobility of international students across borders has many advantages to both host countries and international students. the internationalization of higher education also yields appreciable economic benefits for host institutions, and this in turn can support research and increase resources and staffing (horta 2009) . a less tangible but no less valuable benefit is the positive interaction and cultural exchange that typically occurs between international students and domestic students (ammigan 2019) . a deeper understanding of different cultures can facilitate cooperation, tolerance and international peace (gebregergis et al. 2019; sam 2001; taušová et al. 2019) . residing and studying in a foreign country enables international students to have rewarding experiences that contribute to increased confidence and self-growth (taušová et al. 2019) , although the mobility of international students tends to be motivated more by financial goals than by deliberate cultural exchange (feng and horta 2020 ). an international student may acquire foreign language competency, multicultural experiences and enriched cultural literacy, thereby increasing their human capital of knowledge and skills (wen and hu 2019) , as well as the scope of their employability (nerlich et al. 2018 ). however, overseas students also face numerous adaptation challenges related to their social, cultural, and learning experiences, and not all their experiences are necessarily positive (bamber 2014) . china entered the globalization movement of higher education at a later stage than the west, and although the country is increasingly attracting international students, it faces stiff competition from more established host countries such as the u.s., the u.k., australia and new zealand, as well as from neighboring countries like south korea, japan and singapore. china's success in attracting overseas students can be attributed in major part to its affordable tuition fees and living expenses, which are lower than those in most western countries (dervin et al. 2018) . china also has accessible scholarships, government grants and supportive policies for employment opportunities after graduation (dervin et al. 2018) . the opportunity to acquire chinese language competence is also an attraction because of its value in the global business world (ding 2016) . other factors that potential students take into account include china's political stability, its less strict admission policies, the economic boom and their own interest in chinese culture and history (jiani 2017; liu et al. 2013; wen and hu 2019) . the internationalization of higher education and the recruitment of overseas students has been high on the chinese government's agenda. the moe formulated its program for study in china to strengthen educational exchange and cooperation between china and other countries. it aimed to develop study opportunities that would be accessible by international students. the plan set a target for china to become the asian country with the largest number of foreign students by 2020; this number is anticipated to reach 500,000. 3 this was reinforced by the launching in 2013 of the one belt one road initiative (obor, centered on a silk road economic belt and a maritime silk road), which further opened china up to the rest of the world. critical importance was attached to strengthening educational cooperation and exchange with several countries in asia and in central and eastern europe (lu and tian 2018) . the obor initiative is suggestive of how international education development can be seen as an exercise of soft power: it does not focus on revenue benefits (although it may generate a revenue stream), but rather is part of china's global strategy to influence attitudes overseas and increase awareness of china's culture and its economic and creative growth (wen and hu 2019; yang 2015) . in this context, it is relevant to note that international recruitment mostly targets students in developing countries. these students will likely become part of the governing elite in their countries and will potentially be more sensitive to chinese viewpoints and interests after their stay in china, given their acquired knowledge of the chinese language and china's society, culture, history and politics (yang 2015, p. 25) . by april 2017, china had signed 45 cooperation agreements with countries along the "belt and road" and had signed mutual recognition agreements for academic degrees with 24 countries (china youth network 2017). although these initiatives are important, future growth in the number of international students in china is bound to be more dependent on growing a reputation for the quality of the education and student experiences provided by chinese universities. the fact that such students have to pay fees for their studies means they will always weigh whether they are getting value for money. host nations must therefore be mindful of the need to meet their expectations (karaman and watson 2017) . life satisfaction is defined as "a global assessment of a person's quality of life according to his [or her] chosen criteria" (shin & johnson, 1978 , cited in diener et al. 1985 . it is also considered to be "a generic concept representing an individual's contentment with his or her life"-that is, the level of an individual's subjective appraisal as to whether his or her aspirations and goals have been achieved (jacobsson & lexell, 2013 , cited in nilsson 2015 . the life satisfaction of international students can be understood as the extent to which their experiences in the host country satisfy their expectations in terms of personal goals being realized and appropriate acculturation taking place. in other words, it is a subjective evaluation of a student's perceived level of satisfaction and adjustment to a new sociocultural environment. throughout the literature, international students' assessment of life satisfaction is closely linked to their acculturation experiences in the host country (zhang and goodson 2011) . for example, they may encounter unexpected challenges and adjustment problems in an unfamiliar culture, and may experience mental or psychological stress as a consequence. acculturation is defined as "the dual process of cultural and psychological change that takes place as a result of contact between two or more cultural groups and their individual members" (berry 2005, p. 698) . international students and members of the host culture they connect with undergo acculturative changes through contact, adaptation and accommodation (smith and khawaja 2011; zhang and goodson 2011) . psychological adaptation refers to affective responses (sense of well-being, confidence and self-esteem) as well as physical well-being (ward et al., 2001 , cited in smith & khawaja, 2011 . sociocultural adaptation is "based on behavioral responses related to how effectively an individual links in to the new society" (ward et al., 2001 , cited in smith & khawaja, 2011 . having a clear understanding of international students' acculturation processes is helpful for understanding the factors that may predict their degree of life satisfaction in a new environment. a number of quantitative and qualitative studies have examined acculturation among international students. in their review of 94 studies on acculturation that had been published before 2011, smith and khawaja (2011) thoroughly investigated the acculturation experiences of international students, although almost all the studies they included were from western countries. they identified five of the most common stressors facing such students: (i) language difficulties, (ii) educational factors, (iii) sociocultural differences, (iv) discrimination, and (v) practical problems, such as financial difficulties and accommodation or transportation issues. in western host nations where english is the native language, a low level of english proficiency naturally affects international students' academic performance and social adjustment (smith and khawaja 2011) . it can be deduced from their review that any similar setting in which a student has to cope with a language other than their first language is challenging. the mental strain brought about by unsatisfactory educational experiences or difficulties socializing and making friends can add to the stress. a mismatch between a student's expectations regarding the ease or difficulty of academic courses and the reality of such contributes significantly to stress no matter the circumstances, but this stress can be compounded when it happens in a foreign country (smith and khawaja 2011) . access to counseling services can be important in helping students handle these stressors, but the problem can be exacerbated if the quality of the counseling and support services offered by the host educational institution is poor (smith and khawaja 2011) . in such circumstances, acculturative stress can become debilitating and can significantly undermine a student's level of life satisfaction. in their review, smith and khawaja (2011) identified sociocultural stressors associated with overseas students' attempts to build new social networks in their host country. they found that international students' individual characteristics, such as their personality or attachment style, affected their psychosocial adaptation (smith and khawaja 2011) . a possible example of this finding is that asian students from a collectivist culture may find it hard to accept western social norms and attitudes that seem to conflict with their own. the same kinds of difficulties can occur in reverse, where a student from an individualized culture must cope in a collectivist society. this can be an obstacle to forming friendships, and if friendships are less accessible in the host community, loneliness may occur and be a negative predictor for life satisfaction (salimi 2011; yalçin 2011) . discrimination against international students, sometimes evident through prejudice, verbal insults, physical assault or hostility, has also been recognized to be a significant acculturative stressor. international students were found to be more vulnerable to discrimination, in both covert and overt forms, on and off campus (smith and khawaja 2011) . discrimination not only discourages international students from making friends with local people but can also affect their mental health and wellbeing. in extreme cases, this can also cause a student to withdraw from study and return home. other practical stressors that international students were found to be at risk of experiencing included financial, accommodation and transportation issues (smith and khawaja 2011) . naturally, however, data on the relevance and severity of these more practical factors may vary from country to country depending on conditions in different settings. zhang and goodson (2011) reviewed 64 peer-reviewed american journal articles published between 1990 and 2009 to investigate predictors of international students' psychological adjustment to life in the u.s. consistent with smith and khawaja's (2011) findings, factors such as general stress, social support, language proficiency, the characteristics of the origin country, social connectedness with locals, gender, self-efficacy, personality and length of stay in the host location were most frequently reported as influences on adjustment. it should be noted that the reviews by smith and khawaja and by zhang and goodson were published in 2011, so more recent data are needed, as the past nine years have seen rapid growth in international student mobility. mesidor and sly (2016) contributed a more recent review (albeit drawing also only on u.s. sources) on the variables that contribute to international students' adjustment. unsurprisingly, factors enumerated in their review overlapped with those identified by smith and khawaja (2011) and zhang and goodson (2011) . they identified factors associated with academic issues, language barriers, sociocultural issues, financial problems, and lack of social support, but mesidor and sly (2016) also added more personal factors: emotional intelligence, resilience, locus of control, self-efficacy, coping styles and engagement in leisure activities. an investigation of international students' satisfaction in life and its predictors was carried out by sam (2001) at the university of bergen, norway, where universities widely use english as the medium of instruction. his paper provided a comprehensive review of earlier findings that focused on seven aspects that contribute to satisfaction in life: language proficiency, demographic characteristics, financial situations, academic standards, social activities and social relationships, and the clarity of information received prior to enrollment. in addition, in sam's (2001) study, students from europe and north america studying in norway appeared to have a higher level of satisfaction than their counterparts from asia and africa. this may have been a reflection of students' english language proficiency and the tendency of asian and african students on campus to experience more obvious discrimination and to have fewer friends. in addition, some of the latter students may have received inadequate information about courses prior to enrolment, leading to stress when faced with the reality of study demands. sam (2001) recognized that, overall, the factors identified, together with difficult financial situations, seriously influenced the life satisfaction of the international students involved. in the same study, there were also apparent differences between international students from developing countries and those from developed countries (sam 2001) . for example, interestingly, financial issues appeared to be of greater concern to students from developed countries than to those from developing countries. one possible reason for this difference, according to the researcher, may be that the latter group were typically recipients of financial support from their governments, while the others were not. these differences highlight the fact that influences on the life satisfaction of international students may vary considerably from one geographical, economic, or cultural setting to another. studies since 2011 in both western and non-western contexts have tended to confirm the influence of all the factors previously identified. language, educational experience, financial insecurity, discrimination, social integration, stress, living expenses, and accommodation continue to be common influential factors. for example, ammigan (2019) pointed out that the main factors that predict international students' institutional satisfaction in the u.s., australia, and the u.k. include social networking, work opportunities, suitable accommodation, and compatibility with local culture. more recently, the following influences have gained attention in the literature, with seemingly more sophisticated variables being given attention: cultural distance/proximity, individual coping competence, campus support services, prior sojourn experience, and predeparture preparedness all seem to be particularly important for international students' adjustments (alemu and cordier 2017; ammigan and jones 2018; hennings and tanabe 2018; kosheleva et al. 2015; taušová et al. 2019 ). taušová et al. (2019), in their netherlands-based study of psychological adjustment among international students from 62 countries, investigated four factors of perceived cultural distance: (i) "subjectively perceived discrepancy between cultures in areas such as values, habits, norms, religion and leisure" (p. 13), (ii) perceived growth initiative, which refers to "striving to grow and achieve positive changes in personally important life domains" (p. 13), (iii) english and dutch language proficiency, and (iv) length of residence. as with norway, english rather than dutch is the language most used both in academic and social spheres in the netherlands. another study focused on singapore (nasirudeen et al. 2014 ), a multicultural society where the main language spoken is english. international students from neighboring malaysia were found to have less stress and to experience less discrimination than students from china and myanmar. this was mainly because the students from malaysia had better english competence and shared similar cultural norms to the host nation. this finding reinforces the importance of having adequate proficiency in the language used in the higher education institution of any host country. language proficiency is clearly a major influence on how quickly a foreign student is likely to become acculturated and to achieve a positive level of life satisfaction (nasirudeen et al. 2014) . in a large-scale study of 45,701 international students from 96 universities in australia, the u.k. and the u.s. undertaken by ammigan (2019), the english proficiency of the lecturers was found to be very important for overcoming students' stress associated with the demands of study. in their study of international students in korean universities, alemu and cordier (2017) found that international students from east asian countries tended to be more satisfied with their experiences than other international students, largely because of their greater cultural proximity to the host nation. based on this research, it can be deduced that foreign students who decide to study overseas are likely to encounter the most difficulties if they opt for a country with a different language and very different cultural norms from their home country. research studies have paid less attention to students' coping competence as a factor influencing how they adapt to a new environment. smith and khawaja (2011) acknowledged that most acculturation models recognize that an individual's coping mechanisms play a key role in how they adjust to and integrate into an unfamiliar host culture. a comparative study of determinants of life satisfaction among chinese students in finland and russia indicated that those who were more able to cope with change and were open to new experiences tended to fit into the new culture faster, and with greater confidence and satisfaction (svergun 2016) . saha and karpinski (2016) found that international students who coped by using social media platforms such as facebook, linkedin, twitter, qq, or skype to seek social support from families and friends at home also tended to have higher levels of life satisfaction, which contributed to their overall academic performance. it also appears that marital status can influence how easily a foreign student adjusts to a new country. for example, bhandari (2012) found that married nepalese participants who lived with their spouse in south korea experienced less acculturative stress than did single students. obviously, having such close support and shared experiences is helpful psychologically for international students, and it can also be claimed that couples might be more likely to become engaged with locals through social invitations extended to them as a couple. it is not yet clear whether students' age, gender, and length of residence are linked to acculturative stress, so more research is merited in this area. for example, sam (2001) found that female international students were less adaptive than males and that age and length of stay were positively correlated with students' adaptation abroad. other studies have shown conflicting results in regard to the impact of these factors (nasirudeen et al. 2014; salimi 2011; yalçin 2011) . the degree and quality of campus support services have been recognized in several studies to influence the life satisfaction of international students; these services can determine how well students cope while studying abroad. a quantitative study of the experiences of more than 45,000 international students from 96 institutions across australia, the u.k. and the u.s. revealed that support from the international office, services from the financial department, and assistance from tutoring services had a significant influence on international students' levels of satisfaction (ammigan and jones 2018) . similarly, in their study involving international students at an american university, cho and yu (2015) found a positive relationship between university support, a reduction of psychological stress, and an increase in life satisfaction. the large-scale survey of 45,000 international students from 96 different institutions in australia, the u.s. and the u.k. by ammigan (2019) also revealed that international students' satisfaction with the level of support services (especially related to learning) influenced the decisions of prospective applicants. effective advising (e.g., personal counseling and preparing potential students for their choice of overseas institution) was also found to be important for international students' satisfaction, engagement, persistence, retention, and academic performance (bista 2015; mamiseishvili 2012; zhang 2018) . it is clear that counselors also play an integral role in supporting international students' academic success and well-being (arthur 2017) . counseling can help international students address their immediate concerns, overcome problems, and thus strengthen their adjustment and decision-making abilities (arthur 2017 ). banjong's (2015) u.s. study found that amid the challenges of language, loneliness and financial difficulties, seeking help from a counseling center helped improve international students' academic success. one aspect of "fitting in" to a new educational environment is being able to use all available resources that are provided. library services, for example, are a core resource used by international students upon arrival to transition into the new academic realm, with libraries being critical for engaging with academic content, which is central to the purpose of students being in the host nation. it should be standard procedure for host universities to help international students develop required scholastic skills, such as searching for information online and in print (hughes et al. 2018; toner 2019) . students' prior sojourn experiences and pre-departure preparedness have been found to positively relate to adjustment and acculturation (alemu and cordier 2017; gebregergis et al. 2019; morrell et al. 2013; park and rubin 2012) . international students who have traveled or worked in other foreign countries are likely to have more realistic expectations about the challenges and difficulties they might experience. typically, they are better able to activate adaptation skills readily and experience less stress within a new environment than those with no previous international experience. pre-departure preparedness in terms of information provided by the host institution has also been found to influence how easily an international student adapts to a new study environment (madden-dent and laden 2016) . this pre-preparation information from host country settings typically includes not only important educational details about academic matters but also information about social life, accommodation, financial budgeting, and the availability of counseling and learning support services (alemu and cordier 2017) . the provision of these resources by the educational institution conveys a sensitivity to the challenges that the incoming international students will be facing, and reassures students that such support will be available to them. in summary, the above review of mainly western research identifies several important factors that affect the life satisfaction of international students in general. the review also reveals that there may be differences in the relative importance of these factors according to the characteristics of the host countries and the students' countries of origin. specific regional elements, therefore, have to be taken into account to avoid the risk of overgeneralizing the set of influences that determine international students' life satisfaction in any new study environment. ding (2016) has pointed out that western countries such as germany, new zealand, the u.k. and australia have conducted surveys of international students to better understand their experiences of living and studying in the host country concerned. the results obtained have then been analyzed with the aim of improving educational services and the experiences of the students. in contrast, fewer studies have investigated the life satisfaction of international students in china, where these types of surveys have not yet been a major priority. instead, the work of attracting students has been emphasized over the evaluation of course quality, support services, and levels of student satisfaction. the available survey research of this type in china is four large-scale questionnaire surveys conducted in shanghai (ding 2010 (ding , 2016 and beijing (wen et al. 2013 (wen et al. , 2018 about international students' experiences and levels of satisfaction. these studies identified several factors that were closely related to the life satisfaction of international students studying in mainland china. one limitation is that these investigations were carried out at universities in the two biggest and most metropolitan cities in china, where education and support services are believed to be more advanced than in other chinese cities. in addition, the reality is that there are many more international students studying in china at educational institutions in what can be regarded as second-tier or third-tier cities. therefore, to obtain more in-depth information about life satisfaction of international students, it is also necessary to review studies conducted in these smaller cities. in larger cities, the questionable quality of education seems to be the most frequently raised factor contributing to international students' dissatisfaction (ding 2010 (ding , 2016 sun et al. 2009; tian and lowe 2018; wen et al. 2013 wen et al. , 2018 . according to ding (2010) , only 32.3% of 457 international students felt "satisfied" or "very satisfied" with the teaching quality they experienced, 25.0% felt "unsatisfied" or "very unsatisfied," and 42.8% perceived the teaching quality to be "average." the explanations given for these ratings were largely associated with instructors' teaching methods, which were perceived to be rigid spoon-feeding that required too much rote learning and lacked interactions between teachers and students. these perceptions often led to low class attendance rates among international students. it is possible that international students' levels of satisfaction could have been even lower if not for the fact that most international students in china originate from asia, where similar types of learning (e.g., rote learning) are also prevalent. this potentially explains why the greater proportion of international students chose the option "average" in regard to teaching quality. in a more recent study, ding (2016) conducted a mixed-methods study involving a question-based survey and individual interviews with international students studying in universities in shanghai. the survey found that 67.1% of 1,892 international students at 28 universities in shanghai reported they were satisfied with studying and living in shanghai, and 59.2% of them were willing to recommend studying in china to their social network. however, less than half of the students reported being satisfied with the education they received. class sizes, outdated teaching materials, limited interaction with teachers in class, and unsatisfactory teaching methods were the most frequent targets of their complaints (ding 2016) . for instance, comments were made that teachers were more inclined to lecture and have students memorize material than to encourage self-motivated learning. wen et al. (2013) found that international students complained most about unreasonable curricula and obsolete teaching methods. for example, some courses were too researchoriented to be useful for practical work, or were arranged and structured only to comply with graduation requirements. limited teacher-student interaction was another target of complaint, as were the lack of english used in teaching processes and the limited provision of technical support. for instance, most of the postgraduate seminars and tutorials were conducted in chinese, barring international students with limited chinese proficiency from participating. in a more recent study, wen et al. (2018) conducted a large-scale survey among 1,674 international students enrolled in universities in beijing to investigate the quality of their educational experience. the results indicated that, overall, the international students were satisfied with their learning experience in china, with 83% of them feeling "very satisfied" or "satisfied." yet the researchers also received reports from international students studying in english-medium programs that faculty members teaching the courses were not sufficiently proficient in english. students opined that these teachers were not motivated to invest their time in improving their english because of the greater value given at their university to research rather than to teaching and learning (wen et al. 2018) . similarly, tian and lowe (2018) also found that international medical students thought their teachers had expertise in subject knowledge but were weak in english language proficiency, hindering them from giving clear explanations and answering questions. their teaching style typically involved simply reading slides from overhead projectors. international students considered this ineffective, and this led to dissatisfaction and frustration (and even at times protest) among them. overall, regarding program quality, yang (2018) suggested that chinese institutions were often attracted to the prestige of "internationalization" without the level of commitment and resources needed to strengthen their programs. the second domain about which international students in china tended to report less satisfaction was campus support services. to begin with, international students noted a scarcity of adequate and transparent information (ding 2010; tian and lowe 2018) . they complained, for instance, that the english websites of institutions often failed to reasonably meet students' information needs, and that administrative services sometimes failed to promptly inform them of changes to arrangements, preventing them from making adequate preparations. similarly, tian and lowe (2018) found that international students thought that information on university websites was insufficiently clear. some students also reported that their local agencies at home had provided them with inaccurate information before they reached their host university in xi'an. apart from perceived problems with information and communication, other support services were also perceived to be inadequate for international students' actual needs (ding 2016; tian and lowe 2018; wen et al. 2013; zhou et al. 2018) . in one study, more than 30% of international students reported that they did not receive assistance with airport pick-ups, opening a bank account, or finding an apartment to rent (ding 2016) . many international students reported feeling disappointed by the lack of activities on campus because their desire to interact socially with chinese students could not be fulfilled (tian and lowe 2018) . in terms of guidance for seeking employment, there was a lack of information specifically for international students who desired to work in china after graduation (wen et al. 2013) . in one study, most students found that the career planning help from the campus employment guidance office was not open to international students (wen et al. 2013) . zhou et al. (2018) undertook a focused investigation on the extension of university library services to meet the needs of international students in china. in-depth interviews with 23 international students at a wuhan university revealed their dissatisfaction with the library services. five recommendations emerged from the interviews. first, it was thought to be essential to build a "bilingual library environment" with regard to library signage, management regulations, rules and restrictions, and website design. next, it was recommended that high priority be given to increasing librarians' english skills and cultural sensitivity. third, it was recommended that collaborative relationships be established between the library and the school of international education. fourth, it was considered necessary to improve library space allocation. finally, there was seen a need to develop diversified library collections. from this study, it can be inferred that tailoring library services to international students in mainland china still has a long way to go. accommodation appears to be another salient issue among international students studying in china (tian and lowe 2018; wen et al. 2013) . in beijing, survey results indicated a relatively high percentage (77%) of international students feeling "satisfied" with or "very satisfied" with accommodation, but this figure was below an international standard of 86% (wen et al. 2013) . international students complained about a wide range of problems, such as slow internet speed, a hot water supply shortage, old dormitory facilities, limited visitor time, unprofessional attitudes of dormitory staff, a shortage of single rooms, fees for internet and electricity use, and the separation of international students' accommodation from that of local students (tian and lowe 2018; wen et al. 2013) . international students also expressed dismay with the quality and efficiency of administrators, with some administrators appearing to them to be unfriendly and unhelpful. there also seemed to be a lack of collaboration between departments, with international students finding it difficult to find help solving their problems, which often cut across several areas of responsibility (ding 2010) . international students in china usually have separate coursework and accommodation from chinese peers. in the research, this separation was found to hinder international students from connecting with and getting to know chinese students, thus limiting their studies and quality of life (ding 2016; tian and lowe 2018; wen et al. 2013) . interestingly, chinese students were at times perceived by international students to be unfriendly (ding 2016) . this observation might be attributed to the degree to which chinese students are typically preoccupied with their own studies and their lack of motivation for befriending foreigners. ding (2010) found that international students often wished to do part-time jobs to support themselves because most were relying on parents or relatives for their expenses. even students on scholarship sometimes found their funding insufficient to cover both study and living expenses. sun et al. (2009) noted that some international students grappled with tuition and accommodation fees and tended to experience more anxiety and insecurity when a part-time work permit could not be obtained, with this often being restricted by chinese law. wen et al. (2013) found that international students were more satisfied with tuition fees, living expenses and scholarships in beijing because the municipal government there had invested more funding to support international student education. acculturation sun et al. (2009) surveyed 450 international students from 13 universities in the cities of kunming, beijing, guangzhou and shanghai on the three dimensions of (i) social adaptation, (ii) psychological adaptation, and (iii) cultural adaptation. the findings indicated that the students' social adaptation was satisfactory on the whole, but they at times reported unsatisfactory encounters with, and poor services from, people from government departments (sun et al. 2009 ). they felt most comfortable making friends and going shopping, and they expressed satisfaction with their accommodation. the study also found that the majority of the students had positive experiences of psychological adaptation, but there were some students who felt lonely, sad and disappointed. more women than men reported negative emotional experiences. the findings also showed that international students who lacked social support had the lowest levels of adaptability, while those who experienced social support, mainly from co-nationals and chinese people, had the best levels of adaptability (sun et al. 2009 ). regarding cultural adaptation, sun et al. (2009) noted that international students often found some chinese traditions and rituals hard to understand. according to the researchers, the adaptability of european and american students studying abroad was higher than that of asian students. they attributed four possible reasons for this: (i) foreign students' intrinsic motivation to study in china vs. the pressure of asian parents' expectations, (ii) more adequate psychological preparedness of european and american students for adaptation vs. inadequate preparedness among asians, (iii) individualistic cultures in the west vs. collectivist cultures, and (iv) strong financial background vs. weaker financial status. it should be noted that the study by sun et al. was undertaken a decade ago in 2009, and it provided a broad understanding of international students' acculturation and satisfaction with life and learning in china at that time. it is hoped that the situations they described have improved since then, as indications are that china has progressed considerably in terms of public services and the quality of higher education. however, it is important for topical research to be undertaken to determine whether such changes are reflected in the experiences and perceptions of international students studying in mainland china at present. tan (2013) addressed the adaptability of foreign students to studying in china and suggested that it was natural for international students who had lived in china for longer periods to be better adapted than new arrivals. the latter faced difficulties with social integration, a lack of orientation or counseling services, and communication with chinese staff and classmates; with these factors being confirmed as having an impact on acculturation (li 2007; tan 2013) . gebregergis et al. (2019) addressed the issue of acculturation in their study. they explored the correlation of the three variables of prior travel experience, age, and cultural intelligence (understanding a culture that is not your own and the ability to relate effectively across cultures). they then related these to acculturative stress and depression among international students studying in wuhan. their findings showed that students with prior international experience had lower acculturative stress and were prone to less depression than those without such prior travel experience. age had a negative correlation with acculturative stress and depression in the study, with younger students demonstrating more openness to new experiences and more flexibility in their coping strategies. cultural intelligence was found to be negatively correlated with acculturative stress and depression. international students with more multicultural awareness or experience were less prone to acculturative stress and depression. findings from an empirical study by li et al. (2015) revealed that students from china's neighboring asian countries found it less difficult to integrate into chinese society than international students with more cultural distance from china. nonetheless, the chinese language proved to be a major barrier for them, hindering them from communicating easily with chinese people. for example, even in beijing, a cosmopolitan city where english is more commonly used, international students who lacked either chinese or english language proficiency tended to experience more communication obstacles (wen et al. 2018 ). it was not surprising to find that international students who had good proficiency in a local dialect spoken in the city where they studied, or who spoke chinese more in daily life, were more likely to have adapted well and to express higher levels of satisfaction (sun et al. 2009 ). tian and lowe (2018) found that the perceived wealth of international students' countries of origin seemed to influence administrators' attitudes toward the students. it was also reported that black students were more vulnerable to racist attitudes, although this was not widespread (tian and lowe 2018) . there is a need to research more deeply the issues of discrimination and racism that seem to occur (although they are often hidden) when international students of different backgrounds interact with peers and staff in host educational institutions. because of the deeply personal impact of such attitudes and behaviors, such experiences inevitably have a major effect on life satisfaction and adaptation. international students committed to a religion that they openly practiced tended to encounter more difficulties making friends and maintaining religious activities, and this was a potential source of considerable stress (bin et al. 2014) . for example, muslim international students reported that they were not allowed to pray in public, but only in their dormitory corridors (tian and lowe 2018) . furthermore, muslim and hindu students who had food restrictions found it difficult to eat with chinese students, and those who could not find food that tasted familiar expressed higher levels of homesickness (li et al. 2015) . this review of the literature on international students attending universities in mainland china seems to suggest that factors contributing to life satisfaction and adjustment are similar to those found in other countries. these factors include language barriers, issues with acculturation, financial hardship, cultural proximity/distance, perceived discrimination, campus support services, age, gender, marital status, length of residence, prior sojourn experience, predeparture preparedness, and individual coping skills. yet studying in china seems to pose particular challenges in the form of language barriers and the limited availability of campus support services. the language barrier is especially significant because international students have to speak chinese for everyday tasks, such as shopping, renting an apartment, seeing a doctor, traveling, using banking services, consuming entertainment, and eating in a restaurant, apart from tasks within the crucial educational context. those who are unable to speak chinese adequately tend to be constantly challenged and typically struggle more with life satisfaction and adaptation. ding (2016) and wen et al. (2013) have pointed out that the overall satisfaction level of international students in china is lower than that recorded in other host countries, especially compared with countries like the u.s., the u.k., australia and new zealand. there seem to be four possible reasons for this situation: (i) educational institutions in other settings may be more revenue-driven and thus may attach more importance to maintaining international students' satisfaction; (ii) the universities in these other host countries may have developed better styles of teaching; (iii) the settings mentioned use english as the mode of instruction; and (iv) the settings mentioned have better curricula and well-designed course materials. their years of experience as providers of courses to an international market have taught them the importance of support service systems for student applications, student orientations, and counseling services (smith & khawaja, 2011) . in comparison, china seems at this stage to fall short in these aspects, and chinese universities seem to rely too heavily on outside agencies to recruit, inform, and prepare potential students. it seems evident from the studies reviewed here that chinese institutions may be unaware of the need to improve their services, are resistant to change, or are in the process of adapting but are still in the early stages of doing so. china has created a number of favorable policies designed to attract international students, and chinese universities have made considerable efforts to increase enrollments, but it is important to ensure that current practices and offerings are valued by international students and are sustainable in the long term. china's eagerness to build a reputation as a world-class provider of university education is undermined by international students' perceptions of the quality of experience currently provided. too little attention has been given to researching and addressing the needs of foreign students. china has not actively evaluated its higher education practices, so improvements have been slow to eventuate. with the global advancement of higher education, competition for enticing students at home and abroad will become even more fierce. because hosting international students is crucial to china's soft power strategy, it is important for the chinese government and higher education institutions to more successfully meet the expectations of overseas students, and to ensure that their life satisfaction is strengthened rather than undermined by their experiences in china. the life satisfaction of international students studying in china is clearly influenced by multiple factors. to increase the satisfaction of international students, efforts should be made to improve four main domains: (i) the quality of the courses offered, (ii) the greater use of english language to communicate at all levels, (iii) support services development, and (iv) counseling services. first and foremost is the need to improve the quality of education provided in chinese universities, particularly in terms of teaching style, curriculum content, and study materials. research has confirmed that to ensure ongoing recruitment and retention, high-quality education must be provided that meets the needs and expectations of international students (alemu and cordier 2017; ammigan and jones 2018; ding 2016; jiani 2017; tian and lowe 2018; wen and hu 2019) . it is imperative that regular surveys of international students' evaluation of teaching and support services be conducted, and any deficiencies detected should be addressed at both government and institution levels. second, it is clearly essential that lecturers and teaching assistants in china receive adequate training in english proficiency and oral presentation skills. they need to observe examples of interactive teaching methods that engage students more effectively and establish good teacher-student rapport. this can be achieved through international teacher-exchange programs and video viewings of good practice. the lecturers also need to acquire better cultural knowledge of international students' backgrounds and their expectations. course textbooks and materials should be updated regularly, and full use should be made of e-learning and related technology. thirdly, support services should be developed and extended to meet international students' needs. areas most in need of strengthening include better website pre-enrollment information, better orientation programs for new students, ongoing counseling services, and employment guidance. international students are now used to accessing information and services online, so chinese universities should make full use of the internet to provide fast and convenient communication channels between lecturers and students and between students themselves. it is also recommended that policies be formulated and funding provided to improve accommodation and strengthen logistical services appropriate for international students. international students' social interactions with chinese students can be enhanced in all chinese universities through activities such as cultural exchange programs and chinese language courses. improved chinese language skills help overseas students function more effectively in the local community and within the institution. developing friendships with chinese students and fellow international peers is important for international students to reduce their loneliness and help them feel more engaged (hendrickson et al. 2011; smith and khawaja 2011) . fourthly, strengthening counseling services would greatly assist international students (yalçin 2011) . counselors can help to improve students' level of optimism and motivation through professional work and the establishment of a social support network. counseling support services are a basic necessity in chinese universities, but currently they are very inadequate compared to services in western settings. at present, student counseling services at china's higher education institutions are mostly limited to employment guidance and psychological counseling for local students (li 2010) . however, chinese colleges and universities have begun to attach greater importance to psychological counseling for all students. most of these institutions are establishing centers equipped with facilities (both rooms and resources) for individual counseling and group sessions. counselors are allocated a certain number of students each year, and telephone counseling, online counseling, peer counseling, and other methods are being used (su and wang 2015) . to meet the needs of students, some colleges have administered vocational ability tests to assist with career planning and provided psychological counseling for job candidates. each institution has specific counseling services tailored to their student population. overall, multiple counseling methods are being developed, and they should take full account of students' feedback, as has been made evident in this paper (su and wang 2015) . issues facing counseling services in chinese universities merit particular attention. first, chinese students tend to have misconceptions about counseling, associating it with mental illness or psychological abnormalities. for this reason, they fear that they may be stigmatized by their classmates or teachers if they seek counseling (su and wang 2015) . their reluctance may also be fed by the collectivist cultural influence that constrains help seeking (raunic and xenos 2008) . another possible explanation for students' reluctance is that counseling services at this point seem to fall short of achieving their aims on campus. students doubt whether counselors are able to communicate with them and really understand their worries or problems (sun 2019) . thereupon, eliminating students' misconceptions and mistrust of counseling may be an ice-breaking step towards addressing their needs. secondly, there is a shortage of professional counselors with appropriate skills, training and experience, with this causing counseling-type duties to fall upon other staff members. often, these personnel are student affairs professionals who take general care of a large number of students for such matters as studies, extra-curricular activities, dormitory checks, management matters, and discipline. the shortage of professional counselors, if not addressed, may add more work burden to student affair professionals and deter from achieving the objective of professional counseling in a real sense. third, the emergence of counseling services for international students has been slow at some universities in china, and services for career planning or employment guidance for such students are barely visible. the counseling and consulting services that are available tend to focus on providing the general student population with socialist political guidance, life values, and education about vocational ideals, rather than on providing guidance to individual students for their problems and stress management needs (li 2010) . additionally, students perceive the counseling environment to be insufficiently private, which greatly affects their levels of openness in psychological counseling communication. english language proficiency is also a challenge for counselors, and speaking only in chinese with foreign students who have limited chinese language skills can hardly achieve desired outcomes (xu et al. 2016) . the most important goal for the immediate future is for chinese higher education institutions to acknowledge international students' counseling needs. resources should be invested to build professional counselor teams to serve this increasingly large group of students if chinese universities and colleges are to continue to attract overseas enrolments. multicultural awareness training and language training are essential for all lecturers and counselors because international students comprise a heterogeneous group with unique cultures, religions, traditions, values, expectations, and ways of thinking (arthur 2017) . intercultural competencies for counselors could help them better understand students' perspectives when they are involved in cross-cultural counseling (pattison and robson 2013) . more counseling services are expected to be provided in future, including academic advising, language tutoring, career planning, and employment guidance. modes of online and offline counseling can be combined flexibly and innovatively (sun 2019) . this should help to enhance students' quality of life and levels of satisfaction. for instance, during the current situation of the covid-19 pandemic, in which face-to-face counseling is not feasible, online counseling should be offered to mitigate international students' anxiety and distress. although knowledge of the influences that affect international students studying abroad is significant, several areas would benefit from further empirical research in the chinese context. 4 first, it would be valuable to explore precisely what matters most to international students when they study in the chinese context and which services should be provided or improvedfor example, academic and career counseling, psychological counseling, and career guidance. second, studies should investigate how international students' academic success is affected by the very formal teaching methods traditionally used in china. does this approach hinder them from achieving their goals and expectations? a third important issue to investigate is how individual students differ in the way they cope with the challenges they face. such information would help those who counsel and support international students. it could lead to the development of powerful ways of assisting students in developing coping strategies and resilience. in addition, in the context of the covid-19 global pandemic, it is important to study the impact of the crisis on international students' desire to study abroad, the learning progress of those affected by the pandemic, and their psychological stress and corresponding coping strategies. factors influencing international student satisfaction in korean universities institutional satisfaction and recommendation: what really matters to international students improving the student experience: learning from a comparative study of international student satisfaction supporting international students through strengthening their social resources. studies in higher education: the international student experience: voices and perspectives what motivates chinese women to study in the uk and how do they perceive their experience? higher education international students' enhanced academic performance: effects of campus resources acculturation: living successfully in two cultures stress and health related quality of life of nepalese students studying in south korea: a cross sectional study acculturative stress and influential factors among international students in china: a structural dynamic perspective roles of international student advisors: literature and practice in american higher education ministry of education: in the past five years, more than 350,000 people from china have gone to study in countries along the "belt and roles of university support for international students in the united states: analysis of a systematic model of university identification, university support, and psychological well-being international students in china: a dream come true the satisfaction with life scale what kind of education do international students in china need: on the basis of data from four universities in shanghai exploring the experiences of international students in china brokers of international student mobility: role and processes of education agents in china emerging destination mobility: exploring african and asian international students' push-pull factors and motivations to study in china cultural intelligence, age and prior travel experience as predictors of acculturative stress and depression among international students studying in china an analysis of friendship networks, social connectedness, homesickness, and satisfaction levels of international students study abroad objectives and satisfaction of international students in japan global and national prominent universities: internationalization, competitiveness and the role of the state the role of the university library in supporting international student transition: insights from an australian-american case study why and how international students choose mainland china as a higher education study abroad destination examining associations among achievement motivation, locus of control, academic stress, and life satisfaction: a comparison of u.s. and international undergraduate students stress factors among international and domestic students in russia. procedia -social and behavioral sciences, 200(c) research and countermeasure on intercultural cultural psychology adaptation of foreign students in china educational research and new exploration of non-degree students studying in china. asia-pacific education a comparative study of student guidance and consulting services in chinese and american universities a study on the intercultural adaptability of foreign students: taking a university survey as an example why do foreign students choose to study in china: an empirical analysis based on survey data from beijing universities an analysis of factors influencing international students' choice of education in china pre-departure cultural preparation for international students: addressing adjustment needs before study abroad international student persistence in u.s. postsecondary institutions. higher education factors that contribute to the adjustment of international students statistics of studying in china past experience, cultural intelligence, and satisfaction with international business studies international higher education and public diplomacy: a case study of ugandan graduates from chinese universities acculturative stress among asian international students in singapore australian students in china: making the foreign familiar life satisfaction among outbound students in northern sweden the mediating role of acculturative stress in the relationship between acculturation level and depression among korean immigrants in the u internationalization of british iniversities: learning from the experiences of international counselling students university counselling service utilisation by local and international students and user characteristics: a review the influence of social media on international students' global life satisfaction and academic performance social-emotional loneliness and life satisfaction satisfaction with life among international students: an exploratory study a review of the acculturation experiences of international students research on the status of college students' psychological counseling services a survey of acculturation of foreign students in china and some suggestions analysis and countermeasures on the demands of developmental psychological counseling in higher colleges the determinants of life satisfaction among chinese students in finland and russia several questions about the adaptability of foreign students to china the role of perceived cultural distance, personal growth initiative, language proficiencies, and tridimensional acculturation orientations for psychological adjustment among international students international student recruitment as an exercise in soft power: a case study of undergraduate medical students at a chinese university the role of the university library in supporting international students: a survey of practices across uk universities a comparative study of international students' experiences and satisfaction in beijing international students' experiences in china: does the planned reverse mobility work? the emergence of a regional education hub: rationales of international students' choice of china as the study destination a research on practices of mental health service for overseas students social support and optimism as predictors of life satisfaction of college students commentary: international students in china-what we know, what we don't, and what next china's strategy for the internationalization of higher education: an overview. frontiers of education in china china's soft power projection in higher education state formation and higher education (he) policy: an analytical review of policy shifts and the internationalization of higher education (ihe) in china between 1949 and 2019. higher education predictors of international students' psychosocial adjustment to life in the united states: a systematic review home away from home: extending library services for international students in china's universities using bronfenbrenner's ecological approach to understand academic advising with international community college students publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations acknowledgements the paper is based on qinxu jiang's doctoral research under the supervision of hugo horta and mantak yuen at the university of hong kong faculty of education. conflict of interest the authors declared that they have no conflict of interest. key: cord-327502-za6x4reh authors: tian, lin; liu, nian cai title: inward international students in china and their contributions to global common goods date: 2020-03-11 journal: high educ (dordr) doi: 10.1007/s10734-020-00522-5 sha: doc_id: 327502 cord_uid: za6x4reh this study identifies the (global) common goods produced and augmented by inward student mobility and its relevance to national policies and strategies in china by interviewing 27 policy-makers, university leaders, and academics, as well as international students. most importantly, it develops a new framework for interpreting inward student mobility through the lens of (global) common goods, with benefit, influence, and balance as the three key elements. the findings of this study indicate that inward international students contribute to global common goods in the aspects of cultural diversity, global talents, improved policies, and practices, etc. (benefit). key policies related to inward international students are generally supportive, and meanwhile compatible with these global common goods to a large extent (influence). though some problems and tensions are observed in this study, constructive solutions have already been proposed by both the chinese government and heis (balance). under the influence of globalization since the beginning of the twenty-first century, the internationalization of higher education has become an unstoppable and irresistible trend. international student mobility, in its multiplicity of forms, continues to be a high priority of internationalization, which refers to students moving from a country of origin to a country of education for study in a limited time (knight 2012) . more specific, inward student mobility is viewed from the perspective of the chosen country of study (that is, the favorite destinations), which brings a potential "brain-gain" (murphy-lejeune 2008) . there were about 5.3 million international students pursuing higher education overseas in 2017, comparing with 3.3 million in 2008 (oecd 2019). they have the potential to bring significant academic and economic implications, and their numbers are expected to grow continuously (knight 2012) . however, perhaps as important as the rising numbers of international students is the fact that the traditionally dominant destination countries for international students (e.g., the usa, the uk, australia, etc.) face growing competition from newly industrialized countries like china, russia, singapore, and malaysia. these countries, which once sent large numbers of students abroad, are gradually becoming recipients of international students, as a result of their efforts in promoting inward student mobility (de wit et al. 2013; abdullah et al. 2017) . for instance, in 2017, china sent 608,400 students to study abroad while at the same time hosted 489,200 international students (ministry of education of china, 2018) . in the same year, according to oecd (2019), russia took the fourth largest market share of international education globally, with a percentage of 5%, following the us (18%), uk (8%), and australia (7%). the primary motives for these countries' push on inward student mobility can be summarized as: enhancing international influence, promoting international exchanges, accelerating the process of building leading universities, and potentially generating economic gains (cao et al. 2016; abdullah et al. 2017; choudaha 2017) . though the economic benefit of inward international student flows is not a main focus for certain countries (e.g., china) , it is the primary pursuit for many developed countries including the uk and australia. therefore, when imaging inward international students, the notions of "global education market" and "global student market" are often used, which underpin the commercial value of inward student mobility and ignore its function as cooperation and a "win-win" (marginson 2016a) . in fact, inward student mobility is not solely market-driven. it brings various academic and social benefits not only to the host country but to other countries, including the countries of student origin. in light of this, (global) common goods can be a more inclusive lens to understand inward student mobility, since this concept highlights collective endeavor, shared participation/engagement, diversity, inclusion, and responsibility (unesco 2015) . the idea of "common goods" is of significance in describing (higher) education, as (higher) education requires active participation and collective endeavor in the process, which is compatible with the definition of common goods (unesco 2015; tian and liu 2019) . although closely related to the idea of public goods, the concept of common goods has its own specific meaning and implications 1 (see table 1 ). common goods can be defined as goods that are characterized by a binding destination and necessary for the realization of the fundamental rights of all people, irrespective of any public or private origin (unesco 2015, p.77) . "the good realized in the mutual relationships in and through which human beings achieve their well-being" (hollenbach 2002, p.81) , which is inherent to the relationships that exist among the members of a society tied together in a collective endeavor (unesco 2015, p.78) . these goods are necessarily grounded in the cultural and social dimensions of a specific community and are identified for their contribution to the "general interest", also to conditions of justice and well-being (locatelli 2018a) . deneulin and townsend (2007) argue that a celebratory dinner, an orchestral, or a team sport can be thought as common goods. they then take the orchestra as an example to illustrate how a common good is produced and how its benefits can be enjoyed. the good itself, an orchestra, cannot exist without each musician playing their respective parts and performing collectively for the whole audience. in other words, the good exists in the shared action which generates it. also, benefiting from such good is by participating in it, whether in the orchestra or audience. hence, the shared action is both intrinsic (without the various musicians' participation and performance, the orchestra cannot exist at all) and instrumental (it is necessary, efficient, and convenient to perform as an orchestra) to the good itself, and its benefits (an orchestra; beautiful music; excellent performance) are generated from the course of that shared action. such kinds of goods are intrinsically common in their production and their benefits, reflecting the distinct characteristics of intrinsic value and shared participation (deneulin and townsend 2007; unesco 2015) . in other words, the benefits and burdens for the realization of common goods are intrinsically shared among all participants, thus the common quality of these goods is not preexisting but is dynamic and contingent, emphasizing a process of participatory democracy (locatelli 2018a) . however, common goods may have a boundary and be confined to a given group/ community, since it is often socially embedded. their creation and production are processes of collective participation. people who participate in these processes can benefit from them, and these participants form a community with common interest (tian and liu 2019) . in light of this, global common goods are related to all people worldwide, highlighting the participation can be enjoyed as individual goods necessarily shared, both regarding production and benefits non-excludable and non-rivalrous in terms of consumption of a commodity non-excludable and non-rivalrous in terms of participation and generation of the goods themselves the public quality is predetermined the common quality is dynamic and not pre-existing governance public governance justified on the basis of externalities which extend to all shared governance justified on the basis of the cultural and social value of a specific good result of the action of public institutions result of the interaction of the different components of society top-down approach bottom-up approach passive role of those who benefit active role of those who benefit value global public goods are quasi-universal in terms of accessibility to countries, people and generations global common goods imply the empowerment of all actors in a world society who have a right to a fully informed and critical participation instrumental, can be treated as economic resources cannot be reduced to economic resources or to factors of production because of intrinsic social and relational value source: adapted based on locatelli (2018b) of all persons in a diverse and differentiated, yet solidaristic and collaborative, world society (deneulin and townsend 2007, p. 29) , and perhaps fostering social inclusion, integration, tolerance, equality, and human rights at a global level (unesco 2015; tian and liu 2019). thus, higher education itself can be regarded as a (global) common good, because receiving higher education can be seen as a specific activity, and heis, educators, students, and other stakeholders involved in the process could jointly reap the benefits through shared participation (tian 2019) . broadly speaking, the notion of common goods suggests the transformation of heis through greater engagement of communities, civil society, and other non-state actors in the decision-making and practices in order to build a more democratic educational system (locatelli 2018a; boyadjieva and ilieva-trichkova 2019) . apart from this, higher education as a common good is closely related to such concepts as equity, justice, solidarity, and inclusion (walker and mclean 2013, marginson 2016b) . also, the idea of common goods in higher education is especially important in the process of globalization and internationalization, because it refuses to be closed and conservative, encouraging global universities and educators to collaborate jointly and participate actively (tian 2019) . the previous literature draws a picture of the general situation of the internationalization of higher education, but it does not give as much attention to inward student mobility from a common goods lens, and the (global) common goods which are created and increased by it are rarely spelled out. through semi-structured interviews, this study aims to explore the (global) common goods generated and augmented by inward student mobility in china as well as relevant policies, regulations, and practices. this paper begins with the broad literature on the history and development of inward student mobility in china as well as relevant policies and strategies. then, it presents the research method, procedures, and empirical data. the paper concludes by linking the findings to the previous literature, to deepen and extend the understanding of this topic. inward student mobility in china: history, development, and key policies the recruitment of education for international students in china began in 1950 and went through a 70-year history, which falls into the following five periods with relevant national policies (see table 2 ). until now, china has educated a group of people who understand and respect the chinese culture, and at the same time expand china's international influence, helping to enhance its international image (jiang 2010b) . it can be assumed that before the reform and opening-up in 1978, the recruitment of and education for international students in china mainly served as a diplomatic tool for international exchanges and assistance. after the reform and opening-up, policies for international students studying in china gradually moved from closed to open, and the decision to accept and educate international students in china shifted from government to heis. meanwhile, more than a few heis began to recruit self-funded international students, and the number of these students rose rapidly (jin 2012) . later in the 1990s, several improvements were made in the international education system. for instance, in 1992, the hsk (hanyu shuiping kaoshi, which is a standard chinese language proficiency test) was used as a standard test for evaluating the language ability of international students. in 1996, the ministry of education established the china scholarship council, which specialized in organizing, funding, and managing chinese students studying abroad and international students studying in china. during the present period, policies for international students studying in china are more comprehensive, and efforts related to promoting inward student mobility yield positive results (cheng and huang 2008; fang and wu 2016) . the number of international students in china has increased markedly, and their study in china is greatly supported by a number of policies. china (1950-present) periods changes and development i: initial practice (1950) (1951) (1952) (1953) (1954) (1955) (1956) (1957) (1958) (1959) (1960) (1961) (1962) (1963) (1964) (1965) • china conducted more international exchanges with socialist countries in the former soviet union and developing countries in asia and africa. as a duty to fulfill international aid, china actively accepted a large number of international students from third world countries of asia, africa, and latin america. china regulations on recruiting self-financed international students (1989) iv: establishing a new system (1990) (1991) (1992) (1993) (1994) (1995) (1996) (1997) (1998) • with the formation of a self-operation system under the guidance of the government, international education in china entered the first period of rapid development. three hundred thirty-nine heis in china accepted 234,691 international students from 164 countries, an increase of more than 15 times compared to the previous period. in 2018, the number of international students studying in china was 492,185, about 11 times when compared with 44,711 in 1999 (see fig. 1 ). china has entered the 10 highest-ranked destination countries in the world of international education, with the majority of international students coming from south korea, thailand, and pakistan in 2018 (see table 3 ). in terms of policies, the administrative measures for the enrolment and cultivation of international students by schools issued in 2017 clarified various aspects of inward student mobility, concerning degree certificates, training methods, curriculums, and so on, further improving the management system for international students studying in china. in october 2018, china published the first and latest quality standards of higher education for international students studying in china, expecting to improve the quality and management of international education in china. knight (2012), riaño et al. (2018) consider that international student mobility has brought diverse and plentiful benefits to students, home and host institutions, communities, and countries, for instance, international students can be portrayed as precious human capital to boost the host country's economy and also as sources of income for the higher education sector. specifically, in the chinese context, fang and wu (2016) consider that international students improve the level of internationalization of chinese higher education. also, they assume that international students not only increase the direct economic benefits but also stimulate the growth of related service industries such as catering, transportation, and tourism. similarly, jiang (2010a) notes that international students studying in china have the potential to create economic benefits, though this has not received sufficient attention in china and it lacks reliable official statistics. han (2014) , ma and zhou (2018) agree that international education in china has cultivated a large number of professional talents and develop fig. 1 the increase of international students in china (1999 china ( -2018 source: ministry of education of china (2000 china ( -2019 higher education table 3 the top 10 countries of sending students studying in china (1999 china ( -2018 year rank source: ministry of education of china (2000 china ( -2019 cooperative relations with other countries. moreover, in recent years, under the background of belt and road initiative, chen and wen (2018) suggest that international students play a central role in improving diplomatic relations and these students are human resources for chinese enterprises overseas. however, research touches little on the common good(s) of inward student mobility in china, though some chinese scholars have put forward that higher education contributes to (global) common goods in four respects: research outputs; public services; cultural inheritance and innovation; and talents (with global perspectives) (tian and liu 2019) . the mobility of these global talents and cultural exchanges are conducive to building a community of shared future for humanity (ren lei ming yun gong tong ti) (yang 2018) . in summary, the previous research has explored the history, development, and benefits of international students studying in china. this prior research lays a foundation for the present project. however, most research on inward students in china is theory-based, with less emphasis on empirical investigation; though some relevant research is policy-based and data-based, both the policies and data are out of date; also, there is an absence of attempts to explore inward student mobility and relevant policies in china through a lens of common goods, which would generate both theoretical and practical contributions. hence, in order to deeply understand the inward student mobility in china and (global) common goods, we formulated the following research questions: (1) what are the (global) common goods of inward student mobility in china? (2) how do the key policies, strategies, and regulations concerning inward international students relate to (global) common goods in china? according to kehm and teichler (2007) , institutions, people, and knowledge are the core elements of internationalization in higher education, and there is also a strong political undercurrent in the form of institutional strategies and national policies embedded in internationalization activities. accordingly, in order to have a better understanding of inward student mobility in china, which is a key to higher education internationalization, the research method for this project mainly involves semistructured interviews (people-concerned), while at the same time focuses on policy documents related to international students mentioned by interviewees (policy-focused). learning through policy texts from interviewees is a form of "lesson drawing", which can be defined as the voluntary act of transfer by rational actors working in specific political contexts (benson and jordan 2011) . this project adopted a purposive sampling to identify participants who were directly involved in inward student mobility. they were invited by the researchers through email. twenty-seven chinese people from the government and universities participated in the research. they were divided into four groups according to their affiliated institutions and positions (see table 4 ) to ensure coverage of all relevant groups of people who might have a good understanding of inward international students and common goods in china. the study chose two universities at two different levels in china: a top research university that is also a "double first-class" 2 university in china, with a higher level of internationalization (s university 3 ) and a local university with a lower level of internationalization (z university 4 ), with participants from different disciplines. admittedly, these two cases cannot represent the whole of china, but the investigation of perspectives from them can enable us to make a comparison of the policies, strategies, regulations, and relevant practices (e.g., management) about international students between universities at two different levels. due to the differences of the participants, there were four sets of interview questions for the above-listed participants in the study. interview questions for each group were slightly different, but they can be generally divided into the following three broad topics: (1) policies and strategies related to inward international students; (2) management and regulations of international students; and (3) each interview lasted between 30 and 70 min, depending on the interest of the participants and the natural pace with which the interview moved. interviews were conducted in both chinese and english (according to the respondents' requirements) and then transcribed into written form by researchers. participation in this research was on a voluntary basis, and participants were guaranteed anonymity and confidentiality. universities and conducted a wide range of research cooperation with overseas partners (source: university website). 4 z university is a comprehensive research university with international connectedness and a strong regional mission. it is a provincial university, being listed in the 301-400 in the arwu and 50-60 in the best chinese universities ranking. by the end of 2018, the percentage of international students in z university is about 2.4%. z university also has exchange programs and research cooperation with overseas universities in certain fields (source: university website). nvivo12.0 qualitative software and qualitative content analysis (qca) were used to analyze the interview data. qualitative content analysis, also called thematic analysis, is one of the most commonly used methods for analyzing qualitative data, with the focus on working with categories (codes) and developing a category system (coding frame) (mayring 2014 ). transcripts were coded according to participants' responses to each question and to the most salient categories (also called themes) emerging across the set of interviews. an initial list of main categories corresponding to the questions asked in the interview was formed deductively; then, as a second step, data were coded with the main categories (e.g., policies and strategies; global common good contributions). next, compiling text passages of the main categories and forming subcategories inductively on the material; assigning text passages to subcategories (e.g., "scholarship policy" was a subcategory under "policies and strategies"). last, all categories were refined under more general headings related to the research questions, and a category system was formed. in the paragraphs that follow, participants in the interviews were referred to by different code names, for the purposes of both ensuring anonymity and facilitating tracing references from the data (see examples of code names for participants in table 5 ). considering the contributions made by inward student mobility to (global) common goods, participants suggested that, academically, it cultivates human resources, retains high-level talents (serving local society), and improves international cooperation; politically, it accelerated international exchanges, which improves mutual understanding and respect; scientifically, it strengthens international scientific research cooperation; and economically, it can be seen as a potential economic growth point (though china has not yet achieved this result) and may expand the influence of destination countries and cities. in general, based on marginson's (2018) global goods framework, the individualized and common goods at both national and global levels can be summarized in table 6 . specifically, for the host country, international students promote cultural exchanges, and they could become cultural ambassadors in the future. for students' home countries, they can bring back advanced knowledge and research outputs, and students studying abroad can also the second participant in s university, who is a university leader. the third participant in z university, who is an academic. the fifth participant in z university, who is an international student. notes: p, participants; ga, participants from the government; s, s university (a top research university with a higher level of internationalization); z, z university (a local university with a lower level of internationalization); l, university leaders; a, academics; s, international students be regarded as role models for other students who hoped to study abroad. eight international students interviewed expressed their willingness to stay for employment or continue further studies in china. i have a lot of things that i owe china…i probably will stay here, at least for several years, but that depends on whether china will accept me, but of course, i don't take this education for granted, and i hope there is a chance i can make a contribution to china. most participants suggested the advantages brought by inward student mobility outweigh the disadvantages. however, university leaders and academics considered the potential disadvantages of inward student mobility include: (1) problem of unfairness (n = 4), since international students occupy the opportunities for domestic students, excessively consuming the energy of the administrators as they often pay more attention to international students rather than the domestic students; (2) brain drain of the source countries (n = 1). in contrast, two government officials believed that inward student mobility could promote social equity, as it advances the sharing of educational resources, and helps some backward countries' educational development. i believe inward student mobility in china promotes social equity rather than damages it, because it is good for the sharing of educational resources, and it is also a way to help some countries with less-developed education. (pga4) also, most international students (n = 7) did not consider it to be unfair for people like them to access to the experiences and benefits of international education, because the selection of candidates to study in china depends on the ability of the applicants, and the opportunity for international education is on a global basis, and it is beneficial for all. at the national level, government officials mentioned that china's strategic goal is to host 500,000 international students by 2020. meantime, most participants (n = 18) mentioned that the funding policy oriented to international students studying in china are very supportive (e.g., chinese government scholarships and provincial funding). in these years, under the framework of "people-to-people exchange", the belt and road initiative as well as other important policies includes international students studying in china as a strategic part (n = 7). our strategic goal is to recruit 500,000 international students by 2020… this was stated in the plan of studying in china… the policies, especially the funding policy, are quite supportive… the belt and road initiative also includes international students studying in china as a strategic part. (pga1) eight participants mentioned the decree no. 42, i.e., the administrative measures for the enrolment and cultivation of international students by schools issued in 2017, which covers all aspects of international student management. in addition, the assimilative management of international students studying in china becomes a trend, which implies heis encourage chinese and international students to live together thereby increasing mutual respects and understandings (n = 10). by doing so, china hopes to have more students who have better knowledge about china, become friends to china, and are on intimate terms with china. china is now much closer to the centre of the world stage than ever before… we hope more students come to china so as to better understand china, become familiar with china, and finally form friendly relationship with china. (pga2) at the university level, policies are more specific, that is, increasing the support for international students; conducting assimilation management to avoid isolation; introducing new types of scholarship; and opening bilingual courses (n = 10). s university's current working guidelines for international students are: expanding size, optimizing structure, improving management, and ensuring quality. as a "double first-class" university in china, internationalization is a key component in the development of s university (n = 6). z university focuses on increasing the number of international students and the recruitment of these students from the belt and road countries (n = 4). in 2015, z university introduced an internationalization strategy, and highlighted international postgraduate students' recruitment (n = 3). since there is no government scholarship in z university, it plans to set up its own scholarship for international students. our policy and strategy are definitely conforming to the strategy of national development... we pay much attention to belt and road countries, so we are now working on attracting students from these countries... we will also establish some additional scholarships. (pzl5) the majority of government officials and university leaders (n = 10) pointed out that the government strategically guides the internationalization of heis, and heis serve the national goal of attracting international students. all officials and university leaders (n = 14) considered relevant policies and strategies to be effective, achieving the expected results. the most frequently mentioned policies and strategies by the interviewees are the administrative measures for the enrolment and cultivation of international students by schools (2017) (n = 6); belt and road initiative (n = 5); and the plan of studying in china (n = 5). at the national level, all four government officials agreed that all international students who are accepted by chinese heis do not have any difficulties in their first entry, and it is necessary for all international students to abide by national laws and regulations. the latest requirement in the decree no. 42 allows work-study experiences for international students during their study period, and they are encouraged to have internships in china and spend part of their study time to learn more about chinese society. after graduation, students need to go through certain procedures to obtain a work visa, or they must return home immediately. participants mentioned that china has relatively tight regulations on foreign nationals' long-term residence, but it welcomes high-level talents to stay (n = 5). undergraduate students cannot stay in china after graduation, and they cannot obtain work permits. the policy for postgraduate students is relatively flexible, and students can obtain an employment visa in free trade zones 5 in some coastal metropolitan cities (n = 9). however, some international students (n = 4) are still concerned about the employment issues in china: we have an information group and the teacher promptly provides relevant employment information... but this is not enough for us, we still face a lot of problems in finding a good job in china. (pss5) international students do have some influence on policy-making, for example, the right of residence, entry and exit, and employment, and they play a positive role in promoting policy reforms (n = 6). three interviewees pointed out that when international students are studying and living in china, they are involved in a series of issues related to medical care, social insurance, and other public services; thus, when formulating policies, it is reasonable for the government to consider the needs of international students for their daily life. international students do have an impact on some policies, for example, the right of residence, entry and exit, and employment issues, and they have played a positive role in promoting these reforms, which is a positive sign. (pga4) at the university level, both universities have regulations and requirements on international students based on national policies (n = 12). for international students' first entry, they need to have a student visa and go through medical examinations; for in-university supervision, they must obey relevant laws and regulations (n = 14). most international students in our study thought they had favorable experience in chinese universities which matched their expectations (n = 8). is this a good place? i must say this place can be described as heaven… i find out that china has already entered the modernized stage… there are so many things we can learn… i guess china goes far beyond my expectations... (pss1) both universities offer specific courses and services for international students. each international student dormitory has an assistant, and there are also online information groups specifically for international students (n = 9). all nine interviewed students received generous scholarships from the chinese government or the universities. they considered both the chinese government and universities to be very supportive to international students. basically, international students have few problems on campus, though some of them (n = 3) encountered minor problems, for example, they think the administrative procedures are too complicated, and the accommodation is poor in z university. all nine students put forward that they have never faced issues related to welfare, human rights, personal agency, and dignity, and they feel safer and more comfortable in china than other places. i don't feel marginalized or disempowered, because i believe "ru xiang sui su" (do in rome as rome does), and i am obeying the laws of china and trying to see the world through chinese people' eyes … in many ways, i am freer, safer and happier here, which i've never felt in my home country… (pss3) discussion framework for inward student mobility from a common goods lens the objective of the current research is to identify the (global) common goods produced and augmented by inward student mobility and its relevance to national policies and strategies in china. the findings of this study illustrate that inward student mobility contributes to (global) common goods, and meanwhile policies are compatible to these perceived global common goods to a large extent. although some problems and tensions among policies, practices, and (global) common goods are observed in this study, measures have been taken by both the chinese government and heis. as inward student mobility involves different participants (host countries and universities, sending countries, university staff, international students, etc.) simultaneously, and its benefits flow to places at different levels (individual, institutional, local, national, and global) ; by using the common goods lens, we can interpret inward student mobility within at least the following three sets of relations (see also table 7 ): (1) common and individual. this term (common goods) may differentiate the common and individual benefits accruing from inward student mobility; (2) governmental and institutional. this term is useful to explain policies and initiatives in the national context, thus government's influences, support, regulations on heis can be reflected; (3) national and global. this concept is multidimensional, and it is helpful to explore the balance, consistency, and tensions between policies and practices concerning inward student mobility at different levels (deneulin and townsend 2007; unesco 2015; marginson 2018; tian and liu 2019) . as we have defined in this paper, common goods are multilevel, including local, national, and global common goods. based on the research findings, inward student mobility in china contributes to (global) common goods in the following five aspects, which are highly valued by interviewees from both the government and heis: (1) talents with global perspectives and respect to other cultures. through international education, international students will become global citizens who care for the whole world and make contributions to form a more inclusive, tolerable, and equal world. (2) the wider sharing of educational resources. inward student mobility intensifies educational cooperation between different countries and accelerates the process of mutual recognition of credit and certificate, which boosts the sharing of educational resources. (3) increased cultural exchanges, deeper mutual understandings, and diverse campus culture. international students help to form a more diverse and inclusive campus culture, and they can also be portrayed as cultural ambassadors to press ahead cooperation and exchanges between the host country and their home countries. (4) the improvement of policies and practices. the presence of international students may shape higher education policies and practices to be more efficiency-, service-, and reputation-driven. (5) the potential economic growth point. inward student mobility can be regarded as economic agent or driver of knowledge and eventually of economic growth. for some universities, recruiting international students is an alternative source of revenue to ensure their financial sustainability. in fact, the economic aspect of inward student mobility is particularly strong among anglo-saxon countries (geddie 2015) and extensively studied as the "marketization approach" to international education (findlay et al. 2017 ). however, our results suggest that china gives less weight to the economic aspect of inward student mobility at this moment, agreeing with jiang (2010b) that the economic benefits of international students are not a focus of higher education in china. nevertheless, chinese interviewees highlight the increased soft power brought by inward student mobility, which can be regarded as a national common good within a national boundary. inward student mobility intensifies academic and scientific exchanges which are central to soft power theory, and this can also be inspired by observations of how the us has gained friends among political elites of hostile countries by means of student exchanges (nye 2004; ma and zhou 2018) . this theory suggests that international students with pleasant firsthand experiences of studying abroad will admire the host country's academic, social, and political system and, in turn, nudge or steer cooperation between their home countries and the host country, push education, society, and politics at home in the direction desired by the country they studied and lived in (haugen 2013 ). this is not only an opportunity for chinese language and culture to enter the global platform but also for soft power expansion. in general, these are the perceived potential national and global common goods created by inward student mobility in china, which undoubtedly reveal the unique value of inward international students. despite that, the research findings suggest that attracting and educating international students in china is never a pure business transaction with only market and economic orientations, as both the chinese government and heis attach great importance to the development and welfare of international students who desire educational, social, personal, and professional development that underpins the process of "becoming" and "self-formation" (marginson 2014; tran 2016) . therefore, international education in china tends to be a practice of reciprocity, within the concept of the "ethics of care" (blackburn 1997; abdullah et al. 2017) , which suggests both parties (international students and heis) should ensure each party is involved in the benefits of internationalization from their interaction and have genuine concern over the well-being of each other in the provision of international education. in this sense, international education in china could be viewed as a global common good itself, since both parties need each other to achieve their intended goals, which thinks highly of collective endeavor, shared participation/engagement, inclusion, and responsibility, meshing with unesco's (2015) definition of global common goods. hence, international students should never be simply treated as "cash cows" and their desire to have a life-changing experience (through international education) should be valued. in turn, the cared for (international students) should acknowledge the energy and efforts invested by the carer (heis) to enhance the value of the relationship between both parties. in light of this, the narrative of ethical caring could build a conducive intercultural ecosystem to support the internationalization activities, boosting greater learning between students and university staff, and ensuring mutuality in strategies developed for cooperation and collaboration with foreign partners, which is a key virtue in internationalization from the outset (blackburn 1997; abdullah et al. 2017) . the results of this study show that in the chinese context, inward student mobility has not only emerged as a significant favorable pull factor for the internationalization of higher education but also contributes significantly to both national and global common goods. by examining various recent national policies and university strategies, it is easy to conclude that the inward student mobility has been given a special priority in the chinese context. this reflects the positive attitude of both the chinese government and heis, and contradicts the idea that the emphasis laid on international students is less in china, and the view that international students are less important to chinese universities, because they are not seen as human resources due to their generally low academic performance (jokila 2015) . however, some researchers believe that the chinese government's efforts to promote inward student mobility are only to achieve the policy objectives of diplomacy and education, while ignoring the contributions of international students in culture, politics, and economy (cheng et al. 2013) . obviously, this comment is not objective, because the findings of this study provide evidence that promoting mutual understandings and improving the quality of higher education through international education is an established strategy for the development of higher education in china, and this strategy does not exist in isolation. it falls in line with china's development goals of improving national soft power and cultural influence (wang et al. 2014) . also, the neoliberals believe that china's efforts for educational improvement are characterized by export (zheng 2010) . this comment seems to simply equate the internationalization of higher education with a commercial trade activity, overlooking the internationalization of chinese higher education contains unique cultural appeal. chinese culture advocates "harmony in diversity" and believes that different value systems should coexist harmoniously. according to our research findings, the dissemination of chinese culture through international education is currently an important tool for china to build better political and economic relations with other countries, and the "bridge" is the international students who understand and respect chinese culture, show kindness to china, and are on intimate terms with china. this is aligned with schulte's (2012) idea that chinese education actually takes place at the interface of global ideas and national strategies. internationalization and nationalism are not necessarily in conflict with each other in china; instead, special cultural aspects could benefit from international relations (gu 2001) . specifically, this study uncovers the different policies and strategies of internationalization between the two case universities. due to the different institutional needs and developmental goals in the process of internationalization, the university with lower degree of internationalization lays more emphasis on the quantity of international students, while the university with higher level of internationalization has shifted its focus from merely increasing the number of international students to improving the quality of international students as well as diversifying students' source countries. this strategy is supported by wang et al.'s (2014) study, which propose that the effectiveness international education strategy mainly hinges on the quality, not the number, of international students. tellingly perhaps, the research results also illustrate that z university gives close attention to attracting students from the belt and road countries. on the one hand, this is due to z university's geographic location (it is located near the southern chinese border) which makes it have a unique advantage to attract international students from neighboring nsr countries; on the other hand, this reflects that countries working with china on the belt and road initiative are likely to see more opportunities in educational cooperation and involvement (chen and wen 2018) . however, there are some problems and tensions among the policies, practices, and (global) common goods that can be observed in the results and findings, for example, the employment issue. though both the chinese government and heis endeavor to attract more international students studying in china and build a global talent pool, the employment mechanism is not in place, failing to retain talents who hope to work and stay in china. but very recently, a few coastal metropolitan cities in china have re-tailored their policies on international student employment (but only for postgraduate students) to secure highly skilled international graduates, and potentially, these policies may be expanded to the whole country within the coming years. the present research is both people concerned and policy focused, addressing issues by investigating perspectives from people directly involved in the process and examining some key policy documents related to inward international students in china. the findings of this study reveal that inward student mobility brings benefits to china and contributes to global common goods flowing worldwide, which can also be regarded as a reason for the supporting of national and institutional policies and strategies on international students. though there are also some tensions in the policies and practices for international students (e.g., employment and management issues), relevant efforts have been exerted to address them. most importantly, this study suggests that international education in china, as an essential part of higher education internationalization, can be seen as a global common good, which emphasizes collective endeavor, shared participation, inclusion, and responsibility. nevertheless, the current coronavirus (covid-19) threat may cast a shadow over booming international education in china and affect china's ability to attract more international students in a certain period. this puts forward new requirements for both countries and universities (not only china and chinese universities) to raise awareness of the need for risk management and crisis response strategies to ensure sustainability that highlighted by unesco (2015) as an indispensable role and responsibility of diverse stakeholders in the collective quest for the common good. what are the policies and strategies of your institution in relation to inward student mobility? 2. in your hei's internationalization strategy, what priority do you place on inward student mobility? how internationalized is your hei? what is the profile of international students in your institution at present? how does this compare to other institutions? has it changed over time? 4. [only for those leaders responsible for detailed administration of international students] what are the requirements of migration policy and national security as they directly affect inwardly mobile cross border students, including (1) initial entry, (2) regulation during the student sojourn are there other areas of government policy, apart from policies on education, migration, and national security, which, in your opinion, affect inwardly mobile cross border students? 6. how have policy changes in recent years impacted the student profile, and the student experience? looking at inward student mobility as a whole, not just your hei, to what extent are inward student mobility policies and strategies implemented successfully, given the policy objectives what are the benefits of student mobility and risks/disadvantages for the hei? 8. what are the main global common goods, benefits flowing not just to your country but to other countries, including the countries of student origin, that are created or augmented by inward student mobility in your nation? what are the experiences of inwardly mobile international students in the country of education, in relation to issues of welfare, human rights, personal agency and dignity? what problems and limitations face these students? 11. [only for selected leaders likely to tackle the question effectively] what are the benefits of a degree from your institution, when students return to their home countries? what do they learn at your institution, or gain in other ways, that is not available in their home country? do graduate cross-border students keep in touch with your institution after they leave? 12 the stories they tell: understanding international student mobility through higher education policy what have we learned from policy transfer research? dolowitz and marsh revisited internationalization: some ethical challenges from conceptualization to measurement of higher education as a common good: challenges and possibilities. higher education a survey of the influencing factors for international academic mobility of chinese university students three waves of international student mobility (1999-2020). studies in higher education yi dai yi lu" chang yi xia lai hua liu xue sheng jiao yu: shi ming, tiao zhan he dui ce [international education under the belt and road initiative: missions, challenges and solutions lun lai hua liu xue sheng jiao yu li shi fen qi wen ti strategic choices of development of china international education in the early twenty-first century public goods, global public goods and the common good international student mobility: european and us perspectives. perspectives: policy and practice in higher education gao deng jiao yu lai hua liu xue sheng de bian hua qu shi yan jiu-ji yu jin shi wu nian tong ji shu ju de fen xi marketization, marketing and the production of international student migration policy mobilities in the race for talent: competitive state strategies in international student mobility education in china and abroad: perspectives from a lifetime in comparative education jing ji quan qiu hua shi jiao xia de guo ji ren li zi yuan guan li wen ti yan jiu-ji yu lai hua liu xue sheng jiao yu guan li de si kao [a study on international human resource management from the perspective of economic globalization-based on the education management of international students in china china's recruitment of african university students: policy efficacy and unintended outcomes. globalization, societies and education the common good and christian ethics lai hua liu xue sheng jiao yu de ping jing wen ti ji jie jue cuo shi [the bottleneck problems and solutions of lai hua liu xue sheng jiao yu de zhan lue ding wei: ji yu duo yin su de fen xi gai ge kai fang yi lai zhong guo gao deng jiao yu guo ji hua zheng ce de shan bian: ji yu shu ju yu zheng ce de lian jie [the evolution of china's higher education internationalization policies since reform and opening up: based on the connection of data and policies the internationalization of higher education with chinese characteristics: appadurai's ideas explored research on internationalization in higher education student mobility and internationalization: trends and tribulations education as a public and common good: reframing the governance of education in a changing context higher education as a public and common good yi dai yi lu" yan xian gao duan liu xue sheng mian lin de tiao zhan ji qi dui ce [the challenges and solutions for high-level student self-formation in international education the outline of project 1.2: internationalization of he as a public good. london: centre for global higher education higher education and the common good world higher education under conditions of national/global disequilibria, cghe working paper 42 qualitative content analysis: theoretical foundation, basic procedures and software solution concise statistics for international students in china. beijing: department of international cooperation and exchange in the ministry of education the student experience of mobility soft power: the means to success in world politics new directions in studying policies of international student mobility and migration. globalization, societies and education world culture with chinese characteristics: when global models go native world-class universities: towards a global common good and seeking national and institutional contributions rethinking higher education in china as a common good mobility as 'becoming': a bourdieuian analysis of the factors shaping international student mobility rethinking education: towards a global common good? paris: unesco retrieved professional education, capabilities and the public good: the role of universities in promoting human development quan qiu hua bei jing xia de guo ji xue sheng liu dong yu zhong guo zheng ce xuan ze [international student mobility and china's policy choice under the background of globalization gai ge kai fang 30 nian de lai hua liu xue sheng jiao yu tui dong gao deng jiao yu quan qiu liu dong jian she ren lei ming yun gong tong ti [promoting the global mobility in higher education and building a community of shared future for mankind neoliberal globalization, higher education policies and international student flows: an exploratory case study of chinese graduate student flows to canada publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations acknowledgments this study was partly funded by the esrc (economic and social research council) and hefce (higher education funding council for england) (grant number: es/m010082/1). interview questions for university leaders key: cord-327721-y39751g4 authors: zhang, yan; cao, xiaochen; wang, pu; wang, guixiang; lei, guanghui; shou, zhexing; xie, simiao; huang, fei; luo, na; luo, mingyan; bian, yueran; zhang, jingyuan; xiao, qiang title: emotional “inflection point” in public health emergencies with the 2019 new coronavirus pneumonia (ncp) in china date: 2020-07-19 journal: j affect disord doi: 10.1016/j.jad.2020.07.097 sha: doc_id: 327721 cord_uid: y39751g4 background: the outbreak of the new coronavirus pneumonia (ncp) in wuhan, hubei, has caused very serious consequences and severely affected people's lives and mental health. the outbreak will cause bad emotions such as tension, anxiety, fear, and so on. college students who have returned home from school face infection, isolation, and delay in starting school, and thus, their emotional stress should be observed. methods: this study used self-designed questionnaires and artificial intelligence (ai) to assess and analyze the emotional state of over 30,000 college students during the outbreak period in january (t1) and home quarantine in february (t2). this survey used online questionnaire (www.wjx.cn) to investigate the emotion information of college students. results: in the t1 survey, the "typhoon eye effect" appeared. college students in hubei are calmer than those outside hubei in t1. however, in t2, an emotional "infection point" appeared, there was an "exposure effect", the negative emotions of students in hubei largely increased and became higher than students outside hubei. conclusion: this survey found that there is an emotional "infection point" in february among college students, especially in the hubei area. college students in hubei are calmer than those outside hubei in t1. in contrast, college students in hubei were more nervous and scared than those outside hubei in t2. this epidemic has caused the students to experience significant pressure and negative emotions. therefore, universities and society should pay attention to their emotional adjustment, there are some suggestions such as establish the mental health organizations, test students' emotion status regularly. since december 2019, there has been a large-scale outbreak of new coronavirus pneumonia (ncp) in wuhan (chen, n. et al., 2020) . patients with pneumonia have been diagnosed in all 34 provincial administrative regions . the scale and severity of the pneumonia outbreak caused many cities, including wuhan (lupia et al., 2020) , to close, especially the prefecture-level cities in hubei province (kai & jon, 2020) . this public health emergency made wuhan, hubei, china the focus of the world (wilson & chen, 2020) . on february 11, 2020, the world health organization (who) proposed naming the new coronavirus covid-19; and on february 8 (team, 2020) , the chinese state council announced that the new coronavirus-infected pneumonia was temporarily named "novel coronavirus pneumonia", or ncp for short. public health emergencies are major infectious disease outbreaks that cause or are likely to cause serious damage to and affect public health, including mass unexplained diseases, major food and occupational poisoning, and other events. infectious diseases have become one of the major global public health threats in the 21st century (who, 2007) . emergencies are divided into four levels in china according to the severity of the event, and this new coronavirus pneumonia belongs to the particularly significant level i. the last public health emergency in china was sars in 2003 . the sars epidemic of 2003 spread rapidly to over 30 countries with more than 8000 reported cases, resulting in 774 deaths worldwide (who, 2005) . however, the current pneumonia outbreak is even more severe than that of sars in 2003 (riou & althaus, 2020) . compared with other diseases, cocid-19 has a longer incubation period, is more prone to mutation, has a wider spread, and is more infectious, it will cause more serious physical and mental problems. fortunately, our experience with sars has provided valuable information about how to make a rapid response (wenshe, s, tyler, & shiqing, 2020) . china has moved quickly to contain the outbreak (pan et al., 2020) . live animal markets throughout the country have been suspended (khan, 2020) . previous studies showed that public events would devastate communities in terms of lives and property, and they are difficult to recover from without outside help. for example, a study of ptsd after earthquake over half of people exhibited internalizing problems requiring clinical intervention (jang, lee, & kim, 2020) . another study showed that the exposure to an outbreak of a severe infectious disease can, like other disaster exposures; lead not only to ptsd but also to other psychiatric conditions, such as alcohol abuse/dependence (wu et al., 2008) . other studies also revealed that the residents in high disaster-prevalent regions consistently developed more intense symptoms of intrusion, and avoidance (lee, chi, chung, & chou, 2006) . in previous studies, many researchers have studied the psychological state of people after the sars epidemic, such as the psychological stress of the medical staff after sars (feng et al., 2020; lee, et al., 2005) and covid-19 (wu, zhang, &wang, et al., 2020) . raphael (1980) documented that rescue workers often suffer from psychiatric and posttraumatic distress. a study of the long-term psychiatric morbidities among sars survivors by mak (mak, chu, pan, yiu, & chan, 2009 ) revealed that ptsd was the most prevalent long-term psychiatric condition. ptsd refers to individuals who have experienced, witnessed, or encountered one or more actual deaths involving themselves or others; who are threatened with death or severely injured; or whose physical integrity is threatened, resulting in delayed appearing and persistent mental disorders (kashdan, uswatte, & julian, 2006) . some scholars (andreski, chilcoat, & breslau, 1998; north, kawasaki, spitznagel, & hong, 2004) have perceived that pain severity was a factor associated with ptsd at 30 months post-sars, which is consistent with previous disaster studies. another significant factor currently associated with ptsd is the functional impairment checklist disability score. it was shown that the disability score was a stronger predictor for chronic ptsd compared with the symptoms score, which measures subjective lung function and fatigability (mak et al., 2010) . in the follow-up research on the psychological stress of college students in the future, these two methods can be used. the 2019-ncov infection caused clusters of severe respiratory illness similar to severe acute respiratory syndrome coronavirus and was associated with icu admissions and high mortality . the researcher chew found that forty-one percent had scores indicative of a posttraumatic stress disorder (ptsd) after sars; about 30% had likely anxiety and depression (chew et al., 2006) . therefore, we need to pay more attention to the ptsd caused by public health emergencies. emotion is a collective term for a series of subjective cognitive experiences, and it refers to a person's attitude and experience towards objective things and corresponding behavioral responses. public events will influence emotions, a previous research found that participants were exposed to unprecedented horror following the indian ocean tsunami in khao lak, thailand (sattler, assanangkornchai, moller, kesavatana-dohrs, & graham, 2014) . in addition, some researchers studied adolescent emotion status following 2013 boston marathon terrorist, which demonstrate that adolescents who engaged in catastrophizing had greater ptsd symptoms postattack, with more tension and depression (jenness et al., 2016) . although emotion is a subjective psychological reaction, it often affects people's external performance and people's cognition. sylvie and warren (2007) said "the emotional valence of experience is thought to distort our subjective sense of time". in contemporary society, people are under great pressure, and crises such as this new coronavirus pneumonia are prone to cause psychological stress (kjaer, hanspeter, & ulrich, 2004) . some researchers (obergriesser & stoeger, 2020) found that joy or boredom can affect students' learning strategies to a certain extent. therefore, this study takes the emotional reactions of college students in the face of public health emergencies as the research content, and discusses their emotional states, degree of response, emotional responses, and self-regulation to give relevant suggestions. in this survey, college students are selected as the research samples. as of the end of 2018, wuhan has 84 ordinary colleges and universities, which makes it the capital city with the largest number of colleges and universities in china and the largest number of students in the world. the college students surveyed in this survey will be the mainstay of society in the future. their mental health is the focus of our attention, and it deserves the attention of society. in addition, young people (aged 15-29 years) are more susceptible to outside influence (li, dorstyn, & jarmon, 2020) , which will cause more emotion changes. at the same time, the working forms and methods of mental health education is weak, which causes the emerging problems of mental health that college students have been confronted (li, 2020) . prior to this study, a number of scholars have investigated the social stress responses after a public crisis event, but their research samples of college students were very small; therefore, this investigation is necessary. in past research, researchers have discovered a very special phenomenon of regional perceived risk: typhoon eye effect (gilovich, medvec, & savitsky, 2000) , which is when the recognition of risk events by groups at the center of risk events is lower than the perceived risk of the surrounding areas. domestic researchers have made similar findings on the public's perceived risk during the sars period. chinese researchers found that people's risk awareness and psychological stress during the peak period of sars were lower than those during the low peak period. and there is another "exposure effect", which means that as the epidemic became worse, residents at the center of epidemic exposed a more severe negative emotion. according to the theory of the "typhoon eye effect" and "exposure effect", we can suppose that there is an emotional "infection point" among college students in the hubei area, it refers to an emotion change among college students (negative emotion: hubei is lower than outside hubei in t1, hubei is higher than outside hubei in t2.) in the second week after wuhan was closed, that is, january 29-30, 2020, a total of 17,935 people filled out the questionnaire, of which 47.99% were located in hubei province (n=8607, age 19.56±1.79, 54.19% female) and 52.01% were located outside hubei province (n=9328, age: 19.98±2.05, 58.55% female). in the fourth week, february 17-18, 2020, 16,428 people around the country filled out the questionnaire. among them, 54.03% were in hubei province at that time (n=8876, age: 20.99±2.19， 57.59% female), and 45.97% were outside hubei province (n=7562, age: 20.02±2.85, 57.23% female). the study was designed in accordance with the tenets of the declaration of helsinki. approval from the ethical authority of the school of educational science, huazhong university of science and technology was granted. confidentiality was ensured by assigning anonymous codes to the self-reported questionnaires, and participants signed a statement confirming informed consent. this survey used online questionnaire (www.wjx.cn) to investigate the emotion information of college students in t1 and t2. we distributed questionnaires online through wechat, qq, and sina microblog. participants provided informed consent, then provided socio-demographic information and completed measures of emotion. two surveys utilized the same questionnaires. we used a self-designed questionnaire to collect emotional state data. socio-demographic information the statistical analyses were performed using spss 20.0 for windows (spss inc., chicago, illinois). the statistical methods and data analysis results are described in detail as follows. as for the survey used the online questionnaire (www.wjx.cn), we can choose to refuse any empty questions during the questionnaire preparation, so the survey can ensure that each questionnaire collected is valid. moreover, we utilized artificial intelligence (ai) technology to count the data of 34 provinces, import the averages and percentages of variables in t1 and t2, and python to export the maps of statistic automatically. in the t1 survey on emotional states, the descriptive analysis of spss revealed that the emotional states of most students was calm, which indicates that most students can maintain a relatively calm state of mind in the face of sudden epidemic situations without excessive panic. then the chi square test found that in t1, college students in hubei province were significantly calmer than students outside hubei province (percentage: 0.71 > 0.70, p < 0.05). at the same time, the percentage of variable "bored" of students in hubei province was also significantly higher than students outside hubei province (0.62 > 0.58, p < 0.001). it was found in the investigation that the students in the most serious epidemic area, hubei, were more bored and calmer than those outside hubei. however, other negative emotions exist among college students outside hubei, such as horror (0.10 > 0.08, p < 0.001), fear (0.14 > 0.11, p < 0.001), anxiety (0.39 > 0.34, p < 0.001), tension (0.22 > 0.18, p < 0.001), anger (0.07 > 0.05, p < 0.001), sadness (0.13 > 0.11, p < 0.01), moving (0.16 > 0.15, p < 0.05) and optimism (0.07 > 0.06, p < 0.05), which are significantly at higher levels than those of the students in hubei. it indicated that students outside hubei province are more sensitive to the epidemic in t1, and they are under greater psychological pressure and negative emotions than students from hubei province. this is the typhoon eye effect of disaster psychology. the subjects in the most serious epidemic areas had lower anxiety levels than those in the peripheral epidemic areas (fig. 1) , which is correspond to the hypothesis. in addition, more than half of the college students are bored. a considerable number of students (36.25%) also expressed their concern. they were most worried about the following: newly diagnosed and suspected cases (69.38%), a shortage of medical supplies (67.03%), and infected persons who were not isolated (64.7%). about one-third of the students have large negative emotions and need active guidance and care. using the same chi square test to analyze the data of t2, we found that the results were reversed. in t2, the students in hubei province had significantly higher levels of horror (0.10 > 0.09, p < 0.05), nervousness (0.21 > 0.20, p < 0.01), and fear (0.11 > 0.10, p < 0.01) than those outside hubei. the positive emotions of students in hubei, such as calmness (0.77 < 0.80, p< 0.001) and optimism (0.11< 0.12, p< 0.05), are significantly lower than those of college students outside hubei. the negative emotions easily generated by bystanders such as anger (0.11< 0.14, p < 0.001) and sadness (0.17 < 0.19, p< 0.01) are significantly lower than outside hubei province. from these data, it indicated that during the period of home isolation, college students in hubei province showed more negative emotions due to their long-term exposure to the epidemic. this is called the "exposure effect", the multi-view effect, (simple) exposure effect, (pure) contact effect, etc. from the comparison of the results of the t1 and t2 sentiment surveys through chi square test, we can see that fear (0.12 > 0.11, p < 0.01), anxiety (0.35 > 0.31, p < 0.001), tension (0.21 > 0.20, p < 0.05), anger (0.10 > 0.08, p < 0.001), sadness (0.15 > 0.14, p < 0.001), helplessness (0.18 > 0.17, p < 0.05) increased significantly. however, boredom decreased significantly (0.39 < 0.42, p < 0.001). to contrast emotion changes between t1 and t2, we used artificial intelligence (ai) technology to calculate the data of the 34 provinces, autonomous regions, municipalities around the country, and the hong kong and macao special administrative regions regarding tension (fig. 2) , anxiety (fig. 3) and optimism (fig. 4 ). in the t1 survey, college students in hubei were more optimistic than those outside hubei, the "typhoon eye effect" appeared, the students in hubei did not realize the severity of the epidemic and felt that the virus was far away from them (zheng et al., 2015) . the hubei provincial government did not take strong measures in time (zhang, huang, & wei, 2020) . the information received by people is not symmetrical with the facts, it will cause greater panic later (forsell et al., 2019) . this sent a false signal to the people: this new disease is not serious and can be prevented and controlled. thus, the true situation of the epidemic was concealed. furthermore, the students outside the province appeared to be more panicked due to the asymmetry of information, and the media reported that the epidemic was very serious. in addition, humans by nature are more sensitive to dangerous atmospheres, fear can activate long-term memory, and people can remember bad things for longer (rafiq, batool, liaquat, & haider, 2020) , so psychological negative emotions, such as tension, fear, anxiety, and horror are generated. therefore, the emotional responses of students outside hubei province are stronger. the t1 survey results also showed that more than half of college students are bored. this may be caused by the outbreak and having to stay at home while quarantined, but to some extent, it also exposes the problems that college students are incapable of organizing their own time and that their daily lives are relatively empty (maccann, fogarty, & roberts, 2012). however, regarding the "emotion map" in t2, some negative emotions' distributions were reversed and there was an "exposure effect". negative emotions such as fear, horror, and tension among the students in hubei province started to increase. the negative emotions of college students in hubei exposed because they were in it, the one near the disaster were prone to be in panic (derrick, green, & wand, 2019; johnson, mcleish, shear, & privitera, 2018) . while the negative emotions except anger and sadness of the participants outside hubei province began to decline. this is because after t2, the central government set up an investigation team to go to wuhan to discover the truth for the public. meanwhile, anger and sadness of students in t2 increased, these emotion usually appear on bystanders (buttelmann & buttelmann, 2017) . only then did the public realize that the number of confirmed cases had skyrocketed, and that students in hubei had gradually realized the severity of the outbreak. people in the hardest hit area of the epidemic started thinking seriously about the dangers they may be facing, they became more nervous and scared than before (birstein & gusky, 2020) . due to the outbreak of the epidemic, coupled with the restrictions on medical treatment in wuhan, many confirmed diagnoses and suspected cases were not counted, the citizens were in danger (gao, veeresha, baskonus, prakasha, & kumar, 2020) . a large number of cases were cleared on february 13, and more than 10,000 new cases were exposed on that one day. in particular, the media frequently reported the news of the deaths of the frontline medical staff, which made students in the severely affected areas no longer dare to despise the epidemic, and they became more nervous and fearful. after the state put forward protective measures requiring home isolation for fourteen days, cutting off transportation, banning group activities, and installing roadblocks in urban villages, these measures have created intense tension among the people living in groups. there is also the stress symptom of "seeming as being infected" caused by too much browsing of the relevant news every day, which directly affects the emotions of students, they became more sensible and anxious to disease, this is a mental tension (peng et al., 2019) . in particular, they see the resumption of work in various places and the resumption of classes in other areas being just around the corner, but the resumption of classes in hubei universities is far away, which makes students of hubei universities feel anxious. most importantly, students in hubei's hardest-hit areas still have many difficulties. for example, living resources are scarce, and medical resources and health protection still have problems. in addition, from time to time, unbelievable false news becomes popular, severely disturbing the emotions of college students in hubei who cannot distinguish between true and false and resulting in high levels of wide-ranging anxiety. this study have found an emotional inflection point during the outbreak of epidemic in february, this was an innovation, and full of value. we can seize the emotion changes to guarantee our mental health, which is valuable. after february 4, the number of confirmed cases around the country, except hubei province, continued to decline. by february 23, there had been no new confirmed cases in 24 provinces. in addition, the appearance of cured patients in february injected people with hope, and the number of cured patients increased. on february 22, more than 20,000 people were cured nationwide, which has increased people's confidence in fighting the epidemic. college students across the country have generally experienced a decline in the negative emotions such as fear, anxiety, helplessness, and boredom and a continuous rise in moving, optimistic, and calm emotions. however, the establishment of a large number of square-level hospitals, the fixed-point support of multiple medical teams across the country, the high number of confirmed patients in hubei province in t2, and the news of medical staff deaths cause the emotions of fear, tension, and sadness to constantly increase. in addition, some students said that because of the epidemic situation, some people discriminate against hubei citizens, and the information was leaked, which made hubei college students angry. the survey also showed that in the description of students' recent state, the word most frequently mentioned was "stress". the stressors include worrying about infecting family members, feelings of uncertainty, inadequate staffing levels (taylor, white, & muncer, 1999) , the virulence of the disease and inappropriate equipment (carolyn & karen, 2003) , personal danger, and nosocomial spread. the severity of the new coronavirus pneumonia is self-evident. college students are full of anxiety and concerned about the epidemic. although most students can maintain calm emotions in the face of such major public health events, the government and society still cannot ignore the groups with higher trauma exposure. in particular, students in the worst-hit areas, hubei province, have witnessed and experienced the epidemic every day, and even the relatives and friends around some students have been diagnosed. they need to intervene in advance in psychological crises, and their traumatic stress responses must be treated. since the outbreak of the epidemic, a lot of negative information about the epidemic has appeared on the internet and even on many mobile phone software programs. we know that terrible information will bring pressure and worry to people and will affect people's emotions. so the way college students handle negative news is particularly important. some researchers have found that people will usually want positive news rather than negative news (alex & stephen, 2004; fredrickson, tugade, waugh, & larkin, 2003) . though the current study has found that the majority of college students could properly handle bad news, a part of students are irrational, we should pay attention to this group. the researcher lyrakos (2014) has found that social events and education can reduce stress, worry and emotions. the outbreak area of this emergency was in hubei province, which is the hardest hit area. there are a large number of new cases every day. as things happen near them, students in the hardest hit area are more likely to feel fear and worry, especially when they did not return to their hometown in the new year. importance should be attached to whether students that are stranded at school can be guaranteed to be able to live normally. the government and universities should provide favor to students. we provide the following major suggestions. (1) universities should set up psychological counseling hotlines to provide psychological counseling for students in the hardest hit areas. (2) psychological protection teams should be set up in each community to regularly assess the psychological situations of students in the hardest hit area. (3) students are recommended to strengthen their psychological construction, exercise more, maintain regular work and rest, and stabilize their emotions. long-term health problems and the loss of function of students in the worst-hit areas may play an important role in forming ptsd (carr et al., 1997) . (4) people should talk and exercise when they are in a bad mood and seek intervention and other methods to guide them. (5) officials should follow up on the psychological condition of the students in the severely affected area in a timely manner after the epidemic, and prepare necessary post-traumatic psychological stress treatment programs. the outbreak of the new coronavirus pneumonia disrupted the original life and study plans of college students. this study has found that more than half of the students expressed that they were bored at home, which also exposed some problems to a certain extent, such as college students do not reasonably arrange their free time. if school or family education can teach students how to arrange their time independently and how to find their own interests and long-term development, this problem may be solved. they should use positive emotions to bounce back from negative emotional experiences (tugade & fredrickson, 2004; veruska et al., 2013) . thus, we provide the following suggestions: (1) cultivate students' core literacy, increase students' self-learning section, and enable students to learn to arrange their study time rationally; and (2) cultivate students' personalized interests and hobbies, develop them in an all-round way, and guide students to persist in their hobbies for a long time. stress is an emotional response that may be caused by emergencies. in any biological disaster, themes of fear, uncertainty, and stigmatization are common and may act as barriers to appropriate medical and mental health interventions (xiang et al., 2020) . the number of confirmed cases increasing every day, urgent needs for medical supplies in major hospitals, and other negative news will bring pressure and negative emotions to college students. stress and trauma research has traditionally focused on the negative sequelae of adversity (meyerson, grant, carter, & kilmer, 2011) . therefore, letting students learn to relieve stress and find suitable channels is also what contemporary society and schools have to do. the education department should add mental health courses at all levels of schools and educational institutions; organize group counseling once a week; and teach students how to properly treat their emotional problems, how to control themselves, and how to seek help if they cannot control it. through this survey, we know that there will be an emotional inflection point during public events, so the government and schools of all levels should emphasize students' mental health and grasp the emotional inflection point, to prevent mental diseases. the government and schools should: (1) pay attention to students' mental health regularly; (2) contact with the group as a unit, communicate troubles, and timely report to schools. (3) schools and families should communicate closely, and problems should be resolved in time. this survey found that there is an emotional "infection point" in february among college students, especially in the hubei area. college students in hubei are calmer than those outside hubei in t1. in contrast, college students in hubei were more nervous and scared than those outside hubei in t2. this epidemic has caused the students to experience significant pressure and negative emotions. therefore, universities and society should pay attention to their emotional adjustment, there are some suggestions such as establish the mental health organizations, test students' emotion status regularly. lgh. this study was supported and granted by the "national social science fund emergency management system construction research special project (20vyj040) and national natural science foundation", "central china think tank special key projects (2020hzzk031)", and "natural science foundation of hubei province (2019cfb425)" to yz. the study was designed in accordance with the tenets of the declaration of helsinki. approval from the ethical authority of school of educational science, huazhong university of science and technology was granted. confidentiality and the statement confirming informed consent was managed by placing anonymous coding of one self-report questionnaires. this manuscript has not been published or presented elsewhere in part or in entirety. all study participants provided informed consent, and the study design was approved by the appropriate ethics review boards. all the authors have approved the manuscript and agree with submission to your esteemed journal. there are no conflicts of interest to declare. all authors have no conflict of interest. positive change following trauma and adversity: a review post-traumatic stress disorder and somatization symptoms: a prospective study treatment of panic disorder by trigeminal nerve manipulation: a case series the influence of a bystander agent's beliefs on children's and adults' decision-making process responding to the severe acute respiratory syndrome (sars) outbreak: lessons learned in a toronto emergency department psychosocial sequelae of the 1989 newcastle earthquake: ii. exposure and morbidity profiles during the first 2 years post-disaster epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in wuhan, china: a descriptive study from sars-cov to wuhan 2019-ncov outbreak: similarity of early epidemic and prediction of future trends quality of life and psychological status in survivors of severe acute respiratory syndrome at 3 months postdischarge assessing and responding to anxiety and panic in the emergency department chinese medical personnel against the 2019-ncov development of a very brief scale for detecting and measuring panic disorder using two items from the panic disorder severity scale-self report what good are positive emotions in crises? a prospective study of resilience and emotions following the terrorist attacks on the united states on a new study of unreported cases of 2019-ncov epidemic outbreaks the spotlight effect in social judgment: an egocentric bias in estimates of the salience of one's own actions and appearance clinical features of patients infected with 2019 novel coronavirus in wuhan post-traumatic stress disorder and behavioral problems of parents and children after the 2015 nepal earthquakes catastrophizing, rumination, and reappraisal prospectively predict adolescent ptsd symptom onset following a terrorist attack panic and epilepsy in adults: a systematic review will novel virus go pandemic or be contained? gratitude and hedonic and eudaimonic well-being in vietnam war veterans coronavirus in china. the lancet. respiratory medicine does acute stress disorder predict post-traumatic stress disorder in traffic accident victims? analysis of a self-report inventory facing sars: psychological impacts on sars team nurses and psychiatric services in a taiwan general hospital ageing and psychological response during the post-sars period instructional research on mental health education for college students from the perspective of positive psychology identifying suicide risk among college students: a systematic review 2019-novel coronavirus outbreak: a new challenge epa-1595 -factors affecting negative emotions of depression, anxiety and stress in online social networking services strategies for success in education: time management is more important for part-time than full-time community college students long-term psychiatric morbidities among sars survivors risk factors for chronic post-traumatic stress disorder (ptsd) in sars survivors posttraumatic growth among children and adolescents: a systematic review the course of ptsd, major depression, substance abuse, and somatization after a natural disaster students' emotions of enjoyment and boredom and their use of cognitive learning strategies -how do they affect one another? learning and instruction lessons learned from the 2019-ncov epidemic on prevention of future infectious diseases brain structural abnormalities in emotional regulation and sensory processing regions associated with anxious depression blockade of muscarinic receptors impairs reconsolidation of older fear memory by decreasing cholinergic neurotransmission: a study in rat model of ptsd disaster: the helper's perspective euro surveillance : bulletin europeen sur les maladies transmissibles indian ocean tsunami: relationships among posttraumatic stress, posttraumatic growth, resource loss, and coping at 3 and 15 months how emotions colour our perception of time nurses' cognitive structural models of work-based stress note from the editors: world health organization declares novel coronavirus (2019-ncov) sixth public health emergency of international concern. euro surveillance : bulletin europeen sur les maladies transmissibles resilient individuals use positive emotions to bounce back from negative emotional experiences the role of positive emotion and contributions of positive psychology in depression treatment: systematic review. clinical practice and epidemiology in mental health learning from the past: possible urgent prevention and treatment options for severe acute respiratory infections caused by 2019-ncov summary of probable sars cases with onset of illness from 1 the world health report 2007: a safer future: global public health security in the 21st century travelers give wings to novel coronavirus (2019-ncov) psychological stress of medical staffs during outbreak of covid-19 and adjustment strategy timely mental health care for the 2019 novel coronavirus outbreak is urgently needed. the lancet geographical distance to the epicenter of covid-19 predicts the burnout of the working population: ripple effect or typhoon eye effect? the more involved in lead-zinc mining risk the less frightened: a psychological typhoon eye perspective key: cord-164964-vcxx1s6k authors: kharkwal, himanshu; olson, dakota; huang, jiali; mohan, abhiraj; mani, ankur; srivastava, jaideep title: university operations during a pandemic: a flexible decision analysis toolkit date: 2020-10-20 journal: nan doi: nan sha: doc_id: 164964 cord_uid: vcxx1s6k modeling infection spread during pandemics is not new, with models using past data to tune simulation parameters for predictions. these help understand the healthcare burden posed by a pandemic and respond accordingly. however, the problem of how college/university campuses should function during a pandemic is new for the following reasons:(i) social contact in colleges are structured and can be engineered for chosen objectives, (ii) the last pandemic to cause such societal disruption was over 100 years ago, when higher education was not a critical part of society, (ii) not much was known about causes of pandemics, and hence effective ways of safe operations were not known, and (iii) today with distance learning, remote operation of an academic institution is possible. our approach is unique in presenting a flexible simulation system, containing a suite of model libraries, one for each major component. the system integrates agent based modeling (abm) and stochastic network approach, and models the interactions among individual entities, e.g., students, instructors, classrooms, residences, etc. in great detail. for each decision to be made, the system can be used to predict the impact of various choices, and thus enable the administrator to make informed decisions. while current approaches are good for infection modeling, they lack accuracy in social contact modeling. our abm approach, combined with ideas from network science, presents a novel approach to contact modeling. a detailed case study of the university of minnesota's sunrise plan is presented. for each decisions made, its impact was assessed, and results used to get a measure of confidence. we believe this flexible tool can be a valuable asset for various kinds of organizations to assess their infection risks in pandemic-time operations, including middle and high schools, factories, warehouses, and small/medium sized businesses. as the events of 2020 have shown, pandemics due to novel viruses can lead to unimaginable disruption in society [1] . the impact has been two-fold; the direct impact of the pandemic on physical health and mortality and the indirect impact of lock-downs and social distancing on mental health [1, 2, 3] and the economy [2] . a specific example of a critical societal function facing disruption is higher education, especially for starting freshman for whom an important formative experience, namely of transitioning from home to an independent life, has been severely disrupted [4] . higher education in the us contributed an estimated $528 billion 1 to the national gross domestic product (gdp) in 2020 [5] and employed roughly 3 million people [6] . disruptions in the education sector have long-term ramifications in terms of an inadequately prepared workforce for the future [7] . by mid-march 2020 most colleges and universities across the us either cancelled in-person classes or shifted to remote-only instruction and this mode of instruction may continue for an unknown amount of time. in a recent survey of nearly 3,000 institutions [4] , only 21.3% said that they are considering fully or primarily in person model for fall 2020 and beyond. the campus environment has some unique features as compared to other places. figure 1 shows types of interactions on university campus, namely groups, queues and rivers. groups can further be classified into on-campus interactions. that can be monitored and controlled and off-campus interactions that cannot be monitored. the former includes classrooms, study areas, student life activities under the university's purview, e.g. dorms, extra-mural sports, clubs, etc. off-campus activities include private housing, social activities, grocery shopping, and a myriad of other life activities. queues appear at various kinds of service locations on campus, including those offered by the university, e.g. bookstores, student services, etc., as well as those offered by others, e.g. cafes, banks, etc. rivers include pathways where students cross each other. this classification of interactions allows for more accurate modeling of disease spread. given the nature of the pandemic, groups are the most risky types of interactions, involving sufficiently large number of people in close proximity for long periods of time. further, it is comparatively easier to reduce community transmission in queues using appointments only service and in rivers using one-way rivers, ventilation, masking and physical distancing rules. thus, among all different types of interactions, the on-campus group interactions are the ones that have high community transmission risk, are observable and controllable. therefore, it is reasonable to have a sophisticated model for community transmission through on-campus group interactions, in particular classrooms and a simple model of infections through other types of interactions consistent with related work [25] . in our work, we include a detailed stochastic network model of classes, especially since there is growing evidence of virus transmission through aerosol spread [26, 27] . since classes have fixed schedules, they can be modeled as processes happening at specific times, with batched arrivals of students and instructors. classrooms are assumed to be completely cleaned and sanitized between any two consecutive classes thus the transmission is limited within classes and mixing across two classes is only due to the same people in the two classes and not due to the shared classrooms. infections outside the classroom settings are random with chances of infection depending upon the community prevalence rates [25] . based on our review of various discussions in the media and other sources over the past 6 months, and confirmed by our interactions with university administration, we identify the following key decisions (dimensions) to be considered: 2. physical distancing: student classroom density, in sqft/student, based on the physical distance between students. 3. class modality: in-person or online classes. 4. testing: testing policy, i.e. symptomatic or asymptomatic, and whether to do contact tracing. any operational policy consists of a set of choices, one for each of the decisions outlined above. an example of a policy is minnesota's sunrise plan [28] , details of which are presented in section 4. the university administration needs to evaluate the cost of implementing a policy and the benefits obtained from them. implementation costs for policies are usually estimated based upon the predictions of behavior. for example, class modality decisions can incur technology, infrastructure and support costs, as well as revenue loss due to changes in student enrollment. our work provides a method of estimating the benefits of the operational decisions and policies. in this paper, we present a flexible framework that allows for various models for human behavior and socialization patterns, decisions and choices, infection transfer models, disease progression models, etc. to simulate the outcomes under different policies. a simulation to evaluate policy impact requires several models. for the problem of evaluating university operations policy, four models are key, namely (i) a model of social behavior and interaction patterns among people, (ii) a model of infection transfer from people to people, which can be direct (person-to-person contact) or indirect (contact intermediated by temporally and spatially co-located visits to a location where infected people deposit the infection, and the susceptible people pick up the infection), (iii) a model of disease progression in an individual once infected, and (iv) the management policy being used. once a set of models are selected from the model library and a management policy is chosen, a scenario needs to be executed to assess the impact. in our simulation framework there are three components that achieve this, namely the person-location visit generator, the infection transfer generator, and the disease progression generator. each of these uses the corresponding model selected for the execution. for the infection transfer generator and the disease progression generator, several models exist, and we give brief descriptions of one of each kind in section 2.2 and section 2.3. for the person-location visit generator there are no good models and based on our ongoing work we propose to build new ones. these are described in detail in section 2.1. the framework we developed is flexible since it allows a policy analyst to experiment with various types of models and policies. in addition, it is expandable because new kinds of models, policies, as well as metrics and visualizations can be added. new model of social contact one important contribution of the paper is a new model of social contact. disease spread depends upon the pathogen properties as well as the contact structure in the population. we introduce a peopleplace network model for social interactions that replaces the unstructured population model used for pandemic modeling. the disease spread prediction models for covid-19 so far have ignored the heterogeneity and randomness in the contact structure of the population [14, 12, 11, 21] . the models are based upon the variations of compartmental models [29, 30, 31, 32, 33, 34] that assume populations with homogeneous interactions and give rise to simple ordinary differential equations. the models were originally developed by medical doctors in early twentieth century and later studied by mathematicians, engineers and social scientists. there is limited use of structure in interactions, only at a coarse level in [1] based upon [35] . case study of a major university's policy: over summer 2020, we interacted with the university of minnesota administration administration to track various decisions, and analyzed a range of choices for each discussion, to help inform the decisions. we believe this case study, in addition to showing the usefulness of our approach, also provides helpful guidance for the future. the rest of the paper is organized as follows: section 2 describes the design of the toolkit, section 3 provides an evaluation of the choices for each decision and its impact, section 4 provides a detailed case study of the university of minnesota's sunrise plan, and section 5 concludes the paper, with potential directions for future work. we now present a flexible stochastic simulation framework for evaluating university operational decisions that contains libraries for (i) structural and behavioral models of human contact calibrated using real data, (ii) disease transmission models in buildings, and (iii) disease progression models. current covid-19 spread models are strong on modeling infection spread and disease progression, but relatively weak in modeling human behavior and social contact in various campus activities, e.g. attending classes. this leads to estimates that have a high degree of deviance from reality [13, 1] . proposed approach uses an agent based model (abm) for human interaction, and stochastic models for physical and biological processes. the abm models human interactions as a network, which provides better predictions of disease spread than traditional sir and seir models, which are population based. given the airborne transmission mode of covid-19, i.e. aerosolized droplets containing the pathogen stay suspended in air for long periods of time [26, 36] , indirect human contact must be modeled in addition to direct contact. therefore, we introduce a person-location network, a bipartite graph that captures human interaction indirectly though locations visited. figure 2 shows the architecture of the simulation framework, consisting of three system components: the person-location visit generator, the infection transfer generator, and the disease progression generator. there exist several models for each of these components developed at different times as the knowledge about the disease evolved, along with available data such as list of courses for fall 2020, course selections, mask use policy, number of in person courses, and number of students, faculty, and staff on campus. in the following, we describe the latest models which we have implemented for each component. the person-location visit generator creates a sequence of visits, i.e. events of individuals visiting locations. in the following, we define the network and describe the generation process of the network and the event sequence. the basis of the person location visit generator is the person-location bipartite network g(p, l), with p and l the set of people and locations, respectively. this network captures all connections between people and locations, i.e. existence of an edge between i and j means that person i visits location j at some point. in the simulation, this network g(p, l) is a realization from some random network generation process. network generation: to generate the network g(p, l), we consider the case where there are n people and m locations in the bipartite network, i.e. |p | = n and |l| = m . input to the generation process includes the set of nodes p and l, degree sequence of nodes in p , denoted as d 1 , · · · , d n , and the degree sequence of nodes in l, denoted as w 1 , · · · , w m . these two degree sequences can either be obtained from data, or generated as random samples from certain degree distributions. note that degree distributions for the people side and the location side can be different. also, we may need additional adjustments of the two degree sequences to ensure w j , so that they are valid sequences for the bipartite network. the bipartite network is then generated as a realization from the configuration model [37, 38] with given set of nodes p and l, and desired degree sequences d 1 , · · · , d n (for nodes in p ) and w 1 , · · · , w m (for nodes in l). the pseudo code is highlighted in algorithm 1. this algorithm returns the person-location bipartite network g(p, l). input: set of nodes p and l; degree sequences d 1 , · · · , d n (for nodes in p ) and w 1 , · · · , w m (for nodes in l). output: return the generated person-location bipartite network g(p, l). choose one half-edge from the people side and one half-edge from the location side, both uniformly at random across all half-edges on the same side; connect the two half-edges to form an edge, and add it to the edge set of g(p, l); s = s-1. end event sequence: the input data for our simulation is an event sequence g(p, l, t ), where t is the discretized time range for the whole simulation. for each time t ∈ t , g(p, l, t) represents the actual visits between people and locations at time t and is a subgraph of g(p, l). we can see that g(p, l, t) is still a bipartite network. g(p, l, t ) can be obtained from data or generated as random samples. one simple way to create g(p, l, t ) as random samples can be: for each time t, we sample the set of edges in g(p, l) with probability p uniformly at random, and denote the resulting subgraph as g(p, l, t). the parameter p captures the sociability of people and locations. within a region, certain areas may have higher sociability factor p, while other areas may have lower sociability factor p. many factors, such as user behaviors, regional characteristics and customs, and geographic and weather conditions, can be integrated into this parameter p. more importantly, p can also be modified to capture the impact of some public policies due to the outbreak of covid-19. for example, shutdown or reduced operations of business as well as shelter-in-place can be modeled as reducing the value of the sociability factor p. since we are particularly interested in the use case of university re-opening, we will present the campus specific person-location visit generator in this section. as discussed in section 1.1, by assumption, we will only model the interactions within classes. in modelling a campus, we generate a student / instructor-class bipartite network g(s ∪ i, c), where s denotes the set of students, i denotes the set of instructors, and c denotes the set of classes. g(s ∪ i, c) models which student is taking which class as well as which instructor is teaching which class. each student has a student profile indicating their department and academic level. similarly, each class has a class profile indicating its department and difficulty level. each instructor is assigned to exactly one class and this forms the instructor-class part of the bipartite network. the student-class part of the bipartite network is generated by a modified configuration model, where students are assigned to classes following certain restrictions. we assume that each student chooses 2 to 5 classes (this forms the degree sequence of the students) subject to the capacities of classes (this forms the degree sequence of the classes). with probability p 1 , students choose classes within their own department with difficulty levels matching their academic levels; with probability p 2 , students choose classes within their own department with difficulty levels not matching their academic levels; with probability p 3 , students choose classes outside their own department. in general, p 1 > p 2 > p 3 and we also require p 1 + p 2 + p 3 = 1. these restrictions can be imposed by integrating networks from multiple configuration model processes. we point that generating the network g(s ∪ i, c) from such random processes is useful for analysis prior to course enrollment; if we have the exact student-class enrollment data and instructor-class data, then we can create a deterministic bipartite network g(s ∪ i, c) from the data. figure 3 shows the topology of the bipartite network g(s ∪ i, c). based on the bipartite network g(s ∪ i, c), and the teaching schedule of all classes throughout the semester, a visit schedule of who (students and instructors) visits which class and when will be created. if we assume that students will always attend classes, then this visit schedule becomes the natural event sequence g(s ∪ i, c, t ) for the simulation, where t denotes the set of days within the simulated academic semester. otherwise, we can introduce an attendance rate p to generate the event sequence. the attendance rate p captures the students' attendance activity: with probability p, a student will attend a scheduled class; with probability 1 − p, a student will skip a scheduled class. for day t, we sample the set of edges in the visit schedule associated with that day with probability p uniformly at random, and denote the resulting graph as g(s ∪ i, c, t). the event sequence g(s ∪ i, c, t ) can be obtained as {g(s ∪ i, c, t) : t ∈ t }. the infection transfer generator generates a sequence of infections using the event sequence generated by the person-location visit generator and a disease transmission model. the probability of infection is computed using the wells-riley equation [39, 40] commonly used for modeling indoor airborne disease transmission. the wells-riley equation is used to calculate the probability of each susceptible individual of getting infected when indoors (as in classrooms) with other infectious people. the probability of infection depends upon the physical environment, including room volume, ventilation rate, time spent in the room, pulmonary ventilation rate, and the infectiousness of the disease (quanta of pathogen). the probability calculation by the wells-riley equation is given by the following: where p is probability of infection, c is the number of newly infected people, s is the number of susceptible people, i is the number of infectors, p is the pulmonary ventilation rate of susceptible (m 3 /h), q is the room ventilation rate (m 3 /h), q is the quantum generation rate (quanta/h), and t is the exposure time. the equation demonstrates how changes to both the physical environment and infection control procedures may potentially impact the spread of airborne infections in indoor environments such as classrooms. the original wells-riley equation is used for fast-moving infections. it assumes that during the scenario that a group of people are in an indoor environment, there is a chance that susceptible individuals exposed to the pathogen produced by infected individuals may get infected and start adding pathogen to the environment. this is unlikely in classroom settings because the class time is much small than the usual incubation period of covid-19. alternatively, the disease quantum generation rate is very small, making the infection transfer within a classroom slow-moving. we introduce a simple solution to this problem. we use the first-order taylor approximation of the wells-riley equation (1) to model the transmission probability. we ignore the higher-order terms in the taylor series that are not valid because of the high incubation time and low quantum generation rate of covid-19. in particular, we are using the following equation to model the probability: this approximation of the wells-riley equation is extremely close to the original equation because the exposure time in classrooms is small and a linear approximation of an exponential function is very accurate when the argument of the function is small. the disease progression generator library generates the disease state transitions of each agent. the key states of the library include susceptible, infected, transmitting/infectious, asymptomatic, symptomatic, severely ill, dead, and recovered, which are highlighted in figure 4 . it shows a standard finite state epidemiological model of disease progression states [8] , where arrows imply the direction of change of state. our simulator follows this epidemiological model and obtains the distribution of time spent at different states from existing literature. the transition from the susceptible to the infected state primarily occurs according to the infection transfer generator and is probabilistically determined by the first-order taylor approximation of the wells-riley equation (2) inside the campus, secondly, it also happens spontaneously through outside infection, whose rate is a variable that depends upon the disease prevalence around the campus. once infected, the agent starts his / her incubation period, which follows a weibull distribution w eibull(0.11, 1.97) with a mean of 8.29 days and a median of 7.76 days, based on [41] . people become infectious and start transmitting viruses 1-3 days (uniformly distributed) before their incubation period ends [42] . once an agent starts transmitting, it becomes asymptomatic (this includes agents that are pre-symptomatic). when the incubation period ends, the agent either remains asymptomatic or transitions to being symptomatic with 65% probability [43] . we assume that the symptomatic or asymptomatic agents remain contagious until recovered. the distribution of this period has a mean of 7.8 days and is best fit by the gamma function gamma(3, 7.8/3) [44] . these are parametric values that can be modified as we change our understanding of the disease or implement different testing policies. asymptomatic or symptomatic agents with incorrect test results (false negative rate is assumed to be 3.3% of total tested based on the test mentioned in section 3.3.4) go around spreading the disease to locations they visit and eventually will transition to the recovered state at the end of their contagious period [45] . agents with positive test results are taken out of the simulation and put in effective quarantine based on the number of days left in their contagious period. a portion of them, defined by a probabilistic parameter, will develop a severe illness or even mortality and will not be able to come back on campus throughout the semester. for those without severe illness or mortality, once the quarantine period ends for them, they are put back into the simulation as recovered state, which entails that state for the remainder of the semester and thus they will not get infected again. we have based this on the informed assumption that antibody immunity lasts for three months which is more than the entirety of our semester [46] . we point out that our disease progression generator library is general enough to model the complete disease progression. for example, we can also include states like hospitalized, shortness of breath, respirator, icu, dead, etc. into the library and perform further analysis. however, controlling the spread of the disease is the primary concern for universities, therefore we omit those states and focus more on the infection and transmission of the disease. for this study, we analyze the cumulative infected students due to community transmission of covid-19 in section 3, hence the fraction of agents who leave the system (severe illness or mortality) or get recovered is immaterial for our simulations because neither of the states impact new infections. recovered patients are immune and don't act as vectors whereas patients who leave are isolated from the system. the simulator, available in github 4 , was used to evaluate the effectiveness of different operational interventions for reducing infections during the semester. the university of minnesota is chosen, to represent many big universities in the country. following describes the data and assumptions used for the experiment in section 3.1 and section 3.2 and analyzes the impact of different operational interventions in section 3.3. the simulator uses actual student class enrollment data from the university of minnesota and contains 46,782 students, 5,570 classes and 5,570 instructors. these are from fall 2019 official enrollment statistics report 5 . we consider students from each department except college of continuing and professional studies, since students from that college primarily take online courses. thus, our event sequence is g(s ∪i, c, t ), where |s| = 46782, |i| = 5570, |c| = 5570, and |t | = 7 × 12 = 84 days. each student can enroll in 2 to 5 courses as per university guidelines to maintain student status, which defines the degree of each student in the network g(s ∪ i, c). we assume that students can only take classes within departments of their own school / college in the results we present. fall 2019 registration information of umn 6 is used to provide the set of classes offered, number of students enrolled in each class, timings, and the instructors to the simulation. with such information, we construct the person-location bipartite network and event sequence according to section 2.1. although the current focus is on the pandemic operations of a major university, the framework is flexible enough to analyze the spread of infectious diseases involving human interactions in a big campus if any kind, given relevant models and parameters. as we develop the understanding of the disease, we regularly update the parameters involved in the framework based on the current studies. below we discuss some of the major parameters used in the framework. initial infection: we start our simulation with a sample of the population being initially infected. for our analysis, we assume this value to be 1% according to the minnesota's weekly covid-19 reports [47] . we randomly select the initially infected people in the simulation, and uniformly distribute them into different groups based on the number of days since they have been infected (maximum 5 days). outside transmission: students lead a significant portion of their life outside the university and this is not precisely modeled in the simulation, especially since it cannot be controlled. further, 75% -80% of students live in off-campus private housing, details of which are outside of the university's purview. detailed modeling of this has not been done. instead, an assumption is made that 5 non-quarantined susceptible students are spontaneously infected every day due to presumed transmission from non-university contact. this is consistent with the analysis by the upenn/swarthmore team [25] . we expect modelers to have better estimate of this parameter as the semester progresses. indoor transmission: we consider indoor transmission of covid-19 to be airborne, based on the assumption that tables, chairs, equipment and other surfaces inside the classrooms are being systematically sanitized by the university cleaning staff. we use an approximation of the wells-riley equation (2) , which already has well defined parameters for airborne transmission inside a classroom. the pulmonary ventilation rate of susceptible, defined as p, is set at 0.48 m 3 /h [39] . the quantum generation rate q, or the amount of infection produced by a covid-19 patient per hour, is assumed to be 20 quanta/h [40] . we also assume that different types of masks have various efficiency in filtering the quanta generated (covid-19 virus released) and pulmonary ventilation (air intake). this is presented in the detailed analysis of masking impact in section 3.3. the room ventilation rate q is assumed to be the product of ventilation rate 4 ac/h [39] and room volume (explained in section 3.3.2). in this section we analyze the effects of individual policy dimensions, including masking, physical distancing, class modality, and testing. specifically, we estimate the cumulative infected students due to the community transmission of covid-19 within the university campus. to analyze the impact of each policy dimension, we vary values of parameters associated with that policy dimension, keeping the other dimensions as low as possible (i.e., equivalent to pre-covid-19 times). for each set of simulation parameters we ran 1,000 simulations, plot the mean of cumulative infected students and the 95% confidence interval for the mean. across different simulations the results are concentrated and the standard deviations are relatively small as compared to the mean values. thus, the 95% confidence intervals are close to the mean values. the masking policy is an aggregation of four parameters, namely, student mask type, student mask compliance, instructor mask type, and instructor mask compliance. we study the impact of this policy from two aspects, namely the student mask compliance (the percentage of student population that wear a mask) and the student mask types. for the study of student mask compliance, students and instructors are assumed to be wearing cloth masks. the instructor mask compliance is set to 100%, while the student mask compliance is set as a variable. for the study of student mask types, we consider three types of masks, namely cloth masks, medical masks, and n95 masks. the effectiveness of different mask types was modeled based on a study in [48] , comparing the filtration efficiency of small aerosols on different types of masks. the study [48] found that, on average, n95 masks filter efficiency was 95%, medical mask filter efficiency was 55%, and general cloth mask filter efficiency was 38%. we also assume that all classes are in-person, physical distancing is 2 feet, and only symptomatic people are tested. figure 5 shows the impact of different student mask compliance on the cumulative number of infected students due to community transmission within campus. a decrease of 23.25% in cumulative infected students can be observed when there is a change from no student mask compliance to strict mask policy adherence. in figure 6 , we fix the student mask compliance to be 100% and vary the mask types. we observe that the regular use of n95 masks by students will significantly reduce the spread. however, it's also expensive for them to use these masks every day. figure 6 : impact of different mask types on cumulative infected students due to the community transmission of covid-19 within university campus covid-19 spreads among people in close proximity for sufficient time. in our simulation, we use physical distance as a radius to calculate the area of a circle which acts as a substitute for the area per student. this value is used to determine the room volume of a class based on the total number of students attending the class. the product of room volume and ventilation rate parameter (4 ac/h) is used as the room ventilation rate parameter in the wells-riley equation (2) . center for disease control (cdc) recommends a physical distance of 6 feet [49] . to analyze physical distancing policy, we vary the physical distance from 2 (personal space [50] in normal times) to 6 feet. we analyze the physical distancing dimension by assuming scheduling of all classes in-person, no adherence on wearing mask among students, and only symptomatic people are tested. as seen in figure 7 , an increase from 2 to 6 feet in physical distance can decrease the cumulative infected by 70.50%, which suggests that it's crucial to avoid close contact with other people even though they are not showing any symptoms. large gatherings, like classes, increase the likelihood of spreading the virus [51, 52] . therefore, we analyze the impact of class modality by varying the maximum class size. we set the mask compliance among students to be 0%, physical distancing is reduced to 2 feet, and only symptomatic infected people are tested. in particular, we vary the maximum in-person class sizes to 30, 60 or all in-person, as shown in figure 8 . a change from all classes being in-person to restricting classes with more than 30 students to be online, can bring a decrease of 72.99% in cumulative infected. large classes act as hubs where typically students from various departments study together, which increases their potential exposure to other students. consequently, it creates short paths between students of different departments (communities) [53] , which exacerbates the community spread. therefore, avoiding large classes can be very effective in controlling the spread of the disease. in the simulation, we can implement testing policies to detect infection spread in the network. on a given day, each person has a test state of tested positive, tested negative, or not tested. the testing capacity each day is assumed to be limited due to constraints from time, labor, money, manufacture of testing supplies, etc. available tests are preferentially used to test symptomatic people. people who turn symptomatic go for testing the next day. based on the testing results on each day, we then create a list of classes attended by the positively tested people, called the 'contact traced' (ct) classes. the rest of the classes come under 'non-contact traced' (nct) classes. we can implement various policies to in the simulation, we use the test accuracy statistics from a publicly available infection testing manufactured by inbios 7 , with 96.7% sensitivity and 100% specificity. if an individual has a false negative test result, he / she can still attend classes and spread the virus. if an individual has a false positive test result, he / she will be quarantined in the simulation. we also define a 'testing gap day' parameter which stops people from getting tested within a particular time frame. in the simulation we set the 'testing gap day' as 3 days, because this is the average number of days after which an infected person becomes infectious. this value can be changed depending upon the availability of tests and other policies. in addition, we set the mask compliance among students to be 0%, physical distancing to be 2 feet, and all classes to be held in-person in our simulation. to study this policy dimension, we vary the test capacity by 2000, 5000 and 10,000 tests per day. as shown in figure 9 , 5x testing capacity (from 2000 to 10000) can lead to 36.45% decrease in cumulative infected, which is less effective as compared to other individual dimensions we studied. the higher education community finds itself in uncharted territory due to covid-19, which has limited the functioning of the community. operational policies being implemented also carry social and monetary costs, which are presently unclear, but will impact the future. thus, implementing cost-effective policies with high impact on controlling the spread of the disease is extremely important. masking and physical distancing are economically cheaper to implement. however, as shown in figure 5 and 7, when these policies are implemented in isolation, 71.57% and 27.50% student population respectively still gets infected. these two policies also have high social cost for implementation. in march, when understanding of the pandemic was nascent, policies such as self-isolation and social distancing were recommended to flatten the curve. feelings of loneliness and isolation that are exacerbated during social distancing have caused mental health to suffer and lead to increased substance use, and elevated suicidal ideation [54, 55] . masking as a voluntary policy would likely lead to insufficient compliance, would be perceived as less fair, and could intensify stigmatization such as negative emotional responses, social labeling, or prejudicial attitudes [56] . on the other hand, policies like mass testing and shifting classes with huge enrollments online bear the financial burden and may not be as effective as expected in isolation, unless all other recommendations are being strictly followed by the figure 10 : sunrise plan decision timeline students. scientists at the university of illinois developed a quick, inexpensive saliva test and started doing 10,000 to 15,000 tests per day. however, mass testing and contact tracing alone doesn't guarantee a control over the spread, as shown in our simulation results in section 3.3.4, which also results in 58.3% population being infected by the end of the semester. students' defying quarantining/self-isolation when being testing positive has also led to another spike in infections [57] . these policies when implemented individually don't seem to be enough to control the disease spread, but they can work well in conjunction with each other. we present a case study of a combination of these policies in our next section. here we present a detailed case study of the impact of various decisions taken by the university of minnesota, one of the largest universities in the country, as it went through the process of developing and implementing its sunrise plan the sunrise plan is summarized in figure 10 . under umn's sunrise plan, recommendations for the use of masks and social distancing were made as early as may 1. cloth masks are being provided to all students and employees who are on campus and required in specific settings. as of june 22, based on the physical distancing requirements outlined by the minnesota department of health [58] and umn's own medical and public health experts 8 , decision on maintaining 6 feet physical distance in general-purpose classrooms was adopted. course delivery modality was entered into the university course scheduling system by the end of july 2. it was also decided that any in-person class meetings will end right before thanksgiving (nov 26), and the remainder of the class meetings, including final exams, will be entirely online. led by umn's health emergency response office (hero), a testing and tracing advisory team was formed and the team introduced an elaborate testing policy called mtest 9 on july 30. on aug 21, it was announced that all classes will be wholly online for at least first two weeks of the fall semester. as the university considered and implemented alternative policies over time, we modeled the impacts of those decisions, including requiring masks, the extent of physical distancing in classrooms, introduction of class modalities as to designate classes as in-person, blended, remote, or online, and testing protocols. this was done over a roughly 4-month period from may 1st to august 21st, 2020. as each decision was considered, the impact of various alternatives for it were evaluated. since decisions were made one at a time, in evaluating the impact of downstream decisions, the system also incorporated the earlier decision made in the sunrise plan. in table 1 and figure 11 , we show the combined results of various policies adopted at different stages of sunrise plan. the results can be applied to get a measure of confidence for each decision. our results show that, even just the proper adherence of masking and physical distancing can bring a drastic difference in the cumulative infected in comparison to individual policies. covid-19 has presented administrators of higher education institutions with a completely new problem, i.e. what are the right decisions to make to for operating an educational campus during the pandemic, such that educational objectives can be met, while ensuring health safety for the entire community. the history on this problem is short, i.e. only around 6 months, since the last pandemic with this level of societal impact, i.e. the spanish flu of 1918, happened at a time when higher education was not much of an integral part of the society. this paper is among the first set of efforts to address this problem. our approach is unique in that it presents a flexible simulation system that contains a suite of model libraries, one for each major system component. the simulation system merges agent based modeling (abm) and stochastic network approach, and models the interactions of individual entities, e.g. students, instructors, classrooms, residences, etc. in great detail. for each decision to be made by administrators, the system can be used to predict the impact of various choices, and thus enable the administrator to make a suitable decision. a detailed case study of the university of minnesota's sunrise plan [28] was presented. specifically, as various decisions were made in sequence, their impact was assessed using this system. the results were used to get a measure of confidence in each decision. we believe this flexible tool can be a valuable asset for various kinds of organizations to assess their infection risks in pandemic-time operations. economists see uneven jobs recovery mental health and the covid-19 pandemic here's our list of colleges' reopening models projections of education statistics to 2026. nces 2018-019 projected number of participants in educational institutions, by level and control of institution education during covid-19 and beyond the sir model for spread of disease: the differential equation model. loci.(originally convergence modeling the spatial spread of infectious diseases: the global epidemic and mobility computational model transims: transportation analysis and simulation system forecasting covid-19 impact on hospital bed-days, icu-days, ventilator-days and deaths by us state in the next 4 months forecasting the impact of the first wave of the covid-19 pandemic on hospital demand and deaths for the usa and european economic area countries. medrxiv covid act now. america's covid warning system impact of non-pharmaceutical interventions (npis) to reduce covid19 mortality and healthcare demand pattern of early human-to-human transmission of wuhan 2019-ncov. biorxiv early dynamics of transmission and control of covid-19: a mathematical modelling study. the lancet infectious diseases transmission interval estimates suggest pre-symptomatic spread of covid-19 feasibility of controlling covid-19 outbreaks by isolation of cases and contacts. the lancet global health nowcasting and forecasting the potential domestic and international spread of the 2019-ncov outbreak originating in wuhan, china: a modelling study covid-19 hospital impact model for epidemics (chime). penn medicine, the university of pennsylvania responding to covid-19-a once-in-a-century pandemic policies and guidelines for covid-19 preparedness: experiences from the university of washington covid-19: epidemiology, evolution, and cross-disciplinary perspectives simulating covid-19 in a university environment consideration of the aerosol transmission for covid-19 and public health evidence of short-range aerosol transmission of sars-cov-2 and call for universal airborne precautions for anesthesiologists during the covid-19 pandemic a dynamic model of market share and sales behavior diffusion of innovations discussion: the kermack-mckendrick epidemic threshold theorem three basic epidemiological models the mathematics of infectious diseases nonlinear dynamics and chaos with student solutions manual: with applications to physics, biology, chemistry, and engineering multiscale mobility networks and the spatial spreading of infectious diseases aerosol and surface stability of sars-cov-2 as compared with sars-cov-1 the asymptotic number of labeled graphs with given degree sequences a critical point for random graphs with a given degree sequence. random structures & algorithms modelling the transmission of airborne infections in enclosed spaces association of infected probability of covid-19 with ventilation rates in confined spaces: a wells-riley equation based investigation. medrxiv estimation of incubation period distribution of covid-19 using disease onset forward time: a novel cross-sectional and forward follow-up study. medrxiv temporal dynamics in viral shedding and transmissibility of covid-19 covid-19 pandemic planning scenarios modeling the impact of social distancing measures on the spread of sars-cov-2 in minnesota sars-cov-2 infection induces robust minnesota department of health. minnesota department of health weekly covid-19 report 8 comparison of filtration efficiency and pressure drop in anti-yellow sand masks, quarantine masks, medical masks, general masks, and handkerchiefs center for disease control and prevention. social distancing the hidden dimension/edward twitchell hall public health response to the initiation and spread of pandemic covid-19 in the united states clustering and superspreading potential of severe acute respiratory syndrome coronavirus 2 (sars-cov-2) infections in hong kong the small-world network of college classes: implications for epidemic spread on a university campus mental health, substance use, and suicidal ideation during the covid-19 pandemic-united states the impact of the covid-19 pandemic on suicide rates social and behavioral consequences of mask policies during the covid-19 pandemic a university had a great coronavirus plan, but students partied on minnesota department of health the authors acknowledge krishnamurthy iyer, kelly searle, and rachel croson for their constructive feedback at various stages of this research. key: cord-344113-tjvxslok authors: fatani, tarah h. title: student satisfaction with videoconferencing teaching quality during the covid-19 pandemic date: 2020-10-31 journal: bmc med educ doi: 10.1186/s12909-020-02310-2 sha: doc_id: 344113 cord_uid: tjvxslok background: the coronavirus disease 2019 pandemic prompted the pediatric department at king abdulaziz university to continue students’ educational activities by offering courses online that utilized web video conferencing (wvc). given the uncertainties of wvc educational quality and the challenge of shifting to an online environment, this study aimed to evaluate student satisfaction with the teaching quality of case-based discussion (cbd) sessions conducted through wvc. methods: one hundred sixty-two undergraduate medical students in pediatrics completed the reduced students’ evaluation of educational quality (seeq) survey with a five-point likert scale over 5 weeks. the wvc cbd sessions were facilitated by 50 faculty members. results: 82% of respondents were highly satisfied with the wvc cbd session’s teaching quality. the majority agreed that the sessions were intellectually challenging, that the instructors were dynamic, and encouraged students to participate. no statistically significant correlation was found between student satisfaction and technical issues (r = 0.037, p = 0.003). conclusions: wvc teaching had an overall positive outcome on student satisfaction, and teaching quality relied on teaching, cognitive, and social presence rather than technology. however, technology remains an important platform that supports teachers’ educational activities. thus, implementing a blended pediatric course to augment future course delivery is optimal. the coronavirus disease 2019 (covid-19) pandemic has changed medical education; the united nations educational, scientific, and cultural organization (unesco) announced that 1.2 billion students across the planet have been affected by school and university closures due to the pandemic [1] . saudi arabia is one of the many countries that made the unprecedented decision to close schools and universities to contain the spread of the deadly virus as social distancing has been the most effective preventative strategy for covid-19 pending development of a vaccine, treatment, or both [2] . at king abdulaziz university, this closure dramatically interrupted medical students' educational activities and clinical training, prompting the institution to move the course of study online and contemplate new approaches to distance education. faculty members in the pediatric department shifted from in-person to distance education to facilitate students' education. however, for many in the department, web video conferencing (wvc) technologies and online education were new and challenging. the educators had to meet four major online education challenges: demonstrating pedagogical skills in an online classroom, addressing their managerial role, establishing relationships with students, and providing technical support [3, 4] . despite the challenges, the instructors had to effectively facilitate case-based discussion (cbd) via wvc while maintaining a high-quality, authentic learning experience with high student satisfaction rates. distance education is defined as learning activities within formal, informal, or non-formal domains that are facilitated by information and communication technologies to lessen distance, both physically and psychologically, and to increase interactivity and communication among learners, learning sources, and facilitators [5] . distance education has become the basic global mode of course delivery, and the quality of this delivery is essential. the five elements in the sloan quality framework for effective online education are student satisfaction, learning effectiveness, faculty satisfaction, student access, and institutional cost-effectiveness [6] . furthermore, the root causes of online learning success are motive and leadership, focus on the program, faculty support, students (satisfaction, services, outcome), and growth in student enrolment [7] . thus, the quality of online learning is highly influenced by teaching rather than technology [7] . a systematic review of randomized controlled trials concluded that digital problem-based learning (dpbl) was potentially more effective in improving knowledge and skills than traditional problem-based learning (pbl) due to greater student interaction and engagement in dpbl [8] . however, the review found mixed data concerning student satisfaction outcomes [8] . various studies on student satisfaction with online learning have identified the following factors: instructor interaction, communication, active learning, students' ability to initiate and control their actions in the learning environment, efficient assessment of academic progress, technology, and the learning environment [9, 10] . wvc is a synchronous model for interactive voice, video, and data transfer between two or more groups. it enables real-time, two-way video and audio communication as well as content sharing and messaging between instructors and students [11, 12] . additionally, it allows immediate feedback and supports collaborative learning among students [11, 12] . wvc allows learners and instructors to participate in web-based discussions that are not bound to certain hardware or software, utilizing services such as zoom, skype for business, and goto-meeting [12] . as technology advances, wvc has become one of the most common tools used for synchronous online teaching [13, 14] . it promotes dynamic collaborative efforts, although it does face challenges with technical issues [12] . the findings on student satisfaction with wvc have been mixed. dawson showed that synchronous communication increases the sense of classroom community and, therefore, student satisfaction [15] . additionally, in a study by dogget, over 90% of students responded favourably to the instructor's use of wvc and his/her encouragement to ask questions; however, 80% stated that they would have been more comfortable in a conventional classroom setting, and 57% agreed that wvc technology was a barrier to interacting with the instructor [16] . smith et al. [17] demonstrated that wvc was an effective method for satisfactorily teaching surgical tutorials to medical students. in contrast, giesbers et al. [18] compared the learning experience of students using discussion forums (df) versus students using both df and wvc (which is expected to increase social presence) and found that wvc was not associated with enhanced student learning during their four-year experience with wvc. the only exceptions were the clarification of the module's goals and tasks [19] . cbd is an interactive, student-centered learning approach closely related to pbl. cbd utilizes real-life scenarios that the students would have faced during the clinical phase of their pediatric rotation [18] . it aims to enable students to learn from patients' cases, to enhance their problem-solving skills, and to encourage them to think analytically about the patient's case from various angles. cbd has several advantages. it promotes active, self-directed, life-long learning; integrates basic medical knowledge with clinical cases; emphasizes clinical reasoning when approaching a case; and develops student communication skills [18] . information about the case from history, physical examination, and investigation is given in a step-wise manner to allow students to analyze the information and generate differential diagnoses. the students then strive to reach the most likely diagnosis and appropriate management. wvc is an optimal choice to facilitate cbd sessions as it will increase social presence which help the acquisition of information, focus on learner-centered opportunities, engage students and allow small group interactions, allow instructors to observe and assess students working in real-life learning experience, develop positive communication skills, demonstrate leadership, and shared responsibility as they analytically think about how to approach a pediatric case. however, problems using wvc for conducting the sessions are destined to occur such as experiencing technology issues, confidence navigating the platform by both instructor and students, ensuring student engagement, and delivering creative and effective educational practices in an online learning environment [20] . the community of inquiry (coi) framework constitutes three critical elements of online education-cognitive, teaching, and social presence-as well as the internal dynamic relationships among them [21] . presence is defined as a state of alert awareness, receptivity, and connectedness to the mental, emotional, and physical workings of the individual and the group in the context of their learning environments, as well as the ability to respond with a considered and compassionate best next step [22] . cognitive presence is the extent to which leaners are able to construct and confirm meaning through sustained reflection and discourse. it is implemented through the practical inquiry model consisting of four phases: triggering event, exploration, integration and resolution [21] . teaching presence is the design, facilitation, and direction of cognitive and social processes, while social presence is the ability of the learners to identify with the community, communicate purposefully and develop inter-personal relationships by way of projecting their individual personalities [21] . these three interdependent elements are essential for delivering a successful asynchronous (i.e. not simultaneous) online course and creating an effective educational experience [21] . the seeq questionnaire shares the same concepts as the coi framework (cognitive, teaching, and social presence); questions exploring student learning reflect cognitive presence, questions exploring enthusiasm and organization reflect teaching presence, and questions on group interaction and individual rapport reflect teaching and social presence combined. the relevance of the coi framework to synchronous (i.e., simultaneous interaction) online learning with wvc technology needs to be further verified. according to moore, using interactive teleconference media offers the opportunity for dynamic inter-learner dialogue and student engagement, thereby enabling instructors to bridge the psychological and communication distance between instructors and learners and among learners. it also provides the opportunity to develop the cognitive skills of analysis and autonomy [23] . given the novel exposure to distance education for both students and faculty members, this study aimed to evaluate student satisfaction with the teaching quality of case-based discussion (cbd) sessions conducted through wvc. furthermore, students' ongoing feedback about their satisfaction with the teaching quality will substantially impact future curriculum planning and design as further online pediatric courses are implemented. this study enrolled 162 fifth-year male saudi undergraduate medical students who were in the pediatric rotation course at king abdulaziz university for 12 weeks from march to may 2020. the students were divided into 14 groups with 10-12 students in each group. all students were between 21 and 25 years old. the students' attendance at each cbd was compulsory and monitored throughout the course. the pediatric course at king abdulaziz university used the online platform blackboard ultra before the covid-19 outbreak; this platform integrates course learning outcomes, student study guides, assignments, and assessments. traditionally, the pediatric course is a 12-week rotation at the end of the academic year during which fifth-year medical students are exposed to clinical inpatient and outpatient environments and participate in face-to-face cbd and clinical sessions in a hospital. none of these educational activities was provided via distance education before the covid-19 outbreak, and the cbd sessions were conducted by the students and facilitated by the teachers to enhance active learning. in order to prepare the faculty members for successful online course teaching, king abdulaziz university provided faculty members with nonmandatory blackboard development courses on online education and teaching, course delivery and navigation, and technical requirements. the covid-19 outbreak occurred 2 weeks through the rotation, and the university reduced the course to a duration of 9 weeks. this left 5 weeks to deliver the course online and 2 weeks to complete the assessment and final examinations. to ensure that the remaining course objectives were met, 10 faculty members collaborated to identify the most crucial topics in the fifth-year undergraduate medical curriculum. they also decided whether the educational activity should be delivered asynchronously or synchronously. these faculty members identified 11 asynchronous lectures and 17 synchronous cbd to be delivered online over 5 weeks by 50 pediatric faculty members according to their area of interest or subspecialty. for 5 weeks, each instructor facilitated 1 or 2 cbd sessions weekly to cover 17 cbd topics. each session was allocated a 2-h duration and involved a set of 1 or 2 groups of students (10-12 students per group). the wvc options were blackboard ultra and zoom. because blackboard ultra is the learning management system (lms) platform used by the university for course delivery, it faced high pressure during this period since most university instructors were utilizing it. after piloting both platforms, the instructors reported easier navigation and fewer interruptions, connection dropouts, and audio-visual problems using zoom. zoom is an innovative meeting company that unifies cloud wvc, simple web meetings, and group collaboration in one easy-to-use platform. it has been one of the most extensively used tools for facilitating ongoing synchronous educational activities during the covid-19 pandemic [24, 25] . furthermore, it allows wireless screen and whiteboard sharing, has an interactive chat room, and is free for up to 40 min. thus, it is an optimal choice for facilitating synchronous teaching activities to provide an optimal learning experience with enhanced student and instructor engagement [26] . in preparation for the cbd sessions, instructors received the cbd learning outcomes and the list of students participating in the discussion. additionally, a full orientation on using zoom was provided for all academic pediatric faculty members through a half-day workshop. after the workshop, an e-mail link was sent to both students and faculty members with a demonstrative video recorded by one of the pediatric faculty members providing simple explanations on utilizing zoom in arabic. the reduced version of the students' evaluation of educational quality (seeq) was chosen as the survey tool to evaluate the teaching quality of the wvc cbd sessions. the seeq was developed by marsh in 1987 to assess the level of student satisfaction with teacher effectiveness to improve teaching quality [27] . the reduced seeq has been internationally validated for research studies [28] . it has a robust quality, excellent reliability and internal consistency, and reasonable validity, and has the flexibility to fit individual teaching contexts. it is useful for rapidly acquiring data and minimizing the risks of item non-response, and it has been extensively utilized in many courses worldwide at both the graduate and undergraduate levels [29, 30] . the seeq questionnaire examines nine characteristics of effective learning: learning, individual rapport, enthusiasm, examinations, organization, breadth, group interaction, assignments, and overall rating. following an initial pilot questionnaire completed by one group of 10 students and reviewed by three pediatric faculty members, the questionnaire categories were tailored to be more relevant to the wvc cbd sessions. we eliminated 2 questions from the original reduced seeq questionnaire, one under the category of enthusiasm related to using humour, and the second under the category of individual rapport related to the instructor being friendly toward individual students. moreover, we added 2 technology questions relevant to the use of wvc. the final questionnaire contained the following 21 items: 1 question on the assigned student group; 2 questions about the cbd session (topic, date, and time); 14 reduced seeq factors focusing on learning (q4-q7), enthusiasm (q8-q9), organization (q10-q12), group interaction (q13-q15), and individual rapport (q16-q17); we added 2 questions relevant to wvc technology usage (faculty creativity and audio/visual technical issues); 1 question about student satisfaction with the cbd session quality; and 1 open-ended question that allowed students to comment on their experience. a 5point likert scale ranging from 1 (strongly disagree) to 5 (strongly agree) was used. after approval from the research ethics committee at king abdulaziz university hospital, an attached invitation letter was sent via weblink explaining the purpose of the study and students were asked to complete the survey for one cbd on a weekly bases for 5 weeks duration, and to rate the degree to which they agreed or disagreed with each item on the survey. the surveymonkey was sent over the course of 5 weeks to all group leaders to be distributed among their fellow students. the students' participation was voluntary and anonymous, and they were able to withdraw at any time. data were collected using surveymonkey and then analyzed using statistical package for social science (spss) version 22.0 to determine the means and standard deviations of the questionnaire items. the analysis relied on having no group differences. the mean for each of the items was compared between the high-satisfaction group (hsg) and low-satisfaction group (lsg) with a 95% confidence interval (ci) using independent sample t-tests. a student was assigned to the hsg if the overall cbd satisfaction response rated as agree or strongly agree on the 5-point likert scale or to the lsg if he rated the response as strongly disagree, disagree, or neutral. pearson's correlation coefficient was used to assess the correlations between student satisfaction and technology usage. a total of 662 male student responses were received (an overall response rate of 82%). table 1 displays the descriptive analysis of the of reduced seeq responses by degree of overall student satisfaction. as shown, students rated highest mean score in the items under the category of teaching presence "students were encouraged to participate" and "faculty members' explanations were clear" with a mean score of 4.23 (± 0.78), and 4.20 (± 0.85) respectively. under the technology category "during this cbd session, there were audio or visual technical issues" and "the faculty member creatively used the whiteboard, chat room, and videos" students had the lowest mean scores of 3.95 (± 0.99) and 2.82 (± 1.35), respectively. under the learning category, which was equivalent to cognitive presence, the majority of students agreed that the sessions were intellectually challenging, and that they understood the subject materials, with a mean score of 4.08 (± 0.863) and 4.12 (± 0.836), respectively. moreover, under the category of group interaction and individual rapport, which reflected both teaching and social presence, the students agreed that the instructor invited them to share knowledge and had a genuine interest, with a mean score of 4.19 (±0.829) and 4.08 (± 0.874), respectively. furthermore, 82% of the students were in the hsg versus 18% in the lsg, with a statistically significant difference in the mean scores in all categories (p < 0.001) except for the item "facing technical issues" (p = 0.198). a statistically significant difference was found in student satisfaction between the hsg and the lsg, with a mean score difference of 1.848 (95% ci, 1.744-1.953; p < 0.001). no statistically significant correlation was found between student satisfaction and technical issues (r = 0.037, p = 0.003). the covid-19 pandemic compelled the educational system to enter a new era of online learning. this study investigated the novel exposure to distance education for both students and faculty members. to the best of the authors' knowledge, this was the first study to explore student satisfaction with pediatric wvc cbd sessions' teaching quality. it provided insight into the strengths and weaknesses of student satisfaction with the teaching quality of wvc cbd sessions. students scored the highest mean under the category of teaching presence while scored lowest under the category of technology. this indicated that students were satisfied with teaching presence despite faced challenges with technical issues, and that faculty members could improve their wvc navigation. all students had self-acess to smart devices and internet connection. infrastructure support was provided for both students and faculty members when blackboard platform was utilized including navigation and technical support, but not with zoom. of the participating students, 82% had a high satisfaction rate in all of the categories on the seeq questionnaire. the students' high satisfaction with the sessions' quality was not surprising considering the sessions' compatibility with the coi framework. conrad in his literature review article explored the impact of video technology on each of the three elements of the coi framework. he points out that wvc helps with building a coi framework by establishing social presence for both the instructor and the student, engaging learners and learner-learner interactivity, increases teaching presence by allowing robust feedback and conveying of emotions to the students, and enhances cognitive presence by allowing a flipped classroom model of teaching, which encourages critical thinking and processing of information [31] . the results supported those of other studies that showed that instructor presence and an interactive teaching style were significant determinants of student satisfaction and the effectiveness of online learning environments [32, 33] . additionally, the findings of the present study were in line with duygu's study, which revealed a positive student perception of the teaching content, effectiveness, interaction, and idea-sharing of wvc despite experiencing technical problems related to sound quality and connection [34] . moreover, the findings were consistent with a study by doggett [16] that showed a positive student experience with the instructor's use of wvc technology and teaching skills despite students' 21 do you have any comments ? "dr. x was very enthusiastic during teaching. it was a very beneficial cbd". "one of the most beneficial cbds i took this year". "i couldn't ask all my questions, dr. x immediately logged off the app and didn't wait to see if we had questions" "was a good experience with dr. x but she took so long around 3hours, overall she was a really good tutor" "dr. x was very helpful and encouraging". "dr. x is a good example of how a doctor should encourage her students to be strong and to study hard. she has a good method of delivering the information to the students in an easy way". perception that wvc technology was a barrier to interacting with the instructor. several other studies during the covid-19 pandemic similarly highlighted the effectiveness of wvc for delivering online teaching, engaging students, and establishing a sense of community and social presence, thereby promoting a positive student experience despite visual and audio issues [35] [36] [37] . the study found that teaching effectiveness and quality relied on teaching, cognitive, and social presence and not on technology. however, technology remains an important platform that supports teachers' educational activities. additionally, this study illuminated the importance of satisfying the growing demands for online education while maintaining a worthwhile student learning experience. therefore, standards should be established for a blended-learning pediatric course that combines traditional clinical skills, face-to-face training, and innovative educational methods. faculty development workshops and training on distance education and teaching skills and competencies should be organized, and training on navigating wvc platforms should be provided for both students and faculty members. furthermore, set expectations should be established for the wvc netiquette system, and technical issues should be addressed by utilizing the online technical support system. additionally, the effect of such implementations on future pediatric course quality and student and faculty satisfaction should be monitored. finally, this study has the potential to enhance wvc teaching effectiveness and enrich online course delivery in other medical and healthcare fields. wvc teaching had an overall positive outcome on student satisfaction, and teaching quality relied on teaching, cognitive, and social presence rather than technology. however, technology remains an important platform that supports teachers' educational activities. thus, implementing a blended pediatric course to augment future course delivery is optimal. the study had certain limitations. first, it was conducted on one group of male students rotating in pediatrics in a single center. therefore, inferences cannot be made about other courses or faculty members for which the education culture and environment may be different. second, the survey responses may have been limited by inaccuracy and subjected to recall bias, since some students did not complete the survey immediately after the cbd. third, different faculty members facilitated the cbd sessions and information on previous online course teaching was not collected. therefore, not all students were exposed to the same orientation (such as online netiquette system and expectations) or online teaching style, skill or competency. fourth, the study did not elaborate on student retention or success rates since the students' performance evaluations were set as pass or fail grades, and 98% of the students completed and passed the course successfully. however, this rate of completion reflected positively on the success of the teaching quality of the wvc cbd sessions. covid-19 educational disruption and response 2019 novel coronavirus-important information for clinicians issues and challenges for teaching successful online courses in higher education improving teacher professional development for online and blended learning: a systematic meta-aggregative review from distance education to open and distance learning: a holistic evaluation of history, definitions, and theories the sloan consortium quality framework and the five pillars. the sloan consortium implementing best practices in online learning digital problem-based learning in health professions: systematic review and meta-analysis by the digital health education collaboration student satisfaction with online learning: is it a psychological contract? online instruction, e-learning, and student satisfaction: a three year study technology, e-learning and distance education a scoping review of videoconferencing systems in higher education performance and satisfaction of resident and distance students in videoconference courses promoting and evaluating evidence-based telepsychology interventions: lessons learned from the university of kentucky telepsychology lab a study of the relationship between student communication interaction and sense of community the videoconferencing classroom: what do students think? multi-site videoconference tutorials for medical students in australia the effectiveness of case-based learning in health professional education. a beme systematic review: beme guide no why increased social presence through web videoconferencing does not automatically lead to improved learning videoconferencing: a new opportunity to facilitate learning. clearing house researching the community of inquiry framework: review, issues, and future directions theoretical principles of distance education continuing medical education during a pandemic: an academic institution's experience medical and surgical education challenges and innovations in the covid-19 era: a systematic review visit zoom. com and follow @zoom_us a longitudinal perspective of students' evaluations of university teaching: ratings of the same teachers over a 13-year period reduction of the students' evaluation of education quality questionnaire assessing the students' evaluations of educational quality (seeq) questionnaire in greek higher education. high educ the evaluation of the student evaluation of educational quality questionnaire (seeq) in uk higher education community of inquiry and video in higher education: engaging students online an investigation of factors that contribute to student satisfaction from online courses: the example of an online accounting course examining social presence in online courses in relation to students' perceived learning and satisfaction students' perceptions of video-conferencing in the classrooms in higher education medical students' perceptions and an anatomy teacher's personal experience using an e-learning platform for tutorials during the covid-19 crisis virtual learning during the covid-19 pandemic: a disruptive technology in graduate medical education videoconferences of infectious diseases: an educational tool that transcends borders. a useful tool also for the current covid-19 pandemic publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations the author would like to thank all undergraduate medical students for their feedback and cooperation during those unprecedented times. the author would also like to thank dr. heidi al-wassia for the scholarly discussion which contributed to developing this article. the authors declare that they have no competing interests.received: 18 august 2020 accepted: 15 october 2020 key: cord-028437-lza8eo9n authors: shabaninejad, shiva; khosravi, hassan; leemans, sander j. j.; sadiq, shazia; indulska, marta title: recommending insightful drill-downs based on learning processes for learning analytics dashboards date: 2020-06-09 journal: artificial intelligence in education doi: 10.1007/978-3-030-52237-7_39 sha: doc_id: 28437 cord_uid: lza8eo9n learning analytics dashboards (lads) make use of rich and complex data about students and their learning activities to assist educators in understanding and making informed decisions about student learning and the design and improvement of learning processes. with the increase in the volume, velocity, variety and veracity of data on students, manual navigation and sense-making of such multi-dimensional data have become challenging. this paper proposes an analytical approach to assist lad users with navigating the large set of possible drill-down actions to identify insights about learning behaviours of the sub-cohorts. a distinctive feature of the proposed approach is that it takes a process mining lens to examine and compare students’ learning behaviours. the process oriented approach considers the flow and frequency of the sequences of performed learning activities, which is increasingly recognised as essential for understanding and optimising learning. we present results from an application of our approach in an existing lad using a course with 875 students, with high demographic and educational diversity. we demonstrate the insights the approach enables, exploring how the learning behaviour of an identified sub-cohort differs from the remaining students and how the derived insights can be used by instructors. the use of online learning systems provides a rich set of data that makes it possible to extract information about student learning behaviours. this information provides an opportunity for understanding and improving education, which has motivated many universities to invest in learning analytics dashboards (lads) [6, 28, 41, 48] . these dashboards generally provide visualisations of student data, collected from a variety of educational systems, to assist educators in making decisions [41] . however, the increasing popularity and improvement of online learning systems over the years has resulted in a significant increase data in terms of its volume, velocity and variety. consequently, making sense of data in lads has become more challenging compared to earlier years [43] . in some domains, a common approach to navigating large complex multidimensional data sets is to use drill-downs [39] . a drill-down operation, in an educational setting, allows users to explore the behaviour of sub-cohorts of students by progressively adding filters. manual drill-down operations can generally be used by instructors to effectively investigate curiosity-driven questions that are related to student attributes. for example, it is possible to use a drill-down filter to find how international or female students have performed compared to other students. however, instructors may also be interested in finding which drill-down filters lead to insightful results. as an example, an instructor may be interested in finding drill-downs that identify a sub-cohort of students who have significantly different behaviour or performance compared to the rest of the class. given the availability of a large number of potential drill-downs, manually finding drill-downs that provide insights is a challenging task [1, 42] . in this paper, we report on extending lads with a functionality that provides recommendations of insightful drill-downs. our approach takes a process mining lens to examine students' learning process considering three aspects of their learning behaviour: performed learning activities, the frequency of each activity and the order in which the activities are performed. utilising the learning process, rather than focusing on aggregated engagement metrics which is the common approach in lads [41] , is increasingly being recognised as essential to understanding and optimising learning [33, 46] . in our approach, the notion of an insightful drill-down is defined as a set of filtering rules that identify a sub-cohort of students whose learning processes are most differentiated from the rest of the students. our key contribution is the design and development of an algorithm, which we refer to as learning process automated insightful drill-down (lp-aid). lp-aid employs a process mining method called earth movers' stochastic conformance checking (emsc) [29] to compute the distance between learning processes of different cohorts to recommend insightful drill-downs. we present a practical application of lp-aid in an existing lad called course insights that provides users with a manual drill-down functionality. specifically, we apply lp-aid to data from a course with 875 students, with high demographic and educational diversity, to demonstrate the drill-down recommendations and to explore the possible insights that can be derived from them. our initial findings, and instructor feedback on our approach, suggest that lp-aid can be integrated into lads to provide automated and insightful drill-down recommendations. learning analytics dashboards (lads). several recent systematic literature reviews have been published on lads [6,41]. schwendimann et al. [41] provide a comprehensive picture of the common data sources that are used by lads, which include clickstream logs (e.g., [12, 14, 25, 34] ), data related to learning artefacts (e.g., [11, 16, 20, 24, 45] ), survey data (e.g., [4, 35, 40] ), institutional databases (e.g., [9, 19, 23] ), physical user activities (e.g., [16, 31, 44] ) and data captured from external educational technologies (e.g., [10, 26, 27, 36] ). to make sense of these data lads provide a variety of visualisation options. schwendimann et al. [41] outlines the different types of visualisations that are commonly used in lads, which include bar charts, line graphs, tables, pie charts, and network graphs. while these visualisations simplify the process of making sense of large data sets, they naturally abstract away much of the details related to learning processes, which are essential to understanding and optimising learning [17] . we aim to address this challenge by employing process mining approaches to guide drill-down operations and identification of insightful data. smart drill-down approaches. the concept of a drill-down operation was initially introduced in the context of olap data cubes. they enabled analysts to explore a large search space to identify exceptions and highlight interesting subsets of data [39] . in recent years, drill-downs have also been employed in analytical dashboards. while their use has enabled users to explore large datasets, they provide users with too many drill-down choices and also the potential for incorrect reasoning due to incomplete exploration [1] . several attempts to address these challenges have been made. many of the proposed methods for discovering insightful drill-downs focus on detecting anomalies in small data portions (e.g. [1, 37, 38] ) while some focus on identifying interesting differences in larger data subsets (e.g. [21] ). in this paper, we take a similar approach as [42] by letting lad users request drill-down recommendations at a level of granularity they are interested in, thus reducing drill-down choices without affecting user autonomy. while [42] recommends drill-downs based on the difference between cohorts' attribute values, this paper bases the recommendations on the difference between cohorts learning processes. process mining aims to derive information from historical organisational behaviour, recorded in event logs [2] . educational process mining uses data from educational contexts to discover, analyse, and visualise educational and learning processes, for instance to analyse whether students' behaviour corresponds to a learning model, to detect bottlenecks in the educational process, to identify patterns in processes [7] , to study administrative processes [18] and to study student learning through their interactions with online learning environments [3, 8, 49] . prior work [7] indicates that current educational process mining solutions have not adequately provided support for allowing users to identify and investigate cohorts of interest. next, we introduce our method for recommendation of insightful drill-down criteria in lads, by first introducing relevant concepts and defining our problem statement formally, presenting our approach, and illustrating it with an example. assume that a lad has access to an event log l that captures a collection of traces t = {t 1 , . . . t n }, each representing a student. a trace t i has a unique identifier (e.g. a student id), a set of features presents v being assigned to feature f i for user s i and a sequence of events e i = e i1 , . . . e ili representing the learning path taken by student s i , where the trace length l i can vary for each student. each event e ili has a timestamp and a label representing the learning activity. a rule r expresses a condition on a feature (e.g., 'program' = 'computer science' ). for a feature with numerical values in an event log l, the corresponding rule value can be a range instead of a single value (e.g., 'age' > 25). a drilldown criterion σ is defined as the conjunction of a set of rules (e.g., 'program' = 'computer science' ∧ 'age' > 25). a drill-down criterion σ is said to cover a student s n , if all rules in σ are satisfied for the corresponding features of s n . consequently, applying σ to l leads to the selection of a set of students s ⊆ s such that σ covers each s n ∈ s . we define the coverage of a drill-down criterion c σ as |s | |s| , which is the fraction of students s covered in the resulting sub-cohort s . using this notation, our problem can be formalised as follows: formal problem statement: given an event log l, a set of features f ⊆ f , a constant 0 ≤ α ≤ 1 and a constant k, find a set of drill-down criteria σ = {σ 1 , . . . σ k } that uses features in f such that each criterion σ k : (1) has a larger coverage than α (i.e., c σ k > α), (2) selects a sub-cohort of students s that deviates most from the remaining students on their taken learning path l in terms of events, relative frequency of each different learning path and the order in which the activities have been triggered (i.e. the distance between the sub-log l and the remaining students l \ l ). we present our approach by first providing a high-level overview of the underlying algorithm, and then describing the automatic drill-down process using an example. our algorithm takes the students event log as an input and returns a set of drill-down criteria annotated with the learning process distance and students' population coverage as the output. the algorithm examines all the possible drill-down actions to find the drill-downs that result sub-cohorts with the most deviated learning processes. algorithm 1 provides the high-level pseudocode of our proposed approach. it takes four parameters as input: the event log l, the features f , the minimum coverage α and the number of drill-down criteria to be recommended k. the output of the algorithm is a set of top k scored drill-down criteria represented by σ. the algorithm consists of three main blocks as described in the remainder of this section. create drill-down tree. the buildt ree function takes two parameters as input: the event log l and the list of selected features f , and returns a drill-down tree. the function obtains all the values of each feature in f that exist within algorithm 1. finding a set of k smart drill-down criteria sort and return the top k drill-down criteria return nodet odrilldown(topk) end function function pruneandscore(log l, node parentn ode, log parentl, minimal coverage α) score nodes and prune the tree pruneandscore(l, childn ode, cohortl, α) end if end for end function l and generates a tree-like collection of nodes t , where each node represents a splitting rule r for one feature. each path in the tree consists of a set of feature-value pairs. score nodes and prune the tree. the tree embodies all possible drill-down paths, of which not all will necessarily result in a cohort with the required minimum size (i.e. α). p runeandscore traverses the tree recursively to examine all the possible drill-down actions. obtainsublog takes each node, which is a pair of feature/value pairs, and its parent's event log parentl as input and filters parentl to obtain a sub-log cohortl containing only the data of the sub-cohort. the sub-cohort's size is checked for the covered fraction of the student population to not be smaller than α and not greater than 1 − α. if the condition is met, the main event log l is filtered to obtain the event log of the rest of students remainderl. otherwise, the node is pruned (if coverage ≤ α) or discarded from scoring (if coverage coverage ≥ 1 − α). for each drill-down path, computedistance takes the pair of the sub-cohort and the remaining sub-logs as input and computes the distance between them using earth movers' stochastic conformance checking [29] . sort and return the top k drill-down criteria. topdistances takes the scored drill-down tree t and k as input and returns k recommendations. to pick the k nodes, this function uses a solution set ranking function that maximizes diversity, similar to the approach by [47] . as an alternative we could pick the k highest scored nodes. however, diversifying the recommendation allows us to provide a wider range of insightful drill-downs. our algorithm converts the chosen nodes to a set of drill-down criteria σ, each annotated with distance score and returns them as a recommendation to users. in this section, we illustrate our approach using an event log with a small set of 6 students, and k = 1 and α = 0.2. we explain how our algorithm is used to find the most insightful drill-down criteria (namely the criteria that identify a sub-cohort with the highest distance) for the event log given in fig. 1a ,b with students {s1 · s6} and the feature set: {residential status, assessment} as f . our example course has learning activities of: {lecture 1, lecture 2, quiz a, lecture 3, lecture 4, quiz b and lecture final}, which were made available to students weekly in the mentioned order. the trace of triggered learning events by each student is shown in fig. 1a . each event is represented by an activity label and the timestamp. our algorithm initially extracts all values of f that are present in the event log and generates the drill-down tree t . next, the tree is traversed depth first; based on each node's filtering criteria, the event log is divided into the sub-cohort's sub-log and the remaining students' sub-log. the nodes covering less than α = 0.2 of the student population are pruned. for instance, the node [assessment='mid grade'] is pruned as only one student (i.e. 0.16 coverage) adheres to this criteria. as a result, 5 actionable drill-down paths remain (shown in fig. 1c) our algorithm computes the distance between the sub-logs for each drill-down path and annotates each node by the distance d and the coverage (as shown in fig. 1c) . the drill-down path p5, which has the highest difference (57%), is the resulting recommendation. figure 1d shows the lp-aid interface in course insights, representing the input and the resulting recommendation, including the drill-down criteria, coverage and distance. to understand the difference between the learning behaviour of the subcohort and the remaining students, here we used disco [15] to visualise the underlying learning processes of each group. disco generates a process map in which: boxes represent activities, numbers in the boxes represent frequency of each activity, arrows represent sequence the activities were performed in (i.e. the control flow), numbers on the arrows represent frequency with which the two connected activities were performed, and thickness of the arrows the activities represent relative frequencies. for the demonstration purpose we highlighted the activities that were performed in a different order in red. to compare the two modelled learning processes, we look at the difference between the activities, their frequencies and their order. for instance, fig. 1e shows that lecture 3 was skipped by one of the two students in the cohort, while fig. 1f shows that the remaining students have done this activity. from a control flow perspective, quiz a and quiz b were performed as the last activities by the cohort while the remaining students performed these quizzes during the semester. this section presents an application of our approach using an existing lad called course insights, which is equipped with manual drill-down functionality 1 . we first provide background on course insights and its main segments. we then use data from a course that was integrated with course insights to: 1) explore the recommended drill-downs generated by lp-aid; 2) visualise the process deviation for an example drill-down, and 3) report on the comments and feedback that was provided by the course coordinator upon reviewing our recommendations. course insights. course insights (ci) is a lad that provides filterable and comparative visualisations of students' aggregated daily activities. ci aims to provide actionable insights for instructors by linking data from several sources, including a student information system, blackboard [5] , edx edge [32], and embedded learning tools such as echo360 [13] and kaltura [22] to create a multidimensional educational data set. ci is embedded in the learning management system of the university of queenslandand is available to all instructors. it is equipped with filtering functionality to enable instructors to drill-down into the data to explore the behaviour of sub-cohorts of students. figure 2a illustrates the filter interface, which allows users to select attributes from demographic, assessment, engagement and enrolment features. when a filter is applied, statistical data and a graph representing the filtered versus unfiltered distribution of the target feature is presented (as shown in fig. 2b) . we applied our technique to an introductory calculus and linear algebra course offered in 2019 to 875 undergraduate students from 16 programs. following our data cleaning process, we were left with a dataset on 739 students. as the input for our approach, the event log includes three types of learning activities: (1) accessing course materials: access to course materials by chapter. (2) submission of formative quiz: submitting chapter based practice quizzes. practice quizzes were formative assessments and thus optional. (3) review summative assessment solutions: access to chapter based workbook solutions, released weekly. workbooks were summative assessments, assigned weekly with a weekly requirement to submit their answer-sheets (paper based submissions). as the features f , we selected the attributes brand new, final exam, gender, program, and residential status. a total of 2447 drill-down actions were possible for this data set. table 1 presents the recommendations generated for this course using respectively small (α = 0.05), medium (α = 0.1) and large (α = 0.03) coverage. to investigate what insights can be derived from the recommended drill-downs, we used process discovery methods for the identified sub-cohort and the remaining students. here, we demonstrate the insights derived from the recommended drill-down (1) (shown in table 1 ). this drill-down results in a sub-cohort of: brand new = 'yes' and residential status = 'international' and final exam = 'high' and gender = 'male'. according to the lp-aid result, this sub-cohort's learning process is 72% different from the remaining students. to investigate the difference between the two learning processes we visualised the underlying process of the sub-cohort (shown in fig. 3a ) and the remaining students (fig. 3b) . each box in the map is an activity which is labeled by the action type and the relevant chapter (e.g., formative quiz-chapter1). to more clearly visually distinguish the three types of learning activities in the process map, we use color coding. in the sub-cohort's process, the arrows in between the three different types of activities indicate switching between the types of learning tasks. such switching can be an indication that the three types of tasks were being performed every week before the next chapter's activities were made available. in contrast, the underlying process of the remaining students shows that each activity type related to chapters 9 to 18 (highlighted in fig. 3b ) are mainly performed sequentially, which is indicative of students performing them at the end of the semester when all tasks were available. to further investigate our initial findings, we used disco's events' graph to compare the distribution of the events over the semester. figures 3c and d demonstrate that the sub-cohort was more active during the semester compared with the remaining students. furthermore, the average number of events per student was 36 in the sub-cohort and 25 for the remaining students, which is significantly different (p = 0.0006). to conclude our analysis, the identified subcohort had a high rate of activities throughout the semester compared to the remaining students. one of the common features of this cohort was their high performance in the final exam, which might be correlated with their developed learning process. some other differences perceived by comparing the two process maps are that the formative quiz of chapter 8 was not performed by any students of the sub-cohort, solution review of chapters 2, 7, 8 and 9 were the highestrated activities by the sub-cohort, and that solution review of chapters 1, 2, 6, 7, 8 and 9 were the highest-rated activities by the remaining students. feedback from the instructor. we presented the reported drill-down recommendations and the process visualisations to the instructor of the course to capture their feedback and comments on the findings. their feedback can be summarised as follows: (1) while the instructor had access to course insights throughout the semester, they rarely used it and generally found it to be overwhelming. they considered the large number of potential drill-down options within the platform as the main reason that made using the platform overwhelming; (2) findings of behaviour that have led to successful outcome can be used for positive deviance [30] purposes. the instructor indicated they would like to share fig. 3 as a recommended pattern of successful learning with their students as evidence that consistent engagement with learning activities throughout the semester is related to better outcomes. (3) providing the ability to receive drill-down recommendations based on a rule (e.g., 'midterm' < 50) would be useful. the instructor indicated that they would like to understand deviations in low performing and at-risk students to help them pass the course. the olap drill-down operation is commonly used in data-driven dashboards to enable users to meaningfully zoom in to explore data in more detail. for lads, this operation can be used to enable educators to identify a sub-cohort of students who deviate from class norms and who may require special attention. in this paper, we provide an automated method called lp-aid for finding and recommending a set of insightful drill-down actions to guide data exploration. to support understanding of student learning approaches, we take a process mining lens to examine and compare student learning behaviour in terms of their learning activities, the relative frequency of each different learning path and the order in which the activities were performed. it examines all drill-down paths and uses earth movers' stochastic conformance checking to score the 'insightfullness' of each path by examining the distance between learning behaviours of two cohorts. furthermore, we use a solution set ranking function that maximizes diversity to rank and select the drill-down paths for instructors to consider. we illustrated how lp-aid can be used as part of a lad to guide the discovery of insightful drill-downs. the learning processes of students based on the recommended drill-downs were visualised and compared, highlighting how the learning process of the identified sub-cohort deviates from the remaining students. feedback from the instructor of the course suggests that manual drilldowns without guidance can be overwhelming, and that insights gained from the recommendations can be shared with students to encourage change (i.e. application of positive deviance). future work aims to embed lp-aid in course insights and to partner with course instructors through co-creation to investigate (1) the practical implications of our approach and refine it accordingly; (2) the most effective way to present the drill-down recommendations to instructors and (3) the most appropriate visualisation method(s) to present the learning process deviation of sub-cohorts to instructors. avoiding drilldown fallacies with vispilot: assisted exploration of data subsets process mining -data science in action comparative process mining in education: an approach based on process cubes perceptions and use of an early warning system during a higher education transition program open learner models and learning analytics dashboards: a systematic review a survey on educational process mining clustering for improving educational process mining eco d2. 5 learning analytics requirements and metrics report improving teacher awareness through activity, badge and content visualizations graph-based visual topic dependency models: supporting assessment design and delivery at scale design and implementation of a learning analytics toolkit for teachers echo360 inc.: echo360 towards a system of guidance, assistance and learning analytics based on multi agent system applied on serious games a framework to support educational decision making in mobile learning discovering time management strategies in learning processes using process mining techniques educational process mining: a systematic literature review socially augmented argumentation tools: rationale, design and evaluation of a debate dashboard academic dashboard for tracking students' efficiency interactive data exploration with smart drill-down kaltura software company: kaltura video platform empowering l&d managers through customisation of inline learning analytics topic dependency models: graph-based visual analytics for communicating assessment data using learning analytics to investigate patterns of performance and engagement in large classes ripple: a crowdsourced adaptive platform for recommendation of learning activities development and adoption of an adaptive learning system: reflections and lessons learned effects of learning analytics dashboard: analyzing the relations among dashboard utilization, satisfaction, and learning achievement earth movers' stochastic conformance checking the power of positive deviance tscl: a conceptual model to inform understanding of collaborative learning processes at interactive tabletops analytics of learning strategies: associations with academic performance and feedback towards textual reporting in learning analytics dashboards lim app: reflecting on audience feedback for improving presentation skills addressing learner issues with stepup! an evaluation user-adaptive exploration of multidimensional data user-cognizant multidimensional analysis discovery-driven exploration of olap data cubes learning process analytics for a self-study class in a semantic mediawiki perceiving learning at a glance: a systematic literature review of learning dashboard research automated insightful drill-down recommendations for learning analytics dashboards application of big data in education data mining and learning analytics-a literature review using learning analytics to visualise computer science teamwork integrated analytic dashboard for virtual evaluation laboratories and collaborative forums from local patterns to global models: towards domain driven educational process mining defining and optimizing indicator-based diversity measures in multiobjective search learning analytics dashboard applications recompiling learning processes from event logs. knowl.-based syst key: cord-030613-i4rdhipz authors: jankowski, natasha a. title: guideposts for assessment during covid‐19 date: 2020-08-03 journal: nan doi: 10.1002/au.30222 sha: doc_id: 30613 cord_uid: i4rdhipz nan guideposts for assessment natasha a. jankowski u ncertainty is the word of the day as institutions of higher education move from covid-19 triage responses-entailing a pivot to remote learning-to planning for an unknown fall 2020. with questions of the quality of the educational experience in view (eaton 2020a) , the role of student learning outcomes assessment is more crucial than ever to counteract future concerns over transfer, quality of degree completion, and alignment with quality standards. judith eaton (2020b) of the council for higher education accreditation states that higher education has an opportunity to shift toward innovation for the longer term and asks of accreditation, "what will be appropriate norms for quality when remote learning is carried out on a larger scale over a long period of time?" (p. 2). the same question could be asked of assessment. what are the appropriate norms for assessment when remote instruction will be carried out into the fall or longer and while students, faculty, and staff live, learn, and work in a global pandemic? there is no assessment pandemic playbook, and while online assessment "best practices" provide a useful reference point, not all practices are applicable to the current situation. best practices in online assessment assume things that do not currently hold. the first is about course design and the second about students who partake in online learning. best practices imply that courses have been intentionally designed with clear alignment between learning outcomes, content, assessments, and activities-all led by faculty members fluent in the online system and clear on their role as learning support. connected with intentional design are the students who willingly signed up for online learning, who readily have all the required technology in order to fully participate, and who are clear on the expectations required of an online learner. put those two pieces together and you have the makings of an impactful online program. yet, faculty teaching in the shift to remote instruction have differentiated comfort levels and knowledge bases related to technology supports, and students have differentiated access to technology and comfort levels with online instruction. some students do not have a computer at home, and some do not have access to reliable internet or data plans to partake in online learning experiences. some do not have space to work or focus on schooling at home, are taking care of family, are homeschooling children, and are dealing with changes in employment such as shifting work hours or lost jobs, and concerns over health-mental and otherwise. some might not even have a home to which they could return. what norms do we enforce around assessment and learning in a context such as this? should students be penalized for simply being a student in a pandemic? should credit transfer be hindered due to decisions made about pass/fail options to support students in stressful times? should additional barriers to student engagement in learning be included in order to block cheating? what is decided at this juncture will say quite a bit about assessment-what it is and what it isn't-to faculty, staff, and students. further, as much as this is not a test of online learning by faculty who shifted to remote instruction, this is also not a test of online learning for students. staff at the national institute for learning outcomes assessment have been examining unfolding practices and covid-19 responses within the assessment community. it is from this examination that several possible norms for assessment in a global pandemic are offered, along with several to avoid: 1. return to assessment basics. faculty who taught courses that were intentionally designed from clear learning outcomes-with alignment between learning outcomes, assignments/assessments, and evaluative criteria-were in a better position to make the abrupt shift to remote instruction. a return to examine the learning outcomes not yet addressed, identify relevant assignments, and then send clear signals to students on what to focus their attention provided the necessary means to pivot. clarity in design, driven by learning outcomes, allowed for relevancy as opposed to quantity in assessments. further, the return to learning outcomes allowed for curricular conversations about the role of a specific course within a larger learning trajectory, including conversations on which learning outcomes students would apply elsewhere. these programmatic conversations are relevant because the learning outcomes desired by programs do not happen in a single course in a single semester. students have to run into it on multiple occasions in different ways; thus, assessment is a means of not only educational design but future programmatic planning for learning based on holistic alignment (jankowski and marshall 2017) . 2. lead with equity. inequities in the educational system were starkly apparent in covid-19 responses, as well as for which learners could actively participate in remote instruction. within institutions, there was disparity in understanding of circumstances of students and learners, sometimes furthering inequities. websites created by institutions to assist students and faculty during a crisis had titles such as "keep learning" and "keep educating." few if any responses included a clear statement that it was ok to struggle, that it is fine to take time to breathe, or that the mental health of students and faculty mattered to institutions. instead, the message was "take care of you … but log in, do the work, and turn on your zoom camera after you find a computer." outreach to better understand student needs did occur through surveys and phone calls to students, and assignments were modified in terms of flexible deadlines and removal of unnecessary assignment constraints. however, student access to reliable wi-fi was not well-addressed. some institutions offered student access to the internet through wi-fi parking lots-assuming students had a car, or a home situation that allowed them to go to a parking lot, a computer to use while at the parking lot, and a comfort level with being outside their home in order to access the internet. further, remote instruction assessment policies were implemented without concern for bandwidth access, data plans, software, and home situations. to avoid further disparities and do no harm, leading with equity should be a norm in our assessment practice moving forward. 3. involve students in the solutions. students are the experts of their lived experience. further, they can provide insight into how they are capable and able to document or show their learning. the onus is not upon the faculty member to find creative solutions to enable student learning demonstrations, but instead to co-create with students the means by which students can successfully indicate their learning. faculty and staff need to widen their lens of what falls into the category of evidence of student learning. peer assessments, reflective assignments, students with the time and interest to create tutorials or videos, phone call debates, and other solutions emerged from students in response to covid-19. involving students in discussions of how they are able to demonstrate their learning during a global pandemic may provide new and interesting solutions. 4. transparency, transparency, transparency. the prior norms are effective when there is clarity, clear communication, and transparency amongst all parties involved in learning. transparency to students involves ensuring students are aware of learning outcomes and the value of those learning outcomes to their assessments or assignments, future courses, and career paths. transparency allows for examination of how learning connects across courses, experiences, and employment. it also ensures that students are aware of what they are learning in a given course-something increasingly important in conversations on the cost of remote instruction. 1. do not allow compliance to drive decisions for enabling and supporting learning. guidance is flexible in response to decisions institutions make in support of learners. instead, clearly document decisions and reasons for decisions. 2. do not require a higher level of proof of learning in an online class than face-to-face. period. 3. do not view students as adversaries in their own learning. it is not the case that left to their own devices and given the opportunity (say, for instance, without a grade to drive them) that students will simply stop trying or learning, do not want to learn, or will not work hard. 4. do not design policy from a deficit view of students as cheaters, uninterested in their education, and not driven to succeed. design to enable learning, not block cheating and outlier students who-if they are going to game the system-will game it regardless of what is put in place to stop them. an outlier-driven policy adds unnecessary barriers in front of already overburdened students trying to be successful. a final note of caution for covid-19 norms is to avoid focusing assessment of student learning solely on reporting and meeting reporting mandates and timelines. while assessment of student learning is ongoing, the reporting of assessment might take a different form or different timeline, focused on providing formative information to feed forward into planning. reporting might occur through surveys of faculty, virtual focus groups, or phone-call check-ins. our current times require innovative and compassionate solutions. no examples currently exist. practitioners are designing them as they go, with shared open resources. there will be a story to tell about the decisions made during this time, why they were made, and their impact on students and learning. there will be stories of loss and tragedies. only time will tell what they might entail, but the norms presented here might serve as guideposts. ■ accreditation and the future of quality accreditation, quality and fall 2020: a framework for action degrees that matter: moving higher education to a learning systems paradigm jankowski is the executive director of the national institute for learning outcomes assessment at the university of illinois key: cord-035075-hx74thrr authors: nussbaum, miguel; barahona, camila; rodriguez, fernanda; guentulle, victoria; lopez, felipe; vazquez-uscanga, enrique; cabezas, veronica title: taking critical thinking, creativity and grit online date: 2020-11-09 journal: educ technol res dev doi: 10.1007/s11423-020-09867-1 sha: doc_id: 35075 cord_uid: hx74thrr technology has the potential to facilitate the development of higher-order thinking skills in learning. there has been a rush towards online learning by education systems during covid-19; this can therefore be seen as an opportunity to develop students’ higher-order thinking skills. in this short report we show how critical thinking and creativity can be developed in an online context, as well as highlighting the importance of grit. we also suggest the importance of heuristic evaluation in the design of online systems to support twenty-first century learning. this paper is in response to the article "designing for 21st century learning online: a heuristic method to enable educator learning support roles" (nacu et al. 2018) . in this paper, the authors outline a framework for heuristic evaluation when designing online experiences to support twenty-first century learning. twenty-first century skills can be key to success in a modern knowledge society. among these skills, critical thinking is important not only at work, where problem solving is essential, but also in any social setting where adequate decision making is required (dwyer and walsh 2020) . additionally, creativity helps ensure that the outcomes of critical thinking can be both culturally ingenious as well as treasured (yeh et al. 2019b ). this is achieved by embracing cognitive abilities in order to create new combinations of ideas (davis 1969) . technology has been shown to facilitate the development of higher-order thinking skills in learning (engerman et al. 2018) . however, in general, schools have failed to take advantage of this by incorporating adequate use of technology into their practices (olszewski and crompton 2020) . therefore, the rush towards online learning by education systems during covid-19 can also be seen as an opportunity to develop students' higher-order thinking skills. one potential drawback with online learning is the distance it creates between peers, thus hindering student engagement and the development of higher-order thinking skills (dwyer and walsh 2020) . we show how this barrier can be overcome when developing critical thinking and creativity in an online context. critical thinking includes the ability to identify the main elements and assumptions of an argument and the relationships between them, as well as drawing conclusions based on the information that is available, evaluating evidence, and self-correcting, among others. it is seen as a self-regulated process that comes from developing skills such as interpretation, analysis, evaluation and explanation; going beyond technical skills. it can therefore be considered a metacognitive process (saxton et al. 2012; facione 1990) . by taking learning online, both self-study and teacher-led sessions can be enhanced through a problem-based learning strategy. in the first stage, students build on a question or topic posed by the teacher, e.g. a mathematical problem or an essay writing assignment. in the second stage, students peer-review their classmates' responses or essays using a rubric provided by the teacher. students break down their classmates' responses and see how they relate to the objective of the activity. they then compare this analysis with the rubric in order to provide feedback. in a third stage, the students develop a new response based on their initial response, the experience of giving feedback, and the feedback they received. this process develops self-evaluation as the students compare their own response with their classmates' and discover any gaps in their knowledge. it can also develop metacognition as they integrate various sources of knowledge (initial response, feedback received and the experience of giving feedback) when developing a new response. in the final stage, the teacher discusses the different responses with the class. the teacher then compares the students' work with the expected response and provides a general summary, transferring the responses to different domains. while stages 1 through 3 are asynchronous and computer-aided, stage 4 can be synchronous and supported by the use of a web-based video conferencing tool. active student participation and teacher mediation are both key since interactive and instant feedback has been shown to improve critical thinking (chang et al. 2020) . in addition to the problem-based strategy presented here, other active learning strategies can also be used to develop critical thinking, e.g. structured questioning, role playing, and cooperative learning (cruz and dominguez 2020) . how these might be implemented online is still open to discussion, though heuristic evaluations may be a good alternative given the possibilities presented by online learning as a resource provider, learning broker and learning promoter (nacu et al. 2018 ). creativity is an essential element of the problem-solving process. creative people often find ways of addressing a problem that others cannot see, while also having the ability to overcome barriers where others may otherwise give up (kaufman 2016 ). there are different techniques for developing creativity. in-depth learning is facilitated when students represent concepts based on their own personal perceptions (liu et al. 2018) . in this sense, analogy can be a powerful tool for boosting creativity. analogical transfer includes the idea of making analogies by analyzing objects, ideas or concepts across domains, i.e. information is transferred from the known (the original domain) to the unknown (the new domain) by searching for similarities (shen and lai 2014) . we propose an analogical transfer strategy. in the first stage, the teacher identifies a concept with examples from different domains. this might include showing a video that not only introduces the concept but also provides a context that is both familiar and relatable for the students. in the second stage, students reflect on situations from their own lives where they can apply the concept that is being studied. here, the use of open-ended questions allows the students' creativity to be explored in greater depth, while adapting to their different backgrounds and levels of prior knowledge. in the third stage, which is mediated by the teacher, the students discuss their responses from stage 2. the teacher should focus on original responses from different domains, or responses where it is not clear whether the solution is correct. stages 1 and 2 can be conducted asynchronously and scaffolded using technology through the inclusion of multimedia and student guides. however, stage 3 should be synchronous and supported by the use of a web-based video conferencing tool. in this way, technology facilitates the development of creativity by facilitating the discovery process, the collection of ideas, and the integration of knowledge (yang et al. 2018) . mediation in stage 3 is therefore key (giacumo and savenye 2020) . effective teacher-student dialogue can improve the teacher-student relationship and enhance the creative process. heuristic evaluation can therefore help us understand this relationship by looking at these interactions on the online platform (nacu et al. 2018 ). as with any learning process, critical thinking and creativity require students to be both present and focused, which in turn requires grit (yeh et al. 2019a) . in other words, the way in which students approach their schooling is just as important as what and how we teach them (tissenbaum 2020) . grit should therefore not only be considered an essential element of academic achievement but also as a mental process that activates and/or directs people's behavior and actions (datu et al. 2018, lan and moscardino 2019) . this is particularly relevant in a covid-19 context, where the pandemic is affecting the wellbeing and mental health of many students, families & communities (oecd 2020). in order to achieve effective student engagement, the objective must be attainable, interesting and accessible (i.e. in their zone of proximal development). the means used to complete the task must be attractive and feel more like a reward than an assignment. finally, the teacher should work on the students' persistence, not just in order to complete the task but as an essential quality for everyday life (barnes 2019) . teacher grit may also be key. as haderer (2020) suggests "why do some teachers stay when others run from the challenges?" in this sense, reflection has been shown to be relevant for teacher efficacy and grit (haderer 2020) . heuristic evaluation methods may therefore allow the educator to understand the learning system as a whole (nacu et al. 2018 ). as indicated in (nacu et al. 2018) we are "faced with the need to create youth-centered spaces that also provide adult facilitation of learning". heuristic evaluation can therefore help connect online platforms with students, teachers and twenty-first century skills needs. perseverance in mathematical reasoning: the role of children's conative focus in the productive interplay between cognition and affect from experiencing to critical thinking: a contextual game-based learning approach to improving nursing students' performance in electrocardiogram training engaging students, teachers, and professionals with 21st century skills: the 'critical thinking day' proposal as an integrated model for engineering educational activities the triarchic model of grit is linked to academic success and well-being among filipino high school students training creativity in adolescence: a discussion of strategy an exploratory quantitative case study of critical thinking development through adult distance learning. educational technology research and development games for boys: a qualitative study of experiences with commercial off the shelf gaming. educational technology research and development critical thinking: a statement of expert consensus for purposes of educational assessment and instruction executive summary "the delphi report asynchronous discussion forum design to support cognition: effects of rubrics and instructor prompts on learner's critical thinking, achievement, and satisfaction exploring the relationship between teacher efficacy and grit. doctoral dissertation creativity 101 direct and interactive effects of perceived teacher-student relationship and grit on student wellbeing among stay-behind early adolescents in urban china enhancing language learning through creation: the effect of digital storytelling on student learning motivation and performance in a school english designing for 21st century learning online: a heuristic method to enable educator learning support roles. educational technology research and development educational technology conditions to support the development of digital age skills a framework to guide an education response to the covid-19 pandemic of 2020 the critical thinking analytic rubric (ctar): investigating intra-rater and inter-rater reliability of a scoring mechanism for critical thinking performance assessments formation of creative thinking by analogical performance in creative works i see what you did there! divergent collaboration and learner transitions from unproductive to productive states in open-ended inquiry examining creativity through a virtual reality support system mindful learning: a mediator of mastery experience during digital creativity game-based learning among elementary school students enhancing creativity through aesthetics-integrated computer-based training: the effectiveness of a face approach and exploration of moderators victoria guentulle is an engineer who is doing a phd at the school of engineering of the felipe lopez is an engineer who is doing a phd at the school of engineering of the enrique vazquez-uscanga is an engineer who is doing a phd at the school of engineering of the veronica cabezas is an assistant professor at the school of education, pontificia universidad católica de chile, and associate researcher at millennium nucleus of social development. she is co-founder of teach for all in chile the research results informed in this report were supported by anid/fondecyt 1180024. the different research projects underlying this report received approval from the university's ethics committee. the participation was voluntary and the students signed an informed consent form. key: cord-013263-xw611i8k authors: dederichs, melina; weber, jeannette; muth, thomas; angerer, peter; loerbroks, adrian title: students’ perspectives on interventions to reduce stress in medical school: a qualitative study date: 2020-10-15 journal: plos one doi: 10.1371/journal.pone.0240587 sha: doc_id: 13263 cord_uid: xw611i8k the mental health of medical students remains to be a matter of concern. numerous setting-based and individual-based interventions for student mental health have been proposed in the literature. however, the student perspective on those interventions has been largely neglected. this study aims to explore how medical students perceive different interventions and if they desire any additional changes with regard to their studies. eight focus groups with 71 participants were conducted at a large german medical school. focus groups were recorded, transcribed and content-analyzed using maxqda 18. we found that medical students prefer setting-based interventions. most proposed interventions were on a setting-based level. for instance, students asked for more information on the university’s psychosocial counseling services and for better information management regarding contact persons. interventions proposed in the literature received mixed reactions: several participants did not favour a pass/fail grading system. students considered a peer-to-peer mentoring program for freshmen very helpful. students had diverse attitudes towards balint groups. they approved of several self-management courses, most of them being related to time or stress management. interestingly, the most urgently wanted interventions appear to be rather easy to implement (e.g. a mentoring program). this study explored the medical student perspective on student mental health interventions. additionally, our study illustrates the benefit and feasibility of involving students early on in the conception of interventions. further research with a representative sample is needed to obtain broader information on the acceptance of the suggested interventions. medical students show a high prevalence of mental illnesses worldwide, such as high levels of depression and depressive symptoms [1] .to address medical students' poor mental health, setting-based and individual-based interventions have been proposed [2] [3] [4] [5] . setting-based interventions aim to improve health by modifying environmental factors, e.g. curricular changes in medical schools [3, 6, 7] . individual-based approaches, by contrast, seek a change in the individual, e.g., by providing skills that help to cope with stress [8, 9] . although some interventions different perspectives. we therefore conducted a qualitative study with focus groups to gather insights and specific explanations for why students deemed specific interventions useful or not. our study aims to answer the following questions: 1. what interventions do medical students themselves suggest and wish for? 2. how do medical students perceive interventions that have been described in the literature (pass/fail grading, a peer-to-peer mentoring program, balint groups, and self-management courses)? participants were recruited through social media or advertisement on campus of heinrich-heine-university of düsseldorf in germany. the only criteria for participation were to be currently enrolled as medical student and not having participated at our previous focus groups [29] . participants of one focus group were recruited through a stress-management seminar. participation was explicitly on a voluntary basis. all participants gave their informed consent. students were compensated for their participation with two cinema tickets and one cinema discount card. focus groups were conducted until data saturation was reached. students were grouped into focus groups according to their period of study. this maximised the likelihood that participants could relate to each other in terms of experience regarding classes and exams. four groups included students in the preclinical part of their studies (year 1 to 3) and four groups included students both from the clinical part (year 4 to 5) as well as students in their clinical internship year (after the second state examination). the focus groups took place in june and july 2019 at the medical faculty of heinrich-heine-university düsseldorf. in 2013, the medical faculty introduced a new curriculum. traditionally, medical students take one final exam organised by the state (state-examination) for each of the three periods of study: the first after the preclinical phase (after year 3), the second after the clinical phase (after year 5) and the last one after one year of practical internship at a hospital. with the new curriculum, the first state examination takes place after the first two years. permission to conduct this study was given by the ethics committee of the medical faculty of heinrich-heine-university of düsseldorf. after provision of written informed consent, participants were asked to fill out a questionnaire covering demographic data. the focus groups were conducted by tm, a faculty member and psychologist by training, while md, a junior researcher and trained psychologist, took notes. the focus groups followed a topic guide developed by md and tm (s1 file). both facilitators provide lectures for medical students and are involved in research on medical students' wellbeing. participants were informed about the facilitators' research focus. first, participants were asked about their experience in medical school so far and in which situations they had encountered obstacles or experienced stress. secondly, students were asked about potential changes and interventions that could contribute to stress reduction in medical school. previous research at our faculty already pointed out high stress levels and specific stressors of our medical students [29, 30] . to answer the second research question (how the students perceived literature-based interventions), we discussed interventions proposed in the literature with medical students. the topic guide contained a list of the interventions (see introduction section), which were explained by the facilitator and then discussed by the students one after the other. if necessary, the facilitator asked follow-up questions to further explore students' views. every single participant was encouraged to contribute to discussions at an early stage. all focus groups were audio-recorded, transcribed and content analyzed by md and jw according to mayring [31] using maxqda 18. jw is experienced with occupational health and qualitative research [29] . research questions were transformed into deductive main categories (students' suggestions, interventions from literature). during analysis these categories were further broken down into sub-categories by inductive category formation. after md completed the first coding round of 25% of the data, al, an experienced qualitative researcher [29, 30, 32, 33] , reviewed the coding scheme. the coding scheme was then adapted and a second coding round was performed by md. jw coded the data of four groups independently. both versions were compared and discrepancies resolved. after that, md recoded the remaining four groups and transferred changes made to the coding system in the previous discussion with jw. finally, jw and md discussed the focus groups once more. there was no need for further recoding since there were only minor differences between the analyses of md and jw. due to logistic constraints, corrections and feedback on transcripts and research findings were not obtained from study participants. the completed checklist of consolidated criteria for reporting qualitative research (coreq; [34] ) can be found in s2 file. eight focus groups consisting of 5 to 16 medical students were conducted. focus groups lasted between 90 and 130 minutes. in total, 71 medical students (56 women) participated. with n = 2816 medical students being enrolled at the medical faculty in the summer semester of 2020, we included 2.5% of the population in our focus groups. participant age ranged between 19 and 39 years (mean = 23.71; sd = 3.50). all participants engaged in a comprehensive discussion about what they would like to change in medical school to improve student wellbeing. in the following, we will present their proposed starting points for interventions. regulations for absence of lecturers. students reported that especially in the clinical study phase, many lecturers did not show up for teaching. students asked for a substitution or at least a designated contact person. attendance. attendance rules were perceived to be too strict. according to the study regulations, an attendance of at least 85% presence per subject is required. however, if one subject consists of three appointments for example, all three seminars have to be attended. by attending only two of them, the presence would be 67%, thus below 85%. students therefore requested that if one misses class due to sickness or death of a family member, an exception should be granted. according to the participating students, there should also be the option to complete a compensatory task, for instance, holding a presentation on the subject that was missed. as another solution for not missing a mandatory event, participants suggested switching groups with their peers for the required event only. due to the high number of mandatory classes during the preclinical phase, students wished for less mandatory attendance during this phase. coordination of practical and theoretical phases. the düsseldorf medical curriculum combines phases of theoretical studying (lectures) and practical experience (clinical traineeship). students wished that the curriculum is adapted so that theoretical and practical lectures as well as corresponding tutorials are better aligned and deal with the same subjects. this was believed to increase their learning effect. students noted that the workload between theoretical and practical phases fluctuates significantly with high workload during theoretical phases and low workload during practical phases. therefore, they wished for shorter practical phases to distribute the workload more evenly. often, the lecture-free week takes place after the clinical traineeship. however, students would rather have a week off for studying after completing the theoretical lectures when they find to have more content to repeat and memorize. counseling. participants pointed out that they would benefit from a broader psychological support system. regarding the on-campus counseling service, students wished for longer opening hours as well as opening hours that are compatible with their schedules. free of cost confidential counseling services were perceived as useful for reaching out to students to address mental health issues. overall, students wished for more information regarding mental health problems and interventions. possible solutions students brought up were for instance a mandatory lecture about stress related to medical school, coping strategies and support contacts. additionally, students suggested that a newsletter should be emailed close to the first state examination to present information on emergency contacts and counseling services. registration processes for elective subjects. students explained that when they want to register for a subject, they have to log in to an online portal. there they would have to wait until midnight until the registration process opens. they further explained that registrations are allocated on a first come, first serve basis. therefore, students stated that without access to a fast and reliable internet connection at that time they would not be able to take the class they prefer. students would like this procedure be replaced by a selection based on preference. in this procedure, students could indicate their preference within a certain period. then they would be matched according to their choice. if there is no personnel to do this task, some students suggested that at least the first come-first serve procedure was opened during daytime. information management. students reported a lack of information management in medical school. they requested a comprehensive online portal containing information on classes and exams. moreover, students expressed that they would benefit from an introductory lecture for each new topic block that contains all relevant administrative information, e.g. on how to register for an exam. a faq that addresses most urgent questions concerning the first state examination was considered useful. online lectures. many students raised the topic of online lectures. they perceived online lectures as an efficient measure to reduce stress and improve flexibility. especially commuters were thought to profit significantly from it. online lectures were thought to be a great learning tool and aid with the preparation of future seminars. one student requested a contact person in case any questions arise during a session. "i think it would be great to have this opportunity. i personally learn at night. in the morning, i don't pick up much, but at night i am really diligent. and it were a lot easier to combine family, job and university. that would be a huge advantage." refresher courses. students stated that they would benefit from refresher courses before the second state examination. "i believe it would be pretty fair, for instance, to have courses that prepare you for the state examination in practical terms during the semester. that would be, i think, not bad." examinations. students reported that their exams cover several subjects at once. students asked for a restricted number of subjects within the same exam. they reported that this could be achieved if exams included more questions on the same subject per exam instead of the same subject posing only a few questions per exam over several exams. "now for instance [ they also suggested grades acquired in seminars should be added to the exam score and act as a buffer. it was discussed whether more open-ended questions instead of multiple-choice questions would be beneficial. some students stated that they could explain their knowledge better in open-ended questions. moreover, it would motivate them to learn more details. others preferred the existing multiple-choice format. teaching content. participants made a range of suggestions to improve the quality of teaching. for instance, students wished for a contact person to report verbal harassment of students like depreciative comments by teaching staff. exact teaching guidelines containing learning goals for every subject were considered potentially helpful. they also wished for more covering of content relevant for the second state examination. a perceived lack of interest and skill of teaching staff was thought to be solved through employing senior students or external referents. they asked for more interactive and clinically orientated teaching. elective subjects. in total, 14 elective subjects have to be completed over the course of five years at the medical school in düsseldorf. some of the subjects are graded. participants raised the idea to reduce the total number of required courses to reduce stress and provide more time for state exam preparation. "it is good that we have elective subjects. but there are too many. and it is stressful, to have eight elective subjects here and another six elective subjects there. it would be great if we had less." alternatively, additional credit for scientific projects, clinical lectures or extracurricular activities was suggested. clinical traineeship. students asked to move the clinical traineeships towards the end of the medical curriculum, because students felt that they are often confronted with medical conditions for which they are not prepared yet and often clinicians presuppose content they have not dealt with yet. students reported frequent difficulties in their clinical traineeships. for instance, clinicians that are supposed to teach them are often not available and, if they are present, students felt that they do not take time to teach them adequately. more available contact persons or a central complaints office that assists them were mentioned as a possible solution to this issue. preparation for the second state examination. students wished for lecture free and exam free time for preparation before the second state examination. they requested to have at least 100 days to dedicate completely to studying. this way they stated they could follow the learning schedule they wanted to use (so called "100-day study plan" which is well-known and widely used by medical students in germany). "it is a question of planning. then, if you have these one hundred days, i think that would help a lot. and it would be nice if it wasn't exactly one hundred days but a couple of days more because maybe you don't study seven days a week." participants had many ideas on how to provide more time to study. participants proposed shortening clinical trainings, or to move them towards the end of the medical curriculum to avoid that students have to pass other exams right before the second state examination. students were willing to complete their clinical training during semester break or to begin the semester early to provide more preparation time for the second state examination. measures proposed in the literature pass-fail system. there was no consensus among the students in our focus groups related to the usefulness of pass-fail systems. some were in favour of eliminating grades because it would reduce pressure and stop demotivation through bad grades. others in favour of the pass-fail system argued grades did not have any informative value regarding their ability as a physician anyway and were thus of no importance. students opposing the pass-fail system reported to appreciate the incentive they received from good grades and to appreciate the feedback to better understand whether they had learned enough. students depending on a scholarship or on receiving particularly good grades raised concerns regarding their ability to compete and prove their performance. scholarship holders among the students reported that in order to keep their scholarship, they had to prove to belong to the top students in their semester. without grades, this might be difficult to prove and students from other universities might have an advantage. in düsseldorf, the final grade for the first state examination is calculated as a cumulative sum score of grades from the first to the third year and of the performance on the day of the state examination. students argued that this way they did not solely depend on one's day performance and therefore the stress on the day of the first state examination was reduced. some preferred keeping grades to maintain this system. however, students suggested alterations of the pass-fail system. they were in favour of eliminating grades from some subjects to reduce stress but maintain the current system of collecting grades for the first state examination. mentoring. almost all participants were in favour of a mentoring program. they appreciated it for its easy access and voluntary nature. the idea of having a permanent contact person that provides not only informational and emotional support but also further networking opportunities to higher semesters was welcomed. the personal experience and organizational knowledge of a senior student was thought to reduce stress significantly. "i would have liked having someone who told me 'everything is fine, this is normal. i experienced it too.' especially at the beginning of medical school. . . i had just moved here. . . and everything was new. having someone who can answer questions related to studying or simply help you." many students in senior semesters expressed the desire to become a mentor themselves. those who did not want a mentoring program stated they already established such a contact to a senior student by themselves. there were many suggestions regarding the design of a mentoring program. a pool with volunteers, possibly on an online-based platform was proposed. students further proposed that mentors should receive some form of compensation, either monetary or in a non-monetary form, like an honorary position. students argued that their willingness to participate in a mentoring program also depended on the mentor's personality. they feared that overly performance-driven students might be a bad role model in terms of stress management. students who would not want to participate in a balint group stated that they preferred being mentored by their peers or supervising physicians. they reported to rely on their own social networks for emotional support. some students expressed fear of stigmatisation due to participation in a balint group. they worried about talking freely about unpleasant experiences and being stigmatized by their peers. therefore, they would not want to participate. self-management courses. students were interested in learning self-management strategies or competences to strengthen their resilience. in this context, they wished for subjects on stress-management and relaxation techniques. moreover, students wished for a time-management course and more capacities in an existing elective subject deals with mind-body-medicine. participants stated they would benefit from courses that covered efficient learning strategies and presentation skills. there was little consensus among participants regarding the question when and how those classes should be implemented in their curriculum. in general, students perceived lack of time as a barrier to participate in a non-obligatory class. some students suggested that it should be an ongoing offer over a long period of time where students could decide for themselves every week whether they wanted to participate or not. students preferred to schedule such courses at the beginning of medical school and shortly before the first state examination. general dissatisfaction with already existing self-management courses on campus was expressed. students either perceived lack of information about those classes, lack of quality of available classes or limited capacity to enroll all interested students. the aim of the present study was to explore which interventions students suggest to improve their mental health and to discuss interventions suggested in the literature. in our eight focus groups, students suggested specific solutions to their perceived obstacles in medical school. interventions that students proposed most frequently pertained to setting-based aspects such as curricular changes, improved information management and new regulations concerning absence of teaching staff and students. many of the suggested interventions, such as online lectures, appear rather easy to implement. due to the current covid-19 pandemic (time of publication; [35] ), our faculty addressed the wish for more online lectures by creating new digital learning opportunities and structures. eventually these structures will persist after the university switches to conventional lectures once again. it is also striking that many of the suggested interventions might not only reduce stress, but also improve several aspects of teaching (e.g. the coordination of practical and theoretical phases). this suggests that improvement of well-being and improvement of academic outcomes go into the same direction. in contrast to students' wish for setting-based interventions, most interventions being proposed in the literature focus on the individual [8] . however, such interventions are believed to not tackle the root of the problem [36] . it is argued that instead of teaching students how to cope with stress, the causes for stress need to be addressed [36] . some individual-based interventions (e.g. stress-management courses) might even have the opposite effect by being an addition to existing classes and workload. it is further of interest that we observed that a significant number of our participants did not favor the interventions suggested by the literature. for instance, several students did not prefer a switch to a pass/fail grading system. these students perceived grades as helpful, were concerned that the absence of grades would have a negative impact on their academic performance, or would cause increased stress at the first state examination. however, this latter concern is not supported by the data [12, 37] . it is possible that the idea of a pass/fail grading system elicits discomfort among our students because it is rather uncommon in germany. when the pass/fail system was first introduced in the us similar concerns were raised [38] . a shift to a pass/fail system was perceived as a disadvantage for students and students coming from a university with pass/fail grading were thought to experience difficulties finding a residency placement. therefore, the authors did not recommend such a transition [38] . however, a recent study showed that a pass/fail grading does not curtail career prospects [29] . specifically, there were no differences in residency placement (receiving a placement at all and percentage of students that receive top specialty choices) and overall academic performance between pass/fail and tiered grading system cohorts [29] . besides pass/fail grading being uncommon in germany, it would also require a change of the persisting score calculation for the first state examination. additionally, it would require a cultural shift towards a non-competitive environment among all students in order to be successful. therefore, instead of fully switching to pass/fail grading only, a mixed approach might be more feasible in which pass/fail grading could be adopted in some subjects while other subject are still graded. in this approach, grades still contribute to the result of the state examination. this would address medical students' preference to accumulate marks and thereby reduce stress during the first state examination without a prompt change that requires time for adaptation. possible mentoring programs by senior students were well received by the participants in our study. more advanced students were believed to provide invaluable informational and emotional support. since such a program has relatively low costs and was also favored by potential mentors, we propose starting a peer-to-peer mentoring program at our university as recommended by others [39] . balint groups received mixed reactions. this is in line with previous research suggesting indifferent to mildly positive student attitudes towards balint groups [21, 22] . a qualitative study from finland reported a more positive evaluation of balint groups [23] . here, students reported being satisfied with their participation and benefiting from the groups [23] . in our present study, especially those students who had already encountered a difficult situation with a patient stated that they would be grateful for this kind of opportunity and support. therefore, the implementation of a balint group with voluntary participation on request as suggested by students could be considered. it could build on already established peer-support-programs such as the programs at brigham and women's hospital in boston, which has repeatedly served as a model for peer-support-programs [40] . students themselves requested a variety of self-management courses. most were interested in learning relaxation techniques and how to deal with stress. they stated that existing offers, like a time management class, are less attractive because their study curriculum does not allow for the implementation of taught strategies in this class. overall, they expressed the wish that classes to improve resilience should be accessible to everyone. some stress management courses (e.g. "stress management" or "mind-body-medicine") are only offered as elective subjects and are therefore only available for a fraction of the students. we suggest that all students receive a basic stress-management training and psychoeducation, preferably at the beginning of the curriculum. at some universities, such as the monash university medical school in australia, mindfulness and stress management have been part of the curriculum for many years [41] . since some studies found reliable short-term effects [8, 42] , students could benefit from it in most stressful times without the concern of losing time to study. for the latter reason we also suggest that more classes that address mental health are offered. in addition to face-toface training, e-mental-health-solutions could be made accessible for students for quick support. students' wish for e-learning opportunities suggests that they are open to digital formats that grant them more independence in terms of time and location of use. furthermore, information on specific counseling opportunities should be made more accessible. it is possible that raising awareness of medical students' mental health will reduce fear of stigmatization by peers which was mentioned as a concern in the context of balint groups. overall, we find that students proposed more setting-based interventions than individualbased interventions. this is in sharp contrast with the persistent emphasis on individual-based interventions in medical schools [8] [9] [10] 43] . importantly, while setting-based interventions are sometimes considered expensive or difficult to implement [7] , most ideas in the focus groups (e.g. attendance rules, a new course selection procedure, teaching guidelines) seem easily feasible and resource-friendly and will not only improve wellbeing, but also academic performance. a strength of this study is its rich data, which were collected among as much as 71 students from a broad range of semesters in eight focus groups until data saturation was reached. however, the generalizability of our findings may be limited since only students from the medical school at the university of düsseldorf were included in this study. our findings might be specific to issues and obstacles encountered at universities in germany and therefore only somewhat transferable to universities in other countries. when we asked the participants about their opinions on a pass/fail grading system, we did not clarify in detail whether they would still like feedback on their performance if a pass/fail grading system was implemented. thus, we do not know to what extent students perceive a pass/fail grading system and feedback on their performances mutually exclusive. further, we cannot rule out selection bias. one might argue that only those students suffering from stress or those who are especially dissatisfied with medical school attended the focus groups. on the other hand, one might assume that those students experiencing a high level of stress choose not to participate because of the additional workload. the fact that focus group facilitators were members of the teaching staff and that one focus group was held within a class on stress management could also have affected the observations. md's and tm's position as a teaching staff member might have elicited a social desirability bias and reduced willingness to share sensitive topics. however, also in this seminar, we felt that students spoke very openly about their issues. the contents of this focus group did not differ thematically from the other focus groups. this study explored which type of interventions students consider acceptable and useful. based on our data we are however unable to evaluate the feasibility and effectiveness of proposed interventions. however, we believe that only interventions that are favored by students and address their specific needs will be successful. in contrast to interventions to improve medical students' mental health that are proposed in the literature, we find that medical students mostly proposed interventions on a setting level rather than on an individual level. importantly, many interventions suggested by the students are low-cost and easy to implement. we believe that considering the student perspective is a key factor in designing mental health interventions. further research with a representative sample is needed to obtain more generalizable information on the acceptance of the proposed interventions and to test them in terms of feasibility and effectiveness by using both qualitative and quantitative approaches. prevalence of depression, depressive symptoms, and suicidal ideation among medical students: a systematic review and meta-analysis medical student mental health: culture, environment, and the need for change medical student mental health 3.0: improving student wellness through curricular changes. academic medicine: journal of the association of american medical colleges medical school strategies to address student well-being: a national survey. academic medicine: journal of the association of academic psychiatry: the journal of the american association of directors of psychiatric residency training and the association for academic psychiatry holistic and sustainable health improvement: the contribution of the settings-based approach to health promotion setting-based interventions to promote mental health at the university: a systematic review effectiveness of universal programmes for the prevention of suicidal ideation, behaviour and mental ill health in medical students: a systematic review and meta-analysis. evidence-based mental health mindfulness interventions in medical education: a systematic review of their impact on medical student stress, depression, fatigue and burnout association between learning environment interventions and medical student well-being: a systematic review the benefits of pass-fail grading on stress, mood, and group cohesion in medical students impact of pass/fail grading on medical students' well-being and academic outcomes differences in medical students' academic performance between a pass/fail and tiered grading system a systematic review of the literature describing the outcomes of nearpeer mentoring programs for first year medical students introducing first-year students to medical school: experiences at the faculty of medicine of erasmus university students helping students: vertical peer mentoring to enhance the medical school experience the doctor, his patient and the illness research on balint groups: a literature review the impact of a physician awareness group and the first year of training on hematology-oncology fellows stressors perceived by the para-clinical undergraduate medical students teaching the clinical encounter in psychiatry: a trial of balint groups for medical students reflecting on our practice: an evaluation of balint groups for medical students in psychiatry a qualitative analysis of student balint groups in medical education: contexts and triggers of case presentations and discussion themes. patient education and counseling self-management education: history, definition, outcomes, and mechanisms. annals of behavioral medicine user involvement: a review of the benefits and challenges. behaviour & information technology acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. bmc health services research helping medical students develop lifelong strategies to cope with stress a comprehensive medical student wellness program-design and implementation at vanderbilt school of medicine. academic medicine: journal of the association of american medical colleges stressors and resources related to academic studies and improvements suggested by medical students: a qualitative study medical students' perceptions of stress due to academic studies and its interrelationships with other domains of life: a qualitative study qualitative content analysis: theoretical foundation, basic procedures and software solution what are the perceived influences on asthma self-management at the workplace? a qualitative study psychosocial working conditions and diabetes self-management at work: a qualitative study consolidated criteria for reporting qualitative research (coreq): a 32-item checklist for interviews and focus groups meeting between federal chancellor merkel and the minister-presidents of the lä nder to discuss the coronavirus-press release 2020 medical student perspective on stress: tackling the problem at the root evaluating a grading change at ucsd school of medicine: pass/fail grading is associated with decreased performance on preclinical exams but unchanged performance on usmle step 1 scores pass fail grading-a disadvantage for students applying for residency stressors and starting points for health-promoting interventions in medical school from the students' perspective: a qualitative study peer support for clinicians: a programmatic approach the health enhancement program at monash university medical school enhancing the health of medical students: outcomes of an integrated mindfulness and lifestyle program academic psychiatry: the journal of the american association of directors of psychiatric residency training and the association for academic psychiatry we would like to thank johanna schwerdt for her invaluable help understanding the medical school curriculum and putting the students' perspectives into the right context. loerbroks. key: cord-335582-30rws724 authors: bonal, xavier; gonzález, sheila title: the impact of lockdown on the learning gap: family and school divisions in times of crisis date: 2020-09-15 journal: int rev educ doi: 10.1007/s11159-020-09860-z sha: doc_id: 335582 cord_uid: 30rws724 the lockdown of schools in spain to confront the effects of covid-19 caused an enormous impact at both societal and educational levels. schools and families had to react rapidly to a new teaching and learning scenario without the benefit of previous planning or government guidelines. in this context, some schools were better able to adapt to the new circumstances than others. likewise, the structure and size of families’ economic, social and cultural capital produced significant differences in the learning opportunities for children from different backgrounds. this article assesses the impact of the school lockdown on the learning gap between children from different social backgrounds in catalonia. based on 35,419 responses to an online survey administered between 26 and 30 march 2020 to families with children aged between 3 and 18, the authors’ analysis shows that learning opportunities varied significantly. middle-class families were able to maintain higher standards of education quality in a critical context, while children from socially disadvantaged families had few learning opportunities both in terms of time and learning experiences (schoolwork and maintenance of after-school activities). results differed by type of school (public/private) where students were enrolled, family economic, social and cultural capital, and family living conditions. in the final part of the article, the authors highlight the importance of the role of the school in ensuring learning opportunities for children from low socioeconomic backgrounds, and they discuss some policy implications of their findings. l'impact du confinement sur les écarts en matière d'apprentissage : disparités familiales et scolaires en période de crise -la fermeture des écoles en espagne durant le confinement pour faire face aux effets de la covid-19 a eu d'immenses répercussions sur les plans sociétal et éducatif. les écoles et les familles ont rapidement réagi au nouveau scénario de l'enseignement et de l'apprentissage sans pouvoir s'appuyer sur une planification préalable ou des directives gouvernementales. dans ce contexte, certaines écoles ont réussi mieux que d'autres à s'adapter à ces nouvelles circonstances. de même, la structure et la taille du capital économique, social et culturel des familles a montré qu'en fonction de leur milieu, les enfants avaient des possibilités d'apprendre très inégales. cet article évalue l'impact de la fermeture des écoles en espagne durant le confinement sur les écarts en matière d'apprentissage chez des enfants de différents milieux sociaux en catalogne. s'appuyant sur 35 419 réponses à une enquête en ligne menée entre le 26 et le 30 mars 2020 auprès de familles avec des enfants âgés de trois à dix-huit ans, l'analyse des auteurs révèle de considérables disparités concernant les possibilités d'apprendre. les familles de la classe moyenne ont réussi à maintenir un niveau d'éducation élevé dans cette situation critique, tandis que dans les familles défavorisées sur le plan social, les possibilités des enfants étaient restreintes, tant en termes de temps que d'expériences éducatives (devoirs et maintien d'activités extrascolaires). les résultats étaient différents en fonction du type d'établissement (public/privé) où les élèves étaient inscrits, de la situation économique de la famille, du capital social et culturel de cette dernière et de ses conditions de vie. dans la dernière partie de l'article, les auteurs soulignent l'importance du rôle de l'école pour garantir la possibilité d'apprendre aux enfants de milieux socioéconomiquement faibles. ils abordent en outre un certain nombre de conséquences qu'entraînent leurs constatations pour les politiques en matière d'éducation. el impacto del cierre escolar en la brecha de aprendizaje: divisiones familiares y escolares en tiempos de crisis -el cierre de escuelas en españa para hacer frente a los efectos de la covid-19 causó un enorme impacto tanto a nivel social como educativo. escuelas y familias tuvieron que reaccionar rápidamente a un nuevo escenario de enseñanza y aprendizaje sin contar con planificación previa o con directrices gubernamentales. en este contexto, algunas escuelas fueron capaces de adaptarse mejor a las nuevas circunstancias que otras. asimismo, la estructura y el tamaño del capital económico, social y cultural de las familias produjeron diferencias significativas en las oportunidades de aprendizaje de los niños y niñas de diferentes orígenes. este artículo evalúa el impacto del cierre de las escuelas en la brecha de aprendizaje entre el alumnado de diferentes orígenes sociales en cataluña. sobre la base de 35.419 respuestas a una encuesta en línea realizada entre el 26 y el 30 de marzo de 2020 a familias con hijos e hijas de entre 3 y 18 años, el análisis muestra que las oportunidades de aprendizaje varían significativamente. las familias de clase media pudieron mantener niveles más altos de calidad educativa en un contexto crítico, mientras que los niños de familias socialmente desfavorecidas tuvieron pocas oportunidades de aprendizaje, tanto en términos de tiempo como de experiencias de aprendizaje (tareas reducing physical contact has been the most common strategy adopted by governments to reduce the spread of covid-19. it has led most countries around the world to close their schools for periods of time. the lockdown of schools in spain, one day before the declaration of a state of alarm 1 on 14 march 2020, has had an enormous impact at both societal and educational levels. schools and families had to rapidly adjust to a new teaching and learning scenario without the benefit of previous planning or guidelines from the spanish ministry of education or regional departments of education. in this context, some schools were better able to adapt to the new circumstances than others. likewise, the structure and size of families' economic, social and cultural capital 2 (resources) produced significant differences in the learning opportunities for children from different socioeconomic backgrounds. while the covid-19 pandemic has no precedent in terms of effects on the economy and social life, schools have previously experienced periods of closure. for example, large-scale outbreaks other diseases (such as ebola and influenza), teacher strikes, natural disasters and violent conflicts have already forced schools to stop their activity in the past. researchers and international organisations have studied the effects of school closures on students' learning and found a measurable loss in the acquisition of basic skills, particularly for the most disadvantaged children (quinn et al. 2016; cattaneo et al. 2017) . it is highly likely that the current pandemic will have a dramatic long-term impact on students' competencies and increase existing education inequalities. in this article we reflect on how school closure produces unequal learning opportunities in terms of both formal and non-formal education 3 for children and youth from different socioeconomic backgrounds attending different types of schools in catalonia. during the covid-19 catalan lockdown, the instructional time received by students from different social backgrounds has been unequal, as has the educational value of the activities developed at home in non-school time. we investigated what learning opportunities were available to students from different backgrounds by administering an online survey during the second week of the state of alarm (i.e. between 26 and 30 march 2020). hypothesising that exposure to learning (both formal and non-formal) among children from lower-income households was likely to be lower than for their peers from higher-income families, we expected this to irremediably increase the existing learning gap between them. based on the results of our survey, this article reflects on how students' social background and conditions of confinement were associated with their learning opportunities. these unequal opportunities, in turn, are likely to exacerbate existing inequalities in skills acquisition and academic performance. we begin our article with a review of existing evidence on learning losses and their unequal impact on different groups of students due to past periods of school shutdown. the next section provides information on the context in which we conducted our own survey and the methodology we used to analyse the data. we then present our key findings regarding inequalities in formal and non-formal education, while the final section draws conclusions and discusses some policy implications of our findings. although it has only been a few months since schools in many countries around the world closed due to the covid-19 pandemic, research on the educational effects from that closure has already been notably rich. obviously, most of these studies have measured the short-term effects of the lockdown. it is certainly too early to know whether these effects can be considered a learning loss or just a temporary effect as a result of disuse that can be easily regained with practice (coe et al. 2020) . beyond this recent evidence, several previous studies based on student absenteeeism and past school closures have focused on the impacts of being out of school on learning outcomes (abadzi 2009; eef 2018) . these research studies can help us understand the potential effects of current school closures on learning and the mechanisms by which educational inequalities occur. some of the reasons causing schools to reduce or even stop their activity in the past range from summer school holidays, student or teacher absenteeism and teacher strikes to violent conflicts. extant research evidence based on these experiences identifies the existence of a general loss of learning connected to school absence, which can be more or less severe depending on students' previous performance, family characteristics, age, and education pathway, among other factors. school closure, even when distance (remote) learning is offered, usually implies a reduction in instructional time 4 and, by consequence, also a decline in learning time. 5 past studies have consistently identified a positive relationship between learning time and student achievement -along with other educational outcomes (abadzi 2009; hanushek 2015; scheerens 2014) . therefore, some studies have used instructional time as a proxy for "opportunities to learn". variables that measure exposure to curriculum seem to show a greater effect on learning than variables related to teacher behaviour or school climate. students' prolonged and repeated exposure to stimuli and feedback has been identified as a key educational resource. even though evidence is not always conclusive (cattaneo et al. 2017) , most research suggests that students with a migrant background, and those who are socioeconomically disadvantaged, will be slower learners. thus, these students are likely to benefit more from an increase in learning time than their peers (dahmann 2015; gromada and shewbridge 2016; huebener et al. 2017 ). consequently, not going to school reduces learning opportunities for all, but particularly for students from low-income backgrounds and less-skilled children. student (or teacher) absenteeism can also be understood as a lack of instructional time. student absenteeism results in poorer academic achievement, gaps in skills development, abilities and behaviours necessary for educational success, and an increased likelihood of school dropout. effects go beyond the school environment: absence has also been connected to risky behaviours such as smoking, juvenile delinquency, alcoholism, drug use, risky sexual behaviours or unwanted pregnancies, as well as other effects in adulthood (e.g. unemployment, alcoholism) which diminish life opportunities for young absentees (abadzi 2009; coe et al. 2020; gonzález motos 2020) . in spain, as in other european countries, the school lockdown between march and june 2020 seamlessly merged with the beginning of the summer holidays, resulting in students' absence from school lasting six months. past studies comparing students' learning progress during the school year with the summer break have consistently shown that achievement tends to slow or decline over the summer holidays. although there is no agreement on the magnitude of the learning loss during holidays (von hippel 2019), it is clear that it is steeper for mathematics than reading, and it is especially acute for upper-grade students . moreover, research points out that this loss does not affect all students equally; during the summer period, educational inequalities between socially advantaged and disadvantaged children increase (alegre 2016) . beyond the evidence related to breaks in the ordinary school year (e.g. absenteeism, summer break, instructional time), other research studies have analysed the effects on students' learning caused by external factors. for example, the literature on weather-related school closures (e.g. as a result of heavy snowfall or hurricanes) also contributes to a better understanding of the potential unexpected consequences of disruptive closures. most of the evidence is based on small samples, is geographically specific and refers to short break periods. however, in all cases, studies have found a clear effect of each day of school cancellation on achievement, especially among students attending less-resourced schools or coming from low-income families (goodman 2014; marcotte and hemelt 2008) . teacher strikes have also forced a sudden cancellation of classes. strikes affect student learning outcomes (mainly in terms of achievement and grade repetition) by reducing the time that children attend school, but they also affect the quality of schooling and increase the likelihood that students may engage in risky behaviour (jaume and willén 2019). in addition, research has found heterogeneous (uneven) effects, with children from vulnerable families being most affected by school disruptions (jaume and willén 2019; belot and webbink 2010). the main difference between school closures examined by previous studies and the current lockdown is that this time schools have managed to maintain some contact with students, mostly through online instruction. however, despite the efforts to support remote learning, many students have not had access to it. existing data about the digital gap and surveys conducted during the lockdown indicate that teachers have been unable to contact a significant number of students, mainly because of lack of an internet connection or adequate devices to engage in distance learning ; van lancker and parolin 2020). even when students have been able to connect, remote learning during school closures seems to have widened the existing attainment gap between students from different socioeconomic backgrounds (coe et al. 2020) . for instance, a recent study undertaken by researchers from brown and harvard universities into the use of an online mathematics programme (zearn) before and during lockdown in the united states (chetty et al. 2020 ) shows a sharp decline of student progress in mathematics in classrooms located in low-income zip (postal) code areas, while in high-income zip code areass no changes were observed (goldstein 2020) . before the covid-19 pandemic, in-school learning had already been proven to be more effective than distance learning. furthermore, student outcomes resulting from online learning have been shown to be poorer, on average, than outcomes resulting from face-to-face instruction (heppen et al. 2017) . conditions for effective remote learning (good internet connection, and clear explanations, scaffolding and feedback from teachers) are not easy to accomplish. the combination of a digital gap with teacher inexperience in providing high-quality distance learning makes it difficult to improve students' learning opportunities . moreover, there is now a significant risk that vulnerable students have less access to quality teaching than their peers, widening the attainment gap due to the school lockdown (coe et al. 2020; kay et al. 2020) . if it has already been proven that students from low-income families experience more interruptions and disruptions of their instructional time under normal school conditions (abadzi 2009; alegre and benito 2012) , then poor distance learning can be understood as a new obstacle to effective learning. apart from school organisation or distance learning strategies developed by teachers, parental engagement in children's learning, practised in some families alongside school attendance, becomes more important when school is replaced by at-home instruction. educational studies have demonstrated that parental involvement and the quality of learning at home improve academic outcomes. research in this field has consistently observed greater academic achievement among students whose parents are actively involved in the educational process. it has also found a clear relationship between engagement in the learning process and parental background in terms of social class and ethnicity (oreopoulos et al. 2006) . unequal parental capacities to help children with their homework and different uses of family time have been extensively documented (meyer et al. 2017; mora and escardíbul 2018) . therefore, family reactions to school lockdown are likely to produce significant differences in the learning opportunities for children from different types of social background (burgess and sievertsen 2020) . regardless of their causes and mechanisms, previous crises have had a more intense and negative effect on student learning outcomes in contexts with higher proportions of disadvantaged families (borse et al. 2011; chen et al. 2011; iqbal et al. 2020; sadique et al. 2008; shores and steinberg 2018) . previous research also suggests that the learning loss can not only have a limited short-term effect, but may in fact result in cumulative losses (abadzi 2009 ). likewise, other researchers suggest that the current school lockdowns will widen the learning gap between vulnerable children and their peers, possibly even reversing the progress made during the last decade to narrow this gap (coe et al. 2020 ; van lancker and parolin 2020). on 13 march 2020, the catalan government ordered the closure of all schools and colleges (universities) due to the covid-19 pandemic. more than one million students in compulsory education 6 were asked to stay at home. during the first two weeks of confinement, the catalan department of education advised schools not to provide new curriculum content, in an attempt to contain the adverse effects of the technological gap on educational inequalities. despite the department of education's order, some schools reacted rapidly to the new circumstances and switched to remote learning strategies. other schools stopped their activities entirely. between these two extremes, there were a range of reactions and responses. between 26 and 30 march, after two weeks of school closure and before the spanish government ordered a more restricted lockdown, we (the authors of this article) conducted an online survey with families whose children were aged between 3 and 18. the survey was addressed to families with children enrolled in preschool (3-5 years old), primary school (6-12 years old), lower secondary education (12-16 years old) and post-secondary academic or vocational education (16-18 years old). 7 organised into two main sets of questions, our survey included 78 items, with a variable number of subquestions that branched out to adapt to respondents' particular contexts. the first group of items characterised the different social, spatial and technological conditions in which families had to respond to the school lockdown (such as the size of their home, the availability of outdoor spaces, access to the internet and to digital devices). the second set of questions was designed to assess children's learning opportunities in (a) formal education (contact with school teachers, frequency and types of school tasks, feedback provided by schools), (b) informal education (uses of time, family activities, support to carry out school tasks) and (c) non-formal education practices from home (participation in after-school activities, types of activities and forms of participation). in addition, our survey included variables to characterise respondents' social and economic conditions, such as family structure, parental education attainment, income, ethnic origin, gender and work status. to test the survey, we conducted a brief pilot. due to the exceptional circumstances, the pilot was implemented using an informal strategy: we asked 10 families with children of different ages and on different school pathways to complete the survey. after the pilot was completed, we distributed the final version of the survey through various online communication channels (twitter, facebook and whats app). this strategy helped us to improve the survey's visibility among different social groups as well as reducing sampling bias linked to the use of social networks (blank 2017) . we also mobilised strategic contacts in the most deprived areas of catalonia to reach those families less likely to answer online surveys. in total, 79,668 people clicked the link to the survey, approximately 58,000 started to answer it and more than 40,000 families completed it. after excluding some respondents for reasons like incomplete answers, children's ages outside our survey's scope, families not living in catalonia, the final sample included information from 35,419 families (with a total of 59,167 children aged between 3 and 18). as expected, high-income and highly educated parents were over-represented. due to the lockdown and the urgent need for data collection, we were unable to complement our online survey with other strategies such as telephone calls or face-toface interviews. we opted instead to use non-response weighting to bring the sample closer to the population distribution. to do this, we weighted the sample by the level of parental education attainment, as a key variable of social and cultural differentiation, and as a reliable and accurate source of census data. table 1 shows both the weighted and unweighted samples. of our survey respondents, 86% were female, 89.3% lived in heterosexual biparental households, 9.6% lived in single-parent households and 1.1% were part of a same-sex couple. more than half of the households (53%) were composed of four members, three-member households represented 27% of the sample, and 16% of the households had five or more members. sample distribution according to school variables accurately reflects the distribution between the public and private school sectors. 8 in catalonia, the proportion of children in compulsory schooling who attend public schools is 68% (this was 69.9% in our sample), while 29% of students attend private subsidised schools (27.5% in our sample), and only 2% are enrolled in private independent schools (2.6% in our sample). our sample was also well balanced regarding children's ages, with a slight over-representation of younger students. 9 for our data analysis, we split information across three databases according to different units of analysis. first, a "household database" included information about family members living together during the confinement. second, a "student database" collected information about the learning activities of children aged between 3 and 18. third, the "after-school activities database" contained information about the extracurricular activities children were involved in before and after the lockdown. we also constructed some subsamples to carry out specific analyses. in this article, we present some data based on subsamples by children's age or educational level. in this section, we present and discuss the data from our survey regarding social differences in the learning conditions of catalan children and the impact of these conditions on learning opportunities during lockdown. inequalities in families' economic, social and cultural capital impacted on student learning opportunities by different means, including school responses to the lockdown, access to digital facilities and the level of parental learning support. this section reviews how these factors are drivers of inequalities in the three domains of learning: formal, non-formal and informal. learning conditions at home differ on the basis of a number of variables. for example, the amount of physical space and access to technological devices both have an impact on the learning conditions of children. our survey included questions about the size of the household and the outdoor spaces available, to serve as indicators of the physical conditions of confinement. the responses showed that most homes of confined families had a balcony or other outdoor space/s. however, there were significant differences among residents depending on the size of the municipality (families living in cities had less access to outdoor spaces) and other social indicators, such as parental education attainment. for instance, 13% of families with an adult who had completed compulsory education did not have outdoor spaces, while this was only the case for 4% of households in which at least one adult held a university degree. likewise, 45% of households in the poorest income quintile (q1) 10 had less than 80 square metres of space, reducing to 14% in the case of the richest income quintile (q5). students also had different internet connectivity conditions and unequal opportunities to access technological devices to carry out their schoolwork. since we could only implement an online survey, we were unable to grasp the full extent of the actual digital divide. data from the catalan department of education estimate that there are currently approximately 55,000 school children without access to an internet connection (vallespín 2020) , which would situate the digital gap at between 10% and 15% of all children in the catalan education system. our survey revealed that 3.5% of our sample had only a mobile phone internet connection or no connection at all. most families in our survey had access to a high-speed internet connection. however, there were important differences regarding the "possibilities of use" of technology. for example, 15.3% of respondents declared that they had access to a single device at home or only had access to a mobile phone. we calculated the number of devices per person and found that in 56% of cases there was less than one device per person. the digital divide can also be expressed in terms of access to devices based on the composition of the household. in our survey, 12% of households with two children had only one device available; 32% of households with three children had two or fewer devices available; and 48% of households with three children had three or fewer devices available. access to technology depends on the level of family income. while 25% of surveyed families in q1 (the lowest income quintile) had access to only one digital device, for families in q5 (the highest quintile), the corresponding figure was only 4%. furthermore, while 20% of families in q1 had access to four or more devices, this applied to 54% of families in q5. taking into account the size of the household, 71% of families in q1 did not have access to one device per person. among families in q5, the number of households with less than one device per person reduced to 37%. figure 1 shows the proportion of households for each income quintile that had access to one or more device per person. differences are sorted by income, with an inequality factor of 2.5 between q5 and q1. in addition, the unequal impact of the crisis on parents' working situation also altered the social and psychological conditions that ensured an adequate learning process. our survey revealed that before lockdown, 13% of adults were unemployed, while 80% were working full-time. when we asked about the impact of the covid-19 crisis on their working situation, these figures changed dramatically: 23% of respondents indicated that they had lost their job. of those who were still working, only 21.5% were able to go to work "normally". the rest (39%) were working from home, either with the same schedule or with a more flexible schedule. of those still working, 17% considered it likely that they would lose their job. of those who were already or became unemployed, 50% knew that they were entitled to unemployment benefits, while the other 50% either knew that they were not entitled or did not know. our survey included questions regarding how much time children invested in schoolwork every day since the beginning of the school lockdown, how frequent the contact was with their school and teacher/s, how often they received online teaching lessons, whether they had to complete specific homework tasks, and how often these tasks were reviewed and returned to children. taking into account the intensity of all these tasks, we composed an index of opportunities to learn (otl). to compose the index, which ranged from 0 to 100 points, we normalised indicators and aggregated (combined) frequencies. we only used this index for a subsample of the older children aged between 10 and 18, since assigned school tasks or online teaching were more unlikely for younger age groups. in terms of the index, 28.3% of students had an otl equal to 0, meaning that they dedicated less than one hour a day to school tasks, had almost no communication with teachers and did not have homework to do or to be reviewed. at the other end of the index, 7.7% of students had a maximum otl of 100, meaning that they dedicated more than four hours per day to schoolwork, had frequent contact with their teachers and received regular feedback for school assignments. the majority (80.2%) of all surveyed students in this subsample had an otl score below 60 points. interestingly, we found the otl score to be positively related to a number of variables. having greater access to digital devices, being enrolled in more advanced courses (older students had a higher otl), being native to spain or living in a higher-income household were all factors associated with higher otl scores. students enrolled in private schools, both independent and private subsidised ones, had significantly higher otl scores than those enrolled in public schools. there are explanations for this difference. for example, the catalan department of education announced that the first two weeks after the approval of the state of alarm would be a non-school period. therefore, a significant number figure 2 opportunities to learn (otl) index, by school sector and educational level note: baccalaureate refers to two years of optional upper secondary education preparing students for tertiary level of public schools did not develop school tasks during these two weeks, waiting for new instructions from the department. despite this announcement, private subsidised and independent schools did not stop their teaching activity. one of the plausible explanations for this difference lies in the economic dependency of private schools on fees. they needed to keep providing a service to users despite the exceptional circumstances. figure 2 shows the distribution of the otl index across school sectors for different educational levels. the bar chart reveals strong differences by school sector for students enrolled in the last years of primary education and for those in lower secondary education. the chart also shows how students in vocational education and training (vet) had the lowest otl of post-compulsory education. we also found the otl index to be clearly related to the level of parental education attainment and to family income. for instance, 49% of children in families from the richest quintile (q5) had an otl score of 80 points or higher. this reduced to 33% in the case of the poorest quintile (q1). the absence of schooling increases the importance of families as teacher substitutes in the learning process. our survey included questions regarding whether adults in the family helped students in their school tasks during the relevant lockdown period. an initial remarkable result appeared in terms of gender: while 79% of female adults stated that they supported their children to do schoolwork, only 43% of male adults did. as expected, gender differences were clearly observed, and family support was higher in the case of younger children. figure 3 shows that for children undertaking primary education, the support of mothers who had completed compulsory figure 3 adult support for school tasks, by gender and highest parental education attainment education was comparable to that of mothers with higher educational (in bourdieu's terms) cultural capital. however, for students enrolled in lower secondary education, the differences increased dramatically: only 35% of mothers who had completed compulsory education helped with homework, while 48% of the most educated mothers did. providing support for school tasks to students in post-compulsory education declined to close to 20% for all groups. there are several reasons why support for schoolwork may not have been provided by adults in a student's household. these reasons are remarkably different depending on the level of parental education attainment. in those households in our survey with children enrolled in lower secondary education, 92% of the families with an adult who had a university degree and who did not provide support for schoolwork argued that the child did not need it. this reason was only argued by 69% of respondents from those households with adults who had completed only compulsory education. in this case, a lack of knowledge was proffered as a significant reason for not giving support (by 28% of respondents), which reduced to 2% for those parents with a university degree. differences in cultural capital are therefore reflected in the capacity and possibilities of families to help children with their school tasks. in addition, (and not just in times of school closure) families' cultural capital and everyday informal practices have effects on children's learning experiences and opportunities. interestingly, confinement, which maximises the interactive time between family members, provides ideal research conditions for assessing informal learning activities. figure 4 reveals that in those households with children aged between 3 and 8, there were three activities in which social differences were especially acute. first, accompanying children in reading was more frequent by far in families with high cultural capital: 59% of families with an adult who had completed university studies responded that their children aged between 3 and 8 read with an adult every day. this reduced to only 37% in the case of families with an adult who had completed compulsory education. second, differences were also visible in foreign language informal practices: 44% of families with an adult who had completed university studies responded that foreign language practice was done daily or several days a week, while this practice was carried out by only 36% of families with an adult who had completed compulsory education. third, sporting activities were practised several times a week by 81% of families with university degrees and only by 66% of families with compulsory education. by contrast, other activities were more frequent among families with lower cultural capital. the most relevant was playing video games: 37% of families with an adult who had completed compulsory education reported that their children of preschool age played video games every day or several times a week. this applied to only 21.5% of families with parents who had completed university studies. the differences were also significant in the frequency of watching educational television programmes (52% of households with parents who had completed compulsory education and who had children enrolled in preschool education reported that their children did so every day, versus 30% of parents with a university degree), and in following and carrying out educational activities available via social media. in such an extreme situation of school absence, it appears that families with less cultural capital made more use of external resources to support their children's learning activities, while families with more cultural capital were more confident in their own abilities to respond to their children's learning needs. in the case of older children, the same differences were observed, although the gap increased in musical and other artistic activities and was less pronounced in playing video games. participation in after-school activities is a source of differential learning opportunities for children from different socioeconomic backgrounds (bradley and conway 2016; gonzález motos 2016; lauer et al. 2003; potter and morris 2017) . our survey compared whether children participated in one or more after-school activities before and after the beginning of the lockdown. we found that the effects of covid-19 on these kinds of activities have also increased the gap between socially advantaged and disadvantaged children. before lockdown, after-school activities were more frequent for students enrolled in primary education. this was particularly the case for children whose parent/s had completed a university degree (66%), compared to children from households with parent/s who had completed compulsory education (43%). this polarisation increased particularly for children enrolled in lower secondary education with at least one unversity-educated parent (81%) versus those whose parent/s had completed compulsory education (45%). we also found families' economic, social and cultural capital to be associated with the number of after-school activities in which children participated. for example, for families with two children in lower secondary education, the average number of after-school activities for highly educated households was 3.3, compared to 1.5 for households with lower educational levels. sports, foreign languages and artistic practices were the most frequently reported after-school activities. by income level, students in families in the lowest quintile (q1) were more represented in sports and compensatory education activities, 11 while children from the richest quintiles were over-represented in foreign language and musical after-school activities. as expected, we found that most after-school activities (70%) were interrupted during lockdown. however, economic and cultural inequalities were also visible in the probability of maintaining after-school activities despite the confinement. figure 5 shows that children whose parent/s had lower educational levels already had lower participation in after-school activities before lockdown and were more likely to be unable to continue them after schools closed than children from families with higher educational levels. there are two main reasons for this difference in after-school activities after the closure of schools. first, families with higher economic, social and cultural capital participated in activities that were more likely to continue online (e.g artistic activities, foreign languages), compared to the activities more commonly practised by families with lower capital (e.g. sports). second, voluntary interruption of afterschool activities was also higher among families with lower ecoomic, social and cultural capital, due to difficulties in coping with the costs during times of crisis. for instance, compensatory education activities -which could be undertaken online -were voluntarily interrupted by 25% of families who had previously engaged in figure 5 after-school activities before and after lockdown, by parental education attainment 11 compensatory education, in this context, refers to those after-school classes that some students attend to reinforce their learning competencies. these classes are usually offered to students with marked learning difficulties. them. this voluntary interruption was particularly high among children whose parent/s attained compulsory education (80%), compared to those whose parent/s had a university degree (62%). despite schools' efforts to maintain learning activities during lockdown, our analysis reflects significant inequalities in exposure to school learning depending on family characteristics (income, level of educational attainment) and school characteristics (educational level, school sector). the absence of schooling neutralises the benefits of socialisation provided by early childhood education for the most vulnerable children. it also disrupts those processes of guidance and accompaniment which are especially important for adolescent students in their study, work and life transitions. in addition, the digital divide and visible differences in access to technological devices among students have left some children and young people without options to connect to learning for at least three months or, more probably, for six months. inequalities in being able to maintain school-based learning are not the only existing ones. our analysis shows that families with a lower level of parental education attainment have fewer resources and less knowledge to help their children with school tasks. these limitations become more acute when support from the school has been reduced and demands for autonomous work by students increase. likewise, family economic, social and cultural capital also influences the adoption of everyday informal learning practices that are more or less aligned with school logic, which readily increases differences in learning opportunities and familiarity with what is considered valid knowledge. after-school activities are also more likely to be maintained among children from wealthier families, which also increases the existing inequalities in this area. in sum, if the pre-covid-19 school system already had significant limitations in its ability to eliminate existing social inequalities, the closure of schools during this period of lockdown poses enormous challenges for developing effective policies to compensate for learning losses and learning inequalities. school lockdown has not affected all children in the same way, and significant material and human resources will be necessary to ensure that the most vulnerable children can catch up. an educational emergency plan with social and educational objectives will be needed to restore children's socio-emotional and cognitive skills. the individual and social costs of not intervening are just too high, unfair and unequal. it is the responsibility of states and countries to ensure the right to education in these difficult times. instructional time loss in developing countries: concepts, measurement, and implications are summer programs effective in improving learning and educational outcomes in students? barcelona: ivàlua-fundació jaume bofill climas y (sobre todo) culturas escolares: cómo se explican y qué permiten explicar familia y relaciones intergeneracionales: un espacio de oportunidades para la educación de los hijos e hijas [family and intergenerational relations: a space of opportunity for the education of children do teacher strikes harm educational attainment of students? the digital divide among twitter users and its implications for social research closing schools in response to the 2009 pandemic influenza a h1n1 virus in new york city: economic impact on households the forms of capital a dual step transfer model: sport and non-sport extracurricular activities and the enhancement of academic achievement schools, skills, and learning: the impact of covid-19 on education. vox, cepr policy portal the more, the better? the impact of instructional time on student performance social and economic impact of school closure resulting from pandemic influenza a/h1n1 how did covid-19 and stabilization policies ffect spending and employment? a new real-time economic tracker based on private sector data impact of school closures on the attainment gap: rapid evidence assessment. london: education endowment foundation (eef) how does education improve cognitive skills? instructional time versus timing of instruction. german socio-economic panel study (soep)paper series the attainment gap // 2017. london: education endowment foundation research shows students falling months behind during virus disruptions. the new york times what impact do after-school activities have on children's and youngsters' learning? what works in education? series are programmes to combat school absenteeism effective? what works in education? series flaking out: student absences and snow days as disruptions of instruction time. nber working paper series constitución española [spanish constitution student learning time: a literature review time in education: introduction the struggle to pass algebra: online vs. face-to-face credit recovery for at-risk urban students increased instruction hours and the widening gap in student performance scientific and cultural organization) (n.d.-a). instructional time. ibe-unesco glossary of curriculum terminology learning time. ibe-unesco glossary of curriculum terminology population figures [online resource we should avoid flattening the curve in education: possible scenarios for learning loss during the school lockdowns. education for global development the long-run effects of teacher strikes: evidence from argentina remote learning: rapid evidence assessment. london/cambridge: education endowment foundation (eef)/edtechhub. retrieved projecting the potential impacts of covid-19 school closures on academic achievement the covid-19 slide: what summer learning loss can tell us about the potential impact of school closures on student academic achievement the effectiveness of out-of-school-time strategies in assisting low-achieving students in reading and mathematics: a research synthesis unscheduled school closings and student performance patterns of literacy learning in german primary schools over the summer and the influence of home literacy practices home environment and parental involvement in homework during adolescence in catalonia (spain) the intergenerational effects of compulsory schooling family and schooling experiences in racial/ethnic academic achievement gaps: a cumulative perspective seasonal dynamics of academic achievement inequality by socioeconomic status and race/ethnicity: updating and extending past research with new national data estimating the costs of school closure for mitigating an influenza pandemic effectiveness of time investments in education. springerbriefs in education series the impact of the great recession on student achievement: evidence from population data educació detecta 55.000 famílies sense ordinador o sense connexió per poder seguir les classes a distància covid-19, school closures, and child poverty: a social crisis in the making is summer learning loss real? education next publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations he has been a member of the eu network of experts in social sciences and education (nesse) and is an editorial board member of several international journals on education policies and educational development. xavier has widely published in national and international journals and is the author of several books on the sociology of education, education policy and globalisation, and education and development. he has worked as a consultant for international organisations such as unesco, unicef, the european commission, and the council of europe she is a member of the institute of government and public policy (igop) and the globalisation, education and social policies (geps) research centre. her research interests include public policy analysis and social inequalities, with a special focus on education and migration. sheila has conducted several research projects in this field, and she has experience in policy analysis and policy evaluation the data that support the findings of this study are available from the corresponding author, upon reasonable request. authors' contributions: all authors contributed to the study conception and design. material preparation, data collection and analysis were performed by xavier bonal and sheila gonzález. all authors read and approved the final manuscript. key: cord-263549-hmdyb7hi authors: dewitt, dawn e. title: fighting covid-19: enabling graduating students to start internship early at their own medical school date: 2020-04-07 journal: ann intern med doi: 10.7326/m20-1262 sha: doc_id: 263549 cord_uid: hmdyb7hi this perspective proposes one way to ensure enough doctors, nurses, and pharmacists during the covid-19 pandemic: enable graduating students to start internship early at their own medical school. tems affiliated with their medical school. first, medical schools could point students toward existing "readiness for internship" content. many medical schools have a "capstone" course in the final weeks before graduation that prepares students for internship, with content geared toward meeting common patient care challenges. most schools could deliver much of that content online now or via specific programs (3) (4) (5) . in addition, some commonly required certificate courses, such as advanced cardiac life support and pediatric advanced life support, can be completed online in less than a week (6) . the united states could provide loan repayment or other federal payment programs for any senior students willing (and competent, as judged by their medical school) to begin early. the average u.s. medical student graduates with approximately $200 000 of debt (7), so generous repayment programs would be welcome-and fitting-with potentially less bureaucracy than officially hiring students through health care systems short term. an alternative would be to pay the students a stipend equal to a tuition rebate plus the equivalent of a resident's salary funded by medicare (8) . further, delays in transition to residency for this year's students due to chaos, credentialing, and other barriers might generate more problems for graduating students and short-staffed health care systems. we should urgently prepare these all-but-graduated students to help us address the looming workforce shortage as junior physicians during the next few weeks. however, they also should get credit for the experience they will gain and the service they will provide. the last big advantage of this plan is that the new junior interns would be working on home turf rather than adjusting to a different hospital or place, as happens for many interns who move across states, or across the country, to start in different health systems. starting at their home institutions would vastly decrease credentialing and barriers to electronic health record access. we would have to rapidly address financial and logistic issues. potential guarantees for loan repayment and tuition refunds would be key to success. health profession schools would have to signal which students have the competency to begin working with more independence and agree to supervision requirements similar to those for residents. supervision might be expanded to appropriate recently retired physicians or this article was published at annals.org on 7 april 2020. those whose health risks due to covid-19 make them unable to work on the front lines. health systems would need to authorize access so that competent students could write orders and access electronic medical records from home. graduate medical education (gme) leaders would need to discuss potentially giving participating students "credit" toward residency completion. these are bold but relatively straightforward requests, which i am certain academic medicine could tackle nationally in concert with gme leadership. breaking down bureaucratic barriers must be a priority-a national effort could save many thousands of lives, not to mention being a substantial uplift for exhausted health care providers. despite the logistic challenges, definitive and organized collective action now may give the united states an edge that we desperately need in this fight. italy rushes to promote new doctors to relieve coronavirus crisis accessed at www .ama-assn.org/delivering-care/public-health/covid-19-states-call-early -medical-school-grads-bolster-workforce on 5 accessed at www.wisemed .org/wise-oncall the key role of a transition course in preparing medical students for internship advanced cardiovascular life support (acls) course options house of representatives committee on small business. the doctor is out. rising student loan debt and the decline of the small medical practice medicare payments for graduate medical education: what every medical student, resident, and advisor needs to know. accessed at www.aamc.org/data -reports/faculty-institutions/report/medicare-payments-graduate -medical-education-what-every-medical-student-resident-and -advisor key: cord-352431-yu7kxnab authors: langbeheim, elon; perl, david; yerushalmi, edit title: science teachers’ attitudes towards computational modeling in the context of an inquiry-based learning module date: 2020-08-25 journal: j sci educ technol doi: 10.1007/s10956-020-09855-3 sha: doc_id: 352431 cord_uid: yu7kxnab this study focuses on science teachers’ first encounter with computational modeling in professional development workshops. it examines the factors shaping the teachers’ self-efficacy and attitudes towards integrating computational modeling within inquiry-based learning modules for 9th grade physics. the learning modules introduce phenomena, the analysis of measurement data, and offer a method for coordinating the experimental findings with a theory-based computational model. teachers’ attitudes and self-efficacy were studied using survey questions and workshop activity transcripts. as expected, prior experience in physics teaching was related to teachers’ self-efficacy in teaching physics in 9th grade. also, teachers’ prior experience with programming was strongly related to their self-efficacy regarding the programming component of model construction. surprisingly, the short interaction with computational modeling increased the group’s self-efficacy, and the average rating of understanding and enjoyment was similar among teachers with and without prior programming experience. qualitative data provides additional insights into teachers’ predispositions towards the integration of computational modeling into the physics teaching. electronic supplementary material: the online version of this article (10.1007/s10956-020-09855-3) contains supplementary material, which is available to authorized users. al harrison. "maybe we've been thinking about this all wrong" paul stafford, "how's that"? al -"maybe it's not new math at all" katherine johnson -"it could be old math, something that looks at the problem numerically and not theoreti-cally… math is always dependable" al -"for you it is." (leaving the room) katherine -"euler's method!" paul -"euler's method? but that's ancient!" katherine -"it is ancient, but it works…" (scene from theodore melfi's film "hidden figures," 2016). this excerpt describes the breakthrough of the nasa team working on the calculation of the re-entry coordinates for john glenn's first orbital flight outside the atmosphere. when faced a problem that cannot be solved analytically, katherine johnson, the woman who worked as the team "calculator," suggested to use the euler method-a step-by-step calculation for solving the differential equations of motion (develaki 2019) . today, the euler method is implemented in computer models that are investigated in introductory physics courses (e.g., chabay and sherwood 2008) and allows educators to expand the scope of problems that students can tackle. developing and using computational models reflects an important goal for k-12 science education, within a range of scientific practices such as designing and employing experimental investigations, analyzing measurement data and communicating them (nrc 2012) . indeed, there is growing evidence for doing classroom inquiry that entails a dual focus on experimental measurement and computational modeling in primary and secondary schools (e.g., farris et al. 2019; fuhrmann et al. 2018) . the extent to which teachers adopt inquiry-based science teaching practices may be related to their views of inquiry learning (lotter et al. 2007; osborne 2014; blanchard et al. 2009 ), their subject matter knowledge (tseng et al. 2013) and their teaching self-efficacy (lakshmanan et al. 2011 ). however, research on teachers' attitudes towards integration of computational models in the context of experimental investigations is scarce (gerard et al. 2011) , and thus, teachers' implementation of instruction that involves both experimental classroom inquiry and construction of computational models, remains an open field of research. in this paper, we describe science teachers' engagement with inquiry-based curricular modules for 9th grade physics that introduce computational models. the modules include a structured, inquiry-based experimental component, and a theoretical component in which computational models are used to verify the experimental observations. drawing on studies that address teachers' attitudes and self-efficacy as indicators for implementation of inquiry teaching, we ask how do teachers' prior experiences in teaching physics influence their self-efficacy and attitudes towards inquiry-based learning practices? specifically, in relation to computational modeling, we ask how does teachers' prior involvement with programming influence their self-efficacy in, and experience of computational modeling? coordinating theory and experimental evidence: a model-based perspective a scientific investigation integrates two endeavors: the formation or revision of predictive theoretical constructs and the design of experiments that produce reliable observations or measurements (klahr and dunbar 1988) . in studying the development of scientific practices among students, researchers can focus on practices of experimental design such as the selection and control of variables (chen and klahr 1999) or the evaluation of measured data (allie et al. 1998; kanari and millar 2004) , or they can study how students use and conceptualize theoretical ideas. in practice, the two activities are difficult to separate. specifically, unsubstantiated hypotheses, or "implicit theories" (kuhn 2011 ) have a strong influence on students' planning and interpretation of experiments. conversely, flawed experimental design, or misinterpretations of measurements can shape theoretical misconceptions. judging experimental results as implausible in light of learners' "naïve" theories is a crucial aspect of the theoryexperiment interaction (kuhn 2011) . in many introductory college and high school courses, the interaction between theory and experiment plays out in structured labs designed to "prove" or concretize theoretical statements. although the purpose of these labs is to substantiate the theory students learn in lecture courses, studies show that they do no contribute to students' content knowledge, as measured by test performance (holmes and wieman 2018) . moreover, students' perceptions of their own expertise in conducting experiments tend to decline during such structured laboratory courses (wilcox and lewandowski 2018) . an alternative is the inquiry or design lab that entails discovery experiments that extend the theoretical knowledge students acquired in prior learning. for example, in the investigative science learning environment (isle) curriculum (etkina et al. 2006a) , students design experiments to develop and then test model-based explanations of observed phenomena. this approach brings the design of experiments and the analysis of measurement data to the fore, and often challenges students' prior knowledge. to do so, an extended amount of time is allocated to judging the reliability of measurement data. educational research shows that in these transformed labs, there is no decline in students' perceptions of their expertise in conducting experiments as in the structured lab (wilcox and lewandowski 2018) . a specific case of coordinating theory and experiments is the construction and evaluation of models. models are simplified, and often abstracted quantitative representations of the system under investigation, constructed to describe, explain or predict the system's behavior (etkina et al. 2006b ). in introductory physics courses, models are usually instantiations of general, theoretical principles such as newton's laws, in generic systems such as orbital motion under the influence of a centripetal force (halloun and hestenes 1987) . recently, researchers who design upper-level laboratories, suggested a new modeling framework, in which model construction, evaluation and revision is utilized for two inter-connected modeling cycles: the measurement model cycle and the theoretical model cycle (zwickl et al. 2015; dounas-frazer and lewandowski 2018) . this framework provides two possible reactions to a misalignment of theory-based model and measurements: revising the measurement model-the instruments used or the interpretation of the measurement data, or, revising the theoretical model. while this framework has been suggested for upper-level undergraduate labs that often involve sophisticated measurement equipment, it is a useful construct for guiding scientific investigations at various levels. it suggests designing investigations with explicit awareness to both the judgment of measurement data, and to the limitations of the theory-based, physical system models. the measurement and/or the theoretical models can be revised to achieve better alignment of theory and the behavior of the actual physical system. while most examples of theoretical system models are mathematical/analytical (e.g., dounas-frazer and lewandowski 2018), these models can be also realized using computer simulations, as we will show bellow. jong et al. 2013) . in this approach, students manipulate parameters of the simulated system to learn about its mechanism. the simulations are introduced as substitutes of the real system and the interaction with them resembles a structured laboratory experiment. for example, students investigate a simulation of collisions and compare the velocities before and after the collision to deduce the law of conservation of momentum (de jong et al. 1999) . a different educational approach is to engage students in constructing the computational model (vanlehn 2013; wagh and wilensky 2018) . engaging in model construction requires learning environments that allow students to add or change model features. the students need to plan how to add or change certain aspects of the model, and then run the computational model and compare the outcome vis-à-vis other realizations of the theory or experimental results. for example, in a computational model of a falling paratrooper, students decide whether the magnitude of the drag force depends on the velocity or not. they then examine whether their definition of the drag force yields a velocity pattern that reaches a relatively low terminal velocity as expected (develaki 2017) . constructing computational models and especially accessing their code has been explored in secondary school physics (aiken et al. 2013; sherin et al. 1993; langbeheim et al. 2019; hutchins et al. 2020) , and in a few introductory college texts (redish and wilson 1993; chabay and sherwood 2008; orban et al. 2018) . however, there are only few examples of using simulations side by side with experiments. for example, farris et al. (2019) investigated how 4th graders learn the concepts of speed and acceleration by measuring the motion of toy cars and building models of motion in a computational environment. despite its basic level of content, this study shows that even young students are able to think about variations in measurement, and relate real-world observations to computational models. similarly, in fuhrmann et al. (2018) students studied the osmosis of sugar water through an egg membrane experimentally and then explored a computer, particle-based model that mimics the experimental results. they found that both exploring a ready-made model and designing a model in this approach foster the development of conceptual knowledge about the topic. however, only actively designing the model results in developing nuanced understanding of the role of models in scientific investigations. this finding emphasizes the epistemic importance of constructing models, compared to just exploring them. engaging students in building computational models reflects the actual practice of scientists (develaki 2017; winsberg 2010 ) and contributes to the development of creativity and computational thinking (hutchins et al. 2020; popat and starkey 2019) . however, letting students construct models, and especially when construction involves the understanding and manipulation of computer code, is a much greater leap for educators. in order to guide students in understanding and modifying code, teachers need to become proficient in programming themselves and to produce new learning materials for their science and mathematics classrooms. this requires teachers and students to transition from being users of computer programs to being builders of programs, which can be viewed as a substantial change in computing culture (ben-david kolikant and ben-ari 2008) or an educational paradigm shift (disessa 2001). implementing inquiry-based teaching that involves designing experiments, interpreting measurement data, and constructing models is challenging. in this study, we consider two main factors that determine the extent to which teachers implement such teaching in their classrooms: (1) teachers' self-efficacy in teaching the subject matter (e.g., lakshmanan et al. 2011); (2) teachers' attitudes regarding inquiry-based learning and their beliefs in its effectiveness (blanchard et al. 2009 ). self-efficacy is a set of beliefs regarding one's capabilities to perform well in a certain field (bandura 1997) . these beliefs are one of the main driving forces that motivate people to put effort and pursue challenging tasks. self-efficacy beliefs are dynamic, especially during learning of new skills, or beginnings of new professional endeavors. in preliminary acquisition of an expertise, there are several factors influencing self-efficacy, such as observing others demonstrating successful performances, and most importantly, one's own mastery experiences. mastery experiences are successful performances of tasks, such as conducting an effective lesson or lesson sequence. teaching self-efficacy determines teachers' beliefs about their competence and performance in teaching (tschannen-moran et al. 1998 ). teachers' years of experience in teaching are significantly correlated with their teaching self-efficacy (tschannen-moran and hoy 2007) but not with their tendency to implement inquiry-based teaching (marshall et al. 2009 ). although physics is considered a challenging topic, studies of self-efficacy of teaching physics are very rare (e.g., tanel 2013) . the second factor influencing implementation of inquiry-based teaching is teachers' beliefs regarding the importance and fruitfulness of this type of teaching. teachers' who were less inclined to believe that inquirybased teaching methods lead to effective student learning were less likely to implement them in their lessons (lakshmanan et al. 2011) . similarly, teachers who had meaningful views of the pedagogical approaches to inquiry, grounded in learning theories, were more likely to adopt inquiry-based learning practices in their classrooms (blanchard et al. 2009 ). reformed teaching pedagogies and curricula are usually introduced in professional development (pd) workshops, so that successful implementations can be related to the teachers' backgrounds and/or to the pd process. the structure, duration, and content of pds is an important factor influencing the implementation of reformed instruction and curricular innovations. a survey study showed that pds that focus on content knowledge and provide opportunities for active learning have the strongest positive effect on teachers' self-reported increase in knowledge and skills (garet el at. 2001) . moreover, pds that include mastery experiences have the strongest effect on teachers' self-efficacy (tschannen-moran and mcmaster 2009) and consequently on classroom implementation (penuel et al. 2007 ). there are only few examples of teacher training programs that include computer programming (e.g., repenning et al. 2020) . such courses are necessary for boosting their selfefficacy in computational modeling and knowledge of computational thinking (papadakis and kalogiannakis 2019). a growing number of studies indicate that elementary and secondary school students are able to construct computational models and to modify their code in the context of physics (e.g., aiken et al. 2013; langbeheim et al. 2019; farris et al. 2019 ), but less is known about the views of in-service science teachers regarding adopting computational modeling that involves manipulation of computer code. therefore, the goal of this study is to examine science teachers' attitudes towards introducing computational model construction in the context of inquiry-based learning in physics. our main research questions are: 1. how do teachers' prior experiences in teaching physics influence their self-efficacy and attitudes towards inquirybased learning practices in a pd workshop? 2. how does teachers' prior involvement with programming influence their self-efficacy in, and experience of computational modeling that involves coding in a pd workshop? we investigated these questions in the context of workshops that introduced two curricular modules for 9th grade physics: the focus of the first is the oscillations of a springmass system, and the focus of the second is free-fall with airdrag. each module begins with experimental investigations of motion patterns, and concludes with a theoretical section that compares the experimental measurements to the output of computational models. the experimental section of both modules comprised six 90min lessons that are summarized in table 1 . the lessons are based on the following design guidelines: a) capturing student attention-the main purpose of initial lessons is to raise attention to aspects of the phenomenon through problematization (reiser 2004; phillips et al. 2017 ). in the first module, the theoretical exploration focuses on the "peculiar" lack of covariance of the period of the oscillating spring system and its amplitude. in the second, it focuses on the apparent discrepancy between the aristotelian view (that mass influences the falling rate of objects) and the galilean view (that it does not). b) scaffolding measurement and data analysis practices-the measurement and data analysis methods are introduced in a guided manner in the initial lessons. in subsequent lessons, the teacher removes some of the direct instructions: first in asking students to plan and justify an experimental design to answer a common research question (e.g., how does the falling rate of paper cups change when increasing its mass?) and then in planning and justifying an experimental design to study a question that students generate by themselves (see table 1 for examples) c) raising student accountability-students build an evaluation rubric for experimental investigations based on fabricated student reports, in order to develop awareness for reporting standards. then, they share their evaluation criteria to produce a unified rubric. finally, they use the consensus evaluation rubric to provide feedback on the presentations of their peers. after completing the experimental section, and sharing their inquiry projects using posters or presentations, the students proceed to study a theoretical model of the phenomenon. the theoretical section includes a qualitative, conceptual section and the quantitative computational section. the qualitative, conceptual section comprises two lessons in which students construct a paper and pencil theoretical model that explains the phenomenon that they investigated experimentally. for example, in the oscillating spring-mass unit, students analyze the trace of the oscillating mass, using motion patterns produced by the tracker video analysis software (brown and cox 2009 ). their analysis leads to the discovery that the maximal speed of the mass corresponds to the amplitude, or the distance covered by the mass during one period of oscillation. this finding is used to hypothesize qualitatively that the length of the motion path and the average speed of the mass cancel each other out, and therefore, their ratio-the period of one oscillation-is constant. the purpose of the final unit in the module is to introduce a quantitative approach for comparing the measurements and the theory-based model. the analytical equations of motion of both processes we discussed-harmonic oscillations and falling with a drag force-are nonlinear and involve mathematics beyond that of 9th grade. in order to overcome the mathematical complexity, we introduced the theoretical model using a computer program that calculates the trajectory using the euler method (cromer 1981; develaki 2019) . the computational models of motion with air drag and of the oscillating spring-mass systems were realized using trinket.ioa free, online tool for building coding activities and courses. we chose this platform, since it runs vpython-a 3d graphical package for python (scherer et al. 2000) . python was chosen since it replaced introductory level programming languages that were used in the past such as pascal, basic, and fortran. figure 1 illustrates the trinket interface that has the code on the left side and the graphic output on the right. the unit introduced all the necessary programming ideas so that students (and teachers in the pd workshop) were not expected to have prior experience with vpython. the supplementary material provides detailed information on the introduction of the computational model. the euler method presents the theoretically predicted motion pattern using three linear equations, calculated repeatedly every time step. the first equation calculates the net force according to the current location or velocity of the object; the second calculates the new velocity of the object, based on newton's 2nd law; and the third updates the location of the object based on the new velocity. for example, the implementation of these three equations for the falling with air drag problem is shown in lines 14-16 of the code shown in fig. 1 . in addition to using relatively simple linear equations, this approach promotes the analysis of the dynamics of specific moments during objects' motion patterns, thereby facilitating the comprehension of the process as a whole (sherin et al. 1993; sherin 2001) . the two modules were introduced in two summer pd workshops that span four full days and a total of 30 hours of contact. the workshops included an overview of the principles of inquiry-based learning, engaging in measurement cycles and in coordinating the experimental results with the computational model. the workshops were advertised as introducing inquiry for 9th grade physics, and did not as present themselves as focusing on computational modeling. the first 2 days of the pd focused on experimental investigations, and the last 2 days focused on the related theory and computational modeling. teachers worked in small groups and shared their experiences in discussions with the rest of the participants and the workshop instructors. of 49 participating teachers, 41 completed all of the activities. within this group, 26 teachers are considered "experienced teachers"-teachers who taught physics at the 9th grade level for at least 3 years, or at the high school level for at least 1 year (high-school teachers usually come with a bachelor in physics, and teach more physics in 1 year than 9th grade science teachers do in 3 years). fifteen teachers were considered "novice teachers," if they taught physics in 9th grade for less than 3 years. three years is a common cutoff for differentiating experienced and novice teachers in educational research (e.g., tschannen-moran and hoy 2007). this does the mass of an object influence its falling rate? the dispute between aristotle and galileo 2. initial problematization of measurement how to measure the period of oscillation of a mass on a spring when the motion fades? why does the mass of paper cups influence their rate of falling whereas the mass of clay balls does not? 3. joint investigations: drawing conclusions despite measurement uncertainties students systematically investigate the influence of amplitude on the period of oscillation of the mass-spring system students systematically investigate the differences between falling rates of paper cups, while increasing their mass, and discover that the differences decrease with mass 4. developing an evaluation rubric students were given an empty evaluation rubric and three fictitious student reports. they suggested categories, and differentiated between performances on each category. students chose a research question related to the period of oscillation of a spring mass system (e.g., what happens when increasing the suspended mass), suggested hypotheses and built an experimental setup to test their hypotheses. students chose a research question related to falling motion (e.g., how does the diameter of a balloon influence its falling rate), suggested hypotheses and built an experimental setup to test their hypotheses 6. peer evaluation students present their posters/presentations to their peers, and peers provide written or oral feedback based on the evaluation rubrics. group consisted of new teachers, and a few experienced teachers in other fields such as math, who were new to teaching physics topics. the first research question asked how do teachers' prior experiences in teaching physics influence their self-efficacy and attitudes towards inquiry-based learning practices in a pd workshop. to examine this question, we used the following data sources: 1a. physics teaching self-efficacy survey-before the workshop, we administered a self-efficacy survey with six likert-scale items of the same format: "i am confident in teaching quantitative problem solving/ introducing computational models/ textbook experiments, in 9 th grade physics" on a scale of 1 to 5 (5 = strongly agree, 4 = agree, etc.) and a background survey indicating years of experience in teaching physics and experience in computational modeling. the internal consistency of the selfefficacy scale was high-cronbach α = 0.85. 1b. workshop appreciation survey-at the end of the workshop, we administered a feedback survey with six five-point likert-scale items such as "the workshop contributed to my ability to explain fundamental concepts such as newton's 1st law/ textbook experiments etc." (strongly agree, agree, etc.) and open-ended items asking to state aspects of the workshop that they found significant and aspects that were lacking. 1c. attitudes towards physics lab goals-we administered a 15-item survey to evaluate teachers' beliefs about the goals their students should achieve in the physics lab. some of the items were adapted from the e-class -colorado learning attitudes science survey for experimental physics, (wilcox and lewandowski 2018) . the e-class includes items such as "the primary purpose of doing physics experiments is to confirm previously known results (strongly agree, agree etc.)". our adapted version was: "rate the following statements representing goals for students to achieve in the physics lab: to confirm the theory discussed in class." this statement reflects a common goal for the structured lab, while statements such as "to acquire tools for evaluating the work of peers" represents a goal of an inquiry-based lab. the survey utilized a 4-level likert scale. we used exploratory factor analysis to aggregate fifteen statements into two categories: (a) goals of inquiry-based laboratory practices, (b) goals of structured laboratory practices. the full list of items, the corresponding categories and internal consistency scores are shown in table 2. 2. in order to investigate the 2nd research question regarding the influence of teachers' prior involvement with programming on their self-efficacy in, and experience of computational modeling that involves coding in a pd workshop, we used the following data sources: a. programming self-efficacy-before and after the computational modeling activity we administered a self-efficacy survey with three five-point likert-scale items such as: "i think i can understand / write and modify computer code" (strongly agree, agree ,etc.). thirty-eight teachers responded to the pre and post self-efficacy items related specifically to the computational modeling activity. the internal consistency of the self-efficacy scale was substantial-cronbach α = 0.76, but the distribution of the ratings in the posttest was not normally distributed (shapiro-wilk w = 0.94, p = 0.03) fig. 1 vpython code of the computational model (left), and graphical interface (right). the code includes two definitions of graphical objects: the "box" (line 9) which imports the motion diagram of the real object (marked in red dots) and a green sphere (defined as "s" in line 10) which moves according to a computational model. the green trace of the sphere is compared to the red pattern of dots that was produced from a video of a falling paper clip using the tracker software b. programming activity appreciation-three statements addressed specifically the coding activity "the coding activity was interesting," "the coding activity improved my understanding of the physics of the spring-mass system / free-fall with drag," "i understood everything i did in the coding activity" (strongly agree, agree, etc.). the internal consistency of the activity appreciation scale was substantial-cronbach α = 0.74. the full survey can be found in the online supplement. in addition, we used transcripts from the activity itself to corroborate our survey findings. the self-efficacy related to teaching physics in 9th grade was relatively high before the workshop with an average rating of 3.91 out of 5. the average ratings of the novice teachers with less than 3 years of teaching was 3.566, and the experienced teachers average rating was 4.10. this difference is significant (t = − 2.56, p = 0.017). the characterization of teachers' perceptions of the contribution of the workshop was based on two forms of data: likertscale ratings and open-ended questions. the average ratings were calculated and analyzed according on the teachers' experience as shown in table 3 . both groups rated the contribution of computational modeling (item 6) and designing the experimental setup (item 3) higher than the rest of the topics. the contributions to two common aspects of teaching-conducting standard textbook experiments (item 2) and clarifying fundamental physics concepts (item 5)-were rated higher by the novice teachers, and these differences were marginally significant. in the other categories, the differences between the experienced and novice teachers were very small. also, an index measure, combining all of the items, reveals no significant difference among groups (t = − 0.146, p = 0.88) and between workshops (t = − 1.21, p = 0.23). the responses to the open question, corroborate these findings. for example, in their suggestions for two or more aspects of the workshop that contributed to their teaching, vera, an experienced teacher wrote: the inquiry-module for ninth grade, structured and original, will greatly facilitate my work next year. i learned to use excel for analyzing an experimental graph and fitting it to an equation. the main contributing aspects were the parallel investigations, the discussions about classroom inquiry, its meaning and its implementation. also, the acquaintance with the step-by-step method using the software (although i'm not sure i would implement it in class). table 2 the grouping of the items regarding the goals of the physics lab "rate the importance of the following statements representing goals for students to achieve in the physics lab" cronbach's α (pretest) cronbach's α (posttest) (a) inquiry-based laboratory goals "an active, hands-on acquaintance with physical systems" 0.82 0.79 "acquire tools for evaluating the work of peers" "acquire tools to improve group work" "to present and report findings to peers (using presentations/posters)" "to identify sources of measurement error" "to make inferences from experimental findings" "to learn to improve the experimental setup to increase measurement accuracy" "to design an experimental setup to investigate the quantitative relation between two variables" (b) structured laboratory goals "to illustrate and concretize the theory discussed in class" 0.64 0.81 "to learn to write reports of experimental findings" "to confirm the theory discussed in class" "to learn methods for analyzing and presenting data" "to understand the role of experiments in building theories" "to work with the safety regulations related to equipment and materials in the lab" "to search and find information sources" (relevant to the experiment) both teachers mentioned the structured inquiry sequence as a major contribution to their teaching. in relation to the computational tools, vera mentioned excel which was used to analyze the experimental data, and ana mentioned step-by-step computational modeling. overall, teachers mentioned five main components of the pd: the two most common topics, both mentioned by 54% of the teachers, were the structured inquiry sequence and/or the computational modeling activity. in addition, 41% mentioned the measurement error analysis and/or the active learning format of the workshop. finally, 24% of the teachers mentioned the contribution of the workshop to their understanding of fundamental physics concepts. the differences in attitudes towards laboratory experiments before and after the pd were relatively minor as shown in table 4 . the teachers' rating of the importance of lab purposes associated with traditional structured lab (category b.) decreased during the workshop, and goals related to inquiry practices have slightly increased. however, a closer examination shows that these differences were not the same across groups. the ratings of inquiry lab goals among novice teachers increased, whereas the experienced teachers' ratings slightly decreased. this difference is marginally significant (p < 0.1). the average rating of programming self-efficacy before the activity was 3.11 (sd = 1.01), and at the end of the workshop, it increased to 3.65 (sd = 0.93). a wilcoxon signed rank test shows that the difference between pretest and posttest is significant (w = 930, p = 0.030). fifteen teachers stated that they had prior knowledge in programming, and 23 responded they did not. we found significant differences in self-efficacy both pre and post the activity, between teachers that declared to have prior knowledge in programming and teachers who did not. teachers with prior programming knowledge rated their self-efficacy significantly higher than teachers without such knowledge, both in the pretest (t = − 3.49, p = 0.0013) and in the posttest (w = 80, p = 0.0055). a wilcoxon test shows that appreciation of the coding activity was similar among teachers with (m = 3.93, sd = 0.55) and without (m = 3.46, sd = 1.08) prior experience in programming (p = 0.239). we note that the standard deviation among teachers with no background in programming was larger than among teachers with programming background. this variance is also evident in the boxplot in fig. 2 (left) . a scatter plot of the activity appreciation ratings vs. the self-efficacy of the teachers is shown in fig. 2 (right) . the red circle shows seven teachers with no prior programming experience and low selfefficacy prior to the activity, which gave high rating to the programming activity. in order to examine the variation in the appreciation of the activity among the teachers without prior programming knowledge, we present excerpts from the activity of two teachers: vera, an experienced physics teacher with over 20 years of teaching, and ana, a novice teacher in her first year of teaching. the two teachers were trying to modify the code in the first part of the computational modeling activity (see online supplement): v: if i want to do this, and place it (the graphical object) in a different initial location then… (trying to modify the code) it doesn't work. why didn't it change anything? oh! what have i done? i erased everything! a: maybe press here? v: no, here. but i didn't save, so now what? start over? a: no, do this (points to the "remix" button) to go back to the original. that is odd, it leaves the shapes, but not the code… v: i hate this! i really, really, hate this entire thing. it's annoying. a: i really like (unclear) v: i like excel a: excel is really awesome, v: it is helpful. it is the most convenient tool! vera is frustrated with the computational modeling activity because of the technical problem that erased the changes in her code. this incident results in her saying that she "really, really hates this entire thing." ana, on the other hand, responded with more patience, and even said she likes the modeling activity. they both also agreed that excel is a useful tool. vera also rated her appreciation of the activity lower than ana at the end of the workshop. the following excerpt indicates that while vera is an experienced physics teaching and very confident with her physics teaching, she is insecure when it comes to computational modeling: instructor: how is it going? v: it's annoying…i cannot compete with them (the students) in this (points to the screen), they will defeat me. our students learn python programming in 8 th grade... inst: but they cannot defeat you in physics… v: listen, when i teach them excel, i am the (authority)… vera is afraid that her authority would be undermined by the students who know more programming than she does. again, she mentions excel as a software that she is confident in using. ana, on the other hand, is a new teacher and does not feel the tension between her confidence in teaching physics, and introducing this new task. it also reminds her something she seemed to like as a child. v: let's move one a: do you remember basic? (programming language), "if-then, goto" (functions in basic) v: i've heard of it, but never used it a: when i was 12 years old i was in an afterschool program (that introduced basic) the excerpt shows that ana had some experience in programming as a child, whereas vera had none. this minor background detail might also explain their different approaches to computational modeling and indicate the importance of early exposure to computer programming. this study addressed two research questions-the first asked how prior experience in teaching physics influenced teaching self-efficacy and attitudes towards inquiry-based laboratory practices. the findings indicate that prior experience in teaching is strongly related to teachers' self-efficacy of teaching physics. this finding corroborates bandura's theory that self-efficacy emanates from mastery experiences of performance (bandura 1997) . the significantly higher self-efficacy fig. 2 appreciation of the activity was similar and positive among both teachers with prior programming experience, and those without (left). scatter plot of the appreciation and the selfefficacy of the teachers before the activity (right) in teaching physics of the experienced teachers is also in line with prior research on the effect of experience on teaching selfefficacy (tschannen-moran and hoy 2007). while experience was related to self-efficacy, the overall appreciation of the learning modules was positive among both experienced and novice teachers. this is true for the appreciation of the experimental and computational aspects of the modules, and specifically in relation to the computational modeling activity. table 3 shows that for the two main aspects of the workshop-computational modeling and conducting structured inquiry (items 3 and 6)both groups responded similarly, and positively. the workshops' purpose was to introduce a novel approach for coordinating experimental measurements and theoretical, computational modeling. this approach differs from traditional structured labs that aim at illustrating or reinforcing theoretical concepts. teachers' decrease in ratings of traditional goals, and the increase in ratings of inquiry practices related to the coordination of experimental results and theory, indicate that the workshop had some influence on the attitudes towards the main goals of the lab. teachers' ratings of inquiry lab practices such as designing an experimental setup indicate difference between experienced and novice teachers that are marginally significant. the differences may be related to prior acquaintance with inquiry practices: the mastery experience in the workshop might have boosted the novice teachers' belief in the importance of these practices, but not those of the experienced teachers who were already confident. this is similar to the differences in lab ratings of first-year undergraduates and students in more advanced years: first year students had lower initial ratings than students in their second year or beyond (wilcox and lewandowski 2018) . this difference probably reflects a process of self-selection-students or teachers who persist in their degree or teaching-are more likely to have positive attitudes towards the importance of the practices that they acquired (wilcox and lewandowski 2018) . the second research question addressed the relation between prior involvement with programming, self-efficacy, and attitudes towards the computational modeling activity. we found that the self-efficacy related to programming of most teachers has increased during the workshop, although teachers with no programming experience were still less efficacious than the teachers with programming experience at the end of the workshop. in addition, the appreciation of the computational modeling activity was similar among teachers with or without programming experience but the groups differed in the variation of their ratings, which was larger among the novice teachers as indicated in the error bars of the boxplot in fig. 2 . the transcript of the computational activity offers an explanation for this variation-it illustrates how two teachers without prior computer programming knowledge reacted differently to the computational activity. the conversation between the two teachers hints that the difference in appreciation may be rooted in different dispositions towards the adaptation of new technologies: vera expressed resentment towards the computational modeling activity and contrasted it with her satisfaction with excel-a user-friendly interface for data visualization that was also used in the workshop. conversely, ana mentioned that the programming activity reminds her a positive experience from the past. vera's reaction reflects the difficulty in switching from the "culture" of computer users of programs such as excel to the "culture" of computer programmers that requires planning and debugging (kolikant and ben-ari 2008) . one way to overcome such barriers is by introducing programming to children at a young age to prevent the perpetuation of the culture focusing on merely using readymade computer artifacts, to a culture views the computer as tool for developing new artifacts (disessa 2001). vera's comments might also indicate that she does not see the value of the computational model. it might be that she could benefit from a framing activity that would explicate the opportunities afforded by the computational modeling activity, and its difference from the excel. for example, the computational model uses symbols for the variables of force "f" and velocity "v" and allows to perform calculations with them (see lines 14-15 in fig. 1 ) this type of representation is not possible in excel, nor is it possible to simulate motion in a dynamic manner. at the beginning of this paper, we referenced the "hidden figures" movie scene to introduce the computational euler method as a legitimate, and sometimes even necessary, substitute for analytical/mathematical modeling. we believe that since computational modeling plays a central and indispensable role in scientific inquiry, this practice needs to be represented in school science, and the only way to do this is by investing in teacher pd. in this study, we examined science teachers' reactions towards a pd workshop that introduced inquiry-based modules with a computational modeling component. we found that the workshop contributed to both experienced and novice teachers. specifically, we found that most teachers' attitudes towards the computational modeling were positive, and their self-efficacy beliefs regarding computational modeling increased. the main limitation of our study is that it does not describe the actual implementation of the modules by the teachers in their classrooms. this would be a subject for an additional study. nevertheless, we note that despite the positive reactions at the end of the workshop, only 25% of the teachers implemented the modules in their classrooms in the subsequent year. some of the teachers were unable to implement the modules because they did not teach 9th grade physics the following year. others did not implement the module, since it required substantial amount of classroom time that they preferred to dedicate to topics mandated by the standard 9th grade curriculum. similarly, farris et al. (2019) wrote that "evident in the teacher's instructional approach was the tension between supporting students' exploration (of the computational model) and the curricular need for the production of canonically correct representations… that are mandated by the curriculum and that they would need in standardized tests." we conclude that in order to encourage more teachers to implement such time consuming, challenging learning modules, they should be included national curricula and teacher training programs. in our case, we found that the "free-fall" module, which is more aligned with the middle-school curriculum, has a better chance to be implemented than the oscillations module. another way that may be useful for streamlining the learning module is by embedding the programming activities in a learning management system such as moodle. in the workshop, teachers had to split their attention between a paper worksheet, and the vpython code on the computer. in a revised version of the module that was created to support teachers in distance teaching during the covid-19 shutdown, we embedded the vpython code within an online worksheet in a learning management system. this places the programming section as an integral component of the learning unit, and can encourage students and teachers to use it. yet, another way to encourage teachers to implement is by maintaining a professional learning community of the implementing teachers. in a learning community, teachers meet regularly to discuss their successes and challenges in implementation during the school year. this supplements the summer workshop, where teachers experienced most of the learning module as students, and did not have opportunities to rehearse them as teachers. at first, we used a model of individual teacher support that included planning sessions and classroom observations with the pd team. this model of support was only partially successful, since it prevented teachers from sharing stories and reflecting on their implementation challenges with their peers. promising results from studies of professional learning communities of teachers suggest that they have a better chance to induce changes in teaching and increase teaching self-efficacy (akerson et al. 2009; lakshmanan et al. 2011 ). conflict of interest the authors declare that they have no conflicts of interest. ethical statement this research was conducted with compliance of the rules mandated by our institution's internal review board (irb) and received the approval of the chief scientist of the ministry of education in our country. consent statement all of the data sources were consenting adults. all of the subjects in this study were teachers who responded to the surveys by will, and agreed to be video recorded. understanding student computational thinking with computational modeling fostering a community of practice through a professional development program to improve elementary teachers' views of nature of science and teaching practice first-year physics students' perceptions of the quality of experimental measurements self-efficacy: the exercise of control no silver bullet for inquiry: making sense of teacher change following an inquiry-based research experience for teachers innovative uses of video analysis. the physics teacher computational physics in the introductory calculus-based course all other things being equal: children' s acquisition of the control of variables strategy stable solutions using the euler approximation the integration of computer simulation and learning suppojrt: an example from the physics domain of collisions physical and virtual laboratories in science and engineering education using computer simulations for promoting modelbased reasoning: epistemological and educational dimensions methodology and epistemology of computer simulations and implications for science education changing minds: computers, learning, and literacy the modelling framework for experimental physics: description, development, and applications using introductory labs to engage students in experimental design the role of models in physics instruction learning to interpret measurement and motion in fourth grade computational modeling should students design or interact with models? using the bifocal modelling framework to investigate model construction in high school science what makes professional development effective? results from a national sample of teachers professional development for technology-enhanced inquiry science modeling instruction in mechanics introductory physics labs: we can do better c2stem: a system for synergistic learning of physics and computational thinking reasoning from data: how students collect and interpret data in science investigations dual space search during scientific reasoning fertile zones of cultural encounter in computer science education what is scientific thinking and how does it develop the impact of science content and professional learning communities on science teaching efficacy and standards-based instruction extending the boundaries of high school physics: talented students develop computer models of physical processes in matter the influence of core teaching conceptions on teachers' use of inquiry teaching practices k-12 science and mathematics teachers' beliefs about and use of inquiry in the classroom a hybrid approach for using programming exercises in introductory physics teaching scientific practices: meeting the challenge of change evaluating a course for teaching introductory programming with scratch to pre-service kindergarten teachers what makes professional development effective? strategies that foster curriculum implementation problematizing as a scientific endeavor learning to code or coding to learn? a systematic review student programming in the introductory physics course: muppet scaffolding complex learning: the mechanisms of structuring and problematizing student work the rise of the digital polymath: switzerland is crossing the computer science education chasm through mandatory elementary pre-service teacher education vpython: 3d interactive scientific graphics for students a comparison of programming languages and algebraic notation as expressive languages for physics dynaturtle revisited: learning physics through collaborative design of a computer model prospective physics teachers' self-efficacy beliefs about teaching and conceptual understandings for the subjects of force and motion the differential antecedents of self-efficacy beliefs of novice and experienced teachers sources of self-efficacy: four pd formats and their relationship to self-efficacy and implementation of a new teaching strategy teacher efficacy: its meaning and measure how to help teachers develop inquiry teaching: perspectives from experienced science teachers model construction as a learning activity: a design space and review evobuild: a quickstart toolkit for programming agent-based models of evolutionary processes a summary of researchbased assessment of students' beliefs about the nature of experimental physics science in the age of computer simulation model-based reasoning in the physics laboratory: framework and initial results publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-338976-eyerdt16 authors: nguyen, jeffers k.; shah, neal; heitkamp, darel e.; gupta, yasha title: covid-19 and the radiology match: a residency program's survival guide to the virtual interview season date: 2020-07-07 journal: acad radiol doi: 10.1016/j.acra.2020.06.023 sha: doc_id: 338976 cord_uid: eyerdt16 nan on march 11, 2020, the world health organization declared the covid-19 outbreak a global pandemic. since then, government agencies have taken significant measures to contain the spread of the coronavirus, imposing aggressive measures such as travel bans and shelters-inplace. while these aggressive efforts have flattened the curve and curtailed the spread of the virus, they have also produced negative collateral effects on the economy and medical education. tens of thousands of third-year medical students across the united states were abruptly frozen in their progress of choosing and securing medical specialties in which to train after graduation. official guidance on medical student activity was released by the association of american medical colleges (aamc) on march 17, 2020, with recommendations to suspend student participation in direct patient contact [1] . additionally, the aamc suspended both student and program access to the visiting student application service in adherence to broader travel restrictions and social distancing measures. on may 11, 2020, a special work group of the coalition for physician accountability, comprised of numerous stakeholders including the aamc, the national resident matching program (nrmp), and the accreditation council for graduate medical education, released their recommendations regarding away rotations, inperson interviews, and the electronic residency application service (eras) timeline for this application cycle. among these recommendations, the work group stated that for the 2020-2021 cycle, all interviews for prospective residents should be conducted virtually [2] . on may 21, 2020, a position statement was released jointly by the association of program directors in radiology and the association of program directors in interventional radiology, which supported the recommendations of the work group, urging all residency programs in diagnostic radiology and interventional radiology to commit to them [3] . beginning in march 2020, in accordance with social distancing guidelines, the national board of medical examiners (nbme) suspended most licensing exams administered at prometric testing centers, including step 1, step 2 ck, and step 3. this is anticipated to create a backlog of examinations once testing centers reopen. as a result, residency applicants' step 2 ck test dates have been pushed as far back as december 2020, well past the deadline for eras application submission and most programs' application review. the impact of covid-19 on this year's residency interview season will be profound. the move to virtual interviewing means that both students and programs must make ranking decisions without information provided by the in-person interview experience. programs will need to find a reliable alternative means by which to market themselves and obtain additional information about applicants. the aim of this article, written from the trainee's perspective, is to provide recommendations to radiology residency programs for filling the information gaps created by the rapid transition to virtual interviewing. we feel it is critical for programs to concentrate on three primary areas: (1) establishing an active, online presence as a source of reliable information for applicants, (2) employing additional screening tools to better select applicants for interview, and (3) creating a virtual department visit so that applicants and programs can still discover important information they seek in one another. historically, the most important and reliable data used by students to assess residency programs has been obtained at the in-person interview [4] . verbal and nonverbal information is processed by applicants at every step of the interview experience, including the pre-interview dinner with residents, the hotel accommodations, the look and feel of the city, the interview presentation, the formal interview sessions, informal conversations with residents, the campus tour, interactions with department staff, and the noon conference experience. because the inperson interview day will no longer be available, potential sources of program information will now be limited to direct outreach to residents themselves as well as the program websites, ranked two and three in order of importance, respectively [4]. without the benefit of the interview experience to market themselves and inform students of their programs' strengths, residency directors must now find ways of reaching out to accomplish these tasks. programs with a social media presence are certainly at an advantage, as they will already have an established platform for outreach [5] . social media enables two-way communication-both for students to ask questions, and for residents, faculty, and program directors to provide meaningful answers. specifically, twitter (twitter.com, san francisco, ca) is very effective at promoting and marketing a residency program, allowing for the regular advertisement of departmental achievements such as leadership positions, scholarships, awards, published articles, and presentations at national meetings [4]. this information is crucial for medical students as academic reputation and resident happiness are among the most important factors that medical students consider when ranking programs [4, 6] . programs without social media are losing out on these opportunities for outreach. while social media may be helpful for understanding how happy residents are at their programs, departmental websites are the primary sources for students to obtain official program information. unfortunately, a recent study of diagnostic radiology websites revealed that only 55% listed fellowship placement of their residents, and less than 70% described departmental research projects [7] . worse, nearly one third of integrated interventional radiology residency programs lacked a dedicated website [8] . of those that did, only 50% mentioned active research within the department. as students scramble to gather information during this year's virtual interview season, it is imperative that academic radiology departments quickly update their websites with accurate and complete information regarding their educational programs. more than ever before, the information, images, videos, and personal testimony provided on departmental websites will be responsible for shaping student impressions of residency programs. without the luxury of away rotations, in-person interviews, or licensure exam results, programs must innovate new methods of screening and ranking applicants. a 2018 survey conducted by the nrmp identified factors most important to diagnostic radiology program directors for selecting applicants to interview. aside from board-score related metrics, the highest rated factors included prior personal knowledge of the applicant, letters of recommendation in the specialty, the medical student performance evaluation letter, and perceived commitment to specialty [9] . the loss of the away rotation will make it difficult for students to become familiar with and express their interest in programs distant from their home institutions. additionally, students such as osteopathic and international medical graduates that have traditionally relied on away rotations for gathering letters of recommendation due to lack of home-institutional departments will now be hard-pressed to obtain specialty-specific letters and may have to rely on letters from core rotations. for these reasons, other components of the application, such as the personal statement or volunteer work, may be of more value this application cycle, offering the best insight into a student's personality and commitment to the specialty. as clinical experiences for students will be heterogeneous and limited this year, programs should provide reassurance to students by stating publicly on their department websites that: 1) students will not be penalized for canceled away radiology rotations or sub-internships, and 2) admissions committees anticipate variability in the number and mix of letters of recommendation this application cycle. moreover, the cost savings and uncertainty associated with this year's virtual interview season is likely to encourage students to try to interview at more programs than usual, leading to an unprecedented number of applications for most programs. this may create the need for programs to enhance their application screening policies. care should be taken to screen in a way that does not discriminate against students that have yet to receive their board scores, who are from different geographical areas, or who do not have specialty-specific letters of recommendation. in order to ensure fairness, programs should allow extra time for screening and plan to review a greater diversity of applications this cycle. programs that typically require step 2 ck scores for consideration should forgo these restrictions this year to ensure fairness to students that could not reschedule step 2 in time. programs should also consider that many applicants this year will not have the opportunity to demonstrate improvement on step 2 following a suboptimal step 1 performance. the prolonged suspension of student activities during the pandemic has affected many vital metrics within the eras application that program directors traditionally use to compare applicants, such as clinical rotation experiences, step 2 scores, and quality of recommendation letters. programs should seek to level the playing field for student assessment by utilizing alternative screening and interviewing methods, such as holistic application review, supplementary essays, and the multiple mini interview (mmi) technique. holistic review emphasizes a balanced consideration of applications utilizing a strategic, evidence-based approach, recognizing diversity as an essential component to program success. a short supplementary essay, which would require a writing prompt from programs, would give students control of an additional component in their applications with which they could convey interest, passion, and creativity. programs may also choose to put more emphasis on the personal statement this year, as this will be one of the few elements that students truly control. the mmi involves a number of short interviews, each conducted with a different interviewer. applicants rotate through each interviewer in a similar setup to an objective structured clinical encounter (osce) or the traditional oral boards [10] . in comparison to a traditional interview, the mmi involves shorter interviews, each with a pre-determined question or topic to be discussed. with shorter interview lengths, programs can schedule more interviews per day using this technique. assessing applicant responses to the same list of vetted questions may help level the playing field with respect to direct comparison of candidates. with more applicants per program anticipated this year, this technique may help programs cast a wider net and ensure enough ranks to match successfully. a study conducted by phitayakorn et al. in 2015 evaluated factors most of value to applicants when selecting and ranking programs. the results showed that among the 5 most important factors were resident morale, faculty availability, and faculty involvement in resident training [11] . the avenues by which students are able to gauge these measures have traditionally been the interview day and the away rotation. without being able to visit programs in person to meet and interact with the residents and faculty, students will not be able to evaluate the criteria that they say they value most. successful programs will likely be those that can innovate ways to capture these opportunities and provide them to students online in time for the 2020-2021 interview season. this will mean brainstorming department and residency strengths, devising strategies for how to best market those strengths, and then delivering the digital content to students. one such idea would be to offer "virtual away rotations." interested radiology applicants could simply join the day-to-day activities of a department using the same web-based platforms currently being used to conduct resident educational conferences, department meetings, and even virtual social gatherings. these electives could offer access to teaching curricula, departmental educational resources, and live sessions with residents to help foster relationships as students seek information about the program and the city. the american college of radiology has already taken lead, creating a two-week virtual curriculum that provides medical students with independent required reading assignments, online modules, and online scenarios to incorporate evidence-based strategies for imaging services [12] . several radiology (geisel school of medicine at dartmouth and weill cornell medicine) and radiation oncology (stanford university and ohio state university) programs have followed suit, designing and piloting virtual rotations for students. these efforts will likely provide programs a recruiting advantage, showcasing their residents, faculty, and program to applicants. residency programs could also produce short videos available online to provide the kind of information normally sought by students during the course of the interview experience or away rotation. well-produced segments featuring residents or faculty addressing important topics such as faculty involvement in education, resident morale, department support of the educational mission, details of the rotation curriculum, and professional development opportunities would likely allow programs to stand out among the competition. video would also be an effective way to showcase a department's facilities, teaching faculty, residents, campus, and city. brief messages given by department chairs, program directors, chief residents, and other education leaders would help underscore the program's branding. social media platforms such as facebook (facebook.com, menlo park, ca) and twitter could also be utilized to highlight a program's educational and innovative initiatives such as resident team building, academic work, and resident awards and achievements. another idea to help applicants and programs connect is to set up virtual small-group sessions, such as "zoom happy hours", between a program's residents and students, allowing applicants to ask questions and discuss the kind of topics that would normally get covered during the in-person preinterview dinner. the covid-19 pandemic will make the next interview season challenging for radiology residency programs. the suspension of medical students' clinical rotations will lead to great heterogeneity in applications with respect to their clinical experiences, their ability to take usmle examinations, and their success in finding optimal letter writers. the move to virtual interviewing will make it harder for students and programs to learn about one another. updating department websites and establishing a strong social media presence may be the most crucial way for programs to provide vital program information to students. utilizing additional unbiased screening measures and increasing manpower will help programs appropriately find and select best-fit applicants. allowing applicants to visit programs virtually, either through virtual away rotations or the production of high-yield videos, will help fill the information gap left by the loss of in-person interviews. implementing these recommendations are likely to improve outcomes for both applicants and programs during the 2020-2021 interview season and beyond. none. guidance on medical students' participation in direct patient contact activities the coalition for physician accountability work group. final report and recommendations for medical education institutions of lcme-accredited apdr/apdir position statement regarding residency recruitment in the 2020-2021 cycle. association of program directors in radiology ashx?la=en&h ash=5b0b7f09afae487e50aa36d9e4f27c79c9489dce. accessed radiology program directors should have an active presence on twitter evaluating factors and resources affecting ranking of diagnostic radiology residency programs by medical students in 2016-2017 an assessment of radiology residency program websites integrated interventional radiology residency program websites: a development in progress results of the 2018 nrmp program director survey multiple miniinterviews versus traditional interviews: stakeholder acceptability comparison applicants' self-reported priorities in selecting a residency program virtual radiology curriculum for medical students key: cord-271308-nypkr537 authors: drexler, richard; hambrecht, jan m; oldhafer, karl j title: involvement of medical students during the coronavirus disease 2019 pandemic: a cross-sectional survey study date: 2020-08-30 journal: cureus doi: 10.7759/cureus.10147 sha: doc_id: 271308 cord_uid: nypkr537 background: the coronavirus disease 2019 (covid-19) pandemic affects the education of medical students around the world and countries have had differing responses in dealing with this dynamic situation. the role of medical students in fighting this pandemic is controversial and it is yet to be elucidated how they can best be of service. the aim of this study is to evaluate the working fields of volunteering students and the impact of the pandemic on final year students from a student's perspective. methods: an anonymous online survey was conducted amongst 219 medical students from hamburg (germany), using an institutional online data collection program. results: a total of 137 questionnaires (63.5%) were completed. of these, 97 participants were students from academic year three to five (70.8%) and 40 students were in the final year of medical school (29.2%). of the 97 students from academic year three to five, 68 students (70.1%) signed up for voluntary duties during the pandemic. interestingly, only 25.0% of the students were called for voluntary work in hospitals or health authorities. final year students had already been working in hospitals since before the outbreak, with 35.0% of them assisting doctors in the treatment of covid-19 positive patients during their placements. using a 5-point likert scale, the students who volunteered self-assessed their work as more useful and received more gratitude than final year students (p<0.01). conclusions: the majority of medical students are willing to make a significant contribution in the response to covid-19 and do not wish to be overlooked. furthermore, the current pandemic offers novel educational opportunities for medical students. the coronavirus disease 2019 (covid19) pandemic has spread globally with major outbreaks in the usa, italy and spain. although the end is not yet in sight and the final lasting impact of covid-19 is difficult to gauge, it is clear that the current situation is overwhelming for health care systems and economies around the world. as the pandemic progresses, staff shortages will likely occur, and this raises the question of who will step up to the plate in this ongoing crisis. in this context, the role of medical students is unclear and contentious [1] [2] . due to legal requirements, social distancing was implemented and in-person medial classes have been cancelled and replaced with online lectures and virtual teaching for most medical students [3] . however, the educational situation is crucial for final year students, as they are completing clinical clerkships. these students are normally integrated into medical teams as well as clinical routines and gaining clinical experience is a crucial aspect for success in future residencies. nevertheless, student participation in clinical care varies across medical schools and different countries. some countries, such as italy, china and the united kingdom, integrated the medical students into their health care systems or graduated students early [4] [5] [6] . on the contrary, other countries cancelled clerkships and restricted patient contact for medical students [7] [8] . in germany the treatment of final year students varied across hospitals, but non-final year students were encouraged by the german minister of health to volunteer in healthcare facilities [9] . it is clear that there is a dissonance between medical students' roles during the covid-19 pandemic and the progression of patient-centered medical education. in this study, we present medical students' involvement during the covid-19 pandemic. we evaluate the usefulness of volunteering students as a response to the pandemic as well as the assessment from a student's perspective during volunteering. in addition, we describe the impact of the pandemic on final year students and their clinical education. an online survey was designed and sent to 219 medical students enrolled at the campus hamburg (germany) of the semmelweis university budapest (hungary) via personalised institutional email. the survey was conducted using an institutional online data collection program. two surveys were performed, one survey for medical students from academic year three to year five and a separate survey for final sixth-year students. the surveys consisted of single, multiple-choice, and 5-point likert scale questions and recruitment began on may 12, 2020. the survey was open from may 12, 2020, to may 17, 2020, and available in german. inclusion criteria were students currently active from academic year three to six at the medical faculty hamburg of the semmelweis university budapest. ten external students piloted the survey prior to publication, and minor amendments to wording to improve clarity were made. after a period of data collection, three researchers checked the database for errors and false data independently from each other. four main topics were addressed: (1) reasons for volunteering or rejection of volunteering; (2) work circumstances of volunteering students; (3) critical self-reflection of usefulness of volunteering and (4) impact of covid-19 pandemic on planned clerkships and future plans after graduation. students from academic year three to five were initially asked if they had registered as volunteers. individuals who volunteered until may 12, 2020, then answered a more detailed questionnaire about reasons, working field, type of activity, working hours, and contact to covid-19-positive patients. furthermore, they selfreflected their volunteer work answering a 5-point likert scale ("strongly disagree", "disagree", "neutral", "agree", "strongly agree"). the final year students were asked about the changing workload since the beginning of the covid-19 pandemic, contact to covid-19 positive patients, possible infection, and the impact on future residencies. in closing, final year students self-reflected their clerkship answering the 5-point likert scale as described above. all study procedures were reviewed and approved by the ethics committee hamburg, germany (wf-098/20). no patient data was included. we informed participants that their answers would be anonymously used for statistical analyses and that they would not be transmitted to third parties. variables were processed and analysed using ibm spss statistics for mac version 25 (ibm corp., armonk, ny, usa). data were reported as number with percentage. differences in proportions were analysed with the chi-square test or fisher exact test. differences in 5-point likert scale between third to fifth year and final year students were compared, using the two-sample t-test as stated by norman [10] . a two-sided p-value less than 0.05 was considered as statistically significant. the survey was sent to 219 medical students including 58 third-year students (26.5%), 60 fourth-year students (27.4%), and 57 fifth-year students (26.0%). the remaining 44 students (20.1%) were in the final year of medical school. in total, 137 questionnaires (63.5%) were completed. of these, 97 participants were non-final year students (70.8%) and 40 students were in final year (29.2%). we enrolled 97 students from academic year three to five with an equal distribution within the years (p=0.78, table 1 ). these students were affected by the cancellation of in-person medical classes, which were consequently replaced by online lectures. the students were therefore enabled to volunteer in hospitals or aid health authorities during the covid-19 pandemic. of the 97 students, the majority (70.1%) registered as volunteers, while 29 students (29.9%) did not ( table 1) . depending on their first answer, students were asked what motivated them to sign up as volunteers or to decline the opportunity. the volunteering students had quite similar answers with the majority indicating a sense of duty to society (64.7%), interest in medical activity (57.4%), and social commitment (69.1%) as their incentives. the main reason given for not signing up was a lack of time due to studying commitments or part-time jobs (55.2%). however, six students (20.7%) were either part of a high-risk group or had a first-degree family member at risk. finally, 11 students (11.3%) received notification that their clinical clerkships were cancelled due to the on-going pandemic. 1 multiple answers were possible, in total: 35 answers. 2 multiple answers were possible, in total: 154 answers. as previously stated, 68 students signed up for voluntary commitment but only 17 students (25.0%) were called for work. these 17 students were asked further questions regarding their work circumstances in the survey ( table 2 ). the majority (64.7%) were called within a week after signing up. almost half of the students (47.1%) were assisting health authority bodies, which involved telephone consultations or data administration (29.4%). five students (29.4%) volunteered in the hospital, either on the ward, in icu, or in the emergency department. the remaining volunteers worked in the ambulance service (11.8%) or worked in general practices (11.8%). during their voluntary work, six students (35.3%) had physical contact with patients who tested positive for covid-19. however, none of these students were suspected of having or tested positive for covid-19. to evaluate the usefulness and gratitude of the voluntary work during covid-19 pandemic, the 17 volunteering students were confronted with several statements that were answered using a 5-point likert scale (figure 1) . when self-reflecting on their work, 94% of the students felt helpful and 81% were under the impression that other medical staff valued their work. approximately half of the volunteers acquired new skills through their work and only 6% felt overburdened. academic year three to five. the final year students have a special status as they were already working in hospitals at the beginning of the covid-19 pandemic. therefore, a second survey was specifically designed to evaluate the impact of covid-19 on their workload and clinical education. the response rate was 90.9% and consisted of 40 final year students ( table 3) . when comparing the students' workload before and during the covid-19 pandemic, most students (72.5%) experienced a reduced workload. of note, 14 students (35.0%) were directly involved in treating covid-19 positive patients. during their rotations 15 students (37.5%) were suspected to be covid-19 positive, but ultimately tested negative. as previously described for the first survey, the final year students were confronted with different statements regarding their role during the covid-19 pandemic ( figure 2 ). only 23.0% of the students evaluated their work as helpful and felt their contribution was valued by other medical staff. in addition, 5.0% agreed that they developed new skills due to the covid-19 pandemic. most importantly, students felt the pandemic had a negative impact on education; especially bedside teaching for example, was experienced by 87.0% of students. focussing on future perspectives, the majority (62.5%) felt the pandemic had a negative impact on their planned residencies or research activities after graduation. we described the perspectives of two student groups during the covid-19 pandemic. on one hand, the volunteering students from academic year three to year five, and on the other, the final years in their practical placements. both groups were equally involved in covid-19 patient care (35.0% vs. 35.3%, p>0.05). however, final year students were more often suspected of being covid-19 positive (p<0.01). when evaluating the role of both groups using a 5-point likert scale, significant differences were observed. the volunteering students evaluated their work as more helpful and more of them felt they had acquired new skills (p<0.01). in addition, volunteers' work was appreciated and valued more by the medical staff in comparison to final year students conducting their clinical placements (p<0.01). the covid-19 pandemic has spread around the world and poses a significant challenge ubiquitously for all health care systems. in the wake of finding solutions for possible staff shortages, the role of medical students is contentious as they could significantly contribute to the ongoing crisis. to address this eminent topic, we performed an online survey amongst medical students from hamburg, one of germany's epicentres during the pandemic. we evaluated the willingness to volunteer among third to fifth-year medical students. the actual demand for students was also evaluated as well as the practicality for volunteers in hospitals and health authorities. in addition, our study focussed on the ever-changing clinical education experienced by final year students. these students were already in the final months of their practical placements when the pandemic broke out. as the traditional didactic, in-person teaching shifted towards online lectures and virtual seminars since the beginning of the pandemic, most students wanted to contribute to the fight against the covid-19 pandemic [2, [11] [12] [13] [14] . the results of our survey underline the willingness of medical students to contribute to the handling of the crisis as 70.1% of the students signed up for voluntary commitment. of the remaining students who were unwilling to volunteer, nearly 40% were either already employed in a part-time job in hospitals or unable to sign up due to a higher risk of severe illness from covid-19. however, it was remarkable that only 25.0% of the students had been called for voluntary work, mainly in the realm of hospital care or health authorities. this low number begs the question if hospitals in germany really need student volunteers and gives the impression that hospitals are capable of solving staff shortages without the helping hands of medical students. one explanation might be that lower numbers of infections were experienced here than had been anticipated. however, when looking at current reports from several countries it becomes clear, that this abrupt conclusion cannot simply be drawn and medical students are indeed able to have a positive impact during the covid-19 pandemic [15, 16] . this is reiterated by the fact that 35% of the final year students in this study were already involved in the treatment of covid-19 patients and the role of medical students in general could prove to be indispensible in the on-going fight against the pandemic. furthermore, it is noteworthy that staff shortages and the demand for medical students correlates with the extent of the pandemic outbreak and the strain on the respective country's healthcare system. as major outbreaks were experienced in italy and the uk, medical students played a central role in dealing with the crisis in comparison to germany [4] [5] [6] . however, bearing in mind the unequivocal will of medical students to participate in the fight against covid-19, the role of medical students must be considered. focussing on our results, volunteering students supported doctors in patient care or worked at a covid-19 hotline service. students who took over these tasks unburdened doctors from administrative duties so that they could entirely shift their focus to essential care of critically ill patients. in our study, volunteering students were encouraged to self-reflect on the usefulness of their tasks via a 5-point likert scale. the majority of students considered their work as useful and felt it was valued by medical staff. this emphasizes that students could undertake certain tasks with a symbiotic advantage for both doctors and students, and is consistent with ideas by miller and colleagues [2] . however, it is doubtful if these students are sufficiently trained to undertake roles with more responsibility, such as assisting with invasive ventilation of patients [17] . the student group most affected by the covid-19 pandemic is undoubtedly students in their final year, due to restricted patient contact, reduced bed-side teaching and even cancelled clerkships [8, [18] [19] [20] [21] . therefore, it is unsurprising that 87.0% of the final year students in this study reported a negative influence on their clinical education. in addition, the dissonance in suitable roles for final year students during the pandemic has lead to huge dissatisfaction in our study cohort. final year students have gone through years of rigorous training, are nearing graduation, and therefore could have a decisive role and make a significant contribution during this pandemic. even though clinical education is currently not the main focus, the integration of final year students into critical care of covid-19 patients could be of indispensable value for teaching certain skills, such as critical care or ventilation therapy. furthermore, it can prove useful for crisis management preparation with regard to possible future pandemics. the ongoing pandemic is a challenging time for maintaining clinical education for students on the cusp of graduation, but could be regarded as a worthwhile opportunity to educate them beyond the traditional curriculum. innovative ideas are urgently needed in order to implement these novel-learning opportunities and simultaneously integrate final year students as full members into critical care or emergency teams. there have been a few promising approaches introduced in recent literature [15, 16, 22] . rasmussen and colleagues initiated fast-track courses in ventilation therapy and nursing assistance for medical students attending aalborg university and successfully integrated the majority of these students into medical care teams [15] . another promising approach came from klasen and colleagues who developed a training curriculum including all necessary aspects for working in a triage-test-center for diagnostic swab-testing. after educating medical students according to this new curriculum, they were assigned to emergency teams that evaluated hundreds of patients daily and provided over 6700 swabs during a five-week period [16] . these concepts combine the contribution of medical students to the pandemic with the utilization of these novel learning opportunities. these newly acquired unique clinical skills further the student's opportunities in their future career as a physician. more importantly, rasmussen and klasen demonstrate how medical students' role was crucial to the fight against the covid-19 pandemic. the involvement of medical students is entirely dependent on the gravity of the situation in the respective country. our study showed the current situation in one of the worst-affected cities in germany and evaluated the medical students' involvement in the covid-19 response from the students' perspective. in addition, ramussen and klasen et al. introduced concepts functioning in denmark and switzerland [15] [16] . it would be favourable to share further experiences from various other countries, which would show the integration of medical students into clinical settings. this could prove invaluable in the ongoing fight against covid-19 and much can be learned from different countries' approaches. nevertheless, the students' point of view must be taken into consideration in order to benefit future generations of medical students and patient care. it is our hope that the covid-19 pandemic leads to the development of alternative plans in how medical students could be positively utilised in future pandemics and in other exceptional circumstances, so that hospitals and medical schools are better prepared for future crises. we are convinced that pandemics, especially the current one, could offer various educational opportunities for medical students and should not lead to educational disadvantage and feelings of overall dissatisfaction. the majority of medical students are willing to make a significant contribution in the response to covid-19 and do not wish to be overlooked. furthermore, the current pandemic offers novel educational opportunities for medical students. however, it is mandatory that medical schools and hospitals implement functioning concepts in which students are integrated into clinical settings. disclosures medical students are not essential workers: examining institutional responsibility during the covid-19 pandemic the role of medical students during the covid-19 pandemic virtual bedside teaching rounds on patients covid-19: medical students and fy1 doctors to be given early registration to help combat covid-19 covid-19: medical schools are urged to fast-track final year students guidance-on-medical-students-participation-in-direct-patient consequences of covid-19 on international medical graduates and students applying to residencies in the united states likert scales, levels of measurement and the "laws" of statistics let us help'-why senior medical students are the next step in battling the covid-19 pandemic we signed up for this! -student and trainee responses to the covid-19 pandemic coalition for medical education-a call to action: a proposition to adapt clinical medical education to meet the needs of students and other healthcare learners during covid-19 senior medical students in the covid-19 response: an opportunity to be proactive. acad emerg med. 2020 medical students for healthcare staff shortages during the covid-19 pandemic wab team instead of swat team -students as front-line force during the covid-19 pandemic covid-19: medical students should not work outside their competency, says bma the impact of covid-19 on medical education medical student involvement in the covid-19 response letter: covid-19 impact on the medical student path to neurosurgery a bold response to the covid-19 pandemic: medical students, national service, and public health students supporting critical care -a contention plan to prevent the decompensation of icus in the covid-19 pandemic: translating bjorn ibsens' polio-lessons to modern times human subjects: consent was obtained by all participants in this study. ethics committee hamburg, germany issued approval wf-098/20. all study procedures were reviewed and approved by the ethics committee hamburg, germany (wf-098/20). no patient data was included. we informed participants that their answers would be anonymously used for statistical analyses and that they would not be transmitted to third parties. . animal subjects: all authors have confirmed that this study did not involve animal subjects or tissue. conflicts of interest: in compliance with the icmje uniform disclosure form, all authors declare the following: payment/services info: all authors have declared that no financial support was received from any organization for the submitted work. financial relationships: all authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. other relationships: all authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work. key: cord-354959-0iwdqbu6 authors: saddik, b.; hussein, a.; sharif-askari, f. s.; kheder, w.; temsah, m.-h.; koutaich, r. a.; haddad, e. s.; al-roub, n. m.; marhoon, f. a.; hamid, q.; halwani, r. title: increased levels of anxiety among medical and non-medical university students during the covid-19 pandemic in the united arab emirates. date: 2020-05-13 journal: nan doi: 10.1101/2020.05.10.20096933 sha: doc_id: 354959 cord_uid: 0iwdqbu6 introduction: the covid-19 pandemic is likely to increase anxiety levels within the community and in particular medical students who are already considered psychologically vulnerable groups. since the covid-19 outbreak, no study has yet estimated the effect of this pandemic on university students in the uae or its impact on the psychological well-being of medical students. methods: in this study, we surveyed 1485 medical (comprising medical and dental) and non-medical university students across 4 emirates within the uae. we used an online platform to assess knowledge, sources of information, changes in hygienic behavior, perceptions of fear and worry and anxiety levels using the generalized anxiety disorder 7 (gad-7) scale. the gad-7 score was measured at three time points; during hospital visits for medical/dental students, before the introduction of online learning and after online learning for all students. results: the majority of students demonstrated high levels of knowledge and utilized reliable sources of information. non-medical students exercised higher compliance with social restrictions, while medical students practiced better hand hygiene. almost half of students reported anxiety levels ranging from mild to severe with females reporting higher anxiety scores during hospital visits (or=2.02, 95% ci,1.41 to 2.91) and medical students reporting lower anxiety levels in comparison to dental students (or=0.61, 95% ci,0.45 to 0.84). medical students reported higher levels of anxiety during their clinical rotations which decreased with the introduction of online learning, yet, non-medical student's anxiety levels increased with online learning. conclusions: this is the first study to provide important information on the initial response and anxiety levels in university students across the uae during the covid-19 pandemic. the findings from our study can be used to support the development of effective screening strategies and interventions to build psychological resilience among university students during the covid-19 pandemic or any other public health emergencies in the future. study, we aim to assess medical student's psychological distress and concerns during the recent 85 covid-19 pandemic, their degree of perceived information about the disease, and their overall 86 attitude, practices and behaviors during the outbreak. we also evaluate students' perceptions on 87 the precautionary measures in place and the effectiveness of educational strategies in reducing 88 anxiety levels in the united arab emirates. this makes our study the first in the uae to discuss 89 this aspect in the current pandemic. committee (rec-20-03-03-02) prior to participant recruitments. 104 sample size 105 a study conducted in saudi arabia during the mers-cov outbreak found that 77% of medical 106 students reported minimal anxiety levels [11] . using this proportion and a confidence level of 95% 107 and margin of error of 5%, we calculated the minimum required sample size to be 278 for each of 108 the medical and non-medical groups. to account for non-response the sample size was increased 109 by 20%, making the minimum sample size required for this study 333 students for each of the 110 medical and non-medical groups. to not worried at all, and their opinion about the public fear associated with covid-19 being 121 justifiable or dysfunctional, measured on a 5-point likert scale, ranging from strongly agree to 122 strongly disagree, 5) perception of receiving enough information on covid-19 on a 5-point likert 123 scale, ranging from strongly agree to strongly disagree, 6) the sources of information students 124 resorted to for information on covid-19 and 7) measurement of anxiety levels. . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted may 13, 2020 . . https://doi.org/10.1101 /2020 in order to measure anxiety levels, we used the generalized anxiety disorder scale (gad-7) [14] 126 which is a 7-item questionnaire asking participants how often they were bothered by each symptom 127 such as feeling nervous, trouble relaxing, irritable and afraid that something awful might happen 128 during the last 2 weeks. response options were "not at all," "several days," "more than half the 129 days," and "nearly every day," scored as 0, 1, 2, and 3, respectively. a score of 10 or greater 130 represents a reasonable cut point for identifying cases of anxiety with a sensitivity of 89% and 131 specificity 82%, internal consistency (cronbach α=.92) and test-retest reliability (intraclass 132 correlation=0.83) [14] . the gad-7 has also been identified as a screener for panic disorder, social 133 phobia and ptsd ( with a cutoff score of 8 sensitivity 77% and specificity 82%) [15] . in our study, 134 the gad-7 score was totaled for each student and classified into cut-off points of (0-4 minimal, 5-135 9 mild, 10-14 moderate and 15-21 severe) levels of anxiety. levels of anxiety were measured using 136 the gad-7 scale at three different time points. in addition to calculating the mean gad-7 score, 137 scores were classified into 3 categories (minimal, mild and moderate/severe anxiety). during the launch of data collection for this study, the uae ministry of education declared 139 university campuses to move to online learning to prevent the spread of covid-19 and classes 140 moved online on march 8. in order to determine levels of anxiety of students during the transition 141 to online learning, medical and dental students were asked to respond to the gad-7 questions 142 reflecting back to the time they were attending hospital visits within the previous two weeks. this 143 was to assess whether higher levels of anxiety were associated with hospital visits. all students 144 were asked to respond to the gad-7 questions reflecting on "before the introduction of online 145 learning", and "after the introduction of online learning". the final version of the questionnaire was piloted to ensure clarity and consistency between survey 147 items. to ensure face and content validity of the survey instrument, the survey was sent to a group 148 of 9 experts which consisted of students, tutors, faculty and a psychiatrist who reviewed the survey 149 for content accuracy, clarity and comprehensiveness and whether the survey met its objectives. as descriptive statistics, including means, medians, frequencies and percentages were used to 154 summarize data and to illustrate the demographic and other selected characteristics of students. normality of data was tested visually using the q-q plots and statistically using the smirnov test. bivariate analysis using chi-square ( 2 ) and mann-whitney u tests explored the 157 associations between student demographic characteristics and anxiety levels. spearman's 158 correlation coefficient, r, was used to evaluate the association between knowledge score and gad in total, 1484 students responded to the questionnaire and data were analyzed for 1385 completed 166 surveys (completion rate 93.3%) from 4 different emirates across the uae. the mean age of 167 students was 20 years and most participants were females (72%). almost three quarter of the 168 . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 13, 2020. . https://doi.org/10.1101/2020.05.10.20096933 doi: medrxiv preprint students were studying medicine or dental medicine and from those students a third (35%) were in 169 their clinical years of study and were completing clinical ward rotations. of these students, (12%) 170 reported being in contact with an infected or suspected case of covid-19. demographic 171 characteristics are shown in table 1 . the majority of students reported increased hand hygiene (85%), increased use of hand sanitizer 174 (85.5%), avoiding people with flu like symptoms (80%) and decreased visits to crowded places 175 (77%). a lower proportion of students reported wearing gloves and masks (58%), decreased social 176 visits (62%) or decreased hand shaking (58%). we combined medical and dental students together 177 as the medical group, and all other specialties made up the non-medical group. when medical and 178 non-medical students were compared, more non-medical students avoided being in contact with 179 people with flu-like symptoms, had decreased social visits, decreased visits to crowded places and 180 use of public facilities as displayed in figure 1 . 182 most students agreed they had heard enough information on the symptoms, prognosis, 183 transmission and prevention of covid-19. when we compared medical and non-medical 184 students, significantly more medical students reported having heard sufficient information on 185 covid-19 prognosis,  2 (2, n=1385) = 55.20, p<0.001 and covid-19 transmission,  2 (2, 186 n=1385) = 17.88, p<0.001. (table 2 ). overall, students had good knowledge of covid-19 with 187 over 70% of students answering correctly on the covid-19 knowledge questions with a median 188 score of 5 (from a maximum score of 7). when we compared participant's' knowledge scores by 189 field of study, gender and contact with covid-19, we found that medical students, females and 190 students who had been in contact with covid-19 cases had significantly higher knowledge scores 191 than their counterparts, as shown in table 2 . student sources of information on students' main sources of information were official websites, press releases from the ministry of the median gad-7 scores for medical students during their hospital visits, for all students before 200 introduction of online learning and after online learning were 4, 4 and 3, respectively. when we 201 compared gad-7 scores for medical students by gender, specialty (dental or medical), contact 202 with covid-19, rotation type and clinical phase of study, we found that females, dental medicine 203 students, students who had been in contact with covid-19 patients and students in their clinical 204 phase of study reported statistically significant higher anxiety levels. more students (27%) in low-205 risk ward rotations reported mild anxiety while the majority of students who reported moderate to 206 severe levels of anxiety (26%) were attending high risk ward rotations  2 (3, n=1385) = 10.92, 207 p=0.027 (table 3) . . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 13, 2020. . https://doi.org/10. 1101 /2020 overall, females reported higher levels of anxiety in gad-7, both before and after online learning. the medical students reported higher levels of anxiety before online learning in comparison to 210 non-medical students (table 3) . 211 interestingly, anxiety levels significantly decreased for females and for medical students after 212 switching to the online learning, with  2 (2, n=1385) = 12.22, p<0.001 and  2 (2, n=1385) = 20.99, 213 p<0.001, respectively, with a higher percentage of decreased anxiety among dental students; whilst 214 non-medical students reported higher levels of anxiety after the introduction of online learning ( 2 215 (4, n=1385) = 36.38, p<0.001) (table 3) . the majority of students (73%) reported being worried about transmitting covid-19 to one of 217 their family members or friends, while 65% of them were worried about catching the virus 218 themselves. almost half (49%) of medical students reported being worried about transmitting 219 covid-19 to others and were less likely (29%) to worry about catching it themselves, in 220 comparison to non-medical students (35%), although the differences were not statistically 221 significant ( 2 (2, n=1385) = 2.85, p=0.24). most students (61%) agreed that the public fear was 222 justifiable and (46%) of students did not perceive it as dysfunctional . a cumulative odds ordinal logistic regression with proportional odds was performed to determine 224 predictors of anxiety levels during the different time points. the effect of gender, specialty, contact 225 with covid-19, rotation type, clinical phase of study and knowledge score on the gad-7 score 226 during hospital visits was modelled. we included knowledge score in the regression model because 227 we found a positive, but weak, correlation between knowledge score and anxiety score using 228 spearman's correlation (rs=0.086, p=0.015). the odds of females having higher anxiety scores 229 during hospital visits was 2.02 (95% ci, 1.41 to 2.91) times that for males, and medical students 230 reported lower anxiety levels in comparison to dental medicine students (or=0.61, 95% ci, 0.45 231 to 0.84). furthermore, being in a rotation and demonstrating higher knowledge scores predicted 232 higher gad-7 scores during hospital visits (table 4) . similarly, gender, field of study and 233 knowledge score predicted higher gad-7 scores before online learning, but only gender and field 234 of study predicted gad-7 scores after switching to online learning (table 4) . overall, students in our study demonstrated good knowledge of covid-19 and reported using 247 reliable sources such as official statements and press releases from the ministry of health and the 248 who website. although the use of these sources was higher among medical students, social media 249 was the main source of information for both medical and non-medical students. this finding is 250 . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 13, 2020. . https://doi.org/10. 1101 /2020 consistent with the literature and emphasizes the role that social media can play in risk perception 251 and dissemination of reliable information during a pandemic such as . however, other studies have also reported that young people tend to obtain a large amount of 253 information from social media which can easily be a trigger for stress and anxiety [22] . good knowledge may also explain students' compliance with hygienic practices in our study. most 255 students reported significant change in their hygienic behavior since the covid-19 outbreak, 256 particularly for increased hand hygiene, avoiding crowded places and avoiding being in contact 257 with people with flu like symptoms. however, even though levels of knowledge were significantly 258 higher among medical students, compliance with hygienic practices were similar for both groups. the results from the current study confirm that anxiety levels due to covid-19 among university 276 students are high, ranging from mild to severe, especially amongst females, which is in accordance 277 with previous research [22, 30, 31] . the majority of students in our study were females, which 278 reflects the gender imbalance in higher education within the uae. more than two thirds of students 279 reported mild anxiety levels and one third of students reported severe anxiety. what makes this 280 study unique, is that medical and non-medical students were compared, and within the medical 281 student group, we compared medical and dental students. furthermore, this study assessed anxiety 282 levels at three different time points: during hospital visits for medical and dental students, before 283 online learning and after switching to the online learning for all students. among the 284 medical/dental group, students in their clinical phase of study and who had rotations in high risk 285 wards or who had been in contact with covid-19 patients reported significantly higher (moderate 286 to severe) levels of anxiety, demonstrating that high risk perception of covid-19 may contribute 287 to higher levels of anxiety. medical/dental students continued attending the hospitals at the early 288 onset of covid-19 when fear and worry associated with the outbreak would have been at their 289 peak and before lockdown restrictions were in place, which could also explain their high 290 compliance with infection control measures. medical students high risk perceptions associated 291 with attending hospitals during infectious disease outbreaks has been reportedly associated with 292 higher levels of anxiety [12, 21] . however, when we compared between medical and dental 293 students, dental students reported higher levels of anxiety and these remained significant with 294 further multivariate analysis. previous studies have established that medical and dental programs 295 . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 13, 2020. . https://doi.org/10.1101/2020.05.10.20096933 doi: medrxiv preprint are highly competitive and students generally experience high levels of stress during their training 296 [32, 33] . yet, dental students suffer from greater levels of perceived stress than medical students 297 due to their role as providers of care and earlier exposure to patients in the dental clinics [34] . 298 furthermore, dental students are in very close proximity to patients and are dealing directly with 299 patients' dental care during a potentially highly transmissible respiratory virus. this may explain 300 the higher levels of anxiety among dental students in our study and the overall decrease in anxiety 301 among all medical students following the introduction of online learning. psychological support 302 should be tailored to each students' needs and incorporated into the online remote curriculum. screening university students on a regular basis with tools such as the gad-7 could help faculty 304 in the early identification of highly anxious students and guide students to receive help from 305 targeted interventions that promote psychological well-being, offer counselling, mental health 306 support and coping mechanisms [35] . non-medical students reported higher levels of anxiety before and after online learning in 308 comparison to medical students and whilst medical student anxiety levels decreased following the 309 introduction of online learning, non-medical student's anxiety levels increased. this may be due to the best of our knowledge, this is the first study to provide important information on the initial 331 response and anxiety levels in university students across the uae immediately following the 332 period covid-19 was declared a global pandemic. more than half of university students reported 333 mild to severe anxiety levels with a quarter of students reporting severe anxiety. specifically, 334 medical students reported higher levels of anxiety during their clinical rotations which decreased 335 with the introduction of online learning, yet, non-medical student's anxiety levels increased with 336 online learning. the findings from our study can be used at the government and university level to 337 develop effective screening strategies and to formulate interventions that improve mental health 338 of students. such strategies will reduce unnecessary stress and anxiety among university students, 339 . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 13, 2020. . https://doi.org/10. 1101 /2020 as well as build psychological resilience during the covid-19 pandemic or any other public health 340 emergencies in the future. the authors declare that the research was conducted in the absence of any commercial or financial 343 relationships that could be construed as a potential conflict of interest. is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 13, 2020 . . https://doi.org/10.1101 /2020 cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 13, 2020 . . https://doi.org/10.1101 /2020 . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. the copyright holder for this preprint this version posted may 13, 2020. . https://doi.org/10. 1101 /2020 . cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted may 13, 2020 . . https://doi.org/10.1101 /2020 cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted may 13, 2020. cc-by-nc-nd 4.0 international license it is made available under a is the author/funder, who has granted medrxiv a license to display the preprint in perpetuity. (which was not certified by peer review) the copyright holder for this preprint this version posted may 13, 2020 . . https://doi.org/10.1101 /2020 coronavirus infections-more than just the common 356 2. who. world health organisation director general's opening remarks at the media 358 briefing on covid-19 psychological distress among 364 medical students in conflicts: a cross-sectional study from syria. bmc medical education the global prevalence 368 of anxiety among medical students: a meta-analysis. international journal of environmental 369 research and public health a brief measure for assessing generalized 397 anxiety disorder: the gad-7. archives of internal medicine anxiety disorders in primary 400 care: prevalence, impairment, comorbidity, and detection. annals of internal medicine 16. ibm corp. released 2017. ibm spss statistics for windows va, ny: ibm corp peer mentoring for medical 405 students during covid-19 pandemic via a social media platform middle east respiratory syndrome (mers): comparing the knowledge, attitude and practices 409 of different health care workers media's effect on 22 a nationwide survey of psychological 426 distress among chinese people in the covid-19 epidemic: implications and policy 427 recommendations. general psychiatry functional fear predicts public health 431 compliance in the covid-19 pandemic. international journal of mental health and addiction 2019-ncov epidemic: address mental health care to 435 empower society covid-19 and mental health: a review of the existing literature stabilization and sustainability framework: an integration of the maslow hierarchy of needs disaster medicine and public health preparedness accepting personal restrictions to fight the covid-19 pandemic: results from a danish adult 447 long-449 term psychological and occupational effects of providing hospital healthcare during sars 450 outbreak. emerging infectious diseases psychological 454 responses to the sars outbreak in healthcare students in hong kong prevalence and predictors of ptss 458 during covid-19 outbreak in china hardest-hit areas: gender differences matter immediate psychological responses curriculum versus a case-based integrated curriculum a comparative study of 474 professional student stress mental health of students of medicine: exploraty study in the 477 use of rapid online surveys to assess people's perceptions during 480 infectious disease outbreaks: a cross-sectional survey on covid-19 the authors would like to thank all students who participated in the study. the survey used to collect data for this study is provided as supplementary material. key: cord-028436-ahmpe981 authors: azad, sushmita; chen, binglin; fowler, maxwell; west, matthew; zilles, craig title: strategies for deploying unreliable ai graders in high-transparency high-stakes exams date: 2020-06-09 journal: artificial intelligence in education doi: 10.1007/978-3-030-52237-7_2 sha: doc_id: 28436 cord_uid: ahmpe981 we describe the deployment of an imperfect nlp-based automatic short answer grading system on an exam in a large-enrollment introductory college course. we characterize this deployment as both high stakes (the questions were on an mid-term exam worth 10% of students’ final grade) and high transparency (the question was graded interactively during the computer-based exam and correct solutions were shown to students that could be compared to their answer). we study two techniques designed to mitigate the potential student dissatisfaction resulting from students incorrectly not granted credit by the imperfect ai grader. we find (1) that providing multiple attempts can eliminate first-attempt false negatives at the cost of additional false positives, and (2) that students not granted credit from the algorithm cannot reliably determine if their answer was mis-scored. workplace demand for computing skills [19] has led to large enrollments in introductory programming classes [6] . these courses, however, have had historically large failure rates [2, 29] . some evidence suggests that this is due to a premature emphasis on code writing instead of reading-oriented activities [4, 14, 32] . one important reading skill is the ability to describe the high-level behavior of code [14, 17, 18, 31] . questions to assess this skill-"explain in plain english" (eipe) questions-aren't widely utilized due to the workload of manually grading natural language responses. figure 1 (a) shows an example prompt of one of our eipe questions. in this work, we describe our initial efforts in deploying an nlp-based ai grader for eipe questions and our transition from low-stakes to high-stakes environments. initially, simple nlp-based ai graders were trained using a small amount of survey data collected from course teaching assistants and upperlevel undergraduate computer science students. these simple ai graders were example formative feedback given after student submits answer b fig. 1 . an example mid-semester automated eipe exercise (a) in a python-based intro cs course. after a student submits their answer, they are shown example solutions (b) so that they can learn. non-trivial code fragments are deconstructed so as to show the correspondence between the code and the natural language description. deployed in a low-stakes homework context for which we had two goals: 1) we wanted students to improve their ability to provide natural language descriptions of code, so we provided both immediate correct/incorrect feedback and example correct answers as shown in fig. 1(b) and 2) we wanted to collect additional training data which could be used to train improved nlp-based ai graders. positive results with the homework deployment emboldened us to deploy our ai grader on an exam. to our knowledge, this deployment is unique in the research literature for (imperfect) ai-based graders because it was both high stakes-this question was on one of three midterm exams each worth 10% of students' final grades-and high transparency-the question was graded interactively and students are shown correct answers in a way that permits them to evaluate their submitted answer in light of the correct answers. a high-stakes, high-visibility deployment of an imperfect ai grader, if not well managed, has the potential for student dissatisfaction on a large scale. as such, we wanted to understand what precautions can be taken to prevent students from feeling that they were harmed by such an imperfect grader. to this end, we were willing to tolerate some number of false positives in order to minimize false negatives, and we were willing to employ some manual labor. all things being equal, however, we sought to minimize false positives and the amount of manual labor required. we brain-stormed two strategies to minimize false negatives and, hence, student unrest. first, because our exam was graded interactively on a computer, we could permit students to attempt the question multiple times if the ai grader didn't award them credit on their first attempt. this would hopefully permit students to re-word their answers into a form that could receive credit automatically from the algorithm. second, we could provide students an appeal system where they could, after they are shown the correct answer, request a manual re-grade for an eipe question, if they believed the ai grader had scored them incorrectly. these two strategies led to two corresponding research questions: does providing students with multiple attempts enable false negatives to earn credit without manual intervention? rq2: can students correctly recognize when the ai grader has failed and appropriately appeal for a manual re-grade? our findings can be summarized as follows: 1. the two techniques were effective at avoiding large-scale student dissatisfaction. 2. re-training the ai grader using student responses drawn from the homework deployment improved the accuracy from 83.4% to 88.8%. 3. providing three attempts (at full credit) enabled all first-attempt false negatives to automatically earn credit from the algorithm. it did, however, have the consequence of yielding additional false positives. 4. appeals were useful for morale, but were not effective for distinguishing false negatives from true negatives. 5. students' perception of the grading accuracy of our nlp-based ai grader was lower than that of deterministically-correct auto-graders for true/false, multiple-choice, and programming questions, but only to a modest degree. this paper is structured as follows. section 2 briefly reviews related work. section 3 describes our data collection and ai grader training, while sect. 4 reviews the ai grader's performance and results. we conclude in sect. 5. automatic grading of free response questions is largely split into two areas of focus: automatic short answer grading (asag) and automatic essay scoring (aes). we review briefly the recent work in both areas below. a review of recent, competitive machine learning asag shows only 11% of asag papers were focused on computer science [11] . most of the recent studies are laboratory studies or model evaluations on public or sample data sets [11, 16, 20, 22, 25, 26, 33] . the closest to a high-stakes course exam featured automatic grading for very short answer-defined as four or less wordsquestions, but in a not-for-credit exam-like context rather than on a for-credit exam [23] . the educational testing services (ets) c-rater is deployed for some ets standardized exams, but is not high-transparency and focuses on concept mapping [13, 24] . asag feature selection includes lexical, semantic, syntactic, morphological, and surface features [3, 11, 26] . most recently, dialog based systems and intelligent tutoring systems [20, 22, 25] and end-to-end models have been used for asag [16, 33] . to our knowledge, no asag work has reported on the deployment of ai graders in a high-stakes, high-transparency environment like ours. aes work is more familiar with high-stakes environments. the ets e-rater receives yearly updates and is used in both high-stakes settings like the gre and low-stakes such as the sat online test [21] . however, these systems are not hightransparency as students are provided no means to judge the validity of their scores and there is no process to contest scores. further, aes' major impact is reduction of human labor, with the evaluation of essays focusing broadly on how essay features correlate to human-grader provided marks rather than specific content grading [12] . recent aes approaches include glmms [8] , autoencoders [7] , statistical classifiers [15] , and various deep-learning neural network approaches [1, 9, 10, 27 ]. in fall 2019, we developed and deployed automated eipe questions in an introductory cs course for non-technical majors at a large u.s. university. this 600student course introduces basic principles of programming in both python and excel to a population largely without any prior programming experience. the course was approaching gender balance with 246 women and 355 men. we constructed our eipe ai graders using logistic regression on bigram features. these graders were initially trained with minimal data from a series of surveys. each survey asked participants to provide two correct responses and two plausible incorrect responses for each of the eipe questions. these surveys were completed by the course's instructor and tas and a collection of upper-level cs students who were compensated with an amazon gift card for each survey. these surveys resulted in approximately 100-200 responses per question. survey data was manually reviewed by a research team member to perform any necessary re-categorization of the responses. this survey-data-trained ai grader was deployed on four homework assignments during the first half of the semester. the questions were deployed using the prairielearn [30] online learning platform, the course's primary assessment system. each assignment included a pool of 10-12 eipe questions, and each time a student attempted a question they were given a random draw from the pool. to tolerate the ai grader's inaccuracy in this low-stakes, formative context, students could attempt the activity as many times as they wanted; points were granted for any correct answers with no penalty for incorrect answers. as such, any false negatives would only delay (rather than prevent) students from getting points. furthermore, the ai graded eipe questions were one of many activities on the students' weekly assignment, and they could ignore the activity completely and earn the week's homework points through answering other questions instead. we next deployed the auto-graded eipe questions as one of 24 questions on a proctored, computer-based mid-term exam in the 12th week of the course (also run using prairielearn). we selected the pool of eipe questions deployed on the homework during the 5th week of the course. prior to deployment, two members of the research team manually labeled the students' homework responses to these questions and used as additional training data to improve the grader. the ai graders deployed on the exam were trained with 500-600 labeled responses per question. four of the problems in the pool were not included on the exam because they exhibited a noticeable difference in difficulty from the rest. students were randomly assigned one of the remaining eight problems. students were given three attempts to submit a correct answer, receiving correct/incorrect feedback on each submission and were shown correct answers (as shown in fig. 1(b) ) once all attempts had been used or their answer was scored as correct. the students submitted a total of 1,140 responses. after the exam was completed, for the purpose of this research, two members of the research team familiar with the course content independently scored each response without knowing the ai grader's score. for any responses where these two scores matched, the score was considered the final ground truth. final ground truth for the remaining responses was established by a process of discussion and reconciliation between both scorers and a third research team member until consensus was reached. necessary grade corrections were made for all students who had incorrectly been denied credit. all further analysis in this paper has been done on this set of 1,140 auto-graded exam responses. to understand how students perceived the accuracy of auto-graded eipe questions as compared to other types of auto-graded questions, we asked students to fill out a survey in the week after the exam with the eipe question. using a 1-5 likert scale, students were asked: "for each type of question, rate it based on how reliably accurate you feel the grading for that kind of question is". comparing ai grader and human performance. 51% of students had their eipe question scored as correct by the reconciled human graders, and the ai grader achieved an accuracy of 89%, with a 12% false positive (fp) rate and a 9% false negative (fn) rate. we used cohen's kappa to compare the inter-rater reliability of humans and the ai grader. cohen's kappa between the two experienced human graders was 0.83 ("almost perfect" agreement [28] ) and between the ai grader and the ground truth (reconciled human graders) was 0.74 ("substantial" agreement [28] to understand how much training data is needed for obtaining a reasonable ai grader and whether there is a qualitative difference between survey data and student homework data, we trained graders with different subsamples of data and show the mean of the grader's performance in fig. 2 . there are three main sources of training data: (1) a subset of the survey data, (2) a subset of the student homework data, and (3) both, meaning all of the survey data and a subset of the student homework data. although more data consistently lead to better performance, the student homework data seems qualitatively better than survey data, suggesting that the course staff and senior students creating the survey data were only somewhat able to generate realistic training data. students perceived the grading of ai graded eipe questions as being less accurate than that of other kinds of questions to a statistically significant degree (p < 0.001). compared to the nextlowest question type (programming), code-reading questions were d = 0.48 standard deviations lower, a "medium" effect size [5] . mean likert scores for each type of question are shown in fig. 3 with 95% confidence intervals. we failed to find any correlation between students' perception of the eipe ai grader and whether it mis-graded their answers on the exam. instead, a student's perception of accuracy for all kinds of questions is weakly correlated with the student's performance on that kind of question (mean r = 0.22). how reliably accurate was the grading for each kind of question? fig. 3 . responses to a survey question auto-grader accuracy by question type. choices were from 1 = "very unreliable" to 5 = "very accurate". multiple-attempt accuracy. we need to differentiate between the ai grader's performance on a single student submission versus the net performance over all student submissions to a question. to describe the latter, we define the multi-attempt-k outcomes as shown in table 1 . whenever we use terms like false positive (fp) without the prefix of "multi-attempt", we are referring to the performance on a single-submission level. table 1 . definitions of "multi-attempt" terminology. multi-attempt-k true positive within the first k attempts, student submits at least one correct answer and ai grader awards points for some submission multi-attempt-k false positive within the first k attempts, student submits no correct answer but the ai grader awards points for some submission we visualized how multiple attempts impact the performance metrics in fig. 4 . we see that as students attempted the question more times (moving from ma-1 to ma-3), the true positive rate increased somewhat (93.2% to 97.7%), but at the expense of a substantially higher multi-attempt false positive rate (14.9% to 32.9%). the reference roc curve is for the ai grader evaluated on only the first-attempt responses, and we see that the multi-attempt performance is always worse than this. trajectories with multiple attempts. figure 5 shows the trajectories students took through the multiple attempts at the eipe questions. this reveals several features. first, all students who were falsely graded as incorrect (fn) on table 1 ). the blue roc curve is for the ai grader on the first-attempt data only. (color figure online) the first attempt were able to use the multiple attempts to eventually be graded as correct (as tp or fp). a majority (73%) of these students needed a second attempt to be graded correct, and only 27% needed three attempts. second, students who were falsely graded as incorrect (fn) re-attempted the question at a higher rate than students who were truly graded as incorrect (tn) (100% versus 96%, p = 0.013). third, the ratio of falsely-graded incorrects (fn) to truly-graded incorrects (tn) decreased as students used more attempts (4.7% to 3.2%, p = 0.015). strategies with multiple attempts. students marked as incorrect by the ai grader on either first or second attempt deployed two correction strategies: (1) reword, where students rephrased their previous answer, and (2) change, where students submitted a response different in meaning from their previous answer. figure 6 plots the paths through these strategies taken by the student population. from a standpoint of strategy selection, we see that students who had an actually-correct answer (fn) used the reword strategy at a higher rate than students who did not (tn) (57% vs 42%, p = 0.022). considering strategy effectiveness, we observe that for fn students the reword strategy was more successful for receiving points than the change strategy, but not significantly so (75% versus 25%, p = 0.22), whereas for tn students the change strategy was significantly more effective (81% vs 19%, p = 0.036). appeals to human graders. out of the 203 students who were graded as incorrect by the ai grader, 69 appealed for a human re-grade and 4 of these were truly correct (rate of 5.8%). among those that did not appeal, 3 were truly correct (2.2%). the difference in rates of true-correctness was not statistically significant between students who appealed and those that did not (p = 0.20). these initial results suggest automatically grading "explain in plain english" (eipe) questions may be a simpler task than other asag contexts. even using just bigrams, our results (accuracy of 88.78%) are competitive with other asag results using much more sophisticated algorithms. we believe that this high accuracy is the result of specific elements of disciplinary vocabulary (e.g., "count", "even") being effective markers of when students have correct answers. it is not surprising that the student homework responses were more effective than survey data for training the algorithm to predict student exam responses. the surveys did enable us to deploy the algorithm in the low stakes homework context to collect that homework training data, but our conclusion is that we could get by with fewer survey responses, especially if we were to quickly score early homework responses and re-train the model. while students' perception of accuracy of our nlp model was statistically significantly below their perceptions of accuracy for the other question types, we were surprised by how small the difference in perceptions was. in our minds, the deterministic autograders and our nlp model are categorically different things. the students rated the deterministic autograders much lower than we anticipated (means near 4 out of 5) and the nlp model only d = 0.48 standard deviations below the deterministic autograders. while the answer to rq1-does providing students with multiple attempts enable false negatives to earn credit without manual intervention?-is yes, there are a number of caveats. first, while all first attempt fn students automatically earned credit on subsequent attempts, a few did so through submitting fp answers, which will potentially hinder those students' learning. second, rather than merely reword their answer, many students used the multiple attempts to submit conceptually different answers. that is, while fn students primarily used the multiple-attempt feature to rephrase their answer for clarity (as intended by us), tn students appear to be aware that they don't know the answer, and used the multiple-attempt feature as a way to take more "shots in the dark", changing their answer in the hope that they'd strike the correct response and gain credit. because some of these "shots" resulted in fp, giving students multiple attempts negatively impacted the fp rate. this distinction between rewording and changing answers is important, because they have different implications on how much credit a student should receive. a student whose answer was correct, but needed rewording to be accepted by the algorithm, presumably deserves full credit. in contrast, a student that hedges by changing their answer on each submission, probably has a more fragile understanding and may deserve only partial credit. if we were to use multiple attempts again, we would probably: 1) provide only two attempts, since the majority of fns were able to self correct within by their second try, and 2) have a small penalty (10-30%) for credit earned on a second attempt. that said, in our current implementation providing a single attempt and just shifting the implementation along its roc curve may provide a better fn/fp trade-off. the answer to rq2-can students correctly recognize when the ai grader has failed and appropriately appeal for a manual re-grade?-appears to be no. students that appealed had a statistically equivalent rate of being correct as the whole population of students that didn't earn credit from the algorithm. relying on students to self report appears to be an inequitable strategy that rewards "noisier" students. one important caveat is that appeals were evaluated in a context with multiple attempts; appeals could be more useful in a singleattempt context where more fns are present. in short, in this first report on strategies for deploying imperfect ai graders in high stakes, high visibility contexts, we found that our strategies were ultimately successful. there was no obvious student discontent and only 0.5% (3 out of 600) of students would have incorrectly not received credit (fn) had we not manually scored all responses. while our strategy was passable, there remains a lot of opportunity for improvement. because perfect auto-graders will not be achievable for many important problems, it is important to explore hybrid ai/human systems that can mitigate algorithmic shortcomings with minimal manual effort. automatic text scoring using neural networks failure rates in introductory programming the eras and trends of automatic short answer grading patterns and pedagogy statistical power analysis for the behavioral sciences computing research association: generation cs: computer science undergraduate enrollments surge since autoencoders for educational assessment automated summarization evaluation (ase) using natural language processing tools augmenting textual qualitative features in deep convolution recurrent neural network for automatic essay scoring automatic features for essay scoring -an empirical study machine learning approach for automatic short answer grading: a systematic review automated language essay scoring systems: a literature review c-rater: automated scoring of short-answer questions further evidence of a relationship between explaining, tracing and writing skills in introductory programming evaluating machine learning approaches to classify pharmacy students' reflective statements automatic short answer grading via multiway attention networks relationships between reading, tracing and writing skills in introductory programming explain in plain english' questions: implications for teaching a machine learning grading system using chatbots understanding mean score differences between the e-rater® automated scoring engine and humans for demographically based groups in the gre® general test sentence level or token level features for automatic short answer grading?: use both very-short-answer questions: reliability, discrimination and acceptability c-rater: automatic content scoring for short constructed responses improving short answer grading using transformer-based pre-training automatic short answer grading and feedback using text mining methods a neural approach to automated essay scoring understanding interobserver agreement: the kappa statistic failure rates in introductory programming revisited prairielearn: mastery-based online problem solving with adaptive scoring and recommendations driven by machine learning an australasian study of reading and comprehension skills in novice programmers, using the bloom and solo taxonomies a theory of instruction for introductory programming skills automatic chinese short answer grading with deep autoencoder acknowledgments. this work was partially supported by nsf due-1347722, nsf cmmi-1150490, nsf due-1915257, and the college of engineering at the university of illinois at urbana-champaign under the strategic instructional initiatives program (siip). key: cord-029292-imt8451i authors: d’antoni, loris; helfrich, martin; kretinsky, jan; ramneantu, emanuel; weininger, maximilian title: automata tutor v3 date: 2020-06-16 journal: computer aided verification doi: 10.1007/978-3-030-53291-8_1 sha: doc_id: 29292 cord_uid: imt8451i computer science class enrollments have rapidly risen in the past decade. with current class sizes, standard approaches to grading and providing personalized feedback are no longer possible and new techniques become both feasible and necessary. in this paper, we present the third version of automata tutor, a tool for helping teachers and students in large courses on automata and formal languages. the second version of automata tutor supported automatic grading and feedback for finite-automata constructions and has already been used by thousands of users in dozens of countries. this new version of automata tutor supports automated grading and feedback generation for a greatly extended variety of new problems, including problems that ask students to create regular expressions, context-free grammars, pushdown automata and turing machines corresponding to a given description, and problems about converting between equivalent models e.g., from regular expressions to nondeterministic finite automata. moreover, for several problems, this new version also enables teachers and students to automatically generate new problem instances. we also present the results of a survey run on a class of 950 students, which shows very positive results about the usability and usefulness of the tool. computer science (cs) class enrollments have been rapidly rising, e.g., cs enrollment roughly triples per decade at berkeley and stanford [12] or tu munich. we thank emil ratko-dehnert from prolehre tum for the professional help with the student survey; tobias nipkow and his team for allowing us to conduct the user survey in his class; christian backs, vadim goryainov, sebastian mair and jan wagener for the exercises they added as part of their bachelor's theses; julia eisentraut and salomon sickert-zehnter for their help in developing this project; the tum fund "verbesserung der lehrmittelsituation" and the cav community for caring about good teaching. loris d'antoni was supported, in part, by nsf under grants cns-1763871, ccf-1750965, ccf-1744614, and ccf-1704117; and by the uw-madison ovrge with funding from warf. both online and offline courses and degrees are being created to educate students and professionals in computer science and these courses may soon have thousands of students attending a lecture, or tens of thousands following a massive online open course (mooc). at these scales, standard approaches to grading and providing personalized feedback are no longer possible and new techniques become both feasible and necessary. current approaches for handling this growing student volume include reducing the complexity of assignments or relying on imprecise feedback and grading mechanisms. simpler assessment mechanisms, e.g., multiple-choice questions, are easier to grade automatically but lack realism [8] . designing better techniques for automated grading and feedback generation is therefore a necessity. recent advances in formal methods, including program synthesis and verification, can help teachers and students in verifiably correct ways that statistical or rule-based techniques cannot. for example, formal methods have been used to identify student errors and provide feedback for problems related to introductory python programming assignments [17] geometry [9, 11] , algebra [16] , logic [2] , and automata [3, 6] . in particular, for this last topic, the tool automata tutor v2 [7] has already been used by more than 9,000 students at more than 30 universities in north america, south america, europe, and asia. in this paper, we present automata tutor v3, an online 1 tool that extends automata tutor v2 and uses techniques from program synthesis and decision procedures to improve the quality and effectiveness of teaching courses on automata and formal languages. besides being part of the standard cs curriculum, the concepts taught in these courses are rich in structure and applications, e.g., in control theory, text editors, lexical analyzers, or models of software interfaces. concrete topics in such curricula include automata, regular expressions, context-free grammars, and turing machines. for problems and assignments related to these topics automata tutor v3 can automatically: (1) detect whether the student's solution is correct. (2) detect different types of student's mistakes and translate them into explanatory feedback. (3) if possible, generate new problems together with the corresponding solutions for teachers to use in class. automata tutor v3 greatly expands its predecessor automata tutor v2, which only provides ways to pose and solve problems for deterministic and nondeterministic finite automata constructions. this paper describes the new components introduced by automata tutor v3 and how this new version improves on its previous one. the key advantages to its competitors are the breadth, automatic generation and grading of exercises, infrastructure allowing for use in large courses and a useful feedback to the students, compared to text-based interfaces used by autotool [13] , rudimentary feedback in jflap [14] and none in gradience [1] . since automata tutor has already been well received by teachers around the world, we believe that the readers from the cav community will find great value in knowing about this new and fundamentally richer version of the tool and how it can extensively help with teaching the automata and formal languages courses, a task we know many of the attendees have to face on a yearly basis. our contributions are the following: -twelve new types of problems (added to the four problems from the previous version) that can be created by teachers and for which the tool can assign grades together with feedback to student attempts. while the previous version of automata tutor could only support problems involving finite automata constructions, automata tutor v3 now supports problems for proving language non-regularity using the pumping lemma, building regular expressions, context free grammars, pushdown automata and turing machines, and conversions between such models. -automatic problem generation for five types of problems, with the code modularity allowing to add it for all the others. this feature allows teachers to effortlessly create new assignments, or students to practice by themselves with potentially infinitely many exercises. -a new and improved user interface that allows teachers and students to navigate the increased number of problem types and assignments. furthermore, each problem type comes with an intuitive user interface (e.g., for drawing pushdown automata). -an improved infrastructure for the use in large courses, in particular, incorporating login systems (e.g. ldap or oauth), getting a certified mapping from users to students and enabling teachers to grade homework or exams. -a user study run on a class of 950 students to assess the effectiveness and usability of automata tutor v3. in our survey, students report to have learned quickly, felt confident, and enjoyed using automata tutor v3, and found it easy to use. most importantly, students found the feedback given by the tool to be useful and claimed they understood more after using the tool and felt better prepared for an upcoming exam. in our personal experience, the tool saves us dozens of thousands of corrections in each single course. automata tutor is an online education tool created to support courses teaching basic concepts in automata and formal languages [7] . in this section, we describe how automata tutor helps teachers run large courses and students learn efficiently in such courses. learning without automata tutor. figure 1 schematically shows a studentteacher interaction in a course taught without an online tutoring system. the teacher creates exercises, grades them manually, and (sometimes) manually provides personalized feedback to the students. this type of interaction has many limitations: (1) it is asynchronous (i.e., the student has to wait a long time for what is often little feedback) and does not scale to large classrooms, posing strenuous amount of work on teachers, (2) it does not guarantee consistency in the assigned grades and feedback, and (3) it does not allow students to revise their solutions upon receiving feedback as the teachers often release a solution to all students as part of the feedback and do not grade new submissions. another drawback of this interaction is the limited number of problems students can practice on. because teachers do not have the resources to create many practice problems and provide feedback for them, students are often forced to search the internet for old exams and practice sheets or even exercises from other universities. due to the lack of feedback, this chaotic search for practice problems often ends up confusing the students rather than helping them. learning with automata tutor. figure 2 shows the improved interaction offered by automata tutor v3. here, a teacher creates the problem instances with the help of the tool. the problems are then posed to the students and, no matter how large a class is, automata tutor automatically grades the solution attempts of students right when they are submitted and immediately gives detailed and personalized feedback for each submission. if required, e.g. for a graded homework, it is possible to restrict the number of attempts. using this feedback, the students can immediately try the problem again and learn from their mistakes. as shown in a large user study run on the first version of automata tutor [6] , this fast feedback cycle is encouraging for students and results in students spontaneously exploring more practice problems and engaging with the course material. additional practice is supported by the automatic problem generation, with the same level of detailed and personalized feedback as before without increasing the workload of the teacher. furthermore, automatic problem generation can assist the teacher in creating new exercises. finally, whenever necessary, the teacher can download an overview of all the grades. improved user interface. automata tutor is an online tool which runs in the most used browsers. a new collapsible navigation bar groups problems by topic, facilitating quick access to exercises and displaying the structure of the course (see figure 6 in [5, appendix b]). to create a new exercise, a teacher clicks the "+"-button and is presented the view of fig. 3 . in this case, the drawing canvas allows to easily specify the sample solution pushdown automaton. similarly, when students solve this exercise, they draw their solution attempt also on the canvas. after submitting, they receive their personalized feedback and grade (see example in fig. 4) . for the automatic problem generation, a dropdown menu to select the problem type and a slider to select the difficulty is displayed together with the list of all problems the user has generated so far (see the screenshot in figure 7 in [5, appendix b]). while automata tutor can be used for independent online practice, one of the main advantages is its infrastructure for large university courses. to this end, it is organized in courses. a course is created and supervised by one or more teachers. together, they can create, test and edit exercises. the students cannot immediately see the problems, but only after the teachers have decided to pose them. this involves setting the maximum number of points, the number of allowed attempts as well as the start and end date. to use automata tutor, students must have an account. one can either register by email or, in case the university supports it, login with an external login service like ldap or oauth. when using the login service of their university, teachers get a certified mapping from users to students and enabling teachers to use automata tutor v3 for grading homework or exams. students can enroll in a course using a password. enrolled students see all posed problems and can solve them (using the allowed number of attempts). the final grade can be accessed by the teachers in the grade overview. in this section, we list the problem types newly added to automata tutor v3. they are all part of the course [10] and a detailed description of each problem can be found in [5, appendix a], including the basic theoretical concept, how a student can solve such a problem, what a teacher has to provide to create a problem, the idea of the grading algorithm, and what feedback the tool gives. finding words in or not in the language of a regular expression, context free grammar or pushdown automaton. re/cfg/pda construction: given a description of a language, construct a regular expression, context free grammar or pushdown automaton. re to nfa: given a regular expression, construct a nondeterministic-finite automaton. myhill-nerode equivalence classes: there are two subtypes: either, given a regular expression and two words, find out whether they are equivalent w.r.t. the language, or, given a regular expression and a word, find further words in the same equivalence class. pumping-lemma game: given a language, the student has to guess whether it is regular or not and then plays the game as one of the quantifiers. find derivation: given a context free grammar and a word, the student has to specify a derivation of that word. cnf: given a context free grammar, the student has to transform it into chomsky normal form. cyk: given a context free grammar in cnf and a word, the student has to decide whether the word is in the language of the grammar by using the cocke-younger-kasami algorithm. while to tm: given a while-program (a turing-complete programming language with very restricted syntax), construct a (multi-tape) turing machine with the same input-output behaviour. automatic problem generation (apg) allows one to generate new exercises of a requested difficulty level and problem type. this allows students to practice independently and supports teachers when creating new exercises. while apg is currently implemented for four cfg problem types and for the problem type "while to tm", it can be easily extended to other problem types by providing the following components: -procedure for generating exercises at random either from given basic building blocks or from scratch. -a "quality" metric qual (e) for assessing the quality of the generated exercise e, ranging from trivial or infeasible to realistic. -a "difficulty" metric diff (e) for assessing the difficulty of e. given these components, automata tutor generates a new problem with a given minimum difficulty d min and maximum difficulty d max as follows. firstly, 100 random exercises are generated. secondly, automata tutor chooses exercises e with the best quality such that d min ≤ diff (e) ≤ d max . concretely, for the cfg problem types, cfgs with random productions are generated and sanitized. resulting cfgs that do not accept any words or have too few productions are excluded using the quality metric. the difficulty metric always depends on the number of productions; additionally, depending on the exact problem type, further criteria are taken into account. for the problem type "while to tm" we use an approach similar to the one suggested in existing tools for automatic problem generation [15, 18] : we handcrafted several base programs which are of different difficulty level. in the generation process, the syntax tree of such a base program is abstracted and certain modifying operations are executed; these change the program without affecting the difficulty too much. e.g. we choose different variables, switch the order of if-else branches or change arithmetic operators. then several programs are generated and those of bad quality are filtered out. a program is of bad quality if its language is trivially small or if it contains infinite loops; since detecting these properties is undecidable, we employ heuristics such as checking that the loops terminate for all inputs up to a certain size with a certain timeout. automata tutor v3 is open source and it consists of a frontend, a backend, and a database. it also provides a developer's manual for creating new exercises. the frontend, written in scala, renders the webpage. the drawing canvases for the different automata and the turing machines rely on javascript. the frontend and backend communicate using xml objects. the backend, written in c#, contains methods to unpack the xml of the frontend to compute the grade and feedback for solutions. it is also used to check the syntax of exercises and for the automatic problem generation. it relies on automatadotnet 2 , a library that provides efficient algorithms for automata and regular expressions. the database keeps track of existing users, problems and courses. it uses the h2 database engine. all the new parts of automata tutor v3 were developed and tested over the last 3 years at tu munich, where they were used to support the introductory theoretical computer science course. this local deployment served as an important test-bed before publicly deploying the tool online at large scale. due to its modular structure, the tool is easily scalable by having multiple frontends and backends together with a load distributor. this approach has successfully scaled to 950 concurrent student users; for this, we used 7 virtual machines: 3 hosting frontends, 3 hosting backends (each with 2 cores 2.60 ghz intel(r) xeon(r) cpu and 4 gb ram), and 1 for load distribution and the database (with 4 such cores and 8 gb ram). we will scale the number of machines based on need. large-class deployment. in the latest iteration of the tu munich course in 2019, we used automata tutor v3 (in the following denoted as at) in a mandatory homework system for a course with about 950 students; the homework system also included written and programming exercises. in total, we posed 79 problems consisting of 18 homework and 61 practice problems. the teachers saved themselves the effort of correcting 26,535 homework exercises, and the students used at to get personalized feedback for their work 76,507 times. on average, each student who used at did so 107 times. at the end of the course, we conducted an anonymized survey, based on the system usability survey [4] . 14.6% of the students in the course answered the survey, which is an ordinary rate of return for an online questionnaire, especially given that there was no incentive. the students were given statements to judge on a likert scale from 1 to 5 (strongly disagree to strongly agree). we define "the students agreed with the following statement" to mean that the average and median scores were at least 4 and less than 10% of the students chose a score below 3. dually, if the students disagreed with the statement with median and average score that was at most 2 and less than 10% having a score greater than 3, we say that they "agreed with the negation of the statement". for all statements that do not satisfy either of the criteria, we report mixed answers. the full survey results can be found in [5, appendix c] . usability. regarding the usability of the tool, the students agreed with the following statements: -i quickly learned to use the at. -i do not need assistance to use the at. -i feel confident using the at. -the at is easy to use. -i enjoy using the at/the at is fun to use. however, there were lots of valuable suggestions for improvements, many of which we have implemented since then. moreover, the survey also revealed space for improvement, in particular for streamlining as documented by the following statements where the answers were more mixed: -the at is unnecessarily complex. -the canvas for drawing is intuitive. -the use of at is self-explanatory. usefulness. regarding how useful at was for learning, the students agreed with the following statements: -i understand more after using the at. -i prefer using the at to using pen and paper exercises (12.9% disagreed, but median and average are 4). -the feedback of the at was helpful and instructive. -the exercises within the at are well-designed. -the at fits in well with the programming tasks and written homework. -the at did not hinder my learning. -i feel better prepared for the exam after using at. -the feedback of the at was not misleading/confusing. note that there are no statements with mixed or negative answers regarding the usefulness. additionally, as shown in fig. 5 , when we asked students about their preferred means of learning, at gets the highest approval rate, being preferred to written or programming exercises as well as lectures. overall, this class deployment of automata tutor v3 and the accompanying surveys were great successes, and showed how the tool is of extreme value for both students and teachers, in particular for such large a course. this paper presents the third version of automata tutor, an online tool helping teachers and students in large automata/computation theory courses. automata tutor v3 now supports automated grading and feedback generation for a wide variety of problems and, for some of them, even automatic generation of new problem instances. furthermore, it is easy to extend and we invite the community to contribute by implementing further exercises. finally, our experience shows that automata tutor v3 improves the economical aspects of teaching greatly as it scales effortlessly with the number of students. earlier versions of automata tutor have already been adopted by thousands of students at dozens of schools and we hope this paper allows automata tutor v3 to help even more students and teachers around the world. gradiance online accelerated learning automatically generating problems and solutions for natural deduction automated grading of dfa constructions sus-a quick and dirty usability scale automata tutor v3. corr how can automatic feedback help students construct automata? automata tutor and what we learned from building an online teaching tool national research council: how people learn: brain, mind, experience, and school: expanded edition synthesizing geometry constructions introduction to automata theory, languages, and computation solving geometry problems using a combination of symbolic and numerical reasoning why are english majors studying computer science? the leipzig autotool system for grading student homework. functional and declarative programming in education (fdpe enhancing jflap with automata construction problems and automated feedback generating dfa construction problems automatically automatically generating algebra problems automated feedback generation for introductory programming assignments problem generation for dfa construction key: cord-032012-lmlvbfrl authors: bucholz, e. k. title: creating a welcoming and engaging environment in an entirely online biomedical engineering course date: 2020-09-17 journal: biomed eng education doi: 10.1007/s43683-020-00024-x sha: doc_id: 32012 cord_uid: lmlvbfrl in the spring of 2020, brick and mortar colleges had to abruptly adapt to the reality of covid-19 and transition to entirely online environments in a manner of weeks. this required a rapid (< 2 weeks) acquisition of knowledge and flexibility in using technology, most commonly zoom. upon completion of the semester, and after debriefing with numerous faculty teaching required courses across the biomedical engineering department at ____ university, the most common comments were that the online engagement and the online environment were nothing close to the traditional classroom environment: students were less likely to show up; less likely to enable their cameras so you could see their faces; and less likely to ask or answer questions in class. to address these issues, in this paper we make easily implementable recommendations that will increase engagement and improve the inclusivity of online courses that can be applied to any biomedical engineering course offered in an online environment. student engagement is especially important for promoting student learning as has been well documented in education research (taylor and parsons in curr. issues educ, 2011. retrieved from http://cie.asu.edu/; armstrong in the best schools: how human development research should inform educational practice. association for supervision and curriculum development (ascd), alexandria; 2006). the four recommendations for improving the engagement in the online classroom are: (1) make it a priority to stress the importance of having their cameras on during lecture and explain why it’s important; (2) have a “daily question” at the start of lecture where all students respond in chat followed by a brief discussion; (3) make use of the polling feature in zoom to frequently assess student understanding and upon discovering student confusion on any given question, send them to breakout rooms to discuss, incorporating a quick active learning exercise that can help reduce the monotony of online lecture; and (4) create handouts that are circulated prior to class for students to take different pieces of a particular problem, solve them in their teams in individual breakout rooms, and then discuss the content and results back in class with all teams together. these techniques were shown to increase engagement and attendance and fostered a welcoming environment in the online classroom, contributing positively to student experience and in turn allowing students to achieve student learning outcomes despite the difficult circumstances. electronic supplementary material: the online version of this article (10.1007/s43683-020-00024-x) contains supplementary material, which is available to authorized users. in the spring of 2020, brick and mortar colleges had to abruptly adapt to the reality of covid-19 and transition to entirely online environments in a manner of weeks. this required a rapid (< 2 weeks) acquisition of knowledge and flexibility in using technology, most commonly zoom. this adjustment was fraught with technological, academic, and pedagogical challenges. traditional brick and mortar engineering faculty are not trained nor accustomed to providing significant online content and the fast turnover rate meant faculty showed up for the first day of online teaching underprepared. after debriefing numerous faculty across the biomedical engineering department at duke university following the conclusion of the spring 2020 semester, most faculty noted that the online engagement and the online environment was nothing close to the traditional classroom environment: students were less likely to show up; less likely to have their cameras on so you could see their faces; and less likely to ask or answer questions in class. faculty mentioned anecdotally that students were so disengaged that when class was over, many students failed to leave the zoom classroom even 30-45 min after class had completed, evidence that they had left the physical room a long time ago and simply did not notice that class had finished. these are problems that faculty have never experienced in an in person class. several classes did not have these problems and from these experiences we propose easily implementable recommendations to other faculty to improve the engagement and inclusivity of their respective biomedical engineering courses. student engagement is especially important for promoting student learning as has been well documented in education research. 1, 6 we will examine one such class to understand how these techniques can be implemented and the outcomes they created. bme 303l is taught mostly to juniors in the spring semester and had 65 students in the spring of 2020. as with many other faculty, due to covid-19, the course transitioned to online learning in the middle of the semester. on the first day of online class, the faculty member teaching the course, upon seeing all the blank windows with names but no faces, asked students to turn their cameras on. the system setup for the faculty member was such that she could view both the material she was screen sharing as well as the students faces. such a setup is imperative as if faculty are not able to see their students' faces they are missing all of the visual cues they normally receive in the traditional classroom setting; without video faculty are literally flying blind and unable to adapt to their audience. upon request, students quickly turned on their cameras and the professor reminded them every day for the first several weeks, making it an initial focus and explaining to students why it was important, until it became a habit and expectation that the students knew their camera should be on and the request did not need to be made anymore. as has been well documented when utilizing active learning techniques or flipped classroom pedagogies, explaining the rationale for a particular request goes a long way with getting student buy-in. 5 after debriefing with other faculty who taught required courses in the spring of 2020, it was clear that if over 50% of students turned their cameras off, it became a positive feedback loop where fewer and fewer students turned their cameras on, a form of peer pressure with each student not wanting to stand out. therefore, it is imperative that faculty focus on getting as many students as possible to turn their cameras on for the duration of lecture and that expectation is set at the outset of the course (and perhaps written in the syllabus). while the faculty member does not under any circumstances recommend shaming, or calling students out for not turning on their cameras, explaining the reasoning behind wanting cameras to be on should go a long way to allay fears as well as allowing students an opportunity to communicate their privacy concerns to the instructor either privately or anonymously. it should also be mentioned that students may have limited access to the internet in their home environments and it should be the responsibility of the university to do their best to survey students' needs and provide broadband internet and/or prioritize university housing to those who are particularly at a disadvantage. in the fall, the faculty member intends to survey all the students in class prior to the start of the semester and for students who answer that they are unable to turn on their video, the faculty member plans to have the tas monitor the zoom site and write down the names of the students who fail to turn their video on. the faculty member then will reach out to the students to set up special zoom office hours to check in on these students as a group to help make sure they feel part of the classroom community. the faculty member will also encourage students to post a static picture of themselves so they can at least put a face to their voices, which will help the faculty member recognize students. having been the unfortunate recipient of a ''zoombombing'' this past spring, making sure the zoom classroom is secure from attacks can give students additional comfort that their privacy is being protected. after the incident, the faculty member created waiting rooms and required user authentication to access the classroom, essentially only allowing students who had a university provided netid to access the course. as an added benefit, only allowing students who have university credentials to login to zoom also makes it easier to pre-assign breakout rooms. this strategy would be problematic for faculty wishing to have lots of guest lectures, or faculty who needed to login to zoom with multiple devices. the first day this security was implemented students also had trouble accessing the classroom, something faculty members should anticipate. at the start of every lecture, to facilitate open dialogue with students and to work on the online classroom dynamic, the faculty member initiated a ''daily question'' and expected students to answer the question in the chat window. examples of daily questions are shown in table 1 . most of the questions were positive, focusing on mental wellness, and encouraged open ended answers from students. over 33% of students typically responded and students then began to chat and encourage each other with follow ups, etc. oftentimes birthdays were mentioned as were special projects students had completed that they were particularly proud of. using chat enabled students to see everyone's responses and for more than one student to respond at a time, which is a limitation of talking in an online environment such as zoom. often the faculty member would have a short (< 2 min) conversation asking students follow up questions about their statements and that significantly improved the dialogue between students and the faculty member. once the daily question was implemented, over 85% of the students who attended left their video cameras on and felt comfortable being seen by their peers and professor. in addition, after the daily question was implemented more students asked questions and felt comfortable volunteering answers than prior to implementation. while the emphasis in the spring was on mental health and overall well being given the stress of covid-19, it is anticipated that this technique could be employed not just to build community and open communication in an online environment, but also to encourage students to reflect on their learning at the outset of class with more learning and course specific focused questions, examples of which have been provided in the table but are italicized as they were not implemented in the spring. other techniques initiated in the online classroom were polls that asked multiple choice questions about the material being covered using the zoom poll tool. while in a normal environment, the faculty member would have used iclicker, in the online environment it was better to use the built in zoom tool as it reduced the number of applications both the faculty and students had to interface with, thus reducing complexity. to set this up, the faculty member prepopulated questions in the zoom poll prior to every class, which required visiting the zoom room prior to the start of the online lecture. while the responses could not be recorded for the students or attributed to any particular student, the faculty member could determine how many students were participating (generally > 95% of attendees) and it was a good way to ascertain if everyone was following along with the content. if a large percentage of the students were incorrect in answering the question, the faculty member would send the students to breakout rooms where they were tasked with convincing their peers they knew the correct answer. they were then brought back into the main room and the poll was offered again to see how many minds were changed. this allowed students to make a guess, receive immediate feedback from their peers, and then find the correct answer. most of the time, students were able to arrive at the correct answer after the breakout rooms, but occasionally more discussion would be needed to explain the correct answer to all students. this method can be implemented in any biomedical engineering course to determine how well students are grasping the content and it is an easy, low stakes way to encourage students to interact with the material and each other, deepening their knowledge of the course content. the faculty member also made frequent use of the breakout rooms. ahead of time, the faculty member would prepare a.pdf (included in supplemental materials) that she would then email to students and post on the course piazza site. in the handout, there were typically assignments for the students to complete in their respective breakout rooms. for example, one breakout room team would need to simulate the relaxation curve with a set of constants, while another breakout room team would simulate the same relaxation curve with a different set of constants. students then were asked to draw their resultant graphs on the.pdf using the annotate function in the zoom tool and describe them to the class. this was then discussed with the whole class and students were then given another set of constants to simulate another condition and sent back to their breakout rooms. in this way, the students were kept engaged and active most of the time during lecture, much like they were in the traditional classroom. active learning and creating an inclusive and welcoming environment have long been known to be important contributors to students achieving the learning outcomes specified by the course. 3 ,2 students that feel valued and appreciated in class work harder and feel more connection to the material and are more motivated to learn. 4 these personal connections to other students in the class and the faculty member teaching it are possibly the most valuable piece of the learning experience as learning is not unidirectional or solely communication of content, but rather a process that is give and take; that encourages interaction and exploration of the content with their peers and the faculty member as it encourages students to work higher up the blooms taxonomy pyramid. with many courses by necessity starting online in the fall, creating a welcoming environment that is 100% online for the duration of the semester will be all the more challenging since there will be no face to face time as was the case in the spring semester. after reflecting on the good and the bad of the spring 2020 semester, there are many ways we can improve the experience for the students to do a better job of teaching them in the new reality we currently find ourselves in. prior to the start of the online class session, much time was devoted to thinking about ways for students to actively engage with the course in the new virtual environment. by implementing these recommendations, the faculty member had very high attendance for the course (> 80% or more than 52 out of 65 enrolled students came to class every session). in addition, around 90% of the attendees or around 45 of the 52 students who attended class had their video on. for the polls, the professor got over 95% of the students participating as could be determined by the number of answers received in the zoom poll software. with the daily question, usually only 30% of the students responded with their answers before students began to comment on one another's responses and people stopped answering the question and instead asking follow up questions or congratulating others. even though not everyone wrote a response, it was an olive branch that made students feel more relaxed and comfortable in the classroom environment and is an easy daily activity that can open students up to learning and make them more likely to ask and answer questions, fostering a positive work environment for the students. while in the spring the faculty member was focused on the overall well-being of the students, in the fall, with students more adjusted to the new normal, instructors could consider asking a bme specific course question that encourages reflection and dialogue about the course content and examples of questions that might be used are included in italics in table 1 . there are several things that could be improved or streamlined for the following semester. first, the faculty member spent significant time preparing zoom polls for the course which involved going to the zoom room that was to be used and prepopulating it with the polls for the day. later the faculty member realized she could simply have one default poll with answers a, b, c, d, and e and then give the question and answer in either powerpoint or write it on the doc cam or handout she had provided to the students in electronic form. this would also allow the faculty to adapt on the fly and ask questions she hadn't planned to ask, as she sometimes does in in person class. this would strike a nice balance and would reduce the preparation required for each online class. for larger classes, the breakout rooms used should be assigned prior to the start of class. for the first several weeks of online class, the faculty member randomly assigned them every time in class to streamline the process and in a large online class (65 students) that was not as successful as having more regular groupings of students. with such a large class, students would interact with one student and by random probability never interact in a breakout room with that student again. for classes less than 20, assigning breakout rooms randomly could be beneficial as all students would get a chance to interact with one another and would help foster a sense of community, which is developed through frequent interactions. for students to have a positive breakout room experience, they do need to develop familiarity with one another and so encouraging ice breakers at the start, particularly in an entirely online course would be beneficial. towards the end of the spring semester, the faculty made breakout rooms in zoom ahead of time that were the same for the last 3 lectures. the students commented that this was a positive change and resulted in better collaboration in the breakout rooms. in the fall version of the online class, which is anticipated to be 50 or more students, the faculty member expects to have students pick seats and assign groups at the beginning of the semester based on the seating arrangements which is how it would function in a normal classroom environment. the faculty member is going to ask students to come up with a team name relating to the course content in the fall and then name the breakout rooms for the rest of the semester. in this way the faculty will be able to call on students of one cohort but naming the teams will make them more memorable for both students and faculty. while there is benefit to having students mix, ultimately students what is one thing you haven't missed as much as you thought you would? what is the best thing you did yesterday? what are you most excited about today? what was the most surprising thing that happened yesterday? what is your favorite time/activity of the day? what do you appreciate now that you didn't appreciate as much prior to covid-19? what would be your most useful advice for other students on the homework due in several days? what have you found most interesting about fourier transforms (e.g. fill in topic here)? how do you think the topics we are covering in class will affect your choice of occupation (it's okay to be honest)? why do you think, we make you learn about convolutions (e.g. fill in topic here)? felt less comfortable opening up to people they didn't know and their work product was affected when they were in random groups of students they were unfamiliar with. in smaller classes, random assignment might work but in larger classes it was possible that students only work one time with a given student, which is not long enough for them to form a relationship or develop trust. during the first part of the spring 2020 online experiment, assigning breakout room teams was not possible using the zoom tool through sakai but given recent developments that is likely to have changed. there were also some technical challenges with the ta assisting in the breakout rooms. making the ta host instead of co-host would allow the ta to help assign/adjust breakout rooms instead of the faculty member who was busy fielding questions from the in class exercise and conducting lecture. another improvement that would help is instead of sending.pdfs to the students, the faculty member in the fall plans to send a google slides link during class for two example activities that have been proposed. in the first activity, students would access the table and complete their problem with their teammates at the same time as everyone is filling in the table. after the breakout rooms, the whole class would look at the answers and see if they agree with their classmates. in the 2nd proposed activity, in their breakout rooms, students would scroll to their team slide and plot the solution of the discrete convolution requested. students would edit their respective google slide for their breakout room and answer any questions and then when the teams are brought back into the main zoom classroom, the faculty member can quickly scroll through the plots and responses for the different problems and students can explain their reasoning. in addition, all students would have access to these google slides for studying purposes. students often mention they want more practice problems and given that each breakout room would have a slightly different problem, this would provide additional materials that were curated by other students and verified in class. the feedback the faculty member received for the spring 2020 was very positive. students commonly mentioned the welcoming, friendly environment and one student mentioned coming to class felt like coming to a ''family gathering.'' students by and large appreciated the synchronous content both for how it made the days more routine and because it was good to see other people as it made the class feel more like a normal class. when asked to use one word to describe the overall course the most popular answer was interesting 4 followed by rewarding, 5 challenging, 5 difficult, 5 enlightening, 5 and fun 5 to name a few of the most common answers given. making the changes recommended in this paper in an online class will significantly enhance the classroom dynamic in the fall semester which in turn will allow students to achieve the learning outcomes of the course by creating a positive work environment where students are open and receptive to learning which has been well documented to increase student performance and retention of material. the best schools: how human development research should inform educational practice. alexandria: association for supervision and curriculum development the role of engagement in inspiring teaching and learning. innovations in education and teaching international increasing student engagement in higher education student perceptions of active learning flipped learning: a guide for higher education faculty. sterling: stylus publishing improving student engagement. curr issues educ publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations the online version of this article (https://doi.org/10. 1007/s43683-020-00024-x) contains supplementary material, which is available to authorized users. key: cord-263423-ac46njml authors: iqbal, muhammad zafar; alradhi, hussain ibrahim; alhumaidi, abdulsalam abdulkarim; alshaikh, khalid hussni; alobaid, abdulraouf mohammed; alhashim, mohammed taher; alsheikh, mona hmoud title: telegram as a tool to supplement online medical education during covid-19 crisis date: 2020-06-17 journal: acta inform med doi: 10.5455/aim.2020.28.94-97 sha: doc_id: 263423 cord_uid: ac46njml introduction: instant messaging applications are known for their potential to enhance learning. however, to date, there has been little investigation into the implications of the ‘telegram’ application for online medical education and training. aim: this study explores the potential benefits and disadvantages of integrating telegram into undergraduate medical education during the covid-19 pandemic. methods: an exploratory study was conducted between march and may 2020, with 203 undergraduate medical students recruited through purposive sampling. data was collected through seven open-ended questions, followed by thematic analysis using atlas.ti. results: most students participating in this study reported multiple utilities of the application, including easy access to educational resources and the ability to add unlimited members, as well as files in all formats and sizes. additionally, the application assisted students to engage with collaborative learning, maintain their wellbeing, and ensure their security. however, they also reported some drawbacks, i.e., a complex interface, information overload, and a tendency to distract them, causing time wastage. conclusion: this study concludes that telegram provides an effective mobile learning platform for medical students during the current crisis, in particular. moreover, it offers considerably more functionalities and fewer potential drawbacks than alternative applications. the novel covid-19 pandemic has resulted in widespread disruption for all sectors, including education (1) . the implementation of emergency lockdowns and social distancing has led to campus-based education being suspended in most parts of the world. these unforeseen circumstances have challenged medical institutes, both globally and regionally, to rethink ways of ensuring the smooth delivery of high-quality education, resulting in a paradigm shift from face-to-face to online teaching and learning (2) . teachers are now using a variety of technology-based modalities, i.e., zoom, microsoft teams, google meet, adobe connect, webex and skype to continue their students' education, depending upon their convenience and institutional resources (3) (4) (5) . alongside these web-based programs, additional support is being provided by instant messaging and social media applications to streamline medical education during the current crisis. smartphone applications (i.e., whatsapp, youtube, linkedin, messenger, and facebook) (6) known for improving students' clinical decision making, working efficiency and performance in examinations, are already well integrated into medical education and practice (7) (8) (9) . a relatively new application, known as telegram, has recently gained in popularity among medical students as a resourceful educational platform. telegram is a free, cloud-based messaging application capable of being used on both smartphones and desktops. the application's official website (www. telegram.com) states that it allows its users to create groups and channels formed of as many as 200,000 members. the application also facilitates the sharing of files of multiple formats, including powerpoint presentations, audios/videos, pdfs, and word documents. much has been previously written concerning the role of instant messaging applications for supplementing educational practices (10) (11) (12) . however, an in-depth exploration of the utility of telegram as a mediating tool for online teaching and learning has not been reported yet. its increased use during the ongoing pandemic has highlighted the need to explore and document both the merits and demerits of telegram to obtain evidence and enable its users to make informed decisions (13). this study, therefore, examined the perceptions of medical students concerning the utility of telegram in online education. the research question guiding this project was as follows: what is the impact of the online application telegram on the education of students during the covid-19 pandemic? this exploratory study was conducted between march and may 2020 in the college of medicine of imam abdulrahman bin faisal university, saudi arabia. the college of medicine has, in response to the covid-19 pandemic, been completely transferred its courses to online delivery. moreover, all students have been advised to install and use the telegram application, as this is being used to create official class groups and channels to facilitate distance learning. homogenous purposive sampling (14) was undertaken to recruit volunteers, focusing on undergraduate students of our medical college. as this was not an experimental study, we only recruited those students who already had telegram installed on their smartphones and excluded all students with no prior familiarity with the application. after undertaking a thorough literature review, we developed an inventory of seven open-ended and semi-structured questions to examine students' perceptions in a more meaningful manner. the inventory was then pilot tested on five students to determine the relevance, face validity and clarity of the questions. the responses received from pilot testing were then used to make minor corrections. see appendix 1 for the list of validated survey questions. procedure using the validated inventory, an online survey link was developed and distributed by means of ques-tionpro® (survey analytics llc, oregon, usa). we collected the data over four weeks, and a parallel template analysis was conducted, using both inductive and deductive coding until saturation was achieved (15) . qualitative data were analyzed using atlas.ti (version 8; sci-entific software development, berlin, germany) qualitative data analysis software, whereas the number and frequency of responses were calculated through microsoft excel 2019. ethical approval for the study was obtained from the institutional review board of imam abdulrahman bin faisal university, under the irb # irb-ugs-2020-01-168. an informed consent was also obtained electronically from all participants, agreeing to the use of the collected data for the anonymous dissemination of information. a total of 698 students were approached, of which 203 (29.03%) agreed to participate. the majority of the participants (n=81, 39.90%) had been using telegram for over five years, while n=59 (29.06%) had been using it for between two and five years and n=63 (31.03%) for less than two years. in addition, 27.58% had between one and five channels for educational purposes, while 61.08% had between six and ten, and 11.33% had over ten channels. the participants reported multiple educational benefits of the application, as discussed below. the most frequently reported benefit was the ease of access to educational resources, with the most frequently accessed being: (1) lecture slides (77.2%); (2) ebooks (66.3%); (3) online question banks (44.6%); (4) websites (33.7%); (5) notes (31.7%); (6) educational podcasts (15.8%); (7) discussion forums (14.9%); (8) mind-mapping (9.9%); (9) journal articles (8.9%); (10) the saudi commission for health specialties (national) guidelines (6.9%); and (11) clinical calculators (3%). the students valued the diversity offered by the application to support distance learning, with one stating: i like the app very much. we have special channels where my class fellows and teachers regularly upload content in different formats. when the app notifies me, i go through the reading material, all in one place. unlimited sharing capacity another distinctive feature of telegram noted by students was that (unlike other applications) it allows the unlimited addition of group members and file sharing without any limitations of size or type. furthermore, within the group, each member is given administrative rights, enabling all students and teachers to send and receive educational resources freely. one participant commented: we often have large video lectures, presentations and graphics, which require more than the allowed space in whatsapp. then i prefer switching to telegram because it has no size limit. communicating with peers many of the participants stated that they found telegram beneficial for communicating with their colleagues and class leaders, in particular, when it came to obtaining information regarding timing, schedules, and announcements. the students also highlighted the usefulness of channels, informing that they used specific channels to access information and create polls to make decisions. one student said, wellbeing many students stated that their teachers maintained their engagement through different educational activities by posting quizzes and informative material on the class channel. the students also noted that the application helped them to pursue collaborative peer learning, which prevented them from feeling isolated. one student shared: we have created a group in telegram where we share information and study material. we also use this group to support each other emotionally because of the corona situation. security the students stated that the application's encrypted chats and the ability of messages to self-destruct made them feel secure. this feature gave them the liberty to be extroverts and freely discuss a topic or problem without worrying about making mistakes. additionally, the majority of the female students said that they felt safer using this application than many others, as it allows anonymous access to channel resources. one student noted: the drawbacks alongside the advantages of telegram, the participants also highlighted some disadvantages which are capable of having a negative influence on student learning. some students found the application a source of disturbance and distraction while studying, with one stating: it's a good thing that the app has so much to offer, but sometimes it is annoying too, especially when i am not using it. i keep receiving notifications from groups and channels, which distract me a lot and waste my time too. some students also reported that they found telegram to be complex and that its interface is difficult and lacks user-friendliness. a small number of students also criticized the app for causing information overload. this study explored the use of the telegram application to facilitate online medical education during the current global pandemic. the results identified that the application was generally perceived as being beneficial, with multiple utilities reported by the respondents. the features most frequently highlighted were: the ease of accessing education resources; the ability to add unlimited members; the facility for uploading or downloading files in all formats and sizes; the ability to undertake peer collaboration; the promotion of wellbeing; and security. the findings of this current study accord with that of faramarzi et al. (16) who reported similar utilities of telegram when exploring its feasibility for the distance teaching of language courses. however, one significant difference is that the current participants reported the resource sharing feature more valuable than stated in other studies (16) (17) (18) . a plausible reason for this finding could be the nature of medical education and training that demands a wide range of learning resources. in addition, the telegram channels also enabled the students to search for journal articles and ebooks by subscribing to different libraries and without the need for support from other browsers, websites and/or applications (18, 19) . this feature not only facilitates student learning but also lessens the need for the university library. a further benefit of this application, particularly for female students, was user anonymity while accessing channel resources. this finding could be attributed to local cultural norms that place particular emphasis on female privacy and modesty (20) . this study acknowledges that several other applications (i.e., whatsapp and facebook messenger) also have comparable functionalities to telegram. however, these have some limitations, including restrictions on the number of group members, limited file sharing capacity and type, and the need to link with other accounts (7, 12, 16, 18) . these limitations may have result in the participating students expressing a preference for telegram over other instant messaging applications. the current crisis arising from the covid-19 global pandemic has put all activities on hold, which has resulted in many challenges for students, including maintaining their psychological wellbeing. in such testing times, students need to remain interconnected and continue to engage with educational activities, to prevent any negative psychological impacts (21) . our own study showed that the dedicated class groups within the application enabled the students to communicate with each other and faculty staff regarding the curriculum, schedules, examinations, and learning queries. this feature has the potential of reducing the communication gap between students and their teachers and preventing student isolation in the current lockdown. however, further research is also required to test the full benefits of mobile learning for engaging students and maintaining their psychological wellbeing. this study contains a number of limitations. since the study was conducted in a single institution, its results may not be generalizable for other educational contexts. this study explored only the merits and demerits of telegram in online pedagogy. further research is required to establish whether the application's educational use can also be supported through the lens of educational theories. a further limitation of this study concerns the low response rate, which could be attributed to the current covid-19 crisis. however, since we achieved data saturation, we do not see any negative impact of the low response rate on our findings. this study concludes that telegram offers more functionalities and fewer potential drawbacks than many other applications. following the principles of constructivism, the application supports learners in scaffolding new information on prior knowledge, instead of relying as a tool to supplement online medical education during covid-19 whatsapp messenger as a tool to supplement medical education for medical students on clinical attachment social network sites as educational factors whatsapp enhances medical education: is it the future? effectivity of e-learning through whatsapp as a teaching learning tool the role of whatsapp® in medical education: a scoping review and instructional design model managing resident workforce and education during the covid-19 pandemic: evolving strategies and lessons learned comparison of convenience sampling and purposive sampling the utility of template analysis in qualitative psychology research telegram: an instant messaging application to assist distance language learning mobile learning: innovation in teaching and learning using telegram evaluating the educational usability of telegram as an sns in esap programs from medical students' perspective using social network of telegram for education on continued breastfeeding and complementary feeding of children among mothers: a successful experience from iran cultural knowledge of non-muslim nurses working in saudi arabian obstetric units covid-19 pandemic: impact of quarantine on medical students' mental wellbeing and learning behaviors the use of social media in graduate medical education: a systematic review ten steps to complex learning: a systematic approach to four-component instructional design preparedness and readiness of countries in the eastern mediterranean region technology enhanced assessment (tea) in covid-19 pandemic curriculum delivery in medical education during an emergency: a guide based on the responses to the covid-19 pandemic advantages, limitations and recommendations for online learning during covid-19 pandemic era covid-19 : considerations for medical education during a pandemic mobile technologies in medical education: amee guide no use of smart phones and social media in medical education: trends, advantages, challenges and barriers completely upon teachers (6, 12, 22) . however, as reported by some students, too much information can also cause a cognitive overload that can potentially impede learning (23) . thus, telegram's users need to maintain a balance if they wish to achieve the optimal learning benefits from the application. we advocate that telegram provides an effective mobile learning platform, particularly during the current crisis. in this appendix, we provide the list of validated questions which were used for data collection.1. for how long you have been using telegram? 2. do you use telegram for studying, chatting, or both? 3. how many channels do you have for educational purposes? 4. which educational resources do you access in telegram groups and channels? 5. do you see any potential use of telegram in your online education during the covid-19 pandemic? if yes, then please explain them 6. do you see any potential drawbacks of telegram in your education? if yes, then please explain them 7. is there any unique feature in telegram that supports your learning more other applications?• acknowledgement: acknowledgments: we would like to acknowledge students who participated in our study• conflict of interest: nothing to report• financial support: none• disclaimer: we confirm that the submitted article has not been published in parts or full and that it has not been submitted simultaneously for publication elsewhere. l. key: cord-353886-wvxohfum authors: sevilla vallejo, santiago; ceballos marón, natalia andrea title: theoretical and applied study of the psychological and educational effects of lockdown in primary school students in argentina date: 2020-12-31 journal: social sciences & humanities open doi: 10.1016/j.ssaho.2020.100039 sha: doc_id: 353886 cord_uid: wvxohfum abstract emotions are complex states, which have a fundamental role for children’s mental health and learning. the proper self-regulation of emotions allows to tolerate frustrations, improves social skills and healthy bonds with peers and adults in the environment (bisquerra, 2003, p. 12). the lack of regulation causes discomfort that leads to a decrease in academic performance (graziano, reavis, keanes & calkins, 2006) and particularly in reading comprehension, because this competence requires motivation and cognitive processes that will allow to process the linguistic code, interpret it and represent it symbolically (alonso-tapia, 2005, p. 64). in this work, we show the essential relationship between emotional regulation and reading comprehension in primary school students. after that, we study the effect of lockdown on these two processes. the aim is to have preliminary information about the challenges and effects of the current situation. to do this, a semi-structured survey has been conducted of parents and teachers in order to assess the psychological and educational effects of the current health situation. the results of 40 semi-structured telephone surveys are analyzed, 20 of parents of children with learning disorders and 20 of their classroom teachers in traslasierra, córdoba-argentina. the results inform that the students and their parents have difficulties to regulate their emotions and this emotional struggle has a negative impact on comprehension of texts. the problems that previously presented in these two areas have increased due to the lack of adaptation of activities by teachers, the lack of knowledge of their parents to help them and the difficulty in accessing digital technologies. both parents and teachers express that lockdown affects learning processes, although parents are also concerned about the effects on social life. emotions are complex states that have a fundamental role for children's mental health and learning processes. to express emotions is to communicate to others what we are feeling at a certain moment, a task that is often not easy (ramos, 2011, p. 15) . a suitable way of expressing emotions is called self-regulation of emotions. this allows children to tolerate frustrations, improve social skills and healthy bonds with peers and adults in the environment (bisquerra, 2003, p. 12) . in early childhood, emotional regulation starts out precarious and depends entirely on protective adults. around the age of 8, cognitive development makes children achieve self-regulation in a more conscious way, although early experiences and the established attachment to parents will continue to be very important (p aez, fern andez, campos, zubieta and casullo, 2006; garnesfski, rieffe, jellesma, terwogt & kraaij, 2007) . therefore, a good cognitive development will be key in the childhood. campos (2010) expresses that emotional processes give rise to a good brain functioning and executive components, and they will be the ones who promote the ability to regulate, reason, memorize, make decisions in learning processes (p aez, fern andez, campos, zubieta and casullo, 2006) . in this work, we show the relationship of these psychological processes with linguistic performance. according to the results obtained by graziano, reavis, keanes & calkins (2007) and alonso-tapia (2005, p. 64) , the lack of regulation causes discomfort that leads to a decrease in academic performance and particularly in reading comprehension (ceballos-mar on & sevilla-vallejo, 2020). this is because this competence requires motivation and cognitive processes that will allow the linguistic code to be processed, interpreted and represented symbolically. then, we will see how the current situation of lockdown or quarantine and preventive and compulsory social isolation affect both the regulation of emotions and the reading comprehension specifically of primary school students with learning disorders. 1 we are concerned about the importance of dealing with emotions to help students to improve in human and academic terms. this is even more necessary to be considered when we work with students having learning disorders. the aim of this study is to know in which sense emotional regulation and reading comprehension are affected by lockdown. it is essential to have some guidelines to improve the psychological and educational care of students. a comparison will be made between the gaze of the teacher and the student's parents. this study is justified by the collaboration between teachers and families which has become especially important in the current situation. both the former's work and the latter's accompaniment have been transformed into a situation of social isolation. the teachers are in charge of the academic training and students' human development. in the family case, parents play a fundamental role in children's life and development because they can further promote an integral children's growth not only in their affective development but also in their regulation of their emotions. besides, they can accompany academic learning. regarding to learning, children's case with difficulties in academic learning, it is of great importance that parents have confidence that they will be able to achieve it and improve themselves day by day (ramos, 2011, p. 30) . this study will focus on the effects of the health crisis in argentina, because we have access to students, parents and teachers there. this allows us to collect testimonies of emotional regulation and reading comprehension to offer results that allow interventions in the current situation. likewise, a comparative study of the responses to telephone interviews given by parents and teachers will be offered to analyze the effects on emotions and students' reading comprehension. this will allow us to know to what extent parents and teachers are finding ways to collaborate to overcome this crisis. teachers and psychologists must realize the complex relations between emotional regulation and reading comprehension. the former affects wellbeing and learning processes. the latter is an essential skill for any subject and it is important beyond education. we will start showing the relation between the necessary processes to regulate emotions and the necessary processes to read. in the case of children, processes such as motivation, self-esteem, and emotional regulation depend largely on the bond and attachment to the primary figures (p aez, fern andez, campos, zubieta, and casullo, 2006, p. 340) . so, it must be taken into account that the current situation also has an impact on parents who are living, at this particular moment, an atypical situation that affects them emotionally, physically and financially. we consider that at this uncertain moment the consequences for students could be reflected especially in reading comprehension, because the tasks that involve it demand a great deal of concentration. reading is a complex task that requires adequate emotional regulation in order to be carried out satisfactorily. first, reading involves active meaning-building processes such as interpreting, discriminating, classifying, examining, criticizing, contrasting, and constructing representations of the information received (sevilla vallejo, 2018, p. 5) . secondly, the following must be born within the mind: "motivation and processes are the two pillars on which understanding is based 2 " (alonso-tapia, 2005, p. 2). motivation will be given by the interests of the students (sevilla vallejo, 2017, p. 291) . reading can be a source of discovery for them: "our cognitive and emotional engagement with fiction is to a high degree and emotional engagement with fiction is to a high degree dependent on surprise" (nikolajeva, 2016, p. 36) . according to jesús alonso tapia, motivation depends on the beliefs that readers have not only regarding the objective that they should achieve but also regarding what understanding implies. the teaching community is aware of the necessity to promote active reading among students and for this, we must keep in mind that "we always read with a purpose" (alonso-tapia, 2005, p. 14). the student needs to regulate his or her own emotions to maintain reading motivation and to carry out the cognitive processes involved. as panadero and alonso-tapia (2014 p. 454) have pointed out, although this model incorporates emotion to a lesser extent than cognition, it represents the psychological elaboration process necessary to manage complex tasks, such as reading. alonso-tapia (2005) adapts the previous model to the linguistic levels of reading: 1. identification of graphic patterns. graphemes are discriminated and are associated with their respective phonemes to recognize the words. it depends on learning factors, such as the identification of distinctive features of the letters or the practice of identifying them; but it is also related to neurological aspects such as "the necessity for the inhibition of perceptual representation mechanism to be intact, if this inhibition does not occur, the images overlap" (p. 67). 2. recognition of the lexicon. this factor is about the amount of vocabulary, the closeness or familiarity of that vocabulary and the semantic context in which it is used and the use of that context. 3. construction and integration of the meaning of the sentences. the meaning is constructed in cycles that correspond to the phrases. in these, each representation is connected and integrated with the previous one, as well as with prior knowledge. it depends on the recognition of the syntactic structure of the phrase (subject, predicate and hierarchical relations). 4. integration of the cycle. the ideas of each proposition or phrase must be integrated in a coherent way. this would form a last set of ideas that would synthesize all the others that are called macro-propositions. to achieve this integration, one must rely on thematic progression to establish an understandable relationship between ideas. 5. construction of the global meaning of the text. finally, the ideas contained in the text must be integrated and simplified. for this, some strategies can be followed: selection: set aside the sentences that do not provide necessary information or are accessory. generalization: substitute specific phrases for more general ones that include them. construction: substitute one series of sentences for another, which does not appear in the text, but which synthesizes the content of the others. this theory allows us to divide the problems in reading comprehension into the levels of construction of the text (recognition of signs and understanding of words, sentences, ideas and of the text as a set of ideas). up to this moment, there are no scientific studies on the effects of lockdown on reading comprehension; however, we hypothesize that this situation increases previous difficulties at any of these levels. although the concision of this study does not allow us to delve deeper, it is evident that emotional regulation and reading comprehension are two related and educationally relevant aspects that have been influenced by lockdown. nevertheless, we would like to provide some preliminary information about how this situation provokes a particular sense of lack or emotional regulation and increases problems of reading comprehension in specific linguistic levels. as we mentioned before, in the case of argentina, the national government decided to suddenly implement that each student receive at home the same activities that they would have in their classrooms. as a result, teachers have to accompany their students in a virtual way, without in many cases having previous experience in 1 in a previous work, we studied the relation between emotional awareness and reading comprehension: el efecto del aislamiento social por el covid-19 en la conciencia emocional y en la comprensi on lectora. estudio sobre la incidencia en alumnos con trastornos de aprendizaje y menor acceso a las nuevas tecnología. revista internacional de educaci on para la justicia social, 9(3), número extraordinario. 2 all translations are the author's. digital technologies and in didactic procedures to do so. suddenly, our society experienced physical restrictions, which implied compulsory social isolation or lockdown in order to avoid massive contagions. family routines underwent a profound change, among which is that children must take classes at home. teachers send the activities to the families through different virtual means that have been stipulated by the school institutions and the different teachers. as psychiatrists and psychologists who attend to this crisis situation have pointed out, this alteration causes feelings of discomfort at the individual and social level, physical and emotional disorders such as anxiety, anguish, fear, ocd … (diario p agina 12, 2020). specifically, maría de los angeles l opez geist, psychoanalyst, expresses the following: "there are immeasurable loneliness of children and adolescents due to parents overburdened with work or lack thereof." 3 it also states that children have a childish view of their parents, which means that they do not trust that their parents are capable of protecting them and this in turn causes suffering and aggressive reactions from children. it is necessary for parents to manage their emotions and accompany their children in this process in order to avoid negative emotions that affect their personal and school life. our own teaching and psychological experience tells us that the difficulties to acquire the reading comprehension with confidence and quickness are related to the difficulties presented in the regulation of emotional states, because these affect behavior, bonds and social skills and, especially, regulation of the cognitive processes necessary for reading. now, we should ask ourselves what is happening at this special moment with those children who have some type of learning disorder and are at home receiving tasks from teachers. it is worth asking if they learn and they continue a similar process that they would have had at the school in normal circumstances. the international classification of diseases describes in its latest version, icd 11, learning disorder as a significant and persistent disorder in academic skills that include reading, writing and arithmetic. the intellectual performance of the child with this disorder is under what is expected at their chronological level, causing deterioration at the academic level at the present time and possible professional difficulties in the future. this problem usually appears during the first school years and it is not due to intellectual, sensory or neurological causes. ramos (2011) also adds that children with these difficulties have problems in expressing emotions, thoughts and behaviors, as well as in understanding society when the context is changing (p. 16). it is relevant to mention that, in the case of argentina, on april 1, the national disability agency authorized the relaxation of social isolation for boys and girls diagnosed with mental, cognitive and psychosocial disabilities, as well as those with autism. that resolution became void hours later and it is expected that the ministry of health and cabinet will be able to take measures in this regard. on a national broadcast, on april 10, the president commented that this decree is being studied at this moment to achieve its possible implementation, since it will decrease the emotional consequences on the mental health of children (diario p agina 12, reprint april 12, 2020). this is an example of the significant importance of isolation for students with learning disabilities. regarding the use of technologies, it can be specified that the national government of argentina encourages all students to own computers and thus achieve inclusion in a more formal education system, ensuring that everyone achieves general knowledge about the use of computer tools to give a new way of thinking and reflecting. the economic differences that hinder access to new technologies are very important for educational training (finquelievinch, 2000) and are having a decisive influence on the current situation of social isolation. in the province of c ordoba, a platform was announced for teachers to work with students at different levels. it aims to continue school activities from home through the website. 4 this is updated weekly and has essential contents for students (government of c ordoba, 2020). these tools are not only intended to establish the means for the interaction of teachers, students and their families, but also, they transform the identities of students and they also modify their access to texts because it is opened up the possibility of dialogical education: this particular theory of dialogic education implies an ontological vertical dimension of growth in education from a monologic ontology at one extreme and towards a dialogic ontology at the other extreme. a monologic ontology assumes identities with locations and boundaries. a dialogic ontology, on the other hand, asserts that every apparent identity is in dialogue with every other apparent identity (wegerif et al., 2017, p. 2). the current situation presents an educational challenge, but also an opportunity for an approach in which the student investigates and builds knowledge. the digital technologies can allow dialogic learning to be implemented because the forms of interaction are expanded. in this way, it could be possible for students to regulate their own emotions and access to reading. however, we are going to study how the current situation can also hinder the proper use of technologies to achieve these objectives. this study consists of 40 interviews, 20 with parents with children with learning disorders and 20 with their classroom teachers. in addition, students have comorbid disorders: 7 of them also have asd, 6 adhd, 2 epilepsy, 2 behavioral disorders, 2 fragile x disorders, 1 language disorder and 1 childhood psychosis. the sample is form out of the patients attended in traslasierra, c ordoba-argentina. it is not expected that this sample represents the situation of argentina because it would be necessary to have a bigger number of cases and to choose them from different areas. however, we consider that it is more relevant to have some preliminary results during the current crisis rather. this study could help to wider and more rigorous following studies. we will not either differentiate between these comorbid diagnoses. we simply point them out to indicate that these are students who require very careful educational and emotional attention. we developed a semi-structured survey with questions about the effects on emotional regulation and reading comprehension (appendix a). the survey is composed by 13 questions. questions 1 to 3 ask about the general effect of lockdown and the support that students receive from the therapeutic team or special education teacher, the school and parents. questions 4 and 5 are about the emotional effects of lockdown on students and their parents. it was decided because we wanted to know in depth the situation of families. we have the hypothesis that the lockdown is making more difficult the emotional regulation of those people who are living together. primary students are influenced by the emotions of their parents. questions 6 to 12 are related to the educational adaptation to current situation and the lockdown effects on reading comprehension. this section ranges from quite open questions to more precise questions to know parents and teachers' concerns of the general situation and the specific effect on linguistic competence. finally, question 13 is open to other comments to receive other information that we could have missed in the previous questions. the objective is to compare the views from teachers and parents in order to propose ways to improve collaboration. we are going to analyze the results according these sections. the first question of the survey asked for the implications of lockdown. the two most frequent responses of parents are delayed learning 3 "hay soledades inconmensurables de niños y adolescentes por padres sobrecargados de trabajo o por falta del mismo". 4 the domain is the following: https://tuescuelaencasa.isep-cba.edu.ar/. (8 cases) and lack of social bonds (8 cases). in other words, these two problems have an equivalent incidence in the sample taken. to a lesser extent, parents transmit problems regarding routines (2 cases) and see an improvement in assignments performance in the situation (2 cases). in other words, the first ones find it difficult to manage study routines and understand the educational needs of their children. they express the lack of tools to help their children ("i don't understand how to help him" 5 ) and also that they have needed a long period of adaptation until they could contribute to the education of their children ("only now [20 days have passed] i understand how to help them to download the content and to enjoy it" 6 ). however, the second ones have children who find it difficult to adapt to the educational center, and therefore they see a benefit in studying at home. this is, for example, the case of students diagnosed with asd, in which it is found difficulties in admission and permanence in the school. on the other hand, when teachers are asked the same question, they consider that the greatest problem is the delay in learning (13 cases) and, secondly, the social bond. in other words, parents are equally concerned with academics and social issues, while teachers give more importance to the former. in relation to support from teachers, 17 consider that there is accompaniment, and 3 that there is not. on the other hand, 13 of the teachers consider that families are accompanying their children with their homework, 5 comment that only some families do it and 2 do not answer. this indicates that the majority of families perceive school accompaniment and the majority of teachers receive family accompaniment. this is striking in relation to what has been said about the comprehension problems and emotional difficulties mentioned above. regarding the knowledge that teachers have about their students, 12 answered that they do recognize all the students in the classroom, 7 answered that they know some of them and 1 answered that she does not know her students. most of the teachers comment the following: "we only attended 2 weeks of classes and we have been in virtual classes for a month". 7 the teachers who answer that they do know their students express that they were able to internalize the diagnoses of those who have a school integration process and the rest were unable to read the documentation. the teachers indicate the difficulty of the situation due to the short time they had to work with their students. regarding the question of how their children felt, after approximately 20 days of quarantine, from the total parents, 25% answered that their children are "exhausted", "frustrated" and "irritable"; another 25% "distressed" and with feelings of uncertainty, 15% "sad"; 20% "happy" and another 15% "calm" and "bored". 8 in addition, parents who expressed that their children were frustrated or distressed commented that the children presented maladaptive behaviors: "he cries silently", "constantly asks if he and the whole family are going to die", others refuse all the proposals from therapists and teachers: "he doesn't want to do anything", "he is distressed, i don't know what to do" 9 . they also express that they experience other disorders related to emotions: sleep problems ("they fall asleep late", "they sleep until afternoon"), anxiety ("they bite their nails") and eating disorder ("they eat more quantity", "skip meals" 10 ). regarding parents' emotions, 40% of parents answered that they feel sad and/or distressed, 30% they are frustrated, exhausted and feel collapsed, 15% feel good, 10% do not know what is emotionally happening to them and 5% felt bad but it is improving. likewise, 65% of them answered that these emotions affect their children, 20% that it does not affect because "i try not to seem worried" 11 and 15% answered that they do not know. this is coherent with the negative emotions of the students which were discussed previously. most of the parents have emotional regulation problems, since only 3 of them have managed them properly. these difficulties in regulating their emotions have a negative impact on their children. in turn, it coincides with what the teachers have stated: 70% express that they perceive the families stressed and/or exhausted, 5% scared, 5% want to leave, another 5% who are calm and 15% do not know how they feel. that is, negative emotions in parents predominate both according to parents themselves and according to the teachers. regarding the accomplishment of the tasks, the majority of parents (70%) express that their children find it difficult to have to carry them out at home with their help and only 20% consider that their children are able to solve them without difficulty. avoidance behaviors can be observed: the majority answered that their children "are apathetic", "they get angry", 12 they avoid sitting down and doing their homework, they make excuses: "she opens the folder and gets sick because it is difficult. she wants to go to school." 13 these answers don't coincide with the ones given by the teachers, who express to a greater extent that the students have adapted to the new modality partially (40%), completely (25%), while the rest do not know if they adapt (25%) and only 10% consider that the students have not adapted. when asked if the children understand the task that their teachers send, 65% families answer no, 20% of them do understand it and 15% do it but with difficulty. families who answer that their children do not understand the activities to be carried out ask the special education teacher to explain it to them and then transmit it to their children. the problem is that the teachers wrote on the blackboards in capital letters and the students worked on copying and, at the current situation, parents express that they do not have the necessary tools such as blackboards and do not know how to adapt the tasks or how to teach their children. on the contrary, many teachers send activities to a whatsapp group without such adaptation. most of the students are not used to practice dictation with their parents, so they express ideas such as "we don't have a blackboard at home", "they don't know how to read the letters". 14 on the other hand, teachers consider that students do not read with enough attention the tasks that are sent to them. we also asked parents about the actions they take when they don't understand the tasks. 50% answered that they consult with the special education teacher, 30% ask close relatives and neighbors, 15% search the internet and only 5% of them ask their questions to the classroom teacher. therefore, parents ask more frequently to the special education teacher and it is striking that, they ask to lesser extent to classroom teachers. from 100% of parents, 45% answered that children did not read at home before social isolation, 40% of them did read and 15% only on some occasions. during quarantine, this behavior has increased in 65% of cases and it hasn't changed in 35% of cases. before quarantine, reading at home was not very frequent, since 45% cases is almost half of the sample; and, 5 "yo no entiendo c omo ayudarlo". 6 "reci en ahora [pasaron 20 días] me doy cuenta c omo bajar el contenido y lo disfrute". 7 "asistimos s olo 2 semanas de clases y llevamos un mes de clases virtuales". 8 5 respondieron "agotado", "frustrado" e "irritable"; 4 "angustiado" y con sentimientos de incertidumbre, 3 de ellos "triste"; 4 tienen días en que se sienten bien y días mal; 4 "alegres" y 3 "tranquilos" y "aburridos". 9 "llora en silencio", "pregunta constantemente si el y toda la familia nos vamos a morir", otros se niegan a todas las propuestas de los terapeutas y escolares: "no quiere hacer nada", "se angustia, no s e qu e hacer". 10 problemas de sueño ("se duermen tarde", "duermen hasta pasado el mediodía"), ansiedad ("se come las uñas") y trastorno de alimentaci on ("comen m as cantidad", "saltean comidas"). 11 "trato que no me vea mal". 12 "est an sin ganas", "se enojan". 13 "abre la carpeta y se pone mal, porque le cuesta. ella quiere ir a la escuela". 14 "no tenemos pizarr on en casa", "no sabe leer la letra". based on the responses, reading behavior has improved quantitatively. on 18 occasions, parents comment that their children have a hard time reading, on 1 occasion there are minor difficulties and on 1 occasion there are no problems at all. about the main difficulties that children have when they read, in 50% of the cases they answer in vocabulary and text comprehension, 30% in sentences, 15% in vocabulary and 5% in pronunciation. these roughly coincide with the results of the teacher interviews. on 14 occasions they indicate difficulty in understanding and 6 occasions in vocabulary. according to the families, half of the students have a general problem to read a text, which affects all levels of comprehension. secondly, the highest incidence is at the syntactic level. according to the teachers, the main problem is also the understanding as a whole, but the second aspect is the lexicon. most of the teachers value the reading comprehension of texts through auditory or audiovisual means: audio and video on whatsapp (11) and telephone calls (1). there are some teachers that send images of texts (3), who consider that if the students have answered correctly there must have been reading comprehension. the rest don't have a method: 1 of them still does not know how to evaluate reading ability, and 4 did not answer the question. when receiving the teachers' assignments, 8 families respond that their children accept it very well, 6 indicate that they get upset and 6 do not answer. it can be observed that, with high frequency, receiving daily activities provokes emotional conflict in students. teachers not only face the challenge about not knowing their students well but also about having technological difficulties in communicating with them. regarding the ways used to send assignments by most teachers, 17 use whatsapp, 3 virtual platforms, 2 mail and 1 uses both a platform and whatsapp. on this matter, teachers are very often using technological means. instead, it is more debatable whether the use is appropriate. families comment that, at first, the teachers were "inflexible", they use whatsapp broadcast lists. this way of communication did not allow teachers to answer questions. however, after that, teachers opened other spaces to interact. in other words, it was not possible to take advantage of the dialogical character defined by wegerif et al. (2017) from the beginning of the isolation. teachers use these means to send photos of their folders, word files and other activities in pdf. parents were not very happy with these means at the beginning, although the situation has improved. parents point out that it is difficult to interact with the teachers and they do not have the adequate means to follow the teachers' instructions. on the other hand, teachers also have difficulties in communicating with families through technological tools on a continuous basis because many parents cannot receive the homework files and this establishes a significant inequality in their students. the results presented by the carried-out surveys lead us to conclude that both, teachers and families, are making an additional effort due to isolation. it can be seen that both are using digital technologies to overcome the current situation. however, the responses indicate that reading comprehension has worsened and that students and parents have more difficulty to regulate their emotions. it has been observed that, although the number of students who read at home has increased in isolation, the proportion continues being low. this is very relevant in order to solve this situation because reading is essential as a tool for acquiring academic knowledge and for self-knowledge (sevilla vallejo, 2019, p. 243) . in this work, the sample is made up of students with learning disorders, whose requirement to achieve significant learning demands adjustments to access the curriculum. these students had problems with reading comprehension before lockdown, due to the diagnosis presented. likewise, in the psychological work carried out with them, it is observed that not only do they present these difficulties, but they also lack their regulation on an emotional level. in addition, it is verified that the students show negative emotions when they receive their tasks. this may be caused by a lack of understanding of them. the circumstances in which social isolation has occurred have caused teachers not to know the diagnosis of the students and have not had time to prepare for the adaptation of their subjects, because the measure was established just 2 weeks after classes began. although teachers use digital technologies to communicate with students and their families, parents rarely ask them and some teachers have not decided with which methodology to work students' reading comprehension. parents prefer to contact special education teachers or close family members and neighbors first, to help them to solve the activities. in other words, due to the answers given, part of the classroom teaching responsibility has been transferred to the special education teachers. furthermore, it is observed that families are equally concerned about the effects of the situation on sociability and on students' training, while teachers are more concerned about the latter. it can be seen that teachers have more difficulties accompanying students in the aspects most related to emotions. we can conclude that isolation is causing a negative emotional impact on students with learning disorders, since it is currently observed that feelings of exhaustion, frustration, irritability, sadness, anguish and uncertainty prevail. likewise, emotional instability is observed, which also affects academic performance. the lack of regulation of children's emotions is proportionally linked to that of their parents', who have equivalent emotions and have a negative impact on their children. in relation to reading comprehension, it has been observed that a generalized worsening occurs. although students have increased their reading at home quantitatively, the emotional difficulties and the change in the situation in which they receive their training have qualitatively worsened their reading comprehension, which has been impaired in many cases in all linguistic levels that comprise it. the problem of some students' access to digital technologies is also very relevant in order to face the current situation of social isolation. this study shows the necessity to make some changes to reduce the effect of lockdown on emotional regulation and reading comprehension. it is necessary to build a real dialogical education based on a deeper knowledge of the digital technologies and the access to them. students, parents and teachers are not using these tools as effective and they have difficulties to understand their own roles in the current situation. the authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. credit authorship contribution statement santiago sevilla vallejo: writing -original draft. natalia andrea ceballos mar on: writing -original draft. name and surname: child's name: diagnosis of the child: grade and school: 1 does your child have a therapeutic team to accompany him or her at this time? 2 what are the educational consequences at the time of compulsory social isolation? 3 do you think that the teacher and the school are supporting them adequately in the current situation? in what way? 4 how does your child feel right now? 5 how do you feel emotionally right now? do you think it affects your children? 6 how would you evaluate the accomplishment of the school tasks? 7 do you understand the tasks to be carried out? 8 what do you do when you don't understand the activities? do you read generic bibliography, ask the teacher, search the internet, etc.? 9 how often do your children read at home? has this behavior changed? 10 what are the main difficulties that your child has when reading? with the lexicon, with the sentences, with the meaning of the text? 11 is your child having a harder time understanding texts these days? in what sense? 12 how does your child reacts when receiving the teacher's activity? 13 do you want to comment on any other data of interest? name and surname: age: school and grade: 1 do you know all of your students and if any have a school inclusion process? if positive, what is the child's diagnosis? 2 what are the educational implications at this time of compulsory social isolation? 3 are families supporting students appropriately? 4 how do you think your student with a process of school inclusion feels emotionally? and his or her family? 5 how do you accompany them at this time? 6 how would you evaluate your student accomplishment of the school tasks? 7 what is your teaching work method like at the moment? 8 do you think your students are adapting well to the change to remote methodology? 9 how do you value the reading comprehension of the students? have you change the methodology in the current situation? 10 what are the main difficulties that your students have when reading? with the lexicon, with the sentences, with the meaning of the text? 11 these days, do you think students are having a harder time understanding texts? in what sense? 12 how does your students reacts when receiving the teacher's activity? 13 do you want to comment on any other data of interest? claves para la enseñanza de la comprensi on lectora educaci on emocional y competencias b asicas para la vida neuroeducaci on: uniendo las neurociencias y la educaci on en la búsqueda del desarrollo humano el efecto del aislamiento social por el covid-19 en la conciencia emocional y en la comprensi on lectora. estudio sobre la incidencia en alumnos con trastornos de aprendizaje y menor acceso a las nuevas tecnologías [the effect of social isolation by covid-19 on emotional awareness and reading comprehension cuarentena: piden flexibilizar las condiciones para niñez con discapacidad [quarantine: request to make conditions for children with disabilities more flexible educar en argentina de la era digital cognitive emotion regulation strategies and emotional problems in 9-11-years-old children: the development of and instrument gobierno de c ordoba la provincia lanz o la web tu escuela en casa [the province launched the web your high-school at home el papel de la regulaci on de la emoci on en el exito acad emico temprano de niños. revista de psicología escolar navigating fiction: cognitive-affective engagement with place in children's literature apego seguro, vínculos parentales, clima familiar e inteligencia emocional: socializaci on, regulaci on y bienestar ¿c omo autorregulan nuestros alumnos? revisi on del modelo cíclico de zimmerman sobre autorregulaci on del aprendizaje [how do our students self-regulate? review of zimmerman's cyclical model on self-regulation of learning habilidades sociales en niños y niñas con discapacidad intelectual why should teachers tell stories at class? narration for educative and identity purposes la aventura interminable. algunas claves sobre la motivaci on y los procesos de lectura [the neverending adventure la lectura viva. criterios psicol ogicos y did acticos para fomentar el descubrimiento en los textos [living reading. psychological and didactic criteria to exploring the ontological dimension of dialogic education through an evaluation of the impact of internet mediated dialogue across cultural difference key: cord-332682-c6iql84p authors: liesman, daniel r; pumiglia, luke; kemp, michael t; alam, hasan b title: perspectives from rising fourth year medical students regarding strategies to counteract the effects of covid-19 on medical education date: 2020-07-13 journal: j med educ curric dev doi: 10.1177/2382120520940659 sha: doc_id: 332682 cord_uid: c6iql84p the covid-19 pandemic has put those who oversee medical education in a challenging position. medical school administrators, students, and national governing bodies have been forced to make difficult decision as a result of public health concerns and government-enforced restrictions. we, as rising fourth-year medical students, would like to shed light upon the hard work that many of those in leadership positions have done as well as lay out some concerns that medical students who are preparing to apply to residency have. additionally, we would like to suggest several potential approaches that attempt to address some of the problems arising from the pandemic. continuing to balance education with the hurdles presented by covid-19 will require a multi-faceted and coordinated approach. we believe that implementing virtual rotations, delaying the opening of the application, decentralizing clinical skills evaluations, and modifying graduation requirements are possible options among many that could aid in addressing some of the current challenges presented by covid-19. medical students and administrators across the country are currently faced with the difficult and harsh realities of the covid-19 pandemic. at many medical schools, rising fourth-year medical students have been pulled from rotations during a critical period of career exploration and residency application building. the pandemic has created anxiety from medical students all the way up to hospital administration on how to continue with medical education amid the uncertainty. as 2 rising fourth-year medical students, we understand that students may be anxious regarding their ability to build relationships with faculty mentors, explore all possible specialties, and bolster a residency application due in the fall of 2020. already, attention has been paid to covid-19's implications on the general surgery match and its effect on subinternships and visiting rotations. 1 we would like to expand further on this topic and offer additional perspectives that are generalizable to all medical students interested in various specialties. the association of american medical college's (aamc) concern for student safety and recommended removal of students from clinical rotations is laudable. there is no doubt that this recommendation was made with the utmost consideration of all involved parties. however, these clinical limitations will have clear implications for students during the upcoming residency application cycle. as the country continues fighting the pandemic, decisions must account for not only continued uncertainty but also their potential long-term implications and ripple effects. graduating competent physicians takes remarkable coordination between schools, health systems, physicians, and support staff outside the context of a pandemic. therefore, we hope to contribute to this early discussion to identify possible avenues that residency programs, administration, and students may find useful as they approach some of the developing issues of clinical learning, licensing examinations, and graduation requirements in the wake of covid-19. although many institutions have similar core rotations or clerkships required of students, they do not include all potential specialties. in order to explore interests outside of the core rotations, a student needs to allocate time after completion of core clerkships. for example, physical medicine and rehabilitation and neurosurgery are not included as core rotations at many institutions. therefore, time must typically be allocated after finishing core rotations to explore these specialties. in addition, rotations completed after core clerkships often are where students build meaningful relationships with faculty in their intended specialty as well as work on developing basic intern-level skills related to floor management, calling consults, and the patient discharge process. most importantly, the faculty relationships provide mentorship, career advising, research opportunities, and can be essential in facilitating the transition from student to a junior member of that particular field. as a result of the limitations imposed by covid-19, the removal from clinical rotations means that students may be unable to fully explore their clinical interests prior to applying for residency and therefore less able to build relationships with several faculty mentors. additionally, following core rotations, many students aim to perform visiting rotations in their specialty of choice to gain further exposure and experience in other health systems. recently, the coalition for physician accountability, comprised of representatives from many bodies, including the aamc and accreditation council for graduate medical education (acgme), released recommendations that no away rotations should take place during this residency application cycle except in extenuating circumstances. 2 given that some students rely on visiting rotations for their only exposure to a specialty unavailable at their home institution, the loss of these away rotation opportunities will weigh particularly heavy on certain students. the concurrent loss of core clinical and visiting rotations may put students in the unenviable position of applying to a specialty they may not have explored. it may also mean that they do not have faculty mentors to help guide them through the residency application process. since telehealth has expanded across the country, medical schools may consider mitigating the effects of losing away rotations by building virtual away rotations in some specialties to provide opportunities to students who would otherwise have no exposure to those fields. already at our home institution, we are involving our students in virtual patient care. one other important consideration is how the aamc guidance will impact the diversity of the medical student residency applicant pool for various specialties. for example, the effect that the loss of visiting rotations will have on diversity is unknown. already there is conflicting evidence on whether or not visiting rotations promote equity. the cost of visiting rotations is prohibitive for many students as the average amount paid for one rotation is $958. 3 furthermore, some students complete several visiting rotations for specialties considered to be competitive making the cumulative cost of visiting rotations much higher. in addition, the current differential statewide travel restrictions could allow only a fraction of the student population the privilege of traveling for away rotations. 4 these differential restrictions coupled with the already reduced population of students eligible to travel for away rotations due to recent aamc/acgme recommendations could likely further exacerbate existing disparities 2 . conversely, evidence exists that suggests visiting rotations can be an important tool for improving the diversity of underrepresented minorities who practice medicine. in emergency medicine, for example, 30 programs exist nationally that aim to give visiting underrepresented students spots in their rotation. 5 furthermore, outreach programs and funded rotations are available at johns hopkins for underrepresented students in medicine interested in pathology. as a direct result of this program, there were 18 underrepresented rotators in july 2019 in comparison to 1 in july 2013. 6 these data suggest that, when structured appropriately, visiting rotations can play a role in increasing the diversity of the physician workforce. given the uncertainty around the effects that covid-19 will have on diversity, future investigations should assess outcomes to develop strategies that can be implemented even outside of the pandemic context. in light of the loss of clinical time and significant limitation in away rotation opportunities, the same working group has recommended delaying the opening of the electronic residency application service (eras). 2 this delay could provide crucial time for students to complete rotations at home and outside institutions as well as develop basic clinical skills that will prove invaluable when they start their intern year. however, the benefit of this delay will only be realized if clinical rotations or similar opportunities return. if this happens, the clinical learning gained during the delay would be beneficial for students making decisions about their applications. it may also improve the ability of residency programs to distinguish between candidates. in order to maintain the amount of time necessary for residency programs to interview applicants and for applicants to explore all potential programs, a similar delay in rank submission and match day may be necessary. a later match day would impact the amount of time that students have to explore their potential new city and find housing. however, we believe that the benefit of extra time to explore clinical interests would outweigh the cost of less time post-match day. the social distancing limitations have also had significant impacts on the scheduling of licensing exams. for example, the the clinical skills evaluation committee (csec) has suspended the step 2 clinical skills (cs) exam for the next 12-18 months. 7 assessing a student's ability to effectively communicate with and physically examine patients is vital. given the current circumstances, it is likely many members of the graduating class of 2021 will not take step 2 cs by the time they graduate. rescheduling this buildup of examinees unable to take the test during this suspension could be a tremendous burden on the csec and current medical students. according to the liaison committee on medical education (lcme) accreditation standards, accredited institutions must have a system in place to assess a student's core clinical skills (e.g. history-taking and physical examination). 8 therefore, with national testing possibly becoming less available or accessible, it may be worth considering temporary school-administered clinical assessments as part of a hardship modification. we are not doubting the csec's ability to reschedule the students who need to take this exam, if that becomes necessary. rather, school-administered assessments could reduce the burden on students, the csec, and its testing capabilities. such a solution would take advantage of existing infrastructure to continue ensuring clinical competency of graduates. in addition, with the current economic downturn, school-administered exams could help lessen the economic impact of travel on students. the cost of the examination (approximately $1300) liesman et al 3 coupled with varying expenses associated with flights, hotels, and food can be burdensome for some students. due to the high rate of unemployment nationally, these costs may be insurmountable for some students, particularly those who rely on financial support from others who may now be unemployed. school-administered exams could help reduce this cost associated with examinations. finally, medical school-based assessments could help mitigate any further spread of covid-19 in the future as it would prevent students from convening in the 5 testing centers. due to current limitations, reforming educational approaches and clinical graduation requirements may be necessary for students to graduate on time. our institution is considering reducing the number of in-person clinical months required for graduation. a plethora of online learning platforms and courses exist that medical schools can utilize to educate students on clinical hiatus. these online resources are opportunities for education on core competencies required of effective physicians such as systems-based practice, ethical reasoning, and leadership. in many cases, they are also opportunities to grow students' clinical knowledge and reasoning foundations through virtual patient cases and group discussions. at our institution, students are practicing oral presentations and case discussions via online platforms to maintain preparedness and clinical acumen. since a diversity of valuable online clinical learning exists, consideration with regard to how these experiences will be valued on student transcripts is warranted. for example, our radiology department has assembled a graded 4-week virtual clinical rotation for the different services on radiology that fulfills a graduation requirement. since quality learning can be done outside of traditional clinical duties, it may be worthwhile for the lcme to temporarily modify graduation requirements so that that this learning can be equally considered when determining eligibility for graduation. these are unprecedented times. negotiating the uncertainty brought by covid-19 has been, is, and will continue to be extremely difficult. medical educators and administrators are faced with significant challenges and are rising to the occasion. as students, we believe that implementing virtual rotations, delaying the opening of the application, decentralizing clinical skills evaluations, and modifying graduation requirements are possible options among many that could aid in addressing some of the current challenges presented by covid-19. implications of covid-19 on the general surgery match the coalition for physician accountability's work group on medical students in the class of 2021 moving across institutions for post graduate training. final report and recommendations for medical education institutions of lcme-accredited the prevalence and cost of medical student visiting rotations driving and travel restrictions across the united states. the new york times racial and ethnic diversity in academic emergency medicine: how far have we come? next steps for the future the "race" toward diversity, inclusion, and equity in pathology: the johns hopkins experience usmle suspending step 2 clinical skills (cs) examination function and structure of a medical school d.r.l. and l.p. contributed equally and are co-first authors. luke pumiglia https://orcid.org/0000-0002-8179-2403 key: cord-028394-oq4z0nhc authors: al-doulat, ahmad; nur, nasheen; karduni, alireza; benedict, aileen; al-hossami, erfan; maher, mary lou; dou, wenwen; dorodchi, mohsen; niu, xi title: making sense of student success and risk through unsupervised machine learning and interactive storytelling date: 2020-06-09 journal: artificial intelligence in education doi: 10.1007/978-3-030-52237-7_1 sha: doc_id: 28394 cord_uid: oq4z0nhc this paper presents an interactive ai system to enable academic advisors and program leadership to understand the patterns of behavior related to student success and risk using data collected from institutional databases. we have worked closely with advisors in our development of an innovative temporal model of student data, unsupervised k-means algorithm on the data, and interactive user experiences with the data. we report on the design and evaluation of first, finding interesting stories about students, that provides an interactive experience in which the advisor can: select relevant student features to be included in a temporal model, interact with a visualization of unsupervised learning that present patterns of student behavior and their correlation with performance, and to view automatically generated stories about individual students based on student data in the temporal model. we have developed a high fidelity prototype of first using 10 years of student data in our college. as part of our iterative design process, we performed a focus group study with six advisors following a demonstration of the prototype. our focus group evaluation highlights the sensemaking value in the temporal model, the unsupervised clusters of the behavior of all students in a major, and the stories about individual students. as artificial intelligence in education becomes increasingly prominent, there is a growing need to consider augmented intelligence. this is the idea that artificial intelligence can and should be used to enhance human intelligence and a. al-doulat and n. nur-these authors contributed equally. abilities rather than attempt to replace it. the 2016 national artificial intelligence research and development strategic plan stated that "the walls between humans and ai systems are slowly beginning to erode, with ai systems augmenting and enhancing human capabilities. fundamental research is needed to develop effective methods for human-ai interaction and collaboration" [1] . popenici and kerr further emphasize the importance of recognizing education as a "human-centred endeavor" and the idea that "solely rely[ing] on technology is a dangerous path, and... that humans should identify problems, critique, identify risks, and ask important questions..." [2] . therefore, we should take on a human-centered approach in the era of ai. human-centered ai is a viewpoint that ai systems and algorithms "must be designed with an awareness that they are part of a larger system involving humans" [3] . ai research should not just be technological, but humanistic and ethical as well [4] . one aspect of humancentered ai is to create systems that help humans understand the system itself [3] . therefore, the goal is not simply to provide results through a black-box model. the focus is to help users understand those results and how those results are derived. we explore sensemaking in learning analytics (la) as an example of humancentered ai and present how we address this challenge for advisors that are presented with large amounts of data and analytics about their students. la is an interdisciplinary field that emerged to make sense of unprecedented amounts of data collected by the extensive use of technology in education. la brings together researchers and practitioners from two main fields: data mining and education [5] . effective presentation of analytical results for decision making has been a major issue when dealing with large volumes of data in la [6] . many systems for early alerts on student performance provide results without providing necessary explanations as to how the system derived those results. if an early warning system gives a result that is inconsistent with the expectations of a teacher or an advisor, and there is no information to explain how the system arrived at the prediction, it can easily cause educators to discount or mistrust the prediction [7] . human sensemaking relies on developing representations of knowledge to help serve a task, such as decision-making, and on the design of ai approaches to better aid these tasks. we discuss the design, implementation, and evaluation of an interactive system designed to help advisors better understand student success and risk. in contrast to many la systems designed to support student awareness of their performance or to support teachers in understanding the students' performance in their courses, our interactive system is designed to support advisors and higher education leadership in making sense of students' success and risk in their degree programs. our approach to interactive sensemaking has three main parts: (1) a temporal student data model, (2) data analytics based on unsupervised learning, and (3) storytelling about the student experience. in this section, we review related research in two interdisciplinary threads: (1) sensemaking in la, and (2) data storytelling techniques. sensemaking is process of understanding connections to anticipate their trajectories and to act effectively [8] . van et al. [9] stated that sensemaking is a core component of la dashboard interventions, as the purpose of these tools is to provide users with the ability to become aware of, reflect upon, and make databased decisions. echeverria et al. [6] proposed a learning design-driven data storytelling approach where they support user sensemaking by directing the user's attention to the critical features of the students' data using visualizations with data storytelling components. their user study suggests that adding storytelling elements to the la dashboards has the potential to help users make sense of the critical features of students' data with less effort. calmsystem [10] is another example of a la system that supports sensemaking, awareness, and reflection. it was developed on top of an intelligent tutoring system to give a learner insight into the learner model. klein et al. [11] proposed a model of student sensemaking of la dashboards to show how data and visualization inform user sensemaking and action. verbert et al. [11] introduced a la system for learners and teachers visualizing learning traces with four distinguished stages for the process model -(i) awareness is only concerned with the students' data presented using various visualizations, (ii) reflection focuses on usefulness and relevance of the queries by the users, (iii) sensemaking is concerned with users' responses in the reflection process and the creation of new insights, and (iv) impact is concerned with the induction of new meaning or changing behavior by the users. additionally, researchers made contributions to better prediction and sensemaking of student progress trajectories. learning management systems (lmss) storing students' temporal data have been leveraged in various works to analyze students' progression throughout their whole program [12] [13] [14] [15] [16] and within a course level [12, [17] [18] [19] . stories are capable of conveying essential information to users more naturally and familiarly for them [20] . data storytelling aims to make data more understandable and memorable by human users by presenting data in the form of stories. several research studies created natural language presentations of tabular or numeric data ranging from summarizing statistical results [21, 22] , stock market trends [23] , and environmental data [24] . many applications of natural language generation (nlg) have been used to generate stories from data to promote the user sensemaking. notable examples of tools that generate textual forecast from structured data include the forecast generator (fog) [25] , mul-timeteo [26] , and the sumtime system [27] . such systems increase interpretability and reduce routine writing tasks performed by human forecasters. nlg is also used in medicine. topaz [28] , creates reports of blood cell and drug dosages for lymphoma patients. it uses a schema-based generation system that generates a textual report read by clinicians. other systems that generate medical reports include suregen [29] , narrative engine [30] , and stop [31] . these systems tend to facilitate the users' sensemaking of homogeneous data through brief textual summaries. first is capable of generating stories to support advisors' sensemaking of complex, temporal, and heterogeneous student data. the goal of first is to better communicate analytics results by guiding the user through sensemaking tasks and interactive la. sensemaking tasks consist of information gathering, developing insights, and performing knowledge discovery [32] . in the sensemaking process, domain experts such as the educational leaders, teachers, and academic advisors decide on the existing challenges and expected outcomes for their institution. most of the learning management tools involve data scientists in the knowledge discovery process to design the student data model, analytics approach, visualizations, and a reporting system to understand students' patterns of success or failure. next, domain experts design intervention methods based on the analytics. the analytical process, essential to knowledge discovery, needs substantial data science skills. domain experts do not engage in the discovery process since the analytical model is a black box to them. in first, domain experts can select features from the temporal data model, see the stories about students, and explore which factors are major contributors to a student's performance and behaviors. our system is designed to allow advisors to engage in sensemaking by interacting with temporal data, reviewing aggregate analytics, and reading stories. figure 1a shows the interface for the user to select the student features in the temporal model. the selected features are used when generating stories for each student. the user can change their preferred features at any point, which will consequently change the content of the stories. it is also possible for the system to automatically generate stories based on what it selects as the most appropriate features. however, allowing the user to select the features is important to sensemaking. figure 1b shows the user experience with the results of unsupervised learning, and fig. 1c shows the user experience for interacting with the automatic story generator. first differs from existing la tools in the following ways: -the user can leverage their insights about student behavior and participate in model construction, giving them the flexibility to change the features to be used in the analytic models and automatically generated stories. -the user is presented with automatically generated stories to complement the results from analytic models. first uses a temporal data model that uses time segments to group heterogeneous sources of data and form sequences of information for each student [16] . this allows the analytic models to consider the temporal dependencies of students throughout their enrollment. the temporal model gives flexibility in defining the duration of the temporal node, contextualizing information within a node, and interpreting sequences of nodes as stories. the data model contains one sequence per student that starts with their enrollment and ends with when the student graduates or leaves the university. each node in a sequence represents a period (e.g., a single semester) and contains a vector of features (variables, such as courses taken in that semester). there are three types of temporal nodes for each student: the background node with demographic information, the semester node with semester-wise activities and information, and the outcome node with the value of the performance variable. the student data model is shown in fig. 2a . first uses unsupervised learning to identify patterns of student behavior and then maps that behavior onto performance. the user can select from options for the student performance variable, such as gpa, and can select filters to include a subset of the total population, such as male or female students or a period of time. figure 1b shows the results of clustering all students according to their course progression with the performance variable of gpa, where 2.0 is a minimum value to be successful. course progression is an example engineered temporal feature, which is the average value of the first digit of a course level for each semester. for example, if a student took three courses with levels 1200, 1212, and 3000 in his/her first semester, this feature will take a value of 1.7 (average of 1, 1, and 3) for the first semester. we then formed a 2d (two-dimensional) feature vector for each student in which each row has the values for one of the engineered features for each semester. we used the k-means clustering algorithm [33] on several engineered features and found that course progression, for example, was able to cluster students with high "purity" in terms of the defined outcome variable. we used the elbow method [34] to determine the optimal number of clusters. we analyzed each cluster to see if they were "coherent" in terms of student performance. for example, after we applied the k-means approach to the "course progression" feature, the result could separate the successful and risky student reasonably clearly. our primary hypothesis for this feature is that it should be either increasing or steady along the semesters for those successful students. if it is decreasing or steady for a long time, the student did not progress to higher-level courses or the student was repeating lower-level courses. figure 1b presents the clustering results with 3 clusters for the engineered feature "course progression through semesters". in the blue cluster with 483 students, successful students are the most dominant with a percentage of 90.27%. as we see the intercept and the slope of this blue line in fig. 1b , it has a higher average course level in each semester compared to the other two clusters. in addition, the average course level is consistently increasing. this suggests that this cluster of students consistently takes courses at a higher level and starts to progress early on. the green cluster also has a higher percentage of successful students than the orange cluster. if we compare their intercepts and slopes, the green line stays above the orange one and makes more "linear" progression than the orange counterpart. in this analysis, we define student success as obtaining the final gpa last semester higher than 2.0. if we changed the gpa threshold, the clustering results would be different. the user can select each cluster and further review the data for each student who belongs to that cluster. the bar chart shows the average gpa for each cluster. the user can select an individual student or groups of students in the analytic interface and review their temporal data. the selected students in exploring the analytic results are saved and available on the storytelling page. we use clustering since more students are successful than unsuccessful:: a supervised learning approach could overfit and impose an accuracy paradox due to a higher number of majority class examples caused by the imbalance. equalizing class membership by adjusting the within-class imbalance and using random sampling can introduce unrealistic patterns in the data [35] . we use clustering to separate and classify samples. the clustering results provide insight into the engineered features that discriminate on percentages of successful students compared to students at risk. this classification describes characteristics of cohorts of students and how they behave in the clusters. in the future, we will consider a guided re-sampling and classification method to overcome over-fitting. for this reason we adopted an unsupervised clustering approach to find patterns of student behavior that map onto success criteria. in the future, we plan to incorporate the cluster results into a predictive model to apply our knowledge about patterns of behavior in cohorts of students to develop early alerts or predictions for individual students. first automatically generates stories for each student using the features selected in the temporal data model. these stories present a summary of the student's experience in a narrative. figure 1c shows the user experience for interacting with the student stories. when the user selects a student from the left panel, the timeline and story sections are updated. the storytelling algorithm uses user-selected and standard features. the stories are generated from the data in the temporal model shown in fig. 2 . figure 2a shows the nodes in the temporal data model, fig. 2b shows the features selected from each node, and fig. 2c shows the sentences that are constructed from each feature. the text in black is from a predefined template while the text in red is generated from the features. after generating the sentences for each of the selected features, these sentences are used to generate the story as discussed below. an example of a generated story can be shown in fig. 1c . figure 3 illustrates the 3 stages in the process of generating stories: raw data source and user selection inputs, story synthesis, and story analysis. we describe each stage of story generation. fig. 3a , the input data for story generation comprises: (i) features in the temporal data model, (ii) the results of the analytics, and (iii) the user selected features and outcome. the features in the data model are used in a set of predefined template rules, the analytics results are used to compare the current student with other similar students, and the user-selected variables are used to make the story more customized for the user. story synthesis. the goal of this stage is to determine and sort the content presented in the student's story. as shown in fig. 3b , synthesis has two tasks: content determination and story structuring. -content determination: this is the task of choosing what is included in the story. the selection is based on these factors: â�¢ user-selected features: we include the features selected by the user as illustrated in fig. 1a . â�¢ performance rules: we identified a set of rules that either inspect any sudden changes of the students' performance over time (e.g., a student's gpa suddenly dropped or increased), or abnormal information compared to most students (e.g., the number of attempted, passed, or failed courses for a semester is higher, or the number of d-scored courses is higher). â�¢ comparison with other similar students: we used clusters to look for students that are similar and successful to inspect if the student per se is an outlier in terms of some variables. -story structuring: this is the task of deciding the order of information in which it is presented to the reader. we order the information based on the student temporal data model, in which the story starts with the background information about the student, then with the semester information, and ends with the outcome information. story analysis. this stage improves the language of the stories so they are more human-readable and coherent. as shown in fig. 3c , this includes 2 tasks: sentence aggregation and lexicalization. -sentence aggregation: clusters multiple pieces of the same kind of information together into a single sentence instead of several ones. for instance, if we have a set of candidate sentences as "student achieved an a in the course x", and "student achieved b in course y", these sentences should be aggregated into one sentence "student maintained all his grades at b or above". -lexicalization and linguistic realization: lexicalization is choosing the proper words and phrases to transform the data into natural language text. linguistic realization is inserting punctuation, functional words and other elements required for the text to be fluid and coherent. a focus group study was conducted with the goal of learning what users find important in a tool to support advising. in the focus group session, we demonstrated first and then asked questions about the value of the student data model, analytics, and storytelling. we recruited six professional and faculty advisors whom are already familiar with multiple tools that provide data, analytics, and risk scores for the students that they advise. a focus group study was selected for its effectiveness in collecting user opinions and attitudes through group discussion and dynamic conversations. some preliminary questions were asked to collect information related to the current technology used during advising and the useful features of those tools. the participants revealed that they often ignored the risk score provided by the analytics in their advising tool because the process behind the calculation is not clear to them. they mentioned that although the student reports generated by the existing tool were useful, they would like more flexibility to customize the information for different cohorts of students. the group discussed that one goal for such tools is to be prepared for advising before the student arrives for the advising appointment. first was demonstrated to the group with scenarios for specific students. the participants asked questions about the system and the facilitator demonstrated additional interactive features. then the participants were asked to answer questions to assess the sensemaking they performed through the demonstration: (i) what insights were you able to gain about students through viewing this tool? (ii) what are the differences between what you learned about the students from the analytics versus the stories? (iii) what is the value of the analytics results and the stories? (v) how can the student stories help you with advising? and (vi) can you think of other good predictors(features) of student success? two researchers reviewed the transcript and identified emerging themes independently and through discussion they agreed on three higher-level themes. these three high-level themes were then used to revisit and code the transcript according to the themes. -selecting features for student models: participants appreciated that they could select the features they thought should be part of a predictive model of risk or part of the student story. they also like a number of features that were included, such as students' financial need status, family life, housing options, and mailing addresses. many expressed surprise that the university actually had a lot of data that would be useful for advising that was not available in the other tools. -value of aggregate analytics and temporal data: participants agreed that aggregate analytics is essential for understanding students, especially a targeted group of students. they found the presentation of the student data as a temporal progression is useful since it presents the overall students' progression through semesters. -value of student stories: the participants agreed that student stories were useful and effective to provide a high-level overview or snapshot of the student. they mentioned that the stories would be helpful for understanding a specific student quickly. they agreed that stories provide a good understanding of students in terms of their demographic information as well as their academic performance. one participant said: "i like the stories the best -knowing that the story was created using analytics is reassuring". one comment to extend first is the suggestion to tell stories about groups of students that lie in a single cluster. in this paper, we present first, an interactive la system designed to support advisors using a temporal data model, unsupervised models, and storytelling. first enables the advisor to select specific features, review the aggregate analytics based on unsupervised learning algorithms, and interact with stories about specific students. the student stories are automatically generated using user-selected features, the features that indicate significant changes, and additional data about the student using rules that present a more complete story. the process for generating stories has 3 stages: sourcing the data, selecting and structuring story components, and text-processing the sentences. a focus group study was conducted to evaluate first and gather feedback. the participants highlighted the sensemaking value of storytelling and the increased access to student data compared to other tools. the aggregate analysis was reported to be enhanced by the storytelling since the user can switch between the story and the visual analytics. the results of the focus group confirm our hypothesis that storytelling complements dashboard-style analytics. in the future, we plan to do a longitudinal study of the use of first to learn more about the changes in the advisors' understanding of their students with and without first. strategic plan: the national artificial intelligence research and development strategic plan exploring the impact of artificial intelligence on teaching and learning in higher education human-centered artificial intelligence and machine learning toward human-centered ai: a perspective from human-computer interaction piecing the learning analytics puzzle driving data storytelling from learning design artificial intelligence applications to support k-1 2 teachers and teaching making sense of sensemaking 2: a macrocognitive model analytics for learning and teaching calmsystem: a conversational agent for learner modelling learning analytics dashboard applications course signals at purdue: using learning analytics to increase student success student success system: risk analytics and data visualization using ensembles of predictive models predicting at-risk university students in a virtual learning environment via a machine learning algorithm student network analysis: a novel way to predict delayed graduation in higher education a sequence data model for analyzing temporal patterns of student data developing predictive models for early detection of at-risk students on distance learning modules learning analytics at low cost: at-risk student prediction with clicker data and systematic proactive interventions data mining in course management systems: moodle case study and tutorial storytelling ontology model using rst helping people with visual impairments gain access to graphical information through natural language: the igraph system generation of extended bilingual statistical reports design of a knowledge-based report generator automatic production of multilingual environmental information using natural-language processing to produce weather forecasts interactive generation and knowledge administration in multimeteo sumtime-mousam: configurable marine weather forecast generator combining physiologic models and symbolic methods to interpret time-varying patient data suregen-2: a shell system for the generation of clinical documents building a large-scale commercial nlg system for an emr lessons from a failure: generating tailored smoking cessation letters the cost structure of sensemaking some methods for classification and analysis of multivariate observations clustering with the k-means algorithm using unsupervised learning to guide resampling in imbalanced data sets key: cord-028477-guvc9aa0 authors: hlosta, martin; papathoma, tina; herodotou, christothea title: explaining errors in predictions of at-risk students in distance learning education date: 2020-06-10 journal: artificial intelligence in education doi: 10.1007/978-3-030-52240-7_22 sha: doc_id: 28477 cord_uid: guvc9aa0 despite recognising the importance of transparency and understanding of predictive models, little effort has been made to investigate the errors made by these models. in this paper, we address this gap by interviewing 12 students whose results and predictions of submitting their assignment differed. following our previous quantitative analysis of 25,000+ students, we conducted online interviews with two groups of students: those predicted to submit their assignment, yet they did not (false negative) and those predicted not to submit, yet they did (false positive). interviews revealed that, in false negatives, the non-submission of assignments was explained by personal, financial and practical reasons. overall, the factors explaining the different outcomes were not related to any of the student data currently captured by the predictive model. identifying correctly at-risk students has emerged into one of the most prevalent topics in learning analytics (la) and education in general [1] . the identification of at-risk students using predictive learning analytics (plas) and followed by a subsequent intervention targeting flagged students (e.g., phone call) could tackle this problem. many published papers focused on achieving the highest prediction performance, often comparing several learning algorithms. machine learning models are more likely to exhibit some sort of error hence, the need to understand and explain these errors. in a cross-disciplinary field such as la, not having the best model could still help understand or even improve student learning. kitto et al. [2] argued that having imperfect models does not necessarily mean that these should not be deployed. as the la field is maturing, it becomes essential to understand how models are behaving and how errors occur [3, 4] . only few studies have examined errors up to now. this paper aims to explain errors in predictions through 12 in-depth interviews with undergraduate online students wrongly predicted as being/not being at risk of failing their next assignment. we treated false positive (fp) and false negative (fn) errors separately. following [5] , we refer to fp as students predicted as being at-risk but succeeded, and fn as students that failed despite predicted to succeed. we build on the work of calvert et al. [6] that investigated within a single online course why some fp students passed despite predictions showing the opposite and our early quantitative results from [7] . to analyse the predictive model errors, we used a mixed-methods approach (see fig. 1 ). we focused on first year stem courses and predictions for the first assignment only (a1), when dropout is more likely to happen [8, 9] . the predictions were enhanced by additional data: course context (e.g. the length of the course) and future data from the weeks following the predictions, unknown during the prediction's generation. predictions for each course were put together in one matrix and only predictions with confidence ≧0.85 were selected. a decision tree was constructed to distinguish between (1) fp and true positive (tp) and (2) fn and true negative (tn). after getting a favourable opinion from the university's ethical committee, we conducted 12 semi-structured interviews with students lasting 20 to 40 min. the interview schedule was developed by two of the authors, piloted with one student, and the analysis followed inter-rater reliability principles. students were not new to the university, assuming that they would have devised strategies on how to successfully complete their assignment without accessing the virtual learning environment (vle). gift vouchers were offered. we grouped participants to (a) students predicted to submit yet they did not submit (fn; n = 7) and (b) students predicted not to submit yet they submitted (fp; n = 5). following other published work [10] , we analysed students as individual case studies creating a distinct profile picture for each student. the following themes emerged from the thematic analysis [11] : motivations for taking the module, studying patterns, reasons for not submitting the assignment, factors that helped or hindered submission, tutor contact, student contact, recommendations for other students so that they submit and proposed module changes. we then plotted this information on a table and identified similarities and differences within and between the two cohorts of students 1 . considering the predictions two weeks before the deadline of a1, we analysed 38,073 predictions in 17 courses in 62 presentations between 2017-2019, having 29,247 students (383 fp, 1,507 fn, 2,671 tp and 33,512 tn). the roc auc over all predictions was 0.8897. for confident not submit predictions, the decision tree classified correctly 50.91% of the fp errors with 75.29% precision (195 students). the strongest attribute was the number of clicks one week before the deadline of the first assignment (confidence 0.82). for the confident submit predictions, the model distinguished 18.73% of the fn errors with precision 68.73% (1,036 students). the strongest attribute related to a dramatic decrease in students' activity in the last week before assignment 1 (a1) in courses with high activity (confidence 0.83). both types of errors were associated with a change of student activity after the predictions were generated, and it is worth further examination. participants (see table 1 ) were older than those invited to take part in the study, more successful in their previous courses, female repeating the same course. fn -predicted to submit but did not submit: participants were motivated to study either because they were driven by completing a qualification/degree or out of interest. their studying patterns were rather random, with no strict schedule. the reasons explaining non-submission were related to family matters/issues (i.e. caring responsibilities), practical issues (i.e. no internet connection at the time of submission) or they were restricted to submit because of student financial issues. fn_2 who took the course out of interest, found it pointless to submit her assignment as it only weighted 7% of the final grade and it was too easy for her. on the contrary, fn_5 found it difficult to submit because of her lack of digital skills and absence of detailed guidance. further, student contact with tutors was minimal and related to requesting an extension to submitting the assignment. fn_11 mentioned that although she contacted her tutor via email, the tutor never replied. two interviewees reported that their tutor support was helpful with the tutor proactively getting in touch and communicating with them. interacting with other students was not common for four interviewees. fp_3 reported though that she helped other students and fn_4 used facebook groups and forums to communicate socially. fn students made suggestions for future students to follow the online study guidance and plan ahead for submitting assignments on time. fn_2 who had prior knowledge suggested that assignments should have optional questions for the needs of more advanced students. two participants would like to have online tutorials with a tutor to guide the assignment submission. fn_5 suggested that the course should be more accessible by adding detailed guidelines on technical aspects for submission. most participants took the course for the first time apart from one interviewee. fp -predicted not to submit but submitted: all participants were motivated to take their course in order to get a qualification/degree. their studying patterns varied mostly studying in the evenings. the reasons they managed to submit related to the fact that this was not the first time they were taking the course. two of them took the course for the second time. fp_3 was determined to submit as it was their third time taking the course. two interviewees took the course for the first time. fp_7 on the other hand, did not prepare for the assignment, yet answered the assignment questions as they had some prior knowledge. the other two interviewees submitted after watching videos, consulting books, or with help from external networks. contact with tutors was minimal. fp_7 only contacted their tutor for an extension. no interactions with other students were reported. in terms of recommendations, fp_3 suggested that asking for support from their tutor is important although they did not initiate that. fp_8 and fp_9 suggested looking at the vle material in a timely manner and prepare early on. they proposed more contact with teachers and suggested that audio recordings would be a good addition. interestingly, the interviewee who was taking the course for the third time, mentioned that assignments should be given more weight towards the final grade. fp_7 suggested that students with prior knowledge or expertise on a topic should be allowed to skip an assignment. none of the predictive errors could be fully explained by only looking at the course data. errors were explained by factors not currently captured by the university data sets, including personal, technical and financial issues students faced before submission. the factors reported are rather hard to capture automatically and in a timely manner to support students with difficulties. hence, the role of teachers becomes critical; pastoral and proactive care could identify and resolve such issues on time and enable students to succeed. existing studies already showcased the significance of teachers' monitoring and intervening with students at risk for better learning outcomes [12] . a university-wide policy accompanied by relevant teachers' training as to when and how teachers should get in touch with their students would ensure that academic connection and social presence are established [13, 14] . given that we do not gather data from external systems, errors might be hard to prevent in the future. yet, we could add error explanations especially for students submitting their assignment (e.g. taking the course for a second time). quantitative and qualitative analysis of the learning analytics and knowledge conference 2018 embracing imperfection in learning analytics modeling and experimental design for mooc dropout prediction: a replication perspective likely to stop? predicting stopout in massive open online courses speaking the unspoken in learning analytics: troubling the defaults student feedback to improved retention: using a mixed-methods approach to extend specific feedback to a generalisable concept why predictions of at-risk students are not 100% accurate? showing patterns in false positive and false negative predictions success and failure in higher education on uneven playing fields ouroboros: early identification of at-risk students without models based on legacy data implementing predictive learning analytics on a large scale: the teacher's perspective using thematic analysis in psychology empowering online teachers through predictive learning analytics a modified model of college student persistence: exploring the relationship between astin's theory of involvement and tinto's theory of student departure how students' perceptions of support systems affect their intentions to drop out or transfer out of college key: cord-256972-u0gh7ov2 authors: ahlburg, dennis a. title: covid‐19 and uk universities date: 2020-07-11 journal: polit q doi: 10.1111/1467-923x.12867 sha: doc_id: 256972 cord_uid: u0gh7ov2 universities uk (uuk) has suggested that there may be very significant losses to higher education as a consequence of covid‐19. however, losses are likely to be substantially lower than the potential losses estimated by uuk. but the magnitude of losses is very uncertain. the uuk’s proposal to restrict undergraduate enrolment per university to stop institutions poaching students is not in the interests of the most highly regarded universities, or that of students. some rationalisation of the sector should be the price of further government support. now is also the time to reconsider how university research is funded. universities uk (uuk) asked the government for £2.2 billion to help the sector deal with the impact of the covid-19 pandemic. uuk estimated that the education sector annually generates more than £95 billion for the uk economy, employs the equivalent of 940,000 people, and earns £13.1 billion in export earnings. uuk further estimates that it will lose £790 million in revenues in 2019-20 and potentially £6.9 billion in 2020-21 if foreign students, who are critical to the financial health of the sector, fail to enrol. 1 uuk claims that without the support requested, some institutions will fail and others will be forced to reduce provision. the institutions most likely to fail are those with higher levels of external funding, lower levels of cash reserves, and a higher proportion of bame students. that is, those institutions that were in a more fragile financial state before the pandemic. competition for students will 'be destabilising, creating pressure to switch from their chosen institution'. without help from the government 'access to higher education would be decreased, disadvantaged students would be worse off and less able to select a university that best suits their learning needs'. 2 uuk also argues that research and stem programmes will be particularly hard hit because they are cross-subsidised by the higher fees paid by foreign students and the global position of uk higher education will suffer as a consequence. in exchange for additional government support, the universities have promised to cut costs, accept restructuring, and rein in predatory admissions policies. although higher education (he) provides social and cultural benefits to students and society, uuk has emphasised its economic benefit and so here we focus on economic arguments. the labour party is sympathetic to uuk and has called on the government to guarantee that no university be allowed to go bankrupt, because this would cost jobs, reduce social mobility, and limit the training of key health sector staff. 3 although the department for education supports the request from uuk, the treasury does not. the treasury has agreed only to short-term stabilisation by advancing the sector up to £2.6 billion in tuition fees and £100 million in research money. this is not new money: it is an advance. there will be no guaranteed bailout of institutions and further emergency help will be on a case by case basis, and only as a last resort. the treasury will continue to monitor the financial situation of he, but at this stage only short-term assistance has been offered, with no safety net. he is just one sector among many requesting additional assistance. uuk's statement raises many questions about british universities and the impact of covid-19. is he important enough that it deserves special attention? is the financial loss to he likely to be as great as uuk the political quarterly implies? which institutions are most severely affected? will they fail? should they be saved? will students be unable to select a university that best suits their learning needs? is the treasury's measured support appropriate? is he important to individuals and society? the institute for fiscal studies (ifs) has shown that 80 per cent of students gain financially from attending university (85 per cent of women and 75 per cent of men). 4 that is, their earnings are higher, on average, than if they had not gone to university. but one in five students would have been better off not attending university. for some subjects and universities, the figures are worse. for example, on average-and controlling for academic preparation and family background-for women the net discounted lifetime returns from studying creative arts is zero and for men it is negative. on average, attending university pays off financially for most students in many subjects and most institutions, but there is considerable variation around that average. the ifs also estimated the economic benefit to the nation. although he is expensive to the taxpayer, the average financial gain to the nation is £110,000 per male graduate and £30,000 per female graduate. thus, there is solid evidence that he is a good investment for most students and the government, but there is also solid evidence of courses and institutions for which there is little or no economic return. and this was before covid-19. although programmes with no economic return may be restored, covid-19 should not be an excuse for retaining these zombie courses and institutions. restructuring of he has been offered by uuk and the treasury has indicated that it expects restructuring to take place. it is time to act on recommendations in the augar report on how one can justify the continued support of programmes or institutions where the graduate ends up financially worse off. 5 disadvantaged and bame students are not made better off by enrolling in courses that have no economic return-nor is social mobility furthered by such action. i have argued in this journal that higher education is not exactly like other businesses, but this does not necessarily mean that it is exempt from all policies that are applied to other businesses. 6 will the financial loss be as great as uuk claims? while universities will suffer a financial loss from covid-19 in 2019-20 from a reduction in accommodation, catering, and conference income, it is unclear how large a loss they might suffer in 2020-21. although uuk states that the universities could lose £6.2 billion in revenues if all foreign students stay away, that is highly unlikely. a study by london economics for the university and college union estimates a 16 per cent decline in domestic students costing £612 million, and a 47 per cent decline in eu students causing a loss of £350 million in 2020-21. 7 not all institutions were estimated to be affected equally by covid-19. another estimate, from times higher education, estimated a loss of around £3 billion if enrolment drops by 40 per cent, about £1.7 billion for a 20 per cent drop, and around £1 billion for a 10 per cent drop. 8 so, estimates to date suggest a potential fall in tuition fee income of between £2 billion and £3 billion. some universities are primarily teaching institutions, heavily reliant on student fees, and some are diversified research and teaching institutions. some are highly selective and have an international reputation, some are not. the london economics study emphasised the potential loss of teaching income not total income. if universities are to be compensated for the impact of covid-19, should they be compensated on the basis of the percentage of student fee income lost or the percentage of total income lost? the ordering of recipients for support would be very different depending upon the criterion used. the estimates of reduced student enrolment in the london economics study are driven by lower enrolment of foreign students, but also part-time students and graduate students. london economics did not find any association between graduate enrolment and declines in economic activity, but decided-given the likely depth of the recession-to assume there would be an impact on graduate enrolment. the estimated decline in foreign student enrolment is based on a finding that a 1 per cent decline in global gdp was associated with a 0.485 per cent decline in enrolment of international students in uk he. this assumption is particularly important for more highly ranked universities which attract many international graduate students. it is critical to understand how sensitive the london economics estimate is to the underlying assumptions. unfortunately, we do not have this information. there are also other factors that could offset the predicted decline in enrolment and funding. demographic change will help he because the number of eighteen year-olds is forecast to increase from 600,000 in 2020 to 760,000 in 2030. for example, postgraduate work rights are a major attraction for international students and an extension of such rights, as advocated by former minister for universities chris skidmore, could help attract more students to the uk. another factor that is likely to affect international enrolment is how well each receiving country is seen to handle the pandemic (the 'fear factor'). at present, the us and uk are lagging behind canada, australia, and new zealand. thus, there is enormous uncertainty about the financial impact of covid-19. it is likely to be far less than that implied by uuk and there are reasons to believe that the london economics estimate is too high. given this significant uncertainty, it is no surprise that the treasury was not forthcoming with additional support at this time. future support is likely to be contingent on the sector's actual financial position and what structural reforms it agrees to make. international student numbers have received by far the greatest attention, but it is also important to consider the enrolment of uk-domiciled students. will it fall? nick hillman, head of the higher education policy institute, and david willetts, former minister for universities, both argue that domestic enrolment will not fall. as hillman noted 'recessions tend to mean people want more education because the alternativesunderemployment or unemployment-are worse and having more skills can protect you against economic chill winds'. 9 this view is supported by research in the us and the uk. in the us an enrolment surge occurred during and immediately after the great recession, and a recent analysis published by the brookings institution found that a one percentage point increase in the us unemployment rate is associated with a 1.6 percentage point increase in university enrolment. 10 damon clark estimated 'youth unemployment effects that are positive, statistically significant, and large in magnitude -at least twice as large as previous estimates'. 11 clark's study is important because it encompasses earlier recessions-those of 1980 to 1983 and that of the early 1990swhen employment fell by more than in the great recession, and it focusses on the effects of local labour markets for youth. when job prospects are poor in the local labour market, students enter or stay on in higher education. thus, london economics' assumption of a small increase in enrolment of uk-domiciled students is probably too pessimistic and the agreed caps on enrolment will not help young people when the labour market for them is likely to be very poor. while covid-19 has clearly shown the financial dependence of many uk universities on international-especially chinesestudents, the british council had already forecast declines in chinese students studying in the uk because of increasing quality of chinese institutions, slower growth in household incomes, and a declining number of university-age people. care must be taken not to compensate for revenue losses that were to occur anyway. this could be an opportunity to adjust to possible future declines in chinese students. if the government decides to offer further support to he but it is insufficient to cover all institutions, who should be first in line? is it wise to protect lower ranked institutions at a cost to higher ranked institutions? the london economics study uses the categories established by a study by vicki boliver, who asked whether it was possible to identify distinctive clusters of higher and lower status uk universities. she found that oxford and cambridge stand out among the 'old universities' and form an 'elite tier' based upon research activity, economic resources, academic selectivity, and social mix, but score much more modestly on teaching quality. 12 the russell group universities cluster with the majority of the old universities to form a middle status tier. a quarter of the new post-1992 universities form a distinct bottom tier and the rest a third tier. expansion of higher education, restructuring, marketisation, internationalisation, and different educational policies across countries within the uk, have not altered the hierarchy of universities, because this hierarchy is deeply embedded in social structures and wider processes of social selection and social reproduction. and the same hierarchy is deeply embedded in the minds of foreign students. one could argue that any support offered to the he sector because of covid-19 should be concentrated on the higher status institutions, for that is where the international students are concentrated, where they crosssubsidise research, and it is these institutions upon which the uk's reputation for excellence is based. however, these institutions would generally suffer lower percentage falls in total income (although not necessarily in discretionary income). institutions that primarily serve uk-domiciled students should have relatively small losses and, if enrolment increases, may actually be in a stronger financial position than they expected to be. however, uuk wants to protect all institutions, at least in the short term. the treasury is unlikely to agree. saving institutions that were failing before covid-19 is not in the national interest, nor in the interest of students. what would be in the interest of students is courses and institutions that suit their needs, where those needs often consist of getting a good job at the end of the course. those courses and institutions do not all need to be pale imitations of the old universities. some universities are moving towards serving regional markets and providing applied courses. this should be encouraged, as should asking the question: 'do all university courses need to be three years long?' and, 'are there students who would benefit from further education instead of university?' this may be seen as elitist or going back to the pre-1992 he system, but it may be in the best interest of students and the nation. without enrolment caps, stronger institutions will attempt to replace foreign students by recruiting more domestic students. to do so, universities would enrol students who would have gone to a lower ranked institution in the absence of covid-19. such 'poaching' reduces the financial impact on more highly ranked universities, but it increases the losses of lower ranked institutions unless total enrolment increases. one can see that from an institutional perspective, capping enrolment is a cost to more highly ranked institutions, but a potential benefit to lower ranked institutions. but from a student's perspective, it takes away the possibility of attending a more highly ranked institution and, for some, the possibility of attending he at all. given that lifetime earnings are positively related to the quality of university attended, capping enrolment by institution provides support to lower ranked institutions at the expense more highly ranked institutions and students. the most disadvantaged students may benefit most from a drop in international students and may actually be able to attend a university that better suits their needs than was likely pre-covid-19. if universities can admit more uk-domiciled students, about 1,000 high-attaining disadvantaged students could be placed at better universities than if the currently agreed enrolment caps are put in place. if universities are able to replace overseas students with uk-domiciled students, disadvantaged students may benefit by being admitted to more selective universities. imposing enrolment caps, as proposed by uuk and agreed to by the treasury, will reduce upward moves, particularly for the most disadvantaged students. the he sector in the uk is clearly of significant economic importance. on average, there are significant private and public returns from he. uuk has predicted the likelihood of very significant losses to the sector in income from international students as a consequence of covid-19. however, losses will probably be substantially lower than the potential losses suggested by uuk. but the magnitude of losses is very uncertain. the losses from a fall in international students are concentrated on the most highly ranked universities, but because many of them are diversified, the percentage impact on total income is much less than the percentage impact on student fee revenues. for this reason, there may be pressure on wealthier institutions, as there has been in the us, not to accept government money. the uuk's proposal is to restrict undergraduate enrolment per university to stop institutions poaching students from universities less selective than themselves. this proposal is not in the interest of the most highly regarded universities or that of students. some rationalisation of the sector should be the price of any further government support and some suggestions for such rationalisation are contained in the augar report. covid-19 has shown the value of university research to the health of the nation, but it has also shown the fragility of that research because it is cross-subsidised by tuition fees from international students. now is the time to reconsider how university research is funded. the he sector is of clear importance to the nation, but so too are many other sectors. treasury support is likely to be selective and limited, so he needs to outline why the sector is more deserving of additional government support than other sectors. and he needs to be clear about how it will use any funds from the government and what it will give in return. cent in export earnings, and 31 per cent direct and indirect contributions to local economies) universities' plea for £2bn bailout falls on deaf ears in treasury social mobility and elite universities', hepi policy note 20 impact of undergraduate degrees on lifetime earnings, london, institute for fiscal studies post-18 review of education and funding: independent panel report higher education. a market like any other? skunks in an english woodland: should england embrace forprofit higher education? uk universities "face £2.6bn coronavirus hit with 30,000 jobs at risk"', times higher education poor handling of the pandemic "could damage student recruitment"', times higher education covid-19 could be a curse for graduates but a boon for universities coronavirus poses serious financial risks to us universities the uk labour market do recessions keep students in school? the impact of youth unemployment on enrolment in post-compulsory education in england are there distinctive clusters of higher and lower status universities in the uk? key: cord-033740-1axkt1jc authors: pradhan, archana; bliss, susan; buery-joyner, samantha; craig, latasha; everett, elise; forstein, david; graziano, scott; hopkins, laura; mckenzie, margaret; morgan, helen; page-ramsey, sarah; hampton, star title: to the point: integrating the obstetrics and gynecology core clerkship into a longitudinal integrated curriculum in us medical schools date: 2020-10-14 journal: med sci educ doi: 10.1007/s40670-020-01093-x sha: doc_id: 33740 cord_uid: 1axkt1jc this article, from the “to the point” series that is prepared by the association of professors of gynecology and obstetrics (apgo) undergraduate medical education committee, is a review of issues to consider when integrating obstetrics and gynecology into a longitudinal integrated clerkship (lic) model. the background, objectives, and outcomes related to lic are discussed. specific challenges and solutions for structuring an lic in the context of obstetrics and gynecology are examined, which include student/faculty issues as well as process matters such as general infrastructure and specific curriculum concerns. the origins of the lic are rooted in the rural physician associate program (rpar) at the university of minnesota in the 1970s as a means to address rural physician workforce shortage [1] . despite the success of rpar, it was not until the 1990s that a handful of medical schools attempted to replicate the lic model. the lic curriculum replaces a medical student's traditional core specialty-based clerkships with an integrated clerkship model. lic students are exposed to multiple specialties simultaneously. all lic models have three basic tenets: (1) students need to be exposed to all core clinical training issues to which clerks on the traditional clerkship model are exposed, (2) students should participate in the comprehensive care of patients over time and (3) clinical clerks ought to develop mentoring relationships with these patients' clinicians [2] . the carnegie foundation report in 2010 recommended students have more integration in content, a better mechanism for professional identity formation, and means to foster scientific inquiry and individualized learning. lics can offer many of these opportunities [3] . now that early adopter schools have produced 40+ cumulative years of positive outcomes data, many medical schools across the usa are attempting to implement the lic curriculum as a possible solution to address the national shortage of primary care physicians, to enhance patient-physician communication and continuity, to improve academic results, and to develop more meaningful relationships and feedback with faculty mentors and patients [4, 5] . for many of us, participation in the lic curriculum as obstetrics and gynecology educators is not optional and can be a challenge to implement. the fast-paced nature of the specialty and varied faculty staffing models can make continuity of care and continuity of student-faculty interactions difficult to schedule. however, it is important to keep in mind that the implementation of the lic curriculum may be a strategy to increase the number of students choosing obstetrics and gynecology as a specialty [6] . additionally, it gives us the opportunity to appropriately train primary care physicians in basic women's health issues. many outcome studies have evaluated lic programs with regard to academic performance [7] . academic results (as measured by summative student assessments) of students on lic tracks were better or equivalent to students on traditional clerkship tracks. academic results that were evaluated were summative student assessments such as institution developed subject examinations, national board of medical examination (nbme) subject scores, and united states medical licensing examination (usmle) performance [8] . for example, a comparison of ob-gyn mean nbme scores between students at the yankton 12-month integrated longitudinal program compared with students from traditional obgyn clerkships at the university of south dakota showed no significant difference. additionally, the students in the integrated program also had better long-term learning retention [9] . similarly, students on the traditional ob-gyn clerkship at the university of north dakota had an average nbme score of 73.9, while their lic counterparts had an average score of 74.8 [10] . few studies have drilled down and compared other parameters specific to the obstetrics and gynecology clinical clerks on the lic track versus the traditional clerkship track. a study which compared student experience on the traditional obstetrics and gynecology clerkship with single supervisor/ elimination of fragmented activities experience showed that students on the revised experience performed twice the amount of obstetric procedures and a significantly enhance student-faculty relationship [11] . a study out of the university of hawaii that tracked lic students versus traditional block students confirmed that students on the lic obgyn clerkship logged in more clinical experiences than traditional block ob-gyn clerkship students [12] . many studies have examined overall clinical performances, learner experiences, career choices, clinical supervision, and patient perception between all students on the two tracks. objective standardized clinical examination (osce) scores as a proxy for clinical performance showed that lic curriculum students scored higher than traditional track students [13] . lic students report improved learner experiences in several different ways. "students report having increased patientcentered skills including better developed clinical communication skills, and demonstrate a deeper understanding of the psycho-social component of the biopsychosocial model." [8] . preceptors note a high level of student productivity as the lic experience progresses [14, 15] . clinical supervision outcomes from the lic faculty perspective are extremely positive [16] . at ucsf, with regard to clinical supervision from the student viewpoint, lic students rated clerkship experiences with regard to faculty observation, faculty teaching, feedback, and clerkship overall higher than their traditional clerkship counterparts [17] . faculty familiarity with students allows educators to tailor their teaching to an individual student's needs. additionally, lic students report increased confidence in their clinical skills and feel that they are better prepared for internship [18, 19] . importantly, these benefits of an lic education are reported to endure through residency and into practice [20] . as far as career choice is concerned, students in the lic track are more likely to ob-gyn as a specialty than students in the traditional track [21] . furthermore, patients who were taken care of by lic students valued continuity relationships with students much as they valued continuity relationships with providers [22, 23] . the overwhelmingly positive outcomes data has led to the call for the development of lic curricula at medical schools across the country. in order to implement an lic program successfully, several issues need to be considered prior to committing to develop this type of curriculum. many of the lessons learned are from colleagues in other specialties who have led lic curriculum implementation across the world. these topics can be categorized as student/faculty issues, general infrastructure logistics, and specific curriculum concerns. if these issues are not addressed from the outset, it is highly unlikely that a medical school can sustain an lic track. choosing the correct students and providing appropriate support to students in the lic cohort is critical [24] . lic students must be able to adapt through quick changes and be resilient through times of confusion en route to developing professional competence and confidence [25] . students who are enrolled in the lic track may experience initial anxiety at the prospect of learning core material in multiple disciplines simultaneously. lic students do not receive final formal assessments until the end of the year and this too can make students question whether they are acquiring the core knowledge required of a junior clinical clerk. lic students also need to be self-directed learners as they are given the freedom to follow individual patients to various specialty visits and procedures. an appropriate selection process that seeks out mature learners who have expressed a sincere interest in pursuing a career in a primary care specialty is a key component of developing a successful lic program [17, 26] . similarly, identifying faculty members who have "bought in" to the goals of the lic is important to the success of this non-traditional curriculum [27] . some medical schools may have faculty who are resistant to change such as a new curriculum, and thus will need support from many levels, including the dean's office and administration [28] . the traditional immersive curriculum is how most physicians were trained. transitioning to a new model requires education for educators and administrators and an understanding of the benefits and challenges of the longitudinal curriculum [29] . preceptors initially may need increased faculty development to understand how to deliver the same content in this new format. an innovative study texted brief faculty development podcasts to community-based lic preceptors; 70% reported that the intervention affected their teaching style and they were likely to listen to future podcasts [30] . as lic students are learning about several core specialties simultaneously, preceptors need faculty development on how to monitor and ensure that these students develop and retain specialty-specific skill sets. faculty development focused on teaching around continuity, connection, communication, coaching, commitment, care, and community has been identified as a practical framework for lic preceptors [31] . a unc-asheville study interviewed 4 cohorts of lic students and developed the following 7 themes to develop teaching tips for faculty: enhance teaching; create bring-backs (learning topics to bring back to preceptors); support patient ownership; structure clinic; improve feedback; engage learners; and develop a relationship [32] . implementation of an lic curriculum also needs to account for impact on faculty preceptors. budget to attract and compensate these preceptors should be established [17] . serving as an lic clerkship site is a rewarding experience; however, preceptors have reported seeing 1-3 patients less per session or starting patient hours earlier to accommodate for the increased time required to teach [33] . because of the one-onone teaching aspect of an lic, concerns that it may be more costly to implement have arisen. hudson and colleagues found that students can be cost-neutral or even favorable to an office if the student is there for at least two months, since the student becomes an integral member of the team [34] . faculty do report a high level of satisfaction with the student-preceptor relationship [14, 15] . additionally, a study of faculty and nurses who participated as lic preceptors reported increased reflective practice, improved value of professional identity, and increased enthusiasm for interprofessional learning [35] . creating an infrastructure that can support the learners and the teachers is a critical element in the development of an lic. community settings provide more opportunities for lic students to engage in patient care activities [36] . in addition, community settings may allow part-time physicians to have a more active role in medical student education, especially if the student is only with them a half-day per week. when implemented in a tertiary care setting, students may be placed in specialty or subspecialty clinics which may not afford the students the opportunity to experience the common patient conditions. it is important to have built in time where students can have the flexibility to enrich their patient encounters and schedule clinical experiences which they may not have previously seen. for example, a student may have a patient with fibroids who is being scheduled for a uterine artery embolization in radiology. the student's ability to observe the patient's procedure will enhance the learning. because of the varied schedule, it may take students longer to "get in the flow"; however, once they do, their experiences become more solidified with long-lasting results. lics have been successfully implemented in both rural and urban settings [37] . in any setting, it is important that the clinical site (and its administrative support team) is aligned with the mission of the medical school, while having the financial support to host a medical student. scheduling the varied aspects of an lic can be complicated. each student may have a different weekly schedule, which needs to be juggled. having support staff who understand these complexities is invaluable in maintaining and adjusting schedules as needed. they can also keep track of preceptors' availability, and how many preceptors are needed in each specialty at any time. some challenges which were reported [38] in the initial pilot year included underestimating time requirements, finalizing the details of the model (# of preceptors, scheduling weekly activities), and the transition from one process to the next. the leadership team needs to be able to adapt and be flexible, in the initial phase. there have not been any studies looking at the optimal schedule to learn the ob-gyn material; thus, there is room for adapting the schedule to optimize the strengths at an individual institution. for example, some programs have ob-gyn purely as a longitudinal experience, while others have it mixed with an inpatient block rotation. the scheduling structure can be variable based on whether a medical school has lic learners and traditional learners at the same site or a completely different location. this is dependent on an institution's curriculum objectives, as well as faculty/affiliation resources, i.e., does a particular regional site have all the core specialties represented? lic learners are immersed in clinical experiences during the clerkship phase of an individual medical school's curriculum, so these students are usually second or third year medical students. scheduling educational sessions throughout the preclerkship and clerkship years will permit lic students to (1) develop a community within the students on the lic track and (2) delve into educational cases which they have not yet experienced. interdisciplinary simulations between obstetrics and pediatrics can efficiently address requirements in multiple fields and promote deep learning [39] . for students who are placed in the lic track, encouraging and monitoring the educational process can be quite intensive. students want and need regular feedback, and this can be provided on a regular basis with the continuity of preceptors who will observe a student's growth over time. periodic (perhaps quarterly) feedback to assess the student's overall performance including patient care, medical knowledge, professionalism, lifelong learning, systems-based practice, and communication skills should be provided [40] . documenting comparability between clerkship sites within a medical school is critical for lcme accreditation [28] . an important element of this is making the goals and objectives clear, and thus, the experiences expected are transparent. frequent monitoring of students' experiences allows for early intervention to assure that they are achieving the required experiences in their longitudinal curriculum. with more electronic tracking resources available, this can be achieved and monitored with a central system. ideally, students can log their clinical experiences and faculty can review them regularly. an electronic system in which the students can view their progress and evaluations in a timely fashion could facilitate prompt feedback with correction of errors. the use of tablets can also support a student's individual learning style and help the student be a self-regulated learner [41] . it is important to keep in mind that students on the traditional or lic ob-gyn clerkship are not required to have the exact same opportunities-but they do have to have comparable experiences. apgo has a plethora of resources such as medical student objectives, a curriculum building tool, and a logbook which can be used to ensure that students on lic and traditional track receive comparable educational experiences [42] . student assessment should be based on the same goals and objectives as in a traditional-based curriculum. since there may be fewer individual evaluators for any one student, preceptor development is important to ensure objective evaluation. utilizing a standardized evaluation form with behaviorally anchored data points and comments can help with objectivity. lic students tend to get more direct and immediate feedback, since they are working with the same preceptor over time. students perceive this feedback as authentic and constructive because of the continuity of the preceptor [43] . the curriculum should be evaluated regularly, which allows leadership to address any gaps or unanticipated oversights that may have arisen during the implementation of the lic. just as a traditional curriculum measures its efficacy regularly, so should the lic. knowledge, skills, and attitudes of the students are all important domains to assess. objective measures of student knowledge can be assessed and compared over time, using standardized exams, including nbme subject exams and usmle step 2 ck. the nbme comprehensive science examination has been shown to be comparable to the individual subject examinations [44] . students' skills are generally tracked using case logs, objective structured clinical examinations, and usmle step 2 cs. students' attitudes can be measured using validated tools such as the professional quality of life scale (proqol 5) [45] which measures compassion satisfaction and compassion fatigue; ethics position questionnaire (epq) [46] which measures ethical ideology along two dimensions, relativism and idealism; and the communication, curriculum, and culture survey (c3) [47] which measures the professional learning climate or hidden curriculum. implementation of a successful lic requires planning, buy-in from the stakeholders, and ongoing monitoring of the program to assess its effectiveness and comparability to a traditional block program. a final consideration for lic implementation is the anticipation of situations in which a student needs to take extended time off, and how to manage when they will re-enter the curriculum. remediating the struggling student must also be considered and incorporated into the planning. planning for unforeseen circumstances for curriculum interruption in advance will alleviate stress when this inevitable situation arises [48] . the lic model can be an extremely rewarding and beneficial way to deliver clinical experiences to medical students. it is an immersive educational experience in which students benefit from the continuity of a learning community comprised of preceptors, patients, and peers over time. studies show that the lic track is at least as effective as the traditional model when we measure medical knowledge and objective assessments. this model appears to promote students' independent learning and decrease ethical erosion and exposure to the "hidden curriculum," thus addressing some of the pressing concerns regarding the future of medicine. data availability not applicable. conflict of interest the authors declare that they have no conflict of interests. ethics approval not applicable. code availability not applicable. the rural physician associate program: successful out-comes in primary care and rural practice consortium of longitudinal integrated clerkships educating physicians: a call for reform of medical school and residency better learning, better delivery systems: possibilities from a case study of longitudinal integrated clerkships diffusion of innovation and longitudinal integrated clerkships: results of the clerkship directors in internal medicine annual survey longitudinal continuity learning experiences and primary care career interest: outcomes from an innovative medical school curriculum academic outcomes of a community-based longitudinal integrated clerkships program outcomes of longitudinal integrated clinicians and society comparison of third-year student performance in a twelve-month longitudinal ambulatory program with performance in traditional clerkship curriculum performance of medical students in a nontraditional rural clinical program 1998-99 through relationships matter: enhancing trainee development with a (simple) clerkship curriculum reform the impact of a longitudinal curriculum on medical student obstetrics and gynecology clinical training educational outcomes of the harvard medical school-cambridge integrated clerkship: a way forward for medical education demonstrating the value of longitudinal integrated placements for general practice preceptors it's all about relationships faculty experience and engagement in a longitudinal integrated clerkship development of a longitudinal integrated clerkship at an academic medical center comparing a longitudinal integrated clerkship with traditional hospital-based rotations in a rural setting effectiveness of a rural longitudinal integrated clerkship in preparing medical students for internship into the future: patient centredness endures in longitudinal integrated clerkship graduates obstetrics and gynecology medical student outcomes: longitudinal multispecialty clerkship versus traditional block rotations patient views of continuity relationships with medical students patient perceptions of innovative longitudinal integrated clerkships based in regional, rural and remote primary care: a qualitative study towards an understanding of medical student resilience in longitudinal integrated clerkships transition processes through a longitudinal integrated clerkship: a qualitative study of medical student experiences who should choose a rural lic: a qualitative study of perceptions of students who have completed a rural longitudinal integrated clerkship twelve tips for designing and running longitudinal integrated clerkships cracking the nut on lcme standard 8.7: innovations to ensure comparability across geographically distributed campuses unfreezing the flexnerian model: introducing longitudinal integrated clerkships in rural communities texting brief podcasts to deliver faculty development to community-based preceptors in longitudinal integrated clerkships teaching in longitudinal integrated clerkships: the seven c's preceptor teaching tips in longitudinal clerkships burden, responsibility, and reward: preceptor experiences with the continuity of teaching in a longitudinal integrated clerkship medical students on longterm regional and rural placements: what is the financial cost to supervisors? rural remote health what is the impact of longitudinal rural medical student clerkships on clinical supervisors and hospitals? australian consortium of longitudinal integrated clerkships clinical participation of medical students in three contemporary training models a typology of longitudinal integrated clerkships challenges in transformation of the "traditional block rotation" medical student clinical education into a longitudinal clerkship model shoulder dystocia and neonatal resuscitation: an integrated obstetrics and neonatology simulation for medical students perceptions of evaluation in longitudinal versus traditional clerkships using tablets to support self-regulated learning in a longitudinal integrated clerkship student perceptions of assessment and feedback in longitudinal integrated clerkships comparability of the national board of medical examiners comprehensive clinical science examination and a set of five clinical science subject examinations treating compassion fatigue a taxonomy of ethical ideologies characterizing the patientcenteredness of hidden curricula in medical schools: development and validation of a new measure creating a longitudinal integrated clerkship with mutual benefits for an academic medical center and a community health system publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-337060-jkzbyqtm authors: alqurshi, abdulmalik title: investigating the impact of covid-19 lockdown on pharmaceutical education in saudi arabia – a call for a remote teaching contingency strategy date: 2020-08-03 journal: saudi pharm j doi: 10.1016/j.jsps.2020.07.008 sha: doc_id: 337060 cord_uid: jkzbyqtm background: covid-19 lockdown has forced pharmacy education to be conducted remotely for approximately half of the second semester in the year 2019/2020. this sudden shift to distance learning has put the pharmacy education system through an extraordinary experience that may impact its future. objective: to investigate the effect emergency remote teaching has had on pharmacy education in saudi arabia, and to provide recommendations that may help set in place a contingency strategy. methods: two cross-sectional likert-scale based questionnaires targeted at students and teachers separately, designed to explore stakeholders' satisfaction in three areas of emergency distance teaching/learning: the use of virtual classrooms, completion of course learning outcomes (clos) and assessment via alternative methods during the covid-19 lockdown period. furthermore, phone interviews were conducted with teachers and students to discuss results from both questionnaires for further clarity on teacher and student views. results: over 700 pharmacy students, from 19 different local colleges, and 74 faculty members from 10 different local colleges have participated in this study. while it was challenging for the majority of teachers (>60%) to delivery complex scientific concepts over virtual classrooms, >35% of students and 60% of teachers have expressed concerns on the lack of student–student and student–teacher interactions. a factor that has shown a significantly negative correlation with student overall satisfaction (p < 0.01). emergency remote teaching has forced teachers to alternative assessment methods, which the majority (70%) believe had a positive effect on students' overall skills. almost half of students (45%) were concerned by the lack of guidance accompanied by unfamiliar methods of assessments. conclusions: based on statistically analysed results from cross sectional likert-scale questionnaires aimed at stakeholders of pharmaceutical education, this study concludes with a number of recommendations that may help pharmacy colleges seize this unique opportunity to further enhance the quality of pharmacy education in saudi arabia. prior to 2001, the college of pharmacy at king saud university was the only college that offered an undergraduate pharmacy related degree program in saudi arabia (alhamoudi and alnattah, 2018) . since then, a sharp increase was observed in the number of pharmacy colleges (alhamoudi and alnattah, 2018) . according to a report published by the saudi commission for health specialities (scfhs, 2018) the number of pharmacy colleges in saudi arabia has reached 27 in 2018 hosting over 14,000 students (scfhs, 2018) . this came as a result of a government funded and managed mission of advancing higher education in the country to meet national demands in various sectors, such as healthcare and the pharmaceutical industry (alhamoudi and alnattah, 2018) . while pharmaceutical education in saudi arabia is relatively young, compared to that of other countries such as the uk and the usa, it has witnessed a significant growth and development in its quality, due to the continuous revision and standardization process encouraged by various boards and councils (asiri, 2011) such as the national centre for academic accreditation and evaluation (previously known as ncaaa). furthermore, experts in the field in saudi arabia, have recently started to publish their experiences in developing pharmaceutical education as part of acquiring national and international accreditations (alkatheri et al., 2019) . according to the accreditation council for pharmacy education (acpe), so far, four doctor of pharmacy (pharm d) programs offered in saudi arabian institutions have been accredited internationally (acpe, 2020) . the challenge all educators in saudi arabian institutions and around the world are currently tackling is the sudden shift to distance learning enforced as a measure to prevent the spread of covid-19. from the 8th of march 2020 the saudi arabian ministry of education has directed all schools, colleges, and universities in the country to indefinitely suspend student attendance, and to activate distance learning as an alternative. henceforth, teaching was carried out remotely and replaced by virtual classrooms. this was possible via online learning management systems such as blackboard and microsoft teams. universities around the country spent great efforts launching a variety of online training sessions and providing basic manuals to equip academics for online teaching. this all started taking place after 7 weeks into the second term, where a minimum of 8 weeks of teaching remained. as directed by the minister of education and university presidents, program management teams across all universities in saudi arabia have held emergency meetings to handle this sudden change of teaching strategy. the common goal was to ensure quality of teaching is maintained by assessing all course learning outcomes (clos) and using appropriate methods that account for the limitations of remote teaching. furthermore, to ensure student satisfaction, the ministry of education, through university admissions, has directed all program management teams to modify the distribution of assessments, assigning 80% of the total course mark to coursework related activities, with the remaining 20% allocated to final assessments. lecturers were given a wide range of evaluation methods to choose from. for coursework related activities, examples include reports, essays and student presentations. the final assessment included oral examinations, open book examinations, and online multiple-choice questions. although most pharmacy program management teams did not have a contingency plan in place for such circumstances, with the guidance of the ministry of education and university managerial teams, the teaching and assessment periods of the second term in the academic year 2019/2020 were successfully completed. whilst measuring the level of success for this accomplishment may be possible through established program key performance indicators, this study investigates the effects these extraordinary circumstances have had on pharmacy education. here the author acquires both student and teacher feedbacks on the experience, its difficulties and successes. this study aims to help in forming a basis for developing a contingency strategy for emergency remote teaching in the field of pharmaceutical education in saudi arabian universities. a cross sectional pharmacy student-targeted online questionnaire was developed using google forms. in attempts to ensure only pharmacy students' response, an opening question of the program name was used to filter out non-pharmacy students. followed by a few demographical questions as well as determining the institution's name and program year in which the participant is currently in. the questionnaire was designed to measure students' satisfaction, using 5 point likert-scale questions (joshi et al., 2015) , in three different areas: virtual classrooms, completion of course learning outcome (clos) and alternative assessments. the experience of graduating students was also explored via questions on pre-registrational training during the lockdown phase. in each area, the questionnaire was designed to identify major challenges and obstacles students have faced during the distance learning. furthermore, participants were provided with an ending segment to add any relevant comments. as part of the development process, the questionnaire was revised and amended post pilot testing on a group of volunteer students. a second teacher-targeted cross-sectional questionnaire was developed to measure their satisfaction in the same three areas of interest. to avoid participations from academics in all areas, the questionnaire was adapted with an opening filtering question considering only faculty members that have participated in teaching and assessing pharmacy students, in saudi arabian institutions, during the lockdown. the questionnaire included ending segments to allow teachers to further add any comments or suggestions they see fit. the questionnaire was put through a pilot test by a group of volunteer academics to revise and amend. the study was reviewed and approved by the ethics committee of taibah university. using the electronic platform ''google forms" both questionnaires were sent to 27 different institutions in saudi arabia. the teacher-targeted questionnaire was directed to pharmacy program managerial teams for distribution amongst faculty members, whilst the student-targeted questionnaire was directed to student group leaders to distribute amongst pharmacy students from all program years. both questionnaires were conducted during may 2020. awarded student marks, from the past three years, in four randomly selected courses in different areas of pharmaceutical sciences, were collected from consenting volunteer faculty members. student names and id numbers where erased to protect their privacy. to further clarify the students' and teachers' voices and comments from open-ended questions, individual interviews were conducted remotely with three students and four teachers from the college of pharmacy in taibah university. in the interviews, results from both questionnaires were discussed with all participants. conclusions were drawn, recorded, and used in both the results and discussion parts of this study. diverging stacked bar charts were used to help present results from likert-scale questions in both questionnaires. awarded marks as well as student satisfaction scores were presented using a box plot. coefficient analysis between students' satisfaction and variables of likert-scale ordinal output data, was conducted using non-parametric spearman's rank correlation on ibm ò spss statistics software. responses from the student-targeted questionnaire included 703 students from 19 different saudi arabian colleges, covering most saudi arabian regions, 21% of which were from taibah university (table 1) . this included students from all levels and stages of undergraduate programs, pharm d and bachelor of pharmacy (b pharm), including the pre-registrational training year (table 1) . student sample size represents approximately 5% of the population, as the total number of undergraduate pharmacy students in saudi arabia is estimated at 14,004 (scfhs, 2018). the teacher-targeted questionnaire on the other hand, yielded only 74 academics from 10 different colleges (table 1) . assuming a student to teacher ratio of 18 to 1, as stated by the ministry of education to be the average student to teacher ratio in saudi arabian colleges (moe, 2011) , the author estimates the total number of members in pharmacy faculties to be 778, therefore the teacher based sample in this study represents an estimated 10% of their population. although a higher percentage of teachers than students have participated, the number of colleges represented in the teacher-targeted questionnaire output is much lower. this may indicate that teachers were occupied with post-teaching activities such as the submission of course portfolios. additionally, the author has noticed many circulated research-based questionnaires investigating varies aspects in relation to covid-19, this may have lowered participation interest in general. results from the 5-point likert-scale questionnaires were presented using diverging stacked bar charts in figs. 1 and 2 for students and teachers, respectively. participants with neutral responses were positioned at the centre of the horizontal axis. agreeing and disagreeing participants where positioned on the right and left sides of the horizontal axis, respectively. responses to various statements were grouped under 4 main categories: virtual classrooms, course learning outcomes (clos), alternative assessment methods (fig. 1&2) and training via alternative activities ( fig. 1 only) . individual interviews were conducted remotely with three students and four teachers all from the college of pharmacy at taibah university. these interviews included a discussion on questionnaires output, specifically comments and ideas posted by participants in open-ended questions. results from interviews has been incorporated into the following sections. results show > 20% of students and > 30% of teachers reported difficulties with internet connection, and thus attending and maintaining presence in virtual classrooms (fig. 1&2 ). while attendance may be an issue for some, most teachers (>60%) agree that it was difficult to use distance learning tools in delivering complex scientific concepts (fig. 2) . student-teacher interactions were reported to be limited, by > 35% of students and almost 60% of teacher participants, causing difficulties for students to understand lecture content (fig. 1&2 ). interviews with some of the student representatives have also revealed that, in virtual classrooms, student-student interactions have also been limited, thereby limiting peer collaborations which otherwise many students depend on in normal circumstances. approximately 35% of students agree that it was more difficult to concentrate during virtual classrooms, rendering them less effective ( fig. 1) . furthermore, student representatives in an interview have explained that although a student maybe signed into a virtual classroom, they may be distracted by other applications especially if they are attending via their smart phones. while very few teachers have reported previous experience with virtual classrooms, for the majority, this was their first attempt of being on the teaching side ( fig. 1 ). learning how to use new tools and coping with the pressures of distance teaching has split teacher opinion on whether such tools are appropriate for pharmacy related subjects (fig. 1 ). due to not being able to attend laboratories, psychomotor based clos were not completed. in addition, approximately 20% of teachers have admitted to not being able to completely cover clos (fig. 2) . >35% of students believe they have gained limited knowledge and skills, similarly, almost 40% of teachers believe that students have not gained all essential knowledge and skills during the lockdown period (fig. 1&2 ). >57% of students and 45% of teachers have shown interest in the idea of conducting revision sessions for essential course learning outcomes next semester (fig. 1&2) . table 1 demographic details of participants in both questionnaires (n = 777). data collection was completed through google forms. as the links to the teacher-based questionnaire and student-based questionnaire were sent directly to program management teams and student leaders respectively. out of the 27 targeted pharmacy colleges in the country, 703 students from 19 different institutions and 74 academics from 10 different institutions have responded. student sample is representative of the population in terms of gender, program title and year of program. the academics sample includes all teaching positions. no. of participants (%) gender while the most common conventional method of assessment may be written examinations, the lockdown period has necessitated alternative remote methods. in addition to online multiplechoice question (mcq) exams, teachers have resorted more to assessment via written assignments and online oral examinations. this new experience has alerted teachers to the benefits of these alternatives, as 70% believe they can enhance student overall skills (fig. 2 ). this has prompted the majority of teachers (>60%) to consider a reduction in the number of exams while increasing alternative methods of assessment for courses in the future (fig. 2) . while teachers were split on the practicality of online examinations, most students did not favour written assignments and oral examinations (fig. 1&2 ). student representatives in interviews explained that many students had limited experience in writing essays and or reports, therefore undergoing such assessment required a substantial effort, especially when a plagiarism check is required. furthermore, they explain that oral exams were very difficult to prepare for, as the scope of the exam can be very wide. in many cases students were not presented with what to expect in an oral exam, thus preparations and revision were less effective. a lower proportion (<25%) of students were unfavourable to assessment by online seminar presentations, while some may have struggled due to poor internet connections (fig. 1) . student representatives attribute this response to the limited experience students have in giving presentations in general. students have also shown concern towards the high number of assignments given during this lockdown. student representatives explained in interviews that a sudden influx of assignments was pushed in a very limited period. a student representative stated the following in an interview: ''we believe that each teacher was unaware of the assignments given to us by his colleagues". suggesting they wanted some informed knowledge, as assurance, on college oversight of academic staff in terms of managing the total number of assignments. approximately 25% of students claimed to face difficulties in understanding the requirements of some assessment methods (fig. 1) . furthermore, several student interview participants have claimed that the evaluation criteria for some assessments were unclear, which they believe caused some of the marking processes to be susceptible to subjectivity. others have commented that not fully understanding the reasons behind loss of marks has caused students distress and the feeling of unfairness. when students were asked whether assessments performed during lockdown were fair and proportionate, almost 40% answered ''no" (data not presented). however, when comparing student marks for the past three years (fig. 3) the cohorts assessed during covid-19 fig. 1 . diverging stacked bar chart to show results from a 5-point likert-scale student-targeted questionnaire to measure student level of agreement with several statements in 3 categories concerned with the educational process during covid-19 lockdown, these included: virtual classrooms, completion of clos and alternative assessment methods (n = 363). graduates were also asked to participate with their feedback on training via alternative activities (n = 340). full statements from the questionnaire were abbreviated above due to limited spacing. full statements are as follows: (s1) i struggle with attending virtual classrooms due to internet connection problems; (s2) with remote teaching i have had limited interactions with teachers causing difficulties in understanding the lecture materials; (s3) virtual classrooms are ineffective as one can easily be distracted; (s4) during covid-19 lockdown i was unable to absorb all knowledge and skills associated with this term and as a result i fear it will affect my performance as a pharmacist in the future; (s5) number of assignments requested during the covid-19 lockdown were very high with limited time for completion; (s6) witten assignments such as essays and reports were not suitable for this duration as they're very time consuming and require a lot of effort; (s7) oral exams were a suitable alternative to written exams as they allow demonstration of knowledge and skills with ease; (s8) online seminar presentations are suitable alternative methods of assessment; (s9) i had difficulty understanding the required actions for the assessment methods used during the lockdown; (s10) completing training program requirements with alternative activities will negatively impacts my future professional performance; (s11) completing training program requirements with alternative activities will reduce my chance of obtaining a job. lockdown 2020 appear to have gained significantly higher marks in all courses (p < 0.05). marks awarded during lockdown were densely populated in the upper part of the grading scale (80% -100%), thus indicating less discrimination between student marks and perhaps some inaccuracy in measurement of clos (fig. 3) . this comes in agreement with teachers' responses as a majority (>70%) believe that assessments this semester were less accurate in measuring student attainment of clos, and therefore do not entirely reflect students' knowledge and skills (fig. 2) . both pharmacy programs, b pharm and pharm d, include a training period that involves working in hospitals, community pharmacies and pharmaceutical industrial facilities. results from the student-based questionnaire indicate that, because of the lockdown, most trainees (96.5%) (data not presented) were asked to stop attending training locations and instead were assigned alternative activities and assessments that mimic training programmes. while most students do not believe it will have a significant effect on their performance as future pharmacists, at least 26% of trainees believe that it may reduce their chances of employment in the foreseeable future (fig. 1) . using a 5-point scoring system, with 1 being the lowest part of the scale, students were asked to express their level of satisfaction with the performance of their college in managing educational processes during the lockdown period (fig. 4) . results show 9 colleges (abbreviated with letters b, c, d, h, j, k, n, p and r) with majority responses ranging from 5 to 3, thus indicating a fairly good student satisfaction. six colleges (abbreviated with letters a, e, g, l, m and o) seem to range between satisfied and unsatisfied students. out of the 18 colleges, only 3 (abbreviated with letters f, i and q) seem to include some very unsatisfied students. responses for some of the colleges included a very wide range (1-5) such as the college abbreviated with the letter q. this may indicate inconsistency in management performance, while colleges with a score range of 2 points or less may have been consistent with their performance. results from a coefficient analysis of non-parametric spearman's rank correlation (table 2) indicate that major factors negatively influencing student satisfaction include limited student -teacher interaction and perceived ambiguity in assignment instructions (p < 0.01). although unrelated to university performance, statistical analysis shows internet connection problems as well as difficulty with concentrating in virtual classrooms, has lowered student satisfaction to some degree (p < 0.01) ( table 2) . diverging stacked bar chart to show results from a 5-point likert-scale teacher-targeted questionnaire to measure teachers' level of agreement with several statements in 3 main categories concerned with the educational process during covid-19 lockdown, these included: virtual classrooms, course learning outcomes (clos) and alternative assessment methods (n = 74). full statements from the questionnaire were abbreviated above due to limited spacing. full statements are as follows: (s1) prior to the covid-19 quarantine phase, i had very little experience in using virtual classrooms; (s2) after using virtual classrooms this semester, i believe it to be an appropriate teaching tool for most pharmacy related courses; (s3) i struggle with internet connection and thus find it difficult to stay connected to a virtual classroom; (s4) explaining complex scientific concepts to students through virtual classrooms is difficult and requires more time than conventional teaching; (s5) interacting with students in virtual classrooms to ensure they have understood certain concepts is difficult to achieve; (s6) during this phase, i believe i was able to cover all learning outcomes, except for psychomotor ones; (s7) most students have gained all required knowledge and skills; their performance as pharmacists in the future will not be negatively affected by the covid-19 quarantine phase; (s8) distance learning tools, such as the blackboard, are impractical & ineffective for exam assessments; (s9) due to covid-19 quarantine, it was very difficult to assess students' abilities and performance with regards to course learning outcomes; (s10) awarded grades this semester are not an accurate reflection of students' knowledge and skills; (s11) prior to covid-19, assessment methods were mainly limited to mcq based exams; (s12) mcq exams are less accurate in the assessment of student skills; (s13) introducing students to a wide range of alternative methods of assessments this semester will have a positive impact on their overall skills; (s14) i recommend that methods of assessment, stated in course specifications, are changed to include less exams and more assignments. the covid-19 pandemic has forced college academics across the world to remotely teach and assess students in all levels and fields of higher education institutions. the ministry of education in saud arabia has supervised the transition across the country and contributed to its efficiency. the presence of a strong infrastructure for distance learning such as internet connectivity, subscriptions to various e-systems and the readiness of experts in every institution has made it possible to continue teaching during the lockdown period. many faculty members in health care education had limited experience with distance learning tools, however, administrative teams made a swift and extraordinary effort to train all on the basic use of software such as creating virtual classrooms and online exams on blackboard and microsoft teams. although the efforts were outstanding, faculty members were short for time, as many were split between learning how to operate new technologies and teaching. results from this study indicate that pharmacy faculty members have gained considerable experience in the basic use of distance learning tools, yet there remain many features that can enhance that experience and perhaps help teachers and students overcome some of the observed challenges. both teachers and students have identified the lack of human interactions to influence their ability to remotely teach and learn, respectively. statistical analysis has shown that the lack of interaction has been a significantly negative factor in student satisfaction. research in education has proposed that interactions between students themselves and between students and teachers is fundamental for effective distance learning (moore, 1989; hunter, deziel-evans, and marsh, 2003) . furthermore, contemporary education values student-student interactions considerably, as learning can be enhanced when students share their findings with each other (hunter, deziel-evans, and marsh, 2003) . embedding interpersonal skills in health education has also been known to help prepare caring, competent practitioners (bischoff et al., 1996) . this challenge may be overcome by utilizing available features in distance learning e-systems, such as discussion forums, to help boost student-teacher and student-student interactivity. this in turn may help students engage and focus better in virtual classrooms, an issue many students have faced during the lockdown. adopting collaborative teaching and learning strategies can encourage group-learning and prevent learning in an isolated environment. unlike traditional teaching, remote teaching has the advantage of technology incorporation, where various teaching strategies may easily be implemented. one of which is the ''flipped classroom", where lectures may be pre-recorded and posted for students to watch at their convenience (mclaughlin et al., 2013) . in place of virtual classrooms, a virtual discussion room can be utilized to accommodate students' collaborative efforts in learning. problem-based learning (pbl), amongst student-centred teaching strategies, is another example for a suitable method to use in distance learning. not only would it encourage exchanges, but it can also propel students to be independent learners as opposed to memorizing information of traditional lecture formats (cyr, 1999; camp, 1996) . published studies further showed students taught using the pbl strategy to be motivated and self-directed life-long learners, essential characters for pharmacist practitioners (hunter, deziel-evans, and marsh, 2003) . (mcgill, tukey, and larsen 1978) comparing marks of 4 randomly chosen courses, with different pharmaceutical fields, over a period of three years (i.e. 3 cohorts of students). awarded marks in 2020 represents assessment during the covid-19 lockdown. the horizontal borders of each box represent the 1st and 3rd quartiles of student scores (wickham and stryjewski, 2011; mcgill, tukey, and larsen, 1978) . upper and lower whiskers represent the highest and lowest awarded marks for each cohort respectively (wickham and stryjewski, 2011; mcgill, tukey, and larsen, 1978) . the mean ± std (n > 10) values are pinned to each group. outliers were identified using the interquartile range (iqr) rule (schwertman, owens, and adnan 2004) . mean and median values are relatively close, thus indicating normal distribution (krzywinski and altman, 2014) . using an unpaired two-tailed unequal variance t-test (kim, 2015) , the 2020 student marks, for all courses, were found to be significantly different from the 2019 and 2018 cohorts (p < 0.05). a comparison between the 2018 and 2019 cohorts shows no significant difference in courses a and c (p < 0.05). the most challenging part of conducting distance education during the lockdown has been the remote assessment. although many distance learning systems provide platforms for online exams, teachers have a negative perception as students are not under monitoring and thus may be able to cheat (kaczmarczyk, 2011) . furthermore, the use of distance learning tools for online exams required learning more about the formatting needed, which can be complex. many teachers have resorted to alternative assessment methods, this included written assignments, where students were asked to write a report or an essay through which a demonstration of attaining a learning outcome is measured. results from this study, however, shows that many students had difficulties understanding the required action. while in many cases, written instructions where provided to students, the lack of experience in dealing with such assignments has left students frustrated. to add to this, the concept of plagiarism is new to many, as a result, students have struggled to write in their own words. results from this study also show that perceived ambiguity in assignment instructions has been one of the significant factors in influencing student satisfaction (p < 0.01). interviews with students and teachers have revealed the feeling of unfairness many students share. despite this, analysis of student grades, during the lockdown, has revealed a significant increase when compared to the past two years. although only four courses were investigated, this was further confirmed by all interviewed academics. regardless of struggles, the majority of teachers believe that alternative assessment methods will have a positive impact on students' overall skills. thus, many support changing plans of assessment methods, stated in course specifications, for post-lockdown teaching. while exams are an excellent tool of a summative assessment, these alternates may be more suited for ongoing formative assessments. traditionally, a greater percentage of the total course mark is dedicated to exams and only a small fraction (% 10%) is reserved to alternative assessments. while this may reduce the workload on academics, and especially if exams are of mcq nature, student attainment of various clos based skills are not accurately measured. this includes cognitive, interpersonal, timemanagement, communication, and information technology skills. in many cases, such skills contribute to half the clos while only 10% of the course marks were conventionally used to assess them under the ambiguous category of ''activity". this has indirectly guided students to focus on memorizing lecture materials and ignore other sources of information. fig. 4. box plot (mcgill, tukey, and larsen, 1978) to demonstrate and compare students satisfaction, on a 5 point scoring system, with the performance of their colleges in managing educational processes, during the covid-19 lockdown period, grouped by institution. names of institutions were anonymously substituted for a letter to protect their privacy. the horizontal borders of each box represent the 1st and 3rd quartiles of student scores (wickham and stryjewski, 2011; mcgill, tukey, and larsen, 1978) . upper and lower whiskers represent the highest and lowest values scored by each group of students, respectively (wickham and stryjewski, 2011; mcgill, tukey, and larsen, 1978) . the mean ± std (n ! 6) values are pinned to each group. outliers were identified using the interquartile range (iqr) rule (schwertman, owens, and adnan, 2004) . for both groups, b and r, student satisfaction values were densely populated on scores 3 and 5, respectively, causing the highest and lowest scores along with the 1st and 3rd quartiles to be of the same value, therefore reducing the box and whiskers to a single line. student groups with<6 respondents were excluded (n = 702). results of a correlation analysis, using non-parametric spearman's rank coefficient, between student satisfaction with the college performance in managing educational processes during the covid-19 lockdown period, and responses to questions from fig. 1 . spearman's rank correlation coefficient was used in this instance as the output of likert-scale questionnaire is of ordinal data. (n = 363). correlation with student satisfaction while exams may be used to assess knowledge and cognitive skills, the process of building ones knowledge through research, interpretation, analysis and evaluation followed by sharing his or her findings through a concise well-written document or an engaging presentation, can stimulate critical thinking and collaborative learning. where teachers can not only act as information providers but also help as modulators and facilitators. integrating alternative methods of assessment and collaborative student-centred teaching strategies may be the responsibility of teachers and program management teams. pharmacy students must also be encouraged to adopt a proactive learning strategy, where complete dependence on teachers for information is discouraged. although some students may have unrealistic expectations of the need to be ''spoon-fed" information and be awarded full marks simply by regurgitating the information back in an exam, it is the teacher's responsibility to push students towards taking responsibility for their own learning. an emphasis should be placed on testing the depth of student comprehension of critical clos. teaching remotely during the lockdown period was an excellent opportunity to revise teaching strategies and assessment methods used in normal circumstances. this study does not only encourage a contingency plan for emergency remote teaching, but also for a change in the everyday educational process that can positively impact teacher and student performance in all learning settings. in addition to taught students, this study investigated the impact covid-19 lockdown had on pharmacy trainees. most colleges have pulled back their students, in fear for their safety, for approximately half a term. for this period, trainees were offered alternative online activities. while this period may represent a small portion of the pharm d training duration, it represents approximately a quarter of the b pharm training duration. majority of graduates believe this will not have a negative impact on their practice as pharmacists. however, a small proportion (% 26%) of graduates are concerned by the limited interactions they have had with employers in the pharmaceutical industry, and thus fear a lower chance of employment in the foreseeable future. furthermore, the impact of lockdown on employment rates in the pharmaceutical industry is still unknown. this study shows that during the lockdown period, several clos have not been completely covered, especially psychomotor based ones. in addition, many teachers believe that students have not been able to attain all essential knowledge and skills to cover all basis. both teachers and students have expressed interest in holding revision sessions in the upcoming year, program management teams are encouraged to design and schedule a number of temporary short courses to mend this issue. employment rates of graduates is a program key performance indicator and must be monitored for ncaaa program accreditation. nevertheless, alumni committees must increase their efforts in guiding fresh graduates seeking employment. 3. recommendation for developing a contingency strategy for emergency remote teaching: although pandemics are thankfully rare, other situations may also call for emergency remote teaching, another recent example for saudi arabia is the impact of the gulf war (1990-1991) on educational institutions. while recorded news of school and university closures are available, limited research was performed on the impact of the war on education as well as future plans to adapt with similar circumstances. to avoid missing such opportunity, and in preparation for a possible lockdown extension, the author encourages educational program management teams across the country to explore the possibility of putting in place a contingency strategy for emergency teaching. this may include: creating a contingency course specification (ccs), a short document that outlines appropriate student-oriented teaching strategies and assessment methods to be used for each clo in crises times. ccs may also be used to define clos that may be successfully completed under lockdown circumstances, while outlining a plan for the completion of other learning outcomes postlockdown. for example, such clos may be incorporated into related courses or made into a short compulsory course. creating an overall timeline map of all required assessments to be completed by students from each level or program year, thus, to help coordinate coursework between different courses and to ensure sufficient time is given for completion without overloading students. to further investigate attainment of program learning outcomes, an additional post-lockdown traditional assessment may be utilized. analysis from such assessment may be used to design short comprehensive summer courses to further correct student deviation from the program plan. long term recommendations to enhance the efficiency of contingency strategy may include: further improving the infrastructure for distance learning, based on stakeholders' experiences during the lockdown period. providing training and technical support to all faculty members in effective use of distance educational tools for studentoriented teaching and assessment strategies. this may come in the form of a compulsory online course in alignment with preparing new phd graduates to teach at university level. integrating available learning management systems (lms) into the normal routine of daily teaching and assessment process. this will facilitate a better digital environment for student-teacher and student-student interactivity, as opposed to using emails and phone-based communications. while this will help monitor in-process quality of education, it will also allow for data collection and thus research and development of education. to better equip final year students for pre-registrational training under special circumstances, program managements are encouraged to introduce compulsory requirements of volunteer training hours for students of senior levels. 4.1.2. board of deans for the colleges of pharmacy in ksa 4.1.2.1. recommendation for collaborative initiatives:. education based collaborations between colleges of pharmacy in saudi arabia are still developing. a great initiative of collaboration has started after the formation of a board of deans for pharmacy colleges in saudi arabia. this had a positive impact on the standardisation of goals for pharmacy programs. nevertheless, a greater interaction is needed between pharmacy colleges to further push pharmacy education forward to maintain a high calibre of pharmacy gradu-ates. therefore, the author recommends the formation of a broader organisation such as ''saudi association of colleges of pharmacy" (sacp). this may also further quality of teaching and research efforts as well help to absorb the impact of crises. the covid-19 pandemic had a significant impact on pharmaceutical education in saudi arabia. while all teaching processes were forced to be distant, the ministry of education has led educational institutions across the country to a successful completion of the semester. this extraordinary situation called for a study to investigate student satisfaction with the emergency education processes conducted during this time. the use of online virtual classrooms, as a substitute for face-to-face traditional teaching, came with its challenges. the most significant was the limitations of student-student and student-teacher interactions. to overcome this challenge, the author recommends the implementation of various student-centred teaching strategies where proactive learning is encouraged. assessment via alternative methods during the lockdown has alerted teachers to the importance of their use in normal circumstances, as they can have a positive effect on students clo based skills. nevertheless, as students have less experience in working with alternative assessment methods, it is important they are accompanied by a student friendly guide to help them better understand the required actions and the marking scheme. the author of this study declares no competing financial interests or personal relationships that may have influenced the study performed in this paper. accreditation council for pharmacy education', accreditation council for pharmacy education pharmacy education in saudi arabia: the past, the present, and the future implementation of an acpe-accredited pharmd curriculum at a saudi college of pharmacy emerging frontiers of pharmacy education in saudi arabia: the metamorphosis in the last fifty years transactional distance and interactive television in the distance education of health professionals problem-based learning: a paradigm shift or a passing fad? overview of theories and principles relating to characteristics of adult learners assuring excellence in distance pharmaceutical education likert scale: explored and explained accreditation and student assessment in distance education: why we all need to pay attention t test as a parametric statistic points of significance: visualizing samples with box plots variations of box plots pharmacy student engagement, performance, and perception in a flipped satellite classroom performance indicators of higher education in saudi arabia saudi health work force over the next 10 years a simple more general boxplot method for identifying outliers 40 years of boxplots the author would like to thank the college of pharmacy at taibah university (al madinah al munawarah, ksa) represented by prof. saleh bahashwan the college dean, and the vice deanery for educational affairs represented by dr. sultan s. althagfan the vice dean for academic affairs.the author is very grateful to the recently graduated pharmacist azzam alghamdi and the undergraduate pharmacy students mr. muhanad algamdi and mr. ahmad haidar al-hrazi for participating in the student interviews. the author is also thankful to dr. sultan althagfan prof. khairy gabr, dr. ahmed effat and dr. rawan saeed bafail for taking part in the teachers' interviews.the author is also very thankful to dr. fatma al qudsi, mrs. somaiah alqurashi, mr. ibrahim alqurashi, mis. abrar alqurashi and mr. yahya alqurashi for their early contributions in developing the questionnaire of this study. key: cord-171792-fyly84zu authors: sockalingam, nachamma; liu, junhua title: designing learning experiences for online teaching and learning date: 2020-10-26 journal: nan doi: nan sha: doc_id: 171792 cord_uid: fyly84zu teaching is about constantly innovating strategies, ways and means to engage diverse students in active and meaningful learning. in line with this, sutd adopts various student-centric teaching and learning teaching methods and approaches. this means that our graduate/undergraduate instructors have to be ready to teach using these student student-centric teaching and learning pedagogies. in this article, i share my experiences of redesigning this teaching course that is typically conducted face-to-face to a synchronous online course and also invite one of the participant in this course to reflect on his experience as a student. teaching is about constantly innovating strategies, ways and means to engage diverse students in active and meaningful learning. each and every lesson is dynamic. even without the unprecedented situation of covid-19 and lockdown of educational institutions, we recognize that teaching cannot be just didactic and passive lecturing. while knowledge transfer/ acquisition is necessary, we now understand that our learners need to go beyond and be future-ready so that they are able to solve problems, think critically, work in diverse teams collaboratively, be techno-savvy and be flexible to meet the unforeseen demands of the volatile, uncertain, complex and ambiguous (vuca) world. in line with this, sutd adopts various student-centric teaching and learning teaching methods and approaches. for instance, sutd uses team teaching; that is a team of faculty instructors, graduate/undergraduate teaching assistants come together to teach a cohort of 50 students. this ensures that the faculty to student ratio is kept low (from 1:11 to 1: 16 ratio) and makes it possible for students to get individualized attention. this means that our graduate/undergraduate instructors have to be ready to teach using these student student-centric teaching and learning pedagogies. even though there is a common belief that only the gifted can teach, we take the perspective that teaching can be learnt by anyone if we try to gain a deeper understanding of what teaching entails. we recognize the need to support our graduate teaching assistants in preparing to teach and run a strategic 18 hour, 6-week course to prepare them to be graduate teaching assistants so that it benefits them as instructors, the faculty instructors they work with and also their students. in this article, i share my experiences of redesigning this teaching course that is typically conducted face-to-face to a synchronous online course and also invite one of the participant in this course to reflect on his experience as a student. 2 pedagogical frameworks / models underpinning the redesign of online course the "teaching@sutd" course is typically conducted face-to-face and embraces various active learning teaching methods [1] . however, we had to conduct this course completely online due to the lockdown situation. this course was run from 18 may 2020 to 22 june 2020 for a batch of 25 students. arxiv:2010.15602v1 [cs.cy] 26 oct 2020 given the limited time and resources and quick turnaround time, i decided to "augment and modify" the course instead of completely "redefining" the course as per ruben putendra's samr model which stands for substitution, augmentation, modification and redefinition ( figure 1 ). the samr model provides a classification of the various types of transformation of face-to-face to blended/online lessons [2] . typical online teaching and learning lessons that we experience such as that in corporate trainings, or even massive open online courses (mooc) courses refer to provision of structured reading materials, audio and video resources followed by assessments such as quizzes or assignments for independent and self-directed learning. the advantages of such a mode of online teaching and learning are the conveniences of learning anytime, anyplace, anywhere, being able to revisit the learning materials repeatedly, cheaper cost to students and the wider reach to audience. however, this mode of online teaching and learning is meant for self-directed learning and mainly knowledge acquisition, and may not cover humanistic aspects such as collaboration and communication skills when used solely in an asynchronous/self-directed learning mode. hands-on sessions such as lab work or practical work will also be limited. so it important that when we redesign our teaching practices for online learning, we do not simply substitute our mode of delivery to be online, but consider the purposes of our activities and the learning outcomes from each of the activities to "modify" or ideally "redefine" the lessons. in my case, i needed my students to be able to deliver an online/blended learning that incorporated sutd's active and interactive learning in groups. so, i decided on " augmentation and modification" depending on the activities. while the samr model is useful for classifying the transformation type, it does not guide us in what factors we need to consider in the redesign process. the "fit for purpose" teaching and learning design framework for blended/online teaching and learning [3] i had developed addresses this and helps to plan the technology tools for online teaching. the "fit for purpose" redesigning framework basically breaks the teaching and learning activities into purposeful chunks and proposes that we select technology tools according to the purposes of the teaching and learning activity chunks ( figure 2 ). the framework considers various factors like (i) learning outcomes, (ii) teaching and learning activities, (iii) assessment, and (iv) appropriate technology tools in redesigning our lessons, and it can even help us in considering the sequence of our lessons. it combines various concepts such as bloom's taxonomy [4] , constructive alignment [5] , backward design [6] , tpack model [7] in one framework. the recommendation is that we use the framework for individual lesson plan and build up to the module plan. this framework is also shared with our teaching faculty in our eptl website and you will see the examples of implementation in various courses. let us now see how the framework was used. figure 3 gives a quick overview of the teaching at sutd course and maps it to the "fit for purpose" framework. sutd's learning management system, edimension, and emails were the primary mode for classroom management and communication. content delivery was conducted either synchronously using video conferencing tool zoom or asynchronously using youtube videos/reading materials in edimension. collaboration and group work amongst students were done using video conferencing tool zoom chat, online word processing tool google doc as well online interactive presentation tool mentimeter in zoom sessions. google doc was useful as it allowed for collaborative and simultaneous editing and writing. mentimeter allowed for interactive question and answers for quick polls, quizzes and icebreaking activities. both formative and summative assessments were conducted using tools such as online synchronous quizzes in edimension, open-ended reflections in edimension and student presentations in zoom with peer and instructor feedback. while we had used tools such as google doc and mentimeter in previous runs of the face-to-face classes as well, we used these tools more extensively in the online course this time, especially in classroom discussion ( figure 4 ). typically, this collaborative activity would have been a role play activity of 3 students where one student plays the role of a teacher, another of a student and the third one serves as the observer. the teacher is to teach a certain concept using socrative questioning method, and after the 15 minute activity of planning and executing, students will reflect on their experiences to discuss how the teaching activity could have been improved. they typically use a vanguard sheet for their script. but this time, the team of students wrote out the their socrative questioning in googledoc. i found this useful as i had a concrete script to provide feedback on even after class. it also meant that other student groups could read each other's scripts and feedback to learn from. the main difference in terms of the technology use this time was the use of zoom platform to host the synchronous lessons. we used the breakout rooms in zoom for small group discussion. however, due to the technical limitation of the breakout room, which only allows the instructors/hosts to move from room to room, we could not simulate one of the activities in lesson 2 effectively. this was the jigsaw activity, where group members rotate and go through 6 stations to peer learn 6 teaching methods to construct and formulate their understanding. to adapt this to online format, and since it was not possible for students to figure 3 : an overview of the online "teaching at sutd" course using the fit for purpose" framework move from one station to another, we sorted student names into different breakout rooms and asked students to peer teach. in this case, students did not go from station to station; but students from different stations were sorted into one group. while this is technically the same as moving from station to station, we found that this activity was not as effective as in classroom setting due to various reasons. for example, not all students had read up, somewhere not speaking up and some others were a little confused with the activity since they were unfamiliar. generally, the energy and buzz was also not there. in my views, this activity had worked better in classroom. this is possibly because there would be an expert stationed in one place, and after repeating the explanation 6 times, they would have identified their gaps and would have been able to fill their gaps upon returning to their original station and group. in this way, each group would at least knew one teaching method very well and since the others students were learning from experts, they would have picked up the essentials. the advantage was learning in and from groups. but now, everyone had to be an expert and needed everyone's cooperation. as an instructor, i was also not able to monitor the energy level in the various groups simultaneously and move from group to group to energize them by moving around the classroom space as i would have in a physical classroom. interestingly, the other online activities that were mostly non-physical and those that focused on cognitive engagement ( e.g., designing an assessment) went well. the assessments were kept as the same format as in face-to-face classes although i modified the questions to suit the context this time. for example, the first reflection questioned on what it means to be an online teacher. the synchronicity of lessons, open-endedness in assignment questions, and focus on the thinking process helped to make the assignments suitable for online assessments so that there is originality and copy-pasting is minimized. i did not see any of these. the final assessment of microteaching was completely online and this was probably the most challenging for the students in my opinion compared to the previous terms. senior lecturer oka kurniawan from istd also facilitated the microteaching sessions as an assessor. students had to be creative in designing their online teaching activities and teach within 5 minutes. however we did give additional 5 minutes of buffer time since this was online. the students had to focus more on cognitive activities and minimize on physical activities for their microteaching. in the previous years, many students would include physical activities and demos in their microteaching. i recollect a student teacher asking the class members to imagine themselves to be odd/even numbers and group us according to instructions to illustrate overall, while we have the various frameworks and models to redesign the face-to-face to online lessons, this is not a simple process. we have to juggle with the technical and technology limitations and also set new social norms in online teaching and learning. for instance, we had to resort to switching off the online videos to avoid streaming lags but that also meant that we could not keep track of student engagement continuously, and so we substituted with zoom chats and emoticons. also, it took a slightly longer time to build the social connection with the students to gain their trust and understanding, and seek the cooperation. building the social connection is particularly important in group projects. to get a better perspective, i sought weekly feedback from students. the learners' sentiments of the online class varied. while some liked the online activities, some others found the activities a little monotonous and limited. the discussions and collaborative activities were found to be challenging especially in the early weeks when participants were still warming up but became better over the weeks. some noted in their reflections that they still prefer face-to-face learning over completely online learning. even though blended online learning, that is a blend of synchronous and asynchronous online learning may offer some relief over completely self-directed and independent online learning, students seemed to actually prefer a blend of face-to-face and online learning. there were some rare extremes of students actually preferring completely online (but this misses the importance of learning from peers in collaborative groups). i too agree that the value of personal and social connection in face-to-face meetings is important. in my personal views, teaching is not all just about just the learning outcomes, skills and values; many tacit values, habits (e.g., being organized, being positive), mentorship, professional relationships are formed through teacher-student and student-student connections and these are difficult to foster and maintain online. overall, the course participants indicated that they felt positive and confident at the end of the course about active teaching and learning in an online environment. all of the graduate students demonstrated active, student centric teaching methods in their online teaching (even though many were not familiar with such methods at the onset) and some were given additional opportunities to refine their work when needed. we continue to improve the course based on student feedback. here are some tips for designing online learning experiences based on my experience. 1. establish communication channels and connect with students for better understanding and relationships. 4. redesign using "fit for purpose" activities and tools. 5. minimize physical activities if technology is limiting and redesign activities. for instance, consider virtual tools for labs and reflections to focus on process skills. 6. blend synchronous and asynchronous activities, and connect the synchronous and asynchronous activities. 7 . make assessments open-ended and focus on process skills. 8. use a good mix of formative and summative assessments, and provide prompt feedback, involving the peers and even industrial experts. 9. give opportunities for students to resubmit work where possible. 10. get informal feedback from students and continually modify where possible. this experience of going completely online in such a short time frame was indeed positive and fruitful. i really appreciate and value my students' engagement and learnt from them as well. their constructive comments on improving the course gave me refreshing ideas. reflecting through this article also helped me to decide on which activities to keep and modify for the next run. i end the article with one of the reflections by our graduate student liu jun hua, who is an ai entrepreneur with deep interest in teaching and learning. we both concur that focusing on the learner and connecting with the learner is the most element for successful teaching -especially in an online context. the 6-week gta course was conducted during the covid-19 outbreak, where physical classes were not possible. it was the first time that i participated in a class completely online. fortunately, the online classes went well. i had great joy going through the materials and interacting with the instructor and teachers on zoom. most of the students participated in the discussions and class activities actively. through the microteaching experience, i realised that it is certainly a challenging task to prepare class materials for online classes that help students achieve learning goals while encouraging active participation. from the entire course and the final microteaching sessions, i observed some patterns or methods that the various presenters had used in engaging the audience and encouraging participation. these are summarized as follows. focus on the audience, not the content. pushing all the prepared content to the audience is perhaps the easiest way to conduct a class, but not necessarily the most effective one. focusing on the audience is really important, especially for live classes. be aware of whether majority of the audience are following and being engaged. for instance, are they paying attention? are they responding to my probing? are they thinking and taking notes? learning online can be distracting. therefore, it is crucial that the instructor keeps in mind students' attention during the class. orchestrate the flow of content. designing a class is like planning for a music performance -we need to anticipate the emotion of the audience though the whole play to create an impact [8] . we can variate the rhythm and intensity of the content to orchestrate the audience's emotion to keep them engaged. use of real-world examples. while knowledge is developed constructively, connecting abstract and non-trivial content to real-world examples will tremendously help the audience create connection and see the purpose of learning [9] . furthermore, making the content light-weighted and fun will certainly help with the engagement -who doesn't like humour? learning to teach as graduate teaching assistants (gtas) at sutd online teaching for academic continuity during emergency situations the samr model: six exemplars. retrieved bloom's taxonomy enhancing teaching through constructive alignment. higher education the understanding by design guide to creating high-quality units what is technological pedagogical content knowledge (tpack)? self-evolving adaptive learning for personalized education innovations in software engineering education: an experimental study of integrating active learning and design-based learning key: cord-104142-0nfprn2a authors: azmi, maryam a.; palmisano, nicholas j.; medwig-kinney, taylor n.; moore, frances e.; rahman, rumana; zhang, wan; adikes, rebecca c.; matus, david q. title: a laboratory module that explores rna interference and codon optimization through fluorescence microscopy using caenorhabditis elegans date: 2020-10-19 journal: biorxiv doi: 10.1101/2020.10.17.344069 sha: doc_id: 104142 cord_uid: 0nfprn2a authentic research experiences are beneficial to students allowing them to gain laboratory and problem-solving skills as well as foundational research skills in a team-based setting. we designed a laboratory module to provide an authentic research experience to stimulate curiosity, introduce students to experimental techniques, and promote higher-order thinking. in this laboratory module, students learn about rna interference (rnai) and codon optimization using the research organism caenorhabditis elegans (c. elegans). students are given the opportunity to perform a commonly used method of gene downregulation in c. elegans where they visualize gene depletion using fluorescence microscopy and quantify the efficacy of depletion using quantitative image analysis. the module presented here educates students on how to report their results and findings by generating publication quality figures and figure legends. the activities outlined exemplify ways by which students can acquire the critical thinking, data interpretation, and technical skills, which are beneficial for future laboratory classes, independent inquiry-based research projects and careers in the life sciences and beyond. scientific teaching content learning goals gain experience working with c. elegans understand the process of rna interference and importance of codon optimization learn basic microscopy techniques and image analysis learn how to properly use the scientific method enhance critical thinking skills learning objectives students will be able to: lab 1 and 2: identify specific larval stages of c. elegans synchronize c. elegans larvae using alkaline hypochlorite treatment understand codon usage formulate hypotheses and design a controlled experiment lab 3 and 4: acquire images using an epifluorescence microscope effectively communicate results and formulate conclusions from data describe what rnai is and how it affects gene expression/activity calculate mean fluorescent intensity from acquired fluorescence micrographs perform statistical tests to determine the significance of results generate publication quality figures and figure legends inquiry-based learning is a form of active learning where students obtain the skills they need 97 to problem solve and make unique discoveries about the natural world(1-3). in contrast to teacher-98 centered instruction, where facts are disseminated to students, inquiry-based learning encourages 99 students to foster their own independent learning with the assistance of the instructor(1-3). in 100 addition, inquiry-based learning puts emphasis on students developing science process skills, such 101 as making observations, developing hypotheses, and formulating conclusions(1-3). 102 course-based undergraduate research experiences (cures) are a form of inquiry-based 103 learning that provide students with a genuine research experience. students enrolled in cures 104 develop or are given a research question with an unknown outcome, use the scientific method to 105 address the question, collect and analyze data, and communicate their results (3) (4) (5) . students that 106 participate in a cure learn the necessary skills and techniques they need to carry out the tasks 107 required(6-8), and at the same time gain confidence in their ability to engage in the scientific 108 process(9-11). assessment of student learning gains reveal that cures improve student ability to 109 think critically, interpret data, communicate results, and collaborate as a team, compared to 110 traditional lab courses (12) (13) (14) (15) (16) (17) . a critical aspect of independent research is obtaining the foundational 111 skills and introductory training needed for understanding a specific system and/or research topic of 112 interest. 113 here we describe a simple laboratory module employed in the first half of our upper division 114 undergraduate cure on developmental genetics, which is used to prepare students for independent 115 inquiry-based group research projects that occur in the second half of the course. in this module, 116 students are introduced to the research organism, caenhorhabditis elegans (c. elegans), to explore 117 the concepts of rna interference (rnai) and codon optimization. c. elegans offers many advantages 118 that make it an ideal research organism, such as a fast life cycle, large brood sizes, and easy access to 119 genetic manipulation by forward and/or reverse genetic approaches (18) (19) (20) . additionally, they are 120 transparent, which allows for visualization of all tissue types, and the real-time visualization of 121 fluorescent-tagged reporter proteins expressed in various tissues of interest (21, 22) . using the 122 protocols outlined in this paper, students will cultivate and utilize c. elegans to conduct an rnai 123 experiment where they will visualize first-hand how rnai depletes a gene of interest and how codon 124 optimization significantly impacts gene expression. 125 the importance of this module is that it not only teaches concepts in molecular genetics and 126 introduces students to a model system, it also promotes higher-order thinking. with the instruction 127 provided here, students will gain experience and expertise in working with c. elegans, basic 128 microscopy, data analysis, and science communication. students will have an intuition for probing a 129 research question, developing hypotheses, planning experiments, thinking about possible results, 130 and formulating conclusions. therefore, modules like the one presented here have a positive impact 131 on student development and at the same time provide the prerequisites needed for success in stem-132 related fields and beyond. 133 overall, this module can be used at various educational levels to enhance students' interest 134 in science and provide the groundwork needed for independent scientific research. the module 135 incorporates active learning styles of instruction that significantly enhance student performance and 136 encourage engagement. finally, this module can also be used as a "stepping-stone" or "bootcamp" 137 exercise to provide students with a set of skills and tools for the inquiry-based module of a cure 138 using c. elegans as a model organism. intended audience this laboratory module was employed in the first half of upper-level undergraduate 142 developmental genetics at stony brook university. most students enrolled in the course were juniors 143 or seniors; however, the module can be implemented at various educational levels (i.e. freshman or 144 sophomores or first-year graduate students as a "bootcamp" exercise). it can also be adapted for use 145 in a high school setting where students can be introduced to a model system commonly studied in 146 the life sciences, learn concepts that pertain to gene expression and regulation, and reinforce their 147 prior knowledge of the scientific method. 148 149 the module requires four lab sessions of approximately 3 hours each. we found this was 151 ample time for students to become accustomed to working with c. elegans and proficient in the 152 necessary skills needed to complete the module. however, the module timing can be adjusted as 153 needed to any desired length of time. and animal development. it is highly encouraged that students have familiarity with basic laboratory 158 procedures, such as micropipetting and sterile techniques. prior to the module, all necessary 159 materials and information needed to complete the assignments will be provided, and students will 160 receive an introduction to c. elegans, rnai, and basic microscopy. activities, polling questions, and independent-learning activities. to complete the life history stages 175 assignment and gfp rnai experiment discussed below, students were arranged into groups, which 176 fostered peer-to-peer communication, teamwork, and promoted student engagement. moreover, 177 prior to the gfp rnai experiment, students were assigned a gfp rnai worksheet (see below) which 178 prompted independent thinking about the experiment. by completing the gfp rnai worksheet, 179 students had an opportunity to complete a task on their own without supervision and troubleshoot 180 through any obstacles they may have encountered. student assessments were conducted at multiple levels throughout the module. during the short 184 introductory lectures given, students were asked a series of polling questions incorporated into the 185 lecture (supplement 1) and were assessed based on their answers. students were evaluated on their 186 ability to perform lab tasks and follow directions given by teaching assistants and instructors. 187 additionally, students were graded based on the quality of their lab assignment, which included data 188 analysis, figure generation and figure legend writing (supplement 2). inclusive teaching 191 we have designed this module to be all-inclusive by differentiating content and lesson material to 192 reach various types of learners. the hands-on activities of this module capture the attention and 193 engagement of kinesthetic and tactile learners. our short lectures that contain images, provide 194 written instruction, and facilitate discussion amongst the class are accommodating to both visual and 195 auditory learners. in addition, we encourage splitting the class into groups to include both men and 196 women, as well as students of different ethnic backgrounds to foster an inclusive instructional 197 environment. thus, the module ensures equity by reaching all types of learners and encourages 198 diversity through the formation of disparate groups. overview of the module 204 in the module discussed in detail below, various concepts that are central to the 205 understanding of gene expression and gene regulation will be explored. using c. elegans as a model 206 organism, students will understand how codon optimization of a nucleotide sequence significantly 207 impacts the effectiveness of rnai-mediated gene depletion. 208 specifically, students will work with two gfp-expressing c. elegans strains, where one strain 209 expresses a non-codon optimized (nco) gfp tag (gfpnco), while the other strain expresses a codon 210 optimized (co) gfp tag (gfpco). the gfpnco and gfpco tags are each fused to the histone protein, his-211 58 (h2b), that is driven by a ubiquitous promoter, eft-3, which promotes expression in all cells. 212 students will treat each strain with an empty vector (control) rnai bacterial clone or an rnai 213 bacterial clone that produces dsrna specific to only the non-codon optimized gfp variant (gfpnco) 214 (review timmons and fire, 1998 for a detailed description on how rnai works in c. elegans). 215 through fluorescence microscopy, students will visually see differences in gfp expression in each 216 strain due to codon optimization, and they will observe that significant depletion occurs only in the 217 strain expressing eft-3>h2b::gfpnco. from their understanding of rnai and codon optimization, we 218 anticipate that students will be able to accurately predict these results and explain why these 219 outcomes occur. 220 prior to the module, we present students with a series of lectures that include an introduction 221 to c. elegans and discussion about gene regulation (supplement 1) (review corsi et al., 2015 for a 222 comprehensive overview of c. elegans). we discuss the topic of rnai, which is a biological process 223 that in the presence of exogenous double stranded rna (dsrna) results in post-transcriptional gene 224 silencing (19, (27) (28) (29) . one method used to administer c. elegans with dsrna is to feed them with e. 225 coli expressing a vector capable of producing dsrna, that is complementary to a target gene of 226 interest(30, 31). c. elegans are unique in that they have a systemic rnai response, meaning that 227 dsrna spreads throughout all tissues, with the exception of most neurons(32, 33). thus, loss-of-228 function phenotypes for genes of interest can be assessed in almost any tissue of interest using rnai. 229 we also engage our students by having a discussion on codon optimization, or the 230 modification of a sequence of dna such that the frequency of codons used by a particular organism 231 for a specific amino acid is taken into account(34-36). codon optimization significantly enhances the 232 expression level of a particular protein due to the correlation between codon usage and trna 233 abundance, and mrna stability(37-40). thus, the expression levels of codon optimized genes will be 234 more robust than those of non-codon optimized. in all, we anticipate this module will fulfill several goals, which include student proficiency in 236 using the scientific method and development of critical thinking skills. after completing this module, 237 students will be able to conduct controlled experiments using a model organism. in addition, they 238 will be able to explain what rnai is and how it can be used to assess loss-of-function phenotypes for 239 any gene of interest. lastly, students will be able to state the importance of codon optimization as it 240 pertains to gene expression. and 4). an additional lab session was devoted to an introduction to compound light microscopy and 262 a tutorial on microscopy. after the conclusion of these introductory sessions, students demonstrate 263 their ability to work with c. elegans and operate a compound light microscope available in class, by 264 imaging the life history stages (different larval stages) of the worms. additionally, students compile 265 a figure containing images of the life stages along with a descriptive figure legend (supplement 2 and 266 3). we find this exercise extremely valuable for students to master important worm husbandry 267 techniques (i.e. worm picking), identify larval stages, and become familiar with microscopy 268 techniques, all of which will be necessary to successfully perform the gfp rnai experiment. to prepare the students for the experiment, we presented a short lecture on rnai and codon 273 bias (supplement 1) and devised a "gfp rnai worksheet" (supplement 5). the goal of this worksheet 274 is to drive students to formulate hypotheses as to whether the gfpnco rnai clone will efficiently 275 knock down gfp intensity levels in the strain expressing h2b::gfpco or h2b::gfpnco. in this 276 worksheet, the students are provided with the nucleotide and amino acid sequences for the codon 277 and non-codon optimized h2b::gfp tags, as well as the short-interfering nucleotide sequence from 278 the gfpnco rnai clone (supplement 5). using the sequences provided, students will make a pairwise 279 sequence alignment, using emboss needle (https://www.ebi.ac.uk/tools/psa/emboss_needle/). 280 they will then compare the percent similarities between the different sequences and determine 281 whether the short-interfering nucleotide sequence for gfpnco rnai is most similar to h2b::gfpco or 282 h2b::gfpnco. through this process, students will see that the sirna sequence (in dna form) encoded 283 by the gfpnco rnai clone is 100% identical to the gfpnco sequence and not the gfpco sequence. 284 students will also appreciate that the control rnai clone is called "empty vector" because it does not 285 produce a dsrna product. 286 to conduct the rnai experiment, the students should grow up both the eft-3>h2b:: gfpco when ngm plates are full of gravid adults (>100 adults on each plate or food source is near 296 depletion), students should treat each strain with alkaline hypochlorite solution (figure 1 step 297 1)(supplement 3 and supplement 4 section v) to create synchronized l1s. approximately 50-100 l1 298 animals should be pipetted onto control and gfpnco-specific rnai plates (figure 1 step 2). 299 individual rnai plates should have no more than ~50-100 worms to prevent overcrowding and 300 depletion of the e. coli food source (figure 1 step 2). the l1's are then cultured on the rnai plates 301 at the desired temperature until the l3 or l4 stage is reached (figure 1 step 3). once the desired 302 stage is reached, students mount the animals on 5% agarose slides containing a droplet of m9 buffer 303 and sodium azide to anesthetize the animals (figure 1 step 3). we recommend that students pick 304 ~10 animals for imaging at a time. we also encourage instructors to ensure students handle sodium 305 azide with care as it is toxic(44)(supplement 3 and supplement 4 section vi). 306 307 students quantified h2b::gfp fluorescence depletion using two wide-field epifluorescence 309 microscopes, the accu-scope or leica dmlb fluorescence microscopes (figure 1 step 4 imaged for each rnai treatment (control and gfpnco). from the data acquired by the students, 314 several qualitative observations were made (figure 2 a and b) . first, the overall fluorescence 315 intensity of the gfpco strain was visually much brighter than the gfpnco strain. second, treating the 316 gfpnco strain with gfpnco rnai strongly reduced the fluorescence intensity of gfp, whereas treating 317 the gfpco strain with gfpnco rnai did not (figure 2 a and b, eft-3>h2b::gfp column) . third, in the 318 gfpnco strain treated with gfpnco rnai, although the fluorescence intensity of gfp was strongly 319 reduced, some nuclei still showed high levels of gfp, these correspond to the cells that are insensitive 320 to rnai, most notably neurons (figure 2a , eft-3>h2b::gfpnco; gfpnco rnai) 321 to analyze the data quantitatively, we instructed students to quantify whole-body gfp 322 fluorescence intensity for 10 animals from each strain grown on control and gfpnco rnai, using 323 fiji/imagej2(26). briefly, the entire body of each worm was outlined and the mean fluorescence 324 intensity (mfi) was then measured for both gfp and an area of background. the background mfi 325 measurement was then subtracted from the gfp mfi measurement to reduce background noise and 326 obtain a mean gray value (mgv). mean gray values were normalized by dividing the mfi in rnai-327 treated animals by the average mfi in control-treated animals (supplement 6, supplement 4 section 328 vii, tutorial videos 1-4, supplement 7). the mean gray values obtained from each imaging system 329 (microscope) are plotted next to their respective micrographs (figure 2 a' and b') . by plotting the normalized mgv, students were able to clearly see that treating the gfpnco 331 strain with gfpnco rnai significantly reduced the expression of gfp compared to control-treated 332 animals (figure 2a, 2a' , and 2b, 2b', nco strain; control rnai vs. gfpnco rnai). moreover, the 333 students noted that treatment with gfpnco rnai had no effect on gfp expression levels in the codon-334 optimized strain (figure 2a, 2a', creating a publication quality figure similar to that described for the life history stages, which also 339 included a table of their raw data and description of their results (supplement 2 and 6). from these 340 results, and the results obtained from the gfp rnai worksheet, it becomes evident that rnai 341 specificity is largely dependent on the sequence homology/similarity between the target gene 342 sequence and the sequence of the sirna produced by the rnai clone itself. extended results (optional) compared to wide-field epifluorescence microscopy, high resolution fluorescence 346 microscopy greatly improves the detail and resolving power of fluorescence micrographs, such that 347 unwanted out-of-focus light is significantly reduced, and the detail of cellular objects is greatly 348 enhanced(45). thus, to show students high quality images of nuclear dna labeled with h2b::gfp, we 349 acquired spinning-disk confocal images for both the eft-3>h2b::gfpco and eft-3>h2b::gfpnco strains 350 ( figure 2c, 2c', 3, and 4) . importantly, these spinning-disk confocal images served to better 351 illustrate some of the key concepts discussed in the lab module, such as codon optimization and 352 lineage specific differences in rnai susceptibility. from the confocal fluorescence micrographs, it becomes more apparent that treatment with 354 gfpnco rnai significantly reduces gfp fluorescence intensity in the gfpnco strain, but not in the gfpco 355 strain ( figure 2c , 2c'; co strain vs. nco strain; gfpnco rnai vs. control rnai). to highlight the 356 differences in expression levels between codon optimized and non-codon optimized h2b::gfp fusion 357 proteins, we took spinning disk confocal images of the c. elegans germline. in general, codon 358 optimized transgenes are more robustly expressed in the germline than non-codon optimized 359 transgenes (46, 47) . in line with this, h2b::gfp fluorescence expression was more robust in germ cells 360 when gfp is codon-optimized as opposed to when it is non-codon optimized (figure 3 ; co strain 361 vs. nco strain). in c. elegans, certain cell lineages show different sensitivities to exogenous dsrna. for 363 example, neurons are less sensitive to rnai compared to other somatic tissues (32, 48) . this is 364 because neurons lack the dsrna-gated channel, sid-1, which promotes the uptake of dsrna into cells 365 that express it(49). to emphasize this concept to students, we acquired spinning-disk confocal 366 images of nuclei from various cell lineages commonly studied in c. elegans (figure 4a) with gfpnco rnai significantly reduced gfp fluorescence intensity levels in the gfpnco strain, but not 370 in the gfpco strain ( figure 4b-e) . however, with respect to the gfpnco strain treated with gfpnco 371 rnai, the percent decrease in gfp intensity levels in the pharyngeal neurons was much less than the 372 decrease found in the other cell types examined ( figure 4b compared to figures 4c-e) . thus, these 373 observations can be used in the classroom to clearly illustrate to students that certain cell types are 374 more or less sensitive to exogenous dsrna, and this sensitivity is in part dictated by the surface 375 proteins these cells express (49). 376 377 the laboratory module presented here teaches a variety of common techniques employed by 379 c. elegans researchers and exposes students to various concepts in molecular genetics and 380 microscopy. during this module, students will become proficient at working with a widely used 381 research organism, be able to conduct controlled experiments, analyze data, produce publication 382 quality images, and have a basic understanding of microscopy. in addition, students will have a solid 383 foundation as to how rnai works, how it can be used to study gene function, and the importance of 384 codon optimization on proper gene expression 385 this module clearly illustrates that certain cell types are less or more prone to the effects of 386 dsrna treatment, and that codon optimization results in improved gene expression in tissues (i.e. 387 the germline). the advantage of using a strain that drives ubiquitous expression of h2b::gfp is that 388 it is extremely bright and nuclear localized, and therefore easily visible on widefield epifluorescence 389 microscopes, which are commonly available in most laboratory classrooms. for classrooms that have 390 access to a high resolution microscope, such as a spinning-disk confocal, laser-scanning confocal 391 microscope, or structured illumination microscopes, this module can be easily adapted for use on 392 those types of microscopes as shown in figures 2c, 3 , and 4. the additional benefit of the strains used 393 in this module is that students can immediately see differences in depletion between h2b::gfpco and 394 h2b::gfpnco upon gfpnco rnai treatment. such is a bonus when attempting to keep students 395 intrigued and engaged. upon completing this module, students will acquire the basic foundational skills needed for 397 independent inquiry-based research projects involving c. elegans. some examples of inquiry-based 398 research projects that can follow this module include a reverse genetics screen to identify genes 399 important for specific processes of interest, such as longevity. in this example, with the assistance of 400 their instructor, students can design a simple research question, such as "do fat metabolism genes 401 play a role in regulating lifespan?". students can search the literature for fat metabolism genes of 402 interest, use either the ahringer or vidal rnai libraries (source bioscience) to isolate clones specific 403 for those genes, and determine if their depletion reduces or enhances longevity. results can then be 404 documented, written up in research paper format, and presented to the class. the independent 405 inquiry-based research projects that follow this module are limitless and can focus on key areas of 406 research implicated in a wide range of cellular processes, such as cell cycle regulation, cellular 407 invasion, stress-resistance pathways, vesicle trafficking, and much more. 408 whereas most lecture and laboratory-based classrooms use expository styles of instruction, 409 classrooms that utilize active learning styles of instruction, such as inquiry-based learning strategies, 410 significantly enhance student performance and learning outcomes (15, 50, 51) . several active 411 learning strategies that can be implemented throughout this module include variations of the jigsaw 412 technique(52) and think-pair-share(53). with respect to the jigsaw technique, students can be 413 divided into several teams, with each team focusing on identifying a specific life history stage as 414 described above. when those teams complete the assignment, the class is divided once again into 415 new groups. each new group consists of one member from the original teams, with each member 416 being responsible for teaching the group how to identify their originally assigned life history stage. 417 we performed a variation of the jigsaw technique and found the students to be engaged and excited 418 with the task provided. to further promote student thinking and awareness, an instructor can also 419 take advantage of think-pair-share. here, an instructor can pose a question, allow the students to 420 think independently about the question, then form pairs or groups to discuss their ideas collectively, 421 and share with the class. this strategy can be easily administered for the gfp rnai worksheet. here 422 students can be given the worksheet as a homework or in-class assignment to think independently 423 about which strain the gfpnco rnai clone will be more efficient against. later in the lab session or 424 during the next lab session, the students can form pairs or groups, discuss their opinions with one 425 another, and then present them to the class. in all, there are various active learning strategies that 426 can be implemented in this module, which fosters peer-to-peer communication, promotes student 427 engagement, and stimulates higher-order thinking. this module can be further adapted for remote teaching and online learning. instructors can 429 teach image analysis alongside with their lectures online and provide students with previously 430 acquired raw data sets from epifluorescence and/or confocal microscopes. lessons can be held 431 synchronously by utilizing the share screen option in video conferencing apps, such as zoom or 432 google meet, or asynchronously by recording lectures and lessons ahead of time, and uploading them 433 onto blackboard, microsoft cloud, or google drive along with our image analysis video tutorials 434 (supplemental tutorial videos 1-5). based on the knowledge gained from the lectures, compiled raw 435 data, and the gfp rnai worksheet, students will be able to formulate their hypothesis and test it by 436 analyzing the supplied data. we adapted this distance learning technique for the second half of our 437 course during the sars-cov2 pandemic in the spring of 2020 and received positive feedback from 438 our students about the adaptability of the course. 439 an additional advantage of this module is that it not only applies to the undergraduate setting, 440 it can be adapted to a variety of educational levels, such as the high school or graduate level. at the 441 high school level, this module can enhance critical thinking, promote independence at an early stage 442 of a student's career, and instill awareness by introducing the field of genetics and organismal 443 biology. moreover, it inherently promotes student engagement, by allowing students to work with 444 an organism that most are unaware exists, work in groups to share ideas, and visualize cellular 445 processes live. additionally, we provide simplified protocols and instructions to make it easy for 446 instructors who have little experience with c. elegans to facilitate this module in their classroom. at 447 the graduate level, this module can be particularly useful for graduate student rotations and can serve 448 as an introductory "boot camp" or "stepping-stone" to introduce the c. elegans and experimental 449 techniques used in c. elegans research. here, entry-level graduate students who have not previously 450 worked with c. elegans will have the opportunity to do so, and can immediately start acquiring data 451 by conducting a reverse genetics screen devised by the principal investigator and/or themselves. 452 over time, these students can become confident enough to develop and plan their own projects. 453 in summary, this module is an excellent resource for instructors interested in conveying a 454 real-life science experience to their students and serves as an excellent opportunity for students to 455 gain the hands-on experience they need in order to pursue a career in biology. 456 457 rnai efficiency and to avoid overcrowding/starvation, ~50 worms per plate will suffice. (step 3) l1 494 larvae are grown until the l3/l4 larval stage and then mounted on 5% agarose pad slides (containing 495 sodium azide (anesthetic) and a drop of m9 buffer) for image acquisition. *growth times will vary 496 based on temperature (see text for more details). (step 4) images are acquired and then analyzed 497 using fiji/imagej to determine the mean fluorescence intensity. results are briefly explained in the 498 lab report and submitted along with a publication quality figure with figure legend. 499 practical advice for teaching inquiry-based science process skills 540 in the biological sciences the effects of discovery learning in a lower-division biology 542 course inquiry-based and research-based laboratory 544 pedagogies in undergraduate science assessment of course-based undergraduate research experiences: a meeting report modeling course-based undergraduate research 550 experiences: an agenda for future research and evaluation a scalable cure using a crispr/cas9 fluorescent protein knock-in strategy in caenorhabditis elegans based undergraduate research experience to introduce drug-receptor concepts curric dev 3 research and teaching: development of 557 course-based undergraduate research experiences using a design-based approach alumni perceptions used to assess undergraduate research 560 experience undergraduate research experiences support science career decisions and 562 active learning becoming a scientist: the role of undergraduate 564 research in students' cognitive, personal, and professional development a high-enrollment course-based undergraduate research 568 experience improves student conceptions of scientific thinking and ability to interpret data. 569 cbe adding authenticity to inquiry in a 571 first-year, research-based implementing a course-based undergraduate research experience 573 to grow the quantity and quality of undergraduate research in an animal science curriculum1 active learning increases student performance in science, engineering, and mathematics mini-course-based undergraduate research experience: impact on student understanding 580 of stem research and interest in stem programs undergraduate biology lab courses: 582 comparing the impact of traditionally based "cookbook" and authentic research-based 583 courses on student lab experiences potent and specific genetic 586 interference by double-stranded rna in caenorhabditis elegans the art and design of genetic screens green fluorescent protein as a 590 marker for gene expression a transparent window into biology: a primer on 592 caenorhabditis elegans revealing the world of rna interference codon optimality, bias and usage in translation and mrna decay codon bias as a means to fine-tune gene 597 expression imagej2: imagej for the next generation of scientific image data inhibition of thymidine kinase gene expression by anti-sense 601 rna: a molecular approach to genetic analysis production of antisense rna leads to 603 effective and specific inhibition of gene expression in c. elegans muscle identification of plant genetic loci involved in a posttranscriptional 606 mechanism for meiotically reversible transgene silencing specific interference by ingested dsrna effectiveness of 610 specific rna-mediated interference through ingested double-stranded rna in 611 caenorhabditis elegans systemic rnai in c. elegans requires the 613 putative transmembrane protein sid-1 systemic rnai in caenorhabditis elegans synonymous but not the same: the causes and consequences of 618 codon bias codon usage in yeast: cluster analysis clearly 620 differentiates highly and lowly expressed genes codon usage and trna content in unicellular and multicellular organisms correlation between the abundance of escherichia coli transfer rnas and 624 the occurrence of the respective codons in its protein genes: a proposal for a synonymous 625 codon choice that is optimal for the e. coli translational system coevolution of codon usage and transfer rna abundance codon optimality is a major determinant of mrna stability an improved estimation of trna expression to better elucidate 632 the coevolution between trna abundance and codon usage in bacteria the life cycle of the nematode caenorhabditis elegans a simplified method for mutant characterization auxin-mediated protein degradation in caenorhabditis 636 elegans sodium azide-induced neurotoxicity mitochondrial inhibitors and neurodegenerative disorders super-resolution imaging in live cells the caenorhabditis elegans transgenic toolbox 645 optogenetic dissection of mitotic spindle positioning in vivo enhanced neuronal rnai in c. 647 elegans using sid-1 sid-1 domains important for dsrna 649 import in caenorhabditis elegans inquiry-based learning to improve 651 student engagement in a large first year topic. student success 6 prescribed active learning increases performance in introductory biology the jigsaw classroom the responsive classroom discussion key: cord-029229-59etmpzs authors: barber, va hayman title: teaching and engaging generation z during the coronavirus date: 2020-06-30 journal: nan doi: 10.1002/dch.30335 sha: doc_id: 29229 cord_uid: 59etmpzs nan committees and task forces for diverse, multigenerational participation. the faculty and administrators we interviewed cited offhand ageist remarks during search committee meetings, particularly in relation to tenure-track positions. an academic administrator described how he had to counsel his dean on the illegality of consideration of age. take, for example, the observations of katherine, a white female sociology professor in a private university. but you have the off-the-books conversations that take place among faculty, because obviously if it is a tenure-track job, you are investing a lot of resources and such into the person, and you kind of hedge your bets. are they going to stay with us? is there longevity with this person? are they going to keep up with the demands of this job in particular? we talk about how we kind of need go-getters in these positions. i have noticed occasionally some of my faculty colleagues making comments like "are they going to be able to keep up with the energy level required of this position?" i would characterize that as an ageist comment because fitness levels come at all times of life … there are some people who have low energy when they are young. i think that is a faulty assumption to make. katherine also notes that it appears that men are given more leeway than women in such ageist considerations. the participation of women, people of color, and lgbt individuals from different generational groups in committee assignments will help overcome stereotypes and enhance group processes and problemsolving. diana, a white female professor, describes how, as the only faculty member over forty years old on a committee, her views were met with condescension. a faculty member accused her of leaving out a major area in her approach, with the implication that her work was outdated. the mere presence of members from different generational groups on committees is not sufficient. their perspectives must be recognized. implement flexible workplace practices that reflect the needs of all members of the workforce. the american federation of teachers and the american association of university professors have issued a comprehensive list of principles for the covid-19 crisis for faculty and their protection against negative teaching evaluations as well as the provision of adequate resources for online teaching (american association of university professors 2020). the guidelines also refer to institutional policies for stopping the clock in tenure attainment and other holdharmless practices. consider alternative online professional development opportunities for faculty and staff that focus on inclusion, cultural change, and strategies for leading a multigenerational workforce. a wide variety of free webinars are now being offered by professional associations such as the american association of colleges and universities and by news sources like inside higher ed and diversity consultants. many of these programs highlight the need for diversity and inclusion in developing effective academic and administrative strategies. in these unparalleled times, department chairs can create collegial environments by actively affirming the importance of listening to all voices, overcoming ageist presuppositions related to both younger and more seasoned faculty, and ensuring intergenerational collaboration in decisionmaking processes. by attending to the disproportionate impact of ageism on women, people of color, and lgbt individuals, chairs can help overcome persistent ageist barriers and build an inclusive, intergenerational talent proposition that enhances student learning. ▲ presenting a workshop at the academic chairpersons conference in savannah, georgia, on generation z. the session focused on the best ways to work with this generation of students who were born between 1997 and 2012. the session yielded several outcomes and, most importantly, a time for chairs and administrators to talk about ways to serve these students through marketing, technology, outreach, admissions, and advising. this was the last trip i took before my home institution, just outside of denver, colorado, started the massive shift of moving about seven hundred course sections to online delivery. the move to remote learning put my knowledge to the test around ways to teach, connect with, and create stability for gen z students who have already experienced many traumatic events in their lives, and this virus will be another defining generational marker. as educators, now more than ever we are faced with how to effectively engage students in remote learning environments. these environments must account for the diversity of learning and accessibility to all students while managing student accommodations and learning needs. the purpose of this article is to explore the impact of the covid-19 pandemic and remote learning on gen z students. it will also share ways to engage students in a postpandemic educational environment. overview of generation z in the past year, i have facilitated several presentations on generation z across colorado for k-12 educators, higher education professionals, and university faculty and staff. through this work, i have developed a passion for this group of driven, motivated, diverse, and digitally connected students. these students are more focused on quality and easily accessible services than they are on campus amenities. they crave a variety of different types of engagement and community and are constantly toggling between virtual and in-person interactions (seemiller and grace 2015) . my roles in higher education have allowed me to engage with gen z students by advising them on internships and career paths and also teaching mba students (along with two gen z kids in my own home). generation z students are a diverse group, with only 57 percent of students identifying as caucasian, 17 percent as black, and 27 percent as hispanic (seemiller and grace 2015) . this generation is about 6 percent smaller than the previous millennial generation. with the invention of the iphone in 2007, this generation has never lived without a technology component in their lives. these students are connected globally and instantly became proficient at multitasking. they have seen the dot.com bust and the one-click amazon search and have lived through the great recession of 2008 and now the pandemic of 2020. according to seemiller and grace (2015) , another pivotal experience for generation z has been the quadrupling of student loan debt in 2017. millennial parents are still paying off a large amount of educational debt and can be skeptical of the value of higher education. gen z students are looking for authentic and meaningful relationships, along with opportunities to connect through a variety of in-person and online platforms. they want services that are easily accessible in person and online and that are available twenty-four hours a day. these students are driven by their passions, are empathic, and are looking for others to be a part of their personal and professional journeys. this generation is also constantly seeking value and return on investment. the united states started a major shift in thinking about spring 2020 classes on march 13 when president trump declared a national emergency. institutions of higher education deepened conversations around moving the learning environment, including the advising, counseling, and key support services that student rely on at their college campuses. these changes in academic learning happened quickly. even with this digitally savvy group, student connection and campus communities started to rapidly transform. students were reporting the need for enhanced technology and access to the internet. putting these support systems in place outside of the classroom will continue to be critical components of this generation's lives. many institutions will continue to see students lose jobs, become ill themselves, or care for family members, along with a series of disappointments around graduations, connections to community, and uncertain futures (mull 2020) . this generation is also open to discussing mental health issues and to seeking out support services (gillihan 2019). it will be our duty to continue to find authentic ways to provide support services for these students well after this pandemic has subsided. connecting with generation z during the pandemic there are several ways to enhance our engagement with students, faculty, and staff during and after the coronavirus. gen z is accustomed to multitasking. the truth is that we have all been multitasking, with technology creating key missed opportunities for meaningful connection, conversation, and coaching with students. your virtual presence as a chair and administrator matters not only to students but also to others around you. as an administrator and adjunct faculty member, i've had to adjust my focus and work space and devote my whole self to meetings by not checking email or working on other projects. gen z students want the same and are trying to navigate the virtual and real worlds while creating meaningful relationships with faculty and staff. so go ahead and show off your cats, your kids, your artwork, or even the two-thousandpiece puzzle my staff member texted me. as the higher education machine recovers and evolves from this pandemic and we move back to teaching in the classroom (virtual and in person), meaningful interactions with students will be critical. many of these students have encountered job losses, mainly in the service industry. gen z students face pressures from parents and family members who were already skeptical about higher education and will have increased anxiety as they reframe what the world will look like. this resistant generation will be faced with decisions around taking a gap year before college or going to a community college instead of living on campus or attending their first-choice institution (mull 2020) . this generation of students is also incredibly passionate and driven. they are thoughtful about ensuring that they meet whatever goals they choose to pursue. now more than ever, they will look for a return on investment around areas like income, work flexibility, lifestyle, and career paths (seemiller and grace 2015) . therefore, the majors that students pick will matter. the details of the major, the kinds of classes, and the jobs they will be qualified for are important to students and their families. as our institutions of higher education have moved to online learning to meet the challenges of the coronavirus, will we see a generation of more doctors, health-care workers, public health professionals, and epidemiologists? these shifts in career paths could mirror the upticks seen in these types of careers pursued by students after september 11, 2001. this pandemic has disproportionately affected communities of color due to a variety of factors, including access to health care, lower incomes, and the high volume of service-related jobs (mull 2020) . these inequities are important to acknowledge as a piece of this generation's story. exploring why covid-19 has impacted people differently is an opportunity for colleges and universities to expand and create majors in these public health areas. students are also seeing how this pandemic has affected things like climate change, supply chain management, and accessibility to health-care systems across the world. these topics are important to generation z, and department leaders can help these students funnel their pandemic experiences into passions and careers. i ask myself every day how we can help and engage students through this pandemic that could extend into 2021 or beyond. this prolonged critical community engagement at colleges and universities will cause students to change their trajectory around higher education. many students were struggling before the coronavirus, working multiple jobs, carrying debt, and often lacking an outlet at home. students need support and encouragement in their educational growth and development. this tech-savvy generation deserves and needs more from us in virtual classrooms, chat spaces, and one-on-one interactions using a variety of technology-based applications. covid-19 has also shed an even brighter light on the inequities students face in terms of access to key learning technology and programs, internet availability, and food insecurities. as faculty, staff, and administrators, we'll all come out of this experience with a much better understanding of things like microsoft teams, webex, and zoom as well as our professional presence in virtual one-on-one meetings coaching faculty and staff. one way to engage this generation is to provide continuous professional development around technology and to create clear and well-structured course context and opportunities for students to learn. it's important that we help students understand that a degree and that what you learn in college really matter. our jobs as chairs and as educators is to constantly reinforce that educational goals, degrees, and certificates create opportunities for students well beyond this pandemic. this generation must feel understood, heard, and supported. this support will come through in-person authentic engagement and multiple virtual platforms so that students know they are valued and that higher education holds hope and a future for them. ▲ scrambled to transition to emergency remote teaching to deliver instruction in a virtual learning environment. as chair of a department at a 100 percent online university, there were no interruptions to operations. faculty were positioned to maintain business as usual in the classroom; however, this did not prevent their lives and that of their students from being completely disrupted and upended. without forewarning and time to prepare, covid-19 created personal and professional challenges for faculty across all colleges and universities. the unexpected impact from the spread of covid-19 has caused institutions-small and large, private and public-to rethink how they deliver educational instruction. for faculty, covid-19 presented a unique challenge: maintaining student learning and engagement while the normalcy of life collapses around you. becoming parentteachers tasked with daily instruction as local schools shut down. maintaining mandates to stay at home and go out only for essential items. anxiety about loved ones near and far. uncertainty about how long the pandemic will last. questions regarding the economic recovery from this global crisis. as a chair, all these considerations were at the center of my mind as i endeavored to support faculty, check on their well-being, and provide resources to help them assist students, as we all were in the same boat navigating uncharted waters together. what i learned throughout this process is that in leading faculty through this crisis and future ones requires an important leadership skill-the ability to lead with empathy. what is empathy and why does it matter, especially in times of crisis? empathy is caring for the feelings of others and having compassion for their real-life trials and challenges. as we look ahead to figure out what the future holds for us as a nation, conversations and interactions with faculty should not solely take the form of to-do lists. prioritizing relationship building, where conversations emerge from a place of genuine concern for the individual, is key. identifying meaningful ways to connect with faculty will create a positive impact in both the short term and the long term. empathetic leaders recognize three things why young people face a major mental health crisis generation c has nowhere to turn generation z goes to college key: cord-156320-xwuz4ma2 authors: hernandez-ortega, javier; daza, roberto; morales, aythami; fierrez, julian; tolosana, ruben title: heart rate estimation from face videos for student assessment: experiments on edbb date: 2020-06-01 journal: nan doi: nan sha: doc_id: 156320 cord_uid: xwuz4ma2 in this study we estimate the heart rate from face videos for student assessment. this information could be very valuable to track their status along time and also to estimate other data such as their attention level or the presence of stress that may be caused by cheating attempts. the recent edbbplat, a platform for student behavior modelling in remote education, is considered in this study1. this platform permits to capture several signals from a set of sensors that capture biometric and behavioral data: rgb and near infrared cameras, microphone, eeg band, mouse, smartwatch, and keyboard, among others. in the experimental framework of this study, we focus on the rgb and near-infrared video sequences for performing heart rate estimation applying remote photoplethysmography techniques. the experiments include behavioral and physiological data from 25 different students completing a collection of tasks related to e-learning. our proposed face heart rate estimation approach is compared with the heart rate provided by the smartwatch, achieving very promising results for its future deployment in e-learning applications. nowadays e-learning is experiencing a period of high growth thanks to the flexibility it provides to students who do not have the possibility to access to traditional education, like users with an employ, geographical limitations, or any other special conditions. trying to reach that increasing market of potential students, most of higher education institutions like stanford, harvard, oxford, and the mit have started to offer new options of virtual education [1] . moreover, episodes such as the covid-19 outbreak in 2020 and the social distancing imposed, have demonstrated the necessity to develop new technologies to improve e-learning platforms. even though e-learning presents many advantages, it also has some drawbacks, being one of the more relevant the difficulty to demonstrate if an online evaluation is really being carried out by a specific student. without this verification step, it is hard to know if a student has acquired the knowledge associated to a certain course, or if he is incurring in some type of fraud/cheating on the evaluation, e.g. asking another person to complete his/her exam. biometric technologies seem to be a perfect choice to enhance virtual education environments. these technologies allow to identify a person by their physiological and behavioral characteristics, rather than traditional methods such as a 1 https://github.com/bidalab/edbb password or an id card that could be lost, forgotten, or used by another person to perform student impersonation [2] . the interaction between the students and the computer or the device in which they are accessing to the educational contents can be used to acquire other information about their state, e.g. their heart rate, their level of attention, and how much stressed they are [3] . these type of factors, i.e. stress, emotional state, motivation, focus, and attention, can affect the effectiveness of the learning process [4] , [5] . a student who is affected by any external agent or emotion will not take as much benefit of the lessons as another that is totally focused. traditional education theory has been centered in how to explain the contents to the students in the best way possible, but usually without considering these context and human factors. for online education, these elements are specially crucial. the main contributions of this study are: • a brief survey of state-of-the-art biometric and behavioral technologies based on human-computer interaction (hci) with potential application to student monitoring. • the acquisition of a dataset consisting of biometrics and behavioral data using the student monitoring platform for e-learning edbbplat [6] . this database (edbbdb) is publicly available for research purposes (see footnote on this page). • an experimental evaluation of heart rate estimation in the edbb framework, and the development of a baseline algorithm for heart rate estimation based on remote photoplethysmography. • application of the developed baseline algorithm to two different scenarios in a simulated e-learning environment: one of them consists in estimating the mean heart rate of the students over a whole session and the other consists in making a continuous heart rate estimation during a session (useful for detecting heart rate alterations). the rest of this paper is organized as follows. section ii introduces behavioral biometrics and their application to e-learning scenarios. section iii provides details about the structure of edbbplat. section iv explains the different challenges related to student monitoring proposed in the edbb framework, being one of them heart rate estimation. section v shows the experimental protocol and the results achieved for the heart rate estimation sub-challenges. finally, conclusions are drawn in section vi. historically, the first approaches for monitoring student evaluations in remote learning have consisted in installing a special software in the student's computer. this software is intended to be connected to an institutional server in which a learning management system (lms) controls that users do not perform any forbidden action during their evaluations, i.e. executing certain applications such as the web browser, making screenshots, running certain commands, etc. the usage of online supervisors, i.e. people that manually supervise each session by webcam, allows to monitor students in real time in a similar way as in a classroom. however, this method is not scalable to a large number of students. the possibilities of biometric-based technologies for monitoring online evaluations have been recently showed in real world applications like the coursera e-learning platform. in this case, the programmers used keystroke methods [7] , [8] for verifying the identity of the students enrolled in a course. behavioral biometrics refers to those biometric traits that describe the way that users perform different actions [9] . behavioral biometrics traits can be extracted from human-computer interaction, in which a person interacts with some devices, such as computers and smartphones, in a manner that can be highly different among them [10] [12] . a machine learning algorithm can learn patterns from hci data. these patterns will be affected by several factors like the acquisition sensors, the tasks that are being captured, or the human condition and behavior. modelling these data (that usually comes from heterogeneous sources) is useful for a multitude of applications such as elearning, security, entertainment, and health. behavioral biometrics is composed by different traits like touchpad interaction [13] , keystroking [8] , mouse dynamics [14] , [15] , handwriting patterns [16] , and stylometry. relevant works in this field of research demonstrate that the information coming from hci can be used not only for user authentication, but also for characterizing other human features like [12] : neuromotor and cognitive abilities [17] , physiological signals such as human pulse [18] , and human behaviors/routines. we employed the platform from [6] , called edbbplat. it has been designed for capturing data for automatic detection of anomalous behaviors in virtual evaluation environments. table i shows the sensors and the types of data captured by the platform. the data is acquired through a set of activities for the students to complete. the acquisition setup consists of (see fig. 1 left): • video: 4 rgb cameras (2 frontal, 1 side, and 1 zenital), 2 near infrared cameras (intel real-sense model d435i), and depth images. • pulse and motion sensors: we employed a huawei watch 2 smartwatch that captures pulse and motion signals including accelerometer, magnetometer, and gyroscope. • a personal computer with microsoft windows 10, a mouse, a keyboard, a microphone, and a screen. the computer is used by the students to complete the tasks, while the screen data, the mouse and keyboard dynamics, the audio, and other pc metadata are being acquired in the background. the activities that conform the platform consist of 8 different tasks categorized in 3 main groups: • enrollment form: meant for obtaining personal data of the students, e.g. name and surname, e-mail address, id number, and nationality. • writing questions: since this type of questions are more complex, they can be used to measure the students' cognitive abilities under different situations such as: solving logical problems, describing images, crosswords, finding differences, etc. additionally, some activities have been designed to induce different emotional states to the participants, e.g. stress or nervousness. • multiple choice questions: these are questions largely used in online assessment platforms and are included to detect the students' attention and focus levels. an example of the employed sensors and of the information that is acquired while a student is completing a task can be seen in fig. 1 . the work in [6] proposed 5 different challenges that are relevant to student monitoring: • challenge 1 -attention estimation: the estimation of the intensity of mental focus or attention of the students. • challenge 2 -anomalous behavior detection: detection of non-allowed activities performed by the students. • challenge 3 -performance prediction: prediction of accuracy and time necessary for the completion of the tasks. fig. 1 . example of the information acquired for heart rate estimation using remote photoplethysmography. the acquisition setup can be seen in the left diagram. the sensors of the realsense camera used in this case are the rgb and the left and right near infrared channels (top-right images). we show two different groundtruth heart rates captured with the huawei watch 2 smartwatch (bottom-right plots). in these plots, the points in which the users were asked to perform physical activity are highlighted. and, finally, the fifth challenge, which is the main focus of the present paper: • challenge 5 -pulse estimation: changes in the human pulse have showed to be related to altered emotional states and the presence of stress. emotional states can affect perception and performance. understanding the emotional state of the student may help in different ways: 1) online adaptation of the session according to the emotional state (e.g. reducing working load and the difficulty or type of the contents); 2) improved performance analysis including emotional features. the objective of this challenge consists in estimating the groundtruth human pulse (obtained from the smartwatch) by using the front camera. alternatively, the nir cameras present in the acquisition setup can be used to analyse the potential of this type of sensors. in this study we propose an accurate estimation of the heart rate through remote photoplethysmography (rppg) techniques applied to face biometrics [18] . the proposed benchmark is divided into the following two different sub-challenges related to the student activity monitoring: • sub-challenge 5.1 -heart rate averaged by session: knowing the mean heart rate of a student for a whole session can be useful for comparing these values across different sessions. this way we can track the student's activity along time for detecting unusual events. the average heart rate during the task, the grade obtained, and the student historic data (previous average heart rate and grades) can serve to obtain a detailed picture of the student's performance. • sub-challenge 5.2 -heart rate continuous monitoring: this challenge consists in dividing each session in shorter temporal windows and estimating the heart rate for each one of them individually. unlike the first sub-challenge, this approach can be useful for analyzing the state of the student throughout a single session and detecting anomalous behaviors within the session. additionally, this information is useful to better understand the potential difficulties faced during the tasks. plethysmography refers to techniques for measuring the changes in the volume of blood through human vessels. this information can be used to estimate parameters such as heart rate, arterial pressure, blood glucose level, or oxygen saturation levels. the variant called photoplethysmography (ppg) includes low-cost and noninvasive techniques associated with imagery and the optical properties of the human body [3] . oxygenated blood absorbs more light at specific wavelengths than the blood with less oxygen, so measuring over time the amount of light reflected by the tissues of a person, we can estimate his pulse signal and other parameters like respiration variability [19] . studies have proven that it is possible to measure the changes in the amount of oxygenated blood through facial video sequences [20] . these techniques are called remote photoplethysmography and their operating principle consists in looking for slight changes in the skin color at video recordings using signal processing methods [18] . remote ppg methods can take advantage of cameras that contain both rgb and near result of the heart rate estimation (right). the highest peak in the acquired heart rate corresponds with a moment in which the student was requested to perform a 20 seconds period of physical activity to get him into an altered state. the mean heart rate for the whole session (sub-challenge 5.1) and the values of the heart rate for 10 second windows (subchallenge 5.2) are also shown. infrared sensors. the nir spectrum band information is highly invariant to light conditions, providing robustness against this external source of variability at a low cost. the nir band can also help to derive depth information that could improve the location accuracy of the regions of interest (roi) at face tracking. our approach is based in the one presented in [21] and consists in four main stages: i) we first locate and track 3 different regions of interest in the student's faces, i.e., the forehead and the right and left cheeks (see figure 2 left); ii) we track the regions during the video and we extract their raw rppg signals; iii) we postprocess the raw rppg signals from the 3 regions using a moving window to isolate the component associated to the pulse by minimizing the other components in the video sequences; and iv) we estimate the value of the heart rate for each temporal window by analyzing the frequency components of the postprocessed rppg signal and we concatenate all these values for obtaining the heart rate estimation for all the video sequence (see fig. 2 we have acquired 25 different students while completing the tasks described in section iii-b. the duration of each video recording is variable, going from 15 to 30 minutes. one session has been recorded for each student. the video sequences have been captured at 30 frames per second with the intel realsense camera (we have used both the rgb and the nir channels), with a resolution of 1280 × 720 pixels. the groundtruth for the heart rate has been acquired with the huawei watch 2 smartwatch at a sampling frequency of 1 hz. an example of the images captured with the realsense camera and the smartwatch can be seen in fig. 1 right. during the acquisition, each student had to perform physical activity in a different moment of the evaluation in order to put him into an altered state with a higher heart rate. with the physical activity we intended to simulate possible situations in which the pulse of the student may vary due to events such as high stress or cheating attempts. we are aware that physiological changes are highly related with the nature of the stimulus. changes in the pulse due to physical activity may show different physiological responses that those caused by stress level for example. however, the resulting changes in the heart rate should be similar. we decided to use the rgb and the nir channels in order to compare the results obtained with each type of images. however, in most acquisition setups, the only available sensor will probably be a rgb camera, so we have centered our analysis in the results obtained with that frequency band. the metric used to report the accuracy in the heart rate estimation challenge is the mean average error (mae) expressed in beats per minute (bpm). mae refers to the mean difference in absolute value between the estimated heart rate and the groundtruth. this metric can give us an idea of the average accuracy we can expect of our heart rate estimation method, thus giving us orientation of its possible applications. there are slight differences in the protocol we followed for each one of the two sub-challenges. the first step is common to both challenges: we divided the video sequences in temporal windows of a fixed length and we computed a value of the estimated heart rate for each one of these windows. regarding the groundtruth heart rate, we computed the mean value of the samples acquired with the smartwatch from each temporal window. 1) sub-challenge 5.1 -heart rate averaged by session: for computing the mean heart rate of a whole session we calculated the average of the heart rate estimations of all its temporal windows. then we used the absolute difference between the estimated mean heart rate and the groundtruth as our error metric in beats per minute (bpm). we have selected values for the window length going from 5 to 20 seconds with an increment of 5 seconds. in this case we took the estimated heart rate and the groundtruth heart rate for each single window and we calculated the absolute difference between them. after that we averaged the error of all the windows inside each video sequence. the results of each session were then combined to produce a single performance measure for the whole dataset, i.e. the mean average error (mae) expressed in bpm. in this case we explored values for the window length going from 5 seconds to 20 seconds with a step of 2 seconds. in this sub-challenge we have calculated the mae values for the estimation of the heart rate for complete sessions. the rppg algorithm used in this work employs information from the three color channels available in rgb videos. however, in nir videos only one channel is available, so we replicated its information into three different channels to imitate a rgb video. in table ii we can observe a clear trend of the heart rate estimations, where the nir videos obtain a higher accuracy when using short video windows, while the rgb-based estimation is the most accurate when using a longer window duration. the accuracy obtained is high for both types of videos, being slightly higher for the nir band when using short windows, and better for the rgb color channel when using a wider temporal window. we think that this may be caused by the fact that the nir band is more robust to external illumination changes that affect severely to the rppg heart rate estimation. however, for longer window sequences, having more information available (three channels instead of one) makes possible to obtain better rppg signals. this sub-challenge may be applicable for monitoring the state of the students between sessions, i.e. knowing in which fig. 3 . temporal evolution of the heart rate in a scenario in which the student has been induced to an altered state by means of physical activity at the beginning of the session. the four plots correspond to the same video sequence but with different temporal window lengths. the figure shows the changes in the accuracy when changing the length of the temporal window. classes or evaluations the mean heart rate is higher or lower. these alterations may be caused by user impersonation, lack of interest, or a high level of stress. 2) sub-challenge 5.2 -heart rate continuous monitoring: table iii shows the performance results for heart rate estimation obtained for different values of the temporal window, going from 5 seconds to 20 seconds, and also for both the rgb and the nir bands. it can be seen that the mae decreases when increasing the temporal window length because the algorithm has more information for extracting the frequency components correspondent to the heart rate. however, when the window duration reaches a limit (close to 20 seconds in both cases) the mae does not further improve due to the variations of the heart rate inside a too long window. other drawback related to the use of a longer temporal window is the lower temporal resolution of the predictions. if the heart rate changes quickly, a long temporal window will not be able of capturing that behavior. similarly to the case of the subchallenge 5.1, in this case the accuracy is slightly higher for the nir band when using short windows and better for the rgb color channel when using a wider temporal window. fig. 3 shows the temporal evolution of the heart rate estimation in a scenario in which the students performed physical activity at some points of the evaluation in order to get their heart rate artificially high. the target is checking if the heart rate estimation algorithm is capable of detecting these changes in the heart rate. by inducing alterations we want to simulate a situation in which a student performs any forbidden or inappropriate action, e.g. cheating, that may lead to an altered heart rate. the four plots in the figure correspond to the same video sequence but with a different temporal window length. the figure shows how the estimation algorithm manages to capture the main behavior of the heart rate during the induced alterations. it also reflects the change in the accuracy for the heart rate estimation for the same video sequence when changing the value of the temporal window. as has been said previously when commenting the results of table iii , a higher value for the temporal window makes the mae to decrease. this is shown in fig. 3 with the plot of the averaged groundtruth and estimated heart rates, that become closer when increasing the temporal window length. however, it can also be seen that even though using smaller windows decreases the general accuracy of the heart rate estimation, it also allows to reflect better the quick changes in the heart rate due to the altered states induced in these experiments. these quick changes in the heart rate can only be captured when using lower values for the temporal window. this way, the decision of what window length must be used depends of the desired application. in this paper, we have: i) discussed the application of behavioral biometrics for remote education, ii) employed edbbplat [6] , a platform of biometrics and behavior for student assessment during virtual education, iii) captured data from sensors that are usually present in remote education (rgb cameras), and also from more advanced sensors like nir cameras and a smartwatch, and iv) used the acquired nir and rgb video recordings for estimating the heart rate of the students using rppg while they are completing a series of virtual evaluation tasks. the type of information acquired in this work can be used for detecting unusual events during an evaluation task in remote education. some examples of events that can be detected are: cheating attempts, a stress level out of the ordinary values, drops in the level of attention of the students, or changes in their heart rate. for future work, we expect to add different types of stimuli that lead to altered states. correlating those altered states with the information from the other basic and advanced sensors of the platform (eeg band, other cameras, test results, etc.) may be helpful for detecting inappropriate behaviors and other factors such as the stress level, the focus level, or even for trying to predict some variables like the student's performance. students' perceptions of teaching and social presence: a comparative analysis of face-to-face and online learning environments biometrics systems under spoofing attack: an evaluation methodology and lessons learned photoplethysmography and its application in clinical physiological measurement academic emotions in students' self-regulated learning and achievement: a program of qualitative and quantitative research emotions in classrooms: the need to understand how emotions affect learning and education biometrics and behavior for assessing remote education keystroke biometrics ongoing competition typenet: scaling up keystroke biometrics 50 years of biometric research: accomplishments, challenges, and opportunities understanding and changing behavior smartphone sensors for modeling humancomputer interaction: general outlook and research datasets for user authentication benchmarking touchscreen biometrics for mobile authentication what can a mouse cursor tell us more?: correlation of eye/mouse movements on web browsing becaptcha-mouse: synthetic mouse trajectories and improved bot detection benchmarking desktop and mobile handwriting across cots devices: the e-biosign biometric database active detection of age groups based on touch interaction a comparative evaluation of heart rate estimation methods using face videos photoplethysmography: beyond the calculation of arterial oxygen saturation and heart rate advancements in noncontact, multiparameter physiological measurements using a webcam time analysis of pulse-based face anti-spoofing in visible and nir key: cord-024381-ckkzu57a authors: zwolski, kamil title: assessing international relations in undergraduate education date: 2020-05-06 journal: eur polit sci doi: 10.1057/s41304-020-00255-0 sha: doc_id: 24381 cord_uid: ckkzu57a this paper advocates a holistic approach to assessing international relations in undergraduate education, which revolves around: (a) essays and (b) active learning-related tasks, such as simulation reflective statements/reports and performance. the paper argues that, on the one hand, academic essays are far from irrelevant and it is difficult to overestimate their practical significance. on the other hand, active learning-related tasks are best utilised as a supplementary assessment, expanding the students’ range of transferable skills. the assessment structure advocated in this paper results from a holistic approach to assessment design, which includes teacher’s own experience, familiarity with pedagogical scholarship and input from students. this last element is the least common even though it makes sense to understand how students see their own assessment. to that end, the paper shares the results of a pilot project run at one of the uk universities, which engaged students as partners in rethinking their assessment. and learning ir in general, rather than scrutinising the assessment element. unsurprisingly, most debates concern innovative approaches to ir teaching and learning, most notably simulations. this paper explores simulations as a potentially beneficial teaching activity and advocates considering this and similar active learning methods in the context of the holistic framework for assessing learning outcomes. in the second section, the paper discusses the proposed holistic approach to designing assessment, which involves building on one's own experience, catching up with the pedagogical scholarship and working together with students in order to obtain a better understanding of the local learning culture. this last point is further developed in the subsequent section, which reports on the experiment in forming a staff-student partnership for enhancing assessment and feedback practices. the final part of the paper explains the two-tier assessment strategy for undergraduate ir teaching, followed by the limitations of the framework presented in this paper accompanied by possible solutions. the question of assessment, including its purpose, format and feedback, has long been at the centre of undergraduate student learning. not only are the examinations, essays and other summative assignments crucial for awarding degrees, but assessment also has a defining role in shaping the patterns of student life at a university. several foundational studies in the usa and uk revealed, as early as in the 1970s, the extent to which students care, or even obsess about assessment. making the grade (becker et al. 1968 ) pointed to the so-called gpa (grade point average) perspective, signifying the finding that very few students are genuinely interested in the process of learning, as opposed to strategising about achieving the highest grades. the hidden curriculum (snyder 1971 ) famously exposed the importance, from the students' perspective, of the informal rules about navigating the landscape of studying for a degree. finally, up to the mark: a study of the examination game (miller and parlett 1974) famously identified students as cue-seekers, cue-conscious or cuedeaf, depending on how well attuned they are to what they think is actually expected in assessment. all those studies, together with the plethora of subsequent publications, point to the fact that students care great deal about assessment. in this context, it is rather surprising that almost no academic publications focus on how to assess ir in higher education. this absence may also affect other academic subjects, but what makes it interesting in ir is the fact that there is substantive literature on the teaching and learning of ir as such. almost all that literature, however, revolves around innovative methods for engaging teaching delivery, with the bulk of analysis focusing on simulations (e.g. kaunert 2009 ). this emphasis on simulations is hardly surprising. after all, nation states serve as a point of departure, as well as remain the focal point throughout the entire ir module delivery. even when the overarching argument is that states are not the only meaningful actors in ir, concepts such as globalisation or global governance are still considered against the core notion of sovereign nation states. just like the nation states, individual students possess different qualities, enjoy some degree of autonomy and are constrained and enabled by a variety of structural and individual "variables". consequently, simulations can offer students a glimpse into the "real" world of international politics, bringing the more abstract contents of an ir module to live. still, setting up and running simulations in a way which supports student learning and learning objectives are far from straightforward and involve risks. for example, one key question concerns evaluating the strengths and weaknesses of available simulation models, such as statecraft. statecraft is a commercial, ready-made and fully online simulation of international politics, making strides into ir classrooms around the world (carvalho 2014; epley 2016; linantud and kaftan 2018; raymond 2014; saiya 2016) . in module evaluations, students repeatedly say that statecraft is fun and engaging (raymond 2014 ), but it is notoriously difficult to confirm its learning-enhancing quality (epley 2016: 214-215) . other possible simulation designs involve adapting the traditional diplomacy (mattlin 2018) board game or placing students in a hypothetical scenario of a zombie attack (horn et al. 2015) . while simulations represent the most popular form of active learning activities in ir classes, there are other non-standard techniques intended to boost student engagement and understanding. among those, films are on the rise, with some classes combining the classical features (e.g. duck soup from 1933 or all quiet on the western front from 1930) with more recent releases, such as lord of the rings and 300 (engert and spencer 2009; simpson and kaussler 2009; valeriano 2013) . other, less common delivery techniques involve civic engagement and service learning (glazier 2015) , as well as teaching ir through arts (ramel 2018) and even dance (rösch 2018) . while all of these in-class teaching activities may truly help to engage students and render abstract concepts more accessible, there is no evidence that active learning activities mitigate the "hidden curriculum" effect, which prompts students to strategise in a way to neglect those aspects of learning which are perceived as not directly relevant to assessment. if anything, modern students may be even more oriented towards outcomes as opposed to the process of learning than those attending universities in the 1960s and 1970s. the discussion whether this process of instrumentalisation of university studies is right, or how to change it, goes far beyond the scope of this paper. the fact is that modern universities act as if assessment is the cornerstone of studying for a degree, as indicated by the amount of time and resources allocated to monitor the "quality" of assessment and decide on student assessment/degree outcomes. at the same time, students' perceptions about the importance of "results" are often shaped even before they come to the university by stories they hear from their parents and their understanding of what it takes to build a successful professional life. consequently, there are good reasons for university assessment to be thought-through, seamlessly integrated into our understanding of what kinds of knowledge and skills our students should acquire. some of the rare contributions on assessment in ir focus on measuring student performance in simulations (kollars and rosen 2013; raymond and usherwood 2013) , with the notable exception of the paper arguing for short paper assignments in classes (mcmillan 2014 ). raymond and usherwood make an interesting point on the limits of reflective statements, widely used in simulation exercises (notably in statecraft), which ask students to reflect on their simulation experience and link it to the theoretical contents of the module. the authors note that drawing any definite conclusions about student learning progress based on their reflective statements is unjustified, citing literature questioning how accurately we can recreate all that happened in the past (including causal links) (raymond and usherwood 2013: 160-61) . kollars and rosen, on the other hand, list the benefits of assessment through simulations. one of the alleged benefits is that simulations allow developing a better understanding of students' true abilities because they allow observing student progress over a longer period. in contrast, examinations may induce stress, which may cause otherwise good students to produce poor results (kollars and rosen 2013: 153) . finally, mcmillan stresses the advantages of short paper assignments over longer research papers, arguing that the former can serve the same purposes as the latter, but short assignments come with a plethora of other benefits, such as closer links with the type of writing students will be doing in their jobs (mcmillan 2014: 109-10) . the purpose of this paper is more encompassing. rather than evaluating assessment options for active learning activities, this paper suggests a comprehensive framework for assessing ir in undergraduate education. an integral part of the framework itself, however, is the method underpinning it. the following section explains the holistic method, which informed the two-tier ir assessment strategy introduced later in this paper. in this paper, i argue that a well thought-through assessment strategy should, ideally, rest on three components: experience, some knowledge of the relevant pedagogical literature and student engagement. experience and theoretical expertise pertain to the classical distinction between knowing how and knowing what. while the latter refers to our knowledge of theories and principles, the former "rests on bodily experience and practice: it is knowledge within the practice instead of behind the practice" (pouliot 2008: 267) . of course, this distinction is ideal, and our understanding of assessment is likely underpinned by some theoretical knowledge. what i propose is to make the distinction between experience and theoretical expertise explicit in our thinking about assessment, even if the boundaries between the two are blurred in our professional development. for example, it is established practice to offer graduate teaching assistants some brief pedagogical training to prepare them for teaching and marking. in the uk context, it has also become common for universities to require their early-career staff to acquire professional recognition through the fellowship scheme of the higher education academy (now advance he). both modes of learning, through acquiring theoretical knowledge and through practice, have their own place in developing effective assessment strategies. the final component concerns student engagement. there are different forms of engaging students in discussions on assessment and feedback, ranging from ad hoc consultations to institutionalised channels for working with students in partnership. conversely, there is also a variety of areas suitable for staff-student partnership schemes, including (a) learning, teaching and assessment; (b) curriculum design and pedagogy; (c) subject-based research; and (d) scholarship of teaching and learning (healey et al. 2014) . those areas often overlap, as they do in my own initiative, which i will now briefly outline. in the academic year 2018/2019, i launched a pilot project intended to engage undergraduate students in my department, across all 3 years, in a sustained and meaningful discussion on assessment and feedback. my intention was not merely to elicit information from students on how they would like to be assessed. instead, i framed the initiative as a way to enhance our learning community, which implied that the learning process was going to take place both ways (as it did) (healey et al. 2014: 25-35) . for implementation, i relied on several online platforms in order to make the initiative flexible. specifically, the microsoft teams collaborative platform played the central role, supported by microsoft stream for recording videos and microsoft forms for collecting student responses. a typical flow of work would involve myself recording a 5-10-min video outlining an issue i would like to discuss, such as students' preferred methods of assessment. afterwards, i would ask students to answer a few questions, such as how they would rate several different assessment forms and which ones were their most/least favourite types. after reading the responses, i would record another video commenting on student responses and offering them some tips and insights into how assessment works from the pedagogical perspective. finally, i would ask students to comment on that follow-up video, which would often lead to the new cycle. overall, between 70 and 90 students actively participated throughout the project, and they made over 500 written contributions on different areas of assessment and feedback. the project ended in april 2019 with very positive student feedback. the question concerning students' assessment preferences was the first one i raised. first, i asked students to rate 15 different forms of assessment according to their preferences, with a system of five stars ratings. students were invited to freely interpret the notion of "preference" and whether it related to value-added to their learning process, perceived difficulty, student familiarity or something else. the subsequent two open qualitative questions, where students justified their most and least preferred choices, allowed them to clarify what it is exactly that they like/ dislike about those assessment types. ninety students responded. my expectation was that students would prefer innovative or non-standard forms of assessment to essays and examinations. in reality, essays emerged on top of the list, followed by essay plans and short response papers. assessment related to problem-based learning (pbl) and simulations were next on the list, with the least favourite assessment forms being poster presentations, oral presentations and examinations. while pbl and simulations can, more appropriately, be classified as assessment activities rather than assessment forms, carol evans rightly includes them on the list of assessment types 1 because they are inherently predisposed to generate a variety of non-standard assessments, such as practical performance or written/oral interpretation of practical experience. for this reason, it is difficult to disentangle pbl and simulations as learning techniques from associated assessment as these non-standard learning activities typically work in tandem with a set of non-standard assessment forms. when justifying their choices, students defended essays as representing the fairest form of assessment, allowing them to demonstrate their true academic potential. arguably, essays allow undertaking a more in-depth approach to researching a specific topic. they support advancing a careful and considerate thought and offer enough time to research and explore the topic. essays allow getting to grips with different arguments. because there is plenty of time available, essays represent a true reflection of the students' ability. finally, essays allow going into the details of a question and carefully study the answer. they are effective in drawing students' attention into the relevant literature and are supportive of students' developing own arguments. based on the combination of practical experience, insights from pedagogical literature and student engagement, i argue in the following section that essays should constitute the bedrock of assessing ir in undergraduate education. in fact, skills associated with academic essay writing have become crucially important-more than anywhere in the past-and it is difficult to overestimate their practical significance. for the purpose of my argument, i do not draw the distinction between longer papers and short response papers. this kind of distinction was indeed relevant for mcmillan (2014), who argues the value of short papers over longer ones in ir, but undergraduate essays come in different sizes ranging from 500 to 600 words short papers all the way up to 5000-6000 words. it is even possible to consider the final undergraduate project itself-the dissertation-a one long essay, especially if it is oriented towards theory or history. instead of the length, what matters is the underpinning characteristics of academic essays. these are long established and include (a) explicitness, in that the academic essay form is devoid of nuances; (b) the rationalist and humanist paradigm, in that the written form is considered crucial for exchanging knowledge and fostering progress; and (c) persuasiveness, in that the essay aims to advance ideas supported by explicit evidence (andrews 2003) . in the uk context, the industry-oriented and economic growth-driven policy agenda has led to the dismissal of the humanities and social sciences subjects, in which essays dominate, and the promotion of science and engineering (stem) subjects instead. as such, and in comparison with the more pragmatic, technical and result-oriented reports, it may be tempting to view essays as "academic" in the pejorative sense of the word, i.e. entailing abstract and hence impractical approach to theorise about ideas. in reality, essays allow to develop and practice skills relevant not only for employment prospects, but also-from the ir perspective-to navigate the completely new reality of the interconnection between international politics, new technologies and populism. among others, academic essays allow to practice "research skills, logical and critical thinking skills, clear expression, independent learning, communications skills, organisational skills, time and task management skills self-awareness, reflective skills (…)" (shields 2010: 20) . one helpful way to distinguish between the value of essays for assessing ir from methods related to active learning is to recognise that they emphasise two different kinds of knowledge: declarative and functioning. the word "emphasise" is appropriate here because the two kinds of assessment do not align themselves neatly with the two kinds of knowledge. essays, for example, can ask students to apply declarative knowledge to an empirical problem, such as an international conflict. simulationrelated assessments, on the other hand, may require students to identify relevant facts or argue their position. both kinds of knowledge can be further divided up into different levels of demonstrated complexity using so-called solo framework, which stands for structure of the observed learning outcome (biggs and tang 2011, pp. 86-90) . the solo framework stipulates that when students learn, their understanding should grow in complexity, progressing across five levels. these learning levels are: 1. prestructural indicating that students only understand individual words, but their sentences miss the point due to lack of knowledge and understanding. 2. unistructural indicating that students can identify one basic aspect of a problem, but missing other dimensions. 3. multistructural indicating that students know various facts (individual trees), but lack an overarching understanding of a problem (the forest). 4. relational indicating that students can tie together different facts into a coherent "whole". different facts and concepts are integrated to present us with the "forest view" of a problem. 5. extended abstract indicating that students can move beyond the immediate confines of the question and conceptualise their response at the higher level, offering the reader a new perspective on a problem. the model will look familiar to the students of rosenau and durfee (1995) and their ladder of abstraction, which stipulates that we should always strive to reach a higher level of understanding by shifting our focus away from simple facts and towards identifying relationships and patterns, making ourselves more theoretically conscious. the bottom line of both propositions is that learning entails improving our comprehension of complexity, which entails thinking in a more structured and logical way, reaching better-informed conclusions or being able to synthesise information based on sound assessment of arguments. those are all components of critical thinking-a foundational skill (still) underpinning the rationale behind higher education. in this context, essays remain a highly suitable and practical way for students to practice their complexity comprehension, as well as to demonstrate their progress to the degree-awarding institution. the solo framework, in turn, allows situating students' progress on a scale from a very basic understanding all the way up to the ability to abstract the problem and see it from a new perspective. although it is difficult to overemphasise the practical implications of practicing and improving the skills promoted by a simple academic essay, one caveat needs to be made here. naturally, each module should begin with setting up learning objectives and those objectives should dictate the form of assessment. in the words of biggs (2003) , assessment should not come as a standalone consideration and instead should be constructively aligned with the learning outcomes. this article specifically focuses on the academic disciple of ir, which has developed a relatively welldelineated scope over 100 years since establishment. if the purpose of higher education, in the words of mccaffery (2019: 35), is to "complement critical thinking (…) with critical self-reflection and critical action", then it is the overarching purpose of ir education to nourish this "formation of critical being" in relation to politics beyond the boundaries of the nation states. critical being in ir requires awareness of key structures and processes constituting the realm of the "international" and how they manifest themselves in political agendas, decisions, actions or inactions. for example, what are the implications of climate change or coronavirus to international politics? how to make sense of brexit or donald trump's foreign policy? regardless of specific learning objectives, few ir teachers will deny these are relevant topics in ir education, and so, the following examples illustrating the relevance of academic essays relate to some of these topics. in foreign affairs, tom nichols (2017) paints a sobering, if not entirely shocking image of the american rejection of experts and expertise. what is particularly disturbing, according to nichols, is not the fact that americans do not know about the world (roughly half of the polled respondents favoured bombing agrabaha fictional country from a disney movie), or even that they distaste intellectuals. what seems new and alarming is that ignorance is increasingly considered a virtue and even the most heated argument exchanges are simply replaced by "shouting matches". one interesting twist to this argument was the observation that the more ignorant the respondents were about the topic, the more definite answers they offered. another one was the fact that respondents seem equally ignorant on both sides of the ideological spectrum-conservative and liberal. in the uk context, a study of voters' knowledge of the european union (eu) revealed one similar trend: contrary to the popular perception among "remainers" that the brexit voters were uninformed, the level of eu knowledge is actually the same on both sides (carl 2019) . in one study, both "remainers" and "leavers" scored an average of 60% of correct answers to the list of 15 questions about the eu. while it is debatable whether this level of knowledge-on both sides-was enough to decide on an international political issue of such an enormous magnitude, other research findings from the study are actually more interesting in the context of academic essay writing. out of the 15 questions, nine were considered by the authors "ideologically neutral", in that the response to those questions was not particularly convenient for either of the sides. six questions, on the other hand, could be considered more "ideologically convenient" for one of the sides, in that they asked about things such as the level of the uk's contribution to the eu budget or whether the uk contributes more to the eu budget than it receives. what is interesting is that factors such as age and higher education affected the level of knowledge related to the first nine "neutral" questions. in contrast, the political ideology of the respondents seems to have gotten an upper hand when answering the more "ideological" questions, in that both sides responded relatively better to questions, which appeared more ideologically convenient to them. for example, "leavers" were more often correct than "remainers" when responding to the statement that "the uk currently pays more money into the eu than it gets back in the form of subsidies and other funds" (which is true). conversely, "remainers" were more often right when responding to the statement that "more than ten per cent of british government spending goes to the eu" (which is false). both the us and uk examples illustrate that people are generally ignorant about the complex matters of international politics. this is hardly surprising. further, their ignorance does not prevent them in any way from articulating definite opinions, participating in referendums or otherwise acting upon their believes. the american and british examples further confirm the power of ideological biases, which are well known in the world of party politics. we could argue that in the current age of enormous political polarisation, the high level of ideological bias is to be expected among both politicians and the population at large. university campuses, especially in the usa, are not immune to it (mazer 2018) . still, if there is any meaningful role that universities can still undertake other than proffer their brand name to their graduates' cvs, it is to encourage students to become more critical thinkers and writers. this brings us back to academic essays, which are rightly at the heart of facilitating this task. returning to the example of brexit, ideological position, combined with some rudimentary knowledge of the eu, prompted uk citizens to cast their vote in a referendum. many of those voters likely operated with the unistructural or-at bestmultistructural level of eu knowledge, identifying a limited number of individual facts about the eu. academic essays, by their nature, encourage research when confronted with a problem, prompting students to seek evidence in support of their argument and nourishing the habit of seeking information from higher-quality resources. at this basic level, they encourage students to progress towards practicing multistructural knowledge, where they can identify multiple facts in support of their argument, but also describe and classify those facts. the value of academic essays is most prominent, however, when they prompt students to progress towards higher levels of comprehension, as indicated by the relational and extended abstract learning levels. here, to continue with the brexit example, students are not only able to identify multiple facts relevant to the discussion on the uk's membership in the eu. they are also able to compare and contrast these facts, analyse and explain them and ultimately argue their case for the uk's relationship with the eu. eventually, students should also be able to move outside of the immediate confines of the brexit question and generalise as well as theorise the problem at the higher level of abstraction by linking it to related concepts such as sovereignty, populism, globalisation or fear. as for the question of ideological bias, the practice of essay writing remains the most promising exercise requiring students to systematically challenge and question their prior believes and value systems. if they come to the university convinced that, for example, excessive migration from eu countries justifies brexit, they may still leave the university holding that core believe. at the same time, however, after writing numerous essays concluded with a long-form research project, they should graduate being able to better appreciate the complexity and interconnectedness of those multifaceted political phenomena. they should also be equipped to better recognise how their own ideological biases affect their position on the questions of national and international politics. to reinforce the value of academic essays for assessing ir is not to dismiss more practical assessment methods, such as those stemming from in-class simulations and other kinds of pbl. here, the paper aligns its argument with most of the writings on ir pedagogy, but it emphasises equal value of both traditional and innovative assessment methods. they are both important because-as noted-essays remain highly practical and relevant. further, they both emphasise distinct kinds of knowledge, although with overlaps. essays emphasise declarative knowledge. they teach students to describe, classify, compare, contrast, explain, argue and analyse. they also teach to theorise and generalise (biggs and tang 2011: 124) . assessments related to problem-based and active exercises such as simulations, on the other hand, help students learn how to apply knowledge, solve immediate and remote problems, reflect and improve practice (biggs and tang 2011: 124) . in the staff-student partnership initiative, assessment through pbl exercises, including simulations, constituted the second most preferred method among students participating in my staff-student partnership initiative. in this instance, many students who participated in the survey also participated in the statecraft simulation, so they understood what kind of assessment was linked with the simulation. it included quizzes, reflective statement and performance. in other educational contexts, where students don't know how a simulation exercise could be assessed, it will help to offer specific examples. as with essays, students justified their preferences in a variety of ways. arguably, simulations make learning more effective because they allow practicing it and applying to problems. simulations are especially relevant for learning theories and ideas. simulations are arguably right for individual expression, and they allow learning without the pressure associated with presentations or debates. simulations are entertaining, enjoyable and a compelling reason to engage with the module. related to that point, simulations raise the level of engagement and focus among students when playing a simulation. another advantage of simulations is their perceived utility for practicing skills important for employability. different students pointed to different reasons as to why pbl methods are a good way to assess ir, but they mostly revolved around these few points. these pro-simulation arguments align with the pedagogical literature. a quick glance at the scholarship on teaching and learning in ir indicates that among pedagogically aware academics, there is a strong inclination to move towards active learning methodologies, most notably simulations. as noted, this is hardly surprising because the field of ir is naturally predisposed to encourage this kind of teaching approach. glazier reinforces this observation, noting that the use of simulations for teaching ir has greatly increased, even if the idea is not new (glazier 2015:266-267) . simulations are supposed to offer students a different and, arguably, more effective, way of learning compared to reading and listening. the concepts, dilemmas, problems and theories familiar to ir scholars become more accessible to students through simulations, allowing students to experience them in a controlled environment (wedig 2010) . the simulations are also seemingly a more appropriate method for millennial students, who are more receptive to problem-based methods and the learning process cantered on their practical experiences (crossley-frolick 2010) . finally, computer-based simulations should also develop students' reflective and critical thinking skills with computers acting "as cognitive amplification tools for reflecting on what students have learned and what they know" (jonassen et al. 1998 ). as noted, simulations, while not representing an assessment type as such, invite a variety of assessment methods which can complement essays and diversify student experience. reflective statements, for example, encourage students to consider their practical experience in the context of the theoretical contents of the module [even if this method has limitations-see raymond and usherwood 2013) ]. assessing simulation performance, on the other hand, encourages students to take the exercise seriously and invest some intellectual effort into it. simulations, along with other pbl approaches (such as solving practical world problems in class), represent so-called high-impact pedagogies in higher education learning. their aim can be identified as to "adequately equip students with the knowledge, capabilities and personal qualities that will enable them to thrive in complex and changing contexts" (evans et al. 2015: 11) . simulations, if properly designed and assessed, can certainly contribute to these noble objectives. at the same time, however, we must also underline the lack of concluding evidence confirming the value of high impact pedagogies for stronger learning outcomes (evans et al. 2015: 11) . j. celeste lay and kathleen j. smarick, whose students simulated a us senate office at a large us university, caution against setting expectations for simulations too high. they reinforce the argument of this paper by recommending that simulations should not be adopted as a single teaching method, but instead as a supplement to more traditional pedagogies (lay and smarick 2006) . i am under no illusion that the two-tier assessment model proposed in this paper, together with the holistic methodology for designing assessment, can be easily implemented across undergraduate ir programs in the uk and elsewhere. consequently, instead of a traditional conclusion, i propose to address the potential obstacles both junior and senior academics may face when experimenting with assessment design along the lines outlined here. first, there is the question of the actual purpose of assessment. the two-tier model is based on a rather idealistic assumption that students generally attend classes and engage with the contents of the module. with that assumption in mind, we can then focus on designing assessment, which, we believe, makes most pedagogical sense based on our intended learning outcomes, pedagogical literature and student preferences. the reality, at least in the uk context, is often very different, however. since lectures are mostly non-compulsory in undergraduate education, some students choose not to attend them. this is a problem for different kinds of reasons, and so, the teaching staff typically reacts by seeking ways to improve attendance. if students strategise their time allocation around assessment, as evidenced in this paper, then essay-based assessment is not optimal because it does not encourage active participation. in fact, anecdotal evidence suggests that it seems perfectly logical for many students to only attend sessions covering the topic of their essays and consider other sessions less relevant. as a result, if lecture attendance and participation is a problem, then other forms of assessment may be considered more suitable to entice student participation. in this case, assessment assumes an additional, disciplinary function. examination naturally comes to mind as a popular form of assessment, which can be hoped to discipline students to attend sessions, provided the lectures are not recorded and made available online. in this instance, the teacher must use judgement and try to reconcile the pedagogical value of different forms of assessment with the cultural setting in which learning takes place. the second challenge concerns replicating the holistic method for assessment design, which involves teacher's own experience, familiarity with the pedagogical research and working in partnership with students. in my own experience, setting up effective staff-student partnership proved most time-consuming, although most of the time was spent in preparation for the initiative. once i got a good grasp of the ms teams platform and how to upload videos, i was able to run the initiative committing no more than 2-3 h every other week. this may still prove too time-consuming for many. the alternative to a structured and technology-enhanced partnership with students could be a less formal and more ad hoc system of chatting with students about their assessment and feedback preferences. the idea is to get a better understanding of how students themselves see assessment and feedback, which may differ across institutions and cultures. another issue is finding time for pedagogical research. in the field of ir, there is a decent amount of the literature on designing teaching, learning and assessment, but it mostly relies on the broader field of higher education pedagogy (active learning, pbl, etc.). one efficient way to gain insights from the pedagogical scholarship is to meet with colleagues working in the field of higher education and discuss ideas. another approach would be to link pedagogical research for assessment purposes with other objectives, such as applying for an advance he's fellowship, a promotion or simply to better fulfil one's administrative role. the final challenge also concerns time pressures faced by modern academics, but this time it relates to executing the two-tier assessment model discussed in this paper. essays are relatively straightforward, and all academics (and students) are familiar with the format. however, one aspect of essay-based assessment, which students in my staff-student partnership raised and which is reinforced by the pedagogical scholarship, can significantly raise the workload for the marking staff. it concerns the need for so-called feedforward, entailing some form of a learning curve for students whereby their initial writing attempts have lower stakes in relation to the final essay. those lower stake assignments could be shorter essay plans or drafts, but they still require someone to read them and offer feedback. similarly, simulations can be prohibitively time-consuming and even fully automated simulation platforms, such as statecraft, require significant time and a learning effort on the part of the instructor. the only practical solution, in this case, may be to rely on teaching assistants for the day-to-day facilitation of the simulation, but the availability of those extra teaching resources will vary across institutions, programs and modules. the end of the essay? teaching in higher education making the grade: the academic side of college life aligning teaching for constructing learning teaching for quality learning at university are leave voters less knowledgeable about the eu than remain voters? brexit and public opinion 2019 virtual worlds can be dangerous: using ready-made computer simulations for teaching international relations beyond model un: simulating multi-level, multi-actor diplomacy using the millennium development goals international relations at the movies: teaching and learning about international politics through film learning by doing: using an online simulation game in an international relations course engaged student learning: high-impact strategies to enhance student achievement teaching international relations engagement through partnership: students as partners in learning and teaching in higher education undead pedagogy: how a zombie simulation can contribute to teaching international relations computers as mindtools for engaging learners in critical thinking the european union simulation: from problem-based learning (pbl) to student interest simulations as active assessment? typologizing by purpose and source simulating a senate office: the impact on student knowledge and attitudes the statecraft effect: assessment, attitudes, and academic honesty adapting the diplomacy board game concept for 21st century international relations teaching wicked problems forum: freedom of speech at colleges and universities the higher education manager's handbook bravo for brevity: using short paper assignments in international relations classes up to the mark: a study of the examination game. london: society for research into higher education how america lost faith in expertise: and why that's a giant problem the logic of practicality: a theory of practice of security communities teaching international relations through arts: some lessons learned can't get no (dis)satisfaction: the statecraft simulation's effect on student decision making assessment in simulations the power of dance: teaching international relations through contact improvisation thinking theory thoroughly: coherent approaches to an incoherent world the statecraft simulation and foreign policy attitudes among undergraduate students essay writing: a student's guide ir teaching reloaded: using films and simulations in the teaching of international relations the hidden curriculum teaching introduction to international politics with film getting the most from classroom simulations: strategies for maximizing learning outcomes publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. zwolski is an associate professor in international politics at the university of southampton, jean monnet chair of european security governance and senior fellow of the higher education academy. his teaching and research revolve around international relations theory and european security governance. key: cord-274774-klystgj4 authors: huang, naizhu; qiu, shaoping; alizadeh, amin; wu, hongchao title: how incivility and academic stress influence psychological health among college students: the moderating role of gratitude date: 2020-05-06 journal: int j environ res public health doi: 10.3390/ijerph17093237 sha: doc_id: 274774 cord_uid: klystgj4 many students suffer from academic stress and uncivil behaviors at colleges and there is a need to identify to what extent these negative phenomena might impact students’ mental health. the main purpose of this study is to examine the relationships between incivility, academic stress, and psychological health, as well as investigate the moderating role of gratitude. the study design of this research is cross-sectional. the final sample consisted of 895 university students in china; the statistical package for the social sciences (spss) version 22 was utilized to conduct statistical analysis. sample t-tests were used to examine whether there were gender differences in terms of four continuous variables: incivility, stress, gratitude, and psychological wellbeing. we also used multiple hierarchical linear regression analysis to test the relationships between the aforementioned four variables and the moderating effect of gratitude. the results of our study indicate that academic stress and incivility are positively associated with psychological distress, and gratitude moderates the relationship between incivility and psychological distress. however, no significant moderating effect of gratitude was found in the relationship between academic stress and psychological distress. the problems of psychological distress are prevalent across the globe and have been the major causes of disproportionately higher rates of disability and mortality [1] . as a set of painful mental and physical symptoms, psychological distress includes anxiety, confused emotions, hallucinations, and depression [2] . statistics on psychological distress are alarming; mental illness accounts for 30% of non-fatal disease burdens worldwide and 10% of overall disease burdens, including death and disability [3] . the proportion of the global population with depression in 2015 was estimated to be 4.4% [4] . the problems of psychological distress are especially prominent among college students, as they are particularly more vulnerable to psychological disorders [5] . for example, in china alone, about 20% of college students suffer from various forms of mental problems, such as depression, anxiety, and stress [6] . therefore, psychological distress of college students has become an issue of utmost concern worldwide [7] . in parallel, the topic of psychological distress has drawn much attention among researchers and practitioners worldwide. research has demonstrated that stress perceived as a personal threat and loss the study design of this research was cross-sectional. data were obtained from students in november 2019 at a large university in south china. since the survey was administered in chinese, and we used instruments originally developed in western countries, we conducted a back-translation to guarantee that the meaning of the instrument items was accurate and culturally appropriate. prior to conducting the survey, a pilot study was carried out using 40 students to assess clarity, length, comprehensiveness, and completion time of the measures. the survey instrument was distributed via the internal email listserv of the university to 19,532 students who came from all provinces of china. a total of 1425 students participated in this study and returned their responses. after excluding incomplete questionnaires, we obtained 895 valid, usable data cases with a response rate of 4.58%. therefore, the final sample consisted of 895 university students. among participants, 695 (77.7%) were female students. the average age was 19.14 years old. in total, 392 students (43.8%) were freshman while 352 (39.3%) were sophomores. in addition, 357 participants (39.9%) reported that their monthly family income was between 3000 to 5000 rmb yuan, while only 37 respondents reported their family earned 11,000 or above rmb yuan per month. most students reported that their father (40.6%) and mother (40.1%) only graduated from middle school. a vast majority of students (493, 55.1%) originally came from the province where the university is located. however, the remaining participants were roughly evenly distributed among other provinces of china except for three autonomous regions. all measures employed were developed and validated and used in previous studies. in addition to ensuring participating students' anonymity and confidentiality, we used a few negatively worded items in the survey to reduce common method variance [29] . incivility was assessed using a sevenitem scale developed by cortina et al. [30] . students were asked to rate the frequency they have been in a situation where any of their classmates exhibited uncivil behavior to them. the rating scale of 1 (never) to 5 (very frequent) was used. an example item was: "how often did my classmates make demeaning or derogatory remarks about me in the last year?". the reliability of this scale was acceptable with an alpha of 0.86. stress was measured using a 10-item scale developed by lepine et al. [31] . students were required to rate the extent to which they experienced academic stress on a scale of 1 (no stress) to the study design of this research was cross-sectional. data were obtained from students in november 2019 at a large university in south china. since the survey was administered in chinese, and we used instruments originally developed in western countries, we conducted a back-translation to guarantee that the meaning of the instrument items was accurate and culturally appropriate. prior to conducting the survey, a pilot study was carried out using 40 students to assess clarity, length, comprehensiveness, and completion time of the measures. the survey instrument was distributed via the internal email listserv of the university to 19,532 students who came from all provinces of china. a total of 1425 students participated in this study and returned their responses. after excluding incomplete questionnaires, we obtained 895 valid, usable data cases with a response rate of 4.58%. therefore, the final sample consisted of 895 university students. among participants, 695 (77.7%) were female students. the average age was 19.14 years old. in total, 392 students (43.8%) were freshman while 352 (39.3%) were sophomores. in addition, 357 participants (39.9%) reported that their monthly family income was between 3000 to 5000 rmb yuan, while only 37 respondents reported their family earned 11,000 or above rmb yuan per month. most students reported that their father (40.6%) and mother (40.1%) only graduated from middle school. a vast majority of students (493, 55.1%) originally came from the province where the university is located. however, the remaining participants were roughly evenly distributed among other provinces of china except for three autonomous regions. all measures employed were developed and validated and used in previous studies. in addition to ensuring participating students' anonymity and confidentiality, we used a few negatively worded items in the survey to reduce common method variance [29] . incivility was assessed using a seven-item scale developed by cortina et al. [30] . students were asked to rate the frequency they have been in a situation where any of their classmates exhibited uncivil behavior to them. the rating scale of 1 (never) to 5 (very frequent) was used. an example item was: "how often did my classmates make demeaning or derogatory remarks about me in the last year?". the reliability of this scale was acceptable with an alpha of 0.86. stress was measured using a 10-item scale developed by lepine et al. [31] . students were required to rate the extent to which they experienced academic stress on a scale of 1 (no stress) to 5 (a great deal of stress). two example items were: "the amount of hassles i need to go through to get projects or assignments done" and "the difficulty of the work required in my classes" cronbach's alpha was acceptable at 0.89. gratitude was evaluated using the gratitude questionnaire-six item form (gq-6) developed by mccullough, emmons, and tsang [32] . participants rated the extent to which they agreed with statements on a scale ranging from 1 (strongly disagree) to 5 (strongly agree). one example item was: "i have so much in life to be thankful for" the measure had an adequate internal consistency (α = 0.83). psychological distress was measured using k6 screening scale developed by kessler et al. [33] . students rated on a scale of 1 (never) to 5 (very often) how often they felt psychologically distressed, such as hopeless. the reliability of this scale was 0.88. control variables included age, gender, and family income. these variables were found to predict psychological distress in previous research [33] [34] [35] . for gender, we coded male as "1" and female as "2". in terms of age, we coded age 18 as "1", age 19 as "2", age 20 as "3", age 21 as "4", age 22 as "5", and age 23 or above as "6". with respect to family income, we coded family monthly income below 3000 rmb yuan as "1", 3000-5000 as "2", 5000-7000 as "3", 7000-9000 as "4", and above 9000 as "5". spss version 22 was utilized to conduct statistical analysis. mplus was used to test convergent and discriminant validities of the main variables. independent sample t-tests were used to examine whether there were gender differences in terms of the four continuous variables: incivility, stress, gratitude, and psychological wellbeing. we also used multiple hierarchical linear regression analysis to test relationships between the aforementioned four variables and the moderating effect of gratitude, controlling for age, gender, grade, and household income. the p-values ≤0.05 were considered throughout as statistically significant. we used harman's single factor test to check whether there was common method variance in the data. results revealed that one single factor only explained 26.86% of the variance, much lower than 50%, indicating no major issues with common method variance. means, standard deviations, reliability, and intercorrelations between study variables are shown in table 1 . as can be seen from table 1 , gender is associated with gratitude (r = 0.15, p < 0.01). however, it has no significant relationship with incivility, stress, and psychological wellbeing. psychological distress is significantly related to incivility (r = 0.37, p < 0.01), stress (r = 0.52, p < 0.01), and gratitude (r = −0.20, p < 0.01), whereas correlation between stress and gratitude is not statistically significant. confirmatory factor analysis (cfa) was performed to test convergent and discriminant validities of main variables (i.e., incivility, academic stress, gratitude, and psychological distress). for this purpose, we compared four measurement models. in the three-factor model, we combined incivility and academic stress because they were two independent variables. in the two-factor model, incivility, academic stress, and gratitude were put together as one variable. the fit indices of all four models are shown in table 2 . as can be seen from this table, the four-factor model provided a good fit with the data and was much better than any other models (χ 2 = 502.17, df = 203, rmsea = 0.04, cfi = 0.96, tli = 0.96, srmr = 0.04). thus, the discriminant validity was established. in addition, the factor loadings in the four-factor model were all greater than 0.50 and all values of average variance extracted (ave) for the four variables were also greater than 0.50. therefore, convergent validities were achieved for all four variables. note: n = incivility; st = academic stress; gr = gratitude; ps = psychological distress. ** p < 0.01. ∆ χ 2 is χ 2 difference between respective and four-factor models. results from independent sample t-tests revealed that there were no statistically significant differences between male and female students in terms of incivility, stress, and psychological wellbeing. however, discrepancies between male and female students were significant, with female participants reporting more gratitude towards others (t = −4.43, p < 0.001). hierarchical multiple regression analysis was performed to examine the relationships between incivility, stress, gratitude, and psychological wellbeing, as well as the moderating effect of gratitude. we mean-centered the values of incivility, stress, and gratitude [36] . results are presented in table 3 . as seen from this table, there was a significant main effect of incivility in both model 2 (β = 0.22, p < 0.01) and model 3 (β = 0.22, p < 0.01), indicating that students feel more psychologically unhealthy under high-level incivility from their classmates. we also found that a significant effect of stress in both model 2 (β = 0.46, p < 0.01) and model 3 (β = 0.46, p < 0.01) revealed that students experience more psychological distress issues under high academic stress. in addition, the effect of gratitude on psychological distress was also negatively significant in both model 2 (β = −0.18, p < 0.01) and model 3 (β = −0.17, p < 0.01), indicating that college students who express more appreciation to others suffer less from psychological distress problems. most importantly, the interaction term between incivility and gratitude was significant in model 3 (β = 0.07, p < 0.05). however, there was no significant moderating effect of gratitude on the relationship between incivility and psychological wellbeing. following the guidelines of aiken and west [36] , we plotted the regression of psychological distress on incivility to assess the moderation effect at two values of gratitude (mean +1 standard deviation and mean −1 standard deviation) ( figure 2 ). as illustrated in this figure, the higher the level of gratitude, the stronger the relationship between incivility and psychological wellbeing. when incivility is low, students with a low level of gratitude experience more severe psychological distress problems. however, as the level of incivility becomes higher, psychological distress issues become less different across these two gratitude groups. following the guidelines of aiken and west [36] , we plotted the regression of psychological distress on incivility to assess the moderation effect at two values of gratitude (mean +1 standard deviation and mean -1 standard deviation) ( figure 2 ). as illustrated in this figure, the higher the level of gratitude, the stronger the relationship between incivility and psychological wellbeing. when incivility is low, students with a low level of gratitude experience more severe psychological distress problems. however, as the level of incivility becomes higher, psychological distress issues become less different across these two gratitude groups. using a sample of 895 students recruited from a university in south china, we investigated the relationships between incivility, academic stress, gratitude, and psychological distress. in addition, using a sample of 895 students recruited from a university in south china, we investigated the relationships between incivility, academic stress, gratitude, and psychological distress. in addition, we examined underlying mechanisms through which incivility and academic stress affect students' psychological distress. that is, we tested how incivility and academic stress interact with gratitude to predict students' psychological distress. moreover, we compared differences between male and female students with respect to incivility, academic stress, gratitude, and psychological distress. results of this study illustrate that incivility is positively associated with psychological distress. our findings are consistent with prior studies [18, 19, 37] . academic stress was also confirmed as related to psychological distress. this conclusion is also in line with previous research that examined the aforementioned relationship [11, 12] . it was also identified that gratitude moderates the relationship between incivility and psychological distress. however, no significant moderating effect of gratitude was found in the relationship between academic stress and psychological distress, which is contrary to our expectations. in low incivility environments, grateful students are less likely to suffer from psychological distress than those with low levels of gratitude. it is sensible because under normal conditions, grateful individuals tend to hold a positive attitude towards life, take more pleasure from benefits in life, and feel happier [38] . as uncivil behaviors or comments become relatively more frequent, grateful students can still keep psychologically healthier. however, as incivility frequency increases, the psychological distress college students experience increases more rapidly for grateful students. a possible reason would be that grateful individuals take uncivil communications more seriously and attempt to resolve conflicts to maintain high-quality interpersonal relationships with others [39] . most probably they attribute sources of these uncivil behaviors and comments to themselves and ruminate on how they could change their thoughts, thus aggravating their psychological problems [40] . in this study, we did not find any gender differences on perceived incivility, perceived level of stress, and reported psychological health. the finding of no gender disparity on perceived stress and psychological distress contradicts the study results of moksnes and lazarewicz [28] . in their research on norwegian adolescents from 13 to 18 years old, they found that boys scored lower than girls on stress and symptoms of depression and anxiety. these inconsistencies may be attributed to the difference of age range. another plausible reason could be the emphasis chinese universities place equally on both male and female students. in our study, the sample participants were chinese college students who were developing or built their own resilience and coping strategies. in chinese universities, there are counselors at both university and department levels dedicated to helping students with personal, emotional, and psychological concerns. when facing stress and adversity, both male and female students are likely to be equipped with the same skills to handle negative situations encountered and to perceive the stress as less severe. however, female college students, compared to male students, were found to be more likely to express gratitude towards others. this finding corresponds with previous studies showing that women tend to report higher levels of gratitude than men [41] [42] [43] . as reasoned by watkins et al. [44] , men generally associate gratitude with weakness in personality. therefore, men tend to avoid expressions of gratitude to protect their masculinity and maintain their social status. this study makes both theoretical and practical contributions. first, research examining the association between incivility, academic stress, and psychological distress in the chinese context is scarce. using chinese college students as a study sample is even more scant. this study adds to the current literature by enhancing our understanding of whether incivility and academic stress influence students' psychological distress in a chinese university setting. additionally, given that little is known about how incivility and stress affect psychological health, testing the moderating effects of gratitude helps us gain an understanding of boundary conditions under which such an association might occur. therefore, this study might fill a theoretical gap in the literature. third, findings of this study could provide insight and timely advice to chinese university students on how to keep psychologically healthy. especially during this difficult time of novel coronavirus outbreak, students are learning online from home. the study of gratitude and stress might offer some useful guidelines to develop positive psychology-based student counseling interventions to help chinese students cope with adversity and hardship. although we used a relatively large sample size and well-validated instrument scales, this study has some limitations. first, the data used in this study were obtained from a single source (i.e., university students). we reduced common method variance by using some negatively-worded items, ensuring anonymity and confidentiality. however, there is still a potential for common method variance to bias our study results. future studies could adopt as many measures as possible, as recommended by podsakoff et al. [29] , to further minimize this issue. second, this study was cross-sectional in nature, which precludes us from making a causal conclusion about the main and moderation effects. if possible, a longitudinal study or experimental design is recommended to interpret the relationships between incivility, academic stress, and psychological distress in a causal way. third, despite the large sample size in this study, most participants came from the province where the university is located and no students from the three autonomous regions participated in this study. this sample distribution, together with a low response rate (4.58%), might bias the study results. future studies could collect more data from other provinces in order to be more representative of the whole chinese student population. next, we only solicited data from one university in china. although students were from all parts of china, the sample may not represent the whole college student population in china. finally, we only used gratitude as a moderator to examine how incivility and academic stress impact students' psychological health, ignoring other possible mediators and moderators. if more variables were examined in the study, such as rumination, personality traits, and students' attribution, we could gain a higher understanding of the underlying mechanisms and boundary conditions about how, whether, and when such effects might be most likely to occur. this study showed that both incivility and academic stress positively affect university students' psychological health. it also demonstrated that gratitude moderates the relationship between incivility and psychological distress after controlling for age, gender, and family income. however, the interaction of gratitude and academic stress does not significantly impact university students' psychological health. for highly grateful students, the relationship between incivility and psychological distress is stronger than those with low levels of gratitude. in addition, female students scored higher on gratitude than male students, whereas there were no differences between these two groups on perceived incivility, perceived academic pressure, and psychological distress. this study contributed insight into the moderating role of gratitude in the incivility-psychological distress relationship. to advance our understanding, future researchers could use more measures to combat common method bias, employ a longitudinal study or experimental design to create causal interpretation, and recruit more representative samples for the study results to be generalized. incivility is a rude or impolite attitude or behavior towards others. considering the vagueness and prevalence of incivility, maintaining civility on campus still remains a great concern for most college administers [45] . incivility interferes with a harmonious and cooperative learning atmosphere, contributing to increased psychological distress among college students. given its widespread effect on both students and college culture, colleges and universities should take measures to tackle this disturbing issue. at the college or university level, administers should create a culture in which each and every student is treated with respect, fairness, and equality. at the department level, college counseling staff and department faculty should make it clear what behaviors students need to follow and what should be avoided. in addition, they should provide counseling to help change students' behaviors. furthermore, students as individuals also need to understand their own roles and assume corresponding responsibilities. they must stand up against any uncivilized behaviors occurring on campus in order to stop such behaviors. in this way, we can facilitate civility and enhance learning effectiveness among college students. psychometric study of depression, anxiety and stress among university students psychological distress: concept analysis making mental health a global priority depression and other common mental disorders: global health estimates; world health organization the prevalence and socio-demographic correlations of depression, anxiety and stress among a group of university students changes in undergraduate students' psychological well-being as they experience university life impacts of socio-cultural environment and lifestyle factors on the psychological health of university students in bangladesh: a longitudinal study qualitative and quantitative analyses of psycho-logical distress: methodological complementarity and ontological incommensurability conservation of resources: a new attempt at conceptualizing stress correlations among work stressors, work stress responses, and subjective well-being of civil servants: empirical evidence from china a meta-analytic test of the challenge stressor-hindrance stressor framework: an explanation for inconsistent relationships among stress and performance the relationship between challenge and hindrance stressors and emotional exhaustion: the moderating role of perceived servant leadership stress at college: effects on health habits, health status and self-esteem college students: mental health problems and treatment considerations how rude! emotional labor as a mediator between customer incivility and employee outcomes uncivil communication in everyday life: a response to benson's "the rhetoric of civility customer incivility and employee well-being: testing the moderating effects of meaning, perspective taking and transformational leadership daily cyber incivility and distress: the moderating roles of resources at work and home an integrative review examining student-to-student incivility and effective strategies to address incivility in nursing education gratitude as a psychotherapeutic intervention is gratitude always beneficial to interpersonal relationships? the interplay of grateful disposition, grateful mood, and grateful expression among married couples an attributional theory of achievement motivation and emotion gratitude works: a twenty-one-day program for creating emotional prosperity religious/spiritual struggles and life satisfaction among young roman catholics: the mediating role of gratitude gratitude moderates the mediating effect of deliberate rumination on the relationship between intrusive rumination and posttraumatic growth perceived psychological well-being among university students: a comparative study by gender the association between stress, resilience, and emotional symptoms in norwegian adolescents from 13 to 18 years old sources of method bias in social science research and recommendations on how to control it incivility in the workplace: incidence and impact challenge and hindrance stress: relationships with exhaustion, motivation to learn, and learning performance the grateful disposition: a conceptual and empirical topography screening for serious mental illness in the general population with the k6 screening scale: results from the who world mental health (wmh) survey initiative perceived health locus of control, self-esteem, and its relations to psychological well-being status in iranian students psychological well-being (pwb) of school adolescents aged 12-18 yr, its correlation with general levels of physical activity (pa) and socio-demographic factors in multiple regression: testing and interpreting interactions linking work-family interference, workplace incivility, gender and psychological distress gratitude and happiness: development of a measure of gratitude, and relationships with subjective well-being gratitude and well-being: a review and theoretical integration ruminative coping with depressed mood following loss gratitude and subjective well-being in early adolescence: examining gender differences what are children thankful for? an archival analysis of gratitude before and after the attacks of general feelings of gratitude, gratitude to god, and hemoglobin a1c: exploring variations by gender the debt of gratitude: dissociating gratitude and indebtedness the level of student incivility: the need of a policy to regulate college student civility this article is an open access article distributed under the terms and conditions of the creative commons attribution (cc by) license we would like to thank the students at a chinese university for their participation in this study. the study received no funding from any institutions. the authors declare no conflict of interest. key: cord-167104-snvq1ol7 authors: castano, adriana mejia; hernandez, javier e; llanos, angie mendez title: kids today: remote education in the time of covid-19 date: 2020-10-14 journal: nan doi: nan sha: doc_id: 167104 cord_uid: snvq1ol7 with the recent covid-19 breakup, it became necessary to implement remote classes in schools and universities to safeguard health and life. however, many students (teachers and parents, also) face great difficulties accessing and staying in class due to technology limitations, affecting their education. using several nationally representative datasets in colombia, this article documents how the academic performance of students in their final high school year is affected due to technologies, aggregated by municipalities. we conclude that internet access strongly affects these results, and little improvement on the internet/computer access will reflect better academic performance. under these conditions, belonging to an ethnic group or high rurality (non-geographic centralized municipalities) has a negative impact. policy implications are discussed. in particular, these implementations could increment the gap in access to quality education in low-income or geographically remote populations, which keeps countries from achieving sustainable development goals from the united nation development programme (zhao, lu, huang & wang, 2010) . despite the need to establish these remote programs, there is little empirical evidence on how technologies in the last years affect their performance (zhao, lu, huang & wang, 2010) and what is the effect of its absence. this is a critical question given that in pandemic times almost all academic activities require internet access and all economic activities resulted affected (presented reductions) due to the pandemic (chetty, friedman, hendren & stepner, 2020) . in colombia, data sets like those provided by the icfes (colombian institute for the evaluation of education) have long comparisons over time about the academic achievement of students in all stages. most of the studies over these data indicate that the covariables that most affect the academic performance (have a good score), according to chica, galvis, and ramirez (2011) are socioeconomic status, parents scholarship, the number of hours in the school, school type (private or public) and gender; but there is little work on how technologies affect it. the current article aims to fill this gap. using nationally representative samples of children in their final high school year from 2014 to 2019, this article addresses two related research questions: ! which municipalities have the most difficulty in implementing the remote academic continuity program? ! what should be the municipalities' prioritization (in each state) for information technology programs investment? we decide to proceed using three classification algorithms and compare them to predict academic performance. solving these questions will help decisionmakers focus on certain geographical areas to develop specific intervention programs to improve the learning experience and results of high school and middle school students. background. in this era of high internet connectivity, which brings positive and negative impacts in our lives, many facets are linked to its access; and education is not an exception. since the early 2000s, it has been noticed (pereira, pleguezuelos, meri, 2007; greenhow, robelia, hughes, 2009 ) that there were necessary changes to take advantage of the online resources. parents and teachers know the importance and necessity of computer usage and internet access in all stages of academic formation (bassok, latham, 2017) . in kindergarten students in the united states, bassok, and latham (2017) , found that computer access increases academic children skills. the results of amorim, jeon, abel, felisberto, barbosa, and martins (2020) , in brazil, show that technologies, specifically using games, improves reading scores. in high school students, specifically in china, the relation between internet self-efficacy, internet accessibility, behavior, academic performance between others was investigated (zhao, lu, huang & wang, 2010; penaranda, aragon-muriel, micolta, 2014) . there is strong evidence that digital inequality exists on the internet, and students with internet access at home had a better academic performance. in the united states, rickles, heppen, allensworth, sorensen, and walters (2018) , found no statistically significant differences in longer-term outcomes between students taking online and face-to-face courses; this could indicate that online courses have the same benefits in some populations (a majority of hispanic and african american students). in college students, junco and cotten (2012) , and sazili and khafidhah (2017) found a negative correlation between academic performance and internet usage, and a weak relationship with computer ownership. in university students, sushma, draus, goreva, leone, and caputo (2014) pointed out that time spent on the internet could be a measure of academic achievement under certain conditions, however, some online media could have a negative effect on university students (junco, 2012; asemah, okpanachi & edegoh, 2013) . according to paulsen and mccormick (2020) , student engagement is necessary to have positive benefits. in countries like malaysia and taiwan, the use of the internet is apparently a distractor. indeed, the results of xu, wang, peng, and wu (2019) indicate that discipline plays a vital role in the academic success of undergraduate students; in particular, the results show that the use of internet data could differentiate and predict student academic performance. therefore, the evidence suggests that in any stage of learning, technology strongly affects academic performance across the world (donald, foehr, 2008) and it is most notorious in pandemic times since all economic activities were seriously affected (chetty, friedman, hendren & stepner, 2020) and internet access became a necessity (chiou and tucker, 2020) . going deeper, it looks like there is a way to predict academic performance based on some covariables (hellas et.al, 2018) , and technologies could be one of them. in colombia there is an academic test provided by icfes, proctors on standardized tests, called saber11, that scores students in their final high school year, and also has self-reported socio-demographic information. analyzing data from 2009, chica, galvis, and ramirez (2011) found that high school graduates who own a computer are more likely to obtain a high grade on saber11, although internet possession was not statistically significant. however, analyzing more recent data, internet access became an interest covariable (see appendix). in particular, family-related variables are the best to predict academic performance (garcia-gonzalez, skryta, 2019). the dataset from icfes also includes information of undergraduate students, and an analysis of academic efficiency about these students shows that social covariables play an important role (rojas 2019; delahoz-dominguez, zuluaga, fontalvo-herrera, 2020). during pandemic times it is necessary to understand how the internet or computer access affects the score of saber11 since that can offer a way to comprehend the effect of these technologies in each municipality and which are the best improvements according to each necessity. the actual existing evidence, while limited, does support the notion that internet/computer access strongly affects the academic performance of saber11. the findings suggest that, as we notice in literature, technologies strongly affect academic performance. however, in this scenario, there are two covariables that need to be in consideration: rurality and belonging to an ethnic group. then upgrades in technology could improve academic performance, but it is not enough: teachers, parents, and the social environment play an important role. there are necessary policies about technologies and, like zhao, lu, huang, and wang (2010) and sazili and khafidhah (2017) pointed out, about increasing learning skills in remote environments. data. saber11 tests have been modified during the last 20 years, we have chosen the period from 2014 onwards since it has the same standardized content, taking tests on 5 different subjects: critical reading, citizenship skill, english, written communication, and quantitative reasoning; score from 0 to 100. their sum gives a global score from 0 to 500 (icfes, 2020) . this is the independent variable that we consider in this study. this data also has socio-demographic information such as internet accessibility, computer, family income, habitability conditions, belonging to an ethnic group, among others. there are 8 municipalities that in this period did not take the saber11 test, which are not included in this work. this data was aggregated by municipality into numerical covariables. given that data about internet ownership is self-reported by students, we also decided to use the data reported by the ministry of information technologies and communications mintic, about the total number of internet subscribers, the type of subscription and technology, indicators of internet reach by region, information on internet providers by demand and participation statistics (mintic, 2020). additionally, there is relevant information by region reported by the dane (national administrative department of statistics), including accurate population values (2018 census). with this information it is possible to know how rural a municipality is, to compare if this rurality is correlated with scores, internet access, and other important covariables (dane, 2020). in the appendix, we collect descriptive statistics from a consolidated data frame aggregate by municipalities. from this exploratory data analysis and since we need covariables that allows governmental investments, we conclude that the main numerical covariables to use in this study are: • code: numeric code that identifies the municipality, • year: year of the proctored test, • internet: percentage of students with self-reported internet, • computer: percentage of students with self-reported computers, • ethnic: percentage of students self-reported as belonging to an ethnic group, • school: percentage of public schools, • global_score: student global score average on saber11, • population: censused population during the test year, • connectivity: access to the internet per 1000 inhabitants, • rural_index: percentage of the population that lives in rural sectors. a first finding of the exploratory data analysis, see appendix, is that if the family owns a computer, the average score is higher but rural index decreases, coinciding with the findings of chica, galvis, and ramirez (2011) . the results showed that scores have a mainly rural composition. comparing this with geographical location (figure 1) , we obtain that good scores are geographically centralized, there are many families without the internet, and it looks like these two variables are correlated. in this kind of study, the variables are commonly analyzed via correlation (sazili & khafidhah, 2017; sushma et.al., 2014) . in figure 2 classification. there are many algorithms, from linear and logistic regressions to neural networks, to help identify and classify covariables/results. however, we wanted to choose interpretative models, this is, models that classify but also give us a statistically significant set of covariables. in the literature, it is common to use linear or logistic regression (bassok & latham, 2017 in particular, in the work of xu, wang, peng, and wu (2019), they have covariables as online duration, internet traffic volume, connection frequency, among others. and using decision trees, neural networks, and support vector machines predict academic performance from these features. in this work, we want to do something similar using another set of covariables with logistic regression, decision forest, and regression forest. therefore, the initial model is a logistic regression given that this model shows statistically significant covariables and how they affect (positive or negative) the independent variable. going deeper into the list of models, we found the random forests, which is an ensemble learning method for classification, regression, and other tasks that operates by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees. random decision forests correct for decision trees' habit of overfitting on the training set. we construct a classification algorithm over the municipalities considering the initial set of covariables, which gives a list of municipalities at different vulnerability levels and a few actionable features on which governments (national, state, and/or municipal) can act upon. to use these algorithms, we need a risk threshold that we define for each year as the global score average minus k times the global score standard deviation (where k can be any real number between 0 and 2; for optimal purposes we choose k=1). also, we split the data frame into training data (from 2014 to 2018) and validation data (2019), to decide about the accuracy. the main results of the logistic regression are: • considering the p-value for each covariable (less than 0.05), we conclude that the relevant variables are internet, computer, ethnic, connectivity, and rural index. • ethnic and internet have positive coefficients which imply these covariables have a positive impact on the global score. • computer and rural index have negative coefficients which imply these covariables have a negative impact on the global score. with this new set of covariables, we perform a regression random forest with depth m (with m>2; we choose m=3) and a classifier random forest of depth l. we compare the three models using the auc, see table 1 (area under the curve roc): the roc curve is a performance measurement for classification problems at various thresholds settings. roc is a probability curve and auc represents a degree or measure of separability. it tells how much a model is capable of distinguishing between classes. the higher the auc, the better the model is at predicting 0's as 0's and 1's as 1's. the choice of parameters k, m, l was made looking for auc between 0.7 and 0.87 since lower auc implies that the model is underfitting and bigger implies that it is overfitting. notice that logistic regression, according to auc, is the best classificatory up to these conditions. if we increase the depth to any random forest, we get greater auc, table 1 . but the complexity of the algorithm increases. each one predicts over each municipality if it will be at risk or not, that is, if the average global score of the municipality is greater or not than the risk threshold. to compare and use at the same time the results of the three algorithms, we construct a vulnerability level called total_risk as the sum of the predicted risk of these three models. this analysis allows to define 4 levels of vulnerability: in figure 3 , we can see a map of colombia with all levels of total_risk. to determine how good the estimations are, we can compute a confusion matrix, which is a specific table layout that allows visualization of the performance of the algorithm. each column of the matrix represents the instances in a predicted vulnerability while each row represents the instances in actual risk, and determines if the algorithm is confusing classes. in this case, the first row of the matrix is (914, 21, 5, 8) and the second row is (83, 14, 9, 58) . for example, the last column (8, 58) implies that 66 municipalities are at serious vulnerability (8 of them are false positives, that is, 8 municipalities are labeled as 'at vulnerability' but they are not). we notice that the quantity of false positives and false negatives (83) is little; therefore, the predictions are accurate. beyond classification. in figure 3 , we also notice some interesting municipalities in the center of the country, in high vulnerability, contradicting the commonly believed that remote locations are always vulnerable. this implies that not necessarily being surrounded by no vulnerability affects completely this covariable. one of the findings was coyaima in tolima, in the center of the country. in tolima, during the last years, the percentage of the population belonging to an ethnic group is near to 5%, and the percentage of rurality is around 50%, but these zones can be a reference to improve on their neighbors. listing states with the count of municipalities in each vulnerability we found that vaupés, chocó, guainía, and amazonas are states that need serious intervention as soon as possible since almost 67% of its municipalities are in serious vulnerability. running a bonferroni to understand each vulnerability level, we notice that low and no vulnerability have significant differences in self-reported internet, computer ownership, belonging to an ethnic group, and connectivity. low and medium vulnerability in connectivity; and medium and serious vulnerability in ethnic group and connectivity. since connectivity and belonging to an ethnic group are common variables, in table 2 , we analyze the average number of connections per 1000 habitants and percentage of students belonging to an ethnic group, in each vulnerability level, obtaining that municipalities at serious vulnerability have a high percentage of students belonging to an ethnic group, opposite to no vulnerability municipalities. with covid isolation measures banning students from classrooms, most learning has moved online, but even those pupils who can successfully connect are likely to fall behind if not possessed of the self-direction and motivation needed for remote education. moreover, research has shown that poorer students perform worse in online courses (chetty, friedman, hendren & stepner, 2020) than face-to-face ones. our analysis found that the most relevant covariables related to an increased academic vulnerability in colombia, related to technologies, are connectivity per 1000 inhabitants (as a measure of reliable broadband internet access) and belonging to an ethnic group (associated with low income). amazonas, vaupés, guainía, and chocó need serious intervention, given the high values shown for the vulnerability factors measured. geographic centralization (associated with increased urbanization, as measured by the rural index), is also a strong predictor of better scores for students. we found some centralized municipalities with high vulnerability as coyaima in tolima and altos del rosario in bolívar, surrounded by municipalities of zero vulnerability. in particular, these municipalities have low access to the internet and computer and high rurality and percentage of belonging to an ethnic group, showing an interesting feature of colombia. according to hartley and bendixen (2010) , improving the academic results and reducing the large gap in academic achievement requires the interventions to focus on maintaining the students' interactions (among themselves and with their teachers), also learning skills in the students are necessary as selfregulatory skills, epistemological beliefs, motivation, self-efficacy, ability, physical challenges, and learning disabilities. for this scenario to be realized, a series of factors have to line up: schools need to have the resources to implement remote learning, students need to have access to computers and reliable internet connections, and parents need to have the ability, time, energy, and patience to turn into home-school instructors. a concerted effort between the state government and parents seems to be the most effective strategy. complementary projects might include: • optimizing accessible solutions to mobile devices given that students have some access to them. • offering limited data plans (access granted only to academic sites, to avoid misuse). • focused policies on the rural ethnic minorities, the communities with higher vulnerability. • lending computers to families. • delivering financial stimulus to the municipalities most at vulnerability, conditional on the improvement of academic results compared with the previous year. • support to parents so they can step up as temporary teachers. the challenges are related to providing students at vulnerability the necessary tools for success, not only laptops and reliable broadband internet access but also to motivate parents to help their children succeed academically. cundinamarca has good scores in almost every municipality, we assume that this happens because bogotá (the country capital) is in this state. we now analyze the levels of vulnerability predicted by our algorithm, remember that the choice of the threshold was made to detect the municipalities most at risk. we found that 90% of municipalities in the country obtain no risk. this could be an indicator that, for future works, the threshold risk could be the average in every year. at low risk is 1%, in medium risk 3%, and in high risk 6%. as mentioned before, the municipalities in high vulnerability are mainly in remote locations, the percentage of computers across time is stable, the percentage of internet access has increased since 2017, but the percentage of students belonging to an ethnic group and rurality are high (near 70% using escribo play video games to improve phonological awareness, early reading, and writing in preschool influence of social media on the academic performance of the undergraduate students of kids today: the rise in children's academic skills at kindergarten entry how did covid-19 and stabilization policies affect spend and employment? a new real-time economic tracker based on private sector data determinantes del rendimiento académico en colombia: pruebas icfes saber11 social distancing, internet access and inequality censo nacional de población y vivienda dataset of academic performance evolution for engineering students". data in brief trends in media use predicting academic performance based on students' family environment: evidence for colombia using classification trees learning, teaching, and scholarship in a digital age: web 2.0 and classroom research: what path should we take now? educational research in the internet age: examining the role of individual characteristics predicting academic performance: a systematic literature review transparencia y acceso a la información pública, resultados pruebas de estado en colombia no a 4 u: the relationship between multitasking and academic performance in-class multitasking and academic performance automatically measuring question authenticity in real-world classrooms internet dedicado, accesos fijos a nivel nacional reassessing disparities in online learner student engagement in higher education description and preliminary evaluation of a program for improving chemistry learning in high school students effectiveness of using blended learning strategies for teaching and learning human anatomy online credit recovery and the path to on-time high school graduation does stem stand out? examining racial/ethnic gaps in persistence across postsecondary fields class relations in the higher educational system and its effects in academic performance: the case of bogota the influence of internet usage on student's academic performance the impact of internet addiction on university students and its effect on subsequent academic success: a survey based study internet inequality: the relationship between high school students' internet use in different locations and their internet self-efficacy prediction of academic performance associated with internet usage behaviors using machine learning algorithms is an adjoint professor of mathematics and statistics at universidad del norte, km 5 via puerto colombia, barranquilla -colombia key: cord-260907-uuaa9ta2 authors: schaffir, jonathan; strafford, katherine; worly, brett; traugott, amber title: challenges to medical education on surgical services during the covid-19 pandemic date: 2020-08-28 journal: med sci educ doi: 10.1007/s40670-020-01072-2 sha: doc_id: 260907 cord_uid: uuaa9ta2 nan the arrival of the covid-19 pandemic to the usa in march of 2020 had a profound and unforeseen effect on education of american medical students. students and faculty in the middle of academic programs were given little advance notice that learning would be disrupted. as the epidemic spread, concerns grew for medical systems becoming overwhelmed with new cases and needing to conserve personal protective equipment (ppe). medical schools moved to cancel classes and restrict students from participating in clinical activities or even setting foot in their affiliated medical centers. with recommendations for social distancing and state-ordered mandates to stay at home, students were additionally forbidden from getting together in groups and had to remain physically separated from anyone other than immediate family. previous disruptions in medical education have been described in the literature, generally in the forms of natural disasters such as floods and earthquakes [1] [2] [3] . unlike these events, several factors made the covid-19 pandemic unprecedented. the disruption was on a global scale-not just affecting one medical school or one community, but institutions across the country and around the world. solutions that helped to circumvent the effects of financial failures and natural disasters, such as the distribution of students to other institutions or programs would not apply in this case [1] [2] [3] [4] . the length of the disruption is likely to be greater than that for other examples as well. floods recede and earthquakes cease, with recovery linked to the time it takes to rebuild and redevelop infrastructure. in the case of the current pandemic, effects on student participation will likely last until the threat of infection abates, which may not occur until a reliable and effective vaccination program is in place, an intervention predicted to require a year or more. for other disasters, there was little or no risk to students beyond the initial insult; students suffered from emotional and financial trauma following natural disasters but did not remain at risk of physical harm. in those cases, students were often enlisted to help with recovery efforts and participate in clinical care, giving them a sense of purpose and a useful hands-on education. in the pandemic scenario, students are isolated and restricted from participation due to health concerns and the need to preserve resources for more extensively trained providers. nevertheless, some lessons learned from responses to natural disasters have been applicable to the pandemic situation. following hurricane katrina, the interruption in medical education necessitated some acceleration of the educational process, with a reevaluation of what requirements were necessary to graduate with an md degree [5] . the pandemic situation is forcing educators to take a hard look at what the minimum requirements should be to complete a field of medical study. medical educators and administrators are asked to make a difficult ethical decision, where the education and safety of trainees are being weighed against the increasing demands of medical personnel needed in high-risk environments. these situations also demonstrate the potential psychological trauma on students and emphasize the need for adequate student counseling and mental health resources [6] . regarding educational content, all of these situations have prompted educators to look for alternative means of instruction outside of the classroom. introduction of remote learning with televised lectures, online learning modules, and interactive online discussions have become important solutions to the lack of available time spent within institutional walls. televised and electronic learning is most appropriate when replacing didactic content, whether related to basic sciences or clinical teaching. however, for rotations that involve large numbers of procedures and hands-on learning, such solutions may prove inadequate. clinical clerkships in surgery, obstetrics and gynecology, and anesthesiology require students to participate in surgeries and procedures to learn technical skills, and then demonstrate such skills to assess competence. in addition to learning the skills required for competence as a physician, students also learn vital lessons about communication and function in a surgical team within these arenas. exposure to the surgical or obstetrical atmosphere is often what guides a student into or away from choosing these specialties as a medical career [7] . unfortunately, these experiences are not as easily replaced by electronic learning as are other clinical arenas in medical education. the remainder of this paper focuses on the potential solutions and barriers to providing adequate undergraduate medical education in procedure-oriented fields in the setting of the covid-19 pandemic. many of the challenges of teaching surgery and obstetrics/ gynecology content have been successfully met with online educational platforms. a central learning objective for students in these fields is gaining the clinical judgment to weigh the relative merits of medical and surgical interventions. working through clinical scenarios of patients with potentially surgical conditions is a tried and true method for mastering this objective [8, 9] . in the absence of having such patients physically present, it is useful to work through constructed cases that describe such scenarios. several online services are available that offer these cases and give students a chance to interactively work through their management. onlinemeded, geeky medics, oxford medical simulation, and aquifer are platforms that walk students through clinical scenarios that require them to decide when surgery is necessary, for example, in the setting of appendicitis or ectopic pregnancy (table 1) . other available resources include case vignettes prepared by professional organizations like the association of professors of gynecology and obstetrics (apgo) [10] and the association for surgical education (ase) [11] (table 1 ). these can be utilized via remote faculty-led sessions to guide students through surgical decision-making similar to case conferences that more typically happen on clinical rounds. a greater educational challenge lies in reproducing the experience of assisting in surgical procedures. part of the experience lies in witnessing the procedures themselves and learning the relevant techniques and anatomy. there are many excellent examples of online surgeries on internet video platforms such as youtube and vimeo that display recordings of surgical cases. they range from demonstrations of classical open approaches to cutting-edge techniques that employ the newest technologies available. these latter examples are often geared towards practicing physicians seeking to expand their repertoire of surgical skills and are not applicable to students learning basic surgical concepts. educators looking to provide examples of surgical videos to their learners need to curate these collections with the knowledge base of the learner in mind [12, 13] . although many videos are narrated to edify their audience, the content and expertise of narration varies greatly. for some, the intent is to point out impressive pathological findings or to highlight potential surgical complications. others, more relevant to the undergraduate medical experience, concentrate on basic anatomy and rationale for surgical techniques. even when the perfect video is found that exposes learners to an appropriate procedure with narration at an appropriate learning level, asking a student to watch it alone can still be isolating. excluding the ability to interact with the surgeon and ask questions diverges substantially from the usual experience of being in the operating room. a potential solution is to enlist an interactive online platform to create an electronic classroom where the experience can be shared. a surgical faculty member can share his/her screen with a group of students as he/she describes the video being viewed. the video may be one found online or one recorded by the surgeon. providing real-time narration ensures that the content matches the faculty's objectives and is given at the correct level of complexity. it also provides a point of contact with a faculty member who may be a resource for future questions about the material or about career exploration. the situation still falls short in the instruction of how to assist at surgery. the importance of providing helpful and knowledgeable assistance is a skill that medical students often fail to appreciate. providing instruction through surgical videos about what role the assistant plays is a viable option, though it fails to reproduce the experience of retracting and suctioning. very few resources are available online that are not designated for professional surgical assistants, whose goals may be quite different from the undergraduate learner. creating educational tools that instruct students in surgical assisting may fill an important need. another established technique used in remote surgical education is the use of simulation. in general, simulation is used to demonstrate procedural techniques on a simulated patient (whether physical or virtual) in order to reduce patient risk and increase available opportunities for practice [14, 15] . telesimulation, which provides the benefits of simulation instruction to learners at an off-site location, has been described in settings where a single faculty may teach students in several disparate locations [16] . this technology may be useful in settings such as developing nations where local faculty lack the expertise to train their learners, and the challenge is in finding a reliable electronic infrastructure to communicate the simulations being explained [17] . in the setting of medical students being physically isolated from their institution, the challenge is not with the equipment used in communication but rather with the equipment used for the simulation itself. without being able to gather centrally at a simulation center, students lack the opportunity to use resources such as manikins and surgical instruments. a newer technology that has been proposed to authentically reproduce live experiences remote from the operating room is virtual reality. devices that incorporate virtual reality vary from partially immersive (where a surgical environment that can be manipulated by user controls appears on a screen) to totally immersive (with the use of a headpiece that places the user within a 360-degree environment). such devices have been used in surgical training, particularly in training for minimally invasive surgery and robotics [18, 19] . virtual reality has been proven to be an effective educational tool even for learners at the early stage of medical training [20] . while virtual reality devices may someday be affordable and available outside of the institutional setting, they are currently too expensive to put in the hands of every student learning from home. other less expensive resources are more realistic. providing each student with a prefabricated kit of surgical tools may be cost-effective, depending on the nature of the materials and the size of the class. at minimum, a hemostat, a suture removal kit, and a pack of suture could be provided to train students in suturing and knot-tying on the surface of his or her choice. alternatively, a student could create simulation models using implements at home. several do-it-yourself simulation kits have been proposed to educate students on topics ranging from cervical exams in labor [21] to performing hemorrhoidectomy [22] using household items. students choosing to pursue such instructional methods would be remarkable for high levels of motivation. objectives for surgical education that are more abstract are difficult to teach remotely. a key element of training in surgery is teaching trainees the importance of professionalism and communication. much of the education provided in the clinical arena is by modeling appropriate behaviors and encouraging students to reproduce positive examples of communication that they witness. being physically separated from their instructors, students at home may lack opportunities to observe professional behaviors. sharing web-based video examples of appropriate communication techniques has been shown to be effective in teaching surgical trainees [23] . use of an immersive virtual reality device to teach communication skills has also been proposed [24] . choosing what routes of instruction to pursue may be influenced by students' perceptions of each teaching method's value. students at our institution who were asked which remote teaching methods they considered the most helpful gave similar ratings to interactive online cases, remote lectures, and faculty-guided surgical videos. several students commented that the latter were superior to teaching they received live in the operating room, since the faculty spent more time teaching and the views of the surgical field were clearer. another challenge in having students forced to stay at home during the pandemic is finding optimal means to assess their clinical performance. assessment of student performance is crucial to ensure that students graduate with adequate knowledge to address clinical conditions they will encounter and adequate skills to safely and effectively treat them. assessment is also important to identify struggling students who may require remediation and exemplary students worthy of commendations or honor programs. assessment usually includes written, detailed performance feedback from supervising residents or faculty, and objective evaluation of knowledge and critical thinking based on oral and/or written examination. this tiered system has been used for decades, but recent evidence suggests it may be subject to implicit bias [25] . assessment of medical knowledge is carried out with standardized testing that requires little change in either preparation or execution. students may study using a variety of online and textual resources, and the test can be administered remotely, although significant trust is needed for ensuring adherence to an honor code to prevent cheating. modern medical education has embraced the addition of structured patient encounters and note writing assessments into many course grades [26, 27] . these encounters can be accomplished in an online format where patient, student, and grader may be physically remote from one another. it is more difficult to assess clinical performance in a rotation where a student has never set foot in the hospital. the ability to distinguish one student from another and provide the individual assessments so important to the progression on to residency is further challenged. grading performance on surgical rotations also includes proficiency at specific procedural and examination skills. direct observation when students are not present is limited by both the lack of resources to perform such procedures and the time requirement for faculty to set aside time for observation. it is unreasonable to expect assessment of complex procedures done remotely, but a video assessment of basic skills such as suturing and knot-tying is feasible if students are provided with a basic, prefabricated kit as previously mentioned. video assessment of surgical skills has been previously described using monitors as handy as a smartphone or gopro [28] . because the limited interaction with students has made it difficult for faculty to objectively discriminate degrees of mastery, many institutions elected to transition affected courses to a pass-fail format. with students confined at home, it is difficult to judge when they have either motivation or skills that exceed those of peers. professional behaviors such as attention to required attendance and appropriate interactions with standardized patients may be evaluated remotely, but in this setting, it is fairer to define a single standard of competency than to establish gradations of excellence. the use of pass-fail in medical education and its effect on ability of students to make a successful residency match without the distinctions that tiered grading provides was an additional stressor for students and will continue to be debated [29] . while everyone would like to see a return to the usual format of patient care and student education, it has become increasingly apparent that things may never return to what has previously been considered normal. as educators plan for a gradual reentry of students into the clinical workplace, they must carefully balance the need for education with the safety of patients and the students themselves. students participating in surgical and obstetrical procedures require access to personal protective equipment (ppe) to safeguard against covid-19 as well as blood-borne infections. with rationing of ppe to ensure an adequate supply for health care workers, institutional administrators need to judge the importance of student education against the needs of the community. depending on limitations in supply, students may be forced to assume an increasingly distant role from the surgical bedside. the covid-19 crisis has forced training programs, patient care, and workplaces to incorporate tools for simulation and remote engagement in ways that have been unprecedented. many of these tools will prove to be useful, even desirable, for our learning environments once the crisis has passed. remote learning will play an expanded role in our training programs for the foreseeable future. but which experiential activities and assessments are essential to retain, and which could be effectively replaced or supplemented by simulation or by remote activities? we now have an opportunity as educators to test our existing tools, develop new strategies, and collect a body of evidence surrounding remote learning interventions for clinical education. it is incumbent on us as a profession to develop evidence-based best practices to guide curricular development as we move forward in this rapidly changing environment. authors' contributions all authors contributed substantively to the writing of this manuscript. the impact of learning environment disruption on medical student performance medical education in post-katrina new orleans precepting at the time of a natural disaster reflections on a crisis in graduate medical education: the closure of hahnemann university hospital crises and turnaround management: lessons learned from recovery of new orleans and tulane university following hurricane katrina the impact of major earthquakes on the psychological functioning of medical students: a christchurch, new zealand study medical student experience in surgery influences their career choices: a systematic review of the literature impact of interactive vascular surgery web-based educational curriculum on surgical trainee knowledge and interest integrating a radiology curriculum into clinical clerkships using case oriented radiology education association of professors of obstetrics and gynecology (apgo) medical student educational objectives association for surgical education (ase) educational resources the utilization of video technology in surgical education: a systematic review is video-based education an effective method in surgical education? a systematic review simulation-based medical teaching and learning simulation as a surgical teaching model telesimulation: an innovative tool for health professions education introducing an e-learning solution for medical education in liberia systematic review of virtual haptics in surgical simulation: a valid educational tool? the virtual operative assistant: an explainable artificial intelligence tool for simulation-based training in surgery and medicine the effect of degree of immersion upon learning performance in virtual reality simulations for medical education vaginal examination simulation using citrus fruit to simulate cervical dilation and effacement novel low-cost, low-fidelity hemorrhoidectomy task trainers simulated disclosure of a medical error by residents: development of a course in specific communication skills a virtual reality curriculum for pediatric residents decreases rates of influenza vaccine refusal the surgeon as educator gathering validity evidence on an internal medicine clerkship multistep exam to assess medical student analytic ability using a diagnostic osce to discern deficit from disability in struggling students comparison of personal video technology for teaching and assessment of surgical skills impact of pass/fail grading on medical student's well-being and academic outcomes publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-034293-kngqw8q6 authors: west, marion title: ‘i’m not going to tell you cos you need to think about this’: a conversation analysis study of managing advice resistance and supporting autonomy in undergraduate supervision date: 2020-10-27 journal: postdigit sci educ doi: 10.1007/s42438-020-00194-5 sha: doc_id: 34293 cord_uid: kngqw8q6 this article, firstly, critically analyses a face-to-face supervision meeting between an undergraduate and a supervisor, exploring how the supervisor handles the twin strategies of fostering autonomy while managing resistance to advice. conversation analysis is used as both a theory and a method, with a focus on the use of accounts to support or resist advice. the main contribution is the demonstration of how both the supervisor and the student are jointly responsible for the negotiation of advice, which is recycled and calibrated in response to the student’s resistance. the supervisor defuses complaints by normalising them, and moving his student on to practical solutions, often with humour. he lists his student’s achievements as the foundation on which she can assert agency and build the actions he recommends. supervisor-student relationships are investigated through the lens of the affective dimensions of learning, to explore how caring or empathy may serve to reduce resistance and make advice more palatable. by juxtaposing physically present supervision with digitally mediated encounters, while acknowledging their mutual entanglement, the postdigital debate is furthered. in the context of covid-19, and rapid decisions by universities to bring in digital platforms to capture student-teacher interactions, the analysis presented is in itself an act of resistance against the technical control systems of the academy and algorithmic capitalism. this article presents a microanalysis of face-to-face undergraduate supervision. it adds to the literature on how relationality is constituted through advice sequences (shaw 2013 ) but critically examines this topic in a postdigital context. knox (2016) and fawns, aitken and jones (2020a) remind us that in our postdigital age, in which students and staff are multiply connected, not all face-to-face interactions help build meaningful relationships. chris park's (2007) description of doctoral supervision as 'a secret garden' can be equally applied to undergraduate supervision; despite the prevalence of shared offices and overheard meetings, supervision remains a largely hidden encounter. during the recent pandemic, new dimensions might be considered too, as supervisors and students meet across commercial platforms that universities have hastily introduced. this article explores a face-to-face supervision meeting with a media student planning her undergraduate dissertation. it is developed from previous research (west 2019a, b) which problematises who, if anyone, is to judge excellence in supervision, and whose excellence would be measured anyway. it is informed by the conversation analysis (ca) tradition, which is the main method for investigating naturally occurring talk, and its previous applications to institutional talk and more specifically to advice. two research questions (rqs) are addressed. (1) how do supervisors support students' autonomy? and (2) how do supervisors manage resistance? these are prominent themes in the literature in both higher education (he) (bowker 2012 , mayes 2015 , park 2014 , vehviläinen 2009a and 2009b , waring 2007a and 2007b , leyland 2018 ) and other institutional settings potter 2011, muntigi 2013) . this article contributes to work on the negotiation of advice and relationality in supervisory settings, where it is acknowledged that metrics do not record everything that is important (fawns, aitken and jones 2020b) . as one of the first ca studies of interaction between undergraduates and their supervisors in a uk setting, it demonstrates the staging of support for students' autonomy, which varies from a nudge to full direction, where humour is never far away. it reveals how the supervisory dyad build on resistance to tailor advice to the individual (and wonders how this can be done online and for the many). supervision is a process entailing a range of approaches which depend on the supervisor, the supervisee and their particular context. these may vary according to the 'needs, competencies, expectations, and philosophy of the supervisor and the supervisee' (anderson 1988: 12) . traditionally, the role of the supervisor is to 'provide guidance, advice, instruction, encouragement, support' (mac keogh 2006: 20) . however, it is best to refer to roles such as directors, project managers, providers of subject expertise, advisors, mentors and editors rather than a singular role (rowley and slack 2004) . to this list, we can add promoter of self-efficacy (mac keogh 2006) . these roles congregate at different ends of the control versus freedom or offering guidance versus support ranges. marshall (2009) suggests that direction should be staged from higher levels of autocracy at the start, to abdication at the end of the process. masters and in particular doctoral supervision have attracted more research interest than undergraduate supervision. this study draws on some of this literature, while recognising that the goal for doctoral supervision of fully fledged researchers with 'competent autonomy' (gurr 2001) is over-ambitious for first degrees and that 'wellsupported autonomy' (oldmeadow n.d.) seems a safer and more appropriate goal for final year undergraduates. undergraduate research autonomy is both a goal and a staged process (wisker 2018: 1) . the undergraduate dissertation remains the hallmark of 'graduateness' in the uk and elsewhere (derounian 2011: 91) . dissertations vary in length between 8000 and 15,000 words and are worth between 20 and 60 credits. degree classification generally takes the dissertation grade into account, and students cannot graduate with honours without an independent study. there is consensus that the dissertation process is an 'emotional rollercoaster' (shadforth and harvey 2004: 149-150) ; todd et al. (2004) cite silén's (2003) analogy of 'chaos and cosmos' in which students are not only unsettled but also stimulated by the dissertation task. it is therefore not surprising that both pastoral and technical elements of the supervisory relationship contribute to producing a dissertation (derounian 2011) . when derounian (2011) invited undergraduates to rank supervisor qualities, they ranked subject/literature knowledge top, while listing encouragement in 4th position, above structuring/steering/guiding, ranked 6th and placed empathy in 8th position. according to hagenauer and volet (2014) , teacher-student relationships are an under-researched area. when greenbank and penketh (2009) interviewed undergraduates about their dissertation experience, they found that supervisory relationships were the most significant themes, but did not explore how they are enacted. shaw (2013) calls for more research on how relationality is constituted through advice sequences, a sequence being composed of at least two turns at talk, each produced by a different person. one facet of the supervisory relationship not given much attention in the he, as opposed to the interactional literature (benwell and stokoe 2002) on supervision, is humour. this study contributes to knowledge in these areas. despite a growing literature on the deployment of conversational resources in academic settings, there do not appear to be many conversation analytic studies of undergraduate supervision. an exception is henricson and nelson (2017) , whose cross-linguistic study compares advice in undergraduate supervision and writing centre meetings in sweden and finland. other he interactional research has focussed on settings such as peer tutoring (e.g. waring 2007a and park 2014) or writing conferences (e.g. mayes 2015 , leyland 2018 where the power differential between the advice giver and advice receiver is less marked than with lecturers and students. bowker (2012) looks at negotiating understanding and agreement in supervision meetings with international masters' students in scotland. additionally, i would suggest that it could be very interesting to undertake conversation analytic research in an online context, though this is beyond the scope of the current article. there is not the space here to discuss the changing landscape of he, which several sources claim has altered student expectations (e.g. money et al. 2017) . certainly, consumer constructions of students do not adequately capture the subtle complexities of modern-day student realities (tight 2013, o'toole and prince 2015) . transactions between students and tutors are central to 'student engagement', a strong predictor of success (zepke and leach 2010) . however, this term has itself been critiqued for being a generic buzz phrase (hayes 2018) , where such policy discourse fails to engage in itself with the notion of the diversity and autonomy of individual students. researchers such as michael long (1996) see learner autonomy as a fixed trait, while for others, it can be developed (macaskill and taylor 2010) . for the purposes of this study, the operational definition put forward by macaskill and taylor (2010: 357) was used, which suggests that autonomous learners 'take responsibility for their own learning, are motivated to learn, gain enjoyment from their learning, are open-minded, manage their time well, plan effectively, meet deadlines, are happy to work on their own, display perseverance when encountering difficulties and are low in procrastination when it comes to their work'. dissertations are promoted as an effective vehicle to develop the graduate attribute of autonomy (todd et al. 2006) . the challenge in undergraduate supervision is to provide sufficient support to cultivate initiative and self-reliance (todd et al. 2004) . indeed, río, diaz-vazquez and maside (2017: 159) go so far as to claim that 'the undergraduate dissertation is characterised by a conflict between autonomy and support'. but as noted above, the interactional realities have received little attention. hence, the second focus of this study is on how supervisors support autonomy. even in an institutional environment in which advice is expected and sometimes requested, giving advice is still 'a tricky thing to do' (shaw 2013: 217) and may be resisted. this study pays special attention to how resistance is both prevented and managed in advice negotiation. according to brown and levinson's (1987) model of politeness, advice is a face-threatening act in that it may threaten the recipient's positive face (goffman 1976: 5) , or their desire to be respected, as it calls into question their knowledge and competence. it also undermines their negative face, or desire to be unimpeded, in that it sets up some degree of obligation to follow the advice. furthermore, advice which is unsolicited is more likely to be received as a face-threatening act (goldsmith and fitch 1997) . clayman's definition of resistant responses to interviewer's questions as those 'that either decline to fully address the topical or action agenda of the question, or shift to a different agenda, or both' (clayman 2013: 647) can be applied to advice reception. resistance can be done in either an overt or covert way (clayman 2013) . alignment is the process of matching or coordinating talk in such a way as to show support, agreement, convergence or similarity. its converse is misalignment. whereas active resistance clearly produces misalignment, passive resistance is achieved by withholding acceptance by means of a non-response or by diverting the conversation away from the advice (barton et al. 2016) . active advice resistance can also be 'pre-emptive' in that it can be produced before the expert has delivered advice (gill, pomerantz and denvir 2010) . resistance can also be done ambiguously, to signal either passive acceptance or resistance (heritage and sefi 1992) . dispreferred responses that disrupt the progressivity of the shared talk or that challenge underlying assumptions have also been labelled as resistant (muntigi 2013) . preference in conversation analysis is understood in the sense of what is preferred for smooth, untroubled social interaction, in other words progressivity (schegloff 2007) . preference is viewed from an interactional rather than a psychological standpoint. normalising involves labelling something in the interaction as commonplace or interpreting it in an ordinary way. this may make any ensuing advice more palatable and therefore circumvent resistance. svinhufvud et al. (2017) identify three uses for normalising in their study of academic counsellors: to support the student's position, to challenge the student or to present the student's problem as workable. experts use a variety of tactics to reduce resistance, sometimes combining this with support for autonomy. accounts have been conceptualised as 'explanatory talk used to propose a cause, to make plain, and to warrant' (antaki 1994: 2) . studying them can illuminate what speakers choose to make accountable and what kinds of accounts are seen as legitimate and accessible. the advice giver tailors advice to the client or service-user both before and after the actual advice. indeed, the professional's combined selection and sequencing of advice formulations have a bearing on how likely the advice is to be accepted. following waring (2007b: 368) , accounts in advice giving are 'the reasoning provided to bolster the viability of the advice.' accounts are used to counter and forestall resistance and so have an obvious bearing on rq2. they are also used to do pedagogy (waring 2007b) ; the type of account supervisors use may also reveal which role(s) they are orienting to. accounts have been studied in primary care (stivers 2005) , psychiatric consultations (angell and bolden 2015) , peer counselling (waring 2007b) , and academic writing tutorials with teaching assistants (park 2014) . there is a gap in the literature regarding their use in undergraduate supervision, which the current study intends to fill. in her study of the position of accounts in graduate peer supervising, waring (2007b) found that accounts were used to address face threat, to forestall resistance, and to do pedagogy, including socialising students into academic discourse. she found that accounts feature prominently in writing tutors' advice sequences, to the extent that approximately two-thirds involve an explanation. problem-accounts explain what the trouble is, for example, 'you kind of drift a bit' and benefit-accounts point to the advantages of following the relevant advice for example 'that will save you time' (waring 2007b: 372) . waring demonstrates that accounts are routinely found in the following sequential positions: first position accounts: pre-advice second position accounts: immediately post-advice third position accounts: post-problematic uptake fourth position accounts: post-acceptance hybrid: most commonly pre-advice plus post-acceptance (waring 2007b) these are the steps park (2014: 367) identifies: step 1: acknowledgment of tutor advice (e.g., 'yeah', upshot, repeat of key terms) step 2: contrastive conjunction (e.g., 'but', 'the thing is') step 3: epistemic statement (e.g., 'i feel/think x') step 4: account for advice resistance park then applies the stepwise model to the supervisor's further negotiation and fine tuning of advice: step 1: acknowledgment of student concern (e.g., 'right', 'that's true') step 2: contrastive conjunction (e.g., 'but') step 3: specified advice (e.g., reason/example) heritage and sefi (1992) identify three main ways to display receipt of advice, which have been corroborated and extended. (1) marked acknowledgement (ma) is typically signalled by oh (for news receipt) (heritage 1984) and right (for acceptance), a partial repeat (heritage and sefi 1992) or okay (guthrie 1997) . okay is generally combined with other tokens such as right, yeah or a future action, e.g. i'll do x accepting the advice. (2) unmarked acknowledgement (uma), typically signalled by continuers such as mm, hm, yeh or that's right, conveys that the recipient was already aware of the information offered and/or that they do not consider it relevant to themselves. such receipt tokens do not acknowledge or accept that talk as advice. okay and right, unprefaced by oh, can fall into this category too. however, park (2014) citing silverman (1997) and pudlinski (2002) notes that in certain sequential environments, umas may create ambiguity as they can show agreement as well as passive recipiency. tutors may then push students for more marked acknowledgement (bowker 2012) . (3) heritage and sefi (1992) note that advice may be rejected with assertions of competence or knowledge. however, hepburn and potter raise the issue of how far such displays resist a proposed course of action 'rather than aligning with the trajectory of the advice by offering a shared perspective' (hepburn and potter 2011:218) . waring (2007a) examines student orientation to their existing efforts while replying to advice. this serves to display their identity as effortful students who are aware of relevant gaps in their knowledge. waring (2007a) builds on earlier work (e.g. heritage and sefi 1992) by distinguishing between simple and complex advice acceptance. complex advice acceptance does something more than simply accept the advice. acceptance is combined with claims of comparable thinking (pudlinski 2002 ) (e.g. yes i thought the same thing or i understood that when i was doing it but i did not know how to do it) or with accounts, which are explored next. waring (2007a: 133) explores the use of accounts combined with advice acceptance to portray the student's previous action as thoughtful rather than haphazard and hence himself as a competent student. she draws a parallel with the competing preference structure for compliment responses for both acceptance and autonomy established by pomerantz (1978) , in that the turn of the dispreferred action is designed with delay devices and only weak agreement tokens (e.g. yeah) or disagreement tokens (e.g. well). although more research has been undertaken on advice delivery and reception, advice resistance and how it can lead to advice negotiation have received less attention until recently (e.g. waring 2007a , 2007b , vehviläinen 2009b , park 2014 , leyland 2018 . conversation analysis is aimed at discovering the rules governing everyday interaction. ca is both 'a theoretical and methodological approach' (psathas 1995: 3) , grounded in ethnomethodology. ethnomethodology 'often examines the processes through which the social world is made orderly and coherent' (wiggins 2017: 243) . ca aligns with constructionist and interpretivist principles in educational research. its epistemological and ontological assumptions are that social interaction is orderly at all points and that interactants orient to that order, an order which can be discovered and described (heritage 2008) . this study attends to how advice is initiated, received and negotiated through various practices, a practice being any feature of a turn at talk which has a distinctive character, a specific location within the turn or sequence, and is distinctive in terms of the social action implemented by the turn (heritage 2010) . this research is interested in local outcomes, observable in members' practices, rather than distal outcomes, which are not accessible to conversation analysis. interviewing students about their advice take-up in terms of both mental processes and the execution of their project is beyond the scope and aims of this study. according to garot and berard (2011: 131) , '[m]embership categories refer to identities, essentially and mca [membership categorisation analysis] centers around the study of identities as they are achieved or contested, organized and understood, within the practical contexts of social interaction and language use'. sacks maintains that categories are 'relevant for the doing of some activity' (sacks 1992: 597) , thus linking mca to ca's programme for investigating social action. mca shares its ontological base with ca in that it roots itself in members', and not analysts', concerns. nevertheless, the development of ca with its emphasis on sequential analysis largely disregarded mca for some time. however, more recent attention (e.g. stokoe 2012) has focused on its usefulness in combination with other research methods including ca. comparing ca's 'juggernaut' to mca's' milk float', stokoe (2012) maintains that there is no need for ca to derail mca and promotes a method to complement ca with mca. the main steps in mca involve exploring when interactants engage in categorybound activities, which are 'those actions that are expectedly done by members of a particular category' (butler and weatherall 2006: 444) or invoke 'category-tied predicates' or characteristics which may express 'rights, obligations, and knowledge' (stokoe 2012: 281) . categories such as the standardised relational pair of student/ teacher are omni-relevant in that participants prioritise their orientation to their category incumbency (sacks 1972) . fitzgerald, housley and butler (2009: 46) maintain that knowledge displays show their relevance 'beyond the sequence being examined', whereas gardner (2013: 594) argues that teacher/student roles are not as 'fixed and omnirelevant as some studies suggest'. mayes (2015) analyses writing center sequences to see if and how participants orient to a particular membership category such as student-teacher or writer-reader. notwithstanding this, she acknowledges that a pedagogic goal is for teachers to help their students become better writers, which encompasses teachers orienting to the writer-reader, with the result that the roles of student-teacher and writer-reader may coalesce at times. the data, from a wider study on advice sequences in undergraduate supervision meetings, was collected at a uk widening-participation university in autumn 2013. colleagues from law and humanities were approached on the basis of familiarity, but lecturers in the same department were deliberately not asked, as i did not want to combine research with working closely within my module teams. i was an insider to the extent that i am a lecturer of the hei, but an outsider to the extent that the supervisory interactions were from outside of my discipline (trowler 2011) . instead of viewing the insider-outsider in research positioning as a dichotomy, i see these as relative (kerstetter 2012) . supervision may vary by discipline; hence, interaction may be different, a possibility this study acknowledges. i recognise that by approaching lecturers with whom i had a relationship, the sample may be biased in terms of its lack of representativeness of our diverse population. although i had not seen the lecturers i approached in a pedagogic context, they may have been 'like me' in some way. i left the selection of a student up to the supervisor. i deliberately opted for a 'light touch' in my involvement in the recruitment and data collection stages. supervisors contacted their students on my behalf to request their participation, and the meetings were audio-recorded by the supervisor or the student. this does mean, however, that i have very little information concerning how often the particular students and supervisors met, and whether this was their first meeting or not, unless that is obvious from the data. it also means that i have no knowledge of the offline-online dynamic between the pair. furthermore, gender dynamics, though not the focus of the current study, may have been at play. for example, males are socialised to take risks while females are socialised to capitulate in the face of challenge (rosenfeld 2002) . although research across collections of data is common, by focussing on individual sequences, and even a single case, the conversation analyst can gain a fuller picture of the unfolding interaction (bolden and angell 2017) . here, five sequences from one meeting, which lasted nearly 13 min, are analysed in the order in which they occurred. these were selected to demonstrate the negotiation of advice. audio-recording was selected to facilitate data collection; i acknowledge the significance of the embodied turn and the limitations resulting from the decision not to use video. the recordings were transcribed according to the conventions laid down by jefferson (2004) once data is collected, sidnell (2010: 30) suggests looking for patterns such as repetition and word selection, formulations and formats, which are all part of turn design, which takes account of what is being asked, when and to whom. recipient design relates to 'the multitude of ways which display an orientation and sensitivity to the particular other(s) who are the co-participants' (sacks et al. 1974: 727) . sidnell (2010: 30) also recommends using 'keys' to gain an analytic handle on data. firstly, it is important to identify turn construction units (tcus), which may be lexical, phrasal, clausal or sentential. these include possible utterance completion at the transition relevance place (trp) where the turn could pass to another speaker. the turn-taking system allocates only one turn to a participant (sacks, schegloff and jefferson 1974) , so when a speaker anticipates that they will need more than one turn, they may alert the hearer to an upcoming expanded turn with intonational changes or alerts such as well (heritage 2015) . when participants continue past this point, what is achieved? we join the supervisor and student 3 min into the meeting in which they have been thrashing out key issues in the project such as the focus, aims and methodology. as the sequence unfolds, the supervisor's orientation to directing her development as an independent student increases. in fig. 1 , the supervisor displays his stance on it being the student's task to think through the structure of her dissertation. he sets up this particular advice sequence with a query about her thoughts, presumably on how she will structure her dissertation. he does this with an 'and-prefaced question' (lines 1-2), a device used to indicate a question's agenda-based character (heritage and sorjonen 1994) . the student shows that she has given this topic some thought with 'well i was thinking' to preface her question: 'would i still be able to include … or would that be going off' (lines 4-7), thereby orienting to the supervisor's entitlement to pass judgement and her need for validification (vehviläinen 2009a) . the supervisor responds with a negative assessment: 'it seems like it's veering away on the tangential' (lines 12-13). he follows this up with a question about why the student thinks jamie oliver is 'so popular' (lines 13-14). the supervisor meets the student's response 'because he's relatable' with a creaky voiced 'yeah', which may signal that he is less than convinced and wishes the student to continue, followed by a repaired question: 'but why: wh -what is there about him?' aligning to his pedagogic goals to push the student to find answers for herself (lines 17-18). after an awkward interaction (lines 19-24) consisting of the supervisor's quiet but hearable in-breath (line 20), a quiet hesitation marker: 'er:m', an answerable gap of 2.4 s and the student's claim of 'i don't know' (line 23) plus a longer gap of 3.2 s, the supervisor glosses his advice with 'i mean those are the sort of questions you need to be thinking about' (line 25), thereby bringing the need for the student to take responsibility for her learning to the fore. his turn is designed to flush out resistance. the student produces two overlapping 'okays' in relatively weak acknowledgement, the first of which is stretched (line 26) and the second followed by laughter (line 28). these indicate less than full alignment with the supervisor's project. the post-acceptance account, which is combined with further advice: 'i have my ideas but i'm not going to tell you yet cause you need to think about this' (lines 30-32) with its amplification of 'my' and stress on 'tell' serves to playfully underpin the advice by emphasising that it is the student's project, and not the supervisor's (line 34). this is difficult to resist. there are parallels here with the findings of park (2012 park ( : 2006 in that tutors remind students of their project ownership. laughter during talk may signal 'trouble, limitation or insufficiency' (hepburn and potter 2010) . here it modulates the action of both student and the supervisor. of interest is the order of the laughter, with the student laughing first, after an unmarked acknowledgement of the supervisor's directive: 'okay' (line 28). however, we have no access to any visual clues possibly inviting laughter. then, the delivery with intermittent laughter softens and ironises the supervisor's refusal to share his ideas: 'i have my ideas but i'm not going to tell you' (line 30). although 'laughing moments may be rife with power moments' (glenn 2010 (glenn :1497 , here his action also appears to be designed to maintain relationality (benwell and stokoe 2002: 450) . furthermore, the interactional work undertaken by the supervisor prevents resistance. the student acknowledges the combined account and advice with 'okay' (line 33), produced this time with falling intonation in overlap, which represents firmer acknowledgement (bowker 2012) . nevertheless, the supervisor reinforces his point with 'your project' (line 34), to which the student replies with a different response token: 'yeah', again produced with falling intonation. so, although the student produces only minimal acknowledgement both before and after the account, her agreement tokens are produced with more receptive intonation after the tutor's account. the variation in her response tokens after 'your project' from 'okay' in three consecutive slots (lines 26, 28 fig. 1 extract from 3.05 to 4.00 min: (be)cause you need to think about this. tut tutor (supervisor); stu, student and 33) to 'yeah' may signal greater uptake (schegloff 1982: 85) . with 'right' (line 36) delivered with falling intonation, the supervisor accomplishes topic closure, moving to a new topic with 'what's your methodology going to be?' after a gap. he thereby remains in pedagogic mode by closing one topic and shifting to the next on his agenda. events in fig. 2 occur nearly 7 min into the meeting, after the supervisor establishes what the student has read so far. this sequence opens with troubles-telling (jefferson and lee 1981) , itself indicative of the student-tutor relationship as it is evidence of the student feeling sufficiently comfortable to reveal her difficulties. the supervisor normalises this emotion (svinhufvud et al. 2017) . this is the first occasion that the supervisor foregrounds the need to start writing. the student frames her complaint: 'it's really hard to juggle doing all the module stuff' (line 1) with an extreme case formulation (ecf): the selection of 'all the module stuff' emboldens her claim and 'locate(s) truth in the object being discussed, rather than in the eyes of the beholder' (bischoping and gazso 2016: 120) . however, the supervisor receipts this after a gap with a stand-alone, slightly stretched 'y:ea', thus treating the problem description as normal and relatively unnewsworthy. after another silence he produces: 'erm i have a little sympathy. not-', with stress on a 'little' (line 6) and a cut-off 'not'. the normalising is done with a light touch in line with the supervisors in svinhufvud et al. (2017) , but with irony too. the action at this point may be to ostensibly support the student's position, while challenging her thinking. the student responds with laughter (line 7). instead of topicalising the student's feelings, the supervisor moves on with his agenda to forward a workable fig. 2 extract from 6.47 to 7.15 min: it's really hard to juggle. tut, tutor (supervisor); stu, student solution. by fending off potential resistance, he may be able to pull off his forthcoming advice sequence. 'er-but it looks like you've got some-some good material' (lines 8-9) is janus-faced in that it looks back in counter position to her complaint but is also analysable as a normalising account in pre-advice position. her problem is workable. resistance may be minimised and she may be empowered as an autonomous learner to 'gain enjoyment from [their] learning' and 'display perseverance when encountering difficulties' (macaskill and taylor 2010: 357) . but the student's very quiet, slowly delivered 'okay' does not present her as necessarily convinced (line 10). the supervisor delivers his bald advice with a necessity modal (need): 'you need to start writing this now'. the student receipts this with an unmarked acknowledgement: 'e-yeah' on line 12. one interpretation of the action done by the following quietly delivered 'that's what i thought' is that the student resists the supervisor's advice as newsworthy and reclaims her epistemic rights to pre-existing opinions on her own project. this is in line with pudlinski's (2002) claims of prior thinking. an alternative interpretation is that by demonstrating a shared perspective, the student aligns herself with the supervisor's advice (hepburn and potter 2011: 218) . regardless of her stance, the supervisor ploughs on with his benefit-account, in which the 'start' in the proposed action to 'start writing' is both verbally emphasised and echoed in the resulting benefit: 'you're gonna start pulling your ideas together' (line 15). this forward-looking benefit-account is in line with waring's (2007b) findings concerning supervisors solidifying advice. furthermore, this is an example of recipient-designed advice, since much of the prior talk has pivoted around the need to find a focus for the project. in evidence is the supervisor's pursuit of his pedagogic goal to support the student's development towards this focus. his action foregrounds the student's responsibility for 'pulling' her 'ideas together'. the student receipts this with another quiet 'okay', stretched on this occasion and produced in turn-final overlap with the supervisor's first 'so' (lines 18-19). in fig. 3 , the supervisor works up his concluding remarks, having already told the student: 'you need to start writing this now' (fig. 2: line 11) . the supervisor abandons and repairs 'yeah, i think we should try an' after the cut-off 'i-' (line 2). the 'well' signals that what is to come is not straightforward or part of the original plan (heritage 2015) . the supervisor abandons his first action (which is likely to have been heading towards arranging another meeting) in favour of directing the student to undertake 'two things'. from now on, the advice is given baldly with a directive: 'first of all start writing' (line 3) as the supervisor orients to his role as director, which may not be easy to resist. although the student produces a marked acknowledgement in the form of a repeat of the directive, plus 'okay': 'start writing okay' (line 4), her rising intonation does not present her as entirely convinced. so the supervisor continues in overlap with a stronger directive to 'really start writing'. with 'i can't emphasize' (line 5), he invokes high entitlement before launching his authority-account in second position after the advice, implying that other students engage in erroneous thinking: 'cause too many students think i'll read and then i'll write' (lines 6 and 9). through not explicitly including her in this category of students, the supervisor can achieve two goals: he uses his enacted expertise as leverage in warning of the potential trap, while not accusing her of falling into it. oppositional talk (potter and wetherell 1987) helps the supervisor to keep the student on his side and includes her in the category of autonomous learners who are 'low in procrastination' (macaskill and taylor 2010: 357) . he contrasts her with the noxious identities of that category of students who leave the writing until, he implies, it is too late. the advice is packaged as information that ostensibly may not be relevant to the student, though its underlying relevance to the recipient (shaw 2013) should not be ignored. resistance is therefore less likely. the supervisor uses the negatively valanced 'too many', which he vocally emphasises. this is underlined by his broken-off, stressed 'i can't emphasize' (line 5). he uses the device of the present tense 'think' to represent a generalised, iterative present to formulate this as a routine though erroneous plan (edwards 2000) . several features add weight to the misguided separation of the reading and writing process in the delivery of the problem-account: the gap (0.3) between the two actions of reading and writing, the stressed 'i'll read' and 'i'll write' and the combination of the conjunction 'and' with 'then'. a repeated, upgraded directive, combined with a professional authority account, may be difficult to resist. so how does the student react to the warning not to delay writing? analysed as an example of stepwise advice and resistance, fig. 4 provides a clue that it will not be plain-sailing for the supervisor. in line with step 1 of park's (2014) model, the student had acknowledged the supervisor's advice with a repeat of his instructions (silverman 1997) : 'start writing, okay' (fig. 3: line 4) . the supervisor then upgraded his advice with 'really start writing' (fig. 3 : line 5), as we saw further above. initially, the student's overlapping 'y:eahh,' at least partly, aligns with the supervisor's authority account (fig. 3: line 8) . however, her next turn follows step 2 of park's model (fig. 4 : lines 1-2): she introduces her resistance with a disjunctive 'but', succeeded by a repaired epistemic statement (step 3) 'i've got-i'm confused', which serves as an account (step 4) built with her own extreme case formulation in 'all the readings' for why she cannot start writing. we cannot be sure where she was heading with the 'got', though 'confused' seems an obvious candidate. the repair 'i'm confused' highlights her current state of confusion. by explaining her own experience, the student claims epistemic primacy (ekberg and lecouteur 2015) , which may put the supervisor in an awkward position. this is in line with the 'candidate obstacles', which patients put forward as a possible reason for not being able to follow lifestyle advice in diabetes consultations (barton et al. 2016) . so how does the supervisor respond to her resistant display of emotion? partly in line with the variations in park's model (2014), he skips over fully acknowledging the fig. 3 extract from 10.32 to 10.46 min: too many students think i'll read... tut, tutor (supervisor); stu, student student's complaint that she's 'confused' (step 1). in common with the tutors in svinhufvud, voutilainen and weiste (2017) , the supervisor treats her emotion as commonplace and presents her problem as workable. but the transition to problem solving is not entirely smooth, as svinhufvud et al. (2017) found in some cases of normalising. there are perturbations in his abandoned turn 'then er yeah' (line 3) and a gap after the student's latched 'yeah' (line 4). he then signals that his turn will not be straightforward or expected with 'well'. this takes the place of the contrastive conjunction in park's step 2 (2014). he then follows this with specified advice (step 3): 'if you start writing' which is delicately yet emphatically delivered, in that 'writing' is delivered with a creaky voice, possibly in acknowledgement of the student's difficulties. furthermore, 'start writing' is delivered at a slower speed and with emphasis on the 'start' and the first syllable of 'writing', perhaps to underline the significance and challenge of the desired action. the recipient-designed reason 'because i know you've read a lot' (lines 6-7) inserted between the 'if' and 'then' elements utilises his knowledge of her extensive reading and aligns him with her recent account of 'all the readings' (line 2). she is unlikely to resist this account, which invokes her membership of the 'good student' category. thus far in the meeting, the supervisor's authority-accounts have been used to mainly prevent resistance, but now, he uses an account based on shared knowledge of the student's reading: 'because i know you have read a lot' to counter resistance (lines 6-7). this recipient-designed account bears some similarity to the client-attentive accounts angell and bolden (2015) discuss in their study of psychiatric patients, in that it relates to the client's concerns, in this case the state of being 'confused of all the readings'. however, here the supervisor turns the 'problem' on its head by orienting to the benefits of having 'read a lot' as a foundation for the writing he now prescribes. he is able to temper his advice in accordance with her 'candidate problem' (barton et al. 2016) . this move also coincides with that of the counsellors in cromdal et al. (2018) , who acknowledge their client's problem description on a young people's helpline by offering a solution both grounded in that description, yet recast. the client-attentive and authority-accounts are intertwined in that the supervisor both shows he recognises that the student has read and asserts his judgement over the quantity with 'a lot', where 'a lot' is a positive assessment of an activity in which students are expected to engage. the supervisor uses the polarity between clarity and confusion in that the writing he advises will help her 'clarify what you're going to do', thus reducing her confusion. he places 'actually' in a counter-informing position to offset her claim (clift 2001) . he also capitalises on her focus on reading by asserting that the remedy he advises 'actually will-will-focus your reading more specifically on what you're trying to achieve ' (lines 11-12, 15 ). this benefit-account is recipient-designed to play to her strengths (her extensive reading) and goals (completing her dissertation). such positive personalisation of advice should bring her on board. rather than orient to the institutional requirements to produce a dissertation, as he could have done with 'what we're asking you to do', the supervisor foregrounds the student's agency. such highlighting of her agency may serve to both support her autonomy and manage resistance. the supervisor follows this up with a script formulation (edwards 1994 ) of the normal process of reading leading to writing (lines 17-21, 23, 25), on two occasions produced with an agentless 'it should be' (lines 17 and 25). he neatly summarises the dissertation cycle with 'read write' (line 28). thus, he provides a professional authority account based on his expert knowledge of the dissertation cycle, temporarily removing the focus away from the individual student, which may be tricky to resist. the student receipts his advice with a stretched 'yeah' (line 30) and then reports on her dissertation work to date, which has resulted in her being 'more confused than i want to' (lines 34-35). here the student may be resurrecting the problem to show understanding of the recipient-designed solution. this time she refers to her confusion as an account for advice uptake with: 'cause i have found…' (line 30), rather than her earlier contrast-marked 'but then i've got-i'm confused' (lines 1-2). she selects the full form: 'i have found' as opposed to the contracted form: 'i've found'. this may be to underline her ownership of that experience, or to emphasise that experience. instead of focusing on the problem to which he has already provided a solution, which may lead to further resistance, the supervisor crystallises his advice with a repeated directive to 'definitely definitely start writing' (lines 36-37), delivered with high entitlement, which can be difficult to resist. this action shares its solution orientation with the counsellors in svinhufvud et al. (2017) . the student receipts this with 'okayh', delivered with a falling intonation contour to mark the completion of her turn (line 38). however, this is not the end of the sequence, as the supervisor continues his advice in fig. 5 . now he adds to the advice to start writing by reducing the stakes: 'don't worry if it doesn't get into your final submission' (lines 1-2). he follows this up with an objectside assessment (edwards and potter 2017) : 'it's always a valid exercise to do'. this object-side assessment focuses on the writing activity rather than the tutor's disposition towards it, and is delivered with 'always', a 'maximum case proportional measure' (pomerantz 1986: 223) of frequency to underline its invariable utility, regardless of the individual occasion. the student receipts this with another 'okayh' (line 4) delivered with slightly rising intonation, which here may display a weak epistemic stance or a questioning one (hepburn and bolden 2017: 41) . this conveys that the student is not fully convinced. so the supervisor volunteers information about his own writing process, prefaced by 'i mean' perhaps to specify the preceding advice. 'whenever i write an article, i always have a folder that says edits' (lines 5-6) refers to a 'long-standing personal state of affairs' (antaki et al. 2005: 190) . once more, ecfs on this occasion: 'whenever' and 'always', ratchet up the significance of the action and underline the routine. in line with the psychotherapeutic data in leudar et al. (2006: 29) , self-disclosure is rare in this data and is 'hearably relevant to what the client has said'. it may be used to normalise a client's problem, to model possible ways of dealing with it, or to even up the interaction (priebe et al. 2011) . it may be doing all three here, but perhaps particularly modelling a solution. resisting what a supervisor himself practises may be awkward. it could be argued that the supervisor is re-indexing himself as the more expert; people who write articles are more often established academics rather than undergraduates. however, it seems more likely that he is orienting to the autonomous writer category (mayes 2015) , referencing the student's and his co-membership (benwell and stokoe 2006) by sharing some of his pain as well as modelling his strategies. by stressing 'by the time i'm finished' and the relative size of the edits folder, the supervisor portrays his own writing practices with humour and humility. earlier, the student recounted her experience of reading so much that it had left her 'confused'. now in parallel, he offers a matched experience (heritage and lindström 1998) of iteratively pruning his writing. this action may be a vehicle for displaying empathy and relationship building, and designed to lessen resistance. in overlap, the student receipts this revelation with laughter (line 8). the student acknowledges the end of the tutor's turn with 'ohkay's both followed by 0.7 second gaps, (line 11) which may be stretched to take up more interactional space in overlap (hepburn and bolden 2017) . the supervisor then moves into a new sequence suggesting another meeting before christmas (line 13). finally, the student produces a marked uptake of his earlier advice: 'okay then. i shall start writing' (line 19), delivered with a full modal form rather than a contracted 'i'll'. this promise embodies her commitment to a future action recommended by her tutor (stevanovic 2011) . the analysis has shown how the supervisor uses socratic questioning to help the student find her own focus, followed by a humorous refusal to provide solutions (fig. 1) . the student moves from complaining about juggling the demands of the final year (fig. 2) , and later her confused state (fig. 4) , to announcing her intention to 'start writing' (fig. 5) , thereby taking up the tutor's directive. this declaration comes at the end of a sequence in which the tutor has recourse to a number of devices and actions to counter her resistance. these include warning her about the category of students who put off writing (fig. 3) , tailoring the advice to her achievements and goals (fig. 4) , expounding on the writing process with script formulations (fig. 4) and sharing his own writing practices (fig. 5) . woven into the sequences are orientations to supporting the student's autonomy, particularly regarding effective planning and time management. as goffman (1981) notes, laughter is a method for relating on a human rather than institutional level. in figs. 1, 2 and 5, humour appears to lubricate the wheels of the supervisory encounter and manage resistance. while sympathising to some extent with the 'emotional rollercoaster' (shadforth and harvey 2004: 149) involved in juggling the demands of the final year, the supervisor uses humour to defuse the complaint and move the student on to a practical solution (fig. 2) . this action supports the student's developing autonomy in that she is offered the tools to overcome hurdles. if students are to develop their autonomy, they need opportunities to exercise decision-making; this supervisor pushes the student to consider points she does not appear to have given thought to, balancing scaffolding with a humorous refusal to provide her with all the answers. whether or not others would feel comfortable with 'pulling off' such recipient design, is for them to judge, as is how they might do this in technology-mediated ways. managing resistance and promoting autonomy are both conflicting and complementary forces. indeed, 'doing being a supervisor' is often a matter of managing these forces as well as possible. the multiple roles of the supervisor can complicate this task. meanwhile, students' positions can be viewed on two continua: compliance as opposed to resistance, and dependence as opposed to independence (bowker 2012: 18) . as bowker argues, '[t] he role of the supervisor can then be seen as leading the student from dependence to independence whilst avoiding or overcoming resistance ' (19) . whether either party may be tempted to take 'short cuts' in online communication, thus circumventing the careful negotiation of advice, including tailored, stepwise patterns, remains to be seen. the detailed analysis of staff-student interaction provides valuable insights into the messy complexities of undergraduate supervisory relationships, which can be easily ignored in agendas to measure what is more easily measurable. how this fits into the wider sector aims of 'measuring excellence' warrants further discussion. furthermore, amidst a global, market-led, neoliberal drive for universities to dramatically increase numbers of students (jones 2019) , how sustainable is one-to-one supervision? a critical postdigital perspective (peters and besley 2019) should help us make judgements about how best to combine both individualised and group, online and offline, supervision. further research should explore the mutual entanglements between online and offline student-supervisor communication. a starting point could be orientation to different roles and affective displays (pyyhtinen and suoranta 2020) . such research also necessitates further application of politeness theory (brown and levinson 1987) and its developments to both technologically mediated communication and face-to-face encounters in the academy. as both a method and approach, conversation analysis has much to contribute to investigating online communication and analysing this in a postdigital context. indeed, ca 'can offer a unique insight into if, and how, technology impacts upon interactional practices' (meredith 2019) . with the development of media platforms, it will be interesting to see how particular organizational features such as repair and sequence organisation adapt and nest, or even, in the case of turn-taking for online written communication, disappear. i have related the findings of waring's (2007b) study on account positions and park's (2014) work on stepwise sequences, also with peer tutors, to the more asymmetric site of undergraduate supervision, and applied the work of svinhufvud et al. (2017) on normalising in study counselling to an alternative university context. this article has addressed two rqs concerning how tutors support student autonomy and manage resistance. the analysis has explored the tension between directing students and supporting their autonomy (both of which the student may resist), for instance with refusals to provide all the answers and encouragement for the student to own their project. the tutor walks a tightrope between institutional requirements and his professional authority, on the one hand, and support for the student's agency. nonetheless, at this relatively early stage in this student's dissertation, his role as director rather than mentor prevails. i have shown that this student pushes back against advice with claims of epistemic primacy, and more subtly when she delivers unmarked acknowledgments such as 'okay', particularly when these are produced more quietly, with slightly raising intonation or stretched, showing that she may be less than convinced. the tutor counters resistance by not making heavy weather of problems, attending to her goals instead, which also serves to support her growing autonomy. resistance is prevented with benefit accounts, in line with his solution focus. the tutor invokes various categories: the category of students who delay writing, the 'good student' who has 'read a lot' and the autonomous writers (including himself) who edit profusely. rather than topicalising student concerns, this supervisor focuses on alleviating them by providing the student with a solution to work on. amid a 'growing cultural tendency to pathologise everyday emotions and responses' (ecclestone and hayes 2009/2019: xvi) , the humorous, normalising treatment of this student's affective practices (wetherall 2012) is refreshing. this stance serves to both reassure the student and lighten up the interaction, while attending to both the face needs of the student (goffman 1976 ) and the institutional goal to manage resistance and support autonomy during the negotiation of advice. but it appears that not until the student 'feels heard' is the advice palatable, as evidenced by her eventual commitment to 'start writing'. resistance has value in both asserting a student's ownership of their dissertation, and their experience thereof, and for moving students towards independence. it also facilitates the fine-tuning of advice. this article contributes to the literature on the management of affect in institutional settings. it makes visible important affective practices, such as the supervisor's ways of caring, showing empathy and increasing student agency, which would otherwise remain hidden in more quantitative measures of excellence. in itself, it is an act of resistance against the big-datafied surveillance culture of he (knox 2019) . in line with the critical philosophy of the postdigital, it values the relational and event-based process above abstraction and formalization (peters and besley 2019) . i have demonstrated through this analysis that the feedback that occurs between a supervisor and a student is not adequately captured or understood through either the algorithms that collect data or the metrics that this process is based on. this is because there are deeply contextual factors that get overlooked. this is even more important now that so much supervision is (by necessity) taking place online. the study provides readers with insights into certain supervisory interactions, the richness of which may enable them to be applied as appropriate to their own situations (lincoln and guba 1985) . this standpoint reflects the criterion of transferability for qualitative research (bryman 2001 ). it does not necessarily mean changing existing practices. rather, for continuous professional development, sharing transcripts of recorded supervision meetings with supervisors can provide them with a more theorised understanding of situated practice, and of their own competencies. the conversation-analytic role-play method (stokoe 2011) , in which the trainer engages practitioners in a line by line uncovering of the transcript, synchronised with the recording, could be used to stimulate discussion and awareness of the choices available in interaction. this article also encourages readers to consider recording, analysing and sharing their own practice. with the increasing use of online delivery, the facility to record interactions means that they are easily available for our own reflective work and, with permissions, collegiate inquiry. whether, and how, the 'datafied university' (knox 2019) would (know how to) use recordings to evaluate the 'effectiveness' of staff-student rich dialogues (knight and littleton 2015) is another debate. ethical approval all procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional committee (faculty of education, university of wolverhampton) and with the 1964 helsinki declaration and its later amendments or comparable ethical standards. (2004) [ ] brackets are used to represent overlapping talk at precise points in the talk. = an equals sign represents no break or gap within one turn or between two different turns, where the ordinary beat of silence between turns has not been produced. (0.3) a silence is measured to the nearest tenth of a second and placed in parenthesis. a silence that is less than two-tenths of a second (a micropause) is represented as a full stop in parentheses. need underlining of a word or a part of a word is used to mark a stressed production. :: colons represent an extension of the prior sound. ne:ed underling and colons are used in combination to mark up-down contours. where the letter is underlined and followed by a colon, as in this example, the pitch movement goes up and then down through the word. when the colon is underlined, the pitch is rising through the word. arrows are used to represent marked intonation shifts. .,?¿ punctuation markers are used to represent normal shifts in intonation. the question mark represents questioning intonation, the comma marks slightly rising intonation, while the upside down question mark represents rising intonation that is in between the two. a full stop in contrast marks falling intonation. need uppercase is used to represent talk that is louder relative to the surrounding talk.°n eed°degree signs are used to represent talk that is quieter or softer relative to the surrounding talk. <need> the use of the left/right carats in this order to surround talk, is used to represent the talk as being slowed down in comparison to the surrounding talk. >need< the use of the right/left carat in this order to surround talk, is used to represent the talk as speeded up or rushed through. <need a left hand carat at the beginning of a word is used to display that the utterance was 'jump started'. neethe hyphen is used to mark a word as being cut off. need* an asterisk is placed after a consonant to represent a dentalised sound. nee#d a croaky voice is represented by the hash symbol. .hh inbreaths are represented with a row of 'h's that have a full stop placed in front of them. hh outbreaths are represented by a row of 'h's with no full stop. neehhd a row of 'h's within a word is used to display breathiness. nee(h)d an 'h' that is parenthesized is used to represent plosiveness. heh huh laughter is represented by different combinations of 'h's and vowels. a pound symbol is used to mark a smiley voice. (need) single parenthesised words are used to represent an unsure hearing. (( )) double brackets represent the transcriptionist's comments. the supervisory process in speech-language pathology and audilogy justifying medical decisions in mental health care: psychiatrists' accounts for treatment recommendation explaining and arguing: the social organization of accounts self-disclosure as a situated interactional practice patient resistance as a resource: candidate obstacles in diabetes consultations constructing discussion tasks in university tutorials: shifting dynamics and identities discourse and identity analyzing talk in the social sciences: narrative, conversation and discourse strategies the organization of the treatment recommendation phase in routine psychiatric visits okay? yeah? right?': negotiating understanding and agreement in master's supervision meetings with international students politeness: some universals in language usage social research methods no, we're not playing families': membership categorisation in children's play conversation analysis in the news interview meaning in interaction: the case of 'actually'. language basically it's the usual whole teen girl thing': stage-of-life categories on a children's and young people's helpline. symbolic interaction shall we dance? the importance of staff-student relationships to undergraduate dissertation preparation the dangerous rise of therapeutic education script formulations: an analysis of event descriptions in conversation extreme case formulations: softeners, investment, and doing nonliteral some uses of subject-side assessments clients' resistance to therapists' proposals: managing epistemic and deontic status online learning as embodied, socially meaningful experience ecological teaching evaluation vs the datafication of quality: understanding education with, and around, data. postdigital science and education omnirelevance and interactional context conversation analysis in the classroom ethnomethodology and membership categorization analysis pre-emptive resistance: patients' participation in diagnostic sense-making activities interviewer laughs: shared laughter and asymmetries in employment interviews interaction ritual: essays on face-to-bace behaviour the normative context of advice as social support student autonomy and reflections on researching and writing the undergraduate dissertation negotiating the "rackety bridge" -a dynamic model for aligning supervisory style with research student development on the systematic deployment of okay and mmhmm in academic advising sessions teacher-student relationship at university: an important yet underresearched field invisible labour: do we need to reoccupy student engagement policy? learning and teaching giving and receiving advice in higher education. comparing sweden-swedish and finland-swedish supervision meetings transcribing for social research interrogating tears: some uses of 'tag questions' in a child protection helpline designing the recipient: managing advice resistance in institutional settings conversation analysis: practices and methods a change-of-state token and aspects of its sequential placement conversation analysis as social theory well-prefaced turns in english conversation: a conversation analytic perspective dilemmas of advice: aspects of the delivery and reception of advice in interactions between health visitors and first time mothers constituting and maintaining activities across sequences: and-prefacing as a feature of question design motherhood, medicine, and morality: scenes from a medical encounter glossary of transcript symbols with an introduction the rejection of advice: managing the problematic convergence of a 'troublestelling' and a 'service encounter' capital, neoliberalism and educational technology insider, outsider, or somewhere in between: the impact of researchers' identities on the community-based research process dialogue as data in learning analytics for productive educational dialogue posthumanism and the massive open online course: contaminating the subject of global education what does the 'postdigital' mean for education? three critical perspectives on the digital when psychotherapists disclose personal information about themselves to clients resistance as a resource for achieving consensus: adjusting advice following competency-based resistance in l2 writing tutorials at a british university naturalistic enquiry the role of the linguistic environment in second language acquisition supervising undergraduate research using online and peer supervision the development of a brief measure of learner autonomy in university students a handbook for teaching and learning in higher education enhancing academic practice becoming an 'autonomous writer': epistemic stance displays and membership categorization in the writing conference conversation analysis and online interaction undergraduate student expectations of university in the united kingdom: what really matters to them? cogent education resistance in couples counselling: sequences of talk that disrupt progressivity and promote disaffiliation the psychological contract of science students: social exchange with universities and university staff from the students' perspective. higher education research and development supervising undergraduate research projects redefining the doctorate. york: higher education authority seeking advice: epistemic asymmetry and learner autonomy in writing conferences stepwise advice negotiation in writing center peer supervisoring critical philosophy of the postdigital compliment responses: notes on the cooperation of multiple constraints discourse and social psychology: beyond attitudes and behaviour good communication in psychiatry -a conceptual review conversation analysis: the study of talk-in-interaction accepting and rejecting advice as competent peers: caller dilemmas on a warm line lecture as an event in postdigital education: a dyadic teaching diary satisfaction with the supervision of undergraduate dissertations what do we learn about difference from the scholarship on gender? what is the future for undergraduate dissertations? education and training an initial investigation of the usability of conversational data for doing sociology a simplest systematics for the organization of turn-taking for conversation discourse as an interactional achievement: some uses of 'uh huh' and other things that come between sentences sequence organization in interaction the undergraduate dissertation: subject-centred or student-centred? advice giving in telephone interactions between mothers and their young adult daughters conversation analysis: an introduction responsibility and independence in learning -what is the role of the educators and the framework of the educational programme discourses of counselling: hiv counselling as social interaction participants' deontic rights and action formation: the case of declarative requests for action. interaction and linguistic structures parent resistance to physicians' treatment recommendations: one resource for initiating a negotiation of the treatment decision simulated interaction and communication skills training: the 'conversation analytic roleplay method moving forward with membership categorization analysis: methods for systematic analysis normalizing in student counseling: counselors' responses to students' problem descriptions students: customers, clients or pawns? higher education policy independent inquiry and the undergraduate dissertation: perceptions and experiences of final-year social science students. assessment and evaluation in higher education supervising a social science undergraduate dissertation: staff experiences and perceptions. teaching in higher education researching your own institution. london: british educational research association. www.bera.ac.uk/researchers-resources/publications/researching-your-own-institution-higher-education student-initiated advice in academic supervision problems in the research problem: critical feedback and resistance in academic supervision complex advice acceptance as a resource for managing asymmetries. text and talk the multi-functionality of accounts in advice giving managing advice resistance and supporting autonomy in undergraduate supervision: a conversation analytic study i'm not going to tell you cos you need to think about this affect and emotion: a new social science understanding discursive psychology: theory, method and applications frameworks and freedoms: supervising research learning and the undergraduate dissertation improving student engagement: ten proposals for action. active learning in higher education acknowledgements this article is developed from my doctoral thesis (west 2019a) and from a paper presented at the higher education institutional research (heir) conference: 'measuring excellence' in he: approaches and their impact at university of wolverhampton (west 2019b) . the author wishes to thank the supervisor and student who agreed to record their meeting for the purposes of the wider study on which this article is based. she would also like to express her thanks to the university of wolverhampton for research support and the organisers of the heir conference. key: cord-013203-qvuand0s authors: corbin, charles b. title: conceptual physical education: a course for the future date: 2020-10-14 journal: j sport health sci doi: 10.1016/j.jshs.2020.10.004 sha: doc_id: 13203 cord_uid: qvuand0s the conceptual physical education (cpe) innovation began in the mid-20th century as an alternative approach to college-level, activity-only basic instruction classes. in addition to physical activity sessions, cpe courses (classes) use text material and classroom sessions to teach kinesiology concepts and principles of health-related fitness and health-enhancing physical activity. cpe courses are now offered in nearly all college programs as either required or electives classes. two decades later, the high school cpe innovation began, and kindergarten-8 programs followed. in this commentary, i argue that historian roberta park was correct in her assessment that physical education has the potential to be the renaissance field of the 21st century. scientific contributions of researchers in kinesiology will lead the way, but science-based cpe and companion fitness education (fe) programs that align with physical education content standards and fe benchmarks will play a significant role. cpe courses have been shown to be effective in promoting knowledge, attitudes, and out-of-school physical activity and have the potential to elevate physical education as we chart the course of our future. in 2008 corbin and cardinal 1 cited historian roberta park, 2 who suggested that physical education has the potential to become the renaissance field of the 21st century. her thorough historical account describes the emergence of physical education in the 1800s and traces its development as a science-based profession. park notes that medicine was an emerging field during the 1800s but became the renaissance field of the 20th century largely because of research that provided a scientific basis for medical education and the profession. i believe that park was correct in her suggestion that what was true of medicine in the 20th century can become true for physical education in the 21st century. we can become a renaissance profession, but not without changing our course. the purpose of this article is to articulate the importance of knowledge, especially higher-order knowledge, and conceptual physical education (cpe) as a program central to the delivery of knowledge and other aspects of physical literacy in the future. the dominant focus of 19th-century physical education in the us was formal activities imported from european countries (e.g., gymnastics and regimented exercise) 2 . by mid-19th century, physical education teacher preparation programs were established in the us, and in 1885 the association for the advancement of physical education was formed. the profession of physical education was becoming a force in american education. buoyed by research in psychology (e.g., hall) and education theory (e.g., dewey), leaders put forth theories of play, games, sport, and the -recreative‖ value of physical activity. 2 through the early years of the 20th century the debate about how best to prepare physical educators and the nature of the content of school physical education programs continued. over time the -new physical education‖ of cassidy, nash, williams, wood, and others provided direction for programs of the 1900s. 3 expanded objectives for physical education programs such as leadership, teamwork, and sportspersonship based on uniquely american -democratic values‖ found their way into school programs. physical education became much less regimented, and sports and games became a significant part of the physical education curriculum. the european physical education of the 19th century morphed into the sports-dominated physical education of the 20th century. by the mid-1900s the science movement had begun and prompted park to suggest a new look for the 21st century. she details the new science within physical education (kinesiology) and documents the great strides that have been made in our multiple sub-disciplines. this excerpt characterizes her thoughts: 3 there is a -great need for well-trained and dedicated professional practitioners in areas ranging from the health and fitness industry, to public school physical education, to exercise programs for older populations, to the management of recreational and sporting agencies, and much, much more. the attitude that prizes systematized knowledge, constant questioning, and the ability to forge logical links and see interdependencies, however, must infuse the work of the teacher, coach, and clinician as much as that of the exercise physiologist and biomechanist, sport psychologist, sport sociologist, and sport historian‖ (p. 20) . the new science outlined by park sparked many changes within physical education and kinesiology and was an important factor in the emergence of cpe and fitness education (fe) programs. cpe originally referred to physical education programs (typically a semester-long class or shorter units in a class) that focused on teaching concepts, principles, self-management, and consumer skills to promote healthy lifestyles (e.g., regular physical activity, sound nutrition, making good consumer choices) and the outcomes associated with healthy lifestyle adoption (e.g., fitness, health, wellness). unique features of cpe include classroom sessions and the use of a textbook or text-based materials. 4 knowledge gained in the classroom and from text materials is reinforced in physical activity sessions specifically planned for cpe. although not initially labeled as such, early cpe programs became known as fe programs because they were developed for use with fitness assessment. 5, 6 as health-related fitness testing gained traction, the programs were expanded to help students understand the reasons for testing and to help them learn to effectively plan physical activity programs to build health-related fitness. more than 40 years after the introduction of cpe, fe was formally included in the society of health and physical educators (shape america) fitness education framework 7 as -a subcomponent of the total physical education program, focusing on helping students acquire knowledge and higher-order understanding of health-related physical fitness, the product, as well as habits of physical activity and other healthy lifestyles, the process, that lead to good health-related physical fitness, health and wellness‖ (p. 1). 7 in the years since their inception, cpe and fe programs have evolved significantly. some examples are used to illustrate. fitness for life 8 is an example of a cpe program (e.g., text, classroom and activity sessions) that is also considered to be an fe program because of its extensive fitness-related content. however, over time, it has expanded to include content related to all physical education standards (see later sections). physical best 9 is an fe program that focuses on health-related fitness content. because it does not use a student text or classroom sessions, it is considered to be an fe program, but not a cpe program. science of healthful living 3,10 uses text-based materials but does not use classroom sessions. its authors refer to it as a concepts-based physical education 10 program. for the purposes of this article, concepts-based physical education programs will be included as cpe programs. cpe became widely adopted during an era of change. originally conceived as a physical education offering, college classes are now offered in departments with a variety of names (e.g., kinesiology, exercise science, sport science, health and human performance) and many course titles (e.g., fitness for life, personal fitness, concepts of fitness and wellness). regardless of the academic unit or class name, cpe captures the spirit of park's call for physical education to take its place as an enlightened or renaissance field. at the time of its introduction, cpe was not well received. 11 however, the cpe innovation that began at the college level in the mid-20th century became an offering, either as a required class or an elective, at virtually all institutions of higher learning 1, 12, 13 . cpe -merges the practice and science of the field through a lecturelaboratory approach‖ (p. 467). 1 as cpe grew in popularity, the accompanying science movement provided evidence of the link between physical activity and public health. the epidemiological evidence helped to push performance-based youth fitness testing of the late 1950s toward health-fitness testing by the end of the century. social movements (e.g., civil rights, rights for women, student rights) spurred change as students became active in affecting societal change. they fought for student choice in course and curriculum options. these movements aided the college cpe innovation because cpe provided an alternative to traditional required physical education classes. 1 cpe classes offered a science-based personalized approach as an alternative for all students, especially for those who did not relate to the traditional sports-based physical education offerings. in addition, evidence of the effectiveness of cpe in promoting knowledge, attitudes, and physical activity behaviors provided support for its inclusion. 1 cpe programs, and the evidence supporting them, have saved physical education requirements at many institutions since inception. 1 many of the same factors that led to the college cpe innovation fueled the growth of both cpe and fe programs at the high school level. particularly influential were the growth of the science base within kinesiology and the associated public health approach to physical education. 14-22 knowledge in kinesiology expanded rapidly and provided a platform for advancing the profession of physical education. the shift from performancefitness to health-fitness testing 6 occurred simultaneously with the science boom. practical considerations also contributed to the rationale and need for cpe programs. while kinesiology was accepted in the mainstream of science, physical education was gradually being marginalized in schools. survey data from the youth risk behavior surveillance system 23 indicate that daily physical education attendance decreased from 41.6% in 1991 to 25.4% in 1995. it has remained stable since 1995, but attendance is well below what it was early in the 1900s. 24 facing the challenge of being sidelined or eliminated, many schools adopted cpe as an evidence-and standards-based option that helped protect physical education requirements. 25 furthermore, those who advocated for cpe at the high school level pointed out that not all high school students go to college and that non-collegebound students deserve to participate in effective cpe programs. 26 was the first high school cpe program. a number of others followed, including personal fitness for you, 27 personal fitness: looking good/feeling good, 28 and foundations of personal fitness. 29 fitness for life (6th ed.) is the most widely used model. 8 consistent with park's call for a scientific foundation for physical education, cpe programs are based on sound education theories that provide a foundation for building higher-order learning. promoting confidence, intrinsic motivation, and autonomy (social cognitive theory and self-determination theory); promoting the belief that these factors can help in overcoming barriers (health beliefs model); and providing information about moving through several stages of behavior change (trans-theoretical model) all set the stage for helping students achieve higher-order objectives in cpe. 37 the program stairway to lifetime fitness, health and wellness, as illustrated in figure 1 , provides a visual description that illustrates how theory meets practice in moving students from dependence (in elementary school) to independence and autonomy in middle and high school. 8 central to the -stairway‖ metaphor is the notion that learning (achieving literacy) is vertical, not horizontal. early learning provides a foundation for later learning. accordingly, the stairway emphasizes the importance of addressing higher-order objectives in physical education through teaching for independent thinking and autonomy. as ennis and colleagues 38 have demonstrated, conceptual learning at one grade level provides a foundation for learning in subsequent gradesor to put it another way, it helps students learn how to learn. at steps 1 and 2 (level of dependence) young students are dependent on us, the teachers. they typically lack fitness and physical activity knowledge and benefit from a direct teaching style. they participate as directed and benefit as the directed activities allow. at steps 3 and 4 (level of decision making) students begin to understand and apply concepts and principles and to use self-management skills (e.g., self-assessment, goal setting, self-monitoring, self-planning). they begin to analyze and evaluate their own behaviors. at steps 5 and 6 (level of independence) students become independent and autonomous ( fig. 1) . they become problem-solvers capable of making decisions that can enhance their long-term fitness, health, and wellness. 8 national physical education content standards provide a basis for establishing student objectives and outcomes, including higher-order objectives. the first national physical education content standards were published by the national association for sport and physical education (naspe, now shape america) in 1995. 39 the standards were revised in 2004 40 and again in 2013. 41 the current standards are shown in table 1 . early cpe programs focused on the parts of health-related physical fitness, health-related fitness selfassessments and interpretation, the types of physical activities that promote health and health-related fitness, the fitt (frequency, intensity, time, type) formula for building fitness through physical activity, and steps in program planning. coverage of nutrition and stress management were also included, as were self-management skills (e.g., goal setting, self-monitoring, overcoming barriers). they focused primarily on standards that specifically identify knowledge, concepts, and principles as important student outcomes (standards 2 and 3) and the standard related to health-related fitness (standard 3). cpe continues to provide its original fe function, but as suggested by mohnson, 42 it now covers concepts and principles related to expanded sub-disciplinary content as well as all five physical education content standards. beyond content that focuses on exercise physiology concepts and principles, programs now typically include biomechanical and motor learning principles to help students in their efforts to demonstrate competency in motor skills and movement patterns (standards 1 and 2) and sociological and psychological concepts and principles that underlie the development of self-management skills and social emotional learning (standards 4 and 5). correlation tables have been created to show that programs comprehensively address standards and to indicate which materials and lesson plans address which standards. 43 as noted earlier, in 2012 naspe/shape america developed a framework (with benchmarks) for fe for kindergarten-16 7 . the framework acknowledged the importance of teaching content commonly associated with cpe and fe programs and provided benchmarks for student achievements in these programs. many of the benchmarks (table 1) were derived from longstanding cpe content. for this reason, many cpe programs met all, or most, of the fe framework benchmarks prior to the development of the framework. now, many cpe programs have expanded beyond the fe framework to include content from all physical education content standards. as we move forward, physical education standards and fe benchmarks will need to evolve consistently with the new knowledge that can serve our students in the future. in recent years the term physical literacy has gained traction. 44, 45 the international physical literacy association (ipla) describes a physically literate person as one who has -the motivation, confidence, physical competence, knowledge and understanding to value and take responsibility for engagement in physical activities for life‖ (p. 1). 46 shape america has adopted this definition but operationalized it to refer to a physically literate person as one who meets national physical education content standards (table 1 ). in a separate paper, 45 i have expressed my concerns about the many different definitions of physical literacy and the use of the term. these concerns will not be revisited here. both the ipla definition and the shape america standards demonstrate that the development of knowledge is an important characteristic of physical literacy. as we move toward the future, i encourage physical educators to expand their view of knowledge development (especially higher-order knowledge) to include concepts central to current definitions of health literacy. health literacy, as defined by the institute of medicine (now the national academy of medicine), -is the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions‖ (p. 1). 47 cpe is a program designed to help students to obtain, process, and understand information about physical activity and its health benefits. cpe content centers on providing the knowledge to assist students in making well-informed decisions about physical activity, nutrition, and stress management. as ennis 3 aptly notes, content -transcends the physical, relying on a sound foundation of knowledge to guide and lead physical activity choices and participation across one's lifetime‖(p. 122). she further emphasizes that -knowledge is at the heart of physical literacy‖ and that physical literacy -includes not only knowledge for performance but also the ability to apply knowledge and use knowledge for innovation‖ (p. 119). 3 cpe began at the high school level in the late 1970s with a health-related fitness focus. the content was organized in several major strands as depicted in the first column of table 2 . over time, the cpe content expanded based on user demand and alignment with standards and benchmarks (see the second column of table 2 ). today's middle school cpe content is similar to early high school programs, with representative content indicated by asterisks alongside topics in table 2 . lately, energy balance has also been an area of study in middle school cpe. 48 the original model for high school cpe 43 included an 18-week program (a one-semester class) that met in the classroom two days a week and in activity sessions three days a week. in the us, florida's required personal fitness class was implemented in the 1980s using this model. 49 as was the case when the florida cpe was implemented, the one-semester cpe class is often coupled with a second semester of a -required elective‖ to fulfill a one-year physical education requirement. various states and districts have followed this model, but other alternative schedules are common. for example, some high schools have integrated cpe classroom and activity sessions with traditional activity units, thus extending the class over a full school year. 43 at the middle school level, the three-unit, nine-chapter fitness for life model (taking half a semester) has been widely adopted. 30 in the half-semester plan, two textbook lessons are studied with 2 classroom sessions and 3 activity sessions. however, distributing units over different grade levels is a frequent alternative plan. portfolio sheets are used for recording student data. the science of healthful living curriculum is designed to increase middle school students' knowledge and interest in health-related science. 50 it consists of 120 lessons in 2 units over grades 6, 7, and 8. 3, 10 the curriculum, also referred to as -science in the gym‖ emphasizes a 5e system (engagement, exploration/experiment, explanation, elaboration, and evaluation) to integrate cognitive and physical tasks. students use workbooks and journals as a guide to learning tasks. research on the science for healthful living curriculum indicates that using workbooks is an effective way of promoting knowledge development in physical education. 51,52 fitness tests have been around since the 1950s 53 and were developed primarily to test youth fitness. the early skill-related fitness test batteries gradually gave way to health-related fitness tests beginning with the development of the american alliance for health physical education, recreation, and dance (aahperd, now shape america) health-related fitness test in 1980. after years of debate about various fitness tests, fitnessgram ® created at the cooper institute for aerobics research (cooper institute), morphed from a fitness report into a full health-related test battery with computer-based fitness reports. 6 critics have long suggested that tests by themselves do little to encourage fitness improvement and activity participation and may lead some youths to avoid engaging in physical education altogether. 54 cpe programs were employed to address this concern. although there was no formal collaborative agreement, the fitnessgram ® . 55 the help (h = health, e = everyone, l = lifetime, p = personal) philosophy that originated with fitness for life, by agreement, also became the philosophy of fitnessgram ® . the test items were included in fitness for life as self-assessments providing the basis for building a personal fitness profile in program planning. professional development sessions were conducted at the cooper institute for both fitnessgram ® and fitness for life to help teachers implement the programs and to prepare instructor trainers to conduct workshops to help teachers implement both programs. to fulfill the need for programs for k-8, a variety of fe programs were developed. smart choice, included in the first fitnessgram ® test manual, was an early fe program. 8 it included an award for keeping activity logs and meeting goals. in 1994, it's your move activity booklets were created for elementary school students; and in 1995, you stay active was published jointly by aahperd and the cooper institute. 6 you stay active included teacher materials and student handouts for use in physical education to help students better understand why they were taking fitness tests and how fitness test scores can be used to set goals and aid in program planning. the cooper institute currently offers online instructional materials, called smart coach, for teachers. 55 in 1994 fitnessgram ® became the health-related fitness test battery and fitness reporting system endorsed by shape america. 6 aahperd's physical best health-related fitness test battery was converted to an educational program that supported fitnessgram ® . 6 during the 1990s, aahperd sponsored pre-convention instructor trainer sessions for physical best, fitnessgram ® , and fitness for life. physical best is currently -a program of resources and training for k-12 physical educators … to implement health-related fitness education, including curriculum development and health-related fitness activities‖ (p. 1). 9 physical best activity guides are available for elementary, middle school, and high school teachers. 9 other programs-such as the president's youth fitness program (pyfp), which features fitnessgram ® test items. 56 comprehensive school physical activity programs (cspap), which encourages before-, during-, and after-school physical activity. 57 the active and healthy schools program, which encourages activities throughout the day 58 and fitness for life: elementary school, 59 60 school children 61 have become quite effective in using sound management strategies 60,61 for keeping students active while promoting learning consistent with physical education content standards and fe benchmarks. these programs are often very well conducted by dedicated teachers. i do endorse the inclusion of activities that teach concepts and principles because they can help move students up the stairway to lifetime fitness, health, and wellness ( fig. 1 ) and prepare students for secondary cpe programs. i also endorse whole-school or comprehensive school physical activity programs 57 that create and foster healthy and active school cultures. the remainder of this article will focus on secondary school cpe rather on than elementary school programs. the formal exercise that characterized physical education in the 19th century gave way grudgingly to the -new physical education‖ in the 20th century. however, it was well into the 1900s before the -american‖ model that emphasized sports and games became firmly established. by 1994 the model was dominant. data from the school health policies and practices study (shpps) reveal 23 that team sports were the most frequently offered activities in secondary schools. basketball, volleyball, baseball/softball, football, and soccer were the top 5 activities for both middle schools and high schools. dodgeball and kickball were also frequently included in programs. the team-sport approach of the 1900s is still dominant now, at least in the us. table 3 provides data from the shpps. 62 the results look familiar. four of the team sports that ranked in the top five in 1994 are still in the top 5 for both high schools and middle schools, and sports (mostly team sports) are among the most frequently offered physical education activities. it is important to point out that the data reflect the number of schools offering the activities-not the amount of time spent on the activities. still, the statistics indicate that the same sports activities are repeatedly offered from year to year in secondary school physical education. this can lead to the administrative policy of dumping-mixing lower-grade students in the same classes as upper-grade students, resulting in repeating the same instruction and same activities again for upper-grade students. see mccullick et al. 63 and lounsbery et al. 64 for more information about physical education and physical activity policies and common barriers to successful programming. contrast the activities most frequently taught in schools with the activities in which most adults participate 65 (fig. 2) . team sports that are dominant in school physical education rank well below activities such as individual sports, outdoor activities, and fitness activities. my own observations suggest that fitness activities, most common among adults, are often not available to all students in secondary schools. many high schools, for example, have fitness facilities; but elective resistance training classes using these facilities are open primarily to athletes. physical education classes, required or elective, often cannot use these facilities for classes for nonathletes. additional statistics are revealing. shpps data indicate that 78.8% of middle schools and 95.5% of high schools require students to take physical education as a graduation requirement or for promotion. 61 at first glance, it would appear that most students are required to take secondary physical education. but the same survey data show that the percentage of schools that require physical education in each grade ranges from 34% to 26% for grades 6-8 and from 21% to 9% for grades 9-12. the percentage of students taking physical education 3 days a week is 9.1%. 62 as we plan physical education for the future, we should consider these questions. is it necessary to repeat the same activities over and over again, especially activities that are often not used after the school years? can all youth become proficient in all sports and activities? do they want to? would it be more productive to focus on activities that are more likely to be performed later in life? should students at least have a choice? how does repetition of the same activities contribute to all content standards and benchmarks? is the traditional sports-based model the best model for the future? dintersmithin his book what schools could be, 66 quotes john dewey: -if we teach today's students as we taught yesterday's, we rob them of tomorrow‖(front matter). dintersmith 66  purpose-students attack challenges they know to be important, that make their world better.  essentials-students acquire the skills sets and minds sets needed in an increasingly innovative world.  agency-students own their learning, becoming self-directed, intrinsically motivated adults.  knowledge-what students learn is deep and retained, enabling them to create, to teach others. dintersmith does not address physical education, but his principles can be applied to -what physical education could be". certainly there are many outstanding teachers that currently apply the peak principles in their programs. however, a variety of statistics call into question the universal application of peak principles in secondary physical education. cpe has purpose. in cpe, students are challenged to learn concepts, principles, and self-management skills that can be used throughout life. results of project active teen indicate that students who took cpe as 9th-graders use the information 20 years after high school graduation (see later section). cpe develops essential skills (e.g., self-management, consumer, performance) that are useful in the 21st-century world. cpe programs promote physical literacy that provides a foundation for later innovative learning. cpe builds agency. as shown in fig. 1 , students learn to become intrinsically motivated independent (autonomous) problem solvers and decision makers while in school and in later life. cpe enhances knowledge. as park 2 indicated decades ago, the science base (kinesiology) is significant and growing. cpe focuses on higher-order objectives, enabling students to learn to learn. this enables them to continue their learning throughout life. green 67 labels the -pe effect‖ as physical education's ability to promote lifelong participation in physical activity. he further indicates that despite the belief that physical education produces a -pe effect‖, there remains a dearth of evidence to support this supposition. green 67 states that physical educators often -take-for-granted‖ the positive effects of physical education programs (p. 1) and cites the need for more longitudinal research into the pe effect. the evidence for cpe is described in this section, including the longitudinal research that green recommends. the early research related to cpe was conducted at the college level and provided evidence of the effectiveness of programs in building knowledge, attitudes, and active behaviors. researchers have noted the drop in regular physical activity as teens transition to adulthood 69, 70 . but can physical education stop the drop? is there a pe effect? longitudinal studies suggest that there is. project active teen (pat) began in 1991. 71 high school 9th-graders took a yearlong cpe course using the fitness for life model. students used a textbook and had classroom sessions one day a week. they participated in activity sessions that focused on health-related fitness and the activity sessions were coordinated with classroom content as well as traditional physical education activities. teachers participated in regular professional development sessions that provided training in cpe content and methods, and detailed lesson-plan notebooks were provided. over the 24 years of the study, three different progress reports were published. the first study, pat1, assessed activity patterns of participants as juniors and seniors in high school 71 . results indicated that students who took cpe in the 9th grade had lower levels of inactivity and were more likely to meet physical activity guidelines than both those who took traditional physical education (tpe) and a national sample of age-matched peers. the second study, pat2, found similar results several years after participants graduated from high school cpe. 72 the third study, pat3, was conducted 20 years after the cpe students graduated from high school. as with the first 2 studies, results indicated that the cpe students were more active and less likely to be inactive than national peers and maintained high levels of physical activity 20 years after graduation. 73 questionnaire data 73 showed that -56% of respondents indicated that they remembered content from the class, 50% indicated that they still used the information, 47% indicated that they found the class useful after graduation, and 92% indicated that they currently consider themselves to be well informed about physical fitness and physical activity‖ (p. 3). in the 3 pat studies, 50 tests for statistical differences were conducted. all of the 12 significant differences favored those who took cpe. the authors of pat3 73 suggest that cpe -can be a vital part of a total quality physical education program that promotes lifelong physical activity and complements quality traditional physical education programs‖ (p. 5). ennis, chen, and colleagues 3,10,50 created a middle school cpe program that was used in a multi-year, federally funded intervention study (science of healthy living, shl). students in grade 6-8 in the intervention schools participated in a curriculum based on health, fitness, and nutrition that consisted of 120 lessons in two units and associated physical activity sessions. the results were similar to those for the pat project. two years after the initiation of the study, participants in the shl group had higher out-of-school physical activity levels than the participants taking tpe. they also scored better on knowledge tests. 10 research indicates that secondary school students often lack knowledge and/or hold misconceptions concerning healthy behaviors (e.g., physical activity, physical fitness, nutrition) [74] [75] [76] . other studies document a positive relationship between fitness knowledge and participation in physical activity [77] [78] [79] [80] . there is also evidence that cpe programs can improve physical fitness and physical activity knowledge [81] [82] [83] [84] . furthermore, research has shown that knowledge from lower grade levels enhances learning of knowledge at later grade levels, which is evidence of vertical learning 84 . based on the evidence, wang and chen 85 offer a hypothesis that knowledge is a pathway to motivation for physical activity and ultimately to increased out-of-school physical activity. collectively, the evidence is clear: fitness knowledge-and the resulting knowledge improvement-is an important benefit that results from a well-delivered secondary cpe program. these results support the value of knowledge, especially higher-order knowledge, as a powerful factor that can be the glue that cements together the many benefits of physical education. 86 as we move to the future, a strong case can be made for including cpe as a foundation physical education course in secondary schools. cpe programs are based on a sound philosophy (the help philosophy), a whole-person orientation, and sound learning theory. cpe program objectives align with national physical education content standards (physical literacy) and fe benchmarks. they address higher-order objectives that are consistent with the peak principles. 66 cpe programs also address school reentry considerations that were very important during the corona virus pandemic. 87 in addition, cpe has the support of the national academy of kinesiology 88 and members 89 of the medical community. cpe program content is based on kinesiology's sub-disciplinary sciences that park 2 championed as the basis for moving physical education forward in the 21st century. most importantly, cpe programs work. the evidence is considerable and growing (see previous section). and there is more. in addition to the factors described above, advocates suggest that cpe provides academic connections that benefit students in other subject matter areas (e.g., math, science, english language arts); provides formative and summative assessment tools, including student-centered assessments in the cognitive, affective, and physical domains; and enhances teacher self-esteem and program reputation. 25,37 a quality physical education experience could look like the following for k-12 education. after completing a quality elementary school experience that includes fe, middle school students complete cpe units that provide them with a foundation for entry into a one-semester or one-year high school cpe program. the high school cpe experience, in turn, provides a foundation for practicing lifelong healthy lifestyles (e.g., active living, eating well); for becoming a good fitness, health, and wellness consumer; and for making informed decisions about additional quality physical education and later life experiences. the cpe content is streamlined and coherent from elementary school to high school with a vertical sequence. some of the opportunities for expanding physical education beyond the cpe foundation are listed in fig. 3 , and the sections below illustrate the diverse and adaptive cpe programming for different school settings and situations as demanded for a 21stcentury education. as mccullick et al. 63 have shown in their study of school policies, there is a lack of legislative support for physical education requirements. the lack of support is one reason why most teens are not enrolled in physical education in secondary schools. 90 when physical education is required, it is typically for a limited time (one or two years in high school). enrollment is most prevalent among 9th graders, and by grades 11 and 12, fewer than 10% of students are enrolled. middle school teens are more likely to be enrolled than those in high school, but less than one-third of all students are enrolled in grades 6-8. 62 cpe electives offer students opportunities to continue physical education after the foundation stage to keep learning and maintain in-school physical activity and can be an effective method for increasing enrollment in elective physical education. research has shown that students who have completed cpe have a good knowledge base, as well as self-management, consumer, and decision-making skills. they can use these skills to make their own decisions about elective options. if we teach students to make good decisions, we must allow them to make their own choices. in a school that requires one year of high school physical education, a one-semester cpe course could be followed by a required elective course. students choose the activities that they want to pursue. for this to be a legitimate option, schools must offer what students want to take-not what teachers want to teach. if students want yoga, tai chi, resistance training, self-defense, and dance, we must be prepared to offer these activities and be prepared to teach them. in schools with no requirement, we must offer attractive options that will entice students to take elective physical education. we must also consider ways to reduce barriers to participation. there are obstacles to the introduction of new offerings in secondary schools that are often not easy to overcome. nevertheless, several options are proposed here. the work of teachers in the san francisco unified school district (sfusd) to create an advanced cpe class is one example of how change is possible. it goes without saying that professional development opportunities will be necessary to help physical education teachers implement the proposed options. elective cpe. in schools that do not required physical education, a cpe class can be offered as an elective option. if the option is offered, interested students will take advantage of it. advanced cpe. advanced cpe can also be an elective option. for example, sfusd has a high school cpe requirement for all students 91 . upon completion, students can continue cpe by electing to take an advanced class (fit for life 2). this sfusd class is unique in that it meets university of california admission requirements for electives 91 . the class expands on the content included in the required cpe class. another advanced option is an introduction to kinesiology class. offering an introduction to kinesiology class as a science course is defensible and opens doors for students who want careers in fitness, health, wellness, and allied health occupations. honors cpe. irwin and colleagues 88 recommend that honors classes be offered in physical education. they describe honors classes as courses designed for high-achieving students that include greater depth of coverage 92 . irwin et al. 92 maintain that physical education is a legitimate education domain, and it is appropriate that honors classes be available just as they are in other academic areas. advanced cpe and introduction to kinesiology classes are excellent options for honor classes at the secondary level. advanced placement cpe. advanced placement (ap) classes offer students the opportunity to pursue advanced study in an academic area and allow students to earn college credit in advance of college enrollment. as irwin et al. 92 indicate, -our profession is undergoing a critical dialogue to initiate a much-needed paradigm shift or rebranding of who we are and how society regards us. there is work to be done if we are to increase respect for what we do. we all know that our content can be exceptionally rigorous and just as important to student success as other academic content areas. ap classes can also provide opportunities for students who have a passion for our subject matter to challenge themselves. therefore, we are proposing that it is time to begin the process of creating and launching an ap course focused on our subject matter‖ (p. 8). cpe and introduction to kinesiology courses provide good ap options in physical education. in describing what school could be, dintersmith 66 places great emphasis on project-based learning using peak principles. he describes project-based classes in the schools that he visited and suggests that innovative project-based classes can be part of an effective 21st-century educational system. project-based learning is often structured so that groups of students work together on a specific project (e.g., building a robot in science class). however, students can also work on projects individually. key elements are innovation, student choice and decision-making, the use of 21st-century skills, and, most importantly, a relevant reason or goal that drives the project. cpe classes are, in many ways, project-based. students gain higher-order knowledge and decisionmaking skills driven by the opportunity to create their own lifetime program plan. the outcome can be student portfolios that are exhibits of a healthy-living project based on personal needs and interests. beyond cpe, other project-based physical education opportunities can be offered. sport education classes 93 are, in my view, projectbased classes. elective sport education classes in a variety of activities are consistent with peak principles and allow students (in groups) to play many different roles in sport and physical activity settings. likewise, adventure/outdoor education classes have potential as project-based classes, with students doing the planning and organizing. the health-and-fitness club model 94, 95 can also provide opportunities for project-based learning when students doing the planning, organizing, and administering. this option can be especially rich when students are offered the opportunity to apply their special talents in carrying out a project. for example, student musicians can create music (and background music) for exercise routines, student dancers can choreograph, student artists can create art, and students with computer and other technical skills can create active exergames and apps that encourage active living. schools have been reluctant to offer fully online physical education courses partly because of concerns about how to monitor physical activities and partly because of a lack of digital skills and resources for offering such classes. in 2018, shape america published guidelines for k-12 online physical education 96 . the guidelines offer direction for distance/remote learning of many types (e.g., blended learning, fully online learning, synchronous learning, asynchronous learning). as more and more states and school systems mandate classes in all academic areas, online physical education has become more widespread 86 . cpe classes have been the more frequently offered than tpe because cpe's content is more easily adapted to an online format due to its knowledge-centered approach to physical activity promotion. the corona virus disease 2019 (covid-19) pandemic amplified the value of online learning, especially through cpe. teachers and school districts scrambled to provide remote physical education as schools closed. physical education teachers were tasked with providing alternatives to face-to-face learning for students. more than a few teachers and administrators contacted me to get help with materials and training (e.g., webinars, granting access to resources). those who had already been conducting cpe courses made an easy transition when remote learning became imperative. those who had not made the transition embraced cpe because it was a defensible alternative that administrators could accept. all of the benefits of cpe described in this paper have been used to support it during the pandemic. however, questions remain: if cpe is defensible as the -go to‖ program during the pandemic, why has it not been more universally implemented? will it continue to be implemented post-pandemic? as schools adopt online programs, we must answer the following question: -how do we know that students are active in their remote classes?‖ the answer is: -we don't.‖ face-to-face classes are much better for teaching skills and monitoring physical activity. however, video, activity monitors, and other creative options can help us when remote learning is the only option. in the meantime, student activity logs and reported activities on portfolio sheets can be used. students can design their own activity plans and perform them. no doubt, some may not actually do what they describe in their logs. however, students would learn how to plan and what to include in a program. concepts of personal responsibility can be discussed to help students develop ownership of content. they can learn what to do later in life if they choose to do so. a positive outcome with online cpe is the ability of students to do activities in a non-threatening environment, with none of the baggage of locker room bullying, embarrassing shower room events, and lack of time for personal grooming. for some, online programs can be a good option. on the negative side, we have work to do relating to testing and ensuring that the students enrolled in online classes are actually doing the work. using hybrid options that require students to do some activities face to face, especially assessments, would go a long way in solving this problem. the covid-19 pandemic also helped expose the fact that not all students have equal access to the internet and to the computer tools necessary to take advantage of online classes. for example, 14% of k-12 students have no internet access, 17% have no access to a home computer, and as many as 42% face barriers to connectivity 97 . equality options are necessary to make sure that all students have an opportunity to learn remotely. while there is much more that needs to be done in this regard, when access is available, cpe programs provide digital texts and student resources that can facilitate online learning. print materials can be distributed by mail, delivered, or picked up from the school, thus assuring that all students can have access to at least some of the same experiences. the pandemic also has made obvious the lack of equity in providing resources for teaching in physical education. schools that transitioned to the cpe approach often had large class sizes (sometimes 50 or more) and lacked computers, whiteboards, and other digital tools available to teachers in other subjects. texts, student resources, and teacher resources are also required. while physical education programs often have limited budgets, it should be noted that text materials and student/teacher resources for cpe programs are less expensive than purchasing typical sports equipment. it is my contention that cpe provides us with the ammunition to demand instructional equity (e.g., resources). -how you do it‖ matters when delivering a course of instruction. cpe offers many tools for use by professionals as they facilitate student learning. knowing how to use the tools is as important as selecting the appropriate tool for meeting a specific learning objective. some important factors relating to delivery are listed and briefly described in table 4 . as noted in section 3, cpe is differentiated from other fe programs by its use of classroom sessions and text materials. why classroom sessions? different objectives require the use of different methods. when providing instruction with knowledge as the goal, the classroom offers opportunities for best practices for doing presentations (white boards, computers, vetted text materials, videos) and encouraging student discussions. blocks of time are available for in-depth study of content. because conceptual material is covered in the classroom, students avoid standing and listening during activity. textbooks and text materials provide opportunities to learn in all educational settings: in the classroom (texts), in activity sessions (portfolio sheets or workbooks), out of school (e.g., readings, assignments), and online (digital materials). effective middle school programs have used both textbooks and text-based materials (workbooks, portfolio sheets-print and digital), and effective high school programs have used textbooks (print and digital). one new innovation is interactive web text (iwt), which resides on the web and allows text material to be used on all digital devices (e.g., phones, tablets, computers) and virtually anywhere that an internet connection can be made 97 . this is important in situations where students have a variety of digital devices. however, as noted earlier, for all students to benefit they must have access to digital devices and the internet. when teachers implement cpe, the most common question they ask is: -how can i justify having students sit in a classroom when so many students fail to meet national physical activity guidelines?‖ i offer the following answer, using an excerpt from a previous publication: 37 -an overarching goal of physical education programs is to promote lifelong physical activity. to be sure, taking time from physical activity to be in a classroom reduces physical activity on those days. however, cpe is not meant to be all of physical education, rather it is one part of physical education conducted for a limited time period with a limited number of classroom days. the time spent in cpe yields more physical activity later in high school and later in life, more than compensating for the time lost in physical activity while the student is in the classroom. if, in cpe, we can teach students selfmanagement and consumer skills that will help them to be active for a lifetime, the limited classroom time in cpe seems well worth it. the research supports this idea.‖(p. 46). one of the reasons for the low numbers of students taking physical education in secondary schools is exemptions (allowing alternative school activities to count for physical education credit). proponents of alternatives for fulfilling physical education requirements (e.g., band, jr. rotc, athletics) argue that these alternative programs provide activity and therefore are equal to physical education and are acceptable substitutes. physical educators argue that these alternative programs do not meet the criteria for quality physical education such those outlined in this article. regardless of the amount of activity students get in alternative programs, exempted activities such as band, jr. rotc, and athletics do not meet physical education standards and physical literacy requirements. to fulfill a physical education requirement, an online cpe module can be an option. the online module provides the content for meeting all of the requirements for quality physical education (all standards) while allowing alternative activities to count toward physical activity requirements for the class. physical education teachers conduct the online portion of the class and count students as enrolled. to meet the requirements for the cpe portion of the course, additional activities can be assigned to supplement the alternative programs that often provide relatively low amounts of activity 99 . history has shown that for there are steps that can be taken to provide a pathway to success when implementing a cpe program. three very important steps can be taken to ensure teacher commitment and administrative support, provide teachers with professional development opportunities, and ensure the inclusion of all students. teacher commitment and administrative support. for any program to be effective, teachers must be committed to it. a program such as cpe typically requires teachers to attend professional development sessions and to do extra planning and grading. for some, commitment can be difficult, especially for teachers who are also athletic coaches. research indicates that when roles of coach and teacher conflict in terms of time and resources, priority is often given to the "athletics" portion of the job 25. i readily recognized the many coaches are also quality physical education teachers or teachers of other subjects. however, the primary responsibility of teachers is to fulfilling their duties as teacher, in this case committing to carrying out a quality cpe program. administrative support is also essential. based on my observations, too often the head of the physical education department is also the athletic director or the coach of a major sport. if innovative programs such as cpe are to be effectively implemented, it is my view that the department head should be a physical educator who does not have a coaching or other extracurricular assignment. her/his/their assignment is to administer and direct quality physical education programs. part of the administrative assignment must be holding teachers in the program accountable and making sure that they give a priority commitment to the teaching assignment for which the majority of the person's salary is paid. professional development. for those new to cpe, professional development is essential. teachers need updates on new information, including cpe content, methods, assessments, and technology. my surveys from the 44 state conventions that i have keynoted indicate that few secondary school physical educators attend state conventions, often opting for coaching meetings instead. if cpe (and all of physical education for that matter) is to move forward, it is essential that secondary physical educators attend state conventions and other physical education professional development meetings. for students enrolled in physical education teacher education (pete) programs, a special course designed specifically to help them implement cpe should be offered. recommended content for teacher professional development and pete courses includes cpe instruction methods, content knowledge instruction, methods for overcoming barriers to success (e.g., teacher resistance, teacher workload, coach/teacher role conflict, and student resistance) 25 . commitment to inclusion. the help philosophy emphasizes physical education for everyone and allows students to plan personal programs. to be consistent with this philosophy, programs must offer options for all students. cpe programs, for example, are planned using the universal design for learning framework 100 . the framework helps to -improve and optimize teaching and learning for all people based on scientific insights into how humans learn‖ (p. 1). 100 in addition, programs and program planners can commit to diversity, equity, and inclusion consistent with statements such as the following from the american college of sports medicine (acsm): -we implore every human to go about their responsibilities and every other aspect of their daily lives, making decisions with social justice in their hearts and minds. the future of a pluralistic and just world depends on it!‖ (p. 1). 101 park 2 notes that, as physical education develops in the future, -it may be useful to remember that medicine, an ancient and honored profession that many contend is the most valuable and venerated of the 20th century, was in a general state of disarray at the end of the 19th century‖(p. 20). like medicine, the field of kinesiology was in disarray in the mid-20th century but has made great strides in this century. a robust scientific base has been established, including, but not limited to, evidence that regular physical activity not only enhances fitness and performance but also reduces risk of chronic diseases and conditions (e.g., heart disease, diabetes, some cancers, osteoporosis, obesity). the 2020 covid-19 pandemic accentuated the fact that risk of serious complications and death from the virus are substantially higher among those with hypokinetic conditions. other benefits of physical activity (chronic and acute) include enhanced cognitive function, mental health, and wellness. we also have learned much about the science of human performance (mechanical, physiological, psychological, and sociological) and the science of physical education (sport) pedagogy. those of us who did our undergraduate study in health and physical education in the 1950s did not benefit from the science now available to current and future teachers. texts in many of the subdisciplines did not become available until years later. scientific societies and many important journals were just being founded. earlier generations, however, did lay the groundwork for the dissemination of the new science. the modern profession of physical education, if built on a sound disciplinary foundation, has the opportunity to make strides similar to those made by medicine in the last century. it is interesting that the word doctor is derived from the latin word for teacher, docēre. physical educators do not practice medicine, but they teach. like medical doctors, it is important that they have a strong science background and the ability to pass what they know on to their students. cpe provides an effective platform for them to do so. in my 60 years as an educator and researcher, and especially my years as an advocate for cpe, i have often been criticized and labeled as an opponent of skill learning and other worthy physical education objectives. i have also been accused of wanting to take the -physical‖ out of physical education. to be clear, while i do advocate for cpe as 1 important component of a total quality physical education program, i also support participation in physical activities of all kinds. i support all of the objectives of physical education. i applaud quality skills instruction and instructional methods that foster moderate to vigorous physical activity (mvpa) in physical education. i believe in the importance of social-emotional learning programs that foster diversity and social justice. i endorse programs that help students to find meaning and enjoyment in movement and sport. there is room for them all. cpe is not an opponent of these efforts; it is an important partner. cpe programs that emphasize a knowledge base and that are consistent with peak principles do not distract from other programs-they complement them. students who have -learned to learn‖ are not only more likely to use what they have learned-they can generate their own new knowledge that will guide them in the future. as ennis 102 suggests, engaging -students in a quest for knowledge about the effects of exercise on their bodies requires coordinated efforts by scientists and practitioners to build from kinesiology to society‖ (p. 16) . as outlined in this article, there are many reasons for including cpe as part of a quality physical education program. simply put, if secondary physical education is to thrive in the future, we can and should make knowledge-based cpe programs a central component. if we are to become the renaissance profession of the 21st century, we must be strong in our trust in our science (knowledge) and strong in our conviction to chart a new course for the future based on our science. the advice of robert kennedy 103 can guide us: -some people see things as they are and say why? i dream things that never were and say, why not?‖ table 1 . physical education content standards 41 and fitness education benchmarks. 7 national physical education content standards fitness education instructional framework a standard 1: the physically literate individual demonstrates competency in a variety of motor skills and movement patterns. standard 2: the physically literate individual applies knowledge of concepts, principles, strategies and tactics related to movement and performance. standard 3: the physically literate individual demonstrates the knowledge and skills to achieve and maintain a health-enhancing level of physical activity and fitness. standard 4: the physically literate individual exhibits responsible personal and social behavior that respects self and others. standard 5: the physically literate individual recognizes the value of physical activity for health, enjoyment, challenge, self-expression and/or social interaction. technique: demonstrate competency in techniques needed to perform a variety of moderate to vigorous physical activities. knowledge: demonstrate understanding of fitness concepts, principles, strategies and individual differences. physical activity: participate regularly in fitnessenhancing physical activity. health-related fitness: achieve and maintain a health-enhancing level of health-related fitness. responsible personal and social behaviors: exhibit responsible personal and social behaviors in physical activity settings. values and advocates: values fitness-enhancing physical activity for disease prevention, enjoyment, challenge, self-expression, selfefficacy and/or social interaction. nutrition: strive to maintain healthy diet through knowledge, planning and regular monitoring. consumerism: access and evaluate fitness information, facilities, products, and services. a some fitness education instructional benchmarks were edited for brevity. based on school health policies and programs study (shpps) data 23 . table 4 . factors relating to effective delivery of conceptual physical education (cpe). in a mastery environment, the teacher reinforces efforts toward achieving specific learning goals. consistent with the help philosophy, there is an emphasis on the individual (personal learning). students learn and use selfmanagement skills to make personal decisions related to class content. in cpe, fitness assessments are self-assessments and are used as a basis for personal program planning. criterion-referenced, health-based fitness standards provide the basis for personal rather than comparative assessments. additional established assessment guidelines (e.g., confidentiality, not using fitness scores for grading) are adhered to. portfolios (print or digital) that include fitness and physical activity profiles provide evidence of student accomplishment. instruction focuses on higher-order objects as student's advance. students in cpe use previously learned information as a basis for future learning (vertical learning). some repetition is planned to foster mastery. horizontal learning, such as repeating instruction in the same activities, is contraindicated. when committing to a cpe model and adopting materials to carry out program goals, fidelity is important. staying on task is important for program success. note: the 2020 corona virus pandemic accentuated inequities when many schools used online platforms for remote learning. fig. 1 conceptual physical education: the anatomy of an innovation the second 100 years: or, can physical education become the renaissance field of the 21st century? knowledge, transfer, and innovation in physical literacy curricula exercise for a lifetime: an educational effort recreation and dance. health related physical fitness test manual the history of fitnessgram instructional framework for fitness education in physical education fitness for life society of health and physical educators effects of a concept-based physical education on middle-school students' knowledge, motivation, and out-of-school physical activity ‖ more than 30 editions, and a half-century of effort! -an interview with dr. charles b. (chuck) corbin historical perspective and current status of the physical education graduation requirement at american 4-year colleges and universities the progression and characteristics of conceptually based fitness/wellness courses at american universities and colleges youth fitness, exercise and health: there is much to be done commentary on -children and fitness: a public health perspective health-based physical education the pill not taken: revisiting physical education teacher effectiveness in a public health context impact on schools: implications for curriculum physical education and its role in school health promotion a commentary on children and fitness: a public health perspective physical education's role in public health children and fitness: a public health perspective school physical education: secondary analysis of the school health policies programs study viral hepatitis, std, and tb prevention. trends in the prevalence of physical activity and sedentary behaviors national yrbs: 19912017. available at conceptual physical education: a secondary innovation fitness for life personal fitness and you personal fitness: looking good-feeling good foundations of personal fitness fitness for life: middle school a state level update on secondary physical education policies (abstract) teaching physical lifeskills: practical ideas on health-related fitness aptidao fisica e saude nos programas de educacao fisica: desenvolvimentos recentes e tendencias internacionais fitness for life: physical education concepts fitness for life fitness for life canada: preparing teens for healthy, active lifestyles a dozen reasons for including conceptual physical education in a secondary program educating students for a lifetime of physical activity: enhancing mindfulness, motivation, and meaning moving into the future: national standards for physical education, a guide to content and assessment. reston, va: national association for sport and physical education va: national association for sport and physical education; 2004. 41. society of health and physical educators. national standards and grade level outcomes for k-12 physical education concepts of physical education: what every student needs to know fitness for life: teacher's guide physical literacy, physical activity and health: toward an evidence-informed conceptual model implications of physical literacy for research and practice: a commentary international physical literacy association institute of medicine. health literacy: a prescription to end confusion ninth graders' energy balance knowledge and physical activity aehavior: an expectancyvalue perspective impact of education reforms: the quality of florida's high school physical education programs the science of healthful living learning science-based fitness knowledge in constructivist physical education impact of teacher value orientations on student learning in physical education american association for health, physical education, and recreation. youth fitness test manual american assc for health, physical education and recreation texas youth fitness study: a commentary presidential youth fitness program. presidential youth fitness program comprehensive school physical activity programs active and healthy schools fitness for life elementary school: guide for wellness coordinators the spark programs: a public health model of physical education dynamic physical education for elementary school children human kinetics; 2020 center for disease control and prevention. results from the school health policies and practices study an analysis of state physical education policies in physical education school physical activity policy what schools could be mission impossible? reflecting upon the relationship between physical education, youth sport and lifelong participation college physical education: an unrecognized agent of change in combating inactivity-related diseases gender differences in chronic disease risk behaviors through the transition out of high school tracking pedometer-determined physical activity: a 16-year follow-up study can conceptual physical education promote physically active lifestyles? physical activity participation of high school graduates following exposure to conceptual or traditional physical education effectiveness of secondary school conceptual physical education: a 20-year longitudinal study urban minority ninth-grade students' healthrelated fitness knowledge physical activity and fitness knowledge: how much 1-6 grade students know? physical activity and fitness knowledge in middle school physical education (abstract) fitness knowledge, cardiorespiratory endurance and body composition of high school students to move more and sit less: does physical activity/fitness knowledge matter in youth? determinants of exercise among children: ii. a longitudinal analysis health-related fitness knowledge and physical activity of high school students the effects of conceptually based physical education programs on attitudes and exercise habits of college alumni after 2 to 11 years of follow-up changes in healthy behaviour knowledge of rural pupils effects of florida's personal fitness course on cognitive, attitudinal and physical fitness measures of secondary students: a pilot study does cardiorespiratory fitness knowledge carry over in middle school students? two pathways underlying the effects of physical education on out-of school physical activity society of health and physical educators. school reentry considerations: k-12 physical education, health education, and physical activity american academy of physical education (currently the national academy of kinesiology). conceptual physical education. a position statement shape of the nation: status of physical education in the usa physical education website advanced-placement physical education: an opportunity to act a complete guide to sport education motivating middle school students: a health-club approach society of health and physical educators. guidelines for k-12 online physical education fitness for life: middle school interactive web text more than 9 million children lack internet access at home for online learning jrotc as a substitute for pe: really? the udl guidelines physical education curriculum priorities: evidence for education and skillfulness speech at the university of kansas my thanks to ang chen, pam kulinna, bryan mccullick, and hans van der mars for their reviews of early versions of this manuscript. their time commitment and constructive comments are sincerely appreciated. the author declares that he has no competing interests. key: cord-035310-vf0us6hg authors: wu, you; zhang, mingzhen; li, xiaosi; gan, yu; zhao, chao title: augment reality-based teaching practice date: 2020-11-11 journal: biomed eng education doi: 10.1007/s43683-020-00040-x sha: doc_id: 35310 cord_uid: vf0us6hg nan (received 1 july 2020; accepted 28 october 2020) the covid-19 pandemic has shut down society in an unprecedented way. the past spring 2020 semester has witnessed a rapid transition from in-person to online teaching. for teachers and students, such transition has been progressing with challenges, especially for courses that usually require laboratory settings. here, we describe the challenges that we encountered during such transition and discuss a virtual laboratory setting based on augmented reality (ar) to improve online learning. the challenges concentrate on hands-on skill learning, knowledge gain, and social interaction. lack of access to laboratory facilities, a key loss after transition to online teaching, imposes the first challenge particularly in courses requiring a wet laboratory. with a short responding time to covid-19 and a rapid transition to online teaching, we were not able to secure a solution that would enable the students to visualize or mimic the process of data acquisition. instead, we were left with the option that distributes previously acquired data with modifications to students for laboratory reports purpose. consequently, students have to skip experimental setup, which may prevent them from understanding the mechanisms of how the instruments deliver the designated measures. also, students are fed with data to complete laboratory reports, rather than engaging in acquiring data independently or collaboratively. students essentially have the minimal, if not zero, opportunity to strengthen their on-the-ground observational skills. a physical classroom setting enables vivid instruction of concepts/ideas that span into a threedimentional (3d) space with geometry, morphology, and texture features. the instructor could utilize a model or demo to explain during lectures. in an online class, unfortunately, it is converted into a plain, verbal delivery of messages and additional teaching strategy will need to be implanted to stimulate students' conceptual digestion. moreover, it has been shown that distraction becomes easier in audio settings without actual eye contact, 1 a fact that could cause delays for students to receive the messages. also, the online pattern provides minimal physical space change (e.g. classroom from one teaching building to another) but constrains the students in their chosen space and it could worsen tediousness, especially when multiple courses are scheduled in a row with minimal breakups. students show varying degrees of decline in learning satisfaction due to the limited teaching-learning interactions in online classes. 2 two realities impose negative impacts on quality learning. first, instructions from the lecturer and instantaneous responses from students are no longer in an immediate two-way fashion. communications in online classes are largely determined by network speed and real-time performance of the platform (e.g. zoom software). the lecturers could essentially be instructing in a one-way silent mode as responses from students are usually delayed or even lacking. in addition, the use of a camera in an online classroom is generally optional but not required. lacking facial expressions in communication could potentially diminish the message being delivered. 3 in such scenarios, both lecturers and students experience less satisfaction. second, team-work training is essentially not applicable and students reportedly feel that learning in an isolated community does not promote understanding. 4, 5 this reality is deeply challenged in collaborative laboratory projects. even though the availability of online group discussions could serve as a compensating mechanism, the lack of co-operation on the same experimental subject is not replenishable and each student assigned in a group project will have to largely work on his or her own part with minimal or no interaction with partners. to address the above challenges, we propose a novel online teaching solution based on ar technology. the proposed system is shown in fig. 1 . the whole system requires only hardware of mobile device (smartphone or tablet). we expect to implement an ar-based platform with three modules: augmented lab, virtual textbood, and co-lab. we foresee such immersive technologies could provide a ''real'' lecturing and laboratory environment. novelty: first, within immersive technology, we choose ar over virtual reality (vr) due to ar's advantages in cost, accessibility, and user engagement. any student could simply download the application from his or her mobile device and interact with the software as playing an ar game, such as poke´mon go. if it is implemented in vr, a virtual setting that suits the needs of most students will need to be designed, tested, and implemented, a process that will elongate our study and increase cost; additionally, an extra headset, usually pricing at $400~$600, is required thus not widely feasible to implement at home during covid-19. moreover, ar could enhance the user engagement and interaction in a daily house-setting, minimizing of impact of environment changes on students' learning. second, our design explores a new realm of ar application for educational purpose. existing ar technology dominantly focuses on medical education 6 with a demonstration of static visualization anatomy. in contrast, our design focuses on the understanding of engineering laboratory instruments and textbook knowledge with an addition of dynamic practical functionality of social attributes. we will develop an ar module named augmented lab to address the concerns for lacking access to laboratory settings that train hands-on skills. we develop the laboratory by following publicly available resources. 7 benefiting from its open-sources feature, we choose to use the android system as the platform to develop the ar module, we choose unity integrated development environment (ide) as a gaming engine to develop a smartphone application for users. to enhance the visualization, we make augmented models in blender and then export them to unity. we present a preliminary example of this prototype with an ar-based centrifuge (fig. 2, supplementary video) . this virtual centrifuge will be added to real-world household setup (e.g. table). it will allow a set number of students to operate on each student's computer end. in such a scenario, each student will be able to balance his or her microtubes, optimize the centrifuge settings, and run the machine per their experimental protocol (fig. 2b) . in the case of an unbalanced setting (fig. 2c) , the tubes will not be properly centrifuged. more instruments will be implemented within our computational capability to maximize a virtual experimental setup and test. we are currently ongoing the implementation of a virtual textbook to provide better virtual visualization for online teaching. we propose to establish virtual textbooks that are scannable via quick response (qr) codes such that students have access to a 3d demo of the complex concepts such as molecular structure of proteins, polymers, and nucleic acids, etc. we will enhance a scalable visualization for histology such that students can freely zoom-in and zoom-out to visualize pathological microstructure at different scales while reading books or attending online classes. the scalable and rotatable visualization function allows students to visualize the structure with different proportions and angles. such demo could help deepen students' understanding of the textbook and raise students' curiosity in learning new knowledge and building their habits of thinking in 3d or multi-scale visualizations. we will also implement social interaction features to the online teaching software. we will provide an interface with teleconference software to sync ar visualization with online group discussions (one-tomany) and virtual office hours (one-to-one). online group discussions will be added to each course to ensure high-quality communication. for projects that require cooperation, the project leader will be encouraged to organize an online coffee club to enrich the discussion. the virtual office hour will have greater flexibility without specifing a fixed time and location. instead, it could potentially be applied in the format of more frequent email exchanges with instructor, brief but efficient zoom meetings with the instructor or teaching assistant, and more social media communication (if applicable). we practiced in a wet laboratory course in our department, distributing students with previously acquired data to produce laboratory reports, a fashion of behaviorism learning. 8 this leads to reports that lack an in-depth description of experimental protocols and procedure discussions. there was only a minimal training for students to strengthen their observational skills and detect nuances between similar experimental figure 1. system overview. virtual content (virtual lab device or 3d books content) is added to the real world (table). a hardware device (smartphone or tablet) is used to make the content visible for the user. the software also includes interface to teleconference software, such as zoom or skype. procedures. we are thus inspired to develop a virtual co-lab to improve the learning satisfaction. we performed a survey on online learning and expectation for the coming semesters. this survey has been given exempt approval by the institutional review board (irb) committee for the reason that this survey serves as an educational investigation to compare effectiveness of instructional techniques. the inclusion criteria for data analysis are students who participated in an online class in the spring 2020 and attend the fall 2020 semester. we analyze and summarize the survey (n=15) as below in fig. 3 . the three challenges are well supported by survey results. in terms of hands-on skill learning and submitting laboratory reports, only 33% of students rate online classes as equally helpful as traditional classes while 67% rate online less helpful. in terms of knowledge gains, 93% of students do not rate online classes as positive. closely, 87% of students reported that social interaction is heavily impaired and needs improvement. all students express the need for more virtual office hours and online group discussions if online teaching will continue. taken these practices into consideration, 67% of students express expectations for in-person instructions for at least a part of fall semester. in sum, 40% of students rate their spring online learning as positive, albeit the abovementioned challenges were explicitly presented. likely contributing factors to positive experience may include more flexibility in assignment submission, less commute for attending class, and the availability of audio content for off-campus review. we will practice constructivism experiential learning in fall 2020 semester to promote students learning in an active, contextualized process 9 . we also observed that students who had ar-based gaming experience expressed interests in having ar-based virtual labo-ratory settings as a part of new semester, and that 30% of those who had no ar experience showed some interest. we attribute this yes-and-yes-no-maybe-yes phenomenon to that ar experience exposes students with direct visualization and sensory satisfaction, thus anyone may need to have ar experiences to become interested in ar. hence, we will integrate ar into laboratory settings and practice constructivism learning. we set two minimal learning objectives to test whether or not ar-based laboratory enhances learning. first, without operating on actual instruments, ar-lab will deepen students' understanding in setting up and calibrating instruments correctly. for example, students need to virtually balance the centrifuge and load tubes. second, students become capable of identifying virtual experimental readings that cannot be generated due to incorrect operations. upon the implementation of new ar platform, we will assess the success of our initiatives. the attainment of student outcomes of upcoming semesters will be evaluated, scored, and compared with the spring 2020 results. specifically, to assess the success of three ar-based teaching modules, students' laboratory skills, communication skills, shifts in students' attitudes towards online class, the shift in behaviorism/constructivism, and teamwork skills will be evaluated. the detailed performance criteria will include an experimental plan, data collection, data analysis, data presentation, and conclusion. by comparing the student outcomes between with and without ar, we will employ an evidence-based approach to improve the current ar teaching strategy. specifically, we will implement a better ar demo or add new features to the existing demo. in the future, after the covid-19 pandemic is no longer a concern, this practice could still be used as an alternative for those who are not able to make their presence in laboratory. this article is licensed under a creative commons attribution 4.0 international license, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the creative commons licence, and indicate if changes were made. the images or other third party material in this article are included in the article's creative commons licence, unless indicated otherwise in a credit line to the material. if material is not included in the article's creative commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. to view a copy of this licence, visit http://crea tivecommons.org/licenses/by/4.0/. the protocol for the inclusion of student data was reviewed and deemed exempt by the university of alabama's institutional review board (irb). effects of distraction on memory and cognition: a commentary does online education live up to its promise? a look at the evidence and implications for federal policy nonverbal communication: how to use facial expressions and eye contact to enhance teaching and learning. society for information technology & teacher education international conference factors influencing students' perceptions of online teamwork learning and team-based learning practical experience in virtual teams navab n, editors. mirracle: an augmented reality magic mirror system for anatomy education about behaviorism mind and society: the development of higher mental processes publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-030275-saak3ga8 authors: fogg, kaitlin c.; maki, samantha j. title: a remote flipped classroom approach to teaching introductory biomedical engineering during covid-19 date: 2020-08-10 journal: biomed eng education doi: 10.1007/s43683-020-00001-4 sha: doc_id: 30275 cord_uid: saak3ga8 nan 1. knowledge of contemporary issues in biomedical engineering. 2. general knowledge of and ability to apply physiology. 3 . an ability to apply advanced mathematics, science, and engineering to solve problems at the interface of engineering and biology, including those associated with the interaction between living and nonliving materials and systems. 4. an ability to design experiments, make measurements on, and interpret data from living systems. the original course design consisted of five modules (1. pulmonary physiology, 2. cardiovascular physiology, 3. biostatistics, 4. renal physiology, and 5. phar-macokinetics), and three group projects (1. build a matlab model of an artificial lung, 2. design, execute, and analyze an experiment using biosensors, 3. give an in-class presentation on a proposed improvement to a medical device or intervention). traditionally, homework and group projects were completed outside of class, and scheduled class time was used for delivering lectures and having students do hands-on activities and give in-class presentations. additionally, participation was evaluated by students attending class and completing in-class activities. the challenges for transitioning this class to a remote delivery format were the following: 1. engaging students with the lecture content: attending a class in person is a very different learning environment than logging in to watch a lecture online. in order to engage students with the lecture content additional measures needed to be taken to present the information in an informative and interactive way without being able to interact with the students in person. 2. replacing the hands-on learning activities: as the students were no longer able to come into the classroom and actually make measurements on living systems a virtual experiment needed to be created that met learning objective #4. 3. creating a sense of community within the class: this class is the first class that the bioe undergraduates take with only other bioe students and has traditionally helped foster a sense of comradery and belonging to the broader bioe community on campus. additionally, the majority of students were removed from their daily communities and support networks. since students were no longer able to interact with each other in person, additional time needed to be built into the course structure to facilitate peer to peer interactions. 4. fostering teamwork within group projects: traditionally, students developed a sense of cohesiveness through working on their group projects in person outside of class. since that was no longer an option, extra scaffolding needed to be provided in order to encourage students to work together. 5. ensuring accessibility to all students: the majority of students left campus, encountering a wide range of home and learning environments. in order to ensure this course was accessible to students with limited internet access, care giving responsibilities, and/or time restrictions, extra flexibility needed to be built into the course. the overall approach to transitioning to remote delivery was to integrate a flipped classroom course design with remote delivery pedagogy. students observed recorded lectures before class and in-class lecture time was reserved for supported application of the course material. assessments of learning were diverse, providing opportunities to evaluate comprehension of the material, application of the concepts, and deep learning and application of the course content in a team setting (table 1) . instead of delivering lectures over zoom during our scheduled class time, the lectures for the module were recorded and posted in canvas for students to access at least one week before that module was covered in class. this ensured that students could engage with the lecture content at their own pace. lecture slides were made in microsoft powerpoint and uploaded as handouts with two slides per page, ensuring the slides were numbered for later discussion in class. the lecture slides were very detailed, such that students could get all of the information necessary for comprehending the material from reading the slides alone. lectures were recorded in 5-10 min clips, with clear indications in the lecture slides as to which slides went with which narration. a slide saying ''pause'' was at the end of each recorded video, making it easier to record and also easier for the students to follow along. these short videos were clearly labeled by section headings, uploaded along with the lecture handouts, and the section headings were listed under the link to the handout. this made it easy for the students to locate which section of which video held the answers for the homework or quiz associated with that module. students could ask questions on the lecture material at the beginning of class. application of the lecture material was assessed using homework problems, comprehension of the lecture material was assessed using quizzes, and engagement with the lecture material was assessed using participation assignments. tauntimed quizzes consisting of 25 short multiple choice, matching, or bundled true/false questions. quizzes were used to assess student comprehension of lecture material from the current module, and students could go back and engage with lecture videos or handouts while taking the quiz. each quiz could be taken twice participation participation was evaluated on student attendance during scheduled class time participation was evaluated by students completing three activities for each module: uploading a screen shot of a tweet or retweet of an article relevant to the current module, uploading a picture of at least one page of handwritten notes, and choosing between either writing a 250 word summary or posting a one minute video that summarized an element of the current module group projects introduced in class where students had the opportunity to do relevant hands-on activities. group projects were then worked on outside of class worked on group projects in breakout rooms during scheduled class time. time was spent either in manager meetings with the instructor or as a group with the instructor available for questions ken together these course components addressed course learning objective #2: general knowledge and ability to apply physiology. additionally, they collectively addressed the challenge of engaging students with the lecture content. instead of homework being done independently outside of class, virtual lecture time was used to work on the homework together in a discussion format and then homework was turned in as individual assignments. homework assignments helped students meet course learning objective #3: an ability to apply advanced mathematics, science, and engineering to solve problems at the interface of engineering and biology, including those associated with the interaction between living and nonliving materials and systems. homework assignments consisted of long form word problems that required students to apply the mathematical concepts from the lecture notes as well as their prior knowledge of engineering concepts such as mass and energy balances. homework problems were posted online as well as broadcast via screen sharing, and the homework questions were read aloud. students were then placed into breakout rooms of 4-5 students, where they would work on the problem for anywhere from 5-20 min, depending on the length and difficulty of the problem. this helped address the challenge of creating a sense of community within the class. students could hit the ''i have a question'' button and receive assistance from the professor as needed. students would receive a oneminute warning before the breakout rooms were closed, then the breakout rooms would be closed, and everyone would return to the main room. the reaction feature in zoom was used to assess whether or not students completed the problem, and the chat feature was used to ask students if they wanted to share their answer with the class. homework solutions were then worked through line by line, with students asking questions in the chat feature. students then had to work out the problems themselves, either on paper or in microsoft word, and turn them in the following week. lectures were recorded and edited such that the time in the breakout rooms was removed from the final recording. previously, quizzes were taken in class and used to assess comprehension of the lecture material from the previous week. additionally, in previous iterations of this class, quizzes were mostly long form word problems. for remote delivery of a flipped classroom, the quizzes were used to assess student comprehension of the overall concepts within the lectures, whereas the homework was used to assess whether students could apply the concepts to solve mathematical problems. quiz questions were a combination of multiple choice, matching, or bundled true/false questions. quizzes were graded automatically in canvas, were not timed, and could each be taken twice, with students being told which questions they got wrong the first time. answers were shuffled every time so that students had to read through all the options to pick the correct answer. due to concern that students or their family members may fall ill throughout the course, attendance in the zoom lecture was not mandatory. however, in order to assess and encourage student engagement with the material beyond the quizzes and homework assignments, each module had three required participation activities, each accounting for 1% of their overall grade. all participation assignments were graded on completion, not content. this provided a low stakes assignment to engage with the material and earn points to boost their grade. 1. twitter: inspired by randi ulbricht's twitter assignment for her class (biomedical sciences department, missouri state university), students were asked to tweet or retweet an article relevant to the current module with the hashtag #bioe340. they could create a dummy account just for this class, and they needed to take a screenshot of the tweet and upload it to canvas. the overall goal of this assignment was for students to gain awareness of current science news, academic research, and discussions relevant to the topics in each module. additionally, engagement with science twitter allowed students to see that the concepts they were learning in class enabled them to engage with and understand research that is currently being conducted. this assignment helped students meet course learning objective #1: knowledge of contemporary issues in biomedical engineering. 2. notes: students were asked to write a minimum of one sheet of notes, take a picture of it, and upload it to canvas. 3. summary: students were asked to write a 250word summary or a one-minute video summarizing one aspect of the current module. the three group projects are where students must apply their knowledge of mathematics and physiology biomedical engineering society to come up with creative solutions to real-world problems. previously, group projects were introduced in class after students had the opportunity to use various biosensors in class. however, since that was no longer an option, much more detailed information needed to be provided to students beforehand. for all of the group projects, a very detailed grading rubric was provided and was discussed during lecture. nearly half of the class time was dedicated to working on the group projects and zoom breakout rooms were preassigned to be their project groups. additionally, for each group project one day was devoted to ''manager meetings,'' where each group had to fill out a form before class started stating 1. their overall project summary, 2. their recent accomplishments, 3. current challenges, and 4. identify and commit to next steps. both scaffolded and unscaffolded time was provided for each group project-scaffolded being manager meetings, unscaffolded being group work in breakout rooms. these in class group project sessions were provided to help address the challenge of fostering teamwork within group projects. there were specific challenges associated with transitioning each of the three group projects to a remote learning format. for the artificial lung project, since they no longer could interact with a spirometer, a video of how spirometry worked was provided as well as the matlab code that the class would have normally been able to write together in class. since observing individual computer screens would be difficult, the focus of the project was changed from being very matlab heavy to being more focused in creative solutions and communicating their designs effectively. the design and analysis of an experiment using biosensors was the most difficult group project to convert to an online learning environment, since under normal circumstances the students play with the sensors for a class period, decide which sensor they wanted to learn more about, and use it to conduct an experiment on their fellow classmates. since the students were no longer bound by the biosensors owned by the department, students were encouraged to design an experiment using any commercially available biosensor. additionally, the virtual aspect was leveraged as an opportunity to teach students more about sample size calculation and power analysis (since they were no longer limited by how many of their classmates, they could convince to join their study). since they could not actually run the experiment they designed, the focus of the report shifted to generating a hypothesis, designing an experiment to test the hypothesis, presentation of the data, and choosing the correct statistical test. for data ''collection,'' students were asked to use a website that generates data that follow a gaussian distribution given a mean and standard deviation (https://www.random.org/gaussian -distributions/). furthermore, to generate mock data they needed to look up what average values and standard deviations of the metric of interest would be for a control group, as well as what the raw data from the biosensor of interest would look like and how those data would need to be processed in order to get the values reported. this group project helped students meet course learning objective #4: an ability to design experiments, make measurements on, and interpret data from living systems. additionally, changes to this group project helped address the challenge of replacing the hands-on learning activities. the last group project was to give an in-class presentation on a proposed improvement to a medical device or intervention. the prompt for this project stayed the same, but the delivery mode changed. the challenge for this project was replacing the in-class powerpoint presentation and receiving peer feedback. to meet these challenges, students were asked to upload a 10-12 min video that could be any format as long as it was professional. suggested formats: a powerpoint presentation with voice over (similar to the recorded lectures), a video with props pretending you're giving a demonstration of your product, a music video, a sales pitch video to investors, an animation detailing your device, etc. in-class peer review was replaced by a ''gallery assessment'' assignment, where students needed to watch two videos from other groups and provide constructive feedback. in order to ensure it was constructive feedback and not unhelpful criticism, students were provided with a handout detailing how to give constructive feedback. similar to the participation assignments, the gallery assessment was graded on completion. in an effort to understand the environments in which students were trying to learn, students were asked to fill out an anonymous survey indicating if they had unlimited access to the internet, a quiet space to watch lectures, or child or elder care responsibilities. the responses to these questions guided the approach to be as flexible as possible, while still having hard deadlines to encourage students to work on the course steadily throughout the term instead of trying to complete everything at the end. the day that every module was due, an email was sent to students who had not yet turned in their homework or quiz, with a reminder that it was due that night, but they could ask for an extension with no penalties. less than five students asked for an extension for each module, with a different set of students asking each time. students were not required to give a reason as to why they needed an extension, but those that did often cited events that were disruptive to their well-being and/or learning environment. no student cited internet access as a reason for needing an extension. overall, being highly flexible and compassionate with deadlines addressed the challenge of ensuring accessibility to all students. another anonymous feedback survey was taken halfway through the quarter, where students provided feedback on what aspects of the class were going well, what was challenging, what parts of the courses aided their learning the most, and any suggestions for improvement. from this feedback more structure was provided on how to work effectively in groups remotely and more unstructured group project time was built into the last few weeks of the class. this further addressed the challenge of fostering teamwork within group projects. although the delivery of this course differed from previous iterations, overall the course learning objectives were still met by the updated versions of each course component. the only part of the course learning objectives that was not able to be fully met was the ability to take measurements on living systems, since students could not use the biosensors. if this class were to be taught remote again, a small assignment could be created where students took measurements that did not require biosensors such as heart rate or breathing rate on themselves. furthermore, to better replace the hands-on components, virtual labs could be created of a selection of biosensors so that students felt better prepared to choose one for their design of an experiment group project. a flipped classroom approach to undergraduate engineering courses increases engagement with the course content. 4 furthermore, providing the lecture material as both written handouts and recorded videos ensured students could learn the material in a manner that worked best for them. 8 although all of the information was available in the handouts, the majority of the class watched all the videos for each module. lastly, breaking the lectures up into 5-10 min clips compared to one long video had multiple positive effects. students absorb material better when available in shorter clips compared to one long one, 3 and providing the information in well labeled short clips meant students could easily navigate to the video corresponding to the questions they got wrong on their first quiz attempt. the lectures, homework, and quizzes all complemented each other and helped students meet the course learning objectives #2 and #3. approximately 75% of the students attended the zoom lectures where we went over the homework together, and the recorded lectures were watched by over half the students, indicating that even those that came to lectures went back and watched the videos of the homework solutions in order to be able to complete the homework assignments. students did not ask a lot of questions during the homework sessions while they were in breakout rooms, but they did engage with the homework material and ask questions when they returned to the main room to go over the homework solutions. although the students were given the answers, the act of completing the homework themselves in order to turn in individual assignment ensured process comprehension. furthermore, working on the homework in small groups helped build a sense of camaraderie and community within the class. 6 perceived rapport with both instructors and students increases student participation. 2 however, some students provided feedback that they didn't feel challenged by the homework, thus in future iterations of this course the homework might be updated slightly. one option would be to have the students turn in the homework before class, grade based on completion not having the right answers, go over the solutions in class, and then give bonus points for having the correct answers. another option would be to assign one problem per homework that is not gone over in class but is very similar to the most challenging homework question. that way students can see how the method learned from the discussed problem can be applied to a slightly different problem. quiz structure was based off of khan academy due to their well-established format and position in academic learning. not making the quizzes timed allowed students to go through the lecture material at their own pace to find the answers and be mindful of student inclusivity. allowing students to take the quizzes twice encouraged them to identify what they had gotten wrong and find the correct answer, enhancing comprehension. 7 the majority of the students took the quizzes twice, improving their scores to over 90%. cheating was not evaluated as the answers to the quiz questions were in the lecture handouts. this element of the course paired with the prerecorded lectures could be retained for future iterations of this course. the participation activities of engaging with science twitter, taking handwritten notes, and summarizing one aspect of the course material went very well. handwritten notes enhance comprehension, 5 but since the course was being delivered remotely students were not motivated to take such notes. some students uploaded one short page of notes, while others uploaded ten pages or more of very detailed notes for each module. lastly, explaining a concept to someone much like you are teaching someone else, reinforces comprehension. 1 interestingly, very few students chose to upload a video, with the majority writing a short abstract on a topic instead. overall, these three participation activities provided as much as an assessment of attendance for participation if not more and could be retained for future iterations of this course. from the student's point of view the group projects seemed to be the most difficult aspect of this course. there were some elements that went well, and many elements that were improved upon as the course progressed. manager meetings went very well, with students preparing thoughtful questions and provided a good sense of where they were at progress wise. lectures that were devoted to working on the group projects had the highest attendance, whether they were scaffolded or not. students provided feedback that working remotely with group members was very challenging, with issues ranging from finding a time to all meet up on zoom to figuring out how to divide the labor amongst the group members. many of the students thought they needed to be meeting on zoom in order to do any work on the projects, instead of assigning tasks to each group member, working on them individually, and then meeting to put them all together into a final complete project. upon receiving this feedback, much more scaffolding was provided in terms of suggested division of labor and some tips on how to work well in groups remotely. additionally, the first group project had teams of 5-6 students, but there wasn't enough work for 5-6 students to do so many group members expressed frustration that there was an uneven distribution of labor. for the remaining two projects group size was reduced to 4-5 students, which went much better. further efforts to reduce group project frustrations could include monitoring of group dynamics and allowing students to choose their own groups for at least one of the group projects. a survey halfway through each group project could be given to students where they are asked how things are going and what could be improved upon. this would allow the instructor to intervene if there were specific problems arising in one or two groups or broadcast a message to all the groups if they were all experiencing similar issues. as for selecting group members, groups were formed automatically using a tool in canvas that randomly assigned students to groups. this allowed for grades to be entered once per group instead of individually. however, if the students were empowered to select their own groups inter-group dynamics could potentially be improved. overall, despite the challenges of remote delivery, the quarter went well, with students engaging well with the course and all students got high grades across the board on homework assignments, quizzes, participation assignments, and group projects. many elements of this covid-19 version of the course will be kept for future years, whether in person or online. role of expectations and explanations in learning by teaching instructor-student and studentstudent rapport in the classroom short videos improve student learning in online education comparing the effectiveness of an inverted classroom to a traditional classroom in an upper-division engineering course the pen is mightier than the keyboard: advantages of longhand over laptop note taking creating a sense of community in the classroom teaching and learning mathematics through error analysis learning styles and online education. campus-wide inf syst publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations special thanks to the three outstanding graders of the spring 2020 class. the authors have no competing interests. key: cord-260489-9d24cqke authors: shetty, sandeep; shilpa, c.; dey, debayan; kavya, s. title: academic crisis during covid 19: online classes, a panacea for imminent doctors date: 2020-10-17 journal: indian j otolaryngol head neck surg doi: 10.1007/s12070-020-02224-x sha: doc_id: 260489 cord_uid: 9d24cqke introduction: covid 19 made a serious impact on many aspects of everyday life. the world saw a paradigm shift in the education system favouring online learning during the constrains of pandemic. methodology: to assess the attitude of the students towards online learning in subject of ent, we conducted an observational study among 170 third year mbbs undergraduate students of our institute attending online classes through the student portal of our university website. results: our survey revealed students favoured online learning to sustain their academic interest and development during this pandemic. yet, they perceived many challenges during online learning like lack of face-to-face interactions, lack of socialization, distraction by social media, technology related issues etc. students also opted for a combined approach of learning in the post pandemic period. conclusion: this article reflects the challenges faced during online learning and added the innovative methods that can be included to overcome the obstacles of online learning. during this period of covid, one must embrace the alternative to classroom learning to keep up with one’s academic development and can consider an integrated approach of learning after the pandemic. electronic supplementary material: the online version of this article (10.1007/s12070-020-02224-x) contains supplementary material, which is available to authorized users. covid 19 made a serious impact on many aspects of everyday life. world health organisation (who) announced covid19 as a global public health emergency of international concern on 30th january 2020 [1] and declared it a pandemic on 11th march 2020 [2] . the education system is one of the most impacted aspects of routine learning and daily life. the world saw a paradigm shift in the education system favouring online learning during the constrains of pandemic. yet, the effects and efficacy of online education and the capacity to successfully teach digitally is questionable. this sudden and rapid transformation from an environment of conventional learning to virtual learning has made a great impact on the attitude of the students towards learning [3] . in our institute, regular undergraduate online classes are being conducted through student portal of our university website. this enables students to access the digital contents and the recorded lectures even after the class. to assess the attitude of the students towards online learning in subject of ent, we conducted a short survey among third year mbbs undergraduate students of our institute attending ent classes. this will also help us to assess the feasibility and mould our approach towards online classes. an observational, descriptive study based on designed questionnaire was conducted through google forms which was circulated among the third mbbs undergraduate students of our institute. online classes are being conducted. a total of 170 third mbbs undergraduate students participated in the study. the designed questionnaire consisted of set of questions to know the attitude of undergraduate students regarding online vs classroom learning and to know what are the reasons for their likes and dislikes about online and class room learning. initial questions included socio demographic data and the gadget being used for online classes. part a of the questionnaire was based on the opinion of students towards classroom and online classes. part b consisted of statements for which their opinion was obtained by yes/ no/ may be responses and end questions on what kind of approach they would prefer during the pandemic either online or classroom and after this pandemic with the options of online classes, classroom classes and a combined approach were given. our study included 170 students of which 78 were males and 92 were females within the age group of 20-24 years. the average attendance for online class was found to be 96%. 52% of the students opted for using mobile, 36% laptops and 12% of them used tablets as devices for online classes (fig. 1) . part a of the questionnaire (table 1 ) suggested a comparable result in terms of content covered in both online (37%) and classroom learning (41%). 58% of the students felt understanding concepts is better in classroom learning while 52% students felt retention of the topic was better through classroom learning. majority of the students (59%) felt that they had better access to online study materials. students favoured classroom learning in terms of student teacher interaction (54%), punctuality & discipline (77%), acquiring practical skills (90%). students predominantly favoured online classes for doubt clarification (57%). apprehension of exams were nearly equal in both. part b (table 2 ) revealed students preferred online classes in terms of ease of participation, attending classes (91%) and time investment (67%). students felt that the outdoor activities, group projects and development of communication skills were better in classroom learning (82%). 78% of the students sensed that poor internet connections, social isolation and eye strain as the commonest problems faced during online learning. when asked a specified question regarding preference of classes during the pandemic, majority of the students preferred online classes (89%) to sustain their academic interest and development during this pandemic. (fig. 2) . for the opinion of preferred approach of learning after covid 19 pandemic, 31% of students opted for the option of only classroom learning, 16% opted for only online learning where as 53% of the students opted for combined approach of classes (fig. 3 ). since the serious outbreak of this global pandemic covid-19 [1] , majority of the countries practiced lockdown. currently there are in excess of 26 million cases of covid-19. social distancing and restrictive movement policies are being implemented to curb this rising curve of cases [4] . it also has markedly deranged conventional education practices as most of the schools and colleges are being temporarily shut down. this present situation has made us implement an alternative and innovative approach in sustaining academics of medical undergraduates through online classes. due to the constrains of classroom learning in this indeterminate time course of pandemic, online learning has come to the forefront to partly resolve perplexity. online classes are being conducted through the student portal since the outbreak of pandemic. the average attendance for the online classes was found to be 96% over a period of 5 months. most of our students possess a mobile phone, making it the most accessible and feasible platform to attend online classes (52%). 36% and 12% students preferred using laptops and tablets respectively as they might feel more distracted while using mobile phones. students themselves gave the feedback that accessing online classes on mobile was sometimes distracting them. they feel the urge to access social media, check messages, answer calls while using mobile phones which leads to loss of interest and attentiveness during the class. conventional learning has face to face interactions, motivates one to learn, better interactions among student and teacher and most important a feel of togetherness in learning and sharing opinions. [5] some of these aspects lack in online learning. our survey suggested that although both methods of teaching covered almost equal content of a particular topic yet the students preferred classroom learning for understanding and retention of a topic. the students also preferred classroom learning as practical knowledge, punctuality and self-discipline is better acquired through a reciprocal interaction among teachers and student in a classroom. online learning creates a good platform for acquiring newer methods of learning with less apprehension among students in clearing their doubts, ease of participation, good coverage and understanding of the topic by usage of 3d 89% 11% online learning classroom learning animations, less time investment and easy availability of the resources to review the topics through the online portal. [6, 7] our survey revealed a similar preference among the students. they also relished the concept of multiple choice questions incorporation during the class as it added up to their interest and attentiveness during the class. flexibility and lowering the cost of transportation and accommodation with access from any part of the world are engrossing aspects of online learning. [7] long duration of online classes can cause eye related problems, distraction by social media. [3] decreased outdoor activity, group projects, communication skills causes social isolation in an individual. [5] our students also reported that 79% of them lost interest during online classes due to issues with the internet connection when the class went on for more than an hour. sedentary life with decreased outdoor activity and project works was a major disadvantage of online classes as pointed out by 80% of our students. 75% of the students complained eye related issues like eye strain, epiphora and headache while attending long online sessions. technology related challenge and clinical skill training are the two paramount constraining factors. [7, 8] other crucial constraining factors include student interest captivation and emotional wellbeing. these could be overcome by using simulation based training apps, establishing a fast and reliable internet connection, prior training of teachers in using this technology, flexibility in time and a shorter duration of classes. incorporating virtual whiteboards, videos on clinical examination, 3d images, surgical videos if applicable, weekly one to one student teacher counselling sessions, regular feedback from the students may help in overcoming the obstacles and escalating favourability of online classes. in view of present scenario our students preferred online classes during pandemic which is similar to other studies. [3, 6, 9] majority of our students preferred for a combined approach of teaching after the pandemic for their better academic development. this finding is similar to the study done by rajab et al. [10] . one of the participants of our survey rightly said 'there is a difference to play football on ground and in mobile' but in this apprehensive socially distanced period of covid, one must embrace the alternative to classroom learning to keep up with one's academic development. though the replication of classroom learning is not completely attainable through online learning, yet it is a convenient method with ease of participation, sustaining the academics and maintaining the student teacher interaction amidst pandemic. a combined approach can be considered post pandemic for a finer learning with more innovative methods. funding no external funding has been received. conflict of interest the authors declare that they have no conflict of interest. ethical approval appropriate ethical clearance has been obtained from the institute.informed consentinformed consent was obtained from all individual participants included in this study. who: covid 19 public health emergency of international concern who declares covid-19 a pandemic online teaching during covid-19: perception of medical undergraduate students covid-19) advice for the public online learning amid the covid-19 pandemic: students' perspectives student's perception of online learning during covid pandemic online learning: a panacea in the time of covid-19 crisis sustaining academics during covid-19 pandemic: the role of online teaching-learning distance learning in the era of covid-19 challenges to online medical education during the covid-19 pandemic publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations key: cord-272311-91xjkv6m authors: martin, anastasia; blom, iris martine; whyatt, gemma; shaunak, raghav; viva, maria inês francisco; banerjee, lopamudra title: a rapid systematic review exploring the involvement of medical students in pandemics and other global health emergencies date: 2020-09-02 journal: disaster medicine and public health preparedness doi: 10.1017/dmp.2020.315 sha: doc_id: 272311 cord_uid: 91xjkv6m objectives: the role of medical students in the current coronavirus disease 2019 (covid-19) pandemic is rapidly evolving. the aim of this review is to explore the involvement of medical students in past global health emergencies, to help inform current and future scenarios. methods: a rapid systematic review was undertaken, including articles from online databases discussing the roles, willingness and appropriateness of medical student involvement in global health emergencies. data were extracted, appraised and written up as a narrative synthesis. this study was registered with prospero (crd42020177231). results: twenty-eight articles were included. medical students played a wide variety of clinical and nonclinical roles including education and logistics, although medical assistance was the most commonly reported role. challenges included a lack of preparedness and negative mental health impacts. a total of 91.7% of included articles about willingness found medical students were more willing to be involved than not. conclusions: this review shows medical students are capable and willing to be involved in global health emergencies. however, there should be clear protocols for the roles that they play, taking into account the appropriateness. as a rapid review, there were study limitations and more research is required regarding the impact of these roles on medical students and the system. m edical students represent a largely untapped reservoir of potential in global health issues. this group can provide a youth perspective to global health issues, contribute to research in a manner in which busy seniors cannot, and be an addition to the global health workforce in times of need. 1, 2 this last point is the focus of our research. at the time of writing (april 2020), the world was experiencing coronavirus disease 2019 (covid-19); a pandemic with unprecedented social and financial impacts that was placing pressure on healthcare systems around the world. one solution to alleviating this pressure was by asking medical students to help. [3] [4] [5] many medical schools have been suspended or moved to online platforms, leaving students with extra time on their hands; time that can potentially be spent helping. 4, 5 medical students worldwide have already started moving to the frontline. 6 in the united kingdom and brazil, for example, final year medical students have been fast-tracked to join the workforce early. 7, 8 guidance toward medical students helping on the frontline has been published by several organizing bodies, but due to the dynamic situation, this guidance continues to change. 9,10 therefore, it is important to ascertain what exactly would be the best way for medical students to help. the primary role of the medical student is to learn to be a doctor, and deviations from this may have consequences both for medical students and the healthcare system around them. however, having students in the healthcare environment during a pandemic can be an increased burden to clinicians who need to invest time to teach them. consequently, students' education and patients' care may be compromised. 11 currently, there are multiple articles describing the role of medical students in past similar scenarios, their willingness to assist, as well as opinions on what roles medical students should play during covid-19. 12, 13 however, to our knowledge, a systematic review that combines and assesses this information does not currently exist. filling this gap in the evidence base will help better inform what role medical students can play in future global health emergencies, including covid-19. the aim of this review was to systematically assess how medical students can be involved in pandemics and global health emergencies. by outlining the roles medical students have already undertaken or potentially can undertake, the appropriateness of these roles, as well as how willing medical students are to be involved, this review can be used to guide decision-makers to design safe and effective roles for medical students in current and future global health emergencies. a rapid systematic review was used to collate, critically appraise, and synthesize the information, because it answered our question in a timely manner so our research could potentially inform the covid-19 pandemic. prisma guidelines were followed throughout the process. 14 the review protocol submitted on march 30, 2020, accepted april 20, 2020 and can be found in the prospero database [number: crd42020177231]. four independent reviewers (a.m., r.s., i.b., i.v.) conducted a systematic search on march 31, 2020. pubmed, medline, embase, and the global health database were searched for eligible studies with no date restrictions. studies were available from 1946 to march 2020. the full search strategy can be found in appendix 1. bibliographies from eligible articles were screened (ie, snowballing) and the "journal of disaster medicine and public health preparedness" was hand searched through all volumes for further articles by reviewer l.b. papers were initially screened by title and abstract and shortlisted articles were screened for full-text analysis against eligibility criteria by all 6 reviewers and any disputes were discussed. data from eligible articles were inputted into a tailored data collection form which was trialed by 2 reviewers before use. all steps were recorded using the prisma diagram. all studies that discussed roles medical students have played or can play, their willingness to do so, or whether they should do so in an acute global health emergency, including hypothetical studies, were included. "global health emergency" was defined as any event that significantly and acutely affected the capacity or functioning of a health system in 1 or more countries, including but not limited to: infectious disease outbreaks, natural or man-made disasters, and armed conflict. only english language articles were included. the population of interest was medical students, defined as anyone from any country enrolled in a university course (undergraduate or postgraduate) training to be a doctor who has not yet finished their medical education. articles that involved other participants (eg, nursing students) but also mentioned medical students, were included. qualitative research, quantitative research, and mixed methods studies, including systematic, scoping and literature reviews, published editorials, commentaries, and conference abstracts were included. we included gray literature to add a wider perspective. gray literature included unpublished or nonpeer reviewed papers, reports, theses, and technical documentation. 15 studies involving nonacute events and diseases that were deemed nongeneralizable to the covid-19 outbreak, eg, the opioid epidemic and obesity crisis, were excluded. furthermore, studies that did not discuss medical students, eg, studies that exclusively discussed other populations (eg, veterinary, dental, or nursing students) were excluded because they did not include our population of interest. non-english language papers were also excluded due to time constraints of translating these papers. our primary outcome was to collate descriptions of the roles medical students can play in a pandemic. these were predefined in the data extraction proforma as: clinical assistance; testing; helplines; triage; raising awareness; or "other," if a different role was described, which the reviewer then specified. our secondary outcomes included the level of willingness of medical students to help, factors affecting willingness, and appropriateness of the role. data were extracted using the predefined standardized form and included: article, author, year, journal, country, article type, article design, aim, area studied (role/willingness/appropriateness/preparedness), article population, setting/context, the number of participants, methodology, outcomes, key findings, relation to past global health emergency (if applicable), comments on the role; comments on willingness, comments on appropriateness, and critical appraisal. articles were critically appraised globally and briefly judged for risk of bias; however, the full use of quality assessment tools was not feasible due to time constraints. all included papers were critically appraised using the reviewers' judgment, expertise, and by means of discussion among the 6 researchers. the design, outcome measure, and whether the study was peerreviewed or a gray literature study played a role in guiding whether articles are of high or low quality and are reported on in the discussion section. data were analyzed thematically and written up as a narrative synthesis. our search identified 802 articles as well as 28 from extra sources. a total of 365 articles were screened for title and abstract, and 323 of those were excluded. sixty-six articles were screened for the full manuscript, and 28 of those met the inclusion criteria ( figure 1 ). most of the included articles had good quality of evidence but unpublished gray literature (eg, expert opinion pieces) had poorer evidence quality. table 1 shows the baseline characteristics of all included articles. seventeen articles discussed roles, 12 discussed willingness, and 13 directly discussed appropriateness (chen e, goodman kw, fiore rn. involving medical students in disaster response: ethics, education and opportunity [unpublished pdf]. 2014;1-17). as well as 13 articles purely discussing medical students, articles also examined nursing, pharmacy, and dental students. the views of medical students across multiple years were examined with an even spread. the context of the global health emergencies is detailed in table 1 , with 13 articles reporting on past situations that happened and 15 articles discussing hypothetical scenarios (table 1 ). seventeen of the 28 included articles discussed the roles of medical students during global health emergencies ( table 2 ). the most commonly reported role was providing clinical assistance (n = 12). in 2 of these articles, medical students acted as junior doctors. 18, 40 in 1918, during the outbreak of the spanish flu, medical students acted as nurses and interns because the usual medical workforce were away assisting the war troops. 40 during the kashmir earthquake in pakistan, medical students were sent to less accessible places compared with senior doctors due to their young age. 18 28 in 2010, after an earthquake, a tsunami hit chile and medical students provided medical care in remote areas. 39 recently, it has also been hypothesized that medical students can provide clinical care by distributing medication to hospital staff and providing manual ventilation. 20, 31 the role of medical students as educators through raising awareness was also highlighted (n = 3). in 1987, during the hiv epidemic, medical students developed a peer-to-peer teaching system to raise awareness and contribute to better patient care. 21 during the severe acute respiratory syndrome (sars) and ebola outbreaks, students held (online) campaigns to promote healthier behavior. 16, 37 other reported roles entailed nonclinical assistance. in 1945, in the immediate liberation of belsen concentration camp, medical students set up an acute care hospital for emergency treatment of the inmates waiting to be transferred and a pharmaceutical dispensary. 42 during the 2001 wtc terrorist attack, medical students participated as "runners" to carry information between triage stations, prepared food for rescue workers, worked in emergency hotlines to provide information to families of victims, fundraised, donated blood, and assisted in psychiatric disaster services (chen e, goodman kw, fiore rn. involving medical students in disaster response: ethics, education and opportunity [unpublished pdf]. 2014;1-17). 28 medical students who were further in their medical education were more likely to provide medical assistance (n = 6). 18, 19, 22, 27, 28, 40 furthermore, the challenges faced by medical students in their roles during the global health emergency was discussed (n = 2). these challenges included: a lack of supervision, treating children, and prioritizing medical need. 18, 40 the impact on the mental health of medical students was also highlighted, both during and after the emergency event (n = 3). 28, 29, 39 for example, during the provision of clinical care during the 1918 influenza epidemic, medical students experienced psychological distress during their fieldwork, such as fear, anxiety, depressive symptoms, despair, and panic. 40 an article on the 2001 wtc terrorist attack suggested that the type of role played appeared to correlate with how much the students were affected. it was found that students working in fundraisers and hotlines had significantly higher symptoms. medical students who assisted in medical care at hospitals had the fewest symptoms and a greater sense of empowerment. 28 the same article also concluded that involvement in the disaster was associated with a reinforced desire to become a physician. 28 twelve of the included articles evaluated the willingness of medical students to help in disaster situations (table 3) . most (n = 11) reported quantitative data on the level of willingness of medical students to help in a disaster as shown in table 3 . the percentage of medical students willing to help ranged between 36% and 92.5%, with 9 of the 11 articles reporting a willingness percentage greater than or equal to 50%. one article reported the willingness of medical students to be involved on a different scale but equated it as "moderate willingness." 17 one article did not provide quantitative data but reported that "the majority of students responding to the survey were willing to respond to disaster events." 32 seven of the included articles provided reasons to explain the medical students' level of willingness. obligation or social responsibility was stated most commonly, along with altruism, as a reason why medical students were willing to be involved. concerns for personal health and safety, as well as concerns for family health and safety, were the 2 most commonly stated reasons for students being unwilling to help (table 4 ). some (n = 8) also discussed factors that affected the willingness, which are described in table 5 . when comparing willingness levels with medical students' confidence and knowledge in the final 2 y of medical school, willingness was seen to be proportionally higher than knowledge. 17, 24, [34] [35] [36] one article reported over 59% of students willing to help, but only 23.7% of students believed they have the skills to help. 23 thirteen articles discussed appropriateness of the roles. this included looking at medical students' confidence and preparedness for specific roles. five studies measured students' skill levels; confidence to deal with emergency outbreaks ranged between 23.7 and 53.2% (table 6 ). 23, 26, [33] [34] [35] three articles found that students' willingness was high, but when compared with their self-perceived knowledge, they did not feel prepared. [33] [34] [35] the perspective study by starr describes "for me and my classmates, knowledge of the disease we were to face so soon was limited to the contents of that 1 lecture." 40 the need for disaster management training was also highlighted (n = 2), due to the lack of preparation of students. 18, 39 only 3 articles reported training their students before carrying out their role, and reyes described volunteers as being "quickly trained." 30.31.39 there are ethical issues of involving medical students in global health emergencies, including students' safety, medico-legal issues, and health insurance of students. 22, 29, 30 eastwood et al. specifically highlighted the importance of students to be well informed and be able to make the decision of being involved themselves. 22 the review outlines the past involvement of medical students in global health emergencies. this can help to guide decisionmakers in choosing appropriate roles that medical students are willing to do and are prepared to carry out in a global health emergency. historically, medical students have been involved in the response to global health emergencies and pandemics in a variety of ways. 16, 18, 21, 28, 37, 38 our results identified a range of roles that students have played in the past, thus highlighting the potential roles medical students are able to play in the current covid-19 pandemic or future global health emergencies. unlike members of the general public, medical students have attained several relevant clinical skills during their years of training, which can be useful in a situation where the healthcare system is under pressure. for example, in 1 study, students assisted the resuscitation team, a skill students learn during medical school. 28 the current data suggest that the most common roles played were, in fact, clinical roles. however, nonclinical roles, such as runners to carry information, assisting in hotlines, and psychiatric services were also described (chen e, goodman kw, fiore rn. involving medical students in disaster response: ethics, education and opportunity [unpublished pdf]. 2014;1-17). 28 a role that should not be neglected, however, is the normal role of a medical student-to learn. it is important to consider whether these alternative roles are appropriate in a specific situation. of interest, 1 study concluded that being involved in such situations helps strengthen students' desires to become physicians, suggesting that involvement does not only help to serve healthcare systems but the students themselves. 28 other studies have also found clinical volunteering to be perceived positively by medical students, helping them develop clinical skills and collaborate with other healthcare professionals. 43 the range of roles described in the included articles highlights the versatility of medical student involvement. for example, 6 articles also described that medical students further in their medical education were more likely to provide medical assistance. 18, 19, 22, 27, 28, 40 one study found younger students were involved in more educational roles as first year students helped by raising awareness for hiv. 21 this demonstrates that involvement in the response to global health emergencies is not limited to advanced medical students and that younger students, even though they may have less clinical knowledge, can still offer a valuable contribution when given an appropriate role. furthermore, the variety of roles can allow certain challenges to be avoided. for instance, if a hospital is deemed to have a lack of supervisors, a commonly faced challenge even during nonemergency times, then medical students can get involved in a nondirect clinical role. 18 several challenges were described as students undertook these roles, including a lack of supervision, a lack of experience, and a negative impact on their mental health. 18, 28, 29, 39 pro-actively addressing these challenges will not only enable any involvement medical students have in the current or any global health emergency to be efficient and effective, but will also reduce any detrimental effect on the medical students' mental health and medical education. our results demonstrate that such involvement in highly stressful situations can lead to anxiety, depressive symptoms, and emotional distress. 28, 29, 39 consequently, stress can lead 25 increased severity (very dangerous and contagious compared to very infectious) 35, 36 greater level of knowledge 17, 19, 34 pediatric patient care 34 history of a severe illness to them or family member 25 respiratory transmission compared to contact 37 less than 24 y old 25 medical role (compared to admin role) 23 more than 100 h of previous volunteering 25 natural disaster (compared to an infectious disease) 23 ppe availability 36 training before work 23 travel compensation 23 well run organization 23 table 6 overview of the students' confidence in their skills, knowledge, and education to deal with global health emergencies article id context (self-perceived) percentage of students gouda (2019) 23 sufficiently skilled to respond to an emergency outbreak 23.70 kaiser (2009) 26 sufficiently skilled to respond to natural disasters 51.60 sufficiently skilled to respond to pandemic influenza 53.20 mortelmans (2009) 33 sufficiently educated to help in h5n1 pandemic 46 mortelmans (2013) 34 sufficient knowledge on disease management in disaster situations 18 mortelmans (2015) 35 sufficient skills to deal with infectious outbreak 39.20 academic commitments 24 cv improvement and gaining future contacts 23 family health and safety concerns 26 increase self-confidence in sim situations 23 family/social commitments 24 obligation/social responsibility 23, 25 financial implications 24 professional and skills development 23 inefficiency 26 reduce guilt about less fortunate 23 lack of confidence in skills 19 sense of ethics 25 lack of information 24 needless sacrifice 26 personal health and safety concerns 19 work commitments 24 disaster medicine and public health preparedness to a higher prevalence of university student dropout. 44 to avoid negative mental health impacts, adequate support must be provided to medical students if they are to be involved. this also highlights the need for further research that explores the short-and long-term effects on the students involved, and more specifically, the impact on their mental health and their future careers. ultimately, the roles of medical students might be very context-specific. a limitation of this review is the lack of data providing specific recommendations on the most suitable role of medical students in the differing types of global health emergencies. a further limitation is the paucity of data evaluating the roles medical students have previously undertaken. it is important to ascertain how effective medical students were in the emergency response and whether they made a real impact. therefore, further research into the effectiveness of medical student involvement would be beneficial, and perhaps comparing this to global health emergencies that have not involved medical students. eleven of the 12 articles (91.7%) discussing willingness reported a greater willingness than unwillingness among medical students to be involved in the response to pandemics and global health emergencies. only hwang et al. found the opposite pattern of only 36% of students willing to be involved. 25 this article focused specifically on whether medical students would enter a hypothetical closed area with a highly infectious disease and high fatality rate. of interest, only 9% of those not wanting to enter stated fear of safety as their reason, and in contrast, 44% were unwilling to enter due to inefficiency. 25 this highlights the importance of organization when adding medical students to the workforce. obligation to help and social responsibility were the 2 most commonly stated reasons for wanting to help in such situations (chen e, goodman kw, fiore rn. involving medical students in disaster response: ethics, education and opportunity [unpublished pdf]. 2014;1-17). [23] [24] [25] this suggests that medical students believe they "should" volunteer, rather than "want" to. in turn, this can lead to students putting pressure on themselves to help, when they may not feel ready to do so, which will potentially hinder any positive effect they may have on the disaster situation, and could be dangerous for patients. making students aware of both their mental and physical capabilities is essential to ensure that they only volunteer if they are prepared. certain factors encouraged student involvement, including a greater level of knowledge and being trained and receiving travel compensation. the most common discouraging factor was the severity of the event or outbreak. 17, 19, 23, 34, 35 it is important to consider these factors, specifically by emphasizing encouraging factors, such as travel compensation and training before work. this, along with minimizing any discouraging factors where possible, will ensure the maximum number of medical students will be prepared to volunteer and, thus, increase the size of the workforce. it is vital to support medical students to gain the extra help that is needed, as medical students would be going beyond their role of simply learning and being a student in an already stressful situation. considering ways to decrease stress can help reduce negative mental health effects both during and after the event. factors such as age and previous volunteering experience can be used to help target a specific cohort of medical students to volunteer with specific roles. for instance, initially approaching medical students who are already experienced with volunteering, before approaching the remainder of the medical student cohort if required, may be a more beneficial way of organizing a volunteering scheme. several studies found students reported that their willingness was higher than their knowledge and readiness. 17, 23, [33] [34] [35] this illustrates the need to better prepare medical students, who can indeed be a great addition to the workforce in disastrous situations. knowledge about a specific disease is vital to students undertaking both clinical and nonclinical roles, such as raising awareness. 45 this preparation could be spread throughout their medical school education, or delivered immediately before a specific role. 46, 47 the latter, however, would require further resources acutely during a disease outbreak, which may not be available. the studies discussing willingness were limited by their methodology as they only measured medical students' responses and did not compare with the general population. furthermore, most studies were hypothetical situations and, therefore, students may answer differently if the real situation arose. furthermore, there was no standardized survey of measuring willingness between all studies and this, along with the studies not comparing to a general cohort, limited the ability to carry out a meta-analysis. three articles reported that, although medical students may have been willing to help, they may not have necessarily felt prepared or felt that they knew enough. [33] [34] [35] the ethical considerations of the roles of medical students must be thoroughly explored before students are invited to help, especially where patient care may be compromised. during times of nonglobal health emergencies, the primary role of medical students is education and to learn to be a doctor, which can take between 4 and 7 y. the need to help the workforce in times of crisis must be balanced with the educational and wellbeing needs of the medical student to complete their training. whereas "learning on the job" can be an invaluable experience, the safety of both medical students and their patients must be considered. however, there was considerably less data on this aspect in the evidence base. this rapid systematic review provides an overview of the previous involvement of medical students in global health outbreaks, which, to the authors' knowledge, no other systematic review has previously discussed. the inclusion criteria were kept broad, which allows the results to be generalizable for other future global-health emergencies, as well as the current covid-19 pandemic. the predefined aims and objectives were answered and previous roles and their appropriateness were summarized. although completing a rapid review may help inform the current covid-19 pandemic, this study design also has limitations, especially due to time constraints. first, although each included article was discussed and data were extracted by 2 authors, no full formal critical appraisal or risk of bias tool was used. second, due to the data not being sufficiently homogenous, a meta-analysis was not possible. third, gray literature was included, which may yield lower quality evidence; however, on balance, it provided an invaluable insight into previous roles that had taken. the fact that medical students are the authors of this review is both a strength and a limitation. the authors are themselves experiencing the possibility of being involved in the current covid-19 pandemic and, therefore, have insights into the difficulties and lack of data about this process. much has been written on previous roles students have taken in previous situations, and this can be used to inform future policy regarding covid-19 and future global health emergencies. when designing the role itself, the willingness and preparedness of the medical students should be strongly taken into account. this will ensure specific roles are safe and within appropriate student competencies. medical students should be given clinical roles within reason, and educational and social media roles may be given for the less clinically confident. this highlights the importance of co-production and including students themselves when planning these roles. the authors of this review experienced firsthand the difficulty in defining roles for medical students in the current covid-19 pandemic, within their own universities and hospital trusts. this highlights the lack of robust policy and knowledge surrounding this topic, and the consequent unnecessary delay in the use of a skillful resource. we thoroughly encourage governments to have predefined policies for medical schools if such events arise again. an infographic was made to summarize the findings of this review (figure 2 ). future research to both describe and evaluate the effectiveness of medical students' roles during the current pandemic should be carried out to help guide future pandemics. this research should also encompass the safety of these roles and the shortand long-term effects on the medical students themselves. furthermore, there is a dearth of literature on the ethical aspects of medical student involvement and duty during such situations. this has been discussed when looking at residents/ junior doctors participating in such situations, but not for the medical student cohort. 48 finally, this area of research can be expanded to provide information on the role of other healthcare students in global health emergencies, namely nursing, pharmacy, dental, and veterinary students. medical students are a willing and resourceful potential addition to the healthcare workforce during global health emergencies. their involvement is vast and many roles have been identified; however, adequate and proactive support must be provided to help them overcome any challenges they may face. choosing the perfect role is very subjective to each emergency. therefore, it is vital to consider available resources, students' opinions, and the nature of the emergency itself when planning roles. future research should be targeted at filling important gaps in the literature discussed above, including evaluating the effectiveness of different roles undertaken by medical students in global health emergencies and the ethical issues regarding the appropriateness of the medical students' involvement. the international federation of medical students' associations. ifmsa policy proposal meaningful youth participation medical student research: is it necessary and beneficial? covid-19: medical students to be employed by nhs as part of epidemic response uk universities switching to online lectures and exams. the guardian guidance on medical students' clinical participation: effective immediately. association of american medical colleges covid-19: medical schools given powers to graduate final year students early to help nhs covid-19: medical schools are urged to fast-track final year students dispõe sobre a antecipação da colação de grau para os alunos dos cursos de medicina, enfermagem, farmácia e fisioterapia, como ação de combate à pandemia do novo coronavírus -covid-19 guidance on medical students' participation in direct patient contact activities. association of american medical colleges statement of expectation: medical student volunteers in the nhs amee charter for medical educators. an international association for medical education a bold response to the covid-19 pandemic: medical students, national service, and public health the role of medical students during the covid-19 pandemic preferred reporting items for systematic reviews and meta-analyses: the prisma statement the use of grey literature in health sciences: a preliminary survey ebola prevention to recovery: a student-to-student social media campaign mohamed ibrahim mi. assessment of knowledge, attitudes, and readiness to practice regarding disaster medicine and preparedness among university health students why medical students should be trained in disaster management: our experience of the kashmir earthquake preparing for the next pandemic flu: the role of the medical student -a view from within emergency prophylaxis throughput study: an evaluation of medical students' ability to effectively assemble and distribute mass prophylaxis to hospital employees and staff at a closed point of distribution aids education and a volunteer training program for medical students what should the dean do? attitudes of medical students toward volunteering in emergency situations medical students and pandemic influenza would medical students enter an exclusion zone in an infected district with a high mortality rate? an analysis of book reports on 28 (secondary publication) perspectives of future physicians on disaster medicine and public health preparedness: challenges of building a capable and sustainable auxiliary medical workforce medical students' participation in the 2009 novel h1n1 influenza vaccination administration: policy alternatives for effective student utilization to enhance surge capacity in disasters the medical student experience with disasters and disaster response h5n1 influenza pandemic planning: should medical students be left out? the challenges of "continuing medical education" in a pandemic era training medical students in bag-valve-mask technique as an alternative to mechanical ventilation in a disaster surge setting disaster medicine and public health preparedness of health professions students: a multidisciplinary assessment of knowledge, confidence, and attitudes are belgian senior medical students ready to deliver basic medical care in case of a h5n1 pandemic belgian senior medical students and disaster medicine, a real disaster? dutch senior medical students and disaster medicine: a national survey health care student knowledge and willingness to work in infectious disease outbreaks sars and its effect on medical education in hong kong strategies in disaster preparedness for the next generation of healthcare providers students' response to disaster: a lesson for health care professional schools influenza in 1918: recollections of the epidemic in philadelphia perception, attitudes and knowledge regarding the 2009 swine-origin influenza a (h1n1) virus pandemic among health-care workers in australia rescue of the remnants: the british emergency medical relief operation in belsen camp 1945 learning through service: student perceptions on volunteering at interprofessional hepatitis b student-run clinics trajectory of university dropout: investigating the cumulative effect of academic vulnerability and proximity to family support are saudi medical students aware of middle east respiratory syndrome coronavirus during an outbreak? planning and incorporating public health preparedness into the medical curriculum using an h1n1 vaccination drive-through to introduce healthcare students and their faculty to disaster medicine it is time to prioritize education and well-being over workforce needs in residency training or student doctor[title/abstract] or medical student*[title/abstract])) and (pandemic[title/abstract] or disease outbreak[title/abstract] or disaster medicine[title/abstract] or epidemic[title/abstract] or acute respiratory infection[title/abstract] or severe acute respiratory syndrome[title/abstract] or global health emergency ovid: ((medical student or student doctor or medical student*) and (pandemic or disease outbreak or disaster medicine or epidemic or acute respiratory infection or severe acute respiratory syndrome or global health emergency or public health emergency)).ab,ti all authors have completed the icmje uniform disclosure form at http://www. icmje.org/coi_disclosure.pdf and declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 y, no other relationships or activities that could appear to have influenced the submitted work. the lead author affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained. key: cord-028461-8he5e5pu authors: rubio, miguel a. title: automated prediction of novice programmer performance using programming trajectories date: 2020-06-10 journal: artificial intelligence in education doi: 10.1007/978-3-030-52240-7_49 sha: doc_id: 28461 cord_uid: 8he5e5pu online programming courses have become widely available and host thousands of learners every year. in these courses, participants must solve programming exercises by submitting partial solutions and checking the outcome. the sequence of partial solutions submitted by a student constitutes the programming trajectory followed by the student. in our work, we define a supervised machine learning algorithm that takes as input these programming trajectories and predicts whether a student will successfully complete the next exercise. we have validated our model with two different datasets: the first one is a set of problems from the online learning platform robomission with over one hundred thousand exercises submitted. the second one comprises one hundred thousand exercises submitted to the hour of code challenge. the results obtained indicate that our model can accurately predict the future performance of the students. this work provides not only a new method to represent students’ programming trajectories but also an efficient approach to predict the students’ future performance. furthermore, the information provided by the model can be used to select the students that would benefit from an intervention. online programming courses have emerged as a popular way to introduce students to programming [1] . these courses present several advantages: they are easily accessible, and students face interesting challenges. unfortunately, it is not feasible to provide individual support to each student due to the large number of students enrolled in these courses. automatic systems capable of providing adaptive support could enhance the students' experience and improve their success rate [2] . in order to develop these automatic systems, there is a need to develop models capable of detecting students that will likely fail [3] [4] [5] . these models could use the large datasets that students generate when completing programming tasks [6, 7] . students usually submit several partial solutions before solving a task, creating a programming trajectory for each exercise [8, 9] . these programming trajectories can be analyzed by machine learning systems to find general patterns [10] . in this study we present a supervised machine learning model that predicts the student future programming performance. the model takes the programming trajectory followed by the student and estimates the probability of the student successfully completing the next exercise. the model has been validated using two different datasets obtained from two different online programming environments, robomission [11] , and the hour of code challenge from code.org [12] . our results indicate that this model can predict accurately whether a student will be able to successfully complete a programming exercise. the information provided by the model can be used to rank students in terms of their performance. using this ranking one can automatically select a group student that would benefit most from an intervention. in this study we worked with two different datasets. the first dataset is a set of programming trajectories submitted by students while completing one exercise in the hour of code challenge [13] . additionally, for each student the dataset contains information about whether the student successfully completed the next task. the exercises and their solutions are shown in fig. 1 . piech et al. [8] describe this dataset in more detail. the second dataset comprises 85 programming tasks from the robomission programming platform. effenberfer [14] gives a thorough description of the dataset. our goal is to generate a supervised machine learning algorithm capable of predicting whether the student will successfully complete the next exercise. to this end we will use the programming trajectories followed by the students t = {w 0 , w 1 … w n }. where w 0 is the state before the student starts to work, w i are the code snapshots submitted by the student and w n is the last snapshot. the training phase is straightforward: all the programming trajectories present in the training dataset are assembled into a tree. different branches of the tree contain information about different programming trajectories. figure 2 describes the process to integrate a new trajectory {w 0 , w 1 , w 5 } into a tree. for each code snapshot present in the trajectory we check if there is a branch in the tree with matching snapshots. if there is such a branch, we follow it while the partial solutions match. as soon as we find a partial solution (w 5 in this case) that is not present in the branch, a new branch is created. once we have processed all the student trajectories to generate the tree, we store in each node the relevant parameters of the students that ended their programming trajectories in that node. in this study we stored the proportion of students that successfully completed the next exercise. after assembling the tree, we can estimate the probability that a new student with trajectory t i will successfully complete the next exercise. if we want to classify the student, we only need to compare this probability with the threshold that we have selected. we have selected the receiver operating characteristic (roc) curve [15] and the area under the curve (auc) to measure the performance of the classifier. we have used a 10-fold crossvalidation [16] stratified over students to compute them. we will compare our model optimal performance with the results of a simple baseline model. our baseline model expects the performance of both tasks, the one taken as input and the predicted one, to be the same. we start examining whether our model is successfully detecting students who fail the next exercise in the hour of code challenge. the left side of fig. 3 shows that the roc curve is systematically above the identity line (y = x). the area under the curve (auc) of our model in this case is 0.77, with a 95% confidence interval (0.77-0.79). both the auc and the confidence interval are greater than 0.5, indicating that our model is performing better than a random classifier. figure 3 also contain the main results for the baseline model and the optimal threshold. we can see that the baseline model is much closer to the bottom left corner of the figure than the optimal threshold. the right side of fig. 3 shows the auc obtained for each task in the robomission dataset versus the number of students that attempted each task. we performed a loess regression [16] looking for a correlation between auc and the number of students. from the graph we can conclude that there is no such correlation. however, the variability of auc values depends on the number of students. when the number of students is below 500 the auc values show high variability. for values over 500 the variability decreases markedly. in this study we present a machine learning algorithm able to predict the future performance of novice programmers using their programming trajectories in just one exercise. the output of the model can be used to rank students according to their predicted performance. the data used by the model can be easily obtained in online programming environments. we have validated our model using two different datasets from two online learning platforms. our results indicate that the model can classify students with reasonable accuracy. we have also found that the average performance of our model seems to be independent from the number of students attempting the task. codewebs: scalable homework search for massive open online programming courses educational data mining and learning analytics in programming: literature review and case studies lightweight, early identification of at-risk cs1 students a robust machine learning technique to predict low-performing students evaluating neural networks as a method for identifying students in need of assistance clustering and visualizing solution variation in massive programming classes data-driven hint generation in vast solution spaces: a selfimproving python programming tutor autonomously generating hints by inferring problem solving policies programming pathway clustering using tree edit distance stereotype modeling for problemsolving performance predictions in moocs and traditional courses towards making block-based programming activities adaptive hour of code: we can solve the diversity problem in computer science learnable programming: blocks and beyond blockly programming dataset proc: an open-source package for r and s + to analyze and compare roc curves the elements of statistical learning. sss key: cord-272026-2zb7btjd authors: benedictis, luca de; leoni, silvia title: gender bias in the erasmus network of universities date: 2020-09-15 journal: appl netw sci doi: 10.1007/s41109-020-00297-9 sha: doc_id: 272026 cord_uid: 2zb7btjd the erasmus program (european community action scheme for the mobility of university students), the most important student exchange program in the world, financed by the european union and started in 1987, is characterized by a strong gender bias. female students participate to the program more than male students. this work quantifies the gender bias in the erasmus program between 2008 and 2013, using novel data at the university level. it describes the structure of the program in great detail, carrying out the analysis across fields of study, and identifies key universities as senders and receivers. in addition, it tests the difference in the degree distribution of the erasmus network along time and between genders, giving evidence of a greater density in the female erasmus network with respect to the one of the male erasmus network. at its 30 th anniversary celebrations in 2017, the erasmus program counted more than 9 million participants since its creation, increased to more than 10 million in 2018. the program, which allows its participants to study or take an internship in a foreign country, 1 has become very popular among university students whose participation is increasing year after year. its popularity has made it a true cultural phenomenon, and, given the successful outcome of erasmus+, the european commission has proposed, for the 2021-2027 plan, to double the funds allocated to the program in order to support the mobility of 12 million people, making the program more inclusive, allowing the participation of those coming from disadvantaged families background and less inclined to international mobility. 2 with its objective of inclusion, the program has also become a cornerstone for equal opportunities, with many of its projects, for example, directly aimed at the promotion of gender equality. nevertheless, the participation in the erasmus program itself is characterized by a remarkable and long-lasting gender bias, that favors women, since its launch. evidence shows that the number of women participating in the program has been higher than that of men up to the 1990s (maiworm 2001) . this gender gap is reported versities hosting and sending female and male students abroad. we find evidence of gender bias in favor of women, with a ratio between female and male links equal to 1.338 in 2008 and 1.329 in 2013, showing an overall persistence of the bias. compatibly, the graph of female flows presents a higher density than the one of male flows, with a higher number of both active nodes and arcs. considering the incoming flows of students, the most central node in the female graph is the universidad de granada, in spain, confirming its centrality in both years considered; in the graph of male flows, the universitat politecnica de valencia is the most central in 2008, while the top position is reached by the universidad de granada in 2013. considering the outdegree centrality, the univerzita karlova of prague and the universidad de granada are the most central in 2008 respectively for the male and female flows, while again the universidad de granada is on top of both ranks in 2013. further gender differences are made evident through the analysis: almost half of the university exchanges are gender specific, involving only males or only females, the level of inter-gender bilateral university flows is around 50%; the network of female flows tends to be characterized by a higher level of reciprocity and homophily; the female erasmus network presents a lower number of isolated universities and higher connectivity; the female network also shows a higher prevalence of sending universities over receiving universities, unlike the male one. thirdly, gender imbalance is further studied in terms of the degree distribution of the erasmus network. considering the directed and unweighted erasmus network of universities, the analysis explores the possible changes in the indegree and outdegree distributions along time and between gender and tests a power law fitted model to the data. our findings differ from what found by derzsi et al. (2011) , which conclude that the degree distribution of the erasmus network in 2003 follows an exponential distributional model, although in that work the authors study the overall distribution without differentiating for gender. the results of this paper suggest that the indegree distribution has an opposite behavior along time in the tail of the female and male distribution. the former seems not well represented anymore by a power law model in 2013 as it was in 2008; in the latter, instead, the tail appears well described by a power law model. our interpretation is that in the case of the indegree male distribution, the top universities, in terms of connections, are moving away from the body of the distribution, while the opposite is happening for the female, meaning that passing from 2008 to 2013 the universities with the most male links have been growing this kind of connection, while the universities with the most female connections are reducing their difference with the bulk of the distribution, overall resulting in a tendency towards balancing the initial disparity in favor of women. as it is evident, our study is not the first one to apply network analysis to the flow of erasmus students. several studies opened the way to the methodology we adopted. erasmus student mobility among european higher education institutions is also analyzed in breznik (2017) solely for engineering students between 2007 and 2013. the study recognizes the spanish universities as those with the highest mobility of engineering students and points out the top universities in terms of student exchanges. a similar application can be found in breznik and djaković (2016) which analyze the mobility related to slovenian universities in the years 2007/08 and 2011/12 and highlight the most attractive countries and universities for slovenian students as well as the top destinations for erasmus students in slovenia. the same approach is extended by breznik and skrbinjek (2020) with an analysis of hubs and authorities focusing on the countries participating in the erasmus program. the authors provide a general overview of erasmus mobility trends, revealing 3 different groups of countries: good senders and receivers, good senders only, and good receivers only. the hubs and authorities technique is further extended in restaino et al. (2020) which in addition adopt a blockmodeling approach on the network of countries involved in the erasmus mobility both for study and internship abroad. the authors provide also an analysis of homophily behaviors among countries and exploit educational indicators from eurostat to include investments in higher education. results reveal the presence of a core-periphery structure in the student mobility network, where key elements to attractiveness are economic benefits and investments in education. besides the network approach, these works adopt the same dataset exploited in this paper. finally, a global rather than european focus is adopted in shields (2013) which applies network analysis to international student mobility data covering the period from 1999 to 2008, in order to better understand the globalization of education. the author seeks to study changes in the network rather than the network properties. his findings suggest that the network of international students has become more centralized, less densely connected and it shows structural similarities with the networks of world trade and world polity. the main novelty of the present paper resides in i) the analysis of the gender gap in erasmus flows. the literature focusing on this aspect in fact is still quite scarce. if the issue of gender bias has been faced often in the context of international student mobility, to the best of our knowledge, when considering the specific context of erasmus mobility, gender differences have not been thoroughly studied, with the exception of bottcher et al. (2016) , whose findings about female over-representation in the program are in line with the results of this paper. our contribution aims at filling this gap and, in addition, ii) it uniquely combines the analysis by gender, by field and at the university level rather than at country level, as most preferred in the extant literature (restaino et al. 2020; bottcher et al. 2016; breznik and skrbinjek 2020) . the paper is organized as follows: the first section provides a description of the data used in the analysis; the second section quantifies the erasmus program general trend and gender imbalance in student flows across fields of study. in this section, the comparison between stem and non stem sectors can be found, as well as considerations about the overall trend towards gender parity and the changes seen in eastern and some mediterranean counties. the third section analyses the network of universities participating in the program, by gender. it shows strong gender bias in favor of female students, that persisted over the years considered. the section provides a set of centrality measures in the network, its density and highlights the top universities for sending and receiving students. the fourth section compares the network indegree and outdegree distributions over time and between genders, it highlights the change in the distributional model occurred over the years considered and tests for the best model to fit the empirical distribution; finally, the conclusive section summarizes the results of the analysis, draws some final observations regarding the possible future evolution of the gender bias in the erasmus program and provides some suggestions for future research. data used in the analysis are available at the eu open data portal and are freely accessible. they consist of 6 datasets corresponding to the academic years from 2008/09 to 2013/14 and they contain observations for each participant to the mobility in the relative academic year including information on the type of mobility (study or placement), the home country and the host country, the home university and the host university, the field of study coded in the isced 6 1997-2011 classification (uis 2012), the participant gender, the level of study (first or second cycle), the duration of the mobility, the amount of the grant received and the language used in the mobility. although the erasmus program has a long history, micro-data gathered in this format and containing the information described are not available for any further year. for this reason, since our analysis requires information about the field of study, and the unit of analysis chosen for the network analysis is the institutional level, thus universities, the study is limited to the period from the academic year 2008/09 to 2013/14. for simplicity, the reference years will be abbreviated as 2008 and 2013 respectively. data have been aggregated at country level exclusively for the analysis of gender by field of study (the erasmus program section). to consider variations across the period 2008-2013, each analysis performed compares the initial year (2008) with the final year (2013). in order to have an overview on the evolution of the program and gender participation, as shown in figs. 1 and 2, additional information at country level has been retrieved from the annual reports by the european commission (2000) . from the 6 datasets available, we only consider observations related to study mobility type and clean the data from misclassification, mismatched coding and foreign names, and the use of specific unicode characters present in the original datasets. by counting the frequencies of unique connections between pairs of universities, we specify the weight of each connection, that represents the students flow, i.e. the number of students going on mobility from the sending university to the receiving university. data are therefore collapsed so to have observations for each dyad of universities linked by at least one student in mobility, by gender and by field of study. the fields of study are grouped in macro fields according to the isced-f 2013 coding system (uis 2014), which identifies 12 macro fields. eventually, for the whole period of time considered, the datasets count 762304 observations in total, with 3148 universities present in the time span considered. erasmus stands for european region action scheme for the mobility of university. 7 it is a student mobility program created by the european union in 1987. the program started with the idea of allowing european university students to study abroad in a european university, with the legal recognition of the mobility in the home university and providing a scholarship to cover the additional cost for studying in another country of the eu for a period of between three months and one year. the work that led to the official approval of the program saw the involvement of universities from all over europe in order to establish 6 international standard classification of education. 7 the name of the program comes from the latinisation of the name of erasmus of rotterdam. the success of this program has led to the use of naming other european programs after famous personalities from various european cultures such as socrates, leonardo or comenius (corradi 2015) . the legal and financial basis necessary to develop and manage organizational and educational cooperation between universities underpinning the erasmus program (corradi 2015) . the objectives of erasmus+ and the original erasmus program can be summarised in strengthening the european identity, increasing individual skills, and, thus, their employability. by creating opportunities for study, training, work experience, and volunteering abroad, erasmus aims to respond to the problems of unemployment and skills shortages in europe and to modernize education and training systems. with respect to its objectives, the literature has highlighted the positive impact of erasmus and its effectiveness both in terms of european identity and likelihood to find a job after graduation (oborune 2013; jacobone and moro 2015; engel 2010; parey and waldinger 2011; bryła 2015) . 8 in addition, the erasmus program is considered a successful example of european integration and a symbol of the construction of european identity. over time, erasmus has become an essential part of the unified european mobility programs in the area of education socrates i (1994) (1995) (1996) (1997) (1998) (1999) , socrates ii (2000 -2006 and the growth in the number of participating students has followed the growing trend of the countries participating in the program, which in many cases joined erasmus before becoming eu member states. 9 the number of students participating in the mobility saw an unprecedented increase in 2015, going from 213879 to 291383, although the number of participating countries remained almost unchanged. in 2014 the program became erasmus+ and changed its structure (with the eu regulation 1288/2013): it is no longer exclusively dedicated to education, but also to training, youth and sport, and it no longer restricts participation to university students only, but also admits, for example, school and university teaching staff, as well as administrative staff. therefore, it is an integrated program that has incorporated all the funding mechanisms for school and university student mobility implemented by the european union until 2013 (e.g. comenius, leonardo da vinci and others). in 2013 the program reaches the largest number of participants, 34, the 28 eu member states plus switzerland, iceland, liechtenstein, norway, north macedonia, and turkey. since 2014, switzerland no longer enjoys the status of participant to the program, but it is now a partner country, i.e. it has adopted a transitional solution financed with swiss funds which still allows swiss people and institutions to take part in the program. in 2019, participant countries have been 34 again with the official entry of serbia. €14.7 billion were allocated to the erasmus budget for the period 2014-2020, 40% more than the previous programming period, and, as already highlighted, for the period 2021-2027 the european commission has proposed to double the figure to €30 billion (european commission 2018). the program continues to grow with the aim of becoming more powerful and inclusive. figure 2 shows the amount of female and male participants in the erasmus program for some selected years (2007, the academic years from 2008/09 to 2013/14, and 2015: data on the first and the last year come from the erasmus annual reports provided by the european commission (2000) and the relative ratio between female and male students. the number of both female and male students grows following the growing general trend in participation seen in fig. 1 , with females representing a large majority for each year. nevertheless, the ratio f/m slightly decreases over the years considered, signaling wider participation of men (see fig. 2 ). differences between gender can be observed more in detail across fields of study. in this case, to obtain a clearer visualization, we use the symmetric transformation of the ratio f/m based on de benedictis (2005) , given by: where the superscript b stands for bounded. the f/m b index provides a measure of female participation over male participation with a value ranging (bounded) between [ −1, 1] and demarcation value equal to 0, corresponding to the absence of bias. figures 3 and 4 plot this measure for 2008 against 2013 for each macro field of study in each country, respectively for incoming and outgoing students. these plots allow to identify whether prevalence of one gender in a certain field has changed from 2008 to 2013. briefly, for any field: also, whether the cloud of points is positioned above or below the x-axis gives us an immediate picture of the prevalence of one gender over the other in 2013. in case of fullpersistence all the dots would lay on the 45°line. the more they get disperse around the bisector the more variability from 2008 and 2013 is present (e.g. in fig. 3 the services field shows a high level of discontinuance in the gender ratio, while the arts and humanities field appears to be more persistent). the dots above the 45°line indicate an increase in the f/m b ratio along time; the ones below indicate a decrease in f/m b . at first glance, it is clear that there are two male-dominated sectors: engineering, manufacturing and construction, and information and communication technologies (icts). in the former, values for the incoming flows in 2008 show a higher variability than 2013, with a standard deviation σ 2008 = 0.239 decreasing to σ 2013 = 0.188 in 2013, and corresponding to values closer to the mean. for a few countries such as bulgaria, estonia, and hungary the sector once dominated by female students in mobility changed tendency with a predominance of male students in 2013 (fig. 3) . the outgoing flows show more stability over time, except for greece, poland, and romania, where the field once heavily dominated by outgoing male students had a reversed tendency in favor of female students in 2013 (fig. 4) . for the latter, the variability of the ratio also decreases from 2008 to 2013 considering the incoming flows, with bulgaria representing the only case of reversed tendency in favor of females, while cyprus and greece had a change of the bias adjusting looking at the overall fraction of female students in these two fields, without distinction of countries, allows for a few more considerations. table 1 reveals that firstly, in absolute terms, the field of icts offers less mobility to students, probably as a reflection of the smaller number of students enrolled in this sector with respect to engineering. secondly, the share of women over males in engineering is larger than in the icts, with a progress over the years analyzed, while not much changed in the icts. the greater distance from gender equality could, also in this case, simply be the result of the greater male intensity in terms of students enrolled in icts in europe compared to engineering. the rest of the sectors shows a predominance of female over male students, which in some cases remains pretty stable over the time span considered, as for art and humanities and education, where the cloud of countries is concentrated around the bisector of the first quarter, with a few exceptions for the education sector. some eastern countries (see bulgaria, romania, and latvia for instance), in the incoming flows (fig. 3) , show a decrease of the female predominance from 2008 to 2013 and in the outgoing flows the extreme cases of romania and malta had a reverse of the index respectively in favor of men and women (fig. 4) . in other cases, the values reported show less stability over time, as for the field of health and welfare, both for the incoming and the outgoing flows, where volatility decreased from 2008 to 2013. especially for the incoming flows, a group of eastern countries (see latvia, poland, slovakia) saw a decrease in the measure of bias thus in favor of a growing number of male students, up to cancel out the bias in the case of romania. the same observation can be drawn for the incoming flows of students in business, administration and law, and social sciences, journalism and information, where about the same group of eastern countries reduces the gender bias over time in favor of greater male participation. in the outgoing flows, the behavior is opposite; starting with a bias in favor of male students, eastern countries show to be catching up with the trend followed by the rest of countries. finally, in the fields of agriculture, forestry, fisheries, veterinary and services it is not possible to identify a general trend since the points displayed appear rather scattered, showing heterogeneity in the gender balance across countries; in the field of natural sciences, mathematics and statistics, instead, the points appear concentrated around the mean value being close to the x-axis, with their position on the plane suggesting that the variance decreased over time and pointing out that gender disparities have been nearly cancelled, in a sector usually characterized by a larger male presence as common in the stem-fields (botella et al. 2019; oecd 2017 ). an even clearer pattern of gender differences by study area emerges, in fact, when considering stem and non-stem areas. in figs. 5 and 6 the macro fields of study previously analyzed, as identified by the isced-f 2013 classification, are now grouped in stem and non-stem areas. the two areas fully position themselves in opposite quadrants, highlighting a net difference in gender prevalence in these areas: stem subjects are dominated by male students and, vice-versa, non-stem fields show a female prevalence. this can be observed both for incoming and outgoing flows. the clouds of points are crossed by their respective interpolation lines resulting from a robust linear model, that are positioned oppositely to the displayed bisector. the slope of the lines signals a tendency towards convergence that suggests a trend towards a reduction of gender disparity in the incoming flows, since the the stem field is derived by summing the number of students in mobility in engineering, manufacturing and construction, icts and natural sciences, mathematics and statistics, as coded following the isced-f 2013 classification. non-stem disciplines include agriculture, forestry, fisheries and veterinary, arts and humanities, business, administration and law, education, health and welfare, services, social sciences, journalism and information. the fields not known or unspecified and generic programs and qualifications are not included in either the two groups. the fitted lines are robust to outliers and pass through the averages (black dot) of the two macro-fields average value is in both cases below the 45°line, and a reinforcement of the bias in outgoing flows. 10 considering the incoming students (fig. 5) , the group of countries below the bisector is numerous, both in stem and non-stem fields, and comprehends especially eastern european countries. in the case of non-stem subjects, this highlights that eastern countries saw a decrease in the gender imbalance in favor of women, which in 2013 appears flattened to the zero, thus towards parity. in the stem fields, the right wing of the group of dots still mainly represented by eastern countries, as well as greece and great britain, reports that for these countries there has been a reversal in the trend, from a female prevalence in 2008 to a male prevalence in 2013, consistent with what previously observed for the engineering sector (see fig. 3 ). for the outgoing flows, the countries position appears reversed, with a few countries (romania, cyprus, poland) in stem sectors changing gender prevalence from male in 2008 to female in 2013 (fig. 6) . briefly, decomposing the analysis over time by fields of study shows that eastern and mediterranean countries have seen a change in gender participation in the program, with a trend towards equality, as it could be generally observed in fig. 2 . the convergence toward the mean value of f/m b is more pronounced in non-stem fields, as the slope of the regression line shows for both incoming and outgoing student flows. in this section we will visualize, describe and summarize the characteristics of the erasmus program, comparing its structure in 2008, the initial year in our period, with 2013, the last year, keeping the resulting network of student flows separated by gender. the erasmus network in the year t, n t , is a one-mode network defined by three sets: (a) the graph g t = (v t , l t ): corresponding to the binary adjacency matrix, a t , and including the set of nodes (universities, belonging to different countries), v t , and the ordered set l t of directed arcs (capturing the existence of an erasmus exchange program between universities); (b) the edge value function, w t , containing, for every pair of nodes, the flows of students moving from the sending node to the receiving node, and the field of study; (c) and the node value function o t , containing information on universities, and on the country of activity. we examine the network in the two years t table 2 . the overall density is rising, from a value of 0.006 to 0.008, giving evidence of a quite sparse network that is only moderately reducing its level of sparsity: the probability, for two universities chosen at random, to share an erasmus program is 0.6% in 2008 and it becomes 0.8% in 2013. the evolution of n to take the universidad de granada (e granada01) as an example, in 2013, the university was sending female students to 380 foreign universities and was receiving international female students from 392 universities; with 268 universities the exchange of students was bilateral; with the remaining ones it was unilateral. in the same year, the university was sending male students to 315 foreign universities and was receiving international male students from 273 universities; with 173 universities the exchange of students was bilateral; with the remaining ones it was unilateral. 228 foreign universities were sending both female and male students to the universidad de granada; 164 foreign universities were sending only female students; 45 foreign universities were sending only male students. in the same year, the universidad de granada was sending both female and 11 some examples are the stockholms musikpedagogiska institut, the university college of music education in stockholm, specialized in teaching music and related arts, or the shipping and transport college group based in rotterdam and providing maritime transport and logistics education. among these components we find also (private) foundations with a religious nature (e.g. association familiale de l'externat sainte-marie, based in lyon) or institutions providing secondary education in the form of two-years preparation for admission to a university, usually following the compulsory stage of secondary education (known as bachillerato in spain, a levels in the uk, baccalauréat in france). 12 the universidad de granada (e granada01) receives students from 70 german universities; 65 french universities; 62 italian universities; 39 universities from the uk; 24 from poland; 15 from belgium; 11 universities from turkey, portugal, greece, and the netherlands; 10 from austria and less then 63 universities from the remaining countries; with a total 392 universities (in 2013) sending students, males and females, in all fields of study. note: see wasserman and faust (1994) for the definition of the statistics used. m stands for male; f stands for female. degree stands for degree centralization (standardized); closeness stands for closeness centralization (standardized); active connections includes student flows in different fields of study; erasmus university codes have been shortened for visualization purpose: bar03 = barcelo03; bol01 = bologna01; gra01 = granada01; lju01 = ljublja01; lun01 = lund01; mad03 = madrid03; pra07 = praha07; rom01 = roma01; sev01 = sevilla01; val01 = valenci01; val02 = valenci02; war01 = warszaw01. squared parentheses contain the degree value. the assortativity score is [-1,1] ; the reciprocity score is in percentage points male students to 232 foreign universities, in the remaining 88 the university was sending only females, and to 83 was sending only males. the universidad de granada example is paradigmatic: the amount of inter-gender flows is 50.1%, for university out-flows, and 52.2%, for university in-flows; the remaining flows are gender specific. little bit less than half of the university flows are gender specific. in general, less than half of the universities tends to reciprocate, and, in 2013, reciprocity is more present among female flows (41%) than among male flows (34%). moving from ego to structural statistics, the erasmus network is sparse (low density) and, at the same time, shows some relevant hubs, i.e. top sending universities, and authorities, i.e. top receiving universities (kleinberg 1999) , 14 however, those hubs and authorities are connected to less than the 18% of the all active universities in the erasmus network. there is no sign of a hierarchical structure: the closeness centralization is near zero and this implies that we are very far from a star-like structure. as previously emphasized, the whole network is composed of a set of communities of limited dimension. in the female erasmus network, only 4.2% of the universities has more than 100 links and only the 34.2% has more than 10 links; in the male erasmus network, only 2.3% of the universities has more than 100 links and only the 29.4% has more than 10 links. in 2008, the female and the male erasmus networks were also different in terms of homophily: g f 2008 shows sign of mild assortativity, while g m 2008 shows sign of mild disassortativity: universities sending female students tend to have a positive match in terms of degree, in other therms, highly connected universities are connected with highly connected universities; universities sending male students tend, instead, to have a negative match, with highly connected universities being connected with poorly connected universities. this difference disappears in 2013, with both g f 2013 and g m 2013 showing a sign of mild assortativity. g t can also be explored in its weighted version n t , by accounting for the flow of students that each link has. by looking at the country level this time, the analysis allows understanding how, if on the one hand the number of participant countries has been stable in the last decade, as pictured in fig. 1 , the level of participation of each country is instead pretty heterogeneous, with student flows accounting for different proportion over the number of students enrolled at university. table 3 shows the percentage instrength and out-strength measures for each country, weighted by the flow of incoming and outgoing students, averaged over the years 2008-2013 and normalized to the total number of students enrolled in higher education in 2013. 15 for the incoming flows, the in-strength measure highlights the role of countries such as spain, france, great britain, germany, and italy as the destination for european university students; while for the outgoing flows, the out-strength draws attention to the role of small countries, such as liechtenstein and luxembourg, where a considerable part of the student population enrolled in higher education participates in the program. by considering g t , the graph of universities participating in the program, we analyze the degree distribution of this unweighted network by gender and along time, using 2008 and 2013 as initial and final benchmark years. two universities are connected by a female-link (male-link) if there is at least one female (male) student moving from one university to the other one. as the network is directed, we observe both the indegree and the outdegree 14 according to kleinberg (1999) , the authority scores of the vertices of a graph are defined as the principal eigenvector of t(a) · a, where a is the adjacency matrix of the graph. the hub scores are the principal eigenvector of a · t(a). the university of bologna (i bologna01) is the major hub in 2008 and the distribution of the hub scores is power law: the large majority of universities are poorly connected. as far as the authority score, the universidad de granada (e granada01) gets the highest value, and also in the case of the authority score the distribution is power law. if we look at the difference between the male and female erasmus network the comparison between hubs and authorities distributions gets inverted: the distribution of the authority score decreases sharply before the one of the hub score in the female network, the opposite happens in the male network. the gap gets reduced in 2013. however, the difference between the female and male network in terms of the prevalence of sending universities over receiving universities, for females, and of receiving universities over sending universities, for males, persists. more about this in the next paragraph. 15 in the case of countries that joined the erasmus program later than 2008, the average is taken only on the years of participation. distribution. as common in most social networks, the degree distribution appears to be right-skewed in all cases. tables 4 and 5 collect summary statistics for every distribution considered together with the estimated values of skewness and kurtosis. the degree distributions appear to be far from the normal or the exponential model, as instead observed in derzsi et al. (2011) , and thus they can be identified as "heavy-tailed" (clementi 2016 ). stretched. the tail of the distribution appears to have squeezed along the years considered. this is not visible for the outdegree distributions, which seem to have maintained the same shape along time. the same graphs are reported in figs. 9 and 10, this time comparing the distribution for male and female connections in the same plot. the plotted densities include now also lines of fit for a power law distributional model and a lognormal distribution. a power law degree distribution, p(x) ∝ x −α , is observed in the so-called scale-free networks (barabási et al. 1999) , although the empirical distribution usually follows a power law model only in its upper tail, i.e. starting from a threshold x min . the methodology employed for fitting heavy-tailed distributions is developed by gillespie and et al. (2015) and based on clauset et al. (2009) ; it relies on maximum likelihood estimators and goodness-of-fit based approach to determine the cut-off x min . in particular, the optimal choice of x min is determined by minimizing the distance d between the probability distribution of the data and the best-fit power law model, measured by the kolmogorov-smirnov (k-s) statistic: where s(x) is the cumulative distribution function (cdf) of the data and p(x) is the cdf for the power-law fitted model. following the framework proposed by clauset et al. (2009) , the visual inspection and the distribution fit are complemented with a goodness-of-fit test, based again on the k-s. via bootstrapping, a distribution of the k-s statistic is generated following the creation of a large number of power-law distributed synthetic data sets with scaling parameter α and lower bound x min equal to those of the distribution that best fits the observed data. a pvalue is thus generated as the fraction of the time that the k-s statistic is larger than its value for the empirical data. the p-value quantifies the following hypotheses: table 6 collects the estimated values for the scaling parameter α and the threshold value x min for each power law fitted model and shows the relative k-s statistic together with the p-value resulting from the goodness-of-fit test for each case. as in clauset et al. (2009) we decide to rule out the power law hypothesis if p ≤ 0.1; this is the case for the female indegree distribution in 2013 and the male indegree distribution in 2008, which therefore are not well described by a power law. for the sake of accuracy, the analysis is complemented with a vuong's test comparing the power law fit with a lognormal distribution fit, which suggests that there is not a preferred model between those tested (vuong 1989) . although it is not trivial to identify the best distributional model for the degree distributions, some observations can be drawn. for the indegree distribution, we observe an opposite behavior along time in the tail of the female and male distribution. for the former the test conducted supports the hypothesis that the power law model is no longer a good description for the tail of this distribution. on the contrary, the male indegree distribution shows a "heavy" but not "fat" tail in 2008, meaning that its ccdf goes to zero faster than a power law; in 2013, instead, the tail of the male indegree distribution appears to be well described by a power law. this change in the distributional model might suggest that the top universities in terms of male connections are moving away from the body of the distribution, while the opposite is happening for the female, meaning that from 2008 to 2013 the universities with the most male links have been growing this kind of connection, while the universities with the most female connections are reducing their difference with the bulk of the distribution. starting from a situation characterized by a strong gender imbalance in favor of female connections, this could signal a tendency towards increased gender parity in the erasmus incoming connections. on the other hand, the outdegree distribution remains pretty stable over time and does not behave much differently by gender. in each case considered, it seems well described by a power law model. we can hypothesize that concerning male students, whose degree distribution shows a scale-free behavior, when new universities adhere to the program, they follow a mechanism of preferential attachment: they connect with the most popular hubs, with high indegree (as well outdegree) values, not necessarily receiving a connection in the opposite direction, meaning that the outdegree value increases for those observations located in the core of the distribution, while the nodes with high outdegree measures do not move further away from the bulk of the distribution. the erasmus program has been characterized, since recently, by a strong gender bias in favor of female students. this work quantifies this evidence between 2008 and 2013, using novel data on bilateral student flows at the university level. key results can be summarized as follows. firstly, after describing the structure of the program in great detail, a descriptive analysis across fields of study performed at country level has highlighted the prevalence of female students in mobility in the non-stem disciplines, such as arts and humanities, education, business administration and law, and the opposite for the fields of study defined as stem, such as engineering and icts. it is also observed that moving from 2008 to 2013, there is an increasing trend toward gender parity, especially in eastern european and some mediterranean countries. secondly, the study of the participation of the 3148 universities included in the program between 2008 and 2013 through network analysis emphasized that in general the gender bias persisted over time, given the denser network of connections involving female students. in addition, the female erasmus network is characterized by a higher level of reciprocity and homophily, with also a prevalence of sender universities rather than receivers, contrary to the male erasmus network. the universidad de granada in spain stands out as key sender for the female network, in both years analyzed, and in 2013 replaced the univerzita karlova of prague in its role of key sender in the male network in 2008. the universidad de granada is also the key receiver in 2013 for both the female and male network, replacing the universitat politecnica de valencia, in spain as key receiver in 2008 for the male network. the network of universities is characterized by a giant component including the majority of nodes. finally, the paper also tests the difference in the degree distribution of the erasmus network along time and between gender, giving evidence of contrast with respect to the exponential behavior resulting from (derzsi et al. 2011 ) for 2003 and a variation along time of the indegree distribution. considering the female network of universities, the tail of its degree distribution follows a power law model in 2008 while a lognormal distribution could better describe it in 2013; the opposite behavior characterizes the indegree distribution in the male network of universities, signaling a tendency towards balancing the initial strong gap in the incoming connections. in conclusion, evidence from this work underlines a mild trend towards the reduction of the strong gender bias in favor of female students that characterized erasmus in 2008 and suggests that this tendency can foster the convergence of male and female students flows possibly resulting in a future annulment or at least considerable reduction of the bias. the availability of detailed data at the university level on bilateral flows of erasmus students for more recent years will make possible to confirm or deny this early evidence. from a normative point of view, the effect of the observed gender imbalance might be twofold. on the one hand, the bias in favor of female students may have a positive impact on women empowerment, since international experiences could facilitate entry into the labor market, which is known to be ceteris paribus more difficult for women. on the other hand, considering that it is likely that men, in particular, will lead the labor markets in the future, enlarging their international experience could increase their skills and promote an attitude towards international benchmarking in decision making. it is the responsibility of policymakers to understand whether gender equality would be desirable in erasmus student mobility and, consequently, adopt the most appropriate policy tools to orient student flows. the richness of the erasmus data and relevance of the analysis on international formation of human capital call for future research. the time dimension could be analyzed in more detail by checking for possible differences and similarities across the period, not only comparing each year included but adopting a dynamic approach to the network data (batagelj et al. 2014) . a further extension of the present work could be the adoption of a blockmodeling approach, along the lines of (restaino et al. 2020) , to highlight the presence of sub-networks and communities in the network of universities. the longitudinal data could be also exploited in a more model-based analysis. using a gravity framework (monte et al. 2018 ) could help understand the relationship between the bilateral flow of erasmus students and the country and university characteristics both as sender and receiver, and to explore if the distance elasticity has a gender and/or a field dimension. this analysis could be complemented by the inclusion of economic indicators and information about higher education at country level provided by eurostat or at the university level, using international quality ranks of universities. in general, the growth of the erasmus program in terms of student participation suggests that it will continue to grow, considering also the increased funding devoted to the initiative. conversely, the number of participating countries is rather stable, but this reflects the political dynamics of eu member and partner countries. for instance, the recent exit of great britain from the european union raises doubts about possible consequences for the erasmus program, which necessarily requires stable agreements between universities to allow study abroad without incurring in the payment of tuition fees in the host country. future research could explore possible effects that are still uncertain today. finally, the recent covid-19 pandemic might have changed the future of the erasmus program with universities in lockdown and courses entirely provided online. the success of the erasmus experience relies on the active participation of students in university life in the host country, which includes using the university facilities, relating with professors and classmates using a foreign language, and experiencing the everyday life typical of the host country. what will be of this exchange program with the conditions imposed by the pandemic is still unknown. future research could shed light on possible scenarios as well as on the potential choice of students to postpone or give up the erasmus experience. mean-field theory for scale-free random networks understanding large temporal networks and spatial networks: exploration, pattern searching, visualization and network evolution women on the move: gender dimensions of academic mobility gender diversity in stem disciplines: a multiple factor problem gender gap in the erasmus mobility program institutional network of engineering students in the erasmus programme erasmus student mobility flows-the national-level social network analysis of slovenia erasmus student mobility flows the impact of international student mobility on subsequent employment and professional career: a large-scale survey among polish former erasmus students power-law distributions in empirical data heavy-tailed distributions for agent-based economic modelling student mobility in higher education. erasmus and erasmus plus. laboratory of lifelong learning department of education and training three decades of italian comparative advantages topology of the erasmus student mobility network the impact of erasmus mobility on the professional career: empirical results of international studies on temporary student and teaching staff mobility erasmus+ statistics some evidence that women are more mobile than men: gender differences in uk graduate migration behavior fitting heavy tailed distributions: the powerlaw package evaluating the impact of the erasmus programme: skills and european identity hubs, authorities, and communities erasmus: continuity and change in the 1990s the gender of mobility commuting, migration, and local employment elasticities the geography of gender inequality in international higher education becoming more european after erasmus? the impact of the erasmus programme on political and cultural identity studying abroad and the effect on international labour market mobility: evidence from the introduction of erasmus analysing international student mobility flows in higher education: a comparative study on european countries globalization and international student mobility: a network analysis barriers to international student mobility: evidence from the erasmus program international standard classification of education: isced 2011. unesco institute for statistics, montreal uis (2014) isced fields of education and training 2013 (isced-f 2013): manual to accompany the international standard classification of education likelihood ratio tests for model selection and non-nested hypotheses social network analysis: methods and applications the authors wish to thank the editors and the four anonymous reviewers for their helpful comments and suggestions. the authors are grateful also to domenico vistocco for the kind tutorial on how to clean and wrangle the erasmus data, and to maria prosperina vitale for taking the time to evaluate the preliminary version of the article and provide precious feedback. all remaining errors are our own. a previous version of the paper circulated under the title "gender bias in the erasmus students network" (https://arxiv.org/pdf/2003.09167). both authors planned the research project, silvia leoni collected the original data, luca de benedictis prepared the data sets and coded the network analysis in r, silvia leoni ran the degree distribution analysis, both authors contributed to the empirical analysis and to the writing of the paper. both authors read and approved the final manuscript. the research reported was conducted with no funding. the original datasets used during the current study are available in the eu open data portal repository, [https://data. europa.eu/euodp/en/home]. the datasets generated by the authors, r scripts and stata do files are available on request for replication purposes. the authors declare that they have no competing interests. springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. key: cord-143246-f97v2cih authors: paul, aneesh mathews; susanthomas, sinnu title: multifaceted covid-19 outbreak date: 2020-08-26 journal: nan doi: nan sha: doc_id: 143246 cord_uid: f97v2cih the time when everyone is struggling in the cruel hands of covid19, we present the holistic view on the effects of this pandemic in certain aspects of life. a lot of literature exists in covid-19, but most of them talk about the social and psychological side of the covid problems. covid-19 has affected our day-to-day life and its effects are extensive. most of the literature presents the adverse effect of the pandemic, but there are very few state-of-the-art approaches that discuss its beneficial effects. we see the multiple faces of the pandemic in this paper. to the best of our knowledge, this is the first review that presents the pros and cons of the pandemic. we present a survey that surrounds over effects on education, environment, and religion. the positive side of covid-19 raises an alarm for us to wake up and work in that direction. digital transformation in the marketplace. devaux et al. [14] investigated the effects of hydroxychloroquine against sars-cov-2 virus. faridi [15] has studied the effect of middle east respiratory syndrome coronavirus (mers-cov) that has caused havoc in saudi arabia in 2012. the author has seen the effect of mers-cov on male and female in riyadh. the authors in [16] , [17] , [18] , [19] assessed the psy-chological stress of covid-19 on health workers. xiang et al. [20] reported an overview of infected healthcare workers in china and italy during the early periods of the covid-19. the authors in [21] , [22] , [23] studied some social impacts of covid-19. chakraborty and maity [24] studied the covid-19 effect on the economy and global environment. ivanov [25] predicted the impact of covid-19 on global supply chains. xu et al. [26] studied the air quality index to see the effects of covid-19 on the environment. chinazzi et al. [27] studied the effect of travel and quarantine influence on the dynamics of the spread of covid-19. braun [28] narrated examples of the situations of the poor during covid-19. ahmed et al. [29] highlighted the precarious position of postdoctoral fellows in academic positions due to covid-19. staniscuaski et al. [30] projected out the problems faced by academic mothers having many difficulties working at home during covid-19. bouillon et al. [31] discussed the positive side effect of coronavirus on air pollution. suicide rate has increased during the pandemic time [32] , [33] .the situation of covid-19 has diverse effects in india [34] . in this paper, we study the multi-faceted effects of covid-19 on our planet. our contribution in this paper is threefold. 1) the pandemic has affected the entire education sys-tem and a new era of distance learning has emerged. a review on various education systems during the pandemic is looked out. 2) covid-19 has benefits in certain areas such as the environment. the environmental effects are discussed. 3) overall change in religious practices has changed and we review these aspects in this paper. the remainder of this paper is organized as follows. section ii highlights the overall change in the education system during the covid-19 season, and discusses the social and psychological impacts of the pandemic. section iii presents the religious and environmental effects of coronavirus. section iv presents the conclusions of this paper. education system is one of the prime pillars in developing a nation. it constitutes an important ingredient in deter-mining the growth of a country. human development is an important determinant in a person's health and trade. the education system is severely interrupted in most of the countries since the outbreak of this pandemic across the globe. the schools, colleges, and universities are in the total closure mode. billions of academic learners became devoid of their knowledge acquisition during this pandemic. the teachers, students, schools, and families -all became a victim of this bitter truth. the world has gone under complete reorganization during this period be it any sector, the education sector is not left apart. the speed of the pandemic and the closure of schools was so fast that it was difficult to come up with a solution with all facilities. the closure of educational institutes will not only have short term impact, but leave a footprint on economic and societal components. there are number of areas in education that is affected by the pandemic: the landscape of higher education across the world is defined by the cross border movements of the students. globally every year there is an increase of 10% in the number of students studying abroad as shown in fig. 1 . as per the unesco [35] , the students enrolled for higher education for a period of typically a year to seven years. according to the statistics given by organization for economic cooperation and development (oecd), the in-ternational student population with demographic changes is likely to reach 8 million by 2025 [36] . most of the international students prefer either the united states, the united kingdom, germany, france, or australia for their higher education [37] as shown in fig. 2 . as per the statistics in 2019, the top host countries involved in sending students to other countries include china, india, south korea, and france [35] as shown in fig. 3 . the pandemic has brought a sluggish impact on the movement of students across the border. the travel restrictions during lockdown and the fear of pandemic will affect the cash flow at the universities. parents are afraid to send their ward across any border in this situation. the universities in these countries are undergoing extreme pressure on student admission. if this problem persists, there is a possibility of decline in international higher education in the com-ing years. the pandemic has brought a devastating effect on the global education system. the pandemic has shrinked the world under their own home and hometown and cross border movements seem to be a threat to the life of an individual. 2) online learning active learning is not only a source of fun but also a source of formation of cognitive social skills. carlsson et al. [38] emphasized on the increase in cognitive skills with the total number of school days attended. the study carried out in sweden showed that crystallized intelligence can be aug-mented significantly by 1% of a standard devia-tion while attending ten days of extra schooling. the closure of schools for almost a month at the beginning of this pandemic can cause a trivial loss of 3% of the standard deviation. the pandemic has left the learning systems with no options other than embracing a distant or online learning. as per the statistics released by unesco [39] , the pandemic has affected nearly 1.3 billion learners around the world. the recovery of the disruption of the learning process is essential to facilitate the continuity of the education system. when physical presence is a risky situation, an alternative has to be taken at various levels of learning. online learning is a new strategy embraced by the education system in this time of pandemic. the transition from active learning to passive learning was very rapid during this pandemic. the curriculum was not designed for passive learning, so the viewers are losing their interest in the content. the shift in developed countries to the online learning system does not pose any problem, but for developing and under-developed countries -it is a challenging situation. the rural areas of these countries do not have the basic infrastructure to facilitate the online learning. the pandemic has posed a threat to the overall development of the underprivileged in these countries resulting in shattering their economies. the video telephony softwares is being used for distant learning. the concept of keeping the electronic gadgets far from the children has been loosened even for a primary school going child during this pandemic. the online learning has removed the commuting time for the learners but on the other hand, made them addicted to electronics devices leading to many social, psychological, and physical disabilities. online learning brought a paradigm shift in one's own comfort zone. the hassle of traffic jams, pollution, queues, health problems, allergies is halted in this course of time. most of the learners are happy with the online learning system since environmental problems do not leave them void of attending classes. online learning has brought an end to the centuries old practice of chalk and talk. due to the sudden change to the online learning in the education system, the preparedness of the tutors was a concern. an inhibition of this sudden change was found in the tutors during the beginning phase. the course curriculum was not made for passive learning. the 3 sudden shift in the teaching system with inadequate preparation from the learner side was also noticeable. teaching is a knack that everybody is not gifted with, so many tutors are not so effective in an online mode. in countries like india, where there is a huge shortage of technology savvy tutors, this model of learning would not work out. lack of infrastructure and resources in the rural parts of these countries is an obstacle for teachers for a complete preparation of imparting the knowledge. an unavailability of dedicated online platforms is posing a threat for outcome based education. the tutors are adjusting the platforms with the video telephony platforms. if the problem of pandemic persists, there is a need for creating dedicated learn-ing platforms. most of the schools and universities undergo the ad-mission process during the month of may-june for fall semester. due to the severity of the pandemic in many parts of the world, the admission process is hindered. the situation in the admission process is becoming alarming in the foreseeable future with the pandemic situation. traditional admission procedures would not take place in this season. new procedural strategies for admissions should be considered in order to fill the gap in this pandemic. some universities are not con-sidering taking any students the current academic year, while some are luring people with discounts. it is a crucial task for the students to decide which school they would like to attend without visiting respective campuses. the pandemic has forced people to create a virtual world of working at home. the virtual world cre-ates effortless paths to collaborate across the globe. the conferences, academic meetings, classes, and seminars have gone online leaving a space for academic collaborations. we see a lot of unprecedented collaborative work globally among the educators [40] during this pandemic leading to a loss in the travel economy. the cancellation of universityfunded international travel for conferences, blanket bans on any international travel for spring break, canceling study-abroad programs [41] made different academicians closer virtually. collaborations serve a larger purpose as an individual and also as an organization [42] . there are lots of scope for online conferencing platform business. the concept of education will be reformed envisaging the global collaboration. globally, the collaboration has brought a new direction to certification courses and degrees. these collaborations fulfills the need of each other while dividing the work in chunks. the pandemic has brought a halt to the organizational structure making a scarcity in the manpower. the universities are facing challenges to recruit new students, and faculty during this pandemic. the retention is also questionable. the recruitment for the faculty is a worrisome issue for the administration when the risk of losing students is hovering around them. when survival of many institutes is a burden for them, the recruitment of new faculty members increases their load. due to the recession in the corporate sector, the recruitment process for the students is a great disaster. the job offers have been withdrawn creating a havoc in the student community. the global outlook of the pandemic would massively devastate the livelihoods in the entire world. fig. 4 the consequences of a sudden shift in the learning system brought a slowdown to the world economy. the international students from china and india constitute 33.7% and 18.4% of the total international students in the usa higher education sector. the travel restrictions during the pandemic would cut down the admission process leading to an economic burstdown. the conveyance to the institutes are at a halt causing recession in the travel sector. all the learners cannot afford to stay near their institutes, so they stay far and face a time-consuming and costly commute [44] . students spend approximately £100 a month for commutation to their academic institutes. pandemic has saved the pocket of students in higher education. in countries like india, private schools and private vehicles charge a heavy conveyance fare for the commutation. the pandemic has given relief to the parents. same time, the train services and the road services are hit badly. cashflow in these services reduced leading to an economic crunch in these sectors. the students use to take long commutes to the institutes taking away their well-being [45] . they are deprived of their sleep and exercises. to commute long distances, students get up early and the daily routine is hampered. lack of daily exercises make them obese which is a major cause of concern among the youngsters. students carry a heavy load of bags on their backs to the school in countries like india. carrying school bags are back breaking work to the students. heavy loads of school bags have deleterious effects on the spine of children [46] . many measures are taken to reduce the amount of school baggage, but it was all at a minuscule level. the online learning during the pandemic season turned out to be a heavy relief to the students carrying heavy school bags. being in a well-being state is an important aspect of human being. we tend to give rest to the body if it is not in a position to commute. the learners refrain from going to class if they are not well. the pandemic situation takes off all the health issues and helps in smooth learning of classes. the students are free to learn from their home in any physical condition. the structure of the learning system is based on various assessment procedures. the students are assessed based on the merit system. the pandemic hit the world during the key assessment period cancelling many exams. the cancellation of exams would have a long-term consequence on the ca-reer of the student. first, the internal assessment and then the public examinations were cancelled. the grades at the end of the academic year were predicted according to some undefined rules influ-encing the privileged students. education system is shifting to an online assessment system that can cre-ate measurement errors. these errors in an abrupt assessment would increase the differences between the privileged and under-privileged students in the future. the labour market would face the dire consequences of inefficient assessment scores. the entrance exam in higher education is a worst hit in assessment procedures. the entrance exams to top universities are either postponed or cancelled. the exam agencies are coming up with alternative solutions in consultation with the international in-stitutes. 10) strike free education education system is at stake be it teachers strike or any other political strike. these strikes prevent students from attending classes. according to the study at argentina [47], 180 days of teachers strike there is a decline of years of education by 3:1% in an academic year. the teacher strike has a negative effect on student learning and their overall achievement [48]. frequent political strikes or hartals impact the overall education system. according to the statistics in kerala, india [49], there would be one hartal in every four days leading to disruption in the holistic coverage of prescribed syllabus. the online education system is not affected by any sort of socio-political disruptions. education system in the virtual platform eased out the disturbances due to the strike. the education aspects during the pandemic impacted the family in many ways. a) the education system comes with mid day meals for the underprivileged in countries like india and the pandemic situation has taken the bread out of the mouth of some children. children from poor families would come to school with the greed for getting a one time meal. if the pandemic persists, then there is a high chance of drop-outs from the school. moreover, it would be a tremendous challenge to keep up the motivation of the underprivileged children after the pandemic. b) most of the parents in the pandemic era are working from home. it is difficult for most of the parents to handle domestic pressure and work pressure at home. working parents are juggling with children and working at home. global home schooling would pro-duce disparities depending on the ability of the family members to help their children learn. the inequality in each student skill set would overall affect human capital growth. c) the unprecedented learning system needs assistance of basic infrastructure for its smooth conduct. power supply and internet connectivity are the essentials needed without disruption. to avail these resources at home and keep the student without stress is a burdensome work for the parents. in developing countries, it is a difficult situation to maintain the resources around the clock. 5 d) women take care of the children and rela-tives at home when compared to men. they are more insecure in their jobs. women are struggling with their household obligations and work during the pandemic. the juggling between children at home and work would reduce their opportunities and earnings at the workplace. women have to work harder in order to compensate for the workload and at an increased stress during this period. the study says that many women have left their job during the pandemic due to the imbal-ance in the worklife. covid19 is a disaster that would widen the gender inequalities. studies reveal that there is an increase of 60% in usage of electronic gadgets by the impressionable minds. gadget addiction is one of the major drawbacks of the online learning system. irritational behavioural patterns are observed in the students during this pandemic. the long time exposure to electronic gadgets are making them obese. an attachment towards gadgets creates a space for emo-tional imbalances in their personality. students have confined themselves into their own territory keeping them away from the societal component of life. studying and living together with their companions under one roof increases their social abilities but lockdown has created a void space for problem solving and decision making skills. social unawareness and lack of cognitive skills would be more visible. these skills improve their employability, productivity, health, and well-being in the future, and ensure the overall progress of the nation. people around the world are worried about the undergoing changes in the climate. the global temperature is a major concern for many environmen-tal changes. the last five years (2015-2019) were recorded as the hottest years. globally 3°c temperature has increased since the last century. an increase in per capita gross domestic product (gdp) is proportional to global warming. a study conducted by [50] shows the environmental degradation and co2 emission has increased with the economic growth and more production [51] . according to the census in 2019, the countries with the highest co2 emis-sion in the world is shown in fig. 6 . we see that the environmental degradation increases with the increase in production for economic growth. a lot of measures were taken to reduce the hazardous emis-sion, but a substantial decrement was not possible. the co2 or greenhouse gas disturbs the natural regulation of temperature in the atmosphere and leads to global warming and climate change. humans manipulated nature according to his whims and fancies that resulted in paradoxical im-reduced by 8% or 2.6 gigatonnes (gt) which is equivalent to a decade earlier data. there was an average decline of 25% energy demand per week during full lockdown and an average of 18% decline in partial lockdown countries. an unprecedented decline in demand for various fuels is seen during the pandemic as shown in fig. 8 . the crisis of pandemic is paving a way for clean energy transitions. this decline in co2 emission is unprecedented and would be temporary, unless there is a resilient effort to change the structure. balances. humans are responsible for the emission of the greenhouse gas in the atmosphere over the last 200 years. covid19 is the only disaster that has come as a boom to the environment. the major sources of co2 emission are energy, agricultural processing, land use changes, industrial processing, and other waste. electricity and heat is generated by burning fossil fuel, coal, and natural gas. a total of 24.9% greenhouse gases are emitted while burning these fuels and are the leading cause for tempera-ture regulation. industries emit 14.7%, transporta-tion -14.3%, agriculture processing -13.8%, land use change -12.2%, and industrial processes-4.3%. distribution of different sources of greenhouse gas are shown in fig. 7 . before the arrival of pandemic, it was difficult to control the industrial and transportation emission. an impossible action of putting a halt on these hazardous sources was done overnight. accord-ing to estimates published by international energy 2) vibration in the earth crust high frequency seismic waves are propagated into the earth mainly due to the activities of the human. the seismic noise renders the real time estimate of population dynamics. the covid19 pandemic period is the longest seismic noise quiet period ever recorded. according to the royal observatory of belgium [53], the seismic noise of the earth during the pandemic is not prevalent, reducing the vibration of the earth by 50%. the vibrations are reduced by one-third of the normal activity during the lockdown. it becomes easy for seismologists to detect the movement in the earth crust without much of an expedition. the construction projects in some countries were at complete hold during the initial stages of the lockdown. the availability of the workforce and the site constraints halted some of the projects. construction activities create an adverse impact on the environment. the burning of fossil fuel, noise, and the waste of the construction contribute to the regulation of the temperature in the environment. the halt in construction reduced the amount of pm10 by three times in the month of april 2020. air pollution is recorded highest in many cities of india. the annual average pm2.5 concentration during the lockdown was much better than the safer limit [54], [55], [56] , [57] as shown in fig. 9 . under the banner of economic growth, entire industrial and other waste is dumped into the rivers making it difficult to breathe. the aquatic species are becoming extinct due to the pollutants in the river. india is at the top of river pollution. ganges river is the most populated river in the world. the present pandemic has come as a blessing in disguise for rivers. the water pollution has decreased con-siderably during covid19 period. the waters from the rivers in india are tested during covid19 and the results provoke us to take measures to clean the rivers. the ph levels, the conductivity level, dissolved oxygen (do), and the biological oxygen demand (bod) of the water is reduced during the lockdown period [58] . a betterment in standards of drinking water was seen during the lockdown period as shown in fig. 10 . 7 pandemic season was a lockdown for mankind, but on the contrary animals were liberalized. humans were away but animals took over the deserted cities and towns. animals took the advantage of the drop in human activity and came out to explore and play in the public places. scavengers are not around to shoo them away giving a space for wildlife to thrive. mallard ducks, wild deers, herd of goats, troop of monkeys, kangaroos, gangs of turkeys, and many others are taking human spaces. road mortality was a threat to the wildlife population [59] . the mortality has reduced to 58% due to less traffic on roads in the usa. less roadkill reduces the ecological imbalances. some animals have successfully adapted to live alongside humans and their survival is dependent on them. an absence of human activity endangers some wildlife species. some governments mobilize funds to feed and preserve these animals, and the lockdown hindered their progress. according to the livestock census of 2012, there are around 18 million stray dogs in india. these dogs are fed by ngos or leftovers from restaurants. the closed restaurants and the restrictions in the movement made these stray dogs starve. the sustenance of the people in rural places of poor countries became difficult during the pandemic. people are driven to take extreme steps for their livelihood through poaching. the illegal hunting of endangered species in african continent is a threat for the wildlife society. according to study conducted by traffic, the wildlife poaching in india has increased twice during the pandemic pe-riod. it has increased from 22% to 44% during the lockdown period. it may turn disastrous and pave a way for another pandemic. humans struggled from recent pandemics such as aids, ebola, mers, and sars that came as an effect of consumption of animal meat [60] , [61] . it becomes the responsibility of the wildlife conservation society to prevent any pandemic in the future. due to the clean air and lockdown, non-covid diseases are at steep decline in countries prone to all pollution. the behavioural changes during the lockdown has brought a decline in insurance claims by 40% in india. waterborne infectious diseases and respiratory related diseases are being recorded as lowest during the pandemic time. the claims on deadly diseases such as cancer has turned down by 42% as per the statistics of the insurance companies in india [62] . due to decrease in vaccination [63] and disruption in the hospital services, there is a possibility of an outbreak of other diseases. religion makes people follow different practices and form socio-cultural groups. each culture recorded in human his-5) wildlife effects tory practised some organized system of beliefs and prac-tices. we tend to see very few people practicing faith in normal life. for some it seems absolutely mandatory but for some these are obnoxious practices. religion and faith is an integral part of people's lives worldwide, even though it is increasing. religious practices were hampered during lockdown. various aspects of religion during lockdown are discussed in detail: religion is a predominant factor for satisfaction in life, on the contrary the religious tensions can be annoying [64] and affect the economic growth of the country. religious fervency is vigorous in most secularized countries [65] . the polarization towards targeted groups increased in many countries during the earlier stage of the pandemic [66] . since the cases of the virus were aggravated by the religious gathering in some countries, we could see religious bigotries coupled with the pandemic. the virus has morphed itself into an anti-community virus [67] , [68] . the bigotries and xenophobia towards different sects of people can be seen in different countries as shown in fig. 11 . 8 prayer meeting in france, and many more [71] , [72] , [73] . the pandemic spread in various countries was sparked by religious gatherings as shown in fig. 12 religion and politics are a crucial part of life and covid-19 has acquainted the human life without these jargon words. the places that culminated religious polarization at the earlier stages of the pandemic were felt at peace in the later stages of the pandemic. everybody came out in unison to curb this pandemic through their services. charity works and social commitment was seen at large during the pandemic. the role of religious practices in spreading covid-19 was predominant [69] , [70] . the religious lead-ers surpassing the mass gathering orders became a source of virus carriers in the entire nation. some of the early covid19 outbreaks were traced back to religious gatherings such as daegu church in south korea, bnei-brak in israel, oom in iran, tablighi-jamaat in india, tabligh-e-jamaat in malaysia, many people are fervent in religious practices such as visiting places of worship, mass gatherings, religious celebrations, and many more. all these practices are hindered during the pandemic. entire paradigm shift was seen in the religious fraternity. the religious holidays and celebrations were practiced at home. the key moments of rituals were experienced in their own home. religious leaders were bound to ask their followers to stay at home during pandemics. they started releasing double the amount of messages for the community to cope up with the stress during the pandemic. the religious organizations started doing more charita-ble services. people started living with faith rather than religious places. social distancing would be the most tricky in places of worship. the survey concludes that the public has become comfortable staying at home and practising their faith till the resumption of the normal situation [74] . religious leaders are challenged to foster and to bring their services and communities together in these trying times from a distance. the online platforms were used to connect to the community during religious ceremonies. during the pandemic time, the searches for prayer have skyrocketed in google search engines. many spiritual and therapeutic activities, such as yoga, meditation, martial arts, and conscious dance classes have gone online during this pandemic. these temporary solutions are not sustainable solutions as they need physical relationships with people. the places of worship is a source of income for many religious leaders and the common man. these sources of income are hindered by the pandemic. life without religious practices also hit livelihoods of businesses around the places of worship. a loneliness during the pandemic times created furore among the individuals. people were compla-cent in their comfort zone but they were kicked out of that with hopelessness and despair. adapting to a new environment with a u-turn in an individual's life was a difficult task. life is fragile during pan-demic time but increase in spirituality and faith be-came a vital part of their life. religion is considered as a source of solace in terms of pain and scepticism. the role of prayer in the current pandemic situation among the general public is noteworthy [75] . there was an increased interest ever recorded in search of prayer as per the daily data recorded from google for 95 countries. according to tearfund covid19 prayer public omnibus research [76] conducted in the uk during the lockdown period gauged the responses to spiritual practices. the statistics was conducted on 2,101 uk adults aged 18+ and shows that nearly half (44%) of uk adults pray regularly and a quarter (24%) of uk adults attended online religious service during lockdown. one in twenty uk adults (5%) who attended religious service have never gone to church and twothirds (66%) of uk adults agree that prayer changes the world. generally, religion is more appealing to the older generation, but during the lockdown period the reli-gious revival was seen in younger ones. the highest number of quran apps from google playstore was downloaded during pandemic [77] . irrespective of any religion, everybody started seeking hope in their faith and started praying for various topics as shown in fig. 13 . 9 we humans have gone through multiple virus pandemics in different times. pandemic came with human devastation but with times we came over it. covid-19 is a disaster in many aspects of life, but in some it has proved a blessing. this paper describes the multiple faces of virus outbreak. we have looked upon a few possible areas of life which have been affected by covid-19 such as the educational sector, environmental sector, and religious sector. the areas where it is a boom leaves a space to ponder on the living standard of human beings. lot of effort was taken with respect to some serious problems on the earth, but everything was in vain and it was noticed that there was a sudden break in these problems during a pandemic. once the pandemic is over, there is a call by the earth to make it a better healthy living place. comparative pathogenesis of covid-19, mers, and sars in a nonhuman primate model effects of covid 19 pandemic in daily life rolling updates on coronavirus disease (covid-19) predicting covid-19 in china using hybrid ai model a weakly-supervised framework for covid-19 classification and lesion localization from chest ct deep learning covid-19 features on cxr using limited training data sets accurate screening of covid-19 using attention based deep 3d multiple instance learning dual-sampling attention network for diagnosis of covid-19 from community acquired pneumonia diagnosis of coronavirus disease 2019 (covid-19) with structured latent multi-view representa-tion learning wearable sensing and telehealth technology with potential applications in the coronavirus pandemic easyband: a wearable for safety-aware mobility during pandemic outbreak the impact of covid-19 on consumers: preparing for digital sales a comprehensive review of the covid-19 pandemic and the role of iot, drones, ai, blockchain, and 5g in managing its impact new insights on the antiviral effects of chloroquine against coronavirus: what to expect for covid-19? middle east respiratory syndrome coronavirus (mers-cov): impact on saudi arabia the psychological im-pact of covid-19 pandemic on health care workers in a mers-cov endemic country occurrence, prevention, and management of the psychological effects of emerging virus outbreaks on healthcare workers: rapid review and meta-analysis epidemic on medical staff in china: a cross-sectional study mental health impact of covid-19 pandemic on spanish healthcare workers the impact of novel coronavirus sars-cov-2 among healthcare workers in hospitals: an aerial overview covid-19: disease, management, treatment, and social impact mitigating the psychological effects of social isolation during the covid-19 pandemic loneliness and social isolation during the covid-19 pandemic covid-19 outbreak: migration, effects on society, global environment and prevention predicting the impacts of epidemic outbreaks on global supply chains: a simulation-based analysis on the coronavirus outbreak possible environmental effects on the spread of covid-19 in china the effect of travel restrictions on the spread of the 2019 novel coronavirus (covid-19) outbreak the moment to see the poor the precarious position of postdocs during covid-19 impact of covid-19 on academic mothers coronavirus and exceptional health situations: the first disaster with benefits on air pollution can we expect an increased suicide rate due to covid-19? uncomfortably numb: suicide and the psychological undercurrent of covid-19 covid-19 pandemic: mental health and beyond -the indian perspective facts and figures: mobility in higher education 46 study abroad statistics: convincing facts and figures online learning during the covid-19 pandemic centre for monitoring indian economy long school commutes are terrible for kids the effects of teacher strike activity on student learning in south african primary schools kerala suffers 1 hartal every 4 each one burns rs 200-crore hole in state's economy economic growth and carbon dioxide emissions? quieting of high-frequency seismic noise due to covid-19 pandemic lockdown measures air pollution dropped 28% in bengaluru during lockdown air quality in chennai during lockdown -do we have clues to mitigate air pollution reduction in water pollution in yamuna river due to lockdown under covid-19 pandemic impact of covid-19 mitigation on wildlife-vehicle conflict prioritizing zoonoses for global health capacity building-themes from one health zoonotic disease workshops in 7 countries anthropogenic drivers of emerging infectious diseases insurers see up to 40% fall in non-covid medical claims who and unicef warn of a decline in vaccinations during covid-19 religious polarization, religious conflicts and individual financial satisfaction: evidence from india religious polarization: contesting religion in secularized western european countries india: infections, islamophobia, and intensifying societal polarization new center for public integrity/ipsos poll finds most americans say the coronavirus pandemic is a natural disaster statca of increase in anti-asian sentiment, attacks covid-19 and religious congregations: implications for spread of novel pathogens religion and the covid-19 pandemic god and covid-19 public health response to the initiation and spread of pandemic covid-19 in the united states high covid-19 attack rate among attendees at events at a church-arkansas have prayers changed in lockdown? people of faith answer in crisis, we pray: religiosity and the covid-19 pandemic tearfund covid19 prayer public omnibus research how coronavirus is leading to a religious revival key: cord-032733-5v6kxgv4 authors: bozward, d.; rogers-draycott, m. c. title: value creation programmes: lessons from an early-stage implementation date: 2020-09-26 journal: entrep educ doi: 10.1007/s41959-020-00033-5 sha: doc_id: 32733 cord_uid: 5v6kxgv4 this paper explores the authors’ efforts to design and deliver a venture creation programme (vcp) at the university of worcester between 2015 and 2017. the purpose of the paper is to present critical insights into the manner in which learning can be facilitated through vcps using our own experiences as an example. the paper begins with a discussion of the authors' methodology before moving to a literature review addressing vcp’s and related elements of entrepreneurial education best practice. it then shifts to a critical discussion of the authors' experiences based on insights, student and staff evaluations, and course data. in the process of conducting this review the authors found that good pedagogical design, in a vcp context, needs to focus on ensuring that academics promote constructive alignment between the curriculum that is ‘taught’, the teaching methods used, the learning environment chosen and the assessment procedures adopted. only by minimising inconsistencies which link these factors will students’ experiences of venture creation be fully engaging, promoting more effective, experiential and entrepreneurial development. this paper provides educators with an original and experiential insight into the design of a vcp programme and the pedagogical developments required for their successful implementation. there is considerable value herein for higher education institutions which want to design a new vcp, especially with reference to the curriculum design, experiential learning activities, types and methods of assessments and online learning environments for entrepreneurial students. in september 2016 the university of worcester became one of only a small number of institutions in the uk to launch a degree in entrepreneurship where venture creation was a central, assessed, element of learner participation. there are a very limited number (lackéus and williams-middleton 2011) of this type of 'true' venture creation programme (vcp) globally. the authors define a vcp as a practice-oriented degree, combining the creation of a functioning venture with academic study (lackéus and williams-middleton 2018) . the term 'true' is used in this context to reflect the fact that there are numerous programmes which utilise competitions, tools and activities to simulate the experience of start-up processes (pittaway and edwards 2012; scott et al. 2016 ), but these do not embed a mandate for the creation of a functioning venture at their core and/or make the development of this a requirement for the completion of the course. as this was the newest 'true' venture creation programme (vcp) in the uk, the authors felt that their experiences, the pedagogical challenges they faced and their reflections upon these, might make a useful contribution to the global literature, especially for anyone considering creating a similar programme. therefore, the aim of this paper is to present the lessons learnt from this early-stage implementation to invigorate the discussion surrounding vcps. the paper will do so by exploring a number of issues central to their successful implementation of the programme with a particular focus on the aforementioned pedagogical challenges, including curriculum design, teaching methods, experiential learning activities, types and methods of assessments, and online learning environments for entrepreneurial students. the process of education evaluation is an important part of the design and ongoing development of any course. the processes which govern this are well established (molenda 2003) and described in numerous models of education instructional systems design (clark 1995; sortrakul and denphaisarn 2009) . the range of methodologies used in evaluating education programmes is as diverse as the programmes themselves (lewy 1977; dressel 1980) . the field of entrepreneurship education has been the focus of several publications including fayolle et al. (2006) , vesper and gartner (1997) . these have shown that the evolution of students' attitudes and 'mindset' may be a better criteria than purely evaluating the economic impact or the number of businesses created. as this is an early-stage evaluation of a programme, the methodology adopted should provide the ability to review the outcomes from a longitudinal perspective. therefore, the authors adopted donald kirkpatrick's four levels of evaluation (kirkpatrick and craig 1970; kirkpatrick 1975) , which provides a framework that could be used in stages: 1. level 1 researches the reaction to the learning experience by the learner. this starts with the reaction and reflection of the programme by the students, understanding their likes and dislikes to all aspects of the programme. therefore, the key questions of evaluation are what the student's reaction to the learning environment was. 2. level 2 measures the learning by assessing what students have gained in the intended domain. what principles, facts, and techniques were learned? the key question at this level is whether the students achieved the desired learning objectives. 3. level 3 develops an understanding of the changes in behaviour. what changes over time has the programme developed and solidified? this transfer of practice could be a direct or indirect result of a module or the overall programme. the key question at this level is whether the newly acquired skills, knowledge or attitude are being used by the learner after the learning event is completed. 4. level 4 evaluates the results and the long-term impact of the programme graduate outcome. kirkpatrick introduces the return on investment and the tallying of measurable long-term impacts to a company, gathering of data from the first three levels which correlates with things like graduation rates, job placement rates, and success rates in competitive scholarship or graduate school applications. the key question at this level is whether the student achieved the desired outcomes of the programme. this investigation uses levels 1 and 2 of this framework, providing an evaluation tool in the first year of university education and enabling us to tailor the programme for the later years. levels 3 and 4 could be used on completion of the 3-year programme supporting the authors to gain a complete view of the vcp education intervention. the paper presents data, insights and reflections collected during the first year in which the course ran, after this the staff running the course left, so further information cannot be presented. the subjects were under-graduate students on the vcp entrepreneurship degree who represented a broad range of ages (18-52), social economic backgrounds and a relatively equal mix of genders. the data used in the analyses were gathered from several sources including curriculum reviews, course meetings, student and staff reflections, meeting notes and other conversations. it is important to note that reflection is used at a number of levels with students reflecting on their experiences in and around the curriculum, staff reflecting on their practice, the programme design and the students' development. osterman and kottkamp (1993) state that reflective practice is located within the older tradition of experiential learning and also the more recently defined perspective of situated cognition. experiential learning theorists, including dewey, lewin, piaget and kolb's (1984) maintained that learning is most effective, most likely to lead to behavioural change, when it begins with experience and specifically problematic experience. as part of the methodology of designing this programme we were faced with a complex pedagogical challenge which required us as entrepreneurship education practitioners to take the time to reflect on this experience. as part of our own reflective practice (osterman and kottkamp 1993) we wanted to use this as a framework for understanding individual and organisational opportunities for change. the aim of the vcp programme which the authors designed was to encourage students to learn 'for entrepreneurship' (qaa 2012 (qaa , 2018 developing not only their knowledge and understanding of entrepreneurship and its related skills and abilities but also the practice of entrepreneurial activity. the vehicle for this was gibb's (2002 p. 248 ) concept of 'learning by doing', contextualising and individualising learning experiences through start-up activity, so that these could synchronise with key moments in the students venture development lifecycle (bozward and rogers-draycott 2017) to create moments of understanding (rae 2003) , from which critical insights into the students personal and professional development could be realised and reflected upon. from the outset, the authors were clear that an unambiguous definition of entrepreneurship was important to act as a keystone of programme development, encouraging consistency and continuity which would underpin the course design and development. this was drawn from a board literature (bruyat and julien 2001; fayolle 2007; sarasvathy and venkataraman 2011; lackéus 2016; shane and venkataraman 2007; moberg et al. 2012; qaa 2012) with the focus being on authors who had specifically addressed the action of venture creation, ideally in an educational context, and whose approaches (either practically or philosophically) were aligned to the goals for the programme (fayolle 2007; jones et al. 2014; lackéus 2016) ; as a result, entrepreneurship would be defined as follows: 'entrepreneurship is finding and developing opportunities to create value'. this framed entrepreneurial activity in broad terms and focused it on the creation of 'value' (lackéus 2016) a term which has been left deliberately ambiguous to give students an opportunity to develop ventures that encompassed a range of entrepreneurial action; not just the generation of revenue or profit. the definition is also permissive rather than prescriptive, and it assumes nothing and allows students a freedom to explore their entrepreneurial selves without the constraint of a specific direction (farmer, yao and kung-mcintyre 2011) . this meant that the experience of learning through the programme was most definitely not a 'spectator sport' (tinto 2003) , as students were actively challenged to participate in a cycle of venture creation, reflection and evaluation which informed their development and future activity. this work formed the basis of the pedagogical methods applied across all the modules either directly as content, or indirectly through an influence on programme design and structure. biggs (1996 biggs ( , 2000 describes the duty of good pedagogical design as one of safeguarding that there are no inconsistencies between the curriculum we teach, the teaching methods we use, the learning environment we choose, and the assessment procedures we adopt. this he defined as constructive alignment. previous pedagogical developments within vcps have been researched and published by gibb (1993 gibb ( , 2011 , ollila and williams-middleton (2011), lackéus and williams-middleton (2015) , mwasalwiba (2010) . these investigated the alignment of entrepreneurship education objectives, target audience, teaching methods and impact indicators, and found that greater focus should be put on the pedagogical alignment when setting or designing entrepreneurship education experiences. to position the degree in an international best practice context, other vcps were reviewed during the programme design (lackéus and williams-middleton 2011, 2015) . this work was used alongside the authors' definition, input from the business community, and the latest guidance from the subject benchmark statements (qaa 2012 1 ; qaa 2018) to create the academic foundation for the course. it also provided a means for students and the wider community to understand and compare the context and characteristics of the programme against others around the world. a staged learning narrative was then developed to support the students' entrepreneurial learning (bozward, and rogers-draycott 2017) which drew on the work of kuratko, morris, and schindehutte (2015) to break the entrepreneurial efforts into nine specific steps, each step comprising a number of focal competencies which would need to be met as part of the students' entrepreneurial development. a number of other researchers (bird 1995; man 2007; rasmussen et al. 2011; sanchez 2011) have found value in the concept of 'competence' to support the structuring of entrepreneurial education and assessment. competence, in this form, has a wideranging conception encompassing knowledge, skills, attitudes, behaviours, work habits, abilities and personal characteristics (le deist and winterton 2007) . these focal competencies were then used to underpin the requirements of the curriculum and any assessments at each level within the degree. this framework formed the operational core of the degree, an easily identifiable map of key waypoints for educators and students which needed to be successfully navigated for progression to take place. based on this work, the ba (hons) entrepreneurship programme focused on the active experience of business creation underpinned by relevant academic knowledge and the development of a broad base of entrepreneurial skills and capabilities. this was delivered through a mixture of dedicated sessions, simulations, work-based learning, enterprise events and interactions with entrepreneurs. the programme schedule was designed with formal sessions on one full day each week, thereby maximising the time spent 'on task', while other days were set aside for additional support (mentoring, business clinics, events and team work) and for students to develop their ventures through work-based modules, thereby maximising engagement and, hopefully, student outcomes within the curriculum (lingard 2007; mioduser et al. 2002) . the approach and pedagogy positioned the vcp to develop both the student learner and their business. therefore, the 3-year programme was split into two streams which focused on: 1. the entrepreneur and 2. the business venture. in the first year the programme encouraged ideation and venture creation; in the second year, this moved aside to facilitate a focus on marketing, sales and cash flow stabilisation, while the third year was centred around business growth, innovation and investment. the programme was designed to provide flexibility in the final year so that students could tailor their modules and assessments to suit the needs of their developing businesses. these negotiated modules allowed the students to formulate the learning outcomes over the module, recognising that even at this stage both business and personal needs might rapidly develop. this was complemented by the learning environment provided by the university, consisting of not just the classroom but also space whereby students could develop their businesses. every student was therefore provided a place in the university's business incubator, and they have access to this for the duration of the programme. this location acted as a hub for the broader entrepreneurial student community to engage with the students on this programme. the development of a community (zimmer 1986; kilkenny et al. 1999 ) around the programme was seen as very important in embedding the students within the local business ecosystem. the cornerstone of this strategy was recruiting 15 entrepreneurs in residence (eir) from a diverse range of businesses. these business owners acted as entrepreneurial role models (bosma et al. 2012; lafuente et al. 2007) as it has been shown as an important factor in the decision to start and influence (van auken et al. 2006 ) the growth of businesses. in order to foster an authenticity of experience all of the modules within the programme were taught by experienced entrepreneurs (hopkins and feldman 1989) with further engagement from local business practitioners who provided mentoring and other associated support. these role models added to the student experience while supporting the practice-oriented pedagogical approach. the programme is structured around 50% of the credits each year being achieved from work-based learning (wbl) modules (boud and solomon 2001; raelin 1997) which is supporting the application of the knowledge developed within the taught modules. this structure is now explored. table 1 provides a breakdown of the taught modules and how they were divided between the two streams. in these, first-year students as entrepreneurs develop their mindset and communication skills, while for their business, the process of starting a business and developing a plan. in the second year they develop leadership skills, financial management, advertising, sales and a negotiated learning outcomes module. in the final year, as an entrepreneur the curriculum is focused on decision making, performance and innovation management, and the business focused modules develop growth through marketing and international trade. the work-based modules (shown in table 2 ) and simulated projects were seen as crucial to the development of core venture skills (timmons and spinelli 1999; zhong-wei 2008) and reflective capacity (jack and anderso 1999; harvey and evans 1995) linking the theory and practice of entrepreneurship together in practical endeavours. these were structured with varying levels of staff intervention to match the progression of the students. in earlier years there was a greater element of 'control' (jones et al. 2014 ) and risk mitigation managed by staff to provide a 'safe space' (jones et al. 2014) in which the students could experience failure and be guided through the process. as they progress through the degree, these 'training wheels' were progressively removed and students were challenged in a number of new ways including having to negotiate learning outcomes for modules and contented with new assessment challenges alongside the growth of their ventures. each module required the students to complete a journal of critical reflections (cope and watts 2000; mezirow 1990 ) addressing their experiences at regular intervals, bring together the concept of entrepreneurial moments or learning events, especially when initially viewed as discounted (cope 2003) these series of reflections support the development of higher-level learning. this was used to provide a consistent longitudinal vehicle for reflection of their development throughout the programme. assessment strategies in higher education have been experiencing a period of diversification (fry et al. 2008 ) and best practice development (rust 2002) , meaning that a number of sources are available to inform strategic approaches. for example, pittaway and edwards (2012) highlighted several innovative assessment practices which could be used within these types of programme, and these included self-assessment and peer assessment. carey and matlay (2010) also highlighted best practice assessment strategies from the creative industries which could be applied to entrepreneurship education. both studies developed the importance of industry or degree community in the assessment process and that examinations were not always appropriate for entrepreneurship education. across the programme a mixture of assessment types were utilised in order to maximise opportunities for all students to perform and develop skills relevant to both future study and/or venture development. this was done in conjunction with maintaining the constructive alignment of the programme by developing individual competency aligned to the development of the students venture. it was important to use both a diverse and a consistent set of assessment types throughout the programme. a consistent set of assessments allowed the students' development to be meaningfully measured and also ensured the students' ability to master this assessment type was supported and evidenced. a diverse set of assessments helped to challenge the students' development and provided the opportunity for them to evidence their learning in different contexts which was a key part of the degree's philosophy. consequently, this programme used assessment 'for' learning as well as assessment 'of' learning (gibb 1993 (gibb , 2011 with a clear view to ensure inclusivity. figure 1 demonstrates the mixture of assessments types used on the programme. this programme did not use examinations as an assessment type as this was highlighted in a number of papers, such as pittaway and edwards (2012) and carey and matlay (2010) that it may not be conducive to an academic environment where students were trying to develop original ideas, articulate these, and had them assessed, especially when considering the implementation of constructive alignment. through their removal the authors felt that they had moved a step closer to our pedagogical ambition of developing a more coherently aligned (biggs 2000) learning environment which more closely mirrored the realities of entrepreneurial endeavours. this was achieved by supporting entrepreneurial learning activities that were appropriate to the students' context and educational needs and adapting those which were not. for example, prior to the vcp all of the law and accounting modules accredited for use in the business school used examinations as the main assessment type as it was an external requirement and, therefore, careful selection of and/or re-purposing of the modules was needed to ensure they aligned to the programmes philosophy and design. this required the design of the assessment which was based on the student's own business and not simply a case study example. the assessments then became fundamental to the development of the student's venture as they provided staged points by which the student's own development and understanding was evaluated. this analysis of an early-stage evaluation of a programme was using the methodology proposed by kirkpatrick focusing on level 1 and level 2 of the evaluation framework. to accomplish this the authors took materials gathered and their own reflections (diaries, journals, etc.), student assessment data, student's reflection journal and added to this with feedback from the students, colleagues and the external examiner. this process led them to identify three groups of reflective data which emerged as the data were collected: 1. positive reflections-reflections where content and experience were generally positive; 2. neutral reflections-reflections where content and experience were mixed; and 3. negative reflections-reflections where content and experience were generally negative. from these three groups six elements began to coalesce, and these were presented as key learning moments (rae 2012) using the commonly experienced events which have been encountered within the entrepreneurship learning in the first year of the degree. these addressed important concerns, challenges, achievements, and realisations which have affected the programme and will have a distinct impact on its future form and direction (table 3 ) when the level 3 and 4 analysis is conducted. these six key pedagogical elements will now be discussed in more detail. when the authors compared vcps across the world (lackéus and williams-middleton 2018) there seemed to be an accepted balance between the time spent in the classroom and the time spent focusing on the learner's venture. therefore 50% of the module credits were from work-based learning and 50% from academic study and scholarly activities, aligning the teaching methods to the curriculum. this created a clear narrative focusing the student to work in and on their business, developing the core philosophies of the programme throughout. the taught element was scheduled on the same day each week (wednesday for first years), so that students could maximise the time spent on their business while also having dedicated space to study and develop their other skills. tuesday was set aside to support work-based learning, which included webinars, business clinics and mentoring sessions. friday was used for trading exercises. the students liked this arrangement and most attended tuesday and wednesday throughout the year. those who did not attend the tuesday's programme gained support via webinars and telephone mentoring. to deliver the range of teaching methods the course required a programme team who could facilitate the learning environment and understand the alignment required for successful implementation within a vcp. the team was made up of one programme leader, one enterprise manager, and six part-time staff, all of whom had entrepreneurship experience in starting and running businesses and, therefore, were academic entrepreneurs (de silva 2016). the part-time members were also practising entrepreneurs who were portfolio working with a proven academic teaching background. the diversity of their backgrounds and current experience encouraged the students to develop greater empathy and respect with the teaching team from the offset. this supported the interactions with the wider degree community and role models for the reflective practice. setting out a clear structure, timetable and expectations from the start provided a set of assumptions which the students understood and also appreciated. this generated positive feedback on open days and through the 1-2-1 academic tutoring sessions where students discussed planning their time and business meetings. this 'learning agreement' was seen as one of the core successes between the student and programme as it provided a basis for understanding and development (both student and programme) from which both parties could move forward with over the three years. in summary, to ensure the vcp structure had constructive alignment and fulfilled the goals of a vcp, the curriculum was designed around the starting and growth stages of a business; the teaching used a mixture of professional entrepreneurs and educators; and the innovative learning environment was based around the physical spaces and the strong interactions between the local entrepreneurial business community. • setting student expectations from the start (recruitment) provided students with a known schedule and business like interactions; and • developing a personal learning agreement which ensured students could balance learning, their business, and personal life ensured that students received the attention they required and understood the academic development time they must manage. moore (2012) suggested that education may be as much about social engineering as about personal empowerment. this within an action-based entrepreneurship education programme means that we need integrate the programme pedagogy with the local entrepreneurial ecosystem to encourage the development of a social community around the programme. initially, we developed the worcestershire entrepreneurship ecosystem map and shared this with stakeholders, and this promoted awareness of what was happening throughout the region while simultaneously promoting our programme. every one of the 200 organisations on this map we had a working relationship with. the map used the structure and findings from the babson entrepreneurship ecosystem project research (fetters et al. 2010; isenberg 2010 isenberg , 2011 to develop the entrepreneurial capacity in defined localities; bringing together the policies, structures, programs, and climate that foster entrepreneurship. this tool was used in a number of ways: • mentoring sessions: the map was provided to all mentors, so they could provide a consistent narrative and discussion on the available entrepreneurial ecosystem support and also discuss openly what was available with the mentee; and • business clinics: when discussing any one aspect of support which was being supplied, we could first show the entire ecosystem before drilling down into the details of one aspect. this ensured that the students could gain an understanding of the whole. mentoring was an important element within the portfolio of pedagogy constructs within the programme. mentoring has been shown (st-jean and audet 2012) to develop cognitive and affective learning as well as provide additional experience and competency for the novice entrepreneur. the development of the affective or emotional areas supports their development of self-awareness (bacigalupo et al. 2016) and entrepreneurial self-efficacy (chen, et al. 1998 ) which have been shown to be key factors in developing high-performance entrepreneurs. to support this, positions were advertised for volunteer entrepreneurs in residence (eir) to provide support to the programme and the team, and we received seven applications. each was assessed using a practical interview process which consisted of a networking event with staff and current students who were starting a business. the criteria used were from bushardt et al. (1982) which states (1) that person can help you, (2) he or she has your confidence, (3) you can help that person, and (4) he or she has a successful track record of mentorship. students and staff were asked to provide feedback based on these criteria. this event proved very successful, and we decided to take all those that applied, the only issue being there was a limited diversity. at the end of the first year of operations we had 15 eirs who were from very diverse industries with backgrounds and experience. as a group they had a broad and varied network which supported the introduction to almost all businesses within the country and these contacts had benefited the students. these individuals' mentored the students, provided weekly business clinics and a q&a forum service. all students were offered at least one mentor to support their personal growth, one mentor (megginson 2006) to support the developmental stage of their business, and another to provide local support relating specifically to the industry or physical location of their business. they also had 1-2-1 skills development such as telephone cold-calling skills, business process mapping, and even gone with students to supplier meetings. perhaps, most importantly, these people provided invaluable links between the university and the local business community helping the students to network and integrate. this was evidenced by one student's comments 'one of the best things i've got out of this course so far is the amount of access to key people in my industry. the connections i make on a weekly basis through the ba in entrepreneurship course are incredible.' the eirs in the first year of operation invested financially in the entrepreneurship cohort and also in other university graduates, which has also been shown by schwarzkopf et al. (2010) . the diversity of the students, their ideas and their business requirements could not be met by one person or a single organisation and it was only through this community that educators could provide the ultimate student experience and learning outcomes. it was no longer acceptable to just manage a classroom experience. in entrepreneurship there was very often conflicting advice and views and it was only through knowledge, mentoring, experimentation and experience that a way could be found for the individual student entrepreneur. • the development of a cohort of mentors provided numerous benefits as it allowed the education ecosystem to extend from the classroom, to those supporting the students and the wider business community; and • the development of a simple programme philosophy which could be shared and understood by all stakeholders ensured wider buy in within a greater community. the programme assumes that learning is best when it is active, when it incorporates experience and when it can be shared and supported through collaboration. as such, our pedagogy focus is on problem-based learning (hung, jonassen and liu 2008) , use of authentic assessment and reflective writing (kakouris 2015; jack and anderson 1999; harvey and evans 1995) to develop a lifelong learning approach for our entrepreneurial students. critical reviews of the assessment strategies used in universities and of reflective practice in general (elton and johnston 2002; findlay 2008; palomba and banta 1999) assert that reflection and reflective practice are lacking in the development of a large number of students, but that this can be difficult to encourage as it requires the development (on the students part) of a broad range of skills and faculties to enable them to engage with the process effectively. this use of reflective journals in the development of professional best practice, especially in the disciplines of nursing and education, has been embedded within the curriculum for many years, as it encourages students to integrate theory with practice, appreciate the world on their own behalf, and turn every experience into a new potential learning experience (wong et al. 1995) . with that in mind it was decided that a reflective journal would be used in the two work-based learning modules each of the three years. to support this, reflective writing sessions were also added to several modules, even those which occurred relatively early in the students learning journey in an attempt to maximise their exposure and normalise the experience. eventually, these were linked together using the vle's journal entry subsite using additional material and videos to create an information hub from which students could draw support, advice and guidance. based on this work, a number of students began to adopt the habits of a reflective practitioner early in the programme and developed a useful weekly reflective journal which, they felt, supported their entrepreneurial development. these students were getting feedback from four members of staff on a regular basis and, in their writing, acknowledged the benefits that reflexivity had on their learning and also on the development of their businesses. however, there were a group of students for whom reflection did not come so naturally and further development was required. this, in itself, was a 'reflection' on the nature of reflective practice; it was a lifelong process and an experiential one (kolb and kolb 2005) through which individuals were developing and redeveloping learning. no one student approached it from the same starting point, and most required different levels of support to achieve something meaningful. this meaned that even with support de-engagement could be difficult to avoid, in this context it was apparent through poor-quality reflections and a lack of development in some members of the student population. after speaking to the affected students about the reasons why they did not fully engage with the reflective processes we identified several issues. the first was about being judged, both personally and professionally, something which they at times found difficult to accept. the second was that the students' personal lives could take priority over both academic and business development and they found this difficult to share this as they believed it would affect their marks adversely to admit difficulties. at the core of both of these points there was a powerful realisation for the course team that some students did not engage fully with reflective writing because they simply found it difficult to be 'right' and present a positive image of themselves and their endeavours rather than doing, what they felt, was dwelling on the negatives. with this in mind the team planned to add mid-module review sessions both in class and individually during personal tutoring sessions. these discussions would have been directed towards how reflective practice could support the students learning and ultimately their effectiveness (qaa 2012 (qaa , 2018 as an entrepreneur but only as part of an honest and forthright process. recognising that business school staff were not good at developing reflective practice, we intended to use a professional reflective and reflexive facilitator (from a different subject area such as nursing or education) who would organise regular individual and group sessions and more informal review sessions where students would have discussed reflective issues and investigated different ways of approaching their reflective practice. • reflective practice is a critical part of the development of a practitioner and as such should be an integral part of the assessment and development of the entrepreneur. however, its integration requires insight from other disciplines in which reflection is common practice; and • a clear process of support and reflective development is required to ensure the development of a successful reflective practitioner. beetham and sharpe (2013) state that digital technology is 'like previous innovations, they can be assimilated to pedagogical practice without altering the fundamental truths about how people learn'. the amount of content which is available in digital form has increased, and the ability for students to access this on multiple devices ensures numerous learning opportunities both within and outside the classroom. it was therefore decided that a key element of the degree was the use of virtual learning spaces such as the university's vle (blackboard) to host material including pre-recorded lectures, sessional activities, reading and extensions tasks. the idea behind this was to allow for 'flipped' (sams and bergmann 2013) approach to learning through both online and in class delivery maximising the one day that students had on campus by extending the learning environment beyond this physical limitation allowing for more delivery via video or audio, for example. to facilitate this, early in the degree design process, formats for delivery were created and circulated, vle areas were established, and best practice was shared between staff. however, while some lecturers championed the format, some resisted this because of time, comfort or technical skills. this immediately sent mixed messages to the students as delivery and experience were not consistent. as a result, some disengaged with the process at this stage and only re-engaged sporadically or when forced to for assignments meaning that key elements of the 'flipped' programmes had to be re-calibrated at the last minute to ensure that students who were not engaging online did not miss any delivery. this in turn led to repetition and also frustration for the students who as 'early adopters' had engaged fully with the online experience. having spoken to staff and students it became clear that students saw the vle as a positive, but the mixed delivery methods were confusing and staff saw it as too time-consuming even given the formatting help and support that was provided to them. however, the first year of a module had considerable amount of development and subsequent years had less. to mitigate this risk the team intended to provide a video introduction for the students on how to use the vle within the programme homepage and additional support during the sessions throughout the year. also, there were intentions to experiment with a 'flipped light' version of the delivery process to create a lower bar for staff to achieve while retaining a consistent approach across the teaching team; this would be more of a blended approach allowing staff with the skills to do more and staff who found it difficult to be supported and take other routes to achieving an engaging product. the use of the vle, learning technologists, video creation support, and peer educator support through the programme delivery team was an important development for the programme, which has been used subsequently during the planning of the covid-19 response. the key to this would be that it was made clear to the students how each module would approach the issue and that this would remain consistent across the delivery of that module. based on student feedback it was also decided to deliver one of the first modules in the programme via a portfolio assessment based solely within the vle. everyone hoped that this would ensure that the students engaged within the vle more fully during the crucial early part of the degree. to achieve this, we had planned a structure which meant that the students would be fulfilling regular tasks towards their assignment promoting engagement, familiarity and development within the structure. • the professional development of the teaching team so as to provide a consistent pedagogy approach within the vle; and • a dedicated session to introduce the vle and the benefits of using it for students. a learning community has been defined as 'an intentionally developed community that exists to promote and maximise the individual and shared learning of its members (beck and foster 1999) . they provide an ongoing interaction, relationship, and collaboration among the community's members as they work towards a specified common learning goals' (lenning et al. 2013, p. 7) . learning communities offer learning of an interactive nature and targeted to develop multiple social dimensions, such as employability, general enterprise awareness, intrapreneurship, and venture creation, all of which are becoming a major metric within today's uk education environment. the team-based project module began in the first-year induction week so that insurance and other logistics could be resolved before the teams moved forward. students were provided with a module manual which detailed the structure of the module including the standard module information plus additional information such as company structure, insurance requirements, locations, programme timings, team processes, and contact details. however, the module itself did not start until late october, a month, or so later. the delay had been designed to allow the student to gain some theoretical knowledge from the venture creation module which it was felt might support the group work; however, because the link between theory and practice was not yet clear in the students' minds a lot of this work was wasted as they did not really see how the elements linked together. this 'gap' in delivery was further compounded by new students joining the programme late and others leaving it within the first few weeks of term all of which meant that the team formation processes were never really completed; this led to a lack of team structure within the groups throughout the module. additionally, the selection of products to sell on the market stall was left up to the teams. this created problems as the team were newly formed and personalities dictated what got sold and not a logical approach based on customer needs and market demand. each team purchased stock to sell which never sold and detracted from the core tasks of the module of sales and marketing and led the team into a negative mindset and leadership issues. finally, the leadership of the groups moved from one person to another as the module progressed and it was clear that this enabled the team to use the skills to their advantage but also shifted responsibility from one to the next without a clear set of reflections on what went wrong. it was evident that teamwork and this group-based assignment needed further development. therefore, as part of the wider curriculum review, a research project was developed across all entrepreneurship modules to evaluate how to improve entrepreneurship team-based modules which consisted of 131 students. the core findings of the research project have led to the following future actions for this team-based project modules: • a lecture session and video were created to introduce the teamwork aspects of the students. three core aspects will be covered, (1) what makes a good team member (2) what makes a good team, and (3) why good teams get good marks; • a team-building activity will be built into the module. once this is concluded, no other members will be allowed to join; • team size will be between two and four people, starting with four; • facebook will be used for internal communications from the start. (the groups which used this were more successful in their grades); • it is intended to use a similar system as in the apprentice tv programme, where a limited selection of products will be presented to the teams and they then pitch for and then buy the products using their own money; and • a monthly formal board meeting in our boardroom whereby the company reports to the shareholders and publishes minutes. these will then form part of the formative assessment. non-attendance and non-contribution then get reported regularly. • a more structured implementation of the learning communities provides students with how to be a productive member and what others expect from them; and • making the learning community report their actions ensure that all members are engaged in this action-based education. engagement has emerged as a key metric measure (kahu 2013 ) of higher education impact over the last decade. it has become a catch-all term most commonly used to describe a compendium of behaviours characterising students who are said to be more involved with their university community than their less-engaged peers. engagement refers to the time, enthusiasm, and resources students devote to activities both in and outside the curriculum to enhance learning while at university. therefore the pedagogy design of our programmes requires a range of activities which include lectures, tutorials, online vle, webinars, personal academic tutoring, team activities, through to mentoring, business clinics, and events which align with the needs of the student and also the learning outcomes. ahlfeldt, mehta, and sellnow (2005) found that the level of engagement was typically higher in those classrooms with more problem-based learning (pbl) and that engagement increases as programme level increases. as a result it was expected that as the programme was targeted to encourage learning 'for entrepreneurship' (a pbl approach), it would have a better result, but because there had only been a completed a first year, this might be balanced by students 'finding their feet' and not yet fully engaging. it was expected that truly entrepreneurial students will be hard to keep in a classroom and, as a result that we might find it more difficult to foster a learning community. however, we did not anticipate: • one student in week three gained a contract for £18,000 which meant they were away in south east asia for over one month; • two students had childcare issues for over one month which were the result of student loans being delayed; • one student had to go and manage their father's estate after being incapacitated, so they left for another month; and • one student was diagnosed with diabetes and learning difficulties and had over one month off dealing with these issues. student engagement is not a static metric and nor is it a single entity which can be easily quantified. the level of engagement changes on a weekly basis and student engagement is physically, mentally, and also virtually. therefore, student engagement has to be measured and evaluated continuously to gain feedback and develop an ongoing understanding of the situation. the tools used included virtual learning environment, surveys session sign in sheets, class appraisals, posit-it feedback, module evaluation, personal academic tutoring and programme meetings. as in many programmes there was a direct correlation between those attending classes and the grades achieved at the end of the year. extra-curricular activities were attended by around 50% of the students on a weekly basis which for some programmes was extremely high, but as it was part of a work-based learning programme, it was felt this could be improved. to develop both higher student engagement and also be able to monitor this in a less intrusive manner: • the extra-curricular activities had been planned to be added to the reflective journal so that it was part of the assessment, rewarding good behaviour early on in the programme, and developing good habits; • the first year, first semester mindset module would use a portfolio assessment which would require students to attend each session to be able to complete the portfolio. this would ensure students were active throughout the first semester and engaged in this innovative and thought provoking module; and • mentoring sessions would require the completion of form which highlighted the tasks the student would complete before the next session. this would be handed into the enterprise manager, uploaded to the vle and followed up before the next mentoring session. • the use of extra-curricular and in-curricular activities to provide a reflective portfolio for the student to develop good habits and assessment material; and • the use of portfolio assessments within the first year provided an opportunity to build a better rapport and learning relationship with the student. this paper reviewed the pedagogy used to deliver a 'true' venture creation programme (vcp) at the university of worcester. this practice-oriented degree combined the creation of a functioning venture with academic study (lackéus and williams-middleton 2018) . therefore, the purpose of the paper is to provide a critical overview of the ways in which learning can be facilitated through vcps and focused on six core aspects of pedagogy: the programme structure, degree community, reflective practice, virtual learning spaces, learning communities and student engagement. the structure is the first stage in developing a constructively aligned programme, and this was done by setting the programme definitions, taught schedule, and the use of academic entrepreneurs (hopkins and feldman 1989) . however, for a successful implementation the alignment of the students' expectations was critical, especially within a vcp that was creating an environment where managed discourse and opportunity development were the norm. this was a twoway accord with the learning agreement ensuring the commitment from the student. the programme structure must be the stabilising core for all those involved with the vcp, especially the students and the wider business community. the aim of this degree community was to provide a seamless ecosystem of support to the students and was essential in building self-efficacy through mentoring and business opportunities through networking. however, this community needed to gain awareness and understanding of the programme and this had to be done through regular communications. in our case we provided a single programme approach which successfully allowed a wide range of stakeholders to engage. the reflective practice of an entrepreneur is an important aspect for the development of lifelong learning and aligns with our pedagogic focus on problem-based learning. reflective practice has been widely developed in several other disciplines, and therefore, the authors drew on this to propose the use of multidisciplinary teams to develop the effective entrepreneur reflective practitioner. the majority of the reflection was done within the virtual learning spaces as part of an assessment. the virtual learning spaces provide a considerable opportunity to develop resources for students on a vcp. however, the induction for a student starting to use it and the multidisciplinary staff developing material for it, requires considerable thought and management of initially the students, staff and the technology. the use of learning communities within a vcp provides a considerable opportunity for students to work in teams for their greater interaction, relationship, and collaboration among the course members. however, this requires a structured implementation, such as induction sessions, regular reporting, and team-building activities for it to be successfully implemented. student engagement within a true vcp requires a number of tools to build the attendance habit early on, such portfolio assessment within the first semester first year. the development of extra-curricular and in-curricular activities throughout the years provides additional incentives to attend and also engage, providing advantages in terms of education attainment and venture creation. in summary, this paper provides an important insight to the broader ongoing discussion surrounding the development of experiential entrepreneurial education programmes, especially contributing to the development approach of new true vcp and the critical issues around their design. the paper also focuses on how ensuring constructive alignment is a strategic aspect of the pedagogy design for the success of a vcp and its students. measurement and analysis of student engagement in university classes where varying levels of pbl methods of instruction are in use entrecomp: the entrepreneurship competence framework administration and community: considering challenges, exploring possibilities rethinking pedagogy for a digital age: designing for 21st century learning enhancing teaching through constructive alignment aligning teaching for constructing learning towards a theory of entrepreneurial competency entrepreneurship and role models work-based learning: a new higher education developing a staged competency based approach to enterprise creation defining the field of research in entrepreneurship picking the right person for your mentor creative disciplines education: a model for assessing ideas in entrepreneurship education? education + training does entrepreneurial self-efficacy distinguish entrepreneurs from managers places of inquiry: research and advanced education in modern universities. california entrepreneurial learning and critical reflection: discontinuous events as triggers for 'higher-level learning learning by doing-an exploration of experience, critical incidents and reflection in entrepreneurial learning academic entrepreneurship and traditional academic duties: synergy or rivalry? studies in higher education improving degree programs: a guide to curriculum development, administration, and review assessment in universities: a critical review of research the behavioral impact of entrepreneur identity aspiration and prior entrepreneurial experience entrepreneurship and new value creation: the dynamic of the entrepreneurial process assessing the impact of entrepreneurship education programmes: a new methodology the development of university-based entrepreneurship ecosystems: global practices reflecting on reflective practice. pbpl paper a handbook for teaching and learning in higher education: enhancing academic practice enterprise culture and education: understanding enterprise education and its links with small business, entrepreneurship and wider educational goals in pursuit of a new 'enterprise' and 'entrepreneurship' paradigm for learning: creative destruction, new values, new ways of doing things and new combinations of knowledge concepts into practice: meeting the challenge of development of entrepreneurship educators around an innovative paradigm: the case of the international entrepreneurship educators' programme (ieep) strategic windows in the entrepreneurial process changing entrepreneurship education: finding the right entrepreneur for the job problem-based learning how to start an entrepreneurial revolution the entrepreneurship ecosystem strategy as a new paradigm for economic policy: principles for cultivating entrepreneurship. dublin: institute of international european affairs entrepreneurship education within the enterprise culture: producing reflective practitioners claiming the future of enterprise education framing student engagement in higher education entrepreneurship pedagogies in lifelong learning: emergence of criticality? learning reciprocated community support and small town-small business success education training programs evaluation of short-term training in rehabilitation experiential learning: experience as the source of learning and development learning styles and learning spaces: enhancing experiential learning in higher education understanding the dynamics of entrepreneurship through framework approaches value creation as educational practice-towards a new educational philosophy grounded in entrepreneurship venture creation programs: entrepreneurial education through real-life content venture creation programs: bridging entrepreneurship education and technology transfer venture creation programs list | we list venture creation programs regional differences in the influence of role models: comparing the entrepreneurial process of rural catalonia what is competence? human resource development international powerful learning communities: a guide to developing student, faculty, and professional learning communities to improve student success and organizational effectiveness handbook of curriculum evaluation understanding entrepreneurial learning: a competency approach mentoring in action: a practical guide fostering critical reflection: a guide to transformative and emancipatory learning models of pedagogical implementation of ict in israeli schools impact of entrepreneurship education in denmark-2012. odense: the danish foundation for entrepreneurship-young enterprise in search of the elusive addie model. performance improvement teaching and learning: pedagogy, curriculum and culture entrepreneurship education: a review of its objectives, teaching methods, and impact indicators the venture creation approach: integrating entrepreneurial education and incubation at the university reflective practice for educators assessment essentials: planning, implementing, and improving assessment in higher education assessment: examining practice in entrepreneurship education enterprise and entrepreneurship education. guidance for uk higher education providers opportunity centred learning: an innovation in enterprise education? education + training action learning in new creative ventures a model of work-based learning the evolution of entrepreneurial competencies: a longitudinal study of university spin-off venture emergence the impact of assessment on student learning: how can the research literature practically help to inform the development of departmental assessment strategies and learner-centred assessment practices? flip your students' learning university training for entrepreneurial competencies: its impact on intention of venture creation entrepreneurship as method: open questions for an entrepreneurial future how entrepreneurs-in-residence increase seed investment rates a critical perspective on learning outcomes and the effectiveness of experiential approaches in entrepreneurship education: do we innovate or implement? education + training the promise of entrepreneurship as a field of research the evolution of instructional system design model the role of mentoring in the learning development of the novice entrepreneur new venture creation: entrepreneurship for the 21st century learning better together: the impact of learning communities on student success. higher education monograph series the influence of role models on entrepreneurial intentions measuring progress in entrepreneurship education assessing the level of student reflection from reflective journals an empirical study of the relationships between emotional conflicts and entrepreneurship of top management team and business growth performance the art and science of entrepreneurship publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations authors' contributions all authors participated in data collection and analysis including writing and approving the manuscript for submission. conflicts of interest all authors declare that we have no conflict of interest. key: cord-192882-41grtlfh authors: dittrich, jens; maltry, marcel title: database (lecture) streams on the cloud: an experience report on teaching an undergrad database lecture during a pandemic date: 2020-10-09 journal: nan doi: nan sha: doc_id: 192882 cord_uid: 41grtlfh this is an experience report on teaching the undergrad lecture big data engineering at saarland university in summer term 2020 online. we describe our teaching philosophy, the tools used, what worked and what did not work. as we received extremely positive feedback from the students, in the future, we will continue to use the same teaching model for other lectures. in february 2020, many german universities decided to move to virtual teaching due to the unfolding covid-19 pandemic and the public lockdown. as the summer term approached, lecturers had to decide relatively quickly how to 'give all lectures online'. this included our cs department and our research group at saarland university (big data analytics group). we already had quite some experience with teaching online: in 2013, we started creating and using screencast videos for our lectures. from the beginning, we made our videos publicly available on youtube. our channel 1 currently has 11,000 subscribers and is one of the most successful database channels worldwide. we already reported on that endeavor and in particular our inverted classroom philosophy in a previous report [dittrich(2014) ] in this journal. however, since then, and given the experience with numerous inverted lectures, we decided to revisit all design decisions taken in the past. in this experience report, we describe what we did and why we did it. for each decision, we briefly discuss the alternatives and their pros and cons. we do not claim that our model is the single right way to teach online. there is neither a single right way to teach well offline nor online; there are a zillion different ways. the challenge is to find the teaching style that fits you best. in summer 2020, we found a teaching style that fits well for us -and as we learned from the students' evaluations -it also fits for our students. with this work we would like to inspire others seeking to find the right bits and pieces for their own teaching. this paper is structured as follows: we explain our teaching philosophy in section 2. our teaching material is discussed in section 3. the hardware and software used are presented in section 4. in section 5, we proceed to discuss tutorials. we summarize advantages and issues with our teaching model in section 6. finally, we give an outlook to future plans in section 7. a major and common pitfall in online teaching is to first start discussing technical tools, and specific software, and only then consider the pedagogy and teaching philosophy behind it as a second thought. like that, technology and software will define the pedagogy. so let's first think about the teaching philosophy and pedagogy. there are two extremes: 1. flipped (aka blended or inverted) teaching. in flipped teaching, students receive material for self-study ahead of time. the actual class is then used as a lab to work on exercises with the professor and ph.d. students 2 . 2. live teaching. the other option is to run the lecture live over the internet. this may be done as an online version of the physical lecture where content is 'streamed' from the lecturer at the speaker's desk to the people in the lecture hall. of course, all kinds of hybrids in-between these two extremes are possible. however, a major concern with all these approaches is the degree of interaction with and among the students. a considerable strength of flipped teaching is the lab which is heavily interactive and perceived to be very useful by many students. however, during the self-paced self-study phase, questions to the lecturer are not possible: the students are basically alone with the material. if they are stuck, issues might not be resolved quickly. in contrast, a major strength of live teaching is the potential interactiveness: the lecturer can spontaneously ask questions, form short working groups, resolve urgent issues immediately, react to witty comments, and then depart in completely different directions, and so on. like that, live teaching really becomes a live event that goes beyond a unidirectional stream of content from a lecturer to an audience. so what is the 'right' solution here? again, there is no single 'right' solution here. we debated back and forth and eventually decided to go with live teaching. besides the potential of interactive elements, one reason for this decision was also the perceived high effort for creating video material for an entire lecture in a short period of time. until 2013, our lecture relied on material and examples from [kemper and eickler(2015) ]. in 2013, we created our own material based around a hypothetical photo agency. in 2019, we again created new material from scratch based on the following idea: we did not want to primarily organize the lecture along different topics anymore (first er, then relational model, then relational algebra, etc.) as typically done, as we felt that this did not necessarily motivate students to fully appreciate the power of database technology. instead, we decided to motivate certain topics by different applications. every two weeks, we would pick a different application. this means, we would first show an application or problem and then showed planned structure for every two weeks of lectures: 1. concrete application: xy 2. what are the underlying the data management and analysis issues? 3. basics to be able to solve these problems how certain techniques from the database world would solve exactly that problem. then, we would spent quite some time to make the transfer: how does that technology help in this application? this principle structure is summarized in figure 1 . table 1 shows the applications discussed and their mapping to topics. as saarland university decided to shorten summer term 2020 by starting only in may rather than in april, we also had to shorten the material. in addition, all courses had to be designed to allow all students to start in may only. if we provided material before may, we had to make sure to go through it again in may. due to these additional constraints, we decided to offer material for students who did not know python yet as we planned to use jupyter notebooks to explain certain concepts in the actual lecture. that python introductory material was then again repeated in two lectures in may. moreover, throughout the lecture we also recommended old videos from 2013/14 to students in case they wanted alternative explanations. for all material that we created, we paid attention that our notation was consistent with [kemper and eickler(2015) ] in order to enable students to easily lookup yet another explanation in that book. also note that we left out considerable material that we felt like is not up to the reality of modern databases anymore, e.g. normal forms whose importance can be debated in the light of modern non-scalar sql-types 3 . all videos, including the older ones from 2013/2014 are publicly available on our youtube-channel 4 . the pdfs of our slides are available through our website 5 . if you are a lecturer and want to have access to the sources, send us an email. we also created jupyter notebooks 6 which were shown in actual lecture and used for the exercises. python (part 1, videos and/or 5.5.) basics, functions, functional programming imdb (part 1, 7.5.) data modeling, relational model python (part 2, videos and/or 12.5.) object-orientation, unit tests and automatic testing imdb (part 2, 14.5.) relationale algebra nsa (part 1, 28.5.) introduction to sql nsa (part 2, 4.6.) analytical sql, big data arithmetics, big data vs privacy, counter-measures query optimization (part 1, 18.6.) automatic query optimization, physical operators, heuristic optimization query optimization (part 2, 25.6.) cost-based optimization, join order, plan variants, pipelining, physical optimizations trade, banks, ticket system (part 1, 2.7.) database management system (dbms), transactions, serializability theory trade, banks, ticket system (part 2, 9.7.) two-phase locking (2pl), isolationlevels summary (16.7.) table 1 : course agenda and their learning objectives. in 2019, we used one additional application: data journalism. there we explained graph databases and security issues like sql injection. in this section, we describe the hardware and software we used for our lecture. in terms of hardware, we experimented a lot until converging on the following setup. we used an existing 2016 macbookpro, a 32 inch monitor, a thunderbolt dock (elgato thunderbolt 3 pro dock), a dynamic microphone with internal pop-filter (rode procaster), a mike preamp (tritonaudio fethead), a sound interface (focusrite scarlett solo 3rd gen), and two speakers (yamaha hs7 standard). that's it! an existing ipad pro was not used in the lecture 7 . the office used for streaming was a standard office room without any extra soundproofing (one exception: see section 4.3). with this setups the sound quality of the audio is awesome and makes a huge difference over any setup we used before. in particular, previous issues with background noise are gone. the same holds for the camera setup which outperforms any dedicated webcam we tried. an important and surprising issue we ran into early was how to connect to the internet. initially we had quite some issues when connecting the computer via wifi. after experimenting with several setups, it became clear that wifi is simply not stable enough for live streaming purposes. this is not so much a problem of bandwidth but of channel conflicts and electromagnetic interferences from other devices, as well as latency issues. though we carefully debugged the wifi connection, however, eventually we concluded that a wired ethernet connection should be preferred over wifi whenever possible. the same holds for audio and video conferences. as lan was not available in some home offices, we used a lan via power option (avm fritzpowerline 510e set). though powerline may lower the available bandwidth, it is much more stable than wifi. in general, in terms of internet bandwidth note that even a relatively weak but stable uplink of 2mb/sec may be enough to stream a lecture that only shows slides in the video stream. if you want to stream camera input, you need a higher bandwidth though. initially we planned to use zoom for the actual lecture and additionally stream the zoom lecture to youtube. this can directly be done from any zoom meeting. however, shortly before the semester started we decided against using zoom for several reasons: (1) ongoing discussions on privacy and data protection issues with zoom, (2) relatively poor audio/video quality due to heavy lossy compression, (3) impossible to make the youtube stream publicly available if at any time clear text names or webcam videos from students can be seen in the stream. similar concerns are true for ms teams. after some brief investigation it turned out that the privacy and security issues with ms teams were at least comparable to zoom or even worse, e.g. sso with cleartext university password on microsoft.com website. in addition, we experienced that the video and audio quality of ms teams is simply not acceptable: we observed super-heavy compression artifacts, dichotomy of picture and sound. in addition, we found the ui of ms teams to be extremely confusing. for the lecture, we decided to use a solution that fixes all of the above issues: youtube's livestream feature. in order to send a stream from your computer you have to install a streaming software locally. there aren't many options. we decided to use open broadcaster software (obs) 8 , see figure 3 . commercial options are also around, e.g. wirecast. obs worked well for our purposes. the only problem we ran into was that obs is not really optimized for macos. this results in a relatively high cpu load which in turn leads to notable fan noise. we had to fix this by positioning the macbook as far away from the microphone as possible and improvising some soundproofing between the macbook and the microphone (a chair covered by a thick blanket). by default, the head of the youtube stream is cached to be able to cope with network failures. we observed a latency of about 10 seconds. we configured this to a shorter value to allow for more interaction with students (see section 4.4) 9 . we also disabled the livestream chat to avoid spam; video comments were allowed though. this setup has many advantage: we received great audio and video quality (1080p). we could stream audio and video without any notable compression artifacts or distorted audio. as the stream is automatically archived while being streamed, students can also pause the video, watch it time-shifted or at any point in time later. in addition, all video format issues for different devices are automatically handled by youtube. moreover, we do not put any extra load on the university network (if a video is watched from outside the network) which would have to be paid by our university (if billed per volume which is true for saarland university). to allow for interaction with the students, we required an audience response system. we wanted something simple: a moderated chat where students could type in questions, a moderator (the tutor in chief) would select suitable questions, and forward them to the lecturer who would then drag the question to the stream and answer them live during the lecture. many different tools are available in this space, and after some investigation we decided to use frag.jetzt 10 . it worked well for us. for this tool only the lecturer has to log in to make sure that for each lecture students can use the same url. see figure 4 for an example. the only problem we encountered with this tool is the missing possibility for the lecturer to ask back within the tool. as lecture system, i.e. the system where students register, where all links to all materials are available, and where homework is handed in, we used an existing internal system. this system has been in use at the department for more than ten years for physical courses as well. it is in many dimensions superior to other systems like moodle (which we used for three years). the biggest advantage is simplicity. in contrast, moodle's ui is too complex and confusing. it is sometimes hard to find things and in the student's evaluations it was often heavily criticized. throughout the course, we used additional tools when needed. in the following, we summarize three tools and provide use-cases for each of them. we used discord for office hours, tutorials, and mini office hours during the lecture break and right after each lecture. discord is an audio conference tool widely used in gaming community. the idea is to define virtual rooms which depending on your access rights, users are allowed to join anytime. once you enter a room, anyone in that room can hear what you say and you can hear immediately what others in that room are saying. though, at first, this sounds a bit like a standard video conference tool, in fact, it feels more like walking through different (physical) rooms. discord also allows for screen-sharing and video camera support. discord was popular among students due to its familiarity and popularity in the gaming community. to join our server, students needed an invitation link that was only visible for students registered in our cms. we 'boosted' 11 our discord server to improve screen-sharing capabilities with a better resolution. in addition, we installed a paid third-party discord bot called voicemaster+. it allows students to create their own temporary voice channels. this was used by students to watch the lecture stream 'together' to have a bit of a lecture hall feeling, in tutorials to create breakout sub-rooms, and for students' learning groups. moreover, we also implemented multiple-choice questions for text channels (pollbot). this was heavily used in the tutorials. we also considered using zoom or ms teams for tutorials (cf. section 4.3). we announced initially that we would only consider these tools as a backup. in the end there was no need for this and the tutors and most students were happy with discord. sometimes we observed stability issues with discord. in addition, screen sharing and video is still a bit buggy and far behind zoom's abilities. still, discord is a great and very helpful tool that will surely further improve in future versions. we used our cms allowing students to give anonymous feedback. this was used about ten times in total in the entire semester. we provided a textual forum. for this we used discourse 12 . we witnessed several interesting discussions and witty student's comments. we made sure to answer questions quickly and the lecturer involved himself into answering the hard questions. due to covid-19, the forum was more frequently used than in previous years. in the tutorials, our students also used a tool for shared scribbling: a web whiteboard (aww) 13 . obviously, this sounds a bit like we used a zoo of tools -which is actually true. even though this is only a very small zoo, it still is a zoo. to fix that problem, we tried to integrate the tools wherever possible and make everything easily accessible. in the end, all the student-facing tools were web-based anyways, so the divide among the different tools was not as severe as it would have been for different desktop applications. in addition to the lecture, we wanted to give the students the opportunity to deepen their understanding of the course material and ask questions. therefore, we offered tutorials, office hours, and assignments. in the following, we first discuss our requirements for each of the offers and then explain how we implemented these requirements with the tools discussed in section 4. in the past, we offered tutorials for up to 30 students in seminar rooms at our university. students could vote for their favorite time slots in our cms and were then assigned accordingly. this year, tutorials were held on our discord server with a similar number of participating students. for some time slots two tutorials were offered simultaneously. therefore, we created two categories, one for each virtual seminar room. each category was structured as follows: plenum voice/video channel: participants and tutor meet here at the beginning of the tutorial. the tutor shares screen, gives instructions, and moderates the discussion. temporary voice/video channels: students form groups of up to five to discuss exercises. plenum text channel: tutors can send files and give instructions in case students are not in the plenum voice channel. students can ask the tutor to join their temporary channel for questions. in previous iterations of this lecture, tutors presented the solutions to the assignment handed in most recently. however, we observed that students tended to be very passive and simply wrote down the solutions in these types of tutorials. in fact, we could have simply provided a prerecorded video explaining the solutions but we wanted the students to participate in the tutorial actively. therefore, we decided against presenting solutions and instead prepared additional smaller exercises that could be solved during the tutorials. these exercises were explained by the tutors in a presentationstyle screencast. depending on the type of exercise, they were either discussed in the plenum or students were given 10 minutes to form small breakout groups and solve the exercises. we prepared exercises of the following three types: 1. multiple-choice exercises: these exercises were usually discussed in the plenum. tutors would use our pollbot to post a poll on the text channel and students would vote by reacting to the post. these exercises were especially helpful to gain insight on which topics were already well understood. 2. written exercises: these exercises were solved in smaller groups using temporary channels. students had to apply concepts from the lecture to small problems. they often used aww to work together on a solution. 3. discussion exercises: we often asked open questions that engaged the students to participate in a discussion, either in the plenum or in small groups. we learned that providing exercises to be solved in smaller groups significantly helps students to actively participate in the tutorials. we plan to also keep this approach once tutorials are held physically at the university again. from previous iterations of the lecture, we learned that encouraging the students to continuously recap and apply the concepts presented in the lecture leads to successful participation in the course. therefore, we made correctly solving the majority of weekly assignments a prerequisite for admission to the exam. assignments should not only verify that students understood the concepts but also challenge good students to think further. thus, assignments were divided into three parts: 1. application tasks: students had to directly apply concepts from the lecture to familiar problems. 2. transfer task: students had to apply concepts to new problems and argue about implications. 3. programming task: students had to implement algorithms in empty cells in the jupyter notebook from the lectures. we provided basic unit tests for them to check their implementations. assignments were published on, handed in, and graded using our internal cms after each hand-in. since we chose a different approach for our tutorials this year, all solutions to the assignments were also published on our cms. as the lecture is aimed at undergrad students, we wanted to support them in practicing teamwork. we decided that assignments could be handed in by groups of up to three members. since meeting physically to work on the assignments was not an option, we created a workspace category on our discord server where students could create temporary channels for video calls, sharing their screen, and discussing the assignments. two years ago, we introduced an office hour. in contrast to the tutorials that have a fixed structure, we wanted to provide the students with a more open offer to stop by, discuss individual problems, and get help with technical issues. we held office hours once a week for two hours in a seminar room. however, most of the time, participating students just wanted to solve the assignment on-site and have a contact person in case of difficulties. therefore, we decided to offer office hours of one hour each. office hours were held by two tutors on our discord server. participating students would enter a waiting area (i.e. a public voice channel) and tutors would then accompany them to a private table (i.e. a temporary voice channel). the temporary channels were configured such that other students could not join them to avoid distractions and keep conversations private. if a student wanted to join the conversation, the tutor had to move the student actively to the channel. students mainly used office hours to receive help with technical difficulties, mostly setting up our environment for the jupyter notebooks. they shared their screen in the temporary channel where tutors then guided them towards a solution. having shorter virtual office hours also resulted in fewer students with concrete questions and problems taking part. in summary, we are very happy with our teaching model. we also received tremendous positive feedback from our students in two different course evaluations. 6.1 advantages 1. we avoided many of the privacy issues with zoom and teams. 2. the streams and their recordings are easily accessible, support time-shifted viewing, and are automatically archived. 3. there is no need to organize video servers, no load on the university's network (if watched from outside uni), all audio/video-format issues are automatically handled by youtube, e.g. to watch videos on different devices. 4. as we added video agendas in the video's comment sections, students could easily re-find things in the archived streams. 5. the tutorials with discord were sometimes perceived to be better(!) than physical tutorials. the main reason was that when working in small teams, students were not distracted by noise from other groups. 6. the public videos on youtube are a great advertisement for our department and university (and let's hope for relational database technology in general). 7. videos are replacing textbooks. our youtube videos are used by students from other universities: rather than learning material from classical textbooks, they now use videos to prepare for exams. 1. for the lecturer, teaching without seeing the students was extremely exhausting. the main reason is probably that important communication channels for feedback were missing: e.g. the expressions on student's faces, grumbling, laughing, etc., see also "when everyone laughs for themselves" [kã¼hl(2020)]. teaching often felt like talking to a wall. sometimes while teaching you forget that you are speaking to two-hundred people. 2. for students, teaming up was sometimes difficult if they were not physically present on campus. this is not so much a problem in the 4th semester (as in this lecture) if they already found peers to team up with. yet, this seems to be a severe problem for 1st semester students who may not know many peer students yet. in future, though we are not planning to change much, we aim to further improve the concept and fix the remaining issues. here are some thoughts. 1. how to fix the missing visual backchannel problem? it would help the lecturer a lot to be able to see at least the faces of a subset of the students 14 . one solution could be to offer some students to additionally join a video conference call (zoom or discord) where they must switch their camera on. however, it is unclear what the incentive for students could be. in particular, that incentive should be something that does not penalize students not willing to join that video call. 2. in online teaching, there is an epic debate on whether the lecturer should switch the video camera on. what is the added value of this? in this lecture, we did not use the camera. however, we believe that, in particular for new students, it may be more helpful than for older students. therefore we are considering this for an upcoming first semester lecture. 3. due to many requests on youtube, in the long run, we want to make all material available in english. die umgedrehte vorlesung -chancen fã¼r die informatiklehre kã¼hl s (2020) wenn jeder fã¼r sich allein lacht did you ever see students in a physical lecture hall who completely covered their faces? this would not be acceptable in western societies key: cord-031388-rwj3c3vq authors: lee, chung-hao; liu, yingtao; moore, marc; ge, xun; siddique, zahed title: enhancement of stay-at-home learning for the biomechanics laboratory course during covid-19 pandemic date: 2020-09-03 journal: biomed eng education doi: 10.1007/s43683-020-00025-w sha: doc_id: 31388 cord_uid: rwj3c3vq nan the outbreak and rapid spreading of the coronavirus disease 2019 (covid-19) in the u.s. disrupted american education enterprise with a majority of higher-education institutions physically shutdown and students and faculty transitioning from in-person lectures to remote learning during spring 2020. 1 experimental laboratory courses provide biomedical engineering (bme) students essential experiential learning experiences to enhance critical thinking and innovation and to explore deeper engineering concepts. 2, 3 however, the covid-19 pandemic has forced most bme programs to implement social distancing by switching from face-to-face to online instructions. although studies on education during covid-19 have shown effectiveness of online/blended learning environments for traditional lecture-based courses, 4,5 our students expressed their concerns in recent laboratory post-course evaluations. students and faculty felt unprepared after completing laboratory courses as they were only analyzing pre-collected data or provided with visualization of pre-recorded experiments. this may be attributed to the lack of hands-on experiences and peer-interactions in the traditional distance-learning environment. uncertainties remain on how academia needs to react to the pandemic in fall 2020 and beyond. therefore, we aim to address this significant challenge by discussing our ongoing development of a multi-modal experimental platform for the biomechanics laboratory course to enhance student learning in a stay-at-home or reduced-contact educational environment. we will employ and evaluate four educational models to enhance student learning during fall 2020. specific efforts will be focused on the development and implementation of (i) ''stay-at-home'' experiments, (ii) remotely-accessible experiments, (iii) multiscale visualization of biomechanical testing data and complex material's behaviors, and (iv) instructor feedback and peer assessment (fig. 1) . each of these four instructional modes will be discussed in the subsequent subsections. we will first develop low-cost, 3d-printed experimental setups for testing biomimetic samples, such as soft biocompatible polymers, as ''stay-at-home'' experiments. experimental setups of manually-driven testing stages will be 3d printed using commercial fused deposition modeling (fdm) 3d printers (fig. 2) . all the 3d-printed parts and experimental samples will then be mailed to students' home. students will have hands-on opportunities to assemble the 3d-printed parts and build the experimental platforms for mechanical tests. biocompatible polymers, such as polydimethylsiloxane (pdms) sheets and cubes, will be provided so that students can conduct mechanical testing experiments and visualize material's behaviors under different loading conditions (fig. 2a) . moreover, raw materials and instructions for the fabrication of gelatin cubes will also be mailed to students. by adjusting the material's concentration, gelatin cubes with various mechanical properties can be fabricated at home. students will utilize the mechanical testing stage for compression tests and use a ruler to measure the material's axial and transverse deformations under compressive loads (fig. 2b) . fundamental biomechanics concepts, such as the poisson's ratio that is defined as the minus of the ratio of transverse strain to axial strain, can be experimentally studied using the designed tensile and compression tests. it is expected that the proposed ''stay-at-home'' experiments can enhance the understanding of fundamental biomechanical testing procedures and mechanics concepts, and that assembly of the setup will expose students to engaging and genuine experiences in 3d printing and mechanical components. we will next devote significant efforts to developing novel remotely-accessible experiments for more complex biomechanics experiments, such as the stressstrain relationships. lecturers and teaching assistants will set up a camera and online meeting platforms, such as zoom, for students to visualize the experimental procedures at home. in addition, remote control functions of the experimental testing equipment (e.g., the cellscale's univert system 1 ) will be turned on, allowing remote operation of testing by students. we will develop two remotely-accessible laboratory assignments (table 1) . for example, a three-point bending test of chicken bone or artificial bone material can be set up using the univert mechanical testing stage 1 , and the real-time experiment will be broadcasted to each student group using a camera. in these experiments, students will remotely log in to the computer and set up the testing parameters (e.g., loading rate, displacement range and data acquisition rate). students can then launch and abort the experiments, while the instructors and teaching assistants, who are physically next to the testing equipment, will ensure safety and install new samples in between the two experiments. during the remote experimentations, a series of scaffolding strategies will be implemented. first, student groups will be engaged in remote operation, real-time observations of both the tested samples and the recorded biomechanical quantities through the labjoy graphical user interface (gui). 2 then, students will participate in post-experiment group discussions, followed by their video presentations. such instructional efforts are expected to enable remote access of major equipment for biomechanical testing, allowing students to conduct complex experiments that are available at home during the covid-19 pandemic. a certain level of training from these remotely-accessible laboratory assignments will help prepare future biomedical engineering workforce, such as to be familiar with the remote working environments and establish their effective teamwork and group communication skills. we will next deploy a multiscale visualization and experimentation module to enhance students' under-standing of material behavior under complex loading conditions across multiple length scales, ranging from micrometers (micro-level) to millimeters (macro-level). biomaterials, such as artificial skin or harvested bovine tendon tissue, will be tested using micromechanical tensile stages. at the micro-scale, the samples will be observed under an optical microscope, so that microstructural features, such as collagen fiber architecture, can be visualized during testing. at the macroscale, a digital image correlation (dic) system will be employed to obtain the two-dimensional strain fields, allowing students to visualize both the material's responses and microstructural changes simultaneously. all the videos, images, and mechanical testing data will be collected by the instructors and included in a multiscale visualization and experimentation educational module. directions, guiding questions, and examples will also be included in this educational module. by employing the developed educational module, students will be supported to virtually conduct multiscale biomechanics experiments in a simulated environment. the material's relations across the two length scales will be illustrated after the virtual experiments are completed. students will also learn how to calculate the local strain field by using the dic-tracked fiducial marker positions and compare their results with the strain data provided by the instructor, which serves as expert modeling. 6 this effort is expected to assist students to deepen their understanding of material's microscopic behaviors and their biomechanical responses. in spring 2020, we collected some data to understand the impact of ''stay-at-home'' bme laboratory courses on student learning through anonymous midsemester and end-of-semester course evaluations. the mid-semester evaluation indicated that the main negative impact of the virtual laboratory was the lack of peer interactions that stifled motivation to learn. in the past studies, social interactions have been shown to improve student ability to monitor their own understanding of materials and to motivate them to assume responsibility for their own learning. 7 therefore, in the second half of the semester, we added the laboratory activity of group discussions in zoom breakout rooms to increase peer interactions. the lab groups of two to three students worked in a breakout room to analyze the experimental data, while the lab instructors ''moved'' to different breakout rooms, similar to circulating from one group to another in a typical inperson lab setting, and provided each group with feedback and probed their understanding of the course contents. in addition, we also required students to 1 univert from cellscale biomaterials testing: https://cellscale.com/ products/univert/. 2 labjoy is the software of the univert mechanical testing system. conduct peer-assessment as one of the laboratory assignments. 8, 9 students were asked to write lab reports and then review each other's report, guided by a set of question prompts and rubrics for peer-assessment (see an example in supplementary material), which were designed to further enhance student's peerinteractions. 10 from the final course evaluation in spring 2020, the students noted increased peer interactions and motivation to learn as a result of using the zoom group discussions. the course evaluation from spring 2020 provided initial evidence that ''stay-athome'' bme laboratory courses, integrated with peer interactions, peer assessment, and instructor feedback in zoom breakout rooms, could be a feasible alternative solution for student lab experiences in times of crisis when in-person labs are unavailable. overall, the implementation of both breakout session peer interactions and peer evaluations will not only improve student's learning experience, interest and engagement, but it will also help to offset the unavoidable decrease in peer connectedness resulting from stay-at-home isolation during the covid-19 pandemic. these evaluation techniques will be continuously adopted, revised, and integrated with the developed educational modules of multi-modality biomechanics laboratory experiments. in this article, we have presented an innovative instructional model of multi-modality experiments to provide student virtual laboratory experiences remotely from home, which aims at minimizing the impact on student laboratory learning caused by the covid-19 pandemic-a pressing challenge faced by many bme undergraduate programs in the united states and around the world. this model offers a holistic and systematic scaffolding framework 11 that consists of four main integral processes: (i) developing 3d-printed experimental setups; (ii) conducting remotely-accessible experiments, and (iii) multiscale visualization of biomechanical testing data and complex material's behaviors, and (iv) instructor feedback and peer assessments. each of the four processes has its own micro module with scaffolding strategies embedded, such as guiding questions, prompts, instructor feedback and peer interactions. each subsequent process is intended to further foster students' laboratory experience and deepen their understanding of biomechanics theories. although limited and preliminary, the spring 2020 course evaluation indicates that the proposed instructional model and technology platform have the potential to be extended to other bme and general engineering experimental courses, providing a unique framework for the implementation of stay-at-home, online and remote experiential learning of laboratory experiments. we plan to further collect evidence to evaluate and validate the ''stay-at-home'' bme laboratory model in the following two semesters, focusing on the following two questions: #1 does the ''stay-athome'' bme laboratory have the similar or better effects than the traditional in-person laboratory? #2 what is student's learning experience with the ''stay-athome'' laboratory? over the next two semesters, we will collect data from three experimental conditions, and we will set up for each of the laboratory course: (a) ''stay-at-home'' laboratory with peer interactions and peer assessment, (b) ''stay-at-home'' laboratory withthree artificial skin dog-bone specimens (with an effective cross-sectional area = 1.0, 5.0, and 10.0 mm 2 ) will be tested by a group of 3 students to observe the material's response under the linear stressstrain regime students will analyze the data (.csv) in excel to learn the calculations of uniaxial strain and stress -linear stress-strain relationship (i.e., hooke's law) -importance of using the derived stress and strain relationship, rather than the directly-measured forcedisplacement curve, for bioengineering design chicken bones or artificial bone-like materials will be pre-fabricated and used in this experiment mid-span displacement will be gradually applied, and the force will be measured students will use the measured data to make the force-deflection curve -evaluation of the validness of the elementary beam theory student groups will hold post-experiment virtual meetings to discuss their findings and analyzed results, and then prepare a 5-min video presentation to be shared with the entire class on the canvas course webpage.. out peer interactions and peer assessment, and (c) traditional in-person laboratory. the comparisons will allow us to examine the effectiveness of the ''stay-athome'' laboratory and evaluate student learning outcomes in different laboratory learning environments. to answer question #1, we will collect the pre-test and post-test on the learning concepts and procedures indicated in table 1 . in addition, we will also collect data from the three laboratory environments: the lab reports (using the rubrics shown in supplementary materials) as graded by the instructors and student presentations (video presentations for the two ''stayat-home'' laboratory environments). furthermore, we will collect survey data from the two ''stay-at-home'' laboratory environments, with questions focusing on learner interest, motivation, engagement, self-efficacy, self-regulation, peer-interactions, peer assessment, and cognitive load in order to answer question #2. inferential statistical analysis will be conducted to compare the pre-test and post-test and learning experience surveys across the three learning environments. if time and resources permit, we will also collect and analyze interview data from the ''stay-at-home'' laboratory. further evidence is needed by collecting empirical data to validate the proposed laboratory model in the near future. through further research validation, it is hoped that this educational model, including its technology platform, will be adaptable for other future critical incidences like the covid-19 pandemic (i.e., adaptive education for emergencies). this innovative instructional model is also expected to mitigate the longer-term negative consequences on bme student's intellectual development and career preparation. furthermore, our educational modules of multi-modality experiments tightly couple theories to hands-on experiments with an emphasis on involving students in experiential learning. 12 such innovative developments will aid in making the important connection between theories and the real-world phenomena, which is a known issue in stem education such as the learning of advanced mechanics or machine design without laboratory experiments. 13, 14 the online version of this article (https://doi.org/10. 1007/s43683-020-00025-w) contains supplementary material, which is available to authorized users. the covid-19 pandemic: shocks to education and policy responses. the world bank education and covid-19 educational methods and best practices in bme laboratories roles for learning sciences and learning technologies in biomedical engineering education: a review of recent advances remote learning guidance from state education agencies during the covid-19 pandemic: a first look influence of covid-19 confinement in students performance in higher education a cognitive support system to scaffold students' problem-based learning in a web-based learning environment the impact of social interaction on student learning. read horiz peer assessment without assessment criteria. stud high educ the assessment cycle: a model for learning through peer assessment scaffolding students' problem-solving processes in an ill-structured task using question prompts and peer interactions understand learners' challenges and scaffold their ill-structured problem solving in a technology-supported self-regulated learning environment experiential learning: experience as the source of learning and development hands-on experiments in advanced mechanics courses enhancing the teaching of machine design by creating a basic hands-on environment with mechanical 'breadboards' publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations the authors declare no conflicts of interest or competing interests. all authors contributed to the idea curation. the first draft of the manuscript was written by c-hl, yl, mm, and all authors commented on the revisions of the manuscript. all authors read and approved the final manuscript. no funding source to be acknowledge. key: cord-121285-4ni1vv4l authors: zhang, han; nurius, paula; sefidgar, yasaman; morris, margaret; balasubramanian, sreenithi; brown, jennifer; dey, anind k.; kuehn, kevin; riskin, eve; xu, xuhai; mankoff, jen title: how does covid-19 impact students with disabilities/health concerns? date: 2020-05-11 journal: nan doi: nan sha: doc_id: 121285 cord_uid: 4ni1vv4l the impact of covid-19 on students has been enormous, with an increase in worries about fiscal and physical health, a rapid shift to online learning, and increased isolation. in addition to these changes, students with disabilities/health concerns may face accessibility problems with online learning or communication tools, and their stress may be compounded by additional risks such as financial stress or pre-existing conditions. to our knowledge, no one has looked specifically at the impact of covid-19 on students with disabilities/health concerns. in this paper, we present data from a survey of 147 students with and without disabilities collected in late march to early april of 2020 to assess the impact of covid-19 on these students' education and mental health. our findings show that students with disabilities/health concerns were more concerned about classes going online than their peers without disabilities. in addition, students with disabilities/health concerns also reported that they have experienced more covid-19 related adversities compared to their peers without disabilities/health concerns. we argue that students with disabilities/health concerns in higher education need confidence in the accessibility of the online learning tools that are becoming increasingly prevalent in higher education not only because of covid-19 but also more generally. in addition, educational technologies will be more accessible if they consider the learning context, and are designed to provide a supportive, calm, and connecting learning environment. the impact of covid-19 on students in general can potentially be large. worries about fiscal and physical health, rapid change, and increased isolation are all factors that can affect students and general. students with disabilities/health concerns may face greater risks than their non-disabled peers in a wide range of aspects. first, they have higher financial risk due to the need for accessible housing and personal assistance. moreover, they face greater health risks, depending on their specific situation. in addition, the risk to their education is also higher due to potentially inaccessible course content and classroom environments [21, 31] . these risks may increase further in the face of a global health pandemic. an understanding of the impact of covid-19 on students with disabilities/health concerns is important so that we can identify these increased risks early and respond to them. in this paper, we define disability broadly to include anyone who self-identifies as disabled and also those with health concerns who may not identify as disabled [15] . we focus on two specific issues of concern to all students: education and mental health. both are currently under stress, and understanding how and whether those stresses differentially impact students with disabilities/health concerns is critically important. by comparing data about students' experiences during covid-19 we hope to identify areas of concern. to this end, we present descriptive data showing how students with disabilities/health concerns, in comparison to their non-disabled peers, are faring in the covid-19 context. we present an analysis of data from a multi-year study at a large public university that has included approximately 200 students per year, including students with and without disabilities/health concerns [36] . in this paper, we focus specifically on the impact from survey results from spring 2020 (during the covid-19 pandemic), in which we ask 147 students about their physical and mental health, their academic and social experiences, and specific concerns about covid-19. our sample includes 28 students with disabilities/health concerns in 2020. where relevant, we also make comparisons to data collected from a subset of 91 students, of whom 13 have disabilities/health concerns who filled out a very similar survey in spring 2019 (before covid19) . we contribute insights about university students' early responses just before and during covid-19, to characterize patterns of concern, challenging experiences, and stress as well as initial indicators of mental health. our central focus is on students with disabilities/health concerns, and the impact of covid-19 on their education and health. educational impacts. the shift to online learning may impact the types of access technologies and techniques that students use, as well as their dependence on online course materials that may not be accessible [21, 31] . particularly, as many universities made a late decision to teach online, instructors had less time than usual to prepare materials for an online setting, which may have impacted the accessibility of those materials. at the same time, for some students the ability to participate online may increase accessibility. we contribute an analysis of educational concerns and how they differ from before to during covid-19. we compare students with and without disabilities/health concerns, and we analyze differences between different classes of disabilities/health concerns. our findings suggest that students with disabilities/health concerns have increased concerns about classes going online over those without, particularly the impact on their degree goals such as admission to major and graduation timing. impact on mental health. when social systems are placed under stress due to sudden forces such as a global health pandemic, the impact on marginalized populations can be especially severe. outbreaks of widespread infectious diseases, such as sars and covid-19, are associated with psychological distress and eroded mental health, stimulating calls for mental health care to be part of the national public health emergency system [4] . our examination of mental health impacts starts with exposure to stressors. we look at several potential sources of stress, including the use of social media, income loss, quarantine and isolation, loneliness, chronic discrimination, and overall exposure to adversity. at the time our data was collected, just after classes went online and three weeks after it was discovered that community spread of covid-19 was present in seattle (table 1) , we did not find evidence of changes in anxiety, stress, or depression among students with or without disabilities/health concerns. however, students with disabilities/health concerns reported more stress exposure, and more distress, than those without. implications for accessible online education. although our sample is small and the timing early in the covid-19 crisis, this data provides an important snapshot into the consequential nature of students' concerns about online education, which includes the impact not only on grade, but also on admission to major and graduation. the covid-19 epidemic's impact on learning should be a wake-up call to accessibility researchers to study online learning technologies and their impacts, and higher education in general, from a disability perspective. although social distancing may fade into memory, it is likely that online learning will not. even before covid-19, online education was beginning to achieve parity with in-person education [27] . yet this is a topic that has received little attention from computer scientists or accessibility researchers. a title word search for "education or course or teaching" within the assets proceedings found only 29 matches, of which most discuss introducing curricula that would educate technologists about accessibility (e.g., [20] ). while some deal with online educational tools in general (e.g., [8] [12] ), or childrens' access to the mainstream education system [9] , there is much less work published about higher education. although this is a limited search, it highlights the relative lack of attention to online learning, and higher education in general, in the accessibility community (in contrast, the keyword "blind" found 139 matches). our work provides motivation for the importance of improving the accessibility of online classes (and relatedly the ease with which instructors can make such classes accessible). in addition, as ringland et al. argue [33] , accessibility solutions must take a holistic, contextualized view of the person, including consideration of their potential stresses and concerns. thus, we argue that under covid-19 (and, really, in general) an accessible education is also one that contributes to student well-being and resilience. our data help to justify this need by demonstrating that, at least in our sample, students with disabilities are operating under increased stress loads both over their lifetime and specifically since the beginning of the covid-19 pandemic. life as an undergraduate student presents an exciting and challenging opportunity to learn, innovate, and grow. however, exposure to stress is also common among college students [18] . in addition, some stressors differentially impact vulnerable groups, and these impacts can translate into changes in mental health and behavior (e.g., [36] ), particularly when they interact with risk [35] . increasing participation by people with disabilities requires understanding and efforts to lessen the particular stresses and risks they face. the covid-19 pandemic is one obvious stressor that may directly impact students in a multitude of ways. some may be at greater risk of contracting the illness and interrupted access to health services [3] . pandemics can also impact availability of personal assistance, reduce educational accessibility [21] , and create higher risks of exposure than for the overall population [5] . below we explore expected consequences for learning and mental health of these stressors for people with disabilities. one significant impact of covid-19 has been the shift to online learning. online learning has become increasingly common over the last decade [31] , and even before the radical changes imposed by social distancing in the era of covid-19, issues existed with the accessibility of online courses. for example, at one university, 75% of faculty reported having never made accommodations in their online material [31] , most having never been asked by students to do so. student reluctance to ask for accommodation is driven in part by concerns about faculty bias [21] . due in part to the lack of perceived need among professors, the perspective of students with disabilities indicates that online learning is not fully accessible [21] . in addition, the move to increased asynchronous learning can leave students with disabilities behind, as, for example, videos are often not captioned. as of 2015, only blackboard (of all online learning management systems) had been awarded gold certification for its accessibility by the national federation for the blind [21] . in addition, content generators (i.e. faculty) have a big impact on the accessibility of their content [21, 22] and over half (65%) of faculty are unsure or do not know how to make accommodations [31] . this results in many courses being designed and deployed without accessibility support [21] . when faculty additionally have to make very rapid changes to their courses due to pressure to go online quickly, it is even less likely that they will make time to attend to an issue they are unsure of, like accessibility. at the same time, a shift to online learning due to a pandemic could benefit students with disabilities. for example, going online might reduce the need for disclosure, increase flexibility if instructors are being especially accommodating under the circumstances, or reduce barriers for students with mobility-related impairments. even before the pandemic, universities were seeing high and growing levels of mental health struggles among students [17] . further, natural disasters as well as past epidemics such as sars, have been associated with psychological distress, depression and substance abuse [13, 16, 23, 28, 41] . adverse psychological effects of the current pandemic may also be serious, but are not expected to be uniform [30] . for example, many individuals show resilience in disasters; this resilience manifested in a recent study of college students who, contrary to expectations, reported less loneliness in april than in february of this year [10] . daily surveys by kanter et al. of the general population from march 14th through may 5th 2020 (during the covid-19 epidemic) have not shown overall increases in anxiety, depression or loneliness (https://uwcovid19.shinyapps.io/dashboard/). on the other hand, students with a history of stress exposure may have raised allostatic load [25] , lowering the physiological resilience to the additional demands brought on by the pandemic. it is unclear whether students with disabilities/health concerns are likely to experience mental health consequences of covid-19's impact on society. what is expected is that people with disabilities/health concerns are likely to experience more stressors associated with the pandemic. in addition to the risks of covid-19, and educational accessibility, emergency response plans do not always include planning for people with disabilities [5] , social distancing may be harder for people with disabilities who rely on caregivers, and people with disabilities may have to deal with inaccessible communications [5] . these risks may put people with disabilities at greater risk of experiencing distress associated with the pandemic. their concerns about personal safety may be stronger than those of people without disabilities and they may be more vulnerable to stressors during the pandemic (such as disruption of critical health care). for these reasons they may be at greater risk of psychological distress. given the literature thus far, there is cause for concern. the covid-19 context appears to be exacerbating risks related to mental health, educational outcomes, and physical health, particularly for students with disabilities. thus, this paper addresses time-sensitive questions regarding the adversity profiles and well-being of university students during the current pandemic, with particular focus on ways that students with disability needs may differ from nondisabled students. drawing from research related to cumulative disadvantage and marginality [29] , we anticipate that students with disabilities will express justifiably greater vulnerability in the covid-19 context through elevated levels of concern about their educational environment and likelihood of success, as well as features of their family and personal lives affected by the pandemic. finally, we anticipate that students with disabilities may be experiencing greater levels of psychological distress in the form of mental health indicators. our data are derived from a multi-year study, the university of washington experience (uwexp) [36] , which is in its third year of collecting data about college students' behaviors, mental health, education and well-being. this, in turn, was inspired by a study first pioneered by wang et al. [42] . the goals of the uwexp study are to understand the stressors impacting undergraduate college students through a combination of self-reported information about demographics, health and well-being, institutional data about educational outcomes, and behavioral data collected using fitbits and mobile phones. in 2020, we recruited from all past participants in the uwexp study and additionally advertised to the entire university of washington entering first-year student class. in addition, we conducted targeted recruitment in communities that might be marginalized including students with disabilities, and first-generation college students. students are paid to participate in the study, which is irb approved. in this paper, we primarily make use of the uwexp survey data collected in spring of 2020, within the covid-19 context. students were instructed to complete the survey after their last winter quarter final. at that time, the university of washington had been teaching online for at least two weeks, and students knew that half or more of spring quarter would be online. the uwexp baseline survey data was conducted from march 18th through april 8th (end of winter quarter through beginning of spring quarter). the majority (80%) of students completed it between march 18th and march 29th. at the time, social distancing restrictions were increasing city and state-wide, with the initial stay-at-home order for washington issued march 23rd. the full timeline is shown in table 1 . students answered an hour-long questionnaire consisting of demographics and a series of well-established scales to measure depression (ces-d [32] ); perceived stress (pss [7] ); anxiety (stai [39] ); loneliness (ucla [34] ); chronic discrimination and harassment (cedh [26] ); perceived social status (ses, measured using the macarthur scale of subjective social status [1] ) and major life events (mle [36] ). several of these scales ask students to reflect over a period of time: the depression scale items measure the past week; whereas perceived stress and trauma symptomology assessed the past month. students reported major life events (adverse, traumatic or stressful events) over their lifetime. they reported their depression-related feelings that occurred during last week. we measured trait anxiety and loneliness does not provide a timescale, it is thus likely that answers to the loneliness scale reflect current (in-the-moment) status. in addition, since one arm of the uwexp study was concerned with adversity exposure and related symptoms, the data collection protocol includes a scale designed to measure post-traumatic stress disorder (ptsd) symptomology [43] . we note that in a population without known exposure to severe trauma (such as most of our participants), the items in this scale are better conceptualized as a measure of distress rather than indicative of clinical post-traumatic stress. date event 2/28 evidence of community spread discovered in seattle 3/3 first covid-19 related death discovered in seattle 3/6 announcement that classes would officially switch to online 3/13 last day of instruction for winter quarter and announcement that spring quarter will begin online 3/18 announcement that spring quarter will be fully online 3/18 earliest date a student took the baseline survey 3/20 last day of final exams for winter quarter 3/30 instruction for spring quarter begins 4/8 latest date a student took the baseline survey when no established measures were available, we created questions to address variables of interest such as adverse events due to covid-19. this included questions about students' levels of concern about current life issues-both educational and family related; questions about covid-19 specific adversity exposures; and a single item asking if they have a pre-existing condition that makes them vulnerable to covid-19. in addition, the uwexp study includes a student-specific measure of covid-19 related adversities. 152 students participated in the 2020 survey, of whom 147 stayed in the study (96.7% retention rate). we removed the data for the five students who dropped out from our data analysis. we assign disability status primarily based on self-identification. additionally, we added people who gave very high responses (moderate or severe) on at least eight of the 33 items of the cohen-hoberman inventory of physical symptoms (chips) scale [2] , which included energy impairment suggestive of a chronic health condition. our choice to include disabling health conditions is consistent with classification systems and service provision contexts [24, 38] . this also helps address under-identification of disabling statuses; for example, about 25% of people with energy-impairing chronic conditions do not identify as disabled [15] , but often still encounter similar barriers as people who do identify as disabled. analysis was guided by the research questions, largely focused on full sample portrayals of variable distributions as well as between-group tests of difference. we run both t-tests (parametric) and mann-whitney u tests (non-parametric) for significance testing; we had the same significance levels for all the pairwise comparisons, thus, we chose to report the results of t-tests for consistency. most results meet < .01 or better, and the number of repeat comparisons is small, so we did not correct for multiple comparison. the 2020 study included 147 students ( only a few reported a pre-existing condition that made them vulnerable to covid-19 (5 (4.2%)) and the average ses score is 5.9. there are 28 students with disabilities/health concerns. note that students could indicate multiple types of disability and that one student with a high chips score also identified as disabled, which is why there are 24 disabled students and 5 students with high chips scores. students with disabilities/health concerns had higher marginalized status or vulnerabilities than students without in the study: 82.1% (23) are female, and 17.9% (5) male, 67.9% (19) are asian, 39.3% (11) are first-generation college students, 21.4% (3) are lgbtqia+ and the average ses is 5.5. although the majority of our analysis only looks at the 2020 survey participants, we also look at times at data collected in 2019. of the 2020 students, 91 students returned from 2019 including 13 with disabilities/health concerns. these students filled out the same survey in 2019 and 2020, at the same time of year, with the exception of covid-19 specific questions, which were added in 2020. firstgen stands for first generation students (whose parents did not complete college). covid health vulnerability are students who self-identified as having a pre-existing condition that puts them at risk for coivd-19. students self-identified their disability. in this section, we first present students' levels of concern regarding the potential negative effects of the covid-19 context on their educational standing and success. we then summarize and contrast students' adversity exposures, both those directly covid-related as well as from sources such as discrimination, loneliness, and stress profiles via histories of major life adverse events. finally, we compare students with disabilities/health concerns to those without disabilities/health concerns regarding their current mental health statuses. as noted in table 1 , the university of washington courses went online approximately two weeks before the end of winter quarter (early march), and students filled out the survey after taking their last final. part of that assessment was students' concerns about the potential impact of covid-19 on their academic well-being. figure 1 shows the percentages of students with and without disabilities/health concerns across eight indicators. at the time, things were in flux, but students knew that spring quarter would be at least partly online. as is evident, concerns ranged high on many of these factors. more than 50% of students with disabilities/health concerns were very concerned about grades in both winter and spring quarters, as well as whether they would have to move degree requirements, negative impact on courses that could not then go online, admission to preferred major being threatened, and, to a lesser degree, negative impact on graduation and financial aid. students with disabilities/health concerns were substantially more concerned ( = 13.44, = 6.17) than their non-disabled peers ( = 8.25, = 6.28), with means calculated as a sum across all items. this difference is significant ( (145) = 3.94, < .001). given the wide range of disability identities represented in our sample, it is possible that not all students with disabilities/health concerns share the same levels of concern. for example, students with high fatigue may well context. the x axis shows the type of concern. the y axis shows percentage of students, within group. for example, 16 students with disabilities/health concerns were concerned about moving degree requirements (almost 60% of all students with disabilities) and 25 students without disabilities/health concerns (around 20%). prefer to stay home, whereas a student with hearing impairment may need to rapidly change how they access audio material through a transcription or sign language translation service. we anticipated that students with vision or hearing impairments might face greater barriers going online than those with mental health conditions or other disabilities/health concerns. however, of the 27 students who answered this question, we found that there was no toward assessing student vulnerabilities within the early covid-19 context, we turn next to identifying areas of adversity exposure relevant to psychosocial functioning as well as mental health statuses as reflected through clinical measures. we begin by investigating covid-19 related stressors that students were experiencing outside of their educational settings and differences for those with disabilities/health concerns. as can be seen in figure 2 , students with disabilities/health concerns experienced more stress than those without due to a range of financial, family and isolation issues. experiencing tensions within the household during this period of isolation reflected the greatest disparity between students with disabilities/health concerns and their non-disabled peers. we also asked about whether social media use was a source of stress. this was high for all participants (over half agreed or strongly agreed); however the difference between students with and without disabilities/health concerns was not significant. table 3 reports group comparisons on four forms of adversity that could have implications for stress and functioning. in this data, the cedh instrument measures chronic discrimination and harassment over the past twelve months, major life events is capturing stressors over the lifetime, and covid related adversities provides a sum of the 0% 20% 40% 60% 80% personal income loss (10, 24) family income loss (9, 23) self quarantine (10, 23) family in quarantine (7, 12) isolated (17, 71) tension (17, 35) students with d/h students without d/h fig. 2 . comparison of students experiencing stressful circumstances directly related to covid-19 in early spring 2020. the 6 circumstances assessed were major loss in own income, major loss in family income, oneself in quarantine, family member placed in quarantine, quarantine or other covid-19 effect leading you to feel isolated, conflict/tension within household members due to covid-19. the x axis lists relevant concerns, along with the number of students in each category. for example, in the case of income loss, 10 students with disabilities/health concerns (or 36% of those students) were concerned about income loss, while 24 students without disabilities/health concerns (20% of those students) were worried. stressors attributed specifically to covid-19 (i.e. recent stressors) shown in figure 2 . here we see that students with disabilities/health concerns report higher levels of chronic discrimination (e.g., experiencing demeaning remarks or forms of unfair treatment) and higher exposure to major life adversities (e.g., a serious interpersonal conflict, early life or recent maltreatment). students with disabilities/health concerns also report more covid-related concerns. notably, both student groups reported comparable level concerns about isolation, and comparable loneliness. given the greater levels of adversity exposures and variety that students with disabilities/health concerns face, we anticipated that students with disabilities/health concerns would experience increased mental health problems. however, both groups of students experienced comparable levels of current perceived stress, depression, and anxiety at the time they filled out the survey, as shown in table 4 . it is notable that, in comparison to the other mental health measures, students with disabilities/health concerns did have significantly higher scores on the ptsd (distress) scale [43] . we delve more deeply into this result in the next subsection. [7] 19.23 (6.9) 18.88 (5.6) 0.75 cesd (depression) [32] 9.00 (5.48) 8.25 (4.75) 0.67 stai (anxiety) [39] 45.25 (10.48) 44.14 (9.99) 0.51 ptsd (distress) [43] 22.86 (15.99) 13.18 (11.65) 3.02** table 4 . mean (standard deviation) of scores on mental health scales for students with disabilities/health concerns (d/h) and without disabilities/health concerns (no d/h) . t-test values and significance levels are indicated. mann-whitney u tests yielded comparable results. significance is marked as * * < 0.05, ** < 0.01, *** < 0.001. to further explore these results, we undertook a more nuanced examination of student responses. figure 3 shows the items which had the highest level of endorsement from students with disabilities/health concerns. these items are consistent with domains of ptsd reflective of general psychological distress (specifically, domains d and e [11] ). that is, the items that were most frequently rated as highly concerning by students with disabilities/health concerns appear to be reflecting symptoms of hyperarousal and agitation-such as difficulty concentrating, trouble sleeping, and having negative feelings. students were also comparatively cut off from others in this time of isolation, and indicated that as troubling. one possible explanation for the ptsd group difference described above is that our sampling process in 2020 purposefully included more people with disabilities/health concerns and other vulnerabilities. to address this, we looked at the 2019 and 2020 data just for students who had participated in both the 2019 and 2020 studies (a total of 91 students, of whom 13 have disabilities/health concerns). we first examined whether our 2020 results changed when we only looked at the 53 students who participated in note that while the significance is higher here, the means for students without disabilities/health concerns are lower than for students with disabilities/health concerns in both 2019 and 2020. this study provides an early snapshot of the concerns and vulnerabilities of undergraduate students with disabilities/health concerns within the early-stage context of the covid-19 pandemic. such students expressed considerable concern about the educational and financial impacts of covid-19. at this early stage of covid-19, we did not see higher levels of self-reported depression, anxiety, stress or loneliness. however, students with disabilities/health concerns did report overall high levels of stress exposure and exposure to covid-19 related adversities, and we saw evidence of associated distress. by comparing students with and without disabilities/health concerns, we gain insights about the unique vulnerabilities of students with disabilities/health concerns in the covid-19 context. they brought significantly deeper histories of both discrimination as well as serious major life adversity exposures than non-disabled students to this context. in addition, our results show that students with disabilities/health concerns are already experiencing covid-19 related stress. students reported higher or equal exposure to all of the covid-19 specific stressors we measured than their peers without disabilities/health concerns. we also found early signs of distress, particularly difficulty concentrating, insomnia, and isolation, all understandable reactions to conditions of unpredictability and lack of control [44] . the timing of the surveys (early in the covid-19 crisis) leaves open the possibility that the effects we see are due to different stress exposures between students with and without disabilities/health concerns and not specifically to covid-19. however, our comparison of the students who provided survey data in both 2019 and 2020 suggests that exposure to discrimination and distress both increased this year and that our data reflect increased vulnerability of students with disabilities/health concerns in the covid-19 context. prior evidence has established that students entering higher education with adversity backgrounds are at elevated risk of worsening mental health [19] . subsequent stress exposure adds to and, in some cases, may exacerbate the effects of earlier adversities [14] . on the other hand, measures of stress, anxiety, depression, and loneliness all show no change (see tables 3 and 4 ). this could also indicate resilience in the face of these stresses, many of which are areas where people with disabilities may have prior experience [6] . students with disabilities/health concerns in our study report worries about courses going online, and serious consequences of the move online including receiving worse grades, not being able to meet academic requirements online, and having admission to their chosen major impaired. in addition, our study shows that students are not only concerned about online courses, but also exposed to a variety of stressors from discrimination to financial concerns. this suggests that more accessible online education tools should be designed to consider a wider range of concerns, which can potentially exacerbate the impact of barriers to access. prior work has argued for the importance of considering context in the design of accessible technologies [33] . rather than simply making online course materials more accessible, online instructors may need to support students who are coping with multiple stressors outside of class. for example, this might include designing for asynchronous participation, or redundant assessment that is robust to occasional absences. further, in the context of a pandemic, best practices are to convey safety, calm, comfort and connectedness [40] . if educators and educational technology can strive to provide a calming, connecting experience, they can support students rather than compounding their stress. our choice to combine people who identify as disabled with people who report themselves as having significant health concerns is driven by the relatively high numbers of people with health concerns who do not identify as disabled [15] , even though they may experience accessibility barriers. moreover, this group comprised fewer than 20% of the students with disabilities/health concerns and in an analysis comparing subgroups of students with disabilities/health concerns, we did not see major differences in our results. there are philosophical questions raised by this choice about who "counts" as disabled. our view was that, relative to our study's goals, this level of health concern rendered these students comparably vulnerable in the covid-19 context. that said, we recognize that other sources may use differing definitions. finally, we note that we have a relatively small and non-representative sample (for either seattle or nationally with respect to undergraduates). this sample reflects our efforts to reach more vulnerable students and thus our results may be most relevant to students with such vulnerabilities. this article is a first look at the impact of covid-19 on students with disabilities, and as such, it illuminates a range of needs not yet well documented. it also helps generate questions for further inquiry. understanding how covid-19 affects students with disabilities/health concerns can help us to identify patterns of concerns, stress and indicators of mental health, and provide further steps to take or consider in order to respond to an unexpected global health pandemic. our findings show that students with disabilities/health concerns were more worried about the outcomes of the unanticipated change to online learning than their peers. in addition, during our study, students with disabilities/health concerns experienced more covid-19 related adversities and distress. as pandemics (as well as other major stress conditions) continue over time, additional stressors begin to arise (e.g., loss of income, lack of healthcare access, caring for others [30, 37] ). this could in turn translate to larger impacts on the wellbeing of students with disabilities/health concerns. in the future, we hope to compare students' response to covid-19 further into the epidemic to the data described in this paper. further, we plan to conduct interviews to develop a more nuanced, qualitative picture of these impacts. to conclude, we argue that just as students with disabilities belong in higher education, accessibility research belongs in higher education. it is time for the accessibility community to expand our work studying technology use and developing technology to support students with disabilities in higher education. we hope that these insights into the student experience demonstrate how important it is to learn how higher education fails and succeeds to support students with disabilities, and where technology can play a role in improving this. the macarthur scale of subjective social status the cohen-hoberman inventory of physical symptoms: factor structure, and preliminary tests of reliability and validity in the general population the covid-19 response must be disability inclusive 2019-ncov epidemic: address mental health care to empower society preparing for and responding to pandemic influenza: implications for people with disabilities covid-19's isolated world is the norm for people with disabilities a global measure of perceived stress blackboardnv: a system for enabling non-visual access to the blackboard course management system designability: framework for the design of accessible interactive tools to support teaching to children with disabilities have introverts or extraverts declined in social connection during the first wave of covid-19? trauma-informed care in behavioral health services dual educational electronic textbooks: the starlight platform the early psychological impacts of the deepwater horizon oil spill on florida and alabama communities prevalence, severity and risk factors for depressive symptoms and insomnia in college undergraduates energy impairment and disability inclusion: towards an advocacy movement for energy limiting chronic illness sars control and psychological effects of quarantine mental health problems and help-seeking behavior among college students college of the overwhelmed: the campus mental health crisis and what to do about it adverse childhood experiences (aces), stress and mental health in college students evaluating instructor strategy and student learning through digital accessibility course enhancements disability and elearning: opportunities and barriers accessible online content creation by end users stress and psychological distress among sars survivors 1 year after the outbreak tevfik bedirhan ustun, nenad kostanjsek, and somnath chatterji stress, adaptation, and disease: allostasis and allostatic load you would not believe what i have to go through to prove my intellectual value!" stereotype management among academically successful black mathematics and engineering students online, cheap-and elite post-traumatic stress disorder following disasters: a systematic review cumulative disadvantage and youth well-being: a multi-domain examination with life course implications mental health and the covid-19 pandemic online disability accommodations: faculty experiences at one public university a self-report depression scale for research in the general population understanding mental ill-health as psychosocial disability: implications for assistive technology ucla loneliness scale (version 3): reliability, validity, and factor structure preparation. from microaggressions to major problems. cscw (in preparation) passively-sensed behavioral correlates of discrimination events in college students mental health consequences of infectious disease outbreaks disability across the developmental life span: for the rehabilitation counselor manual for the state-trait anxiety inventory covid-19: the impact of a pandemic on mental health sustained increased consumption of cigarettes, alcohol, and marijuana among manhattan residents after studentlife: assessing mental health, academic performance and behavioral trends of college students using smartphones the ptsd checklist for dsm-5 (pcl-5). scale available from the national center for ptsd at www. ptsd. va the impact of stress on body function: a review key: cord-035244-5rj76rvg authors: rooney, sarah ilkhanipour; scott, rebecca a. title: promoting effective student teamwork through deliberate instruction, documentation, accountability, and assessment date: 2020-11-10 journal: biomed eng education doi: 10.1007/s43683-020-00038-5 sha: doc_id: 35244 cord_uid: 5rj76rvg nan team. the course is normally hands-on and studiobased; however, due to covid-19, we transitioned to distance learning right after the students had completed concept selection and were about to begin prototyping. just before the university announced its decision to move to online learning, we administered an anonymous survey to gauge students' expectations for online learning and their prior zoom experience. teamwork was identified as one of the students' primary concerns: 15/45 survey respondents listed collaboration and teamwork in response to ''what concerns do you have about remote learning?'' when our course transitioned mid-semester to a remotelearning format, we faced the challenge of how to foster teamwork in an online setting, particularly with learners who are novices in teamwork. our philosophy is that deliberately training students in teamwork and promoting their success through specific practices can and should transcend instructional format. in this paper, we describe and reflect upon the practices implemented to support student team learning during face-to-face and remote instruction; most of the described approaches can be implemented in either instructional format. because this project did not originate with a research intent and instead is a description of practice, it does not meet the definition of human subjects research that would require an irb determination. in response to remote learning, we adapted the detailed design and testing phases, which are summarized here to provide context for the continued role of teamwork. we transitioned to virtual design with jus-tifications and simulations in lieu of a physical prototype with testing. to test risky design assumptions, teams created engineering drawings using solidworks or onshape, completed motion studies in solidworks, modeled the physical system in algodoo, simulated circuits using tinkercad, and/or justified materials with granta ces edupack. the tas supported this transition by pivoting to become experts on one or two of these software packages. since most of our students had no prior experience with these programs, our tas created a document in the first week of remote classes that summarized each program's capabilities (supplement a). to help students navigate the transition, we provided information about how to set up vpn, use zoom, and access software licenses, as well as changes to the course structure (revised schedule, weekly advisor meetings, ta office hours, and updated assignment expectations and rubrics). we created a video that reinforced: (1) our course goals and that these goals could still be accomplished in a remote format, (2) the unique opportunity to acquire new skills in remote communication and project management, (3) new weekly advisor meetings and ta office hours (described below), and (4) specific instructions for teams in the first week back. in their first week of remote learning, we expected students to meet as a team, review the updated phase documents and rubrics, and develop a team plan, which they were to document in their team norms update (described below) due at the end of that week. in the remainder of the semester, we sent a weekly announcement that summarized what the teams' focus should be during that week. throughout the course, we promoted successful teamwork through (1) teaching about teamwork, (2) documenting teamwork, (3) promoting accountability, and (4) assessing teamwork (table 1) . successful teamwork post-transition to remote learning built upon the foundation laid at the beginning of the semester. prior to the start of the semester, students were assigned to teams based on the catme team-maker tool (www.catme.org). 6 team formation criteria were prioritized to distribute across teams experience with cad, hand-tools, and arduino, as well as writing expertise, and to optimize opportunities for teammates to meet outside of class hours by aligning students' schedules. teaming began on the first day of class, when the 16 student teams participated in a lesson on teamwork that covered the following topics: a warmup activity that emphasized the value of diverse perspectives (supplement b), definitions of ''team'' and ''group'', 5 discussions of gender-based differences in task division on teams, 8 stages of teams (forming, storming, norming, performing, and adjourning), 5 and types of intra-group conflict (task conflict, relationship conflict, and process conflict). 4 stages of teams were addressed so students knew what to expect throughout the semester and to acknowledge that conflict is normal. in the second week of class, our lessons focused on project management methods (timelines/gantt charts, meeting minutes, and document management), role-playing using the ''bet'' and ''bear'' techniques to provide effective feedback, 3 and instructions on how to complete catme peer evaluations 9 and why they are important. these explicit lessons on teamwork provided a common language and specific tools for the students to utilize throughout the semester. building upon the lessons and tools discussed in class, students were required to document their teamwork. at the start of the semester, each team submitted a team norms document using a provided template (supplement c) that outlined individuals' strengths, weaknesses, and learning goals, as well as plans for team meetings, communication, decision making, equitable division of labor, and conflict resolution. at the start of each subsequent phase of the design process (i.e., three times), students refined their team norms document and submitted progress reports (supplement d). when we first transitioned to distance learning, teams were instructed to specifically address in their progress reports how they would maintain communication and work in an online, instead of faceto-face, format. at the end of the course, students individually submitted a final reflection paper that described how they ''function[ed] effectively on a team whose members together provide leadership, create a collaborative and inclusive environment, establish goals, plan tasks, and meet objectives'', 1 as well as what they learned about themselves and teamwork that they plan to apply in the future. in addition to the team norms and progress report documents, students were expected to maintain detailed meeting notes and document division of responsibilities in their project timelines. specifically, teams established project timelines at the start of each design phase to identify tasks, deadlines, and the lead team member(s). meeting notes were used to document and justify key project decisions, as well as detail teammates' progress on assigned tasks. teams organized these teamwork documents, along with all other project documents, in a shared google drive that was accessible to all teammates, instructors, and tas. overall, the multiple teamwork documentation methods were used to track progress, maintain accountability, and promote self-reflection and metacognition. accountability was critical after the switch to remote learning because instructors, tas, and teammates no longer could ''drop in'' on each other during class work times. in a typical face-to-face format, students primarily work in the design studio during class (twice a week for~2 h each session). during this time, the instructors circulate among the student teams to get impromptu progress updates and answer questions. when all teams have been seen and there are no more questions, the instructors typically return to their offices (in the same building as the studio) so students know where to find them when they have additional needs. in the remote instructional format, all lecture content was asynchronous and informal studio meetings could not occur. further, once we transitioned to a remote format, teams did not always choose to work during the official class times. to address these issues and still maintain instructor-team communication, we scheduled weekly zoom meetings. each team was required to schedule a weekly zoom meeting (20-min time slots) with their assigned faculty instructor using google calendar appointment sign-ups. meetings were student-driven and more formal than the in-person drop-in meetings; teams prepared agendas, explicitly demonstrated progress since the prior week's meeting, and created summary notes, receiving practice in running an efficient meeting. teams also submitted weekly documentation, via email, to instructors summarizing three items: their accomplishments for the previous week, their plans for the upcoming week, and what help (if any) they needed from the instructor. these ''weekly updates'' from teams were established during the face-to-face portion of the course and continued after the switch to remote learning as a means of monitoring team progress. tas also held virtual office hours each week (2 h each) via zoom. to further help the instructors monitor team progress and address common concerns, tas filled out weekly reports via shared google documents (supplement e). students were evaluated on teamwork with a rubric in each phase of the design process (table 2) . this rubric was designed to align with abet student outcome 5 1 for subsequent program assessment. the team norms document and progress reports, online file management system, interactions with their faculty instructor, and ta feedback were all used as evidence to inform the teamwork assessment. furthermore, to promote both individual and collective responsibility, a hallmark of successful teams, 5 students completed catme peer evaluations. 9 the numerical adjustment factor generated by catme was used as a multiplier to determine each student's individual grade. students were required to submit detailed, unique comments for each team member (including themself), which were used as justification for quantitative scores. poor or general comments (e.g., ''we worked well together'') resulted in a grade penalty. faculty instructors had the authority to override the quantitative score if deemed necessary. for example, if the catme adjustment factor was 0.94 for a student, that student's individual grade would be 0.94 of the team grade, provided the students were required to provide detailed, unique comments about their teammates and themselves via catme peer evaluations-poor or general comments that did not provide adequate feedback were penalized peer evaluations students completed catme peer evaluations 4 times catme peer evaluations were used to determine individual grades comments submitted by their teammates were justified (e.g., student was regularly late and unprepared for team meetings and completed their assigned tasks with mediocre quality); however, if the teammates did not provide adequate justification for this score, the faculty instructors modified the score to better reflect the individual's contributions. as bme junior design serves as the first truly teambased experience for most of our undergraduate students, it is critical that we provide formal teamwork instruction and tools to promote student success. participants of the 2019 fourth bme education summit noted that teamwork is often not explicitly taught, despite being required in many courses. 10 following the recommendations posed in the bme education summit, we sought to instruct students in teamwork and provide them practice opportunities prior to capstone design. 10 the required team norms documents and progress reports resulted in important dialogue between teammates that was often overlooked in previous offerings of the course. the initial team norms document (pre-covid) revealed that when not meeting face-to-face, 100% of teams planned to use group messaging/texting to communicate. additional communication forms were proposed less frequently (video calls, e.g., face-time-6/16 teams; email-3/16; google documents-3/16; phone calls-2/16; talk between classes-1/16). in response to ''how will you share documents/data and keep track of revision history so you don't accidentally overwrite something?'', 100% of teams indicated in their initial team norms document that they planned to use google drive and google documents. in reviewing the teams' three progress reports (1-pre-transition, 2-at the transition, 3-remote), we found that students largely maintained in the remote format the same communication structure they had established in the face-to-face format. specifically, many teams used a group chat via groupme, imessage, or texts for quick questions and to schedule meetings, and they used shared google documents to comment on specific report sections and to assign and resolve tasks. teams reserved important decisions, delegating tasks, and sharing ideas for group meetings. these meetings switched from face-to-face to zoom when the class moved remote. based on the anonymous survey administered just before the transition to remote learning, students had little prior experience with zoom (24/45 respondents never used zoom, 15/45 used zoom as a participant once or twice, 5/45 used zoom as a participant several times but had never set up a meeting, only 1/45 used zoom to set up a meeting, and 0/45 regularly used zoom). students found the shared screen feature of zoom helpful when working on tasks such as cad, displaying their virtual prototypes, and using the virtual whiteboard. asking students to define their personal learning goals and document their progress toward those goals in the team norms documents aligns with the recommendations of linder et al. 8 [0] teamwork function effectively on a team whose members together provide leadership, create a collaborative and inclusive environment, establish goals, plan tasks, and meet objectives with appropriate documentation (e.g., agendas, meeting notes, organized online file management system). [8] [7] poor team dynamics with obvious unequal division of labor. operated as individuals rather than as a team. unable to arrive at team decisions or resolve conflict effectively. team agendas and meeting notes lack key details. online file management system is disorganized with unclear file names and is difficult for an outsider to navigate. [6] [0] total: / 20 directed practice. 2 when students documented their learning goals, it allowed them to gain the actionable support of their teammates and their instructors. our students do not take a cad course prior to junior design; therefore, most students are novices. because we created teams using catme criteria to distribute skills (including cad), we successfully split up this strength. as revealed through team norms progress reports and catme peer evaluation feedback, some teams distributed tasks based on existing skills and strengths, but many others aligned responsibilities, such as cad, with individual learning goals. additionally, some teams had problems remotely accessing our university's license to solidworks, so they switched to onshape. because onshape is an online platform, it is easier for students to share work, which they found more conducive to a remote teamwork environment. regardless of the program used, zoom's screen-sharing capabilities allowed more experienced teammates to peer-instruct and allowed students to easily share their designs. importantly, revisiting established team norms and learning goals at the beginning of each subsequent design phase provided teammates with a platform to discuss and update their team expectations, which was critical following the transition to a remote-learning format. just like engineering design, teamwork is an iterative process. prior published work [e.g., 2] demonstrates that engineering student teams move through different stages of teamwork. furthermore, each phase of the engineering design process requires different skills. compounding both factors here is the transition to remote learning. the dynamic nature of teamwork necessitates reflection for continuous improvement, and we observed that teams did adapt based on self-reflection. for example, one team began with 3-5 h meetings, but shifted to limiting their meetings to 2-2.5 h after they identified their limit in productivity. in the remote-learning format, the students no longer experienced the same accountability or structured time as with in-person instruction, and teams needed to agree upon new communication, meeting, and task distribution plans, especially since they no longer had 4 hours of required in-person meetings each week. although completing the team norms progress reports formalized these discussions for students, anonymous end-of-semester feedback obtained through university-administered course evaluations indicated that several students felt some assignments seemed like busywork, with one student specifically commenting that the team norms progress reports often felt repetitive. in the future, we recommend evaluating ways to shorten the progress reports. based on the team norms progress reports, the absolute number of times teams met did not change very much as the semester progressed, which is surprising because some teams noted in their first progress report that they wanted to meet more frequently but for less time. given that we no longer had scheduled/ required class time during the remote learning portion of the course, minimal change in the number of meetings suggests that students actually met less frequently as a team when we moved online, although teams did note that they met more frequently as deadlines approached. this less frequent meeting pattern could be a cause or an effect of more efficient delegation. we observed that teams seemed to engage in greater division of labor in a remote setting compared to in-person, which may be attributed to the fact that they arranged for fewer group meetings or because some file formats only allow one active person compared to in-person prototyping, which can accommodate multiple hands. some teams chose to work online simultaneously, but independently, to maintain accountability and keep on task. anonymous end-of-semester student feedback indicated that the weekly zoom meetings with instructors were highly valuable and several students suggested keeping these meetings (whether face-to-face or via zoom) in the future to maintain accountability. in contrast, ta office hours were not as heavily utilized as we expected. on average, 3.0 â± 2.4 teams (out of 16) attended office hours of at least one ta during any given week. ta office hours were held during formal class times; following the transition online, teams did not always choose to work during those times, which may have contributed to their underutilization. teams also indicated that they often used additional outside resources (e.g., youtube) to selfteach, potentially negating their need to attend ta office hours. the transition to remote learning did not increase conflict within teams compared to previous years. two student teams experienced significant internal conflict due to differences in communication preferences and work habits, which required instructor intervention and began prior to the transition to online. following the transition to remote learning, two teams struggled to maintain motivation and equitably divide work; however, this conflict resolved without instructor intervention. review of team norms progress reports revealed that relationship conflict 4 slightly increased toward the end of the semester, which is common based on the instructors' prior experience and did not seem specific to the remote format. process conflict 4 generally decreased as the semester progressed; one group noted it was easier to schedule meetings after transitioning to remote learning due to greater schedule flexibility, and one team identified a mismatch in preferred working hours (morning or evening) fol-lowing the transition to online. a couple teams noted in their team norms progress reports that they needed to be more deliberate in planning meetings in the remote learning format since they no longer could schedule during their time between common classes. analysis of catme peer evaluations revealed that individual teamwork outcomes of ''contributing to the team's work,'' ''interacting with teammates,'' and ''keeping the team on track'' were not altered when comparing design phase 2, which was completed on our final day of face-to-face instruction, to design phase 3, which was completed one month after our course transitioned to a remote-learning format (table 3). to draw comparisons, the average score in each criterion for each student was calculated from the ratings of their teammates, as well as the students' rating of themselves, and a wilcoxon matched-pairs signed rank test was utilized to evaluate changes in individual teamwork outcomes following the mid-semester transition to remote learning. this finding supports that the transition to online did not cause a decline in individual teamwork performance. individual teamwork reflections submitted at the end of the semester revealed that most students demonstrated significant growth, with students commonly citing the importance of communication, being understanding of each other's circumstances, and learning resilience as key drivers of team success in the transition online. however, despite their success, celebrating accomplishments as a team was often challenging, and future efforts should consider ways to boost morale and celebrate achievements in a remote setting. to provide further support for students, future iterations of the course will consider increased instruction on and practice with communication methods and discussion of the five dysfunctions of a team. 7 furthermore, we believe it would be helpful to formally reinforce mid-semester the teamwork lessons that were introduced at the start of the semester; by this time, teams have progressed out of the ''forming'' phase of teamwork and are making their way to the ''performing'' stage, which typically results in passage through the uncomfortable ''storming'' phase. 5 additionally, individual students and teams may have different needs as they work through the various teamwork stages and design process phases, so reinforcement and accumulated, deliberate practice (such as role-playing) would be beneficial. 2 anonymous endof-semester feedback also indicated that several students desired clearer guidelines in their teamwork evaluations; specifically, one student felt that they should not receive a teamwork score and an individual peer evaluation score. several students also commented about the desire for clearer expectations on phase deliverables. we currently provide templates and rubrics in advance, have weekly lectures to summarize these expectations, and meet with teams to answer their specific questions. we will continue to identify ways to improve the communication of our expectations to students; however, part of this feedback may be because this course is typically the students' first experience with a fully open-ended design project, which is very different from a ''textbook problem.'' learning to manage ambiguity is a challenging skill, and something we will continue to seek ways to address. in conclusion, student teams effectively transitioned from face-to-face to a remote-learning format. the activities and templates described in this paper can be implemented in both learning structures. the ample evidence we have collected suggests instructors and students can implement just small changes to accommodate a remote format and still facilitate successful teamwork. further, these techniques are readily translated to other courses that seek to support team function. overall, the teamwork lessons, documentation, accountability, and assessments implemented in this course facilitated positive team dynamics and successful final design outcomes. for each catme criterion, the average score for each student was calculated from the ratings of their teammates, as well as the students' rating of themselves. a wilcoxon matched-pairs signed rank test was conducted to evaluate changes in individual teamwork outcomes following the mid-semester transition to remote learning. specifically, the results obtained from design phase 2, which was completed on our final day of face-to-face instruction, were compared to those of design phase 3, which was completed one month after our course transitioned to a remote-learning format. all criteria have a maximum score of 5.0. how learning works: seven research-based principles for smart teaching teaching the art and craft of giving and receiving feedback intragroup conflict in organizations: a contingency perspective on the conflict-outcome relationship observing engineering student teams from the organization behavior perspective using linguistic analysis of student reflections and focus group interviews design and validation of a web-based system for assigning members to teams using instructor-specified criteria the five dysfunctions of a team: a leadership fable work in progress-taking one for the team: goal orientation and gender-correlated task division the comprehensive assessment of team member effectiveness: development of a behaviorally anchored rating scale for self-and peer evaluation core competencies for undergraduates in bioengineering and biomedical engineering: findings, consequences, and recommendations publisher's note springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations the authors thank dr. julia maresca for inspiring the creation of the team norms template and dr. valerie myers for permission to use the team warm-up activity. both authors contributed to the course instruction. the first draft of the manuscript was written by dr. s.i.r. and both authors edited previous versions of the manuscript. both authors read and approved the final manuscript. not applicable. not applicable (no data are presented, and instructional tools/templates are provided in the supplemental materials). not applicable. dr. sarah i. rooney is a member of the mcgraw-hill education (mhe) biomedical engineering faculty advisory board, and she reports personal fees, outside the submitted work. mhe was not involved in the course design, creation or implementation of instructional materials, or manuscript preparation. this project did not originate with a research intent and instead is a description of practice; therefore, this project does not meet the definition of human subjects research that would require an irb determination. not applicable. key: cord-029031-jtdc9a7w authors: jirapanthong, waraporn title: a tool for supporting the evaluation of active learning activities date: 2020-06-22 journal: advances in swarm intelligence doi: 10.1007/978-3-030-53956-6_43 sha: doc_id: 29031 cord_uid: jtdc9a7w active learning becomes a strategical approach for an educational principle. the student engagement become a wider concern. many researches have been proposed to support the approach. however, one of issues is how to effectively evaluate the performance and progress of students’ learning. although, having student engagement in a classroom is vital, the evaluation of students’ performance is more important. however, keeping up the details or records of students’ progress is a difficult task. we therefore propose a support for instructors to evaluate the performance of their students. in particular, a prototype tool is designed and developed in order to facilitate the evaluation of activities based on an active learning class. the tool also encompasses the web service for a function of face feature recognition. two scenarios of active learning classrooms are created in order to evaluate the prototype tool. we also plan to create a larger number of scenarios which involve different class objectives. the results show that the tool can detect and determine students with high precision values. however, the prototype tool takes a long time to be processed depending on the size and number of photos. the concept of student engagement is becoming a concern. active-learning techniques have emerged as strategies for instructors to promote engagement during learning classes. the engagement of students at high level increases learning and retention of the students. the engagement with students may occur outside the classroom. for in the classroom, student engagement involves the participation of students such as interaction between instructors and students or between students themselves. the advantages of active learning are numerous: i) classroom time can be utilized in discussing higher order thinking skills rather than wasting time on lower order activities; ii) collaboration, communication and leadership skills are enhanced amongst learners; iii) students are actively engaged in a realistic learning experience. even though the active learning is becoming as an important approach for educational learning, it has still some challenges which include resistance from students, unprepared students, lack of access to technology, and heavy workload prior to and during class. moreover, the evaluation method for the active learning approach is also concerned. our research relies on the method and supporting tool for enhancing the active learning approach to be successful. in this paper, we present a tool for supporting instructors to evaluate the learning progress. this is to facilitate the tasks of evaluation on learners. the instructors therefore can identify participating learners and analyse the learners' performances more effectively. in this paper, the background of active learning and facial recognition analysis is provided in sect. 2. our approach is presenting in sect. 3. the experimental results, discussion and future work are following. active learning is generally defined as any instructional method that engages students in the learning process. in short, active learning requires students to do meaningful learning activities and think about what they are doing [1] . while this definition could include traditional activities such as homework, in practice active learning refers to activities that are introduced into the classroom. the core elements of active learning are student activity and engagement in the learning process. active learning is often contrasted to the traditional lecture where students passively receive information from the instructor. active learning methods that can be incorporated in the classroom: i) having multiple pauses during lectures to allow students time to reflect on what is being learnt and to consolidate their notes; ii) having brief demonstrations, or ungraded exercises, followed by a class discussion to increase student engagement; iii) incorporating small group study periods during lectures to stimulate creativity and discuss the subject matter; and iv) using case studies to involve students in doing things and thinking about what they are doing [1] . the researchers introduced engaging activities throughout traditional lectures as they stimulate learning and retention, improve students' attitudes regarding education, and enhances academic achievement [1] [2] [3] [4] [5] . an active learning environment notably increases student participation and collaboration with peers, where knowledge is enthusiastically shared [6] . it has further been known to stimulate creativity by promoting individual and group ideas [6] . therefore, in a time when individuals need to be critical thinkers and problem-solvers, active learning provides students with the necessary tools to develop those life skills that were not necessarily on the forefront with traditional education methods. facial recognition systems become more common. industries and organizations are provides the services embedded with facial recognition systems. however, some experts are still concerned regarding the algorithms, particularly when it comes to performance on faces with darker skin. some research published by the mit media lab found that some facial recognition systems performed worse when identifying an individual's gender if they were female or darker-skinned by mit's joy buolamwini. particularly, in the research, it focused on amazon rekognition [7] that the testing system made no mistakes when identifying the gender of lighter-skinned men. otherwise, it mistook women for men 19% of the time and mistook darker-skinned women for men 31 percent of the time. considering with the facial analysis software built by microsoft, ibm, and chinese firm megvii, they are concerned to improve identifying similar racial and gender biases. recently, a number of tech companies have voiced concern about the problems with facial recognition. as bias in algorithms is often the result of biased training data. however, many researches keep studying to improve the regulation and to ensure higher standards. although, there are little to engage with this performance, many applications and systems are still suggested to apply with the technology. a lot of researches are going on to find out the best outcome for benefits in use. for example, the gender identification test was facial analysis which spots expressions and characteristics like facial hair. basically, the following are the types of detection and recognition that common recognition analysis software are provided. firstly, labels: a label refers to any objects (e.g. flower, tree, or table), events (e.g. a wedding, graduation, or birthday party), concepts (e.g. a landscape, evening, and nature) or activities (e.g. getting out of a car). the software can detect labels in images and videos. however activities are not detected in images. secondly, custom labels: the software can identify the objects and scenes in images that are specific to some business needs by training a machine learning model. for example, a user can train a model to detect logos or detect engineering machine parts on an assembly line. thirdly, faces: the software can detect faces in images and stored videos. the software shall learn where faces are detected in an image or video, facial landmarks such as the position of eyes, and detected emotions such as happy or sad. moreover, some software have provided face searching function which is indexed into a collection of faces. the software can then be matched with faces detected in images, stored videos, and streaming video. particularly, some software can identify the celebrities in images and stored videos. fourthly, people paths: the software can track the paths of people detected in a stored video. some software provides path tracking, face details, and in-frame location information for people detected in a video. fifthly, text detection: the software can detect text in images and convert it into machine-readable text. this allows the analysis is further applied in other systems. sixthly, unsafe content: the software can identify images or videos for adult and with violent content. to support the evaluation of activities during active learning, we proposed a tool to facilitate instructors to evaluate the performance and competency of learners. the prototype tool is developed. it can analyze images that are supplied as image bytes or images stored in a source folder. as shown in the figure, a part of source code allows a user to choose an image and view the estimated ages of faces that are detected in the image. the chosen image is loaded by using the displaying the image. this example shows how to unencode the loaded image bytes. the program then estimated orientation of an image and to translate bounding box coordinates. an image is loaded and its height and width are determined. the bounding box coordinates of the face for the rotated image. the program is implemented with javascript and json. particularly, it encompasses the amazon web service [7] which provides a function of image recognition. the steps to execute the program are: i) provide photos capturing the activities in the classroom in the images folder. ii) provide photos of each student in the classroom in the ref folder. iii) execute the program, entitled node; which implements the algorithm as shown in figs. 1 and 2 . iv) the program calls the web services of face recognition to identify the picture of a face. particularly, the photos including students in various gesture are computed. when single facial features are identified, the face features are determined comparing with the individual photos. the overall configuration can be described by a vector representing the position an size of the main facial features, such as eyes and eyebrows, nose, mouth, and the shape of face outline. and v) the feature matching based on measured distances between features will be then concluded. the sets of photos for each student are created. to work with the program, an instructor has to capture photos during activities of active learning in a classroom. the photos then become resources for the program. as shown in fig. 3 , the photos are taken during the activities in the classroom. to evaluate the prototype, we created two scenarios of classroom. firstly, we have a classroom with 54 students. the class is run based on active learning. totally, the students involved the class twelve hours. the 39 photos are captured during the activities. also, the photos of each student are provided. as shown in fig. 5 , the program computes by analyzing each picture. in particular, it detects the face features of each student based on reference photos. the program then create a folder of photos for each student as shown in fig. 6 . for the first scenario, there are 54 students with 39 taken photos. 54 folders are created by the prototype. the photos of each student are detected based on their face features. there are 51 folders identified the student correctly 100%. otherwise, recognition of other 3 folders are partly incorrect. one folder included 1 mismatched photo. other two folders includes 2 mismatched photos. for the second scenario, there 4 learners with 9 photos taken during the activities. there are four folders created by the prototype. the photos of each student are detected based on their face features completely. they are totally corrected identified. on average, the performance of our approach in terms of precision measurements in two scenarios seems to be consistently high (ranging from 94% to 100%). our research focuses on how to support the active learning approach. one of remaining issues is how to evaluate the performance or participating of learners effectively. basically, recording the details of students' progress is a clumsy and tedious task. our work here is to support the instructors to evaluate the performance of their students. the prototype tool is implemented in javascript in order to facilitate the evaluation of activities based on an active learning class. the tool also encompasses the amazon web service for a function of face feature recognition. the photos of students during participating the activities in the classrooms are captured and the prototype tool then generates the folders of each student collecting only the student's photos. this assists the instructors to determine the performance and participating of each student. they then can evaluate the students' progress more effectively. we plan to create a larger number of scenarios which involve different class objectives. it is therefore believed that the approach could be extended and enhanced to support a better way. in addition, sophisticated techniques for visualization could support the use of the prototype tool more efficiently. however, the prototype tool takes a long time to be processed depending on the size and number of photos. more work needs to be done to optimize the processing time. active learning: creating excitement in the classroom using active learning instructional strategies to create excitement and enhance learning active learning in secondary and college science classrooms: a working model for helping the learner to learn using active learning in college classes: a range of options for faculty the variation of memory with time for information appearing during a lecture transformation of classroom spaces: traditional versus active learning classroom in colleges key: cord-024038-8vbvnn90 authors: letizia, angelo j. title: conclusion date: 2020-04-28 journal: graphic novels as pedagogy in social studies doi: 10.1007/978-3-030-44252-1_6 sha: doc_id: 24038 cord_uid: 8vbvnn90 this chapter tackles the questions of assessment, suitability and further reading. in this chapter, a number of criteria and rubrics are offered so teachers can assess and evaluate the activities in the previous chapters. leadership field trip assignment leadership field trip assignment #2 jam law comic assignment sequence metaphor assignment constellation assignment culminating comics assignment assessment as a former k-12 teacher, whenever i hear the word assessment, i still cringe a little. i taught from 2004 to 2015. my entire teaching career was essentially contemporaneous with no child left behind and the obsession over standardized testing and data. and while it makes me cringe, i know that assessment and evaluation is or should be at the heart of what any great teacher does. mathison (2014, 247) similarly noted that assessment is usually seen as an externally mandated phenomenon which is "defined by others." mathison (2014, 247) urges teachers to resist these external calls and definitions of assessment and practice assessment that "supports a democratic vision of public schooling." a note on terminology is in order. the terms assessment and evaluation overlap, but there are some distinctions (rea, 2020) . assessment refers to the methods of gathering data on student performance and using this data to make decisions regarding learning, while evaluation refers to the judgements about students and teachers made based on those assessments (banks, mcgee-banks, & clegg, 1999; chapin, 2015; rea, 2020) . both are important for our purposes. teachers and professors must use the information gained from the assignments in this book (e.g. ideas from class discussions and student created comics) to continually improve the assignments, adjust them to the unique conditions of individual classrooms, to evaluate the effectiveness of the assignments and teacher practice, and ultimately, help students arrive at their own ideas of citizenship in the hyperreal. in order to accomplish these tasks, teachers need to know if what they are doing is indeed helping students think. a standardized test created by a corporation i think is very limited in this respect. perhaps this is the crotchety teacher coming out in me, but to truly assess and evaluate student thinking, we need to go much deeper. it simply requires more. below, i examine a number of ways that graphic novels, both the reading and creating of them, might be assessed and evaluated. there are not specific rubrics for every exercise, although i do have rubric for the culminating symbolism assignment (the mini-comics assignment) which was presented in chapter 5. i think rubrics for every assignment would defeat the purpose and crush any variability and spontaneity. following kincheloe (2001) , i am wary of anything standardized for classrooms and teachers because this standardization neglects the fact that each classroom is different. i do not want to crush teacher creativity (kincheloe, 2001) . instead, what i offer below are more general guidelines for teachers to make evaluations about student work. these guidelines however do not just serve as assessment guidelines, but also as a way for students to see what is expected of them (for my culminating symbolism project, i showed students the rubric beforehand so they knew what i was looking for in the project). two other important notions related to assessment are formative assessment and the use of different assessment tasks. chapin (2015) examines the importance of formative assessment, which is assessment that is ongoing during teaching. specifically, informal formative assessment, known as "on-the-fly" assessment is crucial (chapin, 2015, 159) . regarding formative assessment mathison (2014, 256) asserts that assessment can be fused with instruction, which can yield a "continuous flow of information about what students do and do not yet know." informal formative assessment includes looking at students work as they complete it to see what they know, teacher questioning of students and observations of student discussions (chapin, 2015; larson, 2017; mathison, 2014) . i utilized these types of formative assessments for the comic assignments. as students were completing the assignments and frameworks, and even as i was explaining and assigning them, i constantly checked for understanding. i talked to students to ensure they understood what was to be done. after we discussed completed assignments, i used the discussions as a type of assessment. one thing i learned about implementing the reading assignments was that i needed to make sure students were paying attention not only to the content, but the comics form. indeed, it is the comics form which differentiates this whole enterprise from just reading regular prose novels. assessment can consist of written, oral or visual categories (2015) . in the written category, stories, diaries and posters can be utilized. for the visual category, cartoons and drawings can be used (chapin, 2015, 161) . there is overlap between the categories (chapin, 2015) . student created comics may be excellent evidence of the visual and written categories in conjunction. the process of action research is also useful here as well. mertler (2020) argues that action research is performed by teachers and for teachers to understand their own specific classrooms and schools. for action research studies, teachers generally first identify a topic that they want to examine and then consult scholarly literature (larson, 2017; mertler, 2020) . next teachers decide on their research methods, implement the intervention and collect data on it and analyze the data (mertler, 2020) . finally, an action plan for the future is developed, the results are shared and the teacher reflects on the process (mertler, 2020) . of course action research is not this linear, but this gives some inkling of the process (mertler, 2020) . action research is cyclical, meaning that an intervention can be performed by the teacher, who then reflects on it, and can then keep implementing the same intervention while incorporating the lessons learned from the last iteration (mertler, 2020 ). an important consideration of action research is the uniqueness of individual classroom settings (larson, 2017; mertler, 2020) . schwarz (2013) specifically argues for something similar to implement graphic novel pedagogy in classrooms. schwarz (2013) argues that teachers have access to their classroom, credibility with their students, flexibility, and the ability to utilize valuable student feedback right away. while large scale studies can be beneficial, schwarz (2013, 153) notes: a wonderful empirical study on graphic novels done by an educational psychologist or expert in educational technology may not be applicable to these students in this place at this time for this purpose. yet, one teacher may find results in a small, qualitative study that resonates with many other educators. this quote perhaps best captures the essence of teacher led research and the assessment and evaluation aims of this book. teacher research is relevant for the teachers (and most likely other teachers in similar situations). teachers can evaluate for themselves how well their intended methods and frameworks are working in the classroom. mertler (2020) and schwarz's (2013) ideas of action research and teacher research helped to inspire my own research process for this book. i wanted to examine how graphic novels (both the reading of them and creation of them) can help students grapple with citizenship in a number of ways. i consulted the literature (see chapter 1 and relevant parts of the other chapters), implemented the activities in my classes, gathered information and feedback, and am using the feedback to guide future implementation. i am also sharing the results (this book, as well as at professional conferences). the data analysis utilizes the formative assessment procedures and rubrics (i will get to these shortly). in line with action research, i view these endeavors as incomplete and cyclical. while it was not feasible for me to implement every assignment in this book, i learned from the assignments that i did implement. i would recommend that teachers implementing the comic assignments utilize similar procedures and consult the literature on action research for more clarification (larson, 2017) . teachers should continually reassess these assignments and their implementation in their own classrooms and use this information to guide future implementation. i am constantly learning from classes how to make the assignments better and more productive for the next iteration. there are at least two levels to this: number one, teachers must learn how to best go about implementing assignments and how the students perceive the assignments in general. secondly, teachers must learn how to best assess and evaluate those assignments. principles from formative assessment and action research can help teachers and professors accomplish both of these tasks. below, i examine some of my experience with both levels indicated above. something that became very clear to me when i began to implement these assignments was the need for flexibility. i had to make many adjustments on the fly. students, many times asked questions which i did not anticipate and which forced me to reconsider aspects of assignments. for instance, in my history 402 class, we examined an excerpt from the graphic novel uncle sam. i used this excerpt to introduce the ideas of symbolism and imagery and part of our political discourse. we had a great discussion in class. afterwards, my students noted how they would have liked a primer on how to read comics. this information was extremely useful as i moved forward and introduced more comics related material. the point is, i think when implementing these assignments, teachers and professors need to listen to their students. teachers and professors must be flexible. many of these assignments are asking students to take on creative, open ended tasks which may create ambiguity. i view all of this from an assessment perspective. my own experiences in implementing comics related pedagogy have been invaluable and constantly inform future practice. every comment and criticism (either formally on class evaluations or informal comments) a student gives to me helps me to refine these assignments for the next go around. obviously no two classes are the same, but formal and informal student feedback has helped me to continue to fine tune these assignments. paying attention to student comments is good practice (carter, 2013) . this is probably good advice for any teacher and all assignments, but i think it especially pertinent with these types of assignments. something else i noticed, to my own chagrin, was the fact that i simply could not get to all details in all lessons. for instance, in one lesson, i drew an example without panels. i simply forgot to include them because i was so focused on conveying symbolism. when i reflected on lessons, it always seemed like there was something that i forgot to include. i admonished myself, but then i realized something. i was asking my students to do so much, usually in a short amount of time. i was asking a lot of myself as well. the point is i think that forgetting things is inevitable, especially when you have limited time. i just reflect on what i missed and try to incorporate it in the next go around. a number of students gave me formal and informal feedback on the comic assignments in various classes, some of which i want to share. in the final course evaluation of my history of education class (edu 402), one student highlighted the assessment potential of the culminating comic project in chapter 5. the student wrote that the comic book assignment (along with another unrelated assignment) were powerful methods of knowledge assessment that did not rely on traditional means, such as multiple choice tests. the assessment potential of the comic is worth noting. i was able to assess the student's knowledge of history in a new a creative format. after my constant harping on how comics can be used in a variety of ways, as well as assigning a number of comics exercises in class, one of my students in my human development class actually used a variation in her own class. i include this as a form of assessment because it helped me see new applications for some of the material in this book. she told me that she began to use comics to teach aspects of the social studies, specifically economics. she began using calvin and hobbes comics to teach concepts of opportunity costs and supply and demand. these are admittedly somewhat boring concepts, especially to fourth graders, but she found a way to use comics to convey these dry social studies concepts in a lively way. i believe her words are worth quoting at length. she wrote in an email to me: we were discussing supply and demand, specialization, competition, and opportunity cost after introducing the industrial revolution and maryland's economy. the comic strip discusses how without competition, prices increase and quality of product may decrease. calvin demonstrates opportunity cost with how he is skimping on environmental and safety issues in order to keep prices lower, despite it not being satisfactory to the buyer. for supply and demand, this illustrates perfectly the consumer is the demand and supplier is whomever fills the need. specialization was a bit trickier, but after discussing maryland's economy and using the abundance of certain natural resources to specialize and build industries, the students actually connected that calvin is using what resources he has to make a product. even if that product is just a lemon thrown in water. while it's not a perfect representation, it gives a picture of how economics is a relationship between the buyer and seller. i had two students act out the cartoon, and we labeled the parts as a class. we then labeled the labels with examples from maryland. the students seemed to enjoy. (e. gore, email communication, may 29, 2019) she believed that her efforts were successful. while this book focuses on using comics from grades 6-12 and at the university level, i think her adopting comics to teach social studies concepts in an elementary classroom is indicative of how teachers can adopt the ideas in this book to their own needs. she even incorporated a performance aspect, where students had to act out their comics, which might be especially enticing. skits and dramatizations are other types of assessment (chapin, 2015) . another student, who was a ph.d. student in my law class, wrote this to me in an email: every year, my students complete an assignment that connects them to multilingual communities. one of the options i give them is to create a bilingual book to be donated to a family in our community. because of what i learned in dr. letizia's class about some of the pedagogical benefits of the graphic novel due to the purposeful sequencing and encapsulation, emotionally-charged images, and transcendence of meaning, i have decided to encourage learners to consider creating a graphic novel for this assignment. (c. dugan, email communication, july 27, 2019) heather, another student of mine (and also a contributor of student artwork to this book), is a principal and teacher at a private school, decided to offer a graphic novel option for her classes. her ela classes complete a virtue journal, where students utilize art and text to analyze specific virtues found in tolkien's lord of the rings. heather added a graphic novel option of this assignment and had me come into her classes and give a presentation on how to utilize the features of the comic's medium to this end. i gave a short lecture and then did comic activities with the class. in the course evaluation of my other summer law class, where i also utilized a number of comic activities with ph.d. students, a student wrote: i highly enjoyed the comic assignments because of the creative aspect, which can typically lack in ph.d. courses. creativity is a modality that gives a clear understanding of concepts learned with students and i enjoy the challenge of those types of projects (others do struggle with this though). it is important to give assignments that allow students to demonstrate their understanding of knowledge in variety of ways, i felt this course did so. as carter (2013) cautions, student comments may be influenced by the instructor's enthusiasm for comics. he noted that he is a comic advocate and this may have swayed their comments. similarly, i am a huge advocate of comics and this may have influenced the comments my students shared with me. nevertheless, like him, i do believe these comments are instructive and important to examine so i have included them here. they also show how nothing in this book is immutable, everything is up for grabs. i want teachers to take what i am doing and mold it to their own purposes which these students did. while i have had success in implementing these frameworks, i would be remiss if i did not mention some of my less successful attempts. these are important to acknowledge and reflect upon. early on, i tried to implement the drawing your neighborhood activity in my social studies methods class. i distributed the assignment and gave them time to complete it, but no one did the assignment! i concluded that i did not give clear directions. after my human development and learning class completed some frameworks, which i felt were successful, a student, who highly enjoyed the assignments, told me i should give clearer directions. so, i learned that clarity is key. sometimes i become overly enthusiastic, and the assignments make sense to me, but some of the instructions are lost on the students. another time, i had planned to implement one of the buddhist assignments, but at the last minute, i declined. i had it printed it out, we discussed buddhist theory and was about to distribute it, but something held me back. i did not feel my class was ready for it, and i did not sense any enthusiasm for it. i could have been wrong, they might have been ready, but i think it important to pay attention to these gut instincts. sometimes, you may not feel ready or you may feel that your class is not ready and that is okay. sometimes it is best to not push these types of assignments if you feel the class is not ready. the informal formative assessment (chapin, 2015, 159 ) may be especially pertinent to judge student work in chapters 2 and 3. teachers can observe their students at work, and ask pertinent questions to check for understanding. this is what i did. i usually break students into groups and visit with each group individually. i would use their discussions and link these back to things we were learning in class. i would also use discussions to generate new questions for students to ponder. the structure of all the comics reading assignments (contained in chapters 2 and 3) in this book are varied, but i condensed some of what i discovered to be the most important points when facilitating a discussion which cut across all the assignments. while conversation and class discussion should be organic and not pigeonholed into boxes and rubrics, i think this list can be a useful guide. content: are students engaging with the content of the assignment? form: are students engaging with the form? are students considering how the comics form helps to convey ideas and information? citizenship: can you guide the discussion of content and form towards any citizenship ideas discussed in the chapter 1? bloom's taxonomy: are students using levels 2-6 in blooms taxonomy, and thinking about the content and form in a number of different ways? another simple method to determine the effectiveness of the comics assignments is to ask yourself how well the products, either in the form of comments, class discussions or written work, helped to achieve the stated class objectives. i began using this method in my college classes. the objectives that i created gave me a benchmark to make this determination. the assessment of the activities in chapters 4 and 5 lend themselves to more formalized rubrics and other similar criteria, since students are creating a tangible product (although i do not grade all of these assignments with formal rubrics). below, i appropriate different criteria and methods of assessment that teachers and professors can utilize to evaluate student comics. barone and eisner (2012) offer criteria by which to examine and judge arts based research. importantly, they opt for criteria rather than standards because unlike standards, criteria are more flexible and do not offer a discreet "quantitative metric" (barone & eisner, 2012, 147) . criteria are more "slippery" and entail judgement of "significance or value" (147). i adapt their ideas to student created work, but also stress that it is important to link these criteria to citizenship in hyperreality. the criteria are: incisiveness (regarding the issue at hand, does the research get to the point?), concision (no excessive verbiage or material), coherence (does the piece work? does it "hang together?" do the elements "work together?"), generativity (the way a piece allows the viewer to see phenomena differently), social significance (the work has a "focus on the issues that a make a sizeable difference in the lives of people within a society") and evocation/illumination ("one begins to feel the meanings that the work is to help its readers grasp") (2012, 148-154). regarding citizenship, student's creations should be concise and incisive. they should help other citizens, especially those of a different political persuasion, to think differently about a topic. the piece should also make sense to readers so they can get something out of it. of course, there can and usually should be multiple meanings of a piece of art (duncombe & lambert, 2018) . the social significance piece easily relates to citizenship-the issues which students tackle should be pressing issues in the republic. epstein (1997) posits some methods that she employs to evaluate student art work that deals with historical topics which i have utilized as well. epstein (1997) had students study primary sources, such as painting and poems, and then create their own poem, picture or song. one criteria epstein (1997) uses is that of representativeness, which she defines as containing "historical accuracy, probability, possibility or plausibility." another criteria epstein (1997) employs is that of empathy instead of logicality. she looks to see if the student created work can generate empathetic understandings for historical persons or situations. epstein also utilizes the notion of expressiveness to evaluate student art. she looks to see if the work can help an evaluator realize an image of the experience portrayed (epstein, 1997) . novak (2014, 162 ) created a rubric to assess student comic creations. this rubric centers on how well students utilized the different aspect of the comics. i have adapted and modified this rubric (the rubric was presented in chapter 5). i have also combined it with aspects from epstein (1997) . the criteria for my rubric are: narrative/historical understanding, use of comics medium, interanimation of words and pictures, and the images themselves. these criteria embody not only literary and artistic qualities, but literary and artistic concerns in conjunction with historical and citizenship notions. i stress to my students that i am not concerned with how well they draw, but rather, how they use art and words to convey meanings of citizenship. if given the rubric beforehand, it may foster creativity and conversations about that creative process and product, somewhat like distributing a playbill prior to a play (barone & eisner, 2012) . i do not see the rubric as an immutable standard that students must attain. this rubric, along with criteria from epstein and barone and eisner, can help teachers evaluate student created art work. further, barone and eisner's (2012) , epstein (1997) , and novak's ideas are related in many ways to the concern of the first chapter and citizenship in hyperreality. they all call attention to the actual construction of the comic, to historical plausibility, to its significance and ability to generate empathy. following epstein (1997) , we want to indulge students' imaginations, but not allow for an anything goes approach. there needs to be some grounding in reality, students cannot simply abdicate into fantasy. at the same time, there must be ample room for creativity and divergent thinking. all student creations are "evidence." each student assignment gives me a window into a student's thought process regarding citizenship. many of the student created comics most likely do not require a formal rubric or some similar type of assessment. many times i only require small comics as part of a classroom activity. for the larger assignment like the culminating symbolism activity in chapter 5, a more formal rubric or something similar might be necessary. nevertheless, even with the more informal assignments, the points above can still help teachers and professors judge student work. teachers can ask of student work: is the work illuminating different aspects of an issue? does the student work help to convey political and historical messages or messages regarding citizenship theories? is the work concise and use minimal verbiage? these loose questions can give teachers and professors some idea as to the worth of the assignments, as well as ways to improve for the next iteration. teachers can look for historical accuracy (when applicable), historical possibility, and empathy (epstein, 1997) as well as incisiveness and social significance (barone & eisner, 2012) in the more informal student comics done as classwork assignments. the guiding questions can get students and teachers thinking explicitly about how the work relates to some of the citizenship ideas in the hyperreal. some examples. i wanted to provide some examples of exactly how assessment and evaluation student created comics may look. the assignment from chapter 4, where students had to draw their own version of reality (vor) is shown in fig. 6 .1. the student who drew this cartoon is an art teacher, so it looks better than most. this assignment had students construct their vor or version of reality (shealy, 2014) . the assignment called for an exploration of how beliefs and experiences cause actions. since this piece is not as historical, i utilized barone and eisner's (2012) criteria. the student who drew this is a lesbian who grew up in a very conservative area of the south. she then moved to the baltimore area as a young adult. the first thing that struck me was the incisiveness and social significance of this piece. this piece touches on lgbtq issues which is obviously a pertinent question in contemporary american society. further, the piece is concise. she uses minimal verbiage to get her point across. there are some speech bubbles but the words are minimal. however, the words are important to the piece, they help to convey her rejection by society and the piece might not be as powerful without those words. the piece is also coherent, it is put together well and makes sense. this piece also speaks to the notion of generativity. arts based research makes you see the world differently, it "possesses the capacity to invite you into an experience that reminds you of people and places that bear familial resemblances to the settings, events, and characters within the work" (barone & eisner, 2012, 152) . in the first panel she is rejected by society that scorns her, but in the second panel we realize that she doesn't really care. i have known some from the lgbtq community (as well as other marginalized and non-dominant communities) who experienced a similar type of rejection. yet the piece is actually happy. she is rejected but overcomes this and starts a new life in a more welcoming community-baltimore. i can see the rejection and happy ending in this piece and i think it is as or more effective than reading about it. it is here where i also began to "feel " the meaning and power of the piece, which is what barone and eisner (2012, 153) call evocation and illumination. this assignment fig. 6 .1 student example of vor assignment let me truly see and feel jennifer's vor. as a heterosexual white male, i may not be able to truly understand what she went through, but at least i can begin to contemplate it. and this contemplation of another human being, another citizen, is an important part of citizenship. i saw how her vor shaped who she is and led her to act the way she did. she encoded elements of her vor into a comic to challenge injustice and invite readers to consider this sensitive topic from a new and more empathetic angle. for the second example ( fig. 6.2) , students had to illustrate a scene from the court cases we read in my law class, the sequential law assignment. we had spent a good deal of deal of time talking about first amendment rights in the classroom which is reflected in the comic. the comic is concise. it does not employ excess verbiage or details. the words are sparse but necessary. some of the verbiage is posted, such as the title of the school and the establishment date on the post (the date is 1791, the date the bill of rights were ratified). much of the factual information is conveyed pictorially, such as the gate itself, which is a reference to the famous quote "it can hardly be argued that either students or teachers shed their constitutional rights to freedom of speech or expression at the schoolhouse gate" in landmark tinker v. desmoines case. the second frame has the children from tinker, who are admitted to the school because their armbands were found to be free speech, while students in later cases were denied. all of these are accurate historical details which are woven into this fictional piece. here epstein's (1997) ideas of historical accuracy and plausibility come to mind, as well as the idea that arts based research can be empirical (barone & eisner, 2012) . the story and activity in the comic are simple and repetitious. there is a school which are some are admitted to and some are not. it is simple and to the point, which relates back to incisiveness. the social significance of this piece was blatantly obvious to me. the question of free speech is ongoing and complex in hyperreality. what constitutes free speech-and what are the parameters in a school setting? the comic forces the reader, by use of skillful encapsulation and historical detail in sequence, to confront this question in a unique manner. the next example is from the culminating symbolism project in my undergraduate class (fig. 6.3) . the rubric from chapter 5 (built from novak's rubric) calls attention to the historical elements of the piece, the use of the comics medium, the interanimation of words and text and the images used, specifically the symbolic nature of the images as well as citizenship concerns. as noted in the last chapter, one theme that we discussed at length in my history class was the notion of education as an institution of liberation or social control and my student drew a comic examining these ideas. she depicted different scenes throughout the history of american education which spoke to the dual nature of education. please note that i only included the last page. for one, her comic involved a good of historical elements, such as the effects of the protestant reformation, american immigration and women's liberation to name a few. so, she exhibited historical knowledge. however, in some cases, she had to make decisions about how people felt and expressed themselves in these different circumstances. she could infer this knowledge from our readings and discussion, but ultimately had to depict it the way she saw fit. she expertly used the comics medium, as well as emotions on the figures she drew to convey her messages. the scenes of social control were left in pencil, while the scenes of liberation were in vibrant color. the scenes of social control were drawn with more constricting panel sizes. in addition, on pages depicting no social control, there were no panels at all. this i thought was an excellent use of the comics medium itself, specifically the use of encapsulation and the page as a unit, to convey messages about historical materials and ideas related to citizenship. while she could have written about social control in a traditional way, i think she captured the essence of social control visually and it resonates deeper with the reader. social control and liberation are two narratives of education and she was able to grapple and present both of them in her comic-and many times we as citizens are forced to deal with simultaneous and competing narratives, especially as they proliferate on social media. in addition, the symbols and imagery she employed were powerful; pencils, priests, schools, american flags as well as people and situations that she created for the comic. she spoke to the reader with symbols, which is an important skill in hyperreality. these examples demonstrate one way to assess and evaluate student drawn comics. students had to weave factual information, emotional contexts and utilize ideas such as symbolism and encapsulation to make a better truth, a truth which may resonate with people in hyperreality. students used art and text to be citizens and provide commentary on important political issues in new ways. these types of efforts may now be especially useful for citizens in hyperreality to get their messages across. as potts (2013, 18) notes, registering the visual information of comics usually happens on "a gut level" and it usually occurs during later readings. this happened to me when reading my students comics. my students are not professional comic's creators, but they are all insightful, smart people who crafted comics with a ton of references, information and symbolism. i first read the comics and registered their information on a gut level, and then i had to go back a re-read the comics multiple time to truly grasp their power. sometimes i did not see things until weeks later. the assessment tools should not be applied rigidly to student work, rather, they are meant as a guide only. teachers may want to read the comics first as a comic, on a gut level, and then go back and dissect them. here, i think it important to link assessment and evaluation of this project to citizenship. what is the link between the assignments in this book and citizenship in the hyperreal? as stated earlier, i do not claim that these assignments will make students better citizens, but the assignments can at least start a conversation as to what this citizenship might look like. below, i have also provided some questions for teachers and students to consider. these questions are a shorthand for all of the complex ideas sketched out in chapter 1 and ask how the use of the comics form helps to portray these notions of citizenship. these guiding questions, in conjunction with the ideas of epstein (1997) , barone and eisner (2012) and novak (2014) , are an attempt to link the assignments in this book, and the products that result, to citizenship in the hyperreal. moreover, i think this relationship must always be flexible and tenuous. citizenship cannot be nailed down into a quantitative score. rather, i think citizenship must be viewed through the things we as citizens create, through our conversations, through actions and relationships to other citizens. professors and students themselves should answer the questions below by giving examples from student work and using these examples to facilitate conversations. hopefully, the rubrics and guiding questions can begin to illuminate these elements. but the result of the rubric and answers to the questions below are not final, rather they are just another intermediate step in the ongoing process of maintaining a healthy republic. 1. how effective is the student's product and use of the comic form in portraying new truths which challenge injustice? 2. does the student's product and use of the comics form help to illustrate the skills and dispositions of citizenship in new ways? or put forth new ones? 3. does the student's product and use of the comic form allow the student to relate to others in the republic in new ways? 4. if democracy is creative, how does the student's use of the comics form help advance democracy? how does the student's use of the comics form problematize or question the status quo? 5. how does the student's product and use of the comics form create something better that exists already or least point in that direction? what new visions or dreams does this work put forth? how might it help us imagine a better future? 6. does the work disrupt power relations of any sort? 7. does the work and its use of the comics form help us to understand the roles of emotions in the republic and how these emotions may be tangled up with the political process? i briefly examine another student's work, which is featured in fig. 6 .4, with the above questions in mind. the point is not to answer the questions, but use them as a basis for reflection and conversation. the comic charts some educational developments of the past in regards to segregation and desegregation, some of the tumultuous present and america's continuing struggles with racism and discrimination through more insidious means (e.g. zoning and hiring practices) and it ends in the year 2040. my student's use of symbolism and imagery is intriguing. she incorporated buses, maps, contemporary signs (e.g. maga hats) to convey her ideas. she used strong symbols and imagery, such as by depicting desegregation with the use of black and white dots as well as the use of a maze. in her own words, she wrote: "i chose to use a maze to show the systemic barriers that prevent black individuals from succeeding in the same way that white individuals do easily." i think her use of a maze could also be related to dispositions-she is depicting dispositions of social justice and equality visually (an inequality). moreover, there is an obvious commentary on power relations throughout the whole work which can make the work a statement on justice oriented citizenship. my student however, in line with ideas sketched out in the first chapter regarding hyperreality, used the comics form to make this statement. please note that i only included one page of this comic. in a wider sense, i think her work also speaks to the notion of creative democracy. american democracy has had a long struggle with discrimination and continues this struggle as it evolves into the twenty-first century. her work is appropriately titled: a question of progress. my student recast elements of the current political situation in the united states into a statement about creative democracy and used the comic form to accomplish this task. her comic challenged me to confront a sobering question: have we really progressed, or has racism and discrimination have just taken new forms? she left her depiction of the future open for the reader to decide. it does not really posit a dream or vision of the future, as much as it challenges the reader to consider what has happened in the past and so as to make a better democracy. i think the comics assignments are participatory and resemble art activism as well (duncombe, 2019; duncombe & lambert, 2018) . students discuss their comics with each other, they reflect on each other's comics and sometimes they work together on joint projects. comics become a class discussion, a group thing which helps to change thinking and values (duncombe & lambert, 2018) . i believe the act of teaching progressive citizenship is inherently activist anyway. students are (hopefully) changed because they are taught to think differently and the hope is that they then will take these new understandings and then participate in the republic. in this case, students are taught to think differently with comics pedagogy. further, if the comics activities in this book are done with current or prospective teachers as i am advocating, those individuals can then take the comics pedagogy and adapt it for their classes, infusing countless students who are citizens in the making in hyperreality. this is happening to me, a number of my students have taken these ideas and adapted them for use in their own class, as i noted earlier. i have intended this book for a broad audience. the audience includes (but is not limited to; 6-12 social studies teachers, social studies methods professors, professors of history and other professors interested in promoting citizenship). with that said, what is suitable in a college classroom may be wildly inappropriate for a 7th grade classroom. so, for one, there has to be great discretion and professional judgment when employing comics in the classroom at the middle and high school level. there may also be curricular concerns at the middle or high school level because these teachers, especially in core subjects like history, are generally more restricted by standardized tests and state curriculums. it may be a hard to sell to bring a batman comic into a class which does not have time to spare and where the teacher is racing through content to teach what is on the end of year test. nevertheless, i think comics can still be utilized in the 6-12 classroom in a variety of ways. there is a great variety of comics and graphic novels, many of which are directly applicable and relevant to the study of social studies and history. graphic novels pertaining to the holocaust and ethnic cleansing in bosnia (christiansen, 2006 ) are a few relevant examples. i do not believe it would be hard to integrate these type of books into a social studies classroom. many of these graphic novels do not read like academic texts, they are engaging and can draw readers into the story. there may be more difficulty in utilizing traditional superhero comics in the classroom, but even here, i do not believe the difficulties are insurmountable. for one, even traditional superhero comics may at times, even tangentially, center on real world events and social issues, or the subject matter itself may be of social concern. further, comics can be understood in historical terms (ricca, 2012; wright, 2003) . social studies teachers can look carefully at comics and find links to their curriculum or the ncss themes and use those links to justify the inclusion of those comics. mathison (2014, 262) does note that while linking assignments to standards is good practice, this should not be an "exercise in compliance." rather, teachers must constantly work though and figure out how their assignments link to the standards but how this action is situated in the wider purposes and goals that the standards promote (mathison, 2014) . teachers can make these links explicit to their students or help guide student's to find those links as part of the overall analysis of the comics. further, teachers can constantly link the comic's assignment to larger social studies goals and to student's decision making in the republic by asking such questions as outlined in the last section. if a teacher uses a superhero type comic or graphic novel as a short, a supplementary activity which takes twenty minutes in class, this would obviously be less problematic than the teacher spending six weeks studying a superman comic. another consideration may be the actual method used to analyze the comic itself. for example, suppose a teacher has to teach about buddhism. if for a supplemental activity, the teacher was to utilize or modify a buddhist framework offered in this book, and has students apply this to a comic or graphic novel with no link to the curriculum (e.g. a batman comic), this could still be justified because the method itself centers on buddhism. here, the justification and link is centered on the method of analysis, not the content itself. these suggestions largely apply to classrooms where time is of the essence and where teachers must race through material in order to prepare students for a state test. however, many times, high schools have elective classes. while i cannot speak for all schools and classes, i can speak to my own experience in teaching electives. i would assume that other teachers may have similar experiences. i taught two electives, introduction to philosophy and african american studies. in these classes, there were no required state tests and there was little oversight of the curriculum. i had almost free reign to do what i wanted. again, i cannot speak to how other schools structure or oversee their electives, but at least for me, i had a great deal of freedom. if teachers teach electives and have freedom like i did, the use of comics to promote citizenship can be very beneficial. indeed, elective classes in high school may be the perfect laboratory for testing alternative and radical approaches like the use of comics. another issue for high schools is suitability of materials. chapin (2015) notes how there may be reluctance to use some graphic novels due to their sexual and violent nature as well as questions by administrators and the community at large as to the appropriateness of graphic novels in the classroom. obviously this is an issue that cannot be overlooked, especially in middle and elementary schools. while i would not advise using explicit materials in an elementary or middle school, there may be some leeway in high schools. for instance, when i taught african american history, i had to send home permission slips when i showed certain movies, such as malcom x. while the permission slip was a practical measure, it could also, as one of the reviewers of this work pointed out, allow for a degree of student choice. of course, depending on the school and the district, it may just be prudent to steer clear of all questionable material or at least check with an administrator. the suggestions above largely pertain to student interpretation of existing comics. however, students using artistic techniques in class and creating comics can be a very useful pedagogical strategy as pointed out throughout this book and may be less problematic to implement. so, finding links with the curriculum and being able to justify these links, the length of the activity and method versus content are all ways that teachers might be able to justify the use of more problematic or seemingly irrelevant comics and graphic novels. in addition, utilizing comics and graphic novels in elective classes, as well as the use of permission slips might be other methods that teachers may use to bring comic and graphic novels into the classroom. now we come to letizia's basement. so named because for most of my life, my comics have been kept in a basement. in this section, i am going to give you a virtual tour of some of my own comics as well as other comics that i am familiar with to show how comics may have political, legal and/or citizenship themes. the purpose of this section is twofold. the first is to enumerate a specific list (actually two lists) of comics and graphic novels for readers. the second is to demonstrate how many graphic novels that may not seem relevant actually may have important themes upon closer inspection, one just has to be cognizant and look for them. i believe that almost all comics and graphic novels conceivably have some value to teach citizenship and raise interesting questions. the first list is a list of graphic novels that deal directly with historical and social themes. i compiled this list from my own readings of graphic novels, with recommendations by a reviewer of this book and by consulting other lists and pieces written about specific comics (kavaloski, 2012; missiou & koukoulas, 2013; ricketts, 2013; wolk, 2007) . specifically, christiansen (2006) offers an excellent list of historically themed graphic novels specifically for the social studies. mctaggart (2008) also offers a wide list of graphic novels of general interest from elementary to high school. she also delineates by subject (e.g. science, history etc.). i also was given recommendations by a former colleague, jessica north, who was an ela teacher and is now a high school library media specialist. some of the works on the list are memoirs and works of comics journalism. duncan, smith, and levitz (2015) point out that memoirs are not straight, factual history. rather, they can be a combination of facts, memory and fiction. other pieces on the list below are examples of comics journalism, such as the sacco pieces. nyberg (2012) asserts that sacco's comic journalism helps call attention to the interpretive nature of journalism in general. this interpretive function of comics journalism and journalism in general, as well as elements of memoir, may need to be discussed with students as they read. the second list is further subdivided. i first enumerate fictional comics and graphic novels (many of them traditional superhero comics and graphic novels) that i used as examples in this book. following this portion of second list, i then give an in-depth analysis of some other comics and graphic novels which i believe can be used to grapple citizenship themes. obviously, no list of comics and graphic novels can ever by exhaustive. list #1 abina and the important men, by trevor getz and liz clarke. this story chronicles the experiences of an african woman named abina, who lives under colonial european rulers and who is illegally enslaved and brings her owner to court. the work was created from court records. this work also has a text section where the author, a historian, details the process of constructing a history from archival records. a jew in communist prague, by vittorio giordano. this is a fictional work which chronicles the experience of a citizen in communist country. this section continues the list of comics and graphic novels that at first glance may not seem to contain any relevant ideas, but upon further inspection, actually might. in this section, i offer some description and analysis. entertaining comics (ec comics) aroused much controversy in the 1950s (wright, 2003) . and while many of those comics were gory and outlandish, wright (2003) points out that many contained amazingly prescient social and political barbs aimed at american society (wright, 2003) . this fact makes them excellent objects of study for this book. many of the ec comics have been reprinted. i have some paperback issues, but there are also hard cover anthologies. one story from weird science-fantasy, no. 6, february called the inferiors is a tale that i read as a kid and to this day haunts me. a group of astronauts are on a mission to answers questions about a lost but highly advanced alien civilization. what we come to find out is that the civilization voluntarily killed itself because it had grown decadent and immoral. yet a small band of the aliens did not want to die, they wanted to live with their imperfections, and so their memories were wiped and they were jettisoned off the planet to start a new civilization-earth. we descended from the immoral and decadent rejects. it is explained that man is the ultimate de-evolution of this once great race. it is a harrowing tale which can be used to call into question humanities grandeur, sense of purpose and self-worth. further, while the question of species suicide may be delicate at the high school level, it is a fascinating concept which can spark questions of societal peaks and declines, and the ultimate question of when and if a society is too immoral or decadent to continue. another ec story, this time from shock suspense stories, no. 13, september, titled blood brothers, can bring up questions of racism. a man named sid finds out his best friend is actually part african-american. sid torments his friend and eventually burns a cross on his lawn, driving the friend to suicide. sid is obsessed with the idea of negro blood contamination. the town doctor explains to sid that the idea of negro blood is nonsense and that when sid was a child, the doctor had to perform an emergency blood transfusion from a black farmhand. the story ends with sid in disbelief that he has had negro blood in him all along and that he drove his friend to suicide. for me, this story always called into mind irrational racist and ethnic hatreds juxtaposed with the ultimate similarity between white and black bodies. it is a harrowing tale that is just as prescient now as it was then. the graphic novel titled no power in the 'verse is a story which chronicles the adventures of the characters from the movie serenity and the series firefly. in this story, there is a tyrannically interplanetary alliance and two rebel groups opposing them. one rebel group is willing to use any means necessary, even killing innocent people, to topple the alliance (and the leader speaks these words with a maniacal face). the other rebel group, led by the pilot mal, want to topple the alliance but question and disagree with the other group's course of action. for me, this story always called into question the price one has to pay to achieve drastic political change (something directly referenced in the book as well). the comic makes people question the uses of violence to achieve political ends and for me, brings to mind historical figures who did advocate for violence to achieve change. further, this violence was not only directed at the opposition, but innocents as well, who became collateral. again, some of these topics might be sensitive with high schoolers, but political violence has been part of history and it needs to be discussed. the events of no man's land, which was a massive story arc in batman comics, can also call attention to citizenship questions. in this story, gotham city is abandoned by the rest of the country. the comics open with "â�¦and after the earth shattered and the buildings crumbled, the nation abandoned gotham city. then only the valiant, the venal and the insane remained in the place they called no man's land." batman: shadow of the bat, no. 86 from june 1999 opens with a description of gang wars. in batman: detective comics, no. 733 from june 1999 has batman, like solomon, decide who a baby belongs too. the point is that law and order are gone, and this situation, new types of citizenship ideas may be needed. it can lead to an interesting discussion of how people should (or might) act in this situation. i also think of the walking dead series which explores these notions as well. in the absence of government, how will people act? how should they act? what would citizenship look like with no government? these questions can be excellent springboards to deep discussion. the graphic novel joker time! is a three-part story written and drawn by bob hall. in one instance, television executives are not compelled to give up joker's location because due to constitutional concerns. joker is giving them a story for a television show. this is over gordon's objection that the joker is a public menace. a few panels later batman, with his mask off, watching the show with alfred laments: "this isn't happeningâ�¦not in a civilized country." to which alfred retorts: "i'm afraid it is sirâ�¦" of course joker dupes everyone and terrorizes the cast because of this decision. but i think the comic can raise interesting points. if the joker or someone like him was real, what protections would he be given? how would the law deal with this? how do you balance freedoms with security? again, the fictional comic, while dramatic and over the top, can raise these types of questions to ponder. finally, there is the graphic novel titled scarlet written by brian michael bendis and illustrated by alex maleev. the back cover sums it up: "scarlett is the story of a woman pushed to edge by all that is wrong with the worldâ�¦a woman who will not back downâ�¦a woman who discovers within herself the power to start a modern american revolution!" scarlett, a young woman, is pushed over the edge by a heinous act of police brutality against her friend. in one scene, scarlett brutally murders a dirty cop. she reflects on this act and notes: "it is my opinion that he gave up his rights as a human being by betraying societyâ�¦" these types of statements can serve as fodder for class discussion on rights, citizenship and accountability. this graphic novel however is extremely explicit with regards to sex and violence, and probably cannot be used in a high school classroom. nevertheless, it can raise important questions. i truly believe that if one looks hard enough, they can find political and citizenship ideas in almost any comic or graphic novel. the works above are important for the ideas they raise, but also how well these ideas are communicated through the comic form. the ec comics depict aliens and spaceships and the joker comics depict a whimsical, evil and colorful joker for example, these are things which the comic form helps to showcase. similarly, the maniacal face of the resistance leader in the serenity comic, the dilapidated cityscape in no man's land and the burning cross in the ec comic are other powerful depictions. again, the point is not simply to examine the political messages and ideas of the comics, but how those ideas are communicated through the comic's medium. in grant morrison's and dave mckean's (1989/2014 ) arkham asylum, serious house on a serious earth, a psychiatrist asserts that the joker's psychotic behavior might be an adaptation to life at the close of the twentieth century. while i do not think we need to be like the joker, i do think we as teachers need to reckon with this volatile age we find ourselves in. and i don't just mean the technology (which is of course an important component). hyperreality challenges our very humanity. it can be a boon or curse, but that depends on how we as society react. what might a failure to adequately teach citizenship in hyperreality look like? i don't know. it may be a worsening of the things we are currently living through, such as citizen apathy and confusion, an inability to articulate at least some form of agreed upon norms, hyperpolarization, and extreme gridlock to name a few things. what would a society look like that has completely dispensed with any type of truth or agreed upon norms? how would a democracy function without truth? are we living in such a democracy? do societies come back from something like this? and how will the coronavirus impact citizenship and education? i don't know the answer to those questions, but i do know we as educators need to do something to stem this prospect and fight back. as noted earlier, i do not claim that this book will make students better citizens. more modestly, i simply wanted to start a conversation on how best to adapt citizenship education to hyperreality and on what truth actually means and can mean. i encourage readers to contact me. if you used ideas from this book, i want to hear your experiences both good and bad. i want to hear your ideas about how you might improve them, or what other assignments you have in mind. we can only move forward, for better or worse. figure 6. 1: this is an example of a student created comic for the vor activity. in this comic, the student reflects on her experiences as a lesbian women growing up in the conservative south, and what impelled her to move to a more welcoming area. figure 6 .2: this is an example of a student created comic for the sequential law assignment. in this comic the student grappled with what types of speech are allowed in schools and which types are not. figure 6 .3: this is an example of a student created comic for the ethical spectacle activity. on this page, the student masterfully used the comics medium to portray some ideas that we talked about tin class. she used the panels to convey a sense of social control and a lack of panels on the opposite page to convey a sense of liberation. figure 6 .4: this is an example of a student created comic for the ethical spectacle activity. here, the student used more abstract symbolism to probe whether we as a society have really progressed in regards to discrimination and racism. teaching strategies for the social studies: decision-making and citizen action arts based research scarlett: book 1 pre-service teachers meet and grapple over graphic novels in the classroom a practice guide to middle and secondary social studies graphic global conflict: graphic novels in the high school social studies classroom the power of comics: history, form and culture dream or nightmare: reimagining politics in an age of fantasy why artistic activism: nine reasons. the center for artistic activism social studies and the arts batman detective comics: crisis of faith batman: it's joker time the weimar republic redux: multiperspectival history in jason lutes' berlin city of stones getting beyond the facts: teaching social studies/social sciences in the twenty-first century batman, shadow of the bat: home sweet home instructional strategies for middle and secondary social studies: methods, assessment and classroom management making assessment work for teaching and learning graphic novels: the good, the bad, and the ugly teaching visual literacy: using comic books, graphic novels, anime, cartoons, and more to develop comprehension and thinking skills action research: improving schools and empowering educators approaching literacy features through the graphic novel logicomix batman arkham asylum: a serious house on serious earth (25th anniversary ed.) teaching graphic novels in the classroom: building literacy and comprehension comics journalism: drawing on words to picture the past in safe area gorazde the dc comics guide to creating comics: inside the art of visual storytelling you say ee-ther and i say eyether: clarifying assessment and evaluation history: discovering the story of jerry siegel and joe shuster manga, the atomic bomb and the challenges of teaching historical atrocity: keiji nakazawa's barefoot gen graphic novels and teacher research in the knowledge society. the educational forum making sense of beliefs and values: theory research and practice reading comics: how graphic novels work and what they mean shock suspense stories the inferiors. weird science-fantasy comic book nation: the transformation of youth culture on america key: cord-026868-z9gozm2w authors: mohammad, abeer title: “she’s dead!” – nursing simulation practices: a discourse analysis approach date: 2020-06-11 journal: j public health res doi: 10.4081/jphr.2020.1784 sha: doc_id: 26868 cord_uid: z9gozm2w background: the literature on nursing education has revealed a growing wave of interest in the use of simulation sessions to promote undergraduate nurses’ learning experiences. this high prevalence of simulation practices in nursing programs has led to opportunities to research this topic from various angles, including its impact on students’ skill performance, self-efficacy, self-confidence, self-satisfaction, and clinical knowledge acquisition. design and methods: participants in this qualitative study included 54 senior female undergraduates enrolled in a critical care nursing course in saudi arabia. recordings were made of six authentic, acute care simulation sessions. one of these sessions was examined in depth using discourse analysis approaches to gain insights into communication in simulation sessions, examining the way students linguistically managed this critical communication, exhibiting their logical, reflective, decision-making, problem-solving, and collaborative work skills and use of communicative strategies. results: the analyses revealed various training and communication issues including the lack of harmony among the team members (e.g., regarding understanding and performing their assigned roles as well as delegating and conducting delegated tasks) and the students’ inability to effectively communicate with the patient as a valuable source of information and to make appropriate and timely clinical decisions regarding patient assessment. conclusions: simulation sessions have been shown to be a promising instructional tool to support nursing education, allowing students to practice in a safe and controlled environment. however, for more effective sessions and to avoid poor simulation sessions, students need to be thoroughly briefed regarding the sessions prior to implementation. recent research on nursing education has revealed a massive wave of interest in the use of simulation practices to promote undergraduate nurses' learning experiences. [1] [2] [3] simulation practices, in this context, refer to a teaching technique that creates a rich teaching environment that mimics real hospital scenarios. 2 today, nearly every nursing education program involves extensive use of this effective interactive teaching tool. 2, 4 according to the literature in the field, simulation practices allow health students to connect theories that they learn in classroom settings to what they will potentially encounter in clinical environments, and these sessions can allow them to freely practice and receive feedback in various clinical scenarios. 2, 5 this clinical training mostly happens within a supervised context, which not only promotes students' learning but also provides a space for making errors that do not jeopardize real patient safety. 2 this widespread use of simulations in nursing programs has led to opportunities to research this topic from many angles, including its impact on students' skill performance, 6, 7 self-efficacy, [8] [9] [10] self-confidence and satisfaction, [11] [12] [13] and, most importantly, clinical knowledge acquisition. 13 collectively, the major findings of this research indicate the positive impact of simulation practices regarding reinforcement of students' learning and professional skills (including providing valuable opportunities for students to spot areas where they need to improve) and improvement of students' self-esteem. based on an investigation involving a similar context in saudi arabia as that involved in the current study, omer surveyed saudi nursing students' satisfaction and self-confidence after clinical simulation sessions and reported students' overall satisfaction with simulation-based training as well as the simulations' positive impact on their acquisition and retention of clinical knowledge. 3 most students reported having confidence in their clinical skills and they demonstrated that they were able to recognize signs and symptoms of diseases and perform appropriate clinical practices to assess patients' needs. 3 clear evidence in the literature strongly supports the use of simulations, and there is now a need for further studies on the best ways to improve these simulations. however, it is worth noting that despite the abundance of research on simulation practices, few actual examples of authentic simulation interactions are available in the literature. examining authentic communication during simulation practices not only provides insights on how this type of communication unfolds, but also offers evidence regarding whether clinical knowledge acquired in classroom settings is smoothly transferable to simulation practices. additionally, analysis of authentic data can provide deep insights and opportunities to improve communication in simulation practices in order to improve actual patient care in hospital settings. to date, simulation practices have mostly been examined descriptively via questionnaires, 3,10 interviews, 14 and observations. 1 this study employed discourse analysis approaches [15] [16] [17] [18] to provide an in-depth analytic interpretation of an authentic simulation session to develop a more instructive understanding of this type of communication. furthermore, the study aimed to examine the way students linguistically managed this critical communication, exhibiting their logical, reflective, decision-making, problem-solving, and collaborative work skills. 2 additionally, the analyses shed light on the students' use of effective and/or less effective communicative strategies, 15 which could help to further improve the quality of simulation training in this context. effectiveness in this paper was operationalized as "the recommended best practices," meaning the extent to which students put in effort to obtain, understand, and negotiate patient-related information and to make appropriate clinical decisions (as expected in the ideal model training scenario). 15, 19 design and methods a convenience sample of 54 senior female undergraduate nursing students enrolled in a critical care nursing course participated in this study. the students were all saudis whose first language was arabic. the participants also included a faculty member who spoke english as her first language, and two simulation lab assistants who were from the philippines and spoke tagalog as their first language. english is the official means of communication among the participants during the simulation sessions. the collected data for this setting included audio recordings of six authentic, acute care simulation sessions along with observations and detailed field notes. there were nine students per simulation session, and the faculty member and the two lab assistants were present in all six sessions. ethical approval was obtained in april 2019 from king abdullah international medical research center (kaimrc) for collecting data in clinical simulation laboratories at a nursing college in saudi arabia. all participants were provided with informed consent forms, which thoroughly explained the purpose of the study as well as the confidential and anonymous nature of the collected data, and they all provided signed informed consent. in this section, a single extended simulation session, which was the most representative of the six recorded simulation sessions, is presented and discussed. during simulation sessions in this context, nursing students are expected to demonstrate their clinical knowledge in accordance with the cognitive skills acquired in classroom settings via lectures, video tutorials, and/or independent reading assignments. similar to a model simulation session, the students involved in this study were assigned to act as members in a fast-track team providing healthcare in an emergency room (er). for control and supervision purposes, the students were assigned in groups (nine students in each of the simulation sessions). following the model, the roles within the team comprise a primary nurse, who lead the team and delegate responsibilities and tasks, assisted by other students who play various roles including a doctor, a pharmacist, a social worker, a patient, a patient relative, and a recorder. the code team, according to the model, comprise of additional staff who assists with the code. in the following script, while the role of some students was identifiable including the role of the patient and the patient relative, there was little evidence of the roles that other students were taking. only nurse 1 was identified as the primary nurse since the faculty member explicitly delegated responsibilities to her at the beginning of the simulation session. thus, in the script, i refer to the rest of the participants as student 2 and student 3 due to the uncertainty of the roles they were playing. in the simulation scenario in this paper, the team must manage a patient who sustained a severe injury on her right thigh in an auto accident. the management of the patient was expected to involve performing various medical interventions during three major scenarios (1: conducting a primary survey; 2: managing sudden hemodynamic changes; and 3: managing cardiopulmonary arrest). the timeframe assigned for the three scenarios was approximately 25 min: 10, 5, and 10 min for scenarios 1, 2, and 3, respectively. it should be noted here that only the first two scenarios are discussed in this paper as none of the teams participating in the six recorded simulation sessions completed scenario 3 (largely due to the teams' failure to perform an expected procedure, i.e., to call a code). scenario 1 aimed to assess students' ability to prioritize the procedures needed to assess the patient, including performing the air, breathing, circulation, disability, and exposure (abcde) primary survey. extract 1 shows the first phase of the simulation session. like all of the other recorded simulation sessions, the simulation commenced with a set-up phase in which the faculty member began aside dialogue with the team, assigned roles, and provided an overview regarding the complete scenario. based on the faculty member's remarks in lines 6-12, it can be noted that she reminded the students of certain critical information concerning the patient. this information set-up the scenario, revealing the primary medical diagnosis of the patient, i.e., trauma in the form of a severe laceration to the right leg. this introductory information, if well responded to by the team, should positively contribute to the commencement and completion of the expected interventions in scenario 1. declaring the beginning of the scenario, with a rising intonation (line 12), the faculty member concluded the set-up phase by delegating responsibility to the primary nurse. following the set-up phase, students were expected to commence the simulation scenario and work as a team to perform the required medical interventions. the scenario began with a dramatic entrance by the student who was taking on the role of a severely injured patient laying on a stretcher, escorted by the two lab assistants. ideally, the team is required to act immediately, receiving the patient and beginning the case management process. as shown in extract 2, the patient was conscious, breathing, and able to inform the team about the location of her injury (lines [13] [14] . however, this valuable information seemed to be overlooked by the team, meaning that the primary nurse could not build on the introductory information by obtaining more information from the patient (e.g., by double-checking the patient's information, the type of incident that happened, the location of the injury, the severity of pain, the level of consciousness, etc.). in line 15, the primary nurse requested an assessment of the patient's vital signs. however, it was unclear whether this request was a delegation of the task to the rest of the team or a declaration that the primary nurse would perform the task herself. accordingly, this request went unheeded, most probably because it was not perceived as a request by the rest of the team. the request lacked the required tone (the student sounded like she was recalling information rather than requesting an action to be conducted), which led to the first ineffective task delegation among the team. the primary nurse repeated her request once again, but this time implicitly with a rising intonation, implying a request, /hurry↑/ (line 20). although the primary nurse attempted to use different request strategies, mostly relying on intonation, both requests seemed to lack not only the necessary paralinguistic features but also the basic syntactic features that are needed for effective speech act performance i.e., a request. 20 due to the passive behavior of the rest of the team, the patient intervened (line 21) by providing the students with a cue, which most probably was intended to direct the team's attention to the wound, so they could begin the exposure assessment in the abcde primary survey. following the model, the team were required to conduct wound management (i.e., undressing and assessing the injury). however, due to there being no immediate uptake to the cue, the patient screamed /please hurry↑/ (line 23), which reflected the slow interactions between the team rather than the expected dynamic, fast-paced environment in the er. 21 the details of scenario 1 (conducting a primary survey) are continued in extract 3, with the primary nurse attempting to measure the patient's vital signs (line 24), as nobody had started this task. next, student 2, who failed to disclose her role in the team, commented on the patient's behavior (line 25), asking the patient to quit her "drama." although the student who took on the role of the patient distinguished herself, creating a realistic er environment and seeming to be highly engaged in her role in the simulation session, most of the other students, unfortunately, could not react as authentically to the interactions initiated by the patient. this could be because the other students could not overlook the fact that they were acting in a non-real situation. the comment made by student 2 about "drama" (line 25) may also be a reflec-tion of the lack of seriousness among the team during the simulation practice. in response to this comment, the patient wittily redirected the communication to the scenario by declaring that she was in pain (line 27). this interactional move guaranteed the continuity of the session. the primary nurse responded with a lower tone /okay↓/ (line 28), and then she stated that she needed to assess the vital signs, a procedure that had been started at the beginning of the scenario. the patient, again, tried to redirect the nurses' attention to the wound in her leg (line 29), this time with explicit cues, i.e., directing the team's attention to her physical problem (the wound) and its location (the right leg). however, once again, the cues were overlooked by the team, and the patient's further contributions were silenced with student 2's high-pitched comment, /dagee-gah/, i.e., wait a minute (line 30). in another desperate attempt (line 31), the patient intervened, asking the team to administer pain medication. with this interaction, the patient redirected the team to the circulation assessment in the abcde primary survey, meaning that iv insertion should be attempted to allow potential administration of medicine. again, this attempt was halted by the primary nurse's claim that the doctor was on the way (line 33). it should be noted that, one of the students should be playing the role of the physician within the team; thus, the primary nurse's comment may indicate her lack of knowledge of who was taking this role. this sequence of interactions showed that the students seemed slightly impatient, and their communicative style silenced the patient instead of encouraging her to share information in order to allow them to undertake an effective clinical assessment. according to the us joint commission, effective communication and teamwork collaboration are essential to guarantee quality patient care. 22 hence, regarding the transcript presented in the extracts, various untoward events should have been addressed with the students to teach them to avoid preventable medical errors. firstly, as demonstrated above, there is an indication of a lack of harmony among the team members, particularly regarding understanding and performing their assigned roles as well as delegating and conducting delegated tasks. secondly, the nurses failed to effectively communicate with the patient, offer her space to tell her story, and build a shared medical knowledge about the case, which is especially important as patients attending an er usually have no written medical histories with them and there are usually no other patient records accessible in the hospital. 18 students need to understand that the patient is a valuable source of vital information that can facilitate the whole healthcare process. furthermore, it should be noted that, in all six recorded simulation sessions, there was a noticeable lack of use of interactional features (e.g., backchannels, discourse markers, hesitation markers, and yes/no and tag questions), which are considered to be effective tools in spoken discourse as well as being important in effective medical discourse. 18, 23, 24 students need to be trained regarding how to effec-tively elicit patient information as they simultaneously perform the physical assessment (e.g., by using yes/no and tag questions) instead of completely ignoring or silencing the patient. lastly, as for skill performance, the nurses at this point failed to demonstrate their ability to effectively assess the patient's vital signs, perform iv insertion, and check and manage the patient's wound appropriately. as shown in extract 4, student 3, whose role in the team is unidentified, requested that the physician be called and physically waved for the code team to join in (line 35). it should be noted here that, according to the model scenario progression, the role of the code team should have begun only in scenario 3 (managing cardiopulmonary arrest). however, in line 37, we again see student 3 explicitly asking the primary nurse to call for the physician, as she seemed to believe that the physician was part of the code team. although student 3 produced a clear and effective request, she provided no clinical justification for this request, and the patient was still conscious. knowing that requesting the code team to join in is the job of the primary nurse, or whomever she delegates the responsibility to, a member of the code team asked that this request be issued by the primary nurse (line 36). student 3 directed another explicit request to the primary nurse to call the physician (line 37). as can be seen in extract 4, these interactions between the two teams were ceased by the faculty member (line 42). the faculty member's interjection at this point shifted the interaction back to the first procedure, i.e., measurement of the vital signs by the nurses (line 44). thereafter, student 2 finally reported the readings of the vital signs and she indicated that the patient was hypertensive (line 45); however, it is unclear how she came to this clinical conclusion. as shown in extract 5, student 3 re-joined the interaction and seemed to recall the management procedures. in line 46, she interjected with the discourse marker "okay," followed by "first↑" with a rising intonation. this made it seem like she was taking the interaction to the very beginning; thus, she was interrupted by the patient, who asked her to hurry (line 47). next, the nurse successfully recalled the first step in wound management procedure, i.e., controlling the bleeding (line 48). the use of the first-person plural pronoun /we/ (lines 48 and 49) may indicate that the nurse was requesting the team's assistance and that they all should work on this collaboratively. though student 3 started the procedure, she then abruptly delegated the task to the primary nurse (line 50). after a short chat between the nurses, the primary nurse seemed to be about to perform the procedure; however, she just re-covered the wound. shortly after this, the faculty member signaled the end of scenario 1 to the patient, who was simulating being unconscious. extracts 4 and 5 illustrate other untoward events, including the contentious confusion about the assigned roles, task delegation, and responsibilities among the team. again, regarding skill performance, the students failed to demonstrate effective management of the patient's wound or reporting of the patient's vital signs with accurate language and clear justification, which indicate the lack of adequate preparation for this practice. in addition, the nurses failed to notice that the once active patient had stopped speaking after line 52; accordingly, they missed identifying the deterioration in the patient's status and therefore missed the opportunity to act appropriately and in a timely manner to save the patient. with the patient unconscious, scenario 2 began. ideally, students are required to identify and manage sudden hemodynamic changes (i.e., a sudden decrease in heart rate, blood pressure, and response level) in the patient. as can be seen in lines 54-55, with the use of the first-person plural pronoun /we/, student 2, suddenly, decided that the patient needed an allergy medication; accordingly, she decided to call the pharmacy. although the rationale behind her decision is vague, it seems that some clarification of this decision was discussed between the nurses. thus, in line 56, the primary nurse implicitly confirmed this decision by deciding to make the call herself. following a brief interjection from the patient's relative, which should have alerted the nurses to the patient's deteriorating status, the primary nurse declared to the team that she would call the pharmacy (line 58). this decision led to an immediate interjection from the faculty member (lines 59-61). with a high pitch, the faculty member stated that the nurses cannot call anyone because they were in the er and, accordingly, the full team (including the doctor and pharmacist) was there. this remark indicates that the physician is among the team surrounding the primary nurse and not with the code team, meaning that one of the students is not performing her assigned role. although the faculty member's interjection served as a reminder to the students of their roles in the team, it indirectly hindered the likelihood of the team attempting to call a code. this was the action that the faculty member was waiting for, and five out of the six recorded simulation sessions had unsuccessful conclusions because the teams failed to complete this action. the faculty member then addressed the code team, which was physically present in the same simulation room, asking them not to interact with the rest of the team as, ideally, they should not be physically present in the er area. all of the nursing students burst out laughing in reaction to the faculty member's comment. the faculty member's interjection at this moment, again, regressed the whole interaction back to an earlier stage, as evidenced by student 2's decision to proceed with cannula insertion (line 68). the interactions in extract 6 ended with a lengthy pause. while it is acknowledged that simulation sessions are learner centered and require minimal faculty member interjections during the practice, a faculty member's teaching expertise and clinical knowledge can highly impact the effectiveness of the whole practice. 2 thus, if the faculty member had paused after various incidents and discussed them, this would have been a valuable learning opportunity for the students to notice and discuss their critical clinical and communication errors. for instance, the students' decision to administer allergy medication in extract 6 clearly reflects inappropriate critical thinking and/or clinical decision-making ability. if there was a pause after this incident, the students would have gotten a chance to qualify the rationale behind their decision and decide whether the medication really fit with the patient's needs or situation. according to the institute of medicine (iom) report, hospitalized patients often experience medication errors, which occur during the prescribing and/or administration phases in hospital. 25 pausing at these critical points in simulation sessions would offer opportunities for the students to develop their clinical reasoning and validate their knowledge with an expert in the room, i.e., the faculty member. in addition, as observed in extracts 4 and 6 and in other recorded simulation sessions, the interjections of the faculty member always caused a deterioration in the interactions, with the participants reverting back to earlier in the scenario. although the reason behind this deterioration is undetermined, based on the transcripts, the interjections could be modified to better impact the whole interaction. if the faculty member systemized her interjections at critical points, e.g., to clear up confusion, clarify misconceptions, provide suggestions, direct the students' attention to specific training issues, and/or instruct the students on how to perform certain actions (e.g., using the phone to call a code), her interjections would facilitate rather than hinder the practice process. the advantages of such facilitation include ensuring the continuity of the simulation sessions (as most of the six recorded simulation sessions did not make it to scenario 3) and increasing self-confidence (as students had the opportunity to learn via this instructional strategy for the first time, while being observed, and were allowed to play various roles other than the role of a nurse). interestingly, most of the six recorded simulation sessions ended in the first 5-10 min of the simulation scenario. as can be seen in extract 7, the student who took the role of the patient's relative attempted to initiate communication with the patient, who had remained unconscious for several minutes. in response to this attempt, in the next turn, student 2 ended the whole session, declaring the death of the patient "/khalas/ [i.e., that's it]. she's dead" (line 70). among all six recorded simulation sessions, this was the only session that was ended by a student. in the remainder of the sessions, the faculty member declared the death of the patient. this death announcement triggered an abnormal reaction, i.e., group laughter. regardless of this declaration, the patient's relative redirected the interactions back to the scenario (line 71). this prompted the primary nurse to continue with the simulation scenario (i.e., the cannula insertion). in response, student 3 interjected, asking the team if they should clean the wound up. with no immediate uptake regarding either the comment about cannula insertion or the one about cleaning the wound, the interactions stopped, with a long pause. this silence was interrupted by student 2, who re-announced the death of the patient, this time twice (line 74). again, this made the whole team burst out laughing. the primary nurse stated that she wanted to check the patient's pulse (line 75), rather than performing the action itself. the faculty member then interjected and ended the simulation session by declaring the death of the patient: "the patient dies <pause:1.0> because you <pause:2.0> oh my god↓" (line 76). as can be seen, the faculty member seemed to start to offer an evaluation regarding the patient's cause of death ("because you") but she did not complete this utterance, closing with the interjection "oh my god↓," which may demonstrate disbelief, shock, or even worry because the simulation session deteriorated to this degree. 24 the final extract ended, as all of the other five recorded simulation sessions, with the simulated death of the patient. the lack of effective teamwork and appropriate and timely clinical decisionmaking, along with the teams' inability to perform the expected procedure (i.e., to call a code), contributed to the termination of many of the simulation sessions, including this one. however, educational value can be gained from seemingly poor endings. one of the major advantages of simulation practices is that errors are allowed because there is no potential risk to a real patient. 2 thus, thorough debriefing following simulation sessions might help trainees to analyze their performance and decide how to improve it in future in-hospital situations. this study aimed to contribute to nursing simulation research by highlighting some of the favorable as well as untoward practices that appear as simulation practices unfold. examining authentic data shows that effective communication and collaborative teamwork between the team members are essential for the success of simulation practices. based on the analyses, faculty members participating in nurse training are highly encouraged to help students to develop and practice strategies to improve team communication, thereby improving the quality of simulation practices. for example, crew resource management (crm) interventions, which have been adopted by many healthcare organizations, can be used to enhance team training in this context. nursing students need to learn how to work in harmony during simulation practices, be active participants, identify critical changes in the patient's status, and rapidly and appropriately intervene to save the patient and limit any possible adverse events. 27 furthermore, it is important for students to comprehend that adequate preparation is vital and that the whole simulation experience depend on each student executing the role he/she is assigned to. 28, 29 the analyses also revealed the role of the faculty member during the simulation practices. even with in-class preparation, this type of experience is still new and can be overwhelming to students; thus, strong support from the faculty member is highly encouraged to comfort students and increase their self-confidence and motivation. it is recommended that faculty members take an active role and be vigilant during training sessions to identify and assess critical communication and/or practice issues. faculty members can take advantage of various opportunities to improve simulation sessions, allowing students to immediately reflect on their critical mistakes, think critically, and evaluate their clinical decisions, especially as most of the fine-grained, critical details might get overlooked during debriefing following simulation sessions. however, faculty members' interjections need to be wellplanned to avoid communication deterioration as shown in this data. although all of the recorded simulation sessions ended with the simulated death of the patient, they still possess an educational value that can be shared and discussed with future students either via debriefings or in classroom settings, especially that students are rarely exposed to such authentic simulation scripts prior to practice. thus, the authentic transcribed sessions in this study can be used to supplement course materials and be used prior to similar simulation sessions to improve students' understanding of this type of instructional procedure and optimize simulation practice outcomes. 21 lastly, as shown in the extracts, although both the faculty member and lab assistants were english language speakers, the students used arabic to communicate, which may jeopardize the effectiveness of the simulation interactions and limit the faculty member's ability to address communication and clinical errors during debriefing sessions. thus, further research needs to explore the impact of this interactional feature on the overall comprehensibility of interactions; meanwhile, it is recommended that students should be advised to speak in english and limit the use of codeswitching, or use it with caution. 19 simulation sessions have been shown to be a promising instructional tool to support nursing education, allowing students to practice in a safe and controlled environment. however, to improve the effectiveness, and to avoid poor simulation sessions, students need to be thoroughly briefed regarding the sessions prior to implementation. an evaluation of advanced simulation in nursing: a mixed-method study enhancing patient safety in nursing education through patient simulation. in: patient safety and quality: an evidence-based handbook for nurses. us agency for healthcare research and quality nursing students' perceptions of satisfaction and selfconfidence with clinical simulation experience evaluation of simulation in undergraduate nurse education: an integrative review new graduate nurses' perceptions of the effects of clinical simulation on their critical thinking, learning, and confidence a computer-based trauma simulator for teaching trauma management skills undergraduate nursing students' acquisition and retention of cpr knowledge and skills effectiveness of simulation-based learning on student nurses' self-efficacy and performance while learning fundamental nursing skills a nurse residency program for competency development using human patient simulation the impact of simulation on pediatric nursing students' knowledge, self-efficacy, satisfaction, and confidence associate degree nursing students' perceptions of learning using a high-fidelity human patient simulator high-fidelity patient simulation and nursing students' self-efficacy: a review of the literature determining the value of simulation in nurse education: study design and initial results learning endotracheal intubation in a clinical skills learning center: a quantitative study clinical handover as an interactive event: informational and interactional communication strategies in effective shift-change handovers continuum companion to discourse analysis. london; bloomsbury publishing chapter 27: discourse and intercultural communication communicating in hospital emergency departments a discourse analysis of nursing handoffs: exploring nurse-to-nurse interactions in two hospitals in saudi arabia strategies for learning and performing l2 speech acts communicating in hospital emergency departments the discourse of nurse-patient interactions: contrasting the communicative styles of us and international nurses the discourse of online consumer reviews to err is human: building a safer health system frame analysis: an essay on the organization of experience crew resource management training in healthcare: a systematic review of intervention design, training conditions and evaluation high-fidelity patient simulation in nursing education simulation scenarios for nursing educators: making it real the author extends her thanks and appreciation to all the participants (the students, faculty member, and lab assistants) who generously participated in this study. this paper was written during the covid-19 epidemic to support and recognize the work of healthcare providers and to highlight the need to improve health-related practices.ethical approval: ethical approval was obtained in april 2019 from king abdullah international medical research center (kaimrc) for collecting data in clinical simulation laboratories at a nursing college in saudi arabia. key: cord-024853-07n3wroj authors: devinney, timothy; dowling, grahame title: what are the strategies of australia’s universities? arenas, vehicles, differentiators, staging and economic logic date: 2020-05-16 journal: the strategies of australia&#x02019;s universities doi: 10.1007/978-981-15-3397-6_9 sha: doc_id: 24853 cord_uid: 07n3wroj because australia’s universities have multiple missions, they also have a broad scope of operations. notwithstanding this, there are a set of institutional mechanisms that ensure a large degree of sameness across the academic footprint of the universities. they lack fundamental differentiation. also, the cost structure of the universities requires some faculties (cash cows) to generate free cash flow to fund the research operations of other faculties. because university research is expensive, universities are trying to convince industry to fund more of this activity. however, universities have many obstacles to overcome in this endeavour. what are the strategies of australia's universities? arenas, vehicles, differentiators, staging and economic logic 9.1 arenas figure 7 .1 earlier suggests that each of australia's universities is probably two enterprises-one for education and the other for research. each university differs somewhat according to the range of arenas (faculties) over which it practises these activities. for most universities their mix of faculties seems to be the result of circumstance as much as a grand design. in contrast, in the post-war years when stanford university started its rise to stardom, the decision was made to concentrate on postgraduate programmes (where national reputations were forged), and a relatively few areas where government funding signalled national importance-one of which was electronics research. the aim was to do a few things with distinction rather than 'creating excellent programs in fields no one cares about'. 1 a strategic decision that is very costly for a university is its choice of arenas. as just noted, stanford chose some of these because the government of the day was prepared to underwrite much of the cost. if a university chooses to do world-class teaching and research in the natural and physical sciences, it can involve significant investments in infrastructure. in contrast, research in many fields of the humanities is far less costly. the number, variety and size of arenas define the scope and academic footprint of a university. from a strategic perspective, when the logic guiding the choice of arenas is not clear it • makes positioning the institution more complicated, • increases the costs of administration (due to complexity) and • complicates decisions about which areas to prioritise for funding. a criticism of australia's public universities is that their scope is too broad. to illustrate this point, consider the profiles in fig. 9 .1. each circle represents an academic faculty scaled to size. their relationship is depicted by their distance from each other. an issue we will raise in chap. 11 is that each of australia's universities looks more like comprehensive university, one with a scattered academic footprint, than focused university where the logic of its collection of faculties is more apparent. and if the numerous research centres and institutes that a comprehensive university has were added to fig. 9 .1, the picture would look much more complex. the major vehicles used to codify education, such as the common range of degrees; deliver education, such as teaching philosophy, curriculum design, lectures, tutorials and web-based materials; assess learning, such as assignments and examinations; and do research, such as academic time and research equipment, are similar across all of australia's universities. thus from both a student and an academic perspective, 'going to university' means that they are joining a fairly well-defined type of institution and committing to a broadly similar experience. yes, each university has some unique aspects, but universities have a high degree of 'sameness'. the vietnamese have a saying that captures this situation: history and what is called isomorphism (imitation) create this sameness. this is the way that universities were organised in the past (history), and this is the way that a university's peers are currently organised, and so each tends to conform (isomorphism). also, the rankings are designed to reflect this structure. so to look like a university, this is the dominant design. the value of this structuring of operations is that it clearly positions all universities as 'a university'. stakeholders, both internal and external, are comfortable with this model. consultants as we noted earlier suggest that it should be shattered. it is interesting to delve deeper into the issue of sameness for two reasons. one is to expose the weakness of some consulting suggestions for change. at the present time many are simply infeasible. the other is to uncover the structural mechanisms that will impede any significant attempts to change. four institutional mechanisms cement the essential sameness of australia's universities. the first is the overwhelming reliance on government funding. it is anchored to the grand bargain mentioned earlier, which comes with strings attached, namely, to be public institutions committed to the three traditional missions of teaching, research and community engagement. no university has the courage to publicly jettison any of these missions. the second factor creating sameness is three powerful agencies. one is the tertiary education quality and standards agency (teqsa). stemming from the bradley review of australian higher education, this government regulator now imposes quality assurance mechanisms to set and review sector standards in the areas of teaching, learning, scholarship, research and research training. the second agency is the australian research council's excellence in research australia (era) programme. as noted earlier, this initiative produces an australian university league table that encourages a degree of sameness of research across the universities by recognising discipline-based journal ranking lists which identify the set of de facto target journals for much university research. these are then used in the individual performance appraisal of staff. thus, the requirements of these journals create the parameters for what is judged to be the type of research one should do and what quality means. the third agency is the national tertiary education union. this union negotiates the basic workplace agreements employed throughout the university system. while each university negotiates a different agreement with this powerful workforce union, they are very similar in structure and content. we would argue that these union agreements are designed more to protect the 'average' employee than to foster excellence across the system. one of their key conditions illustrates this opinion, namely, the duration of the paid working week. currently this is typically 37.5 hours per week. for an academic 37.5 hours is far too short! it is impossible to be an excellent teacher and world-class researcher and then do some university service, and then some industry and community engagement by working 7.5 hours per day. so here is the problem: • work 37.5 hours per week and do part of the job, or • work 37.5 hours per week and do all the job at average quality or • the university is deliberately underpaying its academic staff (and it should be taken to the fair work commission). when one of us first joined the australian university system, our terms and conditions of employment would easily fit a 37.5 hours per week. the employment contract was one page in length. it had three main paragraphs: 1. welcome to the university. 2. your level is xyz and your pay is $pqr pa. 3. if you have any questions please talk to your head of school. in essence, the deal was 'you pretend to work and we will pretend to pay you'. there were no formal university expectations (regular performance appraisals), and some academic staff treated their employment as a form of paid social welfare. and yes, mediocrity was prevalent. in contrast, the last union-negotiated employment contract for one of us was 69 pages long. it was dense with terms and conditions of employment. the third factor cementing sameness is the rise of npm / professional management in universities. much of this is directed at responding to the requirements of the teqsa, the era and the legislation under which each university receives its charter. also, there now seems to be a need to manage an ever-growing number of risks within the university environment, such as oh&s, staff welfare, student welfare, student malpractice, research malpractice, cyber risk, data risk, ethical risk, operational risk, financial risk, regulatory risk, crisis risk, climate risk, reputation risk. professional risk consultants have targeted universities to help them assess their vulnerability, design programmes to manage risk and recover from risk failures. a rich and successful history plus these institutional mechanisms have combined to produce a higher-education sector with few meaningful differences. in commercial situations where there is an inherent sameness across members of a strategic group, such as the big 4 australian banks, one often observes artificial attempts to differentiate one institution from another. corporate identity, branding and communication are often used for this purpose. the websites of the universities are revealing here. they are marketing devices. the first screen looks more like the package of a supermarket product than the front door to an organisation with a deep purpose. and from here they explode to reveal all the complexity of the institution's operations. at our last university there was a group of 28 administrators tasked with managing the institution's marketing and communications, and a further 36 responsible for brand strategy and engagement. as old marketers, we think that 64 people are not necessary to tell students about the university, especially when research suggests that many students get most of their information via wom-word-ofmouth and word-of-mouse. the strategic narratives of most universities suggest that they are constantly making changes to their operations to exploit opportunities and overcome obvious problems. explore the website of any university and it is easy to find examples of new programmes being launched and new people being appointed to run such initiatives. there is no shortage of activity. for example, in the area of education each university seems to have adopted a different approach to learning. their websites reveal learning philosophies like blended learning, block learning, flipped learning, group-based learning, case-study learning, student-led learning, workbased learning, problem-based learning, active learning, active and inquiry-based learning, flipped classrooms, design thinking, moocs and so forth. long gone it seems is the era of 'chalk and talk'. in the area of research while most activity still occurs within faculties, these endeavours have been supplemented with cooperative research centres, university and industry partnerships, and cross or multi-disciplinary research centres. the easiest way to explore new areas of interest seems to be to bolt-on a new research centre. the university of western australia's annual report notes it has 45 of these. from an npm perspective these are an interesting idea because they allow the university's administrators to measure efficiency and performance. it is clear what they cost to run, what independent money they attract and what they produce. and because of their insular nature, they can be closed down if performance does not meet expectations. another staging mechanism is the branch office. some are local, such as a regional university with an office in the near-by capital city, and some are located in a foreign country, such as dubai, hong kong, singapore, thailand, vietnam. many overseas universities have also followed this approach. all sorts of reasons are provided for why such endeavours are a good idea. however, many are not strategically sound. for example, some years ago both of us attended a meeting where the senior administration of the university suggested that a branch office of our school be opened in hong kong. the idea originated with an industry advisory group-'asia is booming, there are lots of students, why aren't you up there!' at the end of the meeting one old strategy professor (who was an exmanagement consultant) summed up the idea as follows: as a small institution with limited financial resources let's be a late entrant into a highly competitive market where it is not clear about the regulatory requirements for accreditation; where some local universities are very good, and where recent overseas entrants like us are not making any money. and, to succeed we will have to redeploy from our current programs some of our best resources in order to establish our quality. this comment was all it took to bring sanity back to the discussion. it was also a good example of the value of harsh academic scrutiny of the 'free advice' from people without deep knowledge of higher education and no skin in the game-the industry advisors were not willing to bankroll the venture. on a number of occasions we have been critical of the lack of a clear line-of-sight between key aspects of strategy and the financial management of the university. while we were at the university of new south wales, one of the vice chancellors (fred hilmer) made the effort to inform the senior academics about the revenues and costs of the university. at our last university this information appears in the annual business plan. also, each university publishes a set of accounts in its annual report. given that the annual report is a public document, we will use it to raise some important strategic issues in the next section. chapter 1 noted that australia's universities are happy to argue that they are under-funded. so a key question is whether this is true? a second question is whether our universities waste some of the funds they receive? answers to both questions help to expose the business model and the profit and loss statement of a university. the university of south australia is one of the few universities to discuss some of these factors. for example, by 2025 their plan is to: • grow the student body to 40,000, with 25% of these learning online. unisa online is the core asset that will support this (ad)venture. • generate 15% of their revenue from research activities, 60% of which will come from industry partnerships and 40% from research grants. now how will they grow student numbers and change the mix of research funding? presumably, the student body will grow via the introduction of online courses. hopefully it will not grow by lowering entry standards. so the first challenge is to make sure that these online courses do not cannibalise the revenue from their current courses. the second challenge is to do this without reducing quality. the third challenge is that they will have to compete against the online programmes of other universities in australia and overseas. on the research front the idea is-'by growing the scale and focus of research by building capacity in areas that have demonstrated excellence and shown potential for growth'. with a large budget this might be feasible, but do they have these funds? they don't say. on the teaching front what puts grand plans like this at risk is price discounting. for example, there is an emerging trend for universities to offer substantial tuition fee discounts, especially to attract international students. phil honeywood, president of the international education association of australia, said that many of these discounts were dressed up as scholarships. 2 so, there is a downward pressure on fees-revenue. competitors are just as desperate as unisa to grow their numbers. on the research front unisa is exposed by its lack of reputation. a quick look at the era rankings doesn't put the university in the top echelon, and as noted elsewhere, many australian academics and their universities do not have a very good reputation for their commercial acumen. so while universities would like to generate more money from industry, many research academics simply don't care what industry wants. getting favourable recognition from peers is far more important than getting research money from industrialists that is targeted to specific projects, and some of which the university may tax. because the economic logic of a university can be partly deciphered from its annual accounts, this is where we turn to next. at the 2018 australian financial review higher education summit, anthony walker, a director of sovereign and international finance ratings at s&p global, said that there is a game being played by the government and the universities. because of budget pressures, the government wants to reduce its funding of universities while they want more funding, so the truth about what is an appropriate level of funding is likely to be somewhere in between. his analysis suggests that many universities have operating margins of between 8 and 10% and quite healthy cash flows. 3 yes, some universities might be in less than good financial health, but there is a moral hazard here-the government is highly unlikely to let one of them become insolvent. so, maybe the overall financial health of the universities is not as woeful as often claimed. at this summit philip clark, a former federal government advisor on education, makes another point about the funding of the universities, namely that relying on the high numbers of overseas students, many of whom come from china, is a risky strategy. there are two financial risks here. one is that if overseas student revenue is used to 'balance the books', a downturn in demand puts the university's short-term revenue at risk. this was dramatically illustrated during the covid-19 pandemic. the other is that if this revenue source is used to fund long-term investments in infrastructure or research, a downturn here puts these at risk. so, even though it is a luxury good in china to have an education from a groupof-eight university, if the chinese government or other circumstances mean that these students stop coming to australia, this revenue steam will stop very quickly. also, an adverse change in the exchange rate or more restrictive student visa requirements quickly reduces the flow of international students. strategists like to look at what they call the revenue model of a business enterprise. in the university context, the focus is on the principal sources of revenue. to illustrate this, we will look at the 2017 annual report of the university of western australia (uwa). here we see that uwa had $977million income for an underlying result (sustainable surplus) of $18 million. approximately 54% of this revenue was sourced from the federal and western australian state governments. fees and charges were $195 million (20%), other income was $100 million (10%), research income was approximately $37 million (4%) and interest was $39 million (4%). in summary, uwa is a big enterprise that relies for about half of its income from government funding, and it generates about one-third of its income from student fees. hence, government relations, cost control and financial management are key skills. there is nothing surprising here. however, think about research. the reputation of the university is only strong enough to generate 4% of income from this source. this is the same amount generated by the investment committee of council. what these figures suggest is that the external demand for uwa research seems to be very small. a little further digging through the annual report reveals another interesting observation. the university had 24,327 students of whom 14,791 were fee-paying undergraduates, and 7,587 were fee-paying coursework postgraduates. these students are serviced by five faculties (one of which is very small). one of the super-faculties comprised the cheap-to-run faculties, namely, arts, business, law and education. this group educates 51% of the university's undergraduates, 37% of coursework postgraduates and 23% of research postgraduates. this pattern suggests that this faculty is the cash cow for the rest of the university. it is like a giant atm from which other parts of the university withdraw their weekly allowance. as a member of this super-faculty, you could feel proud-your efforts fund much of the rest of the university, or unappreciated-your efforts are unrecognised by others. getting the economic mix of this core funding cross-subsidy game correct can make or break the finances of a university. it can also damage the reputation of the university if in a faculty like education, student entry standards are lowered to fill up the numbers. at the time of writing, this was a topic of conversation in the media. when compared to the great universities in the usa, the piece of the revenue model that is largely missing from australia's universities is its endowment. while there are many reasons that might account for the paucity of australian university endowments, the stark fact is that these institutions do not have access to a significant amount of this revenue. much of the money given to universities comes with constraints, such as to be used for a specific piece of infrastructure or type of research. and even sponsored academic positions, such as named chairs, are rare. as government funding becomes more constrained, the need for this type of income becomes greater. imagine the opportunities that could be pursued by an australian university with a couple of billion dollars in its war chest! the cross-subsidy game all australian universities have a group of low-cost faculties and degreesthe cash cows that are used to fund the high-cost degrees-the stars. the star disciplines are generally the ones that underpin the major university reputation ranking systems. they are also the disciplines that tend to make the major scientific breakthroughs that get publicity on the nightly news and in television natural science documentaries. the deans of these disciplines are generally the most powerful academic voices inside the university. at uwa these are all sciences-science, health and medical sciences, engineering and mathematical sciences. notwithstanding the economic significance of the cash cow faculty to uwa finances, in the annual report, the exploits of the sciences are given much more prominence than law, business and education. cross-subsidies lie at the heart of university growth. if an institution decides to grow by increasing the size of its high-cost faculties, it will generally have to generate more revenue from its less costly faculties. this can be done by enrolling more students, raising tuition fees, reducing the delivery costs of teaching and research or clawing back more of the current revenue from these faculties. however, if the institution grows only the less costly faculties, it has free cash flow to spend as it pleases. thus, the economic modelling of these institutions is a critical component of strategy. this is seldom mentioned outside the executive suite. cross-subsidies, however, come at a price to reputation and social equity. consider the following paradox-or as ross gittins, economics editor of the sydney morning herald, calls it 'short-changing their students'. 4 step 1 is to attract students to cheap-to-provide occupational degrees-that have good job prospects like commerce, or a good image but fewer job prospects like law. step 2 is to teach these undergraduates in large classes or substitute technology for face-to-face teaching. step 3 is to agree to charge them a premium price. 5 step 4 is to use the free cash flow to pay the university administrators and cross-subsidise research and teaching in other areas. now the paradox here is that even though our universities wish to project themselves as model social citizens, an unemployed or underemployed student from a cash cow faculty soon discovers that they have been sold a degree that is not worth as much as they thought. just like most of their business counterparts, transparency is not generally one of a university's core values. in big business, and this is what our universities are, you keep your business model out of public view. the alternative to cross-subsidy is internal privatisation. for example, some us and european universities have allowed their business schools to retain their revenue and pay only a minimal contribution to the rest of the university. as david kirp describes for one us school, this allowed it to raise salaries and improve facilities while salaries were frozen and faculty needs were not met in other parts of the university. 6 any faculty with strong student demand and/or demand for its research could advocate for such a strategy. so from a strategic perspective, should a university allow any or some or all its faculties to 'privatise'? you will not find this strategic question discussed in any public statement. however, the pressure to (partially) privatise can be seen in the internal budgeting practices of the university. when universities adopt performance as opposed to activity budgets, it encourages their units to become (more) self-supporting. chief financial officers argue that it brings more discipline to the budget process and greater internal transparency. academics in smaller faculties might argue that it diminishes the overall contribution of the university. vice chancellors need to make some tough strategic choices here. public universities by their nature have adopted an open, distributed model of research transfer through publishing research and educating people who then enter the workplace. while education provides revenue, publishing gives away much of the university's new knowledge to the publishing houses, which then sell it back to them (through subscriptions to their journals). to try to get some return from their research, some universities have entered the publishing game via their university press or by publishing a journal or two or three. if successful, both revenue and reputation flow back to the university. however, in australia the more likely outcome is a university press that publishes short production runs and seeks not to put too much drain on the parent institution's finances. the open model of research transfer puts a considerable strain on university finances. to help counter this, the universities now ask their academics to raise more of their own research funding. securing australian research council competitive research grants is highly prized, both for revenue and for reputation. however, this and other forms of funding come at a cost. the grant writing and peer-review process are very timeconsuming and highly problematic. they generate considerable amounts of stress for the people forced to seek these funds. and then when funds are secured, most academics need help navigating the university accounting, reporting and intellectual property policies. at a national press club address professor ian jacobs lamented about the underperforming state of business-university collaboration in australia. his model is that the entire research pipeline moves from discovery, to translation, to application, to commercialisation. our universities tend to focus on discovery, and industry on commercialisation. his strategy to help fill the gap was to ask government for more help in the form of things like better tax incentives for university-business projects, government co-investment in research projects and specific funding for translational research. the strategy here is to move down the value chain. along these lines, in 2017 the british commercialisation company ip group committed $200 million to help some the group-of-eight universities commercialise their research. 7 because this is such an important issue, it is worthwhile thinking about alternative models of research transfer and the roadblocks to such initiatives. the different initiatives respond to the different needs of organisations. here are some further ideas. one was noted earlier in our description of working in the australian graduate school of management. business (and engineering) schools are closer to industry than many other faculties on a university campus. in the business school where industry collaboration was easiest to achieve, and where it seemed to have most impact, was when we were invited into a company to educate its managers. this generally involved customising our mba course material to address a specific internal need. one crucial design element of many of these in-house courses was that the presenter would emphasise the role of evidence and research as inputs to management decisions. this was a good way to get research inside a company and to get decision-makers comfortable with using research in management practice. this is a good example of industry collaboration based on the perceived strength of the academic community (teaching). however, unlike the harvard business school we did not leverage this activity into writing case studies and selling them to other business schools. our performance appraisal kpis did not reward such 'clinical' activity. they do at harvard. a second approach is for the university to encourage and help its students to start their own enterprises, using what they have learned and discovered at university. this goes beyond offering courses in entrepreneurship. it requires a venture-capital approach to funding and mentoring new enterprise development. the idea here is for universities to formally invest in the new enterprises created by their students. the ip group has also put $4.5 million into australian university research start-ups. a third approach is for universities to promote to their researchers the benefits of working more closely with national laboratories (like the csiro), and government departments and agencies (like the australian institute of sport). there are hundreds of these that span many areas of interest to the academic community. they also have their own funding that will support various types of research projects. these alliances are generally perceived as 'safer' than the next option. a fourth approach is for the universities to develop strategic partnerships with key innovative companies. old companies like cisco, exxon, general electric, hoechst a.g., ibm, monsanto, rolls-royce and siemens have collaborated with universities for years, and new companies like amazon, facebook and google also work with universities. because many companies adopt an open innovation approach to new product development, the basic research done in universities extends the workbench of both parties. 8 the challenge is to get both parties to work together. what will often impede the development of a close working relationship is different goals, expectations and rewards of the parties and, more recently, squabbles over intellectual property rights. 9 a fifth approach is to start a new academic hub. for example, the australian national university has created its 3a innovation institute (autonomy, agency and assurance) to create new applied science focused on managing the artificial intelligence and big data revolution. many universities are 'playing' in this space with similar academic-industry collaborations. many are also trying to entice students to join these initiatives by offering specialist (master's) degrees. a sixth approach is to start or expand university consulting units. this type of work tends to focus more on problem solving than innovation. for example, the university of new south wales has a professional enterprise called unisearch that provides expert witness services. the university of south australia has the ehrenberg-bass institute for marketing science that collects money from companies to gather data on patterns of consumer behaviour, which are analysed to provide insights back to these companies. other universities do sponsored research for specific industrial companies. while this is not without controversy, especially in some fields like pharmacy, in other fields like market research, there are fewer pitfalls. where difficulties arise is when working with industry becomes transformative and some vocal staff claim that the institution's academic values may be compromised. then contracts are required to provide confidence that conflicts can be managed, and leadership is required to endorse the principle that externally funded research is permissible and desirable. a strategic advantage of building up industry engagement is that it often provides insight into future research opportunities. thus, engagement becomes a dynamic capability focused on sensing research opportunities and seizing other-peoples' money to explore these issues. in the usa one of the biggest sources of funding for such future-oriented research is the military. from a goal-setting and incentive point of view, industry engagement activities need to align with university and individual key performance indicators. three current metrics are funding, impact and engagement. so research that is of interest to industry can secure new funding. then to create impact, the university and its research teams need to add an advocacy element to their research. this can be done by a public relations effort to explain who needs to know about the research and what policy is necessary to implement the findings. finally, engagement can be facilitated by the research team 'consulting' with the parties who will use the research findings. they help organisations use the findings to solve their problems. research funding without specific advocacy and engagement often falls short of its promise. while all these initiatives could be helpful, they don't address some of the key reasons why too few academics work directly with industry and why many business enterprises are somewhat reluctant to work with academics and their universities. here are some roadblocks that need to be addressed before a university's goal of significantly greater industryuniversity collaboration is likely to be met: • we don't understand each other: business often find it difficult to identify if a university has any knowledge of interest to them. research academics have their own interests that may or may not have any practical implications. so the base condition for both parties is to politely ignore each other. • training and focus: doing a phd can be a brutal process that imprints a certain way and timescale for doing research. also, the motivation for doing this piece of research is often grounded in the existing scholarly literature rather than in the field of practice. so in many disciplines phd students follow the university mission outlined earlier by brian schmidt, namely, they want to add to the stock of knowledge rather than solve a problem that is troubling industry. • because approximately 50% of new phd graduates now enter industry, various programmes have been designed to help these graduates migrate to a non-university workforce. for example, the apr.intern programme focuses on getting students internships across all sectors of industry. what both employers and graduates are finding is that doctoral training provides many transferable skills that enhance workplace performance. 10 however, many of these skills often need some reshaping before they 'work' in the workplace. • in academic performance appraisals, working with industry tends to rate behind research publications in the top-rated journals. also, few of these journals focus on industry problems. and excellent teaching evaluations are often not related to industry engagement. so if universities really want their academics to work more with industry partners, they might consider rewriting the contracts of some chosen academics. the harvard business school case noted earlier has solved this problem. • taxing industry funding: in order to pay for overheads, there are a number of ways in which universities can 'tax' research money from outside sources. hence, from the point of view of the patron, not all their money goes to the nominated research topic. this can detract from the perceived value of their contribution. • academic reputations: in australia many academics have a reputation for being not very commercially astute. they live and work 'in an ivory tower'. their research is 'just academic'. they are too slow. doing linkage research with an industry partner is too bureaucratic. and so the list goes on. so many australian academics are handicapped by their reputation and that of their parent university. many organisations would rather seek help from dedicated research institutions like the csiro or hire a freshly minted phd who has not been too 'corrupted' by the academic environment. in many cases what makes academic research so powerful is also what makes it less than timely in a commercial setting. good research especially big-bang breakthroughs, takes time and is plagued by missteps, things that industry can't easily accommodate. • track record: over many years we have seen repeated intellectual failures across many sciences. so if open research transfer is laudable but expensive, and industry collaboration is somewhat problematic, there is a lot of work to be done before industry will come rushing to seek help from australia's universities. yes, there will be ad hoc pockets of success, but universities need to take a realistic look at their strategic and financial goals for this activity. like every large organisation universities have a diverse cost base. however, a quick look at the university of western australia's (uwa) annual financial statements reveals that the biggest cost is people. about 60% of total expenditure is classified as 'employee-related expenses'. so the people in administrative roles outnumber the front-line academics (42.9-41.2%). 11 this situation is slightly more in favour of the administrators if we add some of the 10.7% of professional staff in the academic units to the administration count. these percentages reveal direct administrative costs. but it gets worse. because nearly every academic will be involved in some form of faculty administration, then part of their salary should also be allocated to the total university administration cost. for example, in 2015 in the university of sydney there were more than 150 faculty committees involved in administration. 12 another interesting picture emerges if we look at non-salary costs. for example, in 2017 the non-salary budget was just under $244 million. then add to this-student expenses of $48 million, materials and supplies of $28 million, repairs and maintenance of $22 million, finance costs of $6 million and other expenses of $131 million and we get a total of $479 million before a single class is taught or a single research paper published. 13 this is a lot of money to run a medium-size university. most other australian universities look similar. multinational corporations and other large organisations spend millions of dollars and millions of hours of employee time each year on compliance. the belief is that rigorous compliance programmes reduce the motivation for employee wrongdoing. also, if an employee does something wrong, the regulator and the institution's insurance provider will note that at least the organisation actively tried to avoid such a problem. the aims of these programmes are (a) to get employees to behave ethically and adhere to best practice, and (b) to eliminate the time-consuming and distracting regulatory and legal processes that accompany regulatory failure. but as is all too evident, these programmes don't always stop employee wrongdoing. an aggressive organisational culture and financial incentives will often drown out compliance training, especially when meeting stretch goals determine salary and promotion. for example, there are still periodic cases of administration (admissions scandals), academic (falsifying research) and student (cheating) misconduct. all universities require that their staff do compliance training. for an administrator this is part of what they are paid to do. however, for a student or an academic it is not clear what compliance training crowds out of the daily agenda. for example, as most academics will acknowledge, teaching crowds out research and service. anything to do with delivering or administering a course will take priority over other activities, even sometimes one's family. however, what comes next in the priority list depends on who asks for help. in the latter part of our academic careers, we have seen the people in central administration jump up the priority list, not by asking for help, but by demanding compliance. for example, this occurs when say a new teaching or reporting protocol is launched and everybody has to make changes to what they were doing previously. another example of a compliance cost was noted earlier. recall that the university of technology sydney requires training for all staff on issues such as equal opportunity, mental health awareness, privacy essentials, communication and collaboration, consent (unwanted sexual behaviour), health and safety, legal affairs, preventing bullying and safety and well-being. at the time of writing, the consent training was being pushed throughout the university-for all staff and all students. it was mandatory for everybody. the online course took approximately one hour to complete and required a 100% pass on the quiz that followed the training module. students were threatened with the withholding of their grades if they did not successfully complete the course; casual academics were offered one hours' worth of salary for their participation, and old professors were sent escalating emails (first by the hr system, then the head of school, then the dean of the business school, then the vice chancellor) when the system reported that they had not completed the course. leaving aside the legal and moral issues of such a request, the costs of this programme can be estimated as follows. from the university website we see that there were now if we take an hourly rate of $50, we see that this exercise has cost the university more than $180,000 for staff. 14 and at, say, a notional $20 per hour for the students, it also has a cost of $897,740 for the students (minus one hour of their study time). so here is an administrative initiative with about a $1 million price tag. all this to address a social issue that the senior administrators in the university thought would signal the institution's commitment to social justice. to cover themselves, they sought legal advice before embarking on this course of action. there is an important ethical issue here. imagine the cost to a finalyear student who can't get a job because there is an embargo on their grades because they did not complete and pass a non-curriculum course on a social issue for an organisation that they are about to leave. second, there is no research that supports the claim that sexual harassment training actually works. 15 so here we see the university imposing a dead-weight economic cost on the institution and a dead-weight social cost on some of its members. 16 another largely hidden cost in our universities is due to quality assurance programmes. they are used to signal institutional quality. for example, there are a number of organisations that have developed best practice protocols for education. these organisations promote their programmes, and once a couple of institutions 'sign on', they chase other similar universities to 'catch up'. many university faculties sign onto such programmes with little evidence that they work and without due consideration for the total compliance costs. for example, the business schools of many of australia's universities have signed up to various accreditation programmes. the university of new south wales business school advertises its membership of two such accrediting bodies, namely, the aacsb (association to advance collegiate schools of business) and equis (european foundation for management development). 17 the school pays to join, pays to be certified and pays to be recertified every few years. inside the business school, degree programmes need to be (re)shaped to fulfil the accrediting scheme's standards and someone is tasked with checking compliance. then information has to be collected to report compliance each period. the costs easily accumulate into the hundreds of thousands of dollars for each such certification. however, there is little empirical evidence to suggest that these sorts of accreditation lead to more effective or efficient education, that they attract more students or that they allow the institution to charge higher course fees-especially when all competitors have signed onto the same programmes. however, what these programmes do is to lead to a lack of differentiation across the schools who sign on. all of australia's universities are plagued with hidden compliance costs. the key question is whether they deliver noticeably better outcomes. one of the great paradoxes of modern times is that as information technology has advanced in leaps and bounds, it is much harder to find if these advances have had a general effect on increasing productivity. within the university context, the productivity gains from new technology have been patchy. many gains are the result of faster communication between people rather than better communication. but as the overuse of social media has demonstrated, new or upgraded information technology doesn't necessarily lead to better outcomes. on a university-wide basis, many of the changes in technology are introduced because the suppliers of a platform have upgraded their offer. so the university 'keeps up to date' by adopting the latest version. however, the benefits of these upgrades are seldom realistically estimated relative to the compliance costs of requiring thousands of staff, and students make changes to accommodate the new system. hence, an upgrade usually causes some pockets of frustration and costly time in compliance, especially for the people who are not as it proficient as their colleagues. universities could save hundreds of thousands of dollars in cost if they had more staff and student input into what really needs to be upgraded as opposed to being 'led' by their suppliers. is the australian taxpayer getting good value for money from its public universities? if a typical university like the university of western australia costs hundreds of millions of dollars annually before it produces a single key output, is this money well spent? and if not, where might cost savings be found? these are difficult questions. and because universities know more about universities than governments or the media, the universities are best placed to provide the answers. our analysis suggests that the problem is of two parts, namely, that the strategies of the institutions are underdeveloped and thus complexity and coordination costs accumulate. the other related cost is that the institutions become overstaffed and too bureaucratic. so one strategy for the government is to tighten up its funding and make each university find its own solution. universities would do well to avoid this brute-force solution by seeking greater efficiency and effectiveness in their operations. 16. towards the end of 2018 a report to the council revealed that many students found that the course was inappropriate to their concerns and that many had yet to complete it. so teaching staff were asked to remind the students that their grades would be withheld! 17. for example, on the unsw website you can't find what the letters aacsb actually stand for. you have to go to the association website and then search beyond the first page to find what they stand for. so how can these five letters really affect the decision-making of a new student if they are essentially meaningless? campus leadership and the entrepreneurial university: a dynamic capabilities perspective discounts catch on as international lure unis banking upside to defy funding cuts and lift margins', australian financial review we've turned our unis into aimless, money-grubbing exploiters of students for example, while a degree in medicine costs much more than a degree in commerce (at the time of writing down payment made on $200 m start-up bet how to create productive partnerships with universities developing successful strategic partnerships with universities advancing australia's knowledge economy: who are the top phd employers? a note to the annual financial statements indicates that this group's salary is slightly less than that of the academics improving our organizational design some people might want to add $54 million of asset depreciation, amortisation and impairment to this figure at this time, the pay rate for the casual staff was either $46.60 or $55.73 per hour (depending on their qualification) while sexual harassment training is the most traditional approach to preventing sexual harassment, it has not been shown to do so key: cord-273005-kab6f157 authors: longhurst, georga j.; stone, danya m.; dulohery, kate; scully, deirdre; campbell, thomas; smith, claire f. title: strength, weakness, opportunity, threat (swot) analysis of the adaptations to anatomical education in the united kingdom and republic of ireland in response to the covid‐19 pandemic date: 2020-05-09 journal: anat sci educ doi: 10.1002/ase.1967 sha: doc_id: 273005 cord_uid: kab6f157 the covid‐19 pandemic has driven the fastest changes to higher education across the globe, necessitated by social distancing measures preventing face‐to‐face teaching. this has led to an almost immediate switch to distance learning by higher education institutions. anatomy faces some unique challenges. intrinsically, anatomy is a three‐dimensional subject that requires a sound understanding of the relationships between structures, often achieved by the study of human cadaveric material, models, and virtual resources. this study sought to identify the approaches taken in the united kingdom and republic of ireland to deliver anatomical education through online means. data were collected from 14 different universities in the united kingdom and republic of ireland and compared adopting a thematic analysis approach. once themes were generated, they were collectively brought together using a strength, weakness, opportunity, threat (swot) analysis. key themes included the opportunity to develop new online resources and the chance to engage in new academic collaborations. academics frequently mentioned the challenge that time constrains could place on the quality and effectiveness of these resources; especially as in many cases the aim of these resources was to compensate for a lack of exposure to cadaveric exposure. comparisons of the actions taken by multiple higher education institutions reveal the ways that academics have tried to balance this demand. discussions will facilitate decisions being made by higher education institutions regarding adapting the curriculum and assessment methods in anatomy. the coronavirus pandemic is an unprecedented emergency that has affected all global industries, including education (ayittey et al., 2020) . with the widespread implementation of social distancing and self-isolation policies, it is not feasible for educators and students to attend lessons or assessments as they have previously. the covid-19 pandemic has disrupted our long-standing educational practices and has precipitated an urgent need for many institutions to rapidly implement alternative educational and assessment strategies. on 31 st december 2019, the world health organization (who) was informed of a small cluster of pneumonia cases with unknown etiology in the wuhan city, hubei province, china (who, 2020a: zhu et al., 2020) . the novel coronavirus was isolated on 7 th january 2020 and genetically sequenced on 12 th january (who, 2020a; wong et al., 2020) . the virus was confirmed to have subsequently spread to thailand, japan, and the republic of korea, on 13 th , 15 th and 20 th january, respectively (who, 2020a). on january 23 rd 2020, the government of china imposed a modern form of quarantine, shutting down all transportation in and out of wuhan. the lockdown expanded to twelve other cities in the hubei province (rubin and wessely, 2020) . covid-19 was first confirmed in the united kingdom (uk) and republic of ireland (roi) on 29 th january 2020 and 29 th february 2020, respectively (holmes, 2020; hpsc, 2020) . on 11 th march 2020, the who officially declared the outbreak a pandemic (who, 2020b) . to control the outbreak of the virus, public health england (phe, 2020) and the department of health and department of the taoiseach in the roi (dh&dt, 2020) introduced increasingly restrictive policies governing the movement and gathering of people. social distancing was implemented in the uk on 17 th march 2020 (mahase, 2020) and even greater restrictions to movement were implemented on 23 rd march 2020 (phe, 2020). historic pandemics, such as the severe acute respiratory syndrome (sars) also saw changes to educational practices and important lessons were learned. during the sars epidemic, chinese and canadian medical schools were forced to cease their clinical clerkships and electives (ahmed et al., 2020) . chinese medical schools supplemented their students learning experiences with the introduction of online problem-based learning (ahmed et al., 2020) . at the same time in hong kong, university policies decreed that the external examiner must be present for final-year medical distinction examinations (patil and yan, 2003) . these examinations traditionally took place as viva voce examinations; however, because the external examiner could not be present, these examinations took place by phone call. due to concern regarding the stability of the phone connection, the first part of the examination was conducted by an external examiner and no interruptions were reported (patil and yan, 2003) . these solutions could similarly be utilized by anatomists in addressing the challenges that have arisen due to the covid-19 pandemic. education institutes across all levels (pre-primary, primary, secondary and tertiary) have closed in 188 countries across the globe, impacting over 91% of the world's student population (unesco, 2020) . universities in the uk started to close from the 16th of march. universities in the roi were formally closed on 12 th march 2020 (rté, 2020) and face-to-face teaching was suspended from 16 th march, following government advice. universities have moved all teaching activities online and set up communication platforms, such as webpages and email streams to inform students of the ever-changing circumstance. the majority of higher education institutions in the uk and roi begin their academic term in august or september, and this cessation of face-to-face teaching occurred during the second or third semesters. educators were forced to rapidly find solutions to many challenges; for some this meant moving to online delivery within days. governmental education departments released guidance for students and educators in response to this (des, 2020; gov.uk, 2020a, b) . there are over 45,000 medical students in the uk and roi (gmc, 2017; heffron and socha-dietrich, 2019) . approximately half of these students are completing formal anatomical education during the early years of their program (heylings, 2002) . therefore, alternative educational and assessment strategies will impact a large cohort of approximately 20,000 students with immediate effect. for final years, the general medical council (gmc) urged medical schools to fast track the graduation and registrations of these students to supplement the healthcare workforce (msc, 2020). the google hangouts (google meet) video conferencing application (google inc., mountain view, ca) has successfully been used to deliver lessons to medical students assigned to surgical specialties and it has been speculated that these could similarly be implemented to supplement anatomical education (moszkowicz et al., 2020) . at imperial college london, teleteaching and telemedicine modalities have been implemented to supplement the clinical needs of final-year medical students, and these students are currently perfecting their clinical judgment using an online repository of patient interviews and clinical cases (mian and khan, 2020) . in related fields, such as dentistry, there have also been recommendations to shift education toward online platforms (meng et al., 2020) . there are 41 medical schools in the uk and seven in roi. recently, five new medical schools in the uk were created but have not been included in the sample as they are either not currently taking students or are under the umbrella affiliation of another medical school and are hence using their curriculum. the number of hours dedicated to anatomy teaching and the number of anatomical staff members significantly varies throughout the uk. anatomy is taught through systems-based curricula, problem-based curricula, or in the traditional regional format. lancaster university medical school, peninsula college of medicine and dentistry, and university of limerick medical school are the only institutions that do not use human cadavers to teach anatomy (heylings, 2002) . educators across the globe have been forced to replace traditional learning modalities with distance and blended learning approaches. distance learning has been described as an information delivery mechanism where the educator and learner are separated in both time and space (billings, 2007) , whereas blended learning combines traditional classroom methods of learning with online learning modalities (green and whitburn, 2016) . neither approach is novel, and both have been used successfully as part of anatomy education (pereira et al., 2007; ferrer-torregrosa et al., 2016) . however, these approaches are usually implemented after strategic planning, collaboration with other academics, and careful consideration of the pedagogical evidence. in order to utilize online approaches, educators must invest considerable time up-front to learn how to create online learning material. in many cases, academics are required to develop these new skills and create or adapt resources in parallel with a time frame that reflects the normal progression of student learning. the storage, maintenance, and embalming of cadaveric specimens poses a further challenge to anatomy departments with limited on-site access. with the body donation program largely halted in the uk and roi (hta, 2020a), there may be a reduced number of available specimens for teaching in the following academic year. guidance from the international committee of the red cross (icrc) has also suggested that medical schools may provide body storage facilities as a contingency for mortuary overflow (finegan et al., 2020) , with updated guidance on licensing of emergency mortuaries concurrently released by the human tissue authority (hta) to support this (hta, 2020b). this may result in existing cadaveric material being compromised, placing further strain on availability of specimens going forward. due to the unprecedented demand that the covid-19 pandemic has placed on public health services, clinically trained anatomists have been encouraged to return to the clinical workforce (willan et al., 2020) . in addition, non-clinical academic staff may contract covid-19, or have caring responsibilities placing additional demands on the remaining members within a team of anatomists, disproportionately affecting universities with smaller academic teams, or in locations where only one anatomist is employed placing considerable burden on them. the negative impact on the mental wellbeing of healthcare workers due to anxieties and pressure in relation to covid-19 has already been documented (shaw, 2020) . this research aimed to understand the variety of solutions being utilized across institutions in the uk and roi and to facilitate easier decision-making surrounding changes to curriculum and assessment strategy. the two research questions were: (1) what strategies have been adopted by uk and roi anatomy departments in response to what strengths, weaknesses, opportunities, and threats does the instant move to online learning bring? the questionnaire was conceptualized by the researchers for this study (g.j.l., d.m.s., k.d., d.s., and t.c.) in order to obtain information from anatomy departments across the uk and roi on their institutional and departmental responses to the covid-19 pandemic. all questions proposed by the researchers were selected for review. only the questions that were agreed by all contributors were selected. the seven questions that were selected focused around three themes: (1) location of university and size of anatomy team, (2) adaptations to teaching and resources used in lectures and practical sessions and (3) major challenges and opportunities (see supporting information 1). all feedback was solicited by way of free-text responses. a pilot study was completed in the respective institutions of the researchers (g.j.l., d.m.s., k.d., d.s., and t.c), and on review there were no postpilot modifications made. the questionnaire was tabulated in a microsoft word document (microsoft corp., redmond, wa) and emailed to anatomists in higher education institutions in the uk and roi. the list of anatomists and institutions selected for this study was created using a convenience sample of known contacts. a total of 14 responses were received from the 27 medical schools that were contacted (52% response rate). the adaptations to lectures, practical, and assessment were collated and summarized and a thematic analysis was carried out on the opportunities and challenges. all responses were analyzed using thematic analysis by the researchers (g.j.l., d.m.s., k.d., d.s., and t.c.) (glaser and straus, 1967; braun and clarke, 2012) . to achieve this, the six-phase process described by braun and clarke (2012) was followed. first, all authors read through the entire data set to familiarize themselves with the data. initial codes were then manually generated using an open-coding approach by highlighting interesting and relevant aspects of the responses. this was done individually and then any discrepancies discussed in order to minimize bias. codes were then collated into more general themes. these themes were reviewed by all authors to ensure they fully reflected the responses. due to the small sample size, themes were generated without digital programs. the most commonly reoccurring themes were then used to inform the next stage of the analysis, as well as other essential themes as identified by the authors. the authors assigned these themes to either strengths, weaknesses, opportunities, or threats for the purpose of swot analysis, an analytical approach commonly used to inform strategic planning and decision making (helms and nixon, 2010; teoli and an, 2019) . a swot analysis was originally developed as a business tool to aid decision making, but has since been used to analyze strategies in higher education, including medical education (sharma, 2005; burr, 2009; liu, 2017; hazzan et al., 2018; leiber et al., 2018) . due to the urgent and rapidly evolving nature of covid 19, it was not possible to secure ethical approval. to mitigate this, it was made clear that respondents were providing data and by doing so were consenting for the data to be used. the authors are planning a follow-up study from different stakeholders' perspectives and this will go through full ethical approval and the data presented here will be considered as pilot data. fourteen universities provided information on the departmental response to covid-19. twelve universities were uk based and two were based in the roi. the results presented here represent 39% and 33% of medical schools in the uk and roi, respectively. ten of the uk-based universities are located in england, one is located in wales and another one in northern ireland. the two irish universities are located in dublin. lectures. universities opted to replace lectures with recorded presentations and accompanying audio that was uploaded to the virtual learning environment. the most common lecture recording platform used was "panopto" (panopto inc., seattle, wa), with 50% of universities citing its use. thirty-six percent of universities also provided live sessions and tutorials via platforms such as "zoom" (zoom voice communications inc., san jose, ca), "collaborate ultra" (blackboard inc., new york, ny), and "big blue button" (big blue button inc., ottawa, canada). practical sessions. all bar two universities in the sample group used cadaveric material to teach anatomy prior to the pandemic. universities replaced practical sessions by supplementing virtual learning environments with additional resources. twenty-nine percent of universities used digitized cadaveric resources only, seven percent used 3d virtual resources only, and 43% used a combination of cadaveric and 3d virtual resources (table 1 ). digitized cadaveric resources included "acland's video atlas of human anatomy" (acland, 2013) , high-quality cadaveric images, bespoke videos of prosected/ plastinated specimens, youtube (tm) videos (youtube, san bruno, ca), and the visible human project (u.s national library of medicine, bethesda, md) ( table 2) . three-dimensional virtual resources included "visible body" (argosy publishing, inc., newton, ma), "complete anatomy" (3d4medical/elsevier, dublin, republic of ireland), 3d models using "sketchfab" (sketchfab, new york, ny), "anatomy tv" (primal pictures ltd., colchester, uk), and "sectra" (a virtual dissection table; sectra ab, linköping, sweden) ( table 2) . forty-three percent of universities provided further support for practical sessions by hosting live tutorials and sessions using platforms such as "zoom" (zoom voice communications inc., san jose, ca) and "microsoft teams" (microsoft corp., redmond, wa). assessment. fourteen percent of universities did not have a summative practical assessment in their curriculum, a further 36% of universities canceled their assessment and 21% completed online digital spotter examinations (table 3) . written examinations were canceled in seven percent of universities (table 3) . thirty-six percent opted for online multiple-choice questions (mcqs)/extending matching questions (emqs)/ single-answer questions (saqs)/single-best-answer (sbas) examinations and 21% opted for open-book examinations (table 3 ). there were no regional differences noted with regard to assessment. opportunities and challenges. despite chronic disruption to curricula, the covid-19 pandemic has presented many opportunities to universities. the opportunity to develop new online resources was highlighted by 71% of universities. fifty percent of universities cited academic collaboration as an important opportunity, and 29% highlighted the importance of working from home. universities identified other opportunities such as upskilling in new technologies (21%), incorporation of blended learning (14%), development of alternative examination methods (7%), and free access to online resources (7%) ( table 4 ). the most commonly expressed concern was the time investment associated with the development of new resources to replace lectures and practical classes. this challenge was highlighted by 57% of universities. fifty percent of universities were concerned about lack of practical sessions/cadaveric exposure, and 36% of universities identified reductions in student engagement. universities also highlighted concerns regarding the teacher-student relationship (21%), assessment (14%), working from home (14%), suspension of body donor programs (7%), and lack of technical support (7%) ( table 4 ). many universities documented similar pedagogical opportunities and challenges. themes from the responses to top opportunities and challenges are sorted into a swot analysis and used to form the basis of this discussion. reoccurring strengths included the development of new resources and skills. time constraints, lack of exposure to cadaveric material, and changes to assessment were identified as weaknesses. opportunities identified included academic collaboration and working remotely, as well as with the opportunity to implement blended learning in future curriculum development. the reduction in quality of resources, reduced student engagement, and diminished student-teacher relationship were considered threats (fig. 1 ). the approaches taken by higher education institutions and the effect these have on the strengths, weaknesses, opportunities, and threats currently faced are also discussed to elicit which approaches could prove most effective. development of new online resources. in response to the covid-19 lockdown, both digitized cadaveric resources and 3d virtual anatomy platforms were used by 43% of universities, in an attempt to emulate canceled practical sessions. twenty one percent of universities created unique videos. bespoke resources have been shown to lead to a significant increase in student satisfaction (mandernach, 2009) . new resources created and developed by academics include videos, virtual dissections, formative quizzes, and updating practical dissection notes that will be available to students through virtual learning environments. previous studies have elucidated that students appreciate anatomical videos and they have the potential to improve test scores (pereira et al., 2004; dilullo et al., 2006; topping, 2014) . to date, there are limited data on the effectiveness of virtual dissections; however, the results have been positive, when used in conjunction with cadaveric material (yammine and violato, 2015; darras et al., 2019) . a significant number of medical students have reported that they find online resources helpful (smith et al., 2014) this echoes a statement by one academic who stated gaining "free access to high quality resources from publishers" was a positive aspect of the pandemic. interestingly, the usefulness of online resources for individual students has been linked to personality preference, and thus the switch to distant learning may prove harder for certain groups of students (mcnulty et al., 2006) . the context of the anatomical learning environment has undergone a paradigm shift since the lockdown, and this may result in students adopting different strategic approaches to their learning (smith et al., 2014) . thus, it is imperative to ensure that these new resources are based upon sound pedagogical theory in order to foster deep learning strategies (kreber, 2002) . in addition, the number of new resources has the potential to overwhelm the learner, having negative implications for their extraneous cognitive load. extraneous cognitive load is affected by how information is presented by the instructor (leppink and van den heuvel, 2015) . upskilling in new technologies and resources. three institutions (21%) noted that this is an opportunity to develop technological skills. one academic stated this was a "chance to upskill and incorporate alternative software and novel resources into our teaching." this sudden leap in upskilling and innovation by a large proportion of academics has the potential to transform medical education by incorporating online learning to all aspects of the curricula (skochelak and stack, 2017; rose, 2020) . one academic stated that that this was an "opportunity to improve staff awareness of online teaching methods and their confidence in using them." for anatomists, this is a unique opportunity to assess the educational benefits of this software, encouraged by free licenses offered by many companies during this time, such as "visible body" (argosy publishing, inc., newton, ma) and "human biodigital" (biodigital inc., seoul, south korea). furthermore, it has triggered the development of new skills in assessment modalities. as all universities have adopted an online approach to written examinations, new skills in utilizing the full capabilities of virtual learning environments are being explored. virtual learning environments allow academics to easily build question banks and design assessments that can be marked automatically or manually. virtual learning environments also facilitate randomized presentation of questions, time constraints, and prohibit backtracking in order to prevent collaborative answering. for example, "brightspace" (d2l corp., kitchener, canada) facilitates precise keystroke recording-an important feature to prevent cheating during online spotters. the move to online assessment has identified an opportunity to improve examination conditions (dennick et al., 2009) . specifically, this is true for summative continuous assessments that often involve image-based spotter examinations as there is a high workload associated with designing and implementing station-based spotter exams. schubert et al., (2009) reported that designing, implementing, and invigilating a station-based gross anatomy examination encompassed 3.5 days' work (shubert et al., 2009). dennick et al., (2009) showed that student performance on gross anatomical examination is no different between traditional spotter examinations compared to digital online modes. the development of skills in online assessment modalities may be of benefit for future academic practice without endangering academic integrity or student performance (meyer et al., 2016) . time constraints. time is ultimately of the essence when it comes to creating additional anatomical resources in response to covid-19, with universities identifying it as the top challenge in the delivery of anatomical education. one academic noted that the "time needed to learn how to use technologies" was an issue. creating new resources can take at least three times as much work compared to a traditional format (gewin, 2020) . this may be attributed to the steep learning curve required to learn how to use technologies effectively. the added pressure of producing "high-quality resources," in addition to fulfilling other academic duties in a limited time period may put undue pressure on academics. therefore, it is imperative to weigh-up the cost-benefit implications of the time needed to create new resources against the educational benefit for the students. indeed, it was observed that instructor-made videos lead to no measurable improvements in student achievement (mandernach, 2009 ); however, it may have an impact in student engagement as discussed in the section outlining potential threats. furthermore, well-renowned anatomical educational resources already exist, so one must decide if it is worth 're-inventing the wheel', so to speak. for example, "acland's video atlas of human anatomy" (acland, 2013), recurrent themes in the context of the swot paradigm. the frequency of recurring themes was calculated based on respondent survey data (a total of 14 responses from the 27 contacted medical schools in the united kingdom and republic of ireland). the authors subsequently assigned these themes to the most appropriate heading of the swot paradigm. created in 1993, is still used by medical schools today along with other useful online resources as outlined by participating universities ( table 2) . lack of practical sessions and cadaveric exposure. the results show that a major concern for academics is the loss of cadaveric-based teaching, as expressed by 50% of participating universities. cadavers are utilized as the pillar pedagogical tool in many medical schools across the globe (habicht et al., 2018) . indeed, one academic stated that "not physically attending practical sessions will limit haptic understanding and reduce discussions concerning variations and pathology." the benefits of cadaveric-based teaching have been eluded in a number of studies (aziz et al., 2002; estai and bunt, 2016) . cadavericbased teaching has been shown to lead to a deep understanding of the three-dimensional relationships of the human body and allows students to appreciate anatomical variations and pathologies (azer and eizenberg, 2007; smith and mathaias, 2010; fruhstorfer et al., 2011; hafferty and o'donnell, 2013) . cadaveric dissection can be a sensory explosion, arguably one that cannot be stimulated by technology (korf et al., 2008) . in addition, it allows students to develop competencies of medical profession and gain attributes through the hidden anatomical curriculum, for example, encountering death, empathy, and professionalism, in addition to communication and teamwork skills (estai and bunt, 2016; kumar ghosh and kumar, 2019) . additionally, students themselves believe that working with cadaveric material helps them learn anatomy and is an important component of becoming a healthcare professional (smith et al., 2014; flack and nicholson, 2018) . in the uk and roi, fortunately the lockdown occurred nearer the end of the academic term, thus students will have been exposed to cadaveric material for approximately six or seventh months. conversely, this is not the case for other countries, e.g., in australia and new zealand, where the academic term begins in february/march. all medical students may be disproportionately affected by the lack of cadaveric-based classes. although further studies will elucidate how the current covid-19 crisis will affect students' understanding of anatomy, a previous report revealed that students participating in gross anatomy laboratories received significantly higher grades, compared to those taking an online course (mathiowetz et al., 2016) . with many institutions looking to produce new online content to replace cadaveric dissection the hta have released a new statement regarding the use of cadaveric images, reiterating the importance of protecting the dignity of the deceased (hta, 2020a). where cadaveric images are to be used to supplement online teaching, it is imperative to check if the donor provided consent for their remains to be used in images and videos. (hta, 2016; hennessy et al., 2020) . some institutions have also put in measures to limit sharing of cadaveric material now available on virtual learning environments by ensuring students are able to view but not download this content. issues with assessment. twenty-nine percent of the institutions surveyed did not have pre-existing summative assessments based on practical content. of those that did, 36% canceled the summative element, and 21% switched to online assessments to maintain normal student progression (table 3) . however, online assessments pose their own unique challenges. one key principle that needs to be considered is the preservation of constructive alignment of the assessment task to the intended learning outcomes and teaching/learning activities (biggs and tang, 2011) . many higher education institutions teach through cadaveric dissection and assess the practical element of learnings through a traditional spotter style format. this will encourage students to construct their knowledge of anatomy through learning activities with a focus on cadaveric material. however, online assessments cannot replicate the wealth of information available in a cadaveric specimen that can act as a retrieval cue for learning that took place during practical classes (ali et al., 2015) . a switch to online assessment could cause a breakdown of constructive alignment between the teaching learning activity and assessment task. the results highlighted that 14% of universities identified compromised assessment standards as a significant concern, with "questions around validity and practicality of delivering exams online." one way of mitigating this would be to ensure that images used in online assessments reflect those used during teaching and learning activities. research has suggested that assessment modality does not significantly influence student achievement (attardi and rogers, 2015) and highlights the need for carefully chosen images with clear orientation (meyer et al., 2016) . in order to maintain constructive alignment, it is important that new assessment modalities are chosen that best assess the skills students are expected to achieve through their learning. academic collaboration and working remotely. the covid-19 pandemic presented opportunities for online collaboration between academic peers, both within and between institutions to gain "insight and inspiration" from others that "are facing similar challenges." these were ranked as the main opportunities by 50% of the universities in this study. additionally, four institutes commented on the opportunity to create new and effective online working environments which particularly highlights the opportunity for remote working in the future. while there were two responses regarding the negative aspects of working from home, they both reflected the impact of the current covid-19 crisis (i.e., lack of textbooks and childcare responsibilities). already, there has been a wealth of support and resources offered by numerous academics on twitter (tm) (twitter inc., san francisco, ca), facebook (tm) (facebook inc., menlo park, ca), and other social media platforms. in addition, the anatomical society has a collaborative database containing online resources (as, 2020). an online network, particularly in circumstances such as these can have an impact on academic success (maican et al., 2019) . external collaborations can act as a source of support and are particularly important in institutions with small teams of anatomists. nurturing these collaborations and developing an online community of practice in the future is a huge opportunity for the anatomical community. this is especially important for the next academic year as the impact of covid-19 on the body donor program and anatomical education is yet to be fully appreciated. additionally, as many anatomy departments are developing new technological skills in teleconferencing, there is a potential to pool expertise among institutions. it gives departments the opportunity to have "online connections made between peers," to share expertise in anatomical sub-specialties and to showcase cutting-edge anatomical research across satellite campuses, and indeed across institutions worldwide. incorporation of blended learning in future curriculum development. academics reported being excited at the chance to develop new resources as it was the most frequently cited opportunity among the cohort (71% of responders). the decline in the priority of cadavers, and indeed anatomy as a subject, within medical curricula is well documented (drake et al., 2002; drake et al., 2009) . with diminished resources including curricula time, budgets, and staffing, in conjunction with increased student numbers, innovative and novel approaches are needed to solve these pressing issues. perhaps covid-19 could act as a catalyst to do so. fourteen percent of academics suggested that this crisis presented them with an opportunity to develop resources that can be used in the upcoming academic year, allowing them to incorporate more blended learning techniques and review the curriculum. therefore, it is imperative that enough time can be dedicated in order to create high-quality references. reduced student engagement. a frequently expressed concern was diminished student engagement as a function of the rapid implementation of distance learning (36% of responders). on consideration, this may be compounded by the fact that many universities are no longer pursuing typical mandatory attendance policies for teaching sessions. moreover, the current covid-19 crisis may cause an increased risk of isolation, anxiety, and boredom among the population (rubin and wessely, 2020) . this was also highlighted by one institution as a major concern among the student population where they were worried about the levels of "anxiety" that students face on "the future of their education." previous studies have shown a decrease in quality of life and stress negatively impacts student motivation and academic results (artino et al., 2010; lyndon et al., 2017) . specifically, the responses from this study included comments that highlighted their concerns with "creating resources that students will actively engage with" as well as how to "encourage and maintain student engagement." thus, academics adapting their approach to anatomical education should consider the need for support, interactivity, and social engagement with and between the students. a reduction in these elements may ultimately impede academic progress and student satisfaction. ultimately, one academic expressed they wanted to ensure that students "were getting value for money" from their course. additionally, technical issues, for example, unstable internet connections or lack of suitable electronic devices, will also impact student engagement (wimpenny and savin-baden, 2013; ilgaz and gülbahar, 2015) . the unexpected shift to distance learning means that some students may not be technically prepared for distant learning and assessment. therefore, due consideration to accessibility must also be considered as a potential threat to delivery of distance learning and assessment. in conjunction with potential issues in relation to the student population, staff may not have proficient knowledge of pedagogical techniques specific to the delivery of online teaching. some staff members may not have been able to fully evaluate the theoretical and practical implications of distance learning prior to the cessation of face-to-face teaching. this may also impact on their ability to boost student engagement (wimpenny and savin-baden, 2013) . teacher-student relationship. maintaining the teacherstudent relationship was considered a threat (21% of responders) as online resources lead to a perceived improvement in the online instructor social presence (draus et al., 2014) . the online instructor presence effectively reduces the transactional distance between student and instructor (attardi et al., 2018; stone and barry, 2019) by increasing the likelihood of interaction. one way to address this issue is to host synchronous (live) classes (mcbrien et al., 2009; o'flaherty and laws, 2014) . this approach was taken by 36% of universities that hosted live lectures and 43% hosted live tutorials to support practical sessions. many synchronous software packages contain interactive polls that can be used to assess knowledge, encourage engagement, and provide instant feedback to students. studies have shown that timely feedback from instructors is hugely important in maintaining engagement (martin and bolliger, 2018; ragusa and crampton, 2018) . interactive polling can therefore be utilized in a flipped-classroom approach, shown to increase student interactivity as all levels of bloom's taxonomy can be assessed if correctly designed (gilroy et al., 2015) . providing clear guided instructions for problem solving and linking content to clinical practice and prior knowledge are other methods that are proven to improve engagement (wimpenny and savin-baden, 2013; boton and gregory, 2015; ilgaz and gülbahar, 2015; buelow et al., 2018; martin and bolliger, 2018) . thus, within the synchronous classes, staff can potentially utilize anatomical education software programs to incorporate applied anatomy and clinical scenarios into their sessions. in this regard, there is evidence that virtual synchronous conferencing methods may increase peer-peer and peer-teacher interactions (mcbrien et al., 2009; o'flaherty and laws, 2014) . however, in the covid-19 crisis, synchronous teaching may be deemed unsuitable for a proportion of the student population. students living in different time zones or those with parental and caregiving responsibilities may be unable to attend scheduled synchronous sessions. as synchronous sessions offer potential for interactivity, students unable to attend may feel particularly isolated if departments solely relied on synchronous teaching methods. to address this, interactive asynchronous teaching methods have been employed, for example, twitter (tm) and discussion boards or padlet (tm) (padlet, inc., san francisco, ca) (table 1) . a strong teacher-student relationship results in students feeling more comfortable engaging with online communication platforms such as discussion boards (griffiths and graham, 2009; rose, 2009 ). the use of social media has been shown to diminish anxiety and foster a sense of community (hennessy et al., 2016) . however, if these options are unavailable, adapting resources that are already used across ten institutions in our study can include interactive elements within asynchronous teaching. this includes creating a short video content of problem-solving tasks with recorded audiovisual input from the tutor. such an approach permits students to consistently see and hear their tutor, with the potential to aid in student engagement (wimpenny and savin-baden, 2013; boton and gregory, 2015; ilgaz and gülbahar, 2015; buelow et al., 2018; martin and bolliger, 2018) . it is also important to consider the limitations of the methodologies employed and the data presented. first, the presented data are representative of a subset of the total number of medical schools in the uk and roi. it is conceivable that adaptions utilized by an individual institution or minority of universities may have been missed due to inadequate sampling. second, during the data collection process only a single individual was contacted from each institution. it is our presumption that respondents' answers accurately reflect the most commonly utilized adaptations at their respective institutions-though our methodology did not establish this. third, the authors did not sample any universities in scotland. this is important to consider as differing national practices could affect the various adaptations utilized by universities in response to the pandemic. fourth, our survey sought to establish information pertaining to the education of medical students only and did not consider the allied health disciplines. it would be interesting to also identify the adaptations utilized by universities to manage this cohort's education as any niche solution employed could also be translated to the medical cohort. fifth, the survey did not explicitly state that respondents should describe the strengths and weaknesses of the various adaptations implemented by their respective university. the authors believed that incomplete responses would be returned if respondents were required to provide more information. in hindsight, the utility of these adaptations would have been better contextualized if the authors had formally sought for respondents to comment on the strengths and weaknesses of the adaptations implemented at their university. sixth, due to the small sample size, statistics for interval validity were not possible. everyone is currently amid an unprecedented global event. the effects of covid-19 upon anatomical education, and medical education, are not yet fully understood. changes to curriculum normally take years to research, enforce, and evaluate. however, the current crisis has forced academics to make radial adjustments in a short period of time. the swot analysis and embedded pedagogical theory of these changes has been presented in the hope that anatomists feel more confident in their decision-making. additionally, as the change in anatomical education has been so sudden over the course of the next few months, academics' responses may change and new threats and opportunities may arise. currently, negative effects, such as the time pressure, changes to assessment, and implications in students' engagements and relationship, are somewhat balanced by, positive consequences, such as the potential to create new resources and foster academic collaborations, also arise. indeed, this crisis may be a catalyst for the integration of novel technological resources and the development of stronger ties and collaborations between anatomy departments. it is evident that anatomy departments across the uk and roi are putting tremendous effort into the delivery of high-quality education in the midst of the pandemic. it is hoped that the adaptations utilized by universities will ultimately translate to a lasting positive change in the delivery of anatomical education; and as such there are plans to assess the sustainability of these modalities with a follow-up survey. the authors would like to express their sincerest gratitude to the fellow anatomists who contributed to this study. it is our hope that this crisis leads to the beginning of many new collaborations. georga j. longhurst, b.sc., m.sc. s.p. cert., is a lecturer in anatomical sciences at st george's university of london in london, uk. she teaches anatomy to medical and allied health students. she has a research interest in anatomical education. danya m. stone, b.sc., m.sc., f.h.e.a., is a lecturer in anatomy brighton and sussex medical school (brighton), falmer, uk. she teaches anatomy to medical students and contributes to teaching on multiple external courses. her research interest is in medical education. kate dulohery, b.sc., m.sc., ph.d., f.h.e.a., is a lecturer in anatomy in school of medical education at newcastle university, newcastle upon tyne, uk. she teaches anatomy to medical, dentistry and allied health students and her research interests include incorporation of technology in anatomical education. deidre scully, b.sc., m.sc., ph.d., is a teaching fellow in the department of anatomy, school of medicine at trinity college dublin, the university of dublin, dublin, republic of ireland. she teaches medical and allied health students and her research interests include reproductive biology and anatomical education. thomas acland's video atlas of human anatomy ® covid-19 and medical education lancet inf dis the perception of anatomy teaching among uk medical students second-year medical students' motivational beliefs, emotions, and achievement improving online interactions. lessons from an online anatomy course with a laboratory for undergraduate students design and implementation of an online systemic human anatomy course with laboratory economic impacts of wuhan 2019-ncov on china and the world do we need dissection in an integrated problem-based learning medical course? perceptions of first-and second-year students the human cadaver in the age of biomedical informatics supporting learning engagement with online students integrating assessment innovations in medical education teaching for quality learning at university. 4th ed distance education in nursing: 25 years and going strong minimizing attrition in online degree courses apa handbook of research methods in psychology integrated virtual and cadaveric dissection laboratories enhance first year medical students' anatomy experience: a pilot study online eassessment: amee guide no. 39 republic of ireland: government of ireland public health measures in place until 5 may to prevent spreading covid-19 anatomy in a new curriculum: facilitating the learning of gross anatomy using web access streaming dissection videos survey of gross anatomy, microscopic anatomy, neuroscience, and embryology courses in medical school curricula in the united states medical education in the anatomical sciences: the winds of change continue to blow the influence of instructor generated video content on students satisfaction with and engagement in asynchronous online courses best teaching practices in anatomy education: a critical review distance learning ects and flipped classroom in the anatomy learning: comparative study of the use of augmented reality, video and notes icrc advisory group on the management of covid-19 related fatalities. 2020. international committee of the red cross (icrc): general guidance for the management of the dead related to covid-19 what do medical students learn from dissection? the use of plastinated prosections for teaching anatomy-the view of medical students on the value of this learning resource five tips for moving teaching online as covid-19 takes hold enhancing student engagement using the flipped classroom aldine transactions. 282 p. gmc. 2017. general medical council united kingdom. our data on medical students and doctors in training in the uk. in: gmc. the state of medical education and practice in the uk government of united kingdom. coronavirus (covid-19): guidance for educational settings covid-19): guidance for isolation for residential educational settings impact of introduction of blended learning in gross anatomy on student outcomes the potential of asynchronous video in online education the hidden curriculum in health professional education bodies for anatomy education in medical schools: an overview of the sources of cadavers worldwide stem teachers' swot analysis of stem education: the bureaucratic-professional conflict the irish paradox: doctor shortages despite high numbers of domestic and foreign medical graduates organization for economic co-operation and development publishing exploring swot analysis-where are we now? a review of academic research from the last decade social media and anatomy education: using twitter to enhance the student learning experience in anatomy social media guidelines for anatomists anatomy 1999-2000: the curriculum, who teaches it and how? 08 apr covid-19 timeline. the british foreign policy group (bfpg) health protection surveillance centre. covid-19 health surveillance monitor. health protection surveillance centre hta guidelines: filming or photographing of bodies or body parts that have been donated to medical schools or surgical training centres human tissue authority. anatomy sector guidance. human tissue authority 034%20ext ernal %20-%20gui dance %20on%20hta %20lic ensin g%20of%20eme rgenc y%20mor tuari es a snapshot of online learners: e-readiness, e-satisfaction and expectations the dissection course-necessary and indispensable for teaching anatomy to medical students building professionalism in human dissection room as a component of hidden curriculum delivery: a systematic review of good practices teaching excellence, teaching expertise and the scholarship of teaching bridging theory and practice of impact evaluation of quality management in higher education institutions: a swot analysis the evolution of cognitive load theory and its application to medical education swot analysis of the current nursing education in china burnout, quality of life, motivation, and academic achievement among medical students: a person-oriented approach covid-19: uk starts social distancing after new model points to 260 000 potential deaths a study on academic staff personality and technology acceptance: the case of communication and collaboration applications effect of instructor-personalized multimedia in the online classroom engagement matters: student perceptions on the importance of engagement strategies in the online learning environment comparison of a gross anatomy laboratory to online anatomy software for teaching anatomy virtual spaces: employing a synchronous online classroom to facilitate student engagement in online learning personality preference influences medical student use of specific computer-aided instruction (cai) coronavirus disease 2019 (covid-19): emerging and future challenges for dental and oral medicine student performance on practical gross anatomy examinations is not affected by assessment modality medical education during pandemics: a uk perspective creating the medical schools of the future medical students' approaches to learning anatomy: students' experiences and relations to the learning environment the context of learning anatomy: does it make a difference? improving virtual learning interactions: reducing the transactional distance of online anatomy modules swot analysis. ncbi internet bookshelf gross anatomy videos: student satisfaction, usage, and effect on student performance in a condensed curriculum challenges for nhs hospitals during covid-19 epidemic alienation, agency and authenticity: a synthesis of the literature on student engagement covid-19 in singapore-current experience: critical global issues that require attention and action world health organization. who director-general's opening remarks at the media briefing on covid-19 -11 a meta-analysis of the educational effectiveness of three-dimensional visualization technologies in teaching anatomy china novel coronavirus investigating and research team. 2020. a novel coronavirus from patients with pneumonia in china daily medical education for confined students during covid-19 pandemic: a simple videoconference solution advice from medical schools council to uk medical schools on actions surrounding covid-19 nursing student's evaluation of a virtual classroom experience in support of their learning bioscience sars and its effect on medical education in hong kong using videoclips to improve theoretical anatomy teaching effectiveness of using blended learning strategies for teaching and learning human anatomy guidance: staying at home and away from others (social distancing) sense of connection, identity and academic success in distance education: sociologically exploring online learning environments student perceptions of the use of instructor-made videos in online and face-to-face classes medical student education in the time of covid-19 rté. 2020. raidió teilifís éireann. coronavirus in ireland -a timeline the psychological effects of quarantining a city assessment of spatial anatomical knowledge with a 'three-dimensional multiple choice test' (3d-mc) health education in india: a strengths, weaknesses, opportunities, and threats (swot) analysis hopelessness, helplessness and resilience: the importance of safeguarding our trainees' mental wellbeing during the covid-19 pandemic key: cord-126132-5k415xvj authors: swain, v. das; kwon, h.; saket, b.; morshed, m. bin; tran, k.; patel, d.; tian, y.; philipose, j.; cui, y.; plotz, t.; choudhury, m. de; abowd, g. d. title: leveraging wifi network logs to infer social interactions: a case study of academic performance and student behavior date: 2020-05-22 journal: nan doi: nan sha: doc_id: 126132 cord_uid: 5k415xvj on university campuses, social interactions among students can explain their academic experiences. however, assessing these interactions with surveys fails to capture their dynamic nature. while these behaviors can be captured with client-based passive sensing, these techniques are limited in scalability. by contrast, infrastructure-based approaches can scale to a large cohort and infer social interactions based on collocation of students. this paper investigates one such approach by leveraging wifi association logs archived by a managed campus network. in their raw form, access point logs can approximate a student's location but with low spatio-temporal resolution. this paper first demonstrates that processing these logs can infer the collocation of 46 students in 34 lectures over 3 months, with a precision of 0.89 and a recall of 0.75. next, we investigate how this wifi-based coarse collocation reflects signals of social interaction. with 163 students in 54 project groups, we find that member performance shows a correlation of 0.75 with performance determined from collocation of groups through 14 weeks. additionally, this paper presents preliminary insights for other campus-centric applications of automatically inferred social interactions. finally, this paper discusses how repurposing archival wifi logs can facilitate applications for other domains like mental wellbeing and physical health. humans are social by nature; their functioning is informed and explained by behaviors that are interlinked with those of others [24] . one of the ways these behaviors manifest is when people in the same physical space take mutually-oriented actions [52] . in situated communities such as college campuses, a student's social interactions with their peers can be important to describe the academic experiences of students, such as motivation [20, 66] , absenteeism [64] , and social isolation [19] . indeed, these interactions also share a relationship with a student's academic performance [19, 20] . as a result, understanding how students socially interact can help campus stakeholders gain valuable insights to support academic outcomes and support flourishing. however, traditional survey based approaches are limited in representing these dynamic behaviors at a fine temporal and spatil grannularity, that is scalable at the community level. this motivates us to gain an objective insight into social interactions using unobtrusive and automated methods that can be practically deployed at scale. it is possible to capture social interactions through devices possessed by the user [14, 43, 44] . however, it is often impractical to gain a comprehensive picture of a large community's social behaviors because these 2 background and related work social interactions are known to be related to behavioral and psychosocial outcomes [63] . this motivates us to study how meaningful signals of social interactions can be inferred by unobtrusively sensing collocation among related individuals. particularly, this paper adopts the definition for social interactions as described by rummel [52] : "...acts, actions, or practices of two or more people mutually oriented towards each other's selves...". these interactions cannot be determined by physical distance alone [52] . people could walk through crowds without socially acting on each other. people could also be sitting quietly in a room but still socially interacting [45] . rummel's definition refers to people regulating their actions based on others sharing mutual intent, with the purpose of shaping their subjective experience [52] . although these interactions can take place digitally, this paper focuses on synchronous social interactions in the physical world, i.e., the people interacting are in proximity. the ubiquitous computing community, along with other hci researchers, has shown a keen interest in understanding wellbeing of students geographically situated on a campus [4, 12, 54, 69, 71, 72] . wellbeing describes an individual's perceptions of satisfaction, fulfillment, and motivation. these experiences are characterized by phenomena like affect, stress, and performance. as an individual outcome, wellbeing strongly interacts with social support [60] . social circles-friends, family, or even work-related groups-can influence the functioning of students at higher-education institutes. one's peer group serves multiple functions of support, such as collaboration on specific tasks or protection from external stressors, even without directly assisting the individual's tasks [8] . therefore, in order to holistically study wellbeing, it is important for the community to explore methods to identify social interactions. specifically, social support for college students has been found to moderate stressful events [57] and is linked with reduced negative affect [19] and depression [20] . therefore, the interactions of an individual with others can facilitate coping with negative events [57] . in fact, these interactions can also be indicative of greater satisfaction [19] and self-actualization [20] . in the context of student life, these factors often contribute to determining student performance. for instance, the lack of social interactions is related to absenteeism [64] , the habitual absence associated with low motivation. at the workplace, the social interactions of an individual also explain their embeddedness, which has been shown to describe the propensity to perform [66] . and more explicitly, interactions foster collaborations, which are positive for performance [3] . therefore, reduced interactions, or a complete lack of them, not only affects a student's mental health but also impact their academic outcomes. traditional methods of evaluating social interactions rely on survey instruments, but these are limited by recall and desirability biases [1, 34] . moreover, self-reports are static assessments, while social interactions are fluid and vary over time [55] . one approach to studying human phenomena by avoiding such biases is with unobtrusive sensing. these automatic methods have the promise of dynamically sensing human behavior without interfering with an individual's natural functioning and are therefore more practical for gathering reliable insights. automatically sensing social interactions have piqued the interest of the community for over a decade. according to lukowicz et al., one of the opportunities that describes socially aware computing is "methods for monitoring and analyzing social interactionsâăťin particular, with respect to long-term interactions and interactions within large communities and organizations" [38] . prior work in this space has fundamentally focused on two approaches, differentiated by the scope of the spaces and people studied. the first set of approaches have focused on studying face-to-face interactions in small spaces [14, 44] . olguín et al., used wearable badges to study how low-level interpersonal interactions are related to workplace performance [44] . although this methodology is precise, it is limited by the cost of instrumentation and by the obtrusiveness of wearing a foreign device. a known variation is to track social interactions using bluetooth sensors embedded in one's smartphone [41, 70] . however, the convenience of installing applications on an existing wearable [27, 62] or phone [10] still requires user adoption and introduces concerns of privacy violations [59] . this challenges the approach of aggregating individualized sensing to determine social interactions. the second approach has largely focused on studying city-scale "flocks" through gps based localization [14, 17, 29, 68] . while this approach scales, it is privacy-invasive because of its continuous data aggregation [35, 59] . however, gps-based localization varies in accuracy for indoor settings [31] . moreover, gaining these insights without requiring client-adoption would require resources that are not readily accessible to a campus community. campus settings require an approach that can infer interactions within buildings while not compromising individual privacy outside their perimeter. this would mitigate the adoption issues of client-side approaches and the oversight challenges of global sensing. as a result, campuses can consider harnessing data already logged through their network infrastructure without needing any active participant effort or involvement. to understand social interactions in a large community, campus stakeholders require methods that can infer such interactions in a dynamic, scalable and reliable way. this has led to some prior explorations to leverage wifi-based technologies for localization and consequently infer social interaction based on collocation. a common technique for localization with wifi is by fingerprinting or trilateration [67] . for instance, hong et al., have shown that wifi based fingerprinting can help identify ties between groups [25] . however, these works are client-side approaches. alternatively, enterprise solutions with fingerprinting and trilateration capabilities at the infrastructure-side have emerged [42, 47] . to infer location, these technologies store the received signal strength indicator (rssi) values for any client-device within a neighborhood of access points (aps). yet, a common form of wifi infrastructure deployment in university campuses [16, 71] only stores association logs describing which ap a client-device is connected to. although it is relatively coarse [39] , this parsimonious representation of location has been exploited to understand individual behavior. ware et al., have inferred student location based on network logs to assist depression screening by inferring individual dwelling behaviors (e.g., duration, entropy, and rhythms) [71] . similarly, eldaw et al., have used unsupervised methods on similar logs to understand how the pattern of student visits can explain the semantic purpose of certain campus spaces [16] . while these works trace individual dwelling patterns across campus, they do not explicitly assess if specific groups of students were interacting. this paper is motivated by these ideas to use network association logs and extend it to identifying periods when multiple individuals are collocated for meaningful social interactions. even though collocation does not necessitate verbal communication in the strict sense, it does serve a social function [45] . olson and olson have described how "spatiality" is fundamental to human collaboration even if individuals do not communicate [45] . therefore, we seek to determine if this collocation-based information can capture the signals of such collaborations or social interactions by studying the performance of project groups. although singular instances of collocation might be contaminated with spurious events of co-presence where individuals did not interact, by gleaning the right information from such data we can predict social-interaction based outcomes. for instance, prior work has shown that the network association patterns can phenotype people into behavioral groups [37] . even other infrastructure-based coarse location technologies, such as bluetooth, have been used to capture social signals like synchrony within group routines [11] . while these studies implicitly associate individuals together (e.g., distinguish students by dining hall), they do not explore explicit interactions in physical spaces sufficiently. a more direct depiction of social interactions was demonstrated by zakaria et al., who leverage a custom system integrated to the campus network infrastructure to monitor groups and subsequently predict stress [72] . however, these systems either rely on additional augmentation of the infrastructure or a "social interaction" occurs when and where individuals mutually orient themselves [52] . this happens in the physical world when two or more individuals are within proximity, or they are collocated [45] . in the scope of this paper, periods when individuals are collocated is the basic unit for which we infer social interactions. this paper seeks to answer if coarse collocation can determine meaningful social interactions. therefore, it is important to establish an automatic approach to compute these collocation sessions. as illustrated in figure 1 , this section describes a pipeline to determine collocation by leveraging wifi network association logs and an evaluation of its reliability (rq1). to build a reliable processing pipeline we need real test data that represents the students on campus. this is to ground the methodology in how the network logs actually depict the behaviors of real students. 3.1.1 sample association logs. as a testing sample, we obtained consent from 46 students at a large public university in the united states, and we then analyzed their wifi association logs. these students belonged to two sections of a group project intensive course. both sections were taught by the same instructor(s) and had attendance data for each lecture. we refer to these sections as "1a" (22 students) and "1b" (24 students) throughout the paper. the instructor for the course provided each consenting student's attendance and group label, along with the course lecture schedule. the instituteâăźs it management facility provided anonymized network log data for these students. this data was accessed at the end of the semester 1 and contains approximately 14 weeks of data, which spans 34 lectures for each section 2 . table 1 shows an ap in room 209 of building 122s. larger rooms, such as lecture halls, have multiple aps to increase coverage. in the logs, these aps are registered with different mac addresses, but associated to the same room. every entry in the log documents an snmp (simple network management protocol) update in the network. its timestamp denotes when a device associates or responds to an snmp poll request. therefore, the log itself indicates that a device is in the vicinity of an ap, but without information of the client rssi this inference has a low spatial resolution. moreover, the snmp update is irregular because it depends on the connected device's response [61] . this is erratic because of variable connectivity settings in the device agent (e.g., the wifi turns off when the screen is locked). the irregularity in log updates leads to a low temporal resolution. the low spatio-temporal resolution is what introduces "coarseness" to this data. outside of the specific association timestamps -when a device responds to a poll or switches aps because it roams -the device is invisible in the logs. because of this, it is non-trivial to determine the location of users between two raw log entries. this section describes a method to use this momentary log information of presence to determine sustained periods of mobility, dwelling, and consequentially collocation. the raw logs are coarse for assessing location because the snmp updates occur either when a device roams or when it responds to a network poll if it happens to be awake. therefore, there is no fixed interval within which a log occurs. to reliably determine if an individual is dwelling, it is important to determine where they are between two updates. specifically, our focus here is identifying when an individual is dwelling in the same room, i.e., in the proximity of the associated ap(s). for this we propose the following approach: (i) determine if an individual is mobile -since we knew the scheduled class time and location for the regular lectures of sections 1a and 1b, we examine the logs accumulated in the thirty minutes before and after figure 2 depicts the instances when a student's device is logged before, during, and after the lecture times, along with the ap that captured the update for a section 1a class held on april 5, 2019. in this analysis, we consider any entry associated with a student because less than 1% of the log entries show concurrent updates at different aps from two or more devices owned by the student. since snmp updates occur when a device roams, we measure the interval between two successive log entries from a user's devices that associate with different aps. based on the 90th quantile of these intervals (233 seconds), to determine if an individual is mobile we establish the largest interval between two successive updates from different aps. (ii) determine if an individual is dwelling in place -based on the criteria for moving, a user is considered stationary at the location of the earlier log entry when the time between two successive updates at different aps exceeds the threshold. as evident in figure 2 , the log updates before and after class times not only are at a different ap (than that of the lecture room), but they also exhibit higher update frequency in shorter intervals. therefore, we consider any time segment when the user is not mobile, to be when they are dwelling. contiguous dwelling segments where the ap does not change are combined to represent longer dwelling segments. figure 3 shows how the raw logs represented in figure 2 can depict moving and stationary time segments. (iii) filtering out disconnection periods -one confound to this method of determining dwelling time segments is that it can erroneously label time periods where a user was disconnected from the network as a period when they were dwelling. consider an individual that moves through network coverage and then exits out. when they are moving from campus out of it, this would be registered as multiple short interval updates for the changing aps till the last ap connection on campus. this will be followed by a large interval till the user returns into coverage area. this large interval needs to be distinguished from legitimate dwelling periods to avoid false positives. based on the class dwelling time, we that find the longest interval between two successive log entries of a student present each stack depicts where how many students of section 1b were found to be connected to the lecture room's ap, another ap in the same building, to the campus network, or not connected at all in class was 76 minutes. we use this heuristic as a threshold. with this, we mark any periods of dwelling as disconnected (or inactive) where the log entries are timestamped at intervals longer than the threshold. figure 4 shows that the disconnection periods identified were predominantly on weekends and before or after class times. the previous phase identifies dwelling periods for individuals. this phase identifies periods of collocation based on overlapping periods of dwelling near the same ap (or room). simply considering the overlapping dwelling segments could have breaks when one of the collocated members inadvertently switches from the corresponding ap to a different one and then returns (e.g., participant 2,034 in figure 3 ). this could occur either when they took a break or if they are in place but their device intermittently found a better connection to a different ap. since the aim of obtaining collocation segments is to infer meaningful social interactions, we consider a liberal approach to characterize collocation. this decision aligns with rummel's definition of social interactions, which admits interaction between individuals even when they are not within line of site, because behaviors can still be influenced [52] . when an individual takes a brief break from a meeting, for example, it does not signify the conclusion of social interactions. therefore, instead of dissecting the collocation periods around such short lived absences, these gaps in the segments are bridged. in particular, these gaps are characterized by: (i) common members of a group between the collocation periods adjacent to the gap; and (ii) the gap containing a collocation or dwelling segment with a subset of those members. after identifying such overlapping segments, we first find the median duration of these gaps. the median in our data for such occurrences was 11m 7s. any gaps less than this threshold are resolved by considering all members to be collocated throughout, including the break period. to quantify the reliability of this coarse localization and collocation technique, we evaluate the attendance of 46 students in 2 sections for the 34 lectures that occurred in the sample data period. each section had 3 classes a week and the two sections met in different buildings on campus. the instructors provided us with lecture-by-lecture records of each consenting student's attendance for both sections. we use this as the ground truth to evaluate the reliability of our proposed automated method. missing data. first, we would like to address the missing data problem. on certain lecture days, we did not find any entry for some students (including a 30 minute margin before or after). the red stacks in figure 5 show the number of students per lecture with no log entries for section 1b. on comparing this to the attendance records, we learn that 93% of the times a student does not appear in the logs, they are actually present. one possibility is that the student either had all their devices turned off or connected to a different network (e.g,. cellular data). every student in the sample had no wifi log entries on at least one lecture they attended (the median was five lectures). therefore, despite its pervasiveness, leveraging the managed network will still miss out on students who were actually present. this is irrespective of the technique applied or sophistication of logs as it is dependent on the client-side behavior. for such occurrences, the automated method cannot ascertain if a student was present or absent. as result, we exclude these student records (for that lecture) from further analysis. accuracy. we analyze the accuracy by considering every student who connected to the network during the lecture time for each lecture. we consider a student to be in class, if any time during class they were collocated with their peers. for the lecture illustrated in figure 3 this refers to the green segments. for every lecture this technique identifies a student to be in class, they were actually present 89% of the time-precision. we speculate, the false positives that emerge could be the result of a student missing the attendance sign-up sheet. alternatively, for every instance when the student was present, this method infers them to be collocated 75% of the time-recall. this implies the high false negatives, as shown in figure 6 . a false negative could occur when a student's device connects to a different ap on the network. figure 3 denotes these as the orange segments. a device could connect to an ap that is physically further away because the signal from their closest wifi was attenuated [31] . therefore, this uncertainty in location could lead to missing out on students that were actually present. although this might underestimate possible social interactions that took place, it motivates us to see what we can meaningfully learn from the interactions that we do detect correctly. the previous section describes how raw wifi network logs can be processed to detect the collocation of students in a lecture room. this was validated with attendance records. however, attendance only represents occupancy and not necessarily social interactions [52] . since social interactions are known to be related to several aspects of wellbeing [63] , it is important to learn if collocation-detected by repurposing network logs-can be used to infer these interactions (rq2). this section presents a case study to explore how the collocation of project teams outside scheduled lectures can represent social interactions. specifically, social interactions in teams is known to affect performance [3] . this encourages us to investigate the relationship between a group member's' performance and how they collocate with other group members (such as time invested in meetings, the regularity of group activities, and the locations of these meetings). this case study demonstrates the feasibility of leveraging raw logs for one specific application that involves social interactions-predicting the performance in project teams. in this way, it answers our research question-to what extent can wifi-based coarse collocation represent meaningful signals of social interactions? 4.1.1 participants. the participants were enrolled in an undergraduate design course for cs students. the course is offered every semester and is a two-semester sequence typically taken by students in their junior (3rd) year. students in this course are expected to work with a team of four to six students over two semesters (part 1 and part 2) on a single design project. in spring 2019, this course had four sections for part 1 and five sections for part 2. each section had an enrollment of about 40 students. in terms of course structure, part 1 involves both lectures as well as project milestones. in contrast, part 2 has fewer lectures and expects students to allocate scheduled class-times for project-related efforts. students in both parts are expected to collaborate on project work outside scheduled lectures. it is not generally known how often student teams meet outside of class, nor is it known how much those meetings impact team performance. the data used in the previous analysis was from sections of this same course but had lecture wise attendance records (section 3). recruitment. the recruitment took place in spring 2019 in collaboration with the course instructors. recruitment was carried out in april by physically advertising the study during the lectures and online outreach through the instructors. in addition, a large number of the students were recruited during the final demonstration expo that is attended by students of both parts. on enrolling, participants provided consent for the researchers to access their wifi ap log data as well as their course data. the participants were assured that this is retrospective data that is already archived and the insights of our study would not impact their course outcomes. during enrollment, participants also completed an entry survey where they reported their group id along with describing when, where, and how often they interacted with their group members face-to-face for class purposes. participants were remunerated with a $5 giftcard for enrolling. in total we received consent from 186 students ( table 2) . of these, 170 students were in the age of 18-24 years, and 16 were of age 25 and above. among these students, 59 reported female (32%) 3 . privacy. participant privacy was a key concern for the research team given the nature of data being requested. the two core streams of data, course outcomes and wifi ap logs, are both de-identified and stored in secured databases and servers which were physically located in the researchers' institute and had limited access privileges. the study and safeguards were approved by the institutional review board of the authors' institution. the course related data of the consenting students was requested from the different instructors after grading for the semester was completed. we obtained data for the remaining 186 students along with course lecture times (table 2 ). among them, 23 students did not have any other member from their group in our study and thus were dropped from this analysis. this leaves us with 163 students and 54 groups (figure 7) . final score. all instructors provided the final score of students in their section. this represents a numerical score between 0 and 100 that is incorporated into the instructor's grading scheme to assign a letter grade for the course. this final score is dominated by the project but students are assessed individually. these variations are introduced by participation as well as the instructor's subjective assessment of team feedback. this study uses this final score to represent a student's academic performance. peer evaluation. given the group project oriented nature of the course and our interest in studying social interactions in groups, students completed a fairly extensive peer-evaluation battery. this battery was completed by the students at the end of the semester and it captures their perceptions of conflict, satisfaction, and security with the team [15, 28, 65] . it can also assess behaviors like collaboration, contribution, and feedback [37] . in essence, this battery evaluates an individual's experience interacting with their team members. prior work shows that these instruments quantify aspects of social interactions that relate to performance [15, 28, 37, 65] . therefore, we use a participant's responses to these surveys as a gold-standard to predict grades. we feed these responses into a model to compare against the predictions of models trained on automatically inferred behaviors. table 3 summarizes the distribution of scores for each peer-evaluation survey instrument. specifically, the peer-evaluation battery contained the following validated survey instruments: • team conflict -conflict represents the perception of incompatible goals or beliefs between individuals that cannot be trivially reconciled. this battery contains three scales, "task conflict", "process conflict", and "relationship conflict". jehn and mannix have shown that low-levels of process and relationship conflict along with moderate levels of task conflict are optimal conditions to maximize team performance [28] . • team satisfaction -satisfaction reflects the contentment of an individual with their personal situation in terms of their expectations. van der vegt et al., have shown that team satisfaction is associated with interdependence among team members which is indicative of team performance [65] . • psychological safety -this construct captures a "shared belief held by members of a team that the team is safe for interpersonal risk taking " [15] . edmondson has shown that it is associated with both learning progress as well as team performance [15] . • team member effectiveness -this measure encompasses five dimensions: (i) contributing to the project; (ii) interacting with collaborators; (iii) monitoring progress and providing feedback; (iv) expecting quality; and (v) relevant knowledge and skills [37] . these dimensions characterize behaviors related to "team member effectiveness", which is theoretically related to team performance [37] . the wifi access point log data for consenting students was obtained from the institute's it management facility. since this data was already aggregated for maintenance and security purposes throughout the semester, we were able to retroactively obtain this information at the end of the semester. the data spans all wifi access logs by connected devices belonging to consenting students. this data is richer compared to the sample data for processing the raw logs into collocation (section 3.1.2). it includes more individuals and a larger set of aps. the data spans a time frame of 95 days between january 1 2019 and april 5 2019. on average, the time between the first log by a participant's devices and the last is approximately 90 days. figure 8 shows the distribution of connected students throughout the semester. the logs in this study include 204 unique buildings with 4,865 unique aps. we also find multiple aps to be in the same room for 803 rooms. additionally, the 204 buildings were manually categorized to best express the purpose of that space [16, 70] -for example, "academic", "dining", "green spaces", "recreation", and "residential". two researchers referred to campus resources to independently assigned categories to these buildings. only two of the building labels had a disagreement, which was resolved by a third researcher. the raw logs of the consenting students was processed with the technique described in section 3.3 to obtain periods when students in the dataset were collocated. the median time spent collocating with other students in the dataset was found to be about 70hrs. the low spatial resolution of the collocation makes it insufficient to assert from single instances if individuals were interacting during a session in which they were in proximity. however, processing multiple collocation periods over the semester can represent behaviors that indicate these social interactions. for instance, members of the same group are expected to be collocated on a regular basis at a specific type of building. therefore, it is important to engineer features that can represent these behaviors. this phase extracts relevant information at a week-level based on various behaviors labelled semantically with the help of manually annotated or retrieved data (e.g, building categories, group meeting/lecture schedules). we segregat features by "individual" and "group" in order to capture different behavioral signals. the former is meant to characterize individual behaviors which are not explicitly associated with social interactions. the latter captures the behaviors of individuals that are oriented towards their group, such as time spent collocated with other group members. the dissociation between these features is meant to distinguish the explanatory power of the group behaviors from individual ones. this helps provide discriminant validity that coarse collocation-based features indeed captures social interactions and is not confounded by an individual's general behavior, such as the time spent at academic spaces. table 4 summarizes the different features we extracted at a week level. we derive the individual features based on the lecture schedule and semantic labels for buildings. to craft the group features, we use the same information but compute them as both absolute duration and a relative percentage. the former denotes how much time a student spent collocated with their group (at least one other member). the latter describes this behavior relative to the total time spent by that group together to express what portion of time a student participated. in comparison to the individual feature, the group features are crafted to consider when the behavior occurred (1) scheduled: groups reported their regular meetings in a free-form response field during enrollment (section 4.1.1). the meeting locations reported were at a building resolution and respondents typically indicated a primary building (e.g., learning commons) along with a potential backup (e.g., library). however, teams also expressed meetings could take place at undetermined locations on campus. moreover, groups often provided multiple potential meeting times and places for a week. to accommodate all possibilities, this feature captures the collocations between group members that occurred during any of the reported periods. (2) class: this segregates collocations with group members during class times. this distinguishes itself from the attendance feature by considering periods of collocation even outside the assigned lecture room. for instance in the case of part 2 sections, the students were expected to meet among themselves during class time. and based on student reports, teams did not necessarily use all class times in a week for meetings. this feature represents this set of behaviors. (3) other: this is a catch-all bucket to capture all other ad-hoc collocations. only 4 groups in our study reported interacting with group members for non-academic reasons (e.g., "lived together"). therefore, this category can be considered to indicate impromptu interactions motivated by course milestones or other classes in which group members are together. processing. the phase processes these raw week-level features into aggregate features that describe their collocation behavior. all the raw features we extract (table 4 ) from the data are computed at a week-level for 14 weeks-5 × 14 for individual features and (9 × 2) × 14 for group features. this leads to a rather large feature space given the target variable was the final score obtained at the end of the semester. therefore, in order to reduce the feature space we calculate summary features to describe the entire semester of the individual. specifically for each feature extracted at a week level, we compute the median, the mean and the standard deviation for the study period. these are moment statistics that quantitatively depict the distribution of that feature throughout the study period. in addition to these we also compute the approximate entropy of the feature per individual [48] . this statistic is a measure of the regularity of that feature for every individual. this reduces the overall feature count to 20 and 72 for individual and group features respectively. to predict the academic scores, we build multiple models to investigate how the collocation-based features can predict the final scores in comparison to survey-based peer evaluation scores. since the final score is a continuous integer value, we estimate it using regression. social interactions are known to be related to performance within teams [19, 20] . this motivates our analysis to demonstrate the relationship between performance and collocation behaviors to explain the extent to which coarse collocation reflects meaningful signals of social interaction (rq2). m p e denotes the model trained on peer-evaluation scores (section 4.1.2) based on the self-reported survey responses provided by the instructors. this model illustrates the efficacy of peer-evaluation reports in explaining performance and serves as a benchmark because these constructs have been validated to be associated with performance [15, 28, 37, 65] . m iw f refers to the model trained on individual features and therefore is independent of the participant's group. m дw f describes the model trained only on group features based on coarse collocation and is therefore potentially representative of social interactions. on comparing these models to a specific subset of features (individual or group), it is possible to assess the discriminant validity in predicting final course scores with each subset without confounding interaction effects from other features. we evaluate all models through a 5-fold cross-validation process. to estimate the target variable (the final score), for each model described, we train with different estimators to account for variations in the data. particularly, we train a linear regressor [56] to represent linear relationships between variables and a decision tree regressor [53] for non-linear relationships. additionally, we also train a random forest regressor [36] , i.e., an ensemble method and thus more sophisticated learner. first, we compare these estimators the basis of the rmse (root mean square error) [7] of the predictions. then, to determine the predictive utility of the we measure the correlation between the predicted value and the actual values. for internal validation we compare these models to a rudimentary baseline m 0 , which always predicts the median of the target variable from the training set. the transformations are needed to solve problems with missing data and to scale the features to comparable units. both the transformations and selections take place within each fold and therefore we perform all fitting only with the training data of that fold: (1) scaling final scores by instructor -the target variable the models are trying to predict is the final score for the course. since the final score varies based on the instructor, we standardize the final scores based on the distribution of scores for each instructor in the training data. (2) impute missing data -for a few individuals certain features might have missing values. in case of peerevaluations this could be because the student did not complete a particular survey instrument. for some of the group features a project team did report their scheduled meeting times (in total seven students). we impute these missing values with the mean. (3) standardize the features -we convert all features to zero mean and unit variance [33] . (4) mutual information regression -lastly, we employ a univariate feature selection method on the basis of mutual information between the training features and the target variable [32] . the number of features selected varies from 1 to k, where k is the total number of features in the model. we select the k that minimizes the rmse. the choice of k is illustrated in figure 9 . to re-emphasize, these results will show if the collocation behaviors of students can predict their group-based performance. since team performance is linked to social interactions in physically collocated teams [45] , these results can delineate if patterns in coarse collocation can infer social interactions. this section describes the prediction results of the various models (described in section 4.3.1) to explain the explanatory power of the coarse collocation features. table 5 summarizes the results of the predictions with the best estimator for each model. the rmse of m 0 -the arbitrary regressor -establishes a baseline to determine the goodness of the models we analyze. any features that do not reduce the rmse in comparison are not noticeably better than predicting the mean of the distribution. to compare the information in the features, we select only the best estimator algorithm (based on reduced rmse) for each set of features. the rmse of m 0 was 1.075, which can be interpreted as 1.075 standard deviation away from the true value. the smallest rmse of m p e was 1.085 and that for m iw f was 1.068. for both of these input features, linear regression was the best estimator. therefore, the modeling the peer-evaluation features shows no improvement in error-reduction and the predicted results have an insignificant correlation with the actual scores. in contrast, m iw f only produces a relatively better, yet still weak correlation of 0.12. this implies that merely processing the individual dwelling behaviors is more informative than peer-evaluation responses to predict scores in a group-project intensive course. in comparison, the best rmse for m дw f is 0.748, which uses random forest (a 30% improvement). moreover, the predicted values exhibit a correlation of 0.759 with the true values. figure 10 shows the correlation between predicted and actual final scores variables. the results show that the model trained on students' collocation behaviors (m дw f ) outperforms other models in predicting their final scores, in comparison to models trained on peer-evaluation responses and individual behaviors. first, we find the collocation-based behaviors are significantly better predictors than peer-evaluation scores (m p e ). while peer evaluation scores are expected to yield better correlations [15, 28, 37, 65] , the social desirability bias in manually reporting team experiences can wash out the intricacies of actual team behavior [1, 34] . these surveys expect the participants to subjectively interpret and then transform their social interactions into scores. but these students are also aware that these scores might affect the instructor's impression of their team members and possibly their score. in contrast, m дw f incorporated multiple characteristics of the collocation behavior within groups over multiple weeks. these features are devoid of the subjective biases that plague self-report and other manual assessments of social interaction. second, we find that m дw f performs better than a model built on individual behaviors (m iw f ). note that m iw f was also found to be a better model than the peer-evaluations. this already implies that dynamic offline behaviors carry explanatory power to determine academic performance. however, given the collaboration-based nature of the course in determining the final score of an individual, m iw f falls short of m дw f . for instance the individual behaviors of attendance or dwelling in academic spaces were not comparable in explaining the final score. therefore, this result indicates that features in m дw f reflect more than dwelling around an ap. arguably, in a snapshot of time, individuals could be collocated and not socially interact with each other [52] . however, by observing these behaviors over a period of time it might reveal if certain individuals are mutually oriented when in the same physical space [45, 52] . the fact that the collocation based model (m дw f ) predicts the final score better than the dwelling only model (m iw f ) prides evidence that this data can indicate social interaction. finally, to further dissect the model and understand how the collocation-based features explain the final score, we evaluate the feature importance of the selected variables. table 6 shows the top five features in the best model, m дw f with random forest. it is notable that four of these capture relative behaviors (e.g., percentage of time students were present in group meetings). another noticeable aspect is that three of these features are based on the variance in collocations. these features essentially describe the consistency in collocation patterns (e.g., being collocated with the same group every week for a fixed period of time). however, given the importance scores it is evident that not one collocation behavior alone but analyzing a set of them together is what enables a strong prediction of performance. we demonstrate that collocation behaviors of group members predict performance better than individual behaviors and peer-evaluations. this exhibits that our method of determining collocation can be viably used to infer social interactions. the previous two sections demonstrate that infrastructure-based coarse collocation captures some important aspects of social behaviors in students, and that these measures of behavior form a good predictor for group project success. this presents new opportunities to harness archival network logs to scale analyses of social behaviors for larger groups, up to and including an entire campus. for instance, with an empirical understanding of how successful teams socially interact, instructors can tailor recommendations for project courses. this section illustrates other potential use-cases based on insights gathered from the case study. social interactions are crucial to understand the physical and mental wellbeing of an individual. the lack of social interaction, or social isolation, is linked to stress, negative affect, depression and dissatisfaction [19, 20, 57] . the presence of social interactions can uncover ties within a peer group, which can be helpful to understand individual perception and behavior. for instance, weak social ties are related to a lack of motivation to go to work or absenteeism [64] . alternatively, strong social ties can also explain peer influence and its effect on alcohol, drug and tobacco use [9] . as a result, retrospective investigations of who a student interacts with and how these interactions evolve over a period of time can provide insights for supporting student wellbeing. using the data from section 4, we develop network graphs representing the collocation between pairs of individuals in the same section, shown in figure 11 . we compute the graphs week-wise (excluding lecture times), with each edge depicting the duration of collocation between any pair of individuals. figure 11 shows the evolution of these interaction ties between participants from section 1b. every node represents a participant and the color denotes project groups. nodes that are closer illustrate individuals who spent more time collocated together. even in the first week of class, the participants are somewhat closer to their group members than others, and this becomes more pronounced in the later weeks. however, since we consider all collocations it is evident that a participant's ties are not exclusive to their group. this makes automated methods even more valuable so as to identify these informal interactions. in some cases, members of different groups might be connected through other courses or even social circles. similarly, figure 12 represents interaction ties in section 2d aggregated over the whole study period. it reveals the tight connection amongst some members of the red group. this particularly refers to participants 2006, 2009, and 2010. in comparison, two other group members are slightly further apart. this is explained by the disclosure during enrollment that these participants live together. this application is similar to what hong et al., describe as relationship maps to understand intimacy between individuals [25] . by using raw association logs for an entire campus, it presents opportunities to understand where these interactions occur and semantically labels them based on the expected purpose of collocation (e.g., residential ties for roommates, academic ties for project teams, or recreational ties for parties). note that our sample is limited to the 215 individuals that consented to the use of their network logs. more nuanced automated learning techniques can help identify social groups in a completely unsupervised way, making a compelling argument for the application to a broader cohort on campus. accordingly, stakeholders can take actions for a cohort of individuals based on noticeable anomalies in a network, such as when someone from a cohort drops out. another use case is when someone in a network is affected by an ailment-mental (e.g., violent incident) or physical (e.g., contagious disease)-the campus can react by securing the peers first. another approach to visually analyzing the segments identified by the pipeline is to observe where social interactions occur. this can be further expanded by locating the "collocation spaces", which are spots on campus where individuals gather together. figure 13 shows how the collocation changes before the midterm, during the midterm, after the midterm. while there is a slight increase in collocation period at academic spaces towards the end of the semester, it is most notable how the interactions in residential spaces change during the midterm. this excessive collocation could indicate groups working together to collaborate on midterm milestones or simply study for exams. another notable aspect is the decrease of collocation duration at fraternities and sororities. this might be related to fewer parties and gatherings during midterm exams. more importantly, this data provides evidence that social interactions vary over time at different places. by investigating how collocation varies around specific events, say an exam, it is possible to define the purpose of spaces more dynamically. furthermore, this can be extended to gain a better understanding of the purpose of social interactions itself. this would stem from knowledge of the space and community. accordingly administrators could focus on facilitating better interaction in these spaces during times of collaboration. by contrast, they could also choose to carefully regulate the use of these spaces during times of social distancing [40] . social interactions in physical spaces can lead to congestion depending on the purpose and popularity of a space. understanding where these spaces are and when congestion can occur can regulate gatherings for crowdmanagement [17, 68] and even prevent the spread of contagious diseases [40] . therefore, campus stakeholders have an interest in identifying rooms and pathways with high pedestrian concentration. not only could this help pre-determine bottle-necks for flock movement in cases of emergencies, it can also retroactively indicate which spaces were prone to congestion and inadvertent physical contact. this subsection presents some observational evidence from our data showing how sections move in and out of their class room around a lecture. as discussed in section 3.4, one of the instructors (1q) provided us with lecture-by-lecture attendance data (46 students in sections 1a and 1b). for section 1a, lectures took place three times a week from 10:10-11:00, and for section 1b, these lectures took place from 11:15-12:05. the rooms where these lectures took place were both single entry/exit. to gain a better understanding of how congested these points can get, we evaluate the arrival and exit times of the students that attend class. for entering class, we compare the earliest starting time of an individual's collocation with the designated commencement time. similarly for exiting class, we compare the latest ending time with the lecture's expected conclusion time. the median entering time is five minutes and the median exiting time is two minutes ( figure 14) . these values align with the instructors' observation of typical class behavior, i.e., students are typically tardy on entry but are on time for the exit. notably, our analysis considers the only students from the current class and not those who would occupy the same room for the next or previous lecture. however, with richer data, it is possible to understand congestion points not just for class rooms but where any social interactions occur. with these findings, stakeholders could regulate the exit of students even between different rooms to avoid crowding the pathways. these can be proactive interventions to reduce the physical interactions. this work showcases the utility of association logs recorded by managed wifi networks. these logs are archival data that can be easily scaled for every campus community member that is connected to it, if a mechanism to obtain broad informed consent can be devised. beyond the case study presented in this paper, repurposing this data to infer social interaction based on collocation behaviors can inform the design of various applications for different stakeholders. in this section, drawing upon the threads of the other applications discussed above as well as others in the existing literature, we highlight some of the real-world scenarios where this technology can be implemented on the campus. 6.1.1 academic experiences. harnessing data already collected at the infrastructure facilitates long-term analyses of social interactions in a large cohort of students. in section 4.4, we show that modeling the collocation behavior of project group members can convincingly explain their final scores. in other words, signals reflecting social interaction, and its ramifications on academic outcomes, can be captured by understanding collocation. this enables instructors to provide data-driven insights to a new cohort based on actual behaviors of successful teams. however, student experiences are not limited to the classroom. inspecting the collocation patterns inside a campus can help characterize the campus spaces in terms of their social purposes. in fact, it would be possible to trace the campus' evolving "social blueprint" and dynamically approximate the nature of social interactions based on when and where people were collocated (section 5.2). this knowledge could be used to augment the static semantic labels of places. moreover, it can help disentangle social relationships within a community. it is not uncommon for individuals to be collocated in the same space on campus. however, by studying prior collocations and its evolution, it is possible to elucidate if two students were mutually oriented to each other's actions, and thereby socially interacting [52] (section 5.1). for example, ties at the gym are different from ties through parties. processing these logs provides researchers an opportunity to understand social interactions on multiple dimensions which motivate new questions and applications. for instance, how do teams with prior ties work in comparison to teams of strangers [23] , or how different are the social interactions in a new class for a student from a marginalized community [49] . essentially, the network logs are a passive source of information to gather empirical insights of social support. and in a student's life, social support can explain performance [19, 20] , drug use [9] , and even dropping out [22, 49, 64] . 6.1.2 mental wellbeing. the applications related to academic outcomes discussed earlier have implications for a student's mental wellbeing. however, this dedicated section is to describe applications that are agnostic of student success and instead focus on supporting their mental wellbeing requirements. abundant work in psychology and sociology express the importance of receiving social support, and this coping mechanism is fundamental to leading a healthy life. this data makes it possible to evaluate the changing social interactions over time, for both positive and negative outcomes. major events on campus can impact social behaviors linked to mental wellbeing. this could either be a violent incident [5] (e.g., during a shooting) or an enforced lockdown (e.g., during a pandemic). in fact, the absence of interaction can be associated with social isolation, which in turn is related to stress, affect, and depression [19, 20, 57] . although these kinds of analyses might be hard to justify in real-time, post-hoc analysis of these trends can provide insights to support positive trends or mitigate negative ones. for example, with the use of archival data, campus health facilities can incorporate the social interaction information for screenings. ware et al. have already shown that similarly leveraging network logs to infer individual dwelling behaviors (e.g., duration, entropy and rhythms) can assist with depression screening [71] . in the same vein, social behaviors inferred by our method can be analyzed to predict community-scale mental wellbeing concerns and alert campus health facilities. this can help prepare responses following certain expected (e.g., exams) and unexpected (e.g., student death) events on campus. 6.1.3 physical health. social interactions, and the lack thereof, are important behaviors in the context of contagious diseases, something that has become very clear in 2020 with the coronavirus disease (covid19) pandemic that is affecting people globally. literature in epidemiology provides substantial evidence that social distancing helps reduce the spread of influenza [21] and coronaviruses [40] . even though wifi-based collocation is too coarse to determine physical contact at the spatial resolution of 6-10 feet, the behaviors that can be derived from it has applications for both reactive and proactive measures. in terms of the former, similar processing pipelines aid contact tracing by automatically assessing the likelihood of individuals at risk based on the amount of collocation they may have had with a known contagious set of individuals. although this can have false positives, it can still render a risk-based prioritization to help with screening during highly contagious outbreaks, such as was experienced in spring 2020, and we expect it to recur in the coming years. using sophisticated interaction networks as we demonstrate in section 5.1, campus health officials can look at historically-accumulated data to understand which students are associated with infected ones. this can be potentially extended to study multi-hop relationships with greater degrees of separation as well. alternatively, similar data can be leveraged to develop and simulate proactive measures that assist campuses in resuming and continuing safe operations during a period of contagion. by modeling prior data based on congestion and pedestrian traffic (section 5.3), it is possible to determine the specific bottlenecks on campus that should be regulated because of risk through both direct interaction and exposure to contact surfaces (e.g., door handles at exits) [6] . even simpler solutions of applications that depict occupancy of spaces (section 5.2) to students in real-time can help them adopt safer behaviors by avoiding interactions [18] . policies such as instituting one-way walkways, assigned seating in classrooms, hybrid physical-remote class attendance policies to reduce student density in classrooms or other creative measures can be tested to see how much they impact risk of exposure to an individual and an entire campus community. the use of passive sensing technologies captured in the digital infrastructure of a campus can characterize human behavior and holds exciting potential because it can be automated and scaled. this mitigates the limitations of manual sensing such as self-reports of experiences or even requiring all individuals to install and consent to passive sensing on the personal devices. however, since this paper highlights the feasibility of appropriating data archived in existing systems, it also elicits new concerns when considering practical deployments. any ubiquitous technology with the potential of large-scale passive sensing faces privacy concerns [46] . in the scope of our work, the privacy concerns can be related to both the data that is collected (coarse location) as well as what it can infer (interaction with peers) and its eventual implications [35] . from the perspective of data collected, the use of the wifi association logs is more privacy preserving in comparison to installing an application on a client device that accumulates data to a central server. such client-based applications can be perceived as invasive not only because such agents can collect sensitive data-possibly more than what the user is aware of-but also because the aggregation can be continuous and unbounded, e.g., a campus application logs locations even beyond the campus perimeter [59] . on the other hand, infrastructure-based localization is limited only to timestamps of network associations and does not elicit anxieties related to a client-side agent leaking data from other sensors. moreover, these approaches are also localized to the campus. however, automatic computation of where individuals are and who they interact with can be considered sensitive by campus students [50] . therefore, when adapting such approaches to infer interactions, stakeholders need to consider approaches like differential privacy to obfuscate sensitive data [13] . related to the privacy concerns is establishing policy around data access. this paper and prior work showcase the utility of social interactions and how it can be inferred unobtrusively. however, this involves a centralized observer that harnesses location data, and even when anonymous, this can be used to trivially identify individuals [26] . a predator can incisively connect certain dwelling patterns if they choose to, e.g., lecture rooms can reveal a schedule and potentially an individual. to protect against this, more data can be abstracted, i.e., the ap locations can be anonymized as well (while still retaining category, floor and relative information). yet, it still needs to be established which people have the privileges to query for information and what the queries can be. in fact bagdasaryan et al. have proposed a system for managing the privacy of ubiquitous computing systems that limits the use of the data itself [2] . moreover, campuses can adapt existing policies regarding access to student records to protect data related to students' social interactions on campus. finally, we also need to discuss the ethics of such inferences. since the data relies on network association logs, any individual that connects to the network effectively opts-in their data for this analysis. choosing to not connect can be considered an unfair choice that limits a student's right to self-determine [51] . although students already connect to other networks, it is important to understand their rationale in order to weigh the cost and benefits of opting-out of an institute managed network. moreover, even if a student makes that decision, the fact that they will be excluded from social interaction based insights could in itself be unethical. even though, on any given day, 90% of the students in our sample were connected to the network, the students outside coverage will be missed in applications like contact tracing and social isolation. and this missing data can have ramifications for the entire community. even for academic performance, if instructors can use this kind of data for intervening with certain groups during midterms, those that were left out lose the opportunity of improvement. consequently, this raises concerns of fairness and accountability, which need to be considered before incorporating such systems. the most apparent limitation of using these association logs to determine social interaction is its low spatiotemporal resolution. this introduces reasonable uncertainty in determining the exact location of individuals [37, 71] . even with a lack of precision, wifi-based localization does have its advantages. it can be argued that such approaches (e.g., [25] ) provide greater insight into indoor mobility and dwelling than other scalable solutions like gps [17, 29, 68 ]. yet, indoor setups present several challenges that can lead to unexpected device associations [31] . as a result, an individual could be in a room and not be associated with the physically closest ap, but rather another ap node that found a stronger signal to the client. this creates an opportunity to deal with this noise by modeling the probability of displaced connections. individual dwelling and collocation could be described as a probabilistic measure based on their pathway to the location. other pieces of information that could help calibrate the modelling is incorporating the size and configuration of rooms and neighbourhood maps of the aps. furthermore, advanced off-the-shelf methods to study archival data can be developed to make ap nodes aware of other aps visible to a client. these additional pieces of information can still be very valuable without the need of installing applications on user phones or fingerprinting the entire campus. our work shows that collocation behavior over time can indicate social interactions, even if an instance of co-presence between two individuals does not guarantee face-to-face interactions. theoretically, this falls in line with ideas of spatiality [45] -when collaborators are present near each other, they are interacting through observations and increased sense of accountability. however, it is yet to be explored if these notions of will translate to other social relationships. specifically, if the principles of spatiality can be used to identify the social groups in an unsupervised way. social interactions can explain student experiences in terms of stress, motivation and performance. one way these interactions manifest on campus is when students are physically collocated. this paper studied the feasibility of coarse collocation leveraged from wifi network logs to describe social interactions. we established the reliability of computing collocation of students in class. then we demonstrated how collocation behaviors of project team members is related to their performance. additionally we enlisted other opportunities to apply this kind of social interaction data to support the campus community. this paper motivates the use of existing infrastructure data, such as wifi logs, to perform large-scale longitudinal analyses of social interactions on campus to inform applications for academic outcomes, mental wellbeing and physical health. invalidity of true experiments: self-report pretest biases ancile: enhancing privacy for ubiquitous computing with use-based privacy the impact of the âăÿopenâăźworkspace on human collaboration prediction of mood instability with passive sensing school shootings, the media, and public fear: ingredientsfor a moral panic estimating the impact of school closure on influenza transmission from sentinel data root mean square error (rmse) or mean absolute error (mae)?-arguments against avoiding rmse in the literature stress, social support, and the buffering hypothesis from childhood to the later years: pathways of human development groupsense: a lightweight framework for group identification birds of a feather clock together: a study of person-organization fit through latent activity routines unobtrusive assessment of students' emotional engagement during lectures using electrodermal activity sensors the algorithmic foundations of differential privacy reality mining: sensing complex social systems psychological safety and learning behavior in work teams presence analytics: making sense of human social presence within a learning environment citymomentum: an online approach for crowd behavior prediction at a citywide level adaptive human behavior in epidemiological models the factor structure of received social support: dimensionality and the prediction of depression and life satisfaction the relationship of self-actualization to social support, life stress, and adjustment targeted social distancing designs for pandemic influenza influences of self-beliefs, social support, and comfort in the university environment on the academic nonpersistence decisions of american indian undergraduates prior ties and the limits of peer effects on startup team performance social behavior: its elementary forms socialprobe: understanding social interaction through passive wifi monitoring decentralized privacy-preserving proximity tracing classifying social actions with a single accelerometer the dynamic nature of conflict: a longitudinal study of intragroup conflict and group performance participatory sensing: crowdsourcing data from mobile smartphones in urban spaces indoor positioning using gps revisited challenges for social sensing using wifi signals estimating mutual information advanced engineering mathematics determinants of social desirability bias in sensitive surveys: a literature review privacy in ubiquitous computing classification and regression by randomforest development of a theory-based assessment of team member effectiveness from context awareness to socially aware computing enernet: studying the dynamic relationship between building occupancy and energy consumption effectiveness of social distancing strategies for protecting a community from a pandemic with a data driven contact network based on census and real-world mobility data analyzing the longitudinal impact of proximity, location, and personality on smartphone usage accuware wi-fi location monitor extraction of latent patterns and contexts from social honest signals using hierarchical dirichlet processes sensible organizations: technology and methodology for automatically measuring organizational behavior distance matters harnessing smartphone-based digital phenotyping to enhance behavioral and mental health location-based services 4.1 design guide white paper a regularity statistic for medical data analysis correlates of school dropout and absenteeism among adolescent girls from marginalized community in north karnataka, south india student perspectives on digital phenotyping: the acceptability of using smartphone data to assess mental health der wert des privaten understanding conflict and war a survey of decision tree classifier methodology inferring mood instability on social media by leveraging ecological momentary assessments modeling dynamic identities and uncertainty in social interactions: bayesian affect control theory linear regression analysis life stress in various domains and perceived effectiveness of social support a walk on the client side: monitoring enterprise wifi networks using smartphone channel scans four billion little brothers? privacy, mobile phones, and ubiquitous data collection the relationship between social support and subjective well-being across age snmp, snmpv2, snmpv3, and rmon 1 and 2 working-relationship detection from fitbit sensor data social relationships and health: a flashpoint for health policy social support at work and its relationship to absenteeism patterns of interdependence in work teams: a two-level investigation of the relations with job and team satisfaction social embeddedness and job performance of tenured and non-tenured professionals location in ubiquitous computing crowdwatch: pedestrian safety assistance with mobile crowd sensing studentlife: assessing mental health, academic performance and behavioral trends of college students using smartphones smartgpa: how smartphones can assess and predict academic performance of college students large-scale automatic depression screening using meta-data from wifi infrastructure stressmon: scalable detection of perceived stress and depression using passive sensing of changes in work routines and group interactions key: cord-152327-7om38vrd authors: shen, hong; deng, wesley; chattopadhyay, aditi; wu, steven; wang, xu; zhu, haiyi title: value cards: an educational toolkit for teaching social impacts of machine learning through deliberation date: 2020-10-22 journal: nan doi: nan sha: doc_id: 152327 cord_uid: 7om38vrd recently, there have been increasing calls for computer science curricula to complement existing technical training with topics related to fairness, accountability, transparency, and ethics. in this paper, we present value card, an educational toolkit to inform students and practitioners of the social impacts of different machine learning models via deliberation. this paper presents an early use of our approach in a college-level computer science course. through an in-class activity, we report empirical data for the initial effectiveness of our approach. our results suggest that the use of the value cards toolkit can improve students' understanding of both the technical definitions and trade-offs of performance metrics and apply them in real-world contexts, help them recognize the significance of considering diverse social values in the development of deployment of algorithmic systems, and enable them to communicate, negotiate and synthesize the perspectives of diverse stakeholders. our study also demonstrates a number of caveats we need to consider when using the different variants of the value cards toolkit. finally, we discuss the challenges as well as future applications of our approach. machine learning-based decision-making systems have been deployed to address many high-stakes problems in our society, such as college admissions [46] , loan decisions [65] , and child maltreatment prediction [11] , raising many social and ethical concerns. even with the best of intentions, ml development teams -constrained by limited team diversity and lack of necessary training -often struggle to fully comprehend the complicated and nuanced social issues embedded in many contemporary ai systems [18, 31, 62] . the need to teach computer science students -and by extension, future practitioners -the social impacts of the machine learning systems they are going to build has become urgent. as a result, there has been an increase in calls for computer science departments to complement existing technical training with issues related to fairness, accountability, transparency and ethics (fate) [37, 49, 54] . indeed, we have witnessed a growing number of computer science departments seeking to infuse topics related with fate into their curricula [22] . to date, a lot of work that focused on fairness and explainable ai has sought to develop technical solutions in the form of toolkits and systems to help ml practitioners better comprehend, evaluate and debias their machine learning models (e.g., [8, 63] ). however, the growing impact of algorithmic systems in our society has necessitated the need for more research efforts to be devoted to cultivating deeper understanding of the diverse and potentially competing social values embedded in these systems (e.g., [36, 67] ). our work contributes to this line of research. in this work, we present value cards (figure 1 ), a deliberation-driven toolkit for informing computer science students and practitioners -the social impacts of different performance metrics and their potential tradeoffs in machine learning-based decision making systems. instead of viewing human values in ai systems as individual dilemmas that can be calculated as aggregations of individual preferences, we foreground the importance of negotiating social values and making collective decisions via deliberation [53, 67] . the value cards toolkit uses three different artifacts -the model cards, the persona cards, and the checklist cards -to better facilitate and scaffold the deliberation process. in particular, the model cards describe a set of machine learning models, which capture the inherent trade-offs in the design of a machine learning-based decision making system. the persona cards characterize different potential stakeholders and their prioritized values and interests in the system, with each corresponding to a number of specific model cards. the checklist cards enumerate a number of social and technical considerations to better facilitate and scaffold the deliberation process. instructors can adjust and tailor the components as they need for their specific problem domains or settings. in this paper, we document an early use of the value cards toolkit in a college level computer science course. in our class activity, we test four variants of the toolkit: model cards only, model cards + checklist cards, model cards + persona cards, model cards + checklist cards + persona cards. our evaluation shows that, the use of value cards in general improves students' understanding of the technical definitions of and the trade-offs between different performance metrics, and their implications in the context chosen for the study: recidivism prediction. it helps students recognize the importance of considering diverse social values in the development and deployment of machine learning systems. it also enables them to communicate, negotiate, and synthesize the perspectives of diverse stakeholders. our results also reveal a number of pros and cons that need to be considered when using the different variants of the value cards approach. our contributions are three-fold: • first, we introduce a deliberation-driven toolkit -the value cards -to help computer science students and potentially future practitioners better understand the social impacts of machine learning models. • second, we document an early use of our approach in a college-level computer science course and conduct an empirical investigation of the initial effectiveness of our approach in an educational setting. • third, we discuss the challenges of using different variants of the toolkit as well as future applications of our approach. in this section, we outline relevant work in two areas. first, we survey the existing techniques, toolkits and systems related with issues of fate in the fair ml literature, and describe how our work is positioned in this space. next, we present an overview of existing work on educational efforts around teaching fate in computer science, a field that has received increasing attention in facct and describe how our work contributes to this emerging line of research. recently, there have been increasing concerns about algorithmic bias and unfairness in ai systems (see e.g., [4, 6, 21] ). as a result, significant effort in the fair ml community has been devoted to the development of algorithms, toolkits and systems to aid ml development teams in assessing and addressing potential biases (e.g., see [3, 17, 28, 31] ). for example, prospector [34] helped data scientists better visualize and understand how a given feature influences algorithmic prediction. fairvis [13] used a mixed-initiative visual analytics system to help data scientists discover inter-sectional bias. google's people + ai research group (pair) developed the open-source "what-if" tool [63] to help practitioners who are not formally trained in machine learning visualize the effects of fairness metrics. microsoft has also developed the fairlearn toolkit (fairlearn.github.io) based on the work of [1, 2] and ibm has developed the ai fairness 360 toolkit [8] to help ml developers assess and mitigate certain kinds of harms in ml models. more recently, researchers at google developed ml-fairness-gym [19] to simulate the potential long-term impacts of deploying machine learning-based decision systems in social environments, offering ml practitioners the chance to conduct a dynamic analysis instead of a single-step or static one. a different line of research has explored the use of detailed and multidisciplinary documentation techniques to enhance transparency in model and data reporting. for example, the model card approach [40] details information such as the model type, intended use cases, performance characteristics of a trained ml model. datasheets [29] focuses more on clarifying the characteristics of the data feeding into the ml model. drawing from approaches in document collection practices in archives, jo and gebru [32] discuss five data collection and annotation approaches that can inform data collection in sociocultural machine learning systems. in this work, we take a complementary angle, looking at helping computer science students and practitioners better understand the social impacts of machine learning. inspired by value sensitive algorithm design [67] , a design process seeking to better incorporate stakeholder values in the creation of algorithmic systems, we introduce the value cards -a deliberation-driven toolkit to help computer science students and practitioners better comprehend, negotiate and reflect on those diverse and oftentimes competing social values in machine learning-powered algorithmic decision making systems. while teaching ethics and social responsibility has a long history in computer science education (e.g., [39, 43] ), the recent widespread deployment of machine learning-powered algorithmic decision making systems in many high-stakes social domains has lead to a growing attention to issues related with fairness, accountability, transparency and ethics (fate) [37, 49, 54] . indeed, we have witnessed an increasing number of computer science departments seeking to infuse topics related with fate into their curricula. recently, fiesler et al. [22] presented a qualitative analysis of 115 syllabi from university technology ethics courses and noted that there is a great variability in terms of instructors, topics and learning outcomes across different institutions. past research in this space has offered important case studies and techniques, for example, using immersive theater [55] , games [10] , and science fiction [12] for ethics pedagogy in computer science. more recently, reich et al. [50] documented a curricular experiment at stanford university, which combined philosophy, political science, and cs in teaching computer ethics. their found that compared with separating ethics and tech training, students resonated strongly with this multidisciplinary approach. based on their experience in teaching fate/critical data studies (cds) topics at university of sheffield (uk) information school, bates et al. [7] discussed a series of challenges educators face for deeper integration fate topics into the existing curriculum. similarly, saltz et al. [51] specifically reviewed the current state of ml ethics education via an analysis of course syllabi. their analysis demonstrated the need for ethics topics to integrated within existing ml coursework, rather than stand-alone ethics courses. they also discussed a few novel examples of practically teaching ml ethics without sacrificing core course content. our work contributes to this fast growing line of research. in this paper, we developed and evaluated a novel deliberation-driven toolkit -the value cards -to facilitate the education of fate issues within existing cs coursework, rather than as separated ethics classes. our approach strives to complement existing inclass technical training with informed understanding of the social impacts of machine learning-based decision making systems. it has the potential to be adopted in a wide variety of settings. in this section, we describe the general design, rationale and objective of the value cards toolkit. inspired by the envision cards [26] in value sensitive design [25, 27] , we consider the value cards a versatile toolkit that can be used to facilitate the deliberation of different -and often competing -social values embedded in a variety of machine learning-based algorithmic systems. the core of the value cards approach is a deliberation process. instead of viewing human values in ai systems as individual dilemmas that can be calculated as aggregations of individual preferences, we foreground the importance of social values and collective decision making via deliberation [53, 67] . deliberation refers to an approach to politics in which members from the general public are involved in collective decision-making through the exchange of ideas and perspectives via rational discourse [15] . moreover, deliberation and discussion-based approaches have demonstrated benefits for different aspects of student learning, including conceptual learning, especially learning of difficult content [5, 52] , acquisition of argumentative knowledge [58, 60] , and perspective taking [57] . we anticipate that through deliberation, participants may have the opportunity to understand each other's perspectives, challenge one another to think in new ways, and learn from those who are most adversely affected. prior work in cscw and learning sciences has shown the success of various strategies to support effective deliberation and discussion. scripts to scaffold discussion is a notable thread of research [20, 48, 59] , which has demonstrated benefits on student learning in a variety of contexts [5, 52, 58, 60] . for collaborative problemsolving and decision-making, a script is a set of instructions regarding to how the group members should interact and how they should solve the problem together. one categorization of collaborative learning scripts divide them into social scripts, which structure how learners interact with each other, and epistemic scripts, which specify how learners work on a given task [59] . in the design of value cards toolkit, we instantiate the idea of script with three different artifacts, -the model cards, the persona cards, and the checklist cards -to scaffold and facilitate the deliberation process. the model cards describe a set of machine learning models that capture the inherent trade-offs in the development of machine learning application. the persona cards depict different potential stakeholders and their prioritized values and interests. the checklist cards enumerate a number of social and technical considerations to better scaffold the deliberation process. instructors can adjust the components as they need for their specific problem domains or settings. in our design, the persona cards are one form of social script, and both the model cards and the checklist cards are forms of epistemic script. taken as a group, the value cards toolkit is designed specifically to achieve the following three learning objectives. • understand the technical definitions and trade-offs of performance metrics in machine learning, and apply them in real-world contexts. • understand the importance of considering diverse stakeholders' perspectives in the development and deployment of machine learning systems. • being able to communicate, negotiate, and synthesize the perspectives of diverse stakeholders when algorithmic decisions have consequential impacts. first, as past literature [51] has pointed out, current ethics education in ml tend to separate the technical training from the ethical training, here we offer an integrated toolkit to help students understand the technical definitions and trade-offs of performance metrics, and apply them in real-world contexts. in doing so, it also has the potential to bridge the gap between model development and real-world application [56] . second, as yang et al. [64] point out that there is a tendency among practitioners who are not formally trained in ml to overly prioritize accuracy over other system criteria in model building -"many perceived percentage accuracy as a sole measure of performance, thus problematic models proceeded to deployment." the value cards toolkit is designed to tackle this challenge by helping students better understand the significance of considering diverse stakeholders' perspectives in the development and deployment of machine learning systems. third, past literature in hci also suggests that there is a lack of communication skills among developers when facing designrelated tasks, which might result in poorly designed system that can further disadvantage already-marginalized populations [45] . the value cards toolkit also strives to enable computer science students to better communicate, negotiate, and synthesize the perspectives of diverse stakeholders when algorithmic decisions have consequential impacts. we next offer a "proof of concept" example by illustrating each component in the value cards toolkit in the context of recidivism prediction. in this work, we used recidivism prediction as the context for exploring the general research problem of helping computer science students understand the societal aspect of machine learning-based algorithmic systems. recidivism prediction is a high-stakes social context where algorithms have been increasingly deployed to help judges assess the risk of defendants re-offending. as a contentious social domain that involves diverse and competing social and political interests, this topic has also generated one of the most controversial cases so far in the debates around fairness and machine learning [4] . following [66] , we recreated a recidivism prediction tool using a data set provided by propublica [35] . for the purpose of this study, we focused on two demographic groups, i.e., african american and white populations and created a balanced data set, which resulted in a data set of 3,000 defendants (1,500 white defendants and 1,500 african american defendants). inspired by mitchell et al. [40] , the first set of artifacts in our toolkit is the model cards. each model card describes one machine learning model by showing its performance metrics in aggregate and across different demographic groups (see figure 2 for an example). collectively, the model cards capture the trade-offs across different metrics in a machine learning-based decision making system. 1 the aim of the model cards is to both describe the performance metrics and to capture the inherent and often implicit trade-offs across different metrics in a machine learning-based decision making system. optimizing for multiple system criteria is a tricky task: optimizing one criterion often leads to poor performance on others. the use of the model cards is to specifically capture, scaffold and communicate those trade-offs to the readers. in the case of recidivism prediction algorithm, following yu et al. [66] , we selected four sets of performance metrics, namely, accuracy (fraction of correct predictions), false positive rate (fraction of people who are falsely predicted to re-offend, among those who do not re-offend), and false negative rate (fraction of people who are false predicted to not re-offend, among those who re-offend), and the disparities of these three measures between the two demographic groups: african american and white defendants. by adopting the lagrangian-based methods from [1, 33] , we generated a family of predictive models that exhibit a wide range of trade-offs between the four different system criteria outlined above. next, given a family of models, we selected a subset of eight models approximately mapping back to the stakeholder valaues identified in the persona cards, as discussed below. we created eight model cards for these eight models. in addition, we also offered a table summarizing the performance of the eight models to guide the comprehension and discussion ( figure 3 ). we use golden star and red cross to indicate the relative metric performances. we also provide the percentage range of the metric performance and a disclaimer reminding students to always refer back to the model cards for the nuances in each model. the second set of artifacts in the value cards toolkit is the persona cards. the persona cards are a series of cards that characterize different potential stakeholders and their prioritized values and interests in an algorithmic decision making system, with each corresponding to a number of specific model cards. they describe the backgrounds of each potential stakeholder involved in the system, the stakeholders' primary considerations when facing the outcome of the system, and some brief guidelines on how to engage in a productive discussion with other stakeholders (see figure 4 ). the purpose of the persona cards is twofold. first, it offers a compact value overview of a high-stakes algorithmic system by showcasing the key stakeholders and their various perspectives. second, each persona card offers the readers an access to a specific stakeholder's thinking process while the stakeholder is interacting with a machine learning-based decision-making algorithm. we hoped that students would take the persona card as a gateway to develop empathy towards the stakeholders and be ready to engage in the deliberation. in the case of recidivism prediction algorithm, we followed narayanan's stakeholder mapping [42] , dividing potential social groups involved the recidivism prediction case into four different groups of stakeholders: judges, defendants, community members, and fairness advocates, with each prioritizing one performance metric. judges might want to prioritize increasing accuracy when considering the design of a recidivism prediction system. defendants might want to prioritize decreasing false positive rate as they are worried being falsely predicted as "will offend again". community members might want to prioritize decreasing false negative rate as they are mostly concerned about the safety of the community. fairness advocates might want to prioritize decreasing disparity as they want to minimize the differential treatment african american and white defendants. the third set of artifacts in the value cards toolkit is the checklist cards (see figure 5 ). inspired by madaio et al.'s work on co-designing ai fairness checklist with industrial practitioners [38] , we consider the use of the checklist cards as a way to generate productive discussion, formalize ad-hoc processes and empower individual advocates. we derived our checklist cards from previous work [23, 38, 44] , which were all designed as practitioner-facing toolkits to guide the ethical development and deployment of ai systems. using these three checklists as our initial dataset, we performed affinity diagramming [30] iteratively in our research group to cluster similar ideas, identify common themes, and combine different options. our final version of the checklist cards includes three subsets. the first is "understanding societal values in ai, " which offers highlevel points in considering social impacts of ai systems. the second is "identifying stakeholders," which provides a starting point for students to think about who is at risk of experiencing impacts. the third set is "analyzing impacts, " which asks students to identify the type, degree, scale and overall direction of the impacts. we remark that the goal of our study is not to capture all unfairness issues in the domain of recidivism prediction, but to study the initial effectiveness of the value cards as an educational toolkit. due to several limitations in carrying out the activity in a classroom meeting, we choose to focus on a restrictive set of model performance metrics, demographic groups, personas, and their associated priorities. in particular, throughout the development of the persona cards, we are aware that, in the real world, stakeholders have a myriad of identities -race, gender, class, sexuality -that shape their values, political interests, and interactions with a given algorithmic system [61] . however, we consider specifically scaffolding the stakeholder persona necessary in this early use of our approach in a classroom setting, as our student body is relative homogeneous and many might not have sufficient domain knowledge. in our instruction, we remind students that each stakeholder's value offered in the persona cards should only be served as a starting point; and that students are welcome to and encouraged to enrich the background and celebrate stakeholders' intersectionalities. after the class activity, we also conducted an in-class reflection, where we encouraged the students to reflect on issues not captured in the deliberation process. in this section, we describe a case study of using the value cards toolkit in an authentic college classroom to facilitate student learning and deliberation about the social influences of machine learning algorithms. we designed a study in which we ask students to use the value cards toolkit during a class meeting. in order to observe the effects of the persona cards and the checklist cards respectively, we designed four conditions of using different components of the toolkit. we also employ a mixed-methods approach, using both quantitative and qualitative measures to assess the learning outcomes for students. we did the study in a computer science course at carnegie mellon university's human ai interaction class, fall 2020 2 . the goal of the class is to introduce students ways of thinking about how artificial intelligence will and has impacted humans, and teach students approaches to design ai systems that are usable and beneficial to humans and the society. the intended learning goals of the value cards toolkit align very well with the learning objectives of the course, making the course a natural case study candidate for our value cards toolkit. we conducted the study in september 2020, at the beginning of the fall semester, when students were just starting to learn about machine learning concepts and performance metrics. we have acquired irb permission from our institution to perform the study and also obtained students' consent on using their anonymized data. during covid-19, class meetings were held remotely via zoom. students were asked to complete a 5-step activity using the value cards toolkit during one class meeting. all students were assigned to teams and each team was assigned to one of the four conditions. the detailed instructions and the values cards toolkit were given to the students in digital version through canvas [14] . the study has the following five steps: (1) table 1 . we designed and administered four outcome evaluations to assess student learning. this includes a matched pre-and post-quiz, a post-survey, and an open-ended written group proposal. in addition, we anticipate that student learning will be observed in the deliberation process facilitated by the toolkit. we further analyze the student teams' deliberation process to evaluate student learning along all three learning objectives. in this section, we introduce how these four outcome evaluations are designed and implemented. the alignment between the learning objectives and the outcome evaluations is shown in table 2 . 4.3.1 quiz on conceptual understanding. we administered a prequiz and a delayed post-quiz to assess learning objective #1. we gave students a toy dataset and a prediction task and ask students to apply the performance metrics on this dataset. we used the criminal recidivism prediction task in the pre-quiz and the loan application prediction task in the post-quiz. the quiz contained questions about the basic understanding and computation of performance metrics and questions about evaluating the trade-offs between different performance metrics. at the end of the pre-quiz, we also inserted a subjective question asking about students' considerations of performance metrics when designing machine learning algorithms. the pre-and post-quizzes were designed by two experienced instructors of the topic. we pilot tested the quizzes with two students iteratively to improve clarity. student performance data were directly collected via canvas. we designed a postsurvey to evaluate learning objective #2. in the survey, we first included the last subjective question in the pre-quiz again. this allows us to compare whether students' considerations of metrics changed after using the value cards toolkit. we then included 4 likert-scale questions asking students to what extent they understand other stakeholders' perspectives in recividism prediction. we also inserted an open-ended question for students to elaborate on their responses. we included a series of self-evaluation questions and a section on demographics. following the individual post-survey, the instructors hosted a reflection session to collect learning objectives outcome evaluations 1. understand the technical definitions and trade-offs of performance metrics in machine learning, and apply them in real-world contexts. pre-and post-quizzes; deliberation 2. understand the importance of considering diverse stakeholders' perspectives in the development and deployment of machine learning systems. post-survey and reflection; group proposal; deliberation 3. being able to communicate, negotiate, and synthesize the perspectives of diverse stakeholders when algorithmic decisions have consequential impacts. students' takeaways and feedback of the activity via a zoom class meeting. to make sure students offer authentic feedback and to protect student privacy, students' responses from the post-survey and the in-class reflection were collected and recorded anonymously. in order to assess learning objective #3, we asked each team to collaboratively write a group proposal following their deliberation facilitated by the value cards toolkit. we gave students the following instructions: "now your job is to discuss with your teammates on which algorithm you think is producing the best outcomes and would recommend for the policymakers to use. if you're able to agree upon a model, please share your reasons. if you're not able to reach a consensus, please also share your reasons. " student teams wrote the proposals in google docs, and we analyzed the quality of the proposals to evaluate student learning. in addition to the three outcome evaluations we explicitly designed and administered, we also anticipated that student learning could be observed from their group deliberation facilitated by the value cards toolkit. the deliberation process was audio-taped and transcribed. we analyzed the group deliberation to evaluate whether there is evidence of student learning along all three learning objectives. in total, 62 students participated in the study, we removed the data from 6 students who chose to opt-out of the data collection. our final dataset contained 56 responses. it included 55% female participants, 43% male participants, and 2% prefer not to say. 46% of the sample were undergraduate students while 54% of the sample are pursuing a graduate degree. 80.3% of the participants identified as asian, 3.6% as black or african american, 12.5% as white and 3.6% as hispanic, latino/a/x, or spanish origin. the participants ranged in age from 18-44 (64 % 18-24, 32 % 25-34, 4 % 35-44). among the 56 students who participated in the study, two students missed the pre-quiz, and one student missed the post-quiz, leaving us 53 data points to conduct the pre-quiz/post-quiz analysis. all 14 rooms uploaded their discussion audios and group proposals. all students submitted the individual model selection; 55 students submitted their post-survey. we adopted a mixed-methods approach for data analysis. first, we performed a quantitative analysis on students' learning gains from pre-to post-quizzes, their change on metrics consideration after the activity, and their understanding of diverse perspectives as shown by the multiple-choice questions in the post-survey. second, we performed a qualitative analysis on the group deliberation, openended responses in post-survey and in-class reflection, and the group proposals to give a richer and more comprehensive view of student learning and their demonstration of understanding. the main outcome variables are the quiz scores and metrics selections (i.e., their responses to the question of "for any algorithmic decision-making system, what metrics below do you think are the most important ones to consider in tuning the algorithm?"). note that for each participant, we collected their quiz scores and the numbers of the metrics they chose twice, before and after the value card activity. this allows us to evaluate whether the value card activity helped improve students' understanding of the concepts, and reduce the reliance on single performance metrics in the development of machine learning algorithms (learning objective 1 and 2). we conducted paired t test to examine whether the pre-post change were statistically significant. to examine the difference across conditions, we ran regression analysis with the post-activity quiz scores or metrics choices as dependent variable, conditions as independent variables, and pre-activity quiz scores or metrics choices as control variables. analysis. the qualitative dataset used in this study include group proposals, open-ended questions in the postsurvey, in-class reflection and group discussions. the in-class reflection and group discussions were audio-recorded and transcribed using otter.ai [47] . in this case study, our goal is to examine student learning along the three objectives as shown in table 2 . we started from inductive coding [9] to extract codes that show evidence of learning and developed a codebook. in total, we summarized 6 main codes and 38 subcodes. we then categorized the codes into the three learning objectives. in the end we used a deductive coding approach, applying the codebook on the entire dataset to present evidence of learning along the three objectives. our approach builds upon and differs from existing grounded theory [16] and thematic analysis [9] approaches in that the goal of the inductive coding is not to find out emerging themes and summarizing themes, rather to develop a codebook that serves as a vehicle for the subsequent deductive coding to undercover evidence and insights of student learning along the three objectives. in general, our class activity suggests that the use of the value cards toolkit is promising in achieving the three learning objectives outlined in table 2 . however, our results also suggest that there are a number of pros and cons that need to be considered when using the different variants of the value cards approach. below, we report our findings through the quantitative analysis comparing the two quizzes, and the qualitative analysis aggregating the deliberation transcripts, group proposal, in-class reflection and open-ended questions in quizzes and post-survey. we also present students' critiques towards the different variants of the value cards toolkit, especially the persona cards and the checklist cards. we use "r" to represent discussion room; and "p" to identify specific participants. we don't include ids for anonymous surveys and in-class reflection. 5.1.1 students improved their understandings of the technical definitions and trade-offs of performance metrics in machine learning, and were able to apply them in real-world contexts (objective 1). we used pre-and post-quizzes to measure the learning effects on conceptual knowledge. as a reminder to our readers, the pre-quiz (step 1) happened at the very beginning of our in-class activity, after the instructors gave an introduction about the technical definitions in a previous lecture. the post quiz took place 7 days after the activity. the two quizzes are both designed to assess students' in general, students performed better in the post-quiz (avg = 0.84, std = 0.14) than in the first quiz (avg = 0.68, std = 0.18). the difference is significant ( < 0.01 in the paired t-test) (see figure 7 ). this suggests that the use of the value cards toolkit significantly improved students' comprehension of the technical definition and trade-offs of performance metrics. in a traditional machine learning course, the definitions of false negative rate and false positive rate are usually presented by the instructors in a mathematical formalization without engaging students within a specific social context. in our study, the immersive deliberation procedure served as a friendly yet effective environment for students to proactively apply the technical terminologies they've learned into a real-world case study. students helped each other consolidate their understanding of technical ml terms and the metrics' trade-offs. for example, one student in room 5 (checklist only) explained the false positive rate in the context of recidivism prediction as: it's having people who are actually not going to commit another crime test positive. it's like just calling them out as guilty even if they have no plan to commit any crime. (r5, p17) by observing the various metric performances of the eight models we offered, students deepened their understanding of the tradeoff between different metrics. in room 13 (checklist and persona), students had a conversation as below: i mean, i'm taking a higher false positive rate, but it's definitely a big step to lower the false negative rate... towards the end of the deliberation in room 2 (baseline), p6 commented that "i wish we did this part of the exercise before the quiz (step 1) because now i understand everything way better. " 5.1.2 students understood the importance of considering diverse stakeholders' perspectives in the development and deployment of machine learning systems (objective 2). in both the pre-quiz and the post-survey, we asked the students questions regarding the performance metrics they want to prioritize in designing a recidivism prediction algorithm. our quantitative analysis showed that students selected more metrics after the activities (avg = 2.8) compared to before the activities (avg = 2.3); the difference is significant ( = 0.04), which suggested that our activity helped the students better recognize there are diverse social values embedded in the algorithmic system (see figure 7) . in our post-survey, we asked students in all conditions if they understood the perspectives of the judges, defendants, community members, and fairness advocates after the activity, the average score is 5.6 out of 7. we report the variances in perspective taking across different conditions in section 5.2.2. we noticed a large population of our students endorsing the model with the highest overall accuracy at the beginning of the deliberation. however, after the group discussion, they started to recognize that overall accuracy fails to capture the nuance of social consequences the machine learning systems might have. for example, one student commented in the post-survey: (the activity) made me reconsider my previous perspectives that models can be "ranked" on an absolute basis by their accuracy. although accuracy is a great metric, it is extremely general and fails to capture the nuances of societal costs from using a model with high accuracy at all costs. one student commented why they selected more performance metrics in the post-quiz, after hearing everyone's arguments for their point of view, i kind of began to think all of the aspects are very important in tuning an algorithm. the costs are very high no matter what, so it is important to view them all as equally prioritized. another student in room 9 (persona only) also commented referring to the current events happening in the real world, this is why disparity is very important. because if there are big differences between the prediction results in white americans and african americans, it will hurt people, especially recently we have seen much news about discrimination. so i think we cannot hurt people anymore. (r9, p36) in sum, we observed an general pattern emerging in our data across post-survey, discussion and group proposal that students started to recognize the importance of considering diverse social values in building machine learning systems. we further observed that students started to consider, negotiate and reflect on the diverse and competing stakeholders' perspectives in their group discussion. for example, students were able to assertively advocate for a stakeholder's interest, i'm concerned about the safety of my neighborhood. the false-negative rates for those cards you mentioned are really high. it seems like we will disagree with each other. (r8, p29) when facing conflicts, students actively negotiated and compromised in order to reach a common ground. the models are extreme in one way or another. it either has super high accuracy or very low disparity. to satisfy everyone, we need to reach some common ground. we started with the principle that within the united states you are innocent until proven guilty. as such, though we each had varying perspectives, we felt that the false positive rate should be an attribute strongly considered. there were 4 out of 14 teams that did not reach a consensus. even though they failed to achieve consensus, their group proposals reflected their thoughtfulness in the decision making. for example, in room 6 (checklist only), students didn't reach consensus because of the inevitable trade-offs between fpr and fnr. after our overall discussion we came to the conclusion that because of a lack of context, that we had no objective basis to prioritize fns over fps.... accuracy and low disparity between demographics were important to our group. beyond that we all had personal reasons to value low fnr or low fpr, but it's obvious that the systems had some sort of trade-off between the two, thus making fulfilling both minimization implausible. like that, then a false negative is not a very high cost for society to pay in exchange for overall accuracy. (r12, p46) in general, persona cards enabled students to simulate different stakeholders in the discussion, which could potentially scaffold more effective deliberations. as one student commented during the post-survey, i think that discussing the perspective of defendants while trying to choose a model really gave me a moment to step into the shoes of the defendants and understand what the impacts of different decisions could have on them and others. in comparison, teams in non-persona conditions sometimes struggled with identifying stakeholders in discussion. one student in the "checklist only" condition commented in the post-survey: i think in general, we are not given the instructions to think about the aspect of different stakeholders... but if there is a little bit more hint ... right now i was only thinking about race bias and defendants" however, the use of the persona cards also has its limitations. for example, during the post-activity reflection session, when asked whether or not the persona cards helped their discussion, some students commented that they "can't really put myself in my persona's position". some think the design of the persona card is too simplified and is "missing some value". interestingly, several students also mentioned that having a persona card made them more stubborn. one student said, persona card made me defensive. i'm fighting for my persona. i'm not considering other personas' perspectives as much as i would like to. we did observe several overly intense debate among two personas in some of the discussion transcripts. some reasonable approaches to tackle these issues could be presenting the personas narratively in a story instead of in separate cards, or having the students collectively create their own set of persona cards. we suggest additional potential alternative designs of the persona cards in the discussion session. cons of using the checklist cards. the use of the checklist cards also has both positive and negative effects. on the one hand, we noticed positive signs of using the checklist cards in scaffolding the discussion process, in particular, to generate productive discussion around the diverse social values in the development and deployment of machine learning systems. for example, room 4 (checklist only) strictly followed the framework we offered in the checklists card during the deliberation. students in room 4 started with identifying societal values in the context of recidivism prediction, then identified a set relevant stakeholders, and finally analyzed the impacts of different models' outcomes. in their group proposals, they started by mapping out various stakeholders directly or indirectly impacted by the recidivism prediction system, including "releasable criminals," "inmates," "guards," "taxpayers," "politicians," etc. and listed related harms and benefits caused by the system correspondingly. our results also suggest that the use of the checklist cards alone, without the persona cards, might not help students understand the different stakeholders' perspectives. in our post-survey, we asked whether they understood the perspectives of the judges, community members, defendants, and fairness advocates after the activity. students in the "checklist only" condition responded lowest (5.0 out of 7) compared to the other three conditions (5.8 for the "baseline" condition, 5.7 for the "persona only" condition, 6.3 for the "checklist and persona" condition). the difference is significant (p=0.04). one possible explanation is that for the checklist only condition, students spent more time discussing the checklist, and then had less time discussing the perspectives of potential stakeholders. this aligns with the over-scripting [20] literature, which suggests that scripts for collaboration might be ineffective when they disrupt the team's original conversation flows. interestingly, when the checklists and personas were presented to the students simultaneously ("checklist and persona" condition), students' discussions were driven by the personas more than the checklist. specifically, discussions in room 11, 13, and 14 did not refer to the checklists in their discussion at all. room 12 had one student mentioned checklist only for once, to confirm they were in the right track: i think this is a good sign for us to go into part two (on the checklist), which is like, who actually matters, like who the stakeholders are? (r12, p46) during the post-activity reflection session, students mentioned that they didn't check out all the points on the checklists card due to time constraint. one student also complained that the checklists were too verbose, and the team members were too busy arguing for their personas, so they ignored the checklist during the deliberation. one potential way to mitigate this drawback could be revealing the checklist cards in an earlier stage of the activity, so that students have abundant time to read through and internalize the checklists. in this research, we proposed a deliberation-driven toolkit -the value cards -to inform computer science students the social impacts of machine learning-based decision making system. we conducted a classroom activity to evaluate the effectiveness of our approach. the "proof-of-concept" case study demonstrated that the toolkit is promising in helping students understand both the technical definitions and trade-offs of performance metrics and apply them in real-world contexts. it can also help them recognize the significance of considering diverse social values and enable them to communicate, negotiate and synthesize the perspectives of diverse stakeholders in the development and deployment of algorithmic systems. our findings also suggested nuances in using different combinations of the value card set. our results suggest that, although persona cards helped some students quickly gain empathy towards the stakeholders, some other students had difficulty in putting themselves in the persona's positions. persona cards could also make some students more stubborn and less acceptable to other persona's perspectives. checklist can help scaffold the discussion; however, spending too much discussing the checklist itself might disrupt the team's original conversation flow. when the checklists and personas were presented to the students simultaneously, students' discussions were likely to be driven by the personas more than the checklist. in sum, future users might want to tailor the use of the value cards toolkit for their specific settings or domains. interestingly, during the study, we also saw students start to think about things in more ways than what was specifically offered in the toolkit. for example, students started to identity more stakeholders in addition to the ones offered in the persona cards, in r4, p15 commented, "i'm thinking about tax payers, they pay the money to support the system. they're like also indirect stakeholders. " they also started to think about better ways of integrating humans in the loop. for example, in r2, p7 mentioned, "in my head, there will be another layer, you know, you have another person checking this process and the model's job is to make that person feel easier. " they also questioned the source of the biases. in r12, p48 asked: "is it an institutional bias? or is it a bias in the data set? i mean, we have 70% white american in the data set and only 30% african american data points. " as an initial effort, there are a number of limitations of our "proof-of-concept" study that are important to mention. first, in this work, we utilized a top-down approach to create the persona cards, specifically designating four different stakeholders and their competing values, based on the stakeholder mapping offered in [42] . we are aware that this practice might create abstractions and even "objectified assumputions" about the diverse social values involved in the recidivism prediction system [18] . we consider this top-down approach as a necessary starting point to generate productive discussion within a rather homogeneous student body. future work is needed to explore bottom-up approaches in creating the persona cards via real-world community engagement. for example, we can borrow the "create your own" cards practice from the design of the envisioning cards [26] and ask our participants to design and generate their own persona cards. second, it is also important to note that deliberation is just one way of collective decision making. past scholarship has offered important critiques to both the concept of deliberation and its practices (e.g., [24] ). in this work, we used the deliberation process primarily as a way to foreground the social aspects of seemingly neutral technical process of adjusting system criteria in algorithm design. future work is needed to explore alternative methods. for example, mouffe's theories of agnostic pluralism [41] remind us of the importance of contentious expression as an alternative and complement to rational deliberation. third, in this work, we evaluated the initial effectiveness of our approach via a classroom activity. future work is needed to explore the use of the value cards in other educational settings, for example, to complement the existing technical training for ml practitioners in both the public and private sectors. in addition, we can also explore the use of the value cards toolkit in real-world community engagement around algorithmic decision making systems (e.g., in community workshops), with lay people-oriented design of the model cards and checklist cards. fourth, similarly, in this work, our case study explored the design and use of the value cards in the context of recidivism prediction algorithm. future research is needed to replicate and validate our methods and findings in other decision-making contexts. finally, due to the constraints posted by the ongoing covid-19 pandemic, our class activity was held entirely on zoom, which means the value cards were all presented to study participants in the digital formats. this suggests opportunities for future research to test the use of the physical cards in real-world interactions. in this paper, we present the value cards, a deliberation-driven toolkit for informing computer science students and practitioners the social impacts of different performance metrics and their trade-offs. we documented an early use of the value cards in a college-level computer science course and reported the initial effectiveness of our approach. our results suggested that our approach is promising in helping the students comprehend, negotiate and reflect on the social values embedded in machine learning-based algorithmic systems. a reductions approach to fair classification fair regression: quantitative definitions and reduction-based algorithms themis: automatically testing software for discrimination machine bias. propublica friendship, transactive dialogues, and the development of scientific reasoning big data's disparate impact integrating fate/critical data studies into data science curricula: where are we going and how do we get there ai fairness 360: an extensible toolkit for detecting and mitigating algorithmic bias using thematic analysis in psychology the code of ethics quiz show toward algorithmic accountability in public services: a qualitative study of affected community perspectives on algorithmic decision-making in child welfare services how to teach computer ethics through science fiction fairvis: visual analytics for discovering intersectional bias in machine learning canvas. 2020. canvas approaching deliberative democracy: theory and practice constructing grounded theory. sage why is my classifier discriminatory design justice: community-led practices to build the worlds we need fairness is not static: deeper understanding of long term fairness via simulation studies over-scripting cscl: the risks of blending collaborative learning with instructional design automating inequality: how high-tech tools profile, police, and punish the poor what do we teach when we teach tech ethics? a syllabi analysis johns hopkins center for government excellence rethinking the public sphere: a contribution to the critique of acutally existing democracy value-sensitive design the envisioning cards: a toolkit for catalyzing humanistic and technical imaginations value sensitive design and information systems fairness testing: testing software for discrimination universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions improving fairness in machine learning systems: what do industry practitioners need lessons from archives: strategies for collecting sociocultural data in machine learning preventing fairness gerrymandering: auditing and learning for subgroup fairness using visual analytics to interpret predictive machine learning models how we analyzed the compas recidivism algorithm webuildai: participatory framework for algorithmic governance acm taskforce efforts on computing competencies for undergraduate data science curricula co-designing checklists to understand organizational challenges and opportunities around fairness in ai implementing a tenth strand in the cs curriculum model cards for model reporting deliberative democracy or agonistic pluralism? translation tutorial: 21 fairness definitions and their politics social responsibility and computer education united kingdom department of digital culture media and sport computing students' learning difficulties in hci education weapons of math destruction: how big data increases inequality and threatens democracy scripted cooperation in student dyads: a method for analyzing and enhancing academic learning and performance. interaction in cooperative groups: the theoretical anatomy of group learning the ivory tower can't keep ignoring tech teaching computer ethics: a deeply multidisciplinary approach integrating ethics within machine learning courses scripting by assigning roles: does it improve knowledge construction in asynchronous discussion groups? public(s)-in-the-loop": facilitating deliberation of algorithmic decisions in contentious public policy domains tech's ethical 'dark side': harvard, stanford and others want to address it quantified self: an interdisciplinary immersive theater project supporting a collaborative learning environment for cs ethics fairness and accountability design needs for algorithmic support in high-stakes public sector decisionmaking carolyn penstein rosé, and justin reich. 2020. practice-based teacher education with elk: a role-playing simulation for eliciting learner knowledge contrasting explicit and implicit support for transactive exchange in team oriented project based learning epistemic and social scripts in computer-supported collaborative learning learning to argue online: scripted groups surpass individuals (unscripted groups do not) supporting virtual team formation through community-wide deliberation discriminating systems: gender, race and power in ai the what-if tool: interactive probing of machine learning models grounding interactive machine learning tool design in how non-experts actually build models ai helps auto-loan company handle industry's trickiest turn keeping designers in the loop: communicating inherent algorithmic trade-offs across multiple objectives value-sensitive