June 2019, Vol. 11, No. 2 AJHPE 57 Research The National Department of Health (NDoH) makes the following statement regarding the importance of scientific integrity in research projects in its document Ethics in Health Research: Principles, Processes and Structures:[1] ‘The study’s design and methodology are vital for research integrity, regardless of the discipline. Sound design and methodology are likely to result in reliable and valid data and outcomes that address the research objectives. Poor design and inappropriate methods may expose participants to unnecessary risk of harm and burden with little or no compensating benefit in the form of useful knowledge gained.’ An important methodological aspect of any research project is the selection of appropriate participants. The NDoH document states the following in this regard:[1] ‘… recruitment, selection, exclusion and inclusion of participants for research must be just and fair, based on sound scientific and ethical principles. Persons should not be excluded unreasonably or unfairly on the basis of any of the prohibited grounds for discrimination: race, age, sex, sexual orientation, disability, education, religious belief, pregnancy, marital status, ethnic or social origin, conscience, belief or language (s 8 of the Constitution). Similarly, persons should not be unfairly targeted for research merely on the basis of one or other of these grounds.’ At a workshop titled ‘Ethics in educational research’, hosted by the South African Association of Health Educationalists (SAAHE) in Bloemfontein on 16 September 2016, concern was raised regarding the selection of students as research participants, in particular by student researchers. It was suggested that the choice or use of students as research participants could be based on convenience only, rather than on sound scientific or ethical principles. This may be seen as unfairly targeting student populations who are potentially vulnerable in the context of educational research, especially in terms of giving valid informed consent.[2,3] The term ‘over-researched populations’ has often been used in the literature for populations that are recruited for many research projects owing to their geographical location (conveniently close to the researchers), willingness to participate or specific disease profile. By targeting student populations for research participation, such populations could become over-researched. Koen,[4] however, concluded that ‘using the term [over- Background. University of the Free State (UFS), Bloemfontein, South Africa undergraduate medical students perform a research project as part of their training. These projects frequently include students as participants. This could be seen as targeting convenient populations who are potentially vulnerable, raising ethical concerns. Objectives. To review the selection and inclusion of students as research participants in undergraduate medical student projects at the School of Medicine, UFS, 2002 - 2017, to assess ethical conduct. Methods. For this descriptive study all undergraduate medical student projects from 2002 to 2017 were screened for the inclusion of any type of student as participant (458 projects). Information was obtained from research protocols and final reports. Results. Fifty-seven student projects (12.4%; range 0% (2002) - 26.9% (2017)) included students as participants. Participants were mainly undergraduate medical students (50.9% of the 57 projects) or undergraduate residence students (24.6%). In 86.7% of projects with participating medical students, there was evidence of literature or subject motivation for this choice, compared with 42.9% of projects that included undergraduate residence students. Recruitment was mostly done in class (43.4%) by student researchers (84.9%). No incentives for participation were offered (90.6%). Participation generally followed directly after recruitment (58.5%). In 63.2% of projects, anonymous questionnaires were used. Conclusions. The percentage of undergraduate medical student projects that included students as participants increased during the study period, and may necessitate some form of scheduling of researchers’ contact with students. The selection and inclusion of students as research participants appear to be ethically acceptable, with the possible exception of undergraduate residence students. Afr J Health Professions Educ 2019;11(2):57-62. DOI:10.7196/AJHPE.2019.v11i2.1081 The selection and inclusion of students as research participants in undergraduate medical student projects at the School of Medicine, University of the Free State, Bloemfontein, South Africa, 2002 - 2017: An ethical perspective G Joubert,1 BA, MSc; W J Steinberg,2 MB BCh, DTM&H, DPH, Dip Obst (SA), MFamMed, FCFP (SA); L J van der Merwe,3 MB ChB, MMedSc (Chir), DA (SA), PhD (HPE) 1 Department of Biostatistics, Faculty of Health Sciences, University of the Free State, Bloemfontein, South Africa 2 Department of Family Medicine, Faculty of Health Sciences, University of the Free State, Bloemfontein, South Africa 3 Undergraduate Programme Management, School of Clinical Medicine, University of the Free State, Bloemfontein, South Africa Corresponding author: G Joubert (gnbsgj@ufs.ac.za) This open-access article is distributed under Creative Commons licence CC-BY-NC 4.0. mailto:gnbsgj@ufs.ac.za 58 June 2019, Vol. 11, No. 2 AJHPE Research researched] may lead to an obscured understanding of real or perceived ethical transgressions, making it difficult to intervene to address the underlying concerns’. For the purpose of this article, we refer to student populations as being targeted more frequently. During the SAAHE workshop discussion, concerns were also raised regarding practical issues, such as scheduling of researchers’ contact with students as participants, as participation in research projects during classroom contact sessions may affect academic time. At the School of Medicine, University of the Free State (UFS), Bloemfontein, South Africa (SA), undergraduate medical students plan, execute and report on a research project as part of their training (research modules: Epidemiology, Biostatistics and Special Study Module). This is done in small groups under the guidance of a clinical or laboratory study leader after students have received structured training in protocol writing, research ethics and research methodology. It is their first exposure to research, and research outcomes might not be publishable, often due to sample-size constraints. However, sound methodology remains essential, and therefore protocols are required to be approved by the Health Sciences Research Ethics Committee (HSREC) and appropriate authorities. Nonetheless, the concerns raised during the SAAHE workshop discussion regarding the possible targeting of medical student peers or UFS residence students as convenient and accessible target populations for student researchers need to be considered. Such targeting could be inappropriate, as the topic under investigation may be unrelated to medical students (or health sciences students or any student) as study participants. Furthermore, data collection for these projects usually occurs from July to November. The timing and frequency of data collection may place an additional burden on students in an already full and demanding academic programme. Currently, there are no data regarding the extent to which students are approached for participation in research projects at the School of Medicine, UFS. Therefore, as a first step to gain some relevant data, we did an audit of the practices regarding the selection of students as research participants from 2001, when student research projects were introduced into the undergraduate medical programme. By obtaining such information, we could ascertain whether students were exposed to or even encouraged to follow unethical practices in selection of participants during this first experience of the research process. We would also be able to advise staff involved in training and guiding medical students through the research process, as well as the School of Medicine and Faculty of Health Sciences and UFS management regarding research involving students. These findings could assist in developing appropriate guidelines regarding the inclusion of students in research projects in our setting, as has been done in other institutions.[5,6] Objectives The aim of the study was to review the selection and inclusion of students as research participants in undergraduate medical student projects at the School of Medicine, UFS, to assess the ethical conduct with regard to participant selection. For this purpose, the following data were obtained: • number and percentage of projects that included students as participants, and trends over time • a description of students selected, including their academic study year, gender, and other inclusion or exclusion criteria • motivations for the selection of students as participants • participant recruitment procedures, location of participation, data collection procedure followed, type of measuring instruments used and timing of research participation in academic year • nature of consent obtained, use and nature of incentives to encourage participation and response rates achieved. Methods This descriptive study included all undergraduate medical student projects in the research modules from 2002 to 2017. An initial screening regarding the inclusion of any type of student as a participant was done by the first author, based on the project oral presentation programmes. Further information regarding the projects that included student participants was obtained by scrutiny of the research protocols and reports (documents that are archived by the first author as co-module leader of the research modules). Information was noted on data forms, which were compiled based on the objectives of the study. In addition to the research projects that formed part of the undergraduate medical research modules, all other protocols submitted to the HSREC, UFS from 2014 to 2017 were screened for inclusion of student participants. This screening was done by the first author (a member of the HSREC) using the agendas and minutes of HSREC meetings. A pilot study of student projects of 2002 and 2012 was done, and data from these projects were included in the main study. Information from the data forms was analysed by the first author in terms of frequencies and percentages. No names of students (as research participants or researchers), study leaders or departments were noted or reported on. All information was dealt with confidentially, and only summary information was reported. No information (e.g. exact study topic) was divulged that could lead to the identification of a specific project or student research group. Ethical approval The protocol for this study was approved by the HSREC, UFS (ref. no. HSREC 155/2016), the Head of the School of Medicine, the Dean of the Faculty of Health Sciences, the Dean of Student Affairs, and the Vice-Rector Research, UFS. Results Of the 458 undergraduate medical student projects performed from 2002 to 2017, 57 (12.4%) included students as participants. This percentage ranged from 0% (2002) to 26.9% (2017) (Fig. 1). Table 1 outlines details of participants included in these 57 projects. In 4 (7.0%) projects, student records only were used. The student participants were mainly undergraduate medical students (50.9% of the 57 projects), or students staying in UFS residences (24.6%). Both genders were included in most studies (82.5%). Some studies had additional inclusion criteria (24.6%), but few stated exclusion criteria (5.3%). Sampling was frequently done in residence students (78.6%), but seldom in undergraduate medical students (13.3%). In >85% of projects that included undergraduate medical students as participants, there was evidence of literature or subject motivation for this choice. Such evidence was present in fewer than half of the projects that included UFS undergraduate residence students as participants. Table 2 outlines recruitment procedures. Recruitment was mostly done in class or during an academic contact session (43.4%) by the student researchers June 2019, Vol. 11, No. 2 AJHPE 59 Research (84.9%), and no incentives for participation were offered (90.6%). In 43.3% of projects involving students directly, the project study leader was a lecturer of the student participants, but in only 1 of these did the lecturer take part in the recruitment of participants (registrars in the specific case). Participation generally followed directly after recruitment (58.5%). Table 3 summarises details of the measuring instruments used. In 61.4% of the projects, only questionnaires were used, which were mainly anonymous (76.6%). Only 4 projects made use of interviews – 3 were structured and 1 was semi- structured. The latter was the only qualitative study performed with student participants. Fourteen projects (24.6%) included medical students in other year groups than participants in the pilot study, and 9 projects (15.8%) included students from other schools in the faculty in the pilot study. Because of the structure of the module, data collection was planned to be performed during specific months in the second half of the year. However, of the 35 projects for which information regarding data collection was provided in the reports, data collection took place later than the dates stated in the protocol in 20 (57.1%) projects. As the following quote from a 2006 report shows, some students leave data collection very late: ‘The day the questionnaires were handed out to the final-year medical students was their last academic session of the year and the questionnaire could only be handed out at the end of the 3-hour session. The students showed much reluctance to filling out the questionnaires. Many were also in a hurry and anxious about their then approaching exams; this could have affected the way they answered the questions.’ This study had a response rate of 52.3% of the 128 potential participants. For projects with undergraduate medical students as participants, response rates were usually >75% (except for fifth-year students, who frequently had lower or even poor response rates). For projects with undergraduate residence students as participants, response rates were usually <50%. Table 4 summarises projects with student participants submitted to the HSREC from 2014 to 2017, other than those by undergraduate medical student groups. Students were included as participants in 24 (8%) of the submissions in 2017 and 13 (4%) in 2015, mainly from the School for Allied Health Professions. Discussion Research projects performed by undergraduate medical students often include students as participants. Results from this study showed a rise in the percentage of undergraduate medical student projects at UFS that included students as participants, with the three highest percentages occurring from 2014 to 2017. This increase was also seen in general research project applications reviewed by the HSREC, UFS. Approval by review boards is one method of ensuring ethical research practices. Institutional review boards (IRBs) have varying approaches to the evaluation of protocols and the risks involved in research with students as participants.[8] For an interventional educational trial, for example, only 1 of the 7 IRBs that had to evaluate the protocol required full review, 4 followed an expedited process, and 2 indicated that the protocol required no IRB approval.[8] All projects at the Faculty of Health Sciences, UFS undergo full review by the HSREC, i.e. scrutiny by 2 review - ers and discussion by the full committee. No exceptions are made, regardless of the type of project (e.g. record review v. intervention) or the level of the applicants (undergraduate student v. post-doctorate researcher). All projects have to adhere to all requirements and procedures of the HSREC, such as submission of amendments, progress and final reports. All projects also have to be approved by the relevant institutional authorities, such as university authorities in the case of student participants. The HSREC requires detailed information documents for all projects other than record reviews, outlining, in particular, the voluntary nature of participation and that participants can withdraw at any time without prejudice. Although UFS requires full review of all protocols, other changes have occurred over the period included in this study. In the past, written informed consent was required for the completion of anonymous questionnaires, whereas more recently the HSREC considers completion of the anonymous questionnaire as consent (this needs to be indicated to the participant on the questionnaire). Results from our study showed that in these undergraduate medical student projects using students as research participants, anonymity is frequently ensured, incentives are not abused to encourage participation and consent procedures are followed as stipulated by the appropriate ethics committee. Easy access to convenient populations may lead to populations being targeted for inclusion in research projects. Research topics making use of student participants should be relevant and well motivated. Results from this study demonstrated that literature- or subject-based motivation for the choice of student participants was apparent in 75% of the projects. More than 85% of projects that included undergraduate medical students as participants had a literature or subject motivation for this choice. Furthermore, student researchers are in their second year of study when they perform their projects, which is not the most frequently selected year group to participate. The suspicion that medical students may choose other medical students as participants based on Fig. 1. Percentage of undergraduate medical student projects that included students as participants, annually, 2002 - 2017. 0.0 5.9 3.2 17.1 12.9 14.3 8.8 6.9 7.1 15.2 6.1 12.0 17.9 21.9 14.8 26.9 0.0 5.0 10.0 15.0 20.0 25.0 30.0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 Year Pr o je ct s, % 60 June 2019, Vol. 11, No. 2 AJHPE Research convenience only, therefore currently does not seem to have merit. However, the possibility might exist that research topics are preferentially selected, for which medical students are an appropriate population. The selection of topics in such a way could be considered methodologically unsound[9] and ethically unfair.[7] Walsh[7] pointed out that in medical education research, many more projects are done on easily accessible medical student populations than on less accessible groups doing continuing professional development. Certain groups (and thereby topics) are unfairly not included in the research. In projects with UFS undergraduate residence students as participants, more than half had no motivation for including residence students. The literature used in these projects usually referred to university students, not specifically residence students. Only 2 projects targeted all undergraduates; one used a complex cluster sampling of class venues, the other a methodologically less sound use of volunteers at different sites on campus. Student researchers clearly use (often on the advice of the study or module leader) residence students as a proxy for all undergraduate students owing to the convenience of being able to approach these students fairly Table 1. Details of the type of student participants included in projects (N=57) Student participants n (%) Type of participation Students directly involved as participants 53 (93.0) Student records only used 4 (7.0) Study includes only UFS students Yes 42 (73.7) Types of UFS students included Any undergraduate student 2 (3.8) Residence undergraduate students 14 (24.6) Faculty of Health Sciences undergraduate students 1 (1.8) School of Medicine undergraduate students 29 (50.9) School of Medicine registrars 6 (10.5) School of Medicine undergraduates and registrars 1 (1.8) Residence students 1 (1.8) Any student 2 (3.5) Hostel league rugby players 1 (1.8) Study year of students included UFS undergraduate residence students (n=14) Any year 10 (71.4) First year 4 (28.6) Medical students (n=29)* First year 13 (44.8) Second year 12 (41.4) Third year 13 (44.8) Fourth year 9 (31.0) Fifth year 16 (55.2) Gender Male 2 (3.5) Female 8 (14.0) Male plus female 47 (82.5) Further inclusion criteria specified for the selected student population 14 (24.6) Further exclusion criteria specified for the selected student population 3 (5.3) Type of student 43 (75.4) Undergraduate medical students (n=30) 26 (86.7) UFS undergraduate residence students (n=14) 6 (42.9) Study year (n=38) 29 (76.3) Gender (n=10) 10 (100.0) Sampling done within the selected student population Yes 22 (38.6) Undergraduate medical students (n=30) 4 (13.3) Undergraduate residence students (n=14) 11 (78.6) Type of sampling (n=20) Volunteers 5 (25.0) Convenience 3 (15.0) Some form of random sampling 12 (60.0) UFS = University of the Free State. *Six projects included all year groups, 16 projects included 1 specific year group, the remaining 8 included combinations, e.g. 1 and 5; 4 and 5; 1, 3 and 5. Table 2. Details of recruitment procedures (N=53) Recruitment procedures n (%) Place of recruitment Class only 23 (43.4) Class plus other venues 3 (5.7) Departmental meeting 2 (3.8) Residence house meeting 10 (18.9) Residence house meeting plus letter delivered to residence room 1 (1.9) Not clear 1 (1.9) Not stated 5 (9.4) Team practice 1 (1.9) Questionnaire delivered to room 1 (1.9) Clinical rotation groups 1 (1.9) Specially arranged meeting 2 (3.8) Department 2 (3.8) Various public venues on campus 1 (1.9) Who does participant recruitment? Student researchers only 45 (84.9) Student researchers (for students) and study leader (for registrars) 1 (1.9) Student researchers and class/group leaders 2 (3.8) Student researchers or residence primarius 1 (1.9) Residence committee member 1 (1.9) Not clear 1 (1.9) Not stated 2 (3.8) Incentive offered for participation Yes (sweets, raffle for movie tickets) 3 (5.7) Not stated 2 (3.8) No 48 (90.6) Participation follows immediately after recruitment No 14 (26.4) Unclear 4 (7.5) Yes 31 (58.5) Yes, plus some later opportunity/involvement 4 (7.5) Informed consent Implicit (anonymous questionnaire) 27 (50.9) Unclear 1 (1.9) Written 25 (46.2) June 2019, Vol. 11, No. 2 AJHPE 61 Research easily. This is therefore a group that in our context can be considered as being inappropriately targeted. Recruitment of students as research participants may expose them to coercion. Guidelines for student participation at other institutions,[5,6] and concerns raised regarding medical students being a vulnerable group,[7] are mainly aimed at projects in which staff members (as figures of authority) are the researchers. It is encouraging that student researchers do the recruitment themselves; there is no evidence of abuse of authority by, for example, the study leader doing the recruitment, which could lead to coercion and possible non-voluntary participation. Participating in research projects may place a burden on students who have limited time and who experience severe academic pressure. The timing and duration of participation should therefore be considered when ‘scheduling’ of researchers’ access to students is to be done via, for example, the programme director’s office in the case of medical students as participants. Our results showed that recruitment often takes place in class, with participation mostly following directly after recruitment. The impact of this on academic time might be a hidden risk, which cannot be considered to be a minimal ethical risk.[7] We estimate that participation in student questionnaire surveys takes 30 - 45 minutes, covering the introduction of the project to potential participants and the completion and handing in of the questionnaires. Relevant authorities may need to consider putting specific guidelines in place as part of protocol review processes. The UFS School for Allied Health Professions has decided that research on student participants in the School may no longer be done during class time (personal communication – Executive Committee, School for Allied Health Professions, October 2017). From a practical point, note that data collection often does not occur at the time stipulated in the protocol, which complicates potential scheduling. Class attendance is compulsory in the Faculty of Health Sciences, UFS. It is therefore not surprising that projects with undergraduate medical student participants generally did not make use of any sampling, and generally had good response rates. During the clinical years of training (years 4 and 5), all academic and training activities take place in 5 rotation groups, and students seldom have sessions together as an entire year group. Accessing these clinical students is more cumbersome, and response rates are therefore lower in projects that include clinical students as research participants. Despite fifth-year medical students being less easily accessible, this year group is selected for the largest percentage of projects, thus indicating that convenience only is not the motivating factor in the selection of participants. Projects with undergraduate residence students generally used some form of sampling and had lower response rates – clearly linked to size and ease of access to the populations. Study limitations The results of this study were affected by data not being available in reports (e.g. actual time of data collection) or data not being accessible for a few projects owing to missing records or termination of projects because of students leaving the programme. Data collection did not include information regarding research topics or risk assessment specifically and could be expanded in future studies. For other projects with student participants reviewed by the HSREC, UFS, from 2014 to 2017, only the agendas and minutes of meetings were available as source documents. These documents contained limited detail regarding the projects, and the appropriateness of students as participants could not be ascertained. The brief results about these projects were primarily given to provide some broader context. Recommendations Ongoing vigilance regarding the appropriateness of students as research participants is required. Only by constant scrutiny of one’s practices in this regard can ethical conduct be ensured. A checklist of items such as appropriate motivation for selection of the specific type of participant and avoidance of coercion or undue influence (in this case by lecturers as authority figures) would be of value to these projects specifically and all projects in general. A more comprehensive exploration of risk assessment for students as research participants would provide valuable information for review boards Table 3. Details of measuring instrument used Measuring instrument n (%) Measuring instrument (n=57) Data form only 4 (7.0) Data form plus measurements 2 (3.5) Interview 2 (3.5) Interview plus measurements 2 (3.5) Questionnaire only 35 (61.4) Questionnaire plus measurements 12 (21.0) If questionnaire (n=47) Anonymous 36 (76.6) Table 4. Other projects with student participants reviewed by the Health Sciences Research Ethics Committee, University of the Free State, 2014 - 2017 Type of student participant 2014, n 2015, n 2016, n 2017, n Medical students 2 0 2 5 Registrars 1 1 1 2 Nursing students 3 2 4 1 School for Allied Health Professions students 5 4 6 5 Entire Faculty of Health Sciences 1 2 4 3 Residence students 2 1 1 0 Any University of the Free State student 0 1 0 2 A specific sport 1 1 0 4 A specific limitation 0 1 1 1 A specific subject 0 0 1 1 Total submissions, n (%) 15 (4) 13 (4) 20 (7) 24 (8) 62 June 2019, Vol. 11, No. 2 AJHPE Research and educational authorities in determining guidelines for inclusion of students as research participants. Further studies could be done on student perceptions regarding willingness to participate and vulnerability experienced as research participants, the impact of time demands on their participation, as well as their opinions on the value of the contribution to research. Conclusions There is an increase in the use of students as participants in undergraduate medical student research projects at the School of Medicine, UFS, and therefore this practice needs attention. While the choice of undergraduate medical students as participants is generally well motivated, it appears that residence students may be targeted mainly for convenience. Anonymity is frequently ensured, and incentives are seldom offered for participation. Recruitment takes place mainly by student researchers during classroom contact sessions, with participation following directly. Using students as research participants appears to be ethically justified, but some guidelines on the protection of academic time may be necessary. Further research regarding risk assessment and student perceptions of their experiences as research participants will be valuable. Declaration. None. Acknowledgements. The researchers thank Ms T Mulder, medical editor, School of Medicine, UFS, for technical and editorial preparation of the manuscript. Author contributions. GJ conceived the idea, drafted the initial protocol, collected and analysed the data and wrote the first draft of the manuscript. WJS and LJvdM contributed to writing of the protocol, interpretation of the data and writing of the manuscript. Funding. None. Conflicts of interest. None. 1. National Department of Health. Ethics in Health Research: Principles, Processes and Structures. 2nd ed. Pretoria: NDoH, 2015. 2. Wendler D, Grady C. What should research participants understand to understand they are participants in research? Bioethics 2008;22(4):203-208. https://doi.org/10.1111/j.1467-8519.2008.00632.x 3. Ridley RT. Assuring ethical treatment of students as research participants. J Nurs Educ 2009;48(10):537-541. https://doi.org/10.3928/01484834-20090610-08 4. Koen JL. The ‘over-researched community’: An exploration of stakeholder perceptions and ethical analysis. MA thesis. Durban: University of KwaZulu-Natal, 2010. http://researchspace.ukzn.ac.za/bitstream/ handle/10413/5163/Koen_Jennifer_Leigh_2010.pdf ?sequence=1&isAllowed=y (accessed 21 September 2016). 5. Johns Hopkins University. Policy Concerning the Recruitment and Enrolment of Students in Research Involving Human Subjects. Baltimore, MD: Johns Hopkins University, 2005. 6. University of Colorado Boulder. Students as Research Subjects: How to Avoid Undue Influence and Coercion. Boulder, CO: University of Colorado Boulder, 2017. 7. Walsh K. Medical education research: Is participation fair? Perspect Med Educ 2014;3(5):379-382. https://doi. org/10.1007/s40037-014-0120-5 8. Sarpel U, Hopkins MA, More F, et al. Medical students as human subjects in educational research. Med Educ Online 2013;18:1-6. https://doi.org/10.3402/meo.v18i0.19524 9. Joubert G, Katzenellenbogen JM. Planning a research project. In: Ehrlich R, Joubert G. Epidemiology. A Research Manual for South Africa. 3rd ed. Cape Town: Oxford University Press, 2014:50-57. Accepted 4 December 2018. https://doi.org/10.1111/j.1467-8519.2008.00632.x https://doi.org/10.3928/01484834-20090610-08 http://researchspace.ukzn.ac.za/bitstream/handle/10413/5163/Koen_Jennifer_Leigh_2010.pdf?sequence=1&isAllowed=y http://researchspace.ukzn.ac.za/bitstream/handle/10413/5163/Koen_Jennifer_Leigh_2010.pdf?sequence=1&isAllowed=y https://doi.org/10.1007/s40037-014-0120-5 https://doi.org/10.1007/s40037-014-0120-5 https://doi.org/10.3402/meo.v18i0.19524