Microsoft Word - 27-Sau_36514_reference 243 Original Article Biosci. J., Uberlândia, v. 34, n. 1, p. 243-250, Jan./Feb. 2018 QUALITY INDICATORS APPLIED IN A NURSING CONTINUING EDUCATION PROGRAM OF A HIGH COMPLEXITY UNIVERSITY HOSPITAL FROM BRAZIL: II - SATISFACTION INDICATORS INDICADORES DE QUALIDADE APLICADOS A UM PROGRAMA DE EDUCAÇÃO CONTINUADA DE UM HOSPITAL UNIVERSITÁRIO BRASILEIRO DE ALTA COMPLEXIDADE: II – INDICADORES DE SATISFAÇÃO Eliana Borges Silva PEREIRA1,4; Clesnan MENDES-RODRIGUES1,2,3*; Renata Lemos de SOUSA NETO1; Fabíola Alves GOMES1,2,4; Durval Veloso da SILVA1,5; Arthur Velloso ANTUNES1,2; Rosângela de Oliveira FELICE1,4; Guilherme Silva MENDONÇA1,4; Paula Carolina Bejo WOLKERS1; Teresa Cristina Ferreira ALVES1; Rayany Cristina de SOUZA1,2; Dayane Aparecida Cândida FÉLIX1,2 1. Clinical Hospital of Uberlândia, Federal University of Uberlândia, Uberlândia, MG, Brazil; 2. Nursing, Medicine Faculty, Federal University of Uberlândia, Uberlândia, MG, Brazil; 3. Institute of Biology, Statistics - Mathematics Faculty, Federal University of Uberlândia, Uberlândia, MG, Brazil; 4. Postgraduate Program in Health Sciences from Federal University of Uberlândia; 5. Postgraduate Program in Environmental and Worker Health from Federal University of Uberlândia, Uberlândia, MG, Brazil. * Corresponding author: E-mail: clesnan@hotmail.com ABSTRACT: Although satisfaction is a key point in the evaluation of continuing education, few studies have conducted its assessment, particularly in relation to long-term programmes. The objective of this study was to evaluate the evolution of satisfaction indicators applied to a continuing education program in a university hospital of high complexity. For this, a satisfaction form was filled out after each of the 11 Cycles of Training, and the findings were related to the rates of attendance. It was observed that the nursing professionals gave a positive evaluation of the cycles when each course was evaluated individually, with a tendency to change over time. However, dissatisfaction was observed in relation to the organization criteria for the event, which tended to get worse over time. In addition, different nursing professional categories apparently presented different satisfaction levels, although this association cannot be proven. The results proved that the evaluation of satisfaction with the course or program over time can offer quality indicators for the better management of continuing education programs, and future works should take into account the profile of professionals involved with the program. KEYWORDS: Training. Long-term Course. Continuing Education Evaluation INTRODUCTION Assessment of satisfaction in continuing education programs may be a key point for success, since a significant relationship was found between learning satisfaction and continuing learning intention (WU; HSIEH; LU, 2015; BAGAYOKO et. al., 2013). Positive results of satisfaction have been reported as a result of accessing the needs of the involved audience previously and precisely and the standardization and adoption of modern methods of teaching (FARZIAMPOUR; ERMAMI; ESHRAGHIAN, 2009). Despite this, studies that evaluate satisfaction in long-term events or that occur regularly or for long periods are absent or incipient in literature. Quality indicators in the Brazilian context have been restricted to attendance (eg. MENDONÇA et. al., 2016) and training indicators (eg. OLIVEIRA et. al., 2014). On the other hand, indicators of satisfaction in continuing education have been restricted to a few studies like the assessment of short-term courses (SANTOS et. al., 2015). Yet in literature, the comparison of satisfaction among methods of teaching is common (eg. MISSILDINE et., al., 2013). Studies that evaluate the satisfaction in continuing education programs are necessary to better understand the event organization, course criteria and other aspects of the methodology used. Based on this, the aim of this study was to use satisfaction as a quality indicator to evaluate the efficiency of a Continuing Education Program focused on the nursing staff in a Brazilian University hospital. In addition, this study also checked if different characteristics from the program are related to the satisfaction indicator and how this indicator oscillated with time. Received: 17/01/17 Accepted: 05/12/17 244 Quality indicators ... PEREIRA, E. B. S. et al Biosci. J., Uberlândia, v. 34, n. 1, p. 243-250, Jan./Feb. 2018 MATERIAL AND METHODS Research Field and Training Cycle description The study was conducted at a Clinical Hospital of Uberlândia (CHU, “Hospital de Clínicas de Uberlândia - HCU”) of the Federal University of Uberlândia (“Universidade Federal de Uberlândia”), state of Minas Gerais, Brazil. This study is a sequential study from quality indicators applied to the Training Cycle of the CHU (TCCHU) that is a continuing education strategy for the CHU. The Nursing Direction proposed the TCCHU as a monthly activity, lasting three days, with classes in the morning, afternoon and evening shifts. This allowed the professional nine possibilities of participation in each course. Additional details can be viewed in Mendes-Rodrigues et al. (2018). The study had a management and administrative nature. Data were collected and analyzed to be part of the evaluation of management activities and control of the CHU Nursing Department. The study was approved internally in the CHU. Satisfaction Indicators During the TCCHU, forms were used to evaluate the satisfaction of professionals by using characteristics of the event, as organization, planning, and theme of the course. All professionals received the form at the beginning of each TCCHU. The professional could voluntarily complete the form in its entirety or not. The 2nd to the 12th TCCHU were evaluated. All items from the form were evaluated according to a Likert scale type. The items evaluated for the organization of the event were general content, methodology, materials available, duration, expectations of the event, event contribution, event prior information, general organization, location of the event and coffee break. The evaluation of the courses consisted of an assessment carried out for each course from each TCCHU. The items evaluated were subject knowledge, adopted language, posture and answering questions of the speaker. For the Likert scale, the categories used were: Great, Good, Fair and Poor, except for items Event expectations (categories used: exceeded expectations, met, partially met, not met) and the event contribution (categories used: contributed totally, partially and not contributed). For the analysis of the satisfaction indicator in each TCCHU, we used 60 to 120 randomly selected forms. Only forms with all evaluation items filled out were analyzed, which decreased the sample number in some cycles, initially proposed to be 120 (10% from nursing professionals of CHU). The criterium of using 100% completed forms was adopted because we believe that they presented a better view of the TCCHU, though the availability of these forms decreased over time due to fewer professionals taking the courses (MENDES- RODRIGUES et al., 2018). The final sample size per TCCHU was 120 in 9 cycles, 100 in 1 cycle, and 79 in 1 cycle. Satisfaction indicators were included for 11 cycles, totaling 1,259 forms, and averaging 114 forms per cycle. This satisfaction assessment methodology was not applied in the first TCCHU of 2013. For all items the relative frequency of each item in the scale was calculated. The obtained frequencies were correlated with the temporal order of each cycle. The mean and standard deviation of each category (course or event evaluation) of all 11 TCCHU’s was also calculated from relative frequency. Statistical Analysis In the analysis where the temporal order of cycles was considered, this order was obtained from the sequence of cycles in the two years evaluated, regardless of the interval between one cycle and the next. They were ordered from one to four from the 1st to the 4th TCCHU of 2013 and five to 12 from the 1st to the 8th TCCHU of 2014. This was proposed with the intention of knowing the change in the each metric in function of the sequence of Training Cycles offered, not in function of the time between training cycles. We evaluated if the satisfaction indicators showed linear trend with the sequence of the Cycles. For this, the satisfaction indicators of each TCCHU were correlated with the temporal sequence of the cycles (from the 2nd to the 12th Cycle). The analysis began in the second cycle, since the 1st cycle of 2013 did not use the assessment of indicators of satisfaction. We also tested the linear association of the mean relative frequency of attendance obtained by Mendes-Rodrigues et al. (2018), with the mean values for event and course criteria. In these analyses, we summed the frequencies from the nursing assistants and technicians. The linear relation was tested with Spearman correlation and the coefficient correlation significance was tested with Student’s t-test. No other non-linear relation nor the fitting to linear regression models was tested because of the lack of interest in the prediction results or in the parameter model estimative. Previous analyses (visualization of scatterplots) did 245 Quality indicators ... PEREIRA, E. B. S. et al Biosci. J., Uberlândia, v. 34, n. 1, p. 243-250, Jan./Feb. 2018 not indicate the occurrence of other trends that were not linear in the data. RESULTS Satisfaction Indicators Regarding the criteria related to global content, methodology, materials and time duration the overall evaluation of the event was Good (average between 45 and 60%), followed by Great (average between 26 and 41%), with a few Fair and Bad ratings (average between 0 and 19%) (Figure 1A). Regarding the expectations of the event, a mean of 65.41% answered that expectations were met and only 0.53% claimed that the expectations were not met. The professionals reporting that the event contributed fully or in part to their personal or professional development were 57.54% and 40.31% respectively (Figure 1A). About the planning and organization of the event, most of the evaluations were Good for prior transmission of information and general organization, Great for the location of the event (mean between 21 and 58%), with few evaluations as Poor (mean between 0.08 and 1.81%) (Figure 1B). As an exception, most evaluations of the criterion Coffee Break were concentrated between Good and Fair (41.81 and 25.70%, respectively), with an average of 9.76% as Bad (Figure 1B). 0 10 20 30 40 50 60 70 G re at G oo d F ai r B ad G re at G oo d F ai r B ad G re at G oo d F ai r B ad G re at G oo d F ai r B ad E xc ee de d M et P ar ti al ly m et N ot m et T ot al ly P ar ti al ly N ot c on tr ib ut ed Content Methodology Matherials Time Duration Expectations Envent contibution P er ce n ta ge ( m ea n ± st an d ar d e rr or , % ) A 0 10 20 30 40 50 60 70 G re at G oo d F ai r B ad G re at G oo d F ai r B ad G re at G oo d F ai r B ad G re at G oo d F ai r B ad Prior information General organization Location Coffe Break P er ce n ta ge ( m ea n ± st an d ar d e rr or , % ) Criteria for event planning evaluation B Figure 1. Satisfaction Indicators applied to some event planning items of the Training Cycles of the Clinical Hospital of Uberlândia, as part of a continuing education program from a tertiary university hospital from Brazil, in the period of 2013 and 2014. (A) Content, Methodology, Matherials, Time duration, Expectations and Event Contribution; (B) Prior information, General organization, Location and Coffee Break. 246 Quality indicators ... PEREIRA, E. B. S. et al Biosci. J., Uberlândia, v. 34, n. 1, p. 243-250, Jan./Feb. 2018 For individual criteria related to each courses offered in the Training Cycles, all were mainly evaluated as Great (mean between 50 and 66%) followed by Good (mean between 30 and 41%). The Bad evaluation shows low values and the mean ranged between 0.40% and 0.48% (Figure 2). 0 10 20 30 40 50 60 70 G re at G oo d F ai r B ad G re at G oo d F ai r B ad G re at G oo d F ai r B ad G re at G oo d F ai r B ad G re at G oo d F ai r B ad Choosed theme Theme domain Language adopted Posture Doubts clarification P er ce n ta ge ( m ea n ± st an d ar d e rr or , % ) Criterias for course evaluation Figure 2. Satisfaction Indicators for some criteria of courses offered in the Training Cycles of the Clinical Hospital of Uberlândia, as part of a continuing education program from a tertiary university hospital from Brazil, in the period of 2013 and 2014. Satisfaction indicator over time The evaluation of the tendency of the satisfaction indicators over time, inferred by the temporal sequence of the cycles, showed that the satisfaction indicators oscillated linearly over time for some items (Table 1). About the planning of the event, there was a frequency decrease of Great answers for various items (rs = -0.963 to -0.702, all with P < 0.05), except for time duration and sending prior information; there was also an frequency increase of Good answers for most items (rs = 0. 636 to 0.881, all with P < 0.05), except for materials, duration and Coffee Break. The frequency of Regular answers did not oscillate for most items, except for Coffee Break that increased significantly with time (rs =0.764, P < 0.01). However, the frequency of Bad answers grew only for items Content (rs =0.815, P < 0.01) and materials (rs =0.688, P < 0.05). The average response showed negative correlation for Great (rs =-0.818, P < 0.01) and positive for Poor (rs = 0.682, P < 0.05), showing a migration from Great to Bad answers in the overall assessment of the event. For all items related to the evaluation of each course offered in the TCCHU, there was no linear relation between the frequency and the temporal sequence of the cycles (Table 1) demonstrating the absence of time effect in these indicators, except for the item "Doubts clarification," which had a negative correlation to the item Bad (rs = -0.530, P < 0.05), showing a positive evolution of the item. There was also no correlation to the items "Event Expectations" and "Event Contribution", and in all criteria correlations were not significant. 247 Quality indicators ... PEREIRA, E. B. S. et al Biosci. J., Uberlândia, v. 34, n. 1, p. 243-250, Jan./Feb. 2018 Table 1. Spearman correlation among the temporal sequence of cycles offered by the Training Cycles of Clinical Hospital of Uberlândia and satisfaction indicators applied Event evaluation (d.f. = 11) Likert Scale Great Good Fair Bad Global content -0.702 * 0.733 * 0.575 ns 0.815 ** Methodology -0.564 ns 0.785 ** 0.494 ns 0.013 ns Available materials -0.797* * -0.118 ns 0.511 ns 0.688 * Time duration -0.551 ns 0.597 ns 0.262 ns 0.023 ns Prior information -0.077 ns 0.636 * -0.420 ns 0.347 ns Global organization -0.764 ** 0.778 ** 0.257 ns -0.200 ns Location and equipment -0.851 ** 0.881 ** 0.522 ns -0.229 ns Coffee break -0.963 ** -0.159 ns 0.764 ** 0.598 ns Mean of all event items -0.818 ** 0.573 ns 0.591 ns 0.682 * Course evaluation (d.f. = 33) Great Good Fair Bad Chosen theme -0.088 ns 0.246 ns 0.110 ns 0.138 ns Theme domain -0.240 ns 0.280 ns -0.128 ns -0.007 ns Language adopted 0.005 ns 0.327 ns -0.040 ns -0.343 ns Posture 0.074 ns 0.267 ns 0.101 ns -0.270 ns Doubts clarification 0.195 ns 0.264 ns -0.252 ns -0.530 ** Mean of all course items -0.035 ns 0.314 ns -0.139 ns -0.310 ns Event expectations (d.f. = 11) Exceed Met Partially met Not met -0.173 ns 0.355 ns 0.210 ns 0.231 ns Event contribution (d.f. = 11) Totally Partially Not contributed -0.009 ns 0.492 ns 0.075 ns **: P < 0.01; *: P < 0.05 and ns: P > 0.05 based in Student t test. The mean relative frequency for assistants and technicians was associated with the decrease in the criterium Great (rs = -0.818, P < 0.05) and the increase in the criterium Good (rs = 0.700, P < 0.05) for event evaluation; and also the increase in the criterium Good for course evaluation (rs = 0.642, P < 0.05), showing that with the increase in the presence of these categories the criterium Good showed an increase and the criterium Great showed a decrease. The mean relative frequency of attendance of nurses was related to an increase of the criterion Great for event evaluation (rs = 0.818, P < 0.05) and a decrease of the criterion Good (rs =- 0.700, P < 0.05) and for course evaluation there was a decrease of the criterion Good (rs = -0.642, P < 0.05). The mean relative frequency of attendance of other professionals than nursing was only related to an increase of the criterion Bad for event evaluation (rs = 0.700, P < 0.05). (Table 2). Table 2. Spearman correlation among the mean values from event and course evaluation satisfaction criteria and the mean relative frequency of attendance for different Nursing professional categories in the Training Cycles of Clinical Hospital of Uberlândia Nursing Professional Categories Event evaluation Assistants and Technicians Nurses Others Great -0.818 ** 0.818 ** -0.345 ns Good 0.700 * -0.700 * 0.145 ns Fair 0.291 ns -0.291 ns 0.145 ns Bad 0.464 ns -0.464 ns 0.700 * Course evaluation Assistants and Technicians Nurses Others Great -0.318 ns 0.318 ns 0.573 ns Good 0.642 * -0.642 * 0.041 ns Fair -0.173 ns 0.173 ns -0.327 ns Bad -0.661 * -0.661 * -0.556 ns **: P < 0.01; *: P < 0.05 and ns: P > 0.05 based in Student t test. 248 Quality indicators ... PEREIRA, E. B. S. et al Biosci. J., Uberlândia, v. 34, n. 1, p. 243-250, Jan./Feb. 2018 DISCUSSION Satisfaction Indicators The main dissatisfaction of professionals was related to the organizational aspects of the event. All aspects of the course were highly rated by professionals, showing contentment with the course content or theme. The outdated teaching methods, lack of purpose of the course, inexperienced teachers or even the relevance of the course were not considered impediments to participate in Continuing Education (NI et al., 2014). Apparently, for the population studied here, the fact that a course is offered is more important than "quality or satisfaction" itself with the event, reinforcing that those professionals look for training in the workplace and understand the limitations of this method. Satisfaction Indicators over time Increasing dissatisfaction over time with organizational aspects may be a reflection that, for the professionals, the event was not suited to the needs of the participants over time, although the reasons for this dissatisfaction are still unknown to us. The only item that was not influenced by time was the evaluation of the item "Duration of course", probably because it takes place during working hours with low impact on other personal and professional activities. Expectations about the event and the event's contribution to the development of the professional did not oscillate over time either. Added to the fact that there was overall contentment with the criteria of the courses, this further enhanced the satisfaction of professionals in relation to the expectations and the content of the event, regardless of the theme and or quality of each cycle. The only criterium that decreased over time was the Bad evaluation for the item “Doubts clarification”, which showed a positive result over time, i.e. the clarification of doubts improved over time. These results reinforce the idea that, for professionals, the offering of courses is a more important criterion than the relative quality itself, even if there is dissatisfaction with the organizational aspects as observed in other studies (NI et al., 2014). However, nurse educators are increasingly aware of the importance of measuring the Continuing Education Program results beyond the levels of participation and satisfaction of professionals (ANCC, 2014). Additionally, in Program evaluated here, the attendance indicators for professional categories decreased over time (MENDES-RODRIGUES et al., 2018) and could be a result of the dissatisfaction with the organization of the Program. Our studies did not test this association, since the satisfaction form used here did not assess the professional category of the participant. The association from some satisfaction indicators with attendance indicators could reflect that for nursing assistants and technicians the event is inadequate, but the course is adequate for the necessities of this group. However, for the nurses the event is adequate with an opportunity for continuing education, but the courses are inadequate, since they need more complex approaches. Apparently the nurses could perceive that the TCCHU are more appropriate to other categories. This explains the decrease in attendance of the nurses in this program as observed in the period (MENDES-RODRIGUES et al. 2018). The correlation analyses are not causal. More future researches need to consider that satisfaction depends on nursing categories and could reflect different continuing education necessities. In the TCCHU the professional category is directly associated to attendance indicators (MENDES-RODRIGUES et al. 2018). This relation in a program with obligatory participation in work hours, as the TCCHU, is doubtful, since the participation could be more related to permission by coordinators, sizing or other factors, but this needs additional investigation. The lack of organization and support had little effect on the participation of professionals in educational activities (NI et. al., 2014), reinforcing the absence of association between attendance and satisfaction in an obligatory course during work hours. Results suggest that socioeconomic and demographic factors can affect satisfaction with distance continuing education programs (BYNUM et al., 2010). These factors could be present in the TCCHU too. Apparently in our population, the course content is not affected over time contrary to another study where it is a crucial point for continuing learning intention (WU; HSIEH; LU, 2015). Probably other aspects in the program are related to the decrease in attendance. To improve the satisfaction in continuing education programs it is necessary for the public to offer feedback. There is a need to adjust the topics of the courses, the organizational structure and the whole process of the program in light of this feedback. Experience has shown that such feedback is effective in improving satisfaction indicators (GERCENSHTEIN; FOGELMAN; YAPHE, 2002) and that the prior survey of the needs of participants is directly associated with the level of satisfaction 249 Quality indicators ... PEREIRA, E. B. S. et al Biosci. J., Uberlândia, v. 34, n. 1, p. 243-250, Jan./Feb. 2018 (FARZIAMPOUR; ERMAMI; ESHRAGHIAN, 2009). Based on the quality satisfaction indicator proposed, we observed that a negative evaluation is more related to organizational issues of the event or program, but not to the content of the courses. Besides that, the satisfaction indicator could provide quality improvement of work processes at hospital level (HASHJIN et al., 2014), and guide future planning for other long-term Continuing Education Programs. The low attendance rates could be a consequence of dissatisfaction with the Program, but additional researchers need to evaluate this association, taking into account the effect of the professional category in this association. ACKNOWLEDGMENT The authors are grateful to all those who directly or indirectly contributed to the realization of the Training Cycles of CHU in 2013 and 2014, and all participants of the cycles for their kindness in cooperating with the evaluation of the event, especially the nursing body of the CHU. Thanks to Igor de Oliveira Felice and Roger Hutchings for the English version. RESUMO: Embora a satisfação seja um ponto chave na avaliação da educação continuada, poucos estudos têm realizado a sua avaliação, principalmente no que se refere a programas de longa duração. O objetivo do estudo foi avaliar a evolução de indicadores de satisfação aplicados a um programa de educação continuada em um hospital universitário de alta complexidade. Para isto, foram aplicados ao longo de 11 Ciclos de Treinamento formulários de satisfação e os mesmos foram relacionados com as taxas de comparecimento nos mesmos. Observou-se que os profissionais de enfermagem realizaram uma boa avaliação dos Ciclos no quesito avaliação do curso, que não oscilou com o tempo, porém a maior insatisfação foi relacionada aos critérios de organização do evento, que tendeu a piorar com o tempo. Além disso, aparentemente diferentes categorias de Enfermagem apresentaram critérios de satisfação diferentes, embora essa associação não pode ser comprovada. Os resultados comprovaram que a avaliação da satisfação com o decorrer do tempo pode oferecer subsídios para o gerenciamento dos programas de educação continuada, e que devem levar em consideração o perfil dos profissionais envolvidos com o programa. PALAVRAS-CHAVE: Treinamento. Curso de Longa Duração. Avaliação de Educação Continuada REFERENCES ANCC - American Nurses Credentialing Center’s Commission on Accreditation. The importance of evaluating the impact of continuing nursing education on outcomes: Professional nursing practice and patient care. American Nurses Credentialing Center. 2014. BAGAYOKO, C. O.; PERRIN, C.; GAGNON, M-P.; GEISSBUHLER, A. Continuing distance education: A capacity-building tool for the de-isolation of care professionals and researchers. Journal of General Internal, Secaucus, v. 28, Suppl 3, p. S666-70, 2013. http://dx.doi.org/10.1007/s11606- 013-2522-1. BYNUM, A. B.; IRWIN, C. A.; COHEN, B. Satisfaction with a distance continuing education program for health Professionals. Telemedicine and e-Health, New Rochelle, v. 16, n. 7, p. 776-786, 2010. doi:10.1089/tmj.2010.0005. http://dx.doi.org/10.1089/tmj.2010.0005. FARZIANPOUR, F.; EMAMI, A. H.; ESHRAGHIAN, M. R. The satisfaction of medical practitioners from continuing medical education program of Tehran University of Medical Sciences. Iranian Red Crescent Medical Journal, Dubai, v. 11, n. 4, p. 371-376, 2009. GERCENSHTEIN, L.; FOGELMAN, Y.; YAPHE, J. Increasing the satisfaction of general practitioners with continuing medical education programs: A method for quality improvement through increasing teacher-learner interaction. BMC Family Practice, London, v. 3, n. 15, 2002 250 Quality indicators ... PEREIRA, E. B. S. et al Biosci. J., Uberlândia, v. 34, n. 1, p. 243-250, Jan./Feb. 2018 HASHJIN, A. A.; RAVAGHI, H.; KRINGOS, D. S.; OGBU, U. C.; FISCHER, C.; AZAMI, S. R.; KLAZINGA, N. S. Using quality measures for quality improvement: The perspective of hospital staff. PLoS ONE, San Francisco, v. 9, n. 1, p. e86014, 2014. http://dx.doi.org/doi:10.1371/journal.pone.0086014. MENDES-RODRIGUES, C.; PEREIRA, E. B. S.; SOUSA NETO, R. L.; GOMES, F. A.; SILVA, D. V.; ANTUNES, A. V.; FELICE, R. O.; MENDONÇA, G. S.; WOLKERS, P. C. B.; ALVES, T. C. F.; SOUZA, R. C.; FÉLIX, D. A. C.. Quality indicators applied in a nursing continuing education program of a high complexity university hospital of Brazil: I - Attendance indicators. Bioscience Journal, Uberlândia, v. 34, n. 1, p. 231-242, 2018. MENDONÇA, G. S.; CUNHA, C. M.; DIAS, E. P.; RESENDE, T. C.; MENDES-RODRIGUES, C. Attendance indicators for students in long term nursing courses at a university hospital. Bioscience Journal, Uberlândia, v. 32, n. 3, 2016. http://dx.doi.org/10.14393/BJ-v32n3a2016-30114. MISSILDINE, K.; FOUNTAIN, R.; SUMMERS, L.; GOSSELIN, K. Flipping the classroom to improve student performance and satisfaction. Journal of Nursing Education, Thorofare, v. 52, n. 10, p. 597-599, 2013. http://dx.doi.org/10.3928/01484834-20130919-03. NI, C.; HUA, Y.; SHAO, P.; WALLEN, G. R.; XU, S.; LI, L. Continuing education among Chinese nurses: A general hospital-based study. Nurse Education Today, Philadelphia, v. 34, n. 4, p. 592-7, 2014. http://dx.doi.org/10.1016/j.nedt.2013.07.013. OLIVEIRA, J. L. C.; NICOLA, A. L.; SOUZA, A. E. B. R. Índice de treinamento de enfermagem enquanto indicador de qualidade de gestão de recursos humanos. Revista de Enfermagem da UFSM, Santa Maria, v. 4, n. 1, p. 181-188, 2014. http://dx.doi.org/10.5902/217976928772. SANTOS, M. V.; CERQUEIRA, P. V.; CARVALHO, E. C. M.; SILVA, G. R. F.; FALCÃO, L. M. Satisfação de enfermeiros com curso de formação para operacionalização de protocolo sobre úlcera por pressão. Revista da Rede de Enfermagem do Nordeste, Fortaleza, v. 16, n. 4, p. 496-503, 2015. https://doi.org/10.15253/2175-6783.2015000400006 WU, Y.; HSIEH, L.; LU, J. What's the relationship between learning satisfaction and continuing learning intention? Procedia - Social and Behavioral Sciences, New York, v. 191, p. 2849-2854, 2015. https://doi.org/10.1016/j.sbspro.2015.04.148.