Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 An electronic record system in nursing education: evaluation and utilization. González-Chordá VM1*, Mena-Tudela D1, Salas-Medina P1, Cevera Gasch A1, Lapeña- Moñux Y1, Folch-Ayora A1, Orts-Cortés MI1, Maciá-Soler ML1,2. (1) Nursing Department. University Jaume I. Avda Sos Baynat s/n, 12071. Castellón de la Plana. Castellón. Spain. (2) Nursing Department. University of Alicante. Carretera de San Vicente del Raspeig, s/n, 03690, San Vicente del Raspeig. Alicante. Spain. * Corresponding author: E-mail: vchorda@uji.es Phone: (+34) 964.387.744. Received: 2013-12-15; Accepted: 2014-08-04 Abstract The main objective of the present work is to analyze the results of the utilization and evaluation of the LORETO Record System (LRS), providing improvement areas in the teaching-learning process and technology, in second year nursing students. A descriptive, prospective, cross sectional study using inferential statics has been carried out on all electronic records reported by 55 nursing students during clinical internships (April 1º-June 26º, 2013). Electronic record average rated 7.22 points (s=0.6; CV=0.083), with differences based on the clinical practice units (p<0,05). Three items assessed did not exceed the quality threshold set at 0.7 (p<0.05). Record Rate exceeds the quality threshold set at 80% for the overall sample, with differences based on the practice units. Only two clinical practice units rated above the minimum threshold (p <0.05). Record of care provision every 3 days did not reach the estimated quality threshold (p <0.05). There is a dichotomy between qualitative and quantitative results of LRS. Improvement areas in theoretical education have been identified. The LRS seems an appropriate learning and assessment tool, although the development of a new APP version and the application of principles of gamification should be explored. Keywords Nursing; nursing education; software; student records; m-learning. González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 70 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 1. Introduction. Information and Communication Technology (ICT) has become an effective tool in the daily activity of the healthcare professions and the improvement of the teaching-learning processes in higher education. In the case of Nursing, practising professionals work in a changing context within the care environment. Changes in science, technology and healthcare systems as a whole have important implications for nurses' expectations and the impact on users in relation to the provision of safe, quality care. The quality of patient medical record documentation has an important structural aspect both from an ethical and safety assurance perspectives for the provision of healthcare services. In Spain, the Minimum Clinical Data Set in the National Healthcare System (2010), including the nursing care reports, has been approved pursuant to the National Health System Cohesion and Quality Act (Ley de Cohesión y Calidad del Sistema Nacional de Salud) (2003) and the Royal Decree 1093/2010. On the other hand, the projects conducted by the National Health System Quality Agency (Agencia de Calidad del Sistema Nacional de Salud) for the creation of the National Health System Electronic Health Records (2010, 2012) foster the development of computer applications that allow the electronic record of nursing care (González Sánchez et al, 2004; Sánchez Ros et al, 2006; Navarro Armedo et al, 2007; Garrido Bartolomé et al, 2007; Parra-Estrada et al, 2010; Medina-Valverde et al, 2012; Sánchez García, 2013). Moreover, the use of ICT in nursing education is being progressively introduced into the teaching-learning process, facilitating its adoption by future professionals, as Dearnley, Haigh and Fairhall (2008) stated. It is also an emerging line of research (Bogossian, Kellet y Mason 2009; Wu et al, 2011; Clay, 2011) as there is a commitment among universities and healthcare centers for the integration of information technology into their González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 71 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 processes. However, there have not been many experiences with the use of electronic records as skill learning and assessment tool (Gordon et al, 2007; Sayadi y Rokhaforz, 2012; Saba y Feeg, 2005 Boyle et al, 2008), considering this an important aspect in the training of future graduates in nursing, as Woodill and Udell (2011) and Gardner and Jones (2012) stated. A process of integration of electronic records into the student learning process in all the subjects in which skills are acquired through a methodology that integrates theoretical knowledge and practical and clinical skills (Maciá Soler et al, 2013) has been initiated in the Nursing Degree at the University Jaume I (UJI) (Castellón, Spain) during the 2012/2013 academic year. The implementation process begun in the second year, in which the skills that are to be achieved in specialized care are related to basic care. The students' length of stay in this environment is of 3 months. Each student is assigned an electronic device (tablet with Android system) that includes an electronic activity record system (Loreto Record System) with a methodology developed by the professors of the subjects on the basis of the skills involved and adapted to the learning needs. The LORETO Record System (LRS) is based on the International Classification of Diseases, such as the taxonomy of diseases and procedures. The LRS reflects the nursing method, with a structure based on the assessment, care prescription and re-evaluation of each patient according to their functional abilities (Barthel Index). This system also incorporates support tools for clinical decision making related to nursing sensitive outcomes (falls, pressure ulcers, malnutrition, etc.), and offers students an overview of process-based management applied to the nursing practice, while respecting individuality of care. González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 72 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 The objective of this research is to know the use nursing students make of the LRS during supervised clinical practice and analyse the results of the evaluation, establishing qualitative and quantitative criteria in order to provide possible areas for improvement in the teaching and development of the application. 2. Methodology. This is an analytical, prospective, observational study in which inferential statics has been used. The study population consists of all electronic records carried out by 55 nursing students during supervised clinical practice in the subject "Nursing Care in Healthcare Processes" in the second year of the Nursing Degree at the University Jaume I (UJI), which includes the following subjects: Primary Care Nursing, Nursing Care in Osteoarticular Processes, Nursing Care in Digestive, Endocrine and Renal Processes and Nursing Care in Cardiovascular and Respiratory Processes. The supervised clinical practice is conducted in 16 acute inpatient units of 5 public and private hospitals in the province of Castellón (Spain) with which a cooperation agreement exists. 2.1 Inclusion and exclusion criteria The electronic records carried out in the trauma, internal medicine and surgery units of the public hospitals in the province of Castellón (Spain) were included in this study, as these are the ones that best suit the skills to be acquired by students. Those considered data entry errors or with an age <18 were excluded. 2.2 Variables studies 2.2.1 Utilization of records • Records Estimated (RE) (Formula 1) González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 73 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 Formula 1 : Records Estimated (ER) RE = (64 actual days of clinical practice/average stay in unit) * No. student beds • RI/RE Index: records issued/Records Estimated (Formula 2) Formula 2 : Records Issued/Records Estimated (RI /RE Index) RI/RE index = number of records issued/number of records estimated 2.2.2 Completion of electronic records. • Percentage of completed records (% C): 21 variables included in the LRS, which meet the learning outcomes of the subject "Nursing Care in Healthcare Processes", were selected on faculty consensus. Table 1 : Variables selected from LRS. LRS Variables Socio-demographic variables Age Sex Hospital Unit Clinical Variables ICD-9CM Diagnosis Type of process Type of admission Readmission Complexity Assessment at admission Functional Ability Fall risk PU risk Nutritional Risk Body Mass Index (BMI) Social evaluation Cognitive status Pain Primary Caregiver Care prescription and provision Barthel index score every three days Implementation of the standardized care plan every three days Care report at discharge Reason for discharge Continuity of care Functional Ability • Record Rate (RR) of each electronic record (Formula 3) González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 74 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 Formula 3 : Record Rate; RR = (number of variables recorded/21 variables)*100 2.2.3 Assessment of the quality of records The assessment of the quality of electronic records is carried out by means of a tool designed ad hoc on faculty consensus. This includes 10 items that are evaluated between 0 and 1 points and account for 25% of the final practice score. The variables considered are: • Final score for each student (0-10 points). • Identification of the reason and type of admission (0-1 points). • Record of complete health history (0-1 points). • Accurate record of health history (0-1puntos). • Performance of physical examination (0-1 points). • Utilization of assessment tools (0-1 points). • Patient's evolution per shift (0-1 points). • Record of planned activities (0-1 points). • Accurate record of discharge report (0-1 points). • Identification of care diagnosis (0-1 points). • Use of a professional and technical language (0-1 points). 2.3 Statistical Analysis. 2.3.1 Utilization of LRS. An estimate of the number of records to be carried out by the students in each practice unit included in the study and the records issued/records estimated index (RI/RE Index) were calculated, considering a RI/RE index ≥ 0.7 as quality threshold. González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 75 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 2.3.2 Completion of records. The record rate (RR) and computerized record of each of the selected 21variables was calculated considering an average RR ≥ 80% as quality threshold. By means of the t- Student test, the achievement of the quality threshold (80%) was confirmed for the sample as a whole and for each of the units by comparing the unit RR average to the ANOVA or the Kruskal-Wallis test, being the homoscedasticity previously analysed by means of the Levenne test. In order to confirm whether each variable reaches a record proportion of 80%, the z-test for proportions was used, and a study of the possible dependence of the record proportion of the variables based on the clinical practice units was conducted by means of the chi- square test. Fisher's exact test was used when the number of observations per group was n <5. 2.3.3 Assessment of the quality of the records. A descriptive analysis (average, standard deviation, coefficient of variation and percentiles) of the scores obtained by students and of each of the items composing the assessment tool was performed. The Kruskal-Wallis test determined whether there were differences in scores based on the clinical practice units. A statistical analysis of the data was performed by means of the R Commander application of the R 3.0.2 software. A 5% bilateral significance level was assumed in hypothesis tests. 2.4 Data collection. The collection of information was carried out concurrently during the supervised clinical practice, from 1st April to 26th June 2013, in the clinical practice units that met the inclusion criteria. González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 76 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 2.5 Ethical considerations. There is an agreement among the professors of the subjects to use the results of the applied teaching methods for the elaboration of this research. Moreover, the anonymity of patient records is guaranteed under the current legislation. 3. Results. 3.1 Utilization of LRS. A total of 334 computerized records, made in 7 clinical practice units that met the inclusion criteria, were included in the study. The RI/ RE index for the whole sample was 0.55, which did not meet the quality threshold established. A RI/RE index of 0.7 (Table 2) was reached only in two units. 5 registration errors (data entry errors) (1.49%) and one age error (age <18), were excluded from the 334 electronic records, so the final sample consisted of 328 electronic records (Table 2). Table 2 : RI/RE Index. Exclusion criteria. Hospital/units Electronic Records RE1 RI2 RI/RE Index 3 Exclusion criteria. Included Error <18 no % no % no % Hosp 1 Surgery 60 29 0.48 0 0 1 2.9 28 96.5 Hosp 2 Surgery 70 47 0.67 0 0 0 0 47 100 Internal medicine 210 96 0.45 2 2.08 0 0 94 97.9 Traumatology 70 49 0.7 0 0 0 0 49 100 Hosp 3 Surgery 64 49 0.76 0 0 0 0 49 100 Internal medicine 64 35 0.41 0 0 0 0 35 100 Traumatology 64 29 0.54 3 10.3 0 0 26 89.6 Total 602 334 0.55 5 1.4 1 0.3 328 98.2 González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 77 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 3.2 Completion of records. The average record rate (RR) for all electronic records included in the study is 89.82%, exceeding the quality threshold set at 80% (p <0.05). Except for the Hospital 3 Trauma Unit, all units have an RR greater than 80% (p <0.05). There are also statistically significant differences in RR based on the units in which students undertake clinical practice (p <0.05) (Table 3). Table 3 : Record Rate (RR). Units no 1 s t-Student * Levene ANOVA Hosp 1 surgery 28 93.27 3.72 <0,001 0.0016 <0,001 Hosp 2 surgery 47 88.09 8.24 <0,001 Hosp 2 Internal med 94 88.19 7.14 <0,001 Hosp 2 trauma 49 93.46 3.74 <0,001 Hosp 3 surgery 49 90.27 7.01 <0,001 Hosp 3 Internal med 35 91.89 2.94 <0,001 Hosp 3 trauma 26 81.48 14.12 0.2952 TOTAL 328 89.82 7.68 <0,001 The study conducted on the completion proportion of the 21 selected LRS variables shows that all socio-demographic and clinical variables are recorded by students in 100% of the sample, except for the complexity variable (92.99%), although it exceeds the 80% expected (p <0.05) (Table 4). (*) Student t-test results for a sample. P-value in hypothesis testing (p <0.05): H0: mu = 79.; H1: mu ≥ 80 (1) Records estimated: RE= (64 actual days of clinical practice/average stay in unit) * No. student beds (2) Records Issued, (3) RI/RE Index = Records Issued/Records Estimated González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 78 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 All variables included in the patient's admission assessment exceed the threshold established, except for the BMI (62.5%) and the evaluation of the pain level (41.16%), being the record of these variables based on the units in which students undertake clinical practice (p <0.05). The continuity of care is the only variable recorded at discharge that does not reach the quality threshold (14.94%) depending on the units of supervised clinical practice (p <0.05). Table 5 shows how students' use of Barthel Index for assessing functional ability and prescribing care remains above the established threshold whenever the stay is shorter than 18 days. A connection between the increase in the stay and the decrease in the record proportion can be noted in this variable. A similar situation is observed in the record of the implementation of the standardized care plan based on the functional ability, except that in this variable only the record of the care plan at admission exceeds the 80% expected. González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 79 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 Table 4: Completion proportion of the selected LRS variables. LRS Variables %C1 no N INFERENCE Z-test* IC 95% X2 So ci o- de m og ra ph ic va ri ab le s Age 100 328 328 Sex 100 328 328 Hospital 100 328 328 Unit 100 328 328 C lin ic al V ar ia bl es Diagnosis 100 328 328 Type of process 100 328 328 Type of admission 100 328 328 Readmission 100 328 328 Complexity 92.99 305 328 <0,01 0-9% A ss es sm en t a t a dm is si on Functional Ability 100 328 328 Fall risk 96.95 318 328 <0,01 0-5% PU risk 96.95 318 328 <0,01 0-5% Nutritional Risk 96.95 318 328 <0,01 0-5% BMI 62.5 205 328 1 0-41% <0,01 Social evaluation 91.77 301 328 <0,01 0-11% Cognitive status 96.95 318 328 <0,01 0-5% Pain 41.16 135 328 1 0-63% <0,01 Primary Caregiver 96.95 318 328 <0,01 0-5% C ar e re po rt a t di sc ha rg e Reason for discharge 100 328 328 Continuity of care 14.94 49 328 1 0-85% <0,01 Functional Ability 100 328 328 On the other hand, it cannot be said that there is a significant dependency between the record of the Barthel Index or the implementation of the care plan every three days and the units in which students undertake clinical practice, noting as a single exception the record of the implementation of the care plan based on the functional ability on the third day of stay (p <0.05). (1) Completion proportion (*) Z-test results for a sample. P-value in hypothesis testing (p <0.05): H0: Unregistered variable rate = 20%; H1: Proportion of unregistered variables <20% Implementation conditions are not met. González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 80 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 Table 5: Utilization of assessment scales and implementation of the standardized care plan. Variable %C1 no N INFERENCE Z-test* IC 95% X2 Fisher Barthel I. 0 days 100 328 328 F un ct io na l A bi lit y (B ar th el I nd ex ). Barthel I. 3 days 97.37 220 228 <0,01 0-5.5 Barthel I. 6 days 93.98 125 133 <0,01 0-10.3 Barthel I. 9 days 91.36 74 81 <0,01 0-15.2 Barthel I. 12 days 91.38 51 58 0.015 0-16.6 Barthel I. 15days 91.67 33 36 0.04 0-19.1 Barthel I. 18 days 88.46 23 26 0.14 0-25.6 0.08 Barthel I. 21 days 88.89 14 16 0.172 0-28.6 0.078 Barthel I. 24 days 88.71 12 14 0.296 0-35.2 0.197 Barthel I. 27 days 85.71 7 7 0.352 0-45.2 0.428 Im pl em en ta ti on o f c ar e pl an Implementation 0 day 92.99 305 328 <0,01 0-70 Implementation 3 days 63.6 145 228 1 0-40 <0,01 Implementation 6 days 56.39 75 133 1 0-50.7 0.062 Implementation 9 days 55.56 45 81 1 0-53 0.16 Implementation 12 days 63.69 37 58 1 0-46 0.5 Implementation 15 days 52.78 19 36 1 0-47.2 0.81 Implementation 18 days 42.31 11 26 1 0-72.6 0.502 Implementation 21 days 27.78 5 16 1 0-85 0.82 Implementation 24 days 28.57 4 14 1 0-86 0.64 Implementation 27 days 28.57 2 7 0.99 0-88% 0.6 3.3 Assessment of the quality of electronic records. The average score obtained by students in the evaluation of records was 7.22 (s = 0.6, cv = 0.083), with a minimum score of 6 and a maximum of 8, with statistically significant differences based on the units in which students undertake supervised clinical practice (p <0.05). (1) Proportion of completion (*) Z-test results for a sample. P-value in hypothesis testing (p <0.05): H0: Unregistered variable rate = 20%; H1: Proportion of unregistered variables <20% Implementation conditions are not met. González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 81 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 Table 6: Analysis of the assessment tool items used by the faculty. Assessment items 1 s p0 p25 p50 p75 p100 p-value * Identification of the reason and type of admission 0.96 0.08 0.75 1 1 1 1 1 Record of complete health history 0.84 0.15 0.5 0.75 0.75 1 1 0.99 Accurate record of health history 0.68 0.23 0 0.68 0.75 0.75 1 0.41 Performance of physical examination 0.54 0.16 0.25 0.5 0.5 0.75 0.75 <0,01 Utilization of assessment tools 0.9 0.12 0.75 0.75 1 1 1 1 Patient's evolution per shift 0.68 0.19 0.25 0.5 0.75 0.75 0.75 0.39 Record of planned activities 0.62 0.12 0.5 0.5 0.62 0.75 0.75 0.017 Accurate record of discharge report 0.78 0.22 0.5 0.5 0.75 1 1 0.91 Identification of care diagnosis 0.48 0.32 0 0.3 0.5 0.75 1 <0,01 Use of a professional and technical language 0.98 0.06 0.75 1 1 1 1 1 Table 6 shows the results of the descriptive and inferential analysis of the 10 items included in the clinical practice report. The items with a score lower than 0.7 (p-value <0.5) are the following: "Performance of physical examination" (=0.54, s=0.16), "Record of planned activities" (1= 0.62, s =0.12), and "Identification of care diagnosis" (=0.48, s = 0.32). 4. Discussion. There is little evidence on the use of electronic records systems as a tool for learning and assessment in nursing studies. This study presents the results of the utilization and evaluation of an electronic registration system (LORETO Record System) after the first year of implementation in second year student nursing degree. The results provide a dichotomy between quantity and quality of the records. On one hand, the number of records issued is away from the amount of records estimated. In this sense, the novel presence of nursing students using an electronic record system during (*) T-Student test results for a group in hypothesis testing (p-value): H0: mu = 0,7; H1: mu <0,7 González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 82 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 clinical internships with mobile devices, may cause resistance in the implementation process and hinder student access to electronic records (Kowitlawakul, Chan, Pulcini, Wang, 2014; Baillie, Chadwick, Mann, Brooke-Read. 2013). Other aspects that explain the small number of records are due to problems related to interoperability of mobile devices and the central database, the quality of Wi-Fi networks of hospitals and programming errors detected during the collection period data. In addition, there are problems related to the difficulty of using the software, according to Kowitlawakul, Wang and Chan (2013), despite previous training students have received. The quality of the records seems appropriate in the studied variables, except the variables Barthel index score every three days, Implementation of the standardized care plan every three days and Continuity of care. Although LRS incorporates a method of prescribing care based on functional capacity (Barthel index), these results reflect a reality in which nurses, following the initial assessment, use the method of solving problems and meet the needs sought by patients, according to Urquhart, Currell, Grant and Hardiker (2009), without carrying out a prescription of care required. Moreover, papers that use the results of the evaluation of an electronic record system designed for nursing students in order to identify areas for improvement in the learning process have not found in the review of the literature. In our study, evaluation of the records reveals that the physical examination, the importance of patient's evolution, record of planned activities and identification of care diagnosis are issues that must be reinforced in the classroom, owing to their direct relation to nursing process (assessment, planning, implementation and evaluation) (Habermann, 2006) and this process is crucial in the formation of future graduates. Despite these results, the use of software and similar tools can improve nursing students' clinical skills and encourage their learning and implementation of the nursing process in González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 83 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 clinical practice (Sayadi, Rokhafroz 2012, Saba and Feeg 2005) and the patient safety (Jones, Donelle 2011). Furthermore, the use of electronic records systems allows the development of generic skills related to the use of ICT, being fundamental in the future nursing professionals (Kowitlawakul, Wang, Chan 2013; Baillie, Chadwick, Mann, Brooke-Read 2013) . In another way, the learning environment and tutors practices play a key role in clinical learning (Papastavrou et al 2010). They influence the learning opportunities and skills acquisition. According to results of our study, there may be differences in the support for the students according to the clinical practice units, according to Bailie and Curzio (2009) and Baillie, Chadwick, Mann and Brooke-Read (2013) because nursing documentation requires thought, planning and skills in making decisions that should be monitored (Sverinsson, Sanda 2010). Furthermore, the LRS is newly established and tutors need to develop confidence with this tool based on new ways of learning. The information technologies have strongly irrupted in nursing studies. Proof of this is the development of e-learning (Button, Harrington, Belan 2013) and m-learning tools (Skiba 2011), including application for mobile devices (APP) and educational games (Stanley, Latimer 2011). The LRS has been developed as a tool for m-learning with web technology, but the results of this study suggest that develop innovations in software to enhance its educational role is necessary. Currently APPs are considered as new learning tools that are familiar to students (Phillippi, Wyatt 2011), in which principles of gamification can be applied. Gamification is a relatively recent concept in the world of education that involves applying the basic elements that make games fun and engage in things that typically are not considered as games (Gamify.com) concept. Apply principles of gamification to LRS may improve outcomes for students and their learning. Moreover, the results obtained in this study should be treated with caution because it has been carried out after the first year of implementation of the software and this limits the González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 84 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 size of the sample. In addition the tool used for the evaluation of records has not undergone to a study of validity and reliability. Despite these limitations, the results are useful to initiate improvement actions that directly impact the quality of student learning and continue the evaluation and improvement of LRS. 5. Conclusions. The LRS used by students during clinical practice is a useful learning and assessment tool for skill acquisition, and offer areas for improvement within the learning process. Students' skills related within physical exploration, use of autonomy assessment scales and identification of care diagnosis should be reinforced in the classroom and records may require further supervision from mentors during supervised clinical practice. The use of the LRS by students can be considered adequate in terms of quality, although further improvements in terms of interoperability and programming are needed. The development of an APP version of the LRS for the Android operating system and the application of gamification principles are aspects that are being considered in to improve the software. 6. Acknowledgements This work has been supported by the Universitat Jaume I in the frame of the Mobility Research Staff Program (Ref E-2014-39). 7 References Baillie L., Curzio J. 2009. A survey of first year student nurses’ experiences of learning blood pressure measurement. Nurse Education in Practice, 9: 61e71. Baillie L, Chadwick S, Mann R, Brooke-Read M. 2013. A survey of student nurses’ and midwives’ experiences of learning to use electronic health record systems in practice. Nurse Education in Practice, 13:437-441. DOI: 10.1016/j.nepr.2012.10.003 González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 85 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 Bogossian FE, Kellett SEM, Mason B. 2009. The use of tablet pcs to access an electronic portfolio in the clinical setting: a pilot study using undergraduate nursing students. Nurse Educ Today, 29(2),246-53. DOI: 10.1016/j.nedt.2008.09.001 Button D, Harrington A, Belan I. 2013. E-learning & information communication technology (ICT) in nursing education: A review of the literature. Nurse Educ Today. In press Clay CA. 2011. Exploring the use of mobile technologies for the acquisition of clinical skills. Nurse Educ Today, 31(6),582-6. DOI: 10.1016/j.nedt.2010.10.011 Dearnley C, Haigh J, Fairhall J. 2008. Using mobile technologies for assessment and learning in practice settings: a case study. Nurse Educ Pract, 8(3),197-204. DOI: 10.1016/j.nepr.2007.07.003 Papastavrou E, Lambrinou E, Tsangari H, Saarikoski M, Leino-Kilpi H. 2010. Student nurses experience of learning in the clinical environment. Nurse Education in Practice 10: 176–182 DOI: 10.1016/j.nepr.2009.07.003 Gamify.com. What is Gamification? Retrieved from The Gamification Encyclopedia:http://gamification.org/wiki/Gamification Gardner C, Jones S. 2012. Utilization of academic electronic medical records in undergraduate nursing education. OJNI, 16 (2) http://ojni.org/issues/?p=1702 Garrido Bartolome A, Sánchez Martinez V, Muñoz Izquierdo A, Ruiz Hontangas A, Ferrandis Campos. 2007. Estación de enfermería, primeros pasos en la informatización de cuidados. Investigación y cuidados, 5(11), 45-55 http://www.bibliotecadigitalcecova.com/contenido/revistas/cat4/pdf/Investigacion_y_Cui dados_N11.pdf González Sánchez JA, Cosgaya García O, Simón García MJ, Blesa Malpica AL. 2004. Registros de enfermería: convencional frente a informatizado. Unidad de cuidados críticos. Enferm Intensiva, 15(2),53-62. DOI: 10.1016/S1130-2399(04)78145-6 Gordon J, McNew R, Trangenstein P. 2007. The development of an online clinical log for advanced practice nursing students: a case study. Stud Health Technol Inform, 129,1432- 6. González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 86 http://ojni.org/issues/?p=1702 http://www.bibliotecadigitalcecova.com/contenido/revistas/cat4/pdf/Investigacion_y_Cuidados_N11.pdf http://www.bibliotecadigitalcecova.com/contenido/revistas/cat4/pdf/Investigacion_y_Cuidados_N11.pdf Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 Habermann, M. The Nursing Process: A Global Concept. Elsevier Health Sciences, 2006 Kowitlawakul Y, Wang L, Chan W-CS. 2013. Development of the electronic health records for nursing education (EHNRE) software program. Nurs Educ Today, 33, 1529- 35. DOI: 10.1016/j.nedt.2012.12.001 Kowitlawakul Y, Chan W-CS, Pulcini J, Wang L. Factors influencing nursing students' acceptance of electronic health records for nursing education (EHRNE) software program. Nurse Education Today. 2014 (in press). DOI: 10.1016/j.nedt.2014.05.010 Law 16/2003, de 28 de mayo, de cohesión y calidad del Sistema Nacional de Salud. Spain. Boletín Oficial del Estado, Núm128 de 29.05.2003. http://www.boe.es/buscar/doc.php?id=BOE-A-2003-21840 Maciá Soler L, Orts Cortés I, Galiana Sánchez M, Ors Montenegro A. 2013. Implantación simultánea de los niveles de Grado, Máster y Doctorado en Enfermería en la Universidad Jaume I. Castellón de la Plana, España. Investigación y Educación en Enfermería, 31(2). http://aprendeenlinea.udea.edu.co/revistas/index.php/iee/rt/printerFriendly/14121/14036 Medina-Valverde, MJ; Rodriguez-Borrego, MA; Luque-Alcaraz, O; de la Torre-Barbero, MJ; Parra-Perea, J; Moros-Molina, MP. 2012. Estudio evaluativo sobre una herramienta informática de gestión enfermera en el periodo 2005-2010. Enferm Clin, 22,3-10. DOI: 10.1016/j.enfcli.2011.06.001 Ministerio de Sanidad y Política Social. 2010. El Sistema de Historia Clínica Digital del Sistema Nacional de Salud. Instituto de información sanitaria. Agencia de Calidad del Sistema Nacional de Salud. Spain. http://www.msc.es/organizacion/sns/planCalidadSNS/docs/HCDSNS_Castellano.pdf Ministerio de Sanidad, Servicios Sociales e Igualdad y Ministerio de Industria, Energía y Turismo. 2012. Las TIC en el Sistema Nacional de Salud. El programa Sanidad en Línea. Actualización de datos 2011. Red.es. Spain. http://www.red.es/media/2012- 07/1343654162527.pdf Navarro Armedo, JM; Orgier Uranga, PE; de Haro Marín, S. 2007. AURORA 2003: gestor de informes de alta de enfermería en cuidados intensivos. Enfermería intensiva, 18(2), 78-89. DOI: 10.1016/S1130-2399(07)75740-1 Parra-Estrada, AJ; Guerra-Arévalo, J; Domínguez Guerrero, F; Benítez de la Calle, MC. González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 87 http://www.boe.es/buscar/doc.php?id=BOE-A-2003-21840 http://aprendeenlinea.udea.edu.co/revistas/index.php/iee/rt/printerFriendly/14121/14036 http://www.msc.es/organizacion/sns/planCalidadSNS/docs/HCDSNS_Castellano.pdf http://www.red.es/media/2012-07/1343654162527.pdf http://www.red.es/media/2012-07/1343654162527.pdf Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 2010. Proceso de implementación del módulo de cuidados de Diraya (Azahar) en el Hospital de Jerez. VII Congreso Nacional de Formación Continuada en Salud Innovación eImpacto. http://www.juntadeandalucia.es/agenciadecalidadcanitaria/formacionsalud/gestor/es/recur sos_Formativos/material_congreso_malaga/index2.html Phillippi JC, Wyatt TH. 2011. Smartphones in Nursing Education. CIN, 29(8):449-54. Real Decreto 1093/2010, de 3 de septiembre, por el que se aprueba el Conjunto Mínimo de Datos de los documentos clínicos en el Sistema Nacional de Salud. Spain. Boletín Oficial del Estado núm 225, 16.09.2010. http://www.boe.es/buscar/doc.php?id=BOE-A- 2010-14199 Saba VK, Feeg V. 2005. A Bedside PC Charting System for Nursing Students using Clinical Care Classification (CCC) System. Libro de Congresos AMIA http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1560569/ Sánchez García AB, López Montesinos MJ, Fernández Alemán JL. 2013. Dispositivos inalámbricos en la educación enfermera. Investigación y educación en enfermería, 31(1) http://aprendeenlinea.udea.edu.co/revistas/index.php/iee/article/view/11681/13292 Sánchez Ros, N; Reigosa Gago, LF. 2006. Selene. Informatización de la historia clínica electrónica: Implicación sobre el proceso de enfermería. Enfermería Global, 6, 1-10. http://revistas.um.es/eglobal/article/view/458 Sayadi N, Rokhafroz D. 2012. Nursing Students’ perspective about a mobile software on nursing process for bedside use. Iranian Journal of Medical Education, 12(2) http://ijme.mui.ac.ir/browse.php?a_id=2198&sid=1&slc_lang=en Skiba DJ. 2011. On the horizon mobile devices: are they a distraction or another learning tool? Nurs Educ Perspect, 32(3):195-7. DOI: 10.5480/1536-5026-32.3.195 Severinsson, E., Sanda, A., 2010. Evaluation of the clinical supervision and professional development of student nurses. Journal of Nursing Management, 18:669-77. DOI: 10.1111/j.1365-2834.2010.01146.x Stanley D, Latimer K. 2011. “The Ward”: A simulation game for nursing students. Nurse Education in Practice, 11:20-5. DOI: 10.1016/j.nepr.2010.05.010 González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 88 http://www.juntadeandalucia.es/agenciadecalidadcanitaria/formacionsalud/gestor/es/recursos_Formativos/material_congreso_malaga/index2.html http://www.juntadeandalucia.es/agenciadecalidadcanitaria/formacionsalud/gestor/es/recursos_Formativos/material_congreso_malaga/index2.html http://www.boe.es/buscar/doc.php?id=BOE-A-2010-14199 http://www.boe.es/buscar/doc.php?id=BOE-A-2010-14199 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1560569/ http://aprendeenlinea.udea.edu.co/revistas/index.php/iee/article/view/11681/13292 http://revistas.um.es/eglobal/article/view/458 http://ijme.mui.ac.ir/browse.php?a_id=2198&sid=1&slc_lang=en Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2014.2224 Social and Technological Sciences EISSN: 2341-2593 Urquhart C, Currell R, Grant MJ, Hardiker NR. 2009. Nursing record systems: effects on nursing practice and healthcare outcomes. Cochrane Database of Systematic Reviews. http://www.bibliotecacochrane.com/pdf/CD002099.pdf Woodill G, Udell Ch. 2011. The future of mobile learning in medicine and Healthcare. Float Mobile Learning. http://floatlearning.com/ideas-and-insights/mobile-learning- research/the-future-of-mobile-learning-in-medicine-and-healthcare/ Wu PH, Hwang GJ, Tsai CC, Chen YC, Huang YM. 2011. A pilot study on conducting mobile learning activities for clinical nursing courses based on the repertory grid approach. Nurse Educ Today, 31(8), e8-15. DOI: 10.1016/j.nedt.2010.12.001 González-Chordá et al. (2014) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 1 Nº 2 (2014): 70-89 | 89 http://floatlearning.com/ideas-and-insights/mobile-learning-research/the-future-of-mobile-learning-in-medicine-and-healthcare/ http://floatlearning.com/ideas-and-insights/mobile-learning-research/the-future-of-mobile-learning-in-medicine-and-healthcare/