International Journal of Human Capital Management, Vol. 2 (2), December 2018 International Journal of Human Capital Management, Vol. 7 (1), June 2023 104 | P a g e International Journal of Human Capital Available online at Management http://journal.unj.ac.id/unj/index.php/ijhcm E-ISSN 2580-9164 Vol. 7, No. 1, June 2023, p 104-114 CIPPO MODEL EVALUATION ON THE ENGLISH LANGUAGE TRAINING PROGRAM AT THE INDONESIAN NAVY EDUCATION SERVICES Nani Kusmiyati Department of Human Resource Management, Universitas Negeri Jakarta, Indonesia Email: nani1navy@gmail.com Hamidah Department of Human Resource Management, Universitas Negeri Jakarta, Indonesia Email: hamidah@unj.ac.id Kadir UIN Syarif Hidayatullah Jakarta, Indonesia Email: kadir@uinjkt.ac.id ABSTRACT This study describes the implementation of the English Language Education and Training program at the Indonesian Navy Education Service for Indonesian Navy personnel. This study applies a descriptive qualitative method using the CIPPO model (Context, Input, Process, Product and Outcome). To get the data, the researchers conducted interviews, observation, documentation, focus group discussions and questionnaires. From the data obtained through interviews, observation, documentation, focus group discussions and CIPPO model questionnaires using a Likert scale (1 to 5) the context aspect shows 3.81, the training has been carried out based on the objectives, educational plans and activity plans. The input aspect shows 3.59, it supports the training process. The process aspect shows 3.82, the training process can run smoothly. The product aspect shows 3.96, the results of the training are satisfactory. The outcome aspect shows 3.95, the training has a good impact on human resources and institutions in the Navy. Thi s research has the novelty that English Education and Training at DISDIKAL really needs an in - depth evaluation of the implementation of Education and Training. So far, the evaluation is still based on reports on the completion of training. Keywords: CIPPO, context, input, process, product, outcome, the training, DISDIKAL Received: 8 May 2023 Accepted: 28 June 2023 Publish: June 2023 http://journal.unj.ac.id/unj/index.php/ International Journal of Human Capital Management, Vol. 7 (1), June 2023 105 | P a g e How to Cite: Kusmiyati, N, et.al. (2023). CIPPO Model Evaluation on the English Language Training Program at the Indonesian Navy Education Services. International Journal of Human Capital Management, 7 (1), 104-114. https://doi.org/10.21009/IJHCM.07.01.8 INTRODUCTION The Naval Education Services (DISDIKAL) as one of the educational institutions in the Indonesian Navy has organized English language education and training which aims to prepare the Indonesian Navy's human resources towards the World Class Navy. The World Class Navy can be proven when the Indonesian Navy plays a role in the international community in maintaining world peace and maritime borders with friendly countries. English as one of the languages of diplomacy and the language of instruction for the manuals for the main tools of the sophisticated Indonesian Navy's main weapon system must be learned and mastered by the Indonesian Navy's human resources. English as the international language will make it easier for the world community to communicate and interact so that misunderstandings can be avoided. The Indonesian Navy personnel receive many offers from various countries to take part in assignments, specialized education and development abroad, English is one of the requirements for passing the test for these offers. International activities such as seminars, workshops and meetings are also often held both domestically and abroad. So that the implementation of English Education and Training is very helpful for Indonesian Navy personnel to prepare themselves to fulfill the offer and take part in these international activities. The success of the training can be known by an in-depth evaluation of the implementation of the education and training. Researchers evaluated English training at DISDIKAL using the CIPPO model (Context, Input, Process, Product, Outcome). The theoretical benefit of this research is that it can contribute to the development of knowledge about Training Evaluation so that it can become a reference for other researchers. In the field of Human Resource Management (HRM), this research can provide new perspectives or views, especially in developing human resources through training. While the practical benefit is that it contributes to educational institutions as organizers of the English language Training program in making decisions for t he formulation of policies related to the Education and Training program that has been implemented. The evaluation results obtained then serve as a basis for improvement in accordance with the findings and research recommendations. LITERATURE REVIEW Program Evaluation Concept Program evaluation according to Muryadi A.D. (2017) is a tool or procedure used to find out something with predetermined methods and rules. Explicitly evaluation refers to the attainment of goals while implicitly, evaluation is a comparison of the results achieved with the established achievement standards. Program evaluation is also a systematic investigative activity about something of value from an object. Arikunto, Suharsimi (2018) argues, program evaluation is an activity unit that aims to collect information about the implementation of a policy on an ongoing basis in an organization that involves a group of people for decision making. Program evaluation aims to obtain information about program implementation which is then used as a follow-up activity or decision making. International Journal of Human Capital Management, Vol. 7 (1), June 2023 106 | P a g e Stufflebeam (2014), states that evaluation is a process of validating reliability, effectiveness and cost efficiency, security, ease of use, and honesty. Evaluation shows the community about strengthening values, benefits, progress, accreditation, and accountability. Bad programs must end while good programs need to be maintained and expanded. However, evaluation does not always guarantee high service quality because those in power do not necessarily pay attention to the evaluation results and take the necessary corrective actions. Decision makers fail to make decisions because they do not pay attention to or cover up worrisome evaluation information. Program Evaluation Model Evaluation model is a pattern used to collect and analyze the process of an activity to obtain information about the success of the program. Several evaluation models are used to evaluate education and training programs. As revealed by Muryadi A.D et al (2017), there are various evaluation models such as the CIPP evaluation model and the Kirkpatrick evaluation model. The following is an explanation of the evaluation model mentioned above. CIPP Evaluation Model (Context, Input, Process, and Product) Stufflebeam et al (2014), stated that the CIPP model has the core concept of evaluating the context of entities, inputs, processes, and products, which are denoted by the CIPP acronym letters. In context evaluation, evaluators assess needs, problems, assets, and opportunities, as well as relevant contextual conditions and dynamics. Decision makers use context evaluation to determine objectives, priorities and ascertain program objectives for significant needs and problems. In input evaluation, evaluators assist program planners by identifying and assessing procedural plans, staffing requirements, and budgets for feasibility and potential cost -effectiveness in meeting targeted needs. Decision makers using input evaluation are intended to identify and select plans, write funding proposals, allocate resources, assign staff, and work schedules. In process evaluation, evaluators monitor, document, assess, and report on the implementation of program plans. Evaluators provide feedback during program implementation and the extent to which the program is implemented. Program staff use periodic process evaluation reports to track progress, and identify program implementation issues to ensure quality. After the program cycle, specific departments such as program staff, supervisors and constituents can use the process evaluation documentation to assess how well the program was implemented and assess the reasons for the lack of program success due to weak intervention strategies or inadequate implementation strategies. In product evaluation, the evaluator identifies and assesses the costs and results achieved in both the short and long term. Evaluators provide feedback during the program, the extent to which program implementation is carried out and the goals achieved. At the end of the program, product evaluation helps to identify and assess the full performance of the program. Program staff use interim product evaluation feedback to maintain focus on achieving key outcomes and to identify and address gaps in program progress toward achieving key outcomes. Kirkpatrick Model (Reaction, Learning, Behaviour, and Result) Kirkpatrick's Evaluation Model, as suggested by Kirkpatrick, Donald. L, and James (2007) in their book use four levels of evaluation as follows: Reaction International Journal of Human Capital Management, Vol. 7 (1), June 2023 107 | P a g e Reaction evaluation is how the participant's personal feelings and reactions to the training or learning experience relate to the program, teaching materials or materials, instructors, methodologies, facilities, and so on. Examples of reaction evaluation methods and tools are feedback forms based on subjective personal reactions to the training experience, verbal reactions that can be recorded and analyzed, post-training surveys or questionnaires, online evaluations or assessments by participants. Evaluation can be carried out immediately after the training is over, it is very easy to get feedback, to know that the participant is satisfied with the training, and will make a positive impression when relating the participant's experience to others who may be experiencing the same thing. Participant reactions are an important factor for the improvement and continuation of the Program. Learning Evaluation of learning is a measurement of increasing knowledge or intellectual abilities before and after learning. Have participants learned the material being taught, put it into practice, and how far has the progress or change of participants been made after attending the training and whether it is in accordance with the training objectives. Learning evaluation methods and tools can be through assessments or tests given before and after training, interviews or observation before and after although this is time consuming and inconsistent, assessment methods must be closely related to learning objectives, measurement and analysis. Assessment can be done through hard copy, electronic, online or interview. Behaviour Behavioral Evaluation, namely an evaluation of the behavior of participants in carrying out learning and being able to change their behavior after training or a few months later. It depen ds on the situation in which participants can apply their learning when returning to work with relevant skills and knowledge, when these changes affect the performance of trainees after returning to their roles, changes in behavior and new levels of knowledge are maintained, can transfer knowledge to others, and when participants see changes in their behavior, knowledge, skill level. Behavioral evaluation methods and tools, namely observation and interviews over time, are needed to assess changes, the relevance of changes, and the sustainability of changes. Evaluation of behavior change is possible with good support and involvement from line managers or trainees, so it is helpful to involve them early on, and to identify benefits for them. Result Evaluation of Results, namely evaluations that can increase the performance of trainees as a result of business or the environment. As a measure, usually the key performance indicators of a business or organization, such as value, percentage, time span, return on investment, and other quantitative aspects of organizational performance, for example; number of complaints, staff turnover, attrition, failure, waste, non-compliance, quality ratings, achievement of standards and accreditation, growth and retention. External factors greatly influence organizational and business performance, obscuring the true causes of good or bad results. Human Resource Management (HRM) The Definition of Human Resource Management (HRM) Eri Susan (2019) argues that HR is the factor that most supports the success of an organization. So it is necessary to do the management and development of human resources in International Journal of Human Capital Management, Vol. 7 (1), June 2023 108 | P a g e order to become productive human beings. Good HR management and development will help HR to complete the demands of tasks from now and in the future. Employee performance and skills will not be maximized without a debriefing and development process. HR has two meanings, namely micro and macro. Micro human resources are individuals or members who work in an agency or company which are referred to as employees, workers, or laborers. While macro human resources are residents of a country who have entered productive age, both those who have not worked and those who are already working. Currently, employees are not only seen as human resources, but more as capital or assets of an organization or institution of high value that need to be developed and not as a burden. So the perspective of HR as an institutional or corporate investment is increasingly prominent, so that a new term appears called Human Capital (HC). He also quotes from Basir Barthos' book (2012) which states that HRM is a field of management that specifically studies human relations and roles in corporate organizations which includes the development, use and protection of human resources both individually and when in work relationships. Meanwhile, quoted from Malay S.P. (2013) HRM is the art of regulating labor roles or work relations so that they are effective and efficient in helping the realization of company, employee and community goals. Likewise, as quoted from Sedarmayanti in her book Kadarisman (2013), the scope of HRM based on implementation and development tasks includes (1) Pre - service training given to prospective employees who want to work or new employees as provision to be ready to carry out their duties. employment, (2) In-service training conducted by employees when they work in organizations to improve their ability to carry out their work, (3) Post -service training is provided to employees to prepare them for retirement. According to Sondang (2018), HRM is an HR management activity carried out by managers in an organization, starting from job planning to employee retirement. HR needs to be handled professionally by specialists so that HR can be managed properly. Human Capital Management (HCM) The term Human Capital is often debated by many people compared to human resources (HR). Some people argue that Human Capital is a revolutionary way of managing people, but other opinions state that the concept of Human Capital is just one model of HR management. Those who disagree with the term Human Capital state that the term Human Capital does not humanize the human element in a company. While those who agree with the term Human Capital argue that the concept of Human Capital will place humans in an equal position with other capital such as machines and money in a company. Human Capital is currently accepted as an important element of the value of intangible assets. Lantip Diat P., (2017) quoting Becker (1965) states that the analysis of human capital starts from the assumption that individuals decide to attend education, training, medical care and others as knowledge and health by weighing the benefits and costs. These benefits include the cultural and other non-monetary benefits that come with increased income and employment, the costs of which are usually highly dependent on the value lost from the time spent on these investments. The concept used for human capital development in organizations can be seen from human capital indicators, human capital investment results and human capital success measures. Human capital indicators consist of task-related human capital (start-up experience and work management experience), and non-task-related human capital (education and work experience). While the results of human capital investment in the form of skills, knowledge, competence and motivation. While the measure of success can be seen from productivity, performance and welfare. Human Capital Management Strategy Program Design Lantip Diat P.et al (2017) states that the design of a human capital management strategy program is a decision-making process carried out by a leader in the organization by choosing the International Journal of Human Capital Management, Vol. 7 (1), June 2023 109 | P a g e right organizational structure with the organizational strategy and environment. The design of the organizational structure is used as a reference for employee activities in developing themselves and the organization. The design of the human resource management strategy program consists of seven components, namely: recruitment, selection, training, professional development, performance management, compensation and termination of employment. The seven components are structured so that HR can benefit the organization and itself. Improving organizational performance is the achievement of HR/employees, so that the context of leadership policies in the form of organizational strategy and role design (what keys are needed to carry out the strategy) play a very important role in creating competence and motivation in the form of knowledge, skills, attitudes, from role holders (humans) . Strategies for employee talent acquisition, employee development and talent retention can also drive the successful design of HC strategic management programs. Human capital management is the process of obtaining, training, managing and maintaining human workforce so that they can contribute effectively to the company (Prasojo, Mukminin & Mahmudah, 2017). Such an approach is seen as an extension of the traditional way - known as human resource management - to manage people more effectively and make organizations able to survive and face future competition. Development of Indonesian Navy's Human Resources as Human Capital Human resources are one of the greatest assets of the Indonesian nation in realizing Indonesia's national goals. Educated and qualified human resources are able to face global competition and play an active role in national and international activities. The military and civil servants of the Indonesian Navy are human resources who must be prepared to face global competition and create a world-class Navy with various improvements in all fields according to their specialization. The Indonesian Navy continues to strive to improve the quality of its human resources through various human resource development activities such as courses, training, seminars at home and abroad. The Indonesian Navy's English training was held at DISDIKAL as a manifestation of the leadership's concern for developing its human resources and giving them the opportunity to progress further and prepare themselves for an increasingly competitive era. The World Class Navy Salim (2015) said that having a world-class navy would increase Indonesia's position in the global defense capability index. The navy is one of the strategic components that is taken into account in the formulation of foreign policy towards certain countries. Border issues, trading activities, and international transportation across Southeast Asia's seas are increasing rapidly. The growing importance of regional waters is reflected in growing merchant fleets, increasing global dependence on seaborne trade, and greater exploitation of biological and non-biological resources. The global community is pressing Indonesia to improve the security of international sea lanes that pass through its sovereign waters. Yakti (2017) in his article stated that Jokowi's National Medium-Term Development Plan shifted from land-oriented to sea-oriented. Maritime defense and security is the main axis supported by the presence of the Indonesian Navy. The World Class Navy Doctrine was first issued by the Indonesian Navy prior to the declaration of Indonesia as the World Maritime Axis. To make this happen, the Indonesian Navy must achieve a Minimum Essential Force (MEF), namely creating professional soldiers equipped with defense equipment. Various problems must be faced by Indonesia, especially for the Indonesian Navy, because Indonesia is still far from having adequate maritime defense indicators even though the defense budget occupies the highest portion in the State budget (APBN). Two figures of classical thought: Alfred Mahan and Julian Corbe became a reference for Indonesia's achievements in military strategy, because Indonesia's position International Journal of Human Capital Management, Vol. 7 (1), June 2023 110 | P a g e is at the junction of two continents and oceans. Indonesia implements sea control as an effective way to control trade routes that are currently controlled by a small country like Singapore. In terms of sea power, the Indonesian Navy is categorized based on its territorial coverage: navy brown, green and blue water. However, currently the Indonesian Navy is still in the status of a green water navy with its capability only as a means of protecting the coast. The current focus is on the readiness of the Indonesian Navy in accompanying the government's steps in realizing the world's maritime axis and for the realization of a World Class Navy requires a high budget for upgrading defense equipment and developing human resource capabilities and increasing welfare. human resources or Navy soldiers. English Education and Training at DISDIKAL. English Language Training at DISDIKAL is a form of realizing the development of Indonesian Navy human resources. English Training is held every year on an ongoing basis for military personnel and Civil Servants from different units under the Indonesian Navy. The type of English training depends on the needs of the Indonesian Navy at different times. The trainings that will be held from 2019 to 2022 are the TOEFL English Training, Public Speaking English Training and Elementary Level English Training. TOEFL English Training is held to prepare Indonesian Navy personnel, especially officers who will take postgraduate, Ph.D., specialization, development and SESKO (Staff and Command College) education abroad. Meanwhile, English Public Speaking Training is held to prepare Indonesian Navy personnel to participate in international events such as seminars, workshops, meetings and assignments abroad. Elementary level English Training and Education is intended for Navy personnel who want to learn English starting from the basics as well as those who want to learn to a higher level such as intermediate. The number of training participants for each type of training is 24 people. The teachers come from DISDIKAL itself and outside DISDIKAL institutions such as the Center for Language Education of the Ministry of Defense of the Republic of Indonesia Education Service, Indonesian Navy (DISDIKAL) DISDIKAL is a research center that organizes English Language Education and Training. Based on the Regulation of the Chief of Staff of the Navy (2010) states that DISDIKAL is the Central Executive Institution of the Navy which is directly under the Chief of Staff of the Indonesian Navy (KASAL). Its task is carrying out educational function development which includes needs analysis, educational planning, educational development and education organizer as well as supervision, control and evaluation. The composition of DISDIKAL officials in general consists of the Head of the Naval Education Services, Secretary of the Naval Education Services, Head of the Planning Section, Sub Head of the Operational Education, Sub Head of the Educational Development, Sub Head of Education Curriculum, Sub Head of Education Control, and Sub Head of Education Quality. Evaluation Model Used This study uses the CIPPO evaluation model (Context, Input, Process, and Product, Outcome). The results of the evaluation can be used as a reference in implementing the next English Language Education and Training program. International Journal of Human Capital Management, Vol. 7 (1), June 2023 111 | P a g e METHODOLOGY This evaluation study uses a qualitative approach with a descriptive method. Data collection techniques using questionnaires, interviews, observation, focus group discussions and documentation. The questionnaire method is used to evaluate the context, input, process, product and outcome of the training. Questionnaires were given to 53 former English Training participants, 6 organizing staff, 6 teachers, 20 supervisors of former English Training participants from 2019 to 2022. Interviews were also conducted with 13 former Training participants, 6 organizing staff and 6 teachers. The research took place at DISDIKAL, MABESAL, Cilangkap, East Jakarta. Observations were made by observing, recording and searching for supporting documents for program activities. Documentation methods for collecting documents or data archives which include manuals/instructions, minutes, scientific papers and photos of activities. Data analysis used descriptive statistical analysis and Miles and Huberman's analysis which included data collection, data reduction, data presentation and conclusion drawing or verification. The evaluation method used in this study uses a modification of the Kirkpatrik model, as follows. CIPPO MODEL Context Context evaluation aims to evaluate a program as a whole starting from needs, problems, assets, opportunities, conditions and relevant contextual dynamics, program objectives, as well as identifying strengths, weaknesses, diagnosing problems, and providing appropriate solutions to problems that arise when program implementation. At the context stage of the English Education and Training program, the focus of the discussion is gathering information related to th e needs, problems and objectives of the English Education and Training program. Input Input Evaluation to find out program planning including readiness and planning for English Language Training by compiling an activity plan based on the Indonesian Navy's educational plan so that educational goals can be achieved according to standards. With related institutions such as Navy personnel Services and Navy security staff, DISDIKAL works together to have the training participants. DISDIKAL as the education and training organizer also prepares the budget, educational curriculum, teaching staff, program administration staff, instruction packages or teaching materials, and training facilities. Process Process Evaluation with regard to program implementation aims to detect whether the procedures and implementation of the English Education and Training program are in accordance with the program plan, whether the teaching methods that are being applied are appropriate with the Education and Training and are of good quality, whether there are any obstacles during program implementation. In order to obtain information about the implementation of the training, the organizers document the activities, assess and report the implementation of the education and training. Product Product evaluation is an activity to identify and assess the success of the Education and Training program; the obstacles encountered during the implementation of the Education and International Journal of Human Capital Management, Vol. 7 (1), June 2023 112 | P a g e Training program; whether the achievement of the Education and Training program is quite significant and comparable to the budget spent; how the development of human resources is in accordance with the standards set in the program planning and whether this training program can be used again for the next program or needs to be improved. Outcome Evaluation of outcome aims to provide information about the impact achieved from a program or activity compared to the expected results. Evaluation of outcome aims to determine the results of the HR development program for organizational improvement. After completing the training program, the skills of former training participants are expected to be used for organizational progress and to improve their performance in the organization. RESULTS AND DISCUSSION The results of the evaluation of the English Language Education and Training program at DISDIKAL for the Indonesian Navy were carried out through interviews, observation, documentation, focus group discussions and CIPPO model questionnaires (context, input, process, product and outcome) with respondents 53 former training participants, 6 organizer staff and 6 teachers. The results of the questionnaire show: (1) the evaluation of the training context shows an average questionnaire of 3.81. It means that the implementation of English Language Education and Training at DISDIKAL is in accordance with the objectives, education plans, training activity plans; (2) the training input evaluation shows an average questionnaire of 3.59. It describes that the implementation of English Language Education and Training in general has satisfactory input with the certain aspects to be concerned such us the proposals for training participants are not fully based on offers, the basic abilities of training participants are still heterogeneous, some participants attend training because of the orders from superiors. The number of training support staff is still lacking, mess facilities are not optimal and the education and training budget needs to be increased; (3) the evaluation of the education and training process shows an average questionnaire of 3.82. It describes that the process of organizing English language education and training in general can be carried out well with the concern on the role of leaders, organizers, support staff, and the Education and training environment should be improved; (4) evaluation of education and training products shows an average questionnaire of 3.96. It means that the process of organizing English Education and Training is very beneficial for institutions as well as individuals because it can increase self- confidence and English language skills, and (5) the evaluation of Education and Training outcomes shows an average questionnaire score of 3.95. It shows that the implementation of English language training is very helpful for the Indonesia Navy to have the participants who will study, be assigned abroad and take part in international events. The results of the evaluation of the English language training can be seen in the following chart. International Journal of Human Capital Management, Vol. 7 (1), June 2023 113 | P a g e CONCLUSION The results of the CIPPO evaluation show that the implementation of English Language Training at DISDIKAL is in accordance with the objectives, education plans, and training activity plans. The objectives of the education and training can be achieved by the large number of former English training participants attending schools abroad such as Officer Advanced Education, Command Staff College and National Resilience Institute as well as the peace missions assignments in conflict countries. Other activities such as seminars, workshops, joint exercises with friendly countries can also be attended by former English training participants. In general, the implementation of English Language Education and Training has satisfactory inputs. Sev eral things need to be improved namely the candidates for training participants should have the English language skills according to the level of education and training. The number of training support staff must be increased. The mess and library facilities need to be improved. The research can run smoothly with some limitations, for some interviews and questionnaires cannot be carried out directly but by using via email and WhatsApp. It is expected that the results of this research can be used as input for the implementation of English language training at DISDIKAL and other institutions in the Indonesian Navy which organize similar training. In addition, the English language training will continue to be established to improve the English language skills of Indonesian Navy personnel to become the World Class Navy and the education and training can be operated in the separate institutions that focus on education and training. As a result, the organizers, teachers and the learning environment are really not disturbed by offices’ activities. REFERENCES A. Muryadi. (2007). Model Evaluasi Program Dalam Penelitian Evaluasi (Jurnal Ilmiah PENJAS, Vol 3, No 1, January 2007) page 1, 9-10, ISSN: 2442-3874. D.L. Stufflebeam. (2014). Systematic Evaluation, A Self-Instructional Guide to Theory and Practice, e-ISBN-13:978-94-009-5656-8, DOI: 10.1007/978-94-009-5656-8. 3.81 3.59 3.82 3,96 3,95 CONTEXT INPUT PROCESS PRODUCT OUTCOME CIPPO RESULT LIKERT SCALE (1-5) International Journal of Human Capital Management, Vol. 7 (1), June 2023 114 | P a g e D.L. Kirkpatrick and James. (2007). Implementing the four levels, A practical Guide for Effective Evaluation of Training Programs. ISBN 978-1-57675-454-2, PDF ebook ISBN 978-1- 57675-532-7,2007-1. Dalam http://www.ct.gov/ctdn/lib/ctdn/ttt_14_ m5handouts2.pdf. E. Susan. (2019), Manajemen Sumber Daya Manusia, Volume 9, No. 2, P-ISSN: 2407-8107, E- ISSN: 2685-4538, https://jurnal.iain-bone.ac.id/index.php/adara/article/view/429/354. Prasojo Lantip, Mukminin Amirul, Mahmudah F. (2017). Manajemen Strategi Human Capital Dalam Pendidikan.Yogyakarta: ISBN 978-602-5566-02-8. Peraturan Kepala Staf Angkatan Laut, Buku Petunjuk Pelaksanaan Penyusunan Pokok - Pokok Organisasi dan Prosedur serta Organisasi dan Prosedur di Lingkungan TNI Angkatan Laut (PUM-1.100, 2010). Salim. (2015). A Pathway to Indonesia’s maritime future: the role of maritime policy, doc trine and strategy, Soundings, Commonwealth of Australia. July 2015, Issue No 5 page 6-7. S. Arikunto. (2012). Evaluasi Program Pendidikan. (Resume, January 14, 2012) page 1-2. Dalam http://anannur. Blogspot.co.id/2012/01/evaluasi-program-pendidikan-prof dr.html?m=1. Sondang. (2018). Manajemen Sumber Daya Manusia. Jakarta: PT Bumi Aksara. ISBN 979-526- 100-2. Y. P. Darono. (2017). TNI Angkatan Laut sebagai World Class Navy pada Strategi Pertahanan Indonesia dalam Mewujudkan Poros Maritim Dunia. Edisi Jurnal No.2 tahun ke-41 Departemen Hubungan Internasional, Fakultas Ilmu Sosial dan Politik, Universitas Airlangga. (Gema Keadilan, majalah Pemikiran Umum dan Sosial, 2017), ISSN: 0852-0011.