BCP Education & Psychology ESPS 2023 Volume 10 (2023) 180 Construction of Evaluation System for the Effect of Party’s History Learning and Education for College Students -- Taking Guangdong Higher Vocational Colleges as Examples Yaojia Lin a, Zhuomeng Liu b, Di Li c School of Applied Foreign Languages, Guangdong Polytechnic of Industry and Commerce, Guangzhou, 510000, China a 380766475@qq.com, b 2472904922@qq.com, c lidi@gdgm.edu.cn Abstract. This paper conducts an in-depth study on the Party’s history learning and education of college students in Guangdong higher vocational colleges. Through the questionnaire design, the current situation, problems, and countermeasures of Party’s history learning and education have full investigation, understanding, study, analysis and discussion. Taking Guangdong vocational college students as the research object, this article adopts quantitative and qualitative methods, mainly based on the Kirkpatrick model, to construct evaluation indicators for Party’s history learning and education, and uses the AHP model to design the weights of each indicator to conduct consistency testing. And then this study constructs an evaluation system for the effectiveness of youth Party’s history learning and education in Guangdong vocational colleges, providing new ideas for proposing practical paths, and providing reference for the normalization, scientization, and long-term effectiveness of Party’s history learning and education. Keywords: AHP Model; Party’s History Learning and Education; Kirkpatrick Model; College Students. 1. Research Background and Significance The centennial Party’s history embodies the essence of the excellent traditional culture of the Chinese nation, highlighting a series of great spirits such as the spirit of initiative, struggle, dedication, patriotism, and collectivism, and highly fitting the multi-dimensional value orientation of ideological and political education teaching in colleges and universities, such as its educational, historical, cultural, and practical nature. The history of the Communist Party of China is not only the “best textbook” and inexhaustible resource pool for ideological and political education in colleges and universities, but also an important component of the ideological and political curriculum system and a compulsory course in higher vocational education. “We should use history to reflect the reality and look into the future. From the centennial struggle of the CPC, we should understand why we succeeded in the past and how we can continue to succeed in the future, so that we can be more firm and more conscious in remembering our original mission and creating a better future in the new journey.”[1] Currently, in the context of comprehensively constructing the “Great Ideological and Political Pattern”, vocational colleges have successively opened courses related to Party’s history education, gradually enriching the teaching content, diversifying the teaching forms, and gradually expanding the breadth and depth of education. However, there are still some problems behind these achievements: such as a weak foundation for Party’s history research, limited development and application of Party’s history resources, lack of systematic and long-term Party’s history education mechanisms, fragmentation of Party’s history learning, and poor learning and education results. The development of Party’s history learning and education among young people has its special significance and also requires different methods. Practice has proven that repeating what the textbook says can get half the results with twice the effort. Therefore, it is necessary to deeply study the effectiveness of Party’s history learning and education in vocational colleges and establish an evaluation index system for the effectiveness of learning and education. At the same time, based on the requirements of the new era for the normalization of youth Party’s history learning and education, BCP Education & Psychology ESPS 2023 Volume 10 (2023) 181 a more scientific and reasonable evaluation of the effectiveness of youth Party’s history learning and education is conducted to provide reference and decision-making for optimizing learning and education programs in the later stage. 2. Research Introduction 2.1 Research Theory From a theoretical perspective, the models for learning and training evaluation can be roughly divided into two categories. One is the model for the learning and training process, such as the CIRO training evaluation model and the CIPP evaluation model. The other type is a model that focuses on the effectiveness of learning and training. The most widely known, classic, and widely used model in this category is the Kirkpatrick Model for Training Evaluation, which was proposed by internationally renowned scholar and professor Donald L. Kirkpatrick in 1959. Later, with the changes and development of the times, other relevant evaluation models have been derived from this model. This study focuses on the effectiveness of Party’s history learning and education for vocational college students, mainly using the Kirkpatrick model. The Kirkpatrick model holds that the evaluation of training effectiveness is divided into four levels, namely, the reaction level, the learning level, the behavior level, and the result level. 2.2 Overview of Research Objects This study focuses on young college students in higher vocational colleges in Guangdong Province. Guangdong Province has 625 vocational colleges with 2.805 million students, including 93 vocational colleges with more than 1.25 million students, an increase of 19.2% and 83.1% respectively compared to 2011. Undergraduate level vocational schools have sprung up from scratch, including Guangzhou Vocational and Technical University of Science and Technology (Guangzhou) and Guangdong Business and Technology University (Zhaoqing), with 19000 students. It can be said that higher vocational education has developed quite rapidly in Guangdong, accounting for half of higher education in terms of number and scale, with students in college accounting for 50.1%. In terms of the connection of vocational education and training system, in 2022, 52% of secondary vocational schools in Guangdong Province carried out the pilot project of secondary and higher vocational education in 3+2-year divisional systems, with an enrollment plan of 86000 students, 11.9 times that of 2011, and the promotion rate of secondary vocational graduates reached 44%. The pilot program for the collaborative education of students who upgrade from junior college to university in 3+2-year divisional systems has 12000 students, with 65000 students enrolled in upgrading from junior college to university, and 13.2 times the number in 2011. 22.4% of higher vocational graduates are admitted to undergraduate universities. Over the past decade, it has provided about 7.69 million skilled elites to the society, and the employment rate of graduates has remained above 95% for many consecutive years [2]. Over the past 10 years, Guangdong Province has built 14 national “Double High Plan” schools and 29 provincial or above demonstration higher vocational colleges. Regardless of the size of students in college or graduates, such a large amount of data has laid a good sample foundation for research and further demonstrated the importance of research. 2.3 Research Content and Methods 1. Literature analysis: This study uses the literature research method to collect and collate current papers, works, and academic achievements related to this research through multiple channels such as China National Knowledge Internet, university library, China Doctor/Master Dissertations Full-text Database and various journals, to understand the relevant research status and trends as well as the research objects. Through literature analysis, this article collects relevant research on Party’s history learning and education at home and abroad, and provides detailed preparation for the construction of indicators through theoretical and practical cases such as evaluation indicators for the effectiveness of learning and education. BCP Education & Psychology ESPS 2023 Volume 10 (2023) 182 2. Expert interview: This study conducted interviews with domestic experts and scholars in the field of Party’s history education, as well as teachers and grassroots educators with rich experience in ideological and political education in higher vocational colleges, in the form of email, phone calls, and interviews, to obtain theoretical basis and professional experience support related to the research topic, providing detailed information and suggestions for the indicators and impact factors of the effectiveness evaluation system of Party’s history learning and education. 3. Questionnaire survey: A questionnaire survey was conducted among 2500 young college students randomly selected from 10 representative vocational colleges to understand the basic situation of young college students regarding their understanding of Party’s history, teaching and satisfaction with Party’s history, as well as their confusion, problems, challenges, changes after conducting Party’s history learning and education, self-perception, and the evaluation of tutors and head teachers. At the same time, it was also used to understand the impact of Party’s history education on college students’ values, worldviews, and outlook on life. 4. Analytic Hierarchy Process: Analytic Hierarchy Process (AHP) uses quantitative analysis to solve qualitative problems, and tries to eliminate the impact of personal perspectives on processes and conclusions in research to ensure their objectivity. This study conducted a total of 100 sample questionnaires and 20 offline interviews with experts, scholars, and teachers from universities, educational management departments, academies, research institutes, and other related fields. Through the combination of interviews and survey results, validation is conducted to establish weight factors, evaluation hierarchy models and judgment matrices, so as to obtain the weights of each indicator. 5. Innovative experiment: Innovative practice is a major feature of higher vocational education, and students’ interest in practical learning is significantly higher than theoretical courses. Focus on the first and second classrooms, and carry out practical innovation through the provincial high-quality project of ideological guidance for college students and the provincial high-quality project of student work. Discover and summarize the problems and characteristics in practice. Through classroom practice, discussion and assessment, focusing on the current situation and problems of the teaching content and target positioning of Party’s history, and examining them with modern teaching concepts, this paper proposes a comprehensive and hierarchical practice path of “the party, league, class and group”. 3. Construction of Evaluation System for the Effect of Party’s History Learning and Education for College Students 3.1 Introduction to the Evaluation Model for the Effect of Party’s History Learning and Education This study mainly focuses on the evaluation of the effectiveness of Party’s history learning and education. According to the most widely used evaluation model currently, the Kirkpatrick Model is adopted. Based on a large amount of literature research and combing the characteristics of higher vocational students in China, a total of 16 evaluation indicators reflecting the requirements of the Party, the country, universities, and society for young students in the new era have been added through research and interviews with experts, scholars, and frontline educators. This study takes Marxist methodology as the fundamental guidance, constructs a comprehensive, systematic and targeted methodology system, combines traditional methods with emerging methods, and combines quantitative and qualitative analysis to ensure a more scientific and standardized evaluation of the quality of Party’s history learning and education [3]. The selection of evaluation indicators is the core and basic point of constructing an evaluation system for the Party’s history learning and education of college students in Guangzhou Vocational College. It mainly includes four layers: reaction layer, learning layer, behavior layer, and result layer. As shown in Table 1. BCP Education & Psychology ESPS 2023 Volume 10 (2023) 183 The reaction layer is mainly used to observe the reactions of students. From the perspective of students, it mainly includes five evaluation indicators, which includes satisfaction with learning and educational content, satisfaction with learning methods, and satisfaction with lecturers, guarantee of learning and education conditions, participation and completion of learning and education. The learning level is mainly used to assess the learners’ learning proficiency and focus on their learning gains. This study mainly includes three evaluation indicators, including theoretical knowledge, skill knowledge, and ideal and belief education. Traditional classroom learning education evaluates students’ mastery, or teaching objectives, generally from three dimensions: knowledge, skills, and attitude. In addition to the evaluation indicators for Party’s history learning education, evaluation indicators such as ideals, beliefs, and confidence commitments are added. The purpose of learning and education is to strengthen the education of college students’ ideals and beliefs, achieve the effect of establishing morality and cultivating people, and strengthen the education of loving the party, patriotism, and school, fully understand the advantages of the socialist system with Chinese characteristics in the new era, enhance confidence in the future and development of individuals and the country, so as to better put into action. The behavior level mainly measures daily performance before and after learning and education, as well as the use of knowledge learned by students. This study mainly includes four evaluation indicators, including actively approaching the Party and Youth League organizations (submitting the situation of joining the Party and Youth League), doing practical things for teachers and students, acting consciously, as well as the condition of honesty, integrity and self-discipline. Table 1. Evaluation indicators for the effectiveness of Party’s history learning and education for college students in higher vocational colleges Target Layer Criteria Layer Scheme Layer A Evaluation on the effectiveness of Party’s history learning and education for college students in higher vocational colleges B1 Reaction layer C1 Satisfaction with learning and education content C2 Satisfaction with learning and education methods C3 Satisfaction of lecturers (presenters) C4 Learning and education guarantee C5 Participation and completion of learning and education B2 Learning layer C6 Theoretical knowledge C7 Skill knowledge C8 Ideal and belief education B3 Behavior layer C9 Approaching the Party and Youth League organizations (submitting the situation of joining the Party and Youth League) C10 Doing practical things for teachers and students C11 Act consciously C12 Honesty and self-restraint B4 Result layer C13 Class atmosphere and style of study C14 Participation in Party and League activities C15 Teacher evaluation C16 Related awards BCP Education & Psychology ESPS 2023 Volume 10 (2023) 184 The result layer is mainly about the benefits of learning and education. This study considers four evaluation indicators for internal and external effectiveness, including class atmosphere and style of study, participation in Party and League activities, teacher evaluation, and related awards. 3.2 Dimensionless Evaluation of Indicators An expression that represents a physical derived quantity by the product of the powers of several basic quantities is called the dimensional expression of the physical quantity, or simply the dimension. Dimensionless, which uses mathematical methods to eliminate the impact of original variables through mathematical changes, is commonly referred to as data standardization. Standardization (Z-score) As shown in 1: s xx y i i   (1) In above equation:    n i ixn x 1 1      n i i xx n s 1 2)( 1 1 3.3 The Determination of The Weight of Evaluation Indicators Based on the determination of evaluation indicators, this study uses the AHP model to analyze the weight of evaluation indicators for the effectiveness of college students’ Party’s history learning and education, mainly including four steps: constructing a hierarchical structure, calculating and determining indicator weights, consistency testing and weight analysis, and constructing an overall learning and education effectiveness evaluation indicator system. 3.3.1. Construction of Hierarchical Structure Based on the selected evaluation indicators, this study excluded the impact of personal perspectives on the research conclusions, and combined with the questionnaire to determine the weight of each evaluation factor using analytic hierarchy process. According to the Delphi method, the predictive opinions of scholars and frontline ideological and political educators were consulted. In this study, a total of 10 consultation opinions were issued, with 2 people each time, totaling 20 people. Through the collation and analysis of the questionnaire by the project team, good consistency was obtained. This study compared four hierarchical structures, namely, the reaction layer (B1), the learning layer (B2), the behavior layer (B3), and the result layer (B4), and registered and assigned their importance to establish a judgment matrix. The research mainly used the 1-9 scale method proposed by Satty.[4] The scale method was widely used and suitable for this study. Then used Table 2 to calculate the judgment matrix for comparing two pairs. Table 2. Satty Scale Method Scale Connotation 1 ai indicators have the same importance as aj indicators 3 ai indicators are slightly important than aj indicators 5 ai indicators are significantly important than aj indicators 7 ai indicators are more important than aj indicators 9 ai indicators are extremely important than aj indicators 2, 4, 6, 8 between the median of the above two adjacent judgments Reciprocal Indicator / ija a ai j , and /a aj i , so 1/ ji ija a Note: ai represents the ith indicator value, and aj represents the jth indicator value. BCP Education & Psychology ESPS 2023 Volume 10 (2023) 185 3.3.2. Calculation and Determination of Indicator Weights First of all, this study needs to compare the criteria layer and first construct a judgment matrix for the evaluation of the effectiveness of college students’ Party’s history learning and education, as shown in Table 3. Table 3. A Evaluation and judgment matrix for the effectiveness of college students’ Party’s history learning and education A The attraction of national forest parks B1 Reaction layer B2 Learning layer B3 Behavior layer B4 Result layer B1 Reaction layer 1.0000 0.2500 0.2000 0.3333 B2 Learning layer 4.0000 1.0000 0.5000 2.0000 B3 Behavior layer 5.0000 2.0000 1.0000 3.0000 B4 Result layer 3.0000 0.5000 0.3333 1.0000 Single level sorting 1) Sort out and normalize by column to obtain the judgment matrix: 0.0769 0.0667 0.0984 0.0526 0.3077 0.2667 0.2459 0.3158 0.3846 0.5333 0.4918 0.4736 0.2308 0.1333 0.1639 0.1580 2) Phase weighted of rows: 1 1 ij n j W b   =0.0769+0.0667+0.0984+0.0526=0.2946 The rest may be deduced by analogy → 2W =1.1361; 3W =1.8833; 4W =0.6860 3) Vector normalization + + + =4.0000 0737.0 4 2946.0 W 1 1 1    n j jW W ; Similarly, 2W =0.2840, 3W =0.4708, 4W =0.1715, The resulting eigenvector matrix is: W= [0.0737,0.2840,0.4708,0.1715]T Calculating the maximum eigenvalue of a judgment matrix: max 0.0769 0.0667 0.0984 0.0526 0.3077 0.2667 0.2459 0.3158 0.3846 0.5333 0.4918 0.4736 0.2308 0.1333 0.1639 0.1580 0.2946 1.1361 1.8833 0.6860 (AW)1=0.0769*0.2946+0.0067*1.1361+0.0984*1.8833+0.0526*0.6860=0.3198 (AW)2=1.0734; (AW)3=1.9703; (AW)4=0.6365; Normalize (AW)i, and obtain:0.0800;0.2684;0.4926;0.1599; So, max =   n 1 )( i i i nW AW 1.0959+0.9451+1.0463+0.9324=4.0197; 3.3.3. Verification of Consistency To test the accuracy of the judgment matrix, use Equation 2 for consistency testing. BCP Education & Psychology ESPS 2023 Volume 10 (2023) 186 CI CR RI  (2) CR is the random consistency ratio of the judgment matrix. CI is the general consistency index of the judgment matrix determined by formula (3). RI is the average random consistency index of the judgment matrix. When CR<0.1, the consistency of the judgment matrix is considered acceptable, whereas when CR>1, the judgment matrix does not meet the consistency requirements and needs to be corrected again. max 1 n CI n     (3) After calculation: CI=0.00657. According to the research of CSL Research Office, Institute of Software, Chinese Academy of Sciences, the calculation of higher-order average random consistency index (RI) in the analytic hierarchy process (AHP) [5], intercept the average random consistency index RI value when n is from 1-9. As shown in Table 4. When n=4, RI=0.90. Table 4. The value of the average random consistency index RI n 1 2 3 4 5 6 7 8 9 10 RI 0 0 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 So, 007296.0 9.0 00657.0I CR  RI C CR<0.1, so the judgment matrix has a very satisfactory consistency. Therefore, the weights of the criterion layer (reaction layer B1, learning layer B2, behavior layer B3, and result layer B4) are 0.0737, 0.2840, 0.4708, and 0.1715, respectively. 3.3.4 Construct a Judgment Matrix for Four Scheme Layers Using the same method, this study collected questionnaires from survey experts, scholars, frontline workers, etc., constructed a judgment matrix for the remaining four scheme layers, and calculated the weight of the overall indicator system, as shown in Table 5. 3.3.5 Description of Weight Analysis From the perspective of reaction level indicators, C5 the degree weight of participation and completion of learning and education>C3 satisfaction of lecturers (presenters)>C1 satisfaction with learning and education content>C2 satisfaction with learning and education methods>C4 learning and education guarantee. Whether students are fully involved in learning and education and how well they participate in it are the most important issues for universities to carry out this activity. Colleges and universities should take the study and education of Party’s history as the core component of the ideological and political teaching system, redeploy, implement, and strengthen the study and education of party history, and conduct the study and education of party history completely and systematically [6]. From the perspective of learning level indicators, C8 the degree weight of ideal and belief education>C6 theoretical knowledge>C7 skill knowledge. The learning and education of party history first focuses on the education of ideals and beliefs, and then focuses on the learning of theoretical knowledge. “The history of the Party is the most vivid and persuasive textbook. Our Party has always attached great importance to the study and education of the Party’s history, paid attention to using the Party’s struggle process and great achievements to inspire morale and clarify the direction, used the glorious tradition and fine style of the Party to strengthen faith and pool strength, and used the Party’s practical creation and historical experience to enlighten wisdom and sharpen character [7].” BCP Education & Psychology ESPS 2023 Volume 10 (2023) 187 From the perspective of behavior layer, C12 honesty and self-restraint >C11 act consciously >C10 doing practical things for teachers and students>C9 approaching the Party and Youth League organizations (submitting the situation of joining the Party and Youth League). “If the historical perspective is wrong, not only will it not achieve the goal of learning and education, but it will instead lead to a different direction and lead to a misunderstanding [3].” Table 5. Evaluation Effect, Evaluation System and Weight of Party History Learning and Education Criteria Layer B Scheme Layer C Single Weight Final Weight B1 Reaction layer 0.0737 C1 Satisfaction with learning and education content 0.2275 0.0168 C2 Satisfaction with learning and education methods 0.1142 0.0084 C3 Satisfaction of lecturers (presenters) 0.2483 0.0183 C4 Learning and education guarantee 0.0831 0.0061 C5 Participation and completion of learning and education 0.3269 0.0241 B2 Learning layer 0.2840 C6 Theoretical knowledge 0.2059 0.0585 C7 Skill knowledge 0.1046 0.0297 C8 Ideal and belief education 0.6895 0.1958 B3 Behavior layer 0.4708 C9 Approaching the Party and Youth League organizations (submitting the situation of joining the Party and Youth League) 0.0955 0.045 C10 Doing practical things for teachers and students 0.2568 0.1209 C11 Act consciously 0.3021 0.1422 C12 Honesty and self-restraint 0.3456 0.1627 B4 Result layer 0.1715 C13 Class atmosphere and style of study 0.2678 0.0459 C14 Participation in Party and League activities 0.2409 0.0413 C15 Teacher evaluation 0.3234 0.0555 C16 Related awards 0.1679 0.0288 From the result layer, C15 the degree weight of teacher evaluation>C13 class atmosphere and style of study>C14 participation in Party and League activities>C16 related awards. Teacher evaluation here refers to instructors, head teachers, and other teachers who directly participate in the process of students’ Party’s history learning and education activities. Counselors and head teachers have the most intuitive feeling and voice because they have the closest contact with students. 4. Conclusion In the new era, carrying out Party’s history learning and education for college students is a strategic need to strengthen ideals and beliefs, cultivate new people of the era, and promote the great cause of socialism with Chinese characteristics. If young people are strong, then the country is strong. It is BCP Education & Psychology ESPS 2023 Volume 10 (2023) 188 particularly important to make more effective use of Party’s history learning and education to cultivate young students. From the perspective of students and frontline educators, this study uses the Koch training evaluation model to divide the evaluation of the effectiveness of Party’s history learning and training into four levels: reaction level, learning level, behavior level, and result level, and constructs an evaluation index system. It is hoped that it can provide a reference for universities to carry out Party’s history learning and education. References [1] Jinping Xi. Speech at the Congress Celebrating the 100th Anniversary of the Founding of the CPC[N]. The People’s Daily, 2021-07-02(2). [2] Yuyang Yao. Guangdong Holds a Vocational Education Conference: Focusing on High Quality Development of Vocational Education[N]. Nanfang Daily, 2022-09-22. [3] Liangfa Zhou, Yuhan Wu. Solid promotion of multi-dimensional evaluation of Party history learning and education quality [J]. Theoretical Research, 2021(10): 50-56. [4] Thomas L. Saaty, Dynamic Priorities In the AHP/ANP: Generalizing from Points to Functions and from Real to Complex Variables, Mathematical and computer Modelling. [J]Time Dependent Decision making, 2007, 46(7):860-889. [5] Zhiguo Hong, Yan Li, Zhihua Fan, Yong Wang. Calculation on High-ranked RI of Analytic Hierarchy Process[J]. Computer Engineering and Applications, 2002(12):45-47. [6] Hongyong Li. Carry out Party’s history learning and education throughout the entire process of establishing morality and cultivating people[N]. Jiangxi Daily, 2022-04-12. [7] Jinping Xi. Speech at the Party’s History Learning and Education Mobilization Conference [M] Beijing: People’s Publishing House, 2021. [8] Shumin Meng. The Value, Concept, and Method of Studying and Educating Party History for College Students in the New Era[J]. Chinese Higher Education. 2021(17)40-42.