BECE, Vol.9, No.1, 2021 1163 Exploration of the Non-Cognitive Ability Assessment The Scientific Value of the Survey on Social and Emotional Skills Xiaoqiao Cheng School of Institute of Education Science, Nanjing Normal University, Nanjing 210097, Jiangsu, China “The cognitive development of the players is enormously im- portant, as well as social competence and character values such as discipline and teamwork.” –Joachim Low OMPARED with cognitive skills, non-cognitive skills cannot be ac- complished through simple academic level tests. It is more through self- reported research and observation reports of others. These two methods are relatively easy to be proposed and tested, but they are easily affected by so- cial expectations, response style and familiarity, resulting in deviations from the real results (Braun et al., 2001; He & Van de Vijver, 2015). Therefore, non-cognitive ability assessment is more challenging. The PISA test coordinated by the Organization for Economic Coopera- tion and Development (OECD) has become a teaching evaluation test system recognized by countries. The test measures the cognitive skills of 15-year-old adolescents around the world by testing and comparing students’ reading, mathematics, and science every three years. Since most of the content of the test is the cognitive category of the students, it is considered that “too much emphasis on standardized testing, narrowing the content of students’ learn- ing, ..., paying too much attention to the cognitive ability of young people while ignoring the development of non-cognitive ability” (Huang, 2019). © 2021 Insights Publisher. All rights reserved. Creative Commons Non Commercial CC BY-NC: This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (http://www.creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed by the In- sights Publisher. C BECE, Vol.9, No.1, 2021 1164 In recent years, the OECD has also begun to focus on the research and development of non-cognitive capability. It tried to introduce international evaluation standards to test students’ creativity to test the “soft skills” of stu- dents from various countries, that is, social and emotional skills such as crea- tivity, teamwork, communication and negotiation, and named it Social and Emotional Skills. Based on related research, the evaluation framework for students’ social and emotional skills has been constructed, and the evaluation project of “So- cial and Emotional Skills” for adolescents were launched in 2018. This aims to evaluate the social and emotional skills of young people in participating countries and cities, and how to improve these skills through education, so as to promote the balanced development of cognitive skills and social emotions. At the end of 2019, the first round of official evaluation was completed in 10 cities in 9 countries around the world. On September 7, 2021, the OECD of- ficially released the global report on the Survey on Social and Emotional Skills at its headquarters in Paris, France. The evaluation of the SSES project is based on complex measurement methods designed to capture characteristics that are difficult to quantify. It uses the “Big Five” personality characteristics as the fundamental test system. It measures 19 specific social and emotional skills produced by the “five” personality factors of psychology. Furthermore, through a tripartite question- naire of parents, teachers, and schools, collect information about the learning environment of students’ homes, schools, and communities, and finally ob- tain information about the conditions or practices that cultivate or hinder the development of these key skills. In terms of measurement methods, unlike other large-scale international assessments PISA conducted before, the OECD measures social and emo- tional skills based on the student-parent-teacher “triangulation” model (Kankaraš et al., 2019). That is, in addition to the traditional student self- evaluation, the indirect evaluation of the students’ social and emotional skills by parents and teachers has improved the reliability of the evaluation. In or- der to achieve mutual verification between the collected information, the primary guardian is responsible for filling out the student-related information in the parent questionnaire. The teacher questionnaire is completed by the teacher who knows the student best, and each teacher is responsible for no more than 10 students. The school questionnaire is completed by the school principal or person in charge. In terms of measurement tools, in order to solve the validity problem and correct the methodological limitations of self-reporting and observer re- porting, the SSES test combines quantitative and qualitative methods. After many rounds of empirical inspection and testing, the evaluation rubric for each skill is finally developed. The rubric is not composed of simple, straightforward, and simple items of a certain reading level, but contains a series of overall items that share the same orientation and measure the same structure. Multiple rounds of inspection and testing include cognitive inter- BECE, Vol.9, No.1, 2021 1165 view, online survey, topic experimental test, trial test and final formal test (OECD, 2021, September 7). As for the test subjects, the SSES evaluation comprehensively considers the age of the students and the sample size when selecting the samples to en- sure that the samples are highly effective and representative. Compared with PISA, the SSES test object has been expanded, not only for 15-year-old stu- dents, but also for 10-year-old students. Each country participating in the project has no less than 6,000 test subjects. In addition to students, the ob- jects also involve parents of students, teachers who are most familiar with students, and school principals or principals (OECD, 2021). Prior to this, other social and emotional competence tests have been carried out interna- tionally, such as the Program for the International Assessment of Adult Competencies (PIAAC), the International Early Learning Study (IELS) and the Teaching and Learning International Survey (TALIS). TALIS is aimed at teachers, PIAAC and IELS are skills tests for adults and 5-year-old children respectively. Therefore, in addition to more diverse testers, the SSES test al- so fills in the gap in the experience data of 10 to 15-year-old teenagers in the relevant international tests. Compared with the existing research on social and emotional ability in the world (such as the social emotional learning project organized by “Aca- demic, Social and Emotional Learning” in the United States, the “Teacher Emotional Intelligence” training project developed by the Yale Center for Emotional Intelligence at Yale University and the United Nations The UNICEF “Social Emotional Learning” project, etc.), the SSES project achieved a breakthrough in sample size and region. SSES is based on a rep- resentative sample of students and is the first large-scale international study on the social and emotional skills of 10-year-old and 15-year-old adolescents and their impact on learning outcomes. Even though many of the indicators and measurement dimensions it uses are derived from existing research, its greatest value lies in allowing these indicators and measurement dimensions to be measured and comparable on a large scale in different countries and cultural backgrounds. This not only promotes peer learning between coun- tries, but also provides an international model for global non-cognitive abil- ity testing. At the same time, we are also worried about some problems that may exist in the SSES test. For example, did they pay attention to cultural and language differences in the preparation of the test questionnaire and interpre- tation of the results? Will using the same questionnaire or scale between people of different languages and cultures affect the accuracy of the data? Will it cause people to distrust the questionnaire translation? From the previ- ous PISA test, although a series of measures have been taken to improve the reliability and validity of the test tool during the test, the limitations of geo- politics, linguistics and cultural background cannot be completely eliminated (Forestier, & Adamson, 2017). In addition, in terms of the representativeness of sample selection, the first round of testing was conducted in 10 cities in 9 countries around the world. Then there were obvious differences in economy, BECE, Vol.9, No.1, 2021 1166 politics, culture and education among these 10 cities, which will inevitably affect the performance of students in social and emotional skills. How to de- fine the relationship between the social and emotional skills of students and the factors mentioned above is also a question that needs to be explored (Zhou, 2021). Based on the above considerations, how to enhance the effectiveness of cross-regional non-cognitive ability assessment is of great significance. For example, in the evaluation method, the use of intelligent information tech- nology (facial expression, action posture, eye movement, infrared brain im- aging, etc.) can be used to explore diversified evaluation methods. In the evaluation tools, more contextual evaluation tasks are created. Using real problem situations as the carrier, examine students’ creativity, perseverance, emotional regulation ability and social interaction and other advanced and complex constructions (Zhang & Liu, 2021). In addition, in terms of evalua- tion indicators, relying on the general indicators in the SSES project to help build localized evaluation indicators for each country, and conduct multi- level and dynamic monitoring and measurement of social and emotional abil- ity indicators, thereby forming a scientific localized evaluation system (Zu & Kyllonen, 2019). Finally, we should know that non-cognitive ability is different from cognitive ability. It’s not about highlighting specific characteristics to the maximum, but about achieving balance. Therefore, the idea of the SSES pro- ject is not to create rankings, but to provide teachers, educators and policy makers with a “balanced” image in more dimensions. The final result of SSES is not a simple international competition, but aims to find out the effec- tiveness and blind spots of each country’s education policy through evalua- tion, further analyze the problems existing in the education development of each country, and show their respective achievements and advantages to pro- vide countries with experiences from other countries that can be used for ref- erence. References Braun, H.I., Jackson, D.N., & Wiley, D.E. (2001). Socially desirable responding: The evolution of a construct. In The role of constructs in psychological and educational measurement (pp.67-88). Routledge. Forestier, K., & Adamson, B. (2017). A critique of PISA and what Jullien’s plan might offer. Compare: A Journal of Comparative and International Education, 47(3):359-373. DOI: https://doi.org/10.1080/03057925.2016.1273095 https://doi.org/10.1080/03057925.2016.1273095 BECE, Vol.9, No.1, 2021 1167 He, J., & Van de Vijver, F.J. (2015). The value of keeping an open eye for methodologi- cal issues in research on resilience and culture. In Youth resilience and culture (pp.189-201). Springer, Dordrecht. Huang, Z. (2019). The limits of the PISA test: Criticism of international scholars. Educa- tional Measurement and Evaluation, 12(1):11-17+44. [Chinese] DOI: https://doi.org/10.16518/j.cnki.emae.2019.01.002 Kankaraš, M., Feron, E., & Renbarger, R. (2019). Assessing students’ social and emo- tional skills through triangulation of assessment methods. OECD Education Work- ing Papers, No. 208, Paris: OECD Publishing. DOI: https://doi.org/10.1787/19939019 OECD. (2021). Beyond Academic Learning: First Results from the Survey of Social and Emotional Skills, OECD Publishing, Paris, https://doi.org/10.1787/92a11084-en OECD. (2021, September 7) OECD Survey on Social and Emotional Skills Technical Re- port. Retrieved September 10, 2021, from http://www.oecd.org/education/ceri/thestudyonsocialandemotionalskills.htm Zhang, J., & Liu, D. (2021). Three questions about social and emotional ability assess- ment. Journal of the Chinese Society of Education, 42(2):18-24. [Chinese] https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2 021&filename=ZJYX202102012&uniplatform=NZKPT&v=gJzpcD%25mmd2FFP 31nYs5nyy7yPVRIIGpGq0APwzRB%25mmd2BmFHF9lolxyX%25mmd2B2zwc 3RvssG7sU3U Zhou, L. (2021). When the individual national culture meets a transnational research. Sci- ence Insights Education Frontiers, 8(2):1019-1021. DOI: https://doi.org/10.15354/sief.21.ps011 Zu, J. & Kyllonen, P. (2019). The importance of non-cognitive ability and its measure- ment. Journal of China Examinations, 2019(9):22-31. [Chinese] DOI: https://doi.org/10.19360/j.cnki.11-3303/g4.2019.09.003 Correspondence to: Xiaoqiao Cheng, Ph.D. School of Institute of Education Science Nanjing Normal University Nanjing 210097 Jiangsu China E-mail: xqcheng2008@vip.163.com Conflict of Interests: None. Doi: 10.15354/sief.21.co017 https://doi.org/10.16518/j.cnki.emae.2019.01.002 https://doi.org/10.1787/19939019 https://doi.org/10.1787/92a11084-en http://www.oecd.org/education/ceri/thestudyonsocialandemotionalskills.htm https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2021&filename=ZJYX202102012&uniplatform=NZKPT&v=gJzpcD%25mmd2FFP31nYs5nyy7yPVRIIGpGq0APwzRB%25mmd2BmFHF9lolxyX%25mmd2B2zwc3RvssG7sU3U https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2021&filename=ZJYX202102012&uniplatform=NZKPT&v=gJzpcD%25mmd2FFP31nYs5nyy7yPVRIIGpGq0APwzRB%25mmd2BmFHF9lolxyX%25mmd2B2zwc3RvssG7sU3U https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2021&filename=ZJYX202102012&uniplatform=NZKPT&v=gJzpcD%25mmd2FFP31nYs5nyy7yPVRIIGpGq0APwzRB%25mmd2BmFHF9lolxyX%25mmd2B2zwc3RvssG7sU3U https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2021&filename=ZJYX202102012&uniplatform=NZKPT&v=gJzpcD%25mmd2FFP31nYs5nyy7yPVRIIGpGq0APwzRB%25mmd2BmFHF9lolxyX%25mmd2B2zwc3RvssG7sU3U https://doi.org/10.15354/sief.21.ps011 https://doi.org/10.19360/j.cnki.11-3303/g4.2019.09.003 mailto:xqcheng2008@vip.163.com