001.docx DOI: 10.3303/CET2189088 Paper Received: 29 June 2021; Revised: 27 September 2021; Accepted: 10 November 2021 Please cite this article as: Theng L.M., Tan J., Liew P.Y., Tan L.S., 2021, A Review of Manufacturing Sustainability Assessment Tool Selection Criteria: A Quantitative Score-Rating System versus Process-Data Sustainability Assessment, Chemical Engineering Transactions, 89, 523-528 DOI:10.3303/CET2189088 CHEMICAL ENGINEERING TRANSACTIONS VOL. 89, 2021 A publication of The Italian Association of Chemical Engineering Online at www.cetjournal.it Guest Editors: Jeng Shiun Lim, Nor Alafiza Yunus, Jiří Jaromír Klemeš Copyright © 2021, AIDIC Servizi S.r.l. ISBN 978-88-95608-87-7; ISSN 2283-9216 A Review of Manufacturing Sustainability Assessment Tool Selection Criteria: A Quantitative Score-Rating System versus Process-Data Sustainability Assessment Lina Mary Thenga, Jully Tanb, Peng Yen Liewa, Lian See Tana,* a Department of Chemical and Environmental Engineering (CHEE), Malaysia - Japan International Institute of Technology (MJIIT), Universiti Teknologi Malaysia (UTM), Jalan Sultan Yahya Petra (Jalan Semarak), 54100 Kuala Lumpur, Malaysia b School of Engineering, Monash University Malaysia, Bandar Sunway, Selangor, Malaysia. tan.liansee@utm.my The scarcity of natural resources has urged the necessity to adopt sustainability practices into current industrial activities. Manufacturing industries have a significant contribution towards socio-economic development by creating job opportunities for the communities and stimulating economic growth. Manufacturing activities are also well-known for their considerably high consumption of natural resources and waste generation. With these underlying issues, research has been actively carried out to develop a holistic framework in both conceptual and practical manners for sustainability improvements, specifically in the manufacturing sectors. Various methods in the research field often lead to the dilemma of selecting the appropriate methods for application. A comprehensive review is desirable to understand the current research efforts by evaluating the specific research problems, methodology developments, and research findings. This paper aims to evaluate the available sustainability assessment methods that were demonstrated with case studies, thereby defining the common practices adopted by the researchers in opting for their preferred methods. A systematic review was conducted to identify the common selection criteria for sustainability assessment methods. This study could provide a holistic insight to streamline the available options and justify their selection according to the application needs. 1. Introduction The manufacturing industry has been a significant contributor to socio-economic growth by providing job opportunities and promote sectoral relationships through economic activities (Herman, 2016). The increasing demands of the manufactured goods have put pressure on the environment (Poveda, 2017), mainly due to the current linear economy, which practices a take-make-dispose system (Sánchez Levoso et al., 2020). This production process has led to significant environmental impacts such as pollution and climate change (Ahmad and Wong, 2018). With the arising concerns, the United Nations (UN) has introduced an action plan, namely Sustainable Development Goals (SDGs), which comprises 17 SDGs and 169 targets in consideration of the social, environmental, and economic viewpoints (United Nations, 2015). Among all the SDGs, goal number 12, "Responsible Consumption and Production", is deemed one of the prioritised agendas (Rodriguez-Anton et al., 2019). The manufacturers are urged to operate responsibly in compliance with the stringent regulations provided by governance policy (Nörmann and Maier-Speredelozzi, 2016). The importance of sustainability assessment to support decision-making in the sustainability approach is necessary (Poveda, 2017). The assessment could provide insights into the sustainability of a system and promotes strategic planning to prevent time-consuming trial runs and potentially restrain the resources (Waas et al., 2014). Various researches have been conducted, especially in developing sustainability assessment tools, to measure the sustainability performance of the respective application. With the various methodology approaches, selecting suitable assessment tools could be challenging as it is impossible to have a one-size- fits-all method (Bitter et al., 2020). A systematic review of the current state of the research development is desirable to assist the selection process. The common review articles mostly categorised the findings based on methodology approaches and their respective application. The case studies rarely being discussed (Bitter et al., 523 2020). The existing review literature mainly focuses on supply-side information, such as the capability of assessment tool, the demand-side perspectives, which signify the research problems (Zijp et al., 2015). An example that demonstrates the mentioned approach was presented by Ahmad and Wong (2018). The authors summarised the assessment tools based on the scope boundary and assessment level, whereby the research problems were omitted from the analysis. Aware of the knowledge gaps, some articles intended to contribute new insights in categorising the sustainability assessment approaches to assist in selecting assessment methods. Zijp et al. (2015) designed a sustainability assessment identification key (SA-IK) to articulate the research questions as a guideline to assist the method selection process. Bitter et al. (2020) proposed a quantitative-based selection framework to match the requirements with sustainability assessment specifications. As of recent development, Koppiahraj et al. (2021) conducted a weighed factor technique onto each requirement of sustainable manufacturing practices. The best method was selected based on the determined critical factors. The above-mentioned papers attempted to support assessment method selection through supply-demand viewpoints. The concept was mainly built on empirical judgments and general understanding. The present study aims to review the manufacturing-related sustainability assessment to determine the selection criteria in each of the demonstrated case studies. The concept of the method selection criteria can be observed according to the evidence-based applications and present the common practices in selecting the sustainability assessment method. 2. Methodology A systematic screening was carried out to specify the relevant papers to ensure an adequate review analysis. Figure 1 shows the schematic evaluation process adopted in this study. Figure 1: Literature screening strategy (Adapted from Xiao and Watson (2019)) The sustainability assessment is a vital procedure to achieve sustainable manufacturing (Hartini et al., 2020). Based on the Scopus database, the combined keywords search of "Sustainable Assessment" and "Manufacturing" have resulted in a total of 243 records from the year from 2015 to 2020. The selected period is in conjunction with the SDGs establishment in 2015 to ensure the relevant assessment strategies been reviewed. The searches were further refined to consider only research article document type and English publication based on the identified records. An eligibility check was performed on the screened articles to ensure the crucial information was included, particularly in assessment tool selection and only qualitative methods were analysed. The included articles were reviewed as presented in the following section. 2.1 Case application The present paper was focused explicitly on manufacturing-related sustainability studies. The case studies were segregated based on their respective manufacturing focus area. For each demonstrated case, the requirement and consideration of sustainability assessment were recorded for discussion. 2.2 Assessment level The assessment level represents the scope of sustainability evaluation which focused at different levels, such as process, work cell, facility, company, product, sector, national, and global level (Ahmad and Wong, 2018). More input data and time were required with the broad assessment level, while detailed analysis was required given that specific assessment boundary. This paper documented the assessment level demonstrated by the selected case studies before review analysis. 524 2.3 Method approach Due to various available methods, this study mainly focuses on quantitative approaches given their prevailing application. The assessment tool demonstrated in the case study were categorised based on the indicator- based score rating approach and quantitative process data approach. 3. Results and discussion Based on research methodology, the overall review of sustainability assessment studies is listed in Table 1. Table 1: Sustainability assessment review No. Authors Application Method Ɨ Approach Assessment Level 1 Caldeira et al. (2020) Food waste biorefinery LCA Process-data Process 2 Putra et al. (2020) Cement manufacturing LCA+AHP Process-data Process 3 Sharma et al. (2020) Pharmaceutical production LCA Process-data Process 4 Blanco et al. (2020) Semiconductor manufacturing LCA+GSA Process-data Process 5 Saxena et al. (2020) Metal processing TOPSIS Score-rating Process 6 Khan et al. (2020) Steel manufacturing Sus-indicators Process-data Process 7 Khan et al. (2020) Machining process Sus-indicators Process-data Process 8 Yoon and Bae (2020) Shading devices manufacturing Sus-indicators Process-data Product 9 Parisi et al. (2020) Organic dye synthesis Metrics+LCA Process-data Process 10 Hartini et al. (2020) Furniture industry Delphi+Sus- VSM+AHP Score-rating Facility 11 Song and Moon (2019) Machining process Distance-to-Target Process-data Process 12 Favi et al. (2019) Metal processing LCA Process-data Product 13 Sieti et al. (2019) Baby food manufacturing LCA Process-data Product 14 Renteria Gamiz et al. (2019) Biopharmaceutical manufacturing LCA Process-data Process- Sector 15 Ahmad and Wong (2019) Food manufacturing Delphi Score-rating Sector 16 Jiang et al. (2019) Gear manufacturing Emergy+LCA Process-data Process 17 Rahla et al. (2019) Concrete production MARS-SC Score-rating Product 18 Davide et al. (2019) Ceramic tiles manufacturing LCSA Process-data Product 19 Cusenza et al. (2019) Battery production LCA Process-data Product 20 Liang et al. (2019) Machining process PSI Score-rating Process 21 Lin et al. (2018) Semiconductor industry AHP+DEA Score-rating Company 22 Saavalainen et al. (2017) Formic acid production SAT Score-rating Process 23 Hallstedt et al. (2015) Aerospace industry SAVE Score-rating Product 24 Sureeyatanapas et al. (2015) Sugar manufacturing CSA Score-rating Company ƗNote: LCA = Life Cycle Assessment; AHP = Analytic Hierarchy Process; GSA = Global Sensitivity Analysis; Sus = Sustainability; TOPSIS = Technique for Order of Preference by Similarity to Ideal Solution; Sus-VSM = Sustainable-Value Stream Mapping; MARS-SC = Method for the Relative Sustainability Assessment of Building Technologies; LCSA = Life Cycle Sustainability Assessment; PSI - Product Sustainability Index; DEA = Data Envelopment Analysis; SAT = Sustainability Assessment Tool; SAVE = Sustainability Assessment and Value Evaluation; CSA = Corporate Sustainability Assessment In a detailed review, most studies emphasised the scope of sustainability assessment shall include all three sustainability dimensions, particularly in environmental, social, and economic aspects, also known as triple bottom line (Saxena et al., 2020). Saavalainen et al. (2017) defined the relevant sustainability indicators regarding triple bottom line viewpoints using sustainability assessment tool (SAT) as a score-rating system, and the higher score represents the severity of the impacts. Most researchers applied the score-rating method to assess all sustainability dimensions. The existing methodological approach, such as LCA, stressed environmental perspectives, the selected indicators were widely available and can be rapidly quantified with the weighted scores (Ahmad and Wong, 2019). 525 The sustainability assessment in most case studies was used to compare the alternatives for better sustainability performance. According to detailed analysis, the majority conducted a comparative assessment at the process level. Rahla et al. (2019) compared different concrete mixes to identify the optimum alternatives based on their quantified sustainability performance regarding environmental impacts, production cost, and functionality indicators. Yoon and Bae (2020) evaluated different shading devices at the prototype stage to select a sustainable design. The review discovered that the preferred method used for comparative analysis is the quantitative process analysis method which involved specific data in choosing the best alternatives through the quantified results. Another crucial role of sustainability assessment is to assist in decision-making. The intended assessment level is one of the critical factors in selecting an appropriate approach in terms of approach selection. At the product level, Davide et al. (2019) implemented a life cycle sustainability assessment (LCSA) through stakeholder's involvement in the impact analysis to determine the sustainable supply chain for ceramic tiles manufacturing. Sureeyatanapas et al. (2015) employed corporate sustainability assessment (CSA) to support a decision- making system based on the sustainability performance score of sugar manufacturing processes. Most studies would incorporate a multi-criteria selection method to affirm the decision-making. Putra et al. (2020) integrated the analytic hierarchy process (AHP) as a multi-criteria analysis tool on top of LCA methodology to select the most sustainable cement manufacturing plants. The detailed assessment is desirable for sustainability assessment, most of the case studies highlighted the life cycle thinking could provide a comprehensive evaluation to identify the sustainability hotspots. The life cycle thinking approach has been widely adapted in sustainability assessment studies (Solarte-Toro and Cardona Alzate, 2021). LCA is the most applied tool used to assess the sustainability performance of a given study. Favi et al. (2019) adapted the LCA methodology to study various metal processing technologies through life cycle perspectives. There are also studies using a different approach to integrate life cycle thinking in their assessment. Khan et al. (2020) provided life cycle perspectives in assessing sustainability performance using an indicator-based quantitative approach. The life cycle viewpoints require detailed process data and mainly applicable to process-oriented assessment. For extensive manufacturing areas, none of the methods capable of being used in cross-multidisciplinary scenarios. Most of the case studies underlined the need for a process-specific approach. Hartini et al. (2020) weighted the importance of relevant indicators for a specific process is required to ensure quality outcomes and accurate judgments. Process-oriented studies are necessary, especially for a niche application, such as additive manufacturing processes (Jiang et al., 2019). This sustainability assessment's common practice involving detailed analysis requires a well-defined methodological approach such as LCA or any tool that integrates life cycle perspectives throughout the assessment process. From the literature analysis in Figure 2, the scale-rating approach is useful to identify the sustainability hotspot before proceeding with detailed assessment, meanwhile the process data analysis offers data-oriented evaluation that highlighting the potential improvement of a specific process. The combined methods could provide a holistic framework where scale-rating method can be used to streamline the sustainability focus, followed by process data analysis to quantify the sustainability performance to affirm the assessment results. Figure 2: Overview of findings 4. Conclusions The need of selecting an appropriate method approach is vital to ensure a practical sustainability assessment. The presence of various methods has led to an indecisive situation in academia. This paper evaluated the 526 selection criteria that existing sustainability assessment studies have collectively practised, given the identified research gaps. A set of requirements were highlighted in most of the papers, which include: (a) consider triple bottom line elements, (b) effective comparison study, (c) support decision-making, (d) integrate life cycle thinking, and (e) process-specific application. Each of the requirements was evaluated based on significant factors such as case application, assessment level, and method approach. These factors were found to be significantly affecting the method selection in terms of approach preference. The study's implications provide an evidence-based evaluation on the general criteria review specifically for the manufacturing industry and suggesting the assessment approaches based on research needs. The limitations of this study are that the review only observes the general method approach rather than a technical review of a specific tool. This study primarily reviews the quantitative approach. Future research may consider other relevant approaches and expand this review strategy to other industrial sectors. Acknowledgements The financial support from Universiti Teknologi Malaysia, Ministry of Higher Education under Fundamental Research Grant Scheme (FRGS/1/2018/TK02/UTM/01/6) and MRUN TRGS (R.K130000.7843.4L883) are gratefully appreciated. References Ahmad S., Wong K.Y., 2018, Sustainability assessment in the manufacturing industry: A review of recent studies, Benchmarking, 25(8), 3162–3179. Ahmad S., Wong K.Y., 2019, Development of weighted triple-bottom line sustainability indicators for the Malaysian food manufacturing industry using the Delphi method, Journal of Cleaner Production, 229, 1167– 1182. Bitter J., Janssen D., Vossen R., Hees F., 2020, Spectra and specifications of systematisation criteria for sustainability assessment approaches, The International Journal of Sustainability Policy and Practice, 16(1), 15–35. Blanco C.F., Cucurachi S., Guinée J.B., Vijver M.G., Peijnenburg W.J.G.M., Trattnig R., Heijungs R., 2020, Assessing the sustainability of emerging technologies: A probabilistic LCA method applied to advanced photovoltaics, Journal of Cleaner Production, 259, 120968. Caldeira C., Vlysidis A., Fiore G., De Laurentiis V., Vignali G., Sala S., 2020, Sustainability of food waste biorefinery: A review on valorisation pathways, techno-economic constraints, and environmental assessment, Bioresource Technology, 312, 123575. Cusenza M.A., Bobba S., Ardente F., Cellura M., Di Persio F., 2019, Energy and environmental assessment of a traction lithium-ion battery pack for plug-in hybrid electric vehicles, Journal of Cleaner Production, 215, 634–649. Davide S.B., Enrique G.-M.F., Martina P., Lucrezia V., Cristina S., Maria F.A., 2019, Sustainability as source of competitive advantages in mature sectors: The case of Ceramic District of Sassuolo (Italy), Smart and Sustainable Built Environment, 8(1), 53–79. Favi C., Campi F., Germani M., Mandolini M., 2019, A data framework for environmental assessment of metal arc welding processes and welded structures during the design phase, International Journal of Advanced Manufacturing Technology, 105(1–4), 967–993. Hallstedt S.I., Bertoni M., Isaksson O., 2015, Assessing sustainability and value of manufacturing processes: a case in the aerospace industry, Journal of Cleaner Production, 108, 169–182. Hartini S., Ciptomulyono U., Anityasari M., Sriyanto M., 2020, Manufacturing sustainability assessment using a lean manufacturing tool: A case study in the Indonesian wooden furniture industry, International Journal of Lean Six Sigma, 11(5), 957–985. Herman E., 2016, The importance of the manufacturing sector in the Romanian economy, Procedia Technology, 22, 976–983. Jiang Q., Liu Z., Li T., Cong W., Zhang H.-C., 2019, Emergy-based life-cycle assessment (Em-LCA) for sustainability assessment: A case study of laser additive manufacturing versus CNC machining, International Journal of Advanced Manufacturing Technology, 102(9–12), 4109–4120. Khan A.M., Gupta M.K., Hegab H., Jamil M., Mia M., He N., Song Q., Liu Z., Pruncu C.I., 2020, Energy-based cost integrated modelling and sustainability assessment of Al-GnP hybrid nanofluid assisted turning of AISI52100 steel, Journal of Cleaner Production, 257, 120502. Khan A.M., Jamil M., Mia M., He N., Zhao W., Gong L., 2020, Sustainability-based performance evaluation of hybrid nanofluid assisted machining: Sustainability assessment of hybrid nanofluid assisted machining, Journal of Cleaner Production, 257, 120541. 527 Koppiahraj K., Bathrinath S., Venkatesh V.G., Mani V., Shi Y., 2021, Optimal sustainability assessment method selection: A practitioner perspective, Annals of Operations Research, doi.org/10.1007/s10479-021-03946-z Liang X., Liu Z., Liu W., Li X., 2019, Sustainability assessment of dry turning Ti-6Al-4V employing uncoated cemented carbide tools as clean manufacturing process, Journal of Cleaner Production, 214, 279–289. Lin F., Lin S.-W., Lu W.-M., 2018, Sustainability assessment of Taiwan's semiconductor industry: A new hybrid model using combined analytic hierarchy process and two-stage additive network data envelopment analysis, Sustainability (Switzerland), 10(11), 4070. Nörmann N., Maier-Speredelozzi V., 2016, Cost and environmental impacts in manufacturing: A case study approach, Procedia Manufacturing, 5, 58–74. Parisi M.L., Dessì A., Zani L., Maranghi S., Mohammadpourasl S., Calamante M., Mordini A., Basosi R., Reginato G., Sinicropi A., 2020, Combined LCA and green metrics approach for the sustainability assessment of an organic dye synthesis on lab scale, Frontiers in Chemistry, 8, 214. Poveda C.A., 2017, Sustainability assessment: A rating system framework for best practices, Emerald Publishing Limited, Bingley, UK. Putra M.A., Teh K.C., Tan J., Choong T.S.Y., 2020, Sustainability assessment of Indonesian cement manufacturing via integrated life cycle assessment and analytical hierarchy process method, Environmental Science and Pollution Research, 27(23), 29352–29360. Rahla K.M., Mateus R., Bragança L., 2019, Comparative sustainability assessment of binary blended concretes using Supplementary Cementitious Materials (SCMs) and Ordinary Portland Cement (OPC), Journal of Cleaner Production, 220, 445–459. Renteria Gamiz A.G., Dewulf J., De Soete W., Heirman B., Dahlin P., Jurisch C., Krebser U., De Meester S., 2019, Freeze drying in the biopharmaceutical industry: An environmental sustainability assessment, Food and Bioproducts Processing, 117, 213–223. Rodriguez-Anton J.M., Rubio-Andrada L., Celemín-Pedroche M.S., Alonso-Almeida M.D.M., 2019, Analysis of the relations between circular economy and sustainable development goals, International Journal of Sustainable Development and World Ecology, 26(8), 708–720. Saavalainen P., Turpeinen E., Omodara L., Kabra S., Oravisjärvi K., Yadav G.D., Keiski R.L., Pongrácz E., 2017, Developing and testing a tool for sustainability assessment in an early process design phase – Case study of formic acid production by conventional and carbon dioxide-based routes, Journal of Cleaner Production, 168, 1636–1651. Sánchez Levoso A., Gasol C.M., Martínez-Blanco J., Durany X.G., Lehmann M., Gaya R.F., 2020, Methodological framework for the implementation of circular economy in urban systems, Journal of Cleaner Production, 248, 119227. Saxena P., Stavropoulos P., Kechagias J., Salonitis K., 2020, Sustainability assessment for manufacturing operations, Energies, 13(11), 2730. Sharma R.K., Sarkar P., Singh H., 2020, Assessing the sustainability of a manufacturing process using life cycle assessment technique—a case of an Indian pharmaceutical company, Clean Technologies and Environmental Policy, 22(6), 1269–1284. Sieti N., Schmidt Rivera X.C., Stamford L., Azapagic A., 2019, Environmental sustainability assessment of ready-made baby foods: Meals, menus and diets, Science of The Total Environment, 689, 899–911. Solarte-Toro J.C., Cardona Alzate C.A., 2021, Perspectives of the sustainability assessment of biorefineries, Chemical Engineering Transactions, 83, 307-312. Song Z., Moon Y., 2019, Sustainability metrics for assessing manufacturing systems: A distance-to-target methodology, Environment, Development and Sustainability, 21(6), 2811–2834. Sureeyatanapas P., Yang J.B., Bamford D., 2015, The sweet spot in sustainability: A framework for corporate assessment in sugar manufacturing, Production Planning and Control, 26(13), 1128–1144. United Nations, 2015, Transforming our world: The 2030 agenda for sustainable development. accessed 03.06.2021. Waas T., Hugé J., Block T., Wright T., Benitez-Capistros F., Verbruggen A., 2014, Sustainability assessment and indicators: Tools in a decision-making strategy for sustainable development, Sustainability, 6(9), 5512– 5534. Xiao Y., Watson M.,2019. Guidance on conducting a systematic literature review, Journal of Planning Education and Research, 39(1), 93–112. Yoon J., Bae S., 2020, Performance evaluation and design of thermo-responsive SMP shading prototypes, Sustainability, 12(11), 4391. Zijp M.C., Heijungs R., van der Voet E., van de Meent D., Huijbregts M.A.J., Hollander A., Posthuma L., 2015, An identification key for selecting methods for sustainability assessments, Sustainability, 7(3), 2490–2512. 528 088.pdf A Review of Manufacturing Sustainability Assessment Tool Selection Criteria: A Quantitative Score-Rating System versus Process-Data Sustainability Assessment