1 RBCIAMB | v.58 | n.1 | Mar 2023 | 1-10 - ISSN 2176-9478 A B S T R A C T Despite the existence of several waste recycling initiatives in Brazil, the national average of recyclable material recovery rates still remains below 3%. In this context, it is necessary to diagnose the performance of systems to implement improvement actions, considering the particularities of each system. In this study, we describe the application of a method capable of diagnosing the effectiveness of selective collection systems by identifying a general sustainability index, calculated based on 16 performance indicators. The study focused on the city of Volta Redonda and was organized into three stages: identification of the performance evaluation method; data collection and calculation of indicators; and calculation of the sustainability index. The index value found for the city was 0.47, which represents an unfavorable level of sustainability. In this case, Volta Redonda is making insufficient investments in its selective waste collection system. The factors that contributed to this low index are: an outdated solid waste management plan, low collection coverage, low recovery rate of recyclable materials, high rejection rate, high selective collection cost, outdated cost of selective collection service, precarious occupational health and safety conditions of members of waste picker cooperatives, and lack of registration of residences for donation of recyclable materials. Keywords: performance evaluation; selective collection; indicators; recyclable waste; sustainability. R E S U M O Apesar da existência de várias iniciativas de reciclagem de resíduos secos no Brasil, as taxas de recuperação de materiais recicláveis ainda permanecem em patamares inferiores a 3% na média nacional. Nesse contexto, torna-se necessário realizar uma etapa inicial de diagnóstico, voltado a identificar o desempenho do sistema, para que ações de melhoria possam ser implementadas, considerando-se as particularidades de cada sistema. O objetivo deste estudo foi aplicar um método capaz de gerar o diagnóstico de um sistema de coleta seletiva, por meio da identificação de um índice de sustentabilidade, calculado com base em 16 indicadores de desempenho. A pesquisa aplicada ao município de Volta Redonda foi organizada em três etapas: identificação do método de avaliação de desempenho; coleta de dados e cálculo dos indicadores; e cálculo do índice de sustentabilidade. O valor do índice encontrado para o município foi de 0,47, o que representa um nível desfavorável de sustentabilidade. Neste caso, constata-se que o município de Volta Redonda está fazendo apenas um baixo e insuficiente investimento em seu sistema de coleta seletiva. Os fatores que contribuíram para esse baixo índice são: plano de gestão de resíduos sólidos não vigente, baixa cobertura de coleta e baixa taxa de recuperação de materiais recicláveis, alta taxa de rejeito e alto custo da coleta seletiva, taxa não vigente para cobrança pela prestação do serviço de coleta seletiva, precárias condições de trabalho, de saúde e de segurança dos cooperados e falta de cadastro de residências na doação de materiais recicláveis. Palavras-chave: avaliação de desempenho; coleta seletiva; indicadores; resíduos recicláveis; sustentabilidade. Sustainability performance evaluation of municipal selective collection systems applied to a case study Avaliação de desempenho em sustentabilidade de sistemas municipais de coleta seletiva aplicado a um estudo de caso Ricardo César da Silva Guabiroba1 , Pedro Roberto Jacobi1 , Luís Henrique Abegão2 , Gina Rizpah Besen1 1Universidade de São Paulo – São Paulo (SP), Brazil. 2Universidade Federal Fluminense – Niterói (RJ), Brazil. Correspondence address: Ricardo César da Silva Guabiroba – Universidade Federal Fluminense – Rua Desembargador Ellis Hermydio Figueira, 783 – Bloco A, Sala 305 – Aterrado – CEP: 27.213-145 – Volta Redonda (RJ), Brazil. E-mail: ricardocesar@id.uff.br Conflicts of interest: the authors declare no conflicts of interest. Funding: none. Received on: 10/21/2022. Accepted on: 02/10/2023. https://doi.org/10.5327/Z2176-94781482 Revista Brasileira de Ciências Ambientais Brazilian Journal of Environmental Sciences This is an open access article distributed under the terms of the Creative Commons license. Revista Brasileira de Ciências Ambientais Brazilian Journal of Environmental Sciences ISSN 2176-9478 Volume 56, Number 1, March 2021 https://orcid.org/0000-0003-4108-846X https://orcid.org/0000-0001-6143-3019 https://orcid.org/0000-0002-4182-0916 https://orcid.org/0000-0002-7300-0437 mailto:ricardocesar@id.uff.br https://doi.org/10.5327/Z2176-94781482 http://abes-dn.org.br/ http://www.rbciamb.com.br Guabiroba, R.C.S. et al. 2 RBCIAMB | v.58 | n.1 | Mar 2023 | 1-10 - ISSN 2176-9478 Introduction Good management of selective waste collection systems is essential to maintain public health in cities, especially with the growing genera- tion of solid wastes (Souza et al., 2021). Many cities have implemented selective trash collection systems with the participation of informally and formally organized waste pickers that have substantially reduced the negative social and environmental impacts produced by the dispos- al of solid wastes (Ibáñez-Forés et al., 2018; Siman et al., 2020). In general, the selective collection systems in low- and middle-in- come countries are not implemented in a planned form (Melaku and Tiruneh, 2020). These systems are put into operation without a suffi- ciently developed recycling market, without programs to educate the public about the adequate separation and packaging of solid wastes, without evaluating the sustainability of different management models, and without controlling the system’s costs (Berg et al., 2018; Expósito and Velasco, 2018; Pérez-Lopez et al., 2018; Campos-Alba et al., 2021). Institutional, financial, political and social questions hamper the de- velopment of selective collection systems and the effective inclusion of waste pickers and cooperatives, which are rarely supported by the government and are exploited in the recycling market, and whose problems generally involve financial dependence and difficulties of self-management (Dutra et al., 2018; Siman et al., 2020). The previous studies of Oliveira et  al. (2018), Thoden van Velzen et  al. (2019) and Gadaleta et  al. (2022) about selective collection sys- tems in Europe present discussions on how to improve advanced sys- tems from the standpoint of management and operation. These papers evaluate improvements in systems that are already integrated and fully functioning, involving door-to-door collection models and ecopoints as well as other types of treatment, such as mechanical-biological and pay as you throw (PAYT) systems. The PAYT fee is an economic in- strument for management of wastes that applies the “polluter pays” principle, where a fee is charged from the public that varies according to the quantity and type of trash that is discarded (Morlok et al., 2017). Previous Brazilian studies have evaluated municipal selective col- lection systems by means of sustainability indicators. Among these are Barros and Silveira (2019), Fratta et al. (2019), Ibáñez-Forés et al. (2019), Silva et  al. (2019) and Zon et  al. (2020). Common problems have been identified by performance assessment, such as a lack of com- plete and reliable information, the low recovery of recyclable wastes in selective collection systems and low residential coverage, as well as waste picker cooperatives operating under precarious conditions. In this context, performance evaluation is a tool that can provide the necessary support to make well-informed and responsible deci- sions (Melnyk et  al., 2014; Canonico et  al., 2015; Valmorbida et  al., 2018). By considering environmental and socioeconomic aspects, this tool promotes improvements in selective collection systems, such as increased service quality, reduced operating costs, better management, and increased recovery rates of recyclable materials (Romano et  al., 2019; Llanquileo-Melgarejo et  al., 2021; Romano et  al., 2022; Schuch et al., 2023). In Brazil, 65.6 million tons of municipal solid waste were collect- ed in 2021, which corresponds to 0.95 kilograms per person that year (SNIS, 2022). Of the total solid waste collected in 2020, 97.7% were sent to final disposal sites (SNIS, 2022). This disposal can be classi- fied as environmentally sound when the site is a properly managed sanitary landfill, as defined in the National Policy on Solid Wastes (PNRS), meaning that is complies with specific operational standards so as to avoid damages or risks to public health and safety and mini- mize adverse environmental impacts (Brasil, 2010; Veneu et al., 2014; Olivo et al., 2021). The largest portion (73.3%) of solid wastes collected was sent to sanitary landfills in 2021. The remaining 26.7% were sent to units con- sidered inadequate, such as controlled landfills and open dumps (SNIS, 2022). The average coverage rate of trash collection for urban and rural areas was 89.9% in 2021. The average indicators of Brazil’s five mac- ro-regions varied from 79.0% in the North to 95.8% in the Southeast. It was thus estimated that 22 million people nationwide did not have access to regular door-to-door collection service (direct collection) or collection in containers, bins or dumpsters (indirect collection). In 2021, the number of people without access to collection services in- creased by nearly 6% in relation to 2020 (SNIS, 2022). In the case of selective collection, the number of municipalities with some type of formal arrangement was 1,567 in 2021, equal to 28.1% of all municipalities in the country (5,570). According to the National Sanitation Information System (SNIS, 2022), which receives information from 4,900 municipalities in all five regions of the country, this number was 5.8% lower than in 2020. The SNIS’ diagnosis (SNIS, 2022) shows that selective collection initiatives are very incipient in Brazil, and the lack of separation of wastes reflects the overload of the final disposal systems and the extraction of natural resources, which in many cases is nearing exhaustion. The direct consequence of this is that the recovery rates of recy- clable materials, despite the establishment of the National Solid Waste Policy (PNRS) in 2010, remain lower than 3% nationwide (SNIS, 2022). In spite of various initiatives to promote the sector and enable greater utilization of recyclable materials, such recovery rates con- tinue to be low, reflecting the weakness of selective trash collection systems. In 2021, it was estimated that 1.12 million tons or solid recy- clable wastes were recovered in Brazil, representing 5.3% of the total potential (SNIS, 2022). In this scenario, improvement of the performance of selective col- lection systems is still limited by the low technical and financial capaci- ty of municipalities, as well as by insufficient access to technologies and the low technical training of personnel, which makes performance as- sessment even more urgent (Ibáñez-Forés et al., 2021; Santos and Van Elk, 2021). Therefore, the objective of this study is to report the evalu- ation of a selective collection system based on the sustainability index Sustainability performance evaluation of municipal selective collection systems applied to a case study 3 RBCIAMB | v.58 | n.1 | Mar 2023 | 1-10 - ISSN 2176-9478 proposed by Besen et al. (2017), which considers varied performance indicators, such as system efficiency, institutional aspects, cost, rela- tions with society and the occupational health and safety conditions of waste pickers to identify weaknesses in these systems. Methodology Case study: selective collection system in the municipality of Volta Redonda The performance assessment method of Besen et  al. (2017) was applied in the municipality of Volta Redonda, located in the Southeast- ern region of Brazil, more specifically in the state of Rio de Janeiro. In geographical terms and from an economic standpoint, Volta Redonda is strategically located among the three largest Brazilian metropoli- tan regions — those surrounding Rio de Janeiro, São Paulo and Belo Horizonte. The municipality had an estimated population of 273,988 in 2020, and an area of 182.105 km². In terms of population, it is the largest city in the southern region of the state of Rio de Janeiro and the third largest in the state’s interior (IBGE, 2020). Figure 1 presents the current selective collection model of Volta Redonda. There are freelance waste pickers active in all of the municipality’s districts. They typically collect the recyclable materials with higher value (“cherry picking”). The recyclable materials with lower value are collected regularly and transported to a sanitary landfill located in the neighboring municipality of Barra Mansa, which also receives materials from districts in that city not yet served by selective collec- tion, or they are sent to the facilities of three cooperatives established in Volta Redonda. In the areas they serve, these cooperatives are responsible for the entire selective collection operation, involving transport, storage, sort- ing and sale of the recyclable wastes. The door-to-door collection mod- el is adopted. The three cooperatives work in warehouse buildings sup- plied by the municipal government, which also pays the monthly water and electricity bills. The relationship between the municipal govern- ment and the cooperatives is regulated by a contract with each one. The cooperatives’ revenues are comprised of the fee paid by the municipal government for selective collection (service provision) and the amount received from the sale of the recyclable materials to intermediaries. Steps of the sustainability performance assessment Sustainability performance was evaluated based on the method devel- oped by Besen et al. (2017) to determine the sustainability index, as present- ed in Figure 2. In Step (1), we conducted a literature review of articles at the Coordination for the Improvement of Higher Education Personnel (Capes) Periodicals portal, which is linked to other research databases, such as Web of Science (WoS) and Scopus. In Step (2), we calculated a range of indicators from primary and secondary data, by applying the following methods: doc- umental research, in loco observation, and interviews. In Step (3), we used the value of each indicator to calculate the sustainability index. Step 1: literature review At the search page of the Capes Periodical portal, we typed in the keywords “indicators” and “waste management” to find articles with these words in the English title (regardless of the language of the article), published in the previous five years (2017 to 2021). This search resulted in 82 articles. We checked the content of the articles and only six contained specific indicators and methods to measure selective trash collection: Pereira et  al. (2018); Barros and Silveira (2019); Fratta et  al. (2019); Ibáñez-Forés et  al. (2019); Silva et al. (2019); and Zon et al. (2020). In Zon et al. (2020), the authors based their indicators on those suggested by Besen et  al. (2017). Figure 1 – Current selective collection model in Volta Redonda, state of Rio de Janeiro. G: generators of recyclable wastes; D: districts where the generators are located; F: freelance waste pickers; SU: sorting units where cooperatives operate; R: re- cycling intermediaries; I: recycling industries; N: number of generators, waste pickers, intermediaries or industrial companies active in the system; X: number of districts in the city served by selective collection. Figure 2 – Steps for the evaluation of sustainability performance. Guabiroba, R.C.S. et al. 4 RBCIAMB | v.58 | n.1 | Mar 2023 | 1-10 - ISSN 2176-9478 Therefore, to be more complete, we utilized the method described by Besen et  al. (2017), which includes 16 sustainability indicators, presented in Table 1. To carry out the performance assessment, we used the parameters proposed by Besen et  al. (2017) for comparison with each indicator. Table 2 presents this set of parameters and also four ranges of values that indicate the four levels of the sustainability index: very favorable, favorable, unfavorable and very unfavorable. Step 2: examination of primary and secondary data In Step (2), we examined the primary and secondary data to cal- culate each indicator based on the following methods: documental research; in situ observation; and interviews. The interviews were conducted with support of semi-structured questionnaires about the indicators, with flexibility for the respondents to add comments when they deemed pertinent. Table 3 specifies the research methods used to calculate each indicator. The visits and interviews occurred in January and February 2022. The value of each indicator was obtained by three methods, as de- scribed by Besen et  al. (2017). In particular, three rates were obtained from the data made available by the SNIS (2021): selective collection cov- erage rate (indicator ISSC3); recovery rate of recyclable materials (indica- tor ISSC10); and rejection rate (indicator ISSC11). The SNIS requests and receives data from municipal governments in all regions of the country and makes these data available for consultation and diagnosis. Step 3: calculation of the sustainability index In Step (3), based on the value of each indicator, we obtained the sus- tainability index by calculating the weighted average of the 16 indicators applied (Table 1). The weights were assigned to each indicator as sug- gested by Besen et al. (2017). The index allowed identifying the level of sustainability of the selective collection system according to four levels: • very favorable: meaning that the selective collection system in the municipality is near sustainability or has already attained it; • favorable: the municipal government is making reasonable invest- ments in the sustainability of the selective collection system, but has not yet achieved this goal; • unfavorable: the municipal government is making only a small in- vestments in the sustainability of the selective collection system; • very unfavorable: the municipal government is not making any investments in the sustainability of a selective collection system (Besen et al., 2017). Table 1 – Performance indicators of selective collection systems. ISSC: Indicators of sustainability of selective collection. Source: indicators described by Besen et al. (2017). Indicator Description Sources B es en e t a l. (2 01 7) Pe re ir a et  a l. (2 01 8) B ar ro s an d Si lv ei ra (2 01 9) Fr at ta e t a l. (2 01 9) Ib áñ ez -F or és et  a l. (2 01 9) Si lv a et  a l. (2 01 9) Z on e t a l. (2 02 0) ISSC1 Integrated solid waste management plan x x x x ISSC2 Legal instruments in the relationship with the municipal government x x x ISSC3 Responsiveness to the population x x x x x x ISSC4 Self-financing x x x ISSC5 Education/disclosure x x x x x ISSC6 Social participation and control x x ISSC7 Partnerships x x x ISSC8 Inclusion of freelance waste pickers x x x x ISSC9 Adhesion of the population x x x x ISSC10 Recovery rate of recyclable materials x x x ISSC11 Rejection rate x ISSC12 Working conditions for collection of dry wastes x x ISSC13 Workplace conditions at the sorting center x x x ISSC14 Worker health and safety x x x x ISSC15 Costs of the selective collection service x x ISSC16 Cost of the selective/regular collection + disposal x x Sustainability performance evaluation of municipal selective collection systems applied to a case study 5 RBCIAMB | v.58 | n.1 | Mar 2023 | 1-10 - ISSN 2176-9478 Table 2 – Parameters to evaluate each performance indicator. ISSC: Indicators of sustainability of selective collection; C1.1: Existence of a sanitation or solid waste plan at the intermunicipal/regional/micro-regional level, inclu- ding selective collection and participative construction, in execution; C1.2: Existence of a municipal plan with participative construction undergoing implementation; C1.3: Existence of a municipal plan without participative construction, not implemented; C1.4: No plan; C2.1: Existence of a service provision agreement with the municipal government; C2.2: Existence of a working arrangement with transfer of financing; C2.3: Existence of a working arrangement without transfer of finan- cing; C2.4: No agreement or working arrangement; C4.1: Charging of a fee to cover the cost of the solid waste collection service, including selective collection; C4.2: Inclusion of a surcharge in the urban property tax [IPTU] or budget to cover the entire cost to the service; C4.3: Inclusion of a surcharge in the IPTU or budget that does not cover the entire cost to the service; C4.4: Fee revenue only in the budget; C6.1: Existence of one or more instances in functioning with participation of waste pickers; C6.2: Existence and functioning without participation of waste pickers; C6.3: Existence of an instance, but not functioning; C6.4: No existence of an instance with participation. Source: parameters defined according to Besen et al. (2017). Indicator Table 1 Classification (sustainability level) Very Favorable Favorable Unfavorable Very Unfavorable ISSC1 C1.1 C1.2 C1.3 C1.4 ISSC2 C2.1 C2.2 C2.3 C2.4 ISSC3 100% 75.1 to 99.9% 50.1 to 75.0% ≤ 50.0% ISSC4 C4.1 C4.2 C4.3 C4.4 ISSC5 ≥ 80.0% 50.1 to 79.9% 20.1 to 50.0% ≤ 20.0% ISSC6 C6.1 C6.2 C6.3 C6.4 ISSC7 ≥ 80.0% 50.1 to 79.9% 20.1 to 50.0% ≤ 20.0% ISSC8 ≥50.0% 30.0 to 50.0% 10.1 to 29.9% ≤ 10.0% ISSC9 ≥ 80.0% 50.1 to 79.9% 30.1 to 50.0% ≤ 30.0% ISSC10 ≥ 25.0% 15.1 to 24.9% 5.1 to 15.0% ≤ 5.0% ISSC11 ≤ 5.0% 5.1 to 10.0% 10.1 to 29.9% ≥ 30.0% ISSC12 100% 75.1 to 99.9% 50.1 to 75.0% ≤ 50.0% ISSC13 100% 75.1 to 99.9% 50.1 to 75.0% ≤ 50.0% ISSC14 100% 75.1 to 99.9% 50.1 to 75.0% ≤ 50.0% ISSC15 ≤ 200.00 200.00 to 350.00 351.00 to 500.00 ≥ 500.00 ISSC16 ≤ 100% 100.1 to 150 % 150.1 to 199.9% ≥ 200% IGS 0.76-1.00 0.51–0.75 0.26–0.50 0.00–0.25 Table 3 – Research methods used to attribute a value to each performance indicator. ISSC: Indicators of sustainability of selective collection; A: Interview with the person in charge of selective collection of the Municipal Environmental Secretariat; B: Interview with the person in charge of regular trash collection of the Municipal Infrastructure Secretariat; C: Interviews with the people in charge of social actions for the Municipal Community Action Secretariat; D: Interviews with the representatives of the three cooperatives (Reciclar VR, Folha Verde and Cidade do Aço); E: Do- cumental research — contracts with the cooperatives, regular trash collection company, including final disposal, Municipal Law 5762 (CMVR, 2020) and Municipal Decree 14993 (PVR, 2018); F: Data available from the National Sanitation Information System (SNIS, 2021). Indicator Table 1 Method of Besen et al. (2017) Description Methods Used A B C D E F ISSC1 Method 1 Integrated solid waste management plan x x ISSC2 Method 1 Legal instruments in the relationship with the municipal government x x x ISSC3 Method 2 Responsiveness to the population x ISSC4 Method 1 Self-financing x x ISSC5 Method 2 Education/disclosure x x ISSC6 Method 1 Social participation and control x x x ISSC7 Method 2 Partnerships x x x x ISSC8 Method 2 Inclusion of freelance waste pickers x x x ISSC9 Method 2 Adhesion of the population x x x ISSC10 Method 1 Recovery rate of recyclable materials x ISSC11 Method 3 Rejection rate x ISSC12 Method 2 Working conditions in collection of wastes x ISSC13 Method 2 Workplace conditions at the sorting center triage x ISSC14 Method 2 Worker health and safety x x ISSC15 Method 1 Costs of the selective collection service x x x ISSC16 Method 1 Cost of the selective/regular collection + disposal x x x x Guabiroba, R.C.S. et al. 6 RBCIAMB | v.58 | n.1 | Mar 2023 | 1-10 - ISSN 2176-9478 Table 4 – Results of the performance indicators and calculation of the General Sustainability Index. ISSC: Indicators of sustainability of selective collection. Indicator Result Interpretation Value Weight Final Value ISSC1 No existence of a plan Very Unfavorable 0.25 1.00 0.25 ISSC2 Existence of a contract Very Favorable 1.00 0.83 0.83 ISSC3 43.3% Very Unfavorable 0.43 0.90 0.39 ISSC4 Only budget Very Unfavorable 0.25 0.80 0.20 ISSC5 82% Very Favorable 0.82 0.79 0.65 ISSC6 Existence with participation Very Favorable 1.00 0.73 0.73 ISSC7 70% Favorable 0.70 0.62 0.43 ISSC8 32% Favorable 0.32 0.74 0.24 ISSC9 No Information No information 0.00 0.91 0.00 ISSC10 1.78% Very Unfavorable 0.25 0.89 0.22 ISSC11 17.4% Unfavorable 0.83 0.87 0.72 ISSC12 83% Favorable 0.83 0.84 0.70 ISSC13 38% Very Unfavorable 0.38 0.84 0.32 ISSC14 14% Very Unfavorable 0.14 0.84 0.12 ISSC15 R$625.00/ton Very Unfavorable 0.25 0.82 0.21 ISSC16 243.9% Very Unfavorable 0.25 0.81 0.20 Summation 13.23 6.20 Selective Collection Sustainability Index of the Municipality of Volta Redonda 0.47 Results and Discussion The sustainability index obtained for Volta Redonda was 0.47, a re- sult of the weighted average of the 16 indicators used in this study. The value of the index denoted an “unfavorable” performance, but was near the threshold of 0.50 of a “favorable” sustainability rating. Table 4 reports the results with the calculation of each sustainability indicator as applied to Volta Redonda. The municipality does not have an approved integrated solid waste management plan in force. How- ever, there are legal instruments (service provision agreements) with three waste picker cooperatives, namely Contract 065/2021 with Reci- clar VR, Contract 066/2021 with Folha Verde; and Contract 067/2021 with Cidade do Aço. These contracts were established in 2017 and have been updated since then. According to the SNIS (2021), the regular collection coverage in 2020 was 43.3%. It also showed that all the money allocated to finance waste management was included in the annual budget. According to the interview with the representative of the Municipal Infrastructure Secretariat, 4.5% of the budget goes to waste management. The munici- pal government has not yet established a fee or other charge for the reg- ular and selective collection services. Resolution 079 issued in 2021 by the National Water and Basic Sanitation Agency (ANA) specifies that municipalities must create a fee or other charge to cover regular trash collection service. In Volta Redonda, the creation of this fee is under study. For selective collection, no date has been set for the creation of any fee or charge. According to the interview with the representative of the Munici- pal Environmental Secretariat, of the 11 desirable aspects for education and disclosure of the selective collection program, nine were in prac- tice by the municipal government. Some of these are the conduction of campaigns with the preparation and distribution of flyers and activities with students, municipal employees and community members in gen- eral. We also learned of the existence of an “Intersectoral Committee for Monitoring the Selective Collection System”, with the participation of the three waste picker cooperatives. This committee was created in 2018 by means of Municipal Decree 14,993 (PVR, 2018). With regard to the diversity of partnerships articulated, it was possible to verify that, of the ten desirable partnerships, seven are es- tablished. Of these seven, five derive from Municipal Decree 14,993 (which created the Intersectoral Committee). An indirect partnership arrangement also exists with the National Association of Waste Pickers of Recyclable Materials (ANCAT), which is the technical arm of the National Movement of Waste Pickers of Recyclable Materials (MNCR). This association has provided assistance to two of the cooperatives, Re- ciclar VR and Folha Verde. Through interviews with representatives of the three cooperatives, it was possible to verify that 47 waste pickers are registered. In the city, there are 33 Social Assistance Reference Centers (CRAS), where citizens can ap- ply to receive some type of aid. Of these applicants, 47 declared they were waste pickers. Regarding the number of dwellings that have adhered to selective collection, the three cooperatives did not have this information. Sustainability performance evaluation of municipal selective collection systems applied to a case study 7 RBCIAMB | v.58 | n.1 | Mar 2023 | 1-10 - ISSN 2176-9478 For this reason, we attributed a value of zero to indicator ISSC9. We also noted the low recovery rate of recyclable materials, of only 1.78% in 2020, according to the SNIS (2021). Further, according to the SNIS (2021), there was a high rejec- tion rate (17.4%) in the municipality in 2020. We also investigated the working conditions during selective collection. Of the 12 de- sirable aspects, ten were satisfied by the cooperatives. The coop- eratives use both rented and their own trash collection trucks. The waste pickers use adequate personal protection equipment (PPE) and wear suitable uniforms. Besides this, the Vehicle Ownership Tax (IPVA) was current and the other documents were valid during the research period. Another checklist was applied regarding the working conditions at the sorting centers. Of the 13 desirable aspects, only five were satisfied. This checklist revealed the precarious working conditions of the facilities of the cooperatives granted by the municipal gov- ernment. In the case of Folha Verde, the restrooms were not linked to the sewer system. In the case of Reciclar VR, the men’s bathroom had no water. The walls of the buildings had cracks and generally required repair and painting. The final checklist covered questions about the health and safe- ty of the members. Of the seven desirable aspects, only one was sat- isfied (use of PPE by the waste pickers). The other basic questions were being neglected, such as the existence of adequate fire extin- guishers; an emergency plan; and a workplace accident prevention committee. With respect to the cost of the selective collection service, the municipal government pays a subsidy to the cooperatives of R$ 625 per ton sold (demonstrated by the sale invoices of recyclable mate- rial submitted by the cooperatives), as established in the contracts. Thus, the cooperatives have two sources of revenue: the municipal government and intermediaries (amount paid upon sale of the re- cyclable materials). The value of R$ 625 per ton was determined by the municipal government after market research and consultation of contracts established by other municipal governments. Finally, we calculated the ratio between the cost of selective col- lection and that of regular collection including the cost of disposal, expressed as a percentage. The cost of regular trash collection was R$ 186.58 per ton, while the cost of disposal in the sanitary landfill in the neighboring municipality of Barra Mansa was R$ 69.71 per ton. Therefore, the cost of regular collection plus disposal was R$ 256.29 per ton. The total cost of selective collection was R$ 625 per ton. Thus, the ratio was 243.9%. We found that more than half of the sustainability indicators were “very unfavorable”. After analyzing these indicators, it was possible to obtain a diagnosis of the city’s selective collection sys- tem. In general, the system is expensive (reflected by the indicators ISSC15 and ISSC16). This cost is twice the average cost of Brazil- ian municipalities (R$ 313.17 per ton) estimated by the associa- tion called “Compromisso Empresarial para Reciclagem” (Business Commitment to Recycling) (CEMPRE, 2020). Furthermore, the cost of selective collection is more than twice the cost of regular collection when also considering the cost of final disposal. In this scenario, the municipal government pays the total cost of selective collection (reflected by the indicator ISSC4). Despite this, the system is unable to achieve good results, including due to low coverage (reflected by the indicator ISSC3). And this low cov- erage aggravates the low rate of recovery of recyclable materials (re- flected by the indicator ISSC10). Moreover, another problem is the high rejection rate (reflected by the indicator ISSC11). Curiously, the actions among the population for education and disclosure of the selective collection service (indicator ISSC5) are “very favor- able”. This indicates that sufficient actions are being implemented by the municipal government. However, it seems that these actions are not producing the desired effect of encouraging the people to separate their trash for selective collection, which would reduce the rejection rate. This failure to produce an effect applies to the rate of recovery of recyclable materials. From another standpoint, the number of partnerships (indicator ISSC7) is considered “favorable”. Nevertheless, the quality of these partnerships is low, hampering the attainment of positive results. The municipal government has a committee (indicator ISSC6) formed by representatives of the Municipal Secretariats of the En- vironment and of Infrastructure, along with the Public Defender’s Office and Labor Attorney’s Office (the last two federal) and the cooperatives. The members of this committee meet once a month. Thus, it is necessary for the municipal government to pay heed to the recommendations of this committee for actions to improve the system. For this reason, it is necessary to have an integrated solid waste management plan (indicator ISSC1) in force, indicating the performance of the system and the results that must be reached. This involves reflection on the responsibilities for waste manage- ment. Our results show that these responsibilities for the regular and selective collection service are allocated to different municipal secretariats, which hampers the obtainment of an effective, inte- grated, solid waste collection. Furthermore, despite a “favorable” number of cooperatives in relation to enrolled waste pickers (indicator ISSC8) and the fact the cooperatives have formal contracts (indicator ISSC2), the waste pickers who are members of the cooperatives still suffer from the same insecurities and difficulties as the freelance waste pickers. In particular, although the working conditions for collection of recy- clable materials (indicator ISSC12) is considered “favorable”, the facilities of the sorting centers are in precarious conditions (in- dicator ISSC13), offering risks to the workers’ health and safety (indicator ISSC14). These conditions are not being audited by the Guabiroba, R.C.S. et al. 8 RBCIAMB | v.58 | n.1 | Mar 2023 | 1-10 - ISSN 2176-9478 municipal government, and the respective contracts lack clauses in this respect regarding collective collection. Indeed, a fire occurred once at the sorting center where the cooperative Reciclar VR now operates. In previous studies carried out in Brazil’s Southeastern region, such as Barros and Silveira (2019), it was possible to identify a mu- nicipality with a very unfavorable self-financing indicator and an- other with a very unfavorable rating regarding an integrated solid waste management plan. It was also possible to identify, in the study by Zon et  al. (2020), some factors responsible for the poor perfor- mance of the systems: precarious occupational health and safety conditions of waste pickers organized in cooperatives; low recovery rate of recyclable materials; high cost of selective collection; and lack of a fee to finance the collection program. These are problems faced in common by municipalities in different states and in Volta Redonda. Conclusions The low sustainability index obtained indicated that the Volta Redonda municipal government is making only small investments in its selective collection system. The factors underpinning the unfavorable level were lack of an integrated solid waste manage- ment plan in force; low collection coverage; low recovery of recy- clable materials; high rejection rate; high cost of selective collec- tion; no fee indicated for the selective collection service; lack of enrollment and monitoring of residences for donation of recycla- ble materials; and precarious health and safety conditions of the cooperative members. To attain the targets established in the National Solid Waste Plan, it is necessary to revise the actions adopted and insert new strategies for the purpose of eliminating the causes of these prob- lems. It is also necessary to establish better integration of the system, involving other municipalities and cooperatives, besides searching for new partners, such as other entities representing waste pickers, schools and community associations. We found that the existence of a contract does not necessarily make the system more integrated. In this respect, municipalities can avoid executing actions by allo- cating them to the waste picker cooperatives by means of a service provision agreement. The problem in this regard is that these cooperatives can still be in the structuring phase even after the process of formaliza- tion, and cannot execute all the actions required with the prop- er quality. For this reason, it is necessary for the municipalities to establish contracts that take into consideration the structur- ing phase of the cooperatives, without neglecting their ability to work in integration, by implementing strategies to strengthen the operation and management of these cooperatives. Since this is an ongoing improvement process, it is important to evaluate the performance periodically as a tool to attain targets. To overcome the limitations detected in this study, referring to the reliability of the information obtained from the interviews, we consulted other sources of information, such as contracts and decrees, for com- parison with the information from the interviews with the repre- sentatives of the cooperatives and municipal government. Finally, for new studies, we suggest applying the method of Besen et  al. (2017) in other municipalities in the state of Rio de Janeiro. With this, it will be possible to compare the performance of different systems, to detect strengths and weaknesses and opportunities for improvement that can be shared with other municipalities and waste picker cooperatives. Contribution of authors: GUABIROBA, R. C. S.: conceptualization; data curation; formal analysis; investigation; methodology; writing — original draft. JACOBI, P. R.: project administration; resources; software; supervision; validation. ABEGÃO, L. H.: conceptualization; validation; writing — review & editing. BESEN, G. R.: validation; visualization; writing — review & editing. References Barros, R.T.V.; Silveira, A.V.F., 2019. Use of sustainability indicators for the assessment of urban solid waste management in Belo Horizonte’s Metropolitan Region, MG, Brazil. Engenharia Sanitária e Ambiental, v. 24, (2), 411-423. https://doi.org/10.1590/S1413-41522019177499. Berg, A.B.; Radziemska, M.; Adamcová, D.; Zloch, J.; Vaverková, M.D., 2018. Assessment Strategies for Municipal Selective Waste Collection – Regional Waste Management. Journal of Ecological Engineering, v. 19, (1), 33-41. https://doi.org/10.12911/22998993/79405. Besen, G.R.; Günther, W.M.R.; Ribeiro, H.; Jacobi, P.R.; Dias, S.M., 2017. Gestão da coleta seletiva e de organizações de catadores: indicatores e índices de sustentabilidade. São Paulo. https://doi.org/10.11606/9788588848245. Brasil. Governo do Brasil. Subchefia para Assuntos Jurídicos. Casa Civil, 2010. Lei 12.305. Política Nacional de Resíduos Sólidos (Accessed January 15, 2022) at:. http://www.planalto.gov.br/ccivil_03/_ato2007-2010/2010/lei/l12305.htm. Campos-Alba, C.M.; Garrido-Rodríguez, J.C.; Plata-Díaz, A.M.; Pérez-Lopez, G., 2021. The selective collection of municipal solid waste and other factors determining cost efficiency. An analysis of service provision by Spanish https://doi.org/10.1590/S1413-41522019177499 https://doi.org/10.12911/22998993/79405 https://doi.org/10.11606/9788588848245 http://www.planalto.gov.br/ccivil_03/_ato2007-2010/2010/lei/l12305.htm Sustainability performance evaluation of municipal selective collection systems applied to a case study 9 RBCIAMB | v.58 | n.1 | Mar 2023 | 1-10 - ISSN 2176-9478 municipalities. Waste Management, v. 134, 11-20. https://doi.org/10.1016/j. wasman.2021.07.039. Canonico, P.; De Nito, E.; Esposito, V.; Martinez, M.; Mercurio, L.; Iacono, M.P., 2015. The boundaries of a performance management system between learning and control. Measuring Business Excellence, v. 19, (3), 7-21. https:// doi.org/10.1108/MBE-04-2015-0021. Compromisso Empresarial para Reciclagem (CEMPRE), 2020. Pesquisa Ciclosoft 2020 (Accessed March 2, 2022) at:. https://ciclosoft.cempre.org.br/ resumo-executivo. Dutra, R.M.; Yamane, S.; Siman, L.H., 2018. Influence of the expansion of the selective collection in the sorting infrastructure of waste pickers’ organizations: a case study of 16 Brazilian cities. Waste Management, v. 77, 50-58. https://doi. org/10.1016/j.wasman.2018.05.009. Expósito, A.; Velasco, F., 2018. Municipal solid-waste recycling market and the European 2020 Horizon Strategy: A regional efficiency analysis in Spain. Journal of Cleaner Production, v. 172, 938-948. https://doi.org/10.1016/j. jclepro.2017.10.221. Fratta, K.D.S.A.; Toneli, J.T.C.L.; Antonio, G.C., 2019. Diagnosis of the management of solid urban waste of the municipalities of ABC Paulista of Brasil through the application of sustainability indicators. Waste Management, v. 85, 11-17. https://doi.org/10.1016/j.wasman.2018.12.001. Gadaleta, G.; De Gisi, S.; Todaro, F.; Campanaro, V.; Teodosiu, C.; Notarnicola, M.; 2022. Sustainability assessment of municipal solid waste separate collection and treatment systems in a large metropolitan area. Sustainable Production and Consumption, v. 29, 328-340. https://doi. org/10.1016/j.spc.2021.10.023. Ibáñez-Forés, V.; Bovea, M.D.; Coutinho-Nóbrega, C.; Medeiros-García, H.R., 2019. Assessing the social performance of municipal solid waste management systems in developing countries: Proposal of indicators and a case study. Ecological Indicators, v. 98, 164-178. https://doi.org/10.1016/j. ecolind.2018.10.031. Ibáñez-Forés, V.; Bovea, M.D.; Coutinho-Nóbrega, C.; Medeiros-García, H.R.; Barreto-Lins, R., 2018. Temporal evolution of the environmental performance of implementing selective collection in municipal waste management systems in developing countries: A Brazilian case study. Waste Management, v. 72, 65- 77. https://doi.org/10.1016/j.wasman.2017.10.027. Ibáñez-Forés, V.; Coutinho-Nóbrega, C.; Guinot-Meneu, M.; Bovea, M.D., 2021. Achieving waste recovery goals in the medium/long term: Eco- efficiency analysis in a Brazilian city by using the LCA approach. Journal of Environmental Management, v. 298, 113457. https://doi.org/10.1016/j. jenvman.2021.113457. Instituto Brasileiro de Geografia e Estatística (IBGE), 2020. Panorama do municipality of Volta Redonda (Accessed March 2, 2022) at:. https://cidades. ibge.gov.br/brasil/rj/volta-redonda/panorama. Llanquileo-Melgarejo, P.; Molinos-Senante, M.; Romano, G.; Carosi, L., 2021. Evaluation of the Impact of Separative Collection and Recycling of Municipal Solid Waste on Performance: An Empirical Application for Chile. Sustainability, v. 13, (4), 2022. https://doi.org/10.3390/su13042022. Melaku, H.S.; Tiruneh, M.A., 2020. Occupational health conditions and associated factors among municipal solid waste collectors in Addis Ababa, Ethiopia. Risk Management and Healthcare Policy, v. 2020, (13), 2415-2423. https://doi.org/10.2147/RMHP.S276790. Melnyk, S.A.; Bititci, U.S.; Platts, K.; Tobias, J.; Andersen, B., 2014. Is performance measurement and management fit for the future? Management Accounting Research, v. 25, (2), 173-186. https://doi.org/10.1016/j. mar.2013.07.007. Morlok, J.; Schoenberger, H.; Styles, D.; Galvez-Martos, J.L.; Zeschmar-Lahl, B., 2017. The impact of pay-as-you-throw schemes on municipal solid waste management: The exemplar case of the county of Aschaffenburg, Germany. Resources, v. 6, (1), 8. https://doi.org/10.3390/resources6010008. National Sanitation Information System (SNIS), 2021. Diagnósticos SNIS 2020 – Planilhas em Microsoft Excel. Secretaria Nacional de Saneamento, Governo do Brasil (Accessed January 14, 2022) at:. http://antigo.snis.gov.br/ diagnosticos. National Sanitation Information System (SNIS), 2022. Diagnóstico temático de manejo de resíduos sólidos urbanos – ano de referência 2021. Secretaria Nacional de Saneamento, Governo do Brasil (Accessed December 15, 2022) at:. http://antigo.snis.gov.br/diagnosticos. Oliveira, V.; Sousa, V.; Vaz, J.M.; Dias-Ferreira, C., 2018. Model for the separate collection of packaging waste in Portuguese low-performing recycling regions. Journal of Environmental Management, v. 216, 13-24. https://doi. org/10.1016/j.jenvman.2017.04.065. Olivo, V.E.; Prietto, P.D.M.; Korf, E.P., 2021. Actions and policy tools for local governments to achieve integrated sustainable waste management. Brazilian Journal of Environmental Sciences (Online), v. 56, (3), 436-444. https://doi. org/10.5327/Z21769478968. Pereira, S.S.; Curi, R.C.; Curi, W.F., 2018. Use of indicators in urban solid waste management: a methodological proposal of construction and analysis for cities and regions. Engenharia Sanitária e Ambiental, v. 23, (3), 411-423. https://doi. org/10.1590/S1413-41522018162872. Pérez-Lopez, G.; Prior, D.; Zafra-Gomez, J.L., 2018. Temporal scale efficiency in DEA panel data estimations. An application to the solid waste disposal service in Spain. Omega, v. 76, 18-27. https://doi.org/10.1016/j. omega.2017.03.005. Prefeitura de Volta Redonda (PVR), 2018. Decreto Nº 14.993 - Dispõe sobre a criação do Comitê Intersetorial de Acompanhamento do Sistema de Coleta Seletiva no Município de Volta Redonda (CACS/VR), com a participação das Cooperativas de Catadoras e Catadores (Accessed January 06, 2022) at:. http:// www2.voltaredonda.rj.gov.br/smg/leis/mod/leis_decretos/. Romano, G.; Lombardi, G.V.; Rapposelli, A.; Gastaldi, M., 2022. The factors affecting Italian provinces’ separate waste-collection rates: An empirical investigation. Waste Management, v. 139, 217-226. https://doi.org/10.1016/j. wasman.2021.12.037. Romano, G.; Rapposelli, A.; Marrucci, L., 2019. Improving waste production and recycling through zero-waste strategy and privatization: an empirical investigation. Resourcing, Conservation and Recycling, v. 146, 256-263. https://doi.org/10.1016/j.resconrec.2019.03.030. Santos, J.E.S.; Van Elk, A.G.H. P., 2021. Política Nacional de Resíduos Sólidos: Breve Análise do Legado de uma Década. Revista Internacional de Ciências, v. 11, (2), 229-242. https://doi.org/10.12957/ric.2021.54052. Schuch, D.; Lederer, J.; Fellner, J.; Scharff, C., 2023. Separate collection rates for plastic packaging in Austria – A regional analysis taking collection systems and urbanization into account. Waste Management, v. 155, 211-219. https:// doi.org/10.1016/j.wasman.2022.09.023. Silva, L.; Prietto, P.D.M.; Korf, E.P., 2019. Sustainability indicators for urban solid waste management in large and medium-sized worldwide cities. Journal of Cleaner Production, v. 237, 11780. https://doi.org/10.1016/j. jclepro.2019.117802. Siman, R.R.; Yamane, L.H.; Baldam, R.L.; Tackla, J.P.; Assis Lessa, S.F.; Britto, P.M., 2020. Governance tools: Improving the circular economy through the promotion of the economic sustainability of waste picker organizations. Waste Management, v. 105, 148-169. https://doi.org/10.1016/j.wasman.2020.01.040. https://doi.org/10.1016/j.wasman.2021.07.039 https://doi.org/10.1016/j.wasman.2021.07.039 https://doi.org/10.1108/MBE-04-2015-0021 https://doi.org/10.1108/MBE-04-2015-0021 https://ciclosoft.cempre.org.br/resumo-executivo https://ciclosoft.cempre.org.br/resumo-executivo https://doi.org/10.1016/j.wasman.2018.05.009 https://doi.org/10.1016/j.wasman.2018.05.009 https://doi.org/10.1016/j.jclepro.2017.10.221 https://doi.org/10.1016/j.jclepro.2017.10.221 https://doi.org/10.1016/j.wasman.2018.12.001 https://doi.org/10.1016/j.spc.2021.10.023 https://doi.org/10.1016/j.spc.2021.10.023 https://doi.org/10.1016/j.ecolind.2018.10.031 https://doi.org/10.1016/j.ecolind.2018.10.031 https://doi.org/10.1016/j.wasman.2017.10.027 https://doi.org/10.1016/j.jenvman.2021.113457 https://doi.org/10.1016/j.jenvman.2021.113457 https://cidades.ibge.gov.br/brasil/rj/volta-redonda/panorama https://cidades.ibge.gov.br/brasil/rj/volta-redonda/panorama https://doi.org/10.3390/su13042022 https://doi.org/10.2147/RMHP.S276790 https://doi.org/10.1016/j.mar.2013.07.007 https://doi.org/10.1016/j.mar.2013.07.007 https://doi.org/10.3390/resources6010008 http://antigo.snis.gov.br/diagnosticos http://antigo.snis.gov.br/diagnosticos http://antigo.snis.gov.br/diagnosticos https://doi.org/10.1016/j.jenvman.2017.04.065 https://doi.org/10.1016/j.jenvman.2017.04.065 https://doi.org/10.5327/Z21769478968 https://doi.org/10.5327/Z21769478968 https://doi.org/10.1590/S1413-41522018162872 https://doi.org/10.1590/S1413-41522018162872 https://doi.org/10.1016/j.omega.2017.03.005 https://doi.org/10.1016/j.omega.2017.03.005 http://www2.voltaredonda.rj.gov.br/smg/leis/mod/leis_decretos/ http://www2.voltaredonda.rj.gov.br/smg/leis/mod/leis_decretos/ https://doi.org/10.1016/j.wasman.2021.12.037 https://doi.org/10.1016/j.wasman.2021.12.037 https://doi.org/10.1016/j.resconrec.2019.03.030 https://doi.org/10.12957/ric.2021.54052 https://doi.org/10.1016/j.wasman.2022.09.023 https://doi.org/10.1016/j.wasman.2022.09.023 https://doi.org/10.1016/j.jclepro.2019.117802 https://doi.org/10.1016/j.jclepro.2019.117802 https://doi.org/10.1016/j.wasman.2020.01.040 Guabiroba, R.C.S. et al. 10 RBCIAMB | v.58 | n.1 | Mar 2023 | 1-10 - ISSN 2176-9478 Souza, C.C.F.; Saraiva, J.S.; Lopes, M.L.B.; Santos, M.A.S.; Corradi, A.; Assumpção, D.J.F., 2021. Avaliação de desempenho da coleta de resíduos sólidos em municipalities do estado do Pará: proposta de um indicator multivariado. Gaia Scientia, v. 15, (1), 126-140. https://doi.org/10.22478/ ufpb.1981-1268.2021v15n1.55499. Thoden van Velzen, E.U.; Brouwer, M.T.; Feil, A., 2019. Collection behaviour of lightweight packaging waste by individual households and implications for the analysis of collection schemes. Waste Management, v. 89, 284-293. https:// doi.org/10.1016/j.wasman.2019.04.021. Valmorbida, S.M.I.; Ensslin, S.R.; Ensslin, L., 2018. Performance measurement and management accounting: An integrative review of the literature to overcome the difficulties of practical application of performance evaluation in organizational management. Revista Contabilidade, Gestão e Governança, v. 21, (3), 339-360. https://doi.org/10.21714/1984-3925_2018v21n3a3. Veneu, D.M.; Santos F.S.; Pinto V.M.; Pacheco J.S.; Tavares R.S., 2014. Aproveitamento energético dos resíduos sólidos urbanos do municipality of Nova Iguaçu-RJ – Uma visão sobre a nova política nacional de resíduos sólidos. Revista Teccen, v. 7, (1-2), 11-19. https://doi.org/10.21727/teccen.v7i1/2.272. Zon, J.L.N.; Leopoldino, C.J.; Yamane, L.H.; Siman, R.R., 2020. Waste pickers organizations and municipal selective waste collection: Sustainability indicators. Waste Management, v. 118, 219-231. https://doi.org/10.1016/j. wasman.2020.08.023. https://doi.org/10.22478/ufpb.1981-1268.2021v15n1.55499 https://doi.org/10.22478/ufpb.1981-1268.2021v15n1.55499 https://doi.org/10.1016/j.wasman.2019.04.021 https://doi.org/10.1016/j.wasman.2019.04.021 https://doi.org/10.21714/1984-3925_2018v21n3a3 https://doi.org/10.21727/teccen.v7i1/2.272 https://doi.org/10.1016/j.wasman.2020.08.023 https://doi.org/10.1016/j.wasman.2020.08.023