Microsoft Word - 18380 final.edited.published.docx Lentera Hukum, Volume 7 Issue 3 (2020), pp. 301-318 ISSN 2355-4673 (Print) 2621-3710 (Online) https://doi.org/10.19184/ejlh.v7i3.18380 Published by the University of Jember, Indonesia Available online 23 November 2020 Legal Aspects of Artificial Intelligence on Automated Decision-Making in Indonesia: Lessons from the European Union, the United States, and China Denindah Olivia Padjadjaran University, Indonesia denindah17001@mail.unpad.ac.id ABSTRACT This paper analyzes the importance of Indonesia's comprehensive legal framework on automated decision- making empowered by Artificial Intelligence, comparing it to the European Union, the United States, and China. Specifically, this paper inquires about the status quo of the legal protection of automated decision- making In Indonesia. The analysis highlights profiling in an automated decision-making system with the following discussion about personal data protection. In this context, the European Union's member states set out the General Data Protection Regulation (GDPR) that prohibits automated decision-making to a certain extent. In the United States, the practice of automated decision-making is rather usual. Simultaneously, China takes an exceptional measure instead and develops this automation through a social credit system. The analysis concludes that Indonesia has weak legal protection towards personal data and profiling, which later becomes the basis in facilitating automated decision-making. The provision of automated decision-making and profiling is the absolute bare minimum to Indonesia's Personal Data Protection Bill due to insufficient legal certainty. In the end, it is paramount for lawmakers to consider a comprehensive regulation on automated decision-making by adopting the European Union's GDPR framework. KEYWORDS: Artificial Intelligence, Automated Decision-Making, Personal Data Protection. Copyright © 2020 by Author(s) This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. All writings published in this journal are personal views of the authors and do not represent the views of this journal and the author's affiliated institutions. Submitted: August 12, 2020 Revised: November 22, 2020 Accepted: November 23, 2020 HOW TO CITE: Olivia, Denindah. “Legal Aspects of Artificial Intelligence on Automated Decision-Making in Indonesia: Lessons from the European Union, the United States and China” (2020) 7:3 Lentera Hukum 301-318. 302 | Legal Aspects of Artificial Intelligence on Automated Decision-Making in Indonesia I. INTRODUCTION Historically, humankind has evolved in establishing civilization. The way to advance it later had gained dramatic achievement in the dawn of the industrial revolution indicated by the computational system and designed to mirror human thought processes.1 The progress of computational models becomes sophisticated in terms of how computers can simulate the information process of human consciousness and think, particularly after the invention of Artificial Intelligence (AI). However, AI is not equal to human intelligence, which simultaneously reflects human thinking. To some extent, it even may exceed human intelligence.2 In this context, AI can be a tool of problem-solving and later it can be used by the government in decision-making. In some jurisdictions with majorly applied AI, like the European Union and the United States, their legislation concerns introducing regulation to accommodate AI. It includes protecting the typical human job's replacement as a decision-maker by delegating the power to machines. This practice is known as automated decision-making. Despite the benefits of this automation empowered by AI, the use of this technology can potentially harm an individual's right to data protection, including privacy rights. In particular, the collection and process of AI data without the individual's concern and consent can seriously violate the right to data protection.3 This study then highlights government decision-making automation that conflicts between the rule of law and constant technological evolution. It is because decision- making automation can enhance and detract from the rule of law.4 This study finds that the practice of automated government decision-making can intensify the rule of law or vice versa. This research concludes that the evaluation becomes essential to each decision-making process and considers the rule of law altogether. The legal aspect and its consequences on the automated decision-making in Indonesia's legal framework is matter. While applying automated decision-making into our social lives, the intersection of the legal concepts and social norms with technology is also an essential discussion.5 This paper analyzes and compares the legal framework and practice of automated decision-making in some selected countries that have employed this technology. Then, this paper discusses and the importance of introducing a comprehensive legal framework in Indonesia. In so doing, it provides a brief background of AI and automated decision- making before discussing the importance of a comprehensive legal framework of automated-decision making regulation adopted in Indonesia. It compares the AI use in selected countries, inter alia, the European Union countries, the United States, and China. 1 Ahmed Habeeb, “Introduction to Artificial Intelligence” (2017), online: https://www.researchgate.net/ publication/325581483. 2 Stuart Russell, Artificial Intelligence: A Modern Approach (New Jersey: Prentice-Hall, 1995) at 5. 3 European Union, “General Data Protection Regulation” (2016) Official Journal of the European Union. 4 Monika Zalnieriute, Lyria Bennett Moses, and George Wiliams, “The Rule of Law and Automation of Government Decision-Making” (2019) 82:3 Modern Law Review, at 1. 5 Simon Stern, “Artificial Intelligence, Technology, and Law” (2018) 68: Supp 1 University of Toronto Law Journal at 12. 303 | LENTERA HUKUM Then, it clarifies the relationship between AI, automated decision-making, personal data protection, and the rule of law. II. METHODS This study in this paper uses doctrinal research, by analyzing the legal materials as the primary sources and supported by the secondary sources from the library, such as books, journal articles, and other documents. The analysis in this study also uses the comparative study, by comparing Indonesia with the relevant jurisdictions and their laws to address the discourse regarding law and technology on the automated decision- making empowered by artificial intelligence. III. THE ADVENT OF ARTIFICAL INTELLIGENCE AND AUTOMATED DECISION-MAKING A. Artificial Intelligence John McCarthy brought up the term 'Artificial Intelligence' in 1956, where he thought of it as the science and engineering of intelligent machines. As explained by Schalkoff in 1990, AI was a field of study that seeks to explain and emulate intelligent behavior in terms of a computational process. This definition was concerned with AI behavior. Another description came from Haugeland in 1985. AI was an exciting new effort to make computers think, machines with minds, in the full and literal sense. Haugeland’s definition of AI was more concerned with the reasoning and thought processes, making it slightly different from Schalkoff’s definition of AI. These insights were beneficial in shaping our thinking of an approach to understanding AI.6 It is essential to note that AI includes learning from past events, decision-making, and a swift answer. In this context, AI is expected to avoid ambiguity. In AI, machines are programmed into developing their developed programming language to manipulate knowledge more effectively. AI has rather distinctive programs than other programming languages. They are to manipulate primarily qualitative than quantitative information. It aims to stimulate learning processes, reasoning, and understanding of data. Perhaps the most exciting part of AI is that it can induce, deduct, and often suspect data. They can also review decisions made by employing backtracking for solutions. AI possesses four main parts: an expert system deals with an expert's situation and gives outperformance. They include useful human knowledge in the machine memory to provide thoughtful advice, clarification, and justification for its decision. Heuristic problem-solving deals with evaluating a small range of solutions. These solutions may involve some guesswork to discover the best solutions. First, Natural Language Processing is the one component that provides communication between humans and machines in natural language. Lastly is the vision, which is the automated ability to 6 Ibid. 304 | Legal Aspects of Artificial Intelligence on Automated Decision-Making in Indonesia acknowledge shapes and features. Integrating expert systems in AI has proven its efficiency. They have proved to be doing a far better job to make fewer mistakes and be more consistent in their recommendations. They are arguably cheaper compared to human labor. They are made to deal with the mechanical side of experts' repetitive tasks with consistency and, most importantly, enable operations not suitable for humans.7 B. Automated Decision Making Before AI was invented, humans made decisions conventionally, relying on their conclusions merely on variable data. Humans often made mistakes, as what we call human errors. Decision making also could take up hours and hours of processing if were being done traditionally. In contrast, AI provides humans with the ability to create data- based models and simulations to make decisions quicker. AI offers humans revolutionary models of programs for the implementation of automated decision-making. Automated decision-making employs a set of mathematical algorithms that can analyze various factors. However, these algorithms cannot predict the future. The best they can do is calculate the possibility that an event will occur according to existing data.8 As an example, algorithms cannot predict when a loan applicant will return the money. However, these algorithms can conclude that a set of particular components creates some possibility that an outcome will happen. This tool is somewhat flawed, and misinterpretations will occur.9 Automated decision-making is deciding by automated software, tool, or machine without any human involvement. It is commonly used in the pursuit of business analytics and information. The process is through the automation of applying business rules that are generated by business analytics.10 They use rule engines to process a series of business rules using conditional statements to address logical questions.11 According to Article 22 of the General Data Protection Regulation (GDPR), automated decision-making is based solely on automated means that includes profiling. It produces legal effects concerning the individual related or similarly significantly affects them. Furthermore, recital (71) of the GDPR specifies the scope of the protection: “the data subject should have the right not to be subject to a decision, which may include a measure, evaluating personal aspects relating to him or her which is based solely on automated processing and which produces legal effects concerning him or her or similarly significantly affects them.“12 7 Peter Lucas and Linda van der Gaag, Principles of Experts Systems (Amsterdam: Addison Wesley, 1991) at 2. 8 Ari Ezra Waldman, “Power, Process, and Automated Decision-Making Automated Decision-Making" (2019) 88:2 Fordham Law Review at 617. 9 Ibid. 10 Ananthi Sheshasaayee, “A Study of Automated Decision Making Systems” (2017) 7:1 Research Inventory: International Journal of Engineering and Science at 29. 11 Ibid. 12 Article 22 of the GDPR. 305 | LENTERA HUKUM Various technologies are being deployed in the government's automated decision- making process. There are at least two types: (a) the process follows a series of pre- programmed rules written by human labor, and (b) obtained rules inferred by the system from past events or saved data. Expert systems, as explained before incorporated by governments to either improve or to replace traditional decision making. Since 1980, the plans have been manufactured for various governmental contexts, such as welfare benefits.13 They are operating based on human logic with very similar inputs with the same outputs. The expert system is quite distinct from the Automated Decision-Making system that laid its foundation based on rules learned from patterns and the connection of past events or historical data. The system is driven by the automation of the construction of the rules. The automated decision-making is also called a 'machine learning' system. It happens recurrently as an algorithm attempts to revive performances to reach a certain subjective.14 Automated decision-making can produce beneficial outcomes in society. The tool can assess a human being, evaluate events, and generate recommendations to general people. However, the system has a set of risks that can happen even when designing the tool. There is a risk of choosing false mathematical algorithms. There are some challenges to an automated decision-making system, such as possible errors. In comparison, mechanical decision-making benefits include effectiveness and efficiency. It becomes more complicated and more resilient to an investigation.15 Errors can be inevitable in the operational stage. Therefore, the system needs to be examined annually. Moreover, automated decision-making can analyze a wide range of far more factors than human beings can.16 Given that humans are limited to the variables, they can process one at a time. This tool can also learn from past events and produce far more accurate probability over time.17 However, these algorithms are prone to hacking, making them 'privacy-invasive.' This possible breach of privacy is not in line with the democratic principles of autonomy, dignity, and choice.18 Bellovin, a researcher on computer networking and security, explains that machine learning algorithms can deduce information. Simultaneously, it does not correlate with the input information that might be otherwise remained private. It is because of the natural limitations of manual and human- driven investigation.19 It concludes that automated decision-making provides a radical 13 J.R Sherman, et al., “First Generation Expert Systems in Social Welfare” (1989) 4:1-2 Computers in Human Services at 111-122 14 Cary Coliagnese and D. Lehr, “Regulating by Robot: Administrative Decision Making in the Machine-Learning Era” (2017) 105:5 Georgetown Law Journal at 105. 15 Frank Pasquale, The Black Box Society: The Secret Algorithms Control Money and Information (United States: Harvard University Press, 2015) at 16. 16 Grame S. Halford et al., “How Many Variables Can Humans Process” (2005) Psychological Science Sage Journals at 16 17 Harry Surden, “Machine Learning and the Law,” (2014) Washington Law Review, at 105. 18 Charles Duhigg, How Companies Learn Your Secrets (2012), online: . 19 Bellovin, supra note at 25. 306 | Legal Aspects of Artificial Intelligence on Automated Decision-Making in Indonesia change in the discourse of power. Language shapes our understanding and perceptions regarding the matter of legitimacy and legality.20 With all the various privacy discussions, people linkage privacy to anonymity and the "control" over our data.21 The discourse of law becomes engineering. Those who do not have access to technological privacy, discourse making Automated Decision-Making delegitimized in a democratic country.22 Another flaw of automated decision-making is the possibility of being biased. Algorithmic decision-making systems do not ignore partial data. They instead end up cementing those biases in society. This tool can help provide recidivism rates among criminals. However, it also can be prone to be biased to specific data as well. It can overestimate the recidivism risk to certain people of color or POC.23 IV. REGULATIONS AND PRACTICES OF AUTOMATED DECISION-MAKING: A COMPARISON Automated decision-making tools incorporate sophisticated mathematical algorithms to identify meaningful connections and likely patterns in big data sets.24 Algorithms can calculate on mainly qualitative data that an event would occur based on existing information.25 This technology can use data collected directly from the data subject, through observation, or collected from particular other data by deducing or deriving them. It can infer when decisions are made solely on automated means without direct human interference. It is then called automated decision-making. If an output of a decision has the interference of a natural person in evaluating and considering other attributes, it can affect the final decision. In that case, the decision is not solely automated. Therefore, the involvement of a person becomes significant enough to affect the final decision.26 For comparison, this section provides a few examples of how this technology is applied in selected countries. A. The European Union European Union countries are the member of the GDPR. Although there are definite and set guidelines regarding the data protection on the GDPR, the European Union's member countries remain required to introduce it in each municipal jurisdiction.27 They have 20 Foucault, Michael. The History of Sexuality, Volume 1: An Introduction (London: Allen Lane, 1978) at 168. 21 Julie Iness, Privacy, Intimacy, and Isolation (Oxford University Press, 1992) at 99. 22 Ari Ezra Waldman, supra note 8. 23 Julia Angwin et al., Machine Bias: There is Software Used to Predict Future Criminals. And its Biased Against Blacks, (2016), online: . 24 Emily Berman, “A Government of Laws and Not of Machines” (2018) 98 Boston University Law, at 1277-1279. 25 Steven Bellovin et al., “When Enough is Enough: Location Tracking, Mosaic Theory, and Machine Learning” (2013) 8 NYU Journal of Law & Liberty at 558. 26 Office of the Ombudsman Data Protection, Automated Decision-Making Automated Decision-Making and Profiling, online: . 27 Gianclaudio Malgieri, “Automated Decision-MakingAutomated Decision-Making in the EU Member States: The Right to Explain and Other "Suitable Safeguards" in the National Legislation” (2019) 35 Computer Law and Security Review at 1. 307 | LENTERA HUKUM various approaches in the field of automated decision-making, including prohibition and exceptions.28 The advanced technology and the corporation into the governmental system are inevitable. Then, the adaption of safeguards is necessary. The GDPR provides a solution within the set of rules to regulate the automated decision-making process. It is explicitly set in Article 22 (1) of the GDPR: “the data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her similarly significantly affects him or her.” Moreover, the GDPR is strict with individuals' consent, especially regarding sensitive data.29 A data controller is subject to safeguarding the data subject's rights and freedoms and legitimate interests. Consent of the data subject must be explicitly obtained to make the legitimate process. Also, Article 22 (1) explicitly outlines "legal effects concerning…" which has to be considered in a broad sense. Even if it is the case of discrimination of pricing on specific individuals, to some degree, it has legal effects.30 These provisions should be considered in a broad sense, not restricted in providing data subjects accountably and transparently.31 An example from the European Union countries is Germany. As the European Union's member, Germany has applied a sectoral approach to the automated decision-making provision. Article 37 of German Law (BDSG) states explicitly: “In addition to the exception given in Art 22 (2) (a) and (c) of the regulation (referring to General Data Protection Regulation by European Union), the right according to Art 22 (1) not to be subject to a decision based solely on automated processing shall not apply if the decision is made in the context of providing services according to an insurance contract."32 Insurance is an indispensable means of protection and is essential to individuals. Indeed, as mentioned above, the provisions are technical in setting out a condition on automated insurance decision-making. The requirements above mean that under specific cases, the conduct of automated decision-making is eligible. The provisions are appealing, given the fact that the only issue for implementing Article 22 (2) as a whole. They only allow the conduct of making in very few cases. Under strict regulation, the government can apply automated decision-making in general conditions. It is exemplified to contract, a consent obtained with implementing safeguards, restrictions, and data processing transparency. Automated decision-making can be applied under the condition that there is human intervention in the process. The conduct should be carried out by an authority and capable of changing decisions.33The 28 Ibid. 29 Article 9 (1) of the GDPR outlining the Special Categories of Personal Data. 30 Article 29 of the Working Party, "Guidelines on Automated Individual Decision Making." 31 Andrew Selbst, “Meaningful Information and the Right to Explanation” (2017) 7:4 International Data Privacy Law at 235. 32 Bundesdanteschutzgesetz vom 30 2017. 33 Article 29 of the Working Party, supra note 30. 308 | Legal Aspects of Artificial Intelligence on Automated Decision-Making in Indonesia focus of the GDPR is the protection of subject data. It is believed that the processing of data solely based on automation can generate severe risks of the violation of human rights. This practice can lead to group discrimination that can danger and affect an individual or a group. This practice's prohibition does not include systems with a human being to influence and significantly affect the final decision. Therefore, the participation of the automation facilitates the preparation for decision making.34 The automated decision must anticipate the legal consequences that may affect the data subject or influence them similarly. In this context, a decision involving legal effects will alter a data subject's legal status. For example, the person concerned is rejected by legal entitlement. "Similarly affected" will happen when a data subject is affected to receive negative impacts that do not have legal consequences. It is exemplified when a data subject receives terrible terms in a contract regardless of their legal entitlement.35 Despite the provision made in the GDPR, the prohibition of automated decision- making is sensible. This practice becomes conditionally permissible if the decision is generated of a contract between the data subject and data controller. However, the decision does not have to be a subject like in the online credit application. A problem can occur when there is an imbalance between a data subject and a data controller. In contrast, the data controller has a structural advantage in formulating contracts' elements if the laws approve the controller's country's decision. However, the provision must conform to human rights. As an example, the practice can be an exception when it involves a national law to allow the system to prevent tax fraud. Lastly, if the decision is based on the obtainment, the data subject is explicit consent. The data controller obtains explicit consent. According to Article 22 (2) of the GDPR, the provision refers to the approval of the automated decision circumstances.36 B. The United States The United States is one of a few countries that has applied this automated decision- making technology. In practice, AI has penetrated and helped usher regular tasks. It is exemplified from the Risk Assessment Tools to handle criminal cases and MedicAid at providing results of individuals' eligibility to access healthcare. For Indonesia, it will take a while to accelerate this technological advancement. It considers that Indonesia is so diverse with a vast population and unequal in terms of economic development. Simultaneously, it is also important to note that Indonesia has undergone the dramatic acceleration of communication, network, and technology. It provides an opportunity for Indonesia to develop AI to ease the government's task. As it stands, the government needs to ensure legal certainty in adopting this technology. The United States law on automated decision-making is technical. This law introduces human decision-making in various legal aspects. The legal basis of the 34 Stephen Dreyer and Wolfgang Schulz, The General Data Protection Regulation and Automated Decision-making: Will it Deliver (Germany: Bertelsmann Stiftung, 2019) at 18. 35 Ibid. 36 Ibid. 309 | LENTERA HUKUM conduct in New York State, for example, can be found in State Law 49 of 2018 regarding the Automated Decision Systems Task Force. This law outlines explicitly that the system will recommend the government use of automated decision-making. The additional examples are New York and Chicago that have implemented automated decision-making to assess child risk and safety, called Child Risk and Safety Assessment. The outcome of the decision is not ultimate. It is instead an advise whether a reported case of a child has to be further investigated.37 Another further example is the Risk Assessment tool for criminal justice. This technology uses algorithmic systems for already existing data to develop a 'risk score' used in informed decisions. It is a rate of the defendant's risk of conducting crime in the future. Though some sources, the system explains how the tool might be discriminative against black people.38 New York also has a fire risk assessment tool that uses data mining systems. They can predict what and where a fire can occur by analyzing the highest catching fire risks.39 To solve income discrimination, New York also has a tool of analytics on the source of income discrimination. The machine works to analyze public information to identify landlords that are more likely to be discriminative against applicants based on their income.40 At the national level, the United States has a tool that allows government officials to access personal data. These data range from medical records, criminal records, et cetera to identify new suspects for deportation and aid removal.41 Also, MedicAid, a federal and state program that deals with medical costs, has a benefits system to determine an individual's eligibility for access to Medicaid, benefits, termination, et cetera. Arkansas and Idaho are two states that use this algorithm. This machine can cut a person's health care.42 In assisting law enforcement, the United States utilizes public benefits fraud detection systems. These systems use pattern recognition for detecting fraud or abusing public benefits. Thus, the services of big data provide the development of a tool to fight against benefits fraud.43 Another example is a tool called Predictive Policing. It is used to 37 NYC Administration, NYC Administration for Children’s Services ACS Deploys New Technology to Help Frontline Staff protect NYC Children from Abuse and Neglect. Online: . 38 Julia Angwin, supra note 23. 39 Brian Heaton, New York City Fights Fire with Data, Analytics help NYC firefighters track potential hot spots. Justice and Public Safety (2015), online: . 40 Chris Bousquet, How New York is Protecting Affordable Apartments with Analytics, (2018), online: . 41 Spencer Woodman, Palantir Provides the Engine for Trump’s Deportation Machine, (2017), online: . 42 Colin Lecher, What Happens When An Algorithms Cuts your Health Care, (2018), online: . 43 Natasha Singer, Bringing Big Data to the Fight Against Benefits Fraud, (2015), online: . 310 | Legal Aspects of Artificial Intelligence on Automated Decision-Making in Indonesia identify a crime at a given time or identify people who might be a victim or perpetrator. New York Department Police possess this software.44 In Michigan, there is a tool called Fugitive Felony Compliance Systems. The program is automated and cannot cut an individual's public assistance benefits if they appear on SNAP and a list of outstanding felony warrants by law enforcement.45 C. China The Chinese government intends to build trust in Chinese society through the social credit system (SCS). The system is all about the vast database of individuals, corporate and government behavior, which includes lawfulness and the morality of their actions46 being monitored throughout the Chinese government. The system will use big data to build a trustworthy society where individuals and institutions comply with the law. The system will essentially work by issuing social credit scores to individuals and institutions based on their behavior. SCS will offer rewards and provide penalties based on the scores. To date, the Chinese government plans to launch this system nationwide by the end of the year. Even though the system has not been formally in effect, at least 33 million businesses have been put on blacklists. The system claims a rule-based system, having 43 model cities implementing the system differently. For example, in the Roncheng City model, individuals will be assigned a base scoring of 1,000 pts on a credit management system that links four governmental departments. These points later will be added and or deducted in the system with human interference government officials (e.g., it can be a fine for traffic penalties). In total, additional points will be added for the conduct of any of the 150 categories of positive behavior, and deduction points would happen for the conduct of any of 570 negative behavior. As a result, those who have obtained low social credit scores might not apply for specific jobs or even bank loans. The ones with high points might enjoy things such as priority in the hospital queue.47 D. Indonesia Automated decision-making often requires profiling, but it is not mandatory. It means that the use of automated decision-making can generate an output without profiling as an input. When profiling is involved, profiling is ready, and profiling data will be a decisive mechanical decision-making element.48 In this context, Indonesia has not 44 Rachel Waldman, Court: Public Deserves to Know How NYPD Uses Predictive Policing Software, (2018), online: . 45 Tresa Baldas, Court: Michigan Stiffed Deserving People Out of Food Aid (2016), online: . 46 Rogiers Creemers, “Planning Outline for the Construction of a Social Credit System” (2014) 14 China Copyright and Media. 47 Ibid. 48 Information Commissioner’s Office, What is Automated Decision Making and Profiling? online: . 49 Personal Data Protection Bill. 312 | Legal Aspects of Artificial Intelligence on Automated Decision-Making in Indonesia explain which institution will settle the dispute. Most importantly, to what extent the subject data is allowed to object. The clarity and the gravity of the urgency of Article 10 of the GDPR need to be put on the provision to avoid legal uncertainty. According to legislative drafting principles, the requirements should include clarity of objectives, appropriate institutional or forming officials and conformity between types, hierarchy, and material content that can be implemented efficiently, clarity of formulation, and openness.50 Therefore, there is an urgency for lawmakers to set a comprehensive legal framework regarding automated decision-making that includes profiling into Indonesian law. Then, lawmakers should depict this regulation's importance by explaining the law's objectives with clarity and the law's formulation. The possible prospects of how the future automated decision-making regulation should include: (a) the government stance of automated decision-making, whether it is lawful, unlawful, or conditional; (b) classifications on different types of decisions, especially those that have a legal or similarly significant effect on individuals. (c) the institutions that have authority to practice automated decision making; (d) an independent body to oversee the practice of automated decision making, collecting personal data, and profiling throughout the country; (e) a technical manual, safeguard, or standard for institutions that practice automated decision-making; the monitoring of the use of automated decision-making tool processing; (f) to what extent that subject data can object to their involvement in a decision made by automated means; (g) legal institutions that are competent to settle the dispute regarding a raised objection; (h) possible legal remedies that subject data can pursue if they objected; (1) it is paramount to comprise a robust penalty system within the future automated decision-making regulation. The current penalty system on data protection in Indonesia remains weak. To wrap up this section, forming a sound regulation should involve both national and international principles. The regulation on personal data protection should refer to Article 28 (G) of the 1945 Constitution. It clearly states that the right to privacy shall be protected, given that personal data protection is part of the right to privacy. The regulation's forming should also involve the International Covenant on Civil and Political Rights (ICCPR), which was ratified by the Indonesian government. The ICCPR recognizes the protection of the right to privacy of personal data as a fundamental right. It is relevant to Law Number 11 of 2009 on Information and Electronic Transactions.51 V. PROFILING AND THE CONNECTION TO AUTOMATED DECISION-MAKING Profiling requires a collective of personal data and the action of putting them in categories. These categories are based on their attributes and characteristics. Profiling 50 Ferry Irawan Febriansyah, “Konsep Pembentukan Peraturan Perundang-Undangan di Indonesia” (2016) 21:3 Jurnal Perspektif at 222. 51 Sinta Dewi Rosadi, “Protecting Privacy on Personal Data in Digital Economic Era: Legal Framework in Indonesia” (2018) 5:1 Brawijaya Law Journal at 150. 313 | LENTERA HUKUM has aims to identify unknown individuals based on intelligence. It becomes a predictive method to identify unknown individuals searching for law enforcement.52 According to Articles 4 and 22 of the GDPR, profiling is any form of automated data processing to derive, infer, predict, or evaluate particular attributes, demographic information, behavior, or identify a person.53 There are several categories to distinguish different types of profiling, as explained by Valeria Ferraris.54 There are 'individual profiling' that collects information on an individual and/or employ that information to derive, infer, or predict unknown characteristics of the individual's behavior in the future. Another one is 'group profiling' that aggregates information about a group of individuals. The conduct of profiling can collect data on members of a group that shares a particular attribute (distributive profiling) or a group of people without sharing details (non-distributive profiling).55 However, individual and group profiling may be conducted directly or indirectly. If conducted directly, profiling will use data that has been provided by a group or an individual and proceed to use the data to derive, infer, or predict unknown attributes or future behavior. If conducted indirectly, profiling will rely on data from a larger population and identify individuals based on characteristics that have appeared from the larger population.56 The use of profiling comes in handy in providing the knowledge needed by analyzing existing data to assume an individual. Profiling uses past experiences and analysis to provide correlations between particular attributes and specific outcomes; to be involved in (automated) decision making by using the generated correlations.57 Despite the benefits of profiling, they have potential risks that would bring unlawful conduct. Here are the risks: the practice and result of profiling can get stereotypes against people. It can lead to discrimination; the practice of profiling can generate false correlations and may not be accurate for individuals.58 Here is an example form or incorrect profiling: the assumption that 'women grow hair faster than men' is supported by factual researches made in the past. However, any particular man may grow hair more quickly than any specific woman. Any automated decision-making towards women or men based on this assumption would bring out the risk of being inaccurate and can potentially affect these particular individuals of a false premise. Automated decision- making involving profiling can generate potential bias, and unlawful data is based on subjective and unreasonable justifications. It does not comply with fundamental rights. Based on the European Commission against Racism and Intolerance, profiling should 52 European Union Agency for Fundamental Rights, Preventing Unlawful Profiling Today and In the Future: A Guide (Luxembourg: Publications Office of the European Union, 2018) at 15. 53 Ibid. 54 Valaria Ferrari et al., Defining Profiling (2013), online: . 55 Ibid. 56 David-Olivier Jaquet-Chiffelle, Direct and Indirect Profiling in the light of virtual persons, Profiling the European Citizen, Cross-Disciplinary Perspectives (Netherlands: Springer Netherlands, 2008) at 18. 57 European Union Agency for Fundamental Rights, supra note 52 at 16. 58 Ibid. 314 | Legal Aspects of Artificial Intelligence on Automated Decision-Making in Indonesia pursue a legitimate aim. There has to be a reasonable relationship of proportionality between the means employed and the purpose sought after. There are protected grounds that should be made except profiling such as birth, color, ethnicity, genetic features, language, race, religion, political opinion, and member of a national minority. However, these protected grounds may be revealed from other personal data.59 Therefore, profiling will be deemed illegitimate if profiling involved acts of unjustified treatment based on protected grounds or if profiling interfered with an individual's private life without consent and if done unnecessarily.60 Automated decision-making often requires profiling, but it is not always necessary to involve profiling and vice versa. A single processing activity can involve both depending on the indicators, such as the data being applied. However, decisions based on automated processing alone can inquire about profiling. For example, suppose an education institution says a university processes their students' applications when applying to that university. In that case, each individual plays a significant part in the decision-making process preceding the final acceptance decision, such as their high school origin and completing paperwork. These elements would further affect the decision made. A. The Rule of Law As a fundamental idea, the rule of law will instead be an arduous task to achieve. It has to be understood that no country might claim perfect adherence to the rule of law. 61 Furthermore, the rule of law means that the law is the supreme law to all society layers, including the government's officials (either executive, legislative, or judicial power). It also has to be known by the citizens, stable, and predictable for the future. Society has to be able to participate in making laws, at the very least being able to voice their opinions. It also has to protect human rights with just legal processes. Lastly, the power invested in judiciary power has to be independent of other bodies and judge without clouded by interference.62 The law binds both government and individuals. It is the exact opposite of an arbitrary ruling.63 Dicey postulates that the rule of law comprises of three elements: Supremacy of law, which constitutes that the law is above the man, including officials and governments. It is the antithesis of arbitrary power, equality before the law, which includes everyone from all citizen layers subject to the same treatment by the law court. It asserts that there is no exception for the government officials that do not obey the law; a constitution-based on human rights means that the constitution should comprise human rights as fundamental rights.64 Moreover, regarding automated decision-making, which gives individuals legal effects, it has to be in line with the rule of law's values. The incorporation of a machine and software as decision-making tools must be programmed 59 European Union Agency for Fundamental Rights, supra note 52 at 23. 60 Ibid. 61 Robert Stein, “Rule of Law: What Does it Mean?” (2009) 18:2 Minessota Journal of International Law at 303. 62 Ibid. 63 Anthony Valcke, The Rule of Law: Its Origins and Meanings (A Short Guide for Practitioners) (2012). 64 A.V. Dicey, The Law of the Constitution (Oxford University Press, 2013) at 252. 315 | LENTERA HUKUM with applicable law and often renewed and supervised to avoid glitches. Glitch and unsupervised software can cause legal uncertainty for individuals. B. Principles Relating to the Protection of Personal Data The use of automated decision-making and the principles relating to protecting personal data become relevant. Given the fact, the process of a single decision making consists of collecting an individual's data. For instance, under Article 5 of GDPR, personal data processing shall constitute the following principles: legality, fairness, and transparency, purpose limitation, minimization of data; accuracy; storage limitations; integrity, and confidentiality.65 Sinta Dewi, a personal data protection expert, in her book, calls cyber law. She then concludes a set of personal data protection principles. First, the collection restrictions set limits in collecting personal data. The data obtained must use legally and fairly legitimate methods and the data subjects' consent.66 Second, the data quality defines the purpose of the data used, the data's accuracy and the current data.67 Third, the purpose specification that puts forward the objectives why information is needed and the use of data must follow data collection's purpose.68 Fourth, the use of restrictions that state that data must not be disclosed, available to the public, or used for purposes other than specific purposes unless the data owner or general authorities consent.69 Fifth, security measures state that data must be protected with appropriate safeguards to protect it from loss, damage, use, alteration, or exposure.70 Sixth, the openness, which requires a general policy on the disclosure of personal data.71 Seventh, individual participation states that individuals must have the right to obtain information about their data and delete or correct erroneous data.72 Eighth, accountability states that the data controller is responsible for complying with these steps.73 The use of automated decision-making can potentially harm an individual's right to data protection because an individual is subjected to an automated decision. Simultaneously, the necessary data is obtained from previous use without consent. Then, it will breach the principle of purpose limitation under Article 5 of the GDPR. It is also somewhat tricky to implement the fairness and transparency principle into solely automated decision-making by considering the nature of the practice itself as a process by automated means. 65 European Union, General Data Protection Regulation (2016), online: . 66 Sinta Dewi Rosadi, Cyber Law: Aspek Data Privasi Menurut Hukum Internasional, Regional dan Nasional (Bandung: Refika Aditama, 2015) at 29. 67 Ibid. 68 Ibid. 69 Ibid. 70 Ibid. 71 Ibid. 72 Ibid. 73 Ibid. 316 | Legal Aspects of Artificial Intelligence on Automated Decision-Making in Indonesia VI. CONCLUSION Automated decision-making can be a tool for efficiency and effectiveness in generating decisions that conventionally can take quite some time to finish. However, the use of automated decision-making has further challenges. The privacy rights issue to the use of this technology remains unfinished as it needs further legal protection. It is paramount to set a sound legal framework by setting approval, penalty, restrictions, and safeguards of automated decision-making and personal data regulation by adopting the GDPR's framework. An independent body to control and oversee private data collection is also deemed necessary to fill the void of the overall legal protection of automated decision- making in Indonesia. ACKNOWLEDGMENTS None. COMPETING INTERESTS The authors declare that they have no competing interests. REFERENCES A.V. Dicey, The Law of the Constitution (Oxford University Press, 2013). Ahmed Habeeb, Introduction to Artificial Intelligence (2017), online: https://www. researchgate.net/ publication/325581483. Ananthi Sheshasaayee, “A Study of Automated Decision Making Systems: Research Inventory” (2017) 7:1 International Journal of Engineering and Science. Andrew Selbst, “Meaningful Information and the Right to Explanation” (2017) 7:4 International Data Privacy Law. Anthony Valcke, The Rule of Law: Its Origins and Meanings (A Short Guide for Practitioners) “Encyclopedia of Global Social Science Issues, ME Sharp Publishing (2012). Ari Ezra Waldman, “Power, Process, and Automated Decision-Making” (2019) 88:2 Fordham Law Review. Brian Heaton, “New York City Fights Fire with Data, Analytics Help NYC Firefighters Track Potential Hot Spots” (2015) online: . Cary Coliganese and D. Lehr, “Regulating by Robot: Administrative Decision Making in the Machine-Learning Era” (2017) 105:5 Georgetown Law Journal. Charles Duhigg, “How Companies Learn Your Secrets” (2012) online: . 317 | LENTERA HUKUM Chris Bousquet, “How New York is Protecting Affordable Apartments with Analytics” (2018) online:. Colin Lecher, “What Happens When An Algorithms Cuts your Health Care” (2018) online: . David-Olivier Jaquet-Chiffele, Direct and Indirect Profiling in the light of virtual persons, Profiling the European Citizen: Cross-Disciplinary Perspectives (Springer Netherlands, 2008). Emily Berman, “A Government of Laws and Not of Machines” (2018) 98 Boston University Law Review. European Union Agency for Fundamental Rights, Preventing Unlawful Profiling Today and In the Future: A Guide (Luxembourg: Publications Office of the European Union. 2018). Frank Pasquale, The Black Box Society: The Secret Algorithms Control Money and Information (United States: Harvard University Press, 2015). Ferry Irawan Febriansyah, “Konsep Pembentukan Peraturan Perundang-Undangan di Indonesia” (2017) 21:3 Jurnal Perspektif. Gianclaudio Malgieri, “Automated Decision-Making in the EU Member States: The Right to explanation and other "suitable safeguards" in the National Legislation” (2019) 35 Computer Law and Security Review 35. Grame Halford, et al., “How Many Variables Can Humans Process?” (2005) Psychology Science Journal. Harry Surden, “Machine Learning, and the Law” (2014) 89:1 Washington Law Review. Information Commissioner’s Office, “What is Automated Decision Making and Profiling?” online: . J.R Sherman et al., “First Generation Expert Systems in Social Welfare” (1989) 4:1-2 Computers in Human Services. Julia Angwin, et al., “Machine Bias: There is Software Used Across the Country to Predict Future Criminals. And Its Biased Against Blacks” (2016) online: . Julie Iness, Privacy, Intimacy, and Isolation (UK: Oxford University Press, 1992). Karen Yeung, “Algorithmic Regulation: A critical interrogation. Regulation & Governance” (2017) King’s College London Law School. Michel Foucault, The History of Sexuality Volume 1: An Introduction (London: Allen Lane, 1979). Natasha Singer, “Bringing Big Data to the Fight Against Benefits Fraud” (2015) online: . 318 | Legal Aspects of Artificial Intelligence on Automated Decision-Making in Indonesia NYC Administration for Children’s Services. “ACS Deploys New Technology to Help Frontline Staff protect NYC Children from Abuse and Neglect” online: . New York CLU, “Automatic License Plate Readers” online: . Office of the Ombudsman Data Protection, “Automated Decision-Making, and Profiling”, online: . Peter Lucas and Linda van der Gaag, Principles of Experts Systems, Centre for Mathematics and Computer Science (Amsterdam: Addison Wesley, 1991). Rachel Waldman, “Court: Public Deserves to Know How NYPD Uses Predictive Policing Software” (2018) online: . Robert Stein, “Rule of Law: What does it Mean?” (2018) 18:2 Minnesota Journal of International Law. Rogiers Creemers, “Planning Outline for the Construction of a Social Credit System” (2014) 14 China Copyright and Media. Sinta Dewi Rosadi, Cyber Law: Aspek Data Privasi Menurut Hukum Internasional, Regional, dan Nasional (Bandung: Refika Aditama, 2015). Sinta Dewi Rosadi, Protecting Privacy on Personal Data in Digital Economic Era: Legal Framework in Indonesia (2018) 5:1 Brawijaya Law Journal. Simon Stern, “Artificial Intelligence, Technology, and Law” (2018) 68:Supp 1 the University of Toronto Law Journal. Spencer Woodman, “Palantir Provides the Engine for Trump’s Deportation Machine” (2017) online: . Stephen Dreyer and Wolfgang Schulz, The General Data Protection Regulation and Automated Decision-Making: Will it Deliver? (Germany: Bertelsmann Stiftung, 2019). Steven Bellovin, et al., “When Enough Is Enough: Location Tracking, Mosaic Theory, and Machine Learning” (2013) 8 NYU Journal of Law &Liberty. Stuart Russell, Artificial Intelligence: A Modern Approach (New Jersey: Prentice-Hall, 1995). Tresa Baldas, “Court: Michigan Stiffed Deserving People Out of Food Aid” (2016) online: . Valaria Ferrari et al., "Defining Profiling" (2013) online: .