International Journal of Human Capital Management, Vol. 2 (2), December 2018 ‘ Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic Chia-Hua Chang Southern Taiwan University of Science and Technology, Taiwan Husam Almaghalsah Southern Taiwan University of Science and Technology, Jordan Email: husam.maghalsah@gmail.com ABSTRACT The aim of this research intends to investigate Taiwanese egovernment websites with regards to Accessibility and Usability. The automated WAVE Accessibility tool was test Accessibility and Nielsen’s 10 Heurestic Principle based surveys of 100 foreign users of the English egovernment websites were used to test the Usability. The study found poor results that were unanimous across the board for Usability and total of just over 100 errors for Accessibility. The study proves that further research and investigation is necessary if Taiwan wants to meet its own governmental ambitions with regards to egovernment . Keywords: Usability, Accessibility, E-Government, Heuristic Evaluation, Automated Accessibility Evaluation Tool Received: 3 February2021 ; Accepted: 8 February 2021 ; Publish: June 2021 How to Cite: Chang, C.H., Almaghalsah, H. (2021). Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41. https://doi.org/10.21009/JOBBE.005.1.02 https://doi.org/10.21009/JOBBE.004.2.01 17 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 INTRODUCTION The Currently in Taipei there may an elderly person becoming visually impaired where strict social distance practice is being exercised due to the pandemic’s different strains and so has to use the National Health Care website. In another instance a foreigner may need the immigration website to get a current update on travel guidelines and restrictions. A third example may include any disabled person in Taiwan attempting to use any government website. Our current climate has changed and created circumstantial elements like the pandemic and an increasingly aging population to rely heavily on websites to perform more in-person functions as well as accommodate an elderly’s aging needs that deem some of them partially disabled. Although the telephone is still an option it does cost time and money to address basic needs, functions, and information that could all be accessed online. This study will highlight two approaches, first is an automated WAVE ACCESSIBILITY test of egovernment websites, and secondly an English egovernment website foreign user survey based on Nielsen’s 10 Heurestic Principles. The pandemic served to increase the need for usability in three separate ways. First logistically in the circumstantial elements especially for the elderly, and secondly with Accessibility being an urgently dire, life or death, situation for this vulnerable demographic (Dror et al., 2020). A larger and perhaps a less web exposed demographic, the elderly have been the most vulnerable to COVID19. More folks are being forced to now engage with the web due to COVID (Dror et al., 2020). The pandemic has created a work from home phenomena which compounds this as well while it doesn’t seem to be a trend that will be waning in the post pandemic world that is to come. Social distancing and performing tasks remotely due to strict quarantine measures has been the safeguard way Taiwan has been able to defeat this virus not only in infection rates and deaths but also economically. The Taiwanese way of life was largely preserved due to the strictest quarantine measures and contact tracing undertaken by the government of those infected or entering the country (“Taiwan’s success in curbing COVID-19 tops domestic CNA news of 2020 - Focus Taiwan,” n.d.). It’s two fold in that it increases remoteness forcing more seniors, the disabled, and people in general to engage with the web and this in turn places a more urgent spotlight on usability of website and Accessibility of desktop websites especially for government services. The modern pandemic reality makes it urgently imperative to develop a proper egovernment web experience across the board, but also in particular with the more vulnerable such as the disabled, to prevent spread of any current or future viruses. The number of disabled persons isn’t decreasing neither is the need to serve the vulnerable. Accessibility refers to the ability for disabled users such as the visually and or hearing impaired to use a government website effectively. Reasons for adoption of accessibility testing is made and listed as logistical, ethical, and economic. Logistical refers to the increasing number of disabled through old age as well as increasing lifespan of all humans. Ensuring the government can serve its citizens’ needs also necessitates accessibility which is ethical and especially critical due to the nature of many egovernment services being health related. Understanding the economic ramifications of neglecting a standard of service deemed usable to this demographic should also be clear in the spending power this demographic possesses as well as governmental cost saving measures of automation versus in-person service or call. The automated usability test tool of egovernment websites focuses on addressing this need most efficient ly and effectively with the WAVE Accessibility tool since it is the most widely acclaimed and used (Alsaeedi, 2020; Shawar, 2015). The case for the use of automated test tools is made due to its mechanics, variable consistency, environmental factors accounted for just to mention a few and the quick application in avoiding distribution for instance, thus being the most effective to deliver quick results. The expansion of the web technologies not only displays the economic cost saving measures for the disabled but also through the ease of implementation of emerging Accessibility technologies. https://doi.org/10.21009/JOBBE.005.1.02 18 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 The expansion of web technologies also displays the need for a usability testing strategy. The web reality of the modern world presents the immediate need for a complete usable web solution to using government services. The case for a specific user based website usability evaluation survey based on Nielsen’s Heurestics follows the technological evolution of the IT industry, which has also drastically changed and continues to modernize to meet the needs of a new generation of web users, work from home was statistically insignificant a mere decade ago. The more tailored user centered needs that would be niche or customized to the users interaction with the egovernment website requires a more user based approach, such as a survey. This ensures the evaluation of the usability of the egovernment sites based on the specific user’s purpose, as in the case of the English website foreign user. Compounding the case for English website usability testing is made and the Taiwanese government’s pledge to become a fully bilingual society by 2030 makes the case for Nielsen 10 Heuristics Principles based survey have a foreign perspective. The trade driven economy and tourism sectors of Taiwan round off these compelling factors for adopting this study’s techniques and will use a Nielsen criteria based open and close ended satisfaction ranking questions to make future suggestions and recommendations based on the end user (Mahajan et al., 2018, Kaur and Vinda 2016, Hung et al., 2012). LITERATURE REVIEW This study will highlights two approaches first is an accessibility automated WAVE ACCESSIBILITY test of egovernment websites, and secondly an English speaking user survey based on Nielsen’s 10 Heurestic Principles. Previous research has shown a correlat ion between usability and ease of use as well as rate of use including the satisfaction of the user. Research then reveals the reasons for using a foreign user of the English government website presented in accordance to Taiwanese government initiatives as well as economic factors. For Accessibility the research outlines the major guidelines and indicators as well as the reasons of Accessibility testing adoption including the benefits of the test being automated. However, firstly a brief summation of recent development of egovernment in Taiwan is required to understand the current status of egovernment and its potential for development 2.1 Status and Development of egovernment in Taiwan Taiwan’s NII established the island as a major IT player in communications in East Asia (Deakins, Dillon, & Chen, 2007). Gov.tw serves as the central access center of e-government websites with financial transaction capacity, e-certificates, navigation to utility systems and form capability. Following those initiatives came Taiwan’s DIGI+ (Digital Nation and Innovative Economic Development Plan plan from 2017 through to 2025 have goals set in its final year to have grown Taiwan’s cyber economy to over $200 billion USD, and digitization range of over 80% placing the island in the top 10 internationally in IT. This past year saw the end of NDC’s promotion of Taiwan’s Digital Government initiative which was started in 2017, the island’s fifth level of e-government advancement. This 5th level of advancement plans to synergize the latest technologies like AI, cloud, etc into governmental services that would replace the archaic method of users in person or by phone requiring human personnel, thus automizing governmental services. User customized service, civic and civil participation, as well as information based implementation are the central triad of the government initiative. This study postulates that these two approaches to usability testing listed above are undoubtedly critical toward meeting Taiwan’s egovernment user centered development (Toshio, 2018). https://doi.org/10.21009/JOBBE.005.1.02 19 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 2.2 Usability Usability refers to user and technical device interaction and the simplicity or ease of use to effectively and efficiently perform tasks on the device (Määttä, 2007). Effectiveness of a website lies in its usability (De Lima Salgado & Freire, 2014; Khajouei, Zahiri Esfahani, & Jahani, 2017), and frequency of use (Z. Huang & Brooks, 2012a).Therefore usability has been cited in numerous products, projects, and design of different systems. (Lyzara, Purwandari, Zulfikar, Santoso, & Solichah, 2019; Verkijika & De Wet, 2018). The most broad and conventional definition of usability is “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” (Majrashi & Hamilton, 2015; Seffah & Padda, 2006). This has evolved and ranged from “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction, in a specified context of use” to “the capability of the software product to be understood, learned, and liked by the user, when used under specified conditions”. However the ISO definition of e-Government Usability refers to “the extent to which a website can be used by citizens to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified e- Government service context.” Consequently usability plays a vitally important part in the success of egovernment websites. Most studies suggest comprehensive and detailed criteria strategies in web development and design (Z. Huang & Benyoucef, 2014; King & Youngblood, 2016; Nielsen, 1994; Wang & Senecal, 2007). Specifying usability will enable the assessment of usability to evolve from generalized criteria like ease of use to a more comprehensive position that entails the use of 10 usability elements. These elements include visibility of system status; synchronicity between system and the actual world; user control and independence; consistency and standards; error prevention; recognition instead of recall; flexibility and efficiency of use; look and simplistic design; issues recovery; and assist functions. Breaking down usability to these specific factors makes it easier to pinpoint specific factors and narrow down the issues to specific repairable criteria thus resulting in higher degrees of usability. (Z. Huang & Brooks, 2012a, 2012b), showed that an effective and more suitable egovernment site usually refers to high usability. (Chiew & Salim, 2003; Leist & Smith, 2014) state that “usability is now a major element in identifying the effectiveness and consequent objectives of computer software.” Findings from research reveal the electronic systems such as health-care systems (Weerakkody, Irani, Kapoor, Sivarajah, & Dwivedi, 2017), e-banking (Fernandez, Insfran, & Abrahao, 2011; Z. Huang & Brooks, 2012b; Mulvey, 2008), e-commerce (Díaz, Rusu, & Collazos, 2017; Zhang, Meng, Guo, Yin, & Luo, 2015) and digital libraries (Fernandez et al., 2011; Fung, Chiu, Ko, Ho, & Lo, 2016; Lyzara et al., 2019), service or system quality lies in usability as its critical factor (Belanche, Casaló, & Guinalíu, 2012a). Therefore undoubtedly optimum usability systems and websites will be frequently visited. 2.3 How Usability Influences the User Usability has been found to positively and negatively influence the user’s degree of perception, trust, and fulfillment of e-government. This was one of the major findings by a study conducted by (Wirtz & Kurtz, 2016). The elements that revealed to be most notable by users was the fluidity of pages or forms, security, pertinent content, site usability, system stability, and help. Other studies on user’s perception of e-government were conducted by (Wirtz & Kurtz, 2016). Over 5500 users indicated that preferable websites relied on usability like flexibility, easy to find, quick to load, relevant and useful content as well as accessibility. The higher usability will positively shape the user’s perception resulting in higher rate of use of the e-government. These results were revealed in an earlier study by (Mahajan, Abolhassani, McMinn, & Halfond, 2018) but also suggested a comprehensible and applicable formula to compute and interpret the https://doi.org/10.21009/JOBBE.005.1.02 20 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 influencing elements of a user’s embracing e-government websites like satisfactory service, web design, and the user’s fulfillment with the site overall and features. The results showed that web design in accordance with the user’s purpose attained within an expected time frame as the critical element that results in better user engagement and opinion of e-government. Additionally the user felt ignored when industry expert factors or recommendations were the only ones considered for e-government therefore improving web design in accordance with look, information, accessibility, navigation, and flexibility as most positively influential in compelling the individual to use egovernment regularly as 5500 user based studies revealed. To conclude studies above show that the usability of e-government plays a major role in the user’s adoption and opinion of e- government and that the Taiwanese goals of e-government are dependent on usability. 2.3.1 Reasons of English website survey testing adoption The Taiwanese government has pledged to make this island fully bilingual by 2030 (“Taiwan’s 2030 goal to become a bilingual nation(2019-01-11)- Financial Supervisory Commission,” n.d.). Therefore the application of usability testing tools over both English and Mandarin sites will be vital. However in Taiwan this also must be analyzed and studied from the foreign perspective as well. It is imperative therefore once recommendations based on the tool’s assessment be designed and implemented with a foreign perspective in hand. Culturally contextual qualitative solutions have a greater probability of being executed with foreigners that understand their culture better than anyone else would. The manual survey questionnaire would serve to address these and any additional concerns as well as fill in gaps left by an automated test tool that would be quantitative in nature and under stricter expert based parameters therefore web usability testing of egovernment English websites is indisputable.(Bashir & Farooq, 2019; Dourado & Canedo, 2018; Ji, Park, Lee, & Yun, 2006). 2.3.2 Economy and Tourism Taiwan’s economy has an import and export based component to it. In fact foreign trade has seen a significant uptick every year with corporate partners or companies being based out of the UK, Germany, USA, Japan just to name a few. This is also not restricted to business the same exists with tourism as tourist numbers hit a record high in 2019 (Giroud & Ivarsson, 2020). Taiwan’s handling of the pandemic has only increased its visibility and exposure in the world so it is a safe bet to assume that Taiwan’s tourism only continues to increase once post pandemic travel resumes. Taiwan as stated in the preceding section has reemerged on the world stage wit h its reliable and technical knowhow and experience with pandemics and has led the developed world (“Taiwan’s success in curbing COVID-19 tops domestic CNA news of 2020 - Focus Taiwan,” n.d.). It would be ill advised to neglect its responsibility to its citizens, foreigners, students, or other residents by ignoring usability of its websites. Especially in this post pandemic era an Accessibility issue or a usability issue on an English egovernment website related to health could be fatal. The extraordinary efforts and expertise of Taiwan’s Health Ministry cannot afford to fail on any type of user. In summation Taiwan needs to look at adopting an Accessibility test tool for all its government websites whilst ensuring that both Mandarin and English websites’ usability as critically as it views national security since its trade and tourism sectors rely on English websites. 2.3.3 Use of English Speaking Foreigners as Survey Participants Several factors need to be considered in testing the usability of a website in cultural context as well its linguistic and functional efficacy. There are 9 such factors color, GPS, location of video and content, language options, mobility within the website and web design, accurate translation, https://doi.org/10.21009/JOBBE.005.1.02 21 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 content and page size congruency, lettering and character visibility, keywords, acronyms, and abbreviations. Language selection and usability has often been localized by IP tracking which works well when the English speaker is residing in the country of origin. So for instance a MOFA website user from Canada would be well served with IP tracking however for most foreigners residing in Taiwan the IP tracking will render the use of the site dysfunctional therefore the visibility and accessibility of a language transfer link is vital to meeting this objective. Graphic and placement of text needs to considered in cultural context hence the use English speaking foreigners, whereby the horizontal and vertical placement of left to right and up to down will also need to be considered (Carroll & Carroll, 2015). Color is also not immune to cultural context. “For example, testing has in fact shown that efficiency in the intelligibility and interactivity actually increased in Russia when using a black and red combination for a “Call to Action” button whilst in Italy by changing it from red to orange (Arasid et al., 2018; Z. Huang & Brooks, 2012a; Sá, Rocha, & Pérez Cota, 2016). “ (Alassaf & Amr, 1997; Majrashi & Hamilton, 2015; Sauro, 2016) Literal translation is where problems arise under the translation criteria. Abbreviation and keywords such SIDA versus AIDS or 119 as opposed to 911 in the west are some examples of usability problems. Localizing the website would consider the use of Fahrenheit as opposed to Celsius for an American user which could be part and parcel of the language version used, American or Canadian English or UK English or Australian English in this instance only the survey of foreign English speakers would outline the desired language. IP tracking can assist in this capacity as well but for foreign users residing in Taiwan or at locale other than the country of their origin would still leave this issue not remedied. With the decreasing attention span of website users “the satisfaction of the page layout can be made more uniform by adopting the same layout style but also taking into consideration the textual reading conventions of different cultures.” Font size legibility can range from the use of technical verbiage such as a cursive l for liters and kilometers in cultural context. Finally the fitting of text into webpages rounds off the cultural factors to be considered when testing the usability of foreign English websites (Fontdevila, Genero, & Oliveros, 2017; Mukhoryanova, Novikova, Rudich, & Bogushevich, 2016) 2.4 Accessibility “One of the major elements of usability is accessibility that means the easiness of user with disabilities to access the sites.” (Arasid et al., 2018; Da Silva, Da Silva, & De Oliveira Moraes, 2019; Hung, 2012; Lessa & Tsegaye, 2019; Verkijika & De Wet, 2018) Accessibility issues range from restricted or limited visibility to color blindness or even complete permanent blindness as an example. Most disabilities have a wide range of conventional sensory human dysfunctionality. Regardless of the sensory disability or its severity the accessibility “should offer equal opportunity to access information, removing some barriers to the communication and interaction that faced by users with disabilities in the real world” (Domínguez Vila, Alén González, & Darcy, 2018a; Federici et al., 2005; Król & Zdonek, 2020; Kuzma, 2010; Nuñez, Moquillaza, & Paz, 2019). WAVE Accessibility testing enables optimum accessibility. Therefore WAVE will assist in determining whether or not the egovernment website “is both accessible and compliant.” (WAVE Website 2020). Accessibility according to the WCA guidelines have 4 guidelines and indicators and they are recognized as the authority for designing websites for the disabled. The guidelines and indicators are listed here: 1. Perceivable: Content and clickable items must be presented with respects to their disability. In other words it can't be incomprehensible to all of their capacities. https://doi.org/10.21009/JOBBE.005.1.02 22 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 2. Practicable: Mobility with the page and the different parts must be functional. The website should not require a prompt that is beyond the disabled person’s capacity. 3. Comprehensible: Content and functionality of the page must be understandable. Information or functionality should not be inoperable. 4. Strength: Information on the website should be able to withstand any problems with regards to accurate understanding by a range of different types of individuals such as tech support or other IT experts or help automated tools. Regardless of upgrades or emerging technologies or software or platforms should not impact the users’ capacity to obtain the information or service the user requires. (“Introduction to Understanding WCAG 2.1,” n.d.) “If any of these are not true, users with disabilities will not be able to use the Web. Under each of the principles are guidelines and Success Criteria that help to address these principles for people with disabilities. There are many general usability guidelines that make content more usable by all people, including those with disabilities. However, in WCAG 2.1, we only include those guidelines that address problems particular to people with disabilities. This includes issues that block access or interfere with access to the Web more severely for people with disabilities.” (“Introduction to Understanding WCAG 2.1,” n.d.) 2.4.1 Reasons of Accessibility testing adoption Worldwide there are over 750 million disabled people and this number will only increase with medical advances increasing the lifespan of humans everywhere. In Taiwan in particular having one of Asia’s oldest populations it is imperative that Accessibility be addressed as this demographic continues to age. There would be an economic burden forcing these folks to bog down the system by needing direct human support each and every time. The pandemic reality outlined above also further complicates this interaction and leaves the most vulnerable a burden on the Taiwanese economy. All the while the folks needing Accessibility will feel more vulnerable due to the government’s inability to provide services that won’t place them at risk of infection at worst or inconvenience them with cost and time spent on having to come in to an office in person at best. They would also feel further ostracized if not outright discriminated against by their own government through its benign neglect to provide the same degree and level of services as their sons or daughters have access to. Additionally, disabled folks make a significant portion of the economy as consumers and producers. In other words there are disabled folks that have amassed great resources like Stephen Hawkins. To summarize there are humane, economic, and ethical factors for providing attention and resources to Accessibility in egovernment development of websites. 2.4.2 Automated Test Tool This study intends to make the case for automated evaluation of egovernment websites for Accessibility using the WAVE Accessibility test tool on desktops. The automated WAVE accessibility tool is the industry standard for Accessibility problems of desktop websites (Adepoju, Shehu, & Bake, 2016; Ansari, Baqar, Hassan, & Saeed, 2016; Hung, 2012; Karaim & Inal, 2019; Mtebe & Kondoro, 2017). Accessibility on Taiwanese egovernment websites on the whole is a critical element if not a foundational component of designing egovernmental. Accessibility should apply in similar quality across all egovernment websites in Taiwan as it reflects a developed government’s competency to care for one of its most vulnerable constituency, the disabled. Furthermore since the automated WAVE ACCESSIBILITY tool is also used the actual accessibility standards will be conventionally applied on the desktop websites (Campoverde- https://doi.org/10.21009/JOBBE.005.1.02 23 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 Molina, Lujan-Mora, & Garcia, 2020; Domínguez Vila et al., 2018a; Hassouna, Sahari, & Ismail, 2017; Król & Zdonek, 2020; Máñez-Carvajal, Cervera-Mérida, & Fernández-Piqueras, 2019; Nuñez et al., 2019; Paul & Das, 2020). General shortfalls in the use of automated test tool may be the strictly quantitative approach in addition and a uniform solution for all sites which is precisely what usability is trying to achieve, uniformity as far as aesthetics go with text sizes, links, margins and spacing. Seven WAVE Problem indicators have been identified in this study as major Accessibility Errors, Contrast Errors with very low contrast being a commonly appearing problem. Alerts with tabs, access key, redundant titles and alternative text being covered. Features problems with linked image with alternative text including form labels, and language. In Structural Elements Problems undordered lists, navigation, headings, footers are some of the areas included and lastly under Accessible Rich Internet Applications numerous label tab index and descriptions of the technologies being able to be comprehensible to persons with disabilities. RESEARCH METHOD This study will utilize an automated usability test tool the WAVE Accessibility will be applied to gather any accessibility issues from egovernment desktop websites. Additionally, this study will employee Nielsen’s 10 Heuristic Principles of Usability in the survey questionnaire which will collect data from a survey that will be composed of quantitative and qualitative based questions. The two strategies of a thoroughly rounded and comprehensive usability test of egovernment websites will include:  WAVE Accessibility test of egovernment websites.  Nielsen Heuristic based questionnaire survey. 3.1 Automated Accessibility Evaluation Tool WAVE People with disabilities require special technical assistance in the form of accessibility. Accessibility is known as rules and tools that web developers or coders use to make a website functional to individuals with temporary or permanent disability. The general rules are outlined by international web content accessibility guidelines WCAG (Yakup & Kemal, 2016). The use of this tool will outline any violations as well as enhance the quality with regards to accessibility of the website. The WCAG meets the World Wide Web Consortium (WAVE ACCESSIBILITY) standards which is the international web industry standard for websites. Several egovernment websites will be tested using the WAVE accessibility evaluation tool. This tool is defined as “a suite of evaluation tools that helps authors make their web content more accessible to individuals with disabilities. WAVE can identify many accessibility and Web Content Accessibility Guideline (WCAG) errors, but also facilitates human evaluation of web content”(Web Content Accessibility Guidelines (WCAG) 2.1, n.d.). Web accessibility have a 7 criteria checklist which are consistency, transformability, multi- modality, focus and structure, readability and language, orientation and error prevention/recovery, and assistive technology compatibility (C. J. Huang, 2003) . Consistency deals with consistent navigation and predictability with results for similar actions taken on the website. Transformability refers to the text and images being scalable as well as the website being comprehensible even when color, images or different styles are disabled. Multi-modality is concerned with providing content under multiple mediums such as video, audio, or close captioned text as well as pairing icons or graphics with text for comprehension of the content including using images to enhance the understanding of the page overall. Focus and structure deals with the overall use of white space to separate elements as well minimizing distractions or background sound as well as ensuring the content is structured in a manner that highlights the features and options in a manner that is easy to use. Readability and language refer to the comprehension of the text including technical aspects https://doi.org/10.21009/JOBBE.005.1.02 24 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 like text size, color, and spacing just to name a few. Orientation and error prevention/recovery has to do with web page time outs, error messages, the ability to undo to redo certain functions just to mention a few. Finally assistive technology compatibility has to do with the technology that will enable a disabled individual to understand form labels, tables, alternative text, etc. 3.2 Nielsen’s Heuristics Based Survey Questionnaire The desktop English survey will test usability of governmental websites translated in English for both problem areas and areas of qualitative data with a cross cultural component can be captured here. Nielsen’s founding principles are outlined in the following 10 hypotheses: Table 1 Descriptions of variables used to measure usability according to the 10 principles Usability Heuristics (Nielsen, 1994) Clarifications H1. Visibility (V) The user must not be confused with extended lag time so the user should be aware of the site’s functionality. H2. Match between system and the real world (MBS) The user should be able to understand the website’s appearance naturally and logically according to modern real world examples. H3. User control and freedom (UCF) Exit, undo, and redo functionality. H4. Consistency and Standards (CS) General real world consistency and layout functions throughout the page. H5. Error prevention (EP) The user should be assisted with errors and the reoccurrence of them. H6. Recognition (R) Functions, choices, and features should be recognizable and ease of access to tools. H7. Flexibility and efficiency of use (FEU) Flexible to different types of users with regards to age or IT competency and customization of common funcions. H8. Aesthetic and minimalist design (AMD) Content must be the most common and current information that is generally required any additional content or details takes valuable space and attention away from the most pertinent information on the site. H9. Help users recognize, diagnose, and recover from errors (HU) Alerts of errors should be clear and solution oriented steps available for the user. H10. Help and documentation (H) Support and help should accessible and highlight the individual’s mission while providing a practical and workable instructions. Additional variable that was inserted to this study (Shareef, Kumar, Kumar, & Dwivedi, 2011) Clarifications H11. Language Options (L) The capacity for English pages to assist the user in using the page in the language of their choice. 3.3 Mixed Method of Accessibility and English Foreign User Usability The overall usability of a website is based on the website’s capacity to be usable by everyone including the disabled and English language users hence the mixed method being used using the WAVE automated test tool and a survey questionnaire. They use drastically different methods that are based on different independent variables. Accessibility based on web accessibility guidelines and web usability based on Nielsen’s 10 heurestic principles. Some major usability factors like a https://doi.org/10.21009/JOBBE.005.1.02 25 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 dysfunctional link would be caught as an error by one of the 5 indicators as well as by one of the 10 principles. This mixed method strategy employed in this study can provide the government with a more complete picture of their website’s usability with respects to English foreign users without neglecting Accessibility across desktop platforms. Making the website usable to users with disabilities contributes to the overall usability of the website since usability by all kinds of users determines the foundation on which a website would be deemed to be usable. The mixed method also combines addressing the needs of foreign English users through the Nielsen Heuristic based questionnaire survey. In summation Accessibility on desktop websites and usability of English language website on a desktop platform overlap in some major areas whist covering the overall performance of websites as well increasing the visibility of the website (Kwangsawad, Jattamart, & Nusawat, 2019). 3.4 Survey Questionnaire Data Analysis from Desktop and Websites Two separate strategies will be adopted here in order to achieve two separate findings objectives:  Firstly we would like to analyze the desktop egovernment website tested under SmartPLS which is a variant based alternative method of Structural Equation Modeling (SEM) method. The Partial Least Square (PLS) analysis technique is done in three phases, namely: Phase 1) is to carry out an Indicator Reliability test (loading value of a factor), which is to test the validity and reliability of the constructs of each indicator. Phase 2) is to test the structural model, accomplished to ensure that the structural model built is robotic and accurate. Phase 3) hypothesis testing is done by looking at the probability value and t- statistics.  Secondly The automated WAVE accessibility tool is the industry standard for Accessibility problems of desktop websites (Adepoju et al., 2016; Ansari et al., 2016; Hung, 2012; Karaim & Inal, 2019; Mtebe & Kondoro, 2017). The use of this tool will outline any violations as well as enhance the quality with regards to accessibility of the website. This tool is defined as “a suite of evaluation tools that helps authors make their web content more accessible to individuals with disabilities. WAVE can identify many accessibility and Web Content Accessibility Guideline (WCAG) errors, but also facilitates human evaluation of web content”(Web Content Accessibility Guidelines (WCAG) 2.1, n.d.). 3.5 Survey Participant Sampling Size and Population Rationale Participants in the survey are English speaking foreigners in compliance with Taiwan’s 2030 Bilingual Initiative. Therefore the survey questionnaire will be conducted by 100 participants for the Ministry of Foreign Affairs, Bureau of Consular Affairs, and Ministry of the Interior National Immigration Agency. RESULT AND DISCUSSION In this section we will discuss the results of websites usability and the website accessibility in the following order. The discussion section will also explore further research and recommendations. Limitations will also be discussed in this section. 4.1 Websites usability results (Ministry of the Interior National Immigration Agency) https://doi.org/10.21009/JOBBE.005.1.02 26 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 The usability results will be have the assessment of the listed websites according to the 10 Heurestic Principles with a numeric value under each variable that corresponds to each principles. An additional Language variable is also listed here to provide an assessment of Language as its own variable. 4.1.1 Indicator Reliability Indicator Reliability can be seen in (Table 2). The value of indicator reliability is the loading value of a factor in the latent variable with indicators. In the model PLS, an indicator can be reliable if it has loading value 0.70 or higher is preferred. If it is an exploratory research, 0.4 or higher is acceptable (Liu, 2012). Indicator Reliability” (see Table 1). It can be seen that all of the indicators have individual indicator reliability values that are much larger than the minimum acceptable level of 0.4 and close to the preferred level of 0.7. Table 2 Loading Factor Value AMD CS EFU EP HD HU L MBS R UCF V AMD1 0.505 AMD3 0.962 AMD4 0.582 CS1 0.897 CS2 0.619 CS4 0.750 EFU2 0.934 EFU3 0.915 EP1 0.791 EP2 0.887 EP3 0.837 EP4 0.852 HD1 0.648 HD2 0.510 HD3 0.916 HU1 0.889 HU2 0.951 HU3 0.803 HU4 0.825 L1 0.921 L2 0.878 L4 0.723 MBS1 0.913 MBS2 0.800 R1 1.000 UCF1 0.811 UCF2 0.846 UCF3 0.614 V1 0.866 V2 0.596 V3 0.573 V5 0.851 4.1.2 Reliability and Validity 4.1.2.1 Reliability Test https://doi.org/10.21009/JOBBE.005.1.02 27 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 The validity of each variable principle is based on the CR test (Composite Reliability). The CR value should be greater than or equal to 0.7 while 0.6 is acceptable for exploratory research ((Belanche, Casaló, & Guinalíu, 2012b; Nasir & Morgan, 2017). (Table 3) reveals the consistency of all the dependent variables as valid and having good reliability. 4.1.2.2 Validity Test Convergent Validity refers to the capacity of every principle to correspond with an explanation of the study principle variables being measured. The validity of each principle variable is determined through the CV (Convergent Validity) test that measured the AVE (Average Variance Extracted) of each principle and an ideal variable value be equal to or greater than 0.5 (Bagozzi & Yi, 1988). (Table 3) shows all the principles values of each variable loading factor greater than 0.5 then each principle is considered valid. Table 3 Reliability and Validity Variables Composite Reliability Average Variance Extracted (AVE) AMD 0.739 0.506 CS 0.804 0.583 EFU 0.922 0.854 EP 0.907 0.710 HD 0.743 0.506 HU 0.925 0.755 L 0.711 0.713 MBS 0.848 0.736 R 1.000 1.000 UCF 0.805 0.583 V 0.819 0.540 4.1.3 Hypothesis Test The value is determined by testing the model form the T Stat value amidst the independent and dependent variables from the Path Coefficient table. The ideal value of 5% with a T-value of 1.96 (Hair, Risher, Sarstedt, & Ringle, 2019). Hence our hypothesis deems the website as usable if p-value is less than or equal to 0.05 or T-value is greater than 1.96. Table coefficient path and T-Test is listed in (Table 4). Based on path coefficient metrics below, obtained T statistical value for all hypotheses, we can conclude that in all 10 Heurestic Principles were a fail and especially including Language as the extra variable measuring at 0.449 on the T table when it should have been 1.96. The overall P Values in all categories were also a fail, with all the variables measuring well above the standard set at 0.05. Therefore they do not meet ETU (easy to use). The urgent improvement of these 10 Heurestic Principles are imperative to achieving usability within egovernment in accordance with the recommendations of (Z. Huang, 2010; Kumar, Mukerji, Butt, & Persaud, 2007) suggest enhancing website design, specifically with regards to navigation, aesthetics, content, accessibility, customization and in all the Nielsen Heurestic Principles in general are extremely likely to motivate users’ adoption of e government. The Nielsen 10 Heurestic Principles are listed here below including this study’s unique contribution of language as the eleventh variable. AMD Aesthetic and Minimalist Design, CS Consistency and Standards, EFU Flexibility and Efficiency of Use, EP Error Prevention, HD Help and Documentation, HU Help Users recognize, diagnose and recover from errors, our unique Language MBS Match between https://doi.org/10.21009/JOBBE.005.1.02 28 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 System and the real world, R Recognition rather than Recall, UCF User Control and Freedom,V Visibility of System Status. Table 4 Path Coefficient Matrix Variables Original Sample (O) Sample Mean (M) Standard Deviation (STDEV) T Statistics (|O/STDEV|) P Values AMD -> ETU 1.216 0.081 2.228 0.546 0.920 CS -> ETU 4.411 -0.275 8.746 0.504 0.955 EP -> ETU 0.640 -0.197 2.988 0.214 0.495 HD -> ETU -2.153 0.304 6.321 0.341 0.988 HU -> ETU 2.491 -0.255 4.216 0.591 0.973 L -> ETU 0.142 0.029 0.316 0.449 0.961 MBS -> ETU 3.792 -0.480 7.063 0.537 0.862 R -> ETU 0.507 0.094 1.327 0.382 0.972 UCF -> ETU -10.397 1.222 19.902 0.522 0.925 V -> ETU -10.397 1.222 19.902 0.522 0.970 4.2 Websites usability results (Bureau of Consular Affairs) 4.2.1 Indicator Reliability Indicator Reliability can be seen in (Table 5). The value of indicator reliability is the loading value of a factor in the latent variable with indicators. In the model PLS, an indicator can be reliable if it has loading value 0.70 or higher is preferred. If it is an exploratory research, 0.4 or higher is acceptable. Indicator Reliability” (see Table 4). It can be seen that all of the indicators have individual indicator reliability values that are much larger than the minimum acceptable level of 0.4 and close to the preferred level of 0.7. Table 5 Loading factor value AMD CS EFU EP HD HU L MBS R UCF1 V1 AMD1 0.984 AMD2 0.927 AMD5 0.872 CS2 0.934 CS5 0.935 EFU1 0.814 EFU4 0.705 EP1 0.943 EP2 0.877 EP4 0.711 HD1 0.979 HD2 0.663 HD3 0.835 HU1 0.782 HU2 0.889 HU3 0.829 HU4 0.887 https://doi.org/10.21009/JOBBE.005.1.02 29 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 L1 0.966 L2 0.961 MBS1 0.614 MBS2 0.582 MBS3 0.899 R2 1.000 UCF1 0.913 UCF2 0.603 V1 0.865 V3 0.688 V5 0.740 4.2.2 Reliability and Validity 4.2.2.1 Reliability Test The validity of each variable principle is based on the CR test (Composite Reliability). The CR value should be greater than or equal to 0.7 while 0.6 is acceptable for exploratory research (Bagozzi & Yi, 1988). (Table 6) reveals the consistency of all the dependent variables as valid and having good reliability. 4.2.2.2 Validity Test Convergent Validity refers to the capacity of every principle to correspond with an explanation of the study principle variables being measured. The validity of each principle variable is determined through the CV (Convergent Validity) test that measured the AVE (Average Variance Extracted) of each principle and an ideal variable value be equal to or greater than 0.5. (Table 6) shows all the principles values of each variable loading factor greater than 0.5 then each principle is considered valid. Table 6 Reliability Test and Validity Test Variables Composite Reliability Average Variance Extracted (AVE) AMD 0.835 0.627 CS 0.803 0.522 EFU 0.714 0.564 EP 0.806 0.511 HD 0.869 0.692 HU 0.930 0.769 L 0.769 0.774 MBS 0.748 0.691 R 0.837 0.719 UCF 0.912 0.838 V 0.931 0.730 4.2.3 Hypothesis Test The value is determined by testing the model form the T Stat value amidst the independent and dependent variables from the Path Coefficient table. The ideal value of 5% with a T-value of 1.96. Hence our hypothesis deems the website as usable if p-value is less than or equal to 0.05 or T- value is greater than 1.96. Table coefficient path and T-Test is listed in (Table 7). Based on path coefficient matriks above, obtained T statistical value for all hypothesis, we can conclude that only UCF User Control and Freedom and V Visibility of System Status from Nielsen’s 10 Heurestic https://doi.org/10.21009/JOBBE.005.1.02 30 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 Principles were a pass. All other 7 Nielsen Heurestic Principles were a fail and especially including Language was a notable fail. The improvement of these Principles and language are urgent in meeting usability standards as discussed in this paper and reaffirmed by numerous studies up and including (Hvannberg, Law, & Lárusdóttir, 2007; Ivory & Chevalier, 2002; Lavery, Cockton, & Atkinson, 1996) that suggest enhancing website design, specifically with regards to AMD, CS, EFU, EP, HD, HU, are extremely likely to motivate users’ adoption of e government. The Nielsen 10 Heurestic Principles are listed here below including this study’s unique contribution of language as the eleventh variable. AMD Aesthetic and Minimalist Design, CS Consistency and Standards, EFU Flexibility and Efficiency of Use, EP Error Prevention, HD Help and Documentation, HU Help Users recognize, diagnose and recover from errors, our unique Language MBS Match between System and the real world, R Recognition rather than Recall, UCF User Control and Freedom,V Visibility of System Status. Table 7 Path Coefficient Matrix Variables Original Sample (O) Sample Mean (M) Standard Deviation (STDEV) T Statistics (|O/STDEV|) P Values AMD -> ETU 0.121 0.131 0.100 1.209 0.227 CS -> ETU 0.147 0.131 0.083 1.762 0.079 EFU -> ETU 0.052 0.048 0.087 0.597 0.551 EP -> ETU 0.082 0.106 0.101 0.807 0.420 HD -> ETU 0.048 0.024 0.097 0.498 0.619 HU -> ETU 0.137 0.105 0.135 1.019 0.309 L -> ETU 0.192 0.219 0.140 1.370 0.171 MBS -> ETU 0.109 0.113 0.070 1.546 0.123 R -> ETU 0.041 0.035 0.056 0.727 0.468 UCF -> ETU 0.157 0.144 0.066 2.371 0.018 V -> ETU 0.168 0.166 0.068 2.470 0.014 4.3 Websites usability results (Ministry of Foreign Affairs) 4.3.1 Indicator Reliability Indicator Reliability can be seen in (Table 8). The value of indicator reliability is the loading value of a factor in the latent variable with indicators. In the model PLS, an indicator can be reliable if it has loading value 0.70 or higher is preferred. If it is an exploratory research, 0.4 or higher is acceptable. Indicator Reliability” (see Table 8). It can be seen that all of the indicators have individual indicator reliability values that are much larger than the minimum acceptable level of 0.4 and close to the preferred level of 0.7. Table 8 Loading factor value AMD CS EFU EP HD HU L MBS R UCF1 V1 AMD1 0.984 AMD2 0.927 AMD5 0.872 CS2 0.934 CS5 0.935 EFU1 0.814 EFU4 0.705 https://doi.org/10.21009/JOBBE.005.1.02 31 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 EP1 0.943 EP2 0.877 EP4 0.711 HD1 0.979 HD2 0.663 HD3 0.835 HU1 0.782 HU2 0.889 HU3 0.829 HU4 0.887 L1 0.966 L2 0.961 MBS1 0.614 MBS2 0.582 MBS3 0.899 R2 1.000 UCF1 0.913 UCF2 0.603 V1 0.818 V3 0.734 V5 0.853 4.3.2 Reliability and Validity 4.3.2.1 Reliability Test The validity of each variable principle is based on the CR test (Composite Reliability). The CR value should be greater than or equal to 0.7 while 0.6 is acceptable for exploratory research (Bagozzi & Yi, 1988). (Table 9) reveals the consistency of all the dependent variables as valid and having good reliability. 4.3.2.2 Validity Test Convergent Validity refers to the capacity of every principle to correspond with an explanation of the study principle variables being measured. The validity of each principle variable is determined through the CV (Convergent Validity) test that measured the AVE (Average Variance Extracted) of each principle and an ideal variable value be equal to or greater than 0.5. (Table 9) shows all the principles values of each variable loading factor greater than 0.5 then each principle is considered valid. Table 9 Reliability Test and Validity Test Variables Composite Reliability Average Variance Extracted (AVE) AMD 0.950 0.863 CS 0.799 0.610 EFU 0.733 0.580 EP 0.845 0.593 HD 0.872 0.699 HU 0.911 0.718 L 0.748 0.876 MBS 0.748 0.508 https://doi.org/10.21009/JOBBE.005.1.02 32 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 R 1.000 1.000 UCF1 0.741 0.599 V1 0.809 0.528 4.3.3 Hypothesis Test The value is determined by testing the model form the T Stat value amidst the independent and dependent variables from the Path Coefficient table. The ideal value of 5% with a T-value of 1.96. Hence our hypothesis deems the website as usable if p-value is less than or equal to 0.05 or T- value is greater than 1.96. Table coefficient path and T-Test is listed in (Table 10). Based on path coefficient metrics above, obtained T statistical value for all hypothesis, we can conclude that we can conclude that in all 10 Heurestic Principles were a fail and especially including Language as the extra variable was a massive fail. The improvement of these pinciples is in line with the recommendations (Kumar et al., 2007; Roy, Pattnaik, & Mall, 2014; Shareef et al., 2011) that suggest enhancing website design, specifically with regards to navigation, aesthetics, content, accessibility, and customization is extremely likely to motivate users’ adoption of e government. The Nielsen 10 Heurestic Principles are listed here below including this study’s unique contribution of language as the eleventh variable. AMD Aesthetic and Minimalist Design, CS Consistency and Standards, EFU Flexibility and Efficiency of Use, EP Error Prevention, HD Help and Documentation, HU Help Users recognize, diagnose and recover from errors, our unique Language MBS Match between System and the real world, R Recognition rather than Recall, UCF User Control and Freedom,V Visibility of System Status. Table 10 Path Coefficient Matrix Variables Original Sample (O) Sample Mean (M) Standard Deviation (STDEV) T Statistics (|O/STDEV|) P Values AMD -> ETU 0.133 0.768 13.373 0.010 0.992 CS -> ETU 0.105 0.146 7.128 0.015 0.988 EFU -> ETU 0.139 -0.635 12.367 0.011 0.991 EP -> ETU 0.016 0.691 20.257 0.001 0.999 HD -> ETU 0.058 -0.542 12.434 0.005 0.996 HU -> ETU 0.156 0.068 11.577 0.013 0.989 L -> ETU 0.009 0.052 4.060 0.002 0.998 MBS -> ETU -0.055 0.506 11.777 0.005 0.996 R -> ETU -0.042 -0.439 12.824 0.003 0.997 UCF1 -> ETU 0.062 0.509 7.691 0.008 0.994 V1 -> ETU 0.572 -0.713 25.283 0.023 0.982 4.4 Websites accessibility results (Ministry of the Interior National immigration Agency) https://doi.org/10.21009/JOBBE.005.1.02 33 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 The accessibility results will be have the assessment of the listed websites according to the internationally recognized guidelines for accessibility. Each problem area will have a numeric value given and it will assessed for accessibility. Recommendations and further research will also be discussed and suggested if necessary. Table 11 Problems Report Problems Contrast Errors Alerts Features Structural Elements ARIA Details (6) Very low contrast (1) Long alternative text (2) No script element (4) Access key (2) Tab index (12) Redundant title text (15) Linked image with alternative text (1) form label (1) language (1) Heading level 1 (2) Heading level 2 (21) Unordered list (1) Header (3) Navigation (1) Footer (1) Aside (10) ARIA (5) ARIA label (3) ARIA Description (7) ARIA Tab index (2) ARIA Hidden Total 6 21 17 30 27 As clearly made visible in (Table 11), the National Immigration Website has revealed a dire and urgently negative standing with Accessibility Indicators in 5 Problem areas having been highlighted above. Some of these Problem indicator include 6 instances of Contrast Errors of very low contrast to 12 alerts under redundant titles to 15 features problems with linked image with alternative text to 21 problems under undordered list of Structural Elements not including several navigation problems and finally under ARIA 10 instances with ARIA just to name a few. There are numerous errors under numerous subcategories of each Problem indicators except Contrast Errors of the Accessibility Checklist that are shown in the above graphic and chart. https://doi.org/10.21009/JOBBE.005.1.02 34 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 4.5 Websites accessibility results (BUREAU OF CONSULAR AFFAIRS) Table 12 Problems Report As the above in (Table 12) graphic illustrates and shows the National Bureau of Consular Affairs in dire and urgent standing with regards to Accessibility as it fails in 6 categories of the study’s checklist of Accessibility. With a major Error to 45 instances of Contrast Errors of very low contrast to 15 alerts under redundant titles and alternative text to 40 Features problems with linked image with alternative text to 13 problems under undordered list with 10 Heading level 2 subcategories of Structural Elements problems not including several navigation problems and finally under ARIA 32 instances of ARIA Hidden just to name a few. There are numerous errors under numerous subcategories of each Problem indicators except for Constrast Errors under the Accessibility Checklist. Problems Error Contrast Errors Alerts Features Structural Elements ARIA Details 1 Error (45) Very low contrast (7) Redundant alternative text (1) Long alternative text (2) No script element (5) Access key (9) Tab index (1) Very small text (8) Redundant title text (40) Linked image with alternative text (1) From label (2) Language (1) Heading level 1 (10) Heading level 2 (13) Unordered list (4) Navigation (1) Search (4) ARIA label (2) ARAI tab index (2) ARAI alert or live region (32) ARIA Hidden Total 1 45 111 43 29 40 https://doi.org/10.21009/JOBBE.005.1.02 35 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 4.6 Websites accessibility results (Ministry of Foreign Affairs) Table 13 Problems Report The Ministry of Foreign Affairs website shows in (Table 13) numerous Problem areas in dire or urgent need of repair under 4 Problem areas. Some of them include 11 instances of Contrast Errors under very low contrast to a staggering 182 indicators under Alerts problem to a massive 87 cases of Linked image with alterative text under Features problem and 69 unordered list problems under structural elements. There are numerous subcategories with errors under each Problems indicator except Contrast Errors. Discussion Following the data from the research conducted using foreign surveys of English egovernment websites as well as Accessibility of egovernment websites listed in this study reveal some urgently troubling outcomes with regards to our overall model of usabilit y. The model of usability contains numerous components like web usability, and accessibility. Although the websites are functional this study has tested the above websites according to the parameters set in this field of study set by academics and researchers cited below in accordance to the industry Problems Contrast Errors Alerts Features Structural Elements Details (11) Very low contrast (1) No page regions (1) Skipped heading level (3) Suspicious link text (4) Access key (182) Redundant title text (87) Linked image with alternative text (2) From label (1) Language (1) Heading level 1 (1) Heading level 2 (26) Heading level 3 (7) Heading level 4 (69) Unordered list Total 11 193 90 104 https://doi.org/10.21009/JOBBE.005.1.02 36 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 standard testing tools for Accessibility and U Usability. Other industries from governmental agencies to the private sectors like online shopping would benefit largely by making use of this study’s findings whilst expanding on them. The specific findings in this study reinforces the need for further studies of egovernment websites testing using both approaches utilized in this study. The foreign usability survey questionnaires of the three tested websites came back with problem areas in under every single Heurestic Principle only the user control freedom and visibility principles were met by the Bureau of Consular Affairs website only. This indicates that there are major usability issues to address in all three egovernment websites and by extension one could postulate that most of the egovernment website if not all require some level of redesign (Z. Huang & Benyoucef, 2014) if not tweaking to meet the usability standards expected by the user that engages and interacts with website. Furthermore, the improvement of these principles in general are likely to motivate users’ adoption of egovernment (Shawgi & Noureldien, 2015; Jimenez, Lozada, & Rosas, 2016; Z. Huang & Benyoucef, 2014). Although the Principles are industry tested and academically established the actual surveys are user centered making the problem areas identified as constituent based. In other words the websites used in this survey were websites that are of particular interest to the demographic the website was designed to service. Therefore if this were in the private sector it could be said it was customer centered problems that were identified in these surveys. However most troubling was the unanimous failed score under the Language variable in this study which makes it exponentially imperative to address these issues immediately since it is a foreign user based survey of English egovernment websites. The WAVE Accessibility tool used on the egovernment websites in this study reveal a troubling trend with regards to meeting international accessibility checklist and unanimously the findings were numerous in at least 4 or more major problem areas with the problems varying in nature from low contrast to linked images with alternative text problems on every website with well over 100 problem indicators on each egovernment website. This is very critical state of standing for Taiwan’s egovernment websites in accordance to the international Accessibility Guidelines. This study in no way asserts that the egovernment websites are dysfunctional for disabled people (Alahmadi & Drew, 2017; Domínguez Vila, Alén González, & Darcy, 2018b; C. J. Huang, 2003) although the Bureau of Consular Affairs did reveal one major error in Accessibility. Nevertheless the discussion of the assessment of the data revealed by this study as it relates to internationally recognized and set standards for Accessibility reveal a very urgent look at Taiwan’s egovernment websites and have further research to test other websites as well as testing the mobile version of these sites. One could postulate that a majority if not all the egovernment websites in Taiwan need urgent Accessibility research. CONCLUSION This study can conclude that the research data reveals that Accessibility and Usability of English egovernment websites need further evaluation. The study revealed numerous reasons for this research and outlined the vulnerability of the demographic that falls under Accessibility, the elderly and the disabled, as well as the general populations’ reliance on the web due to Covid-19. Additionally the case for Usability testing of English e-government websites was clearly made since Taiwan aspires to be fully bilingual and due to the fact that English has become the international language of use by foreigners. Economic factors for both Accessibility requiring users and foreign users were also outlined from cost saving measures to their spending power. The research data only confirmed this study’s fears that Accessibility of some e-government website is not in line with Taiwan’s digital development initiative. The study is also unique in using a foreigner’s perspective to assess Usability of English e-government websites. The research data indicates that the Usability of the three English e-government websites that were tested fell well short of the user-centered Usability. Therefore the findings of this study should be addressed as well as more comprehensive and complete studies to provide a fuller picture of Usability of https://doi.org/10.21009/JOBBE.005.1.02 37 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 egovernment including mobile friendliness. To conclude the state of egovernment in Taiwan according to this study and the websites tested reveal a fail for Usability of foreign user English websites and Accessibility. Future research should address the limitations of this study and address mobile friendliness as well as test accessibility of mobile egovernment websites. REFERENCES Adepoju, S. A., Shehu, I. S., & Bake, P. (2016). Accessibility Evaluation and Performance Analysis of e-Government Websites in Nigeria. Journal of Advances in Information Technology, 7(1), 49–53. https://doi.org/10.12720/jait.7.1.49-53 Alahmadi, T., & Drew, S. (2017). Subjective evaluation of website accessibility and usability: A survey for people with sensory disabilities. Proceedings of the 14th Web for All Conference, W4A 2017. https://doi.org/10.1145/3058555.3058579 Alassaf, R., & Amr, T. (1997). HCI Report : Website Evaluation. 1–11. Retrieved from www.cs.bham.ac.uk/internal/courses/hci/.../HCI - Evaluation.pdf Alsaeedi, A. (2020). Comparing web accessibility evaluation tools and evaluating the accessibility of webpages: Proposed frameworks. Information (Switzerland), 11(1). https://doi.org/10.3390/info11010040 Ansari, R. F., Baqar, A., Hassan, H., & Saeed, F. (2016). Heuristic, accessibility and usability evaluations of Pakistan’s e-government websites. Electronic Government, 12(1), 66–85. https://doi.org/10.1504/EG.2016.074247 Arasid, W., Abdullah, A. G., Wahyudin, D., Abdullah, C. U., Widiaty, I., Zakaria, D., … Juhana, A. (2018). An Analysis of Website Accessibility in Higher Education in Indonesia Based on WCAG 2.0 Guidelines. IOP Conference Series: Materials Science and Engineering, 306(1), 0–8. https://doi.org/10.1088/1757-899X/306/1/012130 Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16(1), 74–94. https://doi.org/10.1007/BF02723327 Bashir, M. S., & Farooq, A. (2019). EUHSA: Extending Usability Heuristics for Smartphone Application. IEEE Access, 7, 100838–100859. https://doi.org/10.1109/access.2019.2923720 Belanche, D., Casaló, L. V., & Guinalíu, M. (2012a). Website usability, consumer satisfaction and the intention to use a website: The moderating effect of perceived risk. Journal of Retailing and Consumer Services, 19(1), 124–132. https://doi.org/10.1016/j.jretconser.2011.11.001 Belanche, D., Casaló, L. V., & Guinalíu, M. (2012b). Website usability, consumer satisfaction and the intention to use a website: The moderating effect of perceived risk. Journal of Retailing and Consumer Services, 19(1), 124–132. https://doi.org/10.1016/j.jretconser.2011.11.001 Campoverde-Molina, M., Lujan-Mora, S., & Garcia, L. V. (2020). Empirical Studies on Web Accessibility of Educational Websites: A Systematic Literature Review. IEEE Access, 8, 91676–91700. https://doi.org/10.1109/ACCESS.2020.2994288 Carroll, T., & Carroll, T. (2015). Local Government Websites in Japan : International , Multicultural , Multilingual ? 1397(October). https://doi.org/10.1080/10371397.2010.518942 Chiew, T. K., & Salim, S. S. (2003). Webuse: Website usability evaluation tool. Malaysian Journal of Computer Science, 16(1), 47–57. Da Silva, A. B. P., Da Silva, C. G., & De Oliveira Moraes, R. L. (2019). On the use of a continuous accessibility assessment process for dealing with website evolution. Multi Conference on Computer Science and Information Systems, MCCSIS 2019 - Proceedings of the International Conferences on Interfaces and Human Computer Interaction 2019, Game and Entertainment Technologies 2019 and Computer Graphics, Visualization, Comp, (March 2020), 35–42. https://doi.org/10.33965/ihci2019_201906l005 De Lima Salgado, A., & Freire, A. P. (2014). Heuristic evaluation of mobile usability: A mapping https://doi.org/10.21009/JOBBE.005.1.02 38 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 study. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8512 LNCS(PART 3), 178–188. https://doi.org/10.1007/978-3-319-07227-2_18 Deakins, E., Dillon, S., & Chen, W. (2007). A Comparison between E-Government Practices in Taiwan and New Zealand. Communications of the ICISA, 8(2), 1–24. Díaz, J., Rusu, C., & Collazos, C. A. (2017). Experimental validation of a set of cultural-oriented usability heuristics: e-Commerce websites evaluation. Computer Standards and Interfaces, 50(May 2016), 160–178. https://doi.org/10.1016/j.csi.2016.09.013 Domínguez Vila, T., Alén González, E., & Darcy, S. (2018a). Website accessibility in the tourism industry: an analysis of official national tourism organization websites around the world. Disability and Rehabilitation, 40(24), 2895–2906. https://doi.org/10.1080/09638288.2017.1362709 Domínguez Vila, T., Alén González, E., & Darcy, S. (2018b). Website accessibility in the tourism industry: an analysis of official national tourism organization websites around the world. Disability and Rehabilitation, 40(24), 2895–2906. https://doi.org/10.1080/09638288.2017.1362709 Dourado, M. A. D., & Canedo, E. D. (2018). Usability heuristics for mobile applications: A systematic review. ICEIS 2018 - Proceedings of the 20th International Conference on Enterprise Information Systems, 2(Iceis 2018), 483–494. https://doi.org/10.5220/0006781404830494 Dror, A. A., Layous, E., Mizrachi, M., Daoud, A., Eisenbach, N., Morozov, N., … Sela, E. (2020). Revealing Global Government Health Website Accessibility Errors During COVID-19 and the Necessity of Digital Equity. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3685998 Federici, S., Micangeli, A., Ruspantini, I., Borgianni, S., Corradi, F., Pasqualotto, E., & Olivetti Belardinelli, M. (2005). Checking an integrated model of web accessibility and usability evaluatin for disabled people. Disability and Rehabilitation, 27(13), 781–790. https://doi.org/10.1080/09638280400014766 Fernandez, A., Insfran, E., & Abrahao, S. (2011). Usability evaluation methods for the web: A systematic mapping study. Information and Software Technology, 53, 789–817. https://doi.org/10.1016/j.infsof.2011.02.007 Fontdevila, D., Genero, M., & Oliveros, A. (2017). Towards a usability model for software development process and practice. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 10611 LNCS(October), 137–145. https://doi.org/10.1007/978-3-319-69926-4_11 Fung, R. H. Y., Chiu, D. K. W., Ko, E. H. T., Ho, K. K. W., & Lo, P. (2016). Heuristic Usability Evaluation of University of Hong Kong Libraries’ Mobile Website. Journal of Academic Librarianship, 42(5), 581–594. https://doi.org/10.1016/j.acalib.2016.06.004 Giroud, A., & Ivarsson, I. (2020). World Investment Report 2020: International production beyond the pandemic. In Journal of International Business Policy (Vol. 3). https://doi.org/10.1057/s42214-020-00078-2 Hair, J. F., Risher, J. J., Sarstedt, M., & Ringle, C. M. (2019). When to use and how to report the results of PLS-SEM. European Business Review, 31(1), 2–24. https://doi.org/10.1108/EBR- 11-2018-0203 Hassouna, M. S., Sahari, N., & Ismail, A. (2017). University website accessibility for totally blind users. Journal of Information and Communication Technology, 16(1), 63–80. https://doi.org/10.32890/jict2017.16.1.8218 Huang, C. J. (2003). Usability of e-government Web-sites for people with disabilities. Proceedings of the 36th Annual Hawaii International Conference on System Sciences, HICSS 2003. https://doi.org/10.1109/HICSS.2003.1174330 Huang, Z. (2010). Usability and credibility evaluation of electronic governments: users’ https://doi.org/10.21009/JOBBE.005.1.02 39 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 perspective. PQDT - UK & Ireland, (December), 1. Retrieved from https://login.pallas2.tcl.sc.edu/login?url=https://search.proquest.com/docview/1033189124? accountid=13965%0Ahttp://resolver.ebscohost.com/openurl?ctx_ver=Z39.88- 2004&ctx_enc=info:ofi/enc:UTF- 8&rfr_id=info:sid/ProQuest+Dissertations+%26+Theses+Global&rft_v Huang, Z., & Benyoucef, M. (2014). Usability and credibility of e-government websites. Government Information Quarterly, 31(4), 584–595. https://doi.org/10.1016/j.giq.2014.07.002 Huang, Z., & Brooks, L. (2012a). Systematically evaluating usability in web-based electronic government: An empirical study. Lecture Notes in Business Information Processing, 101 LNBIP, 133–148. https://doi.org/10.1007/978-3-642-28082-5-10 Huang, Z., & Brooks, L. (2012b). Usability evaluation and redesign of e-government: Users’ centred approach. Lecture Notes in Electrical Engineering, 124 LNEE(VOL. 1), 615–625. https://doi.org/10.1007/978-3-642-25781-0_90 Hung, M. J. (2012). Building Citizen-centred E-government in Taiwan: Problems and Prospects 1. Australian Journal of Public Administration, 71(2), 246–255. https://doi.org/10.1111/j.1467-8500.2012.00764.x Hvannberg, E. T., Law, E. L. C., & Lárusdóttir, M. K. (2007). Heuristic evaluation: Comparing ways of finding and reporting usability problems. Interacting with Computers, 19(2), 225– 240. https://doi.org/10.1016/j.intcom.2006.10.001 Introduction to Understanding WCAG 2.1. (n.d.). Retrieved February 2, 2021, from https://www.w3.org/WAI/WCAG21/Understanding/intro#understanding-the-four- principles-of-accessibility. Ivory, M. Y., & Chevalier, A. (2002). A study of automated web site evaluation tools. University of Washington, Department of Computer …, 1–14. Ji, Y. G., Park, J. H., Lee, C., & Yun, M. H. (2006). A usability checklist for the usability evaluation of mobile phone user interface. International Journal of Human-Computer Interaction, 20(3), 207–231. https://doi.org/10.1207/s15327590ijhc2003_3 Karaim, N. A., & Inal, Y. (2019). Usability and accessibility evaluation of Libyan government websites. Universal Access in the Information Society, 18(1), 207–216. https://doi.org/10.1007/s10209-017-0575-3 Khajouei, R., Zahiri Esfahani, M., & Jahani, Y. (2017). Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems. Journal of the American Medical Informatics Association, 24(e1), e55–e60. https://doi.org/10.1093/jamia/ocw100 King, B. A., & Youngblood, N. E. (2016). E-government in Alabama: An analysis of county voting and election website content, usability, accessibility, and mobile readiness. Government Information Quarterly, 33(4), 715–726. https://doi.org/10.1016/j.giq.2016.09.001 Król, K., & Zdonek, D. (2020). Local Government Website Accessibility—Evidence from Poland. Administrative Sciences, 10(2), 22. https://doi.org/10.3390/admsci10020022 Kumar, V., Mukerji, B., Butt, I., & Persaud, A. (2007). Factors for Successful e-Government Adoption : a Conceptual Framework. The Electronic Journal of E- Government, 5(1), 63–76. Kuzma, J. M. (2010). Accessibility design issues with UK e-government sites. Government Information Quarterly, 27(2), 141–146. https://doi.org/10.1016/j.giq.2009.10.004 Kwangsawad, A., Jattamart, A., & Nusawat, P. (2019). The Performance Evaluation of a Website using Automated Evaluation Tools. TIMES-ICON 2019 - 2019 4th Technology Innovation Management and Engineering Science International Conference, 1–5. https://doi.org/10.1109/TIMES-iCON47539.2019.9024634 Lavery, D., Cockton, G., & Atkinson, M. (1996). Usability Evaluation Materials. Evaluation. Leist, E., & Smith, D. (2014). Accessibility issues in e-government. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in https://doi.org/10.21009/JOBBE.005.1.02 40 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 Bioinformatics), 8650 LNCS, 15–25. https://doi.org/10.1007/978-3-319-10178-1_2 Lessa, L., & Tsegaye, A. (2019). Evaluation of the public value of e-government services in Ethiopia: Case of court case management system. ACM International Conference Proceeding Series, Part F1481(June), 21–26. https://doi.org/10.1145/3326365.3326369 Liu, Y. X. (2012). A new antihypertensive drug ameliorate insulin resistance. Acta Pharmacologica Sinica, 33(4), 429–430. https://doi.org/10.1038/aps.2012.31 Lyzara, R., Purwandari, B., Zulfikar, M. F., Santoso, H. B., & Solichah, I. (2019). E-Government Usability Evaluation: Insights from A Systematic Literature Review. ACM International Conference Proceeding Series, 249–253. https://doi.org/10.1145/3305160.3305178 Määttä, T. (2007). Human motion simulation for vehicle and workplace design. Human Factors and Ergonomics in Manufacturing, 17(5), 435–443. https://doi.org/10.1002/hfm Mahajan, S., Abolhassani, N., McMinn, P., & Halfond, W. G. J. (2018). Automated repair of mobile friendly problems in web pages. 140–150. https://doi.org/10.1145/3180155.3180262 Majrashi, K., & Hamilton, M. (2015). A Cross-Platform Usability Measurement Model. Lecture Notes on Software Engineering, 3(2), 132–144. https://doi.org/10.7763/lnse.2015.v3.179 Máñez-Carvajal, C., Cervera-Mérida, J. F., & Fernández-Piqueras, R. (2019). Web accessibility evaluation of top-ranking university Web sites in Spain, Chile and Mexico. Universal Access in the Information Society, (0123456789), 1–6. https://doi.org/10.1007/s10209-019-00702-w Mtebe, J., & Kondoro, A. (2017). Accessibility and Usability of Government Websites in Tanzania. The African Journal of Information Systems, 9(4), 3. Mukhoryanova, O. A., Novikova, I. V., Rudich, S. B., & Bogushevich, E. V. (2016). E- government in the western European countries, Asia and in the USA. Indian Journal of Science and Technology, 9(16), 1–13. https://doi.org/10.17485/ijst/2016/v9i16/90757 Mulvey, C. (2008). Web Accessibility of eGovernment websites. (September). Nasir, M. A., & Morgan, J. (2017). Article information : Nielsen, J. (1994). Usability inspection methods. Conference Companion on Human Factors in Computing Systems - CHI ’94, 25(1), 413–414. https://doi.org/10.1145/259963.260531 Nuñez, A., Moquillaza, A., & Paz, F. (2019). Web Accessibility Evaluation Methods: A Systematic Review. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11586 LNCS, 226–237. https://doi.org/10.1007/978-3-030-23535-2_17 Paul, S., & Das, S. (2020). Accessibility and usability analysis of Indian e-government websites. Universal Access in the Information Society, 19(4), 949–957. https://doi.org/10.1007/s10209- 019-00704-8 Roy, S., Pattnaik, P. K., & Mall, R. (2014). A quantitative approach to evaluate usability of academic websites based on human perception. Egyptian Informatics Journal, 15(3), 159– 167. https://doi.org/10.1016/j.eij.2014.08.002 Sá, F., Rocha, Á., & Pérez Cota, M. (2016). From the quality of traditional services to the quality of local e-Government online services: A literature review. Government Information Quarterly, 33(1), 149–160. https://doi.org/10.1016/j.giq.2015.07.004 Sauro, J. (2016). Measuring the Quality of the Website User Experience. ProQuest Dissertations and Theses, 148. Retrieved from https://login.pallas2.tcl.sc.edu/login?url=https://search.proquest.com/docview/1802057542? accountid=13965%0Ahttp://resolver.ebscohost.com/openurl?ctx_ver=Z39.88- 2004&ctx_enc=info:ofi/enc:UTF- 8&rfr_id=info:sid/ProQuest+Dissertations+%26+Theses+Global&rft_v Seffah, A., & Padda, H. K. (2006). Usability Measurement : A Roadmap for a Consolidated Model. Software Quality Journal, 14(2), 159 – 178. Shareef, M. A., Kumar, V., Kumar, U., & Dwivedi, Y. K. (2011). E-Government Adoption Model (GAM): Differing service maturity levels. Government Information Quarterly, 28(1), 17–35. https://doi.org/10.1016/j.giq.2010.05.006 https://doi.org/10.21009/JOBBE.005.1.02 41 | P a g e Chang, C.H., Almaghalsah, H. Journal of Business and Behavioural Entrepreneurship, 5(1), 16-41 Taiwan’s E-Government Usability from Foreign Perspective and Website Accessibility amid COVID-19 Pandemic. https://doi.org/10.21009/JOBBE.005.1.02 Shawar, B. A. (2015). Evaluating web accessibility of educational websites. International Journal of Emerging Technologies in Learning, 10(4), 4–10. https://doi.org/10.3991/ijet.v10i4.4518 Shawgi, E., & Noureldien, N. A. (2015). Usability Measurement Model (UMM ): 5(1), 5–13. https://doi.org/10.5923/j.ijis.20150501.02 Taiwan’s 2030 goal to become a bilingual nation(2019-01-11)- Financial Supervisory Commission. (n.d.). Retrieved November 28, 2020, from https://www.fsc.gov.tw/en/home.jsp?id=253&parentpath=0,4,212,252#:~:text=In December of 2018%2C the,the country’s overall national competitiveness Taiwan’s success in curbing COVID-19 tops domestic CNA news of 2020 - Focus Taiwan. (n.d.). Retrieved January 30, 2021, from https://focustaiwan.tw/society/202012240005 Toshio, P. (2018). THE 14TH WASEDA – IAC INTERNATIONAL DIGITAL GOVERNMENT RANKINGS 2018 REPORT. (October). Verkijika, S. F., & De Wet, L. (2018). A usability assessment of e-government websites in Sub- Saharan Africa. International Journal of Information Management, 39(September 2017), 20– 29. https://doi.org/10.1016/j.ijinfomgt.2017.11.003 Wang, J., & Senecal, S. (2007). Measuring Perceived Website Usability. Journal of Internet Commerce, 6(4), 97–112. https://doi.org/10.1080/15332860802086318 Web Content Accessibility Guidelines (WCAG) 2.1. (n.d.). Retrieved from https://www.w3.org/TR/WCAG21/ Weerakkody, V., Irani, Z., Kapoor, K., Sivarajah, U., & Dwivedi, Y. K. (2017). Open data and its usability: an empirical view from the Citizen’s perspective. Information Systems Frontiers, 19(2), 285–300. https://doi.org/10.1007/s10796-016-9679-1 Wirtz, B. W., & Kurtz, O. T. (2016). Local e-government and user satisfaction with city portals – the citizens’ service preference perspective. International Review on Public and Nonprofit Marketing, 13(3), 265–287. https://doi.org/10.1007/s12208-015-0149-0 Zhang, N., Meng, Q., Guo, X., Yin, C., & Luo, H. (2015). Key e-government issues in China: an empirical study based on the orientation-maturity framework. Electronic Commerce Research, 15(3), 407–425. https://doi.org/10.1007/s10660-015-9190-7 https://doi.org/10.21009/JOBBE.005.1.02