INT J COMPUT COMMUN, ISSN 1841-9836 9(1):37-47, February, 2014. Development of an Agent Based Specialized Multi-Lingual Web Browser for Visually Handicapped R. Ponnusamy, M.S.S. Babu, T. Chitralekha R. Ponnusamy*, M.S. Satish Babu Dept of Computer Science & Engineering, Madha Engineering College, Chennai - 600 069, India, *Corresponding author: rponnusamy@acm.org T. Chitralekha Dept of Computer Science, Pondicherry University, Pondicherry - 605 014, India Abstract: In the modern age everyone needs access to Internet; Visually handicapped are not an exception for that. SPECS (SPEcialized Computer System) is a system developed to give access to the visually handicapped. It has a Braille shell. The user can enter his spoken language. After the selection of the language the user can present his request to the browser through chosen language in Braille. The output generated by the browser is given out as voice message. The effectiveness of this system is measured based on number of requests processed, access speed and precision of the system. Keywords: Visually handicapped, Braille shell, Internet, Multi-Lingual Web Browser. 1 Introduction The impact of visual loss has profound implications for the person affected. The majority of blind people live in developing countries. The number is huge, also due to the sheer size of the population in developing countries. Especially in south and East Asia itself there exist 27% (1,590.80 Millions, 2002 Estimate) [1] [2] [3] of blindness. The first global estimate on the magnitude and causes of visual impairment was based on the 1990 world population data (38 million blind). This estimate was later extrapolated to 1996 world population as 45 million blind, and projected to 2020 world population as 76 million blind, indicating a twofold increase in the magnitude of visual impairment in the world by 2020 [4]. Further, the survey estimated that 3.9% comes under child blindness and incurable categories. In the past decade, the Internet revolution throughout the computing world, catalyzed largely by the World Wide Web (WWW) [5], has enabled the widespread dissemination of information worldwide. However, much of this information is in English or in languages of Western origin. Presently, the Internet is positioned to be an international mechanism for communications and information exchange, the precursor of a global information superhighway. For this vision to be realized, one important requirement is to enable all languages to be technically transmissible via the Internet, so that when a particular society is ready to absorb Internet technology, the language capability comes prepackaged. The term "Multilingual Computing" refers to the use of computer applications in Indian languages. Traditionally, computer applications were based on English as the medium of interaction with the system. In India, when one attempts to use computers for education and literacy, one faces the problem of language where majority of the population that should get the benefit of Information Technology, does not speak English. This is a non-trivial multilingual information-processing problem. There is an urgent need to recognize that the true burden of blindness has changed with the rapid pace of industrialization and technology, and must adopt these people for development. Copyright © 2006-2014 by CCC Publications 38 R. Ponnusamy, M.S.S. Babu, T. Chitralekha Most legally blind people (70% of them across all ages, according to the Seattle Lighthouse for the Blind) do not use computers. Only small fractions of this population, when compared to the sighted community, have Internet access. There is urgent need to develop Information Technology Tools which can be used by blind people knowing only their own languages. Especially the regional visually handicapped people need special methods and tools for accessing the web. It is the responsibility of the technocrats to develop such technology. Today’s research challenge is to give an optimal access to the computers and internet for the visually handicapped people in different languages. In this paper an attempt has been made to design and develop a special system for visually handicapped people. A specialized browser using role based agents for regional visually handi- capped users. Section 2 explains the working nature of Specialized Browser. Section 3 explains the components and architecture of the Specialized Browser. Section 4 explains the design and functionality of Tamil and English Braille keyboard. Section 5 explains the SPECS Machine Learning System. Section 6 explains the simulation experiment. Section 7 gives experiment results and discussion and Section 8 Concludes the paper. 2 Literature Survey Stuart Goose, 2000 [18] presenting a new idea of creation of a hypertext document in both the visual and auditory realms. In this approach an intelligent agent that is able to convert HTML contents to VXML contents to provide voice services for text disabilities via web. Prior to interpreting HTML documents and separating contents, the contents for the conversion must be selected, however, there are no good solutions for selecting the desired group contents. If an audio document is designed straight from the author’s intentions, it may correspond to the author making an explicit recording the user study presenting voice based html structure in audio: user satisfaction with audio hypertext of the document or pieces of the document. Patrick Roth [19] and his group project aims at providing sight handicapped people with alternative access modalities to pictorial documents. More precisely, our goal is to develop an augmented Internet browser to facilitate blind users access to the World Wide Web. The main distinguishing characteristics of this browser are 1.Generation of a virtual sound space into which the screen information is mapped; 2.Transcription into sounds not only of text, but also of images; 3. Active user interaction, both for the macro-analysis and micro-analysis of screen objects of interest; 4.Use of a touch-sensitive screen to facilitate user interaction. Several prototypes have been implemented, and are being evaluated by blind users. Guan neng huang,2007 [20], designed a special web browser called eguidedog is designed for the visually impaired people. this web browser can extract the structure and the content of an html document and represent it in the form of audio. it helps the blind finding out information they concern more quickly. IBM Accessibility browser [21] provide an additional facility to access audio while enjoying a streaming video, visually impaired people can now select the play button by simply pressing a predefined shortcut key instead of searching in the content for buttons that control the video. Users can also adjust the volume of an individual source in order to identify and listen to different sound sources without losing track of the screen-reading software because of the sound of a video. The main problems with these work is accessing the multi- lingual content. Tim Morris [22] and his team have reported on a prototype screen reader that is intended to vocalize the information displayed on the LCD or LED screen of home or office equipment. Tadayoshi Fujiki, 2006 [23], Developed a new tool easy bar to cater the needs of the visually handicapped users. The functions of the Easy Bar are to change the size of web texts and images, to adjust the color, and to clear cached data that is automatically saved by the web browser. Development of an Agent Based Specialized Multi-Lingual Web Browser for Visually Handicapped 39 3 About SPECS The SPECS [6] [7] is the specialized browser for Visually Handicapped Users (VHU). This system allows the VHU to browse the restricted set of regional language and English WebPages. It receives the inputs through Braille/Normal Keyboard and gives the voice output in respective Language of choice. In order to browse the webpage the user must know the specific website in advance. Also, the trainer must train system by giving English website address and its equivalent regional language website through the Braille keyboard attached with the system. The system is able to read only the static websites. Further the user is not able to travel into the complete set of hyperlinks provided in the site and the present browser functionality is restricted to access the text messages alone. The trainer is not Visually Handicapped. The Braille keyboard is designed to work in the two modes. One is the normal mode and another is command mode. The Visually Handicapped Users (VHU) can travel in the website through the command mode. The VHU can type the website address in normal mode of operation and in case of command mode the user can move from the hyperlink to anther hyperlink. This is the main difference between the previous work and the present work. Another main restriction in the present work is that the VHU cannot access the .gif, .bmp, .jpg fixed text contents. This is great challenge to the user. Also the user cannot feel the pictures that appear on the webpage. At present the SPECS is designed to accommodate only two languages. In future the complete design of SPECS is expected to incorporate all Indian languages. 4 Architecture and Component Functionality of SPECS The overall architecture of SPECS is shown in the following figure 1. This architecture of SPECS consists of three layers. These are 1. SPECS Browser Layer 2. Multi-functional Agent Layer 3. Knowledge Base Layer. Further, the SPECS System Interface (IOCS) is built under Windows platform. The Windows API, informally WinAPI, is Microsoft’s core set of application programming interfaces (APIs) available in the Microsoft Windows operating systems. Developer support is available in the form of the Microsoft Windows SDK, providing facilities and tools necessary to build software based upon the Windows API and associated Windows technologies. Various wrappers were developed by Microsoft that took over some of the more low level functions of the Windows API, and allowed applications to interact with the API in a more abstract manner. Microsoft Foundation Class Library (MFC) wrapped Windows API functionality in C++ classes, and thus allows a more object oriented way of interacting with the API. The Active Template Library (ATL) is a template oriented wrapper for COM. The Windows Template Library (WTL) was developed as an extension to ATL, and intended as a lightweight alternative to MFC. Over and above such facilities the SPECS IOCS has been developed to cater the needs. The layered representation of this specialized browser is shown in the following Figure 2. 4.1 Brower Layer The SPECS Browser is the general browser capable of browsing in regional language and in English language. Font availability is the big problem of this layer. This problem is solved through the Machine Learning System. The SPECS browser is designed using the Internet Explorer Active X component [13]. ActiveX is a framework for defining reusable software com- ponents in a programming language independently. Internet Explorer 7 was used to design this system. Software applications can then be composed from one or more of these components in order to provide their functionality. Active X controls (small program building blocks), can 40 R. Ponnusamy, M.S.S. Babu, T. Chitralekha Figure 1: SPECS Architecture, LLAA – Language Learning Adaptation Agent, DA – Dialogue Agent, MHA – Message Handling Agent Figure 2: Layered Representation of Specialized Browser serve to create distributed applications working over the Internet through web browsers. ActiveX controls can then be embedded into other applications. Internet Explorer also allows embedding ActiveX controls onto web pages. Windows Internet Explorer [14] (formerly Microsoft Internet Explorer; commonly abbreviated to IE), is a series of graphical web browsers developed by Microsoft and included as part of the Microsoft Windows line of operating systems. Since its first release, Microsoft has added features and technologies such as basic table display (in version 1.5); XMLHttp Request (in version 5), which aids creation of dynamic web pages; and Internationalized Domain Names (in version 7), which allow Web sites to have native-language addresses with non-Latin characters. Internet Explorer has introduced an array of proprietary extensions to many of the standards, including HTML, CSS, and the DOM. This has resulted in a number of web pages that appear broken in standards-compliant web browsers and has introduced the need for "quirks mode" rendering improper elements meant for Internet Explorer in such browsers. Considering these advantages the IE 7 has been chosen as the suitable component for designing the SPECS browser. 4.2 Multi-Function Agents Layer This layer performs different functions such as language learning, VHU interaction, error messaging, voicing and VHU direction. The study of multi-agent systems (MAS) [16] focuses on systems in which many intelligent agents interact with each other. The agents are considered to be autonomous entities, such as software programs or robots. Their interactions can be either cooperative or selfish. That is, the agents can share a common goal (e.g. an ant colony), or they can pursue their own interests (as in the free market economy). The characteristics [16] of MASs are that (1) each agent has incomplete information or capabilities for solving the problem and, thus, has a limited viewpoint; (2) there is no system global control; (3) data are decentralized; and (4) computation is asynchronous. In the present design the system functions are asynchronous and hence the MAS architecture Development of an Agent Based Specialized Multi-Lingual Web Browser for Visually Handicapped 41 is chosen as the best fit architecture. This layer has different agents to perform all these function- alities, such as Language Learning Adaptation Agent, Dialogue Agent, Message Handling Agent, Prompter Agent and Director Agent. These agents are operating independently performing the operations. Language Learning Adaptation Agent is a simple component. It scans the user language selection. Normally, it permits two types of users as of now. One is the normal user and second user is the VHU users. It understands the user and displays different screen for different users. In the Braille key board the system is designed to operate in a command mode. This command mode first supports the language selection. The system is able to understand the language selection and change the system setup accordingly. Message Handling Agent displays error and other messages from the knowledge base in regional language or prompts the display in English for other user. An error message alerts users of a problem that has already occurred. Error messages can be presented using modal dialog boxes, in-place messages, and notifications. Dialogue Agent gets the input from the visually handicapped user in particular language Braille through special input keyboard attached with the SPECS system. On the other hand it also gets the normal input form the trainer. The design of the Braille keyboard [11] [12] [15] and its components are explained in the Section 4. Prompter Agent gives the voice output after filtering the output from the browser. This prompter gets the sequence of string from the browser in the form of HTML/DHTML and checks the FONTFACE tag, if it is trained language tag, then it stores the sequence of text in file until it encounters FONT tag, otherwise it simply truncates those tags and HTML input. Then the other tags are given to the sound component. The sound component is able to read the given regional language/English words. A system is designed to read the words in regional language and in English. A girl voicer recorded regional language/English alphabets with different rhythm based on their occurrence in the word at different places. The occurrence may be at the beginning or at the middle or at the end. Then the sound component is designed to pronounce the word with different voice synthesizing. Director Agent directs the browser to browse the regional language/English web sites accord- ing to the selection. Even if the user typed words with small mistakes it is able to direct to the correct website. 4.3 Knowledge Base Layer The third layer consists of Knowledge base which stores different types of fonts, Regional Language Voice Database, Various Regional Language web site information, etc. The main mechanism of this knowledge base development is knowledge representation, acquisition, learning and reasoning. The section 5 explains how this knowledge is acquired, represented, learning and reasoning. 4.4 Mutli-Lingual Font Repository This repository stores all the fonts available in each web site and the common regional language fonts of different designers. As soon as the new website is visited it will fetch the new font from that website. 4.5 Regional Language Voice Database A Prerecorded UNICODE character set is stored in the database. The UNICODE character set is available in the http://unicode.org/. This database provides the equivalent voice file for the particular character as soon as it is requested. These voices are recorded by the voicer in 42 R. Ponnusamy, M.S.S. Babu, T. Chitralekha a recording room with three different types of appearance of letters in different places. That is first type of tag is at the beginning of letters, second one is for the middle letters and third for the end letters. The voice chord in Regional Language/English will differ when it appear in different places. 4.6 Multi-Lingual Web Site Database This database contains the multi-lingual web name and the equivalent English web site name. These websites are taught by the normal person during the training phase of the system. Even through the system is designed for the people; a non-blind person can also use the system in a normal way. His / her duty is to teach the equivalent multi-lingual web site name for every known multi-lingual portal. and third for the end letters. The voice chord in Regional Language/English will differ when it appear in different places. 5 Design and Functionality of Multi-Lingual Braille Keyboard Braille is a touch and feel system for the Visually Handicapped person, which uses an ar- rangement of 6 dots called a cell. The cell is three dots in height and two dots wide. Each Braille character is formed by placing one or more dots in specific positions. A printed sheet of Braille normally contains upwards of twenty five rows of text with forty cells in each row. The physical dimensions of a standard Braille sheet are approximately 11 inches by 11 inches. The dimensions of the Braille cell are also standardized but these may vary slightly depending on the country. The dimension of a Braille cell, as printed on an embosser is shown below. Figure 3: Braille Cell Dimensions A sheet of Braille may thus appear to hold information amounting to about a thousand characters (letters of the alphabet). Later we will see that the designers of the Braille system had foreseen the need to present information in compact form so that a set of cells could convey much more information in the string of letters forming the cells. Try reading the following sentence shown in Grade-1 Braille. To aid in describing these characters the positions in the Braille cell are numbered 1, 2, 3 downward and on the left, and 4, 5, 6 downward on the right. It is shown in the following Figure 3. The Braille Keyboard has six keys, a line spacer, a back spacer and a space bar. The six keys correspond to his six dots of the Braille cell. The keys are struck one or more at a time so that one Braille cell is written with each stroke. There are three keys each side of the space bar. The Development of an Agent Based Specialized Multi-Lingual Web Browser for Visually Handicapped 43 Figure 4: Braille Key Board Design left index finger uses the key to the left of the space bar, which strikes dots1; the middle finger, dot 2; and the left ring finger, dot 3. The right finger, middle finger and the ring index finger strike the keys for dots 4, 5 and 6 respectively. Thumb strike on the space bar leaves a blank cell. The Bharathi Braille Tamil fonts and Grade-1 Braille are used to key in the fonts in to the SPECS system. The Bharathi Braille fonts and Grade-1 Braille are shown in the following Figures 5 and 6 respectively. Figure 5: Bharathi Braille fonts Figure 6: Bharathi Braille fonts 44 R. Ponnusamy, M.S.S. Babu, T. Chitralekha 6 SPECS Machine Learning System The SPECS system uses three different learning systems for usage. First learning mechanism applied is the simple rote learning mechanism to understand the English URLs and their equal Multi-lingual Braille names. The trainer must explicitly train the system to understand the equivalent URLs. The idea is that one will be able to quickly recall the meaning of the URL by repeating it often. The second learning is the reinforcement learning mechanism that must understand the occurrences of an alphabet in different words in different places and sequence the sound files according to their requirement. It can be used in cases where there is a sequence of inputs and the desired output is only known after the specific sequence occurs. Reinforcement learning is concerned with how an agent ought to take actions in an environment so as to max- imize some notion of cumulative reward. In machine learning [17], the environment is typically formulated as a Markov decision process (MDP), and many reinforcement learning algorithms for this context are highly related to dynamic programming techniques. The main difference to these classical techniques is that reinforcement learning algorithms do not need the knowledge of the MDP and they target large MDPs where exact methods become infeasible. This process of identifying the relationship between a series of input values and a later output is temporal difference learning. This algorithm is adapted to understand the word and then present the voice to the VHU users. The third learning is the font usage for the different system and again it is a rote leaning mechanism. As soon as the agent finds new fonts it downloads those fonts and stores them in the local database.If you wish to include color illustrations in the electronic version in place of or in addition to any black and white illustrations in the printed version, please provide the managing editor and the editorial assistant with the appropriate files. 7 Simulation Experiment This system is developed using Microsoft Visual C++ under Microsoft Windows 9x platform. The special browser developed can be facilitating the normal user to give the inputs and train the system. All the given components and agents are developed and integrated and installed with the Braille keyboard and a speaker. The system is installed with the normal Hardware; 800 MHz Intel Pentium 3, 256 MB RAM machine and tested. After the installation the normal users trained the system by visiting different Tamil web sites. The fonts from these websites also downloaded and put into the repository. The Language Learning Adaptation Agent takes care of installation of fonts in the respective system font’s directory as soon as it is downloaded. The SPECS browser developed and sample screen is shown in the following Figure 7. Figure 7: A view of SPECS browser Development of an Agent Based Specialized Multi-Lingual Web Browser for Visually Handicapped 45 User type VHU Non-VHU Mean-Time to Access Time (s) 800 200 Table 1: Comparison of Navigation Time of VHU Vs Non-VHU 8 Experiment Result and Discussion In these experiments, the system is trained with 600 web sites by the normal non-blind person. Then four blind persons were brought from the blind school and they were asked to browse in the system using the specialized browser. After getting their opinion the system performance is evaluated in two different ways. In order to evaluate the effectiveness of this browsing [24], system is measured with the well known precision measure used for different query in both normal browser and special browser and is shown by Precision = Total Number of Documents Retrieved /Total Number of Documents Trained The Precision is the probability that a (randomly selected) retrieved document is relevant. Based on the measurement pertaining to precision is compared for both the blind user and with the non-blind users. It is found that the graph with precision taken in both experiments justifying equality in retrieval effectiveness. To evaluate the efficiency of access system, it is essential to find the Mean Time to Access [8] [9] [10] the browser by both the VHU and Non-VHU. This result is compared and is presented in the Table 1. It is found that the VHU takes four times of the access time compared with the Non-VHU-Figure 8. Figure 8: Comparison of Precision for Both VHU and Non-VHU 9 Conclusion In the present work a special browser has been designed and developed for visually hand- icapped persons. The mean-time to access is taken into account for the VHU as well as for Non-VHU (Internet Explorer 7.0 for Non-blind person) and the comparative results are pre- sented. In addition to this the precision has been measured under different situations. This browser is very much helpful visually handicapped persons of regional language to access the Internet without any difficulties. The system is able to read only the static websites. The user is not able to travel into complete set of hyperlinks provided in the site and the present browser functionality is restricted to access the text messages alone. Further work is necessary to take 46 R. Ponnusamy, M.S.S. Babu, T. Chitralekha care of these problems of this system. Bibliography [1] Serge Resnikoff et al, Global magnitude of visual impairment caused by uncorrected refractive errors in 2004, Bulletin of the World Health Organization, 86(3):63-70, Jan 2008. [2] Murthy GV, Gupta SK, Bachani D, Jose R, John N.; Current estimates of blindness in India, Br J Ophthalmol 89:257-60, PMID: 15722298 doi:10.1136/bjo.2004.056937, 2005. [3] Serge Resnikoff et al., Policy and Practice, Global data on visual impairment in the year 2002, Bulletin of the World Health Organization, 82(11):844-852, Nov 2004. [4] Magnitude and Causes of Visual Impairment, Fact Sheet May 2009, World Health Organiza- tion, http://www.who.int/mediacentre /factsheets/ fs282/en/ [5] Leong Kok Yong et al., Multiple Language Support over the World Wide Web, World Wide Web Journal, 2: 165-176, 1997. [6] S. Kuppswami et al, SPECS: Friendly Computer System for the Visually Handicapped – A Proposal, Proc. of the National Conference on Creating Convenient and Friendly Envi- ronment for Education and Training of the handicapped in Technical Institutions, December 1999, Roorkee University. [7] S.Kuppuswami, V.Prasanna Venkatesan, T.Chithralekha, Role Based Agents for Internet Access Through SPECS , CSI Conference, Chennai, Sept 2000. [8] Browser Speed Comparisons, http://www.howtocreate.co.uk/ browserSpeed .html [9] Tenni Theurer, Performance Research, Part 2: Browser Cache Usage – Exposed: A web Blog, http://yuiblog.com /blog /2007/01/04/performance-research-part-2/, 2007. [10] Michael Czeiszperger, Evaluating Apple’s Browser Performance Claims in The Real World, http://www.webperformanceinc.com/library/reports/Safari % 20 Benchmarks/ [11] Bharathi Braille Fonts, http;//acharya.iitm.ac.in/ [12] Standard Grade-1 Braille Fonts, http;//acharya.iitm.ac.in/ [13] R. Ponnusamy; T. Chithralekha; Prasanna Venkatesan; S. Kuppuswami, Development of an Agent Based Specialised Web Browser for Visually Handicapped Tamils, Lecture Notes in Computer Science, Springer-Verlag LNCS 5616, ISSN 0302-9743, Universal Access in HCI, Part III, HCII2009, 778-786, 2009. [14] Windows XP Technical Manual, Microsoft Corporation. [15] http://acharya.iitm.ac.in/sdi.html [16] http://www.aaai.org/AITopics/pmwiki/pmwiki.php /AITopics /MultiAgentSystems [17] Sio-long Ao et. al., Machine Learning and Systems Engineering, Lecture Notes in Electrical Engineering, Springer, 1st Edition, 2010. Development of an Agent Based Specialized Multi-Lingual Web Browser for Visually Handicapped 47 [18] Stuart Goose , Mike Newman , Claus Schmidt , Laurent Hue, Enhancing Web accessibil- ity via the Vox Portal and a Web-Hosted Dynamic HTML-VoxML Converter, Proc. of the 9th int. World Wide Web conference on Computer networks : the int. J. of Computer and Telecommunications Networking, 583-592, June 2000. [19] Patrick Roth et al., AB – Web : Active audio browser for visually impaired and blind users, Int. Conference on Auditory Display, University of Glasgow, UK, November 1-4, 1998. [20] Jing Xiao, GuanNeng Huang,Yong Tang, An Open Source Web Browser for Visually Im- paired, Third Int. Conference on Intelligent Computing, ICIC 2007, LNCS, Vol. 4681: 90-101, 2007. [21] IBM Accessibility Internet Browser for Multimedia, A tool that enables multimedia con- tent on the Internet to be enjoyed by people with visual impairments, September 27, 2007, http://www.alphaworks.ibm.com /tech/aibrowser [22] Tim Morris et. al., Clearspeech: A Display Reader for the Visually Handicapped, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 14(4):492-500 , Dec 2006. [23] Tadayoshi Fujiki et. al., A Tool for Improving the Web Accessibility of Visually Handicapped Persons, J Med Syst, 30: 83–89, 2006. [24] Barbara Leporini and Fabio Patern, Applying Web Usability Criteria for Vision-Impaired Users: Does ItReally Improve Task Performance?, Intl. Journal of Human–Computer Inter- action, 24:17–47, 2008.