PAPER ACCESSIBLE INTERNET-OF-THINGS AND INTERNET-OF-CONTENT SERVICES FOR ALL IN THE HOME OR ON THE MOVE Accessible Internet-of-Things and Internet-of- Content Services for All in the Home or on the Move http://dx.doi.org/10.3991/ijim.v6i4.2190 Dimitrios Tektonidis and Adamantios Koumpis Research Programmes Division, ALTEC Software S.A., Thessaloniki, Greece Abstract— Public authorities and national healthcare service providers across the European Union and beyond are increasingly under pressure to deliver better services with diminishing resources. Users, quite rightly, demand a high standard of service and increasingly also convenience focused on the needs of the users rather than how the services are organised to deliver them. Users, on the whole, also prefer to be supported in their homes, and be free to move as much as possible. While users without impairments and generally well provided for, there are increasingly also users who have special needs, including the aged and those with a range of physical and mental disabilities who are unable to make effective use of such services from their homes due to the lack of proactive and accessible services. The idea we promote in the chapter is to provide better support to users with disabilities and impairments from the comfort of their home by means of providing them with a set of scalable services which can be either offered for free or purchased through some central form of a marketplace repository. Index Terms—Internet of Things, Internet of Content, smart home, smart hospital, smart city, service interoperability, service connectivity, dependable services. I. INTRODUCTION The arrival of Smartphones and Tablet PCs has created the Post-PC era and led to a development of IOS and IOT. However people with disabilities cannot fully take advantage of these benefits. Our vision is to provide better support to users with disabilities and impairments from the comfort of their home by providing: • Advisory services to help manage their health and social care needs and ensure they can effectively access relevant services. • Monitoring services that help users manage on their own and remind them to about essential aspects such as taking their medication and monitoring their condition and generally look after themselves properly. • Alerting services when the users situation suddenly deteriorates and urgent help is needed. • Dependable services that will check whether services are functioning correctly and notify service provider when the connection to special care users is lost unexpectedly. • Accessible Interfaces using suitable interface technology and user interfaces that are appropriate to the special needs of users. • Smart Interfaces that are sensitive to the context and current needs and that ensure appropriate services are made use of. • Orchestration Services that make sure that services are orchestrated effectively where needed to deliver a seamless service at the point of service delivery. • Third party services when the users may want to use services such as online grocery shopping or public transport or use social networking facilities. The concept is built on the fundamental principle that users should be supported as much as possible by accessible services to maintain independence through monitoring and advice and involve available services when needed to address emerging needs. The services can be provided by both public and private providers (and even include wireless connection to devices such as blood pressure monitors, flow meters, glucose level monitors and the like) to ensure that the user is appropriately supported in managing by themselves while ensuring they receive emergency services if needed. This should include other services required for daily living and to facilitate the user managing on their own and giving them the freedom of independent living. It is also envisaged that access to social networking services be provided to mitigate social exclusion suffered by housebound users who live on their own. We believe that making this vision (we call it “IOTC4ALL” – Internet of Things and Content 4 (: for) ALL) reality will have a significant impact on social inclusion and supporting disadvantaged users to lead independent lives with the assurance of support where needed. The vision is to develop an open service model that allows new service providers to enter and which can easily be extended to different member states independent of the technology used. The aim is to enable to a large variety and diversity of users with special needs the usage of complex services through devices that although are considered smart they are difficult in terms of use for several people. Also a key aspect of the aspired architecture is the inclusion of people with disabilities to execute these services: though it is about services, the architecture requires an enriched Service Oriented Architecture by components that will iJIM – Volume 6, Issue 4, October 2012 25 PAPER ACCESSIBLE INTERNET-OF-THINGS AND INTERNET-OF-CONTENT SERVICES FOR ALL IN THE HOME OR ON THE MOVE enable and facilitate the discovery and execution of services from smart and mobile devices. In practical terms we need to offer a platform where service providers can: • Design services • Implement Services • Deploy • Publish Figure 1. Overall conceptualisation to support the IOTC4ALL vision And where the providers of devices and third party service providers can: • Identify relevant services • Compose and configure services for the specific user • Configure interface devices (tablet, PDA, smartphones, web cams) • Add further home-based healthcare devices where required (blood pressure meters, glucose meters etc) To achieve these goals, we have to deal with the following issues: • Service Orientation: Given that we need to get away from the website orientation to a service orientation how services can be exposed so that they can be accessed by special devices for impaired and disabled users that aim to make them accessible to those with special needs (and potentially combine them with other related services to provide a user centric overall service). • Discoverability: Given a variety of services from a variety of providers and the needs and entitlements of individual users, how the services a user can access be discoverable by assistive devices or special interfaces users will use. • Service Entitlement: Given that services are selective in terms of users meeting certain criteria it should be possible to determine the subset the user is entitled to given their personal circumstances and not to confuse users with irrelevant services and rather work with those they can use at that point in time. • Service Composition and Integration: Services should be fashioned around the user rather than the user having to adapt to the services. So for the user the division between services should be transparent; some services may be provided in collaboration or based on joint or shared responsibilities by different organisations (healthcare, public services etc) and need to integrate seamlessly at the point of delivery to the user. This will require suitable approaches for service composition and integration. • Context sensitivity: Given that the user will only requires some of the available services at any particular point in time it will be important to focus on the context and the immediate needs of the user to increase the assistive powers and not overburden the user who may be bewildered by the sheer amount of services available and what to use in any given situation given on the stated or inferred needs (the latter if some home based measuring devices are used for example) • Accessibility: Given that users will not be IT literate or have special requirements with respect to interfaces in term of presentation, navigation and interface type (especially for blind and with motor disabilities) so that users can effectively use the services. This may also have to be combined with a number of special devices for input/data collection or as output devices. We envisage three main provider communities regarding the proposed service execution platform: • Service providers (public authorities and public services) aiming to manage their services (healthcare, social care and public services) delivery to the service users (disabled or aged) integrate them for a seamless service provision to the end user, • Services consumers will be (disabled or aged) customers that use services through apps from their home using mobile devices (e.g. smart phones, tablets etc.) or stationary devices (PC or PC enabled TV) directed at IT illiterate or inexperienced individuals without any technical knowledge. • Third Party Providers - IT solutions and value added service who are involved in the service design, composition (Service designers) and IT service provision. If the original service providers are unable to provide interfaces third party service providers will take on the role of generating devices and interfaces and additional services. In the next section we present a set of informative scenarios of use that can be easily adapted and customised to fit the realities faced by a broad category of consumers and citizens today. Though emphasis is given to categories facing the risk of exclusion due to age or some type of disability, it is obvious that the same scenarios are applicable to the average European citizen and service customer. II. USER SCENARIOS The aim of the IOTC4ALL vision is to assist users with disabilities and impairments in their quest for independent living. The goal is to support them in their independence while offering them support services through accessible smart interfaces so that they can effectively receive the services they need at a given point in time. These services are to adapt to the users current needs and thus be user centric rather than service provider centric (which is what typically happens currently). Orchestrating services around the users’ current needs using accessible devices 26 http://www.i-jim.org PAPER ACCESSIBLE INTERNET-OF-THINGS AND INTERNET-OF-CONTENT SERVICES FOR ALL IN THE HOME OR ON THE MOVE and interfaces is the main goal of such a research effort. In the following we discuss a range of typical scenarios that we support: Scenario1: A blind user who lives on her own and who needs help with managing aspects daily living, managing to get around on foot and with public transport. May also need help with online shopping to reduce the need to get out of the house unnecessarily also during bad weather periods. Also as living on their own needs alerting facility in case they suddenly get unwell. Scenario2: A user with mental impairment who can generally manage living on his own but needs reminding about important medication they have to take regularly and as forgetful to manage aspects of daily living. He may also need help from doctors/nurses/heath visitors to make sure he receives help if needed and to monitor that he is still safe looking after himself. The user may also get socially isolated and may want wanting to interact with others in self-help groups or with friends and family. Scenario3: Elderly user with mobility and ongoing health management problems. She needs to monitor her condition and manage her health. At time she may need to be taken to places such as shops and doctors/hospital and would benefit from help with home shopping but is not computer literate. Scenario4: User with motor problems who can view screens but needs voice activated or simplified input devices as has motor impairment and consequently problems operating keyboards. He also needs special transport as wheelchair bound and needs to receive prepared meals. Scenario5: An obese young patient, by using smart devices (wireless weight scale or heart-rate measurement device), can keep contact with physicians sending to the Smart Hospital System all his physical parameters such as weight and the daily amount of physical exercise. Plus, the System can go back to the child suggesting to do more physical exercise and, eventually, giving him a target to reach by exercise games or by walking with a pedometer. Scenario6: A diabetic patient can find useful, in addition to those already described, also a wireless glucose meter which sends detected data to the System, so the medical staff is always updated about the state of the patient. Scenario7: An elderly person who has to follow a complicated medication plan can use a Smart Cabinet to store medicines: automatically the System can warn the user if he’s going to take a medicine that it shouldn’t be taken with certain foods or at the same times as other medications they are incompatible with (e.g.: an old man who gets confused by his wife’s medicines) and needs reminding about the medication he has to take regularly. What unites these scenarios s the users have some form of disability or impairment which makes it difficult to manage everyday activities and manage their health and social needs. IT is in principle possible to service all of these needs but this may be too demanding for the users in question who would still want to manage on their own as far as possible while having the assurance of help when needed. Through the use of smart assistants with accessible interfaces a number of available services can be managed on a context sensitive basis and tailored to the current needs of the user. These can be categorised into 3 main areas: Disability Related: Users with disabilities and special needs will want to be able to live on their own rather than being in the looked after system, but to do this effectively it is important that they receive support services that allows them to manage and that keeps them safe. Therefore the delivery of services and the integration of the user with these services in the home are needed. For users with disabilities (including aged users who are not computer literate) this presents a challenge. Their disabilities make it difficult for them to effectively use services, because they are provided in a way that is difficult for them to use or confusing or because their impairments such as blindness makes it difficult to navigate complex services. They need suitable interface to overcome accessibility problems and with assistive function to manage and reduce complexity. There is a concrete need for an end-to end solution to provide accessible interfaces on suitable devices, and accessible services that need in some cases to be integrated for a more complete solution which requires in some cases some form of orchestration. Mobility Related: Users with disabilities or impairment may need help concerning their specific impairments and their ability to function in an urban environment and find available services whilst on the move and this scenario aims to provide location based services and direction to those services the user may require in terms of finding them or using transport to get to them from their home and on the way back. This may also include the ability to call upon help in case the user suddenly gets unwell and needs attention. Apart from mobility aspects this also includes aspects of being able to access benefits and other public services the user may be entitled to. Thus it should be possible that these services are tailored to the individual and their circumstances and become immediately usable in an easy- to-use fashion. Healthcare related: Users may have specific healthcare needs but will want to be able to stay in their homes and be supported in their homes rather than occupying hospital beds but which needs more assistance and monitoring at home to keep them safe and avoid the healthcare system being clogged up with unnecessary hospital stays. Technology for this exists but we shall aim to provide an integrated workable and usable solution for the inferm or frail to be supported using available services more efficiently and in a more user focused fashion. The basic driver for this scenario is that a smart hospital is not necessarily an accessible one. So, this second use case in the IOTC4ALL vision aims to offer accessibility to a smart hospital features. Sensor devices enable function centered on patients, and in particular on diagnosing patient conditions, providing real-time information on patient health indicators. Application domains include different telemedicine solutions, monitoring patient compliance with medication regiment prescriptions, and alerting for patient well-being. Heterogeneous wireless access-based remote patient monitoring systems can be deployed to reach the patient everywhere, with multiple wireless technologies integrated to support continuous bio-signal monitoring in presence of patient mobility What is needed is some form of intelligent user interface that: • adapts to the specific form of disability iJIM – Volume 6, Issue 4, October 2012 27 PAPER ACCESSIBLE INTERNET-OF-THINGS AND INTERNET-OF-CONTENT SERVICES FOR ALL IN THE HOME OR ON THE MOVE • is able to determine the changing needs and wishes of the user • is able to identify relevant services • is able to support the user to access and use these services • is able to orchestrate services for a given goal where needed • reminds the user to complete routine tasks. All then scenarios require a common set of functionalities: • Special visual displays/output devices (large screens, voice output, braille etc) • Easy input devices (predictive text, restricted options, voice command, point and click) • Intelligent Interfaces that adapt to needs (using semantics to analyse current situation and context and structure services accordingly, also remind users where needed) • Ability to match needs with available services (using semantics to select services on user need, entitlement and user wishes) • Ability to orchestrate services for a given goal (using an orchestration service to orchestrate services towards a common goal – e.g. book doctors appointment and book ambulance or taxi or advise which bus and when) • Ability to interface with special local devices (e.g. blood pressure meter to monitor condition and advise to take medication or seek medical advice or assistance) There are some common set of requirements across several scenarios. These are to be met by a smart assistant that provides an interface the users are able to manage effectively and that assesses the current context to provide easy access to and orchestration of services to meet the users’ needs and wishes: • Manage medical conditions through monitoring and advice • Manage balance between supporting independence and calling for help when needed • Manage transport needs • Manage support needs in the home • Finding services suitable for the user • Reducing complexity • Online shopping and Home delivery of goods to avoid unnecessary trips to shops. III. ACCESSIBILITY MATTERS: OPENING SERVICE ACCESS TO ALL Accessibility is increasingly becoming and issue as well as assistive technology for both the able bodied and impaired users. Anti-discrimination legislation now requires businesses and public authorities alike to support disabled or disadvantaged users alongside their able- bodied counterparts. This is starting to attract attention also from the commercial community and with patent applications in these areas becoming more frequent, but where the EU is decidedly lagging behind as compared to the US as can readily be seen from the statistics readily available from ISO and EPO. A number of EU projects have begun to address aspects of accessibility, where movement and mental disabilities were more important, and to a lesser degree the sensory ones (including visual and hearing). Current EU-wide disability and accessibility-applicable research address Web, Sensors, Robotics and Brain-Machine Interfaces, as well as motor- related, mental-, and cognitive disabilities. In a recent survey carried out by the authors it came to light concerning the user needs, that respondents rated cognitive, mental and multiple disabilities as important and that health and employment are areas where new eAccessibility solutions will be needed most. An initial Trend Analysis revealed that standard technology for devices and connectivity were less of an issue, as these are now relatively widely available, while virtualisation, suitable HCI Interfaces, Avatars and Robots as companions were identified as potential weak areas in need of further work. The time horizon for those is considerably longer making them less of an easy picking for researchers and industry alike. IOTC4ALL will be addressing healthcare needs of users with disabilities and making them substantially more accessible than what is available at present through much improved HCI interfaces. Among the areas identified in need of development in the survey are solutions for increased independence of older people and active ageing with expectations from the community to meet the rights to participation of people with disabilities. With growing health care expenditure, Growing expenditure on social protection benefits, countermeasures to address fragmentation of (ICT) and develop affordable solutions with significant benefits are needed. What is needed and where IOTC4ALL will make a concrete contribution to novelty will be by developing new models where users have more control over services they consume, New model for evaluating and implementing accessibility. This will promote the business value of investing in accessibility and raise awareness of accessibility issues in the industry. Among the needs that can be identified in this area and which IOTC4ALL will be supporting: Advanced active interfaces: Dialog interfaces are a reality in many situations nowadays. However, they usually work with specific limited vocabulary and in a specific context. We aim to develop proactive interfaces that do not necessarily wait for the user to start the interaction but to prompt and remind users based on an analysis of context. This will be achieved by leveraging semantic technologies for assessment of context and accessing relevant services at that point in time so that the user receives focused assistance. IOTC4ALL will create innovative solutions here that are currently not available in the market nor have been reported in the academic literature. Automatic adaptation of multimodal interfaces: Modern devices such as tablets and smart phones can contain a lot of information about users, such as documents, music, preferences for various applications and so on. So, having a personal profile on a mobile device seems quite natural for the future of ICT. In addition, mobile phones are enhanced with connectivity sensors which allow them to interact with other machines through WiFi, Bluetooth, NFC to include additional monitoring devices such as blood pressure meters, glucose 28 http://www.i-jim.org PAPER ACCESSIBLE INTERNET-OF-THINGS AND INTERNET-OF-CONTENT SERVICES FOR ALL IN THE HOME OR ON THE MOVE meters and pulse meters. Interfaces should be smart and work out the context and adapt to the needs and preferences of the user and who can interact with remote services to support the user. This goes in tandem with advanced active interfaces and will generate interfaces that adapt to the users disability problems and then actively seek to assist the user to assess their current needs and locate and access relevant services making also use of local devices where appropriate to take readings. Again there are no solutions of this type currently available that cover all these aspects even if devices exist we can draw on to develop the proposed innovative service. Promoting discoverable and integratable services: Ubiquitous connectivity and network services together with cloud based services and technologies mean that services can be accessed remotely but the way services are made available (usually using standard websites and applications) which are largely provider centric and the way they are presented often make it difficult to consume and integrate and orchestrate them so that users with disabilities can be readily and effectively be supported. A new service model is needed that does allow them to be discovered and accessed as a service so that appropriate smart interfaces can connect to them remotely and make the services available alongside each other and combinable and orchestratable at the interface level. We create innovation and make a concrete contribution in this area. IV. SETTING UP A SOLUTION The proposed Service Execution Environment based on the vision and the scenarios previously presented aims to provide a new approach that will improve the use of services from user groups that have accessibility problems. The definition of a service will be enriched with all the required features and parameters that are related to the accessibility of the service. Therefore the proposed solution will investigate and develop the following: • An enhanced Service Model that will include additional specifications related to the e-accessibility issues. • Semantics that will enrich existing services in order to increase their accessibility to groups of users that this is required. • A Personalised Application and Service Marketplace where developers will be able to publish their applications and users will be able to find services and applications based on their profile. A. The IOTC4ALL Service Model The provision of the services for IOTC4ALL, will follow the life-event model that has been implemented successfully in the past for e-Government Services1. The life-event approach facilitates discovery of the service because it bases the services discovery in the user profile and on WHAT the user wants to do (“I want to go to the hospital”, “I want to buy groceries”). The execution of these services will be based on the user profile based on his/her needs. Services execution will be differentiated based on the disabilities of the user (“What is the preferable means of transport for a visual impaired 1 OneStopGov project http://islab.uom.gr/onestopgov/ person”, “How can he/she input the necessary data for executing the service”) The user will be able to download the application to his/her device from the IOT4CALL marketplace according to his/her needs. The services will be installed as Apps (Mobile Applications) to his/her mobile device (Tablet/Smartphone). The installed services will use information from the IOTC4ALL User Profile Registry. The execution of the service will involve the IOT4CALL Semantic Discovery Facility that will be able to locate and retrieve information from available information links. We use the term “information link” as in Linkeddata2 to define every available system or service that can be accessed through the web. The IOT4CALL Marketplace will provide a variety of applications that will make accessible a large variety of services capable to cover the scenarios and improving the lifestyle of the people with disabilities. B. Enhancing services components to enrich their accessibility Today as the number of available services grows the percentage of people with disabilities that can actually access these services is reducing. Web Accessibilities guidelines are applied to many web portals (especially governmental). However these guidelines cannot be applied to mobile applications. Therefore the design for the usability of such applications is limited to the specifications of the mobile device (i.e size to the screen) and not to who is going to use is. The problem is not only to the user interface but also to the service that the mobile application is using. Of course, the cost for reengineering the existing service is huge therefore IOTC4ALL propose an approach that will enhance the existing creating wrappers that will include all the additional features that are related with accessibility. For IOTC4ALL the usage of a service breaks down into components that are independent and can be enriched without changing its structure and operation. To facilitate the enrichment of the services components, IOTC4ALL will provide to the service providers semantic descriptions to enrich their service. Following the approach of Linkeddata where software systems are defined as Information Links, IOTC4ALL will use Semantic Web Technologies to define the requirements of enhancing existing services without changing their structure. C. The IOTC4ALL Marketplace Marketplaces are a common practice from mobile applications and in the future for software in general. Existing Marketplaces (iTunes, Android Marketplaces etc) gather information from the user like the device he/she is using, location and other to suggest applications that may be in his/hers interest. For the applications developers, marketplaces enable them not only to sell their application but also to evolve providing new versions and communicating with their customers. In IOTC4ALL we can make use of a Marketplace that will also gather information based on accessibility providing intelligent filtering for the available apps. In addition it will provide all the necessary infrastructure to facilitate both the users (customers) and the developers. 2 http://www.linkeddata.org iJIM – Volume 6, Issue 4, October 2012 29 PAPER ACCESSIBLE INTERNET-OF-THINGS AND INTERNET-OF-CONTENT SERVICES FOR ALL IN THE HOME OR ON THE MOVE D. Smart Devices UI IOTC4ALL’ goal is to enable to a large variety and diversity of people with special needs the usage of complex services through devices that although are considered smart they are difficult in terms of use for several people. To give better supports to users with disabilities and impairments from the comfort of their home or in mobility it is crucial make the above devices fully accessible by means of the definition and implementation of novel interface solutions which are appropriate to the special needs of people. In particular, a holistic approach in defining an adaptive smart interface should be adopted to pursue a global optimization by considering not only the different kind of devices but also their sensitiveness to the context and people needs, in so ensuring services are appropriately used. In order to be useful, the design of user interfaces should consider many aspects of human behaviours and needs. The complexity of the extent of the involvement of a human in interaction with a machine is sometimes invisible compared to the simplicity of the interaction method itself. The existing physical technologies for Human Computer Interaction (HCI) essentially can be categorized by the relative human sense that the device is designed for. These devices are basically relying on three human senses: vision, audition, and touch (Te’eni et al. 2007). Input devices that rely on vision are the most used kind and are commonly either switch-based or pointing devices (Myers 1998)The switch-based devices are any kind of interface that uses buttons and switches like a (virtual or physical) keyboard (Shneiderman 1998). The pointing devices examples are mice, joysticks, touch screens, graphic tablets and pen-based input (Murata 1991). The output devices can be any kind of visual display or printing device (Nielsen 1994). The devices that rely on audition are more advanced devices that usually need some kind of speech recognition (Rabiner 1993). Nowadays, all kind of non-speech and speech signals (Brewster 2003) and messages are produced by machines as output signals. Beeps, alarms, and turn-by-turn navigation commands of a GPS device are simple examples. Recent methods and technologies in HCI are trying to combine former methods of interaction together and with other advancing technologies such as networking and animation. These new advances can be categorized in three sections: wearable devices (Barfield and Caudell 2001), wireless devices (Yacoub 2007). The technology is improving so fast that even the borders between these new technologies are fading away and they are getting mixed together. Few examples of these devices are: radio frequency identification (RFID) products, personal digital assistants (PDA), and virtual tour for real estate business. Current directions and advances of research in user interface design include intelligent and adaptive interfaces as well as ubiquitous computing. 1) Intelligent and Adaptive HCI As mentioned before, it is economically and technologically crucial to make HCI designs that provide easier and more pleasurable experience for the users. To realize this goal, the interfaces are getting more natural to use every day. Evolution of interfaces in note-taking tools is a good example. First there were typewriters, then keyboards and now touch screen tablet PCs that you can write on using your own handwriting and they recognize it change it to text and if not already made, tools that transcript whatever you say automatically so you do not need to write at all. One important factor in new generation of interfaces is to differentiate between using intelligence in the making of the interface (Intelligent HCI) (Maybury and Wahlster 1998) or in the way that the interface interacts with users (Adaptive HCI) (Kirlik 2006). Intelligent HCI designs are interfaces that incorporate at least some kind of intelligence in perception from and/or response to users. A few examples are speech enabled interfaces that use natural language to interact with user and devices that visually track user’s movements or gaze and respond accordingly. Adaptive HCI designs, on the other hand, may not use intelligence in the creation of interface but use it in the way they continue to interact with users. An adaptive HCI might be a website using regular GUI for selling various products. This website would be adaptive -to some extent- if it has the ability to recognize the user and keeps a memory of his searches and purchases and intelligently search, find, and suggest products on sale that it thinks user might need. Most of these kinds of adaptation are the ones that deal with cognitive and affective levels of user activity. Another example that uses both intelligent and adaptive interface is a tablet PC that has the handwriting recognition ability and it can adapt to the handwriting of the logged in user so to improve its performance by remembering the corrections that the user made to the recognised text. Finally, another factor to be considered about intelligent interfaces is that most non-intelligent HCI design are passive in nature i.e. they only respond whenever invoked by user while ultimate intelligent and adaptive interfaces tend to be active interfaces. The example is smart billboards or advertisements that present themselves according to users’ taste. 2) Ubiquitous Computing and Ambient Intelligence Ubiquitous computing (Ubicomp), often used interchangeably by ambient intelligence and pervasive computing, refers to the ultimate methods of human- computer interaction that is the deletion of a desktop and embedding of the computer in the environment so that it becomes invisible to humans while surrounding them everywhere hence the term ambient. The idea of ubiquitous computing was first introduced by Mark Weiser during his tenure as chief technologist at Computer Science Lab in Xerox PARC in 1998. His idea was to embed computers everywhere in the environment and everyday objects so that people could interact with many computers at the same time while they are invisible to them and wirelessly communicating with each other (Riva et al. 2005). Ubicomp has also been named the Third Wave of computing. The First Wave was the mainframe era, many people one computer. Then it was the Second Wave, one 30 http://www.i-jim.org PAPER ACCESSIBLE INTERNET-OF-THINGS AND INTERNET-OF-CONTENT SERVICES FOR ALL IN THE HOME OR ON THE MOVE person one computer which was called PC era and now Ubicomp introduces many computers one person era. 3) From unimodal to multimodal interfaces An interface mostly relies on number and diversity of its inputs and outputs which are communication channels that enable users to interact with computer via this interface. Each of the different independent single channels is called a modality (Jaimes and Sebe, 2007). A system that is based on only one modality is called unimodal. According to the nature of different modalities, they can be divided into three categories: Visual-Based: Facial Expression Analysis, Body Movement Tracking (Large-scale), Gesture Recognition, Gaze Detection (Eyes Movement Tracking) Audio-Based: Speech Recognition, Speaker Recognition, Auditory Emotion Analysis, Human-Made Noise/Sign Detections (Gasp, Sigh, Laugh, Cry, etc.), Musical Interaction Sensor-Based: Pen-Based Interaction, Mouse & Keyboard, Joysticks, Motion Tracking Sensors and Digitizers, Haptic Sensors, Pressure Sensors, Taste/Smell Sensors. Basic visual user interfaces in smart devices are the standard graphic based WIMP (window, icon, menu, and pointing device). Smart devices also have multiple modalities and are able to interpret information from various sensory and communication channels. For instance, smartphones uses touch based interactions and some (e.g.LG Optimus) allow users to use personalized gestures as shortcuts to commands. Multimodal interfaces process two or more combined user input modes (such as speech, pen, touch, manual gesture, gaze, and head and body movements) in a coordinated manner with multimedia system output (Oviatt 2003). They are a new class of interfaces that aim to recognize naturally occurring in the forms of human language and behavior, and which incorporate one or more recognition-based technologies, e.g., speech, pen, vision. Thus, they can provide richer and more natural methods of interactions, e.g., gesture, speech, and even all the five senses. The term multimodal refers to combination of multiple modalities. In MMHCI systems, these modalities mostly refer to the ways that the system responds to the inputs, i.e. communication channels. The definition of these channels is inherited from human types of communication which are basically his senses: Sight, Hearing, Touch, Smell, and Taste. The possibilities for interaction with a machine include but are not limited to these types. Therefore, a multimodal interface acts as a facilitator of human-computer interaction via two or more modes of input that go beyond the traditional keyboard and mouse. The exact number of supported input modes, their types and the way in which they work together may vary widely from one multimodal system to another. Multimodal interfaces incorporate different combinations of speech, gesture, gaze, facial expressions and other non- conventional modes of input. One of the most commonly supported combinations of input methods is that of gesture and speech. An interesting aspect of multimodality is the collaboration of different modalities to assist the recognitions. For example, lip movement tracking (visual- based) can help speech recognition methods (audio-based) and speech recognition methods (audio-based) can assist command acquisition in gesture recognition (visual- based). Some examples of applications of multimodal systems are Smart Video Conferencing, Intelligent Homes/Offices, Driver Monitoring, Intelligent Games, E-Commerce and of special interest for our project Helping People with Disabilities. A good example of Multimodal System for disable people is to address and assist disabled people (as persons with hands disabilities), which need other kinds of interfaces than ordinary people. In such systems, disabled users can perform work on the device by interacting using voice and head movements. 4) Smart devices today In the past mobile devices were difficult to use and had limited user input methods (number keyboard or stylus) with small displays. Thus advanced usage was limited to business applications and platforms like Symbian and Blackberry. With the release of the iPhone this suddenly changed. Multi-touch interaction on a large colourful display enabled a world of new mobile applications affordable und manageable for everyone. The new interaction paradigm was adopted by all players in the market including Google, Nokia and also Microsoft. A few years later with Apple’s iPad this concept has evolved into a new kind of personal computer, the tablet PC. While UMPCs and Netbooks appeared too bulky and complicated, the tablet PC brings the simplified mobile touch interaction of the iPhone into new interaction situations: touchscreen and overall user interface design are intuitive and visually accessible. The much larger screen should improve accessibility for people with low vision and other partial vision impairment. Furthermore, the capacitive touchscreen is more accessible to those with dexterity limitations by introducing an electronic stylus; while those with limited finger movement and/or sensation in their fingertips have greater success using resistive touchscreen devices because of the ability to use a stylus. But, the electronic stylus for the capacitive touchscreen interfaces levels the playing field. Cameras are increasingly used in conjunction with GPS functionality by software applications for augmented reality, mixed reality, and location-based services. Additionally, cameras are becoming an essential feature for devices used in the growing area of remote health monitoring. Being able to take a picture of a wound, infection or other visible condition will enhance the ability of people (with or without disabilities) to avoid the disruption and considerable effort of going to the doctor's office. In addition, easy input devices have been developed, such as predictive text, restricted options, voice command, point and click. V. CONCLUSIONS Smart house – smart hospital – smart city: are there any common threads amongst them? One may think that each one of these application areas is worth a dedicated examination. In a house we live for years so it is easy to see why a person may need to invest on making it (more) accessible. While in a hospital people stay (fortunately) only for short (and usually unexpected) periods of their life – and again there smartness expected from a hospital iJIM – Volume 6, Issue 4, October 2012 31 PAPER ACCESSIBLE INTERNET-OF-THINGS AND INTERNET-OF-CONTENT SERVICES FOR ALL IN THE HOME OR ON THE MOVE may be of totally different nature than the smartness we expect from a home to exhibit to its inhabitants. Finally, for the smart city case, the requirements are also different: people may need to “connect” to different types of information provision services and infrastructures and two basic service types are: (a) what services are offered to me and (b) how can I get . What we can consider as common denominator of all these are: • The users have access to their mobile phone (smartphone, Android or iphone) so the unique access point to information from the environment (house, hospital or city) comes through their mobile phone. • It is through their mobile phone that they may recognize the various addressable entities i.e. the “things” (of the Internet of Things) that are surrounding them and which can be tagged with QR codes / RFID tags or be accessible through NFC, and • It is again through their mobile phone that they will be informed about the availability of the various content elements (of the Internet of Content) that are retrievable and which can be tagged with QR codes / RFID tags or be accessible through NFC What is our understanding of a service in the context of the IOTC4ALL vision? This comes up to a series of highly connected links and ideas: • Any retrieval of location-based (house / hospital / city) content is a service (though a basic one – could we come up with more sophisticated ideas?) • Any access to a “thing” for getting information about its status or for manipulating is also regarded as a service (again this may be regarded as basic ones). • More sophisticated or complicated services may base on the composition of more than one basic services – this way we may support different families of scenarios to be realized. • What we promote as an idea and as a role is this of industrial enterprises (like the ones participating in the project) which will invest on offering such types of composite services to people. So given one house / hospital / city setting, user X may decide to buy the service bundle by value added service provider A, or B or C, or, in case s/he wants may still access ‘manually’ each of the offered ‘content’ and ‘things’ manually. More specifically: o User X may decide to buy a basic service from value added service provider A that enables organized scenario selection according to a (simple) thematic organization. o Or spend some more money and buy the more expensive service B, where there is also increased intelligence i.e. the selection of a particular scenario follows the mood of the user and the user does not have to choose one. This case is of interest to categories like older people or bipolar patients who may be unable to make a decision on their own and so this extra degree of automation helps their independent living. Finally, o User X may choose to buy a premium service from provider C that offers him/her with an advanced level of customer service and care that extends beyond basic service activities offered at level A and slightly more sophisticated or automated of level B, to include a more personalized and customized approach that create a customer-centric atmosphere. In this case, there may be increased interoperability offered for services to bridge gaps between the various environments that the user is changing – i.e. from the home through the city to the office and then back to home. This extra level of semantic processing that helps information flow between the various smart settings (house / city / hospital) is offered if the customer agrees to pay extra money. Below we briefly summarise each of the supported scenario clusters: Smart home scenario: A smart home in the context of this paper is considered as an augmented environment with the ability to consolidate information (“content”) embedded in appliances, as well as micro-systems and multi-modal sensors to offer users enhanced access to information and functionality and an ability to control and mange the environment at a higher level of abstraction. So why shouldn’t the home take initiative to either understand (proactively) the user’s mood and offer the most appropriate setting adjustment or react to user’s wishes (user chooses to watch TV, light settings are adapted and phone calls, etc.) Smart hospital scenario: What is our idea for a smart hospital? In the context of this paper is mainly related with enhancing accessibility of to its patients (as well as the medical staff working). The users in this pilot may be able-bodied patients as well as visually / cognitive / age challenged persons. The idea of accessibility in the hospital context relates with helping patients take their drugs on time (so all blister packs and other drugs’ packages may contain RFID labels) and facilitating their access to medical equipment that embeds RFID tags. The doctors, nurses, caregivers and other staff members wear a “smart badge” storing information (besides their employee ID number) that may help the patient ask for advice or guidance. Same way as doctors, on arrival, each patient receives a wristband with an embedded RFID tag storing a unique identifier, and some information about his / her profile that besides medical information may include also information significant for the types of interaction that the particular patient is capable to perform with services / “content” / “things” available within the hospital environment. Smart city scenario (mobile phone as a “white cane”): Consider a visually impaired person travelling through a city. Similar to a foreigner in a hostile place feeling lost and requiring guidance, the use of a mobile phone to provide such assistance is ideal. The primary aim of this scenario is to enable visually / cognitive / age challenged persons to acquire the necessary guidance and assistance by capturing geo-tagged information using their mobile phone, and return any information, such as a specific pharmacy, the local police station, or even points of interest. Additionally, the GPS capability of the phone is exploited to enhance the locally available services recognition process (service spots). As far as the links with the other two scenarios are concerned we should notice that with the use of the mobile phone, the same way that a visually / cognitive / 32 http://www.i-jim.org PAPER ACCESSIBLE INTERNET-OF-THINGS AND INTERNET-OF-CONTENT SERVICES FOR ALL IN THE HOME OR ON THE MOVE dementia challenged person is enabled to get guidance and assistance for the urban environment for short-range navigation and exploration in the home and hospital settings. REFERENCES [1] W. Barfield and T. Caudell, Fundamentals of Wearable Computers and Augmented Reality, Lawrence Erlbaum Associates, Mahwah (2001) [2] S. Brewster, “Non speech auditory output”, in J.A. Jacko and A. Sears (eds), The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Application, Lawrence Erlbaum Associates, Mahwah (2003) [3] A. Jaimes and N. Sebe, “Multimodal human computer interaction: a survey”, Computer Vision and Image Understanding, 108(1-2), pp 116-134 (2007) http://dx.doi.org/10.1016/j.cviu.2006.10.019 [4] A. Kirlik, Adaptive Perspectives on Human-Technology Interaction, Oxford University Press, Oxford (2006) [5] M.T. Maybury and W. Wahlster, Readings in Intelligent User Interfaces, Morgan Kaufmann Press, San Francisco (1998) [6] A. Murata, “An experimental evaluation of mouse, joystick, joycard, lightpen, trackball and touchscreen for Pointing - Basic Study on Human Interface Design”, Proceedings of the Fourth International Conference on Human-Computer Interaction 1991, pp 123-127 (1991) [7] B.A. Myers, “A brief history of human-computer interaction technology”, ACM interactions, 5(2), pp 44-54 (1998). http://dx.doi.org/10.1145/274430.274436 [8] J. Nielsen, Usability Engineering, Morgan Kaufman, San Francisco (1994). [9] Oviatt, S. L. Advances in Robust Multimodal Interface Design.” IEEE Computer Graphics and Applications, 23 (2003): 52-68 http://dx.doi.org/10.1109/MCG.2003.1231179 [10] L.R. Rabiner, Fundamentals of Speech Recognition, Prentice Hall, Englewood Cliffs (1993). [11] G. Riva, F. Vatalaro, F. Davide and M. Alaniz, Ambient Intelligence: The Evolution of Technology, Communication and Cognition towards the Future of HCI, IOS Press, Fairfax (2005) [12] B. Shneiderman, Designing the User Interface: Strategies for Effective Human-Computer Interaction (3rd edition), Addison Wesley Longman, Reading (1998) [13] D. Te’eni, J. Carey and P. Zhang, Human Computer Interaction: Developing Effective Organizational Information Systems, John Wiley & Sons, Hoboken (2007) [14] M.D. Yacoub, Wireless Technology: Protocols, Standards, and Techniques, CRC Press, London (2002)) and virtual devices (K. McMenemy and S. Ferguson, A Hitchhiker’s Guide to Virtual Reality, A K Peters, Wellesley (2007) AUTHORS Dimitris Tektonidis holds a BSc in Software Engineering and a PhD in Computer Science for research in the area of Enterprise Application Integration. His main fields of research are Enterprise Application Integration (EAI), IS interoperability, Service Oriented Architectures (SOA), Web Services, Semantic Web, e-Commerce, e- Government and Enterprise Systems (ERP, CRM, SCM). (e-mail: dte@altec.gr). Adamantios Koumpis heads the Research Programmes Division of ALTEC S.A., which he founded at 1996 (then as independent division of Unisoft S.A.). Adamantios holds a PhD degree from the University of Kingston, UK and a Bachelor degree from the University of Crete, Greece. (email: akou@altec.gr). Received 17 July 2012. Published as resubmitted by the authors 23 September 2012. iJIM – Volume 6, Issue 4, October 2012 33