International Journal of Interactive Mobile Technologies – ISSN: 1865-7923 – Vol. 13, No. 2, 2019 Paper—Mobile Phone Sensing using the Built-in Camera Mobile Phone Sensing using the Built-in Camera https://doi.org/10.3991/ijim.v13i02.10166 Norliza Katuk (*), Nur Haryani Zakaria, Ku Ruhana Ku-Mahamud Universiti Utara Malaysia, Malaysia k.norliza@uum.edu.my Abstract—Modern mobile phones or smartphones have multipurpose func- tions apart from being used as a device for voice and text communications. They are also embedded with many useful sensors, including camera, barometer, ac- celerometer, and digital compass. Unlike other types of sensor, the smartphone camera has been underutilized. This paper aims to fill the gap by analyzing and reviewing the hardware and software components of smartphones and highlight- ing the potential uses of the smartphone camera to support human daily life ac- tivities. A simple search of research papers indexed in Google Scholar was car- ried out using guided terms. The papers were filtered to match the research ques- tions for this study. Only relevant papers were analyzed and reviewed. The results of the analysis suggested that the rapid development in the smartphone hardware has extended the use of the smartphone cameras beyond personal and social pho- tography. Consequently, applications (apps) based on smartphone cameras emerge, such as barcode readers, document digitizing tools, augmented reality, translation tools, and indoor positioning device. These apps benefit users and fa- cilitate them in their daily life activities. Keywords—Mobile devices, mobile phones, mobile telecommunication sys- tems, sensors, surveys 1 Introduction The hardware and network advancements have caused desktop computing to be replaced by the emerging mobile computing technology in which smartphones and tab- let computers play a significant role in people’s life [1, 2]. Modern mobile phones or smartphones are a multipurpose device that can be used as a camera, video player, mu- sic player, navigation tool, and mini personal computer; more than simply a device for voice call and text messaging. They are also embedded with many sensors such as a digital camera, accelerometer, digital compass, gyroscope, global positioning system (GPS), and microphone which enables the development of new mobile applications that support human daily life activities. The embedded sensors enable the development of mobile applications (apps) in various domains including healthcare, environmental monitoring, and transportation that consequently created an area of research called mo- bile phone sensing [3]. Unlike other types of sensor, the built-in camera has been underutilized for mobile sensing [4]. Recent trend shows that researchers begin to study mobile sensing using 102 http://www.i-jim.org https://doi.org/10.3991/ijet.v14i03.10105 https://doi.org/10.3991/ijet.v14i03.10105 mailto:k.norliza@uum.edu.my mailto:k.norliza@uum.edu.my Paper—Mobile Phone Sensing using the Built-in Camera the built-in camera in various domains. However, the studies are specialized in their respective scientific domains that limit the information distribution among ordinary smartphone users. Also, the technical terms used in reporting mobile sensing studies make the content difficult to comprehend by ordinary users. Therefore, there is a need for a source of information that reveals the potential use of built-in smartphone camera to support human daily life activities. Hence, this paper aims to fill the gap by reviewing the possible uses of the built-in smartphone camera. Two research questions guide this study: • “What are the components of a modern smartphone?” • “What are the uses of modern smartphone camera?” An analysis was conducted on research papers derived from Google Scholar. The key search terms include “mobile phone sensing,” “built-in camera,” “smartphone cam- era,” and “mobile phone camera.” A simple search using the terms returns thousands of research papers. They have been filtered, and irrelevant papers were discarded. The relevant papers were selected to proceed with a content analysis that is presented in Sections 2 and 3 of this paper respectively. The next section of this paper discusses the generic components of a smartphone including the sensors. Then, Section 3 presents the potential use of the smartphone camera to support daily life activities. Finally, con- clusions are drawn in Section 4. 2 The Components of Smartphones This section answers the first research question on “What are the components of a modern smartphone?” A mobile phone with software apps offers advanced computing ability and enhanced connectivity is called a smartphone [5, 6]. Smartphones have be- come a pervasive and affordable device that provides users with various functionalities as offered by desktop computing [7]. It is a small and straightforward communication tool transformed into a device that is comparable to a low-end traditional computer, with interesting apps. Basic functionalities such as browsing the Internet, reading and composing emails, and working with office documents can be carried out through a smartphone or a tablet computer. Like desktop computers and laptops, smartphones are made up of hardware and soft- ware to allow the devices to operate [8]. Although smartphones are thin and small concerning the design; their performance and capability are far better than the last cen- tury desktop computers with credit to semiconductor manufacturing industry that came up with innovative hardware technology for smartphones. The standard hardware of a computer system such as the processor, memory, input devices, output devices, and the network interface is also needed in a smartphone. In addition to the generic hardware, smartphones are embedded with GPS module, subscriber identity module (SIM), and sensors. Daponte, et al. [9] described the generic architecture of smartphones in a com- prehensive review and suggested that a modern smartphone is comprised of eleven hardware components as shown in Fig. 1. The components are application processor, iJIM ‒ Vol. 13, No. 2, 2019 103 Paper—Mobile Phone Sensing using the Built-in Camera internal memory, data transfer interface, 3/4G network interface, GPS module, sensors, power management, external memory, SIM, touch display, and a speaker module. Fig. 1. The hardware components of smartphones [9]. Each of the smartphone hardware components performs specific roles. As shown in Fig. 1, the application processor is the ‘heart’ of smartphone where all other hardware components are connected and working from it. Table 1 summarizes the functions of the components. Table 1. The components of modern smartphones and their functions. Components Function(s) Application processor A system-on-chip (SOC) that executes the instructions in a program (app) in- stalled on smartphones. Internal memory Volatile and temporary storage for keeping the currently executed instructions. Data transfer interface An interface for communicating data with other devices. The technology includes Bluetooth, Wi-Fi, USB, and GIPO. 3/4G network interface An interface that is used to connect smartphones to the cellular networks. GPS module A module that allows the smartphone users to connect to satellite systems for road navigation. Sensors A device or module that detects changes or events and responds to some inputs from the physical environment. Power management A module that manages the smartphone battery consumption. External memory Non-volatile storage to store data in smartphones. The common type is a secure digital (SD) card. Subscriber identity module (SIM) An integrated circuit stores information of mobile telephony subscriber identity. Touch display A digital screen that displays output and receives input through the touch of a fin- ger or stylus. Speaker module A device to produce audio output that could be used with or without a headphone jack. Internal Memory Application Processor Power Management GPS Module 3/4G Network Interface Data Transfer Interface Sensors Speaker Module Touch Display SIM External Memory 104 http://www.i-jim.org Paper—Mobile Phone Sensing using the Built-in Camera Distinct types of sensor are embedded in modern smartphones. These sensors ena- ble the implementation of new apps across various domains including healthcare, trans- portation, environmental monitoring, and safety [3]. The smartphone sensors can be classified as inertial, positioning, and ambient sensors [10]. Inertial sensors can meas- ure the physical motion of a solid object such as accelerometers and gyroscopes. Posi- tioning sensors can sense the user’s location and presence of entities in her proximity using technology such as Bluetooth, Wi-Fi, Global System for Mobile Communications (GSM), and Global Positioning System (GPS). On the other hand, ambient sensors such as a camera, magnetometer, or microphone are used for sensing the surroundings of a user. Su, et al. [11] classified the smartphone sensors into ten types of sensors namely accelerometer, ambient temperature sensor, gravity sensor, light sensor, linear acceler- ation, magnetometer, barometer, proximity sensor, humidity sensor, and gyroscope. On the other hand, another review by Daponte, et al. [9] also specified ten types of sensors that are classified into five categories namely optical, thermal, acoustics, magnetic, and mechanical. The sensors include a camera and video camera, barometer, gyroscope, proximity, digital compass, temperature, microphone, red-green-blue (RGB) light, ac- celerometer, and GPS as illustrated in Fig. 2. Table 2 summarizes the functions of the individual sensors classified by Daponte et al. [9]. Fig. 2. Types of sensors embedded in a smartphone [9]. Table 2. Built-in sensors in modern smartphones [9, 11]. Sensors Function(s) Video camera A sensor that recognizes and capture image for various purposes including image recognition [12], augmented reality [13], and indoor positioning [14, 15]. Barometer A sensor that measures air pressure and can be applied in vehicle states detection [16]. Proximity A sensor that detects the presence of nearby objects and used to detect the presence of ears during a phone conversation for turning off the touch screen and brightness. Digital compass (Magnetometer) A sensor that detects orientation in relation to the Earth's magnetic field. It is used by compass-based apps. Temperature A sensor that can read the surrounding temperature. Microphone A sensor that can recognize sound and audio in our daily life activities such as coughing, eating, and emotions. iJIM ‒ Vol. 13, No. 2, 2019 105 Paper—Mobile Phone Sensing using the Built-in Camera RGB Light A sensor that measures illumination and it is used by default to adjust the screen dis- play brightness. Accelerometer A sensor used for activity and gesture recognition such as fall detection [11]. By de- fault, the sensor is used for automatic orientation of the smartphone screen and camera [3]. Gyroscope A sensor used for motion recognition that can provide the direction of orientation such as identification of driving style [17]. GPS A sensor that detects the current location mainly used for positioning and location- based service (LBS). 3 The uses of Smartphone Camera This section answers the second research question on “What are the uses of modern smartphone camera?” A built-in camera is a useful tool for facilitating photography- related tasks. During the early introduction of a phone camera, it was mostly used for taking photos to achieve social and personal purposes such as enriching the mutual experience, sharing or communicating experiences with those who were absent and creating personal memory [18]. Today, the advanced version of the phone camera found in smartphones allows users to do photography instantly and share the photos through social network apps installed on smartphones. Smartphones have replaced the role of a digital camera for taking photos of many ordinary users due to the handy feature [7]. It has been estimated that 1.2 trillion digital photos were taken worldwide in 2017 and 85% of these photos were taken using smartphones [19]. This number demonstrates the key role of smartphone for ordinary users’ photography. The camera and cellular (or wireless Internet) connections allow various apps and services to be used while users are on the go. These features provide a pervasive and ubiquitous infrastructure for more cost-effective apps for users to improve their life [20]. Innovative use of the smartphone cameras expands their functionalities to a bar- code scanner, document digitizing tool, translation tool, augmented reality tool, indoor positioning, and health monitoring device (as summarized in Table 3). Analysis of the functions of the smartphone camera in this paper focuses on what the devices can do to facilitate human in their daily activities. Even though smartphone penetration is increas- ing globally, the functions of the built-in camera have not been maximized. The users often use smartphone cameras for personal and social photography [19] and rarely use them for other purposes. Further, review on the other various functions of smartphone cameras is not yet available in the literature. Therefore, there is a need to explore the other functions of the smartphone cameras that can facilitate users in their daily activi- ties. 106 http://www.i-jim.org Paper—Mobile Phone Sensing using the Built-in Camera Table 3. The possible uses of smartphone cameras to support daily life activities. Types of ap- plication Example of studies Descriptions Barcode scan- ner [20-24] The use of the smartphone camera for capturing product barcode and its associated information. The information is used for a fur- ther recommendation to users or price comparison. Document dig- itization tool [25-30] The use of the smartphone camera for generating a digital repre- sentation of a document that works like the OCR document scan- ner. Translation tool [31, 32] The use of the smartphone camera to capture words or road signs and get their translation in different languages. Augmented re- ality tool [13, 33] The use of the smartphone camera as an input device to capture external elements and embed them in a modified or enhanced the physical view of a real environment. Indoor posi- tioning device [14, 15, 34, 35] The use of the smartphone camera to guide users to navigate within a building or room. Personal health monitoring [36-39] The use of the smartphone camera as a non-invasive technique for personal health monitoring. The modern smartphone cameras can be used as a portable barcode reader for scan- ning and translating the barcode image into the corresponding item or product identi- fier. Barcode is a common way of identifying items and products at the supermarkets, shopping malls, and grocery stores. It has been used widely since more than a couple of decades ago and demonstrates a well-established industrial standard [40]. The bar- code system can be integrated with smartphone technology enabling more useful ser- vices for the users [22]. The use of smartphone cameras for barcode scanning is becom- ing widespread as a result of the successful implementation of e-commerce services [41]. Mobile apps developed for a barcode scanner are programmed to interpret the barcode images and generate the barcode value which is used to identify the product from the database [21]. The operational procedure for barcode scanning apps is very straightforward. The users can simply turn the apps on and hold the smartphone camera over the barcode of the product [23]. The smartphone cameras will detect the barcode area and scan the barcode image. Then, barcode localization algorithms will decode the captured barcode images and produced the binary number of the barcode [40]. The barcode allows additional information about the product or items that can be retrieved from other connected electronic systems and databases [21]. The barcode scanning apps in smartphones provide automatic and precise input data of products [20] which conse- quently speeds the up searching process and minimize manual input errors. Barcode scanning using the smartphone cameras will be discussed in the subsequent section. Attempts have been made by software developers to extend the use of the digital camera and built-in smartphone cameras for document digitization [25, 26]. Document scanning is among the most downloaded mobile apps from the app stores [27]. The purpose of digitizing paper documents is mostly for sharing them with other parties and for personal documents keeping and archiving [28]. Digitization of printed documents using smartphone cameras benefits users as the device is portable and always available for domestic document scanning tasks when necessary. The use of the smartphone cam- eras can overcome the issue of bulky and difficult-to-maintain flatbed document scan- ners [26]. Although smartphone is unable to replace the scanners, document scanning iJIM ‒ Vol. 13, No. 2, 2019 107 Paper—Mobile Phone Sensing using the Built-in Camera apps installed on smartphones have the capability of performing similar optical charac- ter recognition (OCR) functions of the scanners. Documents captured using digital cam- eras can be degraded [30]. They usually suffer from low resolution, blur, and improper layout of the document image file [29]. However, the imitated OCR technology used in the smartphone cameras could eliminate the problem and produced a better result than the image file. The smartphone cameras read the characters on the papers and save them in a portable document format (pdf). The digitization process makes the text in the scanned documents searchable, and editable; the same way the flatbed scanners work. Using the same device, the scanned documents can be stored in smartphones, cloud storage, or shared with others immediately through readily-installed email or so- cial network apps. It is one of the smartphone functions that provides convenience to users especially when the document scanning facility is urgently needed, or the users are on the go and unable to locate flatbed scanners for immediate use. The smartphone cameras have another useful role it plays for users who travel to foreign countries. For travel, users mainly use smartphones for searching for infor- mation of the visited places from the Internet, make a reservation for accommodation, buy transportation tickets, and help users navigate from one place to another [42]. Apart from being used for information searching and navigation, the smartphone camera can act as a language translation tool. This translation tool is very useful to travelers as the app can read signage in foreign languages and translate into languages that the users understand [31, 43]. It works in a straightforward manner where the users turn the app on, hold the camera on the text or signage, and then receive the translated text or signage in a second. Smartphones are highly portable, always carried on the person, and easy to use, thereby making traveling activities in foreign countries easier. The smartphone translation tools work in the same way as the barcode scanning apps. Instead of recog- nizing the lines of the barcode, the apps recognize characters of the language and per- form the translation. The advancement in image recognition technology has brought in augmented reality language translation where the translated text is overlaying the orig- inal text on the same background [43]. Google Translate, Microsoft Translator, and WayGo are among the popular language translation tools making use of the smartphone cameras [44]. Augmented reality combines virtual objects into the actual three-dimensional envi- ronment and presents them in real time [13]. It brings virtual objects into the real world; thus, enhancing our visual and auditory experiences. For example, home interior design can be more interesting with many ideas and can be visualized using the augmented reality app such as Ikea Place [45]. Initially, users have difficulty in choosing and buy- ing new furniture as they could not visualize whether it is suitable for a room concerning size, design, color, and layout [46]. However, the augmented reality apps facilitate users by giving the real visualization of how the furniture fitted into the room before making any purchase. Augmented reality technology has also been applied for shopping. A survey by Dacko [47] reported that 272 mobile augmented reality apps for shopping were listed in Google Play Store in 2016. From the listed apps, there are many of them designed for everyday used such as Yates My Garden app that assists users in selecting the suitable plants and flower for their garden. Another example of everyday apps is CorfuAR [47, 48]; a travel guide app for the island of Corfu, Greece. It provides detail 108 http://www.i-jim.org Paper—Mobile Phone Sensing using the Built-in Camera information about the point of interests in the island to the tourists. Other than that, the smartphone cameras have also been used for augmented reality app for learning and educational games. With the growing market of smartphones, mobile augmented reality has attracted business operators to develop an augmented reality app as a complemen- tary tool to their products that could increase customers’ interest in buying them. Despite the augmented reality; the smartphone cameras have been used as an indoor position tool. The existing GPS technology on smartphones is not able to locate people accurately in an indoor environment due to signal attenuation effects of roofs and walls [34]. Therefore, another smartphone-based technology that is more suitable for an indoor environment is needed because locating a specific room in a large complex building is difficult and taking time. Without a proper guide, we tend to get frustrated and leave the building before getting what we want. Therefore, indoor positioning apps are essential to guide users more conveniently to reach the intended room or section in a building. Business owners of the supermarket and shopping mall look into indoor positioning apps that can assist their customers to find items on the shelf easily; hence will increase their sales indirectly [35]. One of the technologies for indoor positioning is visible light communication (VLC) where lights (e.g., manufactured by Philips and OSRAM) are being installed on the ceiling of the buildings, and the apps turn on the smartphone camera to pick up the frequency from the lights. Then the apps will help and guide the customers by giving direction to reach the intended store in the shopping mall or the aisle in the supermarket. This innovative project has been implemented in selected grocery stores and supermarkets such as Marc O’Polo stores in Switzer- land, EDEKA in Germany, Target and Walmart in the US, and Aswaaq in the UAE [49]. Indoor positioning using the smartphone camera is also useful for visually-impaired persons to navigate in indoor environments such as shopping malls, hospitals, muse- ums, and airports. A project named Path Recognition for Indoor Assisted Navigation with Augmented Perception (ARIANNA) [50] was developed and patented in Italy to provide a low-cost navigation system that works similarly as the tactile paving for the visually impaired pedestrian. ARIANNA system used strips that is easily adhered to the floor which is cheaper and faster than installing the tactile paving. When the smartphone camera is pointed on the strips, it will guide the visually impaired people to navigate within the building by following the vibrating signal whether to move for- ward, turn left or right. The use of smartphone cameras for indoor positioning and navigation increases in- dependence and social inclusion among visually-impaired persons. The smartphone cameras are also useful in everyday life for monitoring our health condition. The sensor on the smartphone camera can be used for personal health monitoring as reviewed by Lamonaca, et al. [38]. Kumar, et al. [39] performed an analysis of smartphone apps for blood pressure such as Blood Pressure (My Heart), BP Watch, and Real BP Calculator. The smartphone apps work in a very simple way where users simply place one of the fingers on the smartphone camera; then the apps measure their current blood pressure. An example of a smartphone app named HemaApp [36] was introduced as a way for non-invasive blood screening that can provide the users with the blood hemoglobin readings. A non-invasive tool is appealing for users (e.g., suffering from anemia) who iJIM ‒ Vol. 13, No. 2, 2019 109 Paper—Mobile Phone Sensing using the Built-in Camera require frequent blood hemoglobin readings to avoid puncturing the skin repeatedly [36, 37]. These apps are useful for our everyday use; however, medical experts claim that these apps require validation and clinical evaluation for their accuracy [39]. 4 Conclusion Modern smartphone comprises components including processor, memory, network interface and embedded sensors. The sensors that can be utilized to build apps in vari- ous domains that can facilitate human daily life activities. Unlike other types of smartphone sensors, the built-in camera has been underutilized. Apart for personal and social photography, smartphone camera has other potential uses as a barcode scanning tool, document digitizing tool, translation tool, augmented reality tool, indoor position- ing device, and health monitoring device. The review presented in this paper provides the ordinary smartphone users with information on the uses of the smartphone camera beyond the personal and social photography. Concerning the domain of mobile phone sensing, this paper could inspire other researchers to enhance or improve limitations found in each use of the smartphone camera. For example, studies are needed to im- prove the barcode image localization so that the algorithms use to capture the barcode could respond faster and give more accurate results. In the context of human-computer interaction, the use of the smartphone camera could have an impact on the users’ behaviors. A deep study could be established to provide evidence on the benefits and impact of the smartphone camera to a person or a group of people such as elderly or kids. 5 Acknowledgement The research is funded by the Ministry of Higher Education, Malaysia under the Trans-Disciplinary Research Grant Scheme (Ref: TRGS/2/2014/ UUM/01/3/4, UUM S/O Code:13170). 6 References V. Persson and J. Nouri, "A Systematic Review of Second Language Learning with Mobile Technologies," International Journal of Emerging Technologies in Learning (iJET), vol. 13, pp. 188-210, 2018. https://doi.org/10.3991/ijet.v13i02.8094 H. M. Alfawareh and S. Jusoh, "The Use and Effects of Smartphones in Higher Education," International Journal of Interactive Mobile Technologies (iJIM), vol. 11, pp. 103-111, 2017. https://doi.org/10.3991/ijim.v11i6.7453 N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. T. Campbell, "A survey of mobile phone sensing," IEEE Communications Magazine, vol. 48, pp. 140-150, 2010. https://doi.org/10.1109/MCOM.2010.5560598 E. Koukoumidis, M. Martonosi, and L. S. Peh, "Leveraging smartphone cameras for collab- orative road advisories," IEEE Transactions on Mobile Computing, vol. 11, pp. 707-723, 2012. https://doi.org/10.1109/TMC.2011.275 110 http://www.i-jim.org https://doi.org/10.3991/ijet.v13i02.8094 https://doi.org/10.3991/ijet.v13i02.8094 https://doi.org/10.3991/ijim.v11i6.7453 https://doi.org/10.3991/ijim.v11i6.7453 https://doi.org/10.1109/MCOM.2010.5560598 https://doi.org/10.1109/MCOM.2010.5560598 https://doi.org/10.1109/TMC.2011.275 Paper—Mobile Phone Sensing using the Built-in Camera R. Hildebrand, J. L. Hoffmann, E. Gillich, H. Dörksen, and V. Lohweg, "Smartphones as smart cameras-is it possible?," in Colloquium on Image Processing in Automation, Institute Industrial IT, 2012, pp. 1-8. W. Koo, "Usage of Smartphone Applications: A Descriptive Study of Top 100 US Retail- ers," International Journal of Interactive Mobile Technologies (iJIM), vol. 10, pp. 54-58, 2016. https://doi.org/10.3991/ijim.v10i3.5827 S. Kanchi, M. I. Sabela, P. S. Mdluli, and K. Bisetty, "Smartphone based bioanalytical and diagnosis applications: A review," Biosensors and Bioelectronics, vol. 102, pp. 136-149, 2018. https://doi.org/10.1016/j.bios.2017.11.021 T. Brinda and F. Braun, "Which Computing-Related Conceptions Do Learners Have About the Design and Operation of Smartphones? Results of an Interview Study," in Proceedings of the 12th Workshop on Primary and Secondary Computing Education, 2017, pp. 73-81. https://doi.org/10.1145/3137065.3137075 P. Daponte, L. De Vito, F. Picariello, and M. Riccio, "State of the art and future develop- ments of measurement applications on smartphones," Measurement: Journal of the Interna- tional Measurement Confederation, vol. 46, pp. 3291-3307, 2013. https://doi.org/10.10 16/j.measurement.2013.05.006 S. A. Hoseini-Tabatabaei, A. Gluhak, and R. Tafazolli, "A survey on smartphone-based sys- tems for opportunistic user context recognition," ACM Computing Surveys (CSUR), vol. 45, p. 27, 2013. https://doi.org/10.1145/2480741.2480744 X. Su, H. Tong, and P. Ji, "Activity recognition with smartphone sensors," Tsinghua Science and Technology, vol. 19, pp. 235-249, 2014. https://doi.org/10.1109/TST.2014.6838194 V. Kulyukin and T. Zaman, "Vision-based localization and scanning of 1D UPC and EAN barcodes with relaxed pitch, roll, and yaw camera alignment constraints," International Jour- nal of Image Processing (IJIP), vol. 8, pp. 355-383, 2014. R. F. Rahmat, F. Akbar, M. F. Syahputra, M. A. Budiman, and A. Hizriadi, "An Interactive Augmented Reality Implementation of Hijaiyah Alphabet for Children Education," in Jour- nal of Physics: Conference Series, 2018. Y. Li, Z. Ghassemlooy, X. Tang, B. Lin, and Y. Zhang, "A VLC Smartphone Camera based Indoor Positioning System," IEEE Photonics Technology Letters, 2018. https://doi.org/10.1109/LPT.2018.2834930 W. Elloumi, K. Guissous, A. Chetouani, R. Canals, R. Leconge, B. Emile, et al., "Indoor navigation assistance with a smartphone camera based on vanishing points," in International Conference on Indoor Positioning and Indoor Navigation (IPIN), 2013. https://doi.org/10.1109/IPIN.2013.6817911 K. Sankaran, M. Zhu, X. F. Guo, A. L. Ananda, M. C. Chan, and L.-S. Peh, "Using mobile phone barometer for low-power transportation context detection," in Proceedings of the 12th ACM Conference on Embedded Network Sensor Systems, 2014, pp. 191-205. https://doi.org/10.1145/2668332.2668343 D. A. Johnson and M. M. Trivedi, "Driving style recognition using a smartphone as a sensor platform," in Intelligent Transportation Systems (ITSC), 2011 14th International IEEE Con- ference on, 2011, pp. 1609-1615. https://doi.org/10.1109/ITSC.2011.6083078 T. Kindberg, M. Spasojevic, R. Fleck, and A. Sellen, "The ubiquitous camera: An in-depth study of camera phone use," IEEE Pervasive Computing, vol. 4, pp. 42-50, 2005. https://doi.org/10.1109/MPRV.2005.42 F. Richter. (2017). Digital Photography: Smartphones Cause Photography Boom. Available: https://www.statista.com/chart/10913/number-of-photos-taken-worldwide/ K. Dorman, M. Yahyanejad, A. Nahapetian, M. K. Suh, M. Sarrafzadeh, W. McCarthy, et al., "Nutrition monitor: A food purchase and consumption monitoring mobile system," in iJIM ‒ Vol. 13, No. 2, 2019 111 https://doi.org/10.1109/TMC.2011.275 https://doi.org/10.3991/ijim.v10i3.5827 https://doi.org/10.3991/ijim.v10i3.5827 https://doi.org/10.1016/j.bios.2017.11.021 https://doi.org/10.1016/j.bios.2017.11.021 https://doi.org/10.1145/3137065.3137075 https://doi.org/10.1145/3137065.3137075 https://doi.org/10.1145/2480741.2480744 https://doi.org/10.1145/2480741.2480744 https://doi.org/10.1109/TST.2014.6838194 https://doi.org/10.1109/TST.2014.6838194 https://doi.org/10.1109/LPT.2018.2834930 https://doi.org/10.1109/LPT.2018.2834930 https://doi.org/10.1109/IPIN.2013.6817911 https://doi.org/10.1109/IPIN.2013.6817911 https://doi.org/10.1145/2668332.2668343 https://doi.org/10.1145/2668332.2668343 https://doi.org/10.1109/ITSC.2011.6083078 https://doi.org/10.1109/ITSC.2011.6083078 https://doi.org/10.1109/MPRV.2005.42 https://doi.org/10.1109/MPRV.2005.42 https://www.statista.com/chart/10913/number-of-photos-taken-worldwide/ https://www.statista.com/chart/10913/number-of-photos-taken-worldwide/ https://www.statista.com/chart/10913/number-of-photos-taken-worldwide/ https://www.statista.com/chart/10913/number-of-photos-taken-worldwide/ Paper—Mobile Phone Sensing using the Built-in Camera Mobile Computing, Applications, and Services, ed Berlin Heidelberg: Springer, 2009, pp. 1-11. [21] D. Chai and F. Hock, "Locating and decoding EAN-13 barcodes from images captured by digital cameras," in 2005 Fifth International Conference on Information, Communications and Signal Processing, 2005, pp. 1595-1599. https://doi.org/10.1109/ICICS.2005.1689328 [22] E. Y. Daraghmi, C. F. Lin, and S. M. Yuan, "Mobile Phone Enabled Barcode Recognition for Preferences Monitoring," in Advances in Computer Science and Education Applications, ed Berlin Heidelberg: Springer 2011, pp. 297-302. [23] E. Dunford, H. Trevena, C. Goodsell, K. H. Ng, J. Webster, A. Millis, et al., "FoodSwitch: a mobile phone app to enable consumers to make healthier food choices and crowdsourcing of national food composition data," JMIR mHealth and uHealth, vol. 2, p. e37, 2014. https://doi.org/10.2196/mhealth.3230 [24] O. Gallo and R. Manduchi, "Reading 1D barcodes with mobile phones using deformable templates2011," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, pp. 1834-1843. [25] D. Doermann, J. Liang, and H. Li, "Progress in camera-based document image analysis," in Document Analysis and Recognition, 2003. Proceedings. Seventh International Conference on, 2003, pp. 606-616. https://doi.org/10.1109/ICDAR.2003.1227735 [26] J. Liang, D. Doermann, and H. Li, "Camera-based analysis of text and documents: a survey," International Journal of Document Analysis and Recognition (IJDAR), vol. 7, pp. 84-104, 2005. [27] F. Chen, S. Carter, L. Denoue, and J. Kumar, "SmartDCap: semi-automatic capture of higher quality document images from a smartphone," in Proceedings of the 2013 international conference on Intelligent user interfaces, 2013, pp. 287-296. https://doi.org/10.1145/2449396.2449433 [28] N. Nayef, M. M. Luqman, S. Prum, S. Eskenazi, J. Chazalon, and J. M. Ogier, "SmartDoc- QA: A dataset for quality assessment of smartphone captured document images-single and multiple distortions," in 2015 13th International Conference on Document Analysis and Recognition (ICDAR), 2015, pp. 1231-1235. https://doi.org/10.1109/I CDAR.2015.7333960 [29] F. Asad, A. Ul-Hasan, F. Shafait, and A. Dengel, "High Performance OCR for Camera- Captured Blurred Documents with LSTM Networks," in 2016 12th IAPR Workshop on Document Analysis Systems (DAS), 2016, pp. 7-12. https://doi.org/10.1109/DAS.2016.69 [30] P. Ye and D. Doermann, "Document image quality assessment: A brief survey," in 2013 12th International Conference on Document Analysis and Recognition (ICDAR), 2013, pp. 723-727. https://doi.org/10.1109/ICDAR.2013.148 [31] J. Du, Q. Huo, L. Sun, and J. Sun, "Snap and translate using windows phone," in 2011 International Conference on Document Analysis and Recognition (ICDAR), 2011, pp. 809- 813. [32] M. Lee, Kim, S. H., G. Lee, S. H. Kim, and H. J. Yang, "Correction for Misrecognition of Korean Texts in Signboard Images using Improved Levenshtein Metric," TIIS, vol. 6, pp. 722-733, 2012. https://doi.org/10.3837/tiis.2012.02.016 [33] F. Lamberti, F. Manuri, A. Sanna, G. Paravati, P. Pezzolla, and P. Montuschi, "Challenges, Opportunities, and Future Trends of Emerging Techniques for Augmented Reality-Based 112 http://www.i-jim.org https://doi.org/10.1109/ICICS.2005.1689328 https://doi.org/10.1109/ICICS.2005.1689328 https://doi.org/10.2196/mhealth.3230 https://doi.org/10.2196/mhealth.3230 https://doi.org/10.1109/ICDAR.2003.1227735 https://doi.org/10.1109/ICDAR.2003.1227735 https://doi.org/10.1145/2449396.2449433 https://doi.org/10.1145/2449396.2449433 https://doi.org/10.1109/DAS.2016.69 https://doi.org/10.1109/DAS.2016.69 https://doi.org/10.1109/ICDAR.2013.148 https://doi.org/10.1109/ICDAR.2013.148 https://doi.org/10.3837/tiis.2012.02.016 https://doi.org/10.3837/tiis.2012.02.016 Paper—Mobile Phone Sensing using the Built-in Camera Maintenance," IEEE Transactions on Emerging Topics in Computing, vol. 2, p. 2, 2014. https://doi.org/10.1109/TETC.2014.2368833 [34] M. Werner, M. Kessel, and C. Marouane, "Indoor positioning using smartphone camera," in IPIN, 2011, pp. 1-6. [35] Y. S. Kuo, P. Pannuto, K. J. Hsiao, and P. Dutta, "Luxapose: Indoor positioning with mobile phones and visible light," in Proceedings of the 20th annual international conference on Mobile computing and networking, 2014, pp. 447-458. https://doi.org/10.1145/2639108.2639109 [36] E. J. Wang, W. Li, D. Hawkins, T. Gernsheimer, C. Norby-Slycord, and S. N. Patel, "HemaApp: noninvasive blood screening of hemoglobin using smartphone cameras," in Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2016, pp. 593-604. https://doi.org/10.1145/2971648.2971653 [37] E. J. Wang, W. Li, J. Zhu, R. Rana, and S. N. Patel, "Noninvasive hemoglobin measurement using unmodified smartphone camera and white flash," in Engineering in Medicine and Biology Society (EMBC), 2017 39th Annual International Conference of the IEEE, 2017, pp. 2333-2336. https://doi.org/10.1109/EMBC.2017.8037323 [38] F. Lamonaca, G. Polimeni, K. Barbé, and D. Grimaldi, "Health parameters monitoring by smartphone for quality of life improvement," Measurement, vol. 73, pp. 82-94, 2015. https://doi.org/10.1016/j.measurement.2015.04.017 [39] N. Kumar, M. Khunger, A. Gupta, and N. Garg, "A content analysis of smartphone–based applications for hypertension management," Journal of the American Society of Hypertension, vol. 9, pp. 130-136, 2015. [40] O. Gallo and R. Manduchi, "Reading 1D barcodes with mobile phones using deformable templates," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, pp. 1834-1843, 2011. https://doi.org/10.1109/TPAMI.2010.229 [41] G. Sörös and C. Flörkemeier, "Blur-resistant joint 1D and 2D barcode localization for smartphones," in The 12th International Conference on Mobile and Ubiquitous Multimedia 2013, pp. 11-19. https://doi.org/10.1145/2541831.2541844 [42] D. Wang, Z. Xiang, and D. R. Fesenmaier, "Smartphone use in everyday life and travel," Journal of Travel Research, vol. 55, pp. 52-63, 2016. https://doi.org/10. 1177/0047287514535847 [43] V. Fragoso, S. Gauglitz, S. Zamora, J. Kleban, and M. Turk, "TranslatAR: A mobile augmented reality translator," in Applications of Computer Vision (WACV), 2011 IEEE Workshop on, 2011, pp. 497-502. https://doi.org/10.1109/WACV.2011.5711545 [44] A. Kroulek. (2018). 15 Powerful Translation Apps and Devices for Travelers in 2018. Available: http://www.k-international.com/blog/translation-apps-2016/ [45] A. Syrkett. (2017). Ikea and Apple team up on augmented reality home design app: The ‘Ikea Place’ app lets shoppers virtually test drive furniture. Available: https://www.curbed.com/2017/9/12/16297382/apple-ikea-place-augmented-reality-home- design [46] W. Viyanon, T. Songsuittipong, P. Piyapaisarn, and S. Sudchid, "AR Furniture: Integrating Augmented Reality Technology to Enhance Interior Design using Marker and Markerless tracking," in Proceedings of the 2nd International Conference on Intelligent Information Processing, 2017, p. 32. https://doi.org/10.1145/3144789.3144825 iJIM ‒ Vol. 13, No. 2, 2019 113 https://doi.org/10.1109/TETC.2014.2368833 https://doi.org/10.1109/TETC.2014.2368833 https://doi.org/10.1145/2639108.2639109 https://doi.org/10.1145/2639108.2639109 https://doi.org/10.1145/2971648.2971653 https://doi.org/10.1145/2971648.2971653 https://doi.org/10.1109/EMBC.2017.8037323 https://doi.org/10.1109/EMBC.2017.8037323 https://doi.org/10.1016/j.measurement.2015.04.017 https://doi.org/10.1016/j.measurement.2015.04.017 https://doi.org/10.1109/TPAMI.2010.229 https://doi.org/10.1109/TPAMI.2010.229 https://doi.org/10.1145/2541831.2541844 https://doi.org/10.1145/2541831.2541844 https://doi.org/10.1109/WACV.2011.5711545 https://doi.org/10.1109/WACV.2011.5711545 https://www.statista.com/chart/10913/number-of-photos-taken-worldwide/ https://www.statista.com/chart/10913/number-of-photos-taken-worldwide/ https://www.statista.com/chart/10913/number-of-photos-taken-worldwide/ https://www.statista.com/chart/10913/number-of-photos-taken-worldwide/ http://www.k-international.com/blog/translation-apps-2016/ http://www.k-international.com/blog/translation-apps-2016/ https://www.curbed.com/2017/9/12/16297382/apple-ikea-place-augmented-reality-home-design https://www.curbed.com/2017/9/12/16297382/apple-ikea-place-augmented-reality-home-design https://doi.org/10.1145/3144789.3144825 Paper—Mobile Phone Sensing using the Built-in Camera [47] S. G. Dacko, "Enabling smart retail settings via mobile augmented reality shopping apps," Technological Forecasting and Social Change, vol. 124, pp. 243-256, 2017. https://doi.org/10.1016/j.techfore.2016.09.032 [48] CorfuAR. (2018). Welcome to CorfuAR website! Available: http://www.corfuar.com/ [49] R. Leeming. (2017). Ten retail indoor positioning projects you need to know. Available: http://luxreview.com/article/2017/03/ten-retail-indoor-positioning-projects-you-need-to- know [50] D. Croce, L. Giarré, F. La Rosa, E. Montana, and I. Tinnirello, "Enhancing tracking performance in a smartphone-based navigation system for visually impaired people," in Control and Automation (MED), 2016 24th Mediterranean Conference on, 2016, pp. 1355- 1360. https://doi.org/10.1109/MED.2016.7535871 7 Authors Norliza Katuk is a senior lecturer at School of Computing, Universiti Utara Malay- sia. Her research interests cover multidisciplinary areas including information security and privacy, web and mobile technology, disaster management, and human-computer interaction. She is an editorial board member of Journal of Information and Communi- cation Technology. Nur Haryani Zakaria is a senior lecturer at School of Computing, Universiti Utara Malaysia. Her research interests cover areas related to usable security and privacy, in- formation security and human-computer interaction. She is an editorial board member of Journal of Information and Communication Technology. Ku Ruhana Ku-Mahamud is a professor in the Department of Computer Science, School of Computing, Universiti Utara Malaysia. Her research interests include computer systems performance modeling, ant colony optimization and intelligent agent. She is also chief editor of the Journal of Information and Communication Tech- nology and associate editors to several other refereed journals. Article submitted 06 November 2018. Resubmitted 27 December 2018. Final acceptance 16 January 2019. Final version published as submitted by the authors. 114 http://www.i-jim.org https://doi.org/10.1145/3144789.3144825 https://doi.org/10.1016/j.techfore.2016.09.032 https://doi.org/10.1016/j.techfore.2016.09.032 http://www.corfuar.com/ http://www.corfuar.com/ http://luxreview.com/article/2017/03/ten-retail-indoor-positioning-projects-you-need-to-know http://luxreview.com/article/2017/03/ten-retail-indoor-positioning-projects-you-need-to-know http://luxreview.com/article/2017/03/ten-retail-indoor-positioning-projects-you-need-to-know https://doi.org/10.1109/MED.2016.7535871 https://doi.org/10.1109/MED.2016.7535871 https://doi.org/10.1109/MED.2016.7535871