Format Template

Vol. 5, No. 2 | July – December 2021

SJCMS | P-ISSN: 2520-0755| E-ISSN: 2520-0755 | Vol. 5 No. 2 July – December 2021

Analyzing Usability of Mobile Banking Applications in

Pakistan

Hafiz Abid Mahmood Malik1, Abdulhafeez Muhammad2, Usama Sajid2

Abstract:

Usability is a key factor in the quality of the product, which includes ease of use, user satisfaction

and the ability of the user to quickly understand the product without practice. As smartphone

usage increases, most organizations have shifted their services to mobile applications, such as

m-banking. Most of the people uses banking services but hesitate to use m-banking due to

complex interfaces. Usability researchers concentrate on the value of design simplicity so that

users can perform a particular task with satisfaction, efficiency, and effectiveness. If a mobile

app lacks one of these usability features, users may get confused while using the app. This

research examines the key usability issues in existing m-banking after checking the usability

satisfaction level through System Usability Scale. To compare and highlight a number of

usability issues, the researcher used two types of usability evaluation method 'User Testing' and

'Heuristic Evaluation'. In heuristic evaluation expert users used two M-banking apps i.e., Bank

of Punjab (BOP) and Muslim Commercial Bank/ Islamic Bank (MIB) to evaluate them against

Neilson 10 heuristics and extract the usability issues in apps. The user testing is then performed

by novic1e users which includes tasks (translated from extracted problems by heuristic

evaluation). After completion on whole testing users filled the post-test SUS’s questionnaire.

The result shows that the overall success rate of the tasks was 83%, SUS score was 77 and overall

relative time-based efficiency very 54.2%. The expert evaluators found 83% minor errors and

17% major errors. The finding of this paper shows usability problems and recommendations are

provided to increase the usability of mobile banking applications at the end of this paper.

Keywords: Usability testing; Think Aloud; System usability scale; User testing; Mobile Banking

1. Introduction

In the major banking industry, the
integration of information technology has
created significant changes. Banks compete
closely and try to attract more customers by
facilitating them with more and more facilities.
Mobile business is booming and its consumers
are growing steadily. The main reason for this
is that inexpensive telecommunications
services are available.

1 Faculty of Computer Studies, Arab Open University, Bahrain
2 Department of Computer Science, Bahria University, Lahore, Pakistan

Corresponding Author: [hafiz.malik@aou.org.bh]

The definition of m-banking is like a
payment via mobile. M-banking provides
banking clients the opportunity to send SMS
notifications such as deposits and withdrawals,
check account status, credit card details and
provide account operational information. It
also provides the user with the facility to pay
their utility bills, transfer funds, etc [1].

The challenge today is to develop m-
banking that can easily meet all users ' needs.

mailto:hafiz.malik@aou.org.bh

Hafiz Abid Mahmood Malik (et al.), Analysing Usability of Mobile Banking Applications in Pakistan (pp. 25 - 35)

Sukkur IBA Journal of Computing and Mathematical Science - SJCMS | Vol. 5 No. 2 July – Dec 2021

User design involves specific challenges: the
lives, needs and expectations of these users in
developing countries such as Pakistan are
likely to vary considerably from the
perceptions of the designer. In order to avoid
any inconsistencies, the design of m-banking
must be developed in partnership with primary
users and must require a thorough study of their
experiences and problems. The end-user can
also help to continually test and finish projects
throughout the design process by identifying
design flaws.

In addition to the design of these systems
effectiveness, efficiency and satisfaction are
considered important factors in the application
design. The problem that is emerging in the
computer science community is that the
usability of applications is overlooked by many
developers which causes problems [2]. The
need for enhancing usability is therefore very
important, and numerous researchers in this
field have shown that the usability of software
is affected by different factors. Usability is an
important factor in the production of high-
quality and usability products, such as websites
and mobile applications.

The main purpose of interface design is the
usability of a good system. The various
usability models have shown that by examining
them, applications that have a high-quality
interface can be created with great help.

1.1. Usability

ISO 9241-11 states that the extent of
usability is to be used by a specific user to
attain the specific objectives in a specific
context of application in terms of effectiveness,
efficiency and satisfaction [3]. Learning and
ease of use is a created entity for people. The
subject you use may be a program, a web-based
system, a tool, a procedure or anything else
with which you can interact. A series of recent
studies solely focused on mobile usability with
the occurrence and the swift acquisition of
Smartphone tech. The physical restrictions on
cell phones and wireless networks mean that
the right research method must be carefully
chosen and the possible contextual factors to be
known should be reduced if they are not central
to design and mobile usability studies. The

assessment of product development and user
experience is critical to the way in which the
process model, product development and final
outcome have been established. The usability
and experience of the user are distinct, as they
also consider the usefulness. By analyzing
usability, we can better visualize aspects of
software architecture usability before
implementation. Then, it can help you identify
the user interface's 3 major components. “The
required components”, “The user-friendly
components” and “The components to run
them”. Mobile usability involves several
mobility-related challenges such as: mobile
frames, networks, various resolutions, small
screen sizes, and capacity and limited
processing capabilities and processing inputs.
Features of usability make the product or
system usable. In all other, the user must
possess both subjective and objective
experience if a system is to be used.

1.2. Literature Review
In the banking sector, m-banking is a

vibrant topic because rapid technological
changes increase competence. In this section,
the related research literature helps to identify
the variables related to the problem referred to
earlier in the study.

Kaikkonen tested mobile applications'
usability in his study. He performed two
different tests, the first test was conducted in
the laboratory and the second test conducted in
natural environment of users. He found 22
different usability issues in mobile applications
under observation [4].

There are four different application
categories compared by Ryan and Gonsalves.
They consist of the different types of
applications based on web-based PC, device-
based PC, mobile web and mobile devices. The
worst performance has shown by mobile web-
based applications. The low page display speed
is the reason. The fact is data provided from the
web is low. The speed of low-page viewing
adversely affects usability. Ryan and
Gonsalves ' analysis could lead to a significant
gap in the functionality of both a mobile
application and a PC application with the same
features and capabilities [5].

Hafiz Abid Mahmood Malik (et al.), Analysing Usability of Mobile Banking Applications in Pakistan (pp. 25 - 35)

Sukkur IBA Journal of Computing and Mathematical Science - SJCMS | Vol. 5 No. 2 July – Dec 2021

The literature reviewed reveals that the
usability of a user interface is not one "single
dimension" property. Many usability attributes
need to be considered and measured. Shackel
proposed four-dimensional characteristics that
impact product acceptance: efficacy, ability to
learn, adaptability and attitude [6]. Siti Elliyana
and Puspita Kencana Sari carried out a study on
internet banking website of Mandiri bank of
Indonesia with Nielsens's usability heuristics.
The method of research used was a descriptive
analysis, with hundred participants
participating through questionnaires.
According to this study, Mandiri internet
banking does not fully implemented all
heuristics. Heuristics which were judged not
well, those were; error prevention, flexibility
and efficiency of use, and aesthetic and
minimalist design [2].

Adane kebkab used qualitative method
(heuristic evaluation) to measure usability of
banking website. He found different usability
issues mentioned by experts. He also suggested
ways to improve the usability of banking
website by using user centered approach [7]. A
method that examines the usability of mobile
apps and recognizes the possibility of usability
problems has been developed by Biel Grill and
Gruhn. The SATURN method consists of five
activities. These activities include the context
of analysis, determination of scenarios,
evaluation of scenarios, interpretation, review
and tools. Biel Grill and Gruhn have used the
mobile SATURN model and observed major
usability problems [8].

Zereh Lalji and Judith Good studied the
design of a mobile device for illiterate users.
While the study adopted an incremental and
UCD approach. They have tried to explain how
the results from their study can be beneficial to
non-traditional users in the fields of design [9].
Victor Ndako Adama and Ibrahim Shehi Shehu
develop a prototype on m-banking for novice
users on the basis of recommendations and
guidelines found in literature. They involved
novice users and tested their prototypes. The
results showed that the satisfaction level of
users increased with new prototype [10].

Bernhaupt provided a set of the 'classical'
approaches and adds several theoretical

innovations in the field of mobile devices and
applications to test usability. He recommends
incorporating both forms of field assessment
and standard laboratory testing to
accommodate many stages of the UCD and
development procedure [11]. Amin Babazadeh
Sangar presented the smart banking model to
enhance the usability of mobile software. In
this respect, four new "visibility," "design,"
"navigation" and "compatibility" factors were
achieved. The proposed model was presented
in accordance with the factors in the earlier
studies and obtained factors from his research.
They had created an application for a bank
based on this model, following the proposed
usability pattern by increasing the level of
satisfaction [12].

Azham Hussain has shown a range of
usability measures to assess m-banking’s
usability. Measurement and dimension have
been produced through systematic literature
review in the relevant previous studies [13].
Fatih proposed a model that helps solve
complex problems in the evaluation process of
M-banking services and improves the
performance of M-banking operations [14].

2. Methods and Materials
To achieve the goals of this paper, in this

study the researcher selected different mobile
banking apps running in Pakistan to analyze the
usability of these app and to check the issue
faced by the diversity of mobile banking users
so that the researcher can give
recommendations to increase usability of
mobile banking applications.

2.1. Evaluation Methods and Approaches

Following two usability evaluation
methods in this research work.

2.1.1. Heuristic Evaluation
Heuristic evaluation is the evaluation

method. It was formulated by Nielsen [15],
based on a number of guiding principles of
usability or ' heuristics'. It can be described as
an exercise that involves a number of experts
to make use of heuristics to identify usability
issues of an interface with less effort and in
short period of time. "Heuristic assessment is a
widely recognized diagnostic approach for

Hafiz Abid Mahmood Malik (et al.), Analysing Usability of Mobile Banking Applications in Pakistan (pp. 25 - 35)

Sukkur IBA Journal of Computing and Mathematical Science - SJCMS | Vol. 5 No. 2 July – Dec 2021

critical usability challenges and is common in
various disciplines" stated by Magoulas [16].

2.1.2. Approach
In this research 5 experts were chosen to

evaluate the usability of the mobile banking
app of BOP and MIB. Evaluators were given a
set of 30 questions, 3 against each Nielsen 10
heuristics. Evaluators scored each sub-criterion
with a 5-point Likert scale starting from
strongly agree (0) to strongly disagree (4). Each
heuristic violation identified needed to be
explained by the evaluators.

After completion of heuristic evaluation,
the duplicate usability violations were
consolidated and combined. Finally, a severity
rating was carried out to assess the extent of
usability problems. The usability issue was
assessed by each evaluator using the five-point
severity rating scale of Jakob Nielsen [17].
Table 1 shows the scale of the severity rating.

TABLE 1. Nielsen Severity Rating for
Usability Problems

0 This is not a usability problem at all

1 Cosmetic problem only – does not need

to be fixed unless extra time is available

on the project

2 Minor usability problem - fixing this

should be given low priority

3 Major usability problem - important to fix,

should be given high priority

4 Usability catastrophe - imperative to fix

2.1.3. User Testing
In this method of testing, different

novice users were given typical tasks
(translated from extracted problems by
heuristic evaluation) on mobile banking app.
These tasks were consisting on main
functionality of m-banking app and the
researcher used the results to see how the UI of
m-banking app assists users in their tasks. Each
task given to the user, had precise goal to test
the usability goals/ principles on which the

researcher evaluate the system. User testing
includes many methods like Coaching Method,
Co-Discovery Learning Method, Performance
Measurement, Questions asking Protocols,
Thinking Aloud Protocol etc. We used “Think
Aloud” Protocol.

• Think Aloud
“Think Aloud” protocols, in this a user

worked on an interface and we encouraged
them to "think aloud" say what they think and
wonder, at every moment. “Think Aloud”
conventions are of specific esteem since they
center on a user’s issues. This allows researcher
to obtain a detailed picture of the behavior of
users that can be analyzed to highlight usability
issues. This protocol helped to record the user's
time to perform tasks and to calculate the
efficiency. It also helped researchers to monitor
user satisfaction level and application
effectiveness under observation.

List of Tasks

The following tasks were performed by
the users in user testing.

• Balance checking

• Funds Transfer within bank

• Funds Transfer outside the bank

• Mobile Top-ups

• Bill Payments

• Deactivate Card

• Update Settings

• Approach

Users were requested to fill Pre-Test survey to
know essential data about candidate with the
goal that researcher can assess about the
candidates i.e. novice, moderate or master in
this area. Then method of test and what is
expected from them was presented, together
with the idea of "Think Aloud" convention and
the assignments they should performed on the
Mobile Banking Application. This ease-of-use
testing directed so that just a single candidate
at any given moment could play out the
examination, so as to empower the cautious
observing of their activities and conduct. At the
end user have to fill Post-Test survey in which

Hafiz Abid Mahmood Malik (et al.), Analysing Usability of Mobile Banking Applications in Pakistan (pp. 25 - 35)

Sukkur IBA Journal of Computing and Mathematical Science - SJCMS | Vol. 5 No. 2 July – Dec 2021

questions regarding task which facilitates the
research objective given. Each question has a
rating 0 to 5 calculating the satisfaction level of
candidate from “Strongly Disagree” to
“Strongly Agree”. At the end, we calculated the
satisfaction level, effectiveness and efficiency
of task performed.

2.1.4. Questionnaire
This study contains two questionnaires:

pre-test and post-test. The pre-tests were
utilized to collect demographic information
from users, both for "Heuristic Evaluation" and
"User Testing". The post-test survey was used
for two purposes in the user test: 1) the user's
viewpoints were obtained, and 2) the user's
disappointment and/or satisfaction level
calculated, when they using the "Mobile
Banking App". As a post-test questionnaire,
the researcher used SUS questionnaire. A
simple and extensively-used 10 questions
survey is the "System Usability Scale" (SUS)
which provides qualitative calculations of
system usability. In this study users were asked
to rate the 10 declarations half of their
agreement or disagreement positively, and half
of them negative about the Mobile Banking
App. For presenting outcomes, this study uses
the scoring model that transforms a single score
based on Brooke's standard measuring method
into a single score.

• Sample of Candidates

In this research work, different types of
users were chosen i.e. Novice and expert users.
The aim was to measure the difference in
performance among the two groups, which was
achieved by observing attentively and closely
the behavior of users throughout the user
testing. 18 users were chosen for user testing
out of which 9 were novice and 3 were
experienced and 6 were moderate level users
who don’t use mobile banking app frequently.
Users were also chosen from different age
groups to achieve the research goals.

Below are the samples of users we chose
for my testing.

•Businessman (Who frequently use App)
→ Expert level User

•Businessman (Who use only few times

App) → Moderate level User

•Businessman (Who never user App) →
Novice User

•Housewives

•Students

•Professionals

TABLE 2. Gender and Experience User’s
Group Distribution

User Type

User

Gender
Tota

l Mal

Femal

Novice 9 7 2

18
Moderate 6 4 2

Experience

3 2 1

TABLE 3. Gender and Age Wise User’s
Group Distribution

Gender

Age Class

20-

26-

31-

Above

Male 3 2 4 4

Female 1 2 1 1

3. Results and Analysis
The results from the two methods

("Heuristic Evaluation" and "User Testing")
that have been used in this experiment are
shown in this section. It begins by summarizing
each user's problems during the "user testing"
and calculating usability metrics including
satisfaction, efficiency and success rate of each
task. Then researcher calculates the usability of
the mobile banking app with the help of the
system usability scale (SUS).

Hafiz Abid Mahmood Malik (et al.), Analysing Usability of Mobile Banking Applications in Pakistan (pp. 25 - 35)

Sukkur IBA Journal of Computing and Mathematical Science - SJCMS | Vol. 5 No. 2 July – Dec 2021

3.1. User testing Results

3.1.1. Tasks Analysis
It is not essential to evaluate all of the tasks

carried out by the participants in great detail
during the evaluation process. A
comprehensive examination will be enough for
some of the tasks most successfully performed.
A more comprehensive explanation would be
more suitable for the usability problems as well
as other challenging tasks. The concise work
needs to be scanned carefully in order to ensure
they are a true example of mobile banking.
Each of the tasks in this section is examined
and analyzed to find out the possible causes of
the results of the work performed by the
participants.

Fig. 1a. Home page

• Balance checking
This task was the easiest task among all,

almost every user was successful except one

novice female user. Data collected via “Think
Aloud” protocol found that it was the easiest
task for almost all users and completed within
few seconds after login. Most of the Mobile
Banking Apps show their balance in home page
or it is easy to navigate account balance. To
demonstrate that we took screenshots form
Bank of Punjab’s mobile banking application
for reference at figure 1(a,b).

Fig. 1b. Balance Check

3.1.2 Bill Payments
This task was easy for those users who

already were paying bills from their apps so
they took not much time to complete this task.
For those users who have never paid their bills
through app, first needed to add their Bill
before paying it. Adding bill details took some
time for some users but almost all of them were
successful only 5 persons were unsuccessful
because they did not find from were to add bill
details. This is because few apps have different
screens to add bills so some users find
difficulty to navigate (MIB app have this

Hafiz Abid Mahmood Malik (et al.), Analysing Usability of Mobile Banking Applications in Pakistan (pp. 25 - 35)

Sukkur IBA Journal of Computing and Mathematical Science - SJCMS | Vol. 5 No. 2 July – Dec 2021

problem). While most of the apps give facility
to add bill details on same page if bill details
not added. Below print screen Figure 2 shows
add bill option have on same screen of bill
payments while Figure 3 shows no options
found in bill payment screen to add bill details.

Fig.2. Bill payment and add bill

This task was also easy for those users who
already were recharge their mobile balance
from their apps so they took not much time to
complete this task. For those users who have
never recharge their mobile balance though
app, first needed to add their network provider
details to recharge their balance. Adding details
took some time for some users but almost all of
them successful only few were (6 persons)
unsuccessful because they did not find from
were to add bill details. This is because few
apps have different screens to add bills so some
users find difficulty to navigate (e.g. MIB app
have this problem). While most of the apps
give facility to add bill details on same page if
bill details not added. Above print screen
Figure 2 shows add bill option have on same
screen of bill payments while Figure 3 shows
no options found in bill payment screen to add
bill details.

Fig.3. Absent of add bill option

3.1.4 Funds Transfer within bank
During Funds transfer tasks expert users

were in their comfort zone, because they had
done this so many times before while moderate
or novice users find it a bit difficult to complete
this task and took 6 to 10 minutes to complete
this task. Through “Think Aloud” protocol we
observed that most of the users find difficulty
to check if their funds transferred successfully
or not because of poor feedback from mobile
apps. Few of the users also find difficult to
navigate through the app to add beneficiary’s
details. Some novice users also complain about
the name conventions used in some apps was
not easy to understand for them. From Figure 4
we can see the process of funds transfer and
add beneficiaries.

Hafiz Abid Mahmood Malik (et al.), Analysing Usability of Mobile Banking Applications in Pakistan (pp. 25 - 35)

Sukkur IBA Journal of Computing and Mathematical Science - SJCMS | Vol. 5 No. 2 July – Dec 2021

3.1.3 Mobile Top-ups

3.1.5 Update Settings (e.g. change

password)
This task was also easy for both users

(expert or novice) because update setting is
common operation/ term used in mobile app or
web apps. Those users who are using any app
or maintaining any account in any app or
website were familiar to this task. We asked
them to change their passwords most of the
users were done it within 2 to 3 minutes only 2
were unsuccessful in doing this task.

Fig.4. Fund transfer

3.1.6 Overall Success Rate

The following Table 4 shows the no of
participants who have successfully performed
each task on the mobile banking apps.
Although majority tasks have been
successfully completed within the expected
completion period or approximately, some
have been found hard [18]. Nielsen [10]
defines the "percentage of tasks that the user
completes correctly." This is the rate (i.e.
success rate). For calculating the success rate
Nielsen employs the following formula:

Success Rate =
𝑠𝑢𝑐𝑐𝑒𝑠𝑠𝑓𝑢𝑙 𝑡𝑎𝑠𝑘 + (𝑝𝑎𝑟𝑡𝑖𝑎𝑙𝑙𝑦 𝑠𝑢𝑐𝑐𝑒𝑠𝑠𝑓𝑢𝑙 𝑡𝑎𝑠𝑘) ∗ 0.5

𝑇𝑜𝑡𝑎𝑙 𝑛𝑜 𝑜𝑓 𝑡𝑎𝑠𝑘𝑠

TABLE 4. Success rate of tasks performed

Tasks Gender
Age

Class

Novice/

Experience
Successful

Partially

Successful
Failed

Success

Rate

Balance

Checking

Males

and 5

Females

20 to

above

Both

18 0 0 100 %

Bill

Payments
13 1 4 75 %

Mobile

Top-ups
12 3 3 75 %

Funds

Transfer
12 2 4 72%

Update

Password
16 2 0 94%

Hafiz Abid Mahmood Malik (et al.), Analysing Usability of Mobile Banking Applications in Pakistan (pp. 25 - 35)

Sukkur IBA Journal of Computing and Mathematical Science - SJCMS | Vol. 5 No. 2 July – Dec 2021

Overall

Success

Rate

83.2%

3.1.7 Overall Relative Time based
Efficiency

The overall relative time-based Efficiency
of a product defined as the ratio of effective
users' work time to all users' work time. Many
experienced users had the least time to perform
most of the tasks effectively while novice users
took more time complete/ partially complete
the task. Table 5 shows the user testing results
for calculating overall relative table-based
efficiency.

TABLE 5. Overall relative time-based
efficiency

Tasks Successful Failed

Balance Checking 18 0

Bill Payments 13 5

Mobile Top-ups 12 6

Funds Transfer 12 6

Update Password 16 2

Overall Relative

Time based

Efficiency

54.2%

TABLE 6. System usability score

Participant Q1 Q 2 Q 3 Q 4 Q 5 Q6 Q 7 Q 8 SU Score

P1 4 5 2 2 3 4 4 2 40.0

P2 2 4 4 1 2 5 2 3 32.5

P3 3 2 3 3 2 3 2 3 37.5

P4 2 4 4 3 4 4 4 4 37.5

P5 3 3 2 2 2 2 2 3 37.5

P6 5 4 4 4 3 2 3 5 40.0

P7 1 3 3 3 1 2 3 5 27.5

P8 5 3 2 2 3 4 5 5 42.5

P9 5 2 2 3 3 2 5 5 47.5

P10 4 3 5 3 3 3 3 5 42.5

Average 38.5

3.1.8 User Satisfaction by SUS

After user testing candidates filled SUS
questionnaire which had 10 questions with
Likert scale. From the results researchers have
found that the overall average usability score is
38.5 which is below the satisfaction level [19]
[20]. Table 6 shows the System usability score
in detail of each participant.

3.1.9 Heuristic Evaluation Results

For the m-banking apps, the researcher has
established a usability checklist for heuristic
evaluation. Table 7 shows that the ratio of
major error highlighted by expert evaluators is
7%, minor error’s ratio is 40% and cosmetic
error were 54%.

Hafiz Abid Mahmood Malik (et al.), Analysing Usability of Mobile Banking Applications in Pakistan (pp. 25 - 35)

Sukkur IBA Journal of Computing and Mathematical Science - SJCMS | Vol. 5 No. 2 July – Dec 2021

TABLE 7. Heuristic evaluation results

Problem Type Cosmetic Minor Major Catastrophic Total

Visibility of system status 4 5 2 0 11

Match between system and the real

world

5 7 1 0 13

User control and freedom 4 4 0 0 8

Consistency and standards 7 4 1 0 12

Error prevention 6 5 1 0 12

Recognition rather than recall 4 3 0 0 7

Flexibility and efficiency of use 5 2 0 0 7

Aesthetic and minimalist design 3 2 0 0 5

Helps users recognize, diagnose,

and recover from errors

6 3 0 0 9

Help and documentation 5 2 0 0 7

No of problems discovered 49 37 5 0

91 Percentage of problems

discovered

54% 40% 6% 0%

4. Discussions and recommendations

This segment shows users and expert’s
feedback on the challenges experienced during
the two experiments, and provides a set of
recommendations. The results of the test show
that the novices had significantly more
difficulty than the experienced users in
executing certain tasks, but they were able to
cope with that with time.

User satisfaction is among the main
components of usability in order to enhance the
usability of interactive mobile banking
applications. When designing such a program,
one should bear in mind not only for first-time
users but also the retention of existing app
users. As observed during User Testing, most
of the users faced navigations problems,
Feedback issues and naming convention’s
related issues. Few faced efficiency issues like
response time.

Some suggestions for improving mobile
banking applications are provided below.

• Use easy terminologies which are
understandable for users.

• Enhance visibility of the status of
transactions especially in case of
transferring funds or paying bill.

• Provide help to novice users in case
they mistake by proper feedback.

• Provide functionality of adding
beneficiary details on same funds
transfer page in case of the beneficiary
not added.

• Improve navigation so that user is able
to understand where he is and what to
do next.

• Don’t make interfaces a mess, keep it
simple and provide only necessary
information.

• FAQ and help button should be visible
to the user.

• Title of each page should be displayed
at the top.

5. Conclusion and Future work

The main objective of this study was to
recognize and inspect the usability issues of the
m-banking applications to enhance its

Hafiz Abid Mahmood Malik (et al.), Analysing Usability of Mobile Banking Applications in Pakistan (pp. 25 - 35)

Sukkur IBA Journal of Computing and Mathematical Science - SJCMS | Vol. 5 No. 2 July – Dec 2021

interaction with users. SUS survey was used to
check the satisfaction level of a user which was
below the average. Later, usability testing and
heuristic evaluation showed that Mobile
Banking apps are significantly harder for
beginners than for experienced users. Based on
the experiment conducted and expert opinions,
recommendations are provided for UX/UI
developers and designers. Banking sectors
should follow the design principles of usability
while designing apps in order to achieve best
possible user satisfaction, effectiveness and
efficiency of the system. Future work can be
undertaken to evaluate the user interface of the
m-banking application using combining
multiple techniques, based on these integrated
usability guidelines and their impact on the
usability of m-banking applications. Low
fidelity and high-fidelity prototypes can be
developed to get better results. Multi-Criteria
Decision analysis like AHP, Fuzzy AHP and
TOPSIS techniques can also be used to
prioritize m-banking usability issues in future
work. Research on the design of m-banking
applications for disabled or blind users may
also be carried out in future work.

REFERENCES

[1] R. Santhanam and R. Natesan, “Usability
testing to improvise online-banking
experience,” 4th Annu. Int. Softw. Test. Conf.
India, pp. 1–13, 2004.

[2] S. Elliyana and P. K. Sari, “Usability Analysis
of Internet Banking System Based on User ’ s
Perception Usability Analysis of Internet
Banking System Based on User ’ s
Perception,” no. January 2014, 2015.

[3] I. Standard, “Iso 9241-11,” vol. 1998, 1998.

[4] A. Kekäläinen, A. Kaikkonen, A. Kankainen,
M. Cankar, and T. Kallio, “Usability Testing
of Mobile Applications: A Comparison
between Laboratory and Field Testing,” J.
Usability Stud., vol. 1, no. 1, pp. 4–16, 2005.

[5] C. Ryan and A. Gonsalves, “The effect of
context and application type on mobile
usability: An empirical study,” Conf. Res.
Pract. Inf. Technol. Ser., vol. 38, pp. 115–124,
2005.

[6] B. Shackel, “Usability - Context, framework,
definition, design and evaluation,” Interact.
Comput., vol. 21, no. 5–6, pp. 339–346, 2009.

[7] A. K. Kassaye, “I Mproving Usability of
Banking Websites – B Y Implementing User -
Centered,” 2012.

[8] B. Biel, T. Grill, and V. Gruhn, “Exploring the
benefits of the combination of a software
architecture analysis and a usability evaluation
of a mobile application,” J. Syst. Softw., vol.
83, no. 11, pp. 2031–2044, 2010.

[9] Z. Lalji and J. Good, “Designing new
technologies for illiterate populations: A study
in mobile phone interface design,” Interact.
Comput., vol. 20, no. 6, pp. 574–586, 2008.

[10] R. J. Victor Adama, Ibrahim Shehu , Solomon
Adepoju, “Towards Designing Mobile
Banking User Interfaces for Novice Users,”
Springer Int. Publ. AG 2017, vol. 2, no. 2011,
pp. 255–261, 2017.

[11] R. Bernhaupt, K. Mihalic, and M. Obrist,
“Usability Evaluation Methods for Mobile
Applications,” Handb. Res. User Interface
Des. Eval. Mob. Technol., pp. 745–758, 2011.

[12] A. B. Sangar and S. Rastari, “A Model for
Increasing Usability of Mobile Banking Apps
on Smart Phones,” Indian J. Sci. Technol., vol.
8, no. 30, pp. 1–9, 2015.

[13] A. Hussain, H. I. Abubakar, and N. B. Hashim,
“Evaluating mobile banking application:
Usability dimensions and measurements,”
Conf. Proc. - 6th Int. Conf. Inf. Technol.
Multimed. UNITEN Cultiv. Creat. Enabling
Technol. Through Internet Things, ICIMU
2014, no. 1, pp. 136–140, 2015.

[14] F. Ecer, “An integrated fuzzy AHP and ARAS
model to evaluate mobile banking services,”
Technol. Econ. Dev. Econ., vol. 24, no. 2, pp.
670–695, 2018.

[15] J. Nielsen, “10 Heuristics for User Interface
Design: Article by Jakob Nielsen.” 1994.

[16] G. D. Magoulas, S. Y. Chen, and K. A.
Papanikolaou, “Integrating Layered and
Heuristic Evaluation for Adaptive Learning
Environments,” Proc. Second Work. Empir.
Eval. Adapt. Syst. held 9th Int. Conf. User
Model. UM2003, Pittsburgh, no. June 2014, p.
11, 2003.

[17] M. A. u. Haq, H. A. Malik, F. Akram and E. K.
A. Mutawa, "Monetary Benefits of Solar
Energy for Smart Cities Development," 2020
International Conference on Innovation and
Intelligence for Informatics, Computing and
Technologies (3ICT), 2020, pp. 1-5, doi:
10.1109/3ICT51146.2020.9311992.

[18] J. Nielsen, “Severity Ratings for Usability
Problems,” pp. 4–5, 1995.

[19] J. Nielsen, “How to Conduct a Heuristic
Evaluation,” Useitcom. pp. 1–11, 2002.

[20] A. Bangor, T. Staff, P. Kortum, J. Miller, and T.
Staff, “Determining what individual SUS scores
mean: adding an adjective rating scale,” J. usability
Stud., vol. 4, no. 3, pp. 114–123, 2009.