Kurdistan Journal of Applied Research (KJAR) Journal homepage: http://www.spu.edu.iq/kjar ISSN 2411-7684 (print) – 2411-7706 (online) March 2016 │ Volume 1│ Issue 1 Page 76 KJAR Short Article Evaluating e-Court Services Using the Usability Tests Model Case Study: Civil Court Case Management Sarkhell Nawroly1, Hozan Hama Rashid1, Soran Saeed1*, Nizar Ali1 1Department of computer Sciences, Sulaimani Polytechnic University, Sulaimani, Kurdistan Region, Iraq. *Corresponding Author: soran.saeed@spu.edu.iq Received│November 21, 2016 Accepted│March 13, 2016 Abstract Electronic Court (e-Court) is an Initiative to deliver services to citizens and court agencies (mainly lawyers). It enables citizen to view information and court agencies to access court services through online services. Its implementation has improved the efficiency of governance and court services in Sulaimani. However, it is the first evaluation study on the e-Court to reveal the facilities of the e-Court system using the Usability Test Model. This paper aims to evaluate the e-Court services in Sulaimani by using Usability test and analysis. The Usability test model consists of four main parts which include effectiveness, efficiency, learnability and satisfaction. Effectiveness test; means whether a particular task can be done by users. Simply, test for efficiency; means doing the task fast without getting frustrated. Learnability test; focuses on the number of clicks to accomplish tasks and errors that participants make during the test, this can be used to measure learnability. Simply, all participants vote by rate to determine the level of satisfaction. Generally, satisfaction can be defined as to what extent it is enjoyable or pleasant. A specific test is conducted to get data then the data is analysed based on usability tests model. In the test; five different tasks are given to five users. All the tests are recorded. The test only applied to the civil court as a pilot due to all the courts work in the similar process. Key words: e-Court, effectiveness, efficiency, learnability and satisfaction. Introduction E-Court is a strategy for less developed countries, like Iraq, as Sulaimani City is a part of KRG-Iraq, to improve the quality, efficiency, transparency and effectiveness in their services. In addition, e-Court has the potential to build better relation between the Courts and its constituents (Jin-fu and Duo, 2009). Moreover, it will be able to facilitate towards the achievement of the economic, reduce load, environment in Sulaimani; the initiative has to some extend increase the efficiency of the Court in providing better services to its citizens. However, low acceptance against high investment considered the introduction of e- Court is a strategy that needs to be optimized (MAIT, 2008) (Carter and Belanger, 2004). Even though the Court has increased efforts to promote usage among court staff, to promote http://www.spu.edu.iq/kjar mailto:soran.saeed@spu.edu.iq KJAR Short Article Civil Court Case Management, Sarkhell et al, March 2016 March 2016 │ Volume 1│ Issue 1 Page 77 usage among court staff, the acceptance rate remains low. Perceived barriers and benefits influence the agreement rate which include factors such as culture change, and low IT skills. The level of court staff satisfaction will consequently be a vital indicator to further usage and adoption on a large scale basis, by using the Usability Tests Model, this paper will thus evaluate the user satisfaction towards e-Court services. The implementation of e-Court in Sulaimani began in 2014; the budget of the e-Court system is provided by Sulaimani Governorate and implemented by the Aktors, the Estonian Company, also it is monitoring by Sulaimani Governorate IT Board. The vision of e-Court focuses on effectively and efficiently delivering services from the Court to the citizens of Sulaimani, enabling the Court to become more responsive to the needs of its citizens (MAMPU, 2009). The Sulaimani Court has 7 court parts; their descriptions are shown in (Table 1). Research Paper Model The Usability Tests Model has been formulated with the aim of providing a scale of the evaluation on e-Court, which is a web based system, to expose the level of satisfaction derived by the users (Horan and Abhichandani, 2006). The model includes four categories (effectiveness, efficiency, learnability, and satisfaction). Effectiveness is the rate of the completed uncompleted tasks by the users. Efficiency means conducting tasks without problems. Learnability test focuses on the number of clicks to accomplish tasks and errors that participants make during the test. The level of satisfaction is voted by all the participants then the rate is determined (User focus (n,d) 2012). The usability testing is shown below in the (Figure 1). Figure 1: Usability testing (Source: group quality). Materials and Methods A test was used as an instrument to collect primary data from respondents. This section provides the details about the data collection procedure from using the e-Court system and data analysis method 4.1 Data Collection We apply user testing in this research; we give tasks to the users then record the process throughout the testing for each user. Table 2 shows the tasks that given to the users. Task 1 -5 (Explanation) 1: The first task is about registering or recording a new case in the court. 2: The second task; is to find a registered case based on its name, the name should be registered. 3: The third task deal with searching for a recorded case based on its type and then finding the necessary case. 4: The fourth task is about changing the appointment of a case which means to change the date of a case to another date later. KJAR Short Article Civil Court Case Management, Sarkhell et al, March 2016 March 2016 │ Volume 1│ Issue 1 Page 78 Table 1: Description of E-Court Application. Court Registry Data C iv il P ri m a ry Prime register Case number, names of participants, places of residence, type of case, name of judge, case summary; Daily register Case number, hearing date, hour/minute, names of participants, case type, notes Fees Register The fees collected by the court in the lawsuits and matters emanating therefrom shall be recorded in this register. Register of Legal Distributions Date of entry, case number/letter number, name of court (to), signature Register of decisions (separate registry of urgent decisions) Case number, date (month), names of participants, date of decision, Register of incoming cases from the Court of Appeal Case number, names of participants, result of the appeal court proceeding Register of outgoing cases to the Court of Appeal Case number, date of referral, the name of the court (to) Register of incoming and outgoing letters/Register of Files/Register of Documents Date, number of the case, place of issue, document number, date of the document, subject of the document, attachment, notes, recipient and date of delivery for outgoing documents (e.g. judgments) Register of Notifications Number of the case, court, the date of the suit’s notification to be notified to the plaintiff and the defendant, the date specified for attendance before the court, the nature of the suit and the date of delivery, the summoner. The Register of Trusts All bonds/securities and other sums deposited in the court’s treasury and the name of the depositor and the suits’ numbers shall be registered herein. The depositor shall be given a receipt of payment. Table 2: Illustrate test of a number of tasks conduct by the users for primary court. Tasks 1 Add new case 2 Find someone’s case under name ( For example: Hawkar Kamaran Ali) 3 Search for a registered case based its type 4 Change the appointment of any case 5 Show the total number of registered cases from 01/06/2015 - 01/11/2015 KJAR Short Article Civil Court Case Management, Sarkhell et al, March 2016 March 2016 │ Volume 1│ Issue 1 Page 79 4: The fourth task is about changing the appointment of a case which means to change the date of a case to another date later. 5: The last task is about showing the list of cases by using different parts such as loan and by the name of the judge in between two dates. Results and data Analysis Effectiveness Testing Effectiveness means whether a particular task can be done by users (Liu et al. 2010). After the test was finished, all participants accomplished task1 (Add new case) and task 3 (search for a registered case based its type). Three out of Five 60% completed task 5 (Show the total number of registered cases from 01/06/2015 - 01/11/2015). Only the 3rd and the 5th participant were able to accomplish all the tasks. Table 3 illustrates a ratio of completed tasks and the number of accomplished tasks. Approximately, all of the tasks are achieved. Consequently, the web application of e-Court is effective. Table 3: Shows the Result of Effectiveness Analysis. Efficiency Testing The smart phone video recording software recorded the time when participants did the tasks. Some tasks were more difficult to complete than the others. The statistics in (Table 4) reveal the average time spent on tasks in minute. It can be clearly seen that the average time to complete the first task takes more time than the other tasks which is 9 minutes and 20 seconds. However, task 5 consumes the least time to achieve it which is 2 minutes and 27 seconds in average. The average time on task 2 and task 3 take nearly the same time to finish which is 2 minutes and a half. On the other hand, task 4 takes under 4 minutes to accomplish. Secondly, not much difference between the average time to accomplish task 2, 3, 4 and 5. Also, task 1 approximately needs time to achieve as much as the other tasks. To sum up, the library website is efficient enough due to it does not take a long time to conduct some routine tasks. Participant Task 1 Task 2 Task 3 Task 4 Task 5 Finished tasks 1 √ √ - √ √ 4 2 √ √ √ √ - 4 3 √ √ √ √ √ 5 4 √ √ √ √ - 4 5 √ √ √ √ √ 5 Success 5 5 4 5 3 Completion Rates 100% 100% 80% 100% 60% KJAR Short Article Civil Court Case Management, Sarkhell et al, March 2016 March 2016 │ Volume 1│ Issue 1 Page 80 Table 4: Illustrates the Average time on tasks. Tasks P1 P2 P3 P4 P5 Avg time on tasks Task 1 10.2 9.0 8.8 9.0 9.0 9.2 Task 2 3.0 2.3 2.0 2.3 1.5 2.2 Task 3 3.0 2.2 2.2 2.1 1.4 2.2 Task 4 6.0 5.3 1.6 2.0 2.5 3.5 Task 5 3.0 2.0 1.5 1.5 1.4 2.3 Sum of time 25.0 20.8 16.0 16.8 15.8 Avg of sum=18.88 Learnability Testing The number of clicks to accomplish tasks and errors that participants make during the test can be used to measure learnability (Yaghoubi et al., 2011). The sum of the number of clicks for all of the four tasks was hit a peak with participant 1 which is just under 110 clicks. However, the total number of clicks hit a low with the 4th participant and which is nearly 95 clicks. On the other hand, all the participants make mistakes or errors while doing the test especially on task 3 and 5. The summation of errors made by participant 2 and 3 is the same which is 5 mistakes, whereas, the 4th participant just made 4 mistakes. The difference between mistakes of 1st and 5th participant is only 1 mistake. Overall, based on the table and analysis; the e- Court system is learnable because the number of mistakes and errors are tiny. However, all participants made mistakes on task 3 and 5. The data in (Table 5) shows the number and summation of clicks and errors made by the participants. Table 5: Shows the numbers and summation of clicks and errors. Tasks Clicks/P1 P2 P3 P4 P5 Error by P1 P2 P3 P4 P5 Task 1 53 52 49 45 46 1 0 0 0 1 Task 2 13 12 10 11 13 1 1 0 0 1 Task 3 15 13 14 13 14 3 2 3 2 2 Task 4 14 13 10 11 12 0 0 0 0 0 Task 5 12 12 11 13 11 3 2 2 2 3 Sum of clicks/errors 107 102 94 93 96 8 5 5 4 7 KJAR Short Article Civil Court Case Management, Sarkhell et al, March 2016 March 2016 │ Volume 1│ Issue 1 Page 81 Satisfaction Testing Simply, all participants vote by rate to determine the level of satisfaction. Generally, satisfaction can be defined as to what extent it is enjoyable or pleasant (Jiang 2009, Horan, et al. 2006). The ratio in (Table 6) illustrates the participants vote on each task. According to the vote rate and the above statistical equation, the average of the rate shows that all participants were satisfied with the web application of the e-Court. Table 6: Shows the satisfaction rate. Average bar = summation of avg / 5 = 97.52% Participants Task 1 Task 2 Task 3 Task 4 Task 5 P1 95% 98% 95% 98% 95% P2 100% 100% 95% 100% 95% P3 100% 100% 96% 100% 96% P4 100% 100% 96% 100% 92% P5 100% 100% 94% 100% 93% Avg. rate 99% 99.6% 95.2% 99.6% 94.2% Recommendations Users Recommendation  Search purpose: e-Court system need to match reference number of the document that came manually from outside the system as e-Court system gives automatically their number.  Search purpose: user might write important notes in the note field for any application forms, the search need to have an option to show a list of documents that have note written on.  Search purpose: an option to search for waiting list cases, but did not give any results.  Search purpose: search for reference number of document is the best way as it is not easy to look for Arabic names as do not have unified spelling when it written in Kurdish.  Search and efficiency: search filtering need to be added by the user’s permission, even though the user can not open the other cases that been search for, but still show the whole list that take longer time to find what user want.  Accuracy: Clients still bring their document to staff to be scanned, and sometimes one paper document left in the scanner, to overcome this problem feeder scanner or some staff might work in scanning clients document before coming in outside the department office.  Usability, efficiency and accuracy: more Computer Skills Courses in general and on the system need it and typing training skill needs.  Usability: some buttons need to be exchanged up to down as the original document of the court. Increase training course for users.  Usability: the system still accepts numeric in the field of names.  Usability: change seen date of case takes longer KJAR Short Article Civil Court Case Management, Sarkhell et al, March 2016 March 2016 │ Volume 1│ Issue 1 Page 82 as expected, due to partially doing the process manually, staff have to print document and the judge seen it then sign it, and then come back to the staff to scan it into the system to record the new seen date, due to that the fully electronic system need to be implemented.  Usability: the system does not bring summited back to edit it in case of any mistakes occur.  Usability: Even though some cases are submitted the final decisions, but still show in waiting field.  Usability: need to draw some lines in the interface to distinguish between parts, for example: between complaint and the complaint on.  Usability: explanation description on buttons in Kurdish.  Efficiency: developer work on the requested changes by the court in times of the day that not be a pick time work, not to be off line in that time.  Efficiency: all staff has to inform of any changes happen to the system and the way the staff operate and with their permissions.  Usability: the layout of the typist page has to be more user friendly with layout and font colour.  Usability: Close case option need to be informed where it goes, should add some categories while it held till next time opens again until the final decisions be made.  Efficiency purpose: need fast scanners and more printers, especially in the Court Halls.  Efficiency and accuracy: Staff needs to organize their electronic document before the time on seen in the Court Halls.  User Satisfaction: users are happy with the new system, but still do same job manually and electronically, manually do not need as the electronically hit the purpose. Authors Recommendation  Weekly meeting needed to inform all staff what have changed by the system developers and what has staff are learned or experienced. Then staff have a note book to write suggestions and questions and have they experienced to share it with the developers  Digital signature needed to make the process fast, transparent, environmental and easier.  More course on IT skills, on the interface and typing and more IT support staff needed.  Encourage staff by given appreciation letter, certificate and bonuses.  Encourage academics from universities to do evaluation test and do publication on the system as it is beneficial for both of sides.  Availability to access the E-Court system by citizen to use the service online.  Explain to Staff who to ask for different issues, for example people who are dealing with IT issue are different from software issue.  Continuously interview users to get the usability test faster.  Finance department: need an extra column of Date next to the pay to know when it been paid.  Encourage the users by giving certificates or bonus.  Enhance Queuing system as client might disturb KJAR Short Article Civil Court Case Management, Sarkhell et al, March 2016 March 2016 │ Volume 1│ Issue 1 Page 83 staff while working online that might cause mistake in entering information. Expectation: This system might improve the convenience, accessibility and quality of interactions among users. Moreover, it is expected to improve information flow, and processes within Court agencies, to improve the speed and quality of policy development, and to improve coordination and enforcement (MAMPU, 2009, MAIT, 2008). Conclusions Citizen’s, agencies and staff satisfaction is an important indicator in giving a general idea as to how well the Court has transformed its services in accordance with its citizen’s needs. The level of satisfaction will also be a vital indicator to further usage and adoption on a large scale basis. The Usability test model presented in this paper the satisfaction of all participants. By the time the court staff will be much familiar with the system, learn from the mistakes and get more experience, and this experience add a great value to give better services in the limit time to their citizen, and government agencies. References Jin-fu W. (2009) and H. Duo, "Customer-Centered e- Government Service Quality Evaluation: Framework and Case Study," International Colloquium on Computing, Communication, Control, and Management. MAMPU (2009) "“e-Government Malaysia: Way Forward”," 1 e-Government Conference. MAIT (2008). e-Government in Malaysia. Available from: www.mait.com Jiang X. (2009) "Measurement of Public Satisfaction Evaluation on EGovernment: Based on structural Equation Model," IITA International Conference on Control, Automation and Systems Engineering. Liu Y., et al. (2010) "Customer Satisfaction Measurement Model of EGovernment Service," Service Operations and Logistics and Informatics (SOLI), IEEE. Yaghoubi N. M. et al. (2011), "E-Government and Citizen Satisfaction in Iran: Empirical Study on ICT Offices," World Applied Sciences Journal 12, 1084- 1092. Horan T. A. and Abhichandani T. (2006) “Evaluating User Satisfaction in an E-Government Initiative: Result of Structural Equation Modeling and Group Discussions," Journal of information Technology Management. XVII, 187-198. Horan T. A., et al. (2006), "Assessing User Satisfaction of E-Government Services: Development and Testing of Quality-in-Use Satisfaction with Advanced Traveler Information Systems (ATIS)," Proceedings of the 39th Hawaii International Conference on System Sciences. Carter L. and Belanger F. (2004), "Citizen Adoption of Electronic Government Initiatives," in Proceedings of the 37th Hawaii International Conference on System Sciences. Userfocus,(n,d),(2012).“Usability testing” [online] available from: http://www.userfocus.co.uk/consultancy/usabilitytesti ng.html http://www.mait.com/ http://www.userfocus.co.uk/consultancy/usabilitytesting.html http://www.userfocus.co.uk/consultancy/usabilitytesting.html Evaluating e-Court Services Using the Usability Tests Model Case Study: Civil Court Case Management Sarkhell Nawroly1, Hozan Hama Rashid1, Soran Saeed1*, Nizar Ali1 Abstract Introduction Materials and Methods Results and data Analysis Recommendations Conclusions References