http://www.sajim.co.za/peer75.6nr4.asp?print=1 Peer Reviewed Article Vol.6(4) December 2004 Interactive information consulting system for South African small businesses - Part 2 V.C. Baard School of Business and Informatics, Australian Catholic University, New South Wales, Australia vbaard@bigpond.com A. van den Berg School for Entrepreneurship and Business Development, Central University of Technology, Free State, Bloemfontein avdberg@tfs.ac.za Contents 1. Introduction 2. Pilot testing of CONSIT 3. Implementation of CONSIT 4. Results and findings 4.1 Performance 4.2 Provision of effective management principles 4.3 Lack of general knowledge of computers 4.4 Lack of expertise to computerize the SB 4.5 Reliance on outside sources 4.6 Sub-optimal use of IT 4.7 Technical merits of CONSIT 5. Conclusion 6. References Key words: Small business; information technology; information; performance; implementation methodology 1 Introduction Owing to the size and complexity of the study, Part 1 of this article reviewed the literature and identified the research design and methodology (Baard and Van den Berg 2004). It included a report on intervention design, incorporating a discussion on system and general requirement specifications, facilities to be provided by CONSIT and the metamorphosis of CONSIT from a concept into a finished product. The purpose of CONSIT was to escalate performance levels through the combination of effective management principles and a well-planned optimal use of information technology (IT) in the small business (SB) environment. This process would be facilitated through the provision of high quality information from CONSIT to the user as well as the exchange of information between the user and the intervention. In this article the post-intervention design activities are discussed. These activities include: A pilot-testing of CONSIT The implementation of CONSIT into the SB environment A post-implementation evaluation of CONSIT. 2 Pilot testing of CONSIT Conducting a pilot test was a proactive approach to the successful implementation of CONSIT. It provided an excellent opportunity to determine intervention performance prior to implementation, if CONSIT could be implemented by users in natural agency settings and to examine its viability and utility as a practice tool. Pilot testing was also essential to ensure that the implementation methodology would successfully implement the intervention into its intended environment. CONSIT was therefore subjected to pilot testing after design and prior to implementation. The pilot testing for the purposes of this study consisted of two components, namely, the intervention test and the testing of the implementation methodology. The intervention was subject to an integration test and a user test. The integration test was performed to verify that all the elements in CONSIT communicate properly with each other and that CONSIT interfaces properly with all external systems (computer hardware), which facilitate the operation of the CONSIT software. Different users may use different computer hardware components with different configurations and it was fundamental to ensure that CONSIT was able to operate on any hardware requirements that are found within the SB environment. The user test was performed to ensure that CONSIT was able to operate satisfactorily to meet the information needs of the users. A pilot testing of the implementation methodology allowed problems to arise without serious consequences and allowed the researcher to become more confident about implementing the system with the minimal risk of failure. This was important in the absence of a standardized implementation methodology for interventions, resulting in the researcher formulating an implementation methodology based on the principles and methods of project management. It provided opportunities to adjust the implementation methodology prior to live intervention implementation. Pilot testing of both elements occurred within four small firms that conformed to the criteria listed in the National Small Business Act, Nr. 102 of 1996 and which fell within the typology of business services. The businesses were conveniently sampled, were not subjected to any prior assessment and had no knowledge of the design and development of the intervention, its intended implementation or the research purpose in terms of performance escalation. Both the manual and the electronic formats of CONSIT were subject to testing. The pilot practitioners were provided with a chronological self-reporting form that they were required to complete during the pilot test to enable them to note defects present and possible suggested changes. After completion of the pilot testing the chronological logs were reviewed. The researcher also conducted a semi-structured interview with each of the practitioners in which they reflected on the implementation experience with regards to tools and methods utilized. top The pilot test resulted in minimal amendments to CONSIT and the implementation methodology. 3 Implementation of CONSIT The literature on the implementation of interventions was found wanting and that which was available did not advocate a particular methodology for the implementation of an intervention that can be applicable to all situations. Project management (PM) offers a structured approach to managing projects. Based on an in-depth analysis of the discipline, an implementation methodology consisting of four steps utilizing PM techniques, methods and tools, was designed for the implementation of CONSIT (Figure 1). After extensive planning and preparation, the hard and electronic formats of CONSIT were implemented into ten randomly selected small firms, also referred to as the treatment group; seven already used IT and three did not. The control group consisting of 15 randomly select small firms received no exposure to CONSIT at all. The live implementation occurred over a two and a half week period. During this period the treatment group worked through each of the phases and the associated activities of CONSIT and accumulated experience in how to use the system. They then developed a fit between their businesses and the elements of the phases of CONSIT that were applicable to their specific business. The quality of the live implementation was extremely important and was maintained with rigorous monitoring and controlling of performance, time and cost components. Monitoring occurred telephonically every three or four days, depending on the participant. This was supported with weekly formal in-person visits that greatly improved the level of commitment from the participants, since they perceived these visits from the researcher as an exhibition of commitment to the implementation process as well as the small businesses themselves. The intervention was then integrated within the SB environment for use in daily operations in order to improve performance for a further three months. This too was monitored and controlled, however, the frequency with which this occurred was reduced to a fortnightly exercise. This integration went according to plan and proved to be uneventful in terms of the appearance of issues and problems. It was, however, satisfying in terms of the benefits resulting from the use of CONSIT that the implementers were beginning to notice. Figure 1 CONSIT implementation methodology top The closeout or termination of the CONSIT implementation project entailed the following: CONSIT interactive and its associated materials remained with the SB CONSIT became another tool that forms part of the participants’ normal activities A post-test evaluation was submitted to the participant for the evaluation of CONSIT and the SB performance levels. Based on the information collected from monitoring procedures during implementation, CONSIT was deemed to be successful. The following should also be noted: In terms of the results achieved, the application of project management principles proved to be well suited to the implementation of CONSIT The intense approach to implementation planning resulted in the minimization of any problems or issues that were encountered The methods and tools utilized provided a tight control on the implementation process and so contained the implementation within its intended scope There was no rejection of the intervention by any of the ten participants during implementation or the three-month integration period The design of CONSIT facilitates implementation on a piecemeal basis; this feature enabled the successful infusion thereof into the real-life SB environment The implementation methodology should be utilized in other intervention studies to really determine its mettle as a methodology, so that it can assume a standard model with the allowance of a few adaptations to suit the particular study that is using it. It is only after repeated applications that defects may arise that require reparation or re- modelling. 4 Results and findings The purpose of the post-test evaluation was to determine whether CONSIT had improved the performance of the SB or not, whether it contained a suitable degree of effectiveness in its attempt to solve the problems indicated in Part 1 and to highlight any other significant effects of the system. The results of all 25 responses to the post-test evaluation indicated that no failures in terms of the experimental groups had occurred. The discussion of the results and findings for the purpose of this article are limited to the following areas: Performance Provision of effective management principles Lack of general knowledge of computers Lack of expertise to computerize the SB Reliance on outside sources Sub-optimal use of IT Technical evaluation of CONSIT. 4.1 Performance Table 1 indicates descriptive pre- and post-test statistics on the performance variables for all 25 experimental participants. Table 2 presents the treatment and the control groups’ pre-test and post-test mean scores. Table 1 Performance variable comparisons top Table 2 Variance of performance variables for the experimental groups Although not significant, the post-test variances were greater than the pre-test variances with the exception of the number of employees, which stayed constant. The application of the non-parametric Mann-Whitney U and Wilcoxon Signed Rank Tests to the data indicated no statistical significant differences between or within the treatment and control groups. Hence, in terms of these results the escalation of performance in the SB in the three-month period of the adoption of CONSIT did not occur. However, this does not imply that CONSIT is unable to escalate performance and therefore failed as an intervention in terms of its intended purpose. The results clearly indicated a trend among the performance variables, where the mean of the scores for the variables in the treatment group was increasing. No such trend was prevalent in the control group; the results remained unchanged. It could be argued that the trend indicated in the treatment group is due to extraneous variables, which is possible but, due to the nature and rigor of the research design, unlikely. 4.2 Provision of effective management principles The elements comprising this factor include the lack of management skills to integrate strategic and operational management practices (discussed in 4.6 below) and inadequate planning. The planning activity was renamed a 'business review' and contained a simplified mission statement, a basic list of financial and non-financial objectives (performance measures), an internal (strengths and weaknesses) and external (threats and opportunities) analysis. This crucial strategic planning activity is a template that fits on one page and desires the exchange of information between CONSIT and the user. The treatment group unanimously agreed that this facility provided a simple analysis of the status of their business. Respondents from the treatment group stated: 'I really found the business review great. It is almost like a planning schedule, but a very simple one that is easy to do.' 'CONSIT made me look at things differently and think about issues such as planning that I generally do not have the time for.' The finding was, therefore, that the simplification of this essential activity and the renaming thereof changed the small firms’ perception of strategic planning. Support for this finding is found in the evaluation of the technical merits of CONSIT; the treatment group respondents actually requested additional templates for a business plan and the strategic activities of basic sales and expenditure budgets and, more importantly, cash flow forecasting. IT planning elements contained in CONSIT included a future information needs assessment and a costs evaluation module (budgeting information for total computer cost assessment). Absolute agreement from the treatment group in support of the costs evaluation module as a budgeting tool and the future information needs assessment occurred. CONSIT, therefore, has managerial competency and planning information elements that can successfully contribute to the reduction in SB failure. 4.3 Lack of general IT knowledge The literature indicated that a positive attitude toward the adoption of IT into the SB is enhanced through the provision of a greater degree of IT knowledge to the owner-manager. This in turn will improve the likelihood of IT adoption. Table 3 indicates the spread of proportions for the degree of knowledge of computers held by the 25 respondents of the experimental groups. An improvement in the degree of knowledge of the treatment group is reflected. Table 3 Degree of IT knowledge While the application of the Mann-Whitney U Test and the Wilcoxon Signed Rank Test to the degree of knowledge variables indicated no statistical significant differences between or within the treatment and control groups, other results showed a clear trend in the improvement of the degree of IT knowledge of the treatment group. Additional results included: A strong positive correlation (0.827) exists between the degree of IT knowledge in the treatment group and the increase in the use of the computers currently used in the existing functions of the business. This indicates that the higher the degree of IT knowledge held by a participant, the more the computers currently utilized will increase in terms of the existing functions of the business. The majority of the respondents in the treatment group (85.71%) had average and above average IT knowledge after the implementation of CONSIT. A strong positive correlation (0.833) exists between the degrees of IT knowledge in the treatment group and making more effective use of the computers currently existing in the business because of using CONSIT. Ninety per cent of the treatment group respondents claimed that they had gained a greater degree of general knowledge about computers through the utilization of CONSIT. The entire treatment group agreed that information supplied by CONSIT simplified computer jargon that can be confusing when purchasing computers; this too, would have contributed to the increase in the general knowledge of IT. The finding was, therefore, that CONSIT had the ability to increase the degree of general knowledge of IT in the treatment group. 4.4 Lack of expertise for SB computerization Expertise for SB computerization in this study was the process of computer acquisition in the SB. The results indicated the following: The treatment group unanimously agreed that they had confidence in the ability of CONSIT to assist them in the process of purchasing a computer. Almost half of the respondents purchased software after utilizing CONSIT and one respondent purchased a hardware component. They both utilized CONSIT to attempt to buy a computer and saved money doing so, because they had the correct information on hand when attempting this purchase. All the respondents in the treatment group agreed that CONSIT is a simple system that provides sound information to use when purchasing a computer and that it is a very good self-help system that provides 'computer training' for computer selection and acquisition. The finding in this study was that CONSIT had the ability to provide this expertise through the provision of appropriate information and did in fact do so within the SB during its three- month integration in the SB environment. The provision of this computerization expertise was also reflected in the possession of an improved degree of this expertise, enabling the increased and more effective use of the computers currently utilized in the small businesses. 4.5 Reliance on outside sources for SB computerization Part 1 of this article indicated a reliance by the SB on outside sources (consultants and vendors) that have questionable integrity and ability to serve the needs of the SB; this is as a result of a deficiency in skills and tools to interact with these outside sources and a deficiency of computer knowledge. CONSIT had to reduce the vulnerability of the SB owner-manager to these outside sources through the provision of information that would enhance skills and provide tools to facilitate efficient and effective interaction. Results indicated: 70% strongly agreed and 30% agreed that CONSIT enables them to thoroughly assess a vendor from whom their computer purchase may be made 50% strongly agreed and 50% agreed that they would feel more comfortable approaching a consultant for their computing needs after 'consulting' with CONSIT All respondents agreed that CONSIT would help them save money when using a consultant for computerization in their business. A strong positive correlation (0.808) exists between vendor assessment and the average annual operating expenses. The finding was that CONSIT provided the skills and was the tool that can be used to facilitate SB interactions with consultants and vendors. CONSIT enabled the owner- managers to assess the vendors and consultants and reduce the levels of discomfort and intimidation when dealing with them. This finding was reflected in the attitude of a respondent in the treatment group: 'I will not have the wool pulled over my eyes by the computer consultants that cold call on me. We know how to determine whether you can offer us the help we need and whether you are fly-by-nights trying to make fast money.' Finally, CONSIT provided a sound level of preparedness for the SB owner-manager when he or she approached external sources for computerization needs. 4.6 Sub-optimal use of IT The second component required escalated performance, that is, the well-planned optimal use of IT. The optimal use of IT includes three elements: increasing the efficiency of the asset (IT); the integration of strategic and operational variables; and the successful assessment of the information needs of the business. Table 4 and 5 indicate the IT efficiency improvements in terms of hardware and software satisfaction. Table 4 Degree of hardware satisfaction for the experimental groups Table 5 Degree of software satisfaction for the experimental groups Additional results included: The implementation of CONSIT had a strong positive correlation between The degree of hardware (0.982) and software satisfaction (0.985), and The increased use of the computers currently utilized by the treatment group in the existing functions of their business (0.873), and Those respondents who made more effective use of computers currently in their business as a result of using CONSIT (0.897). All respondents agreed that CONSIT enabled them to comprehensively evaluate the computer hardware and software that may suit the needs of their business prior to purchase. The implementation of CONSIT into the SB increased the use of the computers that the SB currently employed in the existing functions of the business. CONSIT thus had an effect on the increased efficiency of IT. CONSIT provided information that introduced strategic variables to the SB to promote the integration of strategic and operational issues when utilizing IT, since the strategic variable contributed toward growth and improved performance. All the respondents agreed that CONSIT enabled them to integrate, strategic, administrative and operational management practices. A strong positive correlation (0.802) between the ability of CONSIT to equip the SB to optimally use computers in their businesses that can result in improved performance and the ability of CONSIT to integrate strategic, administrative and operational management practices supported this. The simplification of strategic variables into basic budgeting, pricing, cash flow forecasting obviated the complexity associated with strategic issues; the SB owner-managers requested additional worksheets to further assist them with these important issues. Finally, this integration not only solved a part of the managerial competency problem, but also led to a more optimal use of computers within the SB. CONSIT enabled the comprehensive assessment of the business information needs and promoted the matching of those needs to the IT requirements. All the respondents in the treatment group agreed that CONSIT provided a formal means to assess what the present and future information needs of their businesses were. Ninety per cent of the respondents agreed that CONSIT enabled the SB to better match its information needs to the computers it already had or required. The other 10% who expressed uncertainty had an extensive degree of IT knowledge and already made extensive use of computers in the management of the business. The finding therefore, was that CONSIT did promote the optimal use of IT in the SB and the SB did begin to optimally utilize IT due to the implementation of CONSIT. Responses from the treatment group also supported the finding: 'I get more use and value out of my IT investment and I invested in more to substantially improve the way we work.' 'I can start seeing some return on all the money I have spent on my computer.' 4.7 Technical merits of CONSIT Table 6 indicates the frequency of responses to the Likert scale including some descriptive statistics. Table 6 Technical evaluation of CONSIT The presentation of the results yielded the following findings: The information content, accuracy and format, as provided by CONSIT, were of a high quality and were presented in a clear, useful way that met the needs of the SB. The manual and electronic formats that CONSIT adopted were suited to the needs of the SB; the SB did not require any special software or hardware or any other special tool to facilitate the use of the intervention. The user did not need to have a computer or have to know how to use one, to be able to use CONSIT. CONSIT was easy to use and did not intimidate the user in any way. The user did not require a training course to fully utilize the system. CONSIT was an inexpensive and practical system that was suited to a small service firm environment. The treatment group found no defects in CONSIT, although the researcher is of the opinion that the repeated application of an already developed intervention generally provides many occasions to detect limitations. Additions to CONSIT were requested as previously indicated in section 4.2, but there was no need for the re-design thereof. 5 Conclusion The results and findings indicated some recommendations, which include the implementation of CONSIT into the SB for a longer period to ascertain a more significant performance escalation and the refinement of CONSIT, incorporating the tools the treatment group desires. Evidence indicates that CONSIT is a tool that provides an education to the SB owner and is a strategic information resource that allows the SB to see the value of technology through the escalation performance and to provide other valuable benefits that the SB environment can exploit, such as the utilization of efficient management practices. 6 References Baard, V.C. and Van den Berg, A. 2004. Interactive information consulting system for South African small businesses – Part 1. South African Journal of Information Management 6(2). [Online]. Available WWW: http://www.sajim.co.za. top top Disclaimer Articles published in SAJIM are the opinions of the authors and do not necessarily reflect the opinion of the Editor, Board, Publisher, Webmaster or the Rand Afrikaans University. The user hereby waives any claim he/she/they may have or acquire against the publisher, its suppliers, licensees and sub licensees and indemnifies all said persons from any claims, lawsuits, proceedings, costs, special, incidental, consequential or indirect damages, including damages for loss of profits, loss of business or downtime arising out of or relating to the user’s use of the Website. top ISSN 1560-683X Published by InterWord Communications for the Centre for Research in Web-based Applications, Rand Afrikaans University