Esprit project 28015 trump trial Usability Maturity Process Cost benefit analysis Nigel Bevan Serco Usability Services



Download 72.37 Kb.
Date conversion29.05.2018
Size72.37 Kb.


ESPRIT Project 28015 TRUMP



Trial Usability Maturity Process

Cost benefit analysis

Nigel Bevan

Serco Usability Services

D3.0

Version 1.1

5-Oct-00

Abstract

Explains how to calculate the cost-benefits of user centred design, by comparing the costs with an estimate of the potential benefits to the organisation during development, sales, use and support. Vendors can benefit in development, sales and support. Purchasers can benefit in use and support. Systems developed for in-house use can benefit in development, use and support.



Keywords: cost benefit analysis, user centred design, usability.
1.DOCUMENT HISTORY



Issue

Date

Author

Description of change

Approved

0.1

4/12/99

N Bevan

Draft for comment




0.11

20/12/99


N Bevan

Feedback from Dorothy Kushner




1.0

30/3/00

N Bevan

Included example




1.1

8/9/00

N Bevan

Executive summary improved





















CONTACTS

This report was produced by.


Nigel Bevan

Serco Usability Services

Alderney House

4 Sandy Lane, Teddington

TW11 ODU, Middx, UK
Tel: + 44 20 8614 3811

Fax: + 44 20 8614 3765

nbevan@usability.serco.com
Contents


2 Executive summary 6

3 The problems (and their costs) 8

4 Potential benefits of usability 9

4.1 Development 9

4.2 Sales 9

4.3 Use 9

4.4 Support and Maintenance 10


5 Estimating benefits 12

5.1 Development 12

5.2 Sales 12

5.3 Use 12

5.4 Support 13

6 Making the cost-benefit case 14

6.1 Calculating cost-benefits 14

6.2 Example of how to calculate cost-benefits 14

6.3 Other barriers to user-centred design 15

References 16




2Executive summary


To calculate the cost-benefits of user centred design:

  • estimate the potential savings during development, sales, use and support

  • estimate the costs that would be incurred.

The extent of the financial benefits will depend on how completely user centred design can be implemented. A balance needs to be obtained so that a convincing case can be made for benefits that are substantially larger than the costs of additional user centred activities.

  • Vendors can benefit in: development, sales and support.

  • Purchasers can benefit in: use and support.

  • Systems developed for in-house use can benefit in: development, use and support.
1.Development

What savings will be made as a result of:

1.1Reduced development time and cost to produce a product which has only relevant functionality and needs less late changes to meet user needs?
1.2Reduced cost of future redesign of the architecture to make future versions of the product more usable?
2.Sales

What increase in revenue will result from:

2.1Increased competitive edge: customers expect products to be easy to use. What percentage increase in sales can be obtained by marketing your product as easier to use than the competition?
2.2More satisfied customers: difficult to use products create dissatisfied customers. What percentage increase will there be in repeat customers more satisfied with the current product?
2.3Higher ratings for usability in the trade press?

Customers will be more satisfies as a result of the potential savings listed below.
3.Use

What savings will be made as a result of:
3.1Reduced task time and increased productivity?
3.2Reduced user errors that have to be corrected later?
3.3Reduced user errors leading to an improved quality of service?
3.4Reduced training time for users?
3.5Reduced staff turnover as a result of higher satisfaction and motivation?
4.Support

What savings will be made as a result of:

4.1Reduced costs of producing training materials?
4.2Reduced time providing training?
4.3Reduced time spent by other staff providing assistance when users encounter difficulties?
4.4Reduced help line support?
5.Calculating cost-benefits

For organisations already committed to user-centred design a cost-benefit analysis is not essential but it can provide a valuable input when formulating a usability plan. The technique can be used repeatedly as a development project progresses to reassess the importance of various activities. The process can also be used to compare different usability methods and so aid selection of the most cost effective method.

3The problems (and their costs)

Landaur (1995) provides evidence that computers have not improved the efficiency and quality of most information work, and that difficult-to-use computer systems are a major factor in restricting economic growth. Productivity growth in the seven richest nations has fallen in the last 30 years, from an average of 4.5% a year during the 1960s, to a rate of 1.5% in recent years. The slowdown has hit the biggest IT spenders – service-sector industries – hardest (Gibbs, 1997). 31% of the computer systems designed for in house use are either cancelled or rejected as unacceptable, primarily because of inadequate user design input. When they are installed "computers often create pathologies and increased costs" (Standish Group, 1995). The result is a loss of approximately $80 billion annually to the economy (Standish Group 1995).

Some of the specific costs created by badly designed and difficult to use systems have been identified. Because computers are often hard to use, companies often have to provide about $3150 worth of technical support for every user (Gartner Group, 1997). In addition, non-technical employees take 4-10% of their time to help co-workers solve their computer problems at a cost of $10,500 a year for each computer (Nolan Norton Institute, 1997). Unproductive activities with computers (support and housekeeping, response time delays, checks for accuracy, etc) cost another $5590 per computer per year (Gartner Group, 1997).

4Potential benefits of usability


The objective of introducing user centred methods is to ensure that real products can be used by real people to achieve their tasks in the real world. This requires not only easy-to-use interfaces, but also the appropriate functionality and support for real business activities and work flows. According to IBM (1999) “It makes business effective. It makes business efficient. It makes business sense”.

4.1Development


Many existing development processes focus exclusively on adherence to technical and process specifications. User centred methods can reduce development time and cost. The user interface may account for up to 66% of the lines of code in an application and 40% or more of the development effort (MacIntyre et al 1990). During software development, usability engineering can reduce the time and cost of development efforts through early definition of user goals and usability objectives, and by identification and resolution of usability issues. A change may cost 1.5 units of project resource during conceptual design, 6 units during early development, 60 during systems testing and 100 during post-release maintenance (Pressman, 1992).

The value of user centred methods in improving the quality of computer systems is confirmed by a survey by Keil and Carmel (1995) which showed that the probability of project success is improved by user-centred design.


4.2Sales

In today's market, usable products are desirable products. Ease of use differentiates them in a highly competitive market place. Ease of use brings an added value that culminates in a higher degree of customer satisfaction, continued business and higher revenues. Customer service and satisfaction provide market differentiation (Jones and Sasser, 1995, Prokesch, 1995). Companies committed to ease of use do more than meet customer expectations, they can actually exceed anticipated earnings (Karat, 1997).

User-centred design provides methods and tools for identifying, and catering for, a wide range of user needs, thus potentially making a product accessible to a wider range of users, including those with special needs.

Products with a higher rate of employee satisfaction along with improved performance, produce a chain reaction that leads directly, and ultimately, to more satisfied customers. And satisfied customers provide the base for business growth and competitive success.

Wixon and Jones (1992) document a case study of a usability-engineered product that achieved revenues that were 80% higher than for the first release developed without usability engineering, and 60% above project expectations.

4.3Use


Companies that purchase or produce usable systems for their employees also see impressive returns on their investment.

  • Improved productivity. The average software program has 40 design flaws that impair employees' ability to use it. The cost in lost productivity is up to 720% (Landauer 1995). Design changes due to usability work at IBM resulted in an average reduction of 9.6 minutes per task, with projected internal savings at IBM of $6.8 Million in 1991 alone (Karat 1990).

  • Better task focus. A good interface to a well-designed product will allow the user to concentrate on the task rather than the tool. If the interface is designed appropriately, it will allow them to operate effectively and efficiently, rather than lose vital time struggling with a poorly designed user interface and badly thought-out functionality.

  • Reduced errors. A significant proportion of so-called “human error” can be attributed to a product with a poorly designed interface that is not closely matched to the users' task needs, or to their mental model of the task. Avoiding inconsistencies, ambiguities or other interface design faults, while adhering to user expectations in terms of task structure and sequencing, has the potential to significantly reduce user error and improve quality.
  • Reduced training. A poorly designed user interface and dialogue can prove a barrier to an otherwise technically sound system. A well-designed system designed with a focus on the end-user can reinforce learning, thus reducing training time and effort. Karat (1993b) documents a case study of a usability engineered product that required a one-hour training session as compared to one week of training for similar systems built and used within the organisation. The investment in usability save the organisation millions of dollars in training costs and in the opportunity costs of the employees' time in the first year alone.


  • Organisations that ensure their employees are furnished with easy-to-use products see dramatic reductions in training time and, subsequently, great reductions in training cost. In certain cases,.

  • Improved acceptance. This is particularly important where usage is discretionary. Users are more likely to use and trust a well-designed, accessible system, which has been designed so that information is both easy to find and is provided in a form that is easy to assimilate and use.

  • Reduced staff turnover resulting in lowered personnel costs associated with the employees who have lower turnover rates and higher satisfaction with their work environment. (Karat 1994a).

4.4Support and Maintenance


When an organisation tries to recover from significant usability problems in a released product, the costs of post-release changes are unaffordable for many organisations.

  • 80% of software life cycle costs occur during the maintenance phase (Pressman 1992).

  • 80% of maintenance is due to unmet or unforeseen user requirements; only 20% is due to bugs or reliability problems (Martin and McClure 1993; Pressman 1992).

  • Microsoft tracks its support call costs and has seen a significant cost savings resulting from improving the usability of its products, such as Word (Reed 1992).

  • Design changes due to usability work on one project at IDS/American Express resulted in estimated savings of $45 million (Chalupnik and Rinehart 1992).
  • Design changes from one usability study at Ford Motor Company reduced the number of calls to the help line from an average of 3 calls to none, saving the company an estimated $100,000 (Kitsuse 1991).


  • Reduced support. Significant savings of help-desk calls and service costs are another added bonus when products are made to meet user needs.

5Estimating benefits


To calculate the cost-benefits of user centred design, first estimate the potential benefits for your organisation during development, sales, use and support.

  • Vendors can benefit in: development, sales and support.

  • Purchasers can benefit in: use and support.

  • Systems developed for in-house use can benefit in: development, use and support.

5.1Development


What savings will be made as a result of:

  • Reduced development time and cost to produce a product which has only relevant functionality and needs less late changes to meet user needs?

  • Reduced cost of future redesign of the architecture to make future versions of the product more usable?

5.2Sales


What increase in revenue will result from:

  • Increased competitive edge: customers expect products to be easy to use. What percentage increase in sales can be obtained by marketing your product as easier to use than the competition?

  • More satisfied customers: difficult to use products create dissatisfied customers. What percentage increase will there be in repeat customers more satisfied with the current product?

  • Higher ratings for usability in the trade press?

Customers will be more satisfies as a result of the potential savings during use listed below.

5.3Use


What savings will be made as a result of:

  • Reduced task time and increased productivity?

This can be estimated by measuring efficiency in a usability test.


  • Reduced user errors that have to be corrected later?

This can be estimated by measuring effectiveness in a usability test.

  • Reduced user errors leading to an improved quality of service?

This can be estimated by measuring effectiveness in a usability test.

  • Reduced staff turnover as a result of higher satisfaction and motivation?

This can be estimated by measuring satisfaction in a usability test.

  • Reduced training time for users?

5.4Support


What savings will be made as a result of:

  • Reduced costs of producing training materials?

  • Reduced training time for trainers and users?

  • Reduced time spent by other staff providing assistance when users encounter difficulties?

  • Reduced help line support?

6Making the cost-benefit case

6.1Calculating cost-benefits

The basic principle is to estimate the total cost of producing, maintaining, and selling the product, and compare current costs with estimated costs after use of user centred design methods.

The extent of the financial benefits will depend on how completely user centred design can be implemented. A balance needs to be obtained so that a convincing case can be made for benefits that are substantially larger than the costs of additional user centred activities.

Most user centred design techniques are relatively simple to apply. The major costs are the time of the people who apply the methods. The methods chosen will depend not only on the budget available, but also on the available skills and experience and practical constraints such as project deadlines and the availability of users.

A full implementation of user centred design would conform to ISO 13407 and use all of the relevant capabilities in the Usability Maturity Model (Earthy 1998). A minimalist approach could implement the methods recommended in Getting started in user centred design (Bevan 1999) and only involve 10-15 person-days effort.

For more information on calculating the costs of methods, see Daly-Jones et al (1998) and Bias and Mayhew (1994).

6.2Example of how to calculate cost-benefits


    After using a range of user-centred design techniques during the development of a system:

  • For each technique, sum the person days required to perform it:

    (preparation time x people) + (application time x people) + (documentation time x people)



  • Multiply the person-days by the appropriate day rate(s) to give a total labour cost, and add any other costs.

  • Decide which of the benefits listed in section 2 the technique contributed to.

  • Describe the specific benefits obtained from each technique.

  • Group together techniques that have been used in combination (typically at the same stage of development) and provide shared benefits.

  • Estimate the financial savings resulting from the use of each inter-related group of techniques (avoiding any double counting).

  • Calculate the cost-benefit ratio for each group of techniques, and the overall cost-benefit ratio.

  • To estimate the cost-benefits of individual techniques (particularly those regarded as of marginal value) re-calculate the estimated cost benefits for a group of techniques when one or more individual techniques has been excluded.

    Use the results to establish a policy for which techniques should be used as a normal part of development, and to decide how to tailor the selection of techniques to particular development situations.


6.3Other barriers to user-centred design

A cost-benefit case alone is not sufficient to introduce user centred design. Organisations frequently do not budget for the total cost of producing, supporting and maintaining a product. The development teams’ main priority is to complete development on time and on budget. Organisations typically only prioritise usability if they suffer from lost revenues, lost bids, dissatisfied customers, and high support costs. Introducing successful user-centred design into a large organisation requires cultural and technical change as well as strategic commitment:


  • cultural: all those involved in the development of a system or product must be aware of the issues and activities involved in user-centred design to effect the best design decisions at a micro- and macro- level. Others affected by the system or product, for example end-users, who previously have not been involved in its development, need to be made aware of their new role in design. They must also put into place mechanisms for achieving the technical and cultural change outlined below.

  • technical: development processes and procedures must include the user centred methods and activities which are appropriate for the organisation and whose purpose and benefit can be clearly demonstrated to the developers. Techniques are required for the selection of appropriate methods for each project, and for the dissemination of lessons learned to other projects in the organisation.

  • strategic: the organisation and its management must set quality in use as a principal objective for systems development, but until recently there has been no way to accurately specify quality in use requirements prior to design. There is often a contractual requirement to deliver a product that matches the specification, but if the specification is not precise about quality in use, there is little incentive to make this a priority in design. Product developers regard it as inevitable that different users will have different perceptions of the quality of the product. As user perceived quality is regarded as a subjective judgement outside the control of the developer, meeting the technical specification becomes the sole objective of design.


References

Bevan N (1995) Measuring usability as quality of use. Journal of Software Quality, 4, 115-130.

Bevan N (1997a) Quality and usability: A new framework. In: van Veenendaal, E, and McMullan, J (eds) Achieving software product quality, p 25-34, Tutein Nolthenius, Netherlands.

Bevan N (1997b) Usability Issues in Web Site Design. In: Proceedings of HCI Internationalí97, San Francisco, 24-30 August 1997.

Bevan N (1999) Getting started with user centred design. http://www.usability.serco.com/trump/ucdstart/index.html

Bevan N and Azuma M, Quality in use: Incorporating human factors into the software engineering lifecycle, in Proceedings of the Third IEEE International Software Engineering Standards Symposium and Forum (ISESS’97), p169-179, 1997.

Bevan N and Curson I (1998) Planning and Implementing User-Centred Design, CHI’98 Adjunct Proceedings.

Bevan N and Macleod M, Usability measurement in context. Behaviour and Information Technology, 13, 132-145, 1994.

Bevan N, Usability is quality of use, in Anzai & Ogawa (eds) Proc. 6th International Conference on Human Computer Interaction, July 1995. Elsevier, 1995b.

Bevan, N (1999) Industry standard usability tests. In: Human-Computer Interaction – INTERACT ’99 (Volume II). S. Brewster, A. Cawsey & G. Cockton (editors). British Computer Society, Swindon, UK.

Bias, G. and Mayhew, D. (Eds) (1994). Cost-Justifying Usability. Academic Press, NewYork.

Chalupnik and Rinehart 1992 cited at http://wwwwseast.usec.sun.com:80/usability/benefits.html

Daly-Jones, O, Thomas, C, Bevan, N. (1997) Handbook of user-centred design. Serco usability Services.

Earthy, J (1998) Usability Maturity Model: Processes. INUSE deliverable D5.1.4p, http://info.lut.ac.uk/research/husat/inuse/f3_assurance.html

Manning, H, McCarthy JC, Souza RK (1998) Why most web sites fail. Forrester Report, 3(7). Forrester Research Inc, Cambridge, MA.

Gartner Group (1997). Cited in Gibbs (1997).

Gibbs, W.W. (1997) Taking computers to task. Scientific American, 7/97. http://www.sciam.com/0797/issue/0797trends.html

Hurley, M (1997) cited in Gibbs, W.W. (1997)

IBM (1999)  Cost Justifying Ease of Use http://www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/23

INUSE, see http://www.npl.co.uk/inuse, 1998.

ISO 13407 (1999) User-centred design process for interactive systems.

ISO 9241-11 (1998) Ergonomic requirements for office work with visual display terminals (VDT)s - Part 11 Guidance on usability.

ISO/IEC FDIS 9126-1 (2000) Software product quality - Part 1: Quality model.

ISO/IEC TR 15504, Software process assessment, 1998.

Jones, T.O., and Sasser, W.E. (1995) Why satisfied customers defect. Harvard Business Review, 6(1), 88-99.

Karat, C. (1990). Cost-benefit analysis of usability engineering techniques. Proceedings of the Human Factors Society. Orlando. Fl.

Karat, C.M. (1992). Cost-justifying human factors support on development projects. Human Factors Society Bulletin, 35 (11), 1-8.

Karat (1993a) Usability Engineering in dollars and cents. IEEE Software 10(3), 88-89

Karat (1993b) Cost-benefit and business case analysis of usability engineering. SIGCHI’93, Amsterdam.

Karat, C. (1997). Cost-justifying usability engineering in the software life cycle. In Helander, M., Landauer, T., and Prabhu, P. (Eds), Handbook of Human-Computer Interaction. Elsevier Science, Amsterdam.

Keil, M, & Carmel, E. (1995). Customer developer links in software-development. Communications of the ACM. 38(5), pp33 - 44.

Kitsuse, A (1991) Why aren't computers... Across the Board (October) 28, 44-48

Landaur (1995) The Trouble with Computers. MIT Press

MacIntyre, F. Estep, K.W., and Sieburth, J.M. (1990). Cost of user-friendly programming. Journal of Forth Application and Research 6(2), 103-115

Martin and McClure (1993) cited at: http://wwwwseast.usec.sun.com:80/usability/benefits.html

Prokesch, 1995 Competing on customer service. Harvard Business Review, November-December 1995.

Pressman, R.S. (1992). Software Engineering: A Practitioner's Approach. McGraw Hill, NY.

Reed, S. (1992) Who defines usability? You do! PC Computing (Dec), 220-232

RESPECT, see http://www.npl.co.uk/respect, 1998.

Standish Group (1995) CHAOS Report. http://www.standishgroup.com/chaos.html

Thomas C and Bevan N, eds (1995) Usability Context Analysis: A practical guide, Version 4. Serco Usability Services, Teddington, UK

Wixon et al 1994

Wixon, D. and Jones, S. (1995). Usability for fun and profit: A case study of the re-design of the VAX RALLY. In: Human-Computer Interface Design: Success Stories, Emerging Methods, and Real-World Context. Marianne Rudisill, Clayton Lewis, Peter G. Polson, Tim McKay (eds) Morgan Kaufmann Publishers.








The database is protected by copyright ©hestories.info 2017
send message

    Main page