1、軟體品質(zhì)管理,Introduction,What is software quality? How can it be measured? How can it be measured before the software is delivered? Some key quality factors Some measurable indicators of software quality,Introduction,Think of an everyday object e.g. a chair How would you measure its “quality”? constructi

2、on quality? (e.g. strength of the joints,) aesthetic value? (e.g. elegance,) fit for purpose? (e.g. comfortable,) All quality measures are relative there is no absolute scale we can say A is better than B but it is usually hard to say how much better For software: construction quality(建造的品質(zhì))? softwa

3、re is not manufactured aesthetic value (美學(xué)上的價(jià)值)? but most of the software is invisible aesthetic value matters for the user interface, but is only a marginal concern fit for purpose? Need to understand the purpose,軟體品質(zhì)因素,Measuring Quality,The Quality Concepts (abstract notions of quality properties)

4、,Measurable Quantities (define some metrics),Counts taken from Design Representations (realization of the metrics),usability,minutes taken for some user task?,time taken to learn how to use?,complexity,count procedure calls?,information flow between modules?,reliability,run it and count crashes per

5、hour?,mean time to failure?,examples.,Four Key Quality Concepts,Reliability designer must be able to predict how the system will behave: completeness - does it do everything it is supposed to do? (e.g. handle all possible inputs) consistency - does it always behave as expected? (e.g. repeatability)

6、robustness - does it behave well under abnormal conditions? (e.g. resource failure) Efficiency Use of resources such as processor time, memory, network bandwidth This is less important than reliability in most cases Maintainability How easy will it be to modify in the future? perfective, adaptive, c

7、orrective Usability How easy is it to use?,McCalls Quality Factors,operation,revision,transition,Correctness reliability usability integrity efficiency,Maintainability Flexibility Testability,Portability Reusability Interoperability,General utility,portability,As-is utility,Maintainability,reliabili

8、ty,efficiency,usability,testability,understandability,modifiability,device-independence,self-containedness,accuracy,completeness,robustness/integrity,consistency,accountability,device efficiency,accessibility,communicativeness,self-descriptiveness,structuredness,conciseness,legibility,augmentability

9、,Source: See Blum, 1992, p176,Product operation,usability,Product revision,Product transition,integrity,maintainability,testability,reusability,portability,interoperability,operability,training,I/O volume,Access control,Access audit,Storage efficiency,consistency,instrumentation,expandability,genera

10、lity,Self-descriptiveness,modularity,machine independence,s/w system independence,comms. commonality,efficiency,correctness,reliability,flexibility,communicatativeness,I/O rate,execution efficiency,Source: See van Vliet 2000, pp111-3,traceability,completeness,accuracy,error tolerance,simplicity,conc

11、iseness,data commonality,Measurable Predictors of Quality,Simplicity the design meets its objectives and has no extra embellishments can be measured by looking for its converse, complexity: control flow complexity (number of paths through the program) information flow complexity (number of data item

12、s shared) name space complexity (number of different identifiers and operators) Modularity different concerns within the design have been separated can be measured by looking at: cohesion (how well components of a module go together) coupling (how much different modules have to communicate),Quality

13、and how to achieve it,Product quality always an issue After WWII, industry in the US and elsewhere has substantially improved quality via extensive testing and statistical quality control Japanese industry has followed a different concept, a.k.a. total quality initiative (Deming, Juran), where quali

14、ty control is an intrinsic aspect of the production process, not a post-production activity In other words, you should design the quality product and build it right, rather than build it and then make sure it has good quality,SEI and the Capability Maturity Model,Software quality improvement has led

15、 to the establishment of the Software Engineering Institute at Carnegie-Mellon University Capability Maturity Model (CMM) a framework to assess the maturity level of an organizations software development and management processes CMM consists of five levels of maturity as measured by a set of guideli

16、nes called the key process areas Higher levels increase competitiveness, reduce risk Levels are monotonic: level n includes all the characteristics of all the levels below, n-1, n-2, etc.,Capability Maturity Model,Level 1Initial,end Software development follows no prescribed process You dont know wh

17、ere you stand, or when will you finish, or what you will get when you finish You have to be happy with what you get at the,Level 2Repeatable,Project management processes and practices are established, in order to track project costs, schedules, and functionality Still, you just passively track whats

18、 going on and dont know exactly what you should do,Level 3Defined,A standard system development process (sometimes called a “methodology”) purchased or developed, and integrated throughout the IT unit of the organization which means that there is a prescribed sequence of steps to follow, plus you mo

19、nitor the outcomes,Level 4Managed,Measurable goals for quality and productivity are established Constant monitoring gives real-time data about the status of the process and product Process parameters adjusted in order to obtain desired outcomes (that is management, after all),Level 5Optimizing,The s

20、tandardized system development process continuously monitored and improved, based on measures and data analysis established in Level 4 Changes can be made to process parameters But we can make change in the process itself (e.g., choose different sequence, change priorities, even introduce additional

21、 steps or delete some that do not contribute to the process/product quality),CMM certification process,CMM assessment performed by qualified personnel from SEI, through interviews and analysis of procedures and documents used in the organization Levels can be awarded to organizations or specific pro

22、jects/project groups within an organization. Each level has its own questionnaire, with a number of mandatory and optional questions In order to qualify for a specific level, Yes answers must be present in each group (though minimum percentages are different),CMM certification process,Each questionn

23、aire focuses on a number of the so-called Key Process Areas (KPAs) Each level of CMM scale has an associated set of Key Process Areas (KPAs) To qualify for a specific level, issues related to KPAs for that level must be addressed To progress from one level to the next one up, improvements in appropr

24、iate KPAs have to be made Levels are monotonic makes no sense to concentrate on KPAs for, say, level 4, if the development process is still at level 1 or 2,關(guān)鍵流程領(lǐng)域（Key Process Area, KPA),Some problems with CMM,(at least initially) Focuses on project/process management, not product development Ignores

25、 use of certain advanced technologies Did not incorporate risk analysis as a KPA Did not define its domain of applicability,But things have evolved/diversified:,Models that the SEI is currently involved in developing, expanding, or maintaining: Capability Maturity Model Integration (CMMISM) SW-CMM C

26、apability Maturity Model for Software P-CMM People Capability Maturity Model SA-CMM Software Acquisition Capability Maturity Model SE-CMM Systems Engineering Capability Maturity Model IPD-CMM Integrated Product Development Capability Maturity Model,ISO9000,ISO9000 series of quality standards, some o

27、f which are related to software and IS development ISO principles: Say what you will do Do as you have said Prove that you did so NOTE: this is a very simplified view the certification process and the documentation required to go through it are quite extensive and detailed ISO certification is perfo

28、rmed through national standards organizations,CMM vs. ISO,Partial overlap, but doubts remain It seems possible that an organization can be certified for ISO9000 and still be at level 1 of the CMM scale, or be at level 3 or 4 and still unable to obtain ISO9000 certification Note: neither ISO nor CMM

29、prescribe any ABSOLUTE level of quality but the improvements in process management are believed to have a beneficial effect on quality,SQA: What is SQA?,SQA is the process of assuring people that every effort has been made to ensure that software products have the desired level of reliability, maint

30、ainability, usability, and salability. SQA是一種執(zhí)行軟體評(píng)估與衡量的活動(dòng) (Baker and Fisher) Set of systematic activities providing evidence of the ability of the software process to produce a software product that is fit to use (G. Schulmeyer and J. McManus, Software Quality Handbook, Prentice Hall, 1998.),Softwar

31、e Quality Assurance,How do you assure the quality of your software? Good processes Good documentation and artifacts Accountability Learn and improve,The Objectives of SQA,Monitoring processes and products throughout the software development lifecycle to ensure the quality of the delivered product(s)

32、 Monitoring the processes Provides management with objective(客觀的) feedback regarding process compliance(承諾) to approved plans, procedures, standards, and analyses Monitoring the products Focus on the quality of product within each phase of the SDLC e.g., requirements, test plan, architecture, etc. O

33、bjective: identify and remove defects throughout the lifecycle, as early as possible,SQA: An SQA Program,Minimizing number of defects in delivered s/w Creating mechanisms for controlling software development and maintenance so that costs and schedules can be met Making certain that the delivered pro

34、duct can be used in its intended marketplace Improving the quality of future products,Software defects,Mistakes made at any point in the software process Requirements, design, coding, maintenance Consequences Inconvenience, loss of service, financial loss, equipment damage, injury, death,The percent

35、age of defects found by various methods,The truth of defects,1. The later in the life cycle that an error is detected the more expensive it is to repair. 2. Errors remain latent and are not detected until well after the stage at which they are made. 54% of errors detected after coding and unit testi

36、ng. 45% of these errors were requirements and design errors. 3. There are numerous requirements errors. Estimates indicate that 56% of all errors are errors during the requirements stage. 4. Requirements errors are typically non-clerical.,Relative Cost to Repair based on when it was found,Requiremen

37、ts - 1 time Design - 3-6 times Code - 10 times Unit test - 15-40 times System test - 30-70 times Field operation - 40-1000 times,When should quality assurance be done?,At every stage in the software process,The Process of SQA,定義品質(zhì)需求,制定SQA計(jì)畫,需求評(píng)估,設(shè)計(jì)評(píng)估,測(cè)試評(píng)估,需求分析,設(shè)計(jì),測(cè)試,評(píng)估客戶滿意需求程度,measurement,feedback,f

38、eedback,功能與完整性,系統(tǒng)完整性與一致性,執(zhí)行效率與正確性,SQA Planning,IEEE Std 730-2002 Standard for Software Quality Assurance Plans 12 pages IEEE Guide for Software Quality Assurance Planning draft P730.2 87 pages,SQA Planning,IEEE SQAP Purpose (Section 1 of the SQAP) Reference documents (Section 2 of the SQAP) Manageme

39、nt (Section 3 of the SQAP) Documentation (Section 4 of the SQAP) Standards, practices, conventions, and metrics (Section 5 of the SQAP) Reviews and audits (Section 6 of the SQAP) Test (Section 7 of the SQAP) Problem reporting and corrective action (Section 8 of the SQAP) Tools, techniques, and metho

40、dologies (Section 9 of the SQAP) Code control (Section 10 of the SQAP) Media control (Section 11 of the SQAP) Supplier control (Section 12 of the SQAP) Records collection, maintenance, and retention (Section 13 of the SQAP) Training (Section 14 of the SQAP) Risk management (Section 15 of the SQAP),C

41、ontents of SQA Plan (sect 1 2.5M:1) Software organizations need to assess this,SQA: Quality Software,People,Process,Management Discipline,SQA,SQA: Pursuing SQA,What organizations are doing Nothing (42%) Slogans(口號(hào)) - “Quality is Job One!” (4%) Improved testing (24%) Focus on defect prevention (20%) Process Improvements (9%) Other. (1%),SQA: Software Reliability (MTBF),Putnams Software Reliabil