MetricsandBenchmarkingPresentedtoWayneKellyPrincipalConsultantQualityAssurancePracticeContentsIntroductiontoMetricsWhatisMetrics?WhyMetrics?ProjectMetricsProgramObjectivesMetrics-CustomerNeedMeasuresandMetricsImprovementCycleMeasures–DataElementsToolsforMeasurementMetricsCalculationMetricsRoadmapMetricsAnalysis-ExamplesItisnotsufficienttoproducesoftwaredefectmetrics-likehowmanyraised,howmanyoutstanding,fixedbyseverityandurgency.Wemustprovethattheactivitiesofthetestteamareacosteffectiveandefficientuseofstaff.TestMetricsWithoutTears-DavidVaughnQuoteIntroductiontoMetricsWhatIsMetrics?Metrics-Derivedfromtheword“metre”-Quantitativeindicatorofperformanceofas/wprocessorproduct.Metricsisacannot-do-withoutprojectmanagementtoolVisibilityintoprojectexecutionAnalysisofprocessstrengthsandweaknessesyoucannotmanageorimprovewhatyoucannotmeasure“Youcan’tcontrolwhatyoucan’tmeasure”-TomDeMarcoEstablishameasurementandcontinousimprovementcultureacrossthecompany.ImprovedqualityofsoftwareacceptedintoproductionHelpmanagementandemployeesmakewell-informedanddecisionsAlertmanagementaboutriskstoprojectgoals,schedulesandsoftwarequalityProvidesanobjectiveassessmentofthestateofthesoftwareWhyMetrics?WhyMetrics?ContinuedMetricsprovideinputsforfutureestimationsandplanningTheactualperformanceofthepastinquantitativetermsbecomesastartingpointforfutureestimatesHelpsinsettinggoalsandtargetsBetterresourceplanning

MetricsidentifytheareasforimprovementsForexample,metricscanidentifythephasewhichintroducesthemaximumnumberoferrorsMetricscanbeusedtoeliminateproblemareasandrootcausesSeniorManagementProjectManagementProjectTeamsBenchmarkingSettingimprovementgoalsPerformanceacrossgroupsHowlargeisproject,ReducedefectsBetterestimatesattasklevelConsistency,predictabilityBetterestimation,projectcontrolEffort,defect,schedule,sizeCostofQuality,Effort&DefectDistribution,Quality,ProductivityEstimationAccuracy,Quality,Productivity,DevelopmentCost,ROIWhyisitMeasuredWhatisMeasured

MeasurementsdrivecontinuousimprovementacrosstheorganizationMetricsProgramObjectivesMustbeclearlydefinedandPOSITIVE

DrivenbycriticalbusinessobjectivesMeasuresattheenterprise,program,projectandapplicationlevelsProvidequantitativeinformationofkeydrivers(i.e.schedule,estimatingaccuracy,defects)thatimprovesdecisionmakingintimetoaffectthebusinessoutcomeandsuccessfulmanagementofaprojectCUSTOMERPERSPECTIVE"WHATCUSTOMERNEEDSMUSTWESERVE?”LEARNINGPERSPECTIVE"TOACHIEVEOURGOALS,HOWMUSTOURORGANIZATIONLEARN,INNOVATEANDIMPROVE?”“WHATFINANCIALOBJECTIVESMUSTWEACCOMPLISH?WHATISTHECOSTPerFUNCTIONPOINT?”FINANCIALPERSPECTIVE“TOSATISFYOURCUSTOMERS,INWHICHINTERNALBUSINESSPROCESSESMUSTWEEXCEL?”INTERNALPERSPECTIVEMultipleperspectivesofaMetricsprogramMeasuresandMetricsMeasuresMeasuresarethefundamentalbuildingblocksofameasurementsystemMeasuresDefectsSizeEffortScheduleCostMetricsMetricsarederivedoutofmeasuresMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityReviewEffectivenessInternalPerspectiveMetricsMetricsarederivedoutofmeasuresMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityRevieweffectivenessReturnonInvestmentTotalCostofOwnershipInternalPerspectiveFinancialPerspectiveImprovementCycleGatherAnalyseActImproveMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityRevieweffectivenessReturnonInvestmentTotalCostofOwnershipLearningPerspectiveMeasures-DataElementsDefects#ofdefectsDefectdescriptionStageinjected&detectedSeverityStatusEffortPlannedvs.actualActivities(Task,review,rework)ResourcegroupsWeekendingdatePhasewisepersoneffortMeasuresMetricsDefectsSizeEffortScheduleCost

ScheduleStartdate(Planned,Actual)Enddate(Planned,Actual)Milestonedate(Planned,Actual)SizeProgramreferenceLanguageusedSourceLinesofCode(InitialandFinal)FPoutofSLOCCostPerPhasePlanVsActGroupMeasures––ToolsUsed

toAutomaticallyCapturetheDataElementsDefectsExcelTemplateClearQuestDefectLogsEffortExcelTemplateTimeTrackingMSProjectPlanViewMeasuresMetricsDefectsSizeEffortScheduleCost

ScheduleWBSExcelTemplateMSProjectPlanViewSizeSLOCcountersdependingonthelanguageLaborUnitCostProjectBudgetsMeasures––SizeExampleoftoolsMeasures––Effort

ExampleofToolsMeasures––Defect

ExampleofToolsMetricsAreCalculatedUsingMeasuresMetricsUnitofMeasureMeasuresFormulaProductivityQualityDefectRemovalEfficiencyCostofQualityReviewEffectivenessFP/pmDelivereddefects/pm%%%SizeEffortDefectsSizeDefectsEffortDefectsSizeoftheproduct/TotalEffortinpersonmonthsDefectsfoundduringAcceptanceTesting/SizeOftheproductTotaldefectsdetectedbeforeQA/TotaldefectsdetectedincludingQAReview+Test+Training+Reworkeffort*100/TotaleffortfortheprojectDefectsdetectedbymeansofreviews/Defectsdetectedbymeansofreviews+testingMetricsRoadmapMetricProgramCriticalSuccessFactorsDefineclearobjectivesandkeepthemPOSITIVEDrivenbykeybusinessissuesEnsurethattheprocessisrolledouteffectively–targetasubset,test,validateandimproveBeginwithmanualmethodsthen,considerautomation(DataDrill)Train,Train,TrainProvidetimefortheprocessestomature;variabilitytendstobeveryhighinitially–expectnoiseandchangeAssisttheprojectteamsinunderstanding,analyzingandimprovingtheprocessesthroughmetricreportingCommunicate,Communicate,CommunicateObtainExecutiveSponsorDefinethebusinessandITobjectivesforthemeasurementandimprovementprogram(mustbedatadriven)ConductJADsessiontoidentifypossiblemeasuresSelectmeasuresthatalignwithprogramobjectivesandimproveITperformanceIdentifytheinformationrequirement,author,datasourceandtoolsIdentifythecoresetorphasedimplementationplanformeasuresandvalidatewithauthor,contributoranduserthattheyalignwithproject,departmentandorganizationalgoalsImplementingAMetricsProgramDevelopthemetriccollectionandreportingsolutionincludingautomation,collection,graphingandreportingprocessesTest,finetune,verifyandpilotallreportsandmeasuresPackagemultiplemeasuresintoametricsreleaseIdentifyandtrainacoreteamofauthors,contributorsandusersintheinterpretationanduseofallmeasurestoimprovebusinessresultsProductionrolloutbeginsMonitor,improve,retireandimplementnewmeasuresastheorganizationmaturesinprocessandmeasurementcapabilitiesCapture,communicateandinformallofProgramresultsIntegrateintoaBusinessperformanceProgramImplementingAMetricsProgramAlignmentwiththeITGovernanceand/orPMOProgramAlignmentwithcertificationobjectives(CMM/CMMI/Iso,etc)DefinitionofmeasurementrolesandresponsibilitiesIntegrationintotheteam,departmentandorganizationalprocessesConsistencyinthecodingandcollectionofmetricsProduction-izingthecollectionandreportingprocessOverallreportdesignandinterpretationTQMleadershipandfacilitationAlignmentwiththeorganizationalscorecardRelevantindusrtyfiguresforbenchmarkingobjectivesMetricsProgramDependenciesTheBusinessPerformanceProgram1Initial2Repeatable3Defined4Managed5OptimizingDatadefinitionandcollectionisstandardizedacrosstheOrganizationDataisusedtounderstandtheprojectQuantitativelyEstablishPCBFocusonreducingvariabilityDataiscollectedandusedinalldefinedprocessesDataissystematicallysharedacrossprojects

StandardizemetricscollectionImprovemetricsreliabilityOrganizationlevelProcessDatabaseIndividualprojectsplanandmanagedataStartcollectionofmeasuresProjectlevelmetrics

Dataisusedtoevaluateandselectprocessimprovements

ImprovemeancapabilityMetricsRoadmapinTermsofCMMLevelsPredictabilityGoesUpLevel2-Whatyoucanexpect……DeploymentMetricsatProjectlevelformonitoringandcontrollingSQAisidentifiedtohelpprojectsinmetricscollectionToomuchoffollow-upfordataRegularstatustrackingofprojectsusingmetricsBenefitsCultureisbeingestablishedProjectleveldecisionsaretakenbasedonmetricsTrackingbecomeseasierinprojectsAwareness/CultureMetricsidentificationanddefinitionOrientationonvariousmetricsAlittlecynicismfromprojectsideFearofmetricsusageagainstindividualsInfrastructureUseofsimpletoolsformetricsManualcollationofdatafromprojectsLevel3-Whatyoucanexpect……DeploymentSupportfromSQAformetricsgetsreducedCapturingmetricsbecomesawayoflifeImprovementingoodnessofdataBenefitsDecisionsaretakenatorganizationlevelbasedonmetricsProcessimprovementsaredrivenbasedonmetricsImprovementsinEffortestimationaccuracy,CostofQuality,Scheduleadherence,Productivity,Quality,etc..Awareness/CulturePurposeofmetricscollectionisknowntoeveryoneCynicismgoesawayManyaretrainedinMetrics,FPcounting,EstimationusingpastprojectsdataInfrastructureITwiderepositoryformetricsStandardizedtoolsacrossprojectsCollationofdatabecomeseasierMetricsAnalysis-ExamplesExample–EffortanalysisExamples–DefectanalysisExamples–CodechangesExamples–DefectanalysisExamples–DefectcategorizationQuestionsandAnswers

YourFavoriteandMostEffectiveGraphsWayneKelly–PrincipalConsultantLizaRoberts–BusinessDevelopmentManager804-334-28959、靜夜四無(wú)鄰鄰，荒居舊業(yè)業(yè)貧。。2022/12/252022/12/25Sunday,December25,202210、雨中黃葉樹樹，燈下白頭頭人。。2022/12/252022/12/252022/12/2512/25/20223:53:02PM11、以我獨(dú)沈久久，愧君相見見頻。。2022/12/252022/12/252022/12/25Dec-2225-Dec-2212、故人江海別別，幾度隔山山川。。2022/12/252022/12/252022/12/25Sunday,December25,202213、乍見翻疑夢(mèng)夢(mèng)，相悲各問問年。。2022/12/252022/12/252022/12/252022/12/2512/25/202214、他鄉(xiāng)鄉(xiāng)生白白發(fā)，，舊國(guó)國(guó)見青青山。。。25十十二二月20222022/12/252022/12/252022/12/2515、比不不了得得就不不比，，得不不到的的就不不要。。。。。十二月月222022/12/252022/12/252022/12/2512/25/202216、行行動(dòng)動(dòng)出出成成果果，，工工作作出出財(cái)財(cái)富富。。。。2022/12/252022/12/2525December202217、做做前前，，能能夠夠環(huán)環(huán)視視四四周周；；做做時(shí)時(shí)，，你你只只能能或或者者最最好好沿沿著著以以腳腳為為起起點(diǎn)點(diǎn)的的射射線線向向前前。。。。2022/12/252022/12/252022/12/252022/12/259、沒沒有有失失敗敗，，只只有有暫暫時(shí)時(shí)停停止止成成功功??！。。2022/12/252022/12/25Sunday,December25,202210、很很多多事事情情努努力力了了未未必必有有結(jié)結(jié)果果，，但但是是不不努努力力卻卻什什么么改改變變也也沒沒有有。。。。2022/12/252022/12/252022/12/2512/25/20223:53:02PM11、成功就就是日復(fù)復(fù)一日那那一點(diǎn)點(diǎn)點(diǎn)小小努努力的積積累。。。2022/12/252022/12/252022/12/25Dec-2225-Dec-2212、世間成成事，不不求其絕絕對(duì)圓滿滿，留一一份不足足，可得得無(wú)限完完美。。。2022/12/252022/12/252022/12/25Sunday,December2