ModelingandEvaluationChapter3Wherewearenow1.DataAnalytics2.DataPreparationandCleaning3.ModelingandEvaluation4.Visualization5.TheModernAudit6.AuditAnalytics7.KeyPerformanceIndicators8.FinancialStatementAnalyticsObjectivesLO3-1Whatdoyouneedtoknowaboutdatamodels?LO3-2Whataresomeunsupervisedandsupervisedapproaches?LO3-3Howdoyouperformprofiling?LO3-4Howdoyouperformdatareduction?LO3-5Howdoyouperformregression?LO3-6Howdoyouperformclassification?LO3-7Howdoyouperformclustering?IntheIMPACTcycle,we’renowgoingtolookatPerformingtheTestPlanandAddressingtheResults.IdentifythequestionsMasterthedataPerformtestplanAddressandrefineresultsCommunicateinsightsTrackoutcomesExhibit1-1TheIMPACTCycleWhatdoyouneedtoknowaboutdatamodels?LO3-1Whatdoyouneedtoknowaboutdatamodels?Atargetisanexpectedattributeorvaluethatwewanttoevaluate.Example:FraudscoreInterestrateAclassisamanuallyassignedcategoryappliedtoarecordbasedonanevent.Example:Accept/RejectFraud/NotfraudWhatdoyouneedtoknowfirst?Anunsupervisedapproachisusedwhenyoudon’thaveaspecificquestion.Example:“Doourvendorsformnaturalgroupsbasedonsimilarattributes?”Asupervisedapproach

isusedwhenyouaretryingtopredictafutureoutcomebasedonhistoricaldata.Example:“Willanewvendorshipalargeorderontime?”Whataresomeunsupervisedapproaches?Clustering

–findundiscoverednaturalgroupingsinthedataCo-occurrencegrouping–eventsthathappentogetherProfiling

–identifytypicalbehaviorinthedataDatareduction–filterorgroupthedatatosimplifytheanalysisWhataresomesupervisedapproaches?Classification

–predictwhetherdatabelongstooneclassoranotherSimilaritymatching–groupdatabyattributesRegression

–predictaspecificvalueLinkprediction–socialnetworksCausalmodeling–aneventinfluencesanotherUseaflowcharttoidentifyanappropriateapproach.Q.Whatisthemaindifferencebetweensupervisedandunsupervisedmethods?Howdoyouperformprofiling?LO3-2Profilingreliesongatheringsummarystatisticsandidentifyingoutliers.Identifytheobjectsoractivityyouwanttoprofile.Determinethetypesofprofilingyouwanttoperform.Setboundariesorthresholdsfortheactivity.Interprettheresultsandmonitortheactivityand/orgeneratealistofexceptions.Followuponexceptions.Whataresomeexamplesofprofiling?Internalauditorsanalyzetravelandentertainmentexpensesforviolationsofinternalcontrols.Managersuseprofilingtocomparevariancesfromtargetranges.Whataresomeexamplesofprofiling?Inthecontinuousaudit,anauditormayuseBenford’sLawtoevaluatethefrequencydistributionofthefirstdigitsfromalargesetofnumericaldata.Q.Profilingisusedinlawenforcementforoffenderorcriminalprofiling.Howdoesthiscomparewithprofilingaccountingdata?Howdoyouperformdatareduction?LO3-3Datareductionisusedtofilterresults.1.Identifytheattributeyouwouldliketoreduceorfocuson.2.Filtertheresults.3.Interprettheresults.4.Followupontheresults.Whataresomeexamplesofdatareduction?InternalauditorsmaywanttolocatepaymentsmadetoSquarevendors.FinancialstatementanalystswilltakeXBRLinstancedocumentsandfilteronspecifictags.Q.HowmightthedatareductionapproachbeusedtosimplifyT&Eexpenses?Howdoyouperformregression?LO3-4Regressionallowstheaccountanttodevelopmodelstopredictexpectedoutcomes.Identifythevariablesthatmightpredictanoutcome.Determinethefunctionalformoftherelationship.Identifytheparametersofthemodel.Dependentvariable=f(independentvariables)Whataresomeexamplesofregression?Inmanagerialaccounting,regressionmaypredictemployeeturnover:Employeeturnover=f(currentprofessionalsalaries,healthoftheeconomy[GDP],salariesofferedbyotheraccountingfirmsorbycorporateaccounting,etc.)Inauditing,regressionmaybeusedtodeterminetheappropriatenessofallowanceaccounts:Allowanceforloanlosesamount=f(currentagedloans,loantype,customerloanhistory,collectionssuccess)Q.Regressionisusedtopredictstockreturnsfollowingarestatementofpastearnings.Whatfactors(independentvariables)doyouthinkmightpredictthechangeinstockprice(dependentvariable)?Howdoyouperformclassification?LO3-5Thegoalofclassificationistopredictwhetheranindividualweknowverylittleaboutwillbelongtooneclassoranother.Identifytheclassesyouwishtopredict.Manuallyclassifyanexistingsetofrecords.Selectasetofclassificationmodels.Divideyourdataintotrainingandtestingsets.Generateyourmodel.Interprettheresultsandselectthe“best”model.Whatelsedoyouneedtoknowaboutclassification?Trainingdataareexistingdatathathavebeenmanuallyevaluatedandassignedaclass.Testdataareexistingdatausedtoevaluatethemodel.Decisiontreesareusedtodividedataintosmallergroups.Decisionboundariesmarkthesplitbetweenoneclassandanother.Whatelsedoyouneedtoknowaboutclassification?Pruning

removesbranchesfromadecisiontreetoavoidoverfittingthemodel.Whatelsedoyouneedtoknowaboutclassification?Linearclassifiersareusefulforrankingitemsratherthansimplypredictingclassprobability.Theseareusefulfordeterminingthereallyimportantvalues,suchasvaluablecustomers,orwhichtransactionsaremostlikelyfraudulent.Whatelsedoyouneedtoknowaboutclassification?Supportvectormachineisadiscriminatingclassifierthatisdefinedbyaseparatinghyperplanethatworksfirsttofindthewidestmargin(orbiggestpipe)andthenworkstofindthemiddleline.Howdoweevaluateclassifiers?Trytoavoidoverfitting,ormodelsthataretooaccurate.Theyareactuallyprettybadapredictingafutureobservation.Lookforthesweetspotwherewemaximizetheaccuracyofthetestingdata.Q.Ifwearetryingtopredictwhetheraloanwillberejected,wouldyouexpectcreditscoretobepositivelyornegativelyassociatedwithloanrejection?Howdoyouperformclustering?LO3-6Howdoyouperformclustering?Clusteringisusedtoidentifygroupsofsimilardataelementsandtheunderlyingdriversofthosegroups.Clusteringalgorithmscalculatetheminimumdistanceofallobservationsandgroupsthoseelements.Whataresomeexamplesofclustering?Internalauditorscanuseclusteringtoidentifygroupsoftransactionsthatmayindicateriskorfraudininsuranceorotherpayments.Q.WhatarethreeclustersofcustomerswhomightshopatWalmart?SummaryInthischapter,weaddressedthethirdstepoftheIMPACTcyclemodel:the“P”for“performingtestplan.”Thatis,howarewegoingtotestoranalyzethedatatoaddressaproblemwearefacing?Basedonourproblemandthedataavailable,weprovidedaflowchartthathelpstheanalyst