培訓(xùn)時間：2025年8月5日培訓(xùn)講師：XXX培訓(xùn)課件

課堂規(guī)則Open—開放的心態(tài)、積極參與、勇敢發(fā)問

Close—封閉的環(huán)境，不要受外界的干擾，請將手機關(guān)機或靜音請勿隨意走動、交談感謝您的配合Whatis

Meta-analysis?NCRMResearchMethodsFestivalUniversityofOxfordProf.HerbMarshMs.AlisonO’MaraDr.Lars-ErikMalmbergDeptofEducation,UniversityofOxfordPrimaryversussecondarydataanalysisTraditionally,socialscienceresearcherscollectandanalysetheirowndata(referredtoasprimarydata).Secondarydataanalysisisbasedondatacollectedbysomeoneelse(or,perhaps,re-analysisofyourownpublisheddata).Thereareatleastfourlogicalperspectivestothisissue:1.Meta-analysis--systematic,quantitativereviewofpublishedresearchinaparticularfield,thefocusofthispresentation.2.Systematicreview

--systematic,qualitativereviewofpublishedresearchinaparticularfield3.SecondaryDataAnalyses--usinglarge(typicallypublic)databases4.Re-analysesofpublishedstudies--ofteninwayscriticaloftheoriginalstudy.4Whatismeta-analysis?SystematicsynthesisofvariousstudiesonaparticularresearchquestionDoboysorgirlshavehigherself-concepts?CollectallstudiesrelevanttoatopicFindallpublishedjournalarticlesonthetopicAneffectsize(the‘dependentvariable’)iscalculatedforeachoutcomeDeterminethesize/directionofgenderdifferenceforeachstudy“Contentanalysis”codecharacteristicsofthestudy;age,setting,ethnicity,self-conceptdomain(math,physical,social),etc.Effectsizeswithsimilarfeaturesaregroupedtogetherandcompared;testsmoderatorvariablesDogenderdifferencesvarywithage,setting,ethnicity,self-concept,domain,etc.5AblendofqualitativeandquantitativeapproachesCoding:theprocessofextractingtheinformationfromtheliteratureincludedinthemeta-analysis.Involvesnotingthecharacteristicsofthestudiesinrelationtoapriorivariablesofinterest(qualitative)Effectsize:thenumericaloutcometobeanalysedinameta-analysis;asummarystatisticofthedataineachstudyincludedinthemeta-analysis(quantitative)Summariseeffectsizes:centraltendency,variability,relationstostudycharacteristics(quantitative)6Benefitsofmeta-analysisComparedtotraditionalliteraturereviews:(1)thereisadefinitemethodologyemployedintheresearchanalysis;and

(2)theresultsoftheincludedstudiesarequantifiedtoastandardmetricthusallowingforstatisticaltechniquesforfurtheranalysis.Thereforelessbiasedandmorereplicable7Benefitsofmeta-analysisIncreasedpower:increasesthechanceofdetectingatruetreatmenteffectImprovedprecision:withmoreinformationthanasinglestudy,thetreatmenteffectestimateisimprovedWhenstudy-to-studyvariationinresults(whichistypical)canevaluatedifferencesinrelationtostudycharacteristics.CandelveintoresearchquestionsnotexploredbytheindividualstudiesEasytointerpretsummarystatistics(usefulifcommunicatingfindingstoanon-academicaudience)8Theessenceofgoodscienceisreplicableandgeneralisableresults.Dowegetthesameanswertoimportantresearchquestionswhenwerunthestudyagain?Theprimaryaimsofmeta-analysisistotestthegeneralisabilityofresultsacrossasetofstudiesdesignedtoanswerthesameresearchquestion.Aretheresultsconsistent?Ifnot,whatarethedifferencesinthestudiesthatexplainthelackofconsistency?9Whyismeta-analysisimportant?GeneralisabilityWhyismeta-analysisimportant?Meta-analysisisanincreasinglypopulartoolforsummarisingresearchfindings;literaturereviewmethodofchoiceinmanydisciplinesWidely-cited.Ifthereisagoodmeta-analysisrelevanttoyourstudy,youhavetociteitRelieduponbypolicymakersImportantthatweunderstandthemethod,whetherweconductorconsumemeta-analyticresearchShouldbeoneofthetopicscoveredinallintroductoryresearchmethodologycourses10Whenismeta-analysisappropriate?Thereexistsacriticalmassofcomparablestudiesdesignedtoaddressacommonresearchquestion.Dataarepresentedinaformthatallowsthemeta-analysttocomputeaneffectsizeforeachstudy.Characteristicsofeachstudyaredescribedinsufficientdetailtoallowmeta-analyststocomparecharacteristicsofdifferentstudiesandtojudgethequalityofeachstudy.11Schulze,R.(2007).Thestateandtheartofmeta-analysis.Zeitschrift

für

Psychologie/JournalofPsychology,215,87-89.

12Thenumberofmeta-analysesisincreasingatarapidrate.Wherearemeta-

analysesdone?1314Psychology:CitationsPsychology:ArticlesPsychology:WhereitallbeganAmato,P.R.,&Keith,B.(1991).Parentaldivorceandthewell-beingofchildren:Ameta-analysis.PsychologicalBulletin,110,26-46.

TimesCited:471Linn,M.C.,&Petersen,A.C.(1985).Emergenceandcharacterizationofsexdifferencesinspatialability:Ameta-analysis.ChildDevelopment,56,1479-1498.

TimesCited:570Johnson,D.W.,&etal(1981).Effectsofcooperative,competitive,andindividualisticgoalstructuresonachievement:Ameta-analysis.PsychologicalBulletin,89,47-62.

TimesCited:426Tett,R.P.,Jackson,D.N.,&Rothstein,M.(1991).Personalitymeasuresaspredictorsofjobperformance:Ameta-analyticreview.PersonnelPsychology,44,703-742

TimesCited:387Hyde,J.S.,&Linn,M.C.(1988).Genderdifferencesinverbalability:Ameta-analysis.PsychologicalBulletin,104,53-69.

TimesCited:316Iaffaldano,M.T.,&Muchinsky,P.M.(1985).Jobsatisfactionandjobperformance:Ameta-analysis.PsychologicalBulletin,97,251-273.

TimesCited:263.15Education:WidelyCitedMeta-analysesDeWolff,M.,&vanIJzendoorn,M.H.(1997).Sensitivityandattachment:Ameta-analysisonparentalantecedentsofinfantattachment.ChildDevelopment,68,571-591.

TimesCited:340Wellman,H.M.,Cross,D.,&Watson,J.(2001).Meta-analysisoftheory-of-minddevelopment:Thetruthaboutfalsebelief.ChildDevelopment,72,655-684.

TimesCited:276Cohen,E.G.(1994).Restructuringtheclassroom:Conditionsforproductivesmallgroups.ReviewofEducationalResearch,64,1-35.TimesCited:235Hansen,W.B.(1992).School-basedsubstanceabuseprevention:Areviewofthestateoftheartincurriculum,1980-1990.HealthEducationResearch,7,403-430.

TimesCited:207Kulik,J.A.,Kulik,C-L.,Cohen,P.A.(1980).EffectivenessofComputer-BasedCollegeTeaching:AMeta-AnalysisofFindings.ReviewofEducationalResearch,50,525-544.

TimesCited:198.16Business/Management:WidelyCitedMeta-analysesSheppard,B.H.,Hartwick,J.,&Warshaw,P.R.(1988).Thetheoryofreasonedaction:Ameta-analysisofpastresearchwithrecommendationsformodificationsandfutureresearch.JournalofConsumerResearch,15,325-343.

TimesCited:515Jackson,S.E.,&Schuler,R.S.(1985).Ameta-analysisandconceptualcritiqueofresearchonroleambiguityandroleconflictinworksettings.OrganizationalBehaviorandHumanDecisionProcesses,36,16-78.

TimesCited:401TornatzkyLg,KleinKj.(1994).Innovationcharacteristicsandinnovationadoption-implementation-Ameta-analysisoffindings.IEEETransactionsOnEngineeringManagement,29,28-4.TimesCited:269.LoweKB,KroeckKG,SivasubramaniamN.(1996).Effectivenesscorrelatesoftransformationalandtransactionalleadership:Ameta-analyticreviewoftheMLQliterature.LeadershipQuarterly,7,385-425.TimesCited:203.ChurchillGA,FordNM,HartleySW,etal.(1985).Title:Thedeterminantsofsalespersonperformance-Ameta-analysis.JournalOfMarketingResearch,22,103-118.TimesCited:189.17MostWidelyCitedMeta-analysesareinMedicineJadadAR,MooreRA,CarrollD,etal.(1996).Assessingthequalityofreportsofrandomizedclinicaltrials:Isblindingnecessary?ControlledClinicalTrials,17,1-12.TimesCited:2008BousheyCj,BeresfordSaa,OmennGs,Et.Al.(1995).Aquantitativeassessmentofplasmahomocysteineasariskfactorforvascular-disease-Probablebenefitsofincreasingfolic-acidintakes.JAMA-journalOfTheAmericanMedicalAssoc,274,1049-1057.TimesCited:2,128AlbertiW,AndersonG,BartolucciA,etal.(1995).Chemotherapyinnon-small-celllung-cancer-Ametaanalysisusingupdateddataonindividualpatientsfrom52randomizedclinical-trials.BritishMedicalJournal,311,899-909.TimesCited:1,591BlockG,PattersonB,SubarA(1992).Fruit,vegetables,andcancerprevention-Areviewoftheepidemiologicevidence.NutritionAndCancer-anInternationalJournal,18,1-29.TimesCited:1,42218ClassicMeta-analysis:

SmithMl,GlassGV(1977):Meta-analysisOfPsychotherapyOutcomeStudies.AmericanPsychologist,32,752-760.TimesCited:840.

GeneGlasscoinedthephrasemeta-analysisinclassicstudyoftheeffectsofpsychotherapy.Becausemostindividualstudieshadsmallsamplesizes,theeffectstypicallywerenotstatisticallysignificant.Resultsof375controlledevaluationsofpsychotherapyandcounsellingwerecodedandintegratedstatistically.Thefindingsprovideconvincingevidenceoftheefficacyofpsychotherapy.Ontheaverage,thetypicaltherapyclientisbetteroffthan75%ofuntreatedindividuals.Fewimportantdifferencesineffectivenesscouldbeestablishedamongmanyquitedifferenttypesofpsychotherapy(e.g.,behavioralandnon-behavioral).19ESRCRDIOneDayMeta-analysisworkshop(Marsh,O’Mara,Malmberg)19Stepsinameta-analysisEstablishresearchquestionDefinerelevantstudiesDevelopcodematerialsLocateandcollatestudiesPilotcoding;codingDataentryandeffectsizecalculationMainanalysesSupplementaryanalyses20DefiningapopulationofstudiesandfindingpublicationsNeedtohaveexplicitinclusionandexclusioncriteriaThebroadertheresearchdomain,themoredetailedtheytendtobecomeRefinecriteriaasyouinteractwiththeliteratureComponentsofadetailedcriteriadistinguishingfeaturesresearchrespondentskeyvariablesresearchmethodsculturalandlinguisticrangetimeframepublicationtypes21LocateandcollatestudiesSearchelectronicdatabases(e.g.,ISI,PsychologicalAbstracts,ExpandedAcademicASAP,SocialSciencesIndex,PsycINFO,andERIC)ExaminethereferencelistsofincludedstudiestofindotherrelevantstudiesIfincludingunpublisheddata,emailresearchersinyourdiscipline,takeadvantageofListservs,andsearchDissertationAbstractsInternational22PilotcodingRandomselectionofpaperscodedbybothcodersMeettocomparecodesheetsWherethereisdiscrepancy,discusstoreachagreementAmendcodematerials/definitionsincodebookifnecessaryMayneedtodoseveralroundsofpiloting,eachtimeusingdifferentpapers23Developcodematerials__ StudyID__ Yearofpublication__ Publicationtype(1-5)__ Geographicalregion(1-7)____ Totalsamplesize___ Totalnumberofmales___ Totalnumberoffemales24Publicationtype(1-5)JournalarticleBook/bookchapterThesisordoctoraldissertationTechnicalreportConferencepaper19921874146CodeSheetCodeBook/manualESRCRDIOneDayMeta-analysisworkshop(Marsh,O’Mara,Malmberg)EffectsizecalculationTheeffectsizemakesmeta-analysispossibleItisthe“dependentvariable”ItstandardizesfindingsacrossstudiessuchthattheycanbedirectlycomparedAnystandardizedindexcanbean“effectsize”(e.g.,standardizedmeandifference,correlationcoefficient,odds-ratio),butmustbecomparableacrossstudies(generallyrequiresstandardization)representthemagnitudeanddirectionoftherelationshipofinterestbeindependentofsamplesize25EffectsizecalculationRepresentsastandardizedgroupcontrastonaninherentlycontinuousmeasureUsesthepooledstandarddeviation(somesituationsusecontrolgroupstandarddeviation)Commonlycalled