Mid-termevaluationoftheGlobalStrategytoEliminateYellowFeverEpidemics(EYE)

2017-2026

?WHO/Noor/BenedicteKurzen

WHO/DGO/EVL/2023.1

?WorldHealthOrganization2023

Allrightsreserved.

Thishealthinformationproductisintendedforarestrictedaudienceonly.Itmaynotbereviewed,abstracted,quoted,reproduced,transmitted,distributed,translatedoradapted,inpartorinwhole,inanyformorbyanymeans.

ThedesignationsemployedandthepresentationofthematerialinthishealthinformationproductdonotimplytheexpressionofanyopinionwhatsoeveronthepartoftheWorldHealthOrganizationconcerningthelegalstatusofanycountry,territory,cityorareaorofitsauthorities,orconcerningthedelimitationofitsfrontiersorboundaries.Dottedanddashedlinesonmapsrepresentapproximateborderlinesforwhichtheremaynotyetbefullagreement.

Thementionofspecificcompaniesorofcertainmanufacturers’productsdoesnotimplythattheyareendorsedorrecommendedbytheWorldHealthOrganizationinpreferencetoothersofasimilarnaturethatarenotmentioned.Errorsandomissionsexcepted,thenamesofproprietaryproductsaredistinguishedbyinitialcapitalletters.

TheWorldHealthOrganizationdoesnotwarrantthattheinformationcontainedinthishealthinformationproductiscompleteandcorrectandshallnotbeliableforanydamagesincurredasaresultofitsuse.

ThepurposeofpublishingevaluationreportsproducedbytheWHOEvaluationOfficeistofulfilacorporatecommitmenttotransparencythroughthepublicationofallcompletedevaluations.ThereportsaredesignedtostimulateafreeexchangeofideasamongthoseinterestedinthetopicandtoassurethosesupportingtheworkofWHOthatitrigorouslyexaminesitsstrategies,resultsandoveralleffectiveness.

ThisisanindependentevaluationpublishedbytheWHOEvaluationOffice,basedontheworkdonebytheindependentevaluationteam.ThispublicationdoesnotnecessarilyreflecttheviewsoftheWorldHealthOrganization.ThetexthasnotbeeneditedtoofficialpublicationstandardsandWHOacceptsnoresponsibilityforerror.Thedesignationsinthispublicationdonotimplyanyopiniononthelegalstatusofanycountryorterritory,orofitsauthorities,orthedelimitationoffrontiers.

i

Contents

1.Acknowledgements iv

2.Abbreviationsandacronyms v

3.Executivesummary vii

4.Introduction 1

4.1Background 1

4.2Rationaleandobjectives 3

4.3Scopeandevaluationquestions 3

4.4Methods 4

4.5Limitations 8

4.6Organizationofthereport 9

5.Evaluationfindings 10

EQ1:HowrelevantwastheEYEstrategyatthedesignphaseanddoesitcontinuetoberelevant?

(Relevance) 10

EQ2:TowhatextentwastheEYEstrategyimplementedefficientlyandcoherentlytomaximize

publichealthgains?(Efficiencyandcoherence) 20

EQ3:WhatresultshavebeenachievedbytheEYEpartnershipintheimplementationofthe

strategy?(Effectiveness) 39

EQ4:HastheEYEstrategydevelopedplans/identifiedaframeworktosecurefundingorto

otherwiseensuresustainabilityofachievementspost-2026?(Sustainability) 87

EQ5:TowhatextenthastheEYEstrategyincludedandaddressedgender,equityandhuman

rightsconcernstoensurethatactivitiesareconsistentlyandmeaningfullyinformedby

considerationsofoverallequity? 103

6.Conclusions 114

7.Recommendations 118

Tableoffigures

Figure1.YellowfeverriskclassificationAfricaandtheAmericas 2

Figure2.Evaluationapproach 6

Figure3:FamiliaritywiththeEYEstrategy–keystakeholdersatthecountrylevel 16

Figure4:AlignmentofEYEwithcountries'needsandpriorities 17

Figure5:EYEgovernancestructure 22

Figure6:WorkplanpriorityareasforEYEin2022-byEYEentity 26

Figure7:WorkplancompletionprogressonEYEworkinggroups,EYEsecretariatandAfricanregional

implementationteam,Nov2021 27

Figure8:PerceptionsoncountrysupportthroughassistancefromWHO,UNICEForGavi 30

Figure9:Numberoflargedisruptiveyellowfeveroutbreaks2017-2021 43

Figure10:Numberofconfirmedyellowfevercases2016-2021perregion 43

Figure11:YellowfevercasesreportedinAfrica1Jan2021-26Aug2022 45

Figure12:NumberofyellowfevercasesinAfrica1Jan2021-12Aug2022,confirmedandprobable

cases 46

Figure13:Numberofconfirmedyellowfevercases2017-2022,Americasmap 46

Figure14:Numberofpeoplereached/plannedtobereachedthroughyellowfevervaccination

campaigns(PMVC,reactivevaccinationcampaign,catch-upcampaigns),Africa,2017-2022 50

Figure15:Progresssince2017towardstargetofreaching478millionpeoplewithyellowfever

vaccineinAfrica 50

MTEoftheglobalstrategytoEliminateYellowfeverEpidemics2017-2026–pre-editedversion

ii

Figure16:2022yellowfevervaccinationcampaigntargetpopulationpercountryandtypeof

campaign 51

Figure17:PlannedPMVCs&routineimmunizationactivitiesinAfrica,2020-2024basedon2021

allocation 52

Figure18:PerceivedbarrierstoPMVCs 53

Figure19:EYEyellowfevervaccineprocurement(UNICEF)2017-2021(andforecastprocurement

2022-2024) 55

Figure20:Reportedstockoutsofyellowfevervaccinesandinterruptionofcampaignsduetostock

outs2016-2021,byregion 56

Figure21:Proportionofyellowfeverhigh-riskcountriesachievingatleast80%routinecoverageof

annualchildcohortbyregion(WUENIC) 58

Figure22:Yellowfevervaccinationcoveragewithinroutineimmunizationacrossyellowfeverhigh-

riskcountriesinAfrica(WUENIC)2016-2021 60

Figure23:Yellowfevervaccinationcoveragewithinroutineimmunizationacrossyellowfeverhigh-

riskcountriesintheAmericas(WUENIC)2016-2021 60

Figure24:Challengesforgeneralroutineimmunizationprogrammesatcountrylevel 61

Figure25:NumberandproportionofPMVCsreaching80%coverage2018-2021(Africancountries)63

Figure26:CoverageratesforPMVCsinvariousdistrictsofNigeria2018-2021 64

Figure27:Differencesinvaccinationcoveragerates,yellowfevervaccineandMCV1,2016-2021

(WUENIC) 65

Figure28:Vaccinationcoveragerates(throughroutineimmunization)foryellowfeverandMCV1in

2021foryellowfeverhigh-riskcountries 66

Figure29:PerceptionsonreasonsfordifferencebetweenyellowfeverandMVC1coverage 67

Figure30:WHOInternationalTravelandHealth(ITH)YellowfeverCountryList2021 68

Figure31:Proportionofyellowfevercasesinvestigatedwithintwoweeksofsymptomsonset-

overallaverage(Africa) 72

Figure32:Proportionofyellowfevercaseswithspecimentakenwithintwoweeksofsymptomonset

(10countriesinAfrica) 73

Figure33:Annualnumberofsuspectedyellowfevercasesreported2016-2020(21countriesin

Africa) 74

Figure34:Averagenumberofdaysfromspecimencollectiontoreceiptinnationallaboratory2017-

2021(Africa) 75

Figure35:Samplestransportedwithin14daysfromlocalleveltonationalreferencelaboratory

(Africa) 75

Figure36:ProportionofIgMtestresultsreportedbynationalreferencelaboratoriesinyellowfever

high-riskcountrieswithinsevendaysafterreceiptofbloodspecimen,2016-2021(Africa) 76

Figure37:ProportionofIgMtestresultsreportedbynationalreferencelaboratoriesinyellowfever

high-riskcountrieswithinsevendaysafterreceiptofbloodspecimenin2021(Africa) 76

Figure38:Averageshipmenttimelinesforyellowfeversamples 77

Figure39:Averagenumberofdaysbetweenonset(indexcase)andcampaignstart2017-2022(year

ofimplementation) 78

Figure40:Proportionofyellowfeveroutbreakswithcampaignsstartingwithin86days 79

Figure41:Yellowfeveroutbreaktimelines2021 79

Figure42:Towhatextentdoyouagreethatthefollowingarecontinuingchallengesforyellowfever

outbreakresponseincludingreactivevaccinationcampaignsinyourcountry? 80

Figure43:Fragile,conflict-affectedandvulnerablecountrieswithyellowfeveroutbreaks 85

Figure44:Perceptionsonrelianceondonorfinancialsupportbeyond2026forspecificyellowfever

activities 90

Figure45:Perceptionsonpriorityareasforyellowfeverinthefuture 92

Figure46:Yellowfeverriskperceptionamongcountry-levelstakeholdersinyellowfeverhigh-risk

countries 95

Figure47:Nationalplansforyellowfevercontrol 98

Figure48:Populationsmostvulnerableorathighestriskforyellowfever 109

MTEoftheglobalstrategytoEliminateYellowfeverEpidemics2017-2026–pre-editedversion

iii

Tableoftables

Table1:Evaluationquestionsandsub-questions 4

Table2:Overviewofkeyinformantsinterviewedforthemid-termevaluation 7

Table3:EstimatedprojectedcostoftheEYEstrategytothemid-termandtheshareofprioritiesfrom

thetotalcost 15

Table4:NumberofprogrammemanagementgroupmeetingsforEYE,2018-2021 25

Table5:Targetgroupsand“EYEonyellowfever”podcasttopics 32

Table6:EYEstrategyfinancialinvestmentsbyglobalpartners2016-2023 36

Table7:Numberoflargedisruptiveyellowfeveroutbreaks2017-2021-global 43

Table8:StrategicIndicatorsforEYE,mid-termprogressagainst2026targets 47

Table9:Towhatextentdoyouagreethatthefollowingarecontinuingchallengestoyellowfever

PMVCsinyourcountry? 54

Table10:Vaccinesupplychallengesasperceivedbycountrylevelstakeholders 55

Table11:Yellowfeverhigh-riskcountries(withyellowfeveraspartofroutineimmunization)

achievingatleast80%yellowfevervaccinationcoverageoftheannualchildcohort(WUENICdata)

2014-2021 58

Table12:Routineimmunizationchallengesrelatedtocoldchain/logistics,supplyofother

commoditiesandvaccinewastageasperceivedbycountrylevelstakeholders,regional

disaggregation 62

Table13:Statusofurbanreadinessplansforyellowfeveratthecountrylevel 69

Table14:MilestonesfortheEYEstrategy,statusoverview 81

Table15:Fundingofyellowfeverinterventions,andfundingchallengesasperceivedbyEYEcountry-

levelstakeholders 89

Table16:HumanresourcescapacitychallengesasperceivedbyEYEcountry-levelstakeholders 89

Table17:Perceptionsondonorfinancialsupportneededbeyond2026 90

Table18:Perceptionsonpriorityareasforsustaining/achievingeliminationofyellowfeveratcountry

level 92

Table19:Gavieligibilityandtransitionphasesforyellowfeverhigh-riskcountries 93

Table20:Multi-antigenvaccinationcampaignsincludingyellowfevervaccine 100

Table21:theEYEstrategyM&Eframeworkattentiontovulnerable/marginalized/at-riskpopulations

105

Table22:Disaggregatedquantitativedatafromtheonlinesurveyrelevanttovulnerablepopulations

110

MTEoftheglobalstrategytoEliminateYellowfeverEpidemics2017-2026–pre-editedversion

iv

1.Acknowledgements

ThisreportisissuedbytheWHOEvaluationOffice.ItisbasedontheindependentevaluationconductedbytheEvaluationTeamfromEuroHealthGroupcomprisingMsJenniferLissfelt(TeamLeader),MsMicheleGross(DeputyTeamLeader),MsMaikenMansfeldJacobsen,MrAbebeAlebachewAsfaw,DrJanetGruberandMsFinjaDaegling.TheWHOEvaluationOfficewouldliketoexpressitsappreciationtotheTeamforitsdedicationandtheexcellentworkdone.

ThisevaluationwasmanagedandqualitycontrolledbyMsMarieBombin,SeniorEvaluationOfficer,withresearchsupportfromMsYeLi,WHOEvaluationOffice.

TheWHOEvaluationOfficewouldliketothankallparticipatingrepresentativesfromMemberStatesfortheirgeneroustime,insights,andcandidfeedbackandcontributions.TheOfficefurtherappreciatesthededicatedsupportfortheevaluationfromtheEYEsecretariatandtheWHOHealthEmergenciesInterventionsteam,andforprovidingtheevaluationteamwithrequestedbackgrounddocumentsanddata.Inaddition,theWHOEvaluationOfficewouldliketothankallmembersoftheEYEleadershipgroupandtheEYEprogrammemanagementgroupforfruitfuldiscussionsandinput,andallcontributingstafffromheadquarters,regionalandcountryoffice,PAHO,UNICEF,GaviandCDC,aswellasstaffrepresentingacademia,yellowfeverlaboratories,vaccinemanufacturers,andcivilsocietyorganizationsforsharingtheirvaluableinsightsandhonestperspectivestoinformtheevaluation.

AspecialthankstoorganizersandfacilitatorsofthetwocountrycasestudiesforthisevaluationinBrazilandGhana:thePAHO/WHOCountryOfficeinBrazilandtheWHOCountryOfficeinGhana,fortheirinvaluableadvice,supportandinput.

MTEoftheglobalstrategytoEliminateYellowfeverEpidemics2017-2026–pre-editedversion

v

2.Abbreviationsandacronyms

AFR

WHOAfricanRegion

AFRO

WHORegionalOfficeforAfrica

BMGF

BillandMelindaGatesFoundation

CAR

CentralAfricanRepublic

CDC

CentersforDiseaseControlandPrevention

COVID-19

Coronavirusdisease2019

CSO

Civilsocietyorganization

DRC

TheDemocraticRepublicoftheCongo

EHG

EuroHealthGroup

ELISA

Enzyme-linkedimmunosorbentassay

EMRO

WHORegionalOfficefortheEasternMediterranean

EPI

EssentialProgrammeonImmunization

EQ

Evaluationquestion

EYE

TheglobalstrategytoEliminateYellowfeverEpidemics2017–2026

EYE.Ops

EYEoperationsteam

FCV

Fragile,conflict-affected,andvulnerable

FGD

Focusgroupdiscussion

Gavi

Gavi,theVaccineAlliance

GE+HR

Gender,equityandhumanrights

GESI

Gender,equityandsocialinclusion

GHS

GhanaHealthServices

GLAI

GlobalArbovirusInitiative

GNI

Grossnationalincome

GPEI

GlobalPolioEradicationInitiative

HQ

Headquarters

HR

Humanresources

HPV

humanpapillomavirus

IA2030

ImmunizationAgenda2030

ICG

Internationalcoordinatinggroup(onvaccineprovision)

IFRC

InternationalFederationoftheRedCross

IgM

ImmunoglobulinM

IHR

Internationalhealthregulations(2005)

IVB

Immunization,Vaccines,andBiologicalsDepartment(WHO)

KI

Keyinformant

KII

Keyinformantinterview

LAC

LatinAmericaandCaribbeanRegion

LG

Leadershipgroup(fortheEYEstrategy)

LTA

Long-termagreement

M&E

Monitoringandevaluation

MCV

Measlescontainingvaccine

vi

MIC

Middle-incomecountries

MoH

MinistryofHealth

MSF

MédecinsSansFrontières

MTE

Mid-termevaluation

NGO

Nongovernmentalorganization

NHP

Non-humanprimates

PAHO

PanAmericanHealthOrganization/WHORegionalOfficefortheAmericas

PCR

Polymerasechainreaction

PMG

Programmemanagementgroup(oftheEYEstrategy)

PMVC

Preventivemassvaccinationcampaigns

PPE

Personalprotectiveequipment

PRNT

Plaquereductionneutralizationtest

RAWG

Riskanalysisworkinggroup

RfP

Requestforproposal

RI

Routineimmunization

RRL

Regionalreferencelaboratory

RT-PCR

Reversetranscriptionpolymerasechainreaction

RVC

Reactivevaccinationcampaigns

SAGE

StrategicAdvisoryGroupofExpertsonImmunization

SDGs

SustainableDevelopmentGoals

SOP

Standardoperatingprocedure

SWOT

Strengths,weaknesses,opportunitiesandthreats

TAG

Technicaladvisorygroup

ToC

Theoryofchange

ToR

Termsofreference

UHC

UniversalHealthCoverage

UN

UnitedNations

UNAIDS

JointUnitedNationsProgrammeonHIV/AIDS

UNFPA

UnitedNationsPopulationFund

UNICEF

UnitedNationsChildren’sFund

VPD

Vaccine-preventabledisease

YF

Yellowfever

VDWG

vaccinedeliveryworkinggroup

WG

Workinggroup

WHE

HealthEmergencyProgramme(WHO)

WHO

WorldHealthOrganization

WS

workstream

WUENIC

WHO/UNICEFestimatesofnationalimmunizationcoverage

ZD

Zero-dose

ZIP

Zero-doseImmunizationProgramme

MTEoftheglobalstrategytoEliminateYellowfeverEpidemics2017-2026–pre-editedversion

vii

3.Executivesummary

Background

In2016,awidespreadyellowfeveroutbreakinAngola,alsoaffectingthecapitalcity,causedunprecedentedspread,affectingneighbouringcountrieswithanurbanoutbreakinKinshasa(intheDemocraticRepublicoftheCongo)andviraemictravellerstoAsia.Thefollowingyear,yellowfeverspreadtocoastalareasinBrazilincludinglargeurbancentresthathadnotseenyellowfeveroutbreaksinseveraldecades.Inresponsetotheseoutbreaksandthethreatofinternationalspread,theWHO,GaviandUNICEFdevelopedacomprehensivemulti-partnerglobalstrategytoEliminateYellowfeverEpidemics(EYE)2017-2026.TheEYEstrategyhasthreeoverallstrategicobjectives,to:

?protectatriskpopulations;

?preventinternationalspread;and

?containoutbreaksrapidly.

FortycountriesconsideredhighriskforyellowfeveroutbreaksaretargetedundertheEYEstrategy.Thisincludes27countriesinAfricaand13countriesintheAmericas.

Purpose,objectivesandmethodology

Themid-termevaluationwasincludedasamilestoneintheEYEstrategy,andintheWHOevaluationworkplan2022–2023whichwasapprovedbytheWHOExecutiveBoardatits150thsessioninJanuary2022.UndertakenincollaborationwithGAVIandUNICEFbytheWHOEvaluationOfficeworkingwiththeRegionalOfficesforAfricaandtheAmericas,theevaluationwascommissionedtoacompetitivelyselectedindependentcompany,EuroHealthGroup,inMay2022.Thepurposeofthemid-termevaluationwastoassesstherelevance,coherence,effectiveness,efficiencyandsustainabilityofthestrategyimplementationtodateandtoreviewinclusionofgender,equityandhumanrightsconsiderations.Thisincludedprogrammedeliveryaspectsaswellasstrategymanagementandgovernanceaspects.

Themainobjectivesoftheevaluationwereto:

?documentkeyachievements,bestpractices,challenges,gaps,andareasforimprovementinthedesignandimplementationofthestrategy;

?identifythekeycontextualfactorsandchangesthatareaffectingyellowfeverspreadandtransmissionriskprofile,andinfluencingprogrammeimplementation;and

?makerecommendationsasappropriateonthewayforwardtoimproveperformanceandimplementation,andtoensuresustainabilityinthefuturebeyond2026.

Theoverallapproachtotheevaluationwastheory-basedandincludeddevelopingatheoryofchangefortheEYEstrategy.Thetheory-basedevaluationwascombinedwithaprocessevaluation,tolookindetailattheimplementationofthestrategytodate.

Thisreportpresentsfindingsforfivehighlevelevaluationquestions(and15sub-evaluationquestions)andrelatedconclusionsandrecommendationsbasedonareviewofcomprehensivedatasetsandmorethan250documents,61keyinformantinterviewscarriedoutataglobal,regionalandcountrylevel;andthroughasurveyof118countrylevelkeystakeholdersacross40yellowfeverhighriskcountries.Inaddition,severalfocusgroupdiscussionswereundertakenaspartoftheevaluation(atheoryofchangeworkshop,a“strengths,weaknesses,opportunitiesandthreats”analysis,andsmallergroupdiscussionswithkeystakeholders).TofurtherdocumentbestpracticesandlessonslearnedforstrategicactionsundertheEYEstrategy,twocountrycasestudieswereconductedwithmissionstoBrazilandGhana.

MTEoftheglobalstrategytoEliminateYellowfeverEpidemics2017-2026–pre-editedversion

viii

Analyticalapproachescomprised:triangulationofdata(bothacrossandwithincategoriesofdatasources);thematicanalysis(thematiccodingandanalysisofsecondarydocuments,keyinformantinterviewsandfocusgroupdiscussionnotes);statisticalanalysisofkeyEYEM&Eindicatorsandresultsoftheonlinesurveyandcontributionanalysis.

Theevaluationmethodologywasbroadlyimplementedasproposedintheevaluationinceptionreport,withnosignificantdeparturesfromthetermsofreference.Thetimingofthedatacollectionperiod-overthemainholidayseason(June-August2022)-causedchallengesrelatedtoavailabilityofkeyinformantsandonlinesurveyrespondents.AnotherlimitationwasrelatedtotheEYEM&Eframework,dataqualityconcernsanddatagapswithfewmid-termtargets,missingbaselinevalues,andunavailabilityofdatafromtheAmericasattheEYEsecretariatlevelonseveralEYEstrategicindicators.

Baselinedatawerereconstructedbytheevaluationteamusing2017datafromvalidatedEYEdatasourceswhereavailable,andtheevaluationreliedonprojectionsupto2026toestablishmid-termtargets.Examplesofunavailabledataand/ordataqualityconcernshavebeenhighlightedthroughoutthereportwhereapplicable.Fortheresults/effectivenessevaluationquestion,themid-termevaluationmainlyconsideredprogressonthe16indicatorsprioritizedbytheEYEpartnershipasstrategicindicatorsforthestrategy.Theevaluationdidnotconsiderperformanceacrossyellowfevermedium-orlow-riskcountriesbecausetheEYEstrategyimplementationspecificallytargetstheyellowfeverhigh-riskcountries.

Keyfindings

Thissectiondescribeskeyevaluationfindingsstructuredaccordingtothefivehigh-levelevaluationquestions.

Evaluationquestion1:HowrelevantwastheEYEstrategyatdesignphaseanddoesitcontinuetoberelevant?(Relevance)

Design

EYEwasdesignedatatimeofgreaturgencywithlargescaleyellowfeveroutbreaksinAngolaandtheDemocraticRepublicoftheCongo.TheevaluationfoundtheEYEstructure,theEYEpartnershipandstrategicobjectivestobeoverallappropriatetotheyellowfevercontextwhenEYEwasdesignedin2017.Thestrategicfocusonachievinghighimmunityforyellowfeverthroughpreventivemassvaccinationcampaignsandroutineimmunization,promotionofvaccineavailability/laboratory/diagnosticsandsurveillanceandpreventionofinternationalspreadwereallrelevanttotheoverallneedsofcountriestoeliminateyellowfeverepidemics.Thereis,however,scopetoconsiderincludingapproachestoembedandconsolidateownershipatcountrylevel,andtoidentifyactionstoenhancegender,equityandsocialinclusion/humanrightsaspects,includingtargetedapproachesforreachingvulnerableandhigh-riskpopulations.

TheEYEM&Eframeworkwasextensivebyfirstdesignandwasmodifiedoverthefirstyearstofocusonfewerindicators.Yet,theevaluationnotedlimitationsinrelationtoavailabilityofdata,particularlyonstrategicobjective2,forwhichkeyactivitiesareyettobegin.TheevaluationalsonotedlimitedmonitoringbytheEYEsecretariatofdatafromtheAmericasonthe16strategicEYEindicatorsaswellaslimitedmonitoringofdisaggregateddata.Overall,severalM&Eframeworkbaselinevaluesweremissing,andsometargetsseemtooaspirational.MilestonesonM&Eframeworkindicatorsand/ormid-termtargetsh