November2024

ForecastingGlobalOilDemand:

ApplicationofMachineLearningTechniques

AhmadM.Aldabbagh,VisitingResearchFellowOIES

OIESPaper

AndreasEconomou,HeadofOilResearchOIESCharitonChristou,ResearchAssociateOIES

2

Abstract

Thisstudyintroducesanovelapproachtopredictingglobaloildemandbyintegratingmachinelearning(ML)techniquestoforecastconsumptionacrosssevenrefinedoilproductsandsevenkeyregions.Byaggregatingtheseforecasts,weofferacomprehensiveviewofglobaldemandtrends.ThepaperexaminestheefficacyofMLmodelsinprovidingrobustandaccuratedemandforecasts.Italsoprovidesatransparentandrepeatableprocesstoforecastoildemand.Acomparisonbetweentheextremegradientboosting(XGBoost)modelandNeuralHierarchicalInterpolationforTimeSeriesForecasting(N-HiTS)modelwasconductedtodeterminewhichisamoreaccuratemodeltoforecastdemand.OurcomparativeanalysisdemonstratesthatN-HiTSperformsbetter.Theaccuracyofglobaloildemandforecastsispivotalforeconomicplanningandpolicymaking.

Thecontentsofthispaperaretheauthors’soleresponsibility.Theydonotnecessarilyrepresenttheviews

oftheOxfordInstituteforEnergyStudiesoranyofitsMembers.

1.Introduction

Accurateforecastingofoildemandiscriticalforstrategicplanning.Traditionaleconometricmodels,whileuseful,oftenstruggletocapturethecomplexdynamicsinfluencedbynumerouseconomicindicatorsandgeopoliticalfactors.Thesemodelstypicallyrelyonlinearassumptionsandmaynotadequatelyaddressthenon-linearrelationshipsinherentinoilmarkets.Toaddressthesechallenges,ourpaperintroducesamethodologythatleveragesadvancedmachinelearning(ML)techniques,specificallyextremegradientboosting(XGBoost)andNeuralHierarchicalInterpolationforTimeSeriesForecasting(N-HiTS),toenhancetheprecisionandreliabilityofoildemandforecasts.

Machinelearningmodelshavedemonstratedsuperiorperformanceinvariousforecastingtasksduetotheirabilitytohandlelargedatasetsanduncoverintricatepatterns.RecentstudiesconsistentlyshowthatMLtechniquesoutperformtraditionaleconometricmethodsintimeseriesforecastingbymodellingcomplex,nonlinearrelationshipsandhandlinglargedatasets.Forinstance,Hopp(2022)1foundthatlongshort-termmemory(LSTM)neuralnetworksprovidedbetterpredictiveaccuracythanBayesianvectorautoregressions(BVAR)fornowcastingUSquarterlyGDPgrowth,especiallyduringeconomiccrises.Deb(2019)2highlightedthatmodelsexploitingheterogeneity,suchasfinitemixturemodels,yieldedmoreaccuratehealthcarespendingforecastscomparedtogeneralizedlinearmodelsandlog-linearregression.Similarly,Lukongetal.(2022)3showedthatlongshort-termmemoryrecurrentneuralnetworks(LSTM-RNN)modelsachievedsignificantlylowermeanabsolutepercentageerror(MAPE)inlong-termelectricityloadforecastingthanlinearregressionmodels.Infinancialtimeseriesforecasting,Liuetal.(2023)4reportedthatensemblemethodslikeRandomForestandLSTMoutperformedtraditionaleconometricmodelsinbothaccuracyandinterpretability.Additionally,Kontopoulouetal.(2023)5reviewedvariousapplicationsandconcludedthatMLalgorithmsgenerallysurpassedautoregressiveintegratedmovingaverage(ARIMA)models,particularlyincapturingintricatedatapatterns,withhybridmodelsprovingmosteffective.Furthermore,comparativeanalysesbyOukhouyaandElHimdi(2023)6alsoillustratethesuperiorperformanceofsupportvectorregression(SVR),XGBoost,LSTM,andmultilayerperceptron(MLP)instockmarketforecasting,withMLmodelsgenerallyoutperformingtheireconometriccounterparts.ThissuperiorityisattributedtoMLmodels'abilitytolearnfromdatawithoutrelyingonpre-definedassumptions,allowingthemtocapturemorenuancedandcomplexrelationships.Thesefindingsunderscoretheenhancedaccuracy,efficiency,andflexibilityofMLmodelsintimeseriesforecastingacrossdiversedomains.

Inthecontextofoildemandforecasting,theintegrationofMLtechniqueshasshownsignificantimprovementsovertraditionalmethods.StudiesbyZhu(2023)7andAlkhammashetal.(2022)8havevalidatedtheeffectivenessofMLmodelsinthisdomain,demonstratingtheirsuperiorperformanceincapturingcomplexpatternsindata.Zhu(2023)conductedanAI-basedanalysisincorporatingbothendogenousandexogenousfactors,findingthatmachinelearningmodelssignificantlyimproveforecastingaccuracycomparedtotraditionalmodels.Similarly,Alkhammashetal.(2022)usedoptimizedmultivariateadaptiveregressionsplines(LR-MARS)topredictcrudeoildemandinSaudiArabia,showcasingtheadaptabilityandprecisionofMLmodelsindynamicenvironments.

XGBoost9,agradientboostingalgorithm,isknownforitsrobustnessandefficiency.Ithasbeensuccessfullyappliedindomainssuchastemperatureforecastingandprecipitationprediction,highlightingitsversatilityandeffectivenessacrossdifferentpredictivetasks.Forinstance,SinghandRawat(2023)10conductedacomparativeanalysisofXGBoostwithotherMLmodelslikesupportvectormachine(SVM)andRandomForestintemperatureforecasting,emphasizingtheimportanceofmodelselectionbasedonspecifictaskrequirements.Similarly,Dongetal.(2023)11utilizedXGBoostforshort-termprecipitationforecasting,demonstratingitscapabilitytoenhanceforecastaccuracybycorrectingbiasesinnumericalweatherpredictions.Inthecontextofoildemandforecasting,XGBoost'scapabilitytocapturecomplexinteractionsbetweenvariablesmakesitapromisingtool.ResearchbyDezhkamandManzuri(2023)12demonstratedtheefficacyofcombiningXGBoostwiththeHilbert-HuangTransformforstockmarketforecasting,adomainwithanalogouscomplexitytooilmarkets.

Thecontentsofthispaperaretheauthors’soleresponsibility.Theydonotnecessarilyrepresenttheviews

oftheOxfordInstituteforEnergyStudiesoranyofitsMembers.

3

Thecontentsofthispaperaretheauthors’soleresponsibility.Theydonotnecessarilyrepresenttheviews

4

oftheOxfordInstituteforEnergyStudiesoranyofitsMembers.

Similarly,theN-HiTSmodel13,aneuralnetwork-basedapproach,hasshownremarkableeffectivenessintimeseriesforecasting.Thismodelleverageshierarchicalinterpolationandhasbeenrecognizedforitsabilitytohandlenon-lineartimeseriesdataefficiently.StudieslikethosebySouzaetal.(2023)14haveemployedN-HiTStopredictCOVID-19casesanddeaths,showcasingitspotentialinhandlingnon-lineartimeseriesdata.TheN-HiTSmodel'sarchitectureallowsittoadapttodifferentscalesofdata,makingitsuitableforvariousforecastinghorizons.Althoughrelativelynew,N-HiTSpromiseshighaccuracy,particularlyforlong-termforecasts,whicharecrucialforstrategicenergyplanning.Itsabilitytodynamicallyadjustandlearnfromnewdatapointsensuresthattheforecastsremainrelevantandaccurateovertime.

Thispaperemploysatop-downapproach,usingeconomicandotherindustry-specificindicatorsasinputstoforecastglobaldemandacrossdifferentrefinedoilproductsinsevenglobalregions.Theseregionsarestrategicallychosentorepresentthemajoroil-consumingandproducingareas,ensuringacomprehensiveanalysis.Theforecastsarethenaggregatedtoformacomprehensiveglobalviewonoildemand.TheperformanceoftheMLmodels,specificallyXGBoostandN-HiTS,isassessedusingmetricssuchasMeanAbsolutePercentageError(MAPE)andMeanSquaredError(MSE),targetingahighaccuracywithaMAPEaround10%forforecastsextendingoneyearout-of-sample.Thesemetricsarecriticalforevaluatingthemodels'predictiveperformanceandensuringthattheforecastsarebothreliableandactionable.

Ourmethodologydemonstratesitsapplicationinthreeways:asashort-termforecastingtool(usingmonthlyinputsandforecastingupto24monthsahead),asamedium-termforecastingtool(usingquarterlyinputsandforecastingupto5yearsahead),andasaforecastscenariosanalysistool.Theseapplicationshighlighttheversatilityofourapproachanditspotentialtoinformstrategicdecisionsacrossdifferentplanninghorizons.

ByleveragingthesuccessesofMLinvariousdomains,thisstudyseekstosetanewbenchmarkinoildemandforecasting.Theultimategoalistoenhancethestrategicplanningcapabilitiesoftheenergyindustry,offeringmoreaccurateandtimelyforecaststoinformstrategicdecisions.TheresearchhighlightstheadvantagesofMLinidentifyingcomplexpatternsandadaptingtonewdata,makingitparticularlysuitableforthevolatileenergysector.

Theremainderofthepaperisorganizedasfollows.InSection2wereviewthedataprocessingstageprovidingthedatadescriptionandelaboratingonthemethodologiesemployedfordatacleaning,datatransformationanddatasplitting.Section3presentsthemethodologyandframeworkintermsoftheapproachusedformodelselectionandtraining.Section4demonstratestheapplicationofourmethodologyinconstructingshort-termandmedium-termforecastsofglobaloildemandandexaminescomparativelyinformationagainstactualsandconsensusforecasts.InSection5,wedemonstratehowtheseforecastscanbesupplementedwithvariousscenarios.Theseforecastscenarios,forexample,canillustratethesensitivityofthebaselineforecaststochangesinassumptionsabouttheglobalmacroeconomy,theadoptionofnewtechnologies,andevolvingpatternsofoilconsumption.Section6drawstheconclusionsanddiscussesareasforfutureresearch.

Thecontentsofthispaperaretheauthors’soleresponsibility.Theydonotnecessarilyrepresenttheviews

5

oftheOxfordInstituteforEnergyStudiesoranyofitsMembers.

2.Dataprocessing

Thedataprocessingstageisinstrumentalintransformingrawdataintoaformatthatisamenabletoeffectivemodelling.Thisprocessencompassesseveralcriticalsteps:datacleaning,datatransformation,anddatasplitting.Eachstepismeticulouslydesignedtoaddressspecificchallengesinherentintimeseriesforecasting,suchashandlingmissingvalues,ensuringdatastationarity,andoptimizingthedatasetfortrainingandvalidationpurposes.Moreover,thesameapproachisfollowedwhetheritisappliedforshort-termormid-termforecastwithsomeminordifferencesthatarehighlighted.Thissectionelaboratesonthemethodologiesemployedinpreparingthedataset,emphasizingthetechnicalstrategiesandtheirtheoreticalunderpinningstoensurethehighestdataqualityforsubsequentmodelling.

Table1.Globaloildemandforecastingmodeldatastructure

Geography

Regions/Countries

OECD

US

OtherAmericas

Europe

APAC

Non-OECD

China

India

Othernon-OECD

Description

UnitedStates

IncludesotherOECDAmericas,namely,Canada,ChileandMexico.

IncludesOECDEurope,namely,Austria,CzechRepublic,Denmark,Estonia,France,Germany,Greece,Hungary,Italy,theNetherlands,Norway,Poland,Portugal,Slovakia,Slovenia,Spain,Sweden,TurkeyandtheUK.

IncludesOECDAsia-Oceania,namely,Australia,Israel,Japan,KoreaandNewZealand.

China.

India.

Includesrestofworld.

Oilproducts

Oilproduct

Description

LPG(d1)

Includesallliquefiedpetroleumgases.

Naphtha(d2)

Includesnaphthaasfeedstocktothepetrochemicalindustryandforgasolineproduction.Excludesnaphthatypejetfuel.

Gasoline(d3)

Includesfinishedmotorgasolineandmotorgasolineblendingcomponentsandadditives.

Jet/Kero(d4)

Includeskerosene-typejetfuelandotherkerosene.

Gasoil/Diesel(d5)

Includesdieseloil,lightheatingoilandothergasoils.

Fueloil(d6)

Includesallresidualfueloils.

Otherproducts(d7)

Includesotheroilproducts,suchascrudeoil,otherNGL,syntheticcrude/fuels,orimulsion,hydrogen,refinerygas,aviationgasoline,naphtha-typejetfuel,whitespirit,SBP,lubricants,bitumen,paraffinwaxes,petroleumcoke,tar,sulphur,aromaticsandolefin.

Source:IEAMODS,OIES

6

Thecontentsofthispaperaretheauthors’soleresponsibility.Theydonotnecessarilyrepresenttheviews

oftheOxfordInstituteforEnergyStudiesoranyofitsMembers.

2.1.Datadescription

OurglobaloildemandforecastingmodelsdisaggregateglobaloildemandintoOECDandnon-OECDregions,aswellassevenprimaryoilproducts,namely:LPG(d1),naphtha(d2),gasoline(d3),jet/kero(d4),gasoil/diesel(d5),fueloil(d6),andotherproducts(d7).DemandforoilproductsinOECDisfurtherdisaggregatedintotheUS,OtherAmericas,EuropeandAsia-Oceania(APAC).Demandforoilproductsinthenon-OECDisfurtherdisaggregatedintoChina,Indiaandothernon-OECD.OECDdataarebasedontheInternationalEnergyAgency’sMonthlyOilDataService(IEAMODS)database.Fornon-OECDdataweutilizevarioussourcessuchasIEA,Argus,China’sNationalBureauofStatistics(NBS),India’sPetroleumPlanningandAnalysisCell(PPAC),andotherindustrysources.ThesampleperiodspamsfromJanuary1990toJune2023forOECD,andfromJanuary2000toJune2023fornon-OECDcountries/regions.Alloildataaremonthly.Table1summarizesthedatastructureforthedependentvariables.

Theforecastingmodelsalsoutilizefourgroupsofindependentdeterminantscoveringglobalprices,globaleconomics,globalindustryandsector-specificindicators.ThesearesummarizedinTable2.ThemainsourceofdataisOxfordEconomics’GlobalDatabanks.Thesignificanceofthesepredictorsinforecastingglobaloildemandisanalyzedandevaluatedinthefollowingsections.

Table2.Descriptionofselectedglobaloildemandpredictors

Globalprices

rpo

Oilprice

Brentprice,periodaverage.

wci

Worldcommodityindex(non-fuel)

Averageofworldfoodprice,worldbeveragesprice,worldagriculturerawmaterialspriceandworldmetalsprice.

Globaleconomics

pop

Population

Totalpopulation.

gdp

GDP

GDP,inUSD$terms.

cpi

CPI

ConsumerPriceIndex:Allitems.

inc

Disposableincome

Personaldisposableincome.

wtr

Worldtradeindex

OxfordEconomics’globaltradeindex.

lkq

LiKeqiangindex

IndexmeasuringChina’seconomybasedonlyonrailfreightvolume,electricityproduction,andbankloans.

Globalindustry

ind

Industrialproduction

Manufacturingproductionindex.

ppi

PPI

Producerpriceindex.

inv

Totalinvestment

Totalfixedinvestment.

out

Grossoutput

Totalvalueadded.

pcars

Stockpersonalcars

Lightvehiclesforpersonaluse.

ccars

Stockcommercialcars

Lightandheavyvehiclesforcommercialuse.

cbui

Constructionofbuildings

Industrialproductioninconstructionofbuildings.

chem

Salesgrossoutput(chem)

OutputchemicalsminusPharmaceuticals.

egen

Electricitygenbyoil

Electricityproductionfromoilsources(%oftotal).

Airpassengerforecasts

airf

Airfaresindex

Averageairfaresindex.Samplestartsin1Q06.

airp

Airpassengers

Totalpassengers.Samplestartsin1Q06.

rpk

Revenuepassengerkm

Revenuepassengerkilometersinmillions.Samplestartsin1Q06.

Source:OxfordEconomics’GlobalDatabanks,OIES

Thecontentsofthispaperaretheauthors’soleresponsibility.Theydonotnecessarilyrepresenttheviews

7

oftheOxfordInstituteforEnergyStudiesoranyofitsMembers.

2.2.Datacleaning

Ourshort-termdemanddataismonthlywhileeconomicindicatorsarequarterly.However,forthemid-termboththedemandandeconomicindicatordataisquarterlyanddonotrequireanyfrequencyconversion.Fortheshort-termforecast,anyquarterlydatamustbeconvertedtomonthlydatatoallowforamonthlyforecastinsteadofaquarterlyforecastfortheshort-term.Thefollowingsectionshighlighttherequiredstepspriortomodeltraining.

ConvertingQuarterlytoMonthlyData:Giventhediscrepancyinthetemporalresolutionofdemanddata(monthly)versusotherindicators(quarterly),itwasimperativetohomogenizethedatasettoamonthlyscaletomaximizeobservationaldatapointsforanalysis.Toachievethis,thePandas15library'slinearinterpolationmethodwasemployedduetoitssimplicityandeffectivenessinestimatingmissingvalues.Thisapproach,particularlysuitablefortimeseriesdata,linearlyinterpolatesmissingorNaNvaluesbasedonthelinearlyspacedvaluesbetweenknowndatapoints.Byutilizingthe.interpolate(method='linear')functiononourdataset,weensureasmoothtransitionbetweenquarterlydatapoints,therebymaintainingthetrendandvariabilityobservedintheoriginaldata.Thismethodassumesthatthechangebetweentwodatapointsislinear,fillinginmissingvalueswithappropriatelyspacedestimatesthatreflecttheunderlyingdatapattern.

Theinterpolationformulausedisgivenby:

,

where:

vinterpolatedisthevaluetobeinterpolated(estimated)forthetargettimepoint.

vstartandvendaretheknownvaluesbeforeandafterthepointtobeinterpolated,respectively.TstartandTendarethetimesatwhichvstartandvendareobserved,respectively.

Tinterpolatedisthetimepointatwhichthevalueistobeinterpolated.

Figure1:Exampleofbackcastingmissingvalues

Source:Authors’analysis

HandlingMissingValuesusingProphet:Toaddressinstancesofmissingdatainourdataset,weintegratedtheProphet16packageforbackcastingpurposes.Prophet,renownedforitsefficacyindiscerningunderlyingtrendsandseasonalfluctuationswithintimeseriesdata,facilitatedtheextrapolationofdatapointsbackwards.Thismethodallowedustoleverageestablishedpatternstoinferandpopulatemissingvalues,therebyensuringthedataset'scompletenessforsubsequentanalysis.TheProphetpackage,anopen-sourcetooldevisedbyFacebook,excelsinautomatic

8

Thecontentsofthispaperaretheauthors’soleresponsibility.Theydonotnecessarilyrepresenttheviews

oftheOxfordInstituteforEnergyStudiesoranyofitsMembers.

forecastingofunivariatetimeseriesdata,simplifyingtheprocessofselectingoptimalhyperparameterstoenhanceforecastaccuracy.Thereasonwhywedecidedtonotdroptherowswithmissingvaluesisthelimiteddatasetwehad.Furthermore,ourresultsshowaboostinaccuracywhenkeepingtherowswithmissingvaluesandimputingasopposedtodroppingtherowswithmissingvalues.Figure1showsanexampleofProphetappliedtobacktwovariableswithmissingvalues.Itisimportanttonotethatthisstepappliesexactlytoboththeshort-termandmid-termwheretheonlydifferenceisthefrequencyofthetime-seriesbeingprocessed.

2.3.Datatransformation

AchievingStationarity:Differencinghasbeenappliedtostabilizethemeanandvarianceofthetimeseriestoensurestationarity.Whiledifferencingisacommonapproachtoachievestationarity,itsminimalimpactonourresultssuggeststhattheMLmodelsselectedwererobusttonon-stationarydata,acharacteristicadvantageousforourforecastingobjectivesasshowninFigure2.

Figure2:EffectofdifferencingusingXGBoost

Notes:USd1.Source:Authors’analysis

Scaling:PreliminaryanalysisindicatedthatscalingthedatatostandardizefeaturerangesdidnotsignificantlyaffecttheperformanceofourMLmodels.Ourtestsinvolvedusingamin-maxscalerinXGBoostonourtrainingsetandcomparingtheeffectwithandwithoutscaling.ForN-HiTS,weusearobustscaler17whichisamethodusedtostandardizefeaturesbyremovingthemedianandscalingwiththemeanabsolutedeviation(MAD)orinterquartilerange(IQR).Thistechniqueisparticularlyusefulwithnoisydatawhereoutlierscanheavilyinfluencethesamplemeanandvarianceinanegativeway.

9

Thecontentsofthispaperaretheauthors’soleresponsibility.Theydonotnecessarilyrepresenttheviews

oftheOxfordInstituteforEnergyStudiesoranyofitsMembers.

2.4.Datasplitting

Thefollowingmethodologyappliestoboththeinputdatafortheshort-termandmedium-termmodelswithanyspecificmodificationexceptforthefrequencyparameter(monthlyversusquarterly)thatneedstobespecifiedinsomeofthesoftwarepackagesthatweused.

Training,Validation,andTestSets:Thedatasetwasdividedintotraining,validation,andtestingsetsfollowingtimeseriesforecastingbestpractices.Thissplitensuresthatthemodelsaretrainedonhistoricaldata,validatedtotunehyperparameterswithunseendata,andfinally,testedtoevaluateperformanceonthemostrecent,unseendata.Specifically,dataupuntil2018wasusedfortrainingandvalidation;and2019datawasusedforoutofsampletesting.Therationalforcuttingoursamplein2019wastoavoidtheimpactofthe2020COVIDshockinevaluatingourmodels’performance,asbothitssignificantexogeneityandmagnitudeweighingonglobaloildemanddrivenbythegovernments’restrictionsonmobility,aswellaspend-updemandinresponsetoliftingtheserestrictionsin2021and2022,wereidentifiedassourcesofdistortionbothformodeltrainingandevaluation.

Inthenextsection,we'lldelveintoModelDevelopmentandEvaluation,detailingthemethodologiesbehindmodelselection,training,evaluationmetrics,andthepost-processingtechniquesemployedtorefinetheforecasts.

2.5.Evaluationmetrics

Toassesstheperformanceofourforecastingmodels,weemploytwoprimaryevaluationmetrics:MeanSquaredError(MSE)andMeanAbsolutePercentageError(MAPE).Thesemetricsarechosenfortheirabilitytomeasureforecastaccuracyincomplementaryways,providingaholisticviewofmodelperformance.

MeanSquaredError(MSE):TheMSEiscalculatedastheaverageofthesquareddifferencesbetweentheactualandpredictedvalues.Itisgivenbytheformula:

iilsv,,iIuupliinlem,lthisaigeiiiit.fsi

deviations.

MeanAbsolutePercentageError(MAPE):TheMAPEmeasurestheaveragemagnitudeoferrorsasapercentageofactualvalues,offeringanintuitiveunderstandingofmodelaccuracy.Itisdefinedas:

MAPEisparticularlyvaluableincontextswheretherelativesizeoftheforecasterrorismoreimportantthantheabsolutesize,providinginsightsintothemodel'sperformanceinpercentageterms.Thismetricisgenerallyeasiertounderstandespeciallywhenwewanttocomparetheperformanceacrossthedifferentproductsthatmighthavevaryingerrormargins.

Thesemetricstogetherallowforacomprehensiveevaluationofforecastaccuracy,withMSEhighlightingmodelsthatmayhavelargeerrorsfordebuggingpurposes;andMAPEofferingapercentage-basedmeasurethatiseasilyinterpretableandusedforreportingandcomparingmodels.

10

Thecontentsofthispaperaretheauthors’soleresponsibility.Theydonotnecessarilyrepresenttheviews

oftheOxfordInstituteforEnergyStudiesoranyofitsMembers.

3.Modelselectionandtraining

Thefollowingsectionsexplainthetrainingandmodelselectionapproachthatwasused.Thisappliestoboththeshort-termandmedium-termforecastingmodels.

3.1.XGBoostfortimeseriesforecasting

XGBoost(ExtremeGradientBoosting)operatesontheprincipleofboosting,anensembletechniquethatcombinesmultipleweaklearners(typicallydecisiontrees)intoastronglearnerinasequentialmanner.Eachtreeattemptstocorrecttheerrorsmadebythepreviousone,withthisprocessguidedbythegradientofthelossfunction.

Decisiontrees,usuallycreateamodelthatpredictthetargetbycreatingtreesusingtheif/elsestatement,andbyusingtheminimumnumberofsuchstatementstheytrytofindtheprobabilityofhavingthecorrectdecision.Suchtreesareusedforclassificationorregression(asinourproblem).

Themodel'scapacitytohandlevariousdatairregularities,suchasnon-linearityandmissingvalues,stemsfromitsrobustlossfunctionoptimizationandregularizationfeatures.XGBoostintroducesaregularizationtermintheobjectivefunction,reducingoverfittingbypenalizingcomplexmodels.Thisfeature,combinedwithitsabilitytoperformautomatichandlingofmissingvalues,makesXGBoostparticularlywell-suitedfortimeseriesforecastingwheresuchirregularitiesarecommon.

Gradientboostingdecisiontrees(GBDT)suchasXGBoostarewellknownfortheirperformanceagainstdeeplearningmodelsintabularproblems.ThereareseveralGBDTpackages,however,inourcasewehavedecidedtoXGBoostasthemostusedbyboth,MLresearchersandacademia.Figure3illustratesthewayXGBoost(andotherGBDTmodels)finalizetheirpredictionswherethefinalpredictionforagivensampleisthesumofpredictionsfromeachtree.

Figure3:SchematicoverviewoftheGradientBoostingMachineLearningprocess

Source:RecreatedfromAWSSagemaker18

Thefiguredepictstheiterativeprocessoftrainingagradientboostingmodel.Initially,thedataset(x,Y)isusedtotrainthefirsttree,F1(x).Theresidualsr1arecomputedtomeasurethediscrepancybetweenthepredictedandactualvalues.Aregularizationparameterα1isthendeterminedtominimizethelosswhenF1(x)andr1arecombined.

Thecontentsofthispaperaretheauthors’soleresponsibility.Theydonotnecessarilyrepresenttheviews

11

oftheOxfordInstituteforEnergyStudiesoranyofitsMembers.

Thisprocessisrepeatedforeachsubsequenttree.Fortheithtree,themodelFi(x)isupdatedto

Fi—1(x)+αihi(x,ri—1),

whereαiandriaretheregularizationparametersandresidualscomputedwiththeithtreerespectively,andhiisafunctionthatistrainedtopredictresidualsriusingxfortheithtree.

TheobjectiveistofindtheoptimalsetofαparametersbyminimizingthedifferentiablelossfunctionL(y,F(x)),whichiscomputedas:

Throughthisiterativeoptimization,thefinalmodelFm(x)isacombinationofeachindividualtree'spredictionsadjustedbytheircorrespondingregularizationparameters,aimedatreducingthelossfunction.

Fortheshort-termforecastapplication(i.e.,2-yearsahead),themodeldevelopmentprocessinvolvedcreatingasinglemodelforeverytimestepinourpredictionhorizonforagivenregion(i.e.,24-stepsahead).Thismeansthatfortheshortterm,weneedtocreate24modelsforeachproductresultinginatotalof167modelstodoashort-termforecastforagivenregion.

Toretai