MITINITIATIVEONTHEDIGITALECONOMYRESEARCHBRIEF2025,VOL.2

GenerativeAIandtheNatureofWork

INTHISBRIEF

?Artificialintelligencehasbeenshowntoimprovehumanproductivity,butcouldthetechnologyalsochangethenatureofworkitself?Toexplorethisquestion,ateamoffiveresearchersconductedoneofthelargest-evernaturalexperimentsofGenerativeAI.

?Overthecourseoftwoyears—July2022toJuly2024—theresearchersobservedthecodingworkofmore

than187,000softwaredevelopersusingGitHubCopilot,aGenAItoolforsoftwaredevelopment.

?Theresearchersfocusedonopensourceinpartbecausethesedeveloperstypicallyworkindecentralizedsettings.They’realsooftenoverburdened,spendingmoretimethanthey’dlikeonprojectdevelopmentandlesstimethanthey’dlikeonactualcoding.

?TheresearchersfoundthattopdeveloperswhoreceivedfreeaccesstoGitHubCopilotduringtheobserved

periodincreasedtheircodingtasksasashareofalltaskstheyperformed.Thedevelopersalsoreducedtheirrelativeshareofactivitiesrelatedtoprojectmanagement.

?Inaddition,GenAIwasfoundtobemosthelpfulforopensourcedeveloperswithlowerlevelsofcodingexperience.Duringtheobservedperiod,lower-abilitydevelopersusingGitHubCopilotincreasedtheircodinganddecreasedtheirprojectmanagementmorethanItheirhighlyskilledcounterparts

?TheexperimentshowsthatGenAIholdsgreatpromiseforallowingopensourcedeveloperstospendmoretimeinthemannertheyprefer—writingcode—whilealsoensuringtheirsoftware’ssecurity,stabilityandusability.What’smore,asdistributedworkbecomesincreasinglycommon,thiseffectislikelytogeneralizetootheroccupationsandsettingsaswell.

RESEARCHOVERVIEW

Amerehandfuloftechnologicalinnovations—amongthemtheprintingpress,internalcombustionengineandgeneral-purposecomputer—havefundamentallychangedthewaypeopleliveandwork.Givenrecentadvances,artificial

intelligencemayjointhiselitecategoryoftechnologies

(Crafts,2021;Goldfarbetal.,2023;Eloundouetal.,2024).

AI’sgreatesteconomicimpactcouldbeimproving

productivityinknowledge-intensiveindustries(Manyikaetal.,2018;Sachs,2023).

However,researchintoAI’simpactisstillnascent.Thisis

especiallytrueforGenerativeAI,asubsetofthetechnology

builtonlargelanguagemodels(LLMs).Earlystudieshave

shownthatGenAIcanmakehigh-levelimpactsonproductivity(Brynjolfssonetal.,2023;Dohmkeetal.,2023;Noy&Zhang,2023;Pengetal.,2023).Lessclear,however,arethe

mechanismsdrivingtheseimprovements.

Onehintcomesfrompriorresearchshowinghow

technologiesthatstreamlinecommunicationanddecision-

makingprocessescanreducetheoverheadofcollaboration,freeingworkerstofocusontheirownworkinisolation(Farajetal.,2011;Aral&VanAlstyne,2011).GenAItakesthat

processastepfurther.Withthetechnology,manyofthesecollaborativecostsaresimplyeliminated.Workthat

previouslyrequiredcommunicationamongmultiplepeoplecannowbedonewithoutanyinteractionatall.

Toexploretheseandrelatedissues,ateamoffiveacademicandbusinessresearchers—ManuelHoffman,SamBoysel,

FrankNagle,SidaPengandKevinXu—designedandconductedanaturalexperiment.

1

2

Theysoughtanswerstoquestionsincluding:

?WhatistheeffectofAItechnologyontaskallocationacrossspecifickindsofcoreworkandprojectmanagement?

?WhenworkersuseAI,dotheyfavorexploitationorexplorationintaskallocation?Thatis,aretheymorelikelytoincreasetheireffortsonprojectstheyarefamiliarwith?Oraretheymorelikelytobranchoutintoprojectsthatthey’veneverworkedonbefore?

?IsittruethatAIhelpslower-abilityworkersmorethanithelpshigher-abilityworkers?

Theresearchersdescribetheirexperimentanditsresultsinarecentworkingpaper,

GenerativeAIandtheNatureofWork

.

THEEXPERIMENT

Toconducttheirnaturalexperiment,theresearchersfirstneededasettingwithtwosharedcharacteristics:One,theworkisdonefromdistributedlocations,andtwo,theworktasksofindividualscanbeobservedingreatdetail.Astheresearchersdiscovered,bothrequirementscouldbemetinopensourcesoftwaredevelopment.

Theresearchersalsoneededabefore-and-aftersettingin

whichtheeffectsofanewAItoolcouldbeclearlyobserved.

Tomeetthisadditionalrequirement,theresearchersselectedtheJune2022publicreleaseofGitHubCopilot,anAI

software-developmenttool.

GitHubofferedseveralclearbenefits.Astheworld’slargest

hubforopensourcesoftwaredevelopers,GitHubprovides

cloud-basedservicesforbothsoftwaredevelopmentand

versioncontrol.Moretothepoint,GitHubhasbeendesignedforusebygeographicallydispersedteams.Also,GitHub

documentsallactivitiesperformedonitssystem.Thisallowedtheresearcherstoobserveingranulardetailtheworktasks

completedbyremoteteamsofsoftwaredevelopers,makingitanidealsettingfortheirnaturalexperiment.

Theresearchersoptedtostudytheimpactofonespecific

GitHubtool:Copilot,aGenAIsoftware-developmenttool

developedjointlybyGitHub,OpenAIandMicrosoft.WhileGitHubCopilotisbasedonapredictivemodelsimilartothatusedbyChatGPT,thetooldiffersinimportantwaysfrom

bothChatGPTandMicrosoft’ssimilarlynamedCopilottool.

DevelopersusingGitHubCopilotcangeneratecodesnippetsthatareeasilyintegratedintoexistingcodebases(Fig.1).It’sapopularapproach;inonerecentsurvey,ninein10U.S.-baseddeveloperssaidtheyuseanAIcodingtool(Shani,2023).

Figure1:GitHubCopilotinaction.First,thehumandeveloperwroteafunction(PanelA).Then,basedonthisprompt,Copilotsuggestedtherest(PanelB).

Source:GitHub,2022

Thenaturalexperimentconsistedofapanelof187,489developers,andtheresearchersobservedthesedevelopersonaweeklybasisfromJuly2022toJuly2024.Overthistwo-yearperiod,theresearchersmadeliterallymillionsofobservationsofthedevelopers’weeklywork.

Theresearchersfocusedspecificallyondevelopersdeemed“topmaintainers”byGitHub,whichmadethemeligibleforfreeaccesstotheCopilottool.(MostdeveloperscanuseGitHubCopilotforfreeonlyduringshorttrialperiods.

Thereafter,theymustpayamonthlyfee.)

Theresearchersfurtherorganizedtheirobservationsalongtwomaincategoriesofessentialdeveloperwork:codingandprojectmanagement.Undercodingtheyincludedthemoretechnicalprocessesofwritinglinesofsoftwarecode.

3

Projectmanagementwastheheadingformostremainingactivities,incIudingassistingotherdeveIoperswith

softwareissues,introducingnewideastothedeveIopercommunity,anddiscussingIong-termobjectives(Fig.2).

ThereareothertasksadeveIopercandothatdonotfaIIin

oneofthesebuckets;sojustbecausecodingincreasesasa

shareofaIIactivities,projectmanagementdoesn)tnecessariIyneedtodecrease.

Figure2:Classificationofsoftwaredevelopers’workactivities.Eachcategoryisdefinedasthesumofitsdisaggregated,granularactivities.

THERESULTS

TheresearchersfirstestabIishedthatGitHub)sprogramfor

topdeveIopersincreasesCopiIotusageforeIigibIeusers.TopdeveIopersusedCopiIotsignificantIymorethanother

CopiIot-adoptingdeveIopersdid,andmoreofthemadopted

CopiIot.

Next,theresearchersexpIoredthecausaIimpactofaccesstoCopiIotonpatternsofdistributedwork.OveraII,theyfoundthatGenAIinducesdeveIoperstoreaIIocatetowardcore

work.AmongthetopdeveIopersobserved,theircodingworkasapercentageofaIIactivityincreasedby5.4%,whiIetheirprojectmanagementworkasapercentageofaIIactivity

decreasedby10%.ThisaIsoimpIiesthatdeveIoperswith

accesstoGenAItooIsareIessIikeIytoseekheIpfromotherdeveIopers.Instead,theyusetheGenAItooItoaddresstheirprobIemsorinquiries.

DuetotheIong-termnatureofthenaturaIexperiment,the

researcherscouIdaIsoexaminewhetherthedeveIopers)useofCopiIotchangedovertime.ItturnedoutthatthestrongesteffectstookhoIdduringthefirstyear.Then,aftersome

experimentation,theimpactswerestabIeforapproximateIytwoyears(theendofthestudy).

AnotherquestionansweredbythenaturaIexperimentwas

whetherGenAIinducesdeveIoperstoworkmore

autonomousIy.OveraII,thedatashowedthatCopiIotaIIoweddeveIoperstoworkbythemseIvesmore;therefore,they

workedwithothersIess.ThatwasmainIyaresuItofthe

GenAItooIaIIowingthedeveIoperstospendmoretimeontheircore(andmoresoIitary)activityofcoding.

YetanotherquestiontheresearchersexpIoredwaswhetherGenAIencouragesdeveIoperstobranchoutinto

experimentaIworkthattranscendstheirestabIishedprojects.Theshortansweris,itdoes.Theresearchersfoundthaton

average,CopiIot-eIigibIedeveIopersengagedwith15morenewprojectsthandidtheirineIigibIepeers.Thesesame

deveIopersaIsoincreasedtheirexposuretonew

programmingIanguagesbynearIy22%reIativetothebaseIine.

OnefinaIquestionwaswhetherGenAIheIpsdeveIopersofaIIabiIityequaIIy.Toanswerthis,theresearchersusedmeasuresthatincIudedprojectworkIoad,contributiondiversity,andpopuIarinterestfrompeers.TheyfoundthatCopiIotheIpedIow-abiIitydeveIopersmorethanitdidthoseofhighabiIity.

CONCLUSIONS

ThisnaturaIexperimenthasseveraIbroaderimpIications.OneisthatmanagersmaybeaidedbyGenAI)sabiIitytochange

themakeupofwork.Formanyorganizations,GenAImay

bringaboutmorestreamIinedproductionprocesses.AnotheristhatGenAItechnoIogyhasthepotentiaItofIatten

organizationaIhierarchies.YetanotheristhattaIented

workersmayuseGenAItorefocustheirworkonbothcoreprocessesandnew,expIoratoryinnovation.

TheresearchersaIsobeIievethatGenAIcanheIpsoftware

deveIopersearnmoremoney.AdeveIopercanuseGenAI

tooIstogainexposuretonewprogrammingIanguage,and

theresearcherssaythatnewskiIIcanincreaseadeveIoper)searningpotentiaIbyanestimated$1,683ayear.Thatfigure,

4

multipliedbythefullsetof300,000developersworkingonopensource,suggestthatCopilotcouldimprovetheir

combinedannualincomebyasmuchas$468million.

Theresearchersconcedethisisaback-of-the-envelope

calculation.YettheybelievethatgivenGenAI-powered

productivityimprovementsandotherexperimentation,thetruevaluecouldbeevenhigher.

REPORT

Readthe

fullworkingpaper

ABOUTTHEAUTHORS

ManuelHoffman

isanAssistantProfessorattheUniversityofCalifornia,Irvine,andaformerPostdoctoralFellowat

HarvardBusinessSchool.

SamBoysel

isaDataScientistatTheLinuxFoundation

andaformerPostdoctoralFellowatHarvard

University’sLaboratoryforInnovationScience.

FrankNagle

isaResearchScientistwiththeMIT

InitiativeontheDigitalEconomy(IDE);AdvisingChief

EconomistforTheLinuxFoundation;andaformer

AssistantProfessoratHarvardBusinessSchool.

SidaPeng

isaResearchEconomistintheOfficeofthe

ChiefEconomistatMicrosoftResearch.

KevinXu

isaStaffSoftwareEngineeratGitHubInc.

REFERENCES

Aral,S.,andVanAlstyne,M.(2011).

Thediversity-

bandwidthtrade-off

.AmericanJournalofSociology,vol.

117,no.1,pp.90-171.

Brynjolfsson,E.,etal.(2023).

GenerativeAIatwork

.

NationalBureauofEconomicResearch,workingpaper31161.

Crafts,N.(2021).

Artificialintelligenceasageneral-purpose

technology:anhistoricalperspective.

OxfordReviewof

EconomicPolicy,vol.37,issue3,pp.521-536.

Dohmke,T.,etal.(2023).

Seachangeinsoftware

development:economicandproductivityanalysisofthe

AI-powereddeveloperlifecycle.

arXiv:2306.15033.

Eloundou,T.,etal.(2024).

GPTsareGPTs:Labormarket

impactpotentialofLLMs.

Science,vol.384,issue6702,pp.1306-1308.

Faraj,S.,etal.(2011).

Knowledgecollaborationinonline

communities.

OrganizationScience,vol.22,no.5,pp.

1224-1239.

Goldfarb,A.,etal.(2023).

Couldmachinelearningbea

generalpurposetechnology?Acomparisonofemerging

technologiesusingdatafromonlinejobpostings.

ResearchPolicy,vol.52,issue1,article104653.

Manyika,J.,etal.(2018).

Thepromiseandchallengeof

theageofartificialintelligence.

McKinseyGlobal

Institutebriefingnote.

Noy,S.,andZhang,W.(2023).

Experimentalevidenceon

theproductivityeffectsofgenerativeartificial

intelligence.

Science,vol.381,issue6654,pp.187-192.

Peng,S.,etal.(2023).

TheimpactofAIondeveloper

productivity:EvidencefromGitHubCopilot.

arXiv:2302.06590.

Sachs,G.(2023).

GenerativeAIcouldraiseglobalGDPby

7%.

Goldman