AI

eatsthe

worldBenedict

EvansNovember2024“Inmylifetime,I’veseentwodemonstrationsoftechthatstruckmeasrevolutionary:theGUIandChatGPT”Bill

Gates,March2023Benedict

Evans

––November2024

2(Microsoft

took20years

to

reach

a$150bn

valuation*)*$150bn

market

cap

in

summer1996.2024dollars.

Benedict

Evans

––

November

2024

3HugeinterestChatGPTreachedmainstreamconsciousnesswithunprecedentedspeedDenmark

USA

UK

Argentina

France

Japan70%60%50%40%30%20%10%0%AwarenessanduseofChatGPTbycountry,May2024Heardofbut

not

usedUsedBenedict

Evans

––November2024Source:Reuters

Institute4Hugeinterest,limitedusesofarGlasshalffull/halfempty-lotsofpeoplehavetriedit,butfewfoundituseful,sofarDenmark

USA

UK

Argentina

France

JapanHeardofbut

not

usedOnceor

twiceMonthlyWeeklyDaily70%60%50%40%30%20%10%0%UseofChatGPTbycountry,May2024Benedict

Evans

––November2024Source:Reuters

Institute5Development/Pilot

60%Production50%40%30%20%10%0%Hugeinterest,limitedusesofarEnterprisesoftwaretakestime,andcomewithearlydisappointmentsSoftware

CustomerserviceMarketingKnowledge

workersOperations

ITFinanceSales

HRLegalSource:

Bain

BenedictEvans––November20246EnterpriseusecaseadoptionratesforgenerativeAI,October2023&

February2024“Wait

a

minute?”AninvestmentsurgeaheadofaprovenmarketpromptstheobviousquestionsBenedict

Evans

––November2024

7Welcome

to

the

hype

cycle?It

always

takes

time

to

reach

the“Pleacom

ofProductity”BenedictEvans––November20248Beyondthenoise-thenextplatformshiftAfterthewebandsmartphones,alltechgetsbuiltaroundgenerativeAIWebOpenSourceSmartphonesCloudBenedict

Evans

––November2024

9GenerativeAIMainframesPCsSQLHowisthisuseful?Howfar

willthis

scale?How

do

wedeploy

this?ButeverythingiswideopenWedon’tknowtheanswers-we’restillworkingoutthequestionsBenedictEvans––November202410Howfar

will

this

scale?BenedictEvans

––November202411Scaling

willslow

down?‘Just’moresoftwareScaling

willkeepworking?LLMsdo‘everything’Foundationalquestion:willLLMskeepscaling?We

got

these

results

by

using

more

and

more

data

and

compute-will

that

keep

working?Sofar:Moredata+morecompute

=betterresultsBenedictEvans––November202412“Idon’tknowthatIwould

lookatthetraining

trends

andextrapolatethreeordersofmagnitudeahead

blindlyfromtoday”SergeyBrin“Justgiveyourselftheoptionthatwhat’sbeen

happeningforsixyears

nowisgoingtocontinue”KevinScott,MicrosoftCTOBenedictEvans––November202413Isitslowingdownrightnow?A

sudden

blip,or

something

more?BenedictEvans––November202414ScalingishardScaling

these

models

has

practical

challenges

and

will

take

time,even

before

the

science

questionsHowmuch

moretrainingdata

is

there?Lead-timesforGPUsandpowerAndwill

theresults

be

better?Execution&engineeringBenedictEvans––November202415We’regoingtofindout-ifonlyforFOMOAnasymmetricbet-over-spendingcapexhaslessdownsidethanlosingthenextplatform?BenedictEvans––November202416“We

have

no

moat”Internal

Google

memo,May2023

…Source:SemiAnalysis

BenedictEvans––November202417“Themodelsthatareintrainingnow…are

closer

in

cost

to$1bn

…and

then

Ithink

in2025and2026,

we’ll

get

moretowards$5bn

or$10bn”Dario

Amodei,Anthropic

CEO,April2024BenedictEvans––November2024

18“TheamountofcomputeneededtotrainLlama4will

likely

be

almost10x

morethan

what

we

used

to

train

Llama3-andfuturemodelswillcontinuetogrowbeyondthat”Mark

Zuckerberg,July2024BenedictEvans––November20241916,000NvidiaH100GPUs$30keach:~$500m54dayscontinuoustraining

timeLargestclustersarenowusing100k

GPUsTraining

Meta’s

Llama3.1SOTA

modelUnprecedentedcomputational(andcapital)intensitySource:

Meta

BenedictEvans––

November202420If

the

moat

is

capital

…Nvidiacan’tkeepupwithdemand-fornow(butsemiconductorsareacyclicalindustry)Mar-15Mar-16Mar-17Mar-18Mar-19Mar-20Mar-21Mar-22Mar-23Mar-24NvidiadatacentreNvidia

gaming&other

Intel403020100Quarterlyrevenuebysegment($bn)Benedict

Evans

––November2024Source:

Nvidia,Intel21The

capex

surge~$220bn

of

capex

at

the

big

four

in2024,up$90bn

from2023,and

all

expect

more

growth

in2025Source:Companies,company

guidance.Includes

capital

leases

Benedict

Evans

––

November

2024

22*Amazon

forecasts$75bn

total

capex,‘the

majority’for

cloud2015201620172018201920202021202220232024e250200150100500MetaAlphabet

AWS*MicrosoftCapex($bn)From

the

edge

to

the

centreRememberwhentelcosbuilttheinfrastructureandsoftwarehadnoassets?Mar-10Mar-12Mar-14Mar-16Mar-18Mar-20Mar-22Mar-2430%25%20%15%10%5%0%Source:Microsoft,Verizon.IncludescapitalleasesCapex/sales(TTM)Benedict

Evans

––November2024

Verizon

Microsoft23Here

come

the

bankersAfloodofcapitalcreatesopportunitiesforcapitalistsBenedict

Evans

––November2024

24EfficiencyTrainingcostInferencecostFootprintBetterresultsScaling!Plus

agents,multimodal,etcAndeverythingisstillchanging

underourfeetAllthescienceandengineeringquestionsarestillmovingThis

isamazing!(Nowwhat?)Benedict

Evans

––November2024

25Benedict

Evans

November2024ThelasttimesoftwarehadmarginalcostTheconsumerinternetmodelof‘launchfree,go

viral,

workoutrevenuelater’doesn’twork

with

today’sLLMcostmodelSource:

IBMHugeefficiencygainsMany

more

models,many

more

specs

to

measure,and

all

converging

on

a

commodityGPT4oGPT4oMiniGPT4TurboGPT4o(May)GPT4$0

$10

$20$30$40OpenAImodelquality(LMARENA)versuspricepermilliontokens(USD),November20241,4001,3001,2001,1001,000Source:Companies,LMARENABenedict

Evans

––November2024O1

Preview27The‘feeds&speeds’phase

of

the

marketMany

more

models,many

more

specs

to

measure,and

all

converging

on

a

commodity$0

$10

$20$30$401,4001,3001,2001,1001,000Modelquality(LMARENA)versuspricepermilliontokens(USD),November2024

OpenAI

Anthropic

Google

MetaSource:Companies,LMARENABenedict

Evans

––November202428Better

or

cheaper-plus

open

sourceBest,or90%asgoodat5%oftheprice$0

$10

$20$30$401,4001,3001,2001,1001,000Modelquality(LMARENA)versuspricepermilliontokens(USD),November2024

OpenAI

Anthropic

Google

MetaSource:Companies,LMARENABenedict

Evans

––November202429“Everyoneintechisgivingsomeoneelse’sbusinessmodel

away

for

free”Meta’sopensourceTurnmodelsintocommodityinfrastructure!Benedict

Evans

––November2024

30“Everyoneintechisgivingsomeoneelse’sbusinessmodel

away

for

free”Apple’sedgecomputingTurnmodelsintojustanotherAPIcall!BenedictEvans––November2024

31The

great

model

boom

of2023-2024Better,faster,cheaper-pick

twoBestcheaper

modelBest

modelAndmoreandbetterevery

fewweeksBest

modelthatfits

ontheedgeBenedict

Evans

––November2024

32“Ifanythinginthislifeiscertain,ifhistoryhastaughtusanything,itisthatyou

can

kill

anyone.”MichaelCorleoneBenedict

Evans

––November2024

33“Ifanythinginthislifeiscertain,ifhistoryhastaughtusanything,itisthatyou

can

kill

anyone.”SemiconductorsarecyclicalCommoditytechgoestomarginalcostBenedict

Evans

––November2024

34“Ifanythinginthislifeiscertain,ifhistoryhastaughtusanything,itisthatyou

can

kill

anyone.”SemiconductorsarecyclicalCommoditytechgoestomarginalcostAnd

every

new

tech

produces

a

bubbleBenedict

Evans

––November2024

35Howisthis

useful?Benedict

Evans

––November2024

362013:how

is

MachineLearning

useful?“That’s

clever

…but

so

what?”Source:

Imagenet

BenedictEvans––November2024372013:how

is

Machine

Learning

useful?

What’stherightlevelofabstractiontounderstandthis?And

so

dolotsofother

unsolved

problemsImagerecognition

worksnow!What

can

weturnintopatternrecognition?ThisispatternrecognitionBenedict

Evans

––November2024

382023:why

is

GenerativeMachineLearninguseful?“That’s

clever

…but

so

what?”Benedict

Evans

––November2024

39Andwhatcan’titdo?‘Answerthisquestion’‘Whatdoanswerstoquestionslikethistendtolooklike?’‘What

would

the

average

person

probably

say?’Benedict

Evans

––November2024

40ErrorawarenessisstilllimitedAndtheapparentfluencyoftextoutputconcealsthenatureofthemodelbehindThosewhoagreethat

“GenerativeAIalwaysproducesfactuallyaccurateanswers”Thosewhoagreethat

“GenerativeAI

responsesareunbiased”50%40%30%20%10%0%Source:Deloitte

Consumer

Digital

Trends,UKAware

of

GenAI

Used

GenAIAware

of

GenAI

Used

GenAIBenedict

Evans

––November20242023202441Handling‘errors’inaprobabilisticsystemIs

this

a

science

problem,or

a

use

case

and

design

problem?Product

designAbstractthepromptandtheoutputUse-casesArethere‘wrong’answers?Are

errors

easy

tosee?Makethemodelsbetter!(Well,duh)Benedict

Evans

––November2024

42Can

you

use

this

for

generalsearch?Do

you

want

navigation,or

just

the

answer?Does

it

matter

if

it’s

wrong?Can

you

tell?Howmuchpreprocessingandfilteringdoesthis

need?(Alphabet

had$56bn

of

FCF

in

the

last12

months,soit’sworthfindingout)Benedict

Evans

––November2024

43Synthesisandsummary?Reasoning?(Not

yet)‘Predictthenexttoken!’Itautomatesabroad

class

ofnewthingsYes,butwhat?2024:how

are

LLMs

useful?

What’stherightlevelofabstractiontounderstandthis?Benedict

Evans

––November202444AIgivesyouinfiniteinternsBenedict

Evans

––November2024

45Canwepredicttheeffectsfromfirstprinciples?Useful-butimaginetryingtodothisexercisefor

‘theinternet’

in1995Content?Internetmade

distribution‘free’-do

LLMs

make(some)creation‘free’?Language?Howdoesseamlesstranslationchangetheweb?Science?Pop

culture?Costarbitrage?LabourversusautomationBenedict

Evans

––November202446Can

we

predict

the

effectstop-down?Imaginedoingthisfor‘the

internet’

in1995Or‘mobile

internet’in2005Whatwouldyouhavegotright,andwrong?Source:The

Labor

Impact

of

Generative

AI

on

Firm

Values,Eisfeldt

et

al

Benedict

Evans

––

November

2024

47“ThejuryisstilloutoutonwhetherCopilotisusefulenoughtojustifythe

cost”Chevron

CIO,July2024Benedict

Evans

––November2024

48Doyouhavethatusecase?"VisiCalc

took20hours

of

work

for

some

peopleandturneditoutin15minutes”-DanBricklinNow,imagine

a

lawyer

seeing

it:”that’s

veryclever,but

I

don’t

have

that

use

case”Source:Computer

History

Museum

Benedict

Evans

––

November

2024

49Customer

support(Butuse

withcaution)EarlyadoptersMarketingErrorseasytoseeNo‘wrong’answersCodingErrorseasytosee20-30%efficiencygains“I

have

that

use

case!”Immediately,obviously

useful

for

some

professions

and

some

workflows

…Benedict

Evans

––November2024

50“I

have

that

use

case!”Glass

half-empty

or

glass

half-full?10%adoption

already

is

great-but

why

do90%not

find

it

useful?Employedpeople,

“DoyouusegenerativeAI?”responsebyoccupation,USA,August2024TOTAL

M'gment

Software

Business/Education

finance50%40%30%20%10%0%Sales

Health

Office/adminWeeklyactiveusersDailyactiveusersSource:

Blick,Blandin&

DemingScience

/

engineeringBenedict

Evans

––November2024Legal

ServicesArts

/

mediaBluecollar51“You’ve

got

to

start

with

the

customerexperienceandworkbackwardstothetechnology”SteveJobsBenedict

Evans

––November2024

52How

do

we

deploy

this?Benedict

Evans

––November2024

53Redefine

marketsSometimes(Toughtopredict)InnovateNewideas,newproductsStartups

use

it

tounbundleAbsorbMake

it

a

featureBoltitonto

theexistingbusinessHow

do

we

always

deploy

new

technologies?StandardpatternsfordeployingandusingnewtechnologiesBenedict

Evans

––November2024

54IsthisanAccenturequestion?OraBain/BCG/McKinseyquestion?A

CIO

question?CFO?CEO?Toplineorbottomlineinnovation?Benedict

Evans

––November2024

55“What’s

our

AI

strategy?”AccenturenowclaimsmoregenerativeAIrevenuethanOpenAI-butalmostallofthisisstillpilotsFeb-23May-23Aug-23Nov-23Feb-24May-24Aug-241,2501,0007505002500Accenturereportedquarterly

‘generativeAIbookings’($m)Benedict

Evans

––November2024Source:Accenture56StandardtechprocurementquestionsHow

do

enterprises

always

deploy

new

technology?What

questions

do

they

always

ask?Singlevendor

ormulti-

vendor?Buy

versusbuild?Opexorcapex?What’s

theEPS

impact?Whichusecasesfirst?Whosebudget?BenedictEvans––November202457ThefuturecantakealongtimeCloudisoldandboring-butstillonly30%ofworkflowsDec-14Dec-15Dec-16Dec-17Dec-18Dec-19Dec-20Dec-21Mar-23Jun-2460%50%40%30%20%10%0%Enterpriseworkloadsinpubliccloud Expected

in3years TodaySource:Goldman

Sachs

CIO

SurveyBenedict

Evans

––November202458ThefuturecantakealongtimeAquarterofCIOshavelaunchedsomething-buthalfdon’tplananythingforatleastayear0%25%50%75%100%2023H12024H2

2024H12025H220252026orlaterCIOexpectedtimingforfirstLLMprojectsinproduction,August2024Source:

Morgan

Stanley

CIO

SurveyBenedict

Evans

––November202459Buteventually,newplatformsmeannewtoolsSaaSenabledahugeexpansioninautomation,unbundlingworkflowsoutofSAP,ExceloremailOperations

Information

Technology

Engineering

Product

Sales

MarketingAveragenumberofSaaSappsusedperdepartment,largeenterprises1007550250Benedict

Evans

––November2024202120222023Source:Productiv60“There

are

two

ways

to

make

money.You

can

bundle,or

you

can

unbundle”JimBarksdaleBenedictEvans––November202461Scaling

willslow

down?‘Just’moresoftwareScaling

will

keepworking?

LLMscan

do

‘everything’Doesthiskeepscaling?UnlesstheLLMcandothewhole

thing?If

the

models

get

good

enough,maybe

we

will

need

much

less

software?Benedict

Evans

––November2024

62New

features,newproductsLLMsarejustanotherAPI?LLMsaretheentireplatformEverythingelse

is

an

API?Are

LLMs

Infra?APIs?Platforms?The

new

UX?WillweuselogicalsystemstocontrolLLMs,oruseLLMstocontrollogicalsystems?Doesthiskeepscaling?Benedict

Evans

––November2024

63“ItIs

not

the

customerIs

job

to

know

whattheywant”SteveJobsBenedict

Evans

––November2024

64LLMs

as

UXUsers

forced

toinvent&imaginetheusecasesClassical

softwareStartupsinvent&imaginetheusecasesDo

LLMs

break

our

use-case-discovery

models?How

do

entrepreneurs

invent

new

use

cases

and

forms

of

self-expression

if

everything

has

the

same

UX?DoLLMsdo

the

wholething?Benedict

Evans

––November2024

65Newusecases,newunbundlersHowmanyAIstartuparereallyabetonunbundlingbothOracleandChatGPT?W2015W2016W2017W2018W2019W2020W

2021W2022W2023W20245004003002001000YCombinatorstartupsbyfieldBenedict

Evans

––November2024Source:Y

CombinatorOtherAI66Sometimes,it

really

is

just

a

feature“AIis

whateverdoesn’tworkyet”Source:

Apple

Benedict

Evans

––November202467‘AI’

tendstoturninto‘a(chǎn)utomatic’Astechnologymatures,itdisappears‘Smart’AI!Justsoftware‘Auto’Benedict

Evans

––November2024

68NewtoolsRead500

10ksand

tell

me

…NewfeaturesRewritemyemailSummarisethereviewsThreemodelsforLLMproduct?How

far

do

LLMs

enable

new

features

and

how

much

do

they

just

replace

apps

entirely?GeneralisedAI?Buymea

houseBenedict

Evans

––November2024

69“‘Intelligence’iswhatever

machines

havenIt

done

yet”Larry

Tesler,1970“Infromthreeto

eightyearswewillhaveamachinewiththegeneral

intelligenceofanaveragehumanbeing”Marvin

Minsky,1970Benedict

Evans

––November2024

70Insertyourquestion

here

…Answer:“It

will

work

likeeveryother

platformshift”Answer:“No-oneknows”Perhaps,all

AI

questions

have

one

of

two

answersWillthisjustbelikeeverythingelse?No-onereallyknowsBenedictEvans––November2024

71Meanwhile

…Benedict

Evans

––November2024

72Ideasfor2030GenerativeAIIdeasfrom2010SaaS,automation,

collaboration,

workflow…Ideas

from2000“Maybe

peoplewillbuy

thingsonline”FromvisiontodeploymentWhat’s

already

big,what’s

being

built,and

what

comes

next?Benedict

Evans

––November2024

73MetaisstillmetaversingMeta

still

believes

in

VR&AR-it’s

invested

at

least$60bn

so

far,and$17.4bn

in

the

last12monthsApple

R&D,8quarters

before

iPhone

launch

Meta

Reality

Labs

operating

loss,

last8quartersQuarterly

investment($bn)543210Benedict

Evans

––November2024Source:Apple,Meta74E-commerce

is

still

thereThemostboringchartintech,

withabriefexception-butthisistrillionsofdollars

ofglobal

valueSource:US

Census.

Seasonally

adjusted

Benedict

Evans

––

November

2024

75*Excluding

cars,car

parts&fuelMar-00Mar-05

Mar-10Mar-15Mar-20Jun-2425%20%15%10%5%0%

Addressableretail*

RetailE-commerceas%US

retail‘Backtothe

trendline’But40%of

UK

non-food

is

now

onlineSource:

ONS

BenedictEvans––November20247620102011201220132014201520162017201820192020202120222023202470%60%50%40%30%20%10%0%

Shareofnon-food

Shareof

total Share

of

foodE-commerceas%

UKretailNewchannels,newwinnersSheinisontracktobethelargestpure-playapparelretaileronearthSource:Companies,press

reports

of

GMV

for

Shein.

Benedict

Evans

––

November

2024

77*

TJX

reports$54bn

including$9bn

of

home

goods.NB

Amazon

has

est.$60-80bn

apparel

salesTJX*Shein

Inditex

H&M

UNIQLO

Gap

lululemon

PVH

Next

Ralph

Victoria's

Americanathletica

Lauren

Secret

EagleLargestglobalpure-playapparelretailersbyrevenue,2023($bn)6050403020100UnbundlingAmazonShopify

powered$270bn

of

e-commerce

in

the

last12months-35%the

size

of

Amazon

GMVMar-15Mar-16Mar-17Mar-18Mar-19Mar-20Mar-21Mar-22Mar-23Mar-24300250200150100500Shopify

GMV,

TTM($bn)Benedict

Evans

––November2024Source:Shopify78BreakinghabitsThe

pandemic

drove

a

step

change

for

grocery,and

Instacart

now

has

a$33bn

run-rateMar-18Mar-19Mar-20Mar-21Mar