State

of

AI

Report2023//~yó??^2023~10oAir

Street

CapitalRYù?ì^?|~?|?y|?o?s?#stateofai

|2_?ty~y?AI?oùüy{?[ù???wtoo[°~y~2s?t???tu_W1uo?ú?^}o?ty~y^r~ao?o???]T~2?o?~?ys?|??NW???Wrí???yo~y??{2Air

Street???}~ó_ty~yó??^2s?ê???^?~bé??goˉ????o?t?o_óNs?ty~yó??w?ogY?÷???2?^o?×??O?y÷?l

ì^?ty~y?W^T}???wW^Y2l

|~?ao?|?wy?2l

??ty~y?÷??w?oYüty~yw???2l

?y??{ü??úgyogty~y???y?s?~g?~?gT?2l

?o?s?ˉ~og12no_ó?N????cs???t?2022~?m?2u

Air

Street

Capital

??v_2?ˉùa???}_÷?qqtech??ì?uvmW???or?2sì?ˉùa??}_÷?qqtech??T]<AI2023=?]?·×o?^PDFz2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|3RYùì^?6y-57y?--?wT?T1_[}?1?ùa}?ü|~t\?GPU?ˉz?n?ó_??u?1O??S?^|?^?2--?ó?v?v??o??Y?ˉw??O?ˉw?TW_ó?O×?ó?v????2--ChatGPT?~?O?GenAIT}t?1??1y1?÷vCoPiloty÷y×???|g?N~??ú?180??S?T?÷íüO^÷í2|~?58y-115y?--GPT-4?w~]{?_oük_t??yh

O??y????\÷_????t??oWn???2--}?_?k?1?}?uo?1???NO?g[?v?-_ok?gy?ú??g?[&uLLaMa-1/2ó?oc2--t_?Oo{tr?uo??tcty~yk]??[O?ot??r2025~????k?^?}uo?????ò\ruo?Yo?N2??O^o???üuo?yoONnto2--???k?ükgk???~V[{ü?{óó~g???|???~}ù??ó?}{2--[kar~??_??boóNy?u[_T?2??116y-127y?--yw??RV?n~?÷?·￥?Oyw?v??]?wc2g?ty~y???k{í?Nz2--ˉwz?oT??2--ty~y??^YN?Wo???W??~ü}^?O^or~×Y2?y?128y-145y?--[T?ˉ??kr~m?~~×òw2--?[úgy????w8?ó92~?íuRLHF?sz?|~t\k???yh

????si×üyot_}?ì?ˉ?2--?wy??óú??SOTAk??ìN??????g???2?oq?oO_?2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|4TtAir

Street

2022~??ostateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|5s??2022~??o??12DeepMindˉ?Nnwo10Bóu?RLk???Gato10]2ó_??_NN?_ì?AGI?t?t?zus?2rt_~k??ío}_?·??y÷?|~2ó_?íot??ys???o?[_ì?AGI?t?oò?÷íú?W?Cohere1InFliectionAIüAdept2SOTA??k??Chinchilla[10]?uoùNˉ???n?uo?Nós?O~??Oo?ó?GPT-4o13TtokensNˉ????Chinchillao14TtokensNˉ??2Meta?Llama-2o2TtokensNˉ??234u2r2023~9o?r÷?yw??ó|_???10O]_ót\2?ùaO^GAFAMU?}ty~yv_tty~y}?÷í??10??S2????g?ElevenLabsüResemble.aiy?ù?o??100O}w22023~1o????UOpenAI÷í?100??S2÷?ó_???wO?[?__[}?÷~ó??N?n_[}?|?v567o???Oíoík\yvO??w-2?O?wg?|50%??|?w-2÷?{?y????BSLā{?}ty~y????óh?r?_??ó?1??vo????oc2?n÷??|?n~óú?O?BSL?oc?ío2?ws?o?r?ty~yy??Y??yb÷~?T??n~^o??1??S÷í?_y?ty~yi×t?2ty~y|~ü?y}?Anthropic2023~

o{??ú?40??S?í?29N???}wr?????RedditāNN?ty~yk???OpenAIā_[}?_WW^\_????w}wr???y??ì?ˉ2OpenAI??·????wTShutterstockˉ?uo?t?1??ü÷^???tsSuoā?6~??28stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|6Nú?ì^stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|7GPU?nz?ó_?￥w?÷?^|?uO??S?^?ó_?2023~?{t￥w?r

[t_?103.2??S

??N{tT?141%??N~_T?171%2

}??}?2022~￥w?r

270??S

??2021~T?61.4%?O^???tcò?a

t2

ó_?ó?^|~1O??S

??10~_?85??S

ú?116]2?RYù?ì^?|~?|?y|?o?s?#stateofai

|8?CoachellaW????????x?yó?Wko

a-GPUCoreWeaveüLambdaon????ó_?\__??ˉ?tü?ìGPUuoo????{qouO?GPU2

Lambdawùí?oó???|9Ou?S

?H100s?~Nn[_÷??Nz2

CoreWeaveo^Ng?GPU?￥WON2

?}??~~??t°÷?è??ó??k?2024~N{t?\\2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|9ù￥}?ko

c

ó_?GPU?~^w_~t?_?RYù?ì^?|~?|?y|?o?s?#stateofai

|10??o

~????÷y?où?y{??o?_???ùa(Kaust)-~?3000[?H100sgt°????ShaheenIII2

?ó????^?2023~?÷u?ì2No\÷????t??\^?ao[?|~b

_ó?FalconLLM?o?_?ó_?×-??ít2g^?{?O??_n??G42N{?O????Cerebras?r

_ˉ?-~?}??|ú?9??S?v

W??????~t°9óT??ty~y????2

og?y_o?[?o?&o?^u?ˉùauvmW???or?2

sì?ˉùa??}_÷?qqtech??T]<AI2023=]?·×PDFz2RYù?ì^?|~?|?y|?o?s?#stateofai

|11??gu?ó_?

A100????~?g?íkó_?

A100

GPU???u?N?T??{{o{??üStability???H

uggingFace????2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|12??gu?ó_?H100??ó?~÷^í?Où￥üN^}?k?_??H100?x?y?}?íkk??ˉ2

jó10ooí??oüInflection^oy÷ó]?s?÷ówT}?W?OpenAI1Anthropic1Meta1Character.ai1Adept1Imbue{yo??y?2

s???O

O

_o?[???N?2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|13ty~y|~??o

}?ó_?ˉw?bo

??ˉw?{ü[19]?~??^o?s?_???ty~y|~??o{?[?_?{}??2

s?óó?ó_?ˉw?_}?ku??[????2

2023~?ó_?GPU?×k???FPGAs[31]??TPUs[150]231x150xstateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|14ó_?ˉwwo

?~??}|?|??N^rmì?rtü??5~÷?2023~?b

o?tYy?oó_?????H100

GPUN??oA100??^??2

?wH100??kt?o?~?ío?y??O|~t\???V1001A100üRTX

30902

ó_????t??}t_êt?2017~ó_?V100t_?woty~y|~og~}?ˉw2

?èn?2020~ó_?A100?y2026~?r

ü|??V100?y_?r

O

?2

?o??H100?y_?s??r

O

NnS~?stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|15ó_?o

???OCerebrass

z?_o

}·}?ywg?ty~yˉw?[°

Cerebras?óN??~n_tk?ˉ?üuo?[tùt??~???t??k?×|~t\?k?2

O{_N?ó_??t???o?????y2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|16?íkO^^óúty~yo

?bs

{ío

o

????oo?ù?ó_?^2024~?ˉ150Oó200O?H100?ú??~???50O?2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|17{???U

yw_Tty~y????s?2022~???g

uo?{????w100

GPU?uó]?2

jó2023~_y

??}????Nn??u1O?H100tr

????r

~??~kg????ON2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|18?g?[??ík}?~??ty~yy_m_ó?y??v|tMetaó_?MTIA??o?}?Nn??_tRISC-V÷????ò?~???????ytn??ók???n2

?ou?o??k?O

íT??íkü]g

g

??GPUó??c?v?tb

?ú?2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|19ó_?1

ó{\üAMDv°U

o

??ó???vˉwy

oó_??ˉó???y?o???só_?uoo?ts??￥w?20-25%(??÷?)2

ó_???wt??k?_ó?O

??óoW?|?ˉw2?

2022~8oOí?ó_??A100üH1004?}?ty~yT}y÷g?ˉw4?Wu??Wó???ó?voW2r_~11o?ó_???_??oA800üH800ˉw?w??O???ù????gy?|2?

ó{\?T??HabanaGaudi2ˉw??zoZ???u?AMD_è???ot2?

?o?o?T?}????-?|?10??S?S?ó_?A800/H800

GPU2_o?óùo??A100/H100

GPUm?obTò?Oík?_?[2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|20??ó_???w??^???÷O?ArmY????[N^Tr

2020~?s??oó_?^?y?r

?Arm?w-2

?~9o?Arm???[Y?N^?_t÷^|?r

600??S

2?

Arm?÷??g

o?wyw99%?~ykˉw??}?kú?Y??O?yty~y^???\?útt}?}?Cruiseüó_?\__ó?Grace

Hopperˉw(waowon?Y?)22?

w???O_N^Tú2?}?￥wNNNˉ~c?25%g?Armo???o?uo?^b??t?[}?2?

?×rwy?^???Arm?yo?óúw?ó?_?_{??O??NOíT??_t??÷?(?RISC-V)?ìé2?

u?ArmO??{ˉw???~ko×?_Y?O?y???y??_c?O?2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|212022~?o

?r_ty~yT}?g?×k?s?2022~?o?<r

2023~9o?r

_÷?yw^|_??10O]_ót\2

=ElevenLabs?ó??üLemble

AI????y????ny?2

÷Nn????????ké?r

_ty~yao???u\??o{??Uizard????ó]?}?2?

ElevenLabsóo??200OìY}w?~^T???2?}?·??n~O}w??t?Nn~O}w??N]2}w?No???10~?÷???2ElevenLabsg_÷U

[_

ü?zW?ók?Tg?ty~y?v1?1{^ü~o??}?2?

uty~yyw?ú?????}?Uizardè{?jó7o23???}?_?320O?S?ARR?~t?~gwu??\?T?13]2?}?4o??ARR?|?100O?S?3no??100O?STó300O?S2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|222022~?o

?r_ty~yT}?g?×k???_oGenAI??ó]?_???2

{?O?og?Synthesiar

??2017~??2020~???ty~y_Y???[_~2

???r

[??Wˉ??Wˉv??í??üO^{}??o2

t??ˉ~o<?=?Synthesiaó?44%?ˉ?100O^}?nüó]1^￥?1?oc1?o?yü?woó2

?2020~???g??ùoó??????960On??2stateof.ai20232020

data

starts

on

1

May.

2023

data

stops

on

1

Sept.RYù?ì^?|~?|?y|?o?s?#stateofai

|23OpenAI?ChatGPTo

ó]g??T???ONj?~????÷?RYù?ì^?|~?|?y|?o?s?#stateofai

|24OpenAI__??μy?êt&&O??o

?N?12noO_?OpenAI{?100??S

í?÷b

Z?￥w?otyr

??[-?2????}?k??wto2?_c?[O

???o?N??

RYù?ì^?|~?|?y|?o?s?#stateofai

|25×ChatGPT?yt?}2?_wó?Cheggk??CheggoN???b

N^?}??_ì?}??nünr

???ChatGPT????×r

Y[2

?}?2023~5oè{?<?~N[~?s?íor

ChatGPT?s??ˉwT??n~Y?s?k???ìY??2

=Tμ?Cheggg?n??~·?_^?????n??ChatGPT2???

Chegg????í?

40%2

2023~8o~ì?ˉ?u?_ˉN?

Cheggè{?<s????}??U{}ty~yg?}w~n?oó2

=T?kNScale

AI\_?t?????k?2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|26×ChatGPT?yt?{o

ONn??}???&&_ót\?Yk??Stack

Overfl

owty~yyoO_?_ó

?~T?{????óyh2

??u?ChatGPT?mì?y×m?y??ùk_ó

Stack

Overfl

owNó_ChatGPTr

??o2stateof.ai2023Left

figure

credit:Andre

RetterathRYù?ì^?|~?|?y|?o?s?#stateofai

|27??o

?GitHubCoPilot~×óú?_ót\?y_m÷??o}oì???3}N?_o(O

??3[?÷?)2

GitHubt????T??{?kCoPilot?·??y?}?2

?n??ouS?ì?yNˉ??g?2?

20

22~9o?GitHub?95]_^_ót\?ì?Nù???^T??V

rn

t?~?_T?}JavaScriptùNnHTTPoó~??[?÷?2?óó?~×??÷óú2?

2023~6o?GitHub?^?934533]CoPilot}w?uo2oˉ?o??wCopilot}w?é?yw??÷~×óúO_u

oO?????]?}w×tg(?÷óút32%)2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|28ChatGPTyóWù_???wvy?Nù?|~ocmì?ù?ChatGPTo??ù_?{{o??<o?_^ù_=2

|~èn?N?wtt??}ChatGPT?ù_

?r

?ó?÷??]?40%???-?óú?18%2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|29÷?O{n~?GenAI}?_·??~×?|_?s???r

í???}wN?vêy

~t?ìTú?yuˉ2

A16zoc?Character.AI{??1.5??S

?A??í???wT}{?O_?w??o????r

2?k2

????[}?o?g?44??????}_?yywv_?{o}?Os?_r

?W^üovN?sz

2

s???r

}w????~t??????^?}?ú?·o???×k?{t?w?w?~?YO??ìg

é2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|30?o

rt??k_?t?òw?u

\???o2022~?wStable

Diffusion?ó_??|g

?N~O^?Midjour

neyüStability?wO

í}????k?2

}??or

t?y÷N?T?c?OpenAI?oó_???~kg}??or

t?k?DALL-E

32

?o?Ideogram????u

??}??[?to?oImagen?_ó

44T??k?{{_tù2

No\÷?s?mì??or

??u??or

t?k???r

?g~×?oAdobe?Firefly1PhotoroomóDiscord2?

Midjourney?wu2022~3oMRR?oc?gwu???r100O?S???2023~^?r2??S2w}wu??200O\?T?ó1480O2|?ìo?o?Midjourney?rDiscordo?}w??Discordoó~Nrt?2oDiscordù??oo??3000Otwoó~N}ty~yT}{??[t???10?nt{?t?2?

_y??ww??y?_[}?Photoroomè{??w2o???r_ty~yoó??}???6noY?wuü}wu???N2Stability’sSDXLMidjourneyv5.2OpenAI’sDALL-E3Ideogramv0.1stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|31Oo

GenAI?êóm??rt_~k?O???}w}Og&NYouTube1Instagram1Tikt

okvWhatsApp{g×k??óoT}t??ChatGPT1RunwayvCharacter.ai{GenAIT}?W}[÷ü??}wu?w?O

2stateof.ai2023Figure

credit:Sequoia

CapitalRYù?ì^?|~?|?y|?o?s?#stateofai

|322022~?o

?N???}wr???NN?_[}??vW^?óyh?_óty~yk?(?OpenAIN)?}?T???y?N?ìˉ?2022~10o??Y??y[ú_ó?WShutterst

ock?_^NOpenAI\_?^DALL-E?ú???_u?ó2

w^2023~7o?n?}??~6~?????_ˉ??_ˉ^Q?OpenAI??Shutterst

ock?t?1??ü÷^???}?k?ˉ??tsS

uo2

o_?Shutterst

ock^~?wó?ty~yt?[_?Y2

?}??NMeta??GenAI?????_ˉ2

?yocGenAI??NShutterst

ock?t??kGetty

Imagesr

?n???^????GenAI??Nù?w2023~2o?Stability

AIó??zg

?g

??o????z

2vs.stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|332022~?o

?N???}wr???NN?_[}??vW^?óyh?_óty~yk?(?OpenAIN)?}?T???y?N?ìˉ?2023~7o?OpenAIü???r

???_ˉ?Q??V?????1985~?g???e?2

No\÷???^·?OpenAIaoü??_^÷????r

_T}2

}???ío?????k??T}{??O???{}ty~y??g??úW?O^ˉ?ü_2????|?2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|34??y?_[?ty~yr???O?\zg

???Y??O?^?o

Nns?\v}?Y?N???w{y?Yw?Nù?[??[?szg

????t_

ˉ?2

?wN?o?y??OY??Y?ó?y??~?2?

???o?T{{?w{y??T????·üò?

StephenThaler???s

20

12~?t?:????Oy;(ót)|?zg??2?

??zg\??ó?qˉ?è?ty~y?zgy?Y?~ó_???zgg??y????é??|ü?÷2?í???_úo?y??t_

?T}{???g?ty~y?}o2?

?ó?Wg??wszg?o?2023~5o?1981~?[??zgh?éóo???gúy?T}?Ny??1?o|???????N?r\v}O?<?W=2???y??k?ˉ?uo|?n}üúo??ì~y?N?wT?2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|35O[n?y??{???zg

?ht??????oüt?r

~{??htkó?ü???ì2

?w??}??ùT???}}v???u?Oo?íèn???y}_÷2?

GettyImagesó?ü????Stability?ù^

??yw?o]v?u~Oww??}vr??zg?o?~g?Stable

Diffusionr?t?~oGettyImages|s??}zo2?

OpenAIüMetak÷~????ùO\ozgn}?ChatGPTüLLaMa}?ˉ?uo?2o?ó?:?t÷?;k?×?OpenAIó????2O

]úo?k??Stability1DeviantArtüMidjourney??~O

?}?}T??_?gˉ?t?r~???[t<?gí_?=2?

ó?ozgy??oüuow?í]?O??tr?W^}??k?Ní]??R??e2o?_o?í]?Oty~yyhí???xk?ó?W??ó?}?ˉ?wk??zgwy?|?(??yaoNwoszg)?

????U

Copilotyw?}w??????ó?_zgY?}?^??_y?T?2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|36?or_}?wg

???üRLHFr

~??ü????k??Tˉyy?ó?ooó?}???Scale

AIüSurge

HQ?^????k??v?_mìo·????T?2

n?}?yoc?tsíy{??w]W??ty~y_[}?r

?O^?w?Yr

???k?|~????Y???2

Scale

AINNk|~73??S

o2021~?óStable

DiffusionüChatGPTòoO_2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|37ó???úytty~yO÷?_tty~ykìóó]Hugging

FaceoN??r

?7~?}??t_??rrso_tty~y??}wO2

?w{?t?b

oty????ty~yk?üuo???{?

Hugging

Face

kT?T?2

???no???1300nk??ó?r

H

ugging

Face?__???k?óìy?OpenLLM

Leaderboard???2023~8o}o??6?kk?O

?2

??k?_~}GradiovStreamlit{yw?t?tT}{?]{SpacesN?oc??????g

ü????v_2

Gradio?o?}wuT??5]??12O(2013~1o)T?r

58O(2013~8o)2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|38WN????k??o

???T}{??_^W???k??Databricks?13??S

??|w-?MosaicML?~?wt????yo????y????k?2

ogO

oNn?b

O

÷?WNu_k???o^?Nty

oO^uov{??óˉ??_^k?2?

w-O_?Mosaic]{??tsíy{?y{ao????_??O?5O?S(??g?]8])?ro?ˉStable

Diffusion????two?NO??t?SOTA

LLMs2?

??wor_ty~yòy??×óo?Nno?gY?÷{2?

Sn

owflake_o?u

?N??AzureN??U

?wó?OpenAIk????g?2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|39t???v?}????ty~ykyU

N?}??_?üo

?mRNA?×???BioNTech?5?ó???|w-?ty~y}?InstaDeep????óty~yN<ìN÷=?T[Nty~yNv?}?Exscientia?r

??|ú?6.74??S

???w???{?NVerge

Genomics?r

??|ú?8.4??S

??w2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|40ó_?????ùW?\___<tY=ó_??_URe

cursionPharmaceuticals֒5000O?S

?_y

?^

???_y

ùW?80%?^|Tò10??S

2

???Tèn?ty~yy???2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|41DeepMind[kYt???o

Google

DeepMind

2.0?ty~yY?}?DeepMindNGoogle

Brain\~^?ó]??or

_ty~y????_?22010201420152023RYù?ì^?|~?|?y|?o?s?#stateofai

|42~t?÷?_?c_?DeepSpeech

2?í?AI[^?<W~=20

15~?~t?}?ty~y??????Nn?y?r????ytn??÷?{??2?ùy_v_?ty??{??m|?ü???:<s?yy?s?os??HPCao?T}??t??_???óú?7]2_o×uo?N?ì?×o?÷?s????Nty_

??_t?wot??2=g?\N????20

17~??:???Nytno??o?;]{?<??=?í?o????oo?ws??yrü}?íkty~y2?[DeepSpeech

2?_ót\?r~?Y?~n}??[?tvCEO?r~??tküts????2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|43×]??:AttentionIs

All

You

Need;?Transformer???o~???????ì??yw?\???íuS

??S????Transformer?t?Y{ù_???b

o_?y?_??o?~t??y?[^}?2Transformer<?~=_??Capitalraisedin2023alone$10.3B$350M$270M$150M$100Mex-ex-stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|44?útt×rGenAIGAIA-1ouWayve~?útt_ó?o90?óu?r

_^}k?2

?{}??1?oüú_?ugr

??tto?~ó???s??ì~üo{??tt?v2

??ˉ??O_??s?vì~ü???o??????g

èó??tsíy{???y??wr

~Nn??k?~??}?ˉ?ü???úttk?2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|45?úttò{?|T??W^WWaymoüCruise?·×??}??T??yy

Y?úttoó2

?_??o_????ì÷?÷?T?^?oo?y2

??ˉùa???o?^ó_^?

Cruise?ttt?ò??\??

?o?útt?NnY÷{2ò?}q?^Y\_?{×oN?W{×o?g^Nn???{×3?u~÷?o·?2Waymo?gì?TekendraMawakanaè{????<o?ws???}W^?￥?k_?=2?

w???ttt???oót??s???ü{m??og?ío??2Waymo7o??\?T???ús???oó??wT}????Aurora?[^w_Y?~t???2?

_ArgoAI

??t[???útt_[}?

StackAV??}????·??

10

??S?

A

??í2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|46<GenAI=o

??<?=?{?ty~y֒t??2022~o

{???u

GenAI?ú2023~N[~?}ty~y[^}??÷í?NN2022N[~c&&??íoíoíuGenAI?u_ty~y÷í^??~O

?40%??b

o[^}??O

?÷~54%2

o?^u?ˉùauvmW???or?2

sì?ˉùa??}_÷?qqtech??T]<AI2023=]?·×PDFz2stateof.ai2023Data

as

of

29

Sept

2023RYù?ì^?|~?|?y|?o?s?#stateofai

|47O??|?}ty~y?ùo

üN^}??{O^?|2021~^?N^}??^|O

??OVON?Okb]??ùo^|?c{??^o?óO

?2

|?ìo?o?2023~o×n\500g

u50%?wtu<Oy}=?ú???1??1ó_?1Alphabet1Meta1{??üTt?2

??yoty~yò????ú?ü×t

2stateof.ai2023Data

as

of

29

Sept

2023RYù?ì^?|~?|?y|?o?s?#stateofai

|482023~???ty~y}?|w?yw70%?ùtío

?ú?2022~?55%???üó?ùtty~y}??÷í\?{????oty~y}??÷íO

??70%?N2stateof.ai2023Data

as

of

29

Sept

2023RYù?ì^?|~?|?y|?o?s?#stateofai

|49??AIt?}?u????c

?Y?o

?üó??w^?2022~_?????t?}u??292nTòr

315n?O^{?|?46??S

Tòó59??S

2

ó?Tò?3nt?}}??O?O^?|?2070??S

T_r

1550??S

2stateof.ai2023Data

as

of

19

Sept

2023RYù?ì^?|~?|?y|?o?s?#stateofai

|50O^?t1

??ùaü{÷?u

o

yw֒g[?ty~y{2010-23~t֒?ty~y{???2022-23~ty~y{r??ty~y_[}?s?w?~V?2022-23~ty~y{??w?stateof.ai2023Data

as

of

14

Sept

2023RYù?ì^?|~?|?y|?o?s?#stateofai

|51}?IPO2023~???O~-^???c

òN2022~?98?t????N???SPACN^?}???Arrival1Roadzen1Triller?O_?ío{[?}_^ú2

w??o?n?w-?MosaicML

+

Databricks(13??S

)?Casetext

+

ThomsonReuters(6.5??S

)???InstaDeep

+BioNTech

(5?oS

)2stateof.ai2023Data

as

of

19

Sept

2023RYù?ì^?|~?|?y|?o?s?#stateofai

|522023~24%?O^T?÷ííuty~y}?2023~?O^Y?^÷íYù?UGenAI2

??^??ty~y}??÷í\??]50%?\÷?cty~y÷í?{??2022~~290??S

?2023~~220??S

?2stateof.ai2023Data

as

of

19

Sept

2023RYù?ì^?|~?|?y|?o?s?#stateofai

|532023~GenAI?í_TòGenAI?r

_ty~y?}?|_??ío2$950M(2010-18combined)stateof.ai2023Data

as

of

2

Oct2023RYù?ì^?|~?|?y|?o?s?#stateofai

|543?GenAI

ík??2023~}÷í?180??S???kso}g?u<?x=v<_?=k_}??ú???}???g

c?S

??-~???y?gˉ?ík??2

?y???ygt_ob

?|?CoreWeave{??23??S

?zó?í??O

o?g

?g-~GPU2stateof.ai2023Data

as

of

2

Oct2023RYù?ì^?|~?|?y|?o?s?#stateofai

|552X0X2X2~?o

?ó_?NN?AGIt?t?zus?ó_?~ío?t??Nys???oty~y???n[??O

?z

u?W??a)÷íùtü}qty~y_Y}??b)Y__^?GPU?ó?W?c)Tò??ì^×?^2SelectinvestmentsGPUcloudprovidersIndustryverticalsRecursion(drugdiscovery)CoreWeaveBioNeMo:GenAIcloudserviceindrugdiscovery.Synthesia(videogeneration)Cohere(LLMs)LambdaPicasso:GenAIcloudserviceforvisualdesign.Adept(processautomation)Omniverse:digitaltwinsoftheworld.stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|56N?g_tìt?ty~y{íúo

?]u??O^]?x?wOBeastround$10BMonsterroundupto$4BMegaround$1.3BSeriesC$270MSeriesD$235MSeriesC$141MRYù?ì^?|~?|?y|?o?s?#stateofai

|57GenAI}?2023~{??33%??y[?o

bo

_[}??130%]_y^}?ìo?y?oty~yˉN÷???üto~O

??2?sì?ˉùa??}_÷?qqtech??T]<AI2023=?]?·×o?^PDFz2?stateof.ai2023Data

as

of

19

Sept

2023RYù?ì^?|~?|?y|?o?s?#stateofai

|58?ú?|~stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|59

3

GPT-4jz?^??-bo

????k???o

?[tGPT-4oOpenAIg?????k?2

N??o?GPT-3?w^?zot??GPT-4o[ka

???k????×?oˉ??_?×t?ˉ????wT?yO_???????t?r

?o2

?ó_÷NO

??t~8192ntoken??uíky÷?????O_g}?GPT-3.52

_w??}RLHFˉ?2

Y_???Y?ao?GPT-4ojóo?^ó_÷????ˉ?gw?}y??ty~yk?2?OpenAIO??y??w??]v?×N??^í?t??????^??1GRE1Leetcode?N?GPT-4?ì?y÷?2?

GPT-4og}??÷2??ó?N?GPT-3.5?y?ó??ó????N?^???GPT-4?V

uo90%??GPT-3.5?V

uo10%2[u?óo?Tò??étt?o?_?Y?O??wTto??~?2?

OpenAI?^ù?}?GPT-4?w×ré??r?O?÷g?uo?N?~?{_ty~yk??r????O_g}?ChatGPTk??k~÷ú?40%2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|60

3

ChatGPTr???úO?t??Wn?RLHF?r~MVP?~??yú??w100y??s???InstructGPTo}?t??Wn?RLHF??_~?OpenAI?k???y??}w?o~?2

}?oN?_???ChatGPT?r

??n??ùaoíkN??ìg

2l

<RLHFí?t????u???k????ìó??}??ó?gnt_}?_ók??w^^w_~_ó?÷g}Wng?u??k?2=wó?_????r2017~?_÷OpenAIüDeepMind?|~t\^wT}?Atari~o?ˉ??vüwTWnT}o2l

RLHFóogY?????k?r??x??|wo3?~êyT}{???????k?2woW?Anthropic?Claude1

?o?Bard1

Meta?LLaMa-2-chat?_w?oOpenAI?ChatGPT2l

RLHF?n×ot?k????ì?üó]?w^?T??_}?ìtk2???yao??1

f?^o_é2??|~t\?~???2TypicalstepsofRLHF,whichfollowaninitialstep

ofsupervisedfine-tuningofapre-trainedlanguagemodel,e.g.GPT-3.stateof.ai20231

We

willcover

other

issuesof

RLHF

inthe

Safety

section.RYù?ì^?|~?|?y|?o?s?#stateofai

|61

3

k?_o

???k??TW??v

RLHF?_?wo

??o[{?|~t\èn?y

o?1?oy?????k?????_????k??ìt~?u?_??k??_N?tsíy{?O??_?O

×~??o2?

|~t\è??N?WO\_??ˉ????k??~yoO\u??uo?ì??ˉ?2??èn?t??k?íkO?}?[?k?uo?×N_????-?2??g???k?×t?}k?uo2?

??}k?__~-???è?_

ˉ~T??[wsì?}??ˉ???Oo??[?k?uo?ìt~?u2?

OO

?^g?RLHFN_??[2??Tt?í?|~?Meta|~t\T??LLaMa-2??o????:<s?W????k??S?ù_y??k?÷??óo??tì?

bèó?3??yoNouRLHF?ú?2=stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|62

3

s??o?|~t\??t?~??RLHF??k]??yhChatGPTO^??[???_?T?Nn???s?yv[t?OpenAI????k?Nü?y?k???^y__?]t?÷w??

Anthropic?ty~y??oó??Wn?s?^?yú???2?

:??o-?ü?u?;?LIMA:LessIsM

oreforAlignment??Meta}N????o?1000n?t?R

?ó{üT2yo?k????ty??LIMA43%???ONGPT-4wo\{t??2?

:???k????sóú;?

LLMscanSelf-improve

?N?o??o?|~t\èn????k?????ˉ??y???góú2w??sg?oNnn÷??nn÷o?k?r?y?g?1

?uü??o?~????ì?v??uwóu2?y÷?÷Nùy_oMeta?:g?T???si×;?

Self-Alignme

nt

withInstructionBacktranslation.?2?

?w??|~t\}g^Nyyy?}GPT-3.5rg?ü???~??Meta?LLaMa-7B2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|63

3

GPT-4ao?^?SOT

A

LLM|~??}N?&&OpenAIóè?N?s?GPT-4?ao?^?wowío?·?_tsˉ????o??o?wty~y|~?gt?^W2

?o?PaLM-2ao?^_?om?????OpenAIt??g??Anthropicío~wClaudek?ó_ao?^2

o?^u?ˉùauvmW???or?2

sì?ˉùa??}_÷?qqtech??T]<AI2023=]?·×PDFz2?

OpenAIarXivNó_?GPT-4ao?^oùó?<?×rt?|\üGPT-4{íkk???yY?o?^íoWs?÷??W?k?_?1|t1ˉ???1uo??t1ˉ?yyv????Not?2=?

_?oó_wg?PaLM2÷??}?ao?^oùó?<k?_ü÷???Not?O_?_}_2=?

?w?oT?ü?y???g?ú(o????t??N)?o?N_?}????×??wg_?|~?O?n?W2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|64

3

…??LLaMa?o??N??2023~2o?Metaó_?N?W]~LLaMa?k?2

ó_÷???±????r

~_y}_?}?uo?Nˉ??y?g?k?2

Metag_?Q?|~t\k???LLaMa

?O???ì·~N}_2?

LLaMa-1?÷}~íTransformer?÷?u

o?W2_

??_W~üìo?ù??óZ?N??}2?o?<_ˉ?Nn65Bóuk?÷?[???]?y2048

A100

GPUü80GB

RAMN]vt380ntokens/?/GPU2?owwW1.4Ttokens?[???]uo?N?ìˉ???t21y

2

=?

LLaMa-1k??gy_?GPT-3????÷??OoInstructGPT?y??~NDeepMind?Chinchillaü?o?PaLMow\?t??2?

LLaMa-1OQ?W^}?_ó?_}?Metak?ó_÷}<_t=N??o?{?2Oo?

???LLaMa-2?ú?[u_t{2stateof.ai2023RYù?ì^?|~?|?y|?o?s?#stateofai

|65

3

LLaMa_ó?N???k??_t?Metaó_?LLaMa-1O^?wT?_òu??t?????k??gY??ú2woN?±?????MosaicML?MPT-30B?

TII

UAE?Falcon

40B

?

Together?RedPajama

?

Eleuther?Pythia2No\÷?÷Nnúakó?_t{_y?uo?N?u?LLaMa?g_zo?~^wT}?uSnO~T}{?2MistralAI?7Bk?g?_r~g?_?k?2?

|?ìo?o?RedPajama?toot~]vLLaMa-1?w?y_t2Falcon

40Bg?