Next-generationCo-Packaged

OpticsforFutureDisaggregatedAISystems

SajjadMoazeni

AssistantProfessor

UniversityofWashington,Seattle

1|

2|

Co-packagedOptics(CPO)

[Broadcom]

Large-scaledata-center

networkingandswitches

&

Riseofdata-intensiveAI/ML

applications

Demandssignificantly

largeroff-packageI/O

bandwidths!

3|

Hyper-scaleData-centers

?ElectricalLinks:Energy/BWdensityscalabilityissues

?Futurerequirements:multi-Tb/s/mm&Sub-5pJ/b

[Broadcom]

SiPhPlatformfor

OpticalI/O

Co-packaged

PICwithDSP

Switchwith

OpticalI/O

202120232025

Co-packagedOpticsfor>51.2Tb/s

4|

DemandedBandwidthsbyAI/ML

AI/MLDemand

DoublingEvery

3.5Month!

Moore’sLaw

[OpenAI]

5|

DemandedBandwidthsbyAI/ML

100x8xH100

[NvidiaDGXSuperpod]

?+10Tb/soff-packagebandwidthswillbesoonrequiredforGPUs

?OpticallyconnectedGPU&NVSwitchesistheonlyviablesolution

?Scalingoutwillbealsoeasierwithopticalinterconnects

6|

DisaggregatedComputeSystems

?Data-center:

–Improvedutilizationanddynamic

resourceallocations

?AI/MLWorkloads:

–DistributedTrainingandParallelism

–NVMe-over-fabric,CXL,…

–DPU,etc.

–Co-packagedOptics(CPO)

[]

7|

Goalsfor

Optics(CPO)

Co-packaged

[NvidiaGTC2022]

[AMDISSCC2023]

?Silicon

Photonics

[NvidiaGTCDec2020]

?Micro-ringresonator(MRM)basedopticaltransceivers(TRx)

[DallyOFC2022]

?Wavelengthdivisionmultiplexing(WDM)

8|

Micro-ringModulator(MRM)

WavelengthWavelength

?Resonancewavelength:λ0=neffL/m,m=1,2,3,...

–Q-factor:Q=λ0/Δλ

?Compactdevice(radiusof5μm)

–Energy&areaefficientmodulator/filter,Onlya20fFload!

?Supportingwavelengthdivisionmultiplexing(WDM)

9

9|

MRMbasedOpticalLinks

?ModulationScheme:

1.Deplete/InjectcarriersusingPNjunctions

10

10|

MRMbasedOpticalLinks

?ModulationScheme:

1.Deplete/InjectcarriersusingPNjunctions

2.ΔfreecarriersΔindexofrefraction[Carrier-PlasmaEffect]

11

11|

MRMbasedOpticalLinks

?ModulationScheme:

1.Deplete/InjectcarriersusingPNjunctions

2.ΔfreecarriersΔindexofrefraction[Carrier-PlasmaEffect]

3.On-OffKeying(OOK)modulation

12

12|

MRMbasedOpticalLinks

?ModulationScheme:

1.Deplete/InjectcarriersusingPNjunctions

2.ΔfreecarriersΔindexofrefraction[Carrier-PlasmaEffect]

3.On-OffKeying(OOK)modulation

*.OMA:OpticalModulationAmplitude

13

13|

MRMbasedOpticalLinks

?ModulationScheme:

1.Deplete/InjectcarriersusingPNjunctions

2.ΔfreecarriersΔindexofrefraction[Carrier-PlasmaEffect]

3.On-OffKeying(OOK)modulation

*.OMA:OpticalModulationAmplitude

14|

Electronic-PhotonicIntegration

?Electronic-PhotonicIntegration

–Monolithic(LowParasitic,OldCMOS)vs.

2.5D/3D(LargeParasitic,AdvancedCMOS)

45nmSOICMOS+Photonics

?Laserintegration:off-packageor

integratedwithsiliconphotonics

[AMD,ISSCC2023]

15|

40Gb/sPAM-4TxinGF45nm

[Moazenietal.,ISSCC/JSSC17]

16|

State-of-the-ArtofCPO

Ranovus+AMD

?TotalBWperfiber:1Tb

?+10pJ/bfor100Gb/sperwavelength

?IntegratedLasers

?0.5Tb/s/mm

AyarLabs+Intel

?TotalBWperfiber:800Gb

?+5pj/bfor20Gb/sperwavelength

?ExternalLaser

?0.5Tb/s/mm

17|

MajorChallenges

?Electronic-PhotonicIntegration

–Monolithicvs.2.5D/3D

–Interconnectparasitics>3xDevicecap

?Energy-efficiency

–Need5-10ximprovement

–Electronic-photonicCo-design

?FiberPackaging

?DWDMLaserSources

[DallyOFC2022]

100Gb/sPAM-4Rx(3.9pJ/b)

[Lietal.,ISSCC21]

40Gb/sPAM-4Tx(0.5pJ/b)[Moazenietal.,ISSCC17]

18|

SolutionsforNext-generationCPO

?Advancedpackagingwithdirectbondinginsteadofmicro-bumps

?Electronic-photonicCo-design(e.g.,OpticalDAC)

Silicon-InterconnectFabric(SiF),UCLA2021

19|

BeyondJustanE/OBridge

[MoazeniOFC2018]

NewarchitectureswillbeunlockedwithCPO…

?Network-level:

–Micro-secondopticalcircuitswitchingnetworks

?Package-level:

–Co-processingontheCPO

–HBMmemoryaccess&controller

20|

CPOswithMemoryController

?BypassingGPUformemoryaccess

–AddCXLmemorycontroller

?ImprovingGPUDirectlatency

UltimateCPO-enabledArchitecture:

[AngLi,PNNL2019]

21|

Conclusion

?Co-packagedOpticscanprovidetheneedsofnextgenerationofGPU/Acceleratorinterconnects

?Next-generationCPOdemands