NiDDAl+DevelopmentDigitalSummit上海站Al+-研發(fā)數字峰會擁抱AI重塑研發(fā)基于視頻生成模型Hallo和Champ的朱思語|復旦大學角色動畫創(chuàng)作者克隆學hedraDreaminaAI魯LEMONSLICEpuppetryAl虛擬主播IHallwayHYPER◆ArgilcoptionsD-IDEDelphi藤hedraMey直播克隆或換臉MeyGenMagicam廣告和營銷CCBeyondPresenceOColossyanANAMCICEROarcadsDcreatifyHeyGeniconYReelFarmcoptionsCreatorKitJOGGMUKUAITikTokVEEDelai.MeyGenIMMERSIVEFOXsynthesiaOneVEEDvidyardYuzuLabs開發(fā)者工具20252025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)20252025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)角色動畫角色動畫hedraDreaminaAI魯LEMONSLICEpuppetryAl虛擬主播lIHallwayHYPER◆aA2EAKOOL20252025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)角色動畫hedraDreaminaAlLEMONSLICEpuppetryRrunwaysync.Al虛擬主播aIA2E8AKOOLANAMaIA2EAKOOLANAMArgilcoptionsD-IDesence2025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)facialanimationmodelsGenerationCodeTalkerOmniHuman20212022YouSaidwav2lipAD-NeRFDiffTalkThat?FaceFormerVASAX2Face模型架構圖例：2025rpe(x|z)XZqφ(z|x)Aa人工智能機器學習深度學習生成式AIo以替代或超越人類智能為目標的計算機智能。機器學習(1990年)o人工智能的一個子領域，通過分析大量數據以識別模式，使計算機系統(tǒng)能自動獲取知識并不斷改進，從而實現準確決策或預測。o深度學習(2012年)o機器學習的一個分支，利用多層神經網絡對數據進行處理和決策。o生成式AI、大模型(2021年)o基于特定提示、數據或條件，生成新的文本、圖像、音頻、視頻2025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)Xox1X2Xox12TDD8Z8ZhAAAAr2BaLayornom·Canvideogenerationrestorethe3Dphysicalworld?2025R25研級數字峰會擁抱Al重塑研發(fā)14·Hardtoreallymodelthephysicalword.Failurecaseinappearanceandgeometry.·Hardtoreallymodelthephysicalword!Failurecaseinmotion.20252025R25研級數字峰會擁抱Al重塑研發(fā)16·Controllablevideoge2025R25研級數字峰會|擁抱Al重塑研發(fā)17·MCMat:Multiview-ConsistentandPhysicallyAccuratePBRMaterialGeneration.AwoodenbeermugMaterialBranchesMaterialBrancheswithametalhandle×jAlbedoDiTBlockBlockRoughnessMetallicUVUnwarpingUVWarpingObjectNormalDiTBlockDiTBlockRefinedMeshRefinedMaterialCoarseMeshCoarseMaterial2025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)2025R25研級數字峰會|擁抱Al重塑研發(fā)19田田Predictz?今今△FFR食t=t?t=T△FFR食t=t?t=TSpatialAtn≤AbeautifulintricatrandomsampleStage2:FinetuningrandomsampleStage2:FinetuningReconstructionG-ObjaverseRenderingsFFFRpe(Z?|pe(Z?|zt-k)zrq(z-12%)q(z-12%)Score-DistillationSampling2t-k-1)Stage3:3D-AwareDenoiseStage3:3D-AwareDenoise2025A25研發(fā)數字峰會|擁抱Al重塑研發(fā)20·VideoMV:ConsistentMulti-ViewGenerationBasedonLargeVideoGenerativeModel.JackSparrowwearingsunglasseshead,photorealistic8KHDhair,headphotorealistic8KHDDetailedportraitofLaraCroftfromTombRaiderhighlightingTheJokerfromGothamonhisfacecapturedinstunningphotorealisticdetail4KHDAcharmingchibistylerenderingofElsafromFrozenin8KresoluAchimpanzeedressedlikeafootballplayerstunningwearingChinesecoat4KHDAbeardressedinAsquirreldressedlikeHenryVIIIkingofEnglandAMinionwearingtheclothsofSpiderman4KHDAraccoonastronaut2025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)·DirectlyGenerateDynamic3D?Nottomention4Dgeneration...Stage2:4Stage2:4DGaussianFieldFittingVideo-to-4DStage1:PseudoSequencesGenerationDeformed4DGaussianInputSequencesDeformed4DGaussianInputSequencesAnchorMulti-viewSequencesDeformationPlanes-Opti.Init.-→SequencesGenerationDensificationDensificationFrameFrameNoise一-MV-SDSLoss&Ref.PhotometricLossTextText-to-4DText-to-ImageGeneratedVideoSynthesized4DSequence((SDXL)4DGenerationVideoDiffusionstyle.”8“Apandaisreadingabookincartoon2025R25研級數字峰會|擁抱Al重塑研發(fā)23為什么難以建模三維物理世界?NiDD6th·Reason#1:Latentspacenotfullysemantic.·Reason#2:Limitedconditions;multi-modalinformationalignmentishard.2025R25研級數字峰會擁抱Al重塑研發(fā)24控制條件多模態(tài)融合難·更好的3D物理控制條件!更好的3D物理條件對齊!thoughtthoughtVideoCoTMotion&Animation·[ECCV2024]Champ:ControllableandConsistentHumanImageAnimationwith3DParametricGuidanceReferenceImageReferenceNetParametrieShapeAlignmentNormalDepthNormalSkeletonSemanticSkeletonPoseSequenceNoise×FMulti-LayerMotionFusion(MLMF)VideoIterativelyDenoisingSpatial-Attention[Cross-AttentionTemporal-AttentionParametrieShapeAlignmentSMPLSMPLSMPLSMPLReferenceImage2025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)2025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)·[ECCV2024]Champ:ControllableandConsistentHumanImageAnimationwith3DParametricGuidanceChamp:ControllableandConsistentHumanImageAnimationwith3DParametricGuidancefudan-generative-vision.github.io/cha..Activity曰Customproperties☆4.2kstars◎216watching望483forksReportrepositoryHallo:通過參數化控制實現數字人臉生成·Hallo:HierarchicalAudio-DrivenVisualSynthesisforPortraitImageAnimationReferenceImageNoiseDrivingAudioPCross-AttnExpression?-LipReferenceNetSpatial-AttentionReferenceNetCross-AttentionAudio-AttentionTemporal-AttentionHadamardProductConcatenationVideoDenoisingTemporalAlignmentUnetAdapter→LastNframesTemporalSelf-AttnFframesnoise2025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)Hallo:通過參數化控制實現數字人臉生成NiDDPortraitAnimationsofDifferentAudioStylesHallo:HierarchicalAudio-DrivenVisualSynthesisforPortraitImageAnimation②fudan-generative-vision.github.io/hallo/Activity曰Customproperties☆8.1kstars81.1kforksReportrepositoryPatchDropDataAugmentationPatchDropDataAugmentationPatchifyDrop,Sampling-GausianNoiseTextEncoder→□□…□一"Themanhasneutralexpression"LNDataAugmentationFaceneutralexpression"LSpatial-Attention□Audio-Attention□Cross-Attention□Textual-Control□Temporal-AttentionConcatenationTextEncoderAudioDenoisingNoise"Themanhas2025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)·[ICLR2025]Hallo2:Long-DurationandHigh-ResolutionAudio-drivenPortraitImageAnimationHallo3:DiT架構+高真實和動態(tài)數字人·[CVPR2025]Hallo3:HighlyDynamicandRealisticPortraitImageAnimationwithVisionDiffusionTransformerReferenceImageT5品NoiseCtextCZrefFaceEncoderCIdentityReferenceNetworkAudioWav2VecMLPCaudioCtextCtextMotionFramesVideo20252025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)JiahaoCuilHuiLi'YunZhan!HanlinShangKaihuiChengYuqiMaShanMu1HangZhou2JingdongWang2SiyuZhu!2BaiduInc.通用視頻控制：參數化軌跡控制·[CVPR2025]Tora:Trajectory-orientedDiffusionTransformerforVideoGenerationgracefullyabovealush,colorfulc+不++個++個Noisepatch2025Al+-研發(fā)數字峰會|擁抱AI重塑研發(fā)·[CVPR2025]Tora:Trajectory-orientedDiffusionTransformerforVideoGenerationUsingtheofficialpromptsofCogVideoX·[CVPR2025]OpenHumanVid:ALarge-ScaleHigh-QualityDatasetforEnhancingHuman-CentricVideoGeneration|Thevideofeaturestwoindividualsinahistoricalsetting.likelyaperioddrama.Thefirstperson…,Thesecondperson,ontheight,isawomanwithblonde,styledhair.wearing...Thebackgroundisnichlydecorated.with_…ThevideofeaturesapersondressedinaformalpoliceiThevideofeaturesapersondressedinaformalpoliceiuniform,includingapeakedcap,walkingacrossagrassyiarea...Theperson'smovementsaresteadyandpurposeful.iandtheyseemtobeengagedinaconversationorinteractioniwithanotherindividualwhoispartiallyvisibleintheiforeground...Thesettingisoutdoors.likelyinaparkorigarden,duringdaylightwithamplesunlightfilteringthroughithetrees.Theoverallatmosphereiscalmandorderly.OpenHumanVidAsinglepoliceofficer,amiddle-agedmanwithdarkhairandaformaluniform,walkssteadilyacrossasereneoutdoorstting.likelyaparkorgarden,withtrcesandfloweringplantsinthebackground.Heisengagedinacoversationwithanotherindividual,partallyvisibleintheforeground.whoholdsabeigebag,amidstacalmandorderlyatmosphere.DatasetnameYearDomain#VideosCaptiontypeMotiontypeResolution2021Open52KShortText360P2024Open70MShortText720POpenVid-1M[36]2024OpenLongText512P2024Open36MLongText720P2012Humanaction26.7ShortText240PNTURGB+D[45]2014Humanaction3Dpose,depthMSP-Avatar[43]2015Humanaction3Speechaudio,landmark,poseAct