再談交互分析平臺(tái)在運(yùn)營商中的應(yīng)用目錄現(xiàn)場(chǎng)疑問解答技術(shù)發(fā)展概述01020304典型行業(yè)應(yīng)用具體建設(shè)內(nèi)容大數(shù)據(jù)技術(shù)架構(gòu)TraditionalAssumptionsANewApproachBenefitsandValueBare-metalContainersandVMsBig-Data-as-a-ServiceDatalocalityHDFSonlocaldisksComputeandstorageseparationSharedstorageAgilityandcostsavingFastertime-to-insights20112012201420142015Source:ChrisHarrold@EMC,SharedInfrastructureforBigData:SeparatingHadoopComputeandStorage,2015.12大數(shù)據(jù)技術(shù)趨勢(shì)Source:@Tableau,TOP10Trendsfor2017BigData,2016.12BigdatabecomesfastandapproachableBigdatanolongerjustHadoopOrganizationsleveragedatalakesfromtheget-gotodrivevalueArchitecturesmaturetorejectone-size-fitsallframeworksVariety,notvolumeorvelocity,drivesbig-datainvestmentsSparkandmachinelearninglightupbigdataConvergenceofIoT,cloud,andbigdataSelf-servicedataprepbecomesmainstreamasendusersbegintoshapebigdataBigdatagrowsup:HadoopaddstoenterprisestandardsRiseofmetadatacatalogshelpspeoplefindanalysis-worthybigdata目錄現(xiàn)場(chǎng)疑問解答技術(shù)發(fā)展概述01020304典型行業(yè)應(yīng)用具體建設(shè)內(nèi)容大數(shù)據(jù)分析平臺(tái)1.0UniversalDatabase

(UDB)復(fù)雜流程中眾多業(yè)務(wù)系統(tǒng)產(chǎn)生的海量數(shù)據(jù)開發(fā)人員對(duì)日志格式的花樣需求眾多數(shù)據(jù)來源和不同的數(shù)據(jù)格式智能化運(yùn)維改善用戶體驗(yàn)方便故障排除、異常分析處理追蹤和評(píng)估收視效果支持各類報(bào)告、告警CallerIDMetadataDistributionSTB

MenusMenu

Entitlement數(shù)據(jù)輸入結(jié)果輸出Source:SplunkInc.,2012大數(shù)據(jù)分析平臺(tái)1.1CollaborativeFilteringUser-basedCollaborativeFilteringItem-basedCollaborativeFilteringModelBasedCFAlgorithmsProbabilistic(e.g.BayesianNetwork)ClusteringRule-basedapproaches(e.g.AssociationRules)ClassificationRegressionLDAContent-basedRecommendationsModelBasedAlgorithmsNovelMethodsLearningtoRankContext-awareRecommendationsTensorFactorizationFactorizationMachinesDeepLearningSimilaritySocialRecommendationsSource:XavierAmatriain@Netflix,CollaborativeFilteringandotherapproaches,MLSS2014大數(shù)據(jù)分析平臺(tái)1.2Source:AndrewLeamon&ChushiRen@Comcast,MachineLearningatComcast,2015.11大數(shù)據(jù)分析平臺(tái)1.3Source:ShekharAgrawal@Comcast,UsingSparkMLtoPoweraDSaaS(DataScienceasaService),2017.02PerpetualSparkEngineRESTfulAPItocontrolallaspectsConnectorstoCassandra,Hbase,MongoDB,Teradata,MySQL,Hive,ES,etcKafka,StormforstreamingdataORC,Parquet,textfilesRolebasedcontrolonwhoseeswhatIntegrationwithmodelingusingPython,R,SAS,SparkML,H2O大數(shù)據(jù)分析平臺(tái)1.3Source:ShekharAgrawal@Comcast,UsingSparkMLtoPoweraDSaaS(DataScienceasaService),2017.02目錄現(xiàn)場(chǎng)疑問解答技術(shù)發(fā)展概述01020304典型行業(yè)應(yīng)用具體建設(shè)內(nèi)容大數(shù)據(jù)應(yīng)用建設(shè)平臺(tái)工具面向應(yīng)用開發(fā)提供易用的大數(shù)據(jù)平臺(tái)開發(fā)工具自助建模面向業(yè)務(wù)人員提供靈活的自助建模工具和模式數(shù)據(jù)服務(wù)面向數(shù)據(jù)使用者提供數(shù)據(jù)支撐提高大數(shù)據(jù)匯聚后的數(shù)據(jù)服務(wù)能力，挖掘多類型數(shù)據(jù)的融合價(jià)值算法模型大數(shù)據(jù)數(shù)據(jù)服務(wù)面向領(lǐng)導(dǎo)數(shù)據(jù)指數(shù)：多維數(shù)據(jù)統(tǒng)計(jì)，呈現(xiàn)業(yè)務(wù)相關(guān)態(tài)勢(shì)數(shù)據(jù)運(yùn)維：呈現(xiàn)大數(shù)據(jù)中心的數(shù)據(jù)狀態(tài)面向業(yè)務(wù)數(shù)據(jù)融合：以人為中心，提供全方位的數(shù)據(jù)鏈數(shù)據(jù)挖掘：多維度數(shù)據(jù)特征挖掘和模型分析數(shù)立方建設(shè)完善全面的數(shù)據(jù)服務(wù)能力數(shù)據(jù)匯聚地?cái)?shù)據(jù)加工廠數(shù)據(jù)指數(shù)多維統(tǒng)計(jì)收視行為，如人均收視時(shí)長、觀看次數(shù)、忠誠度、收視份額等支付行為，如支付金額、支付次數(shù)、支付人數(shù)等消費(fèi)行為，如消費(fèi)金額、ARPU等工單行為，如工單類型、故障類型等融合統(tǒng)計(jì)跨業(yè)務(wù)數(shù)據(jù)融合統(tǒng)計(jì)例如周末出游且喜歡汽車的人群品牌推薦（收視、消費(fèi)）例如微信支付群體的ARPU（支付、消費(fèi)）例如設(shè)備維修后故障消除的成功率（工單、網(wǎng)維）數(shù)據(jù)運(yùn)維數(shù)據(jù)分布平臺(tái)中的分布應(yīng)用中的分布存儲(chǔ)單元分布數(shù)據(jù)熱點(diǎn)熱點(diǎn)應(yīng)用熱點(diǎn)單元活躍應(yīng)用數(shù)據(jù)轉(zhuǎn)換應(yīng)用被訪問應(yīng)用去訪問單元被訪問歷史趨勢(shì)容量變化趨勢(shì)熱度變化趨勢(shì)轉(zhuǎn)換變化趨勢(shì)數(shù)據(jù)融合數(shù)據(jù)服務(wù)吃住行消樂全景畫像多源數(shù)據(jù)串并數(shù)據(jù)鏈人的多維無限層關(guān)系拓?fù)涫找?支付/消費(fèi)/工單刻畫觀看/點(diǎn)播隱形關(guān)聯(lián)軌跡。。。多源數(shù)據(jù)空間軌跡串并人的行為時(shí)間數(shù)據(jù)串并群體關(guān)聯(lián)關(guān)系數(shù)據(jù)串并。。。數(shù)據(jù)挖掘收視特征挖掘多維向量