1、海量數(shù)據(jù)存儲(chǔ)解決方案建立數(shù)據(jù)湖從海量數(shù)據(jù)談起到2020年,互聯(lián)網(wǎng)將連接76億人2000億設(shè)備數(shù)據(jù)量每?jī)赡陼?huì)增長(zhǎng)一倍目前: 20 Zettabytes2020: 4050 Zettabytes1 ZB = 10億 TBs44 ZB 大概是地球上所有沙灘的沙粒數(shù)字總合的50倍數(shù)據(jù)來(lái)源：IDC3201571 EB全球發(fā)售的總?cè)萘糠墙Y(jié)構(gòu)化數(shù)據(jù)所占的比例75%78%80%2016106 EB2017133 EB非結(jié)構(gòu)化數(shù)據(jù)的占比海量數(shù)據(jù)的組成傳統(tǒng)數(shù)據(jù)源結(jié)構(gòu)化數(shù)據(jù)非結(jié)構(gòu)化數(shù)據(jù)Dark Data公共記錄位置信息社交媒體聯(lián)網(wǎng)設(shè)備新興數(shù)據(jù)源需要處理任何不能用傳統(tǒng)系統(tǒng)處理的數(shù)據(jù)集數(shù)據(jù)湖與數(shù)據(jù)倉(cāng)庫(kù)湖泊的無(wú)限可能用于

2、分析各種來(lái)源、原始格式的數(shù) 據(jù)的企業(yè)數(shù)據(jù)管理平臺(tái)將數(shù)據(jù)以原始格式遷移到數(shù)據(jù)湖中數(shù)據(jù)被放進(jìn)數(shù)據(jù)湖后，可以方便的 利用工具直接對(duì)這些數(shù)據(jù)進(jìn)行分析- Gartner數(shù)據(jù)湖的解讀數(shù)據(jù)永遠(yuǎn)處于數(shù)據(jù)湖中生命周期的各個(gè)階段都在數(shù)據(jù)湖中完 成部分?jǐn)?shù)據(jù)有相對(duì)的生命終點(diǎn)歸檔絕大部分?jǐn)?shù)據(jù)都在數(shù)據(jù)湖中周而復(fù)始的生存、演化數(shù)據(jù)湖中的數(shù)據(jù)生命周期精準(zhǔn)農(nóng)業(yè)提高產(chǎn)量智能衣服知道人體的感覺(jué)隱性眼鏡控制血糖溫控器知道你出門(mén)時(shí)間運(yùn)動(dòng)手環(huán) 了解你的運(yùn)動(dòng)量智能眼睛 引導(dǎo)路徑無(wú)人飛機(jī)傳送貨物尋找海量數(shù)據(jù)中的價(jià)值產(chǎn)品性能提升商品推薦客戶(hù)分析供應(yīng)鏈優(yōu)化風(fēng)險(xiǎn)預(yù)測(cè)?需求場(chǎng)景, 分析工具, 存儲(chǔ)/數(shù)據(jù)構(gòu)建數(shù)據(jù)湖的阻礙SourcesSourceHa

3、doop大容量/快速增長(zhǎng)多種數(shù)據(jù)源低效浪費(fèi)數(shù)據(jù)安全海量數(shù)據(jù)存在海量問(wèn)題SourcesSourcesSourceHadoop共享存儲(chǔ)多協(xié)議支持高可靠性橫向擴(kuò)展能力Shared StorageData LakeDELLEMC數(shù)據(jù)湖SMB, NFS, HTTP, FTP, HDFS, S3,Swift, Restnodeinfo nodeinfo nodeinfo nodeinfoMAPReduce MAPReduce MAPReduce MAPReducedata nodeIsilon/ECSname nodename nodename nodename node決策支持?jǐn)?shù)據(jù)庫(kù)網(wǎng)絡(luò)數(shù)據(jù)OLAP數(shù)據(jù)

4、倉(cāng)庫(kù)Step 1:大部分?jǐn)?shù)據(jù)直接存入Hadoop集群Step 2: 運(yùn)行命令Hadoop ClusterIsilon/ECS 數(shù)據(jù)湖基礎(chǔ)Isilon/ECS組合DATA CENTERUSERSSmartPools Policy30 days高性能30 days-2years2 years大容量CloudCloud數(shù)據(jù)自動(dòng)流動(dòng)GEO數(shù)據(jù)湖國(guó)際知名汽車(chē)技術(shù)產(chǎn)品供應(yīng)商汽車(chē)與智能交通，智能駕駛各類(lèi)型測(cè)試的影像、雷達(dá)、傳感器、日志等數(shù)據(jù)客戶(hù)遇到的挑戰(zhàn)數(shù)據(jù)增長(zhǎng)過(guò)于迅速2015/30PB多個(gè)團(tuán)隊(duì)多個(gè)應(yīng)用需要處理相同的數(shù)據(jù)集效率決定效益DellEMC產(chǎn)品3個(gè)滿(mǎn)配的Isilon存儲(chǔ)集群，每個(gè)集群144個(gè)存儲(chǔ)引擎

5、集中化存放所有的非結(jié)構(gòu)化數(shù)據(jù)通過(guò)強(qiáng)大的權(quán)限管理實(shí)現(xiàn)數(shù)據(jù)的安全共享成功案例Bxxxx國(guó)際知名的半導(dǎo)體制造公司全球有十?dāng)?shù)個(gè)晶圓工廠和封裝測(cè)試工廠員工達(dá)數(shù)萬(wàn)人客戶(hù)遇到的挑戰(zhàn)數(shù)據(jù)增長(zhǎng)過(guò)于迅速數(shù)據(jù)類(lèi)型復(fù)雜MES/VDI/Hadoop/IC design/員工個(gè)人數(shù)據(jù)等效率決定效益DellEMC產(chǎn)品多套存儲(chǔ)集群分布于不同工廠業(yè)界內(nèi)第一性能的全閃存儲(chǔ)集群助力客戶(hù)冷熱數(shù)據(jù)智能分層（無(wú)數(shù)據(jù)遷移工作），降低TCO保護(hù)客戶(hù)投資成功案例TXXX國(guó)際知名的消費(fèi)電器、暖通空調(diào)、機(jī)器人及工業(yè)自動(dòng)化系統(tǒng)、智能供應(yīng)鏈（物流）科技集團(tuán)客戶(hù)遇到的挑戰(zhàn)沒(méi)有專(zhuān)業(yè)的非結(jié)構(gòu)化數(shù)據(jù)存儲(chǔ)平臺(tái)非結(jié)構(gòu)化數(shù)據(jù)增長(zhǎng)快而且類(lèi)型復(fù)雜郵件/EBS/個(gè)人共享

6、數(shù)據(jù)等不同應(yīng)用的性能需求各異DellEMC產(chǎn)品不同性能的產(chǎn)品組成的統(tǒng)一集群，多種應(yīng)用使用同一訪問(wèn)點(diǎn)智能分層，同時(shí)處理不同性能需求的數(shù)據(jù)集管理簡(jiǎn)單，而且大量冷數(shù)據(jù)可以通過(guò)SAN實(shí)現(xiàn)NDMP備份成功案例Mxxxx項(xiàng)目概述采用ECS云存儲(chǔ)建設(shè)了企業(yè)數(shù)據(jù)歸檔平臺(tái)，實(shí)現(xiàn)郵件及內(nèi)部OA業(yè) 務(wù)系統(tǒng)歷史數(shù)據(jù)統(tǒng)一集中歸檔管理，以符合上市公司法規(guī)遵從性 要求。項(xiàng)目背景采用獨(dú)立的、孤島化的本地服務(wù)器內(nèi)置硬盤(pán)方式存儲(chǔ)保存非結(jié)構(gòu) 化數(shù)據(jù)資源分散，孤島化、利用率低、管理復(fù)雜（大于100個(gè)服務(wù)器）法規(guī)遵從性要求，上市公司合規(guī)要求數(shù)據(jù)歸檔可查傳統(tǒng)NAS無(wú)法適合海量小文件數(shù)據(jù)類(lèi)型的存儲(chǔ)要求技術(shù)要求橫向擴(kuò)展、跨節(jié)點(diǎn)實(shí)現(xiàn)云存儲(chǔ)架構(gòu)

7、，并支持多活方式全公司8萬(wàn)人，郵件平均300KB，歷史郵件約4億封需要?dú)w檔項(xiàng)目目標(biāo)建設(shè)穩(wěn)定、高效、跨節(jié)點(diǎn)的歷史數(shù)據(jù)歸檔平臺(tái)長(zhǎng)遠(yuǎn)支持企業(yè)內(nèi)部非結(jié)構(gòu)化數(shù)據(jù)歸檔需求EMC的方案滿(mǎn)足郵件歸檔的高速讀寫(xiě)性能與歸檔應(yīng)用廠商集成，實(shí)現(xiàn)OA及email系統(tǒng)的歷史歸檔功能跨幾十公里的雙站點(diǎn)雙活存儲(chǔ)方案，并支持應(yīng)用級(jí)雙活客戶(hù)收益采用云存儲(chǔ)技術(shù)構(gòu)建了企業(yè)級(jí)歷史數(shù)據(jù)歸檔平臺(tái)滿(mǎn)足合規(guī)和法規(guī)遵從性要求實(shí)現(xiàn)OA及email歷史郵件歸檔能力跨站點(diǎn)雙活存儲(chǔ)架構(gòu)滿(mǎn)足數(shù)據(jù)安全性要求成功案例ZxxxxBIG DATAWILL REDEFINEEVERY BUSINESSIntel Advanced Vector Extensions

8、 512 (Intel AVX-512)Skylake-SP introduces 512-bit vector capability up to 2x improvement in peak performance over AVX2Intel AVX(Sandy Bridge & Ivy Bridge)16 SP/8 DP Flops/Cycle256-bit basic FP16 registersNDS (and AVX128)Improved blendMASKMOVImplicit unalignedIntel AVX2 (Haswell & Broadwell)32 SP/16

9、DP Flops/CycleFloat162 256-bit FP FMAs (Fused Multiply-Add)256-bit integerPERMDGatherIntel AVX-512 (Skylake-SP)Up to 2 512-bit FMAs512-bit FP and Integer32 registers8 mask registers64 SP/32 DP Flops/Cycle (SKUs with 2 512-bit FMAs)32 SP/16 DP Flops/Cycle(SKUs with 1 512-bit FMA)Embedded roundingEmbe

10、dded broadcastScalar/SSE/AVX “promotions”Native media additionsHPC additionsTranscendental supportGather/ScatterIntel AVX* and Intel Turbo Boost Technology 2.0AVX-optimized workloads still see significant performance benefit (when compared to non-AVX workloads)Some processors with turbo may not achi

11、eve any or maximum turbofrequency when running Intel AVX* instructionsRated BaseRated TurboAVX BaseAVX TurboFrequency (GHz)Intel Advanced Vector Extensions (Intel AVX)* are designed to achieve higher throughput for certain integer and floating point operations. Due to varying processor power charact

12、eristics, utilizing AVX instructions may cause a) some parts to operate at less than the rated frequency and b) some parts with Intel Turbo Boost Technology 2.0 to not achieve any or maximum turbo frequencies. Performance varies depending on hardware, software, and system configuration and you should consult your system manuf