黃文波移動終端深度學(xué)習(xí)的優(yōu)化實踐匯報人：美麗聯(lián)合集團(tuán)是專注服務(wù)女性的時尚消費平臺，成立于2016

年

6

月

15

日。美麗聯(lián)合集團(tuán)旗下包括：蘑菇街、美麗說、uni、銳鯊、MOGU

STATION等產(chǎn)品與服務(wù)。覆蓋時尚消費的各個領(lǐng)域，滿足不同年齡層、消費力和審美品位的女性用戶日常時尚資訊與時尚消費所需。集團(tuán)簡介日活用戶10

,000

,000

+成交規(guī)模￥

20

,000

,000

,000

+時尚紅人120

,000

+注冊用戶數(shù)200

,000

,000

+女性用戶占比

95%+移動用戶占比95%+整體數(shù)據(jù)主要內(nèi)容背景與現(xiàn)狀模型壓縮與設(shè)計移動端實踐總結(jié)01背景及現(xiàn)狀深度學(xué)習(xí)：從云端到邊緣計算蘑菇街為什么做深度學(xué)習(xí)優(yōu)化？服務(wù)器減少訓(xùn)練、預(yù)測的時間節(jié)約GPU資源，節(jié)約電移動端實時響應(yīng)需求本地化運行，減少服務(wù)器壓力保護(hù)用戶隱私CNN基礎(chǔ)CNN基礎(chǔ)模型越來越大越多的存儲和計算耗費越多能量移動設(shè)備：內(nèi)存有限、計算性能有限、功耗有限深度學(xué)習(xí)：網(wǎng)絡(luò)越來越深，準(zhǔn)確率越來越高Challenge02模型壓縮與設(shè)計PruningQuantizationHuffman

EncodingModel

CompressionWeight-Level

Pruning

for

the

sparse

connectionsHan

et

al,

“Learning

both

weights

and

connections

for

efficient

neural

networks”,

NIPS

2015PruningChannel-Level

Pruning

and

retraining

iterativelyLi

et

al,

“Pruning

filter

for

efficient

convnets”,

ICLR

2017PruningChannel-Level

Pruning

with

L1

regularizationLiu

et

al,

“Learning

efficient

convolutional

networks

through

network

slimming”,

ICCV

2017PruningHan

et

al,

“Deep

Compression:

Compressing

deep

neural

networks

with

pruning,

trained

quantization

and

huffman

coding”,QuantizationHan

et

al,

“Deep

Compression:

Compressing

deep

neural

networks

with

pruning,

trained

quantization

and

huffman

coding”,Huffman

Encodingchannel-level

pruningand

retraining

iterativelychannel-level

pruning

withL1

regularizationPruning:

less

number

of

channelsSummary

of

model

compressionSqueezeNetMobileNetShuffleNetSmaller

CNNs

architecture

designIandola

et

al,

“Squeeze

Net:

Alex

Net-

level

accuracy

with

50

x

fewer

parameters

and

<

0

.5

MB

model

size”,

arXiv

2016SqueezeNetInput641x1

ConvSqueeze161x1

ConvExpand3x3

ConvExpandOutputConcat/Eltwise1286464Howard

et

al,

“MobileNets:

Efficient

convolutional

neural

networks

for

mobile

vision

applications”,

arXiv2017MobileNetsZhang

et

al,

“ShuffleNet:

An

extremely

efficient

convolutional

neural

network

for

mobile

devices”,

arXiv

2017ShuffleNetOverall

Performance

of

Pruning

ResNet50

on

ImageNetOur

practiceModelstrategyTop-1Top-5Model

SizeOriginal-75%92.27%98MPruned-50Pruning72.5%90.9%49MPruned-Q-50Pruning

+

Quantization72.4%90.6%15MPerformance

of

Pruning

ResNet-34

on

Our

DatasetOur

practice(2319

categories,

1200W

samples)ModelTop-1Top-5Inference

TimeModel

SizeOriginal48.92%82.2%96ms86MPruned-6448.27%81.5%45ms31MParseNet

18類(基礎(chǔ)網(wǎng)絡(luò)：MobileNet)Our

practiceModelmIOUPixel-Level-AccuracyModel

SizeParseNet56%93.5%13M03移動端工程實踐TrainingInference移動端服務(wù)端分工DL

frameworksCaffe

Caffe2

MXNet

Tensorflow

Torch

….NCNN、MDLCoreMLTensorflow

LiteFrom

training

to

inferenceConvolutionBNReluConvolution優(yōu)化卷積計算Direct

convolution25*9

9*1im2col-based

convolution優(yōu)化卷積計算Cho

et

al,

“MEC:

Memory-efficient

convolution

for

deep

neural

network”,浮點運算定點化Input(float)Output(float)MinMaxQuantize8

BitMinMaxQuantizedRelu8Bit

DequaMnitnizeMax卷積計算還能怎么進(jìn)化？再牛逼的優(yōu)化算法，都不如硬件實現(xiàn)來得直接通用卷積

VS特定卷積Android端深度學(xué)習(xí)框架NCNN

vs

MDLTensorflow

LiteMobileNet

on

HuaweiP9FrameWork單線程四線程內(nèi)存NCNN370ms200ms25MMDL360ms190ms30MQuantize

MobileNetFloat

Mobilenet85ms400msiOS上的DLCoreML可擴(kuò)展性不強(qiáng)，不適合部署新算法；需要iOS

11+充分利用GPU資源，不用搶占CPU利用Metal開發(fā)新的層很方便Tips：半精度計算；權(quán)重存儲格式為NHWCMPSCNNMPSImageThe

layout

of

a

9-channel

CNN

image

with

a

width

of

3

and

a

height

of

2.Slice2Slice1Slice0MPSCNNkernel

void

eltwise

Sum_array(texture2d_array<half,

access::sample>

inTexture1

[[texture(0)]],texture2d_array<half,

access::sample>

inTexture2

[[texture(1)]],texture2d_array<half,

access::write>

outTexture

[[texture(2)]],ushort3

gid

[[thread_position_in_grid]]){if

(gid.x

>=

outTexture.get_width()

||gid.y

>=

outTexture.get_height()

||gid.z

>=

outTexture.get_array_size())

return;constexprsampler

s(coord::pixel,

filter::nearest,

address::clamp_to_zero);const

ushort2

pos

=

gid.xy;const

ushort

slice

=

gid.z;half4

in[2];in[0]

=

inTexture1.sample(s,

float2(pos.x,

pos.y),

slice);

in[1]

=

inTexture2.sample(s,

float2(pos.x,

pos.y),

slice);

float4

out

=

float4(0.0f);out

=float4(

in[0]+in[1]);outTexture.write(half4(out),

gid.xy,

gid.z);}Metal

Performance

ShaderMPSCNN

VS

NCNN

on

iPhoneDevice:

iPhone

6sFrameWorkTimeNCNN110msMPSCNN45msHow

to

create

a

new

framework優(yōu)化inference網(wǎng)絡(luò)結(jié)構(gòu)GPU加速指令集加速多線程內(nèi)存布局優(yōu)化

NCHW—>NHWC浮點運算定點化ForprofessionalHighly

FlexibleEasy

to

useHigh

Cohesion&

Low

CouplingMoguDLToolkitMogu

Deep

Learning

ToolkitMogu

Deep

Learning

ToolkitCreatelayerInit

layerInferenceClassificationDetectionSegmentation…………Mogu

DL

Toolkit-ExampleMobileNetclass

MobileNet{public:Input

input;Convolution

fc7;int

Init(const

char*

modelpath);int

infer(Mat

&input,Mat

&output);private:Convolution