Adversarial

Example

Detection姜育剛，馬興軍，吳祖煊Recap:

week

3

1.

Adversarial

Examples

2.

Adversarial

Attacks

3.

Adversarial

Vulnerability

UnderstandingIn-class

Adversarial

Attack

Competitionhttps://codalab.lisn.upsaclay.fr/competitions/15669?secret_key=77cb8986-d5bd-4009-82f0-7dde2e819ff8

In-class

Adversarial

Attack

CompetitionIn-class

Adversarial

Attack

CompetitionAdversarial

attack

competition（account

for

30%）必須使用學校郵箱注冊比賽（否則無成績）比賽時間：Phase

1：10月1號–

10月28號Phase

2：評估階段，學生不參與沒卡的同學可以使用Google

Colab：/

按排名算分：第一名30分最后一名15分Adversarial

Example

Detection

(AED)A

binary

classification

problem:

clean

(y=0)

or

adv

(y=1)?An

anomaly

detection

problem:

benign

(y=0)

or

abnormal

(y=1)?

Principles

for

AEDAll

binary

classification

methods

can

be

applied

for

AEDPrinciples

for

AEDAll

anomaly

detection

methods

can

be

applied

for

AEDPrinciples

for

AEDUse

as

much

information

as

you

canInput

statisticsManual

featuresTraining

dataAttention

mapTransformationMixupDenoising…ActivationsDeep

featuresProbabilitiesLogitsGradientsLoss

landscapeUncertainty…Principles

for

AEDLeverage

unique

characteristics

of

adversarial

examplesTwinsStrangersExtremely

close

to

the

clean

sampleFar

away

in

predictionPrinciples

for

AEDBuild

detectors

based

on

existing

understandingsHigh

dimensional

pocketsLocal

linearityTilting

boundaryPrinciples

for

AEDIt’s

is

still

feature

engineering!Challenges

in

AEDThe

diversity

of

adversarial

examples

used

for

training

the

detectors

determine

the

detection

performanceDetectors

are

also

machine

learning

models:

they

are

also

vulnerable

to

adversarial

attacks

The

detectors

need

to

detect

both

existing

and

unknown

attacksThe

detectors

need

to

be

robust

to

adaptive

attacksExisting

MethodsSecondary

Classification

Methods

(二級分類法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布檢測法)Prediction

Inconsistency

(預測不一致性)Reconstruction

Inconsistency

(重建不一致性)Trapping

Based

Detection

(誘捕檢測法)Existing

MethodsSecondary

Classification

Methods

(二級分類法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布檢測法)Prediction

Inconsistency

(預測不一致性)Reconstruction

Inconsistency

(重建不一致性)Trapping

Based

Detection

(誘捕檢測法)Secondary

Classification

MethodsTake

adversarial

examples

as

a

new

classAdversarialRetraining

(對抗重訓練)Grosse

et

al.

Onthe(Statistical)DetectionofAdversarialExamples,

arXiv:1702.06280Secondary

Classification

MethodsClean

samples

as

class

0,

adversarial

as

class

1AdversarialClassification

(對抗分類)Gong

et

al.

Adversarialandcleandataarenottwins,

arXiv:1704.04960Secondary

Classification

MethodsTraining

a

detector

for

each

intermediate

layerCascade

Classifiers

(級聯(lián)分類器)Metzen,JanHendrik,etal."Ondetectingadversarialperturbations."

arXivpreprintarXiv:1702.04267

(2017).Existing

MethodsSecondary

Classification

Methods

(二級分類法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布檢測法)Prediction

Inconsistency

(預測不一致性)Reconstruction

Inconsistency

(重建不一致性)Trapping

Based

Detection

(誘捕檢測法)Principle

Component

Analysis

(PCA)The

last

few

components

differentiate

adversarial

examplesHendrycks,Dan,andKevinGimpel.“Earlymethodsfordetectingadversarialimages.”

arXiv:1608.00530

(2016);

Carlini

and

Wagner."Adversarialexamplesarenoteasilydetected:Bypassingtendetectionmethods."

AISec.2017.Blue:

a

clean

sampleYellow:

an

adv

exampleAn

artifact

caused

by

the

black

backgroundDimensionality

ReductionBhagoji,ArjunNitin,DanielCullina,andPrateekMittal."Dimensionalityreductionasadefenseagainstevasionattacksonmachinelearningclassifiers."arXiv:1704.02654

2.1(2017).Train

on

PCA

reduced

dataExisting

MethodsSecondary

Classification

Methods

(二級分類法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布檢測法)Prediction

Inconsistency

(預測不一致性)Reconstruction

Inconsistency

(重建不一致性)Trapping

Based

Detection

(誘捕檢測法)Distribution

DetectionGrosse

et

al.

Onthe(Statistical)DetectionofAdversarialExamples,

arXiv:1702.06280MaximumMeanDiscrepancy

(MMD)Two

datasets:

Distribution

DetectionFeinman,Reuben,etal."Detectingadversarialsamplesfromartifacts."

arXivpreprintarXiv:1703.00410

(2017).KernelDensityEstimation

(KDE)Adversarial

examples

are

in

low

density

spaceDistribution

DetectionFeinman,Reuben,etal."Detectingadversarialsamplesfromartifacts."

arXivpreprintarXiv:1703.00410

(2017).KernelDensityEstimation

(KDE)Adversarial

examples

are

in

low

density

space

Bypassing

10

Detection

MethodsAdversarialExamplesAreNotEasilyDetected:BypassingTenDetectionMethods.

Carlini

and

Wagner,

AISec

2017.Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018Definition(LocalIntrinsicDimensionality)AdversarialexamplesareinhighdimensionalsubspacesLocal

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018AdversarialSubspacesandExpansionDimension:

Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018Estimatinglocalintrinsicdimensionality.Amsaleg

et

al.KDD

2015EstimationofLID:

Hill(MLE)estimator(Hill1975,Amsalegetal.2015):BasedonExtremeValueTheory:Nearestneighbordistancesareextremeevents.LowertaildistributionfollowsGeneralizedParetoDistribution(GPD).

Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018InterpretationofLIDforAdversarialSubspaces:LIDdirectlymeasuresexpansionrateoflocaldistancedistributions.Theexpansionofadversarialsubspaceishigherthannormaldatasubspace.LIDassessesthespace-fillingcapabilityofthesubspace,basedonthedistancedistributionoftheexampletoitsneighbors.Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018LID

of

adversarial

examples

(red)

are

higherLID

at

deeper

layers

are

more

differentiableLocal

Intrinsic

Dimensionality

(LID)Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018Experiments&Results:DatasetFeatureFGMBIM-aBIM-bJSMAOptMNISTKD78.1298.1498.6168.7795.15BU32.3791.5525.4688.7471.30LID96.8999.6099.8392.2499.24CIFAR-10KD64.9268.3898.7085.7791.35BU70.5381.6097.3287.3691.39LID82.3882.5199.7895.8798.94SVHNKD70.3977.1899.5786.4687.41BU86.7884.0786.9391.3387.13LID97.6187.5599.7295.0797.60Local

Intrinsic

Dimensionality

(LID)CharacterizingAdversarialSubspaceUsingLocalIntrinsicDimensionality.

Maet

al.

ICLR

2018Experiments&Results:Train\TestattackFGMBIM-aBIM-bJSMAOptFGSMKD64.9269.1589.7185.7291.22BU70.5381.672.6586.7991.27LID82.3882.3091.6189.9393.32Detectors

trained

on

simple

attacks

FGSM

can

detect

complex

attacksAn

Improved

Detector

of

LID/pdf/2212.06776.pdf

An

Improved

Detector

of

LID/pdf/2212.06776.pdfMahalanobisDistance

(MD)Mahalanobis,PrasantaChandra."Onthegeneralizeddistanceinstatistics."NationalInstituteofScienceofIndia,1936.

The

MD

of

between

two

data

points:MahalanobisDistance

(MD)Leeetal.“Asimpleunifiedframeworkfordetectingout-of-distributionsamplesandadversarialattacks.”

NeurIPS

2018.

MahalanobisDistance

(MD)Leeetal.“Asimpleunifiedframeworkfordetectingout-of-distributionsamplesandadversarialattacks.”

NeurIPS

2018.MahalanobisDistance

(MD)Leeetal.“Asimpleunifiedframeworkfordetectingout-of-distributionsamplesandadversarialattacks.”

NeurIPS

2018.Experiments&Results:Existing

MethodsSecondary

Classification

Methods

(二級分類法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布檢測法)Prediction

Inconsistency

(預測不一致性)Reconstruction

Inconsistency

(重建不一致性)Trapping

Based

Detection

(誘捕檢測法)Bayes

UncertaintyBayesianUncertainty(BU)

Feinman,Reuben,etal."Detectingadversarialsamplesfromartifacts."

arXivpreprintarXiv:1703.00410

(2017).Feature

SqueezingXu

et

al."Featuresqueezing:Detectingadversarialexamplesindeepneuralnetworks."

arXiv:1704.01155

(2017).Bit

depth

reductionSqueezing

clean

and

adv

examplesReducing

input

dimensionality

improves

robustnessThe

prediction

inconsistency

before

and

after

squeezing

can

detect

advsRandom

TransformationTian

et

al."Detectingadversarialexamplesthroughimagetransformation."

AAAI2018.The

prediction

of

advs

will

change

after

random

transformationsLog-OddsRoth

et

al.“Theoddsareodd:Astatisticaltestfordetectingadversarialexamples.”

ICML2019.Add

random

noise

to

the

input

Log-OddsHuetal.“Anewdefenseagainstadversarialimages:Turningaweaknessintoastrength.”

NeurIPS

2019.原則1：對抗樣本的梯度更均勻原則2：對抗樣本難以被攻擊第二次測試準則1：隨機噪聲不會改變預測結(jié)果測試準則1：再次攻擊需要更多的擾動Existing

MethodsSecondary

Classification

Methods

(二級分類法)Principle

Component

Analysis

(主成分分析法，PCA)Distribution

Detection

Methods

(分布檢測法)Prediction

Inconsistency

(預測不一致性)Reconstruction