T

H

E

C

H

A

L

L

E

N

G

E

S

O

F

B

I

G

D

ATA

S

O

F

T

WA

R

ET

E

S

T

I

N

G

S

C

E

N

A

R

I

O

’

S

C

O

N

S

T

R

U

C

T

I

O

N?

數(shù)

據(jù)

分

析

軟

件

的

測

試

場

景

構(gòu)

建

之

挑

戰(zhàn)內(nèi)

容

簡

介?數(shù)據(jù)近些年成為各個領(lǐng)域的熱?話題，?論是企業(yè)的商業(yè)模式?jīng)Q策的轉(zhuǎn)換，或是?業(yè)領(lǐng)域的物聯(lián)?，還是國內(nèi)興起的互聯(lián)?＋浪潮，都和?數(shù)據(jù)分析密不可分。在?數(shù)據(jù)軟件分析功能需求量不斷增加的軟件?程項?中，?數(shù)據(jù)進?、存儲、分析、展現(xiàn)等場景測試的構(gòu)建難度不斷增加。希望，借助這次CCF舉辦的“軟件?程與研究實踐2016論壇”，拋磚引?，?同探討?數(shù)據(jù)測試與傳統(tǒng)測試的區(qū)別以及挑戰(zhàn)。并可以對以下兩部分內(nèi)容做?較詳盡的實踐分享與總結(jié)，尋找對傳統(tǒng)的測試?法和理論的創(chuàng)新與補充。1）?數(shù)據(jù)的數(shù)據(jù)類型和量級給軟件測試帶來的挑戰(zhàn)數(shù)據(jù)類型包括機器?志數(shù)據(jù)、設(shè)備數(shù)據(jù)等不同來源與類型。有效的管理測試數(shù)據(jù)的多樣性以及數(shù)據(jù)的?命周期，可以幫助測試優(yōu)化資源的使?并獲得所期望的數(shù)據(jù)場景分析結(jié)果。同時，通過對數(shù)據(jù)量與硬件需求的變化關(guān)系可以制定最優(yōu)?數(shù)據(jù)測試資源策略。2）?數(shù)據(jù)分析測試場景的構(gòu)建?數(shù)據(jù)分析測試場景除了數(shù)據(jù)的復(fù)雜性以外，往往涉及分布式計算等復(fù)雜的集群拓撲結(jié)構(gòu)關(guān)系。需要有穩(wěn)定的底層架構(gòu)?持，來實現(xiàn)構(gòu)建過程的最?程度?動化。如何縮短構(gòu)建場景時間，重復(fù)利?場景，盡早釋放占有資源等是解決環(huán)境構(gòu)建問題的重中之重。B

I

G

D

ATA

S

O

F

T

WA

R

E

T

E

ST

I

N

GS

C

E

N

A

R

I

O

’

S

CO

N

ST

R

U

C

T

I

O

Nbigdatatestchallengeswhat

’s

thebigdata?what

’s

thebigdatatest?bigdatatestaspectsBIGD

ATAtestingscenarios’constructionpracticeW

H

A

T

’

S

?

T

H

E

?

B

I

G

?

D

A

T

A

?B

I

G

D

ATA

I

S

N

O

T

O

N

LY

A

B

O

U

T

D

ATAB

I

G

D

ATA

A

N

A

LY

S

I

S

P

R

O

C

E

S

SD

ATA

->INFORMATION

->KNOWLEDGE,THAT’S

WHYFORBIGD

ATA

ANALYSISDATAsourcedataINGESTIONINFORMATIOND

ATA

VISUALISATIONstorageplatforminfrastructurereportingdashboardknowledge*

COMPUTERCONNECTSPEOPLEANDDATA

BYPATTERNS,

PATTERNS

AREKNOWLEDGEW

H

A

T

’

S

?

T

H

E

?

B

I

G

?

D

A

T

A

?

T

E

S

T

?B

I

G

D

ATA

T

E

S

T

I

N

G

I

S

A

B

O

U

TD

ATA

F

L

O

W

-

H

O

R

I

Z

O

N

TA

L

LYD

ATAD

ATAD

ATAD

ATA??????…?completeness/integritycorrectnesstimelinessconsistencyDATA

INTELLIGENCE*itisnotonlyamountofdata,itisdifferentkindsofdataB

I

G

D

ATA

T

E

S

T

I

N

G

I

S

A

B

O

U

TD

ATA

F

L

O

W

-

V

E

R

T

I

C

A

L

LYD

ATA

VISUALIZATIOND

ATA

ANALYSIS1.

data

visualizationvalidation

(style,

co

lor,

layout)1.

data

logicdefinition2.

data

patterning/modelling

3.

data

reportingD

ATA

PROCESSINGD

ATA

STORAGE1.

data

definition2.

data

generation(realtime,

batch)

3.

data

ingestionvalidation1.

dealing

with

non-relational

DB2.

optimaltest

environment3.

highavailability

4.

scalabilityB

I

G

D

ATA

T

E

S

T

I

N

G

I

S

N

O

T

A

B

O

U

TBIGD

ATA

SOFTWARE

TESTCANNOTSOLVE

BELOWD

ATA

PROBLEMSdegreeofexcellence-actualscenariothestateofcompletenessdataqualitythetotalityoffeaturesandcharacteristicsofdatadatavaluestobusinessrequirementsandacceptancecriteriaB

I

G

?

D

A

T

A

?

T

E

S

T

?

A

S

P

E

C

T

Sfunctionaltestperformance

testsecurity

test????TRADITIONALusabilitytest?

data

qualitytest?

failovertest(HA)?

ScalabilitytestD

ATA

SAMPLINGLINERGROWINGD

ATAINHOUSESERVERNEW

WORLDPLATFORMINFRASTRUCTURED

ATAT

H

E

?

C

H

A

L

L

E

N

G

E

S

?

O

F

?

B

I

G

?

D

A

T

A

?

S

O

F

T

W

A

R

E

?T

E

S

T

I

N

G

?

S

C

E

N

A

R

I

O

’

S

?

C

O

N

S

T

R

U

C

T

I

O

N1.

hard

tohavereal

data2.

hard

toshare

real

data3.

hard

tomanagehugedata1.DAT

A3V

->

4V

->

5V…1.

testenvironmentcostishigh2.

createtestenvironmentcostishigh(engineer)3.

maintaintestenvironmentcostishigh2.TEST

COSThow

big

is

big

enough

for

test?1.

configurationcombinationiscomplex2.

misconfigurationhappensalot3.

bugsare

hard

tobereproduced4.

testenvironmentiselasticandscalable3.CONFIGU

RATION“amillion

thing

sthat

can

go

wro

ng

betweentest

andpro

duction”“the

applicatio

ninpro

ductionisn't

what

IT

thinks

itshould

be”“thereare

toomanymisco

nfiguratio

nsto

list,

thatwould

take

abo

okor

t

hree”SEQUENCETHROUGHPUTNETWORKDISASTER4.UNCERTAINT

YTOOLSPEOPLEhow

totestanelephant?

howtosetupacustomer-like

environment??

howtooperating

thetestsef?ciently??

howtomeasure

thetestresult??

howtoreproduce

theissueincustomerenvironment??

……openquestionOTH

ERS?

whetherdata

samplingcould?ndmostofcriticalissuesinbigdata

testing?howtoleverage

datasamplingmethodinbigdata

testdesign??

doestraditionaltestdesignmethodologiesstill?tforbigdata

testing?likeboundary

condition,equitationpartition,

cause-result

diagram

…?

shouldwedesignnegative

testdata

caseinbigdatatesting?invalidtestset,illegaldata

format?

……B

I

G

?

D

A

T

A

?

S

O

F

T

W

A

R

E

?

T

E

S

T

I

N

G

?

S

C

E

N

A

R

I

O

’

S

?C

O

N

S

T

R

U

C

T

I

O

N

?

P

R

A

C

T

I

C

E

SD

ATA

GENERATION[SAMPLE.TUTORIAL1]MODE=

REPLAYSAMPLETYPE=

CSVINTERVAL

=

15BACKFILL=

-15MBACKFILLSEARCH=

INDEX=MAINSOURCETYPE=SPLUNKDCOUNT=

20HOUROFDAYRATE

=

{

"0":0.8,"1":1.0,"2":0.9,"3":0.7,"4":0.5,"5":0.4,"6":0.4,"7":0.4,"8":0.4,"9":0.4,"10":0.4,"11":0.4,"12":0.4,"13":0.4,"14":0.4,"15":0.4,"16":0.4,"17":0.4,"18":0.4,"19":0.4,"20":0.4,"21":0.4,"22":0.5,"23":0.6}DAYOFWEEKRATE

=

{

"0":0.7,"1":0.7,"2":0.7,"3":0.5,"4":0.5,"5":1.0,"6":1.0}RANDOMIZECOUNT=

0.2RANDOMIZEEVENTS=

TRUESEEDOUTPUTMODE=

SPLUNKSTREAMSPLUNKHOST=

LOCALHOSTSPLUNKUSER=

ADMINSPLUNKPASS

=

CHANGEMETOKEN.0.TOKEN=

\D{4}-\D{2}-\D{2}\D{2}:\D{2}:\D{2},\D{3}TOKEN.0.REPLACEMENTTYPE=

TIMESTAMPTOKEN.0.REPLACEMENT=

%Y-%M-%D

%H:%M:%S,%FTOKEN.1.TOKEN=

\D{2}-\D{2}-\D{4}\D{2}:\D{2}:\D{2}.\D{3}TOKEN.1.REPLACEMENTTYPE=

TIMESTAMPTOKEN.1.REPLACEMENT=

%M-%D-%Y%H:%M:%S.%FINDEX,HOST,SOURCE,SOURCETYPE,"_RAW""MAIN","CSHARP-MBP15.LOCAL","/APPLICATIONS/SPLUNK/VAR/LOG/SPLUNK/METRICS.LOG",SP