.TestDesignforBetterTestAutomation.Whoisyour

speaker?LogiGear

Corporation?

SiliconValley

andVietnam?

Testing

andtest

automationservices:?

consultancy,

training?

test

developmentandautomationservices?

"test

integrated"

developmentservices?

Products:?

TestArchitect?,

TestArchitect

for

VisualStudio??

integratingtest

developmentwithtest

managementandautomationHans

Buwaldahans@@hansbuwalda?

Dutchguy,

inCaliforniasince2001?

Backgroundinmath,computerscience,management?

Focusingonkeyword

testing,agiletesting,bigtestingFrom

TheNetherlands(Holland)ZwolleZwolle

(茲沃勒)We

like

to

ridebikesAmericaNetherlandsOften

hard

to

keep

upfor

QASCRUMPROJECTSprintSprintSprintSPOPOPOPODevDevDevDevQAQAQACooperationinateamisimportantfor

QA.Ifthatsuffers

QAwillfall

evenfurtherbehind...Tests

andtheirscalabilityDon'tjust

automate

manualtestingDon'tjust

automate

manualtestingDon'tjust

automate

manualtestingGoodautomatedtestingisnotthesameasautomatinggoodmanualtesting...Example

Scripting(Selenium)WebElementelement=driver.findElement(B(name));element.sendKeys("mysterymagic");element.submit();???(newWebDriverWait(driver,10)).until(newExpectedCondition<Boolean>(){publicBooleanapply(WebDriverd){returnd.getTitle().toLowerCase().startsWith("mystery");}});System.out.println(driver.getTitle());UsingActionsfragmentfromatest

withactionsreadfromtopto

bottom4actions,eachwithanactionkeywordandarguments?

Writethetestasactions,inaspreadsheet?

Eachactionhasanameandarguments?

Automatetheactions,notthetests?

Eachactionisautomatedonlyonce,easytomaintainVariables

andexpressions

withkeywords?

Use">>"to

assignavalue

to

avariable?

Use"#"to

start

anexpressionUseexisting

actionsto

make

new

actions?

We

use"enter","click"and"check"to

make

anewaction?

Thenewactioniscalled"checkbalance"?

Argumentsare

usedwith"#"defineinoneplace:usemany

timesintests:3ways

to

implementactions?

Withactiondefinitions?

useexisting

actionsto

create

newones?

seethepreviousslide?

Low

level

(UI,etc)

actionscan

bebuilt-ininthetool?

like

"click","selectmenuitem",etc?

TestArchitect

has381ofsuchbuilt-inactions?

Canprogram

inaprogramming

language?

like

C#,PythonorJavaExample

actionimplementationinPythonScriptfor

anactionchecksortorder,to

verifywhethertherows

inatableare

sorted:defaction_checkSortOrder():#gettableobject,columnnumberandcolumncountwindowName=LIBRARY.NamedArgument("window")get

argumentsfromthetest

linefindthetableintheUItableName=LIBRARY.NamedArgument("table")columnName=LIBRARY.NamedArgument("column")table=ABT.OpenElement(windowName,tableName)column=table.GetColumnIndex(columnName)rowCount=table.GetRowCount()#checkthesortorder,rowbyrowprevious=table.GetCellText(0,column)foriinrange(1,rowCount):ifavalueissmallerthanbefore,

failthetestcurrent=table.GetCellText(i,column)if

current<previous:LIBRARY.AdministerCheck("order","sorted",

"fails

"

+

str(i+1),

0)returnprevious=currentifallrows

areascending,passthetestLIBRARY.AdministerCheck("order","sorted",

"all

rows

in

order",

1)Usingthenew

action?

By

keeping

anactiongenericitcan

beappliedfor

avariety

ofsituations?

Someexamples

ofusing"checksortorder":Behavior

DrivenDevelopment

(BDD)?

Usesnatural

languagescenarios"sweater"

--毛線衣?

Tools

like

JBehave

andCucumbermaptheseto

code?

Structureis"Given-When-Then"(GWT)?

Example:Given

acustomerpreviouslyboughtablacksweater

from

meAndIcurrentlyhave

three

blacksweaters

left

instockWhenhereturns

thesweater

for

arefundThenIshouldhave

four

blacksweaters

instock(source:Wikipedia)BDDwithactions(example

1)BDDwithActions(example

2)Givenacustomerpreviouslyboughtablacksweater

frommeAndIcurrentlyhave

threeblacksweaters

leftinstockWhenhereturnsthesweater

for

arefundThenIshouldhave

four

blacksweaters

instockRisks

ofkeyword

approaches?

Keywords

are

notamagicsolutionto

allproblems?

Withmany

tests

andkeywords

maintenance

can

stillbehardThemethodIwillshowyou

today:"ActionBasedTesting"Mystatement

ontest

automation...Issuesare

notalways

obvious...DowntonAbbey(唐頓莊園)Why

Better

Test

Design?QualitywillbebetterAutomationwillbeeasierExample

ofaTestisthisagoodtest?isitgoodfor

automation?BDDscenario...GivenUserturnsoff

PasswordrequiredoptionforDrive

TestAndUserhasloggedinby

Traffic

ApplicantaccountAndUserisatthe

Assessments

Take

a

Test

pageAndUserclicksthe

TrafficTest

linkAndUserclickstheNextbuttonAndUserclickstheSheetradiobuttoninModepageifdisplayedAndUserclickstheStartbuttonAndUserwaitsforteststartAndUserclickstheStop

Test

buttonWhenUserclickstheConfirmStop

Test

buttonAndUserentersthecorrectapplicantpasswordAndUserclickstheConfirmStop

Test

buttonThen

The

Test

isOvershouldbedisplayedintheMessagelabelAndthevalueoftheMessagelabelshouldbe

ThetestisoverAnd

TheWelcometo

TrafficTesting

pageshouldbedisplayedTheTest

AutomationDesignParadox悖論Test

ModuleObjectivesInitial-setupTest

casesFinal-cleanup(1)Early

intheproject:identifythetest

modulesthesebecometheclosettester,

engineersandproductownerscandothistogetherwithlabeled(empty)shelvesthatthetestswillbestored

inlater(2)Duringtheproject:fitthetests

into

themodulesbutdon'thidetoomuch,infothatisneededto

understandatest

shouldbevisibleonlyuseactionsandchecksthatfitthescopeofthemoduleavoidunnecessarydetails,hidethemintheactionsOverview

ActionBased

TestingHighLevelTest

DesignTest

Module1Test

Module2ObjectivesTest

ModuleNObjectives...ActionsAUTOMATIONinteractiontestbusinesstestBreakdown

Criteria?

CommonCriteria?

Functionality

(customers,finances,managementinformation,UI,...)?

Architectureofthesystem

undertest

(client,server,

protocol,subsystems,components,modules,...)?

Kindoftest

(navigationflow,

negative

tests,

responsetime,...)?

AdditionalCriteria?

Stakeholders

(like

"Accounting","Compliance","HR",...)?

Complexityofthetest

(putcomplextestsinseparate

modules)?

Executionaspects(specialhardware,

multi-station,...)?

Projectplanning(availabilityofinformation,timelines,sprints,...)?

Risks

involved

(extra

test

modulesfor

highriskareas)?

Ambitionlevel

(smoke

test,

regression,aggressive,…)For

moreinformation,andexamples,seethearticlesonmy

website:Eye

ontheball,Scope?

Always

knowthescope

ofthetest

module?

Thescope

shouldbeclearandunambiguous?

Thescope

determines

many

things:?

whatthetest

objectivesare?

whichtest

casesto

expect?

whatlevel

ofactions

to

use?

whatthechecks

are

aboutandwhichevents

shouldgenerate

awarningorerror(ifa“l(fā)ower”

functionalityiswrong)Usetheright

level

actionsHidedetailsiftheydon'tmatter

(don'tfitthescopeofthetest):however

......showdetailsiftheydomatter.

Thislinefor

example

isunclear:???Tip:have

default

values

for

argumentsACTIONDEFINITIONloginnameargument

userargument

passworddefaultvaluetestertesterpwwindowloginwindow

usernameloginwindow

passwordcontrolvalue#user#passwordenterenterwindow

controlloginwindow

loginclickSometimesexplicitvaluesare

needed:Usuallydefaultvaluesare

goodenough:datepayeemake

payment

1/1/12

GasCo.

85.00amounttextHelloTamaracheckmessageNon-UI?

Examples?

web

services(RESTandSOAP)?

applicationprogramminginterfaces

(API’s)?

embeddedsoftware?

protocols?

files,batches?

databases,SQL?

commandlineinterfaces

(CLI’s)?

multi-media?

Canbetarget

oftests,

and/or

automationinterfaces?

Non-UIautomationcan

speedupfunctionaltests

that

donotaddress

theUI?

butitcan

alsocomplicatethemMultipleSystem

AccessTest

Modules,drivingeitheroneormultipleinterfacesAutomationprotocolaccessUIaccessSystem

UnderTestAPIaccessdatabaseaccessExample

approach:usinganagentdoasmuchaspossibleheredoaslittleaspossiblehereHostDeviceABTAutomationAgentSoftwareUnderTestInterfaceInfoAndroidCarDiagnosticMobileDevicesPoint

ofSaleTerminalsGames,VRSystemsMultimedia:The"checkpicture"Approach?

Approachapplicablefor

pictureslike

graphicsoricons?

Thetester

willaddaline"checkpicture",thatincludesa"question"asoneofthearguments?

Whilethetest

isexecuted

TA

keeps

therecordedpictures?

Afterexecutionthepicturesare

shownto

amanualtestingfor

approval?

Onceapprovedunchangedsamepictureswon'tbepresented

to

themanualtester

againinfuturerunsAbsolute

andrelative

picture

checks?

Relativepicturechecksare

partoftest

module?

Theydepictimagesthatare

specificfor

thatmodule?

Exampleachartbasedoncalculations.Inanothermodulethechartwilltypicallybedifferent

(basedonadifferent

calculation)?

Absolutepicturechecksrefer

to

picturesthatoccurinmultipletests,

basedonconditions,butalways

thesameimage?

Theirnamesstart

with"/"?

Typical

use:iconsorpictograms:?

asatester

you

usuallydon'tcare

aboutthedesignofanicon,butwhetheritappearsw