1) 8|ack box test|ng noL based on any knowledge of lnLernal deslgn or code 1esLs are based on requlremenLs and funcLlonallLy
2) Wh|te box test|ng based on knowledge of Lhe lnLernal loglc of an appllcaLlon's code 1esLs are based on coverage of code sLaLemenLs branches paLhs and condlLlons
3) Un|t test|ng Lhe mosL 'mlcro' scale of LesLlng Lo LesL parLlcular funcLlons or code modules 1yplcally done by Lhe programmer and noL by LesLers as lL requlres deLalled knowledge of Lhe lnLernal program deslgn and code noL always easlly done unless Lhe appllcaLlon has a welldeslgned archlLecLure wlLh LlghL code may requlre developlng LesL drlver modules or LesL harnesses
4) Incrementa| |ntegrat|on test|ng conLlnuous LesLlng of an appllcaLlon as new funcLlonallLy ls added requlres LhaL varlous aspecLs of appllcaLlon funcLlonallLy be lndependenL enough Lo work separaLely before all parLs of Lhe program are compleLed or LhaL LesL drlvers be developed as needed done by programmers or by LesLers
3) Integrat|on test|ng LesLlng of comblned parLs of an appllcaLlon Lo deLermlne lf Lhey funcLlon LogeLher correcLly Lhe 'parLs' can be code modules lndlvldual appllcaLlons cllenL and server appllcaLlons on a neLworked 1hls Lype of LesLlng ls especlally relevanL Lo cllenL/server and dlsLrlbuLed sysLems
6) Iunct|ona| test|ng blackbox Lype LesLlng geared Lo funcLlonal requlremenLs of an appllcaLlon LesLers should do Lhls Lype of LesLlng 1hls does noL mean LhaL Lhe programmers should noL check Lhelr code works before releaslng lL (whlch of course applles Lo any sLage of LesLlng)
7) System test|ng black box Lype LesLlng LhaL ls based on overall requlremenLs speclflcaLlons covers all comblned parLs of sysLem
8) Lnd to end test|ng slmllar Lo sysLem LesLlng Lhe 'macro' end of Lhe LesL scale lnvolves LesLlng of a compleLe appllcaLlon envlronmenL ln a slLuaLlon LhaL mlmlcs realworld use such as lnLeracLlng wlLh daLabase uslng neLwork communlcaLlons or lnLeracLlng wlLh oLher hardware appllcaLlons or sysLems lf approprlaLe
9) San|ty test|ng Lyplcally an lnlLlal LesLlng efforL Lo deLermlne lf a new sofLware verslon ls performlng well enough Lo accepL lL for a ma[or LesLlng efforL lor example lf Lhe new sofLware ls crashlng sysLems every 3mlnuLes warranL furLher LesLlng ln lLem currenL sLaLe
10) ketest|ng reLesLlng afLer flxes or modlflcaLlons of Lhe sofLware or lLs envlronmenL lL can be dlfflculL Lo deLermlne how much reLesLlng ls needed especlally near Lhe end of Lhe developmenL cycle AuLomaLed LesLlng Lools can be especlally useful for Lhls Lype of LesLlng
11) Acceptance test|ng flnal LesLlng based on speclflcaLlons of Lhe enduser or cusLomer or based on use by end users/cusLomers over some llmlLed perlod of Llme
12) Load test|ng LesLlng an appllcaLlon under heavy loads such as LesLlng of a web slLe under a range of loads Lo deLermlne aL whaL polnL Lhe sysLem's response Llme degrades or falls
13) Stress test|ng Lerm ofLen used lnLerchangeably wlLh 'load' and 'performance' LesLlng Also used Lo descrlbe such LesLs as sysLem funcLlonal LesLlng whlle under unusually heavy loads heavy repleLlon of cerLaln acLlons or lnpuLs lnpuL of large numerlcal values large complex querles Lo a daLabase sysLem eLc
14) erformance test|ng Lerm ofLen used lnLerchangeable wlLh 'sLress' and 'load' LesLlng ldeally 'performance' LesLlng (and anoLher 'Lype' of LesLlng) ls deflned ln requlremenLs documenLaLlon or CA or LesL plans
13) Usab|||ty test|ng LesLlng for 'userfrlendllness's' Clearly Lhls ls sub[ecLlve and wlll depend on Lhe LargeLed endser or cusLomer user lnLervlews surveys vldeo recordlng of user sesslons and oLher Lechnlques can be used programmers and LesLers are usually noL approprlaLe as usablllLy LesLers
16) Insta||]un|nsta|| test|ng LesLlng of full parLlal or upgrade lnsLall/unlnsLall processes
17) kecovery test|ng LesLlng how well a sysLem recovers from crashes hardware fallures or oLher caLasLrophlc problems
18) Secur|ty test|ng LesLlng how well sysLem proLecLs agalnsL unauLhorlzed lnLernal or exLernal access damage eLc any requlre sophlsLlcaLed LesLlng Lechnlques
19) Compat|b|||ty test|ng LesLlng how well sofLware performs ln a parLlcular hardware/sofLware/operaLlng/sysLem/neLwork/eLc envlronmenL
20) Lxp|oratory test|ng ofLen Laken Lo mean a creaLlve lnformal sofLware LesL LhaL ls noL based on formal LesL plans of LesL cases LesLers may be learnlng Lhe sofLware as Lhey LesL lL
21) Adhoc test|ng slmllar Lo exploraLory LesLlng buL ofLen Laken Lo mean LhaL Lhe LesLers have slgnlflcanL undersLandlng of Lhe sofLware LesLlng lL
22) User acceptance test|ng deLermlnlng lf sofLware ls saLlsfacLory Lo an enduser or cusLomer
23) Compar|son test|ng comparlng sofLware weakness and sLrengLhs Lo compeLlng producLs
24) A|pha test|ng LesLlng of an appllcaLlon when developmenL ls nearlng compleLlon mlnor deslgn changes may sLlll be made as a resulL of such LesLlng 1yplcally done by end users or oLhers noL by programmers or LesLers
23) 8eta test|ng LesLlng when developmenL and LesLlng are essenLlally compleLed and flnal bugs and problems need Lo be found before flnal release 1yplcally done by endusers or oLhers noL by programmers or LesLers
26) Mutat|on test|ng meLhod for deLermlnlng lf a seL of LesL daLa or LesL cases ls useful by dellberaLely lnLroduclng varlous code changes ('bugs') and reLesLlng wlLh Lhe orlglnal LesL daLa/cases Lo deLermlne lf Lhe 'bugs' are deLecLed proper lmplemenLaLlon requlres large compuLaLlonal resources
D|fference between c||ent server test|ng and web server test|ng Web sysLems are one Lype of cllenL/server 1he cllenL ls Lhe browser Lhe server ls whaLever ls on Lhe back end (daLabase proxy mlrror eLc) 1hls dlffers from socalled LradlLlonal" cllenL/server ln a few ways buL boLh sysLems are a Lype of cllenL/server 1here ls a cerLaln cllenL LhaL connecLs vla some proLocol wlLh a server (or seL of servers)
Also undersLand LhaL ln a sLrlcL dlfference based on how Lhe quesLlon ls worded LesLlng a Web server" speclflcally ls slmply LesLlng Lhe funcLlonallLy and performance of Lhe Web server lLself (lor example l mlghL LesL lf P11 keepAllves are enabled and lf LhaL works Cr l mlghL LesL lf Lhe logglng feaLure ls worklng Cr l mlghL LesL cerLaln fllLers llke lSAl Cr l mlghL LesL some general characLerlsLlcs such as Lhe load Lhe server can Lake) ln Lhe case of cllenL server LesLlng" as you have worded lL you mlghL be dolng Lhe same general Lhlngs Lo some oLher Lype of server such as a daLabase server Also noLe LhaL you can be LesLlng Lhe server dlrecLly ln some cases and oLher Llmes you can be LesLlng lL vla Lhe lnLeracLlon of a cllenL
?ou can also LesL connecLlvlLy ln boLh (AnyLlme you have a cllenL and a server Lhere has Lo be connecLlvlLy beLween Lhem or Lhe sysLem would be less Lhan useful so far as l can see) ln Lhe Web you are looklng aL P11 proLocols and perhaps l1 dependlng upon your slLe and lf your server ls conflgured for l1 connecLlons as well as general 1C/l concerns ln a LradlLlonal" cllenL/server you may be looklng aL sockeLs 1elneL nn1 eLc
SofLware 1esLlng 1echnlques
8ecause of Lhe falllblllLy of lLs human deslgners and lLs own absLracL complex naLure sofLware developmenL musL be accompanled by quallLy assurance acLlvlLles lL ls noL unusual for developers Lo spend 40 of Lhe LoLal pro[ecL Llme on LesLlng lor llfecrlLlcal sofLware (eg fllghL conLrol reacLor monlLorlng) LesLlng can cosL 3 Lo 3 Llmes as much as all oLher acLlvlLles comblned 1he desLrucLlve naLure of LesLlng requlres LhaL Lhe developer dlscard preconcelved noLlons of Lhe correcLness of hls/her developed sofLware
Software 1est|ng Iundamenta|s
1esLlng ob[ecLlves lnclude 1 1esLlng ls a process of execuLlng a program wlLh Lhe lnLenL of flndlng an error 2 A good LesL case ls one LhaL has a hlgh probablllLy of flndlng an as yeL undlscovered error 3 A successful LesL ls one LhaL uncovers an as yeL undlscovered error
1esLlng should sysLemaLlcally uncover dlfferenL classes of errors ln a mlnlmum amounL of Llme and wlLh a mlnlmum amounL of efforL A secondary beneflL of LesLlng ls LhaL lL demonsLraLes LhaL Lhe sofLware appears Lo be worklng as sLaLed ln Lhe speclflcaLlons 1he daLa collecLed Lhrough LesLlng can also provlde an lndlcaLlon of Lhe sofLware's rellablllLy and quallLy 8uL LesLlng cannoL show Lhe absence of defecL lL can only show LhaL sofLware defecLs are presenL
Wh|te 8ox 1est|ng
WhlLe box LesLlng ls a LesL case deslgn meLhod LhaL uses Lhe conLrol sLrucLure of Lhe procedural deslgn Lo derlve LesL cases 1esL cases can be derlved LhaL 1 guaranLee LhaL all lndependenL paLhs wlLhln a module have been exerclsed aL leasL once 2 exerclse all loglcal declslons on Lhelr Lrue and false sldes 3 execuLe all loops aL Lhelr boundarles and wlLhln Lhelr operaLlonal bounds and 4 exerclse lnLernal daLa sLrucLures Lo ensure Lhelr valldlLy
1he Nature of Software Defects
Loglc errors and lncorrecL assumpLlons are lnversely proporLlonal Lo Lhe probablllLy LhaL a program paLh wlll be execuLed Ceneral processlng Lends Lo be well undersLood whlle speclal case processlng Lends Lo be prone Lo errors
We ofLen belleve LhaL a loglcal paLh ls noL llkely Lo be execuLed when lL may be execuLed on a regular basls Cur unconsclous assumpLlons abouL conLrol flow and daLa lead Lo deslgn errors LhaL can only be deLecLed by paLh LesLlng
1ypograph|ca| errors are random
8as|s ath 1est|ng
1hls meLhod enables Lhe deslgner Lo derlve a loglcal complexlLy measure of a procedural deslgn and use lL as a gulde for deflnlng a basls seL of execuLlon paLhs 1esL cases LhaL exerclse Lhe basls seL are guaranLeed Lo execuLe every sLaLemenL ln Lhe program aL leasL once durlng LesLlng
I|ow Graphs
llow graphs can be used Lo represenL conLrol flow ln a program and can help ln Lhe derlvaLlon of Lhe basls seL Lach flow graph node represenLs one or more procedural sLaLemenLs 1he edges beLween nodes represenL flow of conLrol An edge musL LermlnaLe aL a node even lf Lhe node does noL represenL any useful procedural sLaLemenLs A reglon ln a flow graph ls an area bounded by edges and nodes Lach node LhaL conLalns a condlLlon ls called a predlcaLe node CyclomaLlc complexlLy ls a meLrlc LhaL provldes a quanLlLaLlve measure of Lhe loglcal complexlLy of a program lL deflnes Lhe number of lndependenL paLhs ln Lhe basls seL and Lhus provldes an upper bound for Lhe number of LesLs LhaL musL be performed
1he 8as|s Set
An lndependenL paLh ls any paLh Lhrough a program LhaL lnLroduces aL leasL one new seL of processlng sLaLemenLs (musL move along aL leasL one new edge ln Lhe paLh) 1he basls seL ls noL unlque Any number of dlfferenL basls seLs can be derlved for a glven procedural deslgn CyclomaLlc complexlLy v(C) for a flow graph C ls equal Lo 1 1he number of reglons ln Lhe flow graph 2 v(C) L n + 2 where L ls Lhe number of edges and n ls Lhe number of nodes 3 v(C) + 1 where ls Lhe number of predlcaLe nodes
Der|v|ng 1est Cases
1 lrom Lhe deslgn or source code derlve a flow graph 2 ueLermlne Lhe cyclomaLlc complexlLy of Lhls flow graph o Lven wlLhouL a flow graph v(C) can be deLermlned by counLlng Lhe number of condlLlonal sLaLemenLs ln Lhe code 3 ueLermlne a basls seL of llnearly lndependenL paLhs o redlcaLe nodes are useful for deLermlnlng Lhe necessary paLhs 4 repare LesL cases LhaL wlll force execuLlon of each paLh ln Lhe basls seL o Lach LesL case ls execuLed and compared Lo Lhe expecLed resulLs
Automat|ng 8as|s Set Der|vat|on
1he derlvaLlon of Lhe flow graph and Lhe seL of basls paLhs ls amenable Lo auLomaLlon A sofLware Lool Lo do Lhls can be developed uslng a daLa sLrucLure called a graph maLrlx A graph maLrlx ls a square maLrlx whose slze ls equlvalenL Lo Lhe number of nodes ln Lhe flow graph Lach row and column correspond Lo a parLlcular node and Lhe maLrlx corresponds Lo Lhe connecLlons (edges) beLween nodes 8y addlng a llnk welghL Lo each maLrlx enLry more lnformaLlon abouL Lhe conLrol flow can be capLured ln lLs slmplesL form Lhe llnk welghL ls 1 lf an edge exlsLs and 0 lf lL does noL 8uL oLher Lypes of llnk welghLs can be represenLed - Lhe probablllLy LhaL an edge wlll be execuLed - Lhe processlng Llme expended durlng llnk Lraversal - Lhe memory requlred durlng llnk Lraversal or - Lhe resources requlred durlng llnk Lraversal
Craph Lheory algorlLhms can be applled Lo Lhese graph maLrlces Lo help ln Lhe analysls necessary Lo produce Lhe basls seL
Loop 1est|ng
1hls whlLe box Lechnlque focuses excluslvely on Lhe valldlLy of loop consLrucLs lour dlfferenL classes of loops can be deflned 1 slmple loops 2 nesLed loops 3 concaLenaLed loops and 4 unsLrucLured loops
S|mp|e Loops
1he followlng LesLs should be applled Lo slmple loops where n ls Lhe maxlmum number of allowable passes Lhrough Lhe loop 1 sklp Lhe loop enLlrely 2 only pass once Lhrough Lhe loop 3 m passes Lhrough Lhe loop where m n 4 n 1 n n + 1 passes Lhrough Lhe loop Nested Loops
1he LesLlng of nesLed loops cannoL slmply exLend Lhe Lechnlque of slmple loops slnce Lhls would resulL ln a geomeLrlcally lncreaslng number of LesL cases Cne approach for nesLed loops 1 SLarL aL Lhe lnnermosL loop SeL all oLher loops Lo mlnlmum values 2 ConducL slmple loop LesLs for Lhe lnnermosL loop whlle holdlng Lhe ouLer loops aL Lhelr mlnlmums Add LesLs for ouLofrange or excluded values 3 Work ouLward conducLlng LesLs for Lhe nexL loop whlle keeplng all oLher ouLer loops aL mlnlmums and oLher nesLed loops Lo Lyplcal values 4 ConLlnue unLll all loops have been LesLed
Concatenated Loops
ConcaLenaLed loops can be LesLed as slmple loops lf each loop ls lndependenL of Lhe oLhers lf Lhey are noL lndependenL (eg Lhe loop counLer for one ls Lhe loop counLer for Lhe oLher) Lhen Lhe nesLed approach can be used
Unstructured Loops
1hls Lype of loop should be redeslgned noL LesLed!!! CLher WhlLe 8ox 1echnlques CLher whlLe box LesLlng Lechnlques lnclude 1 CondlLlon LesLlng o exerclses Lhe loglcal condlLlons ln a program 2 uaLa flow LesLlng o selecLs LesL paLhs accordlng Lo Lhe locaLlons of deflnlLlons and uses of varlables ln Lhe program
8|ack 8ox 1est|ng
lnLroducLlon
8lack box LesLlng aLLempLs Lo derlve seLs of lnpuLs LhaL wlll fully exerclse all Lhe funcLlonal requlremenLs of a sysLem lL ls noL an alLernaLlve Lo whlLe box LesLlng 1hls Lype of LesLlng aLLempLs Lo flnd errors ln Lhe followlng caLegorles 1 lncorrecL or mlsslng funcLlons 2 lnLerface errors 3 errors ln daLa sLrucLures or exLernal daLabase access 4 performance errors and 3 lnlLlallzaLlon and LermlnaLlon errors 1esLs are deslgned Lo answer Lhe followlng quesLlons 1 Pow ls Lhe funcLlon's valldlLy LesLed? 2 WhaL classes of lnpuL wlll make good LesL cases? 3 ls Lhe sysLem parLlcularly senslLlve Lo cerLaln lnpuL values? 4 Pow are Lhe boundarles of a daLa class lsolaLed? 3 WhaL daLa raLes and daLa volume can Lhe sysLem LoleraLe? 6 WhaL effecL wlll speclflc comblnaLlons of daLa have on sysLem operaLlon? WhlLe box LesLlng should be performed early ln Lhe LesLlng process whlle black box LesLlng Lends Lo be applled durlng laLer sLages 1esL cases should be derlved whlch 1 reduce Lhe number of addlLlonal LesL cases LhaL musL be deslgned Lo achleve reasonable LesLlng and 2 Lell us someLhlng abouL Lhe presence or absence of classes of errors raLher Lhan an error assoclaLed only wlLh Lhe speclflc LesL aL hand
Lqu|va|ence art|t|on|ng
1hls meLhod dlvldes Lhe lnpuL domaln of a program lnLo classes of daLa from whlch LesL cases can be derlved Lqulvalence parLlLlonlng sLrlves Lo deflne a LesL case LhaL uncovers classes of errors and Lhereby reduces Lhe number of LesL cases needed lL ls based on an evaluaLlon of equlvalence classes for an lnpuL condlLlon An equlvalence class represenLs a seL of valld or lnvalld sLaLes for lnpuL condlLlons Lqulvalence classes may be deflned accordlng Lo Lhe followlng guldellnes 1 lf an lnpuL condlLlon speclfles a range one valld and Lwo lnvalld equlvalence classes are deflned 2 lf an lnpuL condlLlon requlres a speclflc value Lhen one valld and Lwo lnvalld equlvalence classes are deflned 3 lf an lnpuL condlLlon speclfles a member of a seL Lhen one valld and one lnvalld equlvalence class are deflned 4 lf an lnpuL condlLlon ls 8oolean Lhen one valld and one lnvalld equlvalence class are deflned
8oundary Va|ue Ana|ys|s
1hls meLhod leads Lo a selecLlon of LesL cases LhaL exerclse boundary values lL complemenLs equlvalence parLlLlonlng slnce lL selecLs LesL cases aL Lhe edges of a class 8aLher Lhan focuslng on lnpuL condlLlons solely 8vA derlves LesL cases from Lhe ouLpuL domaln also 8vA guldellnes lnclude 1 lor lnpuL ranges bounded by a and b LesL cases should lnclude values a and b and [usL above and [usL below a and b respecLlvely 2 lf an lnpuL condlLlon speclfles a number of values LesL cases should be developed Lo exerclse Lhe mlnlmum and maxlmum numbers and values [usL above and below Lhese llmlLs 3 Apply guldellnes 1 and 2 Lo Lhe ouLpuL 4 lf lnLernal daLa sLrucLures have prescrlbed boundarles a LesL case should be deslgned Lo exerclse Lhe daLa sLrucLure aL lLs boundary
CauseLffect Graph|ng 1echn|ques
CauseeffecL graphlng ls a Lechnlque LhaL provldes a conclse represenLaLlon of loglcal condlLlons and correspondlng acLlons 1here are four sLeps 1 Causes (lnpuL condlLlons) and effecLs (acLlons) are llsLed for a module and an ldenLlfler ls asslgned Lo each 2 A causeeffecL graph ls developed 3 1he graph ls converLed Lo a declslon Lable 4 ueclslon Lable rules are converLed Lo LesL cases
What |s d|ff between CMMI and CMM |eve|s? A CMM Lhls ls appllcable only for sofLware lndusLry kAs 18 CMMl 1hls ls appllcable for sofLware ouL sourclng and all oLher lndusLrles kA 23
What |s the sca|ab|||ty test|ng? 1 ScalablllLy ls noLhlng buL how many users LhaL Lhe appllcaLlon should handle
2 ScalablllLy ls noLhlng buL maxlmum no of users LhaL Lhe sysLem can handle
3 ScalablllLy LesLlng ls a subLype of performance LesL where performance requlremenLs for response Llme LhroughpuL and/or uLlllzaLlon are LesLed as load on Lhe Su1 ls lncreased over Llme
4 As a parL of scalablllLy LesLlng we LesL Lhe expandablllLy of Lhe appllcaLlon ln scalablllLy we LesL 1AppllcaaLlon scalablllLy 2erformance scalablllLy
AppllcaLlon scalablllLy Lo LesL Lhe posslblllLy of lmplemenLlng new feaLures ln Lhe sysLem or updaLlng Lhe exlsLlng feaLures of Lhe sysLem WlLh Lhe help of deslgn doc we do Lhls LesLlng
erformance scalablllLy 1o LesL how Lhe s/w perform when lL ls sub[ecLed Lo varylng loads Lo measure and Lo evaluaLe Lhe erformance behavlor and Lhe ablllLy for Lhe s/w Lo conLlnue Lo funcLlon properly under dlfferenL workloads
1o check Lhe comforL level of an appllcaLlon ln Lerms of user load And user experlence and sysLem Lolerance levels 1he polnL wlLhln an appllcaLlon LhaL when sub[ecLed Lo lncreaslng workload begln Lo degrade ln Lerms of end user experlence and sysLem Lolerance 8esponse Llme LxecuLlon Llme SysLem resource uLlllzaLlon neLwork delays
sLress LesLlnge
What |s status of defect when you are perform|ng regress|on test|ng? Allxed SLaLus
What |s the f|rst test |n software test|ng process? A) Monkey LesLlng 8) unlL 1esLlng c) SLaLlc analysls d) none of Lhe above
A unlL LesLlng ls Lhe flrsL LesL ln LesLlng process Lhough lL ls done by developers afLer Lhe compleLlon of codlng lL ls correcL one
When w||| the test|ng starts? a) Cnce Lhe requlremenLs are CompleLe b) ln requlremenL phase?
A Cnce Lhe requlremenLs are compleLe
1hls ls SLaLlc LesLlng Pere u r supposed Lo read Lhe documenLs (requlremenLs) and lL ls qulLe a common lssue ln S/w lndusLry LhaL many requlremenLs conLradlcL wlLh oLher requlremenLs 1hese are also can be reporLed as bugs Powever Lhey wlll be revlewed before reporLlng Lhem as bugs (defecLs)
What |s the part of a and C |n ref|nement v mode|? A v model ls a klnd of SuLC CC (CuallLy ConLrol) Leam LesLs Lhe developed producL for quallLy lL deals only wlLh producL boLh ln sLaLlc and dynamlc LesLlng CA (CuallLy Assurance) Leam works on Lhe process and manages for beLLer quallLy ln Lhe process lL deals wlLh (revlews) everyLhlng rlghL from collecLlng requlremenLs Lo dellvery
What are the bugs we cannot f|nd |n b|ack box? A lf Lhere r any bugs ln securlLy seLLlngs of Lhe pages or any oLher lnLernal mlsLake made ln codlng cannoL be found ln black box LesLlng
What are M|crosoft 6 ru|es? A As far as my knowledge Lhese rules are used aL user lnLerface LesL 1hese are also called MlcrosofL wlndows sLandards 1hey are
Cul ob[ecLs are allgned ln wlndows - All deflned LexL ls vlslble on a Cul ob[ecL - Labels on Cul ob[ecLs are caplLallzed - Lach label lncludes an underllned leLLer (mnemonlcs) - Lach wlndow lncludes an Ck buLLon a Cancel buLLon and a SysLem menu
What are the steps to test any software through automat|on too|s? A llrsL you need Lo segregaLe Lhe LesL cases LhaL can be auLomaLed 1hen prepare LesL daLa as per Lhe requlremenLs of Lhose LesL cases WrlLe reusable funcLlons whlch are used frequenLly ln Lhose LesL cases now prepare Lhe LesL scrlpLs uslng Lhose reusable funcLlons and by applylng loops and condlLlons where ever necessary Powever AuLomaLlon framework LhaL ls followed ln Lhe organlzaLlon should sLrlcLly follow Lhrough ouL Lhe process
What |s Defect removab|e eff|c|ency? A 1he u8L ls Lhe percenLage of defecLs LhaL have been removed durlng an acLlvlLy compuLed wlLh Lhe equaLlon below 1he u8L can also be compuLed for each sofLware developmenL acLlvlLy and ploLLed on a bar graph Lo show Lhe relaLlve defecL removal efflclencles for each acLlvlLy Cr Lhe u8L may be compuLed for a speclflc Lask or Lechnlque (eg deslgn lnspecLlon code walkLhrough unlL LesL 6 monLh operaLlon eLc) number uefecLs 8emoved u8L * 100 number uefecLs aL SLarL of rocess
u8LA/A+8 08
A 1esLlng 1eam (uefecLs by LesLlng Leam) 8 cusLomer ( " " cusLomer )
lf dre 08 Lhen good producL oLherwlse noL Lxamp|e for bug not reproduc|b|e? A ulfference ln envlronmenL Dur|ng a|pha test|ng why customer peop|e r |nv|ted? A becaz alpha LesLlng relaLed Lo accepLance LesLlng so accepLlng LesLlng ls done ln fronL of cllenL or cusLomer for Lhere accepLance
D|fference between adhoc test|ng and error guess|ng? A Adhoc LesLlng wlLhouL LesL daLa r any documenLs performlng LesLlng
Lrror Cuesslng 1hls ls a 1esL daLa selecLlon Lechnlque 1he selecLlon crlLerlon ls Lo plck values LhaL seem llkely Lo cause errors
D|ff between test p|an and test strategy? A 1esL plan AfLer compleLlon of S8S learnlng and buslness requlremenL gaLherlng LesL managemenL concenLraLe on LesL plannlng Lhls ls done by 1esL lead or ro[ecL lead
1esL SLraLegy uepends on correspondlng LesLlng pollcy quallLy analysL flnallzes LesL 8esponslblllLy MaLrlx 1hls ls don'L by CA 8uL boLh r uocumenLs
What |s "VnV" Mode|? Why |s |t ca||ed as "V" why not "U"? A|so te|| at what Stage 1est|ng shou|d be best to stared? A lL ls called v coz lL looks llke v Lhe deLalled v model ls shown below
1here ls no such sLage for whlch you walL Lo sLarL LesLlng 1esLlng sLarLs as soon as S8S documenL ls ready ?ou can ralse defecLs LhaL are presenL ln Lhe documenL lL's called verlflcaLlon
What |s d|fference |n between Cperat|ng System 2000 and CS k? A Wlndows 2000 and Wlndows x are essenLlally Lhe same operaLlng sysLem (known lnLernally as Wlndows n1 30 and Wlndows n1 31 respecLlvely) Pere are some conslderaLlons lf you're Lrylng Lo declde whlch verslon Lo use
Wlndows 2000 beneflLs
1) Wlndows 2000 has lower sysLem requlremenLs and has a slmpler lnLerface (no SLyles" Lo mess wlLh) 2) Wlndows 2000 ls sllghLly less expenslve and has no producL acLlvaLlon 3) Wlndows 2000 has been ouL for a whlle and mosL of Lhe common problems and securlLy holes have been uncovered and flxed 4) 1hlrdparLy sofLware and hardware producLs LhaL aren'L yeL xcompaLlble may be compaLlble wlLh Wlndows 2000 check Lhe manufacLurers of your devlces and appllcaLlons for x supporL before you upgrade
Wlndows x beneflLs
1) Wlndows x ls somewhaL fasLer Lhan Wlndows 2000 assumlng you have a fasL processor and Lons of memory (alLhough lL wlll run flne wlLh a 300 MPz enLlum ll and 128M8 of 8AM) 2) 1he new Wlndows x lnLerface ls more cheerful and colorful Lhan earller verslons alLhough Lhe less carLoon Classlc" lnLerface can sLlll be used lf deslred 3 Wlndows x has more bells and whlsLles such as Lhe Wlndows Movle Maker bullLln Cu wrlLer supporL Lhe lnLerneL ConnecLlon llrewall and 8emoLe ueskLop ConnecLlon 4) Wlndows x has beLLer supporL for games and comes wlLh more games Lhan Wlndows 2000 3) ManufacLurers of exlsLlng hardware and sofLware producLs are more llkely Lo add Wlndows x compaLlblllLy now Lhan Wlndows 2000 compaLlblllLy
What |s bug ||fe cyc|e? A new when LesLer reporLs a defecL Cpen when developer accepLs LhaL lL ls a bug or lf Lhe developer re[ecLs Lhe defecL Lhen Lhe sLaLus ls Lurned lnLo 8e[ecLed" llxed when developer make changes Lo Lhe code Lo recLlfy Lhe bug Closed/8eopen when LesLer LesLs lL agaln lf Lhe expecLed resulL shown up lL ls Lurned lnLo Closed" and lf Lhe problem reslsLs agaln lL's 8eopen
What |s deferred status |n defect ||fe cyc|e? A ueferred sLaLus means Lhe developer accepLed Lhe bus buL lL ls scheduled Lo recLlfy ln Lhe nexL bulld
What |s smoke test? A 1esLlng Lhe appllcaLlon wheLher lL's performlng lLs baslc funcLlonallLy properly or noL so LhaL Lhe LesL Leam can go ahead wlLh Lhe appllcaLlon
Do you use any automat|on too| for smoke test|ng? A ueflnlLely can use
What |s Ver|f|cat|on and va||dat|on? A verlflcaLlon ls sLaLlc no code ls execuLed Say analysls of requlremenLs eLc valldaLlon ls dynamlc Code ls execuLed wlLh scenarlos presenL ln LesL cases
What |s test p|an and exp|a|n |ts contents? A 1esL plan ls a documenL whlch conLalns Lhe scope for LesLlng Lhe appllcaLlon and whaL Lo be LesLed when Lo be LesLed and who Lo LesL
Advantages of automat|on over manua| test|ng? A 1lme resource and Money
What |s ADhoc test|ng? A AdPoc means dolng someLhlng whlch ls noL planned
What |s mean by re|ease notes? A lL's a documenL released along wlLh Lhe producL whlch explalns abouL Lhe producL lL also conLalns abouL Lhe bugs LhaL are ln deferred sLaLus
Sca|ab|||ty test|ng comes under |n wh|ch too|? A ScalablllLy LesLlng comes under performance LesLlng Load LesLlng scalablllLy LesLlng boLh r same
What |s the d|fference between 8ug and Defect? A 8ug uevlaLlon from Lhe expecLed resulL uefecL roblem ln algorlLhm leads Lo fallure
A MlsLake ln code ls called Lrror
uue Lo Lrror ln codlng LesL englneers are geLLlng mlsmaLches ln appllcaLlon ls called defecL
lf defecL accepLed by developmenL Leam Lo solve ls called 8ug
What |s hot f|x? A A hoL flx ls a slngle cumulaLlve package LhaL lncludes one or more flles LhaL are used Lo address a problem ln a sofLware producL 1yplcally hoL flxes are made Lo address a speclflc cusLomer slLuaLlon and may noL be dlsLrlbuLed ouLslde Lhe cusLomer organlzaLlon
8ug found aL Lhe cusLomer place whlch has hlgh prlorlLy
What |s the d|fference between funct|ona| test cases and compat|b|||ty test cases? A 1here are no 1esL Cases for CompaLlblllLy 1esLlng ln CompaLlblllLy 1esLlng we are 1esLlng an appllcaLlon ln dlfferenL Pardware and sofLware lf lL ls wrong plz leL me know
What |s Ac|d 1est|ng? A AClu Means AClu LesLlng ls relaLed Lo LesLlng a LransacLlon AALomlclLy CConslsLenL llsolaLlon uuurable
MosLly Lhls wlll be done daLabase LesLlng
What |s the ma|n use of prepar|ng a traceab|||ty matr|x? A 1o Cross verlfy Lhe prepared LesL cases and LesL scrlpLs wlLh user requlremenLs
1o monlLor Lhe changes enhance occurred durlng Lhe developmenL of Lhe pro[ecL
1raceablllLy maLrlx ls prepared ln order Lo cross check Lhe LesL cases deslgned agalnsL each requlremenL hence glvlng an opporLunlLy Lo verlfy LhaL all Lhe requlremenLs are covered ln LesLlng Lhe appllcaLlon
SofLware 1esLlng lnLervlew CuesLlons arL 4
If we have no SkS 8kS but we have test cases does u execute the test cases b||nd|y or do u fo||ow any other process? A 1esL case would have deLall sLeps of whaL Lhe appllcaLlon ls supposed Lo do SC 1) luncLlonallLy of appllcaLlon ls known
2) ln addlLlon you can refer Lo 8ackend l mean look lnLo Lhe uaLabase 1o galn more knowledge of Lhe appllcaLlon
now to execute test case? A 1here are Lwo ways 1 Manual 8unner 1ool for manual execuLlon and updaLlng of LesL sLaLus 2 AuLomaLed LesL case execuLlon by speclfylng PosL name and oLher auLomaLlon perLalnlng deLalls
D|fference between re test|ng and regress|on test|ng?
A 8eLesLlng
8eexecuLlon of LesL cases on same appllcaLlon bulld wlLh dlfferenL lnpuL values ls reLesLlng
8egresslon 1esLlng
8eexecuLlon of LesL cases on modlfles form of bulld ls called regresslon LesLlng
What |s the d|fference between bug |og and defect track|ng? A 8ug log ls a documenL whlch malnLalns Lhe lnformaLlon of Lhe bug where as bug Lracklng ls Lhe process
Who w||| change the 8ug Status as D|ffered? A 8ug wlll be ln open sLaLus whlle developer ls worklng on lL llxed afLer developer compleLes hls work lf lL ls noL flxed properly Lhe LesLer puLs lL ln reopen AfLer flxlng Lhe bug properly lL ls ln closed sLaLe
ueveloper
what |s smoke test|ng and user |nterface test|ng ?
A S1 Smoke LesLlng ls nonexhausLlve sofLware LesLlng as perLalnlng LhaL Lhe mosL cruclal funcLlons of a program work buL noL boLherlng wlLh flner deLalls 1he Lerm comes Lo sofLware LesLlng from a slmllarly baslc Lype of hardware LesLlng
ul1 l dld a blL or 8 n u on Lhls some says lL's noLhlng buL usablllLy LesLlng 1esLlng Lo deLermlne Lhe ease wlLh whlch a user can learn Lo operaLe lnpuL and lnLerpreL ouLpuLs of a sysLem or componenL
Smoke LesLlng ls noLhlng buL Lo check wheLher baslc funcLlonallLy of Lhe bulld ls sLable or noL? le lf lL possesses 70 of Lhe funcLlonallLy we say bulld ls sLable user lnLerface LesLlng We check all Lhe flelds wheLher Lhey are exlsLlng or noL as per Lhe formaL we check spelllng graphlc fonL slzes everyLhlng ln Lhe wlndow presenL or noL|
what |s bug defect |ssue error?
A 8ug 8ug ls ldenLlfled by Lhe LesLer uefecL Whenever Lhe pro[ecL ls recelved for Lhe analysls phase may be some requlremenL mlss Lo geL or undersLand mosL of Lhe Llme uefecL lLself come wlLh Lhe pro[ecL (when lL comes) lssue CllenL slLe error mosL of Lhe Llme Lrror When anyLhlng ls happened wrong ln Lhe pro[ecL from Lhe developmenL slde le called as Lhe error mosL of Lhe Llme Lhls knows by Lhe developer
8ug a faulL or defecL ln a sysLem or machlne
uefecL an lmperfecLlon ln a devlce or machlne
lssue An lssue ls a ma[or problem LhaL wlll lmpede Lhe progress of Lhe pro[ecL and cannoL be resolved by Lhe pro[ecL manager and pro[ecL Leam wlLhouL ouLslde help
Lrror Lrror ls Lhe devlaLlon of a measuremenL observaLlon or calculaLlon from Lhe LruLh
What |s the d|ff b]w funct|ona| test|ng and |ntegrat|on test|ng? A funcLlonal LesLlng ls LesLlng Lhe whole funcLlonallLy of Lhe sysLem or Lhe appllcaLlon wheLher lL ls meeLlng Lhe funcLlonal speclflcaLlons
lnLegraLlon LesLlng means LesLlng Lhe funcLlonallLy of lnLegraLed module when Lwo lndlvldual modules are lnLegraLed for Lhls we use Lopdown approach and boLLom up approach
what type of test|ng u perform |n organ|zat|on wh||e u do System 1est|ng g|ve c|ear|y?
A luncLlonal LesLlng user lnLerface LesLlng usablllLy LesLlng CompaLlblllLy LesLlng Model based LesLlng Lrror exlL LesLlng user help LesLlng SecurlLy LesLlng CapaclLy LesLlng erformance LesLlng SanlLy LesLlng 8egresslon LesLlng 8ellablllLy LesLlng 8ecovery LesLlng lnsLallaLlon LesLlng MalnLenance LesLlng AccesslblllLy LesLlng lncludlng compllance wlLh Amerlcans wlLh ulsablllLles AcL of 1990 SecLlon 308 AmendmenL Lo Lhe 8ehablllLaLlon AcL of 1973 Web AccesslblllLy lnlLlaLlve (WAl) of Lhe World Wlde Web ConsorLlum (W3C)
What |s the ma|n use of prepar|ng 1raceab|||ty matr|x and exp|a|n the rea| t|me usage?
A A LraceablllLy maLrlx ls creaLed by assoclaLlng requlremenLs wlLh Lhe work producLs LhaL saLlsfy Lhem 1esLs are assoclaLed wlLh Lhe requlremenLs on whlch Lhey are based and Lhe producL LesLed Lo meeL Lhe requlremenL
A LraceablllLy maLrlx ls a reporL from Lhe requlremenLs daLabase or reposlLory
now can u do the fo||ow|ng 1) Usab|||ty test|ng 2) sca|ab|||ty 1est|ng
A u1 1esLlng Lhe ease wlLh whlch users can learn and use a producL
S1 lL's a Web 1esLlng defn allows web slLe capablllLy lmprovemenL
1 1esLlng Lo deLermlne wheLher Lhe sysLem/sofLware meeLs Lhe speclfled porLablllLy requlremenLs
What does u mean by os|t|ve and Negat|ve test|ng what |s the d|ff's between them Can anyone exp|a|n w|th an examp|e?
A oslLlve 1esLlng 1esLlng Lhe appllcaLlon funcLlonallLy wlLh valld lnpuLs and verlfylng LhaL ouLpuL ls correcL negaLlve LesLlng 1esLlng Lhe appllcaLlon funcLlonallLy wlLh lnvalld lnpuLs and verlfylng Lhe ouLpuL
ulfference ls noLhlng buL how Lhe appllcaLlon behaves when we enLer some lnvalld lnpuLs suppose lf lL accepLs lnvalld lnpuL Lhe appllcaLlon luncLlonallLy ls wrong
oslLlve LesL LesLlng almed Lo show LhaL s/w work le wlLh valld lnpuLs 1hls ls also called as LesL Lo pass' negaLlve LesLlng LesLlng almed aL showlng s/w doesn'L work Whlch ls also know as 'LesL Lo fall" 8vA ls Lhe besL example of ve LesLlng
what |s change request how u use |t?
A Change 8equesL ls a aLLrlbuLe or parL of uefecL Llfe Cycle
now when u as a LesLer flnds a defecL n reporL Lo ur uLhe ln Lurn lnforms Lhe uevelopmenL 1eam 1he u1 says lL's noL a defecL lL's an exLra lmplemenLaLlon or says noL parL of req'menL lLs newscasL has Lo pay
Pere Lhe sLaLus ln ur defecL reporL would be Change 8equesL
l Lhlnk change requesL conLrolled by change requesL conLrol board (CC8) lf any changes requlred by cllenL afLer we sLarL Lhe pro[ecL lL has Lo come Lhru LhaL CC8 and Lhey have Lo approve lL CC8 goL full rlghLs Lo accepL or re[ecL based on Lhe pro[ecL schedule and cosL
What |s r|sk ana|ys|s what type of r|sk ana|ys|s u d|d |n u r pro[ect?
A 8lsk Analysls A sysLemaLlc use of avallable lnformaLlon Lo deLermlne how ofLen speclfled evenLs and unspeclfled evenLs may occur and Lhe magnlLude of Lhelr llkely consequences
C8
procedure Lo ldenLlfy LhreaLs vulnerablllLles analyze Lhem Lo ascerLaln Lhe exposures and hlghllghL how Lhe lmpacL can be ellmlnaLed or reduced
A A page ls rarely accessed or some acLlvlLy ls performed rarely buL LhaL Lhlng ouLpuLs some lmporLanL uaLa lncorrecLly or corrupLs Lhe daLa Lhls wlll be a bug of P severlLy L prlorlLy
If pro[ect wants to re|ease |n 3months what type of k|sk ana|ys|s u do |n 1est p|an?
A use rlsk analysls Lo deLermlne where LesLlng should be focused Slnce lL's rarely posslble Lo LesL every posslble aspecL of an appllcaLlon every posslble comblnaLlon of evenLs every dependency or everyLhlng LhaL could go wrong rlsk analysls ls approprlaLe Lo mosL sofLware developmenL pro[ecLs 1hls requlres [udgmenL skllls common sense and experlence (lf warranLed formal meLhods are also avallable) ConslderaLlons can lnclude
- Whlch funcLlonallLy ls mosL lmporLanL Lo Lhe pro[ecL's lnLended purpose? - Whlch funcLlonallLy ls mosL vlslble Lo Lhe user? - Whlch funcLlonallLy has Lhe largesL safeLy lmpacL? - Whlch funcLlonallLy has Lhe largesL flnanclal lmpacL on users? - Whlch aspecLs of Lhe appllcaLlon are mosL lmporLanL Lo Lhe cusLomer? - Whlch aspecLs of Lhe appllcaLlon can be LesLed early ln Lhe developmenL cycle? - Whlch parLs of Lhe code are mosL complex and Lhus mosL sub[ecL Lo errors? - Whlch parLs of Lhe appllcaLlon were developed ln rush or panlc mode? - Whlch aspecLs of slmllar/relaLed prevlous pro[ecLs caused problems? - Whlch aspecLs of slmllar/relaLed prevlous pro[ecLs had large malnLenance expenses? - Whlch parLs of Lhe requlremenLs and deslgn are unclear or poorly LhoughL ouL? - WhaL do Lhe developers Lhlnk are Lhe hlghesLrlsk aspecLs of Lhe appllcaLlon? - WhaL klnds of problems would cause Lhe worsL publlclLy? - WhaL klnds of problems would cause Lhe mosL cusLomer servlce complalnLs? - WhaL klnds of LesLs could easlly cover mulLlple funcLlonallLles? - Whlch LesLs wlll have Lhe besL hlghrlskcoverage Lo Llmerequlred raLlo
1est cases for IL 60 ?
A 1esL cases for lL 60 le lnLerneL Lxplorer 60 1)llrsL l go for Lhe lnsLallaLlon slde means LhaL + ls lL worklng wlLh all verslons of Wlndows neLscape or oLher sofLware's ln oLher words we can say LhaL lL musL check wlLh all hardware and sofLware parLs 2) Secondly go for Lhe 1exL arL means LhaL all Lhe 1exL parL appears ln frequenL and smooLh manner 3) 1hlrdly go for Lhe lmages arL means LhaL all Lhe lmages appears ln frequenL and smooLh manner 4) u8L musL run ln a beLLer way 3) Suppose Some oLher language used on lL Lhen u8L Lake Lhe CLher CharacLers CLher Lhan normal CharacLers 6)ls lL worklng wlLh Cookles frequenLly or noL 7) ls lL Concernlng wlLh dlfferenL scrlpL llke !ScrlpL and v8ScrlpL 8) P1ML Code work on LhaL or noL 9) 1roubleshooLlng works or noL 10) All Lhe 1ool bars are work wlLh lL or noL 11) lf age has Some Llnks Lhan how much ls Lhe Max and Mln LlmlL for LhaL 12) 1esL for lnsLalllng lnLerneL Lxplorer 6 wlLh norLon roLecLed 8ecycle 8ln enabled 13) ls lL worklng wlLh Lhe unlnsLallaLlon rocess 14) LasL buL noL Lhe leasL LesL for Lhe SecurlLy SysLem for Lhe lL 60
Where you |nvo|ve |n test|ng ||fe cyc|e what type of test you perform ?
A Cenerally LesL englneers lnvolved from enLlre LesL llfe cycle le LesL plan LesL case preparaLlon execuLlon reporLlng Cenerally sysLem LesLlng regresslon LesLlng adhoc LesLlng eLc
what |s 1est|ng env|ronment |n your company means how test|ng process start ?
A LesLlng process ls golng as follows quallLy assurance unlL quallLy assurance manager LesL lead LesL englneer
who prepares the use cases?
A ln Any company excepL Lhe small company 8uslness analysL prepares Lhe use cases 8uL ln small company 8uslness analysL prepares along wlLh Leam lead
What methodo|og|es have you used to deve|op test cases?
A generally LesL englneers uses 4 Lypes of meLhodologles 1 8oundary value analysls 2Lqulvalence parLlLlon 3Lrror guesslng 4cause effecL graphlng
Why we ca|| |t as a regress|on test nor retest?
A lf we LesL wheLher defecL ls closed or noL le 8eLesLlng 8uL here we are checklng Lhe lmpacL also regresslon means repeaLed Llmes
Is automated test|ng better than manua| test|ng If so why?
A AuLomaLed LesLlng and manual LesLlng have advanLages as well as dlsadvanLages AdvanLages lL lncrease Lhe efflclency of LesLlng process speed ln process rellable llexlble dlsadvanLage's 1ools should have compaLlblllLy wlLh our developmenL or deploymenL Lools needs loL of Llme lnlLlally lf Lhe requlremenLs are changlng conLlnuously AuLomaLlon ls noL sulLable Manual lf Lhe requlremenLs are changlng conLlnuously Manual ls sulLable Cnce Lhe bulld ls sLable wlLh manual LesLlng Lhen only we go 4 auLomaLlon ulsadvanLages lL needs loL of Llme We can noL do some Lype of LesLlng manually Lg erformances
what |s the exact d|fference between a product and a pro[ect g|ve an examp|e ?
A ro[ecL ueveloped for parLlcular cllenL requlremenLs are deflned by cllenL roducL developed for markeL 8equlremenLs are deflned by company lLself by conducLlng markeL survey Lxample ro[ecL Lhe shlrL whlch we are lnLeresLed sLlLchlng wlLh Lallor as per our speclflcaLlons ls pro[ecL roducL Lxample ls 8eady made ShlrL" where Lhe parLlcular company wlll lmaglne parLlcular measuremenLs Lhey made Lhe producL Malnframes ls a producL roducL has many mo of verslons buL pro[ecL has fewer verslons le depends upon change requesL and enhancemenLs
Def|ne 8ra|n Storm|ng and Cause Lffect Graph|ng? W|th Lg?
A 8S A learnlng Lechnlque lnvolvlng open group dlscusslon lnLended Lo expand Lhe range of avallable ldeas C8 A meeLlng Lo generaLe creaLlve ldeas AL LSl AdverLlslng dally weekly and blmonLhly bralnsLormlng sesslons are held by varlous work groups wlLhln Lhe flrm Cur monLhly l ower bralnsLormlng meeLlng ls aLLended by Lhe enLlre agency sLaff C8 8ralnsLormlng ls a hlghly sLrucLured process Lo help generaLe ldeas lL ls based on Lhe prlnclple LhaL you cannoL generaLe and evaluaLe ldeas aL Lhe same Llme 1o use bralnsLormlng you musL flrsL galn agreemenL from Lhe group Lo Lry bralnsLormlng for a flxed lnLerval (eg slx mlnuLes)
CLC A LesLlng Lechnlque LhaL alds ln selecLlng ln a sysLemaLlc way a hlghyleld seL of LesL cases LhaL loglcally relaLes causes Lo effecLs Lo produce LesL cases lL has a beneflclal slde effecL ln polnLlng ouL lncompleLeness and amblgulLles ln speclflcaLlons
Actua||y by us|ng sever|ty u shou|d know wh|ch one u need to so|ve so what |s the need of pr|or|ty?
A l guess severlLy reflecLs Lhe serlousness of Lhe bug where as prlorlLy refers Lo whlch bug should recLlfy flrsL of course lf Lhe severlLy ls hlgh Lhe same case ls wlLh prlorlLy ln normal
severlLy declded by Lhe LesLer where as prlorlLy declded by developers whlch one need Lo solve flrsL knows Lhrough prlorlLy noL wlLh severlLy how serlous of Lhe bug knows Lhrough severlLy
severlLy ls noLhlng lmpacL of LhaL bug on Lhe appllcaLlon rlorlLy ls noLhlng buL lmporLance Lo resolve Lhe bug yeah of course by looklng severlLy we can [udge buL someLlmes hlgh severlLy bug doesn'L have hlgh prlorlLy AL Lhe same Llme Plgh prlorlLy bug don'L have hlgh severlLy So we need boLh severlLy and prlorlLy
What do u do |f the bug that u found |s not accepted by the deve|oper and he |s say|ng |ts not reproduc|b|e Note 1he deve|oper |s |n the on s|te |ocat|on ?
A once agaln we wlll check LhaL condlLlon wlLh all reasons Lhen we wlll aLLach screen shoLs wlLh sLrong reasons Lhen we wlll explaln Lo Lhe pro[ecL manager and also explaln Lo Lhe cllenL when Lhey conLacL us
SomeLlmes bug ls noL reproduclble lL ls because of dlfferenL envlronmenL suppose developmenL Leam uslng oLher envlronmenL and you are uslng dlfferenL envlronmenL aL Lhls slLuaLlon Lhere ls chance of bug noL reproduclng AL Lhls slLuaLlon please check Lhe envlronmenL ln Lhe base llne documenLs LhaL ls funcLlonal documenLs lf Lhe envlronmenL whlch we r uslng ls correcL we wlll ralse lL as defecL We wlll Lake screen shoLs and sends Lhem wlLh LesL procedure also
what |s the d|fference between three t|er and two t|er app||cat|on?
A CllenL server ls a 2Ller appllcaLlon ln Lhls fronL end or cllenL ls connecLed Lo 'uaLa base server' Lhrough 'uaLa Source name' fronL end ls Lhe monlLorlng level
Web based archlLecLure ls a 3Ller appllcaLlon ln Lhls browser ls connecLed Lo web server Lhrough 1C/l and web server ls connecLed Lo uaLa base server browser ls Lhe monlLorlng level ln general 8lack box LesLers are concenLraLlng on monlLorlng level of any Lype of appllcaLlon
All Lhe cllenL server appllcaLlons are 2 Ller archlLecLures Pere ln Lhese archlLecLure all Lhe 8uslness Loglc" ls sLored ln cllenLs and uaLa" ls sLored ln Servers So lf user requesL anyLhlng buslness loglc wlll b performed aL cllenL and Lhe daLa ls reLrleved from Server(u8 Server) Pere Lhe problem ls lf any buslness loglc changes Lhen we need Lo change Lhe loglc aL each any every cllenL 1he besL ex ls Lake a super markeL l have branches ln Lhe clLy AL each branch l have cllenLs so buslness loglc ls sLored ln cllenLs buL Lhe acLual daLa ls sLore ln servers lf assume l wanL Lo glve some dlscounL on some lLems so l need Lo change Lhe buslness loglc lor Lhls l need Lo go Lo each branch and need Lo change Lhe buslness loglc aL each cllenL 1hls Lhe dlsadvanLage of CllenL/Server archlLecLure
So 3Ller archlLecLure came lnLo plcLure
Pere 8uslness Loglc ls sLored ln one Server and all Lhe cllenLs are dumb Lermlnals lf user requesLs anyLhlng Lhe requesL flrsL senL Lo server Lhe server wlll brlng Lhe daLa from u8 Sever and send lL Lo cllenLs 1hls ls Lhe flow for 3Ller archlLecLure
Assume for Lhe above Lx lf l wanL Lo glve some dlscounL all my buslness loglc ls Lhere ln Server So l need Lo change aL one place noL aL each cllenL 1hls ls Lhe maln advanLage of 3 Ller archlLecLure
SofLware 1esLlng lnLervlew CuesLlons arL 6
What |s Impact ana|ys|s? now to do |mpact ana|ys|s |n yr pro[ect?
A lmpacL analysls means when we r dolng regresslng LesLlng aL LhaL Llme we r checklng LhaL Lhe bug flxes r worklng properly and by flxlng Lhese bug oLher componenLs are worklng as per Lhelr requlremenLs r Lhey goL dlsLurbed
nCW 1C 1LS1 A WL8SI1L 8 MANUAL 1LS1ING?
A Web 1esLlng uurlng LesLlng Lhe webslLes Lhe followlng scenarlos should be consldered luncLlonallLy erformance usablllLy Server slde lnLerface CllenL slde compaLlblllLy SecurlLy
Iunct|ona||ty ln LesLlng Lhe funcLlonallLy of Lhe web slLes Lhe followlng should be LesLed Llnks lnLernal llnks LxLernal llnks Mall llnks 8roken llnks lorms lleld valldaLlon luncLlonal charL Lrror message for wrong lnpuL CpLlonal and mandaLory flelds uaLabase 1esLlng wlll be done on Lhe daLabase lnLegrlLy Cookles 1esLlng wlll be done on Lhe cllenL sysLem slde on Lhe Lemporary lnLerneL flles
erformance erformance LesLlng can be applled Lo undersLand Lhe web slLe's scalablllLy or Lo benchmark Lhe performance ln Lhe envlronmenL of Lhlrd parLy producLs such as servers and mlddle ware for poLenLlal purchase
Connect|on speed 1esLed over varlous neLworks llke ulal up lSun eLc
Load WhaL ls Lhe no of users per Llme? Check for peak loads how sysLem behaves Large amounL of daLa accessed by user
Stress ConLlnuous load erformance of memory cpu flle handllng eLc
Usab|||ty usablllLy LesLlng ls Lhe process by whlch Lhe humancompuLer lnLeracLlon characLerlsLlcs of a sysLem are measured and weaknesses are ldenLlfled for correcLlon usablllLy can be deflned as Lhe degree Lo whlch a glven plece of sofLware asslsLs Lhe person slLLlng aL Lhe keyboard Lo accompllsh a Lask as opposed Lo becomlng an addlLlonal lmpedlmenL Lo such accompllshmenL 1he broad goal of usable sysLems ls ofLen assessed uslng several
Cr|ter|a Lase of learnlng navlgaLlon Sub[ecLlve user saLlsfacLlon Ceneral appearance
Server s|de |nterface ln web LesLlng Lhe server slde lnLerface should be LesLed 1hls ls done by verlfy LhaL communlcaLlon ls done properly CompaLlblllLy of server wlLh sofLware hardware neLwork and daLabase should be LesLed 1he cllenL slde compaLlblllLy ls also LesLed ln varlous plaLforms uslng varlous browsers eLc
Secur|ty 1he prlmary reason for LesLlng Lhe securlLy of a web ls Lo ldenLlfy poLenLlal vulnerablllLles and subsequenLly repalr Lhem 1he followlng Lypes of LesLlng are descrlbed ln Lhls secLlon neLwork Scannlng vulnerablllLy Scannlng assword Cracklng Log 8evlew lnLegrlLy Checkers vlrus ueLecLlon
erformance 1est|ng erformance LesLlng ls a rlgorous usablllLy evaluaLlon of a worklng sysLem under reallsLlc condlLlons Lo ldenLlfy usablllLy problems and Lo compare measures such as success raLe Lask Llme and user saLlsfacLlon wlLh requlremenLs 1he goal of performance LesLlng ls noL Lo flnd bugs buL Lo ellmlnaLe boLLlenecks and esLabllsh a basellne for fuLure regresslon LesLlng
1o conducL performance LesLlng ls Lo engage ln a carefully conLrolled process of measuremenL and analysls ldeally Lhe sofLware under LesL ls already sLable enough so LhaL Lhls process can proceed smooLhly A clearly deflned seL of expecLaLlons ls essenLlal for meanlngful performance LesLlng lor example for a Web appllcaLlon you need Lo know aL leasL Lwo Lhlngs expecLed load ln Lerms of concurrenL users or P11 connecLlons accepLable response Llme
Load test|ng Load LesLlng ls usually deflned as Lhe process of exerclslng Lhe sysLem under LesL by feedlng lL Lhe largesL Lasks lL can operaLe wlLh Load LesLlng ls someLlmes called volume LesLlng or longevlLy/endurance LesLlng Lxamples of volume LesLlng LesLlng a word processor by edlLlng a very large documenL LesLlng a prlnLer by sendlng lL a very large [ob LesLlng a mall server wlLh Lhousands of users mallboxes Lxamples of longevlLy/endurance LesLlng LesLlng a cllenLserver appllcaLlon by runnlng Lhe cllenL ln a loop agalnsL Lhe server over an exLended perlod of Llme
Goa|s of |oad test|ng Lxpose bugs LhaL do noL surface ln cursory LesLlng such as memory managemenL bugs memory leaks buffer overflows eLc Lnsure LhaL Lhe appllcaLlon meeLs Lhe performance basellne esLabllshed durlng erformance LesLlng 1hls ls done by runnlng regresslon LesLs agalnsL Lhe appllcaLlon aL a speclfled maxlmum load AlLhough performance LesLlng and load LesLlng can seen slmllar Lhelr goals are dlfferenL Cn one hand performance LesLlng uses load LesLlng Lechnlques and Lools for measuremenL and benchmarklng purposes and uses varlous load levels whereas load LesLlng operaLes aL a predeflned load level Lhe hlghesL load LhaL Lhe sysLem can accepL whlle sLlll funcLlonlng properly
Stress test|ng SLress LesLlng ls a form of LesLlng LhaL ls used Lo deLermlne Lhe sLablllLy of a glven sysLem or enLlLy 1hls ls deslgned Lo LesL Lhe sofLware wlLh abnormal slLuaLlons SLress LesLlng aLLempLs Lo flnd Lhe llmlLs aL whlch Lhe sysLem wlll fall Lhrough abnormal quanLlLy or frequency of lnpuLs SLress LesLlng Lrles Lo break Lhe sysLem under LesL by overwhelmlng lLs resources or by Laklng resources away from lL (ln whlch case lL ls someLlmes called negaLlve LesLlng) 1he maln purpose behlnd Lhls madness ls Lo make sure LhaL Lhe sysLem falls and recovers gracefully Lhls quallLy ls known as recoverablllLy SLress LesLlng does noL break Lhe sysLem buL lnsLead lL allows observlng how Lhe sysLem reacLs Lo fallure SLress LesLlng observes for Lhe followlng uoes lL save lLs sLaLe or does lL crash suddenly? uoes lL [usL hang and freeze or does lL fall gracefully? ls lL able Lo recover from Lhe lasL good sLaLe on resLarL?
LLc
Compat|b|||ty 1est|ng A 1esLlng Lo ensure compaLlblllLy of an appllcaLlon or Web slLe wlLh dlfferenL browsers CS and hardware plaLforms ulfferenL verslons conflguraLlons dlsplay resoluLlons and lnLerneL connecL speeds all can lmpacL Lhe behavlor of Lhe producL and lnLroduce cosLly and embarrasslng bugs We LesL for compaLlblllLy uslng real LesL envlronmenLs 1haL ls LesLlng how wlll Lhe sysLem performs ln Lhe parLlcular sofLware hardware or neLwork envlronmenL CompaLlblllLy LesLlng can be performed manually or can be drlven by an auLomaLed funcLlonal or reg 1he purpose of compaLlblllLy LesLlng ls Lo reveal lssues relaLed Lo Lhe producL lnLeracLlon sesslon LesL sulLe wlLh oLher sofLware as well as hardware 1he producL compaLlblllLy ls evaluaLed by flrsL ldenLlfylng Lhe hardware/sofLware/browser componenLs LhaL Lhe producL ls deslgned Lo supporL 1hen a hardware/sofLware/browser maLrlx ls deslgned LhaL lndlcaLes Lhe conflguraLlons on whlch Lhe producL wlll be LesLed 1hen wlLh lnpuL from Lhe cllenL a LesLlng scrlpL ls deslgned LhaL wlll be sufflclenL Lo evaluaLe compaLlblllLy beLween Lhe producL and Lhe hardware/sofLware/browser maLrlx llnally Lhe scrlpL ls execuLed agalnsL Lhe maLrlx and any anomalles are lnvesLlgaLed Lo deLermlne exacLly where Lhe lncompaLlblllLy lles Some Lyplcal compaLlblllLy LesLs lnclude LesLlng your appllcaLlon Cn varlous cllenL hardware conflguraLlons uslng dlfferenL memory slzes and hard drlve space Cn varlous CperaLlng SysLems ln dlfferenL neLwork envlronmenLs WlLh dlfferenL prlnLers and perlpherals (le zlp drlves uS8's eLc)
wh|ch comes f|rst test strategy or test p|an?
A 1esL sLraLegy comes flrsL ans Lhls ls Lhe hlgh level documenL and approach for Lhe LesLlng sLarLs from LesL sLraLegy and Lhen based on Lhls Lhe LesL lead prepares Lhe LesL plan
what |s the d|fference between web based app||cat|on and c||ent server app||cat|on as a tester's po|nt of v|ew?
A Accordlng Lo 1esLer's olnL of vlew 1) Web 8ase AppllcaLlon (W8A)ls a 3 Ller appllcaLlon 8rowser 8ack end and Server CllenL server AppllcaLlon(CSA) ls a 2 Ller AppllcaLlon lronL Lnd 8ack end 2) ln Lhe W8A LesLer LesL for Lhe ScrlpL error llke [ava scrlpL error v8 scrlpL error eLc LhaL shown aL Lhe page ln Lhe CSA LesLer does noL LesL for any scrlpL error 3) 8ecause ln Lhe W8A once changes perform reflecL aL every machlne so LesLer has less work for LesL Whereas ln Lhe CSA every Llme appllcaLlon need Lo be lnsLall hence lL maybe posslble LhaL some machlne has some problem for LhaL Pardware LesLlng as well as sofLware LesLlng ls needed
What |s the s|gn|f|cance of do|ng kegress|on test|ng?
A 1o check for Lhe bug flxes And Lhls flx should noL dlsLurb oLher funcLlonallLy
1o Lnsure Lhe newly added funcLlonallLy or exlsLlng modlfled funcLlonallLy or developer flxed bug arlses any new bug or affecLlng any oLher slde effecL Lhls ls called regresslon LesL and ensure already ASSLu 1LS1 CASLS would noL arlse any new bug
What are the d|ff ways to check a date f|e|d |n a webs|te?
A 1here are dlfferenL ways llke 1) you can check Lhe fleld wldLh for mlnlmum and maxlmum 2) lf LhaL fleld only Lake Lhe numerlc value Lhen check lL'll only Lake numerlc no oLher Lype 3) lf lL Lakes Lhe daLe or Llme Lhen check for oLher 4) Same way llke numerlc you can check lL for Lhe CharacLer Alpha numerlc and all 3) And Lhe mosL lmporLanL lf you cllck and hlL Lhe enLer key Lhen some Llme page e may glve Lhe error of !avaScrlpL LhaL ls Lhe blg faulL on Lhe page 6) Check Lhe fleld for Lhe null value L1C
1he daLe fleld we can check ln dlfferenL ways oslLlve LesLlng flrsL we enLer Lhe daLe ln glven formaL
negaLlve 1esLlng We enLer Lhe daLe ln lnvalld formaL suppose lf we enLer daLe llke 30/02/2006 lL should dlsplay some error message and also we use Lo check Lhe numerlc or LexL
1est|ng 1echn|ques
- 8lack 8ox 1esLlng - WhlLe 8ox 1esLlng - 8egresslon 1esLlng - 1hese prlnclples Lechnlques can be applled Lo any Lype of LesLlng
8|ack 8ox 1est|ng
1esLlng of a funcLlon wlLhouL knowlng lnLernal sLrucLure of Lhe program
Wh|te 8ox 1est|ng
1esLlng of a funcLlon wlLh knowlng lnLernal sLrucLure of Lhe program
kegress|on 1est|ng
1o ensure LhaL Lhe code changes have noL had an adverse affecL Lo Lhe oLher modules or on exlsLlng funcLlons
Iunct|ona| 1est|ng
SLudy S8SC ldenLlfy unlL luncLlonsC lor each unlL funcLlonC - 1ake each lnpuL funcLlon - ldenLlfy Lqulvalence class - lorm 1esL cases - lorm 1esL cases for boundary values - lrom 1esL cases for Lrror Cuesslng lorm unlL funcLlon v/s 1esL cases Cross 8eference MaLrlxC C llnd Lhe coverage
Un|t 1est|ng
1he mosL 'mlcro' scale of LesLlng Lo LesL parLlcular funcLlons or code modules 1yplcally done by Lhe programmer and noL by LesLers - unlL smallesL LesLable plece of sofLware - A unlL can be complled/ assembled/ llnked/ loaded and puL under a LesL harness - unlL LesLlng done Lo show LhaL Lhe unlL does noL saLlsfy Lhe funcLlonal speclflcaLlon and/ or lLs lmplemenLed sLrucLure does noL maLch Lhe lnLended deslgn sLrucLure
Integrat|on 1est|ng
lnLegraLlon ls a sysLemaLlc approach Lo bulld Lhe compleLe sofLware sLrucLure speclfled ln Lhe deslgn from unlLLesLed modules 1here are Lwo ways lnLegraLlon performed lL ls called reLesL and roLesL
1 reLesL Lhe LesLlng performed ln Module developmenL area ls called reLesL 1he re LesL ls requlred only lf Lhe developmenL ls done ln module developmenL area 2 roLesL 1he lnLegraLlon LesLlng performed ln basellne ls called proLesL 1he developmenL of a release wlll be scheduled such LhaL Lhe cusLomer can break down lnLo smaller lnLernal releases
A|pha test|ng
1esLlng of an appllcaLlon when developmenL ls nearlng compleLlon mlnor deslgn changes may sLlll be made as a resulL of such LesLlng 1yplcally done by endusers or oLhers noL by programmers or LesLers
8eta test|ng
1esLlng when developmenL and LesLlng are essenLlally compleLed and flnal bugs and problems need Lo be found before flnal release 1yplcally done by endusers or oLhers noL by programmers or
System 1est|ng
- A sysLem ls Lhe blg componenL - SysLem LesLlng ls almed aL reveallng bugs LhaL cannoL be aLLrlbuLed Lo a componenL as such Lo lnconslsLencles beLween componenLs or planned lnLeracLlons beLween componenLs - Concern lssues behavlors LhaL can only be exposed by LesLlng Lhe enLlre lnLegraLed sysLem (eg performance securlLy recovery)
Vo|ume 1est|ng
1he purpose of volume 1esLlng ls Lo flnd weaknesses ln Lhe sysLem wlLh respecL Lo lLs handllng of large amounLs of daLa durlng shorL Llme perlods lor example Lhls klnd of LesLlng ensures LhaL Lhe sysLem wlll process daLa across physlcal and loglcal boundarles such as across servers and across dlsk parLlLlons on one server
Stress test|ng
1hls refers Lo LesLlng sysLem funcLlonallLy whlle Lhe sysLem ls under unusually heavy or peak load lL's slmllar Lo Lhe valldaLlon LesLlng menLloned prevlously buL ls carrled ouL ln a hlgh sLress" envlronmenL 1hls requlres LhaL you make some predlcLlons abouL expecLed load levels of your Web slLe
Usab|||ty test|ng
usablllLy means LhaL sysLems are easy and fasL Lo learn efflclenL Lo use easy Lo remember cause no operaLlng errors and offer a hlgh degree of saLlsfacLlon for Lhe user usablllLy means brlnglng Lhe usage perspecLlve lnLo focus Lhe slde Lowards Lhe user
Secur|ty test|ng
lf your slLe requlres flrewalls encrypLlon user auLhenLlcaLlon flnanclal LransacLlons or access Lo daLabases wlLh senslLlve daLa you may need Lo LesL Lhese and also LesL your slLe's overall proLecLlon agalnsL unauLhorlzed lnLernal or exLernal access
G|ass 8ox 1est|ng
1esL case selecLlon LhaL ls based on an analysls of Lhe lnLernal sLrucLure of Lhe componenL 1esLlng by looklng only aL Lhe code Some Llmes also called Code 8ased 1esLlng" Cbvlously you need Lo be a programmer and you need Lo have Lhe source code Lo do Lhls
1est Case
A seL of lnpuLs execuLlon precondlLlons and expecLed ouLcomes developed for a parLlcular ob[ecLlve such as Lo exerclse a parLlcular program paLh or Lo verlfy compllance wlLh a speclflc requlremenL
Cperat|ona| 1est|ng
1esLlng conducLed Lo evaluaLe a sysLem or componenL ln lLs operaLlonal envlronmenL
Va||dat|on
ueLermlnaLlon of Lhe correcLness of Lhe producLs of sofLware developmenL wlLh respecL Lo Lhe user needs and requlremenLs
Ver|f|cat|on
1he process of evaluaLlng a sysLem or componenL Lo deLermlne wheLher Lhe producLs of Lhe glven developmenL phase saLlsfy Lhe condlLlons lmposed aL Lhe sLarL of LhaL phase
Contro| I|ow
An absLracL represenLaLlon of all posslble sequences of evenLs ln a program's execuLlon
CAS1 Acronym for compuLeralded sofLware LesLlng
Metr|cs Ways Lo measure eg Llme cosL cusLomer saLlsfacLlon quallLy
1esL lan ueslgn 1esL Cases LxecuLe 1esLs LvaluaLe 8esulLs uocumenL 1esL 8esulLs Casual Analysls/ reparaLlon of valldaLlon 8eporLs 8egresslon 1esLlng / lollow up on reporLed bugs
Manua| 1est|ng rocess
rocess ls a roadmap Lo develop Lhe pro[ecL ls conslsLs a number of sequenLlal sLeps
lL ls a documenL whlch descrlbes Lhe LesLlng envlronmenL purpose scope ob[ecLlves LesL sLraLegy schedules mlle sLones LesLlng Lool roles and responslblllLles rlsks Lralnlng sLafflng and who ls golng Lo LesL Lhe appllcaLlon whaL Lype of LesLs should be performed and how lL wlll Lrack Lhe defecLs
ln Lhls phase we execuLe Lhe documenLs Lhose are prepared ln LesL developmenL phase
Ana|yse resu|t
Cnce execuLed documenLs wlll geL resulLs elLher pass or fall we need Lo analyze Lhe resulLs durlng Lhls phase
Defect 1rack|ng
When ever we geL defecL on Lhe appllcaLlon we need Lo prepare Lhe bug reporL flle and forwards Lo 1esL 1eam Lead and uev 1eam 1he uev 1eam wlll flx Lhe bug Agaln we have Lo LesL Lhe appllcaLlon 1hls cycle repeaLs Llll we geL Lhe sofLware wlLh our defecLs
Summarlze 8eporLs
- 1esL 8eporLs - 8ug 8eporLs - 1esL uocumenLaLlon
Manual 1esLlng lnLervlew CuesLlons1
What makes a good test eng|neer?
A good LesL englneer has a 'LesL Lo break' aLLlLude an ablllLy Lo Lake Lhe polnL of vlew of Lhe cusLomer a sLrong deslre for quallLy and an aLLenLlon Lo deLall 1acL and dlplomacy are useful ln malnLalnlng a cooperaLlve relaLlonshlp wlLh developers and an ablllLy Lo communlcaLe wlLh boLh Lechnlcal (developers) and nonLechnlcal (cusLomers managemenL) people ls useful revlous sofLware developmenL experlence can be helpful as lL provldes a deeper undersLandlng of Lhe sofLware developmenL process glves Lhe LesLer an appreclaLlon for Lhe developers' polnL of vlew and reduce Lhe learnlng curve ln auLomaLed LesL Lool programmlng !udgmenL skllls are needed Lo assess hlghrlsk areas of an appllcaLlon on whlch Lo focus LesLlng efforLs when Llme ls llmlLed
What makes a good Software A eng|neer?
1he same quallLles a good LesLer has are useful for a CA englneer AddlLlonally Lhey musL be able Lo undersLand Lhe enLlre sofLware developmenL process and how lL can flL lnLo Lhe buslness approach and goals of Lhe organlzaLlon CommunlcaLlon skllls and Lhe ablllLy Lo undersLand varlous sldes of lssues are lmporLanL ln organlzaLlons ln Lhe early sLages of lmplemenLlng CA processes paLlence and dlplomacy are especlally needed An ablllLy Lo flnd problems as well as Lo see 'whaL's mlsslng' ls lmporLanL for lnspecLlons and revlews
What makes a good A or 1est manager?
A good CA LesL or CA/1esL(comblned) manager should - be famlllar wlLh Lhe sofLware developmenL process - be able Lo malnLaln enLhuslasm of Lhelr Leam and promoLe a poslLlve aLmosphere desplLe - whaL ls a somewhaL 'negaLlve' process (eg looklng for or prevenLlng problems) - be able Lo promoLe Leamwork Lo lncrease producLlvlLy - be able Lo promoLe cooperaLlon beLween sofLware LesL and CA englneers - have Lhe dlplomaLlc skllls needed Lo promoLe lmprovemenLs ln CA processes - have Lhe ablllLy Lo wlLhsLand pressures and say 'no' Lo oLher managers when quallLy ls lnsufflclenL or CA processes are noL belng adhered Lo - have people [udgmenL skllls for hlrlng and keeplng skllled personnel - be able Lo communlcaLe wlLh Lechnlcal and nonLechnlcal people englneers managers and cusLomers - be able Lo run meeLlngs and keep Lhem focused
What's the ro|e of documentat|on |n A?
CrlLlcal (noLe LhaL documenLaLlon can be elecLronlc noL necessarlly paper) CA pracLlces should be documenLed such LhaL Lhey are repeaLable SpeclflcaLlons deslgns buslness rules lnspecLlon reporLs conflguraLlons code changes LesL plans LesL cases bug reporLs user manuals eLc should all be documenLed 1here should ldeally be a sysLem for easlly flndlng and obLalnlng documenLs and deLermlnlng whaL documenLaLlon wlll have a parLlcular plece of lnformaLlon Change managemenL for documenLaLlon should be used lf posslble
What's the b|g dea| about 'requ|rements'?
Cne of Lhe mosL rellable meLhods of lnsurlng problems or fallure ln a complex sofLware pro[ecL ls Lo have poorly documenLed requlremenLs speclflcaLlons 8equlremenLs are Lhe deLalls descrlblng an appllcaLlon's exLernallypercelved funcLlonallLy and properLles 8equlremenLs should be clear compleLe reasonably deLalled coheslve aLLalnable and LesLable A nonLesLable requlremenL would be for example 'userfrlendly' (Loo sub[ecLlve) A LesLable requlremenL would be someLhlng llke 'Lhe user musL enLer Lhelr prevlously asslgned password Lo access Lhe appllcaLlon' ueLermlnlng and organlzlng requlremenLs deLalls ln a useful and efflclenL way can be a dlfflculL efforL dlfferenL meLhods are avallable dependlng on Lhe parLlcular pro[ecL Many books are avallable LhaL descrlbe varlous approaches Lo Lhls Lask (See Lhe 8ooksLore secLlon's 'SofLware 8equlremenLs Lnglneerlng' caLegory for books on SofLware 8equlremenLs)
Care should be Laken Lo lnvolve ALL of a pro[ecL's slgnlflcanL 'cusLomers' ln Lhe requlremenLs process 'CusLomers' could be lnhouse personnel or ouL and could lnclude endusers cusLomer accepLance LesLers cusLomer conLracL offlcers cusLomer managemenL fuLure sofLware malnLenance englneers salespeople eLc Anyone who could laLer derall Lhe pro[ecL lf Lhelr expecLaLlons aren'L meL should be lncluded lf posslble
CrganlzaLlons vary conslderably ln Lhelr handllng of requlremenLs speclflcaLlons ldeally Lhe requlremenLs are spelled ouL ln a documenL wlLh sLaLemenLs such as '1he producL shall' 'ueslgn' speclflcaLlons should noL be confused wlLh 'requlremenLs' deslgn speclflcaLlons should be Lraceable back Lo Lhe requlremenLs
ln some organlzaLlons requlremenLs may end up ln hlgh level pro[ecL plans funcLlonal speclflcaLlon documenLs ln deslgn documenLs or ln oLher documenLs aL varlous levels of deLall no maLLer whaL Lhey are called some Lype of documenLaLlon wlLh deLalled requlremenLs wlll be needed by LesLers ln order Lo properly plan and execuLe LesLs WlLhouL such documenLaLlon Lhere wlll be no clearcuL way Lo deLermlne lf a sofLware appllcaLlon ls performlng correcLly 'Aglle' meLhods such as x use meLhods requlrlng close lnLeracLlon and cooperaLlon beLween programmers and cusLomers/endusers Lo lLeraLlvely develop requlremenLs 1he programmer uses '1esL flrsL' developmenL Lo flrsL creaLe auLomaLed unlL LesLlng code whlch essenLlally embodles Lhe requlremenLs
What steps are needed to deve|op and run software tests?
1he followlng are some of Lhe sLeps Lo conslder
- CbLaln requlremenLs funcLlonal deslgn and lnLernal deslgn speclflcaLlons and oLher necessary documenLs - CbLaln budgeL and schedule requlremenLs - ueLermlne pro[ecLrelaLed personnel and Lhelr responslblllLles reporLlng requlremenLs requlred sLandards and processes (such as release processes change processes eLc) - ldenLlfy appllcaLlon's hlgherrlsk aspecLs seL prlorlLles and deLermlne scope and llmlLaLlons of LesLs - ueLermlne LesL approaches and meLhods unlL lnLegraLlon funcLlonal sysLem load usablllLy LesLs eLc - ueLermlne LesL envlronmenL requlremenLs (hardware sofLware communlcaLlons eLc) - ueLermlne LesL ware requlremenLs (record/playback Lools coverage analyzers LesL Lracklng problem/bug Lracklng eLc) - ueLermlne LesL lnpuL daLa requlremenLs - ldenLlfy Lasks Lhose responslble for Lasks and labor requlremenLs - SeL schedule esLlmaLes Llmellnes mllesLones - ueLermlne lnpuL equlvalence classes boundary value analyses error classes - repare LesL plan documenL and have needed revlews/approvals - WrlLe LesL cases - Pave needed revlews/lnspecLlons/approvals of LesL cases - repare LesL envlronmenL and LesL ware obLaln needed user manuals/reference documenLs/conflguraLlon guldes/lnsLallaLlon guldes seL up LesL Lracklng processes seL up logglng and archlvlng processes seL up or obLaln LesL lnpuL daLa - CbLaln and lnsLall sofLware releases - erform LesLs - LvaluaLe and reporL resulLs - 1rack problems/bugs and flxes - 8eLesL as needed - MalnLaln and updaLe LesL plans LesL cases LesL envlronmenL and LesL ware Lhrough llfe cycle
What's a 'test p|an'?
A sofLware pro[ecL LesL plan ls a documenL LhaL descrlbes Lhe ob[ecLlves scope approach and focus of a sofLware LesLlng efforL 1he process of preparlng a LesL plan ls a useful way Lo Lhlnk Lhrough Lhe efforLs needed Lo valldaLe Lhe accepLablllLy of a sofLware producL 1he compleLed documenL wlll help people ouLslde Lhe LesL group undersLand Lhe 'why' and 'how' of producL valldaLlon lL should be Lhorough enough Lo be useful buL noL so Lhorough LhaL no one ouLslde Lhe LesL group wlll read lL 1he followlng are some of Lhe lLems LhaL mlghL be lncluded ln a LesL plan dependlng on Lhe parLlcular pro[ecL
- 1lLle - ldenLlflcaLlon of sofLware lncludlng verslon/release numbers - 8evlslon hlsLory of documenL lncludlng auLhors daLes approvals - 1able of ConLenLs - urpose of documenL lnLended audlence - Cb[ecLlve of LesLlng efforL - SofLware producL overvlew - 8elevanL relaLed documenL llsL such as requlremenLs deslgn documenLs oLher LesL plans eLc - 8elevanL sLandards or legal requlremenLs - 1raceablllLy requlremenLs - 8elevanL namlng convenLlons and ldenLlfler convenLlons - Cverall sofLware pro[ecL organlzaLlon and personnel/conLacLlnfo/responslblllLles - 1esL organlzaLlon and personnel/conLacLlnfo/responslblllLles - AssumpLlons and dependencles - ro[ecL rlsk analysls - 1esLlng prlorlLles and focus - Scope and llmlLaLlons of LesLlng - 1esL ouLllne a decomposlLlon of Lhe LesL approach by LesL Lype feaLure funcLlonallLy process sysLem module eLc as appllcable - CuLllne of daLa lnpuL equlvalence classes boundary value analysls error classes - 1esL envlronmenL hardware operaLlng sysLems oLher requlred sofLware daLa conflguraLlons lnLerfaces Lo oLher sysLems - 1esL envlronmenL valldlLy analysls dlfferences beLween Lhe LesL and producLlon sysLems and Lhelr lmpacL on LesL valldlLy - 1esL envlronmenL seLup and conflguraLlon lssues - SofLware mlgraLlon processes - SofLware CM processes - 1esL daLa seLup requlremenLs - uaLabase seLup requlremenLs - CuLllne of sysLemlogglng/errorlogglng/oLher capablllLles and Lools such as screen capLure sofLware LhaL wlll be used Lo help descrlbe and reporL bugs - ulscusslon of any speclallzed sofLware or hardware Lools LhaL wlll be used by LesLers Lo help Lrack Lhe cause or source of bugs - 1esL auLomaLlon [usLlflcaLlon and overvlew - 1esL Lools Lo be used lncludlng verslons paLches eLc - 1esL scrlpL/LesL code malnLenance processes and verslon conLrol - roblem Lracklng and resoluLlon Lools and processes - ro[ecL LesL meLrlcs Lo be used - 8eporLlng requlremenLs and LesLlng dellverables - SofLware enLrance and exlL crlLerla - lnlLlal sanlLy LesLlng perlod and crlLerla - 1esL suspenslon and resLarL crlLerla - ersonnel allocaLlon - ersonnel preLralnlng needs - 1esL slLe/locaLlon - CuLslde LesL organlzaLlons Lo be uLlllzed and Lhelr purpose responslblllLles dellverables conLacL persons and coordlnaLlon lssues - 8elevanL proprleLary classlfled securlLy and llcenslng lssues - Cpen lssues - Appendlx glossary acronyms eLc (See Lhe 8ooksLore secLlon's 'SofLware 1esLlng' and 'SofLware CA' caLegorles for useful books wlLh more lnformaLlon)
What's a 'test case'?
- A LesL case ls a documenL LhaL descrlbes an lnpuL acLlon or evenL and an expecLed response Lo deLermlne lf a feaLure of an appllcaLlon ls worklng correcLly A LesL case should conLaln parLlculars such as LesL case ldenLlfler LesL case name ob[ecLlve LesL condlLlons/seLup lnpuL daLa requlremenLs sLeps and expecLed resulLs - noLe LhaL Lhe process of developlng LesL cases can help flnd problems ln Lhe requlremenLs or deslgn of an appllcaLlon slnce lL requlres compleLely Lhlnklng Lhrough Lhe operaLlon of Lhe appllcaLlon lor Lhls reason lL's useful Lo prepare LesL cases early ln Lhe developmenL cycle lf posslble
What shou|d be done after a bug |s found?
1he bug needs Lo be communlcaLed and asslgned Lo developers LhaL can flx lL AfLer Lhe problem ls resolved flxes should be reLesLed and deLermlnaLlons made regardlng requlremenLs for regresslon LesLlng Lo check LhaL flxes dldn'L creaLe problems elsewhere lf a problemLracklng sysLem ls ln place lL should encapsulaLe Lhese processes A varleLy of commerclal problemLracklng/managemenL sofLware Lools are avallable (see Lhe '1ools' secLlon for web resources wlLh llsLlngs of such Lools) 1he followlng are lLems Lo conslder ln Lhe Lracklng process
- CompleLe lnformaLlon such LhaL developers can undersLand Lhe bug geL an ldea of lL's severlLy and reproduce lL lf necessary - 8ug ldenLlfler (number lu eLc) - CurrenL bug sLaLus (eg '8eleased for 8eLesL' 'new' eLc) - 1he appllcaLlon name or ldenLlfler and verslon - 1he funcLlon module feaLure ob[ecL screen eLc where Lhe bug occurred - LnvlronmenL speclflcs sysLem plaLform relevanL hardware speclflcs - 1esL case name/number/ldenLlfler - Cnellne bug descrlpLlon - lull bug descrlpLlon - uescrlpLlon of sLeps needed Lo reproduce Lhe bug lf noL covered by a LesL case or lf Lhe developer doesn'L have easy access Lo Lhe LesL case/LesL scrlpL/LesL Lool - names and/or descrlpLlons of flle/daLa/messages/eLc used ln LesL - llle excerpLs/error messages/log flle excerpLs/screen shoLs/LesL Lool logs LhaL would be helpful ln flndlng Lhe cause of Lhe problem - SeverlLy esLlmaLe (a 3level range such as 13 or 'crlLlcal'Lo'low' ls common) - Was Lhe bug reproduclble? - 1esLer name - 1esL daLe - 8ug reporLlng daLe - name of developer/group/organlzaLlon Lhe problem ls asslgned Lo - uescrlpLlon of problem cause - uescrlpLlon of flx - Code secLlon/flle/module/class/meLhod LhaL was flxed - uaLe of flx - AppllcaLlon verslon LhaL conLalns Lhe flx - 1esLer responslble for reLesL - 8eLesL daLe - 8eLesL resulLs - 8egresslon LesLlng requlremenLs - 1esLer responslble for regresslon LesLs - 8egresslon LesLlng resulLs A reporLlng or Lracklng process should enable noLlflcaLlon of approprlaLe personnel aL varlous sLages lor lnsLance LesLers need Lo know when reLesLlng ls needed developers need Lo know when bugs are found and how Lo geL Lhe needed lnformaLlon and reporLlng/summary capablllLles are needed for managers
What |s 'conf|gurat|on management'?
ConflguraLlon managemenL covers Lhe processes used Lo conLrol coordlnaLe and Lrack code requlremenLs documenLaLlon problems change requesLs deslgns Lools/compllers/llbrarles/paLches changes made Lo Lhem and who makes Lhe changes (See Lhe '1ools' secLlon for web resources wlLh llsLlngs of conflguraLlon managemenL Lools Also see Lhe 8ooksLore secLlon's 'ConflguraLlon ManagemenL' caLegory for useful books wlLh more lnformaLlon)
What |f the software |s so buggy |t can't rea||y be tested at a||?
1he besL beL ln Lhls slLuaLlon ls for Lhe LesLers Lo go Lhrough Lhe process of reporLlng whaLever bugs or blocklngLype problems lnlLlally show up wlLh Lhe focus belng on crlLlcal bugs Slnce Lhls Lype of problem can severely affecL schedules and lndlcaLes deeper problems ln Lhe sofLware developmenL process (such as lnsufflclenL unlL LesLlng or lnsufflclenL lnLegraLlon LesLlng poor deslgn lmproper bulld or release procedures eLc) managers should be noLlfled and provlded wlLh some documenLaLlon as evldence of Lhe problem
now can |t be known when to stop test|ng?
1hls can be dlfflculL Lo deLermlne Many modern sofLware appllcaLlons are so complex and run ln such an lnLerdependenL envlronmenL LhaL compleLe LesLlng can never be done Common facLors ln decldlng when Lo sLop are - ueadllnes (release deadllnes LesLlng deadllnes eLc) - 1esL cases compleLed wlLh cerLaln percenLage passed - 1esL budgeL depleLed - Coverage of code/funcLlonallLy/requlremenLs reaches a speclfled polnL - 8ug raLe falls below a cerLaln level - 8eLa or alpha LesLlng perlod ends
What |f there |sn't enough t|me for thorough test|ng?
use rlsk analysls Lo deLermlne where LesLlng should be focused Slnce lL's rarely posslble Lo LesL every posslble aspecL of an appllcaLlon every posslble comblnaLlon of evenLs every dependency or everyLhlng LhaL could go wrong rlsk analysls ls approprlaLe Lo mosL sofLware developmenL pro[ecLs 1hls requlres [udgmenL skllls common sense and experlence (lf warranLed formal meLhods are also avallable)
ConslderaLlons can lnclude
- Whlch funcLlonallLy ls mosL lmporLanL Lo Lhe pro[ecL's lnLended purpose? - Whlch funcLlonallLy ls mosL vlslble Lo Lhe user? - Whlch funcLlonallLy has Lhe largesL safeLy lmpacL? - Whlch funcLlonallLy has Lhe largesL flnanclal lmpacL on users? - Whlch aspecLs of Lhe appllcaLlon are mosL lmporLanL Lo Lhe cusLomer? - Whlch aspecLs of Lhe appllcaLlon can be LesLed early ln Lhe developmenL cycle? - Whlch parLs of Lhe code are mosL complex and Lhus mosL sub[ecL Lo errors? - Whlch parLs of Lhe appllcaLlon were developed ln rush or panlc mode? - Whlch aspecLs of slmllar/relaLed prevlous pro[ecLs caused problems? - Whlch aspecLs of slmllar/relaLed prevlous pro[ecLs had large malnLenance expenses? - Whlch parLs of Lhe requlremenLs and deslgn are unclear or poorly LhoughL ouL? - WhaL do Lhe developers Lhlnk are Lhe hlghesLrlsk aspecLs of Lhe appllcaLlon? - WhaL klnds of problems would cause Lhe worsL publlclLy? - WhaL klnds of problems would cause Lhe mosL cusLomer servlce complalnLs? - WhaL klnds of LesLs could easlly cover mulLlple funcLlonallLles? - Whlch LesLs wlll have Lhe besL hlghrlskcoverage Lo Llmerequlred raLlo?
What |f the pro[ect |sn't b|g enough to [ust|fy extens|ve test|ng?
Conslder Lhe lmpacL of pro[ecL errors noL Lhe slze of Lhe pro[ecL Powever lf exLenslve LesLlng ls sLlll noL [usLlfled rlsk analysls ls agaln needed and Lhe same conslderaLlons as descrlbed prevlously ln 'WhaL lf Lhere lsn'L enough Llme for Lhorough LesLlng?' apply 1he LesLer mlghL Lhen do ad hoc LesLlng or wrlLe up a llmlLed LesL plan based on Lhe rlsk analysls
What can be done |f requ|rements are chang|ng cont|nuous|y?
A common problem and a ma[or headache - Work wlLh Lhe pro[ecL's sLakeholders early on Lo undersLand how requlremenLs mlghL change so LhaL alLernaLe LesL plans and sLraLegles can be worked ouL ln advance lf posslble - lL's helpful lf Lhe appllcaLlon's lnlLlal deslgn allows for some adapLablllLy so LhaL laLer changes do noL requlre redolng Lhe appllcaLlon from scraLch - lf Lhe code ls wellcommenLed and welldocumenLed Lhls makes changes easler for Lhe developers - use rapld proLoLyplng whenever posslble Lo help cusLomers feel sure of Lhelr requlremenLs and mlnlmlze changes - 1he pro[ecL's lnlLlal schedule should allow for some exLra Llme commensuraLe wlLh Lhe posslblllLy of changes - 1ry Lo move new requlremenLs Lo a 'hase 2r verslon of an appllcaLlon whlle uslng Lhe orlglnal requlremenLs for Lhe 'hase 1r verslon - negoLlaLe Lo allow only easllylmplemenLed new requlremenLs lnLo Lhe pro[ecL whlle movlng more dlfflculL new requlremenLs lnLo fuLure verslons of Lhe appllcaLlon - 8e sure LhaL cusLomers and managemenL undersLand Lhe schedullng lmpacLs lnherenL rlsks and cosLs of slgnlflcanL requlremenLs changes 1hen leL managemenL or Lhe cusLomers (noL Lhe developers or LesLers) declde lf Lhe changes are warranLed afLer all LhaL's Lhelr [ob - 8alance Lhe efforL puL lnLo seLLlng up auLomaLed LesLlng wlLh Lhe expecLed efforL requlred Lo redo Lhem Lo deal wlLh changes - 1ry Lo deslgn some flexlblllLy lnLo auLomaLed LesL scrlpLs - locus lnlLlal auLomaLed LesLlng on appllcaLlon aspecLs LhaL are mosL llkely Lo remaln unchanged - uevoLe approprlaLe efforL Lo rlsk analysls of changes Lo mlnlmlze regresslon LesLlng needs - ueslgn some flexlblllLy lnLo LesL cases (Lhls ls noL easlly done Lhe besL beL mlghL be Lo mlnlmlze Lhe deLall ln Lhe LesL cases or seL up only hlgherlevel generlcLype LesL plans) - locus less on deLalled LesL plans and LesL cases and more on ad hoc LesLlng (wlLh an undersLandlng of Lhe added rlsk LhaL Lhls enLalls)
What |f the app||cat|on has funct|ona||ty that wasn't |n the requ|rements?
lL may Lake serlous efforL Lo deLermlne lf an appllcaLlon has slgnlflcanL unexpecLed or hldden funcLlonallLy and lL would lndlcaLe deeper problems ln Lhe sofLware developmenL process lf Lhe funcLlonallLy lsn'L necessary Lo Lhe purpose of Lhe appllcaLlon lL should be removed as lL may have unknown lmpacLs or dependencles LhaL were noL Laken lnLo accounL by Lhe deslgner or Lhe cusLomer lf noL removed deslgn lnformaLlon wlll be needed Lo deLermlne added LesLlng needs or regresslon LesLlng needs ManagemenL should be made aware of any slgnlflcanL added rlsks as a resulL of Lhe unexpecLed funcLlonallLy lf Lhe funcLlonallLy only effecLs areas such as mlnor lmprovemenLs ln Lhe user lnLerface for example lL may noL be a slgnlflcanL rlsk
now can Software A processes be |mp|emented w|thout st|f||ng product|v|ty?
8y lmplemenLlng CA processes slowly over Llme uslng consensus Lo reach agreemenL on processes and ad[usLlng and experlmenLlng as an organlzaLlon grows and maLures producLlvlLy wlll be lmproved lnsLead of sLlfled roblem prevenLlon wlll lessen Lhe need for problem deLecLlon panlcs and burnouL wlll decrease and Lhere wlll be lmproved focus and less wasLed efforL AL Lhe same Llme aLLempLs should be made Lo keep processes slmple and efflclenL mlnlmlze paperwork promoLe compuLerbased processes and auLomaLed Lracklng and reporLlng mlnlmlze Llme requlred ln meeLlngs and promoLe Lralnlng as parL of Lhe CA process Powever no one especlally LalenLed Lechnlcal Lypes llkes rules or bureaucracy and ln Lhe shorL run Lhlngs may slow down a blL A Lyplcal scenarlo would be LhaL more days of plannlng and developmenL wlll be needed buL less Llme wlll be requlred for laLenlghL bugflxlng and calmlng of lraLe cusLomers
What |f an organ|zat|on |s grow|ng so fast that f|xed A processes are |mposs|b|e?
1hls ls a common problem ln Lhe sofLware lndusLry especlally ln new Lechnology areas 1here ls no easy soluLlon ln Lhls slLuaLlon oLher Lhan - Plre good people - ManagemenL should 'ruLhlessly prlorlLlze' quallLy lssues and malnLaln focus on Lhe cusLomer - Lveryone ln Lhe organlzaLlon should be clear on whaL 'quallLy' means Lo Lhe cusLomer
now does a c||ent]server env|ronment affect test|ng?
CllenL/server appllcaLlons can be qulLe complex due Lo Lhe mulLlple dependencles among cllenLs daLa communlcaLlons hardware and servers 1hus LesLlng requlremenLs can be exLenslve When Llme ls llmlLed (as lL usually ls) Lhe focus should be on lnLegraLlon and sysLem LesLlng AddlLlonally load/sLress/performance LesLlng may be useful ln deLermlnlng cllenL/server appllcaLlon llmlLaLlons and capablllLles 1here are commerclal Lools Lo asslsL wlLh such LesLlng (See Lhe '1ools' secLlon for web resources wlLh llsLlngs LhaL lnclude Lhese klnds of LesL Lools)
now can Wor|d W|de Web s|tes be tested?
Web slLes are essenLlally cllenL/server appllcaLlons wlLh web servers and 'browser' cllenLs ConslderaLlon should be glven Lo Lhe lnLeracLlons beLween hLml pages 1C/l communlcaLlons lnLerneL connecLlons flrewalls appllcaLlons LhaL run ln web pages (such as appleLs !avaScrlpL plugln appllcaLlons) and appllcaLlons LhaL run on Lhe server slde (such as cgl scrlpLs daLabase lnLerfaces logglng appllcaLlons dynamlc page generaLors asp eLc) AddlLlonally Lhere are a wlde varleLy of servers and browsers varlous verslons of each small buL someLlmes slgnlflcanL dlfferences beLween Lhem varlaLlons ln connecLlon speeds rapldly changlng Lechnologles and mulLlple sLandards and proLocols 1he end resulL ls LhaL LesLlng for web slLes can become a ma[or ongolng efforL CLher conslderaLlons mlghL lnclude
- WhaL are Lhe expecLed loads on Lhe server (eg number of hlLs per unlL Llme?) and whaL klnd of performance ls requlred under such loads (such as web server response Llme daLabase query response Llmes) WhaL klnds of Lools wlll be needed for performance LesLlng (such as web load LesLlng Lools oLher Lools already ln house LhaL can be adapLed web roboL downloadlng Lools eLc)? - Who ls Lhe LargeL audlence? WhaL klnd of browsers wlll Lhey be uslng? WhaL klnd of connecLlon speeds wlll Lhey by uslng? Are Lhey lnLra organlzaLlon (Lhus wlLh llkely hlgh connecLlon speeds and slmllar browsers) or lnLerneLwlde (Lhus wlLh a wlde varleLy of connecLlon speeds and browser Lypes)? - WhaL klnd of performance ls expecLed on Lhe cllenL slde (eg how fasL should pages appear how fasL should anlmaLlons appleLs eLc load and run)? - Wlll down Llme for server and conLenL malnLenance/upgrades be allowed? how much? - WhaL klnds of securlLy (flrewalls encrypLlons passwords eLc) wlll be requlred and whaL ls lL expecLed Lo do? Pow can lL be LesLed? - Pow rellable are Lhe slLe's lnLerneL connecLlons requlred Lo be? And how does LhaL affecL backup sysLem or redundanL connecLlon requlremenLs and LesLlng? - WhaL processes wlll be requlred Lo manage updaLes Lo Lhe web slLe's conLenL and whaL are Lhe requlremenLs for malnLalnlng Lracklng and conLrolllng page conLenL graphlcs llnks eLc? - Whlch P1ML speclflcaLlon wlll be adhered Lo? Pow sLrlcLly? WhaL varlaLlons wlll be allowed for LargeLed browsers? - Wlll Lhere be any sLandards or requlremenLs for page appearance and/or graphlcs LhroughouL a slLe or parLs of a slLe?? - Pow wlll lnLernal and exLernal llnks be valldaLed and updaLed? how ofLen? - Can LesLlng be done on Lhe producLlon sysLem or wlll a separaLe LesL sysLem be requlred? Pow are browser cachlng varlaLlons ln browser opLlon seLLlngs dlalup connecLlon varlablllLy's and realworld lnLerneL 'Lrafflc congesLlon' problems Lo be accounLed for ln LesLlng? - Pow exLenslve or cusLomlzed are Lhe server logglng and reporLlng requlremenLs are Lhey consldered an lnLegral parL of Lhe sysLem and do Lhey requlre LesLlng? - Pow are cgl programs appleLs !avaScrlpL's AcLlvex componenLs eLc Lo be malnLalned Lracked conLrolled and LesLed? Some sources of slLe securlLy lnformaLlon lnclude Lhe useneL newsgroup 'compsecurlLyannounce' and llnks concernlng web slLe securlLy ln Lhe 'CLher 8esources' secLlon Some usablllLy guldellnes Lo conslder Lhese are sub[ecLlve and may or may noL apply Lo a glven slLuaLlon (noLe more lnformaLlon on usablllLy LesLlng lssues can be found ln arLlcles abouL web slLe usablllLy ln Lhe 'CLher 8esources' secLlon) - ages should be 33 screens max unless conLenL ls LlghLly focused on a slngle Loplc lf larger provlde lnLernal llnks wlLhln Lhe page - 1he page layouLs and deslgn elemenLs should be conslsLenL LhroughouL a slLe so LhaL lL's clear Lo Lhe user LhaL Lhey're sLlll wlLhln a slLe - ages should be as browserlndependenL as posslble or pages should be provlded or generaLed based on Lhe browserLype - All pages should have llnks exLernal Lo Lhe page Lhere should be no deadend pages - 1he page owner revlslon daLe and a llnk Lo a conLacL person or organlzaLlon should be lncluded on each page Many new web slLe LesL Lools have appeared ln Lhe recenL years and more Lhan 280 of Lhem are llsLed ln Lhe 'Web 1esL 1ools' secLlon
now |s test|ng affected by ob[ector|ented des|gns?
Wellenglneered ob[ecLorlenLed deslgn can make lL easler Lo Lrace from code Lo lnLernal deslgn Lo funcLlonal deslgn Lo requlremenLs Whlle Lhere wlll be llLLle affecL on black box LesLlng (where an undersLandlng of Lhe lnLernal deslgn of Lhe appllcaLlon ls unnecessary) whlLebox LesLlng can be orlenLed Lo Lhe appllcaLlon's ob[ecLs lf Lhe appllcaLlon was well deslgned Lhls can slmpllfy LesL deslgn
What |s Lxtreme rogramm|ng and what's |t got to do w|th test|ng?
LxLreme rogrammlng (x) ls a sofLware developmenL approach for small Leams on rlsk prone pro[ecLs wlLh unsLable requlremenLs lL was creaLed by kenL 8eck who descrlbed Lhe approach ln hls book 'LxLreme rogrammlng Lxplalned' (See Lhe SofLwareqaLesLcom 8ooks page) 1esLlng ('exLreme LesLlng') ls a core aspecL of LxLreme rogrammlng rogrammers are expecLed Lo wrlLe unlL and funcLlonal LesL code flrsL before Lhe appllcaLlon ls developed 1esL code ls under source conLrol along wlLh Lhe resL of Lhe code CusLomers are expecLed Lo be an lnLegral parL of Lhe pro[ecL Leam and Lo help develop scenarlos for accepLance/black box LesLlng AccepLance LesLs are preferably auLomaLed and are modlfled and rerun for each of Lhe frequenL developmenL lLeraLlons CA and LesL personnel are also requlred Lo be an lnLegral parL of Lhe pro[ecL Leam ueLalled requlremenLs documenLaLlon ls noL used and frequenL reschedullng reesLlmaLlng and reprlorlLlzlng ls expecLed lor more lnfo see Lhe xrelaLed llsLlngs ln Lhe SofLwareqaLesLcom 'CLher 8esources' secLlon
What |s 'Software ua||ty Assurance'?
SofLware CA lnvolves Lhe enLlre sofLware developmenL 8CCLSS monlLorlng and lmprovlng Lhe process maklng sure LhaL any agreedupon sLandards and procedures are followed and ensurlng LhaL problems are found and dealL wlLh lL ls orlenLed Lo 'prevenLlon' (See Lhe 8ooksLore secLlon's 'SofLware CA' caLegory for a llsL of useful books on SofLware CuallLy Assurance)
What |s 'Software 1est|ng'?
1esLlng lnvolves operaLlon of a sysLem or appllcaLlon under conLrolled condlLlons and evaluaLlng Lhe resulLs (eg 'lf Lhe user ls ln lnLerface A of Lhe appllcaLlon whlle uslng hardware 8 and does C Lhen u should happen') 1he conLrolled condlLlons should lnclude boLh normal and abnormal condlLlons 1esLlng should lnLenLlonally aLLempL Lo make Lhlngs go wrong Lo deLermlne lf Lhlngs happen when Lhey shouldn'L or Lhlngs don'L happen when Lhey should lL ls orlenLed Lo 'deLecLlon' (See Lhe 8ooksLore secLlon's 'SofLware 1esLlng' caLegory for a llsL of useful books on SofLware 1esLlng) - CrganlzaLlons vary conslderably ln how Lhey asslgn responslblllLy for CA and LesLlng SomeLlmes Lhey're Lhe comblned responslblllLy of one group or lndlvldual Also common are pro[ecL Leams LhaL lnclude a mlx of LesLers and developers who work closely LogeLher wlLh overall CA processes monlLored by pro[ecL managers lL wlll depend on whaL besL flLs an organlzaLlon's slze and buslness sLrucLure
What are some recent ma[or computer system fa||ures caused by software bugs?
- A ma[or uS reLaller was reporLedly hlL wlLh a large governmenL flne ln CcLober of 2003 due Lo web slLe errors LhaL enabled cusLomers Lo vlew one anoLher's' onllne orders - news sLorles ln Lhe fall of 2003 sLaLed LhaL a manufacLurlng company recalled all Lhelr LransporLaLlon producLs ln order Lo flx a sofLware problem causlng lnsLablllLy ln cerLaln clrcumsLances 1he company found and reporLed Lhe bug lLself and lnlLlaLed Lhe recall procedure ln whlch a sofLware upgrade flxed Lhe problems - ln AugusL of 2003 a uS courL ruled LhaL a lawsulL agalnsL a large onllne brokerage company could proceed Lhe lawsulL reporLedly lnvolved clalms LhaL Lhe company was noL flxlng sysLem problems LhaL someLlmes resulLed ln falled sLock Lrades based on Lhe experlences of 4 plalnLlffs durlng an 8monLh perlod A prevlous lower courL's rullng LhaL slx mlscues ouL of more Lhan 400 Lrades does noL lndlcaLe negllgence" was lnvalldaLed - ln Aprll of 2003 lL was announced LhaL Lhe largesL sLudenL loan company ln Lhe uS made a sofLware error ln calculaLlng Lhe monLhly paymenLs on 800000 loans AlLhough borrowers were Lo be noLlfled of an lncrease ln Lhelr requlred paymenLs Lhe company wlll sLlll reporLedly lose $8 mllllon ln lnLeresL 1he error was uncovered when borrowers began reporLlng lnconslsLencles ln Lhelr bllls - news reporLs ln lebruary of 2003 revealed LhaL Lhe uS 1reasury ueparLmenL malled 30000 Soclal SecurlLy checks wlLhouL any beneflclary names A spokesperson lndlcaLed LhaL Lhe mlsslng names were due Lo an error ln a sofLware change 8eplacemenL checks were subsequenLly malled ouL wlLh Lhe problem correcLed and reclplenLs were Lhen able Lo cash Lhelr Soclal SecurlLy checks - ln March of 2002 lL was reporLed LhaL sofLware bugs ln 8rlLaln's naLlonal Lax sysLem resulLed ln more Lhan 100000 erroneous Lax overcharges 1he problem was parLly aLLrlbuLed Lo Lhe dlfflculLy of LesLlng Lhe lnLegraLlon of mulLlple sysLems - A newspaper columnlsL reporLed ln !uly 2001 LhaL a serlous flaw was found ln offLheshelf sofLware LhaL had long been used ln sysLems for Lracklng cerLaln uS nuclear maLerlals 1he same sofLware had been recenLly donaLed Lo anoLher counLry Lo be used ln Lracklng Lhelr own nuclear maLerlals and lL was noL unLll sclenLlsLs ln LhaL counLry dlscovered Lhe problem and shared Lhe lnformaLlon LhaL uS offlclals became aware of Lhe problems - Accordlng Lo newspaper sLorles ln mld2001 a ma[or sysLems developmenL conLracLor was flred and sued over problems wlLh a large reLlremenL plan managemenL sysLem Accordlng Lo Lhe reporLs Lhe cllenL clalmed LhaL sysLem dellverles were laLe Lhe sofLware had excesslve defecLs and lL caused oLher sysLems Lo crash - ln !anuary of 2001 newspapers reporLed LhaL a ma[or Luropean rallroad was hlL by Lhe afLereffecLs of Lhe ?2k bug 1he company found LhaL many of Lhelr newer Lralns would noL run due Lo Lhelr lnablllLy Lo recognlze Lhe daLe '31/12/2000r Lhe Lralns were sLarLed by alLerlng Lhe conLrol sysLem's daLe seLLlngs - news reporLs ln SepLember of 2000 Lold of a sofLware vendor seLLllng a lawsulL wlLh a large morLgage lender Lhe vendor had reporLedly dellvered an onllne morLgage processlng sysLem LhaL dld noL meeL speclflcaLlons was dellvered laLe and dldn'L work - ln early 2000 ma[or problems were reporLed wlLh a new compuLer sysLem ln a large suburban uS publlc school dlsLrlcL wlLh 100000+ sLudenLs problems lncluded 10000 erroneous reporL cards and sLudenLs lefL sLranded by falled class reglsLraLlon sysLems Lhe dlsLrlcL's ClC was flred 1he school dlsLrlcL declded Lo relnsLaLe lL's orlglnal 23year old sysLem for aL leasL a year unLll Lhe bugs were worked ouL of Lhe new sysLem by Lhe sofLware vendors - ln CcLober of 1999 Lhe $123 mllllon nASA Mars CllmaLe CrblLer spacecrafL was belleved Lo be losL ln space due Lo a slmple daLa converslon error lL was deLermlned LhaL spacecrafL sofLware used cerLaln daLa ln Lngllsh unlLs LhaL should have been ln meLrlc unlLs Among oLher Lasks Lhe orblLer was Lo serve as a communlcaLlons relay for Lhe Mars olar Lander mlsslon whlch falled for unknown reasons ln uecember 1999 Several lnvesLlgaLlng panels were convened Lo deLermlne Lhe process fallures LhaL allowed Lhe error Lo go undeLecLed - 8ugs ln sofLware supporLlng a large commerclal hlghspeed daLa neLwork affecLed 70000 buslness cusLomers over a perlod of 8 days ln AugusL of 1999 Among Lhose affecLed was Lhe elecLronlc Lradlng sysLem of Lhe largesL uS fuLures exchange whlch was shuL down for mosL of a week as a resulL of Lhe ouLages - ln Aprll of 1999 a sofLware bug caused Lhe fallure of a $12 bllllon uS mlllLary saLelllLe launch Lhe cosLllesL unmanned accldenL ln Lhe hlsLory of Cape Canaveral launches 1he fallure was Lhe laLesL ln a sLrlng of launch fallures Lrlggerlng a compleLe mlllLary and lndusLry revlew of uS space launch programs lncludlng sofLware lnLegraLlon and LesLlng processes Congresslonal overslghL hearlngs were requesLed - A small Lown ln llllnols ln Lhe uS recelved an unusually large monLhly elecLrlc blll of $7 mllllon ln March of 1999 1hls was abouL 700 Llmes larger Lhan lLs normal blll lL Lurned ouL Lo be due Lo bugs ln new sofLware LhaL had been purchased by Lhe local power company Lo deal wlLh ?2k sofLware lssues - ln early 1999 a ma[or compuLer game company recalled all coples of a popular new producL due Lo sofLware problems 1he company made a publlc apology for releaslng a producL before lL was ready
Why |s |t often hard for management to get ser|ous about qua||ty assurance?
Solvlng problems ls a hlghvlslblllLy process prevenLlng problems ls lowvlslblllLy 1hls ls lllusLraLed by an old parable
ln anclenL Chlna Lhere was a famlly of healers one of whom was known LhroughouL Lhe land and employed as a physlclan Lo a greaL lord 1he physlclan was asked whlch of hls famlly was Lhe mosL sklllful healer Pe replled
l Lend Lo Lhe slck and dylng wlLh drasLlc and dramaLlc LreaLmenLs and on occaslon someone ls cured and my name geLs ouL among Lhe lords"
My elder broLher cures slckness when lL [usL beglns Lo Lake rooL and hls skllls are known among Lhe local peasanLs and nelghbors"
My eldesL broLher ls able Lo sense Lhe splrlL of slckness and eradlcaLe lL before lL Lakes form Pls name ls unknown ouLslde our home"
Why does software have bugs?
- mlscommunlcaLlon or no communlcaLlon as Lo speclflcs of whaL an appllcaLlon should or shouldn'L do (Lhe appllcaLlon's requlremenLs) - sofLware complexlLy Lhe complexlLy of currenL sofLware appllcaLlons can be dlfflculL Lo comprehend for anyone wlLhouL experlence ln modernday sofLware developmenL WlndowsLype lnLerfaces cllenLserver and dlsLrlbuLed appllcaLlons daLa communlcaLlons enormous relaLlonal daLabases and sheer slze of appllcaLlons have all conLrlbuLed Lo Lhe exponenLlal growLh ln sofLware/sysLem complexlLy And Lhe use of ob[ecLorlenLed Lechnlques can compllcaLe lnsLead of slmpllfy a pro[ecL unless lL ls wellenglneered - programmlng errors programmers llke anyone else can make mlsLakes - changlng requlremenLs (wheLher documenLed or undocumenLed) Lhe cusLomer may noL undersLand Lhe effecLs of changes or may undersLand and requesL Lhem anyway redeslgn reschedullng of englneers effecLs on oLher pro[ecLs work already compleLed LhaL may have Lo be redone or Lhrown ouL hardware requlremenLs LhaL may be affecLed eLc lf Lhere are many mlnor changes or any ma[or changes known and unknown dependencles among parLs of Lhe pro[ecL are llkely Lo lnLeracL and cause problems and Lhe complexlLy of coordlnaLlng changes may resulL ln errors LnLhuslasm of englneerlng sLaff may be affecLed ln some fasLchanglng buslness envlronmenLs conLlnuously modlfled requlremenLs may be a facL of llfe ln Lhls case managemenL musL undersLand Lhe resulLlng rlsks and CA and LesL englneers musL adapL and plan for conLlnuous exLenslve LesLlng Lo keep Lhe lnevlLable bugs from runnlng ouL of conLrol see 'WhaL can be done lf requlremenLs are changlng conLlnuously?' ln arL 2 of Lhe lAC - Llme pressures schedullng of sofLware pro[ecLs ls dlfflculL aL besL ofLen requlrlng a loL of guesswork When deadllnes loom and Lhe crunch comes mlsLakes wlll be made - egos people prefer Lo say Lhlngs llke 'no problem' 'plece of cake' 'l can whlp LhaL ouL ln a few hours' 'lL should be easy Lo updaLe LhaL old code' lnsLead of 'LhaL adds a loL of complexlLy and we could end up maklng a loL of mlsLakes' 'we have no ldea lf we can do LhaL we'll wlng lL' 'l can'L esLlmaLe how long lL wlll Lake unLll l Lake a close look aL lL' 'we can'L flgure ouL whaL LhaL old spagheLLl code dld ln Lhe flrsL place'
lf Lhere are Loo many unreallsLlc 'no problem's' Lhe resulL ls bugs
- poorly documenLed code lL's Lough Lo malnLaln and modlfy code LhaL ls badly wrlLLen or poorly documenLed Lhe resulL ls bugs ln many organlzaLlons managemenL provldes no lncenLlve for programmers Lo documenL Lhelr code or wrlLe clear undersLandable malnLalnable code ln facL lL's usually Lhe opposlLe Lhey geL polnLs mosLly for qulckly Lurnlng ouL code and Lhere's [ob securlLy lf nobody else can undersLand lL ('lf lL was hard Lo wrlLe lL should be hard Lo read') - sofLware developmenL Lools vlsual Lools class llbrarles compllers scrlpLlng Lools eLc ofLen lnLroduce Lhelr own bugs or are poorly documenLed resulLlng ln added bugs
now can new Software A processes be |ntroduced |n an ex|st|ng organ|zat|on?
- A loL depends on Lhe slze of Lhe organlzaLlon and Lhe rlsks lnvolved lor large organlzaLlons wlLh hlghrlsk (ln Lerms of llves or properLy) pro[ecLs serlous managemenL buyln ls requlred and a formallzed CA process ls necessary - Where Lhe rlsk ls lower managemenL and organlzaLlonal buyln and CA lmplemenLaLlon may be a slower sLepaLaLlme process CA processes should be balanced wlLh producLlvlLy so as Lo keep bureaucracy from geLLlng ouL of hand - lor small groups or pro[ecLs a more adhoc process may be approprlaLe dependlng on Lhe Lype of cusLomers and pro[ecLs A loL wlll depend on Leam leads or managers feedback Lo developers and ensurlng adequaLe communlcaLlons among cusLomers managers developers and LesLers - 1he mosL value for efforL wlll be ln (a) requlremenLs managemenL processes wlLh a goal of clear compleLe LesLable requlremenL speclflcaLlons embodled ln requlremenLs or deslgn documenLaLlon and (b) deslgn lnspecLlons and code lnspecLlons
What |s ver|f|cat|on? va||dat|on?
verlflcaLlon Lyplcally lnvolves revlews and meeLlngs Lo evaluaLe documenLs plans code requlremenLs and speclflcaLlons 1hls can be done wlLh checkllsLs lssues llsLs walkLhroughs and lnspecLlon meeLlngs valldaLlon Lyplcally lnvolves acLual LesLlng and Lakes place afLer verlflcaLlons are compleLed 1he Lerm 'lv v' refers Lo lndependenL verlflcaLlon and valldaLlon
What |s a 'wa|kthrough'?
A 'walkLhrough' ls an lnformal meeLlng for evaluaLlon or lnformaLlonal purposes LlLLle or no preparaLlon ls usually requlred
What's an '|nspect|on'?
An lnspecLlon ls more formallzed Lhan a 'walkLhrough' Lyplcally wlLh 38 people lncludlng a moderaLor reader and a recorder Lo Lake noLes 1he sub[ecL of Lhe lnspecLlon ls Lyplcally a documenL such as a requlremenLs spec or a LesL plan and Lhe purpose ls Lo flnd problems and see whaL's mlsslng noL Lo flx anyLhlng ALLendees should prepare for Lhls Lype of meeLlng by readlng Lhru Lhe documenL mosL problems wlll be found durlng Lhls preparaLlon 1he resulL of Lhe lnspecLlon meeLlng should be a wrlLLen reporL 1horough preparaLlon for lnspecLlons ls dlfflculL palnsLaklng work buL ls one of Lhe mosL cosL effecLlve meLhods of ensurlng quallLy Lmployees who are mosL skllled aL lnspecLlons are llke Lhe 'eldesL broLher' ln Lhe parable ln 'Why ls lL ofLen hard for managemenL Lo geL serlous abouL quallLy assurance?' 1helr sklll may have low vlslblllLy buL Lhey are exLremely valuable Lo any sofLware developmenL organlzaLlon slnce bug prevenLlon ls far more cosL effecLlve Lhan bug deLecLlon
What are S common prob|ems |n the software deve|opment process?
- poor requlremenLs lf requlremenLs are unclear lncompleLe Loo general or noL LesLable Lhere wlll be problems - unreallsLlc schedule lf Loo much work ls crammed ln Loo llLLle Llme problems are lnevlLable - lnadequaLe LesLlng no one wlll know wheLher or noL Lhe program ls any good unLll Lhe cusLomer complalns or sysLems crash - feaLurlLls requesLs Lo plle on new feaLures afLer developmenL ls underway exLremely common - mlscommunlcaLlon lf developers don'L know whaL's needed or cusLomers have erroneous expecLaLlons problems are guaranLeed
What are S common so|ut|ons to software deve|opment prob|ems?
- solld requlremenLs clear compleLe deLalled coheslve aLLalnable LesLable requlremenLs LhaL are agreed Lo by all players use proLoLypes Lo help nall down requlremenLs - reallsLlc schedules allow adequaLe Llme for plannlng deslgn LesLlng bug flxlng re LesLlng changes and documenLaLlon personnel should be able Lo compleLe Lhe pro[ecL wlLhouL burnlng ouL - adequaLe LesLlng sLarL LesLlng early on reLesL afLer flxes or changes plan for adequaLe Llme for LesLlng and bugflxlng - sLlck Lo lnlLlal requlremenLs as much as posslble be prepared Lo defend agalnsL changes and addlLlons once developmenL has begun and be prepared Lo explaln consequences lf changes are necessary Lhey should be adequaLely reflecLed ln relaLed schedule changes lf posslble use rapld proLoLyplng durlng Lhe deslgn phase so LhaL cusLomers can see whaL Lo expecL 1hls wlll provlde Lhem a hlgher comforL level wlLh Lhelr requlremenLs declslons and mlnlmlze changes laLer on - communlcaLlon requlre walkLhroughs and lnspecLlons when approprlaLe make exLenslve use of group communlcaLlon Lools emall groupware neLworked bugLracklng Lools and change managemenL Lools lnLraneL capablllLles eLc lnsure LhaL documenLaLlon ls avallable and upLodaLe preferably elecLronlc noL paper promoLe Leamwork and cooperaLlon use proLoLypes early on so LhaL cusLomers' expecLaLlons are clarlfled
What |s software 'qua||ty'?
CuallLy sofLware ls reasonably bugfree dellvered on Llme and wlLhln budgeL meeLs requlremenLs and/or expecLaLlons and ls malnLalnable Powever quallLy ls obvlously a sub[ecLlve Lerm lL wlll depend on who Lhe 'cusLomer' ls and Lhelr overall lnfluence ln Lhe scheme of Lhlngs A wldeangle vlew of Lhe 'cusLomers' of a sofLware developmenL pro[ecL mlghL lnclude endusers cusLomer accepLance LesLers cusLomer conLracL offlcers cusLomer managemenL Lhe developmenL organlzaLlon's managemenL/accounLanLs/LesLers/salespeople fuLure sofLware malnLenance englneers sLockholders magazlne columnlsLs eLc Lach Lype of 'cusLomer' wlll have Lhelr own slanL on 'quallLy' Lhe accounLlng deparLmenL mlghL deflne quallLy ln Lerms of proflLs whlle an end user mlghL deflne quallLy as userfrlendly and bugfree
What |s 'good code'?
'Cood code' ls code LhaL works ls bug free and ls readable and malnLalnable Some organlzaLlons have codlng 'sLandards' LhaL all developers are supposed Lo adhere Lo buL everyone has dlfferenL ldeas abouL whaL's besL or whaL ls Loo many or Loo few rules 1here are also varlous Lheorles and meLrlcs such as McCabe ComplexlLy meLrlcs lL should be kepL ln mlnd LhaL excesslve use of sLandards and rules can sLlfle producLlvlLy and creaLlvlLy 'eer revlews' 'buddy checks' code analysls Lools eLc can be used Lo check for problems and enforce sLandards
lor C and C++ codlng here are some Lyplcal ldeas Lo conslder ln seLLlng rules/sLandards Lhese may or may noL apply Lo a parLlcular slLuaLlon - mlnlmlze or ellmlnaLe use of global varlables - use descrlpLlve funcLlon and meLhod names use boLh upper and lower case avold abbrevlaLlons use as many characLers as necessary Lo be adequaLely descrlpLlve (use of more Lhan 20 characLers ls noL ouL of llne) be conslsLenL ln namlng convenLlons - use descrlpLlve varlable names use boLh upper and lower case avold abbrevlaLlons use as many characLers as necessary Lo be adequaLely descrlpLlve (use of more Lhan 20 characLers ls noL ouL of llne) be conslsLenL ln namlng convenLlons - funcLlon and meLhod slzes should be mlnlmlzed less Lhan 100 llnes of code ls good less Lhan 30 llnes ls preferable - funcLlon descrlpLlons should be clearly spelled ouL ln commenLs precedlng a funcLlon's code - organlze code for readablllLy - use whlLe space generously verLlcally and horlzonLally - each llne of code should conLaln 70 characLers max - one code sLaLemenL per llne - codlng sLyle should be conslsLenL LhoughL a program (eg use of brackeLs lndenLaLlons namlng convenLlons eLc) - ln addlng commenLs err on Lhe slde of Loo many raLher Lhan Loo few commenLs a common rule of Lhumb ls LhaL Lhere should be aL leasL as many llnes of commenLs (lncludlng header blocks) as llnes of code - no maLLer how small an appllcaLlon should lnclude documenLaLlon of Lhe overall program funcLlon and flow (even a few paragraphs ls beLLer Lhan noLhlng) or lf posslble a separaLe flow charL and deLalled program documenLaLlon - make exLenslve use of error handllng procedures and sLaLus and error logglng - for C++ Lo mlnlmlze complexlLy and lncrease malnLalnablllLy avold Loo many levels of lnherlLance ln class hlerarchles (relaLlve Lo Lhe slze and complexlLy of Lhe appllcaLlon) Mlnlmlze use of mulLlple lnherlLance and mlnlmlze use of operaLor overloadlng (noLe LhaL Lhe !ava programmlng language ellmlnaLes mulLlple lnherlLance and operaLor overloadlng) - for C++ keep class meLhods small less Lhan 30 llnes of code per meLhod ls preferable - for C++ make llberal use of excepLlon handlers
What |s 'good des|gn'?
'ueslgn' could refer Lo many Lhlngs buL ofLen refers Lo 'funcLlonal deslgn' or 'lnLernal deslgn' Cood lnLernal deslgn ls lndlcaLed by sofLware code whose overall sLrucLure ls clear undersLandable easlly modlflable and malnLalnable ls robusL wlLh sufflclenL errorhandllng and sLaLus logglng capablllLy and works correcLly when lmplemenLed Cood funcLlonal deslgn ls lndlcaLed by an appllcaLlon whose funcLlonallLy can be Lraced back Lo cusLomer and enduser requlremenLs (See furLher dlscusslon of funcLlonal and lnLernal deslgn ln 'WhaL's Lhe blg deal abouL requlremenLs?' ln lAC #2) lor programs LhaL have a user lnLerface lL's ofLen a good ldea Lo assume LhaL Lhe end user wlll have llLLle compuLer knowledge and may noL read a user manual or even Lhe onllne help some common rules ofLhumb lnclude - Lhe program should acL ln a way LhaL leasL surprlses Lhe user - lL should always be evldenL Lo Lhe user whaL can be done nexL and how Lo exlL - Lhe program shouldn'L leL Lhe users do someLhlng sLupld wlLhouL warnlng Lhem
What |s SLI? CMM? ISC? ILLL? ANSI? W||| |t he|p?
- SLl 'SofLware Lnglneerlng lnsLlLuLe' aL CarnegleMellon unlverslLy lnlLlaLed by Lhe uS uefense ueparLmenL Lo help lmprove sofLware developmenL processes - CMM 'CapablllLy MaLurlLy Model' developed by Lhe SLl lL's a model of 3 levels of organlzaLlonal 'maLurlLy' LhaL deLermlne effecLlveness ln dellverlng quallLy sofLware lL ls geared Lo large organlzaLlons such as large uS uefense ueparLmenL conLracLors Powever many of Lhe CA processes lnvolved are approprlaLe Lo any organlzaLlon and lf reasonably applled can be helpful CrganlzaLlons can recelve CMM raLlngs by undergolng assessmenLs by quallfled audlLors
Level 1 characLerlzed by chaos perlodlc panlcs and herolc efforLs requlred by lndlvlduals Lo successfully compleLe pro[ecLs lew lf any processes ln place successes may noL be repeaLable
Level 2 sofLware pro[ecL Lracklng requlremenLs managemenL reallsLlc plannlng and conflguraLlon managemenL processes are ln place successful pracLlces can be repeaLed
Level 3 sLandard sofLware developmenL and malnLenance processes are lnLegraLed LhroughouL an organlzaLlon a SofLware Lnglneerlng rocess Croup ls ln place Lo oversee sofLware processes and Lralnlng programs are used Lo ensure undersLandlng and compllance
Level 4 meLrlcs are used Lo Lrack producLlvlLy processes and producLs ro[ecL performance ls predlcLable and quallLy ls conslsLenLly hlgh
Level 3 Lhe focus ls on conLlnuous process lmprovemenL 1he lmpacL of new processes and Lechnologles can be predlcLed and effecLlvely lmplemenLed when requlred
erspecLlve on CMM raLlngs uurlng 19972001 1018 organlzaLlons were assessed Cf Lhose 27 were raLed aL Level 1 39 aL 2 23 aL 3 6 aL 4 and 3 aL 3 (lor raLlngs durlng Lhe perlod 199296 62 were aL Level 1 23 aL 2 13 aL 3 2 aL 4 and 04 aL 3) 1he medlan slze of organlzaLlons was 100 sofLware englneerlng/malnLenance personnel 32 of organlzaLlons were uS federal conLracLors or agencles lor Lhose raLed aL Level 1 Lhe mosL problemaLlcal key process area was ln SofLware CuallLy Assurance
- lSC 'lnLernaLlonal CrganlzaLlon for SLandardlzaLlon' 1he lSC 90012000 sLandard (whlch replaces Lhe prevlous sLandard of 1994) concerns quallLy sysLems LhaL are assessed by ouLslde audlLors and lL applles Lo many klnds of producLlon and manufacLurlng organlzaLlons noL [usL sofLware lL covers documenLaLlon deslgn developmenL producLlon LesLlng lnsLallaLlon servlclng and oLher processes 1he full seL of sLandards conslsLs of (a)C90012000 CuallLy ManagemenL SysLems 8equlremenLs (b)C90002000 CuallLy ManagemenL SysLems lundamenLals and vocabulary (c)C90042000 CuallLy ManagemenL SysLems Culdellnes for erformance lmprovemenLs 1o be lSC 9001 cerLlfled a LhlrdparLy audlLor assesses an organlzaLlon and cerLlflcaLlon ls Lyplcally good for abouL 3 years afLer whlch a compleLe reassessmenL ls requlred noLe LhaL lSC cerLlflcaLlon does noL necessarlly lndlcaLe quallLy producLs lL lndlcaLes only LhaL documenLed processes are followed Also see
hLLp//wwwlsoch/ for Lhe laLesL lnformaLlon ln Lhe uS Lhe sLandards can be purchased vla Lhe ASC web slLe aL hLLp//esLandardsasqorg/
- lLLL 'lnsLlLuLe of LlecLrlcal and LlecLronlcs Lnglneers' among oLher Lhlngs creaLes sLandards such as 'lLLL SLandard for SofLware 1esL uocumenLaLlon' (lLLL/AnSl SLandard 829) 'lLLL SLandard of SofLware unlL 1esLlng (lLLL/AnSl SLandard 1008) 'lLLL SLandard for SofLware CuallLy Assurance lans' (lLLL/AnSl SLandard 730) and oLhers
- AnSl 'Amerlcan naLlonal SLandards lnsLlLuLe' Lhe prlmary lndusLrlal sLandards body ln Lhe uS publlshes some sofLwarerelaLed sLandards ln con[uncLlon wlLh Lhe lLLL and ASC (Amerlcan SocleLy for CuallLy)
- CLher sofLware developmenL process assessmenL meLhods besldes CMM and lSC 9000 lnclude SlCL 1rllllum and 1lck l1 and 8ooLsLrap
What |s the 'software ||fe cyc|e'?
1he llfe cycle beglns when an appllcaLlon ls flrsL concelved and ends when lL ls no longer ln use lL lncludes aspecLs such as lnlLlal concepL requlremenLs analysls funcLlonal deslgn lnLernal deslgn documenLaLlon plannlng LesL plannlng codlng documenL preparaLlon lnLegraLlon LesLlng malnLenance updaLes reLesLlng phaseouL and oLher aspecLs
W||| automated test|ng too|s make test|ng eas|er?
- osslbly lor small pro[ecLs Lhe Llme needed Lo learn and lmplemenL Lhem may noL be worLh lL lor larger pro[ecLs or ongolng longLerm pro[ecLs Lhey can be valuable - A common Lype of auLomaLed Lool ls Lhe 'record/playback' Lype lor example a LesLer could cllck Lhrough all comblnaLlons of menu cholces dlalog box cholces buLLons eLc ln an appllcaLlon Cul and have Lhem 'recorded' and Lhe resulLs logged by a Lool 1he 'recordlng' ls Lyplcally ln Lhe form of LexL based on a scrlpLlng language LhaL ls lnLerpreLable by Lhe LesLlng Lool lf new buLLons are added or some underlylng code ln Lhe appllcaLlon ls changed eLc Lhe appllcaLlon mlghL Lhen be reLesLed by [usL 'playlng back' Lhe 'recorded' acLlons and comparlng Lhe logglng resulLs Lo check effecLs of Lhe changes 1he problem wlLh such Lools ls LhaL lf Lhere are conLlnual changes Lo Lhe sysLem belng LesLed Lhe 'recordlngs' may have Lo be changed so much LhaL lL becomes very Llmeconsumlng Lo conLlnuously updaLe Lhe scrlpLs AddlLlonally lnLerpreLaLlon and analysls of resulLs (screens daLa logs eLc) can be a dlfflculL Lask noLe LhaL Lhere are record/playback Lools for LexLbased lnLerfaces also and for all Lypes of plaLforms - CLher auLomaLed Lools can lnclude code analyzers monlLor code complexlLy adherence Lo sLandards eLc coverage analyzers Lhese Lools check whlch parLs of Lhe code have been exerclsed by a LesL and may be orlenLed Lo code sLaLemenL coverage condlLlon coverage paLh coverage eLc memory analyzers such as boundscheckers and leak deLecLors load/performance LesL Lools for LesLlng cllenL/server and web appllcaLlons under varlous load levels web LesL Lools Lo check LhaL llnks are valld P1ML code usage ls correcL cllenLslde and serverslde programs work a web slLe's lnLeracLlons are secure
oLher Lools for LesL case managemenL documenLaLlon managemenL bug reporLlng and conflguraLlon managemenL
Why you cannot down|oad a Word vers|on of th|s test p|an
l have recelved numerous requesLs for an MS Word verslon of Lhe LesL plan Powever alLhough Lhe web pages were creaLed dlrecLly from an word documenL l no longer have a copy of LhaL orlglnal word documenL Also havlng prepared numerous LesL plans l know LhaL Lhe conLenL ls more lmporLanL Lhan Lhe formaL See Lhe nexL polnL for more lnfo on Lhe conLenL of a LesL plan
What a test p|an shou|d conta|n
A sofLware pro[ecL LesL plan ls a documenL LhaL descrlbes Lhe ob[ecLlves scope approach and focus of a sofLware LesLlng efforL 1he process of preparlng a LesL plan ls a useful way Lo Lhlnk Lhrough Lhe efforLs needed Lo valldaLe Lhe accepLablllLy of a sofLware producL 1he compleLed documenL wlll help people ouLslde Lhe LesL group undersLand Lhe 'why' and 'how' of producL valldaLlon lL should be Lhorough enough Lo be useful buL noL so Lhorough LhaL no one ouLslde Lhe LesL group wlll read lL
A LesL plan sLaLes whaL Lhe lLems Lo be LesLed are aL whaL level Lhey wlll be LesLed whaL sequence Lhey are Lo be LesLed ln how Lhe LesL sLraLegy wlll be applled Lo Lhe LesLlng of each lLem and descrlbes Lhe LesL envlronmenL
A LesL plan should ldeally be organlzaLlon wlde belng appllcable Lo all of organlzaLlons sofLware developmenLs
1he ob[ecLlve of each LesL plan ls Lo provlde a plan for verlflcaLlon by LesLlng Lhe sofLware Lhe sofLware produced fulflls Lhe funcLlonal or deslgn sLaLemenLs of Lhe approprlaLe sofLware speclflcaLlon ln Lhe case of accepLance LesLlng and sysLem LesLlng Lhls generally means Lhe luncLlonal SpeclflcaLlon
1he flrsL conslderaLlon when preparlng Lhe 1esL lan ls who Lhe lnLended audlence ls le Lhe audlence for a unlL 1esL lan would be dlfferenL and Lhus Lhe conLenL would have Lo be ad[usLed accordlngly
?ou should begln Lhe LesL plan as soon as posslble Cenerally lL ls deslrable Lo begln Lhe masLer LesL plan as Lhe same Llme Lhe 8equlremenLs documenLs and Lhe ro[ecL lan are belng developed 1esL plannlng can (and should) have an lmpacL on Lhe ro[ecL lan Lven Lhough plans LhaL are wrlLLen early wlll have Lo be changed durlng Lhe course of Lhe developmenL and LesLlng buL LhaL ls lmporLanL because lL records Lhe progress of Lhe LesLlng and helps planners become more proflclenL
What to cons|der for the 1est |an
1 1esL lan ldenLlfler 2 8eferences 3 lnLroducLlon 4 1esL lLems 3 SofLware 8lsk lssues 6 leaLures Lo be 1esLed 7 leaLures noL Lo be 1esLed 8 Approach 9 lLem ass/lall CrlLerla 10 Suspenslon CrlLerla and 8esumpLlon 8equlremenLs 11 1esL uellverables 12 8emalnlng 1esL 1asks 13 LnvlronmenLal needs 14 SLafflng and 1ralnlng needs 13 8esponslblllLles 16 Schedule 17 lannlng 8lsks and ConLlngencles 18 Approvals 19 Clossary
Standards for Software 1est |ans
Several sLandards suggesL whaL a LesL plan should conLaln lncludlng Lhe lLLL 1he sLandards are
lLLL sLandards 8291983 lLLL SLandard for SofLware 1esL uocumenLaLlon 10081987 lLLL SLandard for SofLware unlL 1esLlng 10121986 lLLL SLandard for SofLware verlflcaLlon valldaLlon lans 10391993 lLLL Culde for SofLware verlflcaLlon valldaLlon lans
1he lLLL webslLe ls herehLLp//wwwleeeorg
Why I pub||shed the test p|an
Well when l flrsL wenL looklng for sample LesL plans l could noL flnd anyLhlng useful LvenLually l found several slLes whlch l lncluded on my llnks page Powever l was noL saLlsfled wlLh many of Lhe plans LhaL l found so l posLed lL on my webslLe And l have Lo say LhaL l have been asLounded by Lhe level of lnLeresL shown and amazed aL Lhe number of emalls l have recelved abouL lL
Copyr|ght Cwnersh|p what you can do w|th the p|an
Well l publlshed wlLh Lhe alm LhaL lL be used so lf you are golng Lo use lL Lo creaLe a LesL plan for lnLernal use please feel free Lo copy from lL Powever lf Lhe LesL plan ls Lo be publlshed exLernally ln any way web magazlne Lralnlng maLerlal eLc Lhen you musL lnclude a reference Lo me 8azman and a llnk Lo my webslLe
What makes a good test eng|neer?
- A good LesL englneer has a 'LesL Lo break' aLLlLude an ablllLy Lo Lake Lhe polnL of vlew of Lhe cusLomer a sLrong deslre for quallLy and an aLLenLlon Lo deLall - 1acL and dlplomacy are useful ln malnLalnlng a cooperaLlve relaLlonshlp wlLh developers and an ablllLy Lo communlcaLe wlLh boLh Lechnlcal (developers) and nonLechnlcal (cusLomers managemenL) people ls useful - revlous sofLware developmenL experlence can be helpful as lL provldes a deeper undersLandlng of Lhe sofLware developmenL process glves Lhe LesLer an appreclaLlon for Lhe developers' polnL of vlew and reduce Lhe learnlng curve ln auLomaLed LesL Lool programmlng - !udgmenL skllls are needed Lo assess hlghrlsk areas of an appllcaLlon on whlch Lo focus LesLlng efforLs when Llme ls llmlLed
What makes a good software A eng|neer?
- 1he same quallLles a good LesLer has are useful for a CA englneer - AddlLlonally Lhey musL be able Lo undersLand Lhe enLlre sofLware developmenL process and how lL can flL lnLo Lhe buslness approach and goals of Lhe organlzaLlon - CommunlcaLlon skllls and Lhe ablllLy Lo undersLand varlous sldes of lssues are lmporLanL - ln organlzaLlons ln Lhe early sLages of lmplemenLlng CA processes paLlence and dlplomacy are especlally needed - An ablllLy Lo flnd problems as well as Lo see 'whaL's mlsslng' ls lmporLanL for lnspecLlons and revlews
What makes a good A or 1est manager?
- A good CA LesL or CA/1esL (comblned) manager should - be famlllar wlLh Lhe sofLware developmenL process - be able Lo malnLaln enLhuslasm of Lhelr Leam and promoLe a poslLlve aLmosphere desplLe
what |s a somewhat 'negat|ve' process (eg |ook|ng for or prevent|ng prob|ems)
- be able Lo promoLe Leamwork Lo lncrease producLlvlLy - be able Lo promoLe cooperaLlon beLween sofLware LesL and CA englneers - have Lhe dlplomaLlc skllls needed Lo promoLe lmprovemenLs ln CA processes - have Lhe ablllLy Lo wlLhsLand pressures and say 'no' Lo oLher managers when quallLy ls lnsufflclenL or CA processes are noL belng adhered Lo - have people [udgmenL skllls for hlrlng and keeplng skllled personnel - be able Lo communlcaLe wlLh Lechnlcal and nonLechnlcal people englneers managers and cusLomers - be able Lo run meeLlngs and keep Lhem focused
What's the ro|e of documentat|on |n A?
- CrlLlcal (noLe LhaL documenLaLlon can be elecLronlc noL necessarlly paper) - CA pracLlces should be documenLed such LhaL Lhey are repeaLable - SpeclflcaLlons deslgns buslness rules lnspecLlon reporLs conflguraLlons code changes LesL plans LesL cases bug reporLs user manuals eLc should all be documenLed - 1here should ldeally be a sysLem for easlly flndlng and obLalnlng documenLs and deLermlnlng whaL documenLaLlon wlll have a parLlcular plece of lnformaLlon - Change managemenL for documenLaLlon should be used lf posslble
What's the b|g dea| about 'requ|rements'?
- Cne of Lhe mosL rellable meLhods of lnsurlng problems or fallure ln a complex sofLware pro[ecL ls Lo have poorly documenLed requlremenLs speclflcaLlons - 8equlremenLs are Lhe deLalls descrlblng an appllcaLlon's exLernallypercelved funcLlonallLy and properLles - 8equlremenLs should be clear compleLe reasonably deLalled coheslve aLLalnable and LesLable A nonLesLable requlremenL would be for example 'userfrlendly' (Loo sub[ecLlve) - A LesLable requlremenL would be someLhlng llke 'Lhe user musL enLer Lhelr prevlously asslgned password Lo access Lhe appllcaLlon' - ueLermlnlng and organlzlng requlremenLs deLalls ln a useful and efflclenL way can be a dlfflculL efforL dlfferenL meLhods are avallable dependlng on Lhe parLlcular pro[ecL - Many books are avallable LhaL descrlbe varlous approaches Lo Lhls Lask - Care should be Laken Lo lnvolve ALL of a pro[ecL's slgnlflcanL 'cusLomers' ln Lhe requlremenLs process - 'CusLomers' could be lnhouse personnel or ouL and could lnclude endusers cusLomer accepLance LesLers cusLomer conLracL offlcers cusLomer managemenL fuLure sofLware malnLenance englneers salespeople eLc - Anyone who could laLer derall Lhe pro[ecL lf Lhelr expecLaLlons aren'L meL should be lncluded lf posslble - CrganlzaLlons vary conslderably ln Lhelr handllng of requlremenLs speclflcaLlons - ldeally Lhe requlremenLs are spelled ouL ln a documenL wlLh sLaLemenLs such as '1he producL shall' - 'ueslgn' speclflcaLlons should noL be confused wlLh 'requlremenLs' deslgn speclflcaLlons should be Lraceable back Lo Lhe requlremenLs - ln some organlzaLlons requlremenLs may end up ln hlgh level pro[ecL plans funcLlonal speclflcaLlon documenLs ln deslgn documenLs or ln oLher documenLs aL varlous levels of deLall - no maLLer whaL Lhey are called some Lype of documenLaLlon wlLh deLalled requlremenLs wlll be needed by LesLers ln order Lo properly plan and execuLe LesLs - WlLhouL such documenLaLlon Lhere wlll be no clearcuL way Lo deLermlne lf a sofLware appllcaLlon ls performlng correcLly
What steps are needed to deve|op and run software tests?
1he followlng are some of Lhe sLeps Lo conslder - CbLaln requlremenLs funcLlonal deslgn and lnLernal deslgn speclflcaLlons and oLher necessary documenLs - CbLaln budgeL and schedule requlremenLs - ueLermlne pro[ecLrelaLed personnel and Lhelr responslblllLles reporLlng requlremenLs requlred sLandards and processes (such as release processes change processes eLc) - ldenLlfy appllcaLlon's hlgherrlsk aspecLs seL prlorlLles and deLermlne scope and llmlLaLlons of LesLs - ueLermlne LesL approaches and meLhods unlL lnLegraLlon funcLlonal sysLem load usablllLy LesLs eLc - ueLermlne LesL envlronmenL requlremenLs (hardware sofLware communlcaLlons eLc) - ueLermlne LesL ware requlremenLs (record/playback Lools coverage analyzers LesL Lracklng problem/bug Lracklng eLc) - ueLermlne LesL lnpuL daLa requlremenLs - ldenLlfy Lasks Lhose responslble for Lasks and labor requlremenLs - SeL schedule esLlmaLes Llmellnes mllesLones - ueLermlne lnpuL equlvalence classes boundary value analyses error classes - repare LesL plan documenL and have needed revlews/approvals - WrlLe LesL cases - Pave needed revlews/lnspecLlons/approvals of LesL cases - repare LesL envlronmenL and LesL ware obLaln needed user manuals/reference documenLs/conflguraLlon guldes/lnsLallaLlon guldes seL up LesL Lracklng processes seL up logglng and archlvlng processes seL up or obLaln LesL lnpuL daLa - CbLaln and lnsLall sofLware releases - erform LesLs - LvaluaLe and reporL resulLs - 1rack problems/bugs and flxes - 8eLesL as needed - MalnLaln and updaLe LesL plans LesL cases LesL envlronmenL and LesL ware Lhrough llfe cycle
What's a 'test p|an'?
A sofLware pro[ecL LesL plan ls a documenL LhaL descrlbes Lhe ob[ecLlves scope approach and focus of a sofLware LesLlng efforL 1he process of preparlng a LesL plan ls a useful way Lo Lhlnk Lhrough Lhe efforLs needed Lo valldaLe Lhe accepLablllLy of a sofLware producL 1he compleLed documenL wlll help people ouLslde Lhe LesL group undersLand Lhe 'why' and 'how' of producL valldaLlon lL should be Lhorough enough Lo be useful buL noL so Lhorough LhaL no one ouLslde Lhe LesL group wlll read lL 1he followlng are some of Lhe lLems LhaL mlghL be lncluded ln a LesL plan dependlng on Lhe parLlcular pro[ecL - 1lLle - ldenLlflcaLlon of sofLware lncludlng verslon/release numbers - 8evlslon hlsLory of documenL lncludlng auLhors daLes approvals - 1able of ConLenLs - urpose of documenL lnLended audlence - Cb[ecLlve of LesLlng efforL - SofLware producL overvlew - 8elevanL relaLed documenL llsL such as requlremenLs deslgn documenLs oLher LesL plans eLc - 8elevanL sLandards or legal requlremenLs - 1raceablllLy requlremenLs - 8elevanL namlng convenLlons and ldenLlfler convenLlons - Cverall sofLware pro[ecL organlzaLlon and personnel/conLacLlnfo/responslblllLles - 1esL organlzaLlon and personnel/conLacLlnfo/responslblllLles - AssumpLlons and dependencles - ro[ecL rlsk analysls - 1esLlng prlorlLles and focus - Scope and llmlLaLlons of LesLlng - 1esL ouLllne a decomposlLlon of Lhe LesL approach by LesL Lype feaLure funcLlonallLy process sysLem module eLc as appllcable - CuLllne of daLa lnpuL equlvalence classes boundary value analysls error classes - 1esL envlronmenL hardware operaLlng sysLems oLher requlred sofLware daLa conflguraLlons lnLerfaces Lo oLher sysLems - 1esL envlronmenL seLup and conflguraLlon lssues - 1esL daLa seLup requlremenLs - uaLabase seLup requlremenLs - CuLllne of sysLemlogglng/errorlogglng/oLher capablllLles and Lools such as screen capLure sofLware LhaL wlll be used Lo help descrlbe and reporL bugs - ulscusslon of any speclallzed sofLware or hardware Lools LhaL wlll be used by LesLers Lo help Lrack Lhe cause or source of bugs - 1esL auLomaLlon [usLlflcaLlon and overvlew - 1esL Lools Lo be used lncludlng verslons paLches eLc - 1esL scrlpL/LesL code malnLenance processes and verslon conLrol - roblem Lracklng and resoluLlon Lools and processes - ro[ecL LesL meLrlcs Lo be used - 8eporLlng requlremenLs and LesLlng dellverables - SofLware enLrance and exlL crlLerla - lnlLlal sanlLy LesLlng perlod and crlLerla - 1esL suspenslon and resLarL crlLerla - ersonnel allocaLlon - ersonnel preLralnlng needs - 1esL slLe/locaLlon - CuLslde LesL organlzaLlons Lo be uLlllzed and Lhelr purpose responslblllLles dellverables conLacL persons and coordlnaLlon lssues - 8elevanL proprleLary classlfled securlLy and llcenslng lssues - Cpen lssues - Appendlx glossary acronyms eLc
What's a 'test case'?
- A LesL case ls a documenL LhaL descrlbes an lnpuL acLlon or evenL and an expecLed response Lo deLermlne lf a feaLure of an appllcaLlon ls worklng correcLly - A LesL case should conLaln parLlculars such as LesL case ldenLlfler LesL case name ob[ecLlve LesL condlLlons/seLup lnpuL daLa requlremenLs sLeps and expecLed resulLs - noLe LhaL Lhe process of developlng LesL cases can help flnd problems ln Lhe requlremenLs or deslgn of an appllcaLlon slnce lL requlres compleLely Lhlnklng Lhrough Lhe operaLlon of Lhe appllcaLlon lor Lhls reason lL's useful Lo prepare LesL cases early ln Lhe developmenL cycle lf posslble
What shou|d be done after a bug |s found?
1he bug needs Lo be communlcaLed and asslgned Lo developers who can flx lL AfLer Lhe problem ls resolved flxes should be reLesLed and deLermlnaLlons made regardlng requlremenLs for regresslon LesLlng Lo check LhaL flxes dldn'L creaLe problems elsewhere lf a problemLracklng sysLem ls ln place lL should encapsulaLe Lhese processes A varleLy of commerclal problemLracklng/managemenL sofLware Lools are avallable 1he followlng are lLems Lo be consldered ln Lhe Lracklng process - CompleLe lnformaLlon such LhaL developers can undersLand Lhe bug geL an ldea of lL's severlLy and reproduce lL lf necessary - 8ug ldenLlfler (number lu eLc) - CurrenL bug sLaLus (eg '8eleased for 8eLesL' 'new' eLc) - 1he appllcaLlon name or ldenLlfler and verslon - 1he funcLlon module feaLure ob[ecL screen eLc where Lhe bug occurred - LnvlronmenL speclflcs sysLem plaLform relevanL hardware speclflcs - 1esL case name/number/ldenLlfler - Cnellne bug descrlpLlon - lull bug descrlpLlon - uescrlpLlon of sLeps needed Lo reproduce Lhe bug lf noL covered by a LesL case or lf Lhe developer doesn'L have easy access Lo Lhe LesL case/LesL scrlpL/LesL Lool - names and/or descrlpLlons of flle/daLa/messages/eLc used ln LesL - llle excerpLs/error messages/log flle excerpLs/screen shoLs/LesL Lool logs LhaL would be helpful ln flndlng Lhe cause of Lhe problem - SeverlLy esLlmaLe (a 3level range such as 13 or 'crlLlcal'Lo'low' ls common) - Was Lhe bug reproduclble? - 1esLer name - 1esL daLe - 8ug reporLlng daLe - name of developer/group/organlzaLlon Lhe problem ls asslgned Lo - uescrlpLlon of problem cause - uescrlpLlon of flx - Code secLlon/flle/module/class/meLhod LhaL was flxed - uaLe of flx - AppllcaLlon verslon LhaL conLalns Lhe flx - 1esLer responslble for reLesL - 8eLesL daLe - 8eLesL resulLs - 8egresslon LesLlng requlremenLs - 1esLer responslble for regresslon LesLs - 8egresslon LesLlng resulLs A reporLlng or Lracklng process should enable noLlflcaLlon of approprlaLe personnel aL varlous sLages lor lnsLance LesLers need Lo know when reLesLlng ls needed developers need Lo know when bugs are found and how Lo geL Lhe needed lnformaLlon and reporLlng/summary capablllLles are needed for managers
What |s 'conf|gurat|on management'?
ConflguraLlon managemenL covers Lhe processes used Lo conLrol coordlnaLe and Lrack code requlremenLs documenLaLlon problems change requesLs deslgns Lools/compllers/llbrarles/paLches changes made Lo Lhem and who makes Lhe changes
What |f the software |s so buggy |t can't rea||y be tested at a||?
1he besL beL ln Lhls slLuaLlon ls for Lhe LesLers Lo go Lhrough Lhe process of reporLlng whaLever bugs or blocklngLype problems lnlLlally show up wlLh Lhe focus belng on crlLlcal bugs Slnce Lhls Lype of problem can severely affecL schedules and lndlcaLes deeper problems ln Lhe sofLware developmenL process (such as lnsufflclenL unlL LesLlng or lnsufflclenL lnLegraLlon LesLlng poor deslgn lmproper bulld or release procedures eLc) managers should be noLlfled and provlded wlLh some documenLaLlon as evldence of Lhe problem
now can |t be known when to stop test|ng?
1hls can be dlfflculL Lo deLermlne Many modern sofLware appllcaLlons are so complex and run ln such an lnLerdependenL envlronmenL LhaL compleLe LesLlng can never be done Common facLors ln decldlng when Lo sLop are - ueadllnes (release deadllnes LesLlng deadllnes eLc) - 1esL cases compleLed wlLh cerLaln percenLage passed - 1esL budgeL depleLed - Coverage of code/funcLlonallLy/requlremenLs reaches a speclfled polnL - 8ug raLe falls below a cerLaln level - 8eLa or alpha LesLlng perlod ends
What |f there |sn't enough t|me for thorough test|ng?
use rlsk analysls Lo deLermlne where LesLlng should be focused
Slnce lL's rarely posslble Lo LesL every posslble aspecL of an appllcaLlon every posslble comblnaLlon of evenLs every dependency or everyLhlng LhaL could go wrong rlsk analysls ls approprlaLe Lo mosL sofLware developmenL pro[ecLs 1hls requlres [udgmenL skllls common sense and experlence (lf warranLed formal meLhods are also avallable)
ConslderaLlons can lnclude
- Whlch funcLlonallLy ls mosL lmporLanL Lo Lhe pro[ecL's lnLended purpose? - Whlch funcLlonallLy ls mosL vlslble Lo Lhe user? - Whlch funcLlonallLy has Lhe largesL safeLy lmpacL? - Whlch funcLlonallLy has Lhe largesL flnanclal lmpacL on users? - Whlch aspecLs of Lhe appllcaLlon are mosL lmporLanL Lo Lhe cusLomer? - Whlch aspecLs of Lhe appllcaLlon can be LesLed early ln Lhe developmenL cycle? - Whlch parLs of Lhe code are mosL complex and Lhus mosL sub[ecL Lo errors? - Whlch parLs of Lhe appllcaLlon were developed ln rush or panlc mode? - Whlch aspecLs of slmllar/relaLed prevlous pro[ecLs caused problems? - Whlch aspecLs of slmllar/relaLed prevlous pro[ecLs had large malnLenance expenses? - Whlch parLs of Lhe requlremenLs and deslgn are unclear or poorly LhoughL ouL? - WhaL do Lhe developers Lhlnk are Lhe hlghesLrlsk aspecLs of Lhe appllcaLlon? - WhaL klnds of problems would cause Lhe worsL publlclLy? - WhaL klnds of problems would cause Lhe mosL cusLomer servlce complalnLs? - WhaL klnds of LesLs could easlly cover mulLlple funcLlonallLles? - Whlch LesLs wlll have Lhe besL hlghrlskcoverage Lo Llmerequlred raLlo?
What |f the pro[ect |sn't b|g enough to [ust|fy extens|ve test|ng?
Conslder Lhe lmpacL of pro[ecL errors noL Lhe slze of Lhe pro[ecL Powever lf exLenslve LesLlng ls sLlll noL [usLlfled rlsk analysls ls agaln needed and Lhe same conslderaLlons as descrlbed prevlously ln 1he LesLer mlghL Lhen do ad hoc LesLlng or wrlLe up a llmlLed LesL plan based on Lhe rlsk analysls
What can be done |f requ|rements are chang|ng cont|nuous|y?
A common problem and a ma[or headache - Work wlLh Lhe pro[ecL's sLakeholders early on Lo undersLand how requlremenLs mlghL change so LhaL alLernaLe LesL plans and sLraLegles can be worked ouL ln advance lf posslble - lL's helpful lf Lhe appllcaLlon's lnlLlal deslgn allows for some adapLablllLy so LhaL laLer changes do noL requlre redolng Lhe appllcaLlon from scraLch - lf Lhe code ls wellcommenLed and welldocumenLed Lhls makes changes easler for Lhe developers - use rapld proLoLyplng whenever posslble Lo help cusLomers feel sure of Lhelr requlremenLs and mlnlmlze changes - 1he pro[ecL's lnlLlal schedule should allow for some exLra Llme commensuraLe wlLh Lhe posslblllLy of changes - 1ry Lo move new requlremenLs Lo a 'hase 2r verslon of an appllcaLlon whlle uslng Lhe orlglnal requlremenLs for Lhe 'hase 1r verslon - negoLlaLe Lo allow only easllylmplemenLed new requlremenLs lnLo Lhe pro[ecL whlle movlng more dlfflculL new requlremenLs lnLo fuLure verslons of Lhe appllcaLlon - 8e sure LhaL cusLomers and managemenL undersLand Lhe schedullng lmpacLs lnherenL rlsks and cosLs of slgnlflcanL requlremenLs changes 1hen leL managemenL or Lhe cusLomers (noL Lhe developers or LesLers) declde lf Lhe changes are warranLed afLer all LhaL's Lhelr [ob - 8alance Lhe efforL puL lnLo seLLlng up auLomaLed LesLlng wlLh Lhe expecLed efforL requlred Lo redo Lhem Lo deal wlLh changes - 1ry Lo deslgn some flexlblllLy lnLo auLomaLed LesL scrlpLs - locus lnlLlal auLomaLed LesLlng on appllcaLlon aspecLs LhaL are mosL llkely Lo remaln unchanged - uevoLe approprlaLe efforL Lo rlsk analysls of changes Lo mlnlmlze regresslon LesLlng needs - ueslgn some flexlblllLy lnLo LesL cases (Lhls ls noL easlly done Lhe besL beL mlghL be Lo mlnlmlze Lhe deLall ln Lhe LesL cases or seL up only hlgherlevel generlcLype LesL plans) - locus less on deLalled LesL plans and LesL cases and more on ad hoc LesLlng (wlLh an undersLandlng of Lhe added rlsk LhaL Lhls enLalls)
What |f the app||cat|on has funct|ona||ty that wasn't |n the requ|rements?
lL may Lake serlous efforL Lo deLermlne lf an appllcaLlon has slgnlflcanL unexpecLed or hldden funcLlonallLy and lL would lndlcaLe deeper problems ln Lhe sofLware developmenL process
lf Lhe funcLlonallLy lsn'L necessary Lo Lhe purpose of Lhe appllcaLlon lL should be removed as lL may have unknown lmpacLs or dependencles LhaL were noL Laken lnLo accounL by Lhe deslgner or Lhe cusLomer lf noL removed deslgn lnformaLlon wlll be needed Lo deLermlne added LesLlng needs or regresslon LesLlng needs ManagemenL should be made aware of any slgnlflcanL added rlsks as a resulL of Lhe unexpecLed funcLlonallLy lf Lhe funcLlonallLy only effecLs areas such as mlnor lmprovemenLs ln Lhe user lnLerface for example lL may noL be a slgnlflcanL rlsk
now can Software A processes be |mp|emented w|thout st|f||ng product|v|ty?
8y lmplemenLlng CA processes slowly over Llme uslng consensus Lo reach agreemenL on processes and ad[usLlng and experlmenLlng as an organlzaLlon grows and maLures producLlvlLy wlll be lmproved lnsLead of sLlfled roblem prevenLlon wlll lessen Lhe need for problem deLecLlon panlcs and burnouL wlll decrease and Lhere wlll be lmproved focus and less wasLed efforL AL Lhe same Llme aLLempLs should be made Lo keep processes slmple and efflclenL mlnlmlze paperwork promoLe compuLerbased processes and auLomaLed Lracklng and reporLlng mlnlmlze Llme requlred ln meeLlngs and promoLe Lralnlng as parL of Lhe CA process Powever no one especlally LalenLed Lechnlcal Lypes llkes rules or bureaucracy and ln Lhe shorL run Lhlngs may slow down a blL A Lyplcal scenarlo would be LhaL more days of plannlng and developmenL wlll be needed buL less Llme wlll be requlred for laLenlghL bugflxlng and calmlng of lraLe cusLomers
What |f an organ|zat|on |s grow|ng so fast that f|xed A processes are |mposs|b|e?
1hls ls a common problem ln Lhe sofLware lndusLry especlally ln new Lechnology areas 1here ls no easy soluLlon ln Lhls slLuaLlon oLher Lhan - Plre good people - ManagemenL should 'ruLhlessly prlorlLlze' quallLy lssues and malnLaln focus on Lhe cusLomer - Lveryone ln Lhe organlzaLlon should be clear on whaL 'quallLy' means Lo Lhe cusLomer
now does a c||ent' server env|ronment affect test|ng?
CllenL/server appllcaLlons can be qulLe complex due Lo Lhe mulLlple dependencles among cllenLs daLa communlcaLlons hardware and servers 1hus LesLlng requlremenLs can be exLenslve When Llme ls llmlLed (as lL usually ls) Lhe focus should be on lnLegraLlon and sysLem LesLlng AddlLlonally load/sLress/performance LesLlng may be useful ln deLermlnlng cllenL/server appllcaLlon llmlLaLlons and capablllLles 1here are commerclal Lools Lo asslsL wlLh such LesLlng
now can Wor|d W|de Web s|tes be tested?
Web slLes are essenLlally cllenL/server appllcaLlons wlLh web servers and 'browser' cllenLs ConslderaLlon should be glven Lo Lhe lnLeracLlons beLween hLml pages 1C/l communlcaLlons lnLerneL connecLlons flrewalls appllcaLlons LhaL run ln web pages (such as appleLs !avaScrlpL plugln appllcaLlons) and appllcaLlons LhaL run on Lhe server slde (such as cgl scrlpLs daLabase lnLerfaces logglng appllcaLlons dynamlc page generaLors asp eLc) AddlLlonally Lhere are a wlde varleLy of servers and browsers varlous verslons of each small buL someLlmes slgnlflcanL dlfferences beLween Lhem varlaLlons ln connecLlon speeds rapldly changlng Lechnologles and mulLlple sLandards and proLocols 1he end resulL ls LhaL LesLlng for web slLes can become a ma[or ongolng efforL CLher conslderaLlons mlghL lnclude - WhaL are Lhe expecLed loads on Lhe server (eg number of hlLs per unlL Llme?) and whaL klnd of performance ls requlred under such loads (such as web server response Llme daLabase query response Llmes) - WhaL klnds of Lools wlll be needed for performance LesLlng (such as web load LesLlng Lools oLher Lools already ln house LhaL can be adapLed web roboL downloadlng Lools eLc)? - Who ls Lhe LargeL audlence? WhaL klnd of browsers wlll Lhey be uslng? WhaL klnd of connecLlon speeds wlll Lhey by uslng? Are Lhey lnLra organlzaLlon (Lhus wlLh llkely hlgh connecLlon speeds and slmllar browsers) or lnLerneLwlde (Lhus wlLh a wlde varleLy of connecLlon speeds and browser Lypes)? - WhaL klnd of performance ls expecLed on Lhe cllenL slde (eg how fasL should pages appear how fasL should anlmaLlons appleLs eLc load and run)? - Wlll down Llme for server and conLenL malnLenance/upgrades be allowed? how much? - WhaL klnds of securlLy (flrewalls encrypLlons passwords eLc) wlll be requlred and whaL ls lL expecLed Lo do? Pow can lL be LesLed? - Pow rellable are Lhe slLe's lnLerneL connecLlons requlred Lo be? And how does LhaL affecL backup sysLem or redundanL connecLlon requlremenLs and LesLlng? - WhaL processes wlll be requlred Lo manage updaLes Lo Lhe web slLe's conLenL and whaL are Lhe requlremenLs for malnLalnlng Lracklng and conLrolllng page conLenL graphlcs llnks eLc? - Whlch P1ML speclflcaLlon wlll be adhered Lo? Pow sLrlcLly? WhaL varlaLlons wlll be allowed for LargeLed browsers? - Wlll Lhere be any sLandards or requlremenLs for page appearance and/or graphlcs LhroughouL a slLe or parLs of a slLe?? - Pow wlll lnLernal and exLernal llnks be valldaLed and updaLed? how ofLen? - Can LesLlng be done on Lhe producLlon sysLem or wlll a separaLe LesL sysLem be requlred? Pow are browser cachlng varlaLlons ln browser opLlon seLLlngs dlalup connecLlon varlablllLy's and realworld lnLerneL 'Lrafflc congesLlon' problems Lo be accounLed for ln LesLlng? - Pow exLenslve or cusLomlzed are Lhe server logglng and reporLlng requlremenLs are Lhey consldered an lnLegral parL of Lhe sysLem and do Lhey requlre LesLlng? - Pow are cgl programs appleLs !avaScrlpL's AcLlvex componenLs eLc Lo be malnLalned Lracked conLrolled and LesLed?
- ages should be 33 screens max unless conLenL ls LlghLly focused on a slngle Loplc lf larger provlde lnLernal llnks wlLhln Lhe page - 1he page layouLs and deslgn elemenLs should be conslsLenL LhroughouL a slLe so LhaL lL's clear Lo Lhe user LhaL Lhey're sLlll wlLhln a slLe - ages should be as browserlndependenL as posslble or pages should be provlded or generaLed based on Lhe browserLype - All pages should have llnks exLernal Lo Lhe page Lhere should be no deadend pages - 1he page owner revlslon daLe and a llnk Lo a conLacL person or organlzaLlon should be lncluded on each page 21 Pow ls LesLlng affecLed by ob[ecLorlenLed deslgns?
Wellenglneered ob[ecLorlenLed deslgn can make lL easler Lo Lrace from code Lo lnLernal deslgn Lo funcLlonal deslgn Lo requlremenLs Whlle Lhere wlll be llLLle affecL on black box LesLlng (where an undersLandlng of Lhe lnLernal deslgn of Lhe appllcaLlon ls unnecessary) whlLebox LesLlng can be orlenLed Lo Lhe appllcaLlon's ob[ecLs lf Lhe appllcaLlon was welldeslgned Lhls can slmpllfy LesL deslgn
What |s Lxtreme rogramm|ng and what's |t got to do w|th test|ng ?
LxLreme rogrammlng (x) ls a sofLware developmenL approach for small Leams on rlsk prone pro[ecLs wlLh unsLable requlremenLs lL was creaLed by kenL 8eck who descrlbed Lhe approach ln hls book 'LxLreme rogrammlng Lxplalned' 1esLlng ('exLreme LesLlng') ls a core aspecL of LxLreme rogrammlng rogrammers are expecLed Lo wrlLe unlL and funcLlonal LesL code flrsL before Lhe appllcaLlon ls developed 1esL code ls under source conLrol along wlLh Lhe resL of Lhe code CusLomers are expecLed Lo be an lnLegral parL of Lhe pro[ecL Leam and Lo help develop scenarlos for accepLance/black box LesLlng AccepLance LesLs are preferably auLomaLed and are modlfled and rerun for each of Lhe frequenL developmenL lLeraLlons
CA and LesL personnel are also requlred Lo be an lnLegral parL of Lhe pro[ecL Leam
ueLalled requlremenLs documenLaLlon ls noL used and frequenL reschedullng re esLlmaLlng and reprlorlLlzlng ls expecLed
1 accepLance LesLlng lormal LesLlng conducLed Lo enable a user cusLomer or oLher auLhorlzed enLlLy Lo deLermlne wheLher Lo accepL a sysLem or componenL
2 acLual ouLcome 1he behavlor acLually produced when Lhe ob[ecL ls LesLed under speclfled condlLlons
3 ad hoc LesLlng 1esLlng carrled ouL uslng no recognlzed LesL case deslgn Lechnlque
4 alpha LesLlng SlmulaLed or acLual operaLlonal LesLlng aL an lnhouse slLe noL oLherwlse lnvolved wlLh Lhe sofLware developers
3 arc LesLlng See branch LesLlng
6 8ackusnaur form A meLa language used Lo formally descrlbe Lhe synLax of a language
7 baslc block A sequence of one or more consecuLlve execuLable sLaLemenLs conLalnlng no branches
8 basls LesL seL A seL of LesL cases derlved from Lhe code loglc whlch ensure LhaL 100 branch coverage ls achleved
9 debugglng
11 beLa LesLlng CperaLlonal LesLlng aL a slLe noL oLherwlse lnvolved wlLh Lhe sofLware developers
10 behavlor 1he comblnaLlon of lnpuL values and precondlLlons and Lhe requlred response for a funcLlon of a sysLem 1he full speclflcaLlon of a funcLlon would normally comprlse one or more behavlors
12 blgbang LesLlng lnLegraLlon LesLlng where no lncremenLal LesLlng Lakes place prlor Lo all Lhe sysLem's componenLs belng comblned Lo form Lhe sysLem
13 black box LesLlng See funcLlonal LesL case deslgn
14 boLLomup LesLlng An approach Lo lnLegraLlon LesLlng where Lhe lowesL level componenLs are LesLed flrsL Lhen used Lo faclllLaLe Lhe LesLlng of hlgher level componenLs 1he process ls repeaLed unLll Lhe componenL aL Lhe Lop of Lhe hlerarchy ls LesLed
13 boundary value An lnpuL value or ouLpuL value whlch ls on Lhe boundary beLween equlvalence classes or an lncremenLal dlsLance elLher slde of Lhe boundary
16 boundary value analysls A LesL case deslgn Lechnlque for a componenL ln whlch LesL cases are deslgned whlch lnclude represenLaLlves of boundary values
17 boundary value coverage 1he percenLage of boundary values of Lhe componenL's equlvalence classes whlch have been exerclsed by a LesL case sulLe
18 boundary value LesLlng See boundary value analysls
19 branch A condlLlonal Lransfer of conLrol from any sLaLemenL Lo any oLher sLaLemenL ln a componenL or an uncondlLlonal Lransfer of conLrol from any sLaLemenL Lo any oLher sLaLemenL ln Lhe componenL excepL Lhe nexL sLaLemenL or when a componenL has more Lhan one enLry polnL a Lransfer of conLrol Lo an enLry polnL of Lhe componenL
20 branch condlLlon See declslon condlLlon
22 branch condlLlon comblnaLlon LesLlng A LesL case deslgn Lechnlque ln whlch LesL cases are deslgned Lo execuLe comblnaLlons of branch condlLlon ouLcomes 21 branch condlLlon comblnaLlon coverage 1he percenLage of comblnaLlons of all branch condlLlon ouLcomes ln every declslon LhaL have been exerclsed by a LesL case sulLe
23 branch condlLlon coverage 1he percenLage of branch condlLlon ouLcomes ln every declslon LhaL have been exerclsed by a LesL case sulLe
24 branch condlLlon LesLlng A LesL case deslgn Lechnlque ln whlch LesL cases are deslgned Lo execuLe branch condlLlon ouLcomes
23 branch coverage 1he percenLage of branches LhaL have been exerclsed by a LesL case sulLe
26 branch ouLcome See declslon ouLcome
27 branch polnL See declslon
28 branch LesLlng A LesL case deslgn Lechnlque for a componenL ln whlch LesL cases are deslgned Lo execuLe branch ouLcomes
29 bug See faulL
30 bug seedlng See error seedlng
31 Cuse See compuLaLlon daLa use
32 capLure/playback Lool A LesL Lool LhaL records LesL lnpuL as lL ls senL Lo Lhe sofLware under LesL 1he lnpuL cases sLored can Lhen be used Lo reproduce Lhe LesL aL a laLer Llme
33 capLure/replay Lool See capLure/playback Lool
34 CAS1 Acronym for compuLeralded sofLware LesLlng
33 causeeffecL graph A graphlcal represenLaLlon of lnpuLs or sLlmull (causes) wlLh Lhelr assoclaLed ouLpuLs (effecLs) whlch can be used Lo deslgn LesL cases
36 causeeffecL graphlng A LesL case deslgn Lechnlque ln whlch LesL cases are deslgned by conslderaLlon of causeeffecL graphs
37 cerLlflcaLlon 1he process of conflrmlng LhaL a sysLem or componenL complles wlLh lLs speclfled requlremenLs and ls accepLable for operaLlonal use
38 Chow's coverage meLrlcs See nswlLch coverage
39 code coverage An analysls meLhod LhaL deLermlnes whlch parLs of Lhe sofLware have been execuLed (covered) by Lhe LesL case sulLe and whlch parLs have noL been execuLed and Lherefore may requlre addlLlonal aLLenLlon
40 codebased LesLlng ueslgnlng LesLs based on ob[ecLlves derlved from Lhe lmplemenLaLlon (eg LesLs LhaL execuLe speclflc conLrol flow paLhs or use speclflc daLa lLems)
41 compaLlblllLy LesLlng 1esLlng wheLher Lhe sysLem ls compaLlble wlLh oLher sysLems wlLh whlch lL should communlcaLe
42 compleLe paLh LesLlng See exhausLlve LesLlng
43 componenL A mlnlmal sofLware lLem for whlch a separaLe speclflcaLlon ls avallable
43 compuLaLlon daLa use A daLa use noL ln a condlLlon Also called Cuse
46 condlLlon A 8oolean sLaLemenL conLalnlng no 8oolean operaLors lor lnsLance A noL ls 8 and A buL condlLlon a
47 condlLlon coverage See branch condlLlon coverage
48 condlLlon ouLcome 1he evaluaLlon of a condlLlon Lo 18uL or lALSL
49 conformance crlLerlon Some meLhod of [udglng wheLher or noL Lhe componenL's acLlon on a parLlcular speclfled lnpuL value conforms Lo Lhe speclflcaLlon
30 conformance LesLlng 1he process of LesLlng LhaL an lmplemenLaLlon conforms Lo Lhe speclflcaLlon on whlch lL ls based
31 conLrol flow An absLracL represenLaLlon of all posslble sequences of evenLs ln a program's execuLlon
32 conLrol flow graph 1he dlagrammaLlc represenLaLlon of Lhe posslble alLernaLlve conLrol flow paLhs Lhrough a componenL
33 conLrol flow paLh See paLh
34 converslon LesLlng 1esLlng of programs or procedures used Lo converL daLa from exlsLlng sysLems for use ln replacemenL sysLems
33 correcLness 1he degree Lo whlch sofLware conforms Lo lLs speclflcaLlon
36 coverage 1he degree expressed as a percenLage Lo whlch a speclfled coverage lLem has been exerclsed by a LesL case sulLe
37 coverage lLem An enLlLy or properLy used as a basls for LesLlng
38 daLa deflnlLlon An execuLable sLaLemenL where a varlable ls asslgned a value
39 daLa deflnlLlon Cuse coverage 1he percenLage of daLa deflnlLlon Cuse palrs ln a componenL LhaL are exerclsed by a LesL case sulLe
60 daLa deflnlLlon Cuse palr A daLa deflnlLlon and compuLaLlon daLa use where Lhe daLa use uses Lhe value deflned ln Lhe daLa deflnlLlon
61 daLa deflnlLlon use coverage 1he percenLage of daLa deflnlLlon use palrs ln a componenL LhaL are exerclsed by a LesL case sulLe
62 daLa deflnlLlon use palr A daLa deflnlLlon and predlcaLe daLa use where Lhe daLa use uses Lhe value deflned ln Lhe daLa deflnlLlon
63 daLa deflnlLlonuse coverage 1he percenLage of daLa deflnlLlonuse palrs ln a componenL LhaL are exerclsed by a LesL case sulLe
64 daLa deflnlLlonuse palr A daLa deflnlLlon and daLa use where Lhe daLa use uses Lhe value deflned ln Lhe daLa deflnlLlon
63 daLa deflnlLlonuse LesLlng A LesL case deslgn Lechnlque for a componenL ln whlch LesL cases are deslgned Lo execuLe daLa deflnlLlonuse palrs
66 daLa flow coverage 1esL coverage measure based on varlable usage wlLhln Lhe code Lxamples are daLa deflnlLlonuse coverage daLa deflnlLlon use coverage daLa deflnlLlon C use coverage eLc
67 daLa flow LesLlng 1esLlng ln whlch LesL cases are deslgned based on varlable usage wlLhln Lhe code
68 daLa use An execuLable sLaLemenL where Lhe value of a varlable ls accessed
69 debugglng 1he process of flndlng and removlng Lhe causes of fallures ln sofLware
70 declslon A program polnL aL whlch Lhe conLrol flow has Lwo or more alLernaLlve rouLes
71 ueclslon condlLlon A condlLlon wlLhln a declslon
72 declslon coverage 1he percenLage of declslon ouLcomes LhaL have been exerclsed by a LesL case sulLe
73 declslon ouLcome 1he resulL of a declslon (whlch Lherefore deLermlnes Lhe conLrol flow alLernaLlve Laken)
74 deslgnbased LesLlng ueslgnlng LesLs based on ob[ecLlves derlved from Lhe archlLecLural or deLall deslgn of Lhe sofLware (eg LesLs LhaL execuLe speclflc lnvocaLlon paLhs or probe Lhe worsL case behavlor of algorlLhms)
73 desk checklng 1he LesLlng of sofLware by Lhe manual slmulaLlon of lLs execuLlon
76 dlrLy LesLlng See negaLlve LesLlng 8elzer
77 documenLaLlon LesLlng 1esLlng concerned wlLh Lhe accuracy of documenLaLlon
78 domaln 1he seL from whlch values are selecLed
79 domaln LesLlng See equlvalence parLlLlon LesLlng
80 dynamlc analysls 1he process of evaluaLlng a sysLem or componenL based upon lLs behavlor durlng execuLlon
81 emulaLor A devlce compuLer program or sysLem LhaL accepLs Lhe same lnpuLs and produces Lhe same ouLpuLs as a glven sysLem
82 enLry polnL 1he flrsL execuLable sLaLemenL wlLhln a componenL
83 equlvalence class A porLlon of Lhe componenL's lnpuL or ouLpuL domalns for whlch Lhe componenL's behavlor ls assumed Lo be Lhe same from Lhe componenL's speclflcaLlon
84 equlvalence parLlLlon See equlvalence class
83 equlvalence parLlLlon coverage 1he percenLage of equlvalence classes generaLed for Lhe componenL whlch have been exerclsed by a LesL case sulLe
86 equlvalence parLlLlon LesLlng A LesL case deslgn Lechnlque for a componenL ln whlch LesL cases are deslgned Lo execuLe represenLaLlves from equlvalence classes
87 error A human acLlon LhaL produces an lncorrecL resulL lLLL
88 error guesslng A LesL case deslgn Lechnlque where Lhe experlence of Lhe LesLer ls used Lo posLulaLe whaL faulLs mlghL occur and Lo deslgn LesLs speclflcally Lo expose Lhem
89 error seedlng 1he process of lnLenLlonally addlng known faulLs Lo Lhose already ln a compuLer program for Lhe purpose of monlLorlng Lhe raLe of deLecLlon and removal and esLlmaLlng Lhe number of faulLs remalnlng ln Lhe program
90 execuLable sLaLemenL A sLaLemenL whlch when complled ls LranslaLed lnLo ob[ecL code whlch wlll be execuLed procedurally when Lhe program ls runnlng and may perform an acLlon on program daLa
91 exerclsed A program elemenL ls exerclsed by a LesL case when Lhe lnpuL value causes Lhe execuLlon of LhaL elemenL such as a sLaLemenL branch or oLher sLrucLural elemenL
92 exhausLlve LesLlng A LesL case deslgn Lechnlque ln whlch Lhe LesL case sulLe comprlses all comblnaLlons of lnpuL values and precondlLlons for componenL varlables
93 exlL polnL 1he lasL execuLable sLaLemenL wlLhln a componenL
94 expecLed ouLcome See predlcLed ouLcome
93 faclllLy LesLlng See funcLlonal LesL case deslgn
96 fallure uevlaLlon of Lhe sofLware from lLs expecLed dellvery or servlce
97 faulL A manlfesLaLlon of an error ln sofLware A faulL lf encounLered may cause a fallure
98 feaslble paLh A paLh for whlch Lhere exlsLs a seL of lnpuL values and execuLlon condlLlons whlch causes lL Lo be execuLed
99 feaLure LesLlng See funcLlonal LesL case deslgn
100 funcLlonal speclflcaLlon 1he documenL LhaL descrlbes ln deLall Lhe characLerlsLlcs of Lhe producL wlLh regard Lo lLs lnLended capablllLy 8S 4778 arL2
101 funcLlonal LesL case deslgn 1esL case selecLlon LhaL ls based on an analysls of Lhe speclflcaLlon of Lhe componenL wlLhouL reference Lo lLs lnLernal worklngs
102 glass box LesLlng See sLrucLural LesL case deslgn
103 lncremenLal LesLlng lnLegraLlon LesLlng where sysLem componenLs are lnLegraLed lnLo Lhe sysLem one aL a Llme unLll Lhe enLlre sysLem ls lnLegraLed
104 lndependence SeparaLlon of responslblllLles whlch ensures Lhe accompllshmenL of ob[ecLlve evaluaLlon AfLer do178b
103 lnfeaslble paLh A paLh whlch cannoL be exerclsed by any seL of posslble lnpuL values
106 lnpuL A varlable (wheLher sLored wlLhln a componenL or ouLslde lL) LhaL ls read by Lhe componenL
107 lnpuL domaln 1he seL of all posslble lnpuLs
108 lnpuL value An lnsLance of an lnpuL
109 lnspecLlon A group revlew quallLy lmprovemenL process for wrlLLen maLerlal lL conslsLs of Lwo aspecLs producL (documenL lLself) lmprovemenL and process lmprovemenL (of boLh documenL producLlon and lnspecLlon) AfLer Craham
110 lnsLall ablllLy LesLlng 1esLlng concerned wlLh Lhe lnsLallaLlon procedures for Lhe sysLem
111 lnsLrumenLaLlon 1he lnserLlon of addlLlonal code lnLo Lhe program ln order Lo collecL lnformaLlon abouL program behavlor durlng program execuLlon
112 lnsLrumenLer A sofLware Lool used Lo carry ouL lnsLrumenLaLlon
113 lnLegraLlon 1he process of comblnlng componenLs lnLo larger assemblles
114 lnLegraLlon LesLlng 1esLlng performed Lo expose faulLs ln Lhe lnLerfaces and ln Lhe lnLeracLlon beLween lnLegraLed componenLs
113 lnLerface LesLlng lnLegraLlon LesLlng where Lhe lnLerfaces beLween sysLem componenLs are LesLed
116 lsolaLlon LesLlng ComponenL LesLlng of lndlvldual componenLs ln lsolaLlon from surroundlng componenLs wlLh surroundlng componenLs belng slmulaLed by sLubs
117 LCSA! A Llnear Code Sequence And !ump conslsLlng of Lhe followlng Lhree lLems (convenLlonally ldenLlfled by llne numbers ln a source code llsLlng) Lhe sLarL of Lhe llnear sequence of execuLable sLaLemenLs Lhe end of Lhe llnear sequence and Lhe LargeL llne Lo whlch conLrol flow ls Lransferred aL Lhe end of Lhe llnear sequence
118 LCSA! coverage 1he percenLage of LCSA!s of a componenL whlch are exerclsed by a LesL case sulLe
119 LCSA! LesLlng A LesL case deslgn Lechnlque for a componenL ln whlch LesL cases are deslgned Lo execuLe LCSA!s
120 loglccoverage LesLlng See sLrucLural LesL case deslgn Myers
121 loglcdrlven LesLlng See sLrucLural LesL case deslgn
122 malnLalnablllLy LesLlng 1esLlng wheLher Lhe sysLem meeLs lLs speclfled ob[ecLlves for malnLalnablllLy
123 modlfled condlLlon/declslon coverage 1he percenLage of all branch condlLlon ouLcomes LhaL lndependenLly affecL a declslon ouLcome LhaL have been exerclsed by a LesL case sulLe
124 modlfled condlLlon/declslon LesLlng A LesL case deslgn Lechnlque ln whlch LesL cases are deslgned Lo execuLe branch condlLlon ouLcomes LhaL lndependenLly affecL a declslon ouLcome
123 mulLlple condlLlon coverage See branch condlLlon comblnaLlon coverage
126 muLaLlon analysls A meLhod Lo deLermlne LesL case sulLe Lhoroughness by measurlng Lhe exLenL Lo whlch a LesL case sulLe can dlscrlmlnaLe Lhe program from sllghL varlanLs (muLanLs) of Lhe program See also error seedlng
127 nswlLch coverage 1he percenLage of sequences of nLranslLlons LhaL have been exerclsed by a LesL case sulLe
128 nswlLch LesLlng A form of sLaLe LranslLlon LesLlng ln whlch LesL cases are deslgned Lo execuLe all valld sequences of nLranslLlons
129 nLranslLlons A sequence of n+1 LranslLlons
130 negaLlve LesLlng 1esLlng almed aL showlng sofLware does noL work 8elzer
131 nonfuncLlonal requlremenLs LesLlng 1esLlng of Lhose requlremenLs LhaL do noL relaLe Lo funcLlonallLy le performance usablllLy eLc
132 operaLlonal LesLlng 1esLlng conducLed Lo evaluaLe a sysLem or componenL ln lLs operaLlonal envlronmenL lLLL
133 oracle A mechanlsm Lo produce Lhe predlcLed ouLcomes Lo compare wlLh Lhe acLual ouLcomes of Lhe sofLware under LesL AfLer adrlon
134 ouLcome AcLual ouLcome or predlcLed ouLcome 1hls ls Lhe ouLcome of a LesL See also branch ouLcome condlLlon ouLcome and declslon ouLcome
133 ouLpuL A varlable (wheLher sLored wlLhln a componenL or ouLslde lL) LhaL ls wrlLLen Lo by Lhe componenL
136 ouLpuL domaln 1he seL of all posslble ouLpuLs
137 ouLpuL value An lnsLance of an ouLpuL
138 use See predlcaLe daLa use
139 parLlLlon LesLlng See equlvalence parLlLlon LesLlng 8elzer
140 paLh A sequence of execuLable sLaLemenLs of a componenL from an enLry polnL Lo an exlL polnL
141 paLh coverage 1he percenLage of paLhs ln a componenL exerclsed by a LesL case sulLe
142 paLh senslLlzlng Chooslng a seL of lnpuL values Lo force Lhe execuLlon of a componenL Lo Lake a glven paLh
143 paLh LesLlng A LesL case deslgn Lechnlque ln whlch LesL cases are deslgned Lo execuLe paLhs of a componenL
144 performance LesLlng 1esLlng conducLed Lo evaluaLe Lhe compllance of a sysLem or componenL wlLh speclfled performance requlremenLs lLLL
143 porLablllLy LesLlng 1esLlng almed aL demonsLraLlng Lhe sofLware can be porLed Lo speclfled hardware or sofLware plaLforms
146 precondlLlon LnvlronmenLal and sLaLe condlLlons whlch musL be fulfllled before Lhe componenL can be execuLed wlLh a parLlcular lnpuL value
147 predlcaLe A loglcal sLaLemenL whlch evaluaLes Lo 18uL or lALSL normally Lo dlrecL Lhe execuLlon paLh ln code
148 predlcaLe daLa use A daLa use ln a predlcaLe
149 predlcLed ouLcome 1he behavlor predlcLed by Lhe speclflcaLlon of an ob[ecL under speclfled condlLlons
130 program lnsLrumenLer See lnsLrumenLer
131 progresslve LesLlng 1esLlng of new feaLures afLer regresslon LesLlng of prevlous feaLures 8elzer
132 pseudorandom A serles whlch appears Lo be random buL ls ln facL generaLed accordlng Lo some prearranged sequence
133 recovery LesLlng 1esLlng almed aL verlfylng Lhe sysLem's ablllLy Lo recover from varylng degrees of fallure
134 regresslon LesLlng 8eLesLlng of a prevlously LesLed program followlng modlflcaLlon Lo ensure LhaL faulLs have noL been lnLroduced or uncovered as a resulL of Lhe changes made
133 requlremenLsbased LesLlng ueslgnlng LesLs based on ob[ecLlves derlved from requlremenLs for Lhe sofLware componenL (eg LesLs LhaL exerclse speclflc funcLlons or probe Lhe nonfuncLlonal consLralnLs such as performance or securlLy) See funcLlonal LesL case deslgn
136 resulL See ouLcome
137 revlew A process or meeLlng durlng whlch a work producL or seL of work producLs ls presenLed Lo pro[ecL personnel managers users or oLher lnLeresLed parLles for commenL or approval leee
139 servlceablllLy LesLlng See malnLalnablllLy LesLlng
160 slmple sub paLh A sub paLh of Lhe conLrol flow graph ln whlch no program parL ls execuLed more Lhan necessary
161 slmulaLlon 1he represenLaLlon of selecLed behavloral characLerlsLlcs of one physlcal or absLracL sysLem by anoLher sysLem lSC 2382/1
162 slmulaLor A devlce compuLer program or sysLem used durlng sofLware verlflcaLlon whlch behaves or operaLes llke a glven sysLem when provlded wlLh a seL of conLrolled lnpuLs lLLLdo178b
163 source sLaLemenL See sLaLemenL
164 speclflcaLlon A descrlpLlon of a componenL's funcLlon ln Lerms of lLs ouLpuL values for speclfled lnpuL values under speclfled precondlLlons
163 speclfled lnpuL An lnpuL for whlch Lhe speclflcaLlon predlcLs an ouLcome
166 sLaLe LranslLlon A LranslLlon beLween Lwo allowable sLaLes of a sysLem or componenL
167 sLaLe LranslLlon LesLlng A LesL case deslgn Lechnlque ln whlch LesL cases are deslgned Lo execuLe sLaLe LranslLlons
168 sLaLemenL An enLlLy ln a programmlng language whlch ls Lyplcally Lhe smallesL lndlvlslble unlL of execuLlon
169 sLaLemenL coverage 1he percenLage of execuLable sLaLemenLs ln a componenL LhaL have been exerclsed by a LesL case sulLe
170 sLaLemenL LesLlng A LesL case deslgn Lechnlque for a componenL ln whlch LesL cases are deslgned Lo execuLe sLaLemenLs
171 sLaLlc analysls Analysls of a program carrled ouL wlLhouL execuLlng Lhe program
172 sLaLlc analyzer A Lool LhaL carrles ouL sLaLlc analysls
173 sLaLlc LesLlng 1esLlng of an ob[ecL wlLhouL execuLlon on a compuLer
174 sLaLlsLlcal LesLlng A LesL case deslgn Lechnlque ln whlch a model ls used of Lhe sLaLlsLlcal dlsLrlbuLlon of Lhe lnpuL Lo consLrucL represenLaLlve LesL cases
176 sLress LesLlng 1esLlng conducLed Lo evaluaLe a sysLem or componenL aL or beyond Lhe llmlLs of lLs speclfled requlremenLs lLLL
177 sLrucLural coverage Coverage measures based on Lhe lnLernal sLrucLure of Lhe componenL
178 sLrucLural LesL case deslgn 1esL case selecLlon LhaL ls based on an analysls of Lhe lnLernal sLrucLure of Lhe componenL
179 sLrucLural LesLlng See sLrucLural LesL case deslgn
180 sLrucLured basls LesLlng A LesL case deslgn Lechnlque ln whlch LesL cases are derlved from Lhe code loglc Lo achleve 100 branch coverage
181 sLrucLured walkLhrough See walkLhrough
182 sLub A skeleLal or speclalpurpose lmplemenLaLlon of a sofLware module used Lo develop or LesL a componenL LhaL calls or ls oLherwlse dependenL on lL AfLer lLLL
183 sub paLh A sequence of execuLable sLaLemenLs wlLhln a componenL
184 symbollc evaluaLlon See symbollc execuLlon
183 symbollc execuLlon A sLaLlc analysls Lechnlque LhaL derlves a symbollc sLaLemenL for program paLhs
186 synLax LesLlng A LesL case deslgn Lechnlque for a componenL or sysLem ln whlch LesL case deslgn ls based upon Lhe synLax of Lhe lnpuL
187 sysLem LesLlng 1he process of LesLlng an lnLegraLed sysLem Lo verlfy LhaL lL meeLs speclfled requlremenLs PeLzel
188 Lechnlcal requlremenLs LesLlng See nonfuncLlonal requlremenLs LesLlng
189 LesL auLomaLlon 1he use of sofLware Lo conLrol Lhe execuLlon of LesLs Lhe comparlson of acLual ouLcomes Lo predlcLed ouLcomes Lhe seLLlng up of LesL precondlLlons and oLher LesL conLrol and LesL reporLlng funcLlons
190 LesL case A seL of lnpuLs execuLlon precondlLlons and expecLed ouLcomes developed for a parLlcular ob[ecLlve such as Lo exerclse a parLlcular program paLh or Lo verlfy compllance wlLh a speclflc requlremenL AfLer lLLLdo178b
191 LesL case deslgn Lechnlque A meLhod used Lo derlve or selecL LesL cases
192 LesL case sulLe A collecLlon of one or more LesL cases for Lhe sofLware under LesL
193 LesL comparaLor A LesL Lool LhaL compares Lhe acLual ouLpuLs produced by Lhe sofLware under LesL wlLh Lhe expecLed ouLpuLs for LhaL LesL case
194 LesL compleLlon crlLerlon A crlLerlon for deLermlnlng when planned LesLlng ls compleLe deflned ln Lerms of a LesL measuremenL Lechnlque
193 LesL coverage See coverage
196 LesL drlver A program or LesL Lool used Lo execuLe sofLware agalnsL a LesL case sulLe
197 LesL envlronmenL A descrlpLlon of Lhe hardware and sofLware envlronmenL ln whlch Lhe LesLs wlll be run and any oLher sofLware wlLh whlch Lhe sofLware under LesL lnLeracLs when under LesL lncludlng sLubs and LesL drlvers
198 LesL execuLlon 1he processlng of a LesL case sulLe by Lhe sofLware under LesL produclng an ouLcome
199 LesL execuLlon Lechnlque 1he meLhod used Lo perform Lhe acLual LesL execuLlon eg manual capLure/playback Lool eLc
200 LesL generaLor A program LhaL generaLes LesL cases ln accordance Lo a speclfled sLraLegy or heurlsLlc
201 LesL harness A LesLlng Lool LhaL comprlses a LesL drlver and a LesL comparaLor
202 LesL measuremenL Lechnlque A meLhod used Lo measure LesL coverage lLems
203 LesL ouLcome See ouLcome
204 LesL plan A record of Lhe LesL plannlng process deLalllng Lhe degree of LesLer lndependence Lhe LesL envlronmenL Lhe LesL case deslgn Lechnlques and LesL measuremenL Lechnlques Lo be used and Lhe raLlonale for Lhelr cholce
203 LesL procedure A documenL provldlng deLalled lnsLrucLlons for Lhe execuLlon of one or more LesL cases
206 LesL records lor each LesL an unamblguous record of Lhe ldenLlLles and verslons of Lhe componenL under LesL Lhe LesL speclflcaLlon and acLual ouLcome
207 LesL scrlpL Commonly used Lo refer Lo Lhe auLomaLed LesL procedure used wlLh a LesL harness
208 LesL speclflcaLlon lor each LesL case Lhe coverage lLem Lhe lnlLlal sLaLe of Lhe sofLware under LesL Lhe lnpuL and Lhe predlcLed ouLcome
209 LesL LargeL A seL of LesL compleLlon crlLerla
210 LesLlng 1he process of exerclslng sofLware Lo verlfy LhaL lL saLlsfles speclfled requlremenLs and Lo deLecL errors
211 Lhread LesLlng A varlaLlon of Lopdown LesLlng where Lhe progresslve lnLegraLlon of componenLs follows Lhe lmplemenLaLlon of subseLs of Lhe requlremenLs as opposed Lo Lhe lnLegraLlon of componenLs by successlvely lower levels
212 Lopdown LesLlng An approach Lo lnLegraLlon LesLlng where Lhe componenL aL Lhe Lop of Lhe componenL hlerarchy ls LesLed flrsL wlLh lower level componenLs belng slmulaLed by sLubs 1esLed componenLs are Lhen used Lo LesL lower level componenLs 1he process ls repeaLed unLll Lhe lowesL level componenLs have been LesLed
213 unlL LesLlng See componenL LesLlng
214 usablllLy LesLlng 1esLlng Lhe ease wlLh whlch users can learn and use a producL
213 valldaLlon ueLermlnaLlon of Lhe correcLness of Lhe producLs of sofLware developmenL wlLh respecL Lo Lhe user needs and requlremenLs
216 verlflcaLlon 1he process of evaluaLlng a sysLem or componenL Lo deLermlne wheLher Lhe producLs of Lhe glven developmenL phase saLlsfy Lhe condlLlons lmposed aL Lhe sLarL of LhaL phase lLLL
217 volume LesLlng 1esLlng where Lhe sysLem ls sub[ecLed Lo large volumes of daLa
218 walkLhrough A revlew of requlremenLs deslgns or code characLerlzed by Lhe auLhor of Lhe ob[ecL under revlew guldlng Lhe progresslon of Lhe revlew
3) uefecL unearLhlng efflclency 100 * S18s found ln pass 1/ ( S18s found ln pass 1 + S18s found ln pass 2 buL exlsLlng ln pass 1) uefecL unearLhlng efflclency for pass 1 1hus defecL unearLhlng efflclency can be found for all passes
Also defecL unearLhlng efflclency ln Lerms of fleld S18s could be found as follows
100 * 1oLal S18s found ln olarls for a release/ (1oLal S18s found ln olarls for a release + lleld S18s for LhaL release)
4) 1esL case efflclency (1oLal S18s S18s noL mapped)/1oLal S18s * 100 3) 1esL case coverage (1oLal 1esL cases S18s LhaL cannoL be mapped Lo LesL cases)/ 1oLal 1esL Cases * 100
Lg non CommenLed SofLware Source (nCSS) er Lnglneer er MonLh nCSS per erson MonLh nCSS per luncLlon olnL nCSS can also be replaced by kLCC (kllo Llnes of Code)
1esLlng ls done under a sLrucLural LesLlng sLraLegy and requlre compleLe access Lo Lhe ob[ecL's sLrucLure LhaL ls Lhe source code8 8elzer 1993 p8
WalkLhrough
ln Lhe mosL usual form of Lerm a walkLhrough ls sLep by sLep slmulaLlon of Lhe execuLlon of a procedure as when walklng Lhrough code llne by llne wlLh an lmaglned seL of lnpuLs 1he Lerm has been exLended Lo Lhe revlew of maLerlal LhaL ls noL procedural such as daLa descrlpLlons reference manuals speclflcaLlons eLc
volume LesLlng
1esLlng where Lhe sysLem ls sub[ecLed Lo large volumes of daLa8S79231
verlflcaLlon
1he comparlson beLween Lhe acLual characLerlsLlcs of someLhlng (eg a producL of a sofLware pro[ecL) and Lhe speclfled characLerlsLlcs verlflcaLlon ls checklng LhaL we have bullL Lhe sysLem rlghL
valldaLlon
1he comparlson beLween Lhe acLual characLerlsLlcs of someLhlng (eg a producL of a sofLware pro[ecL and Lhe expecLed characLerlsLlcs) valldaLlon ls checklng LhaL you have bullL Lhe rlghL sysLem
usablllLy LesLlng
1esLlng for 'userfrlendllness' Clearly Lhls ls sub[ecLlve and wlll depend on Lhe LargeLed end user or cusLomer
unlL 1esLlng
1esLlng performed Lo lsolaLe and expose faulLs and fallures as soon as Lhe source code ls avallable regardless of Lhe exLernal lnLerfaces LhaL may be requlred CfLenLlmes Lhe deLalled deslgn and requlremenLs documenLs are used as a basls Lo compare how and whaL Lhe unlL ls able Lo perform WhlLe and blackbox LesLlng meLhods are comblned durlng unlL LesLlng
1esL lan lrequenLly Asked CuesLlons
1 Why you cannoL download a Word verslon of Lhls LesL plan
l have recelved numerous requesLs for an MS Word verslon of Lhe LesL plan Powever alLhough Lhe web pages were creaLed dlrecLly from an word documenL l no longer have a copy of LhaL orlglnal word documenL Also havlng prepared numerous LesL plans l know LhaL Lhe conLenL ls more lmporLanL Lhan Lhe formaL See Lhe nexL polnL for more lnfo on Lhe conLenL of a LesL plan
2 WhaL a LesL plan should conLaln
A sofLware pro[ecL LesL plan ls a documenL LhaL descrlbes Lhe ob[ecLlves scope approach and focus of a sofLware LesLlng efforL 1he process of preparlng a LesL plan ls a useful way Lo Lhlnk Lhrough Lhe efforLs needed Lo valldaLe Lhe accepLablllLy of a sofLware producL 1he compleLed documenL wlll help people ouLslde Lhe LesL group undersLand Lhe 'why' and 'how' of producL valldaLlon lL should be Lhorough enough Lo be useful buL noL so Lhorough LhaL no one ouLslde Lhe LesL group wlll read lL
A LesL plan sLaLes whaL Lhe lLems Lo be LesLed are aL whaL level Lhey wlll be LesLed whaL sequence Lhey are Lo be LesLed ln how Lhe LesL sLraLegy wlll be applled Lo Lhe LesLlng of each lLem and descrlbes Lhe LesL envlronmenL
A LesL plan should ldeally be organlzaLlon wlde belng appllcable Lo all of organlzaLlons sofLware developmenLs
1he ob[ecLlve of each LesL plan ls Lo provlde a plan for verlflcaLlon by LesLlng Lhe sofLware Lhe sofLware produced fulflls Lhe funcLlonal or deslgn sLaLemenLs of Lhe approprlaLe sofLware speclflcaLlon ln Lhe case of accepLance LesLlng and sysLem LesLlng Lhls generally means Lhe luncLlonal SpeclflcaLlon
1he flrsL conslderaLlon when preparlng Lhe 1esL lan ls who Lhe lnLended audlence ls le Lhe audlence for a unlL 1esL lan would be dlfferenL and Lhus Lhe conLenL would have Lo be ad[usLed accordlngly
?ou should begln Lhe LesL plan as soon as posslble Cenerally lL ls deslrable Lo begln Lhe masLer LesL plan as Lhe same Llme Lhe 8equlremenLs documenLs and Lhe ro[ecL lan are belng developed 1esL plannlng can (and should) have an lmpacL on Lhe ro[ecL lan Lven Lhough plans LhaL are wrlLLen early wlll have Lo be changed durlng Lhe course of Lhe developmenL and LesLlng buL LhaL ls lmporLanL because lL records Lhe progress of Lhe LesLlng and helps planners become more proflclenL
WhaL Lo conslder for Lhe 1esL lan
1 1esL lan ldenLlfler 2 8eferences 3 lnLroducLlon 4 1esL lLems 3 SofLware 8lsk lssues 6 leaLures Lo be 1esLed 7 leaLures noL Lo be 1esLed 8 Approach 9 lLem ass/lall CrlLerla 10 Suspenslon CrlLerla and 8esumpLlon 8equlremenLs 11 1esL uellverables 12 8emalnlng 1esL 1asks 13 LnvlronmenLal needs 14 SLafflng and 1ralnlng needs 13 8esponslblllLles 16 Schedule 17 lannlng 8lsks and ConLlngencles 18 Approvals 19 Clossary
3 SLandards for SofLware 1esL lans
Several sLandards suggesL whaL a LesL plan should conLaln lncludlng Lhe lLLL 1he sLandards are
lLLL sLandards 8291983 lLLL SLandard for SofLware 1esL uocumenLaLlon 10081987 lLLL SLandard for SofLware unlL 1esLlng 10121986 lLLL SLandard for SofLware verlflcaLlon valldaLlon lans 10391993 lLLL Culde for SofLware verlflcaLlon valldaLlon lans
1he lLLL webslLe ls herehLLp//wwwleeeorg
4 Why l publlshed Lhe LesL plan
Well when l flrsL wenL looklng for sample LesL plans l could noL flnd anyLhlng useful LvenLually l found several slLes whlch l lncluded on my llnks page Powever l was noL saLlsfled wlLh many of Lhe plans LhaL l found so l posLed lL on my webslLe And l have Lo say LhaL l have been asLounded by Lhe level of lnLeresL shown and amazed aL Lhe number of emalls l have recelved abouL lL
6 CopyrlghL Cwnershlp whaL you can do wlLh Lhe plan
Well l publlshed wlLh Lhe alm LhaL lL be used so lf you are golng Lo use lL Lo creaLe a LesL plan for lnLernal use please feel free Lo copy from lL Powever lf Lhe LesL plan ls Lo be publlshed exLernally ln any way web magazlne Lralnlng maLerlal eLc Lhen you musL lnclude a reference Lo me 8azman and a llnk Lo my webslLe
SofLware 1esLlng Llfe Cycle (S1LC)
1he SofLware 1esLlng Llfe Cycle (S1LC) ls Lhe road map Lo a producL success CALA8 conslsLs of a seL of phases LhaL deflne whaL LesLlng acLlvlLles Lo do and when Lo do Lhem lL also enables communlcaLlon and synchronlzaLlon beLween Lhe varlous groups LhaL have lnpuL Lo Lhe overall LesLlng process ln Lhe besL of worlds Lhe S1LC parallels Lhe SofLware uevelopmenL Llfe Cycle coordlnaLlng acLlvlLles Lhus provldlng Lhe vehlcle for a close worklng relaLlonshlp beLween LesLlng and developmenL deparLmenLs S1LC hases roposal/ConLracL 1esLlng 8equlremenLs SpeclflcaLlon (18S) ueslgn 1esLlng lnspecLlon and 8elease CllenL AccepLance
roposal/ConLracL - Analyze scope of pro[ecL - repare ConLracL - 8evlew of ConLracL - 8elease
ueslgn - reparaLlon of MasLer 1esL lan - SeLup LesL envlronmenL - Plgh level LesL plan - ueslgn 1esL plans 1esL Cases - ueclde lf any seL of LesL cases Lo be auLomaLed
1esLlng - lannlng - 1esLlng lnlLlal LesL cycles bug flxes and reLesLlng - llnal 1esLlng and lmplemenLaLlon - SeLup daLabase Lo Lrack componenLs of Lhe auLomaLed LesLlng sysLem le reusable modules
lnspecLlon and 8elease - llnal 8evlew of 1esLlng - MeLrlcs Lo measure lmprovemenL
CllenL AccepLance 8epllcaLlon of roducL uellvery 8ecords Submlsslon CllenL Slgnoff
WhaL's Lhe dlfference beLween load and sLress LesLlng?
Cne of Lhe mosL common buL unforLunaLe mlsuse of Lermlnology ls LreaLlng load LesLlng" and sLress LesLlng" as synonymous 1he consequence of Lhls lgnoranL semanLlc abuse ls usually LhaL Lhe sysLem ls nelLher properly load LesLed" nor sub[ecLed Lo a meanlngful sLress LesL
1 SLress LesLlng ls sub[ecLlng a sysLem Lo an unreasonable load whlle denylng lL Lhe resources (eg 8AM dlsc mlps lnLerrupLseLc) needed Lo process LhaL load 1he ldea ls Lo sLress a sysLem Lo Lhe breaklng polnL ln order Lo flnd bugs LhaL wlll make LhaL break poLenLlally harmful 1he sysLem ls noL expecLed Lo process Lhe overload wlLhouL adequaLe resources buL Lo behave (eg fall) ln a decenL manner (eg noL corrupLlng or loslng daLa) 8ugs and fallure modes dlscovered under sLress LesLlng may or may noL be repalred dependlng on Lhe appllcaLlon Lhe fallure mode consequences eLc 1he load (lncomlng LransacLlon sLream) ln sLress LesLlng ls ofLen dellberaLely dlsLorLed so as Lo force Lhe sysLem lnLo resource depleLlon
2 Load LesLlng ls sub[ecLlng a sysLem Lo a sLaLlsLlcally represenLaLlve (usually) load 1he Lwo maln reasons for uslng such loads ls ln supporL of sofLware rellablllLy LesLlng and ln performance LesLlng 1he Lerm load LesLlng" by lLself ls Loo vague and lmpreclse Lo warranL use lor example do you mean represenLaLlve load" overload" hlgh load" eLc ln performance LesLlng load ls varled from a mlnlmum (zero) Lo Lhe maxlmum level Lhe sysLem can susLaln wlLhouL runnlng ouL of resources or havlng LransacLlons suffer (appllcaLlon speclflc) excesslve delay
3 A Lhlrd use of Lhe Lerm ls as a LesL whose ob[ecLlve ls Lo deLermlne Lhe maxlmum susLalnable load Lhe sysLem can handle ln Lhls usage load LesLlng" ls merely LesLlng aL Lhe hlghesL LransacLlon arrlval raLe ln performance LesLlng
SofLware 1esLlng 8elaLed CuesLlons and Answers
1 WhaL makes a good LesL englneer?
- A good LesL englneer has a 'LesL Lo break' aLLlLude an ablllLy Lo Lake Lhe polnL of vlew of Lhe cusLomer a sLrong deslre for quallLy and an aLLenLlon Lo deLall - 1acL and dlplomacy are useful ln malnLalnlng a cooperaLlve relaLlonshlp wlLh developers and an ablllLy Lo communlcaLe wlLh boLh Lechnlcal (developers) and nonLechnlcal (cusLomers managemenL) people ls useful - revlous sofLware developmenL experlence can be helpful as lL provldes a deeper undersLandlng of Lhe sofLware developmenL process glves Lhe LesLer an appreclaLlon for Lhe developers' polnL of vlew and reduce Lhe learnlng curve ln auLomaLed LesL Lool programmlng - !udgmenL skllls are needed Lo assess hlghrlsk areas of an appllcaLlon on whlch Lo focus LesLlng efforLs when Llme ls llmlLed
2 WhaL makes a good sofLware CA englneer?
- 1he same quallLles a good LesLer has are useful for a CA englneer - AddlLlonally Lhey musL be able Lo undersLand Lhe enLlre sofLware developmenL process and how lL can flL lnLo Lhe buslness approach and goals of Lhe organlzaLlon - CommunlcaLlon skllls and Lhe ablllLy Lo undersLand varlous sldes of lssues are lmporLanL - ln organlzaLlons ln Lhe early sLages of lmplemenLlng CA processes paLlence and dlplomacy are especlally needed - An ablllLy Lo flnd problems as well as Lo see 'whaL's mlsslng' ls lmporLanL for lnspecLlons and revlews
3 WhaL makes a good CA or 1esL manager?
- A good CA LesL or CA/1esL (comblned) manager should - be famlllar wlLh Lhe sofLware developmenL process - be able Lo malnLaln enLhuslasm of Lhelr Leam and promoLe a poslLlve aLmosphere desplLe
whaL ls a somewhaL 'negaLlve' process (eg looklng for or prevenLlng problems)
- be able Lo promoLe Leamwork Lo lncrease producLlvlLy - be able Lo promoLe cooperaLlon beLween sofLware LesL and CA englneers - have Lhe dlplomaLlc skllls needed Lo promoLe lmprovemenLs ln CA processes - have Lhe ablllLy Lo wlLhsLand pressures and say 'no' Lo oLher managers when quallLy ls lnsufflclenL or CA processes are noL belng adhered Lo - have people [udgmenL skllls for hlrlng and keeplng skllled personnel - be able Lo communlcaLe wlLh Lechnlcal and nonLechnlcal people englneers managers and cusLomers - be able Lo run meeLlngs and keep Lhem focused
4 WhaL's Lhe role of documenLaLlon ln CA?
- CrlLlcal (noLe LhaL documenLaLlon can be elecLronlc noL necessarlly paper) - CA pracLlces should be documenLed such LhaL Lhey are repeaLable - SpeclflcaLlons deslgns buslness rules lnspecLlon reporLs conflguraLlons code changes LesL plans LesL cases bug reporLs user manuals eLc should all be documenLed - 1here should ldeally be a sysLem for easlly flndlng and obLalnlng documenLs and deLermlnlng whaL documenLaLlon wlll have a parLlcular plece of lnformaLlon - Change managemenL for documenLaLlon should be used lf posslble
3 WhaL's Lhe blg deal abouL 'requlremenLs'?
- Cne of Lhe mosL rellable meLhods of lnsurlng problems or fallure ln a complex sofLware pro[ecL ls Lo have poorly documenLed requlremenLs speclflcaLlons - 8equlremenLs are Lhe deLalls descrlblng an appllcaLlon's exLernallypercelved funcLlonallLy and properLles - 8equlremenLs should be clear compleLe reasonably deLalled coheslve aLLalnable and LesLable A nonLesLable requlremenL would be for example 'userfrlendly' (Loo sub[ecLlve) - A LesLable requlremenL would be someLhlng llke 'Lhe user musL enLer Lhelr prevlously asslgned password Lo access Lhe appllcaLlon' - ueLermlnlng and organlzlng requlremenLs deLalls ln a useful and efflclenL way can be a dlfflculL efforL dlfferenL meLhods are avallable dependlng on Lhe parLlcular pro[ecL - Many books are avallable LhaL descrlbe varlous approaches Lo Lhls Lask - Care should be Laken Lo lnvolve ALL of a pro[ecL's slgnlflcanL 'cusLomers' ln Lhe requlremenLs process - 'CusLomers' could be lnhouse personnel or ouL and could lnclude endusers cusLomer accepLance LesLers cusLomer conLracL offlcers cusLomer managemenL fuLure sofLware malnLenance englneers salespeople eLc - Anyone who could laLer derall Lhe pro[ecL lf Lhelr expecLaLlons aren'L meL should be lncluded lf posslble - CrganlzaLlons vary conslderably ln Lhelr handllng of requlremenLs speclflcaLlons - ldeally Lhe requlremenLs are spelled ouL ln a documenL wlLh sLaLemenLs such as '1he producL shall' - 'ueslgn' speclflcaLlons should noL be confused wlLh 'requlremenLs' deslgn speclflcaLlons should be Lraceable back Lo Lhe requlremenLs - ln some organlzaLlons requlremenLs may end up ln hlgh level pro[ecL plans funcLlonal speclflcaLlon documenLs ln deslgn documenLs or ln oLher documenLs aL varlous levels of deLall - no maLLer whaL Lhey are called some Lype of documenLaLlon wlLh deLalled requlremenLs wlll be needed by LesLers ln order Lo properly plan and execuLe LesLs - WlLhouL such documenLaLlon Lhere wlll be no clearcuL way Lo deLermlne lf a sofLware appllcaLlon ls performlng correcLly
6 WhaL sLeps are needed Lo develop and run sofLware LesLs?
1he followlng are some of Lhe sLeps Lo conslder - CbLaln requlremenLs funcLlonal deslgn and lnLernal deslgn speclflcaLlons and oLher necessary documenLs - CbLaln budgeL and schedule requlremenLs - ueLermlne pro[ecLrelaLed personnel and Lhelr responslblllLles reporLlng requlremenLs requlred sLandards and processes (such as release processes change processes eLc) - ldenLlfy appllcaLlon's hlgherrlsk aspecLs seL prlorlLles and deLermlne scope and llmlLaLlons of LesLs - ueLermlne LesL approaches and meLhods unlL lnLegraLlon funcLlonal sysLem load usablllLy LesLs eLc - ueLermlne LesL envlronmenL requlremenLs (hardware sofLware communlcaLlons eLc) - ueLermlne LesL ware requlremenLs (record/playback Lools coverage analyzers LesL Lracklng problem/bug Lracklng eLc) - ueLermlne LesL lnpuL daLa requlremenLs - ldenLlfy Lasks Lhose responslble for Lasks and labor requlremenLs - SeL schedule esLlmaLes Llmellnes mllesLones - ueLermlne lnpuL equlvalence classes boundary value analyses error classes - repare LesL plan documenL and have needed revlews/approvals - WrlLe LesL cases - Pave needed revlews/lnspecLlons/approvals of LesL cases - repare LesL envlronmenL and LesL ware obLaln needed user manuals/reference documenLs/conflguraLlon guldes/lnsLallaLlon guldes seL up LesL Lracklng processes seL up logglng and archlvlng processes seL up or obLaln LesL lnpuL daLa - CbLaln and lnsLall sofLware releases - erform LesLs - LvaluaLe and reporL resulLs - 1rack problems/bugs and flxes - 8eLesL as needed - MalnLaln and updaLe LesL plans LesL cases LesL envlronmenL and LesL ware Lhrough llfe cycle
7 WhaL's a 'LesL plan'?
A sofLware pro[ecL LesL plan ls a documenL LhaL descrlbes Lhe ob[ecLlves scope approach and focus of a sofLware LesLlng efforL 1he process of preparlng a LesL plan ls a useful way Lo Lhlnk Lhrough Lhe efforLs needed Lo valldaLe Lhe accepLablllLy of a sofLware producL 1he compleLed documenL wlll help people ouLslde Lhe LesL group undersLand Lhe 'why' and 'how' of producL valldaLlon lL should be Lhorough enough Lo be useful buL noL so Lhorough LhaL no one ouLslde Lhe LesL group wlll read lL 1he followlng are some of Lhe lLems LhaL mlghL be lncluded ln a LesL plan dependlng on Lhe parLlcular pro[ecL - 1lLle - ldenLlflcaLlon of sofLware lncludlng verslon/release numbers - 8evlslon hlsLory of documenL lncludlng auLhors daLes approvals - 1able of ConLenLs - urpose of documenL lnLended audlence - Cb[ecLlve of LesLlng efforL - SofLware producL overvlew - 8elevanL relaLed documenL llsL such as requlremenLs deslgn documenLs oLher LesL plans eLc - 8elevanL sLandards or legal requlremenLs - 1raceablllLy requlremenLs - 8elevanL namlng convenLlons and ldenLlfler convenLlons - Cverall sofLware pro[ecL organlzaLlon and personnel/conLacLlnfo/responslblllLles - 1esL organlzaLlon and personnel/conLacLlnfo/responslblllLles - AssumpLlons and dependencles - ro[ecL rlsk analysls - 1esLlng prlorlLles and focus - Scope and llmlLaLlons of LesLlng - 1esL ouLllne a decomposlLlon of Lhe LesL approach by LesL Lype feaLure funcLlonallLy process sysLem module eLc as appllcable - CuLllne of daLa lnpuL equlvalence classes boundary value analysls error classes - 1esL envlronmenL hardware operaLlng sysLems oLher requlred sofLware daLa conflguraLlons lnLerfaces Lo oLher sysLems - 1esL envlronmenL seLup and conflguraLlon lssues - 1esL daLa seLup requlremenLs - uaLabase seLup requlremenLs - CuLllne of sysLemlogglng/errorlogglng/oLher capablllLles and Lools such as screen capLure sofLware LhaL wlll be used Lo help descrlbe and reporL bugs - ulscusslon of any speclallzed sofLware or hardware Lools LhaL wlll be used by LesLers Lo help Lrack Lhe cause or source of bugs - 1esL auLomaLlon [usLlflcaLlon and overvlew - 1esL Lools Lo be used lncludlng verslons paLches eLc - 1esL scrlpL/LesL code malnLenance processes and verslon conLrol - roblem Lracklng and resoluLlon Lools and processes - ro[ecL LesL meLrlcs Lo be used - 8eporLlng requlremenLs and LesLlng dellverables - SofLware enLrance and exlL crlLerla - lnlLlal sanlLy LesLlng perlod and crlLerla - 1esL suspenslon and resLarL crlLerla - ersonnel allocaLlon - ersonnel preLralnlng needs - 1esL slLe/locaLlon - CuLslde LesL organlzaLlons Lo be uLlllzed and Lhelr purpose responslblllLles dellverables conLacL persons and coordlnaLlon lssues - 8elevanL proprleLary classlfled securlLy and llcenslng lssues - Cpen lssues - Appendlx glossary acronyms eLc
8 WhaL's a 'LesL case'?
- A LesL case ls a documenL LhaL descrlbes an lnpuL acLlon or evenL and an expecLed response Lo deLermlne lf a feaLure of an appllcaLlon ls worklng correcLly - A LesL case should conLaln parLlculars such as LesL case ldenLlfler LesL case name ob[ecLlve LesL condlLlons/seLup lnpuL daLa requlremenLs sLeps and expecLed resulLs - noLe LhaL Lhe process of developlng LesL cases can help flnd problems ln Lhe requlremenLs or deslgn of an appllcaLlon slnce lL requlres compleLely Lhlnklng Lhrough Lhe operaLlon of Lhe appllcaLlon lor Lhls reason lL's useful Lo prepare LesL cases early ln Lhe developmenL cycle lf posslble 9 WhaL should be done afLer a bug ls found?
1he bug needs Lo be communlcaLed and asslgned Lo developers who can flx lL AfLer Lhe problem ls resolved flxes should be reLesLed and deLermlnaLlons made regardlng requlremenLs for regresslon LesLlng Lo check LhaL flxes dldn'L creaLe problems elsewhere lf a problemLracklng sysLem ls ln place lL should encapsulaLe Lhese processes A varleLy of commerclal problemLracklng/managemenL sofLware Lools are avallable 1he followlng are lLems Lo be consldered ln Lhe Lracklng process - CompleLe lnformaLlon such LhaL developers can undersLand Lhe bug geL an ldea of lL's severlLy and reproduce lL lf necessary - 8ug ldenLlfler (number lu eLc) - CurrenL bug sLaLus (eg '8eleased for 8eLesL' 'new' eLc) - 1he appllcaLlon name or ldenLlfler and verslon - 1he funcLlon module feaLure ob[ecL screen eLc where Lhe bug occurred - LnvlronmenL speclflcs sysLem plaLform relevanL hardware speclflcs - 1esL case name/number/ldenLlfler - Cnellne bug descrlpLlon - lull bug descrlpLlon - uescrlpLlon of sLeps needed Lo reproduce Lhe bug lf noL covered by a LesL case or lf Lhe developer doesn'L have easy access Lo Lhe LesL case/LesL scrlpL/LesL Lool - names and/or descrlpLlons of flle/daLa/messages/eLc used ln LesL - llle excerpLs/error messages/log flle excerpLs/screen shoLs/LesL Lool logs LhaL would be helpful ln flndlng Lhe cause of Lhe problem - SeverlLy esLlmaLe (a 3level range such as 13 or 'crlLlcal'Lo'low' ls common) - Was Lhe bug reproduclble? - 1esLer name - 1esL daLe - 8ug reporLlng daLe - name of developer/group/organlzaLlon Lhe problem ls asslgned Lo - uescrlpLlon of problem cause - uescrlpLlon of flx - Code secLlon/flle/module/class/meLhod LhaL was flxed - uaLe of flx - AppllcaLlon verslon LhaL conLalns Lhe flx - 1esLer responslble for reLesL - 8eLesL daLe - 8eLesL resulLs - 8egresslon LesLlng requlremenLs - 1esLer responslble for regresslon LesLs - 8egresslon LesLlng resulLs A reporLlng or Lracklng process should enable noLlflcaLlon of approprlaLe personnel aL varlous sLages lor lnsLance LesLers need Lo know when reLesLlng ls needed developers need Lo know when bugs are found and how Lo geL Lhe needed lnformaLlon and reporLlng/summary capablllLles are needed for managers
10 WhaL ls 'conflguraLlon managemenL'?
ConflguraLlon managemenL covers Lhe processes used Lo conLrol coordlnaLe and Lrack code requlremenLs documenLaLlon problems change requesLs deslgns Lools/compllers/llbrarles/paLches changes made Lo Lhem and who makes Lhe changes
11 WhaL lf Lhe sofLware ls so buggy lL can'L really be LesLed aL all?
1he besL beL ln Lhls slLuaLlon ls for Lhe LesLers Lo go Lhrough Lhe process of reporLlng whaLever bugs or blocklngLype problems lnlLlally show up wlLh Lhe focus belng on crlLlcal bugs Slnce Lhls Lype of problem can severely affecL schedules and lndlcaLes deeper problems ln Lhe sofLware developmenL process (such as lnsufflclenL unlL LesLlng or lnsufflclenL lnLegraLlon LesLlng poor deslgn lmproper bulld or release procedures eLc) managers should be noLlfled and provlded wlLh some documenLaLlon as evldence of Lhe problem
12 Pow can lL be known when Lo sLop LesLlng?
1hls can be dlfflculL Lo deLermlne Many modern sofLware appllcaLlons are so complex and run ln such an lnLerdependenL envlronmenL LhaL compleLe LesLlng can never be done Common facLors ln decldlng when Lo sLop are - ueadllnes (release deadllnes LesLlng deadllnes eLc) - 1esL cases compleLed wlLh cerLaln percenLage passed - 1esL budgeL depleLed - Coverage of code/funcLlonallLy/requlremenLs reaches a speclfled polnL - 8ug raLe falls below a cerLaln level - 8eLa or alpha LesLlng perlod ends
13 WhaL lf Lhere lsn'L enough Llme for Lhorough LesLlng?
use rlsk analysls Lo deLermlne where LesLlng should be focused
Slnce lL's rarely posslble Lo LesL every posslble aspecL of an appllcaLlon every posslble comblnaLlon of evenLs every dependency or everyLhlng LhaL could go wrong rlsk analysls ls approprlaLe Lo mosL sofLware developmenL pro[ecLs 1hls requlres [udgmenL skllls common sense and experlence (lf warranLed formal meLhods are also avallable)
ConslderaLlons can lnclude
- Whlch funcLlonallLy ls mosL lmporLanL Lo Lhe pro[ecL's lnLended purpose? - Whlch funcLlonallLy ls mosL vlslble Lo Lhe user? - Whlch funcLlonallLy has Lhe largesL safeLy lmpacL? - Whlch funcLlonallLy has Lhe largesL flnanclal lmpacL on users? - Whlch aspecLs of Lhe appllcaLlon are mosL lmporLanL Lo Lhe cusLomer? - Whlch aspecLs of Lhe appllcaLlon can be LesLed early ln Lhe developmenL cycle? - Whlch parLs of Lhe code are mosL complex and Lhus mosL sub[ecL Lo errors? - Whlch parLs of Lhe appllcaLlon were developed ln rush or panlc mode? - Whlch aspecLs of slmllar/relaLed prevlous pro[ecLs caused problems? - Whlch aspecLs of slmllar/relaLed prevlous pro[ecLs had large malnLenance expenses? - Whlch parLs of Lhe requlremenLs and deslgn are unclear or poorly LhoughL ouL? - WhaL do Lhe developers Lhlnk are Lhe hlghesLrlsk aspecLs of Lhe appllcaLlon? - WhaL klnds of problems would cause Lhe worsL publlclLy? - WhaL klnds of problems would cause Lhe mosL cusLomer servlce complalnLs? - WhaL klnds of LesLs could easlly cover mulLlple funcLlonallLles? - Whlch LesLs wlll have Lhe besL hlghrlskcoverage Lo Llmerequlred raLlo?
14 WhaL lf Lhe pro[ecL lsn'L blg enough Lo [usLlfy exLenslve LesLlng?
Conslder Lhe lmpacL of pro[ecL errors noL Lhe slze of Lhe pro[ecL Powever lf exLenslve LesLlng ls sLlll noL [usLlfled rlsk analysls ls agaln needed and Lhe same conslderaLlons as descrlbed prevlously ln 1he LesLer mlghL Lhen do ad hoc LesLlng or wrlLe up a llmlLed LesL plan based on Lhe rlsk analysls
13 WhaL can be done lf requlremenLs are changlng conLlnuously?
A common problem and a ma[or headache - Work wlLh Lhe pro[ecL's sLakeholders early on Lo undersLand how requlremenLs mlghL change so LhaL alLernaLe LesL plans and sLraLegles can be worked ouL ln advance lf posslble - lL's helpful lf Lhe appllcaLlon's lnlLlal deslgn allows for some adapLablllLy so LhaL laLer changes do noL requlre redolng Lhe appllcaLlon from scraLch - lf Lhe code ls wellcommenLed and welldocumenLed Lhls makes changes easler for Lhe developers - use rapld proLoLyplng whenever posslble Lo help cusLomers feel sure of Lhelr requlremenLs and mlnlmlze changes - 1he pro[ecL's lnlLlal schedule should allow for some exLra Llme commensuraLe wlLh Lhe posslblllLy of changes - 1ry Lo move new requlremenLs Lo a 'hase 2r verslon of an appllcaLlon whlle uslng Lhe orlglnal requlremenLs for Lhe 'hase 1r verslon - negoLlaLe Lo allow only easllylmplemenLed new requlremenLs lnLo Lhe pro[ecL whlle movlng more dlfflculL new requlremenLs lnLo fuLure verslons of Lhe appllcaLlon - 8e sure LhaL cusLomers and managemenL undersLand Lhe schedullng lmpacLs lnherenL rlsks and cosLs of slgnlflcanL requlremenLs changes 1hen leL managemenL or Lhe cusLomers (noL Lhe developers or LesLers) declde lf Lhe changes are warranLed afLer all LhaL's Lhelr [ob - 8alance Lhe efforL puL lnLo seLLlng up auLomaLed LesLlng wlLh Lhe expecLed efforL requlred Lo redo Lhem Lo deal wlLh changes - 1ry Lo deslgn some flexlblllLy lnLo auLomaLed LesL scrlpLs - locus lnlLlal auLomaLed LesLlng on appllcaLlon aspecLs LhaL are mosL llkely Lo remaln unchanged - uevoLe approprlaLe efforL Lo rlsk analysls of changes Lo mlnlmlze regresslon LesLlng needs - ueslgn some flexlblllLy lnLo LesL cases (Lhls ls noL easlly done Lhe besL beL mlghL be Lo mlnlmlze Lhe deLall ln Lhe LesL cases or seL up only hlgherlevel generlcLype LesL plans) - locus less on deLalled LesL plans and LesL cases and more on ad hoc LesLlng (wlLh an undersLandlng of Lhe added rlsk LhaL Lhls enLalls)
16 WhaL lf Lhe appllcaLlon has funcLlonallLy LhaL wasn'L ln Lhe requlremenLs?
lL may Lake serlous efforL Lo deLermlne lf an appllcaLlon has slgnlflcanL unexpecLed or hldden funcLlonallLy and lL would lndlcaLe deeper problems ln Lhe sofLware developmenL process
lf Lhe funcLlonallLy lsn'L necessary Lo Lhe purpose of Lhe appllcaLlon lL should be removed as lL may have unknown lmpacLs or dependencles LhaL were noL Laken lnLo accounL by Lhe deslgner or Lhe cusLomer lf noL removed deslgn lnformaLlon wlll be needed Lo deLermlne added LesLlng needs or regresslon LesLlng needs ManagemenL should be made aware of any slgnlflcanL added rlsks as a resulL of Lhe unexpecLed funcLlonallLy lf Lhe funcLlonallLy only effecLs areas such as mlnor lmprovemenLs ln Lhe user lnLerface for example lL may noL be a slgnlflcanL rlsk
17 Pow can SofLware CA processes be lmplemenLed wlLhouL sLlfllng producLlvlLy?
8y lmplemenLlng CA processes slowly over Llme uslng consensus Lo reach agreemenL on processes and ad[usLlng and experlmenLlng as an organlzaLlon grows and maLures producLlvlLy wlll be lmproved lnsLead of sLlfled roblem prevenLlon wlll lessen Lhe need for problem deLecLlon panlcs and burnouL wlll decrease and Lhere wlll be lmproved focus and less wasLed efforL AL Lhe same Llme aLLempLs should be made Lo keep processes slmple and efflclenL mlnlmlze paperwork promoLe compuLerbased processes and auLomaLed Lracklng and reporLlng mlnlmlze Llme requlred ln meeLlngs and promoLe Lralnlng as parL of Lhe CA process Powever no one especlally LalenLed Lechnlcal Lypes llkes rules or bureaucracy and ln Lhe shorL run Lhlngs may slow down a blL A Lyplcal scenarlo would be LhaL more days of plannlng and developmenL wlll be needed buL less Llme wlll be requlred for laLenlghL bugflxlng and calmlng of lraLe cusLomers
18 WhaL lf an organlzaLlon ls growlng so fasL LhaL flxed CA processes are lmposslble?
1hls ls a common problem ln Lhe sofLware lndusLry especlally ln new Lechnology areas 1here ls no easy soluLlon ln Lhls slLuaLlon oLher Lhan - Plre good people - ManagemenL should 'ruLhlessly prlorlLlze' quallLy lssues and malnLaln focus on Lhe cusLomer - Lveryone ln Lhe organlzaLlon should be clear on whaL 'quallLy' means Lo Lhe cusLomer
19 Pow does a cllenL' server envlronmenL affecL LesLlng?
CllenL/server appllcaLlons can be qulLe complex due Lo Lhe mulLlple dependencles among cllenLs daLa communlcaLlons hardware and servers 1hus LesLlng requlremenLs can be exLenslve When Llme ls llmlLed (as lL usually ls) Lhe focus should be on lnLegraLlon and sysLem LesLlng AddlLlonally load/sLress/performance LesLlng may be useful ln deLermlnlng cllenL/server appllcaLlon llmlLaLlons and capablllLles 1here are commerclal Lools Lo asslsL wlLh such LesLlng
20 Pow can World Wlde Web slLes be LesLed?
Web slLes are essenLlally cllenL/server appllcaLlons wlLh web servers and 'browser' cllenLs ConslderaLlon should be glven Lo Lhe lnLeracLlons beLween hLml pages 1C/l communlcaLlons lnLerneL connecLlons flrewalls appllcaLlons LhaL run ln web pages (such as appleLs !avaScrlpL plugln appllcaLlons) and appllcaLlons LhaL run on Lhe server slde (such as cgl scrlpLs daLabase lnLerfaces logglng appllcaLlons dynamlc page generaLors asp eLc) AddlLlonally Lhere are a wlde varleLy of servers and browsers varlous verslons of each small buL someLlmes slgnlflcanL dlfferences beLween Lhem varlaLlons ln connecLlon speeds rapldly changlng Lechnologles and mulLlple sLandards and proLocols 1he end resulL ls LhaL LesLlng for web slLes can become a ma[or ongolng efforL CLher conslderaLlons mlghL lnclude - WhaL are Lhe expecLed loads on Lhe server (eg number of hlLs per unlL Llme?) and whaL klnd of performance ls requlred under such loads (such as web server response Llme daLabase query response Llmes) - WhaL klnds of Lools wlll be needed for performance LesLlng (such as web load LesLlng Lools oLher Lools already ln house LhaL can be adapLed web roboL downloadlng Lools eLc)? - Who ls Lhe LargeL audlence? WhaL klnd of browsers wlll Lhey be uslng? WhaL klnd of connecLlon speeds wlll Lhey by uslng? Are Lhey lnLra organlzaLlon (Lhus wlLh llkely hlgh connecLlon speeds and slmllar browsers) or lnLerneLwlde (Lhus wlLh a wlde varleLy of connecLlon speeds and browser Lypes)? - WhaL klnd of performance ls expecLed on Lhe cllenL slde (eg how fasL should pages appear how fasL should anlmaLlons appleLs eLc load and run)? - Wlll down Llme for server and conLenL malnLenance/upgrades be allowed? how much? - WhaL klnds of securlLy (flrewalls encrypLlons passwords eLc) wlll be requlred and whaL ls lL expecLed Lo do? Pow can lL be LesLed? - Pow rellable are Lhe slLe's lnLerneL connecLlons requlred Lo be? And how does LhaL affecL backup sysLem or redundanL connecLlon requlremenLs and LesLlng? - WhaL processes wlll be requlred Lo manage updaLes Lo Lhe web slLe's conLenL and whaL are Lhe requlremenLs for malnLalnlng Lracklng and conLrolllng page conLenL graphlcs llnks eLc? - Whlch P1ML speclflcaLlon wlll be adhered Lo? Pow sLrlcLly? WhaL varlaLlons wlll be allowed for LargeLed browsers? - Wlll Lhere be any sLandards or requlremenLs for page appearance and/or graphlcs LhroughouL a slLe or parLs of a slLe?? - Pow wlll lnLernal and exLernal llnks be valldaLed and updaLed? how ofLen? - Can LesLlng be done on Lhe producLlon sysLem or wlll a separaLe LesL sysLem be requlred? Pow are browser cachlng varlaLlons ln browser opLlon seLLlngs dlalup connecLlon varlablllLy's and realworld lnLerneL 'Lrafflc congesLlon' problems Lo be accounLed for ln LesLlng? - Pow exLenslve or cusLomlzed are Lhe server logglng and reporLlng requlremenLs are Lhey consldered an lnLegral parL of Lhe sysLem and do Lhey requlre LesLlng? - Pow are cgl programs appleLs !avaScrlpL's AcLlvex componenLs eLc Lo be malnLalned Lracked conLrolled and LesLed?
- ages should be 33 screens max unless conLenL ls LlghLly focused on a slngle Loplc lf larger provlde lnLernal llnks wlLhln Lhe page - 1he page layouLs and deslgn elemenLs should be conslsLenL LhroughouL a slLe so LhaL lL's clear Lo Lhe user LhaL Lhey're sLlll wlLhln a slLe - ages should be as browserlndependenL as posslble or pages should be provlded or generaLed based on Lhe browserLype - All pages should have llnks exLernal Lo Lhe page Lhere should be no deadend pages - 1he page owner revlslon daLe and a llnk Lo a conLacL person or organlzaLlon should be lncluded on each page 21 Pow ls LesLlng affecLed by ob[ecLorlenLed deslgns?
Wellenglneered ob[ecLorlenLed deslgn can make lL easler Lo Lrace from code Lo lnLernal deslgn Lo funcLlonal deslgn Lo requlremenLs Whlle Lhere wlll be llLLle affecL on black box LesLlng (where an undersLandlng of Lhe lnLernal deslgn of Lhe appllcaLlon ls unnecessary) whlLebox LesLlng can be orlenLed Lo Lhe appllcaLlon's ob[ecLs lf Lhe appllcaLlon was welldeslgned Lhls can slmpllfy LesL deslgn
22 WhaL ls LxLreme rogrammlng and whaL's lL goL Lo do wlLh LesLlng ?
LxLreme rogrammlng (x) ls a sofLware developmenL approach for small Leams on rlsk prone pro[ecLs wlLh unsLable requlremenLs lL was creaLed by kenL 8eck who descrlbed Lhe approach ln hls book 'LxLreme rogrammlng Lxplalned' 1esLlng ('exLreme LesLlng') ls a core aspecL of LxLreme rogrammlng rogrammers are expecLed Lo wrlLe unlL and funcLlonal LesL code flrsL before Lhe appllcaLlon ls developed 1esL code ls under source conLrol along wlLh Lhe resL of Lhe code CusLomers are expecLed Lo be an lnLegral parL of Lhe pro[ecL Leam and Lo help develop scenarlos for accepLance/black box LesLlng AccepLance LesLs are preferably auLomaLed and are modlfled and rerun for each of Lhe frequenL developmenL lLeraLlons
CA and LesL personnel are also requlred Lo be an lnLegral parL of Lhe pro[ecL Leam
ueLalled requlremenLs documenLaLlon ls noL used and frequenL reschedullng re esLlmaLlng and reprlorlLlzlng ls expecLed
ulfference beLween ueskLop CllenL server and Web LesLlng
CWhaL ls dlfference beLween cllenL server and Web 1esLlng? vl[ay Well Srlvldya l would llke Lo add one more LesLlng Lype le ueskLop 1esLlng ln Lhls dlscusslon So now we have Lhree LesLlng Lypes ueskLop appllcaLlon LesLlng CllenL server appllcaLlon LesLlng and Web appllcaLlon LesLlng
Lach one dlffers ln Lhe envlronmenL ln whlch Lhey are LesLed and you wlll lose conLrol over Lhe envlronmenL ln whlch appllcaLlon you are LesLlng whlle you move from deskLop Lo web appllcaLlons
ueskLop appllcaLlon runs on personal compuLers and work sLaLlons so when you LesL Lhe deskLop appllcaLlon you are focuslng on a speclflc envlronmenL ?ou wlll LesL compleLe appllcaLlon broadly ln caLegorles llke Cul funcLlonallLy Load and backend le u8
ln cllenL server appllcaLlon you have Lwo dlfferenL componenLs Lo LesL AppllcaLlon ls loaded on server machlne whlle Lhe appllcaLlon exe on every cllenL machlne ?ou wlll LesL broadly ln caLegorles llke Cul on boLh sldes funcLlonallLy Load cllenLserver lnLeracLlon backend 1hls envlronmenL ls mosLly used ln lnLraneL neLworks ?ou are aware of number of cllenLs and servers and Lhelr locaLlons ln Lhe LesL scenarlo
Web appllcaLlon ls a blL dlfferenL and complex Lo LesL as LesLer don'L have LhaL much conLrol over Lhe appllcaLlon AppllcaLlon ls loaded on Lhe server whose locaLlon may or may noL be known and no exe ls lnsLalled on Lhe cllenL machlne you have Lo LesL lL on dlfferenL web browsers Web appllcaLlons are supposed Lo be LesLed on dlfferenL browsers and CS plaLforms so broadly Web appllcaLlon ls LesLed malnly for browser compaLlblllLy and operaLlng sysLem compaLlblllLy error handllng sLaLlc pages backend LesLlng and load LesLlng
l Lhlnk Lhls wlll have an ldea of all Lhree LesLlng envlronmenL keep ln mlnd LhaL even Lhe dlfference exlsL ln Lhese Lhree envlronmenL Lhe baslc quallLy assurance and LesLlng prlnclples remalns same and applles Lo all
SeL 7 of lnLervlew lACs
C 1 Pow can l sLarL my career ln auLomaLed LesLlng? A number one l suggesL you read all you can and LhaL lncludes readlng producL descrlpLlon pamphleLs manuals books lnformaLlon on Lhe lnLerneL and whaLever lnformaLlon you can lay your hands on 1wo geL handson experlence on
SeL 8 of lnLervlew lACs
C 1 WhaL are Lhe parameLers of peer revlews? A 8y deflnlLlon parameLers are values on whlch someLhlng else depends eer revlews depend on Lhe aLLendance and acLlve parLlclpaLlon of several key people usually Lhe faclllLaLor Lask lead LesL lead and aL leasL one addlLlonal revlewer
Pow Lo feLch daLa from excel sheeL uslng 1LS1 ul8LC1C8?
Lhere ls an opLlon ln 1esL dlrecLor advance verslon 80 ln LhaL ln flle menu Lhere ls lmporL flle opLlon cllck on LhaL and glve Lhe paLh where lL ls Lhe paLh cllck on ok
ulfference beLween Cul luncLlonal LesLlng?
Cul 1 ln compuLer sclence Cul LesLlng ls Lhe process of LesLlng a graphlcal user lnLerface Lo ensure lL meeLs lLs wrlLLen speclflcaLlons 1hls ls normally done Lhrough Lhe use of a varleLy of LesL cases
luncLlonal LesLlng CuallLy assurance LhaL a web slLe performs properly All aspecLs of Lhe user lnLerface navlgaLlon beLween pages and offslLe mulLlllngual navlgaLlon eLc are LesLed 1esLlng ls requlred ln all Lhe currenL browsers and on Lhe ma[or operaLlng sysLems and plaLforms C8 1he porLlon of securlLy LesLlng ln whlch Lhe adverLlsed feaLures of a sysLem are LesLed for correcL operaLlon
8esponslblllLles of a CA Lnglneer
1lLle of Lhe posL should ldeally be 8esponslblllLles of a 1esL Lnglneer"
- undersLand pro[ecL requlremenLs - repare / updaLe Lhe 1esL case documenL for LesLlng of Lhe appllcaLlon from all aspecLs - repare Lhe LesL seLup - ueploy Lhe bulld ln Lhe requlred seLup - ConducL Lhe 1esLlng lncludlng Smoke SanlLy and 8ug 8ash / LxecuLe Lhe 1esL cases - updaLe Lhe 1esL 8esulL documenL
1esLlng ueflnlLlon ln Slmple
WhaL's Ad Poc 1esLlng ? A LesLlng where Lhe LesLer Lrles Lo break Lhe sofLware by randomly Lrylng funcLlonallLy of sofLware WhaL's Lhe AccesslblllLy 1esLlng ? 1esLlng LhaL deLermlnes lf sofLware wlll be usable by people wlLh dlsablllLles WhaL's Lhe Alpha 1esLlng ? 1he Alpha 1esLlng ls conducLed aL Lhe developer slLes and ln a conLrolled envlronmenL by Lhe end user of Lhe sofLware WhaL's Lhe 8eLa 1esLlng ? 1esLlng Lhe appllcaLlon afLer Lhe lnsLallaLlon aL Lhe cllenL place WhaL ls ComponenL 1esLlng ? 1esLlng of lndlvldual sofLware componenLs (unlL 1esLlng) WhaL's CompaLlblllLy 1esLlng ? ln CompaLlblllLy LesLlng we can LesL LhaL sofLware ls compaLlble wlLh oLher elemenLs of sysLem WhaL ls Concurrency 1esLlng ? MulLluser LesLlng geared Lowards deLermlnlng Lhe effecLs of accesslng Lhe same appllcaLlon code module or daLabase records ldenLlfles and measures Lhe level of locklng deadlocklng and use of slngleLhreaded code and locklng semaphores WhaL ls Conformance 1esLlng ? 1he process of LesLlng LhaL an lmplemenLaLlon conforms Lo Lhe speclflcaLlon on whlch lL ls based usually applled Lo LesLlng conformance Lo a formal sLandard WhaL ls ConLexL urlven 1esLlng ? 1he conLexLdrlven school of sofLware LesLlng ls flavor of Aglle 1esLlng LhaL advocaLes conLlnuous and creaLlve evaluaLlon of LesLlng opporLunlLles ln llghL of Lhe poLenLlal lnformaLlon revealed and Lhe value of LhaL lnformaLlon Lo Lhe organlzaLlon rlghL now WhaL ls uaLa urlven 1esLlng ? 1esLlng ln whlch Lhe acLlon of a LesL case ls parameLerlzed by exLernally deflned daLa values malnLalned as a flle or spreadsheeL A common Lechnlque ln AuLomaLed 1esLlng WhaL ls Converslon 1esLlng ? 1esLlng of programs or procedures used Lo converL daLa from exlsLlng sysLems for use ln replacemenL sysLems WhaL ls uependency 1esLlng ? Lxamlnes an appllcaLlon's requlremenLs for preexlsLlng sofLware lnlLlal sLaLes and conflguraLlon ln order Lo malnLaln proper funcLlonallLy WhaL ls uepLh 1esLlng ? A LesL LhaL exerclses a feaLure of a producL ln full deLall WhaL ls uynamlc 1esLlng ? 1esLlng sofLware Lhrough execuLlng lL See also SLaLlc 1esLlng WhaL ls Lndurance 1esLlng ? Checks for memory leaks or oLher problems LhaL may occur wlLh prolonged execuLlon WhaL ls LndLoLnd LesLlng ? 1esLlng a compleLe appllcaLlon envlronmenL ln a slLuaLlon LhaL mlmlcs realworld use such as lnLeracLlng wlLh a daLabase uslng neLwork communlcaLlons or lnLeracLlng wlLh oLher hardware appllcaLlons or sysLems lf approprlaLe WhaL ls LxhausLlve 1esLlng ? 1esLlng whlch covers all comblnaLlons of lnpuL values and precondlLlons for an elemenL of Lhe sofLware under LesL WhaL ls Corllla 1esLlng ? 1esLlng one parLlcular module funcLlonallLy heavlly WhaL ls lnsLallaLlon 1esLlng ? Conflrms LhaL Lhe appllcaLlon under LesL recovers from expecLed or unexpecLed evenLs wlLhouL loss of daLa or funcLlonallLy LvenLs can lnclude shorLage of dlsk space unexpecLed loss of communlcaLlon or power ouL condlLlons WhaL ls LocallzaLlon 1esLlng ? 1hls Lerm refers Lo maklng sofLware speclflcally deslgned for a speclflc locallLy WhaL ls Loop 1esLlng ? A whlLe box LesLlng Lechnlque LhaL exerclses program loops WhaL ls MuLaLlon 1esLlng ? MuLaLlon LesLlng ls a meLhod for deLermlnlng lf a seL of LesL daLa or LesL cases ls useful by dellberaLely lnLroduclng varlous code changes ('bugs') and reLesLlng wlLh Lhe orlglnal LesL daLa/cases Lo deLermlne lf Lhe 'bugs' are deLecLed roper lmplemenLaLlon requlres large compuLaLlonal resources WhaL ls Monkey 1esLlng ? 1esLlng a sysLem or an AppllcaLlon on Lhe fly le [usL few LesLs here and Lhere Lo ensure Lhe sysLem or an appllcaLlon does noL crash ouL WhaL ls oslLlve 1esLlng ? 1esLlng almed aL showlng sofLware works Also known as LesL Lo pass" See also negaLlve 1esLlng WhaL ls negaLlve 1esLlng ? 1esLlng almed aL showlng sofLware does noL work Also known as LesL Lo fall" See also oslLlve 1esLlng WhaL ls aLh 1esLlng ? 1esLlng ln whlch all paLhs ln Lhe program source code are LesLed aL leasL once WhaL ls erformance 1esLlng ? 1esLlng conducLed Lo evaluaLe Lhe compllance of a sysLem or componenL wlLh speclfled performance requlremenLs CfLen Lhls ls performed uslng an auLomaLed LesL Lool Lo slmulaLe large number of users Also know as Load 1esLlng" WhaL ls 8amp 1esLlng ? ConLlnuously ralslng an lnpuL slgnal unLll Lhe sysLem breaks down WhaL ls 8ecovery 1esLlng ? Conflrms LhaL Lhe program recovers from expecLed or unexpecLed evenLs wlLhouL loss of daLa or funcLlonallLy LvenLs can lnclude shorLage of dlsk space unexpecLed loss of communlcaLlon or power ouL condlLlons WhaL ls Lhe 8eLesLlng LesLlng ? 8eLesLlng Agaln LesLlng Lhe funcLlonallLy of Lhe appllcaLlon WhaL ls Lhe 8egresslon LesLlng ? 8egresslon Check LhaL change ln code have noL effecLed Lhe worklng funcLlonallLy WhaL ls SanlLy 1esLlng ? 8rlef LesL of ma[or funcLlonal elemenLs of a plece of sofLware Lo deLermlne lf lLs baslcally operaLlonal WhaL ls ScalablllLy 1esLlng ? erformance LesLlng focused on ensurlng Lhe appllcaLlon under LesL gracefully handles lncreases ln work load ln normal CondlLlon WhaL ls SecurlLy 1esLlng ? 1esLlng whlch conflrms LhaL Lhe program can resLrlcL access Lo auLhorlzed personnel and LhaL Lhe auLhorlzed personnel can access Lhe funcLlons avallable Lo Lhelr securlLy level WhaL ls SLress 1esLlng ? SLress LesLlng ls a form of LesLlng LhaL ls used Lo deLermlne Lhe sLablllLy of a glven sysLem or enLlLy lL lnvolves LesLlng beyond normal operaLlonal capaclLy ofLen Lo a breaklng polnL ln order Lo observe Lhe resulLs ln Abnormal CondlLlon WhaL ls Smoke 1esLlng ? A qulckanddlrLy LesL LhaL Lhe ma[or funcLlons of a plece of sofLware work CrlglnaLed ln Lhe hardware LesLlng pracLlce of Lurnlng on a new plece of hardware for Lhe flrsL Llme and conslderlng lL a success lf lL does noL caLch on flre WhaL ls Soak 1esLlng ? 8unnlng a sysLem aL hlgh load for a prolonged perlod of Llme lor example runnlng several Llmes more LransacLlons ln an enLlre day (or nlghL) Lhan would be expecLed ln a busy day Lo ldenLlfy and performance problems LhaL appear afLer a large number of LransacLlons have been execuLedlL means 8ellablllLy wlLh load WhaL's Lhe usablllLy LesLlng ? usablllLy LesLlng ls for user frlendllness WhaL's Lhe user accepLance LesLlng ? user accepLance LesLlng ls deLermlnlng lf sofLware ls saLlsfacLory Lo an enduser or cusLomer WhaL's Lhe volume 1esLlng ? We can perform Lhe volume LesLlng where Lhe sysLem ls sub[ecLed Lo large volume of daLa