Anda di halaman 1dari 13

Software Testing Club: Software Testing Checklists:Checklists

Software Quality Characteristics


http://thetes teye.com/pos ters /TheTes tEye_SoftwareQualityCharacteris tics .pdf Go through the list and think about your product/features. Add specifics for your context, and transform the list to your own.

Capability. Can the product perform valuable functions?


Completenes s : all important functions wanted by end us ers are available. Accuracy: any output or calculation in the product is correct and pres ented with s ignificant digits . Efficiency: performs its actions in an efficient manner (without doing what its not s uppos ed to do.) Interoperability: different features interact with each other in the bes t way. Concurrency: ability to perform multiple parallel tas ks , and run at the s ame time as other proces s es . Data agnos ticis m: s upports all pos s ible data formats , and handles nois e. Extens ibility: ability for cus tomers or 3rd parties to add features or change behavior.

Reliability. Can you trust the product in many and difficult situations?
Stability: the product s houldnt caus e cras hes , unhandled exceptions or s cript errors . Robus tnes s : the product handles fores een and unfores een errors gracefully. Stres s handling: how does the s ys tem cope when exceeding various limits ? Recoverability: it is pos s ible to recover and continue us ing the product after a fatal error. Data Integrity: all types of data remain intact throughout the product. Safety: the product will not be part of damaging people or pos s es s ions . Dis as ter Recovery: what if s omething really, really bad happens ? Trus tworthines s : is the products behavior cons is tent, predictable, and trus tworthy?

Usability. Is the product easy to use?


Affordance: product invites to dis cover pos s ibilities of the product. Intuitivenes s : it is eas y to unders tand and explain what the product can do. Minimalis m: there is nothing redundant about the products content or appearance. Learnability: it is fas t and eas y to learn how to us e the product. Memorability: once you have learnt how to do s omething you dont forget it. Dis coverability: the products information and capabilities can be dis covered by exploration of the us er interface. Operability: an experienced us er can perform common actions very fas t. Interactivity: the product has eas y-to-unders tand s tates and pos s ibilities of interacting with the application (via GUI or API). Control: the us er s hould feel in control over the proceedings of the s oftware. Clarity: is everything s tated explicitly and in detail, with a language that can be unders tood, leaving no room for doubt? Errors : there are informative error mes s ages , difficult to make mis takes and eas y to repair after making them. Cons is tency: behavior is the s ame throughout the product, and there is one look & feel. Tailorability: default s ettings and behavior can be s pecified for flexibility. Acces s ibility: the product is pos s ible to us e for as many people as pos s ible, and

Charisma. Does the product have it?


Uniquenes s : the product is dis tinguis hable and has s omething no one els e has . Satis faction: how do you feel after us ing the product? Profes s ionalis m: does the product have the appropriate flair of profes s ionalis m and feel fit for purpos e? Attractivenes s : are all types of as pects of the product appealing to eyes and other s ens es ?

Attractivenes s : are all types of as pects of the product appealing to eyes and other s ens es ? Curios ity: will us ers get interes ted and try out what they can do with the product? Entrancement: do us ers get hooked, have fun, in a flow, and fully engaged when us ing the product? Hype: s hould the product us e the lates t and greates t technologies /ideas ? Expectancy: the product exceeds expectations and meets the needs you didn't know you had. Attitude: do the product and its information have the right attitude and s peak to you with the right language and s tyle? Directnes s : are (firs t) impres s ions impres s ive? Story: are there compelling s tories about the products inception, cons truction or us age?

Security. Does the product protect against unwanted usage?


Authentication: the products identifications of the us ers . Authorization: the products handling of what an authenticated us er can s ee and do. Privacy: ability to not dis clos e data that is protected to unauthorized us ers . Security holes : product s hould not invite to s ocial engineering vulnerabilities . Secrecy: the product s hould under no circums tances dis clos e information about the underlying s ys tems . Invulnerability: ability to withs tand penetration attempts . Virus -free: product will not trans port virus , or appear as one. Piracy Res is tance: no pos s ibility to illegally copy and dis tribute the s oftware or code. Compliance: s ecurity s tandards the product adheres to.

Performance. Is the product fast enough?


Capacity: the many limits of the product, for different circums tances (e.g. s low network.) Res ource Utilization: appropriate us age of memory, s torage and other res ources . Res pons ivenes s : the s peed of which an action is (perceived as ) performed. Availability: the s ys tem is available for us e when it s hould be. Throughput: the products ability to proces s many, many things . Endurance: can the product handle load for a long time? Feedback: is the feedback from the s ys tem on us er actions appropriate? Scalability: how well does the product s cale up, out or down?

IT-bility. Is the product easy to install, maintain and support?


Sys tem requirements : ability to run on s upported configurations , and handle different environments or mis s ing components . Ins tallability: product can be ins talled on intended platforms with appropriate footprint. Upgrades : eas e of upgrading to a newer vers ion without los s of configuration and s ettings . Unins tallation: are all files (except us ers or s ys tem files ) and other res ources removed when unins talling? Configuration: can the ins tallation be configured in various ways or places to s upport cus tomers us age? Deployability: product can be rolled-out by IT department to different types of (res tricted) us ers and environments . Maintainability: are the product and its artifacts eas y to maintain and s upport for cus tomers ? Tes tability: how effectively can the deployed product be tes ted by the cus tomer?

Compatibility. How well does the product interact with software and environments?
Hardware Compatibility: the product can be us ed with applicable configurations of hardware components . Operating Sys tem Compatibility: the product can run on intended operating s ys tem vers ions , and follows typical behavior. Application Compatibility: the product, and its data, works with other applications cus tomers are likely to us e. Configuration Compatibility: products ability to blend in with configurations of the environment. Backward Compatibility: can the product do everything the las t vers ion could? Forward Compatibility: will the product be able to us e artifacts or interfaces of future vers ions ? Sus tainability: effects on the environment, e.g. energy efficiency, s witch-offs , power-s aving modes , telecommuting. Standards Conformance: the product conforms to applicable s tandards , regulations , laws or ethics .

Internal Software Quality Characteristics


These characteristics are not directly experienced by end users, but can be equally important for successful products.

Supportability. Can customers usage and problems be supported?


Identifiers : is it eas y to identify parts of the product and their vers ions , or s pecific errors ? Diagnos tics : is it pos s ible to find out details regarding cus tomer s ituations ? Troubles hootable: is it eas y to pinpoint errors (e.g. log files ) and get help? Debugging: can you obs erve the internal s tates of the s oftware when needed? Vers atility: ability to us e the product in more ways than it was originally des igned for.

Testability. Is it easy to check and test the product?


Traceability: the product logs actions at appropriate levels and in us able format. Controllability: ability to independently s et s tates , objects or variables . Obs ervability: ability to obs erve things that s hould be tes ted. Monitorability: can the product give hints on what/how it is doing? Is olateability: ability to tes t a part by its elf. Stability: changes to the s oftware are controlled, and not too frequent. Automation: are there public or hidden programmatic interface that can be us ed? Information: ability for tes ters to learn what needs to be learned... Auditability: can the product and its creation be validated?

Maintainability. Can the product be maintained and extended at low cost?


Flexibility: the ability to change the product as required by cus tomers . Extens ibility: will it be eas y to add features in the future? Simplicity: the code is not more complex than needed, and does not obs cure tes t des ign, execution and evaluation. Readability: the code is adequately documented and eas y to read and unders tand. Trans parency: Is it eas y to unders tand the underlying s tructures ? Modularity: the code is s plit into manageable pieces . Refactorability: are you s atis fied with the unit tes ts ? Analyzability: ability to find caus es for defects or other code of interes t.

Portability. Is transferring of the product to different environments enabled?


Reus ability: can parts of the product be re-us ed els ewhere? Adaptability: is it eas y to change the product to s upport a different environment? Compatibility: does the product comply with common interfaces or official s tandards ? Internationalization: it is eas y to trans late the product. Localization: are all parts of the product adjus ted to meet the needs of the targeted culture/country? Us er Interface-robus tnes s : will the product look equally good when trans lated? Rikard Edgren, Henrik Emilsson and Martin Jansson - thetesteye.com v1.1 This work is licensed under the Creative Commons Attribution-No Derivative License inspired by James Bachs CRUSSPIC STMPL, ISO 9126-1, Wikipedia:Ilities and more _______________________________________________________________________________________________________

Test Heuristics Cheat Sheet


http://tes tobs es s ed.com/wp-content/uploads /2007/02/tes theuris tics cheats heetv1.pdf

Data Type Attacks


Paths/Files
Long Name(>255 chars ) Special Characters in Name(s pace * ? / \ | < > , . ( ) [ ] { } ; : !@ # $ % ^ & ) Non-Exis tent Already Exis ts No Space Minimal Space Write-Protected Unavailable Locked On Remote Machine Corrupted

Time and Date


Timeouts Time Difference between Machines Cros s ing Time Z ones Leap Days Always Invalid Days (Feb 30, Sept 31) Feb 29 in Non-Leap Years Different Formats (June 5, 2001; 06/05/2001; 06/05/01; 06-05-01; 6/5/2001 12:34, US vs Europe 5/6/2001) ISO week/year (53 week years ) Daylight Savings Changeover Date fields s aved as date/time format (time s aved as "00:00:00" caus es date change on Daylight Savings or Time Z one s hift) Res et Clock Backward or Forward

Numbers
0 32768 (2^15) 32769 (2^15 + 1) 65536 (2^16) 65537 (2^16 +1) 2147483648 (2^31) 2147483649 (2^31 + 1) 4294967296 (2^32) 4294967297 (2^32 + 1) Scientific Notation (1E-16) Negative Floating Point/Decimal (0.0001) With Commas (1,234,567) European Style (1.234.567,89) All the Above in Calculations

Strings
Long (255, 256, 257, 1000, 1024, 2000, 2048 or more characters ) Accented Chars (, etc.) As ian Chars ( ) Common Delimiters and Special Characters ( ` | / \ , ; : & < > ^ ? Tab ) Leave Blank Single Space Multiple Spaces Leading Spaces End-of-Line Characters (^M)

SQL Injection ( s elect from cus tomer ) With All Actions (Entering, Searching, Updating, etc.) Right to Left Languages (Hebrew, Arabic)

General
Violates Domain-Specific Rules (an ip addres s of 999.999.999.999, an email addres s with no @, an age of -1) Violates Uniquenes s Cons traint

Web Tests
Navigation
Back (watch for Expired mes s ages and double-pos ted trans actions ) Refres h Bookmark the URL Select Bookmark when Logged Out Hack the URL (change/remove parameters ; s ee als o Data Type Attacks ) Multiple Brows er Ins tances Open

Input
See also Data Type Attacks HTML/JavaScript Injection (allowing the us er to enter arbitrary HTML tags and JavaScript commands can lead to s ecurity vulnerabilities ) Check Max Length Defined on Text Inputs > 5000 Chars in TextAreas

Syntax
HTML Syntax Checker (http://validator.w3.org/) CSS Syntax Checker (http://jigs aw.w3.org/cs s -validator/)

Preferences
Javas cript Off Cookies Off Security High Res ize Preferences Brows er Window Change Font Size

Testing Wisdom
A tes t is an experiment des igned to reveal information or ans wer a s pecific ques tion about the s oftware or s ys tem. Stakeholders have ques tions ; tes ters have ans wers . Dont confus e s peed with progres s . Take a contrary approach. Obs ervation is exploratory. The narrower the view, the wider the ignorance. Big bugs are often found by coincidence. Bugs clus ter. Vary s equences , configurations , and data to increas e the probability that, if there is a problem, tes ting will find it. Its all about the variables .

Heuristics
Variable Analysis - Identify anything whos e value can change. Variables can be obvious , s ubtle, or hidden. Touch Points - Identify any public or private interface that provides vis ibility or control. Provides places to provoke, monitor, and verify the s ys tem. Boundaries - Approaching the Boundary (almos t too big, almos t too s mall), At the Boundary Goldilocks - Too Big, Too Small, Jus t Right CRUD - Create, Read, Update, Delete Follow the Data - Perform a s equence of actions involving data, verifying the data integrity at each s tep. (Example: Enter Search Report Export Import Update View) Configurations - Varying the variables related to configuration (Screen Res olution; Network Speed, Latency, Signal Strength; Memory; Dis k Availability; Count heuris tic applied to any peripheral s uch as 0, 1, Many Monitors , Mice, or Printers ) Interruptions - Log Off, Shut Down, Reboot, Kill Proces s , Dis connect, Hibernate, Timeout, Cancel Starvation - CPU, Memory, Network, or Dis k at maximum capacity Position - Beginning, Middle, End (Edit at the beginning of the line, middle of the line, end of the line) Selection - Some, None, All (Some permis s ions , No permis s ions , All permis s ions ) Count - 0, 1, Many (0 trans actions , 1 trans actions , Many s imultaneous trans actions ) Multi-User - Simultaneous create, update, delete from two accounts or s ame account logged in twice. Flood - Multiple s imultaneous trans actions or reques ts flooding the queue. Dependencies - Identify has a relations hips (a Cus tomer has an Invoice; an Invoice has multiple Line Items ). Apply CRUD, Count, Pos ition, and/or Selection heuris tics (Cus tomer has 0, 1, many Invoices ; Invoice has 0, 1, many Line Items ; Delete las t Line Item then Read; Update firs t Line Item; Some, None, All Line Items are taxable; Delete Cus tomer with 0, 1, Many Invoices ) Constraints - Violate cons traints (leave required fields blank, enter invalid combinations in dependent fields , enter duplicate IDs or names ). Apply with the Input Method heuris tic. Input Method - Typing, Copy/Pas te, Import, Drag/Drop, Various Interfaces (GUI v. API) Sequences - Vary Order of Operations , Undo/Redo, Revers e, Combine, Invert, Simultaneous Sorting - Alpha v. Numeric, Acros s Multiple Pages State Analysis - Identify s tates and events /trans itions , then repres ent them in a picture or table. Works with the Sequences and Interruption heuris tics . Map Making - Identify a bas e or home s tate. Pick a direction and take one s tep. Return to bas e. Repeat. Users & Scenarios - Us e Cas es , Soap Operas , Pers onae, Extreme Pers onalities

Frameworks
Judgment Inconsistencies
Abs ences , and Extras with res pect to Internal External Specific, or External Cultural reference points . (James Lynds ay, Workroom Productions )

Observations
Input Output Linkage (James Lynds ay, Workroom Productions )

Flow
Input Proces s ing Output

Requirements

Us ers Function Attributes Cons traints (Gaus e & Weinberg Exploring Requirements )

Nouns & Verbs


The objects or data in the s ys tem and the ways in which the s ys tem manipulates it. Adjectives (attributes ) s uch as Vis ible, Identical, Verbos e and Adverbs (action des criptors ) s uch as Quickly, Slowly, Repeatedly, Precis ely, Randomly. Good for creating random s cenarios .

Demings
Cycle Plan Do Check Act 'This cheat sheet includes ideas from Elisabeth Hendrickson, James Lyndsay, and Dale Emery' 'www.qualitytree.com Copyright 2006 Quality Tree Software, Inc.' _______________________________________________________________________________________________________

Heuristic test strategy model


http://www.s atis fice.com/tools /s atis fice-ts m-4p.pdf

General Test Techniques


A test technique is a way of creating tests. There are many interesting techniques. The list includes nine general techniques. By general technique I mean that the technique is simple and universal enough to apply to a wide variety of contexts. Many specific techniques are based on one or more of these nine. And an endless variety of specific test techniques can be constructed by combining one or more general techniques with coverage ideas from the other lists in the Heuristic Test Strategy Model.

Function Testing
Tes t what it can do 1. 2. 3. 4. Identify things that the product can do (functions and s ubfunctions ). Determine how youd know if a function was capable of working. Tes t each function, one at a time. See that each function does what its s uppos ed to do and not whatit is nt s uppos ed to do.

Domain Testing
Divide and conquer the data 1. 2. 3. 4. Look for any data proces s ed by the product. Look at outputs as well as inputs . Decide which particular data to tes t with. Cons ider things like boundary values , typical values , convenient values , invalid values , or bes t repres entatives . Cons ider combinations of data worth tes ting together.

Stress Testing
Overwhelm the product 1. Look for s ub-s ys tems and functions that are vulnerable to being overloaded or broken in the pres ence of challenging data or cons trained res ources . 2. Identify data and res ources related to thos e s ub-s ys tems and functions . 3. Select or generate challenging data, or res ource cons traint conditions to tes t with: e.g., large or complex data s tructures , 4. high loads , long tes t runs , many tes t cas es , low memory conditions .

Flow Testing
Do one thing after another 1. Define tes t procedures or high level cas es that incorporate multiple activities connected end-to-end. 2. Dont res et the s ys tem between tes ts . 3. Vary timing and s equencing, and try parallel threads .

Scenario Testing
Tes t to a compelling s tory 1. Begin by thinking about everything going on around the product. 2. Des ign tes ts that involve meaningful and complex interactions with the product. 3. A good s cenario tes t is a compelling s tory of how s omeone who matters might do s omething that matters with the product.

Claims Testing
Verify every claim 1. 2. 3. 4. Identify reference materials that include claims about the product(implicit or explicit). Analyze individual claims , and clarify vague claims . Verify that each claim about the product is true. If youre tes ting from an explicit s pecification, expect it and the product to be brought into alignment.

User Testing
Involve the us ers 1. 2. 3. 4. 5. Identify categories and roles of us ers . Determine what each category of us er will do (us e cas es ), how they will do it, and what they value. Get real us er data, or bring real us ers in to tes t. Otherwis e, s ys tematically s imulate a us er (be carefulits eas y to think youre like a us er even when youre not). Powerful us er tes ting is that which involves a variety of us ers and us er roles , not jus t one.

Risk Testing
Imagine a problem, then look for it. 1. 2. 3. 4. 5. What kinds of problems could the product have? Which kinds matter mos t? Focus on thos e. How would you detect them if they were there? Make a lis t of interes ting problems and des ign tes ts s pecifically to reveal them. It may help to cons ult experts , des ign documentation, pas t bug reports , or apply ris k heuris tics .

Automatic Testing
Run a million different tes ts

1. Look for opportunities to automatically generate a lot of tes ts . 2. Develop an automated, high s peed evaluation mechanis m. 3. Write a program to generate, execute, and evaluate the tes ts

Scenario Testing
Tes t to a compelling s tory 1. Begin by thinking about everything going on around the product. 2. Des ign tes ts that involve meaningful and complex interactions with the product. 3. A good s cenario tes t is a compelling s tory of how s omeone who matters might do s omething that matters with the product.

Project Environment
''Creating and executing tests is the heart of the test project. However, there are many factors in the project environment that are critical to your decision about what particular tests to create. In each category, below, consider how that factor may help or hinder your test design process. Try to exploit every resource.''

Customers. Anyone who is a client of the test project.


Do you know who your cus tomers are? Whos e opinions matter? Who benefits or s uffers from the work you do? Do you have contact and communication with your cus tomers ? Maybe they can help you tes t. Maybe your cus tomers have s trong ideas about what tes ts you s hould create and run. Maybe they have conflicting expectations . You may have to help identify and res olve thos e.

Information. Information about the product or project that is needed for testing.
Are there any engineering documents available? Us er manuals ? Web-bas ed materials ? Does this product have a his tory? Old problems that were fixed or deferred? Pattern of cus tomer complaints ? Do you need to familiarize yours elf with the product more, before you will know how to tes t it? Is your information current? How are you appris ed of new or changing information? Is there any complex or challenging part of the product about which there s eems s trangely little information?

Developer Relations. How you get along with the programmers.


Hubris : Does the development team s eem overconfident about any as pect of the product? Defens ivenes s : Is there any part of the product the developers s eem s trangely oppos ed to having tes ted? Rapport: Have you developed a friendly working relations hip with the programmers ? Feedback loop: Can you communicate quickly, on demand, with the programmers ? Feedback: What do the developers think of your tes t s trategy?

Test Team. Anyone who will perform or support testing.


Do you know who will be tes ting? Are there people not on the tes t team that might be able to help? People whove tes ted s imilar products before and might have advice? Writers ? Us ers ? Programmers ? Do you have enough people with the right s kills to fulfill a reas onable tes t s trategy? Are there particular tes t techniques that the team has s pecial s kill or motivation to perform? Is any training needed? Is any available?

Equipment & Tools. Hardware, software, or documents required to administer testing.


Hardware: Do we have all the equipment you need to execute the tes ts ? Is it s et up and ready to go? Automation: Are any tes t automation tools needed? Are they available? Probes : Are any tools needed to aid in the obs ervation of the product under tes t?

Matrices & Checklis ts : Are any documents needed to track or record the progres s of tes ting?

Schedule. The sequence, duration, and synchronization of project events.


Tes t Des ign: How much time do you have? Are there tes ts better to create later than s ooner? Tes t Execution: When will tes ts be executed? Are s ome tes ts executed repeatedly, s ay, for regres s ion purpos es ? Development: When will builds be available for tes ting, features added, code frozen, etc.? Documentation: When will the us er documentation be available for review?

Test Items. The product to be tested.


Scope: What parts of the product are and are not within the s cope of your tes ting res pons ibility? Availability: Do you have the product to tes t? Volatility: Is the product cons tantly changing? What will be the need for retes ting? New Stuff: What has recently been changed or added in the product? Tes tability: Is the product functional and reliable enough that you can effectively tes t it? Future Releas es : What part of your tes ts , if any, mus t be des igned to apply to future releas es of the product?

Deliverables. The observable products of the test project.


Content: What s ort of reports will you have to make? Will you s hare your working notes , or jus t the end res ults ? Purpos e: Are your deliverables provided as part of the product? Does anyone els e have to run your tes ts ? Standards : Is there a particular tes t documentation s tandard youre s uppos ed to follow? Media: How will you record and communicate your reports ?

Product Elements
''Ultimately a product is an experience or solution provided to a customer. Products have many dimensions. So, to test well, we must examine those dimensions. Each category, listed below, represents an important and unique aspect of a product. Testers who focus on only a few of these are likely to miss important bugs.''

Structure. Everything that comprises the physical product.


Code: the code s tructures that compris e the product, from executables to individual routines . Interfaces : points of connection and communication between s ub-s ys tems . Hardware: any hardware component that is integral to the product. Non-executable files : any files other than multimedia or programs , like text files , s ample data, or help files . Collateral: anything beyond s oftware and hardware that is als o part of the product, s uch as paper documents , web links and content, packaging, licens e agreements , etc..

Functions. Everything that the product does.


Us er Interface: any functions that mediate the exchange of data with the us er (e.g. navigation, dis play, data entry). Sys tem Interface: any functions that exchange data with s omething other than the us er, s uch as with other programs , hard dis k,network, printer, etc. Application: any function that defines or dis tinguis hes the product or fulfills core requirements . Calculation: any arithmetic function or arithmetic operations embedded in other functions . Time-related: time-out s ettings ; daily or month-end reports ; nightly batch jobs ; time zones ; bus ines s holidays ; interes t calculations ; terms and warranty periods ; chronograph functions . Trans formations : functions that modify or trans form s omething (e.g. s etting fonts , ins erting clip art, withdrawing money from account). Startup/Shutdown: each method and interface for invocation and initialization as well as exiting the product. Multimedia: s ounds , bitmaps , videos , or any graphical dis play embedded in the product. Error Handling: any functions that detect and recover from errors , including all error mes s ages . Interactions : any interactions or interfaces between functions within the product. Tes tability: any functions provided to help tes t the product, s uch as diagnos tics , log files , as s erts , tes t menus , etc.

Data. Everything that the product processes.


Input: any data that is proces s ed by the product. Output: any data that res ults from proces s ing by the product. Pres et: any data that is s upplied as part of the product, or otherwis e built into it, s uch as prefabricated databas es , defaultvalues , etc. Pers is tent: any data that is s tored internally and expected to pers is t over multiple operations . This includes modes or s tates of the product, s uch as options s ettings , view modes , contents of documents , etc. Sequences : any ordering or permutation of data, e.g. word order, s orted vs . uns orted data, order of tes ts . Big and little: variations in the s ize and aggregation of data. Nois e: any data or s tate that is invalid, corrupted, or produced in an uncontrolled or incorrect fas hion. Lifecycle: trans formations over the lifetime of a data entity as it is created, acces s ed, modified, and deleted.

Platform. Everything on which the product depends (and that is outside your project).
External Hardware: hardware components and configurations that are not part of the s hipping product, but are required (or optional) in order for the product to work: CPU's , memory, keyboards , peripheral boards , etc. External Software: s oftware components and configurations that are not a part of the s hipping product, but are required (or optional) in order for the product to work: operating s ys tems , concurrently executing applications , drivers , fonts , etc. Internal Components : libraries and other components that are embedded in your product but are produced outs ide your project. Since you dont control them, you mus t determine what to do in cas e they fail.

Operations. How the product will be used.


Us ers : the attributes of the various kinds of us ers . Environment: the phys ical environment in which the product operates , including s uch elements as nois e, light, and dis tractions . Common Us e: patterns and s equences of input that the product will typically encounter. This varies by us er. Dis favored Us e: patterns of input produced by ignorant, mis taken, careles s or malicious us e. Extreme Us e: challenging patterns and s equences of input that are cons is tent with the intended us e of the product.

Time. Any relationship between the product and time.


Input/Output: when input is provided, when output created, and any timing relations hips (delays , intervals , etc.) among them. Fas t/Slow: tes ting with fas t or s low input; fas tes t and s lowes t; combinations of fas t and s low. Changing Rates : s peeding up and s lowing down (s pikes , burs ts , hangs , bottlenecks , interruptions ). Concurrency: more than one thing happening at once (multi-us er, time-s haring, threads , and s emaphores , s hared data).

Quality Criteria Categories


''A quality criterion is some requirement that defines what the product should be. By looking thinking about different kinds of criteria, you will be better able to plan tests that discover important problems fast. Each of the items on this list can be thought of as a potential risk area. For each item below, determine if it is important to your project, then think how you would recognize if the product worked well or poorly in that regard.''

Operational Criteria
Capability: Can it perform the required functions ? Reliability: Will it work well and res is t failure in all required s ituations ? Error handling: the product res is ts failure in the cas e of errors , is graceful when it fails , and recovers readily. Data Integrity: the data in the s ys tem is protected from los s or corruption. Safety: the product will not fail in s uch a way as to harm life or property. Usability: How eas y is it for a real us er to us e the product?

Learnability: the operation of the product can be rapidly mas tered by the intended us er. Operability: the product can be operated with minimum effort and fus s . Accessibility: the product meets relevant acces s ibility s tandards and works with O/S acces s ibility features . Security: How well is the product protected agains t unauthorized us e or intrus ion? Authentication: the ways in which the s ys tem verifies that a us er is who s he s ays s he is . Authorization: the rights that are granted to authenticated us ers at varying privilege levels . Privacy: the ways in which cus tomer or employee data is protected from unauthorized people. Security holes: the ways in which the s ys tem cannot enforce s ecurity (e.g. s ocial engineering vulnerabilities ) Scalability: How well does the deployment of the product s cale up or down? Performance: How s peedy and res pons ive is it? Installability: How eas ily can it be ins talled onto its target platform(s )? System requirements: Does the product recognize if s ome neces s ary component is mis s ing or ins ufficient? Configuration: What parts of the s ys tem are affected by ins tallation? Where are files and res ources s tored? Uninstallation: When the product is unins talled, is it removed cleanly? Upgrades: Can new modules or vers ions be added eas ily? Do they res pect the exis ting configuration? Compatibility: How well does it work with external components & configurations ? Application Compatibility: the product works in conjunction with other s oftware products . Operating System Compatibility: the product works with a particular operating s ys tem. Hardware Compatibility: the product works with particular hardware components and configurations . Backward Compatibility: the products works with earlier vers ions of its elf. Resource Usage: the product does nt unneces s arily hog memory, s torage, or other s ys tem res ources .

Development Criteria
Supportability: How economical will it be to provide s upport to us ers of the product? Testability: How effectively can the product be tes ted? Maintainability: How economical is it to build, fix or enhance the product? Portability: How economical will it be to port or reus e the technology els ewhere? Localizability: How economical will it be to adapt the product for other places ? Regulations: Are there different regulatory or reporting requirements over s tate or national borders ? Language: Can the product adapt eas ily to longer mes s ages , right-to-left, or ideogrammatic s cript? Money: Mus t the product be able to s upport multiple currencies ? Currency exchange? Social or cultural differences: Might the cus tomer find cultural references confus ing or ins ulting? ''Designed by James Bach, james@satisfice.com, www.satisfice.com, Copyright 1996-2006, Satisfice, Inc.'' _______________________________________________________________________________________________________

Michael Kelly Tours


FCC CUTS VIDS
http://www.tes tingreflections .com/node/view/2823 Feature tour: Move through the application and get familiar with all the controls and features you come acros s . Complexity tour: Find the five mos t complex things about the application. Claims tour: Find all the information in the product that tells you what the product does . Configuration tour: Attempt to find all the ways you can change s ettings in the product in a way that the application retains thos e s ettings . User tour: Imagine five us ers for the product and the information they would want from the product or the major features they would be interes ted in. Testability tour: Find all the features you can us e as tes tability features and/or identify tools you have available that you can us e to help in your tes ting. Scenario tour: Imagine five realis tic s cenarios for how the us ers identified in the us er tour would us e this product. Variability tour: Look for things you can change in the application - and then you try to change them.

Interopeability tour: What does this application interact with? Data tour: Identify the major data elements of the application. Structure tour: Find everything you can about what compris es the phys ical product (code, interfaces , hardware, files , etc...). Based upon an idea by Michael Kelly http://www.michaeldkelly.com _______________________________________________________________________________________________________

HICCUPPS
Consistency (this agrees with that) - an important theme in oracles
History: If a product is incons is tent with previous vers ions of its elf, we s us pect that there might be a problem. Image: If a product is incons is tent with an image that the company wants to project, we s us pect a problem. Comparable Products: When a product s eems incons is tent with a comparable product, we s us pect that there might be a problem. Claims: When a product is incons is tent with claims that important people make about it, we s us pect a problem. User Expectations: When a product is incons is tent with expectations that a reas onable us er might have, we s us pect a problem. Purpose: When a product is incons is tent with its des igners explicit or implicit purpos es , we s us pect a problem. Product: When a product is incons is tent internally as when it contradicts its elf we s us pect a problem. Statutes and Standards: When a product is incons is tent with laws or widelyaccepted or relevant s tandards , we s us pect a problem. Orignal idea from James Bach and Michael Bolton http://www.develops ens e.com/articles /2005-01-Tes tingWithoutAMap.pdf

Anda mungkin juga menyukai