Quality Assurance Testing - helping to ensure the quality of systems

How do we know the software works? Claim - Testing

Software Development Today - This is the structure
Programmer -> Manager (Decision Maker) <- Tester
Test Iteration 1
Programmer - I'm done
Tester - This crashes on startup
Manager - Ok, calm down. We'll slip the schedule a little. Try again
Test Iteration 2
Programmer - I'm done
Tester - There are white screens all over the place.
Manager - Now remember, we're all in this together. Try again.
Test Iteration 3
Programmer - I'm done
Tester - It does the wrong thing in half the test cases.
Manager - Ok, calm down. We'll slip the schedule a little. Try again
Test Iteration 4
Programmer - I'm done
Tester - It still fails some of the test cases we agreed upon.
Manager - Ok, Try again, but please hurry up!
Test Iteration 5
Programmer - I'm done
Tester - Yep, it's done
Manager - Oops, the world has changed. Here the new spec.

Software Development Today - Why do we have this structure? Key assumptions

1. Independent Development and Testing
2. Specifications must be explicit
3. Specifications must change
4. Resources are finite so Organizations need decision makers - headship principle
1. Why have Independent Testing?
Testing is basic to all engineering - design of drugs, bridges, planes, software? Why?
Because our ability to predict behavior is imperfect
We need to check our work because we will make mistakes
2. Why do we need explicit Specifications?
- Software involves multiple people (at least one programmer and one user)
- Usually has multiple programmers and multiple testers
- Any Team effort requires mutual understanding of the goal - a specification
Otherwise, team members will have different goals in mind
3. Why must Specifications change?
- Most software systems are truly novel, not one really know what needs to be built
- What people what changes as they use the software
- The hardware and software components you work with changes
Software specifications are usually in prose, imprecise and out-of-date, so they need updating
4. Why do Organizations need decision makers to handle Finite Resources ?
- Time and Labor are limited
- Not all wishes can be accommodated, so organizations make tradeoffs.
- Most use dollars to express relative cost and competing goals
Manager says" We have 2 months, 5 programmers and 2 testers, Here is a prioritized list of features. A feature
is finished when it passes all the test for that feature. A programmer does not move on to a new feature until all
higher priority features are finished or assigned to another programmer. We start now and ship whatever features
are finished 60 days from now."

Quality Assurance Testing - helping to ensure the quality of systems

Software Development Today 1. Many proposals for improving software quality
2. But everyone testers. 50% of the cost of software development is testing
3. Testing is important, let's take a closer look at the practice.
Purpose of Testing
- Find Bugs - important ones
- Clarify the specification. Good Testers - clarify the specification, this is hard work. There is no hope tools
will automate clarification, this will stay hard work.
- Help determine what risks are involved in releasing code
Manual Testing
- Test cases are a list of instructions - a test script
- Someone manually executes the script, doing each action, step by step
1. Type URL
2. Enter username and password
3. click OK
4. Manually record the results - check yes on Software Life Cycle System
- Low Tech, simple to implement, very widespread. Why? Because some test can't be automated (usability
testing) and some test could be automated, but are not worth the costs of implementation.
Naive Automated Testing - Idea: Record manual test - playback on demand.
- This doesn't work as well as expected.
Test recording is usually fragile (breaks in environment if anything changes)
Automation Tools cannot generalize. They record exactly what happens.
- A hidden strength of manual testing - Since people are doing the testing, they can adapt test to slightly
modified situations. When code changes, test break (change a name of a dialog box)
- Maintain automated tests is expensive - cost proportional to the number of tests
Improved Automated Testing
- Recording and playback is too low level, every test case has name of dialog box
- Need abstract test to replace dialog box with string variable name x. X is maintained in just one place, when
it changes, only x need to be updated. This is structured programming, just as hard as any system design.
Cucumber and other specification to test routine implement this with behavior driven design
This is state of the art. But does not necessary validate the entire testing surface, but make sure software
behavior is invariant under program modification.
The people with whom Ive worked that promote BDD or ATDD using tools like Cucumber (or Specflow for .NET) all generally agree its a
design tool and NOT a quality tool. Converting a requirement or a user story into a testable check forces all involved to analyze whats
being requested. When it works well, a Cucumber test is a very informative metric letting the developer know when theyve coded just
enough to stop. It is not, however, a tool for verifying quality or even the fitness of the requirement under test anymore than a unit test would
be a replacement for an integration or performance test. BDD represents an improvement in product development. But it not testing or
QA. It is a way for product managers (stakeholders),development and even QA/Test to come together to question what would satisfy the
stakeholder of the application. It also brings together these groups at the inception of the product something that, many times, never happens.

What is a Tester's Job?

- Find important Bugs (only part of this is subject to automation)
- Clarify the spec - no automation tool can do this, people must
- Communicate well with developers
Software Development Today - How do we know the software works?
1. We need a specification - most use user stories.
2. Compare the code to the specification
3. Testing is important but expensive, to find bugs and clear up the specification so the product is useable by
others.

Quality Assurance Testing - helping to ensure the quality of systems

How can Testers Improve?
1. Know your application - if you test without knowledge of the requirement, you will not know if a program is
functioning as designed. You will not be able to tell if required functionality is missing. So reading and
getting a clear understanding of the requirements is a must before testing. After than go through all the
features of the application in a systematic way.
2. Think like an end user - Don't think like a technical guy. Think like a customers or the end user. Test your
application as an end user would. Think how an end user will be using your application and what technical
background he has. Personas help with this.
3. Don't make testing a repetitive task - Tester can sometimes get easily bored doing the same things again
and again. The testing job involves repetitive things, so take care to distribute the work among team
members. Systematically spread out the testing work within a team, this way the team will get
knowledge of the whole product. This will help to find more bugs.
4. Take every test case as an opportunity to learn - Without a tester notebook for can't review what you've
done and learn from it. You cannot trace what you have tested and what problems occurred. What can you
keep in a Tester's Notebook? Testing heuristics and test ideas, collections of observations while testing, logs
of testing sessions, meeting notes, one page presentations, mind maps and diagrams, action ideas,
reminders and random notes. Some use index cards for bug reports to keep a easily accessible record of
bugs they have filed.
What should Testers Learn?
1. Learn to write formal test cases/ test plans - a test case writing guide help keep all test cases in a format
that is easy for all Testers to follow. It list the steps for testing and checks for success to allow test cases to
be placed in a Software Life Cycle Tool.
2. Learn session-based exploratory testing - these are strategies for testing that let you systematically work
your way through the application. In session-based exploratory testing, a session is the basic testing work
unit . What we call a session is an uninterrupted block of reviewable, chartered test effort. By chartered, we
mean that each session is associated with a missionwhat we are testing or what problems we are looking fo
3. Learn new technologies - having a solid foundation in technology will bring new testing ideas. This will help
you better understand what the testing surface is.