www.TechWell.com
TD Maxim is an amalgam of Cognizants Test Data Management service and Informaticas ILM TDM product For more information, contact testing.services@cognizant.com
adc-bsc-west.techwell.com
BETTER SOFTWARE
MARCH/APRIL 2013
www.TechWell.com
Looking for cleaner ways to manage your build and deploy process? Learn more about our Dynamic DevOps Suite at
www.Openmakesoftware.com/DevOps-Answers
Test Studio
Test the reliability of your rich, interactive JavaScript apps with just a few clicks. Benefit from built-in translators for the new HTML5 controls, crossbrowser support, JavaScript event handling, and codeless test automation of multimedia elements.
www.telerik.com/html5-testing
20
C O N TENTS
features
12
COVER STORY
To build and sustain credibility, good project managers focus on managing expectations and leave perception management to magicians. Explore the difference and find out why. by Payson Hall
16
12
20
24
24
in every issue
Mark Your Calendar Editor's Note Contributors From One Expert to Another
4 5 6
9 Techwell Spotlight 10
Product Announcements 28 FAQ 31 Ad Index 33
columns
7 TECHNICALLY SPEAKING
IT'S ALL A MATTER OF PERSPECTIVE by Johanna Rothman Everyone has a unique perspective on problems at work. Help your problems make it to the top of the queue by expressing them in terms of business value.
Better Software magazineThe companion to TechWell.com brings you the hands-on, knowledge-building information you need to run smarter projects and deliver better products that win in the marketplace and positively affect the bottom line. Subscribe today at www.BetterSoftware.com or call 800.450.7854.
32
CAREER DEVELOPMENT
A MAJOR AWARD by Lisa Crispin You may remember the major award from the film A Christmas Story as that gaudy leg lamp. But, for Ralphie's Old Man, its indescribably beautiful. Sometimes, the meaning of an award is more important than the award itself.
3
SQE TRAINING
training weeks
www.sqetraining.com/trainingweek Testing Training Weeks March 1822, 2013 Boston, MA April 2226, 2013 San Diego, CA June 1014, 2013 Chicago, IL Requirements Training Weeks September 1620, 2013 Boston, MA Agile Software Development Training Weeks June 24, 2013 Las Vegas, NV
Publisher Software Quality Engineering, Inc. President/CEO Wayne Middleton Vice President of Communications Heather Buckman Publications Manager Heather Shanholtzer Editorial Better Software Editor Joseph McAllister Online Editors Jonathan Vanian Noel Wurst Community Manager David DeWald Production Coordinator Cheryl M. Burke Design Creative Director Catherine J. Clinger Advertising Sales Consultants Daryll Paiva Kim Trott
conferences
STARCANADA starcanada.techwell.com April 711, 2013 Delta Chelsea Toronto, ON STAREAST stareast.techwell.com April 28May 3, 2013 Rosen Shingle Creek Orlando, FL Better Software Conference West bsc-west.techwell.com June 27, 2013 Caesars Palace Las Vegas, NV Agile Development Conference West adc-west.techwell.com June 27, 2013 Caesars Palace Las Vegas, NV
4
BETTER SOFTWARE MARCH/APRIL 2013
STARWEST starwest.techwell.com September 29October 4, 2013 Disneyland Hotel Anaheim, CA Better Software Conference East bsc-east.techwell.com November 1015, 2013 Sheraton Boston Hotel Boston, MA Agile Development Conference East adc-east.techwell.com November 1015, 2013 Sheraton Boston Hotel Boston, MA
CONTACT US Editors: editors@bettersoftware.com Subscriber Services: info@bettersoftware.com Phone: 904.278.0524, 888.268.8770 Fax: 904.278.4380 Address: Better Software magazine Software Quality Engineering, Inc. 340 Corporate Way, Suite 300 Orange Park, FL 32073
www.TechWell.com
Editors Note
A Kind
of
Magic
Its called the art of persuasion, not the blunt instrument of persuasion. However, if youve ever watched kids TV programming after a school day or on a typical Saturday morning, you know that commercials geared toward kids can be noisy, flashy, and seemingly the furthest thing from art. You need a bowl full of totally awesome Sugar Flakes right now! But, beneath the veneer, even the most ostentatious commercials are working a certain kind of subtle magic. For instance, food artists are in charge of making sure that their products look perfect on camera. They manage viewers perceptions through trickery, such as using white school glue in place of milk in a cereal bowl, because the glue keeps the cereal from getting soggy on set during filming and, in the end, the television audience cant tell the difference. Of course, if youve ever eaten a bowl of soggy cereal, you know that perception management only lasts so long. In this issues cover story, Payson Hall addresses the difference between managing perception and managing expectations. Keeping expectations in line is a valuable workplace skill. Pulling the wool over peoples eyes can be useful in the workplace, too, but only if you work as a professional magician. Otherwise, if youre trying to manage the perceptions of your employees, your colleagues, or even your own managers, youd better start honing your magic tricks in your spare time, because you likely wont be working as a manager for very long. In their articles, Matthew Heusser and Brooke Bowie discuss the importance of adaptation in testing, with Matthew focusing on exploration and Brooke on precision. Sunita Purushottam and Vaibhav Bhatia look to the future of green IT in their article on using metrics to show that sustainability is more than just a nice ideaits also a quantifiable asset for your company. Plus, Johanna Rothman teaches us about working our way through problems by keeping an open mind and looking to others perspectives, while Lisa Crispin relates her personal experience of winning a major award that appears to be just a big rock from someones garden but ultimately turns out to be much more. We hope youll enjoy this issues articles, look beneath the surface, and maybe even take away a new perspective or two. Keep your eyes peeled for ways in which people might try to manage your perceptionsespecially if theyre not magicians by trade. Yours abracadabrally,
Contributors
Vaibhav Bhatia is a certified data center associate and green IT professional with nine years of industry experience, most of it in the data center space. He has managed operations of a data center in Bangalore and large-scale data center projects and has worked on several data-center-optimizing and green IT initiatives. Vaibhav has published and presented several papers at various forums. He is currently a senior consultant with the sustainability practice at Infosys Limited.
Brooke Bowie has more than eighteen years of quality assurance and testing experience across various industries. She specializes in transforming and creating testing organizations that fit the unspoken needs of the company culture by creating powerhouse teams that have customized testing and quality toolkits. Brooke is available for corporate coaching and training and independent consulting services. You can contact her at Brooke@testimprovements.com or testimprovements.com.
Lisa Crispin is the coauthor (with Janet Gregory) of Agile Testing: A Practical Guide for Testers and Agile Teams, coauthor (with Tip House) of Extreme Testing, and a contributor to Beautiful Testing. She has worked as a tester on agile teams for the past ten years and enjoys sharing her experiences via writing, presenting, teaching, and participating in agile testing communities around the world. Lisa was named one of the 13 Women of Influence in testing by Software Test & Performance magazine. For more about Lisas work, visit lisacrispin.com. Janet Gregory, coauthor (with Lisa Crispin) of Agile Testing: A Practical Guide for Testers and Agile Teams, specializes in helping teams build quality systems. As tester or coach, she has helped introduce agile development practices into companies and has successfully transitioned several traditional test teams into the agile world. Janet is a frequent speaker at agile and testing software conferences in North America, including the STAR conferences.
Payson Hall is a consulting project manager for Catalysis Group Inc. in Sacramento, Californiaand a magician. Payson consults on project management issues and teaches project management. Email Payson at payson@catalysisgroup.com, and follow him on Twitter at @paysonhall.
Matthew Heusser is a consulting software tester and software process naturalist, who has spent his entire adult life developing, testing, and managing software projects. Matthew blogs at Creative Chaos, is a contributing editor to Software Test & Quality Assurance magazine, and is on the board of directors of the Association for Software Testing. Matthew recently served as lead editor for How to Reduce the Cost of Software Testing. Follow Matthew on Twitter at @mheusser or email him at matt@xndev.com.
Sunita Purushottam is a principal consultant in the sustainability unit at Infosys. She has more than fourteen years of experience as an environmental and sustainability consultant specializing in carbon, sustainability, and supply chain strategies and e-waste management. Sunita is a fellow of the Royal Meteorological Society (UK) with keen understanding of climate change. She has a post graduate degree in physics (with specialization in electronics) and a PhD in meteorology and air pollution models and impacts on humans.
Johanna Rothman helps organizational leaders see problems and risks in their product development, recognize potential gotchas, seize opportunities, and remove impediments. She is the technical editor for Agile Connection and the author of many books, including Behind Closed Doors: Secrets of Great Management and Hiring Geeks That Fit. Johanna is working on a book about agile program management. She writes columns for Stickyminds.com and Gantthead.com and blogs at jrothman.com and createadaptablelife.com. Ed Weller, the principal at Integrated Productivity Solutions, is an SEI-Certified SCAMPI high-maturity appraiser for CMMI appraisals with nearly forty years of experience in hardware and software engineering. This extensive practical background in development has resulted in a no-nonsense, practical approach to process improvement. Integrated Productivity Solutions is a consulting firm that focuses on providing solutions to companies seeking to improve their development productivity. You can reach Ed at efweller@aol.com.
6
BETTER SOFTWARE MARCH/APRIL 2013
www.TechWell.com
Technically Speaking
Mapping iT ouT
The Leading Conference on
S o f t wa r e t e S t i n g a n a lyS i S & r e v i e w
Matt Barcomb
Years in Industry: 14 Email: mgbarcomb@gmail.com
Applying patterns allows us to take actions from principles, instead of mechanically implementing some checklist.
At a high level, when I think of whole-team quality, I think of a When a group of individuals shares the same knowledge of certain patterns, it provides a common language for understanding and working together in some domain. Teams can visualize all sorts of things, like workflow, The team should understand that quality has various aspects and applications depending on context and that quality is more than just testing the software. team norms, product goals, quality initiatives, etc. I find that visualization helps teams come to a shared understanding or build consensus more quickly. cross-functional development team where all members feel responsible for quality and continuously work to understand and improve it.
Team members should work to understand how their particular specialties can improve quality efforts, especially when collaborating with other team members' specialties.
well.tc/FOETA15-2
TechWell Spotlight
Featuring fresh news and insightful stories about topics that are important to you, TechWell.com is the place to go for what is happening in the software industry today. TechWells passionate industry professionals curate new stories every week day to keep you up to date on the latest in development, testing, business analysis, project management, agile, DevOps, and more. The following is a sample of some of the great content youll find. Visit TechWell.com for the full stories and more!
Hollywood Hackers vs. Real-Life Hackers
by Jonathan Vanian Hollywood seems to enjoy portraying hackers as stereotypes akin to troubled geniuses or bands of attractive rogues subverting some sort of powerful corporate or political (bonus points if its a hybrid) dictatorship that gets a kick out of limiting the personal freedoms of beautiful people and the peons (Hollywood extras and commercial actors) that cheer for them. With this in mind, its fascinating when prominent hackers adopt a larger-than-life personality that seems the stuff of action movies. In this case, Im thinking of the infamous Julian Assange, although maybe a naturally captivating character like him is always destined to play this role. Continue reading at well.tc/152-TW-Hackers.
10
BETTER SOFTWARE
MARCH/APRIL 2013
www.TechWell.com
TechWell Spotlight
certified ScrumMaster, certified product owners, certified developers, etc. Continue reading at well.tc/152-TW-Scrum.
about a revolution in the office environment. Businesses are being forced to implement policies (e.g., usability standards and bring your own device) that enhance the employee experience. Yet, usability is only in its infancy in terms of the importance it will play in future software systems. As the world around us becomes more complex, our ability to solve problems without software assistance becomes more limited. While an explosion of specializations in various fields might provide us with answers, this tactic can only take us so far. Continue reading at well.tc/152-TW-Usability.
Each curator is responsible for submitting a minimum of five to ten stories a month. Stories should run 300-500 words, with 400 words being ideal. Stories are built around and should link to articles, videos, blog posts, or other online contentboth from our TechWell Community sites and anywhere in the Internetthat the curator considers interesting and applicable to our audience. You should expect to spend one to two hours developing and writing a story. Because audience engagement is key to the success of a curated site, we ask curators to respond to reader comments and questions.
Stories you write will feature your byline with a link to a profile page containing your photo, bio, and links to your blog , Twitter, LinkedIn, etc. Readers will come to know you, your stories, and your personality. Thought leaders are born this way. TechWell curators receive $100 per story published, up to a total of ten stories ($1,000) per month. In addition, active TechWell curators receive free Wednesday-Thursday conference passes to any SQE conference and half price on pre- and post-conference event sessions (tutorials + summit). Curators submit stories to the TechWell editors, who check them for grammar, style, and punctuation, and then publish them to the siteusually within two business days.
To apply for a TechWell curator position, please contact Heather Shanholtzer at hshanholtzer@sqe.com with the following information: Name Company affiliation Interest area(s) Approximate stories per month you are available to curate
340 Corporate Way | Suite 300 | Orange Park, FL 32073 | 904.278.0524 | www.TechWell.com
11
THINKSTOCKPHOTOS.COM
12
BETTER SOFTWARE
MARCH/APRIL 2013
www.TechWell.com
recently met with a project manager as part of a project review. I think the PM has a clue, but I left feeling uneasy. (For those unfamiliar with external reviewers coming in and conducting what is essentially a real-time project audit, having a lead reviewer feel uneasy is a bad thing.) As I reflected on why, I thought it might serve as a cautionary tale for others, so Im sharing. The heart of the issue is the difference between managing expectations and managing perceptions.
Expectations
Expectations are what people believe they will get out of a project. How much do they believe it will cost? How long do they imagine it will take? What do they think they will get for their investment when the project is complete? A key part of a project managers job is managing the expectations of project sponsors and stakeholders. Before the project begins, when it is merely a gleam in the sponsors eye, everything is possible. This is called the honeymoon phase. Everyone is happy and in love, giving no thought to the hard work ahead. No one knows how much the project will cost, but everyone assumes the cost will be reasonable. No one knows how long it will take, but theyre sure it wont take too long. People usually dont know exactly what they will get for their investment, but they believe that their business problem will be (mostly painlessly) solved by the project. This is the bliss of early project love. The gritty realities that emerge during planning and execution about costs, risks, limitations, competing priorities, resource requirements, and tradeoffs often prove more daunting than originally imagined. One of the principle roles of the project manager is working to assure that these emerging realities do not surprise sponsors and stakeholders. Managing expectations about such things as the uncertainty of new technologies; the variability of early estimates of cost and schedule; and necessary tradeoffs among cost, schedule, functionality, and quality as additional information becomes available is a tough part of the project managers job. This involves keeping people grounded during the honeymoon so the emerging realities arent such a shock. It also involves keeping an open channel for good news, bad news, and changes in the risk profile as the project evolves. Show me a competent project manager who does a good job of managing expectations, and I will show you a good project manager. On the other hand, if you dont manage expectations well, the rest of your performance is often irrelevant. Unpleasantly surprised people are not happy people. Managing expectations honestly, openly, and fairly is an essential skill.
Read those two conclusions carefully. They represent two very different meanings that can be derived from the same data. There is a particular element of perception management that is reasonable, necessary, and helpful: assuring that people understand the context of the data they are receiving. For example, imagine that integration testing in the month of December identifies 2,000 faults with the system being developed. This might sound bad. Helping to manage perceptions in this case might involve saying something like this: 2,000 reported faults in one month sounds like a lot, but I need to remind you that we are talking about a system that has several million lines of code and processes about one hundred unique kinds of transactions with a variety of interface partners. With integration testing just starting in December, we discovered that one of our interface partners had made a slight change to an interface that behaved differently than it had during earlier testing and was not consistent with our agreed-upon specification. That accounted for about 900 of the faults automatically reported by our testing tool. We are working to analyze and categorize the other faults and will have additional information for you next month, but we believe that the initial December numbers were reasonable for a project of this size and complexity and, factoring out the one interface, were actually pretty good. I look forward to providing you with more detailed information about the number, sources, and severity of faults, as well as trends, next month. Perception management of this kind can be valuable and improves communication. However, managing perceptions can be taken beyond simply providing context. When this happens, it can damage credibility.
Perceptions
Perceptions are the meaning we make of the data we observe. Perceptions can be tricky things. An executive overhears one person in the lunchroom saying, The Alpha Project is having trouble, and the meaning the executive might take from that could be one of the following: A) One person in the cafeteria says there is some kind of trouble. B) There is trouble.
13
Sometimes, when a magician appears to give you a choice, it isn't really a choice at all. If it generally appears to be a choice, that is often sufficient.
You might imply from this answer that the project remains on track, but that isnt really what was said, was it? The question about resource issues has not been addressed. Q: Can we please review the risk log and the budget reports this afternoon? A: Great. You may only have time to review one, though. Where would you like to start? Q: Can we start with the risk log? A: Im afraid the risk database is down today. Perhaps we should start with the budget, and we can schedule the risk log for your next visit. Notice that there was about a 50 percent chance that the fact that the risk log wasnt available would not come up. Heres a suggestion: Dont try this. When it works, it works well. When it fails, it often wont go undetected and will be taken for outright dishonesty.
Summary
I think the project manager Im working with is well intentioned but a little green. I intend to work with him to get past what I take as clumsy attempts to manage my perceptions, which I currently attribute to defensiveness rather than malice. Im aware that when Im participating in project reviews, people are sometimes anxious about what I will find and how I will characterize it. What is challenging for me to communicate is that reviewers like me dont expect perfection. Complex projects always have issues. Reviewers look for the project manager to have a handle on where the issues are and a plan to address them. When I feel like someone is trying to manage my perceptions, it is a red flag that drives down credibilityprobably the opposite reaction of what was intended. If you are interested in more information about the perception management methods of con men and magicians, you might find the book The Right Way to Do Wrong by Harry Houdini (yes, that Harry Houdini) to be a fast and amusing read. {end}
payson@catalysisgroup.com
14
BETTER SOFTWARE
MARCH/APRIL 2013
www.TechWell.com
innovative testing
Tune In to
ConferenCe
guide
SundAy sses Begin Multi-day Training Cla MondayTuesday d Full-day Tutorials 31 In-depth Half- an day WednesdayTHurs urrent sessions, 5 Keynotes, 42 Conc g events, the eXPo, networkin ssions, and More se s receptions, Bonu FrIday adership summit Testing & Quality Le ed software lat gu Workshop on re ) sT Testing (Wre
RegisteR by
AND sAVe
Premier
15
THINKSTOCKPHOTOS.COM
16
BETTER SOFTWARE
MARCH/APRIL 2013
www.TechWell.com
"I want to live my life taking the risk all the time that I dont know anything like enough yet. That I havent understood enough. That I cant know enough. That I am always hungrily operating on the margins of a potentially great harvest of future knowledge and wisdom. I wouldnt have it any other way. Christopher Hitchens [1] Inspired by Hitchens, Id like to contrast two ways of looking at software testing. The first is straightforward, predictable, and repeatable, while the second is wild, dangerous, and perhaps a little bit scary. After explaining both, Ill provide some tips and guidelines to help steer testing in an environment that is constantly changing and chaotic.
Example 1: ProcessCorp
ProcessCorp, a large enterprise operating in twenty states, was in the middle of a transition from a waterfall approach to Scrum. For ProcessCorp, the test process was straightforward. Any given story had acceptance criteria. Testers took the acceptance criteria and turned it sideways, creating test cases that operated at the click-type-click-expect level of detail. When the build was ready, the tester would create a test run to record results. In the world of ProcessCorp, bugs came from one place: Test runs did not comply with expected results and could be traced to acceptance criteria. Test automation and training on the application were easy, because anyone could automate the cases and, likewise, anyone could run the tests to get up to speed on the software.
that it is not a real company or that I am making a straw man argument. This is the way a real client recently explained testing to me and, while we may be tempted to mock it, ProcessCorp offers real answers to the questions What is a bug? and Where does a bug come from? AdaptiveCorp does not; it requires us to develop a method that may change over time. Now, Ive painted these as two polar opposites because they represent two different ways of thinking, but the way people actually act is usually not black and white. At ProcessCorp, testers often perform exploratory testing to get results to the programmers fast. They may find half the defects through an exploratory process. Likewise, they are quick to admit that the documentation often fails to cover all combinations of the user interface and that something is happening to help people figure out whether an undocumented behavior is actually a bug or an unplanned, logical consequence.
Example 2: AdaptiveCorp
Another company, roughly the size of ProcessCorp, with something like a half dozen development teams spread over six cities and two continents, AdaptiveCorp was also starting with something like Scrum and adapting it. Instead of viewing specifications as the source of truth, the staff at AdaptiveCorp viewed them as a source of truthas if, at some point, they decided to stop investing time in arguing about what they were building, wrote down the information as best they knew it, and started working. In this model, defects are very different. Instead of coming from the requirements, a bug is, as Michael Bolton has said, anything that bugs someone that matters. [2]
17
having a conversation with a decision maker about what the customer experience should be like. Often, the claims are implied. We may derive customers should never crash the web browser from a vague claim that quality is job number one and may internalize our job as testers as making sure that a web browser crash never happens. For a great deal more on where bugs come from during improvisation, I recommend James Bachs Heuristic Test Strategy Model. [3] Parimala Hariprasads blog post The Power of Mnemonics [4] also outlines many popular collections of these heuristics.
a script that must be completed to call a phase done. Instead, they find value in helping to get a product to market. Without specific direction, they need to find the best way to do that. The bet that AdaptiveCorp makes is that those people, in the moment, know better how to invest their time than someone else who weeks or months ago wrote a script. The bet is that by documenting what to test rather than how, the team will create documentation that is cheaper, less brittle, and possibly more valuable. It is also a bet I would make any day of the week. In a very real way, I have staked my career on it.
matt@xndev.com
References
18
BETTER SOFTWARE
MARCH/APRIL 2013
www.TechWell.com
adc-bsc-west.techwell.com
PMI members can earn PDUs at both events
19
Save up to $400
gRoups of 3+ sAve even moRe!
www.TechWell.com MARCH/APRIL 2013 BETTER SOFTWARE
THINKSTOCKPHOTOS.COM
have always been fascinated with creating methods for efficient delivery, particularly during testing. In the 1990s, I was stretching my theories to the brink and loving the ride. The adoption of evolutionary methods brought about many solutions for better efficiency, including the idea to test smaller, more frequently, and earlier. In todays age of automation and complex integrated infrastructures, we often encounter the unresolved issue of how to get high-value testing within the condensed time-to-market window. Automated frameworks and modularized scripts provide a partial solution, but they are not independently intelligent enough to provide consistently high value or highly efficient testing. To solve this, we need to select tests that require us to examine what is needed to test within each unique increment,
20
BETTER SOFTWARE MARCH/APRIL 2013
cycle, or iteration. Every change, whether done for improvement or remediation, presents an opportunity for the software ecosystem (applications, browsers, web services, and vendor software) to fail. This results in a much greater need on our part to perform high-value testing. High-value testing does not mean that you need to perform all end-to-end testing or run the full suite of tests. This can potentially create a bottleneck and dampen the velocity. To properly perform high-value testing requires a precise and often unique test response for each new change, which entails a medley of testing types, each working in concert to ensure the quality goals. This is a modern-day necessity to fully ensure the end-user experience, the ecosystem stability, and product health.
www.TechWell.com
The goal is for you to create an intelligent testing trove (security tests, functional tests, data accuracy tests, performance tests, usability tests, interoperability tests, etc.) that can be succinctly arranged and rearranged across varying sets of browsers, platforms, and hardware. This variety of intelligent tests is scalable to varying business goals and marries the quality categories to the unique business requirements to create test goals. The adapting tests are always targeted at the most relevant business and quality goals, which yield the most important results for the team to use for decision making.
changes), of which most were small, front-end UI changes to multiple web applications. In this case, the test team executed all the tests and performed regression, and the sprint was given the green light. This was followed by an uneventful implementation. To our chagrin, on the day after implementation we received a call from an executive informing us that one of the two user profiles was redirecting to a broken page after login, and the other had severe performance issues, taking over three minutes to authenticate the user. This particular login page and process had not been changed by the recent implementation, and our previous regression testing had only targeted the login process of one user profile using test data. We did not test the less-common user profile (which resulted in the broken page).
21
When constructing the initial user stories and tests, we knew that the login process was a critical path and should be included in regression. But, we had designed stories for general login with test data since it seemed stable in the test environment. As we learned, however, 70 percent of the generated revenue was connected to this line of business, and the production environment was conclusively different, which could render some of our test results useless. In the retrospective meeting, it was clear to me that a few key areas were being underserved, resulting in a growing problem. For one thing, the testing value was beneath the quality need, meaning that the best test results did not accurately predict the system behavior or confidently indicate that the business goals would be met in production. After I digested this premise, the exact root cause of the issues was less relevant, because we didnt have a process that would allow us to detect errors left or right of the established regression. The established regression comprised previously created user stories and tests. The regression suite was enormously inefficient and took two to three days to execute. The thought of having such an ineffective, time-expensive process boggled my mind. The production problem was revealed to be a service breakdown between the content management system (production instance only) and the middleware, which would have never been caught due to the established coverage gap and lack of testing in the production environment. From this, I concluded that there was a potential of ongoing defect migrations into production as well as unknown issues residing in the production environment that were both just waiting to be encountered by a customer. We could have used adaptive testing during production, as this method would have created a focus on quality goalsin this case, critical process flows, usability, and content.
This would have resulted in higher test efficiency and better test precision based on the goals of data accuracy, security, usability, and content consistency. Both of these experiences resulted from misplaced testing rigor, or the lack of intelligent test design because of low business domain knowledge. Adaptive testing would have allowed the teams to focus on the greater goal of the changes and to creatively fashion test solutions by combining and rearranging tests, types of tests, browser and OS combinations, or hardware configurations, according to the need in different environments. For example, high-value tests for production may encompass 40 percent usability (of both functions and content), 30 percent interoperability (of critical user flows), 20 percent security (user authentication and data flows), and 10 percent performance. It all depends on what the quality goals are for that particular test run and environment. The testing value shifts with different changes and potentially with each unique need of the code promotion.
1. Become self-adapting.
Break out of the pre-defined test scope by creating versatility and flexibility in your testing suite. You can do this by engineering a flexible framework that allows unique combinations of small, executable tests and grouping test assets based on quality goals. This will provide the ability to re-integrate parts of stories (or test cases) into new, high-value runs. You can define the new executions by precise needs and execute them in combination or independently. The core principle here is the flexibility of test assets, which presents endless options for creative execution.
www.TechWell.com
your site, and 40 percent of the existing customers use mobile devices to post product opinions on social media. This data tells you that the tablet presentation (usability and branding) will be important to test, and the ease of launching to social media from a mobile phone (interoperability and performance) should also be precisely targeted by combining tests that focus on these areas.
tion. What were once called regression, performance, and security tests are now combined needs that can be incorporated into a standard testing process. This method serves best when done in a lightweight and self-adapting way. Adaptive testing provides nimble test solutions that bend and shift with the changing needs of the market or the environment. {end}
brooke@testimprovements.com
This article first appeared on AgileConnection.com.
its
Since 1993, ALPI has empowered clients with innovative solutions delivered by our staff of flexible and creative professionals. Trainings are held virtually, at our state-of-the-art facility located just outside of the Nations Capital, or onsite at your company location.
Contact training@alpi.com or 301.654.9200 ext. 403 for additional information and registration details
www.alpi.com
23
THINKSTOCKPHOTOS.COM
24
BETTER SOFTWARE
MARCH/APRIL 2013
www.TechWell.com
refers to environmentally sustainable computing and the optimal use of information and communication technology (ICT) to manage the environmental sustainability of enterprise operations, supply chain, products, services, and resources throughout their lifecycle. [1] Organizations often fail to quantify the green benefits of portfolio optimization and thus miss a significant opportunity to articulate its sustainable value. The Carbon Disclosure Project has developed guidelines to calculate the carbon footprint of ICT-sector companies [2], and the International Telecommunication Union [3] is developing guidelines to help the ICT sector embrace sustainability in all aspects of its operations. These developments should nudge more organizations to look into the greening of their IT operations. As seen in figure 1, most IT projects can be classified into the following broad areas: application, operating system, hardware, network, data center (power, cooling, and facility), monitoring, security, costs, and processes and management. The elements in the border encompass all the individual layers as monitoring can be for the physical data center, network, or application. Similarly cost optimization can be a project for just one of the layers or for overall IT cost. A data center consolidation project may be undertaken to reduce cost and overhead and to provide reliable services to employees and customers. While these are valid reasons for this project, the green benefits are equally important. A project of this scale usually involves rationalizing several applications, discarding hardware, and trimming resources. If no environmental baseline is taken at the beginning of the project, then the sustainability benefits cannot be measured or reported at the end, and an additional benefit will be lost.
In addition, employees generally feel good about contributing to a greener environment through their daily work. Green initiatives can be leveraged for the companys participation in voluntary disclosure frameworks like the Carbon Disclosure Project to improve brand reputation. There is also motivation to achieve a little more while executing projects due to the added measurements and metrics that must be reported, and the process makes a stronger business case for streamlined IT operations. Lastly, organizations will find themselves in a better position to be compliant with upcoming regulations, such as those enforced by market regulators in certain locations that require organizations to publish sustainability reports or carbon taxation norms.
Case Study
A telecommunication service provider wanted to cut costs, increase capacity, and reduce the carbon footprint of its IT infrastructure. The provider understood that the data center and its server, storage, and network assets supported applications. The organizations IT infrastructure and operations group took a unique approach. They focused on rationalizing the portfolio first and then focused on consolidating, optimizing, and virtualizing the remaining infrastructure. The benefits were significant. From January to December, the organization retired 127 applications, decommissioned or redeployed more than 2,239 servers, and freed up 291,042 GB of storage. This translated into $28 million of re-deployable assets, a $20 million reduction in operating costs, and a reduction of data center-related carbon emissions by 10,450 metric tons. [4]
25
Figure 2: The various aspects of an application portfolio that need to be taken into account for the purpose of carbon footprinting (Source: Infosys Research)
Portfolio optimization presents the right opportunity to embrace a greener business application portfolio by introducing responsible e-waste disposal, reduced paper use, and efficient application usage. It also can result in a more efficient data center, data de-duplication, disposal of old assets, faster and more efficient applications, the elimination of underutilized applications, a reduced carbon footprint, and fewer required resources for support and maintenance.
power, facilities, security, monitoring, costs, processes, and others that are beyond the data center. The sustainability impacts of projects in each layer of the data center (see figure 1) are felt by all layers below. Therefore, the higher a project is in the data center layers, the greater the impact and the more measurements will be required. Based on this layered structure, the framework helps to identify which areas are potentially high and low impact. And, because IT projects usually impact one or several of the areas indicated in figure 1, the framework indicates metrics that should be identified and defined across all areas. The framework further describes a process to realize the benefits. Creating a baseline will help the company understand the current IT assets, resources, application landscape, energy use, and carbon footprint of the portfolio. Reviewing the metrics is an integral part of the framework so that all key stakeholders from different parts of the organizationlike IT, sustainability, and businesssign off on the metrics and baseline to avoid later disagreements. Also, each
www.TechWell.com
Figure 3: The sustainability framework for portfolio optimization (Source: Infosys Research)
department will review the metrics and baseline from a different perspective. An effective way of monitoring and tracking the key identified metrics is to create a dashboard, which also helps to create transparency at multiple levels of the organization. Measuring at project milestones is important so that the metrics are regularly updated. An added benefit is that if a particular metric is missed, it can be identified at the checkpoints and added to the dashboard. Reporting toward the end helps in articulating the cost, energy, and resource savings delivered as a result of portfolio optimization. In addition, the sustainability team can use this information to promote brand value and industry recognition. Table 1 depicts sample scenarios in which accurate baselines of the parameters should be created and measured at regular intervals. If the answer to any of the guidelines is affirmative, then there can be an associated sustainability benefit.
Conclusion
A comprehensive IT portfolio optimization initiative can help organizations create a responsive, agile, flexible, and cost-effective IT function with a strong alignment to business and environmental strategy. Embarking on business portfolio optimization provides the right platform to initiate sustainability efforts within the organization and, in turn, provides an added advantage to embarking on the optimization journey. Sustainability disclosures are gaining strength. Stringent upcoming carbon and energy regulations, such as the Carbon Reduction Commitment Energy Efficiency Scheme in the UK and the Australian carbon tax, will indirectly drive organizations to adopt green IT practices by focusing on energy efficiency and efficient operations. The disclosures and metrics described in the framework in this article are likely to become prevalent in many IT projects. Increasing public and industry awareness about sustainability makes it imperative that we talk about sustainability and IT portfolio optimization together. {end}
Table 1
sunita_purushottam@infosys.com vaibhav_bhatia@infosys.com
27
Product Announcements
UrbanCode Launches uRelease
UrbanCode, a build, deploy, provision, and release automation tool company, launched uReleasea release management and coordination platform that lets users plan, execute, and track releases all the way to production. uRelease eliminates the visibility and tracking challenges of managing application and infrastructure changes with different tools. The environment management capabilities within uRelease allow users to allocate environments to phases in a release for development and testing, providing visibility and control for efficient environment utilization. The flexibility to deploy using application-centric deployment processes in lower environments and team-centric processes in production environments accommodates environmental differences, allowing the tool to align with established practices within release management.
Kovair Software Announces Release 7.0 for Enterprise-class ALM and ITSM Products
Kovair Software, Inc. announced release 7.0 of its enterpriseclass Development and IT Tools products generally offered as the ALM Studio, ITSM Studio, and Omnibus Integration Platform. The major focus area for this release is to provide Kovairs current and future users with a Web 2.0 user interface experience that is consistent with the social media UI that users are looking for in all the applications that they use currently. This new release resembles social networking websites and has been designed like that in order to enrich user experience and allow future releases of Kovair 7.0 to run on mobile devices.
blackducksoftware.com/black-duck-suite Splunk, Inc. Rolls Out New Software Development Kits for Java and Python
Splunk, Inc., a software platform for real-time operational intelligence, announced the general availability of new software development kits (SDKs) for Java and Python. SDKs make it easier for developers to customize and extend the power of Splunk Enterprise, enabling real-time, big-data insights across the organization. Splunk previously released the general availability version of the Splunk SDK for JavaScript for Splunk Enterprise 5. The Splunk SDK for PHP is in public preview. The Splunk SDKs for Java, Javascript, PHP, and Python are built on a fully documented and supported REST API and include documentation, code samples, resources, and tools to help developers build on the Splunk platform.
licensedashboard.com/US/license-management-software/license-manager.html
28
BETTER SOFTWARE MARCH/APRIL 2013
www.TechWell.com
Product Announcements
advantage of the flexibility of Git while maintaining control of code quality and development. With improved performance and scale for large, distributed teams, Stash 2.0 rounds out Atlassians integrated enterprise offerings that include JIRA for issues management and Confluence for content collaboration. Stashs new enterprise functionality includes branch permissions that allow development teams to specify and manage an individual or teams access control to code. All code developed separately on a branch can be properly tested and reviewed before being merged into source. The new team collaboration features include @mentions and Markdown support, instantly bringing any team member into a code discussion and providing greater context for richer discussions and faster communications. tween Appcelerator Titanium 3.0 and SOASTAs TouchTest mobile test automation solution and a distribution agreement between the two companies. This integration allows mobile developers at Appcelerators 1,400 enterprise customers to access TouchTest for continuous automated testing as part of the mobile application development lifecycle. TouchTest is the first enterprise test automation solution to support Appcelerators Titanium 3.0. The integrated solution provides a cost-effective, rapid, and continuous product development lifecycle for mobile developers. The new distribution agreement between Appcelerator and SOASTA enables Appcelerator customers to easily add TouchTest to their mobile development environment. As enterprises and developers build mobile applications for a variety of devices and operating systems, they can now rely on TouchTest to simplify the testing process, utilizing SOASTAs patented visual test environment, precision capture, and replay technology. Apps are now continuously testable, thoroughly validated, and fully supported through the development lifecycle by leveraging TouchTests embedded Jenkins plugin for 100 percent hands-free mobile test automation.
icims.com Appcelerator and SOASTA Partner to Bring TouchTest to Mobile Enterprise Developers
Appcelerator and SOASTA announced a new integration be-
Product Announcements
with a free trial of Keynote DeviceAnywhere Test Center Developer. documenting every step they take during the testing process. A subscription-based service, Defect Scribe is installed on the users desktop and records every user action and application screen to improve test session efficiency and create better bug reports. Available for web, Windows, or Java applications, Defect Scribe records activity during manual and exploratory testing. Defect Scribe builds a detailed history of the test session, including descriptions of the user interface controls used and a screen shot of every step with the relevant graphical user interface element automatically highlighted. If a defect is found, the tester can submit a complete and accurate bug reportincluding the recording, associated screenshots, testing notes, and detailed steps to reproducein just one click to a defect tracking tool or via email. This saves significant time with manual data collection, allowing testers to focus on breaking software and enabling developers to quickly reproduce and fix bugs.
GET A TESTING
RESULT YOUR
testplant.com/products/eggplant/mobile
[1]
Capers Jones, A Short History Of The Cost Per Defect Metric, Randall Rice, The Value of ISTQB Certication
30
BETTER SOFTWARE
MARCH/APRIL 2013
www.TechWell.com
by Ed Weller
efweller@aol.com
31
Career Development
A Major Award
The physical award might not always be pretty, but the meaning shines through.
by Lisa Crispin | lisa@lisacrispin.com
North American readers may be familiar with the 1983 movie A Christmas Story (now also a Broadway musical), about a boy named Ralphie growing up in Indiana in the early 1940s. Ralphie longs for a BB gun for Christmas and has visions of protecting his family from marauding cowboy villains with it, but all the adults warn him, Youll shoot your eye out! The father in the movie (brilliantly played by Darren McGavin) constantly enters crossword puzzle contests. In one of my favorite scenes, he finally wins a major award, which is delivered to their house in a huge packing crate. The award turns out to be an incredibly tacky lamp made in the erotic shape of a womans leg, with a fishnet stocking and high-heeled shoe. His wife is horrified, but to the dad, this award is indescribably beautiful. It is the affirmation that hes a great solver of crossword puzzles and should enter even more contests. Back in 1998, I had my first major award experience. At the height of the Internet boom, I took the leap from traditional software development to a startup company producing a web travel site (Trip.com, now CheapTickets.com). Id never tested web software, and my new team had never done any testing at all. We found our courage, experimented, and, though we may not have delivered the best software around, we helped our company succeed. After Id been there a few months, the company founders decided to institute a quarterly award to the employee who rocked the mostthat is, someone who was a solid foundation for team success. Naturally, they called it the Rock Award. I was quite amazed to be named as the first recipient. The award itself was a wooden base with a small brass plaque, and sitting on it was a rock that our COO had picked up from the side of his driveway that morning. Its not quite as tacky as the leg-shaped lamp, though its just about as ugly. But for me, it was affirmation that my efforts to learn how to test web applications had paid off, that I could really add value to my team, and that my team appreciated my collaboration. And, yes, I thought it was indescribably beautiful. The Rock Award gave me the confidence to keep taking chances in my professional career and to share my experiences with a wider community. I wanted to attend testing conferences, so I started submitting papers. With the help of our teams technical writer, I finally got one accepted and my conference-presenting career was launched. In 2000, several of my teammates joined another startup and gave me Kent Becks Extreme Programming Explained to read. I was so excited about XPs focus on quality that I begged them to hire me as their tester and I never looked back. Ive had a series of mostly wonderful jobs on terrific teams since then. Today, Im enjoying a dream job, along with many opportunities to write, participate in conferences, and be part of a wonderful worldwide software community. I credit Trip.coms cofoundersand their initiative to encourage employees with that crazy Rock Awardwith boosting my confidence. Ive continued to develop my professional abilities and skills that helped my team, and Ive worked to exchange skills and experiences with others. Its always a good time to think about your learning and career goals. Im not sure that we all need to win an award, but think of ways that you can encourage your own colleagues to grow professionally. Recognize their contributions to the teams achievements. You dont have to be a manager to nurture a learning culture and help your business stakeholders understand that software teams need time to learn and experiment. Freedom to continually improve not only makes us more productive, it also
32
BETTER SOFTWARE
MARCH/APRIL 2013
www.TechWell.com
Career Development
It's always a good time to think about your learning and career goals.
helps each of us enjoy getting up in the morning and going to work. Sharing our experiences in the larger software community helps us all. What would you and your team like to achieve this year? How can the global community help you? How can you help and encourage others? In A Christmas Story, Santa brings Ralphie his Red Ryder BB gun, and it indeed proves to be a bit dangerous. But in the end, the family has a wonderful Christmas, and the boy feels more confident and happy. Lets all take some risks and find more joy in developing software.
{end}
index to advertisers
ADC/BSC West 2013 adc-bsc-west.techwell.com Inside Front Cover
Display Advertising advertisingsales@sqe.com All Other Inquiries info@bettersoftware.com Better Software (ISSN: 1553-1929) is
adc-bsc-west.techwell.com
19
ALP International
www.alpi.com
23
published six times per year January/ February, March/April, May/June, July/ August, September/October, November/
ASTQB
www.astqb.org
30
December. Back issues may be purchased for $15 per issue plus shipping (subject to availability. Volume discounts available. Entire contents 2013 by Software Quality Engineering (340 Corporate Way, Suite 300, Orange Park, FL 32073), unless otherwise noted on specific articles. The opinions expressed within the articles and contents herein do not necessarily express those of the publisher (Software Quality Engineering). All rights reserved. No material in this publication may be reproduced in any form without permission. Reprints of individual articles available. Call for details.
33
Cognizant
www.cognizant.com/testing-services
OpenMake Software
www.openmakesoftware.com
SQESTARCANADA 2013
starcanada.techwell.com
SQESTAREAST 2013
stareast.techwell.com
15
sqetraining.com/TrainingWeek
Back Cover
Telerik
www.telerik.com/html5-testing
A dd th e c rit ic A l p i e c e t o yo u r c A r eer
San Francisco, CA May 2123 Nashville, TN Las Vegas, NV Chicago, IL Tampa, FL May 2123 June 24 June 1012 June 1820
www.sqetraining.com/certification 34 www.TechWell.com
BETTER SOFTWARE MARCH/APRIL 2013
SQE TRAINING