Boost Application Deliver

FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS
Boost Application Delivery Speed And Quality

With Agile DevOps Practices
Road Map: The Modern Application Delivery Playbook
by Kurt Bittner, Diego Lo Giudice, and Amy DeMartine
March 22, 2016 | Update: April 14, 2016
Why Read This Report
Key Takeaways
Driven by customer demands and competitive

pressures, application development and delivery
(AD&D) leaders have an imperative to deliver
applications faster, yet many struggle with where
to start and what comes next. They feel
overwhelmed by the apparent immensity of the
change. The good news? The path is now well
understood and the journey is an incremental one,
where progress is made step by step, practice by
practice, and team by team. This report describes
the common adoption patterns that organizations
follow on their road to delivering applications
faster and with higher quality.
Pair Solid Dev And Ops Practices To Stabilize

And Simplify Delivery
Versioning everything, introducing continuous
integration, and automating environment
provisioning enable organizations to go faster -eliminating errors, reducing manual effort, and
removing fundamental bottlenecks.
This is an update of a previously published report,

The Eight Tenets Of Faster Application Delivery.
This report completely replaces the earlier report.
FORRESTER.COM
Agile And DevOps Teaming Models Reduce

Handoffs And Improve Collaboration
Teams with diverse skills spanning business,
development, quality assurance, and operations
are more effective and spend less time waiting
for shared resources. They are more accountable,
deliver faster, and produce better results.
To Gain The Greatest Business Benefits,
Business Practices Must Change, Too
Reducing release size reduces risk and
complexity, but, more importantly, it allows
organizations to innovate by quickly trying
new ideas and measuring the result. Business
stakeholders must embrace uncertainty and learn
from their customers to achieve better results.
Boost Application Delivery Speed And Quality With Agile DevOps

Practices
by Kurt Bittner, Diego Lo Giudice, and Amy DeMartine
with Christopher Mines, Jeffrey S. Hammond, Taylor Turrisi, and Matthew Izzi
March 22, 2016 | Updated: April 14, 2016
Table Of Contents
2 Companies Know They Must Deliver Faster,
But Where To Start?
3 Organizations Face Consistent Barriers To
Faster Delivery
4 Introducing Forresters Agile And DevOps
Practice Model
7 Traditional Approaches Let You Release
Quarterly, But Its Not Pretty
7 To Release Every One To Three Months,
Stabilize The Delivery Pipeline
10 To Release Monthly, Make Systemic
Changes To Your Pipeline
Notes & Resources

Forrester interviewed 45 vendor and user
companies, including the American Medical
Association, Disney, General Electric, Hewlett
Packard Enterprise, IBM, Microsoft, and
ThoughtWorks.
Related Research Documents

Faster Software Delivery Will Accelerate Digital
Transformation
Forget Two-Speed IT; DevOps Enables Faster
Delivery Across The Board
Use DevOps And Supply Chain Principles To
Automate Application Delivery Governance
13 To Release Daily, Embrace Cloud Platform

Practices
14 Scaling Agility And DevOps To The
Enterprise Level
Recommendations
15 Start Now To Improve Your Application

Delivery Capabilities
16 Supplemental Material
Forrester Research, Inc., 60 Acorn Park Drive, Cambridge, MA 02140 USA

+1 617-613-6000 | Fax: +1 617-613-5000 | forrester.com
2016 Forrester Research, Inc. Opinions reflect judgment at the time and are subject to change. Forrester,
Technographics, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester
Research, Inc. All other trademarks are the property of their respective companies. Unauthorized copying or
distributing is a violation of copyright law. Citations@forrester.com or +1 866-367-7378
Boost Application Delivery Speed And Quality With Agile DevOps Practices
Companies Know They Must Deliver Faster, But Where To Start?

Customers and competitors have forced most organizations to an uncomfortable conclusion:
Organizations must deliver high-quality software, at a rapid pace, constantly, or they will fail. Most
organizations, in fact, can deliver some kinds of releases quickly, in the form of hot fixes or patches.
Yet the processes that AD&D pros follow are special circumstances; they are not sustainable, and they
expose the organization to too much risk. Their typical release cycles are considerably slower than
necessary (see Figure 1).
We can deliver changes in production in a matter of days . . . but the entire IT floor stops to focus
on that emergency! (VP of application development, large multinational Italian bank)
FIGURE 1 Application Release Cycles Are Still Too Long For Many Organizations
How often does your team or teams release applications?

We release many times per day
2%
We average one release a day
2%
We average one release a week
8%
We average one release a month
21%
We average one release per quarter
26%
We average two releases a year
16%
We average one release a year

(or less frequently)
We time new releases to coincide with new
versions of operating systems or middleware
Other
11%
10%
6%
Base: 635 business and technology influencers and decision-makers whose

priorities include addressing rising customer expectations
(percentages do not total 100 because of rounding)
Source: Forresters Business Technographics Global Developer Survey, 2015
2016 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law.
Citations@forrester.com or +1 866-367-7378
Organizations Face Consistent Barriers To Faster Delivery

Old habits are hard to break. Years of accumulated processes, outmoded practices, ineffective tools,
and siloed organizations have created systemic barriers that prevent organizations from improving their
delivery performance (see Figure 2). These typical bottlenecks prevent fast release cycles by creating
inconsistencies, errors, waste, and delay:
The business wants things that customers dont. Lacking information about what customers
really need and want, organizations make, at best, educated guesses. The result? Too much of
what gets built never gets used or misses the mark and needs to be reworked.
The word enhancement is used to hide a lot of waste. It really means that what was delivered
the first time was wrong and needs to be reworked. Calling it an enhancement suggests that
the original idea worked and now well make it better; thats usually not the case. (Pierre Pureur,
companywide chief architect, Travelers Insurance)
The business packs releases full of features, leading to delays. The more features that they
pack into it, the more they delay the release. The math is simple, but the solution is not: getting
people to let go of their favorite feature in order to get to market quicker.
Its a chicken-and-egg problem. Business bloats releases with functionality because it knows the
next opportunity for more functionality [wont] come [for] months. But actually if it could prioritize
the minimum needed features, and therefore require a smaller release, the opportunity for more
functionality and change would come sooner! (VP of demand, national postal service)
Manual software development and testing processes further delay releases. It is ironic, but
all too common, that developers and testers whose profession focuses on delivering automated
solutions fail to apply automation to their own activities. The culprit is inconsistency in their rush
to get applications out the door, developers and testers dont always take the time to standardize
and simplify their work so that it can be automated.
Effective automation has a fundamental prerequisite: strong design and development skills.
We went out and hired good developers to replace some of our manual testers. (VP of quality
engineering, global systems integrator)
Ops should automate environments and releases but often doesnt. Operations, like
developers and testers, doesnt always apply automation to its own work. When ops handconfigures unique environments and applies manual release processes, it lays the foundation for
future problems caused by inconsistencies. But its often too busy to put in place the processes
and automation that eliminate problems at their source.
Operations has too many manual processes, and the cost of the errors these manual processes
cause comes out in dollars lost for the business. (Gene Kim, author of The Phoenix Project: A
Novel About IT, DevOps, And Helping Your Business Win)
Keeping everyone as busy as possible causes waiting, busywork, and delay. Managers love
it when their employees are busy it makes everyone feel important and needed. But there is a
hidden cost: When one employee needs another employee to get something done, they wait, and
they fill their time with noncritical work. When software delivery is the goal, the cumulative result is
delay. Handoffs, stops, and restarts all lead to slower, less effective releases.
FIGURE 2 Typical Application Release Bottlenecks
Lack of
effective
customer
insight
Too much
WIP
Manual build
and integration
processes
Manual testing
Lack of
Manually
sufficient test
managed
data to make
environments
decisions
UAT/
exploratory
testing
Idea
proposed
Understand
needs and
invent
solutions
Develop,
commit,
and build
Waiting
for build
and test
environments
Functional
testing
Release
decision
Deploy
solution
Customer
value
Load,
performance,
security, . . .
testing
Manual and error-prone app deployments
Error-prone manual handoffs and processes

One-way flow of information
Introducing Forresters Agile And DevOps Practice Model

Lean value stream mapping is a technique, based on Lean manufacturing principles, that AD&D pros
use to understand and improve their application delivery practices by uncovering bottlenecks and
barriers.1 The information it uncovers provides insight into the greatest sources of waste and delay,
which enables AD&D pros to simplify and streamline their application delivery processes. Key DevOps
practices remove these barriers.
To help organizations improve their delivery capability, Forrester has created a model that describes the
central practices that AD&D pros must apply to deliver at specific release cadences (see Figure 3). This
model is based on the experiences of hundreds of organizations across a wide variety of industries
and government entities. It describes the typical practices of these organizations that enable them to
deliver at the associated application release cadence.
FIGURE 3 Key Delivery Practices And The Release Capabilities They Enable
Release cycle
More than
three months
One to three
months
Practices
Characteristics
Largely manual build and integration, environment Chaotic releases driven by personal
heroics
provisioning, testing, and governance
Relatively infrequent releases
Developers working on separate branches,
producing inconsistent results
periodically integrating
High cost of wait time hidden by
Siloed teams, when teams even exist; lots of
high utilization rates; over-focus on
individuals with low collaboration
cost of labor due to predominance
The business is disengaged, Scrum/Agile adoption
of manual activities
is inconsistent.
Code versioned
Adopt enterprise Agile practices such as SAFe and
Nexus; high trust within and across teams
Goals shift from utilization and activity
measurement to customer experience and value.
Continuous integration is used to drive build,
integration, and build verification.
Automated deployments at least to testing
environments
Artifact repository is used to manage binaries.
Teams exhibit good practices in

isolation with some cross-functional
organization but lack a consistent
cross-enterprise approach.
Business case, epics, features,
releases, and sprints are planned at
the enterprise level.
Little control over software supply
chain
Delivery governance and testing
beyond unit testing are still largely
manual.
The business does not yet embrace
faster release cycles; still thinks in
terms of large releases.
Release velocity is constrained by
dependencies on other
applications, shared components.
FIGURE 3 Key Delivery Practices And The Release Capabilities They Enable (Cont.)
Release cycle
One to four
weeks
Daily or faster
Practices
Characteristics
The business plans releases in relatively small

increments.
Adopt enterprise Agile practices such as SAFe and
Nexus; high trust within and across teams
Environments versioned; infrastructure as code
practices adopted.
Loosely coupled architectures enable team
independence.
APIs for everything enables highly automated
testing.
Integrate database administration into continuous
integration process.
Comprehensive automated test suites based on
APIs; tests run complete in minutes
Test data management, service virtualization,
and network virtualization enable truly productionlike environments in dev and test.
Closely manage the software supply chain.
Application-delivery-pipeline-as-a-service
enables consistency and auditability.
Automate releases to production.
Automated application monitoring and alerting
provide delivery teams with early visibility into
potential performance problems.
Adopt Agile engineering practices (e.g., TDD and
refactoring).
Trunk-based development practices
Business plans and release plans

are closely aligned.
Production incidents are rare;
when they do occur, response is
rapid.
Governance achieved as a
byproduct of application delivery
processes.
The quality of every change is
easily and quickly assessed.
Scrap and waste caused
by rework and waiting is reduced.
Releases are frequent, fast, and
high quality.
Reduced time is spent coordinating
teams.
The business is hypothesis-driven, working in

very small batches to drive innovation based
on application analytics.
Application analytics, including customer usage
data, informs and guides work.
Teams continually refine and improve practices
based on feedback, including retrospectives.
Microservices and container-based architectures
enable change in very small increments.
Blue-green and A/B testing practice are used
to test in production.
The continuous delivery pipeline is continuously
improved to optimize throughout.
Dark deployments are used to release but not
expose functionality to users. Gray deployments,
canary deployments, and feature toggles are used
to control change visibility.
The organization is able to

understand customer satisfaction in
real time and respond rapidly to
new opportunities.
Organizations spend less time
speculating what customers want.
They form and test hypotheses
continually.
Organizations waste less time and
effort, leading to lower cost per unit
of product or service.
Applications are more stable
and can be continuously adapted
to meet new needs.
Traditional Approaches Let You Release Quarterly, But Its Not Pretty
Most organizations want to release faster, but they are frightened by the risks and instability they
see when they try do so with their existing practices. The following experiences are common to
organizations using traditional, largely manual practices:
Developers dont consistently version source code. While most developers version source
code, some do not; when they do, they may work on a branch for long periods, creating codemerging complexity.
QA testing is largely manual and disconnected from developers work. Manual testing
consumes significant time, and relying on user interface (UI)-based testing means that defects
are discovered late in the delivery cycle when the original developers context is largely forgotten.
Recreating this context means developers spend substantial time fixing the defect.
Deploying and running applications is largely ops problem. Developers are only held
accountable for preproduction quality; once the application is deployed, operations or quality
assurance (QA) is held accountable. Developers are deprived of vital feedback on the quality of
their applications, and QA and ops are saddled with problems they didnt create.2
Siloed resources maximize their use, which reduces overall speed to market. Lacking a
singular focus on improving customer experience or contributing to the bottom line, employees are
measured by how busy they are and the busier, the better. The problem is that whenever they are
needed, they are usually busy doing something else, which adds delays and increases hidden costs.
The business is largely disengaged from the application delivery process. The business views
participation in application delivery as a side job, not its main focus. Requirements are largely
gathered upfront and using traditional approaches, even when the development team employs
some Agile practices such as a water-Agile-fall approach.3
Personal heroics and rolling crises are common. Inconsistent manual processes mean that
someone always has to step in to fix a problem, and sometimes the problems are big. The lack of
repeatable processes means that knowledge is often tacit and hoarded. In a chaotic organization,
knowledge is power even though it perpetuates dysfunction.
To Release Every One To Three Months, Stabilize The Delivery Pipeline

Organizations take the first steps toward releasing applications faster when they recognize two things:
They cannot continue the way they work today; and with the right practices, faster releases are actually
less risky. AD&D leaders moving their teams to this level need to focus on seven key practices:
Begin to change the culture to focus on customer experience, not utilization. Organizing in
functional silos can create incentives for employees to focus narrowly on their job rather than on
improving customer experiences or business results. Measuring everyone on customer experience
and responsiveness (as measured by cycle time) incents team members from different silos to work
together toward a common goal. Changing culture takes time, but it begins with shared objectives.4
Make developers version all source-code changes and check in code frequently. To increase
the pace of delivery, developers must check in code changes many times a day. They cant work
for days or weeks on a private branch and expect that their code will integrate with all the other
code being developed. Working in smaller increments and integrating frequently lets developers
stay in sync with one another.
Manage open source and third-party components. Open source and third-party components
can help organizations speed development and reduce cost, but they can also increase risk
exposure when organizations do not manage them effectively. Evaluating and approving standard
components helps organizations streamline their software supply chains, improve quality, and
reduce risk and cost.5
Automate software build and unit testing with continuous integration practices. Continuous
integration is the automated assembly line for the application development process. It builds and
tests software automatically every time a change is delivered to the source code repository, but it
can do more. It can verify that the changed code meets quality standards using static analysis tools
and by running unit tests before it is merged into the main code line where it might affect other
developers.6 Push developers to unit-test 100% of the code they write, and dont merge code into
the trunk until they do.
Continuous integration is the enabling foundation for continuous delivery. Without it, teams struggle
to deliver sufficient velocity to enable the feedback loop that is essential to delivering business
value at more rapid cycles. (Mike Dillon, vice president, development services, Financial Industry
Regulatory Association [FINRA])
Automate deployment to test environments. Continuous integration enables automation of more
than just software build and integration; it provides a general-purpose automation framework for
delivery pipeline tasks. Deploying a build to a testing environment once it passes unit tests will
eliminate manual effort and speed testing efforts. At one- to three-month release cycles, testing
beyond unit testing is still largely manual, but at least having unit testing done better and more
often saves teams valuable time and forms the basis for more test automation in the future. At this
release frequency, automating 30% of all functional tests is a realistic goal.
Adopt Agile work management approaches. Agile project management practices help
organizations break down large requirements into small chunks that they can develop, test, and
demonstrate in intervals of just a few weeks. While demonstration is not the same as deployment
to production, doing so forces teams to continuously build and test working software and establish
the working habits that enable them to develop high-quality solutions more quickly than with
traditional approaches.
Bring new IDEAs to forming application delivery teams. Agile teams are composed of people
with diverse skills and who are dedicated to the team, not shared across many projects. They
plan and manage their own work, working flexibly to complete most work within the team without
having to call upon (and wait for) outside resources. They spend less time waiting for others and
more time getting work done. Reducing wait time helps them deliver applications faster and with
less overhead than traditional shared resource teaming models.7
Forresters IDEA team approach extends the Agile teaming model by bringing together the right
people to identify the outcomes that customers or users need to achieve, design the experiences
that will deliver those outcomes, engineer and deliver the solution, and analyze the result, repeating
the loop as necessary (see Figure 4).8
FIGURE 4 IDEA Teams Bring Together Diverse Talents To Deliver Great Results
Identify
personas,
outcomes, and context.
Analyze
results to monitor
performance and
optimize outcomes.
Start small
with a platform
to extend.
Design
the experience.
Engineer
your platforms,
processes, and
people to deliver the
experience.
To Release Monthly, Make Systemic Changes To Your Pipeline

Releasing once a month or faster requires teams to refine and improve practices across the entire
delivery pipeline, including the way they plan releases and organize themselves. There are broad
changes that affect how everyone works:
Focus each release on a specific outcome for a specific persona. When releases were
infrequent, the business would have something for everyone in a release. Increased testing and
better visibility provided by the automated delivery pipeline gives organizations the confidence to
release more frequently. The business needs to respond by targeting a single persona, a set of
users who share the same characteristics, with features that deliver or improve a single outcome to
reduce releases to the smallest size possible.9
Extend Agile management practices with Agile engineering practices. Basic Agile practices
are no longer sufficient to achieve faster delivery; to deliver with a cadence of between one week
and four weeks, you need to master practices such as test-driven development, refactoring, and
collaborative code review, as well as automating both functional and nonfunctional tests.10
Broaden the use of Agile practices within and across teams. Delivering applications quicker
and at scale requires teams to mix and match the best of Scrum, Kanban, and XP practices to
manage the fast pace of work. Enterprise Agile frameworks such as SAFe and Nexus help improve
throughput and align priorities across teams.
Use automated application monitoring to improve customer insight. To deliver the right things
faster, organizations use application analytics tools to gain insights into what customers are doing,
where their needs are met, and where opportunities exist to improve their experiences. Providing
these insights to delivery teams and business professionals lets them focus on building and
delivering the right capabilities.11
Developers And Architects Must Work Together To Refine Their Delivery Practices
The application delivery pipeline is, at its core, driven by the work developers do. Until they make
fundamental changes in the way they work and develop software, other improvements will yield little
benefit. Architects play a role as well, helping evaluate and approve component vendors and their
offerings. Developers and architects need to work together to:
Adopt trunk-based development practices. Trunk-based development means that the
application has a single code line that can be built and deployed at any time. To achieve this,
developers must test and peer-review any code change being merged into the trunk to make sure
the trunk stays clean and defect-free. The most important change for developers is that every code
change must have automated tests that verify the code functions correctly.
10
Evolve to an API-centric, loosely coupled, service-based application architecture. Using

services, complete with robust, versioned APIs for each and every function, enables organizations
to independently develop different parts of applications and to test those parts independently
and as an integrated whole using continuous-integration-driven automation.12 Organizations with
large legacy code bases cant adopt truly loosely coupled service-based application architectures
overnight, but they can evolve toward them over time.13
Use supply chain practices to manage externally sourced components. Open source and
third-party commercial components enable organizations to deliver quickly by reducing the amount
of code they have to write. Just as manufacturers have learned they have to monitor and manage
their suppliers, AD&D pros are learning that they have to manage increasingly complex supply
chains. Every component brings benefits as well as risks, and you must manage those risks by
selecting the best components and suppliers and by making sure delivery teams use only the
latest, most secure versions of selected components.14
Integrate database administration (DBA) into the continuous integration process. Many
applications depend deeply on information in databases and can break when application and
database updates are not synchronized. Manual DBA processes impede rapid delivery. Putting
database changes under version control, and integrating automated DBA into the continuous
integration process, enables organizations to keep applications and database schemas in sync
without sacrificing speed or quality.15
Use application-delivery-pipeline-as-a-service to boost consistency. Organizations just
starting their DevOps journey find that every team has different tools and different delivery
pipelines. Offering teams a pipeline-as-a-service gives them a way to build and test their code
using a common tool chain so they dont have to configure their own. Organizations that have
implemented this govern and instrument their delivery processes by having common pipeline
processes, while letting delivery teams focus on their code.16
Testers Must Embrace Automation And Work With Developers To Shift Left
Manual QA cant keep up with the demands of subquarterly delivery cycles. UI-based test automation
cant keep up, either, because UIs stabilize too late to give essential early feedback. To avoid being the
bottleneck in the delivery pipeline, testers need to:
Embrace API-based test automation. With API-based applications, organizations can shift much
of their application testing to the left i.e., earlier, in the typical left-to-right depiction of process
flows. Continuous-integration-based test automation can perform not just functional testing but
also virtually any kind of test except usability and exploratory testing. This allows organizations to
achieve very high levels of testing in every single build, including performance, scalability, security,
and regression testing. This ensures that quality is continuously assessed, providing feedback that
continually improves the application.
11
The key is that the tests need to run in minutes, not hours. This means that some tests may need
to run in the background in order to provide rapid feedback to developers. If tests take too long,
developers will stop using CI. (Engineering lead, software-as-a-service company)
API-based testing requires coding skills, meaning that testers also need to shift their skills left
by learning how to code. They must work alongside developers when they create APIs to define
executable tests that detect when APIs are broken. Over time, they need to develop deeper
functional tests by writing API tests that string together multiple API calls to perform a function.
Make test environments simulate production with service virtualization technology. Service
virtualization enables organizations to simulate applications and services without having to fully
replicate a production environment.17 Network virtualization lets them simulate the variability of realworld communication networks when testing connected applications, especially when developing
mobile apps. Test data management, though often overlooked, gives organizations the ability to
create realistic test data sets in less time, with less data. When application logic is data-sensitive,
as many commercial applications are, test data management gives organizations confidence that
their application testing is uncovering logic errors.18
Service virtualization also simplifies and reduces team dependencies by simulating the APIs of
components that other teams are developing. Automated unit tests detect API changes and
expose broken API contracts. As long as APIs dont change and the fully realized components are
available by release time, teams can work relatively independently, eliminating complex program
management coordination.
Ops Must Automate Provisioning And Deployment To Increase Delivery Speed
Developers and testers need an environment to do their work, and every time they have a build to test,
it has to be deployed. To enable everyone to do their best work, ops needs to:
Version environment configurations and automate provisioning. Application code needs to
run in environments with specific configurations; when environments arent configured correctly,
applications break. Versioning both together and building standardized environments with
automation helps organizations eliminate hard-to-diagnose errors that cause developers or testers
to say It works fine on my machine when they try to reproduce a defect.19
These infrastructure as code practices typically start with systems but can evolve to extend
to networks and storage. Infrastructure as code also provides a perfect opportunity for AD&D,
infrastructure and operations (I&O), and information security pros to work together to define
environment configurations.20
Production incidents are expensive. When we did a root-cause analysis on ours, we found that
18% were due to configuration errors. Infrastructure as code practices helped us to eliminate
those. (I&O manager, petrochemical company)
12
Automate application deployments to production. Automating deployment makes releases

less risky and more predictable by eliminating manual errors. Automation is essential when teams
deploy software frequently because it also eliminates waiting for manual deployment steps. When
testing environments mirror production environments, teams can execute and test deployment
automation with every build, ensuring that when they use it to deploy to production, it is reliable
and error-free.21
To Release Daily, Embrace Cloud Platform Practices

Not every organization needs to release software in production every day, or many times per day.
Many applications cant be released this fast; mobile apps, for example, dependent on app store
approvals and customer downloads, dont need to be released faster than every few weeks unless a
critical update is necessary. Organizations with applications running in the cloud or on the Web need to
update much faster. To respond more quickly to customers, exploit the dynamic capabilities of cloud
platforms to gain customer insights faster:
Test new ideas quickly by releasing software in rapid cycles. Leading organizations dont try to
define large sets of requirements in one go. They focus narrowly on a specific customer outcome at
a time; they form hypotheses on how they can improve that customer outcome; they develop and
deliver software they think will improve that outcome; they measure the result; and then they repeat
the cycle. By doing this, they gradually but systematically improve customer experiences.
In our Visual Studio Team Services, within each sprint, each feature crew tries several experiments
based on the next set of hypotheses for its area of responsibility and compares them to established
business metrics. In each case, the experimental results are based on data from production
telemetry like usage metrics or quality of service. The speed of experimentation varies. A/B testing
a single web page can be done almost instantly. Determining how a revised user experience affects
customer retention may take a few weeks. (Sam Guckenheimer, group product planner, Microsoft)
Use application analytics to inform and refine customer insights. Application analytics provides
organizations with insight into customer journeys and satisfaction, enabling them to adapt
applications and broader product offerings to better meet customer needs. Knowing how, or even if,
features are being used and knowing where customers are still struggling to achieve what they
would like gives delivery teams the information they need to improve customer experience.22
Continually refine and improve practices based on feedback and retrospectives. Measuring
and diagnosing delivery pipeline delays gives organizations the insight they need to improve
their practices. Higher-than-expected rework, deployment failures, and production problems that
got past testing all these point teams toward needed improvements. Removing waste and
eliminating errors does more than improve cycle time; it also helps organizations reduce cost and
improve application reliability.
13
Increase application modularity using services, microservices, and containers. Modular

applications are easier to modify and deploy than tightly coupled applications. Achieving very fast
delivery cycles requires organizations to release everything in the form of services. Organizations
that release many times a day are, in fact, not releasing entire applications but rather simply
deploying new and changed services. This makes deployment simpler, easier, and less risky.23
Test in production to get faster feedback from real-world users. Testing in production may
sound like complete madness, but it can yield real-world insights that simulated testing cannot.
Advanced deployment techniques like blue-green deployments and canary deployments let teams
gradually expose functionality to users while gathering feedback.24 The goal of this testing is not
to find defects; that is achieved through automated testing early in the delivery pipeline. Instead,
this kind of testing provides feedback about usability and suitability by monitoring how overall
application performance meets customer needs.
Use feature toggles to separate software release from user visibility. Feature toggles let
teams deploy software all the time without letting users see the changes. When the feature is ready
to use, it can be turned on without having to push out new code, providing the business with the
flexibility to release new capabilities when they are ready. Organizations that release every day
(or faster) are pushing new code all the time, but most of it is turned off until the business is ready
to make the change visible. Developers often use feature toggles with advanced deployment
techniques to progressively expose new features to the user community.
Use continuous workflow management techniques to improve throughput. Releasing many
times a day leaves teams no time to plan releases. Feature toggles shift the planning of user
visibility to the business, and software release planning shifts to continuous workflow approaches
like Kanban that improve flow and reduce waste by reducing work in process (WIP).
Scaling Agility And DevOps To The Enterprise Level

Few organizations employ DevOps practices across their whole organization. Even those that widely
apply DevOps practices find that teams release at different cadences depending on business needs
and architectural and delivery pipeline constraints. Organizations that broadly apply Agile and DevOps
practices share the following characteristics:
They scale Agile throughout the enterprise and downstream. To support DevOps at the
enterprise level, organizations have to be more Agile in the way they define business cases,
innovate with new ideas, manage the portfolio of ideas and opportunities, coordinate the work
necessary to realize those ideas using Agile practice, and drive the delivery of solutions using
DevOps practices.25
They support teams with common delivery pipeline services. Having a common delivery
pipeline framework for all teams enables organizations to provide consistent governance and
oversight while at the same time improving team delivery capabilities.
14
They build common application platforms. Common components and frameworks let teams be
more productive while reducing variability across teams. These components come from a variety
of sources: in-house delivery teams, open source projects, and commercial vendors. Application
delivery teams produce better results, faster, when they build on top of common platforms.26
They mediate all team interactions through services and APIs. Modular services, with
associated APIs, let teams focus on building their own software rather than spending time
coordinating with other teams.
They excel in at least the fundamental DevOps practices. These organizations have effective
trunk-based development practices, they excel in continuous integration and infrastructure as
code, and they automate their releases, at minimum. Depending on their needs, some teams may
also use more advanced release cycle practices: While not everyone needs to release many times
a day or use feature toggles, everyone needs to be able to patch applications quickly when new
security vulnerabilities are detected.
Recommendations
Start Now To Improve Your Application Delivery Capabilities

Truly continuous delivery isnt the answer for every application in every firm, but every team can
improve the speed and quality with which they deliver applications. Whether by improving workflows,
reducing handoffs, or eliminating meetings by automating release readiness assessments, AD&D and
I&O pros have countless opportunities to improve their application release capabilities, but those
opportunities will vary from team to team. On your journey toward better delivery capabilities, here are
some strategies for staying on course:
Start small and work in small increments. Big changes are disruptive and can actually
make teams perform worse before they get better. By working in small increments, making
improvements, measuring the benefits, and then finding new opportunities, youll make the
changes easier to absorb, easier to fund, and easier to make.
Introduce improvements team by team. Not all teams will have the same challenges. Most will
benefit from continuous integration or infrastructure as code, but further improvements depend on
the skills of the team and the modularity of the application. Assess and remove barriers at the team
level to yield immediate and concrete results.
Support team improvements with investments in organizational capability. Continuous
integration and standardized, automated environment provisioning are two areas where common,
centralized processes and automation can help everyone deliver faster. Centralizing these functions
frees up team resources to focus on delivering business value, not building infrastructure. The labor
savings will pay for centralizing these functions many times over.
15
Automate governance assessments. Security audits and code quality compliance are two areas
where automation can reduce time and labor cost while improving results. Static analysis tools and
automated tests provide objective and inexpensive means for ensuring quality results without timeconsuming meetings and expensive, and often inconsistent, manual inspections.
Make sure the business is more than along for the ride put it in the drivers seat. Achieving
better results requires everyone to be fully engaged, especially the business. Faster delivery gives
you a closer connection to customers and the ability to adapt quickly to new information, but
only when the business is fully engaged. Faster delivery not only enables new business models, it
demands new business models and new approaches to cocreating innovative solutions, delivering
them to customers, gathering feedback from customers through analytics, and refining solutions
based on a constant stream of new information.
Engage With An Analyst

Gain greater confidence in your decisions by working with Forrester thought leaders to apply our
research to your specific business and technology initiatives.
Analyst Inquiry
Analyst Advisory
Ask a question related to our research; a

Forrester analyst will help you put it into
practice and take the next step. Schedule
a 30-minute phone session with the analyst
or opt for a response via email.
Put research into practice with in-depth

analysis of your specific business and
technology challenges. Engagements
include custom advisory calls, strategy
days, workshops, speeches, and webinars.
Learn more about inquiry, including tips for

getting the most out of your discussion.
Learn about interactive advisory sessions

and how we can support your initiatives.
Supplemental Material
Survey Methodology
Forresters Business Technographics Global Developer Survey, 2015, was fielded to 1,943 developers
located in Australia, Brazil, Canada, China, France, Germany, India, New Zealand, the UK, and the US.
16
Forresters Business Technographics provides demand-side insight into the priorities, investments, and
customer journeys of business and technology decision-makers and the workforce across the globe.
Forrester collects data insights from qualified respondents in 10 countries spanning the Americas,
Europe, and Asia. Business Technographics uses only superior data sources and advanced datacleaning techniques to ensure the highest data quality.
Companies Interviewed For This Report
American Medical Association
IBM
AppDynamics
Infosys
Atlassian
Magenic
Basis Technologies
MasterCard
CA Technologies
Microsoft
Capital One
Nationwide Insurance
Chef Software
New Relic
Clarive Software
OpenMake Software
CloudBees
Optum
CollabNet
Parasoft
Cushman & Wakefield
Perforce
Danske Bank
Practical Large Scale Agile
DBmaestro
Prologis
Deutsche Bank
Puppet Labs
Disney
Serena Software
Electric Cloud
Skytap
FamilySearch
Tata Consultancy Services
FINRA
Tech Mahindra
Gap
ThoughtWorks
GE
Travelers Insurance
Grant Thornton
XebiaLabs
The Hartford Financial Services Group
ZeroTurnaround
Hewlett Packard Enterprise
17
Endnotes
For a summary of how value stream mapping works, and the core steps and milestones needed to define it, see the
Faster Software Delivery Will Accelerate Digital Transformation Forrester report.
Putting developers on call provides them with valuable insights into how their applications are used and how they
operate. See the Brief: Put Developers On The Front Lines Of App Support Forrester report.
Agile adoption varies greatly across organizations. For a deeper view, see the The 2015 State Of Agile Development:
Learn From Agile Expert Firms Forrester report.
Most organizations say that changing their culture is the most challenging aspect of improving their delivery
capabilities. For a more thorough discussion of the cultural and organizational aspects of improving application
delivery capabilities, see the Use DevOps Practices To Create A Lean And Responsive Application Delivery
Organization Forrester report.
DevOps practices actually improve application governance rather than weakening it. See the Use DevOps And
Supply Chain Principles To Automate Application Delivery Governance Forrester report.
A wide variety of continuous integration tools are available, led by the popular open source tool Jenkins; its
commercial extension CloudBees Jenkins Enterprise; and other commercial offerings, such as Atlassians Bamboo,
Electric Clouds ElectricCommander, IBMs AnthillPro and Rational Build Forge, JetBrains TeamCity, Microsofts Team
Foundation Server, and OpenMakes Meister.
For a perspective on the typical roles on Agile teams, see the The 2015 State Of Agile Development: Learn From Agile
Expert Firms Forrester report.
For more information on IDEA teams, see the Organize For Mobile Development Success Forrester report and see
the Mobile Moments Transform Customer Experience Forrester report.
For more information on how design practices improve application delivery capability, see the Bring Design Practices
To Application Development Forrester report.
For more information on how organizations extend basic Agile practices, such as shorter iterations, improved team
communication, transparency for the business, and measurements focused on value with downstream practices, see
the Agile Experts Focus On Downstream Delivery Forrester report.
10
For a deeper view of application analytics, see the Application Performance Management Remains The Same Even
With Mobile Forrester report.
11
APIs are key to DevOps and the digital business strategies they enable. See the APIs Underpin A Digital Business
Platform Forrester report.
12
Modular applications composed of independent services are easier to deliver and evolve over time, but most legacy
applications are built upon monolithic application architectures. Rewriting those applications is often impractical or
too expensive, making incremental modernization the only alternative. See the Application Modernization, Service By
Microservice Forrester report.
13
For a view on other practices that enhance security and reliability, see the The Seven Habits Of Rugged DevOps
Forrester report.
14
Companies like Datical and DBmaestro provide solutions that enable organizations to make rapid and frequent
changes to relational database schemas.
15
For more information on how DevOps improves application delivery governance, see the Use DevOps And Supply
Chain Principles To Automate Application Delivery Governance Forrester report.
16
18
In Forresters 15-criteria evaluation of service virtualization and testing vendors, we evaluated five solutions offered by
CA Technologies, HP, IBM, Parasoft, and SmartBear Software. See the The Forrester Wave: Service Virtualization
And Testing Solutions, Q1 2014 Forrester report.
17
Test data management helps organizations reduce the cost and complexity related to providing clean and consistent
test data. See the Vendor Landscape: Enterprise Test Data Management Forrester report.
18
For an overview of release automation tools, see the Market Overview: Application Release Automation Tools
Forrester report.
19
20
For more information on how DevOps improves application delivery governance, see the Use DevOps And Supply
Chain Principles To Automate Application Delivery Governance Forrester report.
21
For an evaluation of popular release automation tools, see the The Forrester Wave: Application Release
Automation, Q2 2015 Forrester report.
22
Measuring customer experience is a complex subject. For some of the latest insights on the topic, see the How To
Measure Emotion In Customer Experience Forrester report.
Microservices and containers are hot topics. For more information, see the Microservices Have An Important Role In The
Future Of Solution Architecture Forrester report and see the Nine Questions To Ask About Docker Forrester report.
23
Blue-green deployment is a release technique that reduces downtime and risk by running two identical production
environments called Blue and Green. At any time, only one of the environments is live, with the live environment serving
all production traffic. Canary deployments are a pattern for rolling out releases to a subset of users or servers. The idea
is to first deploy the change to a small subset of servers, test it, and then roll the change out to the rest of the servers.
24
25
For more information on this topic, see the Agile Experts Focus On Downstream Delivery Forrester report.
26
For more information on how a common platform enables faster application delivery, see the Case Study: Fidelitys
Journey To A Common Platform For Global Scale And Efficiency Forrester report.
19
We work with business and technology leaders to develop

customer-obsessed strategies that drive growth.
PRODUCTS AND SERVICES
Core research and tools

Data and analytics
Peer collaboration
Analyst engagement
Consulting
Events
Forresters research and insights are tailored to your role and

critical business initiatives.
ROLES WE SERVE
Marketing & Strategy
Professionals
CMO
B2B Marketing
B2C Marketing
Customer Experience
Customer Insights
eBusiness & Channel
Strategy
Technology Management
Professionals
CIO
Application Development
& Delivery
Enterprise Architecture
Infrastructure & Operations
Security & Risk
Sourcing & Vendor
Management
Technology Industry
Professionals
Analyst Relations
CLIENT SUPPORT
For information on hard-copy or electronic reprints, please contact Client Support at
+1 866-367-7378, +1 617-613-5730, or clientsupport@forrester.com. We offer quantity
discounts and special pricing for academic and nonprofit institutions.
Forrester Research (Nasdaq: FORR) is one of the most influential research and advisory firms in the world. We work with
business and technology leaders to develop customer-obsessed strategies that drive growth. Through proprietary
research, data, custom consulting, exclusive executive peer groups, and events, the Forrester experience is about a
singular and powerful purpose: to challenge the thinking of our clients to help them lead change in their organizations.
113382
For more information, visit forrester.com.

Boost Application Deliver

Diunggah oleh

Informasi Dokumen

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Boost Application Deliver

Diunggah oleh

Hak Cipta:

Format Tersedia

FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS