Agile Development

ARKANSAS BLUE CROSS BLUE SHIELD RSFOX@ARKBLUECROSS.COM

(501) 396-4160 Tuesday 9/25, 8:40 - 9:20

Biography
Robert Fox - Data Architect, Arkansas Blue Cross Blue Shield

17 years of data architecture and warehousing experience in the finance, telecom, and health insurance industries. Installed over 50 data warehouses worldwide, some loading more than 4.5 terabytes of new transactional data per day. Guest lecturer in information management masters degree programs at 3 universities. Speaker at national conferences for Oracle, Fidelity, and MDM-CDI.

Abstract - Agile IT is about finding balance

The term Agile Development is used in several ways. Most commonly, the phrase is used in a narrow sense to refer strictly to the development phase of a software development project. In this presentation, I will be using the phrase in a much larger sense: creating an IT organization that is agile in ALL of its parts. I think the term Agile itself is misleading. According to the dictionary, agile means nimble, quick, and flexible. This definition makes it appear that this agile thing we are seeking is an extreme, the opposite of rigid. Many of the proponents of agile development take this view. I believe, however, that agility is not an extreme, but a balancing act between extremes: balancing the desire for continual user feedback with the need to let developers remain focused balancing the need for predictability with the need for flexibility balance the development of strategic, technical infrastructure with the need to respond to immediate business requests balancing the need to provide self-service interfaces with the need for controlling the quality of data and processes All too often, organizations adopting agile development completely abandon their existing processes (which were unbalanced in one direction) in order to adopt an unbalanced polar opposite process. Ideally, mature agile development is about finding the correct balance across the entire organization.

Agenda
Balancing the Building Blocks of an Agile IT Organization
Organization Structure

Agile Development Methodology

Balancing the need for continuous feedback with the need to keep developers focused
Organization Structure

Traditional Development
a.k.a. waterfall, or release
Analysis Design Develop Test Implement
Works well for: Large, complex projects involving many inter-related components or departments Projects which are foundational to future projects

PROs

Supported by most project management software Very clear, up-front understanding of entire timeline Clear lines of responsibility

CONs
Sequential Not iterative Requestor not involved in most of the process Long time between request and delivery

Agile Development
Time-boxed scope

PROs

Cross-functional development
Works well for: Small projects which can be completed in isolation from other resources and projects Projects which, while perhaps critical on their own, are not critical to future projects Small development shops where everyone is working on the same project

Very rapid response to changing needs Requestor involved throughout process

CONs
No clear insight into long-term timeline, resource needs, cost, etc. Not well suited to most project management tools Difficult to coordinate strategy and architecture across sprints and projects Responsibilities less clear

This Is Really Two Issues Finding the right balance of interaction between requestor and developer Finding the right balance of administrative effort necessary for management insight

Lets think about them separately

Balancing Levels of Interaction During Software Development

Working in teams Can keep work from being done wrong, but can also keep any work from being done at all.

Working in isolation Can keep people focused, but they can be focused on the wrong things

Finding the Balance

We need isolation to focus, but we need to come together for vision. We need to get together as soon as there is a question, not at the next weekly meeting. We need to develop an environment where people are not reluctant to get out of their cubes and go sit down with someone else to work things out. Identify your best communicators Give them a title that recognizes and encourages this skill Make sure they are assigned to critical projects Use them to mentor other members of the team

Agile Project Management

Balancing the need for management insight with the need to minimize project overhead
Organization Structure

Why do YOU need PM? The questions isnt as simple as you might think. PM is absolutely necessary for management insight, but in many cases it has become a tool for looking over peoples shoulders because you dont trust them to do their jobs. Ask yourself, If we backed off on project management, whats the worst that could happen?

Is Oversight Needed Because of Cowboy Developers?

Will the developers play with technical toys if you dont have PM oversight?
Ask yourself it they are they willfully violating standards, or is the problem that there are no standards? Maybe you told them what the standards are, but never told them why these standards are needed Most people will do the right thing if they know what it is, and agree that its right in the long run, and its not too much harder in the short term.

Doctor Fun by David Farley, October 2002

Business Goals versus Natural Desires

Do what I tell you to do, or else

Job Security Recognition

If you do things this way, heres what will happen

Business Goal

Social/Fun
Weak Commitment

Strong Commitment

Business Goal

Family
Creativity

It could be that the reason your organization needs so much PM Oversight is that your people havent bought into the vision. Most people will do the right thing, if they understand what and why in terms that are personally meaningful

Finding the Balance Clearly document standards and processes Communicate the vision of the future made possible by these standards, and put in terms relevant to the target audience

Invest in the tools and infrastructure necessary to make following the standards easier than violating them

Oversight Needed Because of Weak Supervisors?

Are the supervisors: Incapable of adapting to changing conditions, and you need PM oversight to steer their projects for them? Incapable of locking down scope creep? Incapable of enforcing standards? In many cases, this is due to the same problem of communication of standards and vision. In some cases, however, it is not.

Agile Organization Structure

Balancing the need for creative thinking with the need for well-defined, repeatable process
Organization Structure

Predictabilty vs Flexability
Two Approaches to Project Management The Factory Floor Research and Development Balancing Lessons From Jack Understand which processes are cookie-cutter and which are not, and manage them differently Put the right people in the right roles

Jack Welch CEO of GE

Fortune Magazine Manager of the Century

Maybe its not weak supervisors, but a weak organization chart

Creative Roles Development Strategic Planning User Interface Design Training Documentation Tool Selection Solution Oriented Quality Roles QA/Testing Change Management Security Operations Audit Compliance Process Oriented

Agile Development Can Be an Organizational Issue

Project Flow
Often, development is not agile because the same resources are asked to focus on too many things. Organization should reflect functional responsibility.

We know better than to let developers test their own code. But do you know to separate creative development from rigid operations? To separate tactical development from strategic architecture?

Know when to hold em, and know when to fold em

Was it an issue with communication of standards or vision? Was it an example of misplaced skillset? Or was it a case where the person needs to be let go?
Adding permanent project management overhead should be a last resort, not a first.

Agile Architecture
Balancing the need to invest in tomorrows infrastructure with the need to meet the demands of the business today
Organization Structure

Today versus Tomorrow

Strategic Determining long range destination and planning a path to it

Tactical Keeping the lights on. Meeting day to day business demands

These should be different roles Navigating Piloting

Finding the balance How do you balance the need to get things done quickly and cheaply with the need to prepare for the future?
Dedicate resources to developing and maintaining a 3-5 year strategy for ALL the functional areas of your organization. Include both what the future looks like, and a tentative roadmap to get there. During budgeting process each year, make sure to ask for funds for any IT-sponsored, strategic projects. When tactical business requests come in, analyze each to see how it can be implemented in a way that aligns with the long term strategy.

Agile Business Intelligence

Balancing the need to remove development bottleneck with the need to ensure data is used appropriately
Organization Structure

Agile Means IT is Not a Bottleneck

Enterprises where IT addresses more than 20% of BI requirements will continue to see the snowball effect of an evergrowing BI requests backlog
Boris Evelson - Forrester Research

BI requirements change faster than an IT-centric model can keep up

Jim Ericson - Editorial director of Information Management

Maybe what we should be looking at isnt turning around IT projects faster, but instead architecting a managed platform where the business is able to self-serve their information needs using safe information.

Finding the balance Implement business rules in the database, not in business reports Build specialized data marts for specific data domains Provide awesome meta data Use tools that can explore prebuilt dimensional cubes, and drill through to prebuilt detail reports Clean up your data Make it easier for your users to user your vetted data than to try to pull it themselves Stamp out private copies of data

Summary Agile IT Organizations

Balancing the need to remove development bottleneck with the need to ensure data is used appropriately
Organization Structure