21 Scott Ambler - Agile Data Workshop

IBM Software Group
Agile and Data:

Strategies for Scaling Agile Software Development
Scott W. Ambler Practice Leader Agile Development
2008 IBM Corporation
IBM Software Group | Rational software
Scott Ambler - Background

Practice Leader Agile Development Senior Contributing Editor, DDJ Fellow International Association of Software Architects www.ibm.com/rational/bios/ambler.html www.ibm.com/developerworks/blogs/page/ambler
Agenda
Introduction Comparing Methodological Approaches Evolutionary/Agile Data Modeling Database Refactoring Database Testing Conclusion
Agenda
Introduction
Warning! Introduction to Agile Software Development Agile Adoption and Success Rates
Comparing Methodological Approaches Evolutionary/Agile Data Modeling Database Refactoring Database Testing Conclusion
Warning!
Im spectacularly blunt at times Many new ideas will be presented Some may not fit well into your existing environment Some will challenge your existing notions about software development Some will confirm your unvoiced suspicions Dont make any career-ending moves Be skeptical but open minded
What is Agile?
An iterative and incremental (evolutionary) approach performed in a highly collaborative and selforganizing manner with just the right amount of ceremony to frequently produce high quality software in a cost effective and timely manner which meets the changing needs of its stakeholders.
Core principles
Fits just right process Continuous testing and validation Consistent team collaboration Rapid response to change Ongoing customer involvement Frequent delivery of working software
How Agile is Different

Focus on collaboration:
Less paperwork and more conversation Stakeholders actively involved
Focus on quality
We have a full regression test suite for our systems We develop loosely-coupled, highly cohesive architectures We refactor to keep them this way
Focus on working software:

Greater feedback makes agile projects easier to manage Less documentation is required Less bureaucracy
Agilists are generalizing specialists:

Less hand offs between people Less people required Specialists find it difficult at first to fit into the team
Agile is based on practice, not theory:

This is a significant change from traditional You need to see how agile works in practice to truly understand it
Agile Values
www.agilealliance.com
We value: 1. Individuals and interactions 2. Working software 3. Customer collaboration 4. Responding to change
Over: 1. Processes and tools 2. Comprehensive documentation 3. Contract negotiation 4. Following a plan
Agile Principles
1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.
www.agilealliance.com
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. Business people and developers must work together daily throughout the project. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Working software is the primary measure of progress. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. Continuous attention to technical excellence and good design enhances agility. Simplicity--the art of maximizing the amount of work not done--is essential. The best architectures, requirements, and designs emerge from self-organizing teams. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Survey Results
Agile Adoption Survey
February 2008 Message sent out to DDJ mailing list 642 respondents: 54.8% were developers, 29.4% were in management 41.6% had 10-20 years IT experience, 37.2% had 20+ years 37.7% worked in orgs of 1000+ people 71% worked in North America, 17% in Europe, 4.5% in Asia
Project Success Survey

August 2007 Email to DDJ mailing list 586 respondents 54% were developers/modelers, 30% were in management 73% had 10+ years in IT 13% worked in orgs of 1000+ IT people 84% worked in commercial firms 69% North American, 18% European
Data, summary, and slides downloadable from www.ambysoft.com/surveys/
Has Your Organization Adopted One or More Agile Techniques?
No 31%
Yes 69%
61% of developers thought they were doing Agile 78% of IT management thought they were doing Agile 18% of respondents indicated theyre still in the pilot stage 15% of No respondents hope to do Agile this year Source: Dr Dobbs 2008 Agile Adoption Survey
Source: DDJ 2008 Adoption Survey
Why Agile/Lean? Its More Successful

Quality: 87% believe that delivering high quality is more important than delivering on time and on budget Scope: 87% believe that meeting actual needs of stakeholders is more important than building the system to specification Money: 80% believe that providing the best ROI is more important than delivering under budget Staff: 76% believe that having a healthy workplace is more important than delivering on time and on budget Schedule: 62% believe that delivering when the system is ready to be shipped is more important than delivering on schedule
Source: Dr Dobbs 2007 Project Success Survey
Agenda
Introduction Comparing Methodological Approaches
Waterfall Mini-Waterfall Agile Discussion
Evolutionary/Agile Data Modeling Database Refactoring Database Testing Conclusion
Organize Into Groups

Please form groups of 4-6 people
Smaller is better, but you need at least 4 people
If possible:
Work with people you have never met before Work with people from different organizations Have fun, but stay focused on the assignments
Greeting Card Assignment: Traditional

1. 2. 3. Assign roles: Stakeholder, Business Analyst, Designer/Developer(s), Tester The stakeholder must come to the front of the room for directions. Everyone else, get to know each other and read the rest of this slide. Follow these steps to create the greeting card:
1. Business analyst interviews stakeholder (2 minutes), producing requirements document. Everyone else may listen but not speak. 2. Business analyst hands over requirements document to development team (designer/developers). BA and Stakeholder now quiet. 3. Development team creates the card (2 minutes). Tester watches. 4. Development team provides card and original requirements document to the tester. 5. Tester inspects card and suggests improvements (30 seconds). 6. Development teams fixes card (30 seconds). 7. Development team provides card to the stakeholder. 8. Stakeholder rates the end product.
Greeting Card Assignment: Mini-Waterfalls

1. Same roles as before 2. Organize your project into three mini-waterfalls:
1. BA interviews stakeholder to understand requirements (30 seconds) 2. Team works on card (30 seconds) 3. Tester inspects card (15 seconds) 4. Team updates card (15 seconds)
3. Stakeholder rates the end product
Greeting Card Assignment: Agile

1. Same roles as before 2. Organize the project into 3 one-minute iterations:
1. Develop the card (45 seconds). During this period you will talk with the stakeholder and create the card as a team 2. Iteration wrap-up (15 seconds). Stakeholder reviews the card and provides feedback. Team, including the stakeholder, discusses how they can work better.
Discussion
What worked best for you? Why? What are the risks with each approach? What are the trade-offs with each approach? Was there overhead between hand-offs? Why? What can you apply to the real world? Was the number of people right for each approach? Be prepared to share one or two interesting points with the class
Have 4-5 points prepared One person should be ready to represent your group
Agenda
Introduction Comparing Methodological Approaches Evolutionary/Agile Data Modeling
What is agile data modeling? The case against BMUF Agile modeling best practices An example Why this works
Database Refactoring Database Testing Conclusion
10
Agile Data Modeling

Data modeling is the act of exploring data-oriented structures. Evolutionary data modeling is data modeling performed in an iterative and incremental manner. Agile data modeling is evolutionary data modeling done in a collaborative manner. Data modeling is only one of many aspects of modeling, see www.agilemodeling.com/artifacts See Agile Model Driven Development (AMDD) www.agilemodeling.com/essays/amdd.htm
The Case Against Big Modeling Up Front

Feature Usage Within Deployed Applications
Detailed requirements specs increase project risk
Always 7% Often 13% Never 45%
Traditional change mgmt approaches are little more than change prevention strategies Stakeholders want software which fulfills their needs, not software that fulfills a spec
Sometimes 16%
Rarely 19%
The real goal is to understand and then implement requirements, not document them
Source: Chaos Report v3, Standish Group www.agilemodeling.com/essays/examiningBRUF.html
11
Best Practices of Agile Modeling

www.agilemodeling.com/essays/bestPractices.htm
Karate School Management System Example
Based on an actual organization Did the same problem two ways: Without an initial conceptual model
www.ddj.com A three-issue series
With an initial conceptual model

www.agiledata.org/essays/agileDataModeling.html
12

Iteration 1 User Stories
Maintain student contact information Enroll student Drop student Record payment Promote student to higher belt Invite student to grading Email membership to student Print membership for student Schedule gradings Print certificate Put membership on hold Maintain product information Sell product Print catalog of products Order product for inventory Order product for student Organize tournament Enroll participant in tournament Send out tournament announcement email to past participants Print tournament announcement letters to past participants
The Initial Requirements (A Karate School Management System)
Initial Domain Model
13
Iteration 1 PDM
Iteration 2 PDM
14
Iteration 3 PDM
IBM Software Group | Rational software Iter. User Stories
Enroll child student Offer family membership plan Support child belt system Enroll student in Tai Chi Support Tai Chi belt system Enroll student in cardio kick boxing Support Tai Chi belt system Support the belt order for each style Maintain product information Sell products Print catalog of products Order product for inventory Order product for student Organize internal special event (special classes, internal tournaments, ) Enroll student in special event Print special event certificate for student
Updated Requirements
6 7
15
Updated Domain Model
Iteration 4 PDM
16
Iteration 5 PDM
Iteration 6 PDM
17
Why This Works

Up front, high-level modeling defines the landscape JIT model storming enables:
You to ask more intelligent questions because you know more about the domain the longer you work in it Stakeholders to answer more intelligently because they understand the solution space better Focus on functionality that is actually needed, not what might be needed
Agenda
Introduction Comparing Methodological Approaches Evolutionary/Agile Data Modeling Database Refactoring
Database Refactoring The Process Examples Why its Hard Why This Works
Database Testing Conclusion
18
61.9% of Respondents Have Production Data Problems: Their Strategy to Address Data Source Problems
8% 18%
33%
No Strategy Don't Make it Worse Big Bang Evolutionary Other

33%
8% Source: DDJ 2006 Data Quality Survey www.ambysoft.com/surveys/
Database Refactoring
A database refactoring is a simple change to a database schema that improves its design while retaining both its behavioral and informational semantics. A database schema includes both structural aspects such as table and view definitions as well as functional aspects such as stored procedures and triggers. Important: Database refactorings are a subset of schema transformations, but they do not add functionality. www.agiledata.org/essays/databaseRefactoring.html
19
The Process of Database Refactoring
Survey: How Long to Rename a Column in a Production Database?

8% 7% 25%
11%
Single Day Within a Week Within a Month Within 3 Months > 3 Months Too Risky To Try
21% 28%
Source: DDJ 2006 Data Quality Survey www.ambysoft.com/surveys/
20
Rename Column
Example: Replace Column
21
Example: Remove View
Example: Merge Columns
22
Why DB Refactoring is Hard

Other Applications You Don't Know About
Other Applications You Know About
Your Application
Persistence Frameworks
Your Database Data Imports Data File Test Code Data Extracts Data File
Other Databases
Why This Works The database schema is evolved in very small steps in a low-risk manner Each refactoring is simple to implement It enables an evolutionary approach while supporting the existing infrastructure
23
Agenda
Introduction Comparing Methodological Approaches Evolutionary/Agile Data Modeling Database Refactoring Database Testing
Current State of Database Testing Test-Driven Development (TDD) Database Regression Testing Possible Things to Test Tools
Conclusion
Current State of Database Testing (DDJ Survey) 95.7% of organizations considered data to be a corporate asset
Of those, only 40.3% had a regression test suite in place Of those, Only 63.3% allowed developers to run this test suite when they needed to Of the organizations that didnt have a test suite, only 31.6% had discussed putting one in place
63.7% of organizations implemented mission-critical functionality in the database

Of those, only 46% had a regression test suite in place Of those, Only 66.3% allowed developers to run this test suite when they needed to Of the organizations that didnt have a test suite, only 38.6% had discussed putting one in place
24
Test/Behavior Driven Development (TDD/BDD)

Automated acceptance testing drives your detailed requirements efforts Automated developer testing drives the detailed design of the system Tests form the majority of your detailed specifications Youll still need some high-level models to provide overviews This is the first time in IT history that developers are motivated to keep the specifications up-to-date An example of single sourcing information
Database Regression Testing

Agilists develop regression unit and acceptance test suites for their applications, why not for databases too? You must test:
Within the database (clear box) At the interface to the database (black box)
You must be able to put the database into a known state, therefore you need test data (generation) May/June 2007 issue of IEEE Software
25
Things to Test For

www.agiledata.org/essays/databaseTesting.html
Database Regression Testing Tools

www.agiledata.org/essays/tools.html
DBUnit
dbunit.sourceforge.net
OUnit for Oracle

www.ounit.com
Microsoft Visual Team Studio

msdn.microsoft.com/vstudio/teamsystem/default.aspx
26
Agenda
Introduction Comparing Methodological Approaches Evolutionary/Agile Data Modeling Database Refactoring Database Testing Conclusion
Other Important Techniques Conclusion
Other Important Techniques

Configuration management of all project artifacts Encapsulate database access Train developers in data-oriented skills Train data professionals in development skills Promote the concept of generalizing specialists
www.agilemodeling.com/essays/generalizingSpecialists.htm
27
Conclusion
Evolutionary development is the norm, not the exception Agile development techniques are quickly being adopted There is no reason why data professionals cannot work in an evolutionary, or even agile, manner The software world has changed. Have you? Visit www.agiledata.org
Critical IBM Agile Resources
www.ibm.com/rational/agile/ www.ibm.com/developerworks/ www.ibm.com/developerworks/blogs/page/ambler
28
Learn more at: IBM Rational software IBM Rational Software Delivery Platform Process and portfolio management Change and release management Quality management Architecture management Rational trial downloads developerWorks Rational IBM Rational TV IBM Rational Business Partners
Copyright IBM Corporation 2008. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBMs sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, the on-demand business logo, Rational, the Rational logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
References and Recommended Reading

www.agilealliance.com www.agilemodeling.com www.agiledata.org www.enterpriseunifiedprocess.com Ambler, S.W. (2002). Agile Modeling: Effective Practices for XP and the UP. New York: John Wiley & Sons. Ambler, S.W. (2003). Agile Database Techniques. New York: John Wiley & Sons. Ambler, S.W. (2004). The Object Primer 3rd Edition: AMDD with UML 2. New York: Cambridge University Press. Ambler, S.W. and Sadalage, P.J. (2006). Refactoring Databases: Evolutionary Database Design. Reading, MA: Addison Wesley Longman, Inc. Larman, C. (2004). Agile and Iterative Development: A Managers Guide. Reading, MA: Addison Wesley McGovern, J., Ambler, S.W., Stevens, M., Linn, J., Sharan, V., & Jo, E. (2003). The Practical Guide to Enterprise Architecture. Prentice Hall PTR.
29
66% of Development Teams Go Around Data Groups: Why?

8% 19% 17%
Don't Know it Exists
Don't Know They Should Work With Them Data Group Too Difficult Data Group Too Slow
20% 36%
Data Group Offers Little Value
Source: DDJ 2006 Data Management Survey www.ambysoft.com/surveys/
Dispelling Agile Myths

Myth
1. No Documentation 2. No Modeling 3. Undisciplined 4. No Planning 5. Not Predictable 6. Does Not Scale 7. Is a Fad 8. Silver Bullet 9. RUP isnt agile 10. Not Fixed Price
Reality
1. Agile Documentation 2. Agile Model Driven Development (AMDD) 3. Requires great discipline 4. Just-in-time (JIT) planning 5. Far more predictable 6. Eclipse is agile 7. Its quickly becoming the norm 8. It requires skilled people 9. RUP is as agile as you make it 10. Agile provides stakeholders control over the budget, schedule, and scope
30

21 Scott Ambler - Agile Data Workshop

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

21 Scott Ambler - Agile Data Workshop

Diunggah oleh

Hak Cipta:

Format Tersedia

IBM Software Group

Agile and Data:

2008 IBM Corporation

IBM Software Group | Rational software

Scott Ambler - Background

IBM Software Group | Rational software

IBM Software Group | Rational software

IBM Software Group | Rational software

IBM Software Group | Rational software

IBM Software Group | Rational software

How Agile is Different

Focus on working software:

Agilists are generalizing specialists:

Agile is based on practice, not theory:

IBM Software Group | Rational software

IBM Software Group | Rational software

IBM Software Group | Rational software

Project Success Survey

Data, summary, and slides downloadable from www.ambysoft.com/surveys/

IBM Software Group | Rational software

Has Your Organization Adopted One or More Agile Techniques?

IBM Software Group | Rational software

Source: DDJ 2008 Adoption Survey

IBM Software Group | Rational software

Why Agile/Lean? Its More Successful

Source: Dr Dobbs 2007 Project Success Survey

IBM Software Group | Rational software

Evolutionary/Agile Data Modeling Database Refactoring Database Testing Conclusion

IBM Software Group | Rational software

Organize Into Groups

IBM Software Group | Rational software

Greeting Card Assignment: Traditional

IBM Software Group | Rational software

Greeting Card Assignment: Mini-Waterfalls

3. Stakeholder rates the end product

IBM Software Group | Rational software

Greeting Card Assignment: Agile

IBM Software Group | Rational software

IBM Software Group | Rational software

Database Refactoring Database Testing Conclusion

IBM Software Group | Rational software

Agile Data Modeling

IBM Software Group | Rational software

The Case Against Big Modeling Up Front

Source: Chaos Report v3, Standish Group www.agilemodeling.com/essays/examiningBRUF.html

IBM Software Group | Rational software

Best Practices of Agile Modeling

IBM Software Group | Rational software

Karate School Management System Example

With an initial conceptual model

IBM Software Group | Rational software

The Initial Requirements (A Karate School Management System)

IBM Software Group | Rational software

Initial Domain Model

IBM Software Group | Rational software

IBM Software Group | Rational software

IBM Software Group | Rational software

IBM Software Group | Rational software Iter. User Stories

IBM Software Group | Rational software

Updated Domain Model

IBM Software Group | Rational software

IBM Software Group | Rational software

IBM Software Group | Rational software