"Agile software development is about iteration not oscillation"
Jim Highsmith (paraphrased), Agile 2009


"Kanban is the science of not trying to do too much at once"
Stephen Palmer, 2012


"We try to solve the problem by rushing through the design process so that enough time is left at the end of the project to uncover the errors that were made because we rushed through the design process"
Glenford Myers (via jJeff De Luca)


"We apply the analytic procedure in order to create a conceptual framework within which we can draw conclusions about the means by which a system solves its tasks. Indeed, we do this with the express purpose of establishing a solid foundation from which we can carry out a subsequent synthesis. This synthesis, in turn, acts to verify the conceptual model as an explanation. The process is an iterative one."
Tom Ritchey, Refactoring, 1991,


"Man will occasionally stumble over the truth, but most times he will pick himself up and carry on."
Winston Churchill


"It usually takes more than three weeks to prepare a good impromptu speech."
Mark Twain


"chaos often results when a project's complexity is greater than its managers' ability to direct meaningful progress toward a goal."
Ken Schwaber, Agile Project Management with Scrum, 2004


"Human brain capacity is more or less fixed, but software complexity grows at least as fast as the square of the size of the program."
Gerald M. Weinberg, Quality Software Management: Volume 1 Systems Thinking, 1992


"The complexity of software is an essential property, not an accidental one. ... Many of the classical problems of developing software products derive from this essential complexity and its nonlinear increases with size."
Frederick P. Brooks, The Mythical Man-Month, 1995


"As the number of people increases, the ways they can interact tend to multiply faster than you can control them."
Gerald M. Weinberg, Quality Software Management: Volume 1 Systems Thinking, 1992


"When people are factored in, nothing is simple. The complexity of individuals and individuals working in teams raises the noise level for all projects."
Ken Schwaber, Mike Beedle, Agile Software Development with Scrum, 2002


"Use models to find out how things work or to find solutions to puzzling dilemmas. Create models to communicate ideas and understand things you can’t see. Recognize models and the countless ways models are used for working, playing, teaching and explaining. Assess models for what they do and don't tell you about the real thing and how useful they are"
Boston Science Museum


"Simplicity is the final achievement. After one has played a vast quantity of notes and more notes, it is simplicity that emerges as the crowning reward of art."
Frederic Chopin


"If you can't explain it simply, you don't understand it well enough."
Albert Einstein


"The structure of a software system will reflect the communication structure of the team that built it."
R. E. Fairley


"I guess that as you go along you try to fill your tool-box, so that when you face these circumstances you have more options to choose from"
Jonny Wilkinson, Tom Fordyce's Blog, 2009


"Speech is conveniently located midway between thought and action, where it often substitutes for both."
John Andrew Holmes


"Information Technology is 80% psychology and 20% technology "
Jeff De Luca, www.nebulon.com


"Deliver frequent, tangible, working results"
Peter Coad


"Perhaps the worst software technology of all time was the use of physical lines of code [for metrics]. Continued use of this approach, in the author’s opinion, should be considered professional malpractice."
Capers Jones, Applied Software Measurement


Notes on eXtreme programming

At the turn of the century, while many in the industry lost sleep over the millennium bug, a wave of new approaches to team-based software development appeared over the horizon. The first to break on the beach of mainstream awareness was Kent Beck's very cool sounding eXtreme Programming or XP for short (no relation to the similarly named version of Microsoft Windows).

In a Nutshell

As described in the first edition of Kent Beck's book, Extreme Programming Explained, published in 1999, eXtreme Programming is originally based on twelve best practices, four core values, and an options-based economic model.

The premise behind eXtreme Programming is that the cost of making changes to software does not need to grow exponentially more expensive throughout a project or product development life-cycle. That the cost of change does increase exponentially over-time is attributed to unnecessary complexity added early in the project product life-cycle; complexity that provides the flexibility to address all the changes that might be asked for in the future.

The assumption in eXtreme Programming is that few of the potential changes catered for early in the life of the project are ever requested. In contrast changes that were not predicted are requested and the unneeded complexity designed to cater for changes not asked for impedes the ability to cater for the unforeseen changes that are asked for. The argument is that time invested adding flexibility early in the project frequently does not realize its expected return on that investment. Therefore, it is better postpone that investment, do the simplest thing that works now, and defer the decisions to add complexity/flexibility until it is proven that they are actually needed.

For this reason, the core values and best practices of eXtreme Programming are designed to keep the cost of change relatively constant throughout a project. If this is achieved then instead of trying to minimize the amount of change later in a project, late changes are accommodated and even welcomed. If the cost of change is reasonable, then a software development team can 'embrace change'.

The four original values of eXtreme Programming are:

  1. Communication - the eXtreme Programming practices are designed to keep communication flowing within the development team and with stakeholders.

  2. Simplicity - eXtreme Programming bets that doing the simplest thing that works today and paying tomorrow if a change is needed, is cheaper than doing a more complicated thing today that adds flexibility that may never be needed.

  3. Feedback - concrete feedback based on production-quality, working code enables better communication.

  4. Courage - the courage to make changes both supports and is supported by the other three values.

The second edition of Extreme Programming Explained adds a fifth value, respect; respect for other members and roles within the project team and respect for all interested parties.

The twelve original best practices of eXtreme programming are:

  1. The Planning Game - The business and development work together to produce a list of estimated, prioritized user stories for development to work on. The business is responsible for scope, priority, and composition and date of releases. Development is responsible for estimates, communicating consequences of technical choices, process, and scheduling of work within releases. The Planning Game is repeated after each iteration and release. A very short book, Planning Extreme Programming, by Martin Fowler and Kent Beck published in 2000 explains this crucial aspect of eXtreme Programming in a little more practical detail.

  2. Small Releases - The aim of the Planning Game is to produce an initial overall plan comprising of small releases each delivering the currently outstanding requirements of most value to the business.

  3. Metaphor - the metaphor is eXtreme Programming's replacement for an initial, high-level architecture, and probably the vaguest, most misunderstood and under-used practice. It is an overall guiding statement for the project. In Domain-Driven Design and Feature-Driven Development, a domain model performs the equivalent role.

  4. Simple Design - select the simplest design that works. Don't design for tomorrow if tomorrow's needs are not fully known.

  5. Testing - provide comprehensive automated unit and functional tests for every feature. The idea is to increase developer and stakeholder confidence in the software making refactoring easier and enabling iterative development.

  6. Refactoring - refactoring is changing the internal design and implementation of features without changing the externally visible behaviour of that code. eXtreme Progrmaming advocates frequent refactoring to simplify design and code to make it easier to add new features.

  7. Pair Programming - all production code in eXtreme Programming is written by two people working together at one workstation. This is intended to provide a form of continuous design and code peer review catching many errors in design and programming logic as they are made.

  8. Collective Ownership - any programming pair may edit any of the code base of the project at any time they need to. This practice is designed to avoid delays caused by requesting, explaining and waiting for someone to make a needed modification to 'their' code.

  9. Continuous Integration - every few hours, the whole system is built and all the automated tests run against it to prove that recent changes have not broken anything or clashed with anyone else's changes. This practice has matured considerably since eXtreme Programming first arrived on the scene. Proponents of the practice have produced a number of popular tools and books devoted to the topic, including Paul Duvall's,  Continuous Integration and more recently Jez Humble has written about extending the practice to encompass continuous delivery.

  10. 40-hour week - to sustain productivity on an iterative project requires team members work at a sustainable pace. eXtreme Programming mandates a 40 hour maximum working week. If a team really must work longer one week to achieve a project goal, then the subsequent week should include slack time to enable the team to recover to full potential.

  11. On-site customer - communication is best done face-to-face and immediately the need arises. Someone permanently sitting with the development team able to answer questions about requirement details eliminates a large amount of communication overhead and delays caused by communicating formally to someone remotely located from the project team.

  12. Coding Standards - collective ownership and pair programming makes it essential that code is written and formatted consistently to an agreed set of conventions.

In the second edition of , eXtreme Programming Explained, Kent Beck did some serious refactoring of the best practices. eXtreme Programming now comprises 13 primary practices with 11 corollary practices.

Primary Practices

  1. Sit Together - everybody involve in the project should be located in the same open space office area.
  2. Whole Team - otherwise known as cross-functional teams; the project team needs to include people with all the skills necessary to complete the project.
  3. Informative Workspace - large, frequently-updated wall posters, lists, pin-boards and charts are used to communicate status and progress among other items
  4. Energized Work - replaces the 40 hour working week practice with something more comprehensive
  5. Pair Programming - unchanged
  6. Stories (user stories) - the use of user stories as a tool for requirements elicitation, elaboration, planning and progress reporting remains unchanged
  7. Weekly Cycle - work in weekly iterations
  8. Quarterly Cycle - the equivalent of other agile advocates concept of release planning. A regular, planned opportunity to step back and perfrom high-level planning, look at improving the process, remove bottlenecks, etc.
  9. Slack
  10. Ten Minute Build - mainitinaing the ability to build and test the system under development within ten minutes so that develoeprs can do this frequently.
  11. Continuous Integration - unchanged
  12. Test First Programming - write tests before writing production code
  13. Incremental Design - same as simple design but with an emphasis on doing a little design every day

Corollary Practices

  1. Real Customer Involvement
  2. Incremental Deployment
  3. Team Continuity
  4. Shrinking Teams
  5. Root Cause Analysis
  6. Shared Code
  7. Code And Tests
  8. Single Code Base
  9. Daily Deployment
  10. Negotiated Scope Contract
  11. Pay Per Use

Strengths

Most of the strengths of eXtreme Programming are almost taken for granted within modern software development approaches now. This was not the case when eXtreme Programming, Scrum and Feature-Driven Development (FDD) where first formulated and developed in the late 1990's.

Highly Iterative

All of the self-styled agile processes including eXtreme Programming divide work into short development iterations or timeboxes of between a few days and four weeks. This has a number of benefits over processes comprising a sequence of lengthy phases:

  • Rapid feedback from short iterations can radically reduce the distance between the introduction of errors and their detection.
  • Short iterations enable the team to experience the feeling of finishing something on a frequent basis, improving job satisfaction and increasing team morale.
  • Short iterations enable a team to get into a habit-forming rhythm.
  • Short iterations provide an opportunity to improve the way the team works together.

Small, Cross-functional Teams

Similarly, all agile approaches emphasize the need for cross functional teams. Like many of these practices, they are not new ideas.  Cross functional teams were proposed by Marlin Hills as surgical-style teams lead by a Chief Programmer and promoted by Fred Brooks in his book, The Mythical Man-Month , first published in 1975, as a means of maintaining the conceptual integrity of a design. eXtreme Programming must be given some credit for pushing this idea once again.

Automated Testing

eXtreme Programming's emphasis on automated testing coupled with the availability of the xUnit family of testing frameworks has without question transformed the unit testing landscape. xUnit-style automated unit testing is now almost taken for granted as a best practice by most development teams. The question is no longer whether automated unit tests should be written but on keeping the tests running quickly and evaluating whether or not tests are earning their keep by providing more value in terms of defect detection than the cost of writing and maintaining them.

Continuous Integration

Improvements in computing performance have enabled teams to quickly and cheaply build and regression test their entire system whenever they want to. Unfortunately the name for this practice is a poor one. The integration is not without interruption as the name implies; it is not continuous. Continual Integration would be a more accurate name.

Weaknesses

Pair Programming

The most controversial practice within eXtreme Programming is pair programming. I have worked on two projects that professed to be using eXtreme Programming. In both cases, the teams had dropped the use of pair programming by the time I joined the project; one team had dispensed with pair programming completely and the other used it only for user stories where they had not done something similar before.

Testing checks quality attributes that are visible to an end-user such as correctness, robustness, and performance. Testing does not check quality attributes that are visible only to developers such as code simplicity, loose coupling, high cohesion, appropriate encapsualtion, compliance with best practice coding conventions, good use of design patterns, etc. Dropping pair programming eliminates a key quality assurance mechanism for these internal quality factors.

Static analysis tools can check and catch an increasing number of these internal quality problems when run within developers' IDEs and as part of the continuous integration build process. However, they cannot catch the more important, more subjective items or identify logic problems. Some form of design and code review or inspection is needed to catch these. Pair programming, while almost certainly not as effective as well run traditional design and code inspections, is eXtreme Programmings peer review mechanism. Without it, the ability to catch important internal design and coding quality issues early is largely lost. And as Bertrand Meyer states in Object-Oriented Software Construction, "In the end, only external factors matter. ... But the key to achieving these external factors is the internal ones."

Where pair programming is practiced properly, it is not a like-for-like replacement for more structured, well run, design and code reviews. Design and code reviews give a chance to step back and pause before the review, they add additional eyes and brains for a short, sharp focused period of time.

Ironically, frequently having two pairs of eyes looking at something that only requires one, could be argued is a form of waste that Lean Software Development requires be removed to increase productivity.

Architecture

The metaphor practice is vague; probably deliberately so. The domain model-oriented approaches such as Domain-Driven Design and Feature-Driven Development are stronger in this area especially when it comes to enterprise software systems for complex business areas such as global commercial financing and insurance.

Scale

eXteme Programming is designed for small co-located teams. Scaling to larger teams and projects or projects involving of multiple development teams puts practices such as collective ownership under unsustainable pressure.Feature-Driven Development's feature teams provides an alternative way to structure larger efforts.

Other weaknesses of eXtreme programming are not so much in what it promotes as best practice but in what it enables developers to easily get away with in terms of poor practice. For example, it is very easy for developers to substitute 'simplest design that works' for 'first design that springs to mind'. In addition, it is too easy for developers to avoid delivering necessary supporting documentation using the excuse that the source code including test code is all the documentation needed. This is simply not true.

Conclusions

In revolutions aimed at countering extremely dysfunctional behaviour, the tendency is to take the counter proposal to the opposite extreme. This is true of eXtreme Programming and makes its name somewhat ironic. For many teams, eXtreme programming is simply too extreme to be a practical solution over the medium to long term. The doubling of the number of practices to over twenty has not helped either because it makes the whole thing much more unwieldy to learn and remember.

Today the popularity of eXtreme Programming is in decline, being largely superseded by Ken Schwaber and Jeff Sutherland's Scrum approach in terms of mainstream popularity. However, it still heavily influences many of the design and coding practices that Scrum teams use.

There is a vast difference between a product that forces you to change the way you work and one that inspires you to work differently. Being forced to do it 'their' way is uncomfortable and usually unproductive. The freedom to discover better ways of working is more enjoyable and inspires new insights into other approaches to get your work done.
Andy Carmichael, Better Software Faster, 2002

Scrum

Scrum splits a project into a number of fixed length time boxes or iterations called sprints. At the beginning of each sprint the project team meets to select a subset of items from a priortised to-do list known as the product backlog. The subset of items chosen from the product backlog are the items that provide the most business value for the customer and which the team believe they can complete by the end of the sprint. This selection process is known as Sprint Planning.
Read more...

Feature-Driven Development

FDD combines key advantages of other agile approaches with model-centric techniques and fluid team organisation. This means that FDD easily scales to much larger teams and projects than generally recommended for other agile approaches. FDD is also characterised by an emphasis on quality throughout the process, and timely, accurate, meaningful status reporting and progress tracking for all levels of leadership.
Read more...

Domain-Driven Design

Domain-Driven Design comes as a bit of a surprise to those who have been taught that agile software development and especially eXtreme Programming do away with the need for analysis and design techniques like object modeling and notations like the Unified Modeling Language (UML). It is actually very obvious that any software team working in a .Net language, Java, Objective-C, C++ or any of the object-oriented dynamic languages such as Python or Ruby has an object model at the heart of their software.
Read more...

Follow me on Twitter...