Hold Core Beliefs -- Part II

In my previous post, "Hold Core Beliefs -- Act on Them", I outlined three core beliefs we act on at my organization.  

• Always be delivering
• Not All Builds are Created Equal
• Consume Your Own Output

This post will will cover the second -- Not all Builds are Created Equal

Build Types

When you make every one of your builds consumable, you open yourself to all kinds of feedback.    Several years ago, I was a bit naive on this topic.   When reviewing my own team's automated test results with one of my consumers, I tried to explain that if ruled out 75% of the builds we produced, I was assisting his team in which builds they should consume.   His response (assuming he was looking at build results instead of automated test results), was, "This is terrible.    I have worked on complex systems before and it built more often than this."   Another consumer said, "I don't understand why each build isn't perfect". 

It was at this time I began studying the project organization and development operations at Eclipse.    They have a nice summary of the different types of builds that they attempt to produce.  You can see the entire list at Eclipse.org, but  In a nutshell they range from:

• Release -- Releases are the right builds for people who want to be on a stable, tested release, and don't need the latest greatest features and improvements
• Stable - builds that have been found to be stable enough for most people to use, such as Milestones and Release Candidate builds
• Integration -- Integration builds are built from these stable component versions that have been specified by each component team as the best version available.
• Maintenance -- for maintenance of a Release
• Nightly -- They are completely untested and will almost always have major problems. Many will not work at all.

It was the bold phrase of the nightly build that I quoted to the consumer that expected perfection from my team.   If a well funded project like Eclipse did not attempt perfection with each and every build, why he expect that from my team?   He still did (or at least said that he he did) and eventually left the company.

While my team has not distinguished between build types, we have captured the essence of "Not all builds are equal" with a concept we call "Build Maturity".    We take every build and put it through a series of increasingly complex automated tests.   As it passes the next more complex test suite, we set a higher maturity.  If we attempt the next round of testing and it does not pass, we record that it failed on this maturity attempt and stop all further testing.    This is very similar to the "Fast Fail" concept of a build system.  The build maturity is fully visible to anyone that wants to consume a build.

It is from the highest level of Build Maturity that our Consumers spend most of their time writing bugs against.    The focus then shifts to addressing all bugs that would stop the product from shipping, while each build is still put through the battery of automated testing to ensure that the Maturity has not regressed.   A release can come from anyone of these builds.

Build Feedback from Consumers

You should also have an issue tracking system and be willing to let anyone report anything to you in this system.  If you ask for feedback you will get it.   However, if you get it, you will need to act on it.  To act on it, your developers will need an issue tracking system that they enjoy using.   They will enjoy it if the tool you choose encourages a collaborative mindset rather than a "bug fix" compliance mindset. Even better if you can use the same tool to distinguish and track feature development.  Collaboration is increased and feedback becomes more fact based rather than emotional statements like, "that build is bad", "that build is buggy", or "that feature is clearly not done."

This kind of collaboration takes trust, understanding, and time.   But with time you can develop a common mindset towards an agile software release process that hits target dates more frequently and produces higher quality software.   Who wouldn't signup for a project with those outcomes?