The Cost of Continuous Delivery

Continuous delivery is a process in which developers can land features on master (or trunk if you're on subversion), have it run through continuous integration (unit tests, user interface tests, service layer tests), have it propagate through the various stages of your deployment pipeline and have it on production in short matter of time. It sounds great because you'll never really need an ops team anymore and really dispels the need to have iterations (in 2 weeks, a feature lands, instead, a feature lands when it lands).

A build pipeline like that saves money for the company in that respect and makes it so that engineers can react more quickly to problems without having to go through layers of communication; however, it surfaces many more issues. As a software engineer on a fast pipeline that is continuously delivering to customers, it is increasingly more important to have test coverage. Below is a list of aspects of testing that need to be seriously thought of

• Unit Testing
• Web Service Testing
• User Interface Testing

Taking a step back, in order to really take continuous delivery seriously, we should always look at the true value of having automated tests. Here are some of the core principles of automation testing:

• To protect our users from misbehaviours in the system (never cause distrust between the product and the customer)
• To be vigilant on any changes to the source tree (new and legacy code can be fractured easily)
• To be able to verify functionality in an automated fashion without increasing engineering manpower (reducing costs -- people are the most expensive aspect of running a business)

With the above, automated tests in a continuous delivery model should now a part of the feature itself. Without tests, it is not a feature and it should not be able to land on production as it would be too much of a risk to the product and the customers. The most difficult part of absorbing the cost of continuous delivery is convincing the stakeholders that the new model is faster but needs more checks along the pipeline. This cost of continuous delivery (Jidoka) includes

• spending time writing unit tests
• spending time writing service layer tests (assuming your platform follows SOA, validating inputs/outputs of data)
• spending time writing user interface tests that detect defective user flows (registration, updating profile, etc)

In the end, following the guiding set of principles of Jidoka allows your engineering teams to produce high quality features by detecting defects along the pipeline and ensuring there's fixes in place to keep the pipeline pushing forward.

Good luck and have fun,
Jaime Bueza

Jaime Bueza is a software engineer in Vancouver, British Columbia, Canada. He has developed web applications for Nintendo, Starbucks, Bacardi, Nike, Electronic Arts, Ritchie Brothers, Kiwi Collections, Cox Communications and Microsoft. When he's not developing useful software that constantly evolves with business requirements, he's writing open source software and helping developers in the Windows Azure NodeJS community.