Push on Green Deployment – (Brain Dump)


The other day I was listening to Kent Beck in this podcast where he was talking about, amongst other things, push on green deployment, basically the idea is that once your test suite goes green, you deploy.

When I heard  this originally I thought :

1) Think about all possible problems -> not only technical( this component is not compatible with the component just rolled out, etc, etc)  but also processes ( and people using them ) this is a real concern, because the pace of change would increase I would guess)

2) What tests? Who wrote them? I would think higher level tests? actually all tests? Possibly the upside of this is that companies that produce quality software will become incredibly popular ( “My software doesn’t brake, we test throughly” could actually be part of the marketing and backed by actual tests, imagine that)

3) The premise for push on green is basically to push Software Development practices up a notch, so that as a developer we can say we are delivering a feature on a particular date and we can do that confidently, How does push on green help us deliver as we promised, would it makes us slower?

Deployment is a painful process, and often neglected topic, as a developer how often do you think about it? And worse, How many times you just don’t know what will happen? Most importantly. How stressed do you get close to going live? (I guess this is the time when we all start thinking about it seriously)

Great, now imagine What if that would happens every hour?

In the beginning it would be scary, but then, the practices need to be polished enough so it becomes something normal, that you are not only used to , but confident about it. After all we all become better at things with practice.

Do i like the idea? Yeah! and Its as scary as it gets 🙂 Lets look into it more in depth

So how can this possibly work?

If I was going insane(er?) and really wanted to do this, as a C# developer, what is possible?

Maybe I would start of an continuous open beta or, edge (somehow warning my users of the fact that the code they are using is “fresh out of the oven”) and once I see some stability (perhaps people could vote for it or similar)then I can fully deploy.

But then thats is just my application, lets imagine I produce code that is not public facing, but that , its an API, consumed by many other applications, how would they be aware and react to interface changes? More importantly, would this make software evolution slower  by persisting everlasting APIs? or would it induce APIs that are very noisy, hence harder to discover?

I m asking this because:   lets forget about the barrier of web or desktop for a minute and focus on dlls, where they live and how we use them and consume them.

If my Operating System vendor/framework provider makes a push do i want my app to get that immediately?

Say I m using an interface something like IEmailSender, that has the methods Send(IEmailTemplate, ISomethingElse) and that is replaced for the better designed IEmailSender.Send(Message) with the Send(Message[]) overload. In that example I agree with the change, I want to use the new code because the contract is better (lets assume we all agree is better ok); however, that would mean changing my application, re testing all those areas where I use IEmailSender and I most likely don’t want that.

I have to add, I would probably want to do an integration test anyway, or should I blindly trust the new IEmailSender implementation because we are all pushing on green now?

So the option here seems to be, create the new contract but maintain the old one around for a while right?, but then your API will have all these Send methods, and that doesnt seem to be going into the better development practices direction

Next I m gonna try heroku in anger and see if I get more ideas from using it, heroku is a service that runs ruby applications that are deployed when you commit.

This is a brain dump on this and would love to hear thoughts from other people, corrections, feedback, etc always welcome 🙂

5 Responses to “Push on Green Deployment – (Brain Dump)”

  1. 1 Andrew

    I guess it really depends on what kind of app you are talking about. Your own web app, where you can freely push out to your own server(s) may be ok. And may make total sense (in fact, don’t sites like StackOverflow do that sort of thing already?) Self contained client apps also probably doable. But automaticallly pushing out changes to libarary that other apps depend on? Not so sure…

    From a technical perspective, I’m sure it’s possible to do. But I would imagine it would require strong versioning of assemblies so the API you depend on doesn’t just dissapear one day (I’d rather have different versions of the same dll, then a dll containing new and obsolete methods)

    Actually just thinking about standalone client line-of-business apps. I’ve worked in places where a new version of anything required user retraining. So, just wouldn’t be a runner. Also, it may work fine in your tests but lots of apps support user customization and it may not be possible to pre-test for every scenario. Some apps will always need an ‘acceptance test’ phase imo.

    As you can tell, not sold on the idea for non-web stuff 🙂 But definitely a different approach and one that would no doubt increase test coverage! Will be interesting to hear how you get on.

  2. 2 Rob

    Nice post – and “food for thought”..

    I think it definitely depends on what you are deploying and where.. I think we all agree in most cases, Web will be the easiest. However, as you say, once we start getting into desktop/public APIs – I think the process could easily start to break down..

    That said – I think the fact that this is so god damn scary is a bit of a turn-on 🙂
    For many – TDD is/was scary because it would mean you see the ugly truth about what a mess a lot of your beloved code is right? But we bite the bullet, start adding tests, refactoring code etc. We suffer some pain because we want to *be better*.

    I think this could very much be a similar situation – wouldn’t it be cool if software was actually improving on a daily basis? Even if you didn’t actually drop to live on every commit or whatever, but just to know that the deployment process is so smooth, you *could* deploy to live at the click of a button?

    Obviously, this kind of practice would have to have some requirements. High test coverage, well organised release mechanism (e.g. DB schema update scripts etc.) and a system that tolerates frequent updates (e.g. app cache being blitzed by frequent writes to binaries) and so on..

    TBH, would love to see more in this direction – but there is a lot of risk to be negated and I am not sure really many people are able to stomach it!

  3. Just to point out that this is the logical progression of Continuous Integration, and is also known as Continuous Deployment – http://radar.oreilly.com/2009/03/continuous-deployment-5-eas.html. Personally, I think this could be a very powerful approach to the automation of software deployment.

  4. This is all well and good from a technical stand point. But surely the fact that “all green” doesn’t mean the product is fit for production deployment.

    How are regression and UI testing factored into this approach?

  5. 5 roundcrisis

    Hi :
    well, I think that the green mean regression and UI test are green too.
    So actually this is code that you would consider production ready, perhaps actually you only push when the acceptance tests ( ie the BDD style tests are passing , hence you consider done, hence you are ready to deploy)
    Does this answer the question?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: