Continuous Delivery with Blue-Green Deployments

The two halves are referred to as "Blue" and "Green". For example, if "Blue" is active at any time, the next deployment switches to "Green".

In our case, the switch is made via an upstream Varnish that offers both reverse proxy and load balancing functionalities. Varnish is excellent for Blue-Green deployment because it can be switched between two configurations with one command. One configuration contains the "Blue" servers, the other the "Green" servers to which new requests are distributed. If requests are still being processed when switching, this is not a problem either: they are processed by Varnish with the "old" configuration.

One of the big advantages is the immediate switch of the whole cluster to the new version: At no time are both old and new versions active in the cluster, so our software does not have to deal with the fact that a new feature is already rolled out on half of the cluster nodes, but not yet on the other half.

In our case, both halves use the same database, so no further action is required to synchronize data during the switchover. We will write another separate blog post about how to deal with possible schema changes of the DB - just this: schema-free document stores are a very big help here.

Once you have solved the problem with possible schema changes, you get another feature as a gift: the possibility to reset the release to the previous state. Since both releases are still available on "their" servers, you can easily and quickly switch back - which we haven't had to do yet, though.

One of the disadvantages of this approach is quite obvious: we need twice as many servers. At the moment, we accept this disadvantage, but we are also discussing alternatives that would allow us to partially avoid these additional costs.

• In our system environment, all servers are virtualized. Duplicate servers do not necessarily mean duplicate hardware: Since the passive servers consume only few system resources, some savings can be made if, for example, main memory and CPUs are "overcommitted". Since such a configuration is "unsightly" and difficult to manage, we have refrained from doing so so far.
• Due to the size of our system environment, we could also set up a pool: Before a deployment, VMs from the pool are reserved, then the deployment takes place. After a certain time, the old environment is then returned to the pool. Again, the configuration is likely to be quite complex and the question is whether it would not be cheaper to keep the extra hardware on hand. In cloud environments, however, this approach could work well.
• Somewhat more seriously, we discussed the variant of configuring two application servers (Tomcat in our case) on different ports on each server: one port would then be "Blue", the other "Green". This way, you only need a little more main memory per server - but you lose the possibility to "just" install a new operating system version or make other major changes.

Overall, our experience with Blue-Green deployments has been very good so far. The deployments run smoothly, no adjustments to the software are necessary (apart from dealing with DB schema changes - but you have to solve that problem anyway) and we are actually able to go into production fully automated at ten in the morning - when all developers are in place. The ability to roll back at any time is also a confidence builder: by being able to immediately roll back to the previous release, people are more likely to dare to deploy even in a hot phase - leading to many small deployments overall, each of which is much less risky than the large, infrequent deployments of "major releases" - done nightly and controlled by an extensive process.