Distributed Builds FTW

Posted May 5th, 2016

As an engineering team, we recently decided to move towards a microservice based architecture and away from the classic monolithic application that struggles to support the modern day distributed architecture which highly available web applications require. Our goals are to write services that are isolated, small, fast, language and technology agnostic, as well as able to run inside of a container based solution (we use Docker). Ultimately we needed a distributed build system for our distributed services. In this article, we will demonstrate how we decided to use Drone CI to improve our developer productivity by reducing the lag time between a developer commit / push to git, and actually deploying that fully built microservice into our QA environment.

What We Have vs What We Need

Currently we use Jenkins for our continuous integration build and deploy tool for the main web application, which is written in C# and deployed to an IIS web server. Jenkins builds and deploys the main web application into our QA environment after a simple git commit and push from a developer. Developers and DevOps (though sometimes we prefer the title NoOps) enjoy this model as it provides a streamlined process for developers to write code and see it running as soon as possible in a QA environment.

We knew that we wanted a similar process with our microservices that enabled fast turnaround times from code to deploy. Naturally we began by trying to use Jenkins. While Jenkins works great for many use cases, we found that it had some pain points that made it difficult when working with microservices. Our first main issue with Jenkins 1.0 is that its configuration is completely centralized on the Jenkins server itself (though that should be changing in 2.0 very soon). We were looking for a CI system that allowed for services to define their own build and deploy configuration that could be version controlled and managed by anyone with access to the repository. Our second main concern with Jenkins was its available plugins for Docker and Kubernetes left much to be desired. We run many of our services in Kubernetes and will eventually be moving most if not all of them inside of a Kubernetes cluster. After wrestling with the Jenkins plugins for a couple hours it was evident that they would be more hassle than they were worth. Enter Drone.

Drone to the Rescue

Drone immediately caught our attention for several reasons.

  1. Its non-cloud solution is completely open source
  2. Can be run on-premises inside of a docker container and in our case inside a kubernetes cluster
  3. Decentralized configuration
  4. Fully container based
  5. Easy to use and contribute to plugin system
  6. Active Gitter channel with the main contributors actively responding
  7. It’s written in Golang (us Devops guys really like Golang)

We were able to get Drone running inside a docker container in minutes and within an hour or two we had it running inside of a Kubernetes cluster right along side the services it would be building and deploying. Next we set to writing our build configuration files for some of our services written in Node and after some time searching through Drone’s documentation (which at times couldn’t always be fully relied on, though thankfully can be made up with the Gitter channel) we were able to trigger a build from a Github web hook that ran completely isolated inside of a Docker container and finished in under 5 minutes. Awesome!

Our next challenge was to figure out how to have Drone deploy our services to Kubernetes similarly to how we use Jenkins to deploy to IIS. We went looking through the Drone plugins list and unfortunately found nothing for Kubernetes. However, since Drone is container-centric it follows that its plugins must also be able to run inside of a Docker container. This allows for their plugins to be technology agnostic as they don’t also need to be written in Golang despite the Drone core being written in Go. So we decided to write our own relatively simple plugin that handles our use case perfectly – deploy a microservice directly to our QA Kubernetes cluster with no downtime and completely automated. The plugin is written in Golang (did I mention we are big fans of Go?) and open source.

All that was left was the ability to notify us of any build or deploy failures. Fortunately, Drone has this built in and allows for notifications to be sent via Email, Slack, Hipchat, and many other messaging mediums. Slack is currently our notification of choice.

Moving forward

At the time of this post we are using Drone to build and deploy multiple services written in both Node and Java and look forward to doing even more. We have seen a drastic improvement in speed of development since implementing and taking advantage of everything Drone has to offer. From a Devops NoOps perspective we have loved the automation and definitely do not miss the days of our manual deploys for each of our microservices.

By: Austen Lacy

P.s. We’re hiring! Check out our careers page for more info on how to get the most out of Geofeedia.