A battle-tested DevOps platform

A battle-tested DevOps platform

A fellow software developer asked a relatively important question on Reddit a while back: How can we improve our development workflow? The question was backed by a detailed description of their current workflow and the original poster's (OP's) ideas about the methods they should be using. People contributed with very informative comments to the discussion, so it is a good read for anyone interested in more effective software development workflows.

The buzzword "DevOps" was mentioned in the comments. The OP wasn't sure how he should introduce DevOps to their organisation and what would be the needed tools. As a remedy, I decided to present one proven example of a DevOps platform in this blog post.

TL;DR: you'll find the example of an efficient DevOps platform by jumping to the end of the post through this link

DevOps in short

Before we look more into the DevOps tools, it would be appropriate to spend a few lines on the whole DevOps concept. This is because having the proper tools only pave the way towards a DevOps organisation but it doesn't make you one.

Defining DevOps in just a few words is impossible because it spreads to so many areas of a modern organisation. It is more of a culture that expands through the whole lifecycle of a software than a single operation. The bottom line is that DevOps breaks down the traditional organisational silos where you have developers in one corner, IT in another, testers in the third, and business executives in the fourth corner. The idea of DevOps is to combine business needs, software development, IT operations, testing, and deployment to an effectively collaborating entity that adds value to the customer. An effective DevOps organisation is able to deliver better software faster.

Different software development models have been around for ages. And the aim has always been to be more effective. So what makes DevOps so different from, let's say Waterfall or Agile?

Waterfall vs. Agile vs. DevOps

In the Waterfall model, you first define the outcome according to customer needs, and then the development process runs until a "finished" product is released. The biggest problem with this approach is that the customer's needs usually change during the development phase. Thus, you end up delivering software that doesn't meet the revised needs, or you spend a lot of time and money in changing plans midway.

The next step in the evolution is the Agile model. The idea in Agile is to develop software in small iterations and be thus able to adapt to the changing customer needs better than in Waterfall. However, this model has its hitches as well, such as 1) the budget goals and deadlines are often missed, 2) completed software components are not compatible which each other, and 3) new features break old functions. Furthermore, there is no or only a little cooperation between development and IT operations crew. The notorious silos are still intact!

DevOps brings more flexibility on top of the Agile model. With continuous integration (CI) and continuous delivery (CD) pipelines, you can make sure that you can release often and the releases actually work and meet the customer needs. Cooperation between the development team and IT operations ensures that also the used tools are streamlined and do not form bottlenecks. With the effective tools, repetitive work can be automated and transparency is improved. Thus all the involved parties have comprehensive visibility into the project.

DevOps production model

The picture above represents how a DevOps production model works. As you can see, business needs are the starting point for the DevOps model. It is important to understand the customer needs first and plan the work accordingly. When the first requirements are defined, software development can start running. Furthermore, the production runs continuously: testing and deployment are automated so new versions can be released in short intervals. Monitoring runs throughout the whole cycle. Thus, we'll constantly know that everything is running smoothly.

If the needs change during the development, they are easy to implement without having to start from scratch. The revised needs are communicated and documented in the requirements management, from where they are delivered to the development team, who implement the new needs. Automated processes ensure that the newly created software works and can be deployed fast.

Effective communication tools ensure that the feedback is constant, and the team is developing software that the customer actually needs.

The battle-tested DevOps platform

Before we can start building the DevOps platform, you should bear in mind, that DevOps is more a culture rather than a single operation. Because of this, a platform that consists of multiple tools with effective integrations is needed. There is no such thing as one DevOps tool that has takes care of all the aspects and needs of DevOps.

The DevOps platform presented below is battle-tested in development organisations that have thousands of users. Naturally, there are other options for the used tools as well. However, this toolkit has streamlined the software development processes in various industries, ranging from finance to logistics. Furthermore, it consists mainly of open source tools so that license fees won't bankrupt you.

Let's start with the requirements management and general project management. For those actions, you want to choose Atlassian's Jira. Naturally, you'll need also a communication and documentation tool from the beginning. Atlassian's Confluence is our choice for that. If you are on a tight budget, you can replace Confluence with the collaboration and documentation features of the best software configuration management tools, and save money on the license fees.

In the more technical area, you'll be using a variety of tools. For version control purposes Deveo is the best choice. "Why not Atlassian's BitBucket?" you might ask. The reason is that Deveo's delegated access management allows you to delegate and automate a lot of the IT's tasks to the project owners. In addition, Deveo integrates very nicely to all of the tools mentioned here. In this phase, also the continuous integration pipeline starts. For that, the tool is Jenkins.

Naturally, the development environments need to run on a server somewhere. If you do not have the physical server running in your basement, you'll want to use Amazon Web Services (AWS) for virtual servers. Unit testing is done with JUnit, which is a simple framework for running repeatable tests where code tests code. SonarQube has been proven to work phenomenally for static code analysis. Artifacts (e.g. binaries compiled from the code for testing) are stored in Artifactory.

Automated acceptance testing is run with Robot Framework. It has an easy-to-use tabular test data syntax and it utilizes the keyword-driven testing approach.

Production deployments and configuration management are taken care by Ansible, which is a simple IT automation engine that automates, e.g. cloud provisioning, configuration management, application deployment, and intra-service orchestration. For software containerization, our chosen tool is of course Docker. If you want to learn more about how we've used Docker, you should visit "Dockerizing the IT infrastructure" blog post.

Finally, the whole process is monitored with Zabbix, which is designed for real-time monitoring on the enterprise level. It works even if you need millions of metrics from thousands of servers. Grafana is used to visualize the metrics to nice and informative dashboards.

Let's combine these tools and put them on the right places in the DevOps production model:

Typical DevOps toolkit

Conclusion

The original poster in Reddit, who wanted to streamline their software development process should definitely look into DevOps. However, the tools are only one part of a DevOps organisation and setting up the platform I've described above doesn't instantly make your organisation a DevOps advocate. You'll need a complete cultural change and commitment from the whole organisation.

If you have already started the cultural shift towards DevOps and want to set up the best possible DevOps environment, just use the example above. You can get started by signing up to Deveo!

Seamless software development.

Code management and collaboration platform with Git, Subversion, and Mercurial.

Sign up for free
comments powered by Disqus