If you’re new to DevOps, check out this five-step process for building your first pipeline.
1. A brief introduction to DevOps and the CI/CD pipeline
If you ask several people, “What is DevOps? you’ll probably get several different answers. DevOps, like agile, has evolved to encompass many different disciplines, but most people will agree on a few things: DevOps is a software development practice or a software development lifecycle (SDLC) and its central tenet is cultural change, where developers and non-developers all breathe in an environment where formerly manual things are automated; everyone does what they are best at; the number of deployments per period increases; throughput increases; and flexibility improves.
While having the right software tools is not the only thing you need to achieve a DevOps environment, some tools are necessary. A key one is continuous integration and continuous deployment (CI/CD). This pipeline is where the environments have different stages (e.g., DEV, INT, TST, QA, UAT, STG, PROD), manual things are automated, and developers can achieve high-quality code, flexibility, and numerous deployments.
This article describes a five-step approach to creating a DevOps pipeline, like the one in the following diagram, using open source tools.
Without further ado, let’s get started.
Step 1: CI/CD framework
The first thing you need is a CI/CD tool. Jenkins, an open source, Java-based CI/CD tool based on the MIT License, is the tool that popularized the DevOps movement and has become the de facto standard.
So, what is Jenkins? Imagine it as some sort of a magical universal remote control that can talk to many many different services and tools and orchestrate them. On its own, a CI/CD tool like Jenkins is useless, but it becomes more powerful as it plugs into different tools and services.
Jenkins is just one of many open source CI/CD tools that you can leverage to build a DevOps pipeline.
|Jenkins||Creative Commons and MIT|
|Apache Gump||Apache 2.0|
Here’s what a DevOps process looks like with a CI/CD tool.
You have a CI/CD tool running in your localhost, but there is not much you can do at the moment. Let’s follow the next step of DevOps journey.
Step 2: Source control management
This is where SCM becomes almost a necessity. A SCM tool helps by storing your code in repositories, versioning your code, and coordinating among project members.
Although there are many SCM tools out there, Git is the standard and rightly so. I highly recommend using Git, but there are other open source options if you prefer.
|Git||GPLv2 & LGPL v2.1|
|Concurrent Versions System (CVS)||GNU|
|Mercurial||GNU GPL v2+|
Here’s what the DevOps pipeline looks like with the addition of SCM.
The CI/CD tool can automate the tasks of checking in and checking out source code and collaborating across members. Not bad? But how can you make this into a working application so billions of people can use and appreciate it?
Step 3: Build automation tool
Enter the build automation tool. No matter which build tool you decide to use, all build automation tools have a shared goal: to build the source code into some desired format and to automate the task of cleaning, compiling, testing, and deploying to a certain location. The build tools will differ depending on your programming language, but here are some common open source options to consider.
Awesome! You can put your build automation tool configuration files into your source control management and let your CI/CD tool build it.
Everything is good, right? But where can you deploy it?
Step 4: Web application server
So far, you have a packaged file that might be executable or deployable. For any application to be truly useful, it has to provide some kind of a service or an interface, but you need a vessel to host your application.
For a web application, a web application server is that vessel. An application server offers an environment where the programming logic inside the deployable package can be detected, render the interface, and offer the web services by opening sockets to the outside world. You need an HTTP server as well as some other environment (like a virtual machine) to install your application server. For now, let’s assume you will learn about this along the way (although I will discuss containers below).
There are a number of open source web application servers available.
|WildFly||GNU Lesser Public||Java|
|GlassFish||CDDL & GNU Less Public||Java|
Now the DevOps pipeline is almost usable. Good job!
Although it’s possible to stop here and integrate further on your own, code quality is an important thing for an application developer to be concerned about.
Step 5: Code testing coverage
Implementing code test pieces can be another cumbersome requirement, but developers need to catch any errors in an application early on and improve the code quality to ensure end users are satisfied. Luckily, there are many open source tools available to test your code and suggest ways to improve its quality. Even better, most CI/CD tools can plug into these tools and automate the process.
There are two parts to code testing: code testing frameworks that help write and run the tests, and code quality suggestion tools that help improve code quality.
- Code test frameworks
|JUnit||Eclipse Public License||Java|
- Code quality suggestion tools
|CodeCover||Eclipse Public (EPL)||Java|
|Emma||Common Public License||Java|
|JaCoCo||Eclipse Public License||Java|
Now that you’ve implemented code testing coverage tools, your DevOps pipeline should resemble the DevOps pipeline diagram shown at the beginning of this tutorial.
2. Optional steps
As I mentioned above, you can host your application server on a virtual machine or a server, but containers are a popular solution.
What are containers? The short explanation is that a VM needs the huge footprint of an operating system, which overwhelms the application size, while a container just needs a few libraries and configurations to run the application. There are clearly still important uses for a VM, but a container is a lightweight solution for hosting an application, including an application server.
Although there are other options for containers, Docker and Kubernetes are the most popular.
In this post, i have explained how to build DevOps pipelines with open source tools.
If you have any questions or feedback, feel free to leave a comment.
As always, if you found this post useful, then click like and share it 🙂