DevOps: A Catalyst for Engineering Excellence
In 2020, over 60% of software development teams had been practicing DevOps for at least one year as a way to accelerate their tests, deployments, and releases, according to GitLab. But what exactly is DevOps?
Over the years, DevOps has been framed as everything from a renewed commitment to collaboration to the implementation of specific technologies like test automation frameworks. In reality, it’s both of those things and a lot more: DevOps is the merging of all engineering (Dev) and IT operations (Ops) capabilities into a single continuous pipeline for the entire lifecycle of a product, system, or platform.
Above all, DevOps is a change in culture, breaking down silos previously separating the work of development and operations teams. Any technologies involved along the way are important but secondary to that principle.
DevOps seeks to automate anything that can be automated, and to move the team toward a state of “continuous everything,” with engineering and operations working closely in tandem at every juncture. Central to this notion of continuous cross-functional collaboration is the DevOps pipeline, which incorporates specific workflows including but not limited to:
- Continuous integration and delivery (CI/CD): Code changes are merged frequently to the shared main (“trunk”) branch each day, to be validated by automated tests and readied in a repository before being moved into production.
- Continuous deployment: A step further than the above, continuous deployment involves automatically deploying any code that has passed all pipeline checks directly to users.
- Monitoring: DevOps teams constantly monitor features and system availability to identify issues. Synthetic monitoring also helps continuously optimize performance and anticipate user issues.
- Tight feedback loops: In DevOps culture, there is clear-cut ownership of code, as well as high exposure to customer feedback. This setup leads to fewer handoffs and much quicker problem resolution than possible in siloed Waterfall environments.
The ideal DevOps pipeline will deliver increased velocity without compromising on software quality or security. The 2019 Accelerate State of DevOps report found that the best-performing DevOps teams deploy over 200 times more quickly than low-performers, while spending far less time recovering from incidents and fixing security issues.
What are the main benefits of DevOps?
At a high level, DevOps empowers cross-functional teams to do more in less time.
More specifically, DevOps yields shorter time to market — and in turn, faster realization of business value — along with a more straightforward path to innovation (e.g., new features), closer team collaboration, and improved stability, security, and quality across software. There are also major concurrent benefits for team culture and morale, as DevOps makes it easier for teams to deal with unplanned work and avoid costly, demoralizing outages.
Let’s explore some of these major benefits in more detail.
Velocity and time to value
In a DevOps culture, software can be built and iterated frequently — at an average rate of four deploys every day for high-performers in the Accelerate report. The automation of a well-functioning CI/CD pipeline, paired with practices like test-driven development, enables teams to build, test, and roll back efficiently. These teams work within a loosely coupled architecture that allows them to test and deploy on demand. Accordingly, problems get identified early on because teams own what they work on and don’t throw it over the wall for someone else to deal with. There’s a “you made it, you run it” mentality.
DevOps helps eliminate the tradeoff between adding new features and maintaining stability. For example, the best DevOps teams spend less time attending to security issues and have a change failure rate that’s seven times lower than their peers. Because they’re not constantly putting out fires or performing routine actions with low added value, these teams have more room to focus on innovation. Similarly, DevOps automation and the use of microservices also make it easier to curb technical debt that can push back feature development.
Collaboration is integral to DevOps. In fact, a 2020 Atlassian survey ranked it as the most important component of a DevOps culture. Personnel who might have otherwise worked within siloed engineering, QA, and ops units instead work as one team. This collaboration can lead to:
- Shared goals.
- Increased visibility.
- Fewer handovers.
- Better optimized code.
- Reduced meeting and email volume.
- More manageable technical debt.
- Less burnout and overall frustration.
On that last point, DevOps is indeed a powerful stress reducer. Through automation, DevOps help tame unplanned work, reduce the amount of time teams have to devote to root cause analyses and to applying fixes outside their typical working hours. Moreover, DevOps allows for better understood and lightweight change approval processes, lessening the procedural overhead that can burn people out.
Stability, security and quality
These three benefits all flow from how DevOps makes the software lifecycle more generally predictable.
CI/CD within DevOps, for instance, ensures that changes are always validated through a standardized testing process before reaching customers. Security is incorporated throughout the pipeline — an approach sometimes called “shifting left” — and can be systematically supported through Infrastructure as Code (IaC). Overall, software built through DevOps is safer and more secure, resulting in happier users and more productive teams.
To deliver these benefits and others, DevOps should be combined with Agile principles and clean architecture. The latter, via its dependency rule, makes it easier to maintain testable code and to be aware of dependencies. In other words, a clean architecture provides a reliable foundation for a DevOps culture.
Analyzing DevOps best practices and tools
Although DevOps is primarily a cultural practice, there are some specific technical processes and tools that help keep its engine running smoothly. CI/CD, which we have already explored in depth, is one of these fundamental DevOps building blocks. It helps avoid “merge hell” by automating/standardizing testing, simplifying conflict resolution, and streamlining delivery. Beyond CI/CD, other DevOps best practices and tools include:
IaC solutions, such as Terraform, allow infrastructure (e.g., virtual machines and networks) to be provisioned declaratively using machine-readable configuration files. These files will create the same environments every time no matter the starting state — somewhat similar to how source code will generate the same binaries on each run. IaC curbs the configuration drift that would otherwise happen due to ongoing changes (many of which don’t get documented) and by doing so it enables the scalability and speed that DevOps requires.
A microservice architecture breaks an application down into numerous small services, each running its own process and connecting to others via the APIs it exposes and through lightweight protocols like HTTP. For DevOps purposes, a microservice architecture is more practical to build, test, and maintain than a monolithic one. DevOps teams can scale and enhance the functionality of one service without negatively impacting another. Applications can be updated without having to have their entire monolith rebuilt each time.
A container bundles an application and all of its dependencies, configuration files, and libraries into one package that is decoupled from the underlying operating system. As such, a containerized application is highly portable, no matter the original environment for which it was developed. DevOps teams get to reduce the time spent on fixing bugs and errors when applications are moved across environments, and instead focus on more strategic work. Containers simplify the DevOps pipeline.
Logging and monitoring
DevOps requires continuous monitoring so that teams get feedback directly from production. Effective logging and monitoring will surface issues quickly. DevOps monitoring tools like Amazon CloudFront can also provide the granular data necessary for optimizing resource utilization and holistically assessing the overall health of current operations.
A DevOps culture thrives on continuous sharing and rapid communication about what’s happening throughout the pipeline. The resulting collaboration and team-wide accountability keep DevOps organizations aligned on a common set of goals, mitigating the risk of siloed workflows.
Building a DevOps team and culture
One of the first steps in building a DevOps team is to observe how engineering and operations personnel currently interact with one another, to reveal pain points and possible room for improvement. This assessment might reveal where handoffs are occurring and who is bearing the brunt of problems.
Typically, the DevOps team will have a 90/10 structure, under which 90% of its members are people who can do “everything,” — i.e. handle tasks across engineering and operations —while the remaining 10% are specialists focused on particular complex and esoteric areas (such as cloud configuration management). Other structures are possible, though. For instance, DevOps may involve a site reliability engineering team or a set of DevOps consultants.
Successfully building a DevOps culture can require some trial and error, especially since DevOps is holistic and as such requires wide-reaching changes to how teams perform. The blueprint for DevOps success will vary from company to company, but a few tips to keep in mind include:
- Start implementing and using DevOps workflows and tools on day one to get everyone used to them.
- Focus on the KPIs that matter to the organization, whether those are deployment frequency, lead time, change failure rate, mean time to recovery, or something else.
- Have experts on hand who can navigate the more complex parts of the DevOps toolchain.
- Cloudify the DevOps pipeline; do not try to make it work on-premises.
- Take advantage of IaC and containerization to better scale, standardize, and accelerate DevOps workflows.
At Transcenda, our experienced team embraces DevOps principles to rapidly deliver high-quality software for your organization. Connect with us to learn more about how we can guide your next project.