By now, most organizations have dipped their toe in the DevOps waters. In most cases the results come up short of their expectations, not to mention the cost of tools, staff and turnover eating up much of the ROI. While many struggle to get started, some organizations are emerging and seeing results.
Over the last 5 years the nvisia DevOps team, in partnership with enterprises large and small, has had a front row seat to many organizational journeys. We're sharing our lessons learned in this blog series, but first, it's important to understand the business value DevOps can bring.
The "why" behind DevOps initiatives
It is hard to deny the results from high performing DevOps organizations. The Accelerate State of DevOps Reports provided by DORA (acquired by Google in 2018), provides an excellent summary of the difference between low and elite performers.
From the Google DevOps site:
Speed of deploymentsThe best teams deploy 973x |
Stability of your softwareHigh performers don’t trade off speed and stability. The best teams recover from incidents 6570x faster and have change fail rates 3x lower. |
Security in from the startHigh performers spend 50% less time fixing security issues compared to low performers. |
What does this really mean? At the end of the day, Elite DevOps performers are able to:
- Commit code to source code control and deploy to production in less than 1 hour
- Perform 2 production deployments in a single day
- Recover (MTTR - mean time to recover) from deployment problems in less than 1 hour
- Deploy at will with 0-15% Failure Rate
Each one of these items saves valuable time of highly paid professionals, reduces costly errors and increases the stability and quality of your production systems.
To be successful in a DevOps initiative, you should measure these sorts of business metrics, understand where your IT teams are experiencing problems, and make efforts to improve the numbers. Installing tools does not magically make you a master of DevOps. Like any well-conceived initiative, a DevOps initiative should target business problems and success must be tied to some measurable outcomes.
Common DevOps anti-patterns we've seen
As we've engaged with struggling organizations, we've noticed some patterns, or really anti-patterns, that led to their issues. For brevity's sake, here are the most common along with some simple recommendations.
Centralized operations teams(Rebrand Jenkins Operations team as DevOps)Many operations teams own central tools like databases and old-school Jenkins pipelines. These teams are understandably looking for relevance in the new distributed, cloud native world. In the new world functional standards and specification for tooling may still lie in the traditional central group, but the running the tools are now distributed within the application team's landing zones. RecommendationInstead of paving the cow path, establish a (cloud) platform engineering capability, responsible for ensuring the consistent deployment of the centralized teams standards and specifications, using infrastructure as code (IaC) and automated pipelines. |
Lead with tools(There is no problem that a tool vendor can't fix)While DevOps related tools are perfectly necessary for things like automation, deployments, security and observability, you need collaborate with actual application teams through an evolutionary process to solicit their input and establish the context of tool usage. RecommendationMany DevOps problems can be avoided altogether with agile implementation and design, informed by the DevOps way(s). |
Science projects in production(The rockstar dev teams that turned GhostOps)Many early DevOps successes in larger organizations were born from high profile application development teams staffed by highly-skilled, passionate technologists with incredible grit. Sounds great, but since very little thought was given to day-2 operations, it becomes unsustainable. Subsequently, team members either burn out or bounce to the next cool team, leaving the application owners in a bad place. RecommendationInfrastructure as code and automation pipelines to declaratively manage your infrastructure are good insurance policies to prevent GhostOps. |
The "DevOps Team"(new function = new team)While it seem reasonable to create a new DevOps team to support DevOps capability, this notion is a tool oriented approach to find a place for the Pipeline tools to live. DevOps is way more than Jenkins, Gitlab, Github or Azure DevOps. RecommendationAgain, DevOps is an organization and cultural movement that spans all of your technology organization. It can't be neatly tucked into a single organizational silo. Explore a matrixed platform engineering capability instead. |
DevOps is way more than tools
While many attempts have been made to articulate this point more politely, rarely has it been stated less ambiguously than this quote:
“Stop talking about the blinky shit!
…it’s about
the people and the culture”- Chris Roberts, DevSecOps Expert Panelist
DevOps culture, with the core concepts built upon lean manufacturing (aka lean software) concepts, is beautifully portrayed in the legendary series of MUST, MUST READ books shown below. The Phoenix Project is a fictional account of DevOps in a gripping story masterfully designed to get you thinking about DevOps concepts. If you have been in IT for any length of time, you will immediately relate to most of the characters in this gripping tale. Next is the DevOps Handbook, the non-fictional guide to getting started with DevOps. Finally, we have Accelerate, research associated with organizations who adopt DevOps practices and culture.
Our recommended approach to your DevOps & Cloud initiative
Now that you understand the goals we're aiming for in a DevOps initiative and the pitfalls we're trying to avoid, read on to the next entry in this blog series for our recommendations on how to achieve the goals: Rockin' Cloud Native DevOps.
Think it sounds like too much work and just want us to build it for you? Click here to learn about Digital Foundations, a service that uses our proprietary Terraform modules and pipelines to accelerate your Cloud Native DevOps Success: Digital Foundations
Conclusion
DevOps journeys are fraught with challenges, but deliver real business value to the organizations that take a smart approach to them.