DevOps Evolution and the Rise of Platform Engineering

As we continue to see rapid technological advancements, organizations must evolve and adapt to maintain a competitive edge. The principles of DevOps - collaboration, automation, continuous integration, and delivery - have emerged as critical success factors in this landscape, enabling organizations to navigate the ever-changing environment. From this, we have seen two critical studies on the importance of DevOps and its role in organizations - Humanitec’s DevOps Benchmarking Study 2023 and Dynatrace’s CIO and DevOps Report.

The speed of DevOps, often referred to as 'DevOps velocity', is a measure of how quickly an organization can move from idea conception to delivering functioning software, embodying a continuous loop of improvement. This acceleration enables organizations to compete effectively in the market, attracting and retaining customers while improving productivity. By embodying the values and methodologies of DevOps, organizations can efficiently respond to market dynamics, continually refining their offerings to meet evolving customer needs.

Yet, as the definition and role of DevOps continues to evolve, we are beginning to see another evolution, one where platform engineering is beginning to guide the future of software development and operations. This gradual shift towards platform engineering is considered the natural progression of DevOps, highlighting the importance of platform-centric approaches such as containerization, AI and machine learning, serverless computing, and cloud-native technologies.

Through our community of industry leaders, we conducted a study to reveal further insights into this progression from DevOps to platform engineering. Our respondents, the majority of whom were from software development, DevOps, and platform engineering roles, reflected a wide adoption and endorsement of DevOps principles. The broad applicability of these practices across different sectors emphasizes the importance and potential impact of the insights gathered from this survey.

From our research, we saw 52% of responses were from DevOps engineers, which correlates to 69.6% of the same audience stating that they have either a “good” or “very good” understanding of DevOps. On the other end, only 40.8% of the same group state that they are familiar with platform engineering. Whilst these responses still lean towards a moderate to high-level familiarity with platform engineering, the gap suggests there is room for further awareness and education, especially since a significant portion of respondents reported low familiarity.

Saiyam Pathak, Director of Technical Evangelism at Civo, spoke about the importance of the patterns between DevOps and platform engineering by outlining that “businesses can achieve greater efficiency, scalability, and collaboration by implementing standardized internal developer platforms and leveraging the principles of DevOps.”

By combining our own research with other studies in the industry, we aim to explore the implications of the transition from DevOps to platform engineering and illuminate how this evolution could shape the future of software development and operations.

Background to the survey

As part of the research for DevOps evolution and the rise of platform engineering, we spoke with over 100 industry experts. This allowed us to gain insight into a wide range of contexts and settings from the industries represented in the responses.

The sample size will be considered throughout the results to achieve a fair understanding of how the industry is progressing whilst comparing it to other research available.

Of those surveyed, our intention was to reach individuals who have experienced transformations in the industry over the past decade. Of all respondents, 22% have been part of the industry for over 7 years, while a notable 25% have ten or more years of experience, thus having a first-hand view of the significant changes in the sector over the last decade.

Key findings

The key findings of our research are presented below. These insights are based on the experiences and viewpoints of organizations now navigating the dynamic DevOps and platform engineering landscape. They highlight developing trends and identify significant industry problems and possibilities for future growth.

1. DevOps Adoption: 85% of organizations have either partially or fully adopted a DevOps culture, emphasizing its continued relevance and growing influence in the industry.
2. Platform Engineering Familiarity: Only 40.8% of the respondents in DevOps roles reported a moderate to high-level familiarity with platform engineering, indicating room for further awareness and education.
3. Impact of Platform Engineering: Among organizations that have adopted platform engineering, 50.4% reported improved standardization, and 43.2% saw faster product delivery.
4. Future of DevOps: While 65% of respondents agree or strongly agree that platform engineering is the next evolution of DevOps, a significant proportion showed neutrality, reflecting uncertainty about the future trajectory of DevOps.
5. Challenges in Implementing DevOps: Resistance to change and lack of standardization emerged as the top challenges for organizations implementing DevOps practices.
6. Emerging Trends: The future of DevOps and platform engineering is likely to be influenced by GitOps, AI and Machine Learning, DevSecOps, developer platforms, and serverless computing.

Adopting practices of a DevOps culture

With growing complexity in software systems, the need for faster and more reliable deployments and the increasing demand for standardization have all contributed to the evolution of DevOps. As acknowledged through an article by Alan Shimel, DevOps is “more alive than ever,” making it essential to understand how DevOps is evolving alongside these changing times.

We found a strong trend toward the integration of DevOps practices in the industry:

• Of those surveyed, 85% said their organization has either partially or fully adopted DevOps culture.
• Automated CI/CD was the most used DevOps tool/technology, with 81% stating their organization uses it.
• 72% said their organization uses version control software as part of their DevOps implementation.

Respondents recognized DevOps's benefits to their organization, such as improved collaboration, faster deployment, and increased productivity. Despite this, it is crucial to consider that, according to Dynatrace, 55% of organizations confess to compromising on aspects such as quality, security, and user experience to accommodate the swift transformations required by DevOps. The correlation between these two data sets indicates the need for a security-focused approach to improve standardization.

While many organizations have broadly adopted categories like "deployment and development practices" and "infrastructure and configuration management," other tools like "collaboration and feedback" have only seen usage from 38.6%. This information indicates that while certain DevOps tools are extensively used within organizations, others don't enjoy the same level of adoption, potentially compromising other operational sectors.

From this, we can conclude that the adoption of DevOps practices brings many benefits; however, it is crucial to understand the potential compromises that might be necessary. By focusing on currently under-utilized areas, organizations can create a more balanced and effective DevOps approach, addressing potential shortcomings, and promoting a more integrated, secure, and high-performing software development ecosystem.

The growth of platform engineering

Our research found a balanced understanding of platform engineering, with a minor skew toward medium to high degrees of familiarity. This suggests that while platform engineering is somewhat comprehended within the industry, there is potential for enhancing knowledge and awareness.

Whilst this outlines the growth of platform engineering, it is important we understand the potential it brings to the industry. We found that for those respondents that have adopted platform engineering within their organization, they have seen significant improvements to their development and operations processes:

• 50.4% said that adopting platform engineering has improved standardization across their organization.
• Faster product delivery was seen by 43.2% of respondents who have adopted the use of platform engineering.
• 30% of organizations are currently evaluating the adoption of platform engineering practices.

Gartner also suggests platform engineering could help organizations grow their services, deliver products faster, and connect systems more effectively. This is because platform engineering offers specific tools that can improve productivity and lighten the workload for development teams. Despite this, it's essential to understand and address the barriers to its implementation.

With less than half of the organizations having an internal developer platform in place, there lies significant potential in investing in such platforms. These platforms can help accelerate a successful platform engineering strategy, as they centralize development automation, increase efficiency and promote standardized practices across the organization.

Understanding the growth and implications of platform engineering is just the first step. Equally important is realizing how this field intersects and aligns with the existing principles of DevOps. To illuminate this, we need to delve deeper into our survey results, specifically focusing on the ways in which platform engineering may address the common challenges faced in implementing DevOps practices.

Bridging the gap between DevOps and platform engineering

In our earlier discussion, we established that over 50% of respondents felt that platform engineering has improved their organizations' standardization. This finding directly correlates to the 54% of respondents who identified the lack of standardization as a primary challenge when implementing DevOps practices. The inference here is clear: platform engineering seems to be addressing a core DevOps implementation challenge, thereby leading to better consistency, fewer errors, and improved efficiency.

Despite these benefits, our survey revealed a divergence of opinion on the future of DevOps. While 65% of respondents agree or strongly agree that platform engineering is the next evolution of DevOps, many participants selected a neutral response. This neutrality underscores a prevailing uncertainty about the future trajectory of DevOps.

Turning to the challenges of implementing DevOps practices, our findings identified resistance to change as a primary obstacle. To mitigate this, organizations could focus on adopting consistent tools, a move that would align with the principles of standardization and help avoid hindering the effectiveness of both DevOps and platform engineering practices. After all, finding and standardizing around a set of tools that aligns with an organization's goals and needs is vital for effective transition and transformation.

Emerging trends shaping the future of DevOps and platform engineering

As we look towards the future, we anticipate seeing the continued evolution of DevOps and platform engineering and their reliance on accompanying technologies and practices increase to reduce the core challenges faced. This will likely involve greater automation, increased AI and machine learning use, further adoption of cloud-native technologies, and an ongoing focus on security.

So, what’s next?

Looking closer at the emerging trends identified by our respondents, several key technologies and methodologies stand out:

• GitOps: Many participants mentioned GitOps as a method of implementing Continuous Deployment for cloud native applications. GitOps provides a declarative approach to automation and can help maintain consistency across environments, enhancing the overall efficiency of operations.
• AI and machine learning: The respondents also envision a significant role for AI and Machine Learning in the future, including MLOps¹ and AIOps². These technologies can automate complex tasks, improving efficiency and enabling more intelligent decision-making, which will be critical in managing the increasing complexity of DevOps and platform engineering processes.
• DevSecOps³: Many respondents emphasized integrating security practices, or DevSecOps, into the DevOps model. This approach is crucial for reducing vulnerabilities and ensuring the reliability of applications, making it a key trend for the future of these fields.
• Developer platforms: Tools like backstage.io and other developer portals or platforms were also mentioned as pivotal for boosting developer productivity and standardization.
• Serverless computing: Some participants indicated the growing importance of serverless computing. This technology allows developers to focus more on writing code and less on managing servers or infrastructure, which could drastically improve productivity and speed up time-to-market.

Conclusion

Given the changing nature of the cloud native landscape, our research findings provide a compelling case for the continued evolution of DevOps into platform engineering. We're witnessing an industry-wide shift that signifies the maturity and expansion of DevOps principles, with platform engineering emerging as the next crucial phase in this progression.

Our findings underline the importance of both DevOps and platform engineering in shaping the future of software development and operations. As organizations continue to adapt and evolve in response to market dynamics, they must embrace the principles of these disciplines, invest in the necessary skill sets, and align their strategies with emerging trends to maintain a competitive edge.

The evolution of DevOps into platform engineering isn't a question of if, but when – and organizations that recognize and prepare for this shift will be best placed to succeed in this new era of software development and operations.

Further resources

• DevOps vs Platform Engineering video from Saiyam Pathak
• Humanitec’s DevOps Benchmarking Study 2023
• Dynatrace’s CIO and DevOps Report
• DevOps all you need to know
• Is DevOps Dead? by Alan Shimel

Key terms

¹ MLOps, short for Machine Learning Operations, is a practice that unifies machine learning system development and machine learning system operations. It aims to streamline the deployment, testing, and maintenance of ML models in production, thus ensuring their reliability and effectiveness.

² AIOps combines data analytics and machine learning to improve IT operations. Its goal is to enhance and streamline IT tasks using automation and advanced analytics. Using artificial intelligence techniques, AIOps aids in problem-solving and offers predictive insights.

³ DevSecOps is a philosophy that integrates security practices within the DevOps framework. It aims to embed security in every part of the development process, ensuring fast, safe code delivery, and reducing security vulnerabilities in the application lifecycle.