Engineering excellence sounds great — but what does an excellent engineering team actually look like? Recent findings in the 2024 DORA Report give us a clue, showing that top tier organizations who have built an operating model of engineering excellence can deploy code 127 times faster and 182 more frequently than those who don’t.
But what actually is engineering excellence? And what metrics can you use to measure it?
Sections:
1. What is Engineering Excellence?
There is no clear definition of what engineering excellence means. There are multiple different perspectives on the topic. For example, Kevin Scott, who led engineering at AdMob and LinkedIn, breaks it down into three key areas:
- How do we make things
- How we operate things
- How we function as a team
Dinker Charak, who co-authored the book Engineering Excellence to Business Outcomes, instead suggests a two-part framework to break down engineering excellence:
- Excellence in software development — quality of the work that happens during the creation of software.
- Excellence of software once in production — quality of software once deployed, such as the maintenance and optimization the software's performance and reliability in the real world.
Combining multiple perspectives together, we believe that Engineering excellence goes beyond writing functional code. It's about creating software solutions that stand the test of time in the real world through thoughtful design, maintainability, and scalability. Every decision, from architecture choices to code reviews, contributes to building robust and reliable systems — ultimately in service of customers and business outcomes.
2. Engineering Excellence Metrics
The software industry has seen a dramatic shift in how teams think about measuring engineering work, particularly since the COVID pandemic. Organizations are increasingly focused on understanding the value they get from their engineering investments, leading to an explosion of interest in metrics and measurement.
However, this surge in metrics adoption brings its own challenges. Research shows there are now 700-800 different product metrics being used across the industry, with teams often creating their own custom measurements. This abundance of metrics has led to some confusion about their purpose. Teams sometimes blur the lines between different measurement goals, such as:
- Identifying value stream inefficiencies
- Supporting individual developer growth
- Evaluating team performance
- Measuring business outcomes
The key isn't just having metrics — it's understanding exactly what you're trying to measure and why. Teams need to be thoughtful about which metrics they track and ensure they're using the right measurements for their specific goals. A metric that's perfect for measuring business impact might not be helpful for supporting developer growth, and vice versa.
So how can we measure Engineering Excellence?
DORA metrics
DORA metrics are considered an industry gold standard that measures software delivery performance and engineering excellence.
Based on research spanning over 36,000 professionals across organizations of all sizes and industries, these metrics have proven to be reliable predictors of organizational performance:
- Change Lead Time: The time it takes to go from first commit to code successfully running in production.
- Deployment Frequency: How often an organization deploys code to production or releases it to end users.
- Failed Deployment Recovery Time (Formerly Mean Time to Recovery): The time it takes to restore service when a deployment causes an outage or service failure in production (whereas previously Mean Time to Recovery also included uncontrollable failure events such as an earthquake disrupting service).
- Change Failure Rate: The percentage of changes that result in degraded service or require remediation (e.g., that lead to service impairment or outage, and require a hotfix, rollback, fix forward, or patch).
- Rework rate: This fifth metric was introduced later in 2024, and together with Change Failure Rate provide an indicator of software delivery stability. Since it's a newer addition, there aren’t yet established quantifiable benchmarks and so this metric tends to receive less focus.
EEBO metrics
The book Engineering Excellence to Business Outcomes, also suggests a set of metrics. Naturally, they suggest that DORA already offers a great way to measure excellence in production deployment (#2 in the above framework).
However, for measuring Excellence in Software Development they suggest 3 key metrics (amongst others):
- Build failure rate — How frequently your builds fail and which pipelines experience the most failures
- Security warnings — Number of security breaches or violations
- Tech debt — Additional rework caused by choosing an easy solution now instead of a better approach that would take longer.
However, EEBO doesn’t suggest that these 3 specific metrics are the holy grail but instead offers several to choose from — based on your organizational priorities. They emphasize instead a philosophy to select the right metrics for what you are measuring.
SPACE Framework
The SPACE framework takes a big picture view of Engineering Excellence, considering the key elements required by developers to be productive by considering 5 key dimensions:
- Satisfaction and Well-being: Measures satisfaction, fulfillment, and well-being, both at work and off work.
- Performance: Outcomes that the organization aims to reach or create value for customers and stakeholders.
- Activity: Combines outputs that are countable, discrete tasks and the time it takes to complete them.
- Communication and Collaboration: Represents the interactions, discussions, and other acts of collaboration that take place within teams.
- Efficiency and Flow: Focuses on the ability of a developer to complete work or make progress on it.
By integrating these dimensions into consideration, the SPACE framework gives managers a holistic view of engineering team performance that enables them to make better decisions. At Multitudes, we like the SPACE framework because it encapsulates the performance aspects of the DORA framework while also acknowledging the importance of a psychologically-safe and trust-based working environment.
3. How to Improve Engineering Excellence
There are several dimensions to improve engineering excellence. At the individual level, this can include:
However, the biggest drivers of Engineering Excellence occurs when there a common set of practices across the team level:
Deployment Speed and Frequency
Fast, reliable deployments are key to delivering value to users quickly. Here's how to accelerate your deployment pipeline:
- Break updates into smaller pieces: This approach improves the delivery speed by allowing easier movement through the delivery process. Google’s research shows the ideal size is one self-contained change, and a Cisco study suggests no more than 400 lines of code at a time.
- Implement automation: Automation could include source control (to manage and track code changes), automated testing, automated integration and delivery (CI/CD), monitoring, and more. As one example, GitLab research demonstrates having CI/CD implemented in your pipelines can lead to superior code quality - Goldman Sachs saw their code builds increase from 1 per fortnight to over 1,000 builds per day.
- Employ feature flags: Improve development efficiency by decoupling deployments from releases. LaunchDarkly's 2024 Research shows 59% teams using feature management deploy several times a week, if not more frequently.
Recovery and Reliability
Building reliable systems requires both proactive planning and the ability to respond quickly when issues arise. Here's how to optimize your approach:
- Enhance monitoring and observability: Datadog's 2023 Annual Observability Report found teams using advanced monitoring tools achieve 40% faster recovery times.
- Develop effective incident response plans: These plans are essential for quick recovery, yet S&P Global found less than 50% of companies have these in place.
- Deploy smaller changes: Simplify testing and recovery processes while reducing bug likelihood through smaller, targeted deployments.
Collaborative Culture focused on Excellence
A strong engineering team thrives on more than just technical skills — it's built on trust, open communication, and productive collaboration that brings out the best in everyone.To foster a collaborative culture:
- Start with trust: Build an environment with psychological safety, where the people on the team feel secured and valued. This starts with leaders role-modeling high-trust behavior to help their team members do the same. [Pull out some action examples from the blog post about how to do so in practice]
- Encourage open communication: Open collaboration creates space for teams to align on shared goals and break down work together. This approach helps teams spot potential issues early and builds meaningful connections across the organization. When everyone feels comfortable sharing ideas and concerns, teams develop stronger working relationships and deliver better results together. Again, this starts with leaders role-modeling good examples.
- Implement cross-functional projects: Working across different teams brings together diverse knowledge and perspectives, enriching how we solve problems. When people with varied expertise collaborate, they share unique insights and learn from each other, leading to more well-rounded solutions that benefit everyone involved.One specific exercise that can help in engineering teams is having a support rotation so all engineers can see the impact of what they build on users. Another option here is having your engineers join in on customer research calls, so they can see first-hand challenges that users might be having with the product.
- Recognize and reward collaboration: Forbes highlights that celebrating teams and individuals who collaborate effectively creates lasting positive impact. When we acknowledge those who share knowledge and support others it encourages more of these behaviors that help everyone succeed together. As an example, some teams do this with an #appreciations channel in Slack where they encourage team members to celebrate each other publicly.
4. Improve Engineering Excellence with Multitudes
To effectively measure and improve engineering excellence metrics, teams can use Multitudes, an engineering insights platform built for sustainable delivery. Multitudes seamlessly integrates with your existing tools like GitHub and Jira to provide a comprehensive view of your team's technical performance, operational health, and collaboration patterns.
With Multitudes, you can:
- Track all key Engineering Excellence metrics in one place
- Identify patterns in your metrics that impact delivery speed and quality
- Measure team collaboration and its effect on engineering performance
By leveraging Multitudes, teams can focus less on metrics collection and more on using these insights to drive engineering excellence.
Our clients ship 25% faster while maintaining code quality and team wellbeing.
Ready to improve your engineering excellence metrics?
