Software projects often begin with promise – a clear plan, a motivated team, and consistent releases. Yet, beneath the surface, trouble brews: deadlines slip, bugs persist, and progress stalls despite ongoing effort.

Managers grapple with familiar questions: More engineers? Better testing? The answers lie within the data – but primary metrics like bug counts or test coverage merely scratch the surface. Without deeper analysis, root causes remain concealed.

This article redefines engineering metrics as a system, concentrating on derivative metrics – the less evident, interconnected signals that disclose what primary metrics cannot. We will examine how these metrics, rooted in core data, turn guesswork into precise, actionable strategies for managing software development.

The System: Four Pillars with a Deeper Lens

Effective metrics blend immediate visibility with long-term insight. We organise them into four familiar categories, but our focus here is the derivative metrics within each:

1. Quality & Stability – Beyond bug counts, derivative metrics like defect clustering or regression failure trends expose systemic weaknesses.
2. Development Efficiency – Past velocity, metrics like code churn rate or PR review latency pinpoint process friction.
3. Delivery Performance – Beyond lead time, change failure density or deployment rollback rate highlight delivery risks.
4. Business Impact – Beyond adoption rates, support ticket escalation rate or feature usage decay tie engineering to value.

Primary metrics (e.g., bug count, lead time) serve as the starting point – they are well-known and widely monitored. Derivative metrics, on the other hand, explore the reasons behind the variations in those numbers, linking different categories together.

Derivative Metrics in Action: The Hidden Story

Derivative metrics shine when primary metrics raise flags but lack context. Here’s how they work together to uncover root causes, predict issues, and drive decisions:

1. Unmasking Root Causes

• Defect Clustering (Quality): A high bug count is a red flag, but defect clustering – bugs concentrated in specific modules – might reveal shaky architecture or overworked teams. Pair it with code churn rate (Efficiency), and frequent rewrites could be the culprit.
• PR Review Latency (Efficiency): While slow lead times for changes are evident, PR review latency highlights this issue further – long review cycles may indicate knowledge silos or understaffing, rather than merely deployment challenges.

2. Bridging Quality and Efficiency

• Regression Failure Trends (Quality): Test coverage looks solid, but a rising regression failure trend shows tests aren’t catching new bugs – perhaps they’re outdated or too shallow. Check the iteration rate (Efficiency) to see if rushed cycles are skipping updates.
• Change Failure Density (Delivery): A high change failure rate grabs attention, but change failure density (failures per module) ties it to defect clustering, exposing weak spots needing rework.

3. Anticipating Trouble

• Code Churn Rate (Efficiency): Velocity’s steady but spiking code churn rate – excessive edits to recent code – hints at unstable requirements or poor planning. Couple it with the deployment rollback rate (Delivery), and you’ll see if those changes tank releases.
• Support Ticket Escalation Rate (Business): Feature adoption is decent, but a climbing support ticket escalation rate – tickets needing higher-tier fixes – suggests stability or usability flaws. Cross-check regression failure trends to confirm.

4. Feeding the Loop

Derivative metrics rely on iteration. Noticing a spike in defect clustering? Examine PRs or modules for patterns. Observing a decline in feature usage (drop-off after launch)? Assess change failure density to determine if quality is affecting it. This cross-referencing transforms data into actionable insights.

Spotlight: Key Derivative Metrics

Here’s a closer look at how to track and use these less obvious metrics:

1. Defect Clustering (JIRA, SonarQube)
   • What It Reveals: Where bugs concentrate beyond raw counts.
   • Watch For: Clusters in new features (rushed work?) or legacy code (tech debt?).
   • Action: Target refactoring or extra testing on hot spots.
2. Code Churn Rate (GitHub)
   • What It Reveals: How frequently code is rewritten shortly after being committed – a canary for stability. 
   • Watch For: Spikes associated with delayed releases – unplanned rework may impede delivery. 
   • Action: Discuss planning or scope creep with the team.
3. Change Failure Density (Jenkins, Sentry)
   • What It Reveals: Failures tied to specific changes or modules, not just totals.
   • Watch For: Patterns with high-risk areas – time to bolster QA there.
   • Action: Adjust testing or rollout strategies for repeat offenders.
4. Support Ticket Escalation Rate (Zendesk)
   • What It Reveals: How often issues escalate beyond essential support – hints at deeper flaws.
   • Watch For: Rises despite stable adoption – usability or bugs might be brewing.
   • Action: Investigate with quality metrics like regression failure trends.

Why It Matters: Strategic Clarity

Derivative metrics shift the game:

• Proactive Planning: Spot trouble – like code churn delaying delivery – before it snowballs.
• Resource Precision: Justify hires or shifts based on PR review latency, not vague “we’re slow” claims.
• Business Alignment: Link support ticket escalation to engineering fixes, proving impact.

Final Thoughts: The Power of Deeper Data

Primary metrics like bug count or lead time are the pulse of a project – vital but basic. Derivative metrics are the diagnostics, revealing why the pulse quickens or falters. As a system, they turn raw numbers into a quality, efficiency, and impact narrative – guiding engineering leaders from firefighting to foresight.

Read more:
Engineering Metrics as a System: Unlocking Insights with Derivative Metrics

The Shift-Left paradigm reshapes software development by integrating quality, security, and compliance into its earliest phases.

Often mistaken as merely a push for earlier testing, its true strength lies in a fundamental change – a rethinking of how teams collaborate, confront risks directly, accelerate delivery, and create software that withstands pressure. Numerous studies highlight its impact, demonstrating clear gains in efficiency and fewer defects for companies that embrace it. The FAANG companies have implemented it in practice, streamlining their processes, reducing errors, and enhancing security.

Real-World Application: Netflix’s Bold Move

Netflix, a firm that thrives on software, offers a striking early lesson in Shift-Left thinking. Their Chaos Monkey tool, part of the Simian Army suite, deliberately throws spanners into the works during development. It might seem mad to break your own systems, but it digs out flaws when they’re more straightforward and cheaper to mend. This forward-thinking tactic has fuelled Netflix’s meteoric rise, keeping their streaming service rock-solid for millions. By stirring up trouble in testing, they push their engineers to design systems that shrug off failure from the outset. It’s a mindset of welcoming hiccups that’s now baked into their culture, ensuring top-notch streaming without a glitch.

Beyond Testing: Smashing Silos for Organisational Overhaul

Shift-Left is often pegged as a testing boost, but its real clout is shaking up how organisations run. The old-school setup where development, testing, security, and compliance teams operate in their own bubbles – breeds friction, crossed wires, and hold-ups. In today’s high-speed digital world, that’s a recipe for trouble. These disconnected teams can miss the bigger picture, letting vital details slip through the cracks, dragging out timelines and increasing the odds of pricey slip-ups.

Shift-Left tears down those walls by knitting security, compliance, and quality checks into the process from day one. It’s about everyone pulling together and owning the software’s strength as a team. That means better chatter and tighter teamwork – groups that once barely spoke now must gel, swapping info and syncing up. It’s a cultural shift, nudging firms toward openness and cooperation. As noted in their work on on-demand scanning, Google’s cloud crew found that catching security snags early can slash weaknesses by up to 40% after launch.

How You Can Do It: Lifting Test Coverage with Developer Input

One headache in software development is ensuring tests cover how the app will be used and where it might crack. Plenty of outfits hit decent test coverage numbers but still face live issues from “hidden” flaws. This snag often points to a weak spot: developers aren’t fully in the loop on test reviews, and there’s a disconnect between them and the testers. Standard testing is handy, but it can miss tricky edge cases or quiet bugs that coders are better placed to spot with their insider grasp of the system.

A fresh tactic here is to get developers leading the charge on test coverage, digging into the source code themselves. The payoff can be eye-opening – unearthing a stack of overlooked test cases, sometimes matching the size of the existing set. That’s a wake-up call to rethink how coverage is judged, dig into the quality of design specs and requirements, and double-check how well teams talk to each other. It’s not just about nabbing more bugs; it’s figuring out why they were missed and sharpening the process.

Other firms’ journeys flag up some big takeaways. First, dragging developers into testing is a must – their code know-how spots risks others might skip. Second, leaning only on numbers – like the percentage of code tested – falls short; you need a proper look at whether tests match real-world use and weak points. Third, testing ties right back to solid requirements and designs – tests miss the mark without clear groundwork. Lastly, keeping the lines open between coders and testers and the rest is vital – clear channels and a chatty culture catch trouble early. This isn’t just tweaking tests – it’s building a fuller, stricter quality setup that cuts live hiccups.

The Role of AI: Pushing Shift-Left Further

Artificial intelligence is flipping the script on Shift-Left, bringing in automation, sharp predictions, and nifty fixes before problems hit. AI isn’t just sprucing up tests or sniffing out flaws – it’s paving the way for clever systems that learn as they go, tweaking workflows for the better. It’s the next leap, dragging teamwork and proactive fixes even earlier into the mix. With AI sifting through heaps of data, spotting oddities, and flagging insights humans might miss, firms can squash bugs sooner and stop them from cropping up at all.

Companies face a fork in the road: whip up their own AI tailored to their quirks or tap into ready-made kits from cloud providers and open-source crews. Custom AI is gaining ground for outfits in tight-rule zones like finance or healthcare, tackling their specific compliance and security headaches. Microsoft’s boffins reckon bespoke AI can boost defect spotting by up to 60%, backing the push for tailored quality tools.

How AI-Driven Shift-Left Reshapes Software Development and Security

AI is turning software quality into a crystal ball game. Machine learning, fed by old defect logs and live system pulses, predicts flops before they land. It watches code shifts, system quirks, and past slip-ups, pinpointing risks and suggesting fixes ahead of time – keeping downtime low and live woes at bay. That frees developers to cook up fresh features and keep customers chuffed rather than firefighting messes. Take Spotify – serving hundreds of millions of users. They lean on AI-powered monitoring to catch hiccups early, keeping tunes flowing without a hitch. Or Google’s Cloud Platform, where AI scans dodge sneaky threats, making their platforms tough as nails. It’s not just catching bugs – it’s about building more innovative, safer software from the off, letting teams focus on what matters.

Read more:
The Shift-Left Strategy: Boosting Development Speed and Crafting Robust Software