Not Measuring Your Software Engineering Maturity? It's Time To Start

Welcome to my LinkedIn newsletter! In each issue of Entrepreneurship and Leadership, I'll be sharing my thoughts on scaling, leading and funding high-growth startups and what the future of innovation looks like. Subscribe here to stay updated.

What if I told you that one of the biggest departments in your company can be a black box? That for team members and for those who write the checks, there’s sometimes little insight into efficiency, what people are doing and how their work measures up to industry standards?

Would you let your sales department run that way? Or marketing, or HR, or any other part of the business?

Yet this is sometimes how things work today with software engineering. Unless you think this is no urgent matter, remember that software is one of the largest teams at most modern businesses. With the average salary of a software developer in the US sitting at $115,000, annual costs for some enterprises can easily run into the billions. 

A business can’t measure what it can’t see. As the founder of three software startups over the past two decades, I’ve seen little progress on this front. For many businesses, the lack of a benchmark and the right tools to measure and improve engineering efficiency represents a costly problem. Here’s a look at why it persists and what to do about it. 

Why companies remain in the dark about software engineering efficiency

There are two main reasons why companies have such a poor grasp of what goes on with their software teams. First, for most businesses, software engineering only recently became a core competency. Over the past decade, what was once a niche confined to technology firms has taken over the world, turning everyone into software players. 

No industry is immune: businesses as diverse as JPMorgan, HBO, General Electric, Hilton International and Delta Airlines now have software at their heart. Unlike time-tested business functions with legacy processes like accounting, sales and HR, software engineering is a new discipline that companies are still learning to master. 

Second, because they know how to write code, software engineers often take it upon themselves to build their own tools for measuring efficiency, rather than seeking support from established vendors. As a result, ironically enough, their tools are often worse than those for HR, marketing or sales.

On top of that, nearly everyone who works with software thinks their technology is unique and impossible to benchmark. For instance, the Netflix development team might argue that its technology is more complex than a bank’s software, so an apples-to-apples comparison is impossible. But the bank’s team might make the same argument, citing governance and other rules as reasons for greater complexity.  

Why should the typical company care about any of this? Think about the cost of software development. The latest estimates peg the annual cost of IT at $4.6 trillion. At many companies, engineering and development costs represent one of the single biggest expenses. If the process is 50% inefficient, that’s a sizable loss. Scale it out to 30,000 developers at a large bank or another enterprise, and the waste is enormous.  

More importantly, for most businesses today, interaction with customers is driven almost wholly by software. For example, when you look at the meteoric rise of neobanks and fintech upstarts in recent years, it's not that they're offering better interest rates or expanded services; it's just that they have a better app.

How to fix software efficiency in three steps

So a company admits it has a software measurement problem. Now, what can they do about it? 

1. Survey your engineers: To start, companies need to hold up a mirror to their own processes and see how they stack up. These kinds of “diagnostics” are already table stakes elsewhere in the company. Take net promoter score (NPS), which is widely used across industries to assign a simple numerical score for customer loyalty.  The good news is that surveys like this are being developed for the software world — for example, by the Engineering Excellence Collective. These frameworks survey engineers on three main attributes of their software development: velocity, quality and quantity. Velocity refers to how responsive a software development team is to market needs. If it takes six months to deliver a requested feature, that’s a challenge. Quality covers software attributes like user experience, performance and security. Quantity measures how much a team actually produces. Armed with these survey results, companies can begin benchmarking — across industries, within industries and even within the same company. For example, a bank would be able to see how efficient its software development process is compared to that of rivals, plus how its various teams stack up against each other. 

2. Use diagnostic tools to quantify the problem: The next step is to move beyond user surveys and take a quantitative pulse of software teams. A growing number of tools can analyze the development process in fine-grained detail, breaking it down into hundreds of small metrics that pinpoint what’s wrong and how to improve — everything from deployment processes to security and governance integration, and discoverability and documentation. 

3. Automate fixes: The critical final step is to fix these blockers. Manual approaches here are quickly being replaced by software platforms that facilitate continuous integration/continuous delivery (CI/CD). Continuous integration automates the testing of code changes as they arise, ensuring that the code is ready to ship. Besides saving time, this improves software quality. Continuous delivery helps speed up development too, by automatically shipping code to production each day. That deployment stage pulls in many other parts of the organization, including security, compliance and cost management.  

Measuring and improving software engineering efficiency may not sound especially exciting or even groundbreaking. But it’s an important fix for an exceedingly expensive problem. Just as every modern company tracks sales numbers and marketing metrics, the time has clearly come to apply a quantitative lens to software development.

Thank you for reading! I'd like to know your thoughts in the comments below. For more insights from my experience as a serial entrepreneur and how we can harness the power of software to change the world, be sure to subscribe to Entrepreneurship and Leadership.