The Real Cost of Nobody Knowing How the System Works
By Sahil Singh, Founder & CEO
Every software organization reaches a point where no single person understands how the whole system works. This is not a theoretical concern. It is the primary reason why software projects fail, and the financial consequences are staggering.
Organizations waste an average of $109 million for every $1 billion they invest in projects and programs, according to the Project Management Institute. That waste does not come from bad engineers or poor technology choices. It comes from a knowledge gap so pervasive that most teams do not even recognize it exists.
I have built engineering teams where the person who designed the payment system left six months ago, the only person who understood the notification pipeline was on vacation during a critical outage, and the PM was writing specs based on documentation that was two years out of date. These problems are universal. And they are expensive.
This article examines why software projects fail at such alarming rates, where the money actually goes, and what organizations can do to stop the bleeding.
The $109M Problem
Understanding why software projects fail starts with the numbers. Let me put that PMI statistic in a context that hits closer to home. If your company spends $10 million a year on software development, you are likely wasting $1.09 million of it. Not on bad tools. Not on lazy employees. On the sheer friction of people not knowing how their own system works.
This waste shows up in specific, measurable ways.
Meetings That Exist Only to Transfer Knowledge
A senior engineer spends three hours explaining how the authentication flow works to a product manager writing a spec. Two weeks later, a different PM asks the same engineer the same question. The engineer now has a recurring "architecture overview" on their calendar every sprint.
This is not collaboration. It is an oral tradition masquerading as a development process.
Specs Written Without Code Awareness
Product managers write requirements based on their mental model of the system. Engineers read those requirements and realize the PM assumed a microservice boundary that does not exist, or missed a dependency that makes the "simple" feature a three-sprint project.
The result: 70% of software project failures can be traced back to issues with requirements, according to PMI research. Not missing requirements. Wrong requirements. Requirements written by people who did not have enough visibility into what already exists.
Duplicated Work Nobody Catches
Two teams build slightly different versions of the same functionality because nobody knew the other team had already started. I have seen this happen with search implementations, notification systems, and permission models. One company I spoke with discovered they had three separate implementations of role-based access control, each built by a different team over 18 months.
Why 70% of Projects Fail
The Standish Group reports that 66% of software projects experience cost overruns, with an average overrun of 1.8x the original budget. McKinsey found that 17% of large IT projects go so badly that they threaten the very existence of the company. These statistics underscore why software projects fail at rates that have barely improved in two decades.
These are not edge cases. This is the default outcome.
The conventional explanation blames scope creep in software development: stakeholders keep adding requirements, engineers keep saying yes, and timelines blow up. That explanation is partially correct but deeply incomplete.
The Real Root Cause
Scope creep is a symptom. The root cause is that the people making decisions about software do not have accurate information about the software they already have.
When a CTO cannot tell the board exactly what features the product contains, the team will scope features that partially overlap with existing functionality. When a PM cannot see which files a change would touch, they will underestimate complexity by 2-3x. When an engineer joins a new team and cannot understand the codebase architecture for six months, their first few projects will take three times longer than expected.
The common thread is not incompetence. It is information asymmetry. The knowledge about how a system works is locked inside the heads of a few senior engineers who have been there the longest.
"The single biggest problem in communication is the illusion that it has taken place." -- George Bernard Shaw
This quote applies perfectly to software organizations. Everyone assumes someone knows how the system works. In practice, that knowledge is fragmented, outdated, and distributed across dozens of Slack threads, Confluence pages last edited in 2023, and the memory of an engineer who left last quarter.
The Visibility Gap Across Roles
The knowledge problem is not uniform, which is partly why software projects fail across so many different team structures. It affects different roles in different ways, and each gap compounds the others.
Product Managers
PMs need to know what features exist, how they work, and what technical constraints apply to new features. Most PMs get this information by asking engineers, reading outdated documentation, or guessing. A PM who writes a spec without understanding the technical reality is playing a game of telephone where the message degrades at every step.
Engineering Managers
EMs need to know where technical debt lives, which modules are fragile, and which team members are single points of failure. Most EMs get this information from sprint retrospectives and incident post-mortems, which means they learn about problems after they have already caused damage.
CTOs and VPs of Engineering
Senior technical leaders need to answer board-level questions: How healthy is our codebase? How does our feature set compare to competitors? What is the real cost of that migration we have been deferring? Most CTOs answer these questions with a mix of intuition and optimism, because they lack the data to do otherwise.
Developers
Engineers new to a codebase spend months building a mental model of how things connect. Stripe found that developers spend an average of 17 hours per week on maintenance tasks, including understanding existing code, rather than building new features. That is 42% of their working time spent on comprehension, not creation.
Each of these gaps feeds the others. The PM writes a bad spec because they lacked code awareness. The engineer builds it wrong because the spec was misleading. The EM misses the risk because they did not see the dependency. The CTO reports confidence to the board because nobody surfaced the problem.
Technical Debt as Invisible Tax
Cost overruns are only part of why software projects fail. Software project cost overrun statistics tell part of the story. But there is a less visible and often larger cost: technical debt's impact on development velocity over time.
Technical debt operates exactly like financial debt. You borrow against the future by shipping shortcuts today. The interest payment is the slowdown your team experiences every sprint as the codebase grows more complex, more fragile, and harder to understand.
The Compounding Effect
A small shortcut in Year 1 becomes a 10% velocity tax in Year 2 and a 30% velocity tax in Year 3. By Year 4, teams are spending more time working around old decisions than building new features.
This is not speculative. Stripe's research found that the global cost of developer time lost to technical debt is $85 billion annually. For individual companies, the tax shows up as:
- Features that used to take one sprint now take three
- Every new hire takes six months to become productive
- "Simple" bug fixes cascade into multi-day investigations
- Nobody volunteers to work on certain parts of the codebase
Why Teams Cannot See It
The fundamental problem with technical debt is that it is invisible to everyone except the engineers currently working in the affected code. Product managers do not see it. CTOs cannot quantify it. Board members do not know it exists.
This invisibility is why technical debt accumulates unchecked. You cannot manage what you cannot measure. And you cannot measure what you cannot see.
The standard approach to making technical debt visible relies on engineers self-reporting during sprint planning. This approach systematically understates the problem because engineers are incentivized to ship features, not flag debt.
How to Fix It
The pattern behind every problem described above reveals the deeper truth of why software projects fail: the people making decisions do not have access to the information they need. The fix is not more meetings, more documentation, or more Jira tickets. The fix is giving every role direct access to what is actually in the codebase.
Step 1: Make the Codebase Queryable
The first step is eliminating the bottleneck of asking engineers. Product managers, engineering managers, and CTOs need to ask questions about the codebase and get accurate answers without waiting for a senior engineer's calendar to open up.
This means tools that can parse a codebase, understand its structure, and answer natural language questions like "How does our authentication work?" or "What files would be affected if we changed the billing model?"
Step 2: Auto-Discover What Exists
Most organizations do not have an accurate inventory of their own features. They have a marketing website that lists product capabilities, but that list diverges from reality within months of being created.
Automatic feature discovery, where the tool analyzes the code and identifies distinct features based on code relationships, eliminates the manual cataloging that nobody maintains.
Step 3: Make Technical Debt Visible to Everyone
Technical debt needs to be surfaced in a format that non-engineers can understand. Not cyclomatic complexity scores. Not code smell counts. Business impact: "This module has a bus factor of 1, takes 3x longer to modify than average, and handles $2M in monthly transactions."
Step 4: Ground Specs in Code Reality
Every spec should reference the actual files, functions, and patterns it will affect. Not "update the user service" but "modify UserService.kt lines 45-120, which currently handles authentication for 3 downstream consumers." This is the difference between effort estimation based on guesses and estimation grounded in code.
Glue was built to solve exactly this problem. It indexes every file, symbol, dependency, and commit in your codebase, then makes that information accessible to every role on the team. PMs can ask questions in plain English and get answers with file references in seconds. Engineering managers can see technical debt quantified by business impact. CTOs can answer board questions with data from their actual codebase, not their best guess.
The platform auto-discovers features using graph-based clustering of code relationships, maps competitive gaps with impact scores, and generates development plans that reference specific files and patterns. It turns the codebase from a black box that only senior engineers can interpret into a shared source of truth the entire product team can use.
If your team is spending more time understanding existing code than building new features, that is not an engineering problem. It is an information access problem. And it is costing you far more than you think.
The organizations that reverse this pattern share a common trait: they stop treating code comprehension as an individual responsibility and start treating it as an infrastructure problem. They invest in tools and practices that make the codebase legible to everyone who makes decisions about it - not just the engineers who wrote it. They measure knowledge distribution the same way they measure uptime. And they recognize that the single most effective way to prevent project failure is giving every role on the product team direct access to the truth about what has been built.
See how Glue gives your team visibility into the codebase →
Frequently Asked Questions
Why do software projects fail?
The primary cause of software project failure is information asymmetry: the people making decisions about what to build lack accurate information about what already exists in the codebase. PMI research shows that 70% of project failures trace back to requirements issues, not technical problems. When product managers write specs without understanding the technical reality, when engineering managers cannot see where debt and risk live, and when CTOs cannot quantify what has been built versus what competitors have, projects are set up to fail before a single line of code is written. Scope creep in software development is usually a symptom of this deeper visibility problem, not the root cause.
How much money is wasted on failed IT projects?
Organizations waste an average of $109 million for every $1 billion invested in projects and programs, according to the Project Management Institute. The Standish Group reports that 66% of software projects experience cost overruns, with an average overrun of 1.8x the original budget. McKinsey found that 17% of large IT projects go so badly that they threaten the company's existence. On a per-developer basis, Stripe estimates that developers spend 17 hours per week (42% of their time) on maintenance and code comprehension rather than building new features, representing billions in lost productivity globally.
What is the main cause of software project failure?
The main cause is a lack of shared understanding of the existing system. While scope creep and poor estimation are commonly cited, both are downstream effects of a more fundamental problem: nobody has a complete, accurate picture of how the codebase works. When PMs, engineers, and leaders are all operating on different mental models of the system, every decision compounds the misalignment. Technical debt accumulates invisibly, requirements miss critical dependencies, and effort estimates are based on assumptions rather than code reality. Solving this requires making the codebase itself accessible and queryable by every role on the product team.