Reading Noise: A Flaw in Human Judgment, by Daniel Kahneman, Olivier Sibony, and Cass Sunstein, brought me back to an exercise I have used in my teaching: examining startup cemeteries, collections of postmortems describing ventures that closed, often after exhausting their seed capital.
I ask students to examine what happened. What did the founders believe? What evidence supported those beliefs? Which assumptions survived longer than they should have? What might someone have questioned before the money ran out?
These accounts are incomplete. They are retrospective explanations, sometimes written by people defending their decisions or trying to make sense of a painful outcome. We rarely get the full customer evidence, board discussions, or alternatives available at the time. Knowing how the story ends also makes earlier mistakes easier to see.
Still, a pattern repeatedly emerges in our discussions: a confident development path ends with some version of “we built something people did not want enough to buy.”
By then, the product exists and the runway does not.
Most of my students reach a similar conclusion. Early in an innovation journey, understanding what customers are trying to accomplish, where they struggle, and which underserved outcomes matter enough to motivate a purchase or a switch deserves serious attention.
That work can often begin before extensive development, hiring, and market entry. Interviews alone cannot establish demand, and some questions require a working product. But investigating customers’ circumstances and existing behavior can expose weak assumptions while changing direction is still relatively inexpensive.
The lesson seems accessible. The cases are public. The methods are familiar. People can explain why building on an untested demand assumption is dangerous.
So why does recognizing the mistake in someone else’s postmortem fail to prevent the next organization from funding it?
My interest in this is also personal. I have encountered these problems in corporate innovation decision boards and as a startup co-founder working with my own advisory board and board of directors (and it was also the topic of my master thesis). Across those settings, I experienced differences in judgment and shared assumptions that I now examine through the lens of noise and bias.
I was part of those judgments too. My proximity to a venture gave me information others lacked. It did not make my interpretation objective.
Noise sharpened a question that connects those experiences: how much uncertainty lies in the venture itself, and how much avoidable error enters through the judgments of the people evaluating it?
One venture, a hundred boards
A postmortem gives students an ending. A live investment proposal gives executives a team, a prototype, encouraging customer comments, commercial projections, and a request for resources.
The decision-makers bring experience. They recognize patterns, recall previous cases, and apply professional judgment. Their colleagues may find the interpretation convincing.
But would another equally qualified board see the same venture in the same way?
What about a tenth board? A hundredth?
Give them the same evidence, objectives, and resource constraints. Ask each to assess what has been demonstrated, what remains assumed, and what commitment is justified. Would their judgments cluster closely or scatter widely?
The board making the actual decision rarely sees that distribution. It sees its own interpretation, discussed among people who may share professional backgrounds, organizational norms, and years of exposure to the same internal information.
The shooting-target illustration in Noise makes an important distinction visible. Shots can scatter around the center, cluster away from it, or combine both problems. Noise concerns unwanted variability in judgment. Bias concerns systematic error.[1]
For a venture, however, the center is partly hidden. The commercial outcome has not happened yet. We can compare judgments before we can establish their accuracy.
Independent errors may partly offset when judgments are appropriately aggregated. But a board does not necessarily aggregate independent assessments. It may follow its most influential member. Its members may also accept the same misleading premise.
A tight cluster demonstrates agreement. It does not establish accuracy.
Nor is every difference a defect. One reviewer may recognize a purchasing barrier others have missed. Another may understand why enthusiasm for a prototype will not translate into switching behavior.
That disagreement is useful if it reveals information or assumptions that improve the decision. Other differences may reflect conflicting objectives, acceptable losses, or unexplained inconsistency. They require examination before being dismissed as poor alignment.
When contrary evidence changes nothing
In corporate innovation, I have presented primary market research that challenged claims made in venture proposals.
What had been described as demonstrated demand looked materially different in my findings. Expensive prototypes existed, but the evidence connecting a relevant customer struggle to willingness to pay or switch remained weak. Proposals to proceed depended on economics, acquisition approaches, or ecosystem arrangements that were still insufficiently established.
These distinctions matter. A useful product does not automatically establish a viable business. Paying customers do not automatically establish a repeatable acquisition model. Equally, a weak description of the customer’s job does not itself disprove demand.
My question was whether the evidence justified the next commitment.
In those cases, the discrepancies were tolerated by the decision board and the projects continued. What remained unresolved was the gap between what was claimed and what the research could substantiate.
Continuing was not necessarily the error. An uncertain venture can deserve another experiment. The problem was proceeding without adequately correcting the account of what had already been established.
“We are testing demand” and “we have demonstrated demand” describe different investment cases.
The first can justify buying evidence. The second can justify expenditure that depends on the evidence already being there.
Contradictory research does not automatically prove that a venture will fail, or that its supporters are biased. Samples, segments, prices, and conditions must be compared. But the discrepancy needs an answer.
If favorable findings repeatedly count as confirmation while unfavorable findings leave the claim untouched, the process is favoring one conclusion.
My experience as a co-founder also makes me cautious about assigning this problem exclusively to boards. Founders interpret the market before presenting it. We decide which observations matter, which objections seem temporary, and which previous cases appear relevant.
Even an honest presentation is a selection of evidence. The board then judges both the venture and the founder’s interpretation of it, sometimes without clearly distinguishing the two.
Experience without reliable feedback
Repeated decisions do not automatically produce better judgment. They can produce greater familiarity and confidence without a corresponding improvement in accuracy.
Kahneman and Klein argue that reliable intuitive expertise requires sufficiently predictable patterns and opportunities to learn through useful feedback. Subjective confidence alone does not establish expertise.[2]
Venture assessment can provide difficult learning conditions. Outcomes arrive late. Markets change. Funding and execution influence results. Rejected opportunities may never produce an observable outcome. Unless original forecasts are recorded, decision-makers may not discover which judgments were consistently too optimistic or too pessimistic.
An executive can become highly practiced at reviewing investment proposals without becoming equally accurate at assessing their prospects.
A board may accumulate experience faster than it accumulates evidence about the quality of its own judgment.
Shared experience can also make a particular interpretation feel obvious. Ross and Ward describe naïve realism: experiencing our own view as objective and expecting other informed, reasonable people to reach similar conclusions.[3]
Inside an organization, a contrary assessment may consequently appear to reveal the outsider’s lack of understanding before it prompts an examination of the prevailing assumptions.
An outsider is not automatically right. Their evidence and methods require scrutiny too. Their value lies in making reasoning visible that the organization may otherwise leave unquestioned.
A single independent question can expose a consequential gap: “What evidence establishes that these customers will switch, rather than merely express interest?”
The answer may reveal disagreement within the board. It may also reveal that everyone accepts the same unsupported inference.
Interested in working together?
Agreement can conceal disagreement
Authority adds another complication. Research by See and colleagues found that greater power was associated with less advice-taking, with elevated confidence helping explain the relationship.[4] That does not diagnose any particular executive. It identifies a risk where confidence carries substantial influence.
Someone leading the discussion may sincerely believe they recognize the case. Someone else may see a weakness but judge the relationship cost of challenging it too high. The resulting silence can resemble confirmation.
Kahneman and Sibony describe how discussion can conceal differences when participants converge around a senior person’s opening assessment.[5]
A board can therefore align on an action without agreeing on why it is justified. One member believes demand is established. Another believes the next expenditure will test demand. A third assumes someone else has checked.
The vote is clear. The investment premise is not.
Genuine alignment allows disagreement to remain explicit. Two reviewers can estimate different chances of commercial success and still support the same bounded experiment because its results will inform the next decision.
The problem begins when agreement on an action is treated as agreement on the evidence, and agreement on the evidence is treated as proof that it is correct.
Meanwhile, commitments accumulate. An assumption about demand supports a revenue forecast. The forecast supports hiring, development, and market entry. Those expenditures increase the cost of being wrong without necessarily strengthening the original assumption.
Resources consumed here are unavailable elsewhere. Learning may remain if the venture closes, but its value depends on subsequent use. An archived presentation does not automatically repay the expenditure.
That is why the postmortem lesson is insufficient. Knowing that customer demand matters does not establish that founders, advisers, or boards can reliably judge the evidence of it.
Nor does a successful outcome retrospectively validate every decision that preceded it. A venture can succeed despite weak reasoning, just as a justified bet can fail.
Before funding the next stage, examine how the board knows what it believes it knows.
The paid section below provides a Venture Judgment Review worksheet and a worked example for exposing consequential differences before another commitment becomes difficult to reverse.
Thanks for reading INNOVATION&! This post is public so feel free to share it.
Start paid learning and application: the Venture Judgment Review
Use this review before a consequential funding decision. It is a practical adaptation of independent assessment, not a validated diagnostic instrument.
Its purpose is to distinguish evidence, interpretation, and investment preference before discussion turns them into one apparently shared conclusion.
