Verify

After we Quantify a company and it clears our high bar, we invest time verifying the story. Up until now, we have trusted data from the company and quick web searches. Now is the time to Verify our information bases. Here are some validations we perform.

• Verify Contracted Revenue

• Verify Customer Enthusiasm

• Verify Market Size

• Verify Governance and Board

• Verify the Team by Playing Advisor for a Day

• Find a Devil’s Advocate

Verify Contracted Revenue

Founders say the darndest things when talking about “revenue” and “sales.” We have seen founders include government grants and even invested capital as revenue. We have seen founders refer to cumulative revenue over multiple years as “revenue” without a “cumulative” qualifier. We have seen founders categorize the full value of a multi-year contract as monthly revenue. The list of inappropriate accounting and pitch-deck representations goes on.

We dig into monthly income statements, request corresponding contracts, and verify the revenue, ARR, MRR, ACV, or other flavor of top-line accounting being used. For SaaS companies, we prefer to look at ACV (annualized contract value). If a contract was signed on December 31, 2019, but billing won’t start until February 2020, the contract will be reflected in 2019 exit ACV but not 2019 revenue. 

“We just signed a contract with BigCo” claimed one founder. When asked for the contract, the founder provided a “Master Service Agreement” (MSA) that was indeed signed, but with no order schedules completed. So it was just a pro forma. After calling the contact at BigCo, we were able to verify that BigCo is indeed interested in potentially doing business with NewCo and that they might ultimately place large orders. Still, the MSA is no guarantee of any future revenue. Further, BigCo was auditioning several of the startup’s competitors for the potentially big orders.

Verify Customer Enthusiasm

One CEO stated: “We have active customer engagement. We have 4 pilot projects currently, and we expect them to convert to big contracts in 6 to 12 months.” When asked to speak with all 4, the CEO backpedaled and clarified that only 2 of the pilot sites had actually tested it so far. We spoke with those two pilot sites, each at a different company. Both commented on how they really liked the founders and were happy to help them out by testing the systems; however, the startup’s technology had real problems in the field. These companies had zero interest in purchasing from the startup since they were deeply committed to an alternative, superior technology.

Another CEO: “We did a 2-system proof-of-concept (POC) at the customer site. The success of that free POC led the site to agree to a paid pilot of 50 systems, one for each piece of equipment to be monitored at the site. They will start the paid pilot in a few weeks and after a few months of data hope to roll this out to their other 7 sites for a total of 350 systems.” A site visit confirmed the story. The pilot sponsor at BigCo described the problem exactly as the startup had represented and was able to show a real example of the problem ongoing at the time of our visit. He confirmed the problem is real and is large and that they don’t have any other solution. He validated the pricing model, verified that the POC was a success, and confirmed that they are excited to start the paid pilot.

Verify Market Size

Market sizes must be confirmed bottom-up, not top-down. Unfortunately, top-down market sizing is prevalent in pitch decks. “Health Care is a trillion-dollar industry. We conservatively estimate that our product can capture 1/10th of 1%; therefore, we have a $1b market opportunity.” Nonsense.

Bottom-up market sizing can take more work but is necessary to avoid delusion. Sometimes the founder has done a great job with a realistic bottom-up market sizing and has nicely documented the assumptions and sources. In this case, verification is easy. Sometimes the founder has the information in their head or scattered across reports. Working with the founder to piece it together can be very educational, providing insights into the market and the founder’s horsepower.

We were investigating a therapy for a nasty pediatric orphan skin disease. The therapy was promising in part because it had demonstrated strong efficacy in humans for a different skin disease. Orphan drugs are, in theory, somewhat immune to market size since the fewer the number of patients, the more a pharma company can charge per patient. The net is that the annual market size for orphan drugs, where ongoing treatment is required, tends to range between $700m and $1b/yr, regardless of the number of patients. However, that assumes you accurately understand the number of patients in your market. If you believe the market is 2x its actual size, your pricing will be ½ what it could have been, and you have cut your market size in half. An accurate understanding of the actual market size is important. There is a non-profit foundation for this orphan disease that provides various helpful advocacy resources for patients and others concerned with the disease. They estimated the prevalence of the disease as N patients in the U.S. Because of the prominence of this organization, when people wanted to know the number of patients with the disease, they turned to this non-profit and parroted the number N. The number N was quoted in financial models by Wall Street analysts, in news articles, by prominent universities, and by the startup. The problem, Joe discovered, was that the number N was inflated by about a factor of 10. It turns out that there is a registry of all patients with the disease. Once every 5 years or so, an unbiased academic, working under a government grant, counts additions to the registry and meticulously attempts to track down all patients added to the registry in the last 20 years to learn their status. It was clear that the academic was an expert in epidemiology (the study of the incidence and prevalence of disease) and that she had been extremely thorough and rigorous in her analysis. We tried to understand the ultimate source of the foundation’s number. Buried in the foundation’s website was a statement on the topic that was not at all compelling. In contrast, the academic wrote a post-script in her latest report explaining the vastly different estimations of prevalence between her findings and the foundation’s. Her critiques were presented with civility; however, they demonstrated with incisive clarity that the foundation had major flaws in their methodology. It turns out that an advocacy foundation has a motivational bias to claim as many victims of the disease as possible. This increases the problem’s perceived size and may lead to more attention and resources being paid to the disease. This CEO had not done their homework.

Verify Board and Governance

In the zombie companies we have seen (stumbling about, not quite dead, not growing), there is often a lack of governance and accountability of the CEO to the board. 

One company has a world-leading technologist as a founder. The CEO added to the board a person who made the first (albeit tiny) investment but who has zero business operating experience. The CEO also added a seasoned engineering manager in the space who also made an early tiny investment. None of the three has experience with developing a sales force. The company continues to struggle with sales as a result.

Recently we were attracted to a company with breakthrough technology in bioengineering. The innovation would greatly reduce the cost and time to market of wide classes of bioengineered therapies. The founder/CEO is a brilliant technologist. However, we didn’t feel he was a great CEO. Progress had been slow, some strategic missteps had been made, and forward strategy was murky. Joe spoke with the board members individually and floated the idea that perhaps the CEO should be made the CTO and a new CEO brought in. They agreed that that would be an improvement, but they lacked the resolve to make it happen. Joe told them to call back if they hire a new CEO (although we would probably want to see a new board as well.)

Before investing, we need to verify that a professional investor is on the board. We serve that role when appropriate.

Verify the Team by Playing Advisor for a Day

In some cases, where more visibility into the team’s intelligence, motivation, or interpersonal dynamics is needed, we will roll up our sleeves and play advisor for a few hours. A CEO can be very polished and on-script during a presentation where they can exude competence and confidence. During the Analyze stage, if we pick up hints of strategic questions, dilemmas, or challenges, we may volunteer an advisory working session to dive into the issue. This can uncover rough spots in team chemistry. It can also uncover competitive issues or other threats that never make their way to pitch decks.

Find a Devil’s Advocate

This final verification is aimed at verifying our understanding and reasoning. Before investing, we write up a deal memo with a summary of our analysis and a quantification (pwMOIC) of the deal. We show the memo’s factual parts to the CEO to see if we have made any errors in understanding. We show the whole memo to a trusted devil’s advocate, such as one of our advisory partners, to challenge our thinking. Often the challenge will force us to go back and verify an issue we had taken for granted. Often our analysis survives the challenge. Sometimes it does not. Either way, the decision’s quality is improved.

Verification is important since the devil is in the details.