The number of listed companies in the U.S. rose 50 percent from 1976 to 1996 and fell 50 percent from 1996 to 2016. This has not happened in other parts of the world, opening a U.S. listing gap. This is important because the U.S. comprises one-half of the value of the world’s stock market.
A company’s decision to list involves weighing costs and benefits. Net benefits appeared to be positive in the first 20 years of this period and have turned negative in the last 20 years. As a result, delistings have exceeded new listings by a large margin since 1996.

Regulation appears to have played a role in two ways. The cost of being public, especially after the implementation of the Sarbanes-Oxley Act in 2002, has risen in the past two decades. That said, the shrinkage in the population of listed companies started well before that law was implemented. Further, relatively accommodative anti-trust enforcement allowed for robust M&A activity.

As a result, listed companies today are on average larger, older, and more profitable than they were 20 years ago. Further, they operate in industries that are generally more concentrated. The overall size and maturity of listed companies means they are more likely to pay out cash to shareholders in the form of dividends and share buybacks than companies were in the past.

We speculate that the maturation of listed companies has also contributed to informational efficiency in the stock market. Gaining edge in older and well established businesses is likely more difficult than it is in young businesses with uncertain outlooks. In turn, the greater efficiency may be one of the catalysts for the shift that investors are making from active to indexed or rule-based strategies.

The chief investment officer (CIO) of an institution in the mid-1970s could gain reasonable exposure to U.S. equities by investing in an early stage venture fund and a large market index such as the S&P 500 (itself not an easy thing to do at the time). Today, that CIO needs to participate in early- and late-stage venture capital, a private equity buyout fund, and the S&P 500. Only a few investors have access to all of these alternatives.

The universe of alternative investments, including venture capital, buyout funds, and hedge funds, has grown sharply in the past 20 years to provide some investors with access to more investment opportunities as well as to employ more sophisticated methods to generate excess returns. The growth of these asset classes has led to lower returns for investors.

Venture capital funds launched in the 1990s outperformed public markets. But funds started since 2000 have underperformed public markets, with an improvement in recent years. Buyout funds with vintage years before 2006 outperformed public markets, but those launched in the last decade have only equaled the returns of the market. Hedge funds have also seen diminishing excess returns in the past decade. The difference between the top and bottom performers is larger in venture capital than in buyout funds.

Eoin Treacy's view

Here is a link to the full report.

Regulation and cost are a major obstacle to listing. Meanwhile the very low interest rate environment means venture funds are flush with cash to bid up the prices of private assets. Together that has contributed to the ability of unlisted companies to accrue massive valuations while also allowing established companies’ source capital for takeovers and buybacks. 

Less supply and stable demand, not least from IRAs and pension funds represent a significant tailwind for stock markets. Meanwhile share buybacks are one of the clearest conduits for easy monetary policy to reach the market. Together they help to explain the persistence of the bull-run. 

The eventual potential for rising rates, higher yields and a shrinking Fed balance sheet represent headwinds for these trends. I watch the PowerShares Buyback Achievers ETF/S&P 500 spread for signs that the environment might be changing. The ETF outperformed spectacularly for as long as quantitative easing was a significant influence on the market. However its performance has been much spottier since 2014. That suggests the funding conduit for the stock market from buybacks is less influential that it was earlier in this bull market. 

The S&P500 Index was due some consolidation of the powerful move posted in February and is now getting just that. This has been a particularly impressive five-month rally and a deeper corrective phase cannot be ruled out over next two quarters. Nevertheless, a sustained move back into the underlying range would be required to question medium-term upside potential. Politics is likely to continue to play an important role in how the trend unfolds.