50 3 Shades of Grey Market Data
By Bijesh Amin, co-founder of Indus Valley Partners
The advent of Big data and Machine Learning as increasingly strategic priorities for the fund management industry has overlooked an important aspect in the underlying evolution of market data.
In its widest sense one can think of market data encompassing a "visibility spectrum" (see graphic below). The traditional "light" market data is well-known, well structured, well analyzed and readily available such as a quarterly earnings report. The "grey" data is accessible to only those players in the market with the means and contacts to access them, for example hedge funds that can tap expert networks in the bio-pharma industry to assess the success of a clinical drug trail. Or an investment bank research analyst speaking with a company CFO about operating margins.
But it is in the realm of "dark" data where things get interesting from the perspective of technology. Examples of the value embedded in these non-traditional data sets could be predicting demand for a new product from online marketplace purchasing data or analyzing parking lot patterns at retail stores to gain insight into store footfall. Unlocking the value in this "dark" data is only possible - and commercially viable - by using a combination of machine-learning techniques, big data and cloud computing. This combination of technological and analytical expertise is exemplified by firms such as Descartes Labs - a start-up that uses satellite imaging data sourced from another startup (Planet) for predicting crop yields prior to the official USDA crop report.
Mainstream asset managers - particularly quantitative traders such as hedge funds are actively looking to tap "dark" data in their pursuit of an investing "edge"; looking to social media-based sentiment analysis to confirm an investment thesis or find long/short trading opportunities.
"The Market Data Spectrum: from Visible (Light) to Non-Visible (Dark)"
The rapid digitization of economic activity means the internet itself is effectively becoming where initial pricing insight can occur across a multitude of sectors, products and commodities. These “signals” may be discernible prior to the more formalized price-discovery process that occurs on established markets and exchanges. And this insight can easily translate into an investment advantage for an investor or trader.
There is a corollary here in the emergence of "dark pools" of liquidity - where price-discovery happens away from traditional stock exchanges and benefits those "inside" market participants such as large fund managers or institutional investors that wish to minimize the market impact of large trades. However, the internet (as it currently stands) is more democratic than a gated intranet such as dark liquidity pool. The "insiders" in this case will be those that possess the tools, techniques and technologies required to unlock and interpret the value hidden in the terabytes and petabytes of "dark" data.