Data quality is deteriorating.
A Neudata consultancy survey found that one-third of investors purchasing data say its quality has declined over the past several years, and some blame artificial intelligence.
Data is essential to any investment operation, and hedge funds have used so-called alternative data—information not derived from conventional sources such as SEC filings or exchange trading records—to inform their bets for years.
AI has prompted a surge of new data vendors, as it has become easier to collect and organize information, while also changing the practices of established sellers.
“There is an enormous amount of data available now, but there are not many experienced data scientists. These newer companies are simply running it through LLMs,” said a fundamental equity investor who buys outside datasets and has observed declining quality.
“Hallucinations still occur,” he said.
Why AI Is the Culprit
AI is damaging data quality in two ways: providers may use it to create or map the datasets they sell, or they may invoke AI as a reason to reduce or reassign staff responsible for quality control, said Daniel Entrup, cofounder of AggKnowledge, a data-products company that works with both buyers and sellers.
Both practices create problems. Buyers say companies that use AI to produce datasets for hedge funds often cannot explain how a final report or dataset was generated—a crucial detail funds need to comply with federal rules governing information access and privacy. The model used by the vendor may also be poor.
“The data may exist, but the LLM may be terrible,” said Daryl Smith, head of research at Neudata.
Hedge funds, which often have larger technology budgets and more advanced teams, would rather process data through their own systems than depend on a vendor’s AI model. In addition, the work performed on raw data is part of many managers’ proprietary edge.
“Funds trust AI with their workflow far more than they trust it with their alpha,” Smith said.
Entrup has also seen more basic data-cleaning problems in recent months as industry attention has shifted toward discovering new AI applications. His company performs data “revision” work that corrects errors in client datasets, and he said this type of work has increased substantially.
“How important is quality? For a hedge fund, it is extremely important,” he said.
Many buyers are frustrated because these AI projects were not initiatives they had asked their data providers to pursue.
“Show me the customer research showing that clients requested this,” Entrup said.
Data vendors that have incorporated more AI into their processes have not seen stronger business results. One buyer at a mid-sized hedge fund said the firm would use a vendor’s LLM if it came at no cost, but not if it required payment. Smith said his company has not observed a meaningful revenue increase among vendors using AI tools.
A single hallucination can compromise an entire dataset.
“Once a signal fails, the data becomes contaminated,” said one hedge fund manager.