Analysis: Bad FX Data Holding Back Machine Learning Developments

Machine learning has long been a favourite topic of conversation in the corp-speak world. Scroll down your LinkedIn feed and you’re sure to see some video, complete with annoying subtitles, talking about “a shocking new machine learning robot disrupting the tech industry.”
But outside of Gary Vaynerchuk’s Twitter feed, is making real headway in some industries. Nowhere is this truer than in the world of financial services.

A new report, published by on Monday, indicates that adoption of the technology is actually greater than previously thought.
In fact, according to the report, 90 percent of firms now deploy machine learning to analyse content with 78 percent of companies also saying that the technology is a core component of their business strategy.
Not all easy
Still, that doesn’t mean it’s all smooth sailing for firms using machine learning. Refinitiv’s report indicates that there have been a number of challenges in adopting the technology and getting it to perform specific tasks.

In the foreign exchange (FX) markets, 39 percent of firms were using market data to apply machine learning-based models. That was compared to 72 percent in equities, 63 percent in fixed income and 40 percent in derivatives. The reason for these disparites is due to the data that firms have access to.
“Like fixed income, there is no standardised messaging language [in the FX markets] so data from different venues and market data providers is reaching the institution in multiple formats,” said Matthew Hodgson, the chief executive officer and founder of FX data provider Mosaic Smart Data.
“For machine learning to provide a view across all channels of activity in the market, you must normalise data from across all of these venues. Doing so is a huge engineering challenge.”
Sorting through the pile
Financial institutions are also struggling to deal with huge volumes of poor quality data. In its report, Refinitiv writes that for data scientists at some firms, normalising and cleaning data, was taking up 90 percent of their time.
To be fair, it appears to have only been a small number of outlying firms that were devoting this much time to sorting through poor quality data. Nonetheless, on average firms are – regardless of business type – spending 30 percent of their time dealing with this particular problem.
For Curtis Pfeiffer, chief business officer at algo-trading tools provider Pragma, FX teams specifically are not only dealing with bad information, but actually lack some of the data necessary to build machine learning-based trading models.
“I think we’re still in early phases for AI in the FX market,” Pfeiffer told Finance Magnates.
“Looking at the Refinitiv report, a lot of attention is being placed on using machine learning to analyse company data, but these kinds of reports usually won’t generate useable signals to power FX decision making. So I think further research is needed before we know whether there are analogous data sources for the FX market.”
More money or better data?
If Refinitiv’s report is to be trusted, communicating these data problems to senior management is going to be an important part of solving them.
A survey carried out by the data provider found that data scientists rank poor quality data as the biggest hurdle to greater adoption of machine learning. For c-level executives, this was secondary to a lack of funding.
These two opinions may not actually contradict one another. C-level executives may think that more funding would equal better data. Thus, if funds are lacking to provide better data, then, for those senior executives, a lack of funds becomes the biggest barrier to machine learning adoption.
But if simply sinking more money into machine learning projects won’t improve data quality, then there’s no point in doing it.
In either case, Refinitiv’s report is indicative of what many people have been saying for a while – machine learning is coming but it’s going to take some time for the technology to start working at full capacity.