The world has been turned upside down by COVID-19 and civil demonstrations during the last few months. The effects on our society, the overall economy, and just about every industry are unprecedented. April and May look absolutely nothing like January or February, and with the situation still unfolding, June and July will be completely different as well. One of the less obvious impacts of this period of rapid change is to the behavior of AI models that play critical roles in our society.
At Arthur, we’ve seen models across many industries affected. Trading algorithms are one clear example, as recently covered in the Wall Street Journal.
“Hedge funds that use artificial intelligence models to suggest trades and stock picks declined when stock markets unraveled in late February... market experts warn the unpredictable nature of the economy these days could trip up some algorithms that continue to rely on data gathered during better times.”
-WSJ, AI Funds Decline—Then Recover—During Market Turmoil, May 1, 2020
Let’s examine why these models are so vulnerable.
The Shortcomings of Machine Learning
Machine Learning (ML) is the most prevalent form of AI today, to the point that the two terms are often used synonymously. ML works by learning patterns in existing data (aka the training set) and then using those learned patterns to reason about future data. It performs quite well when that future data is similar to the training set. As that similarity disappears, the lessons the ML model has learned become less relevant and the model becomes less able to “reason” correctly about the new data it is seeing.
In a recent MIT Technology Review interview, Gary Marcus, a vocal critic of deep learning-based approaches to AI said “I think that we’re living in a weird moment in history where we are giving a lot of trust to software that doesn’t deserve that trust.” So what are these shortcomings and how do they relate to our current situation?
Consider the ancient game of Go. We all watched as AlphaGo proved that algorithms had finally surpassed humans on this extraordinarily complex challenge. But what if we were to replay those games, adding in a little twist. In this new game, every few moves the rules get altered slightly -- additional rows of dots added, the ability to place more than one stone per turn, etc. This time around, the human would best the algorithm because it would never have seen any data like this in the past. The human, while faced with a more difficult task than a routine game of go, would still be able to play relatively effectively.
Simply stated, current approaches to AI do not adapt well to changing situations.
AI in the age of COVID-19
Right now, the automated systems affecting our finances, healthcare, travel, purchasing intents, and many other aspects of our life are making decisions based on patterns they learned prior to COVID-19. So are models that make business decisions in capital markets, high frequency trading, supply chain management, and manufacturing. But those patterns are much less applicable now; the world has shifted.
These shifts and models’ inability to adapt will result in companies losing money at a time when they are already being strained. Meanwhile, consumers and end users who interact with these systems will be on the receiving end of less reliable decisions.
Is there a solution?
There are avenues of research that seek to solve this problem. Reinforcement and online learning focus on models that continue to learn as new data flows in, so they aren’t limited to patterns learned from a “point in time” snapshot training set. But these approaches are often hard or impossible to apply to real world scenarios.
Symbolic AI is an alternative to deep learning that seeks to explicitly model truths about the real world and then allow machines to programmatically apply formal logic to these truths to draw conclusions. It has the huge advantage of being able to reason about scenarios it has never seen before. While it has been out of fashion for many years due to reliability concerns, it has had a resurgence in the last couple of years as researchers grapple with the shortcomings of deep learning.
These types of scientific advances are a ways off and will require time to mature to the point they can be utilized in the real world.
The immediate, pragmatic solution is to make your AI observable. Systematic model monitoring allows you to immediately assess the shifts in the data feeding your models, and understand the degree of impact it is having on model decisioning. With this knowledge you can respond immediately, smartly prioritizing model performance improvements to ensure your business maintains maximum continuity.
Going forward
People who have worked in data science long enough have battle scars from model performance loss. Sometimes it’s quick, such as when you encounter a black swan event. More frequently, it’s gradual. Often the gradual degradation is worse because it flies under the radar for a long time.
A lot will change as a result of COVID-19. People are already questioning social norms that go back centuries -- handshakes, sporting events, working in an office, etc. For those of us in the ML community this is a wake up call to build systems that are resilient. That starts with building AI that is observable. Knowing when your AI is struggling to adapt to rapid change and which models are the most impacted is critical for real world systems. And longer term, continued research into more adaptable approaches to AI such as symbolic AI and reinforcement learning will be paramount. Automated systems are creating extraordinary value -- making them more observable will ensure that continues even when the world shifts.
SHARE
