Hey, it’s Matt. This week’s AI Street is arriving a day early. Happy Thanksgiving! 🦃
🔎 Transformers Take on Time Series
💰️ Model ML Raises $75M to Automate Bank Tasks
🗞️ AI on Wall Street News Roundup
RESEARCH
Training Transformers on Stock Returns
A new study highlights the potential of using the technology behind ChatGPT on time series data
The current AI boom stems from the breakthrough of transformer models trained on language. But the benefits of this architecture extend beyond text. You can train these models on other types of data — the weather, grocery sales, bond portfolios — as long as you have billions of datapoints.
Netflix trained a model on viewer data. Stripe trained a model on payment data.
Results look promising: Netflix says the model learns more from a viewer’s history to provide better user recommendations, while Stripe says the model is better at spotting fraud.
The biggest bottlenecks to training these models beyond language are having enough data and computing power.
A new study cleared these hurdles by training a transformer model on up to 2 billion datapoints on daily stock returns over 34 years and 94 countries for about 50,000 GPU hours.
Researchers at Manchester, UCL, and Shanghai University say this is the first comprehensive study of Time Series Foundational Models in global markets. Time series data is any set of observations recorded in order over time (think temperature, daily sales figures, etc).
Here’s what they did:
The researchers took two popular "foundation models" for time series forecasting (Chronos from Amazon, TimesFM from Google) and asked them to predict next-day stock returns.
They trained these models from scratch by using only financial data. They then compared the model they built against off-the-shelf versions trained on generic time series data. They compared all of this against simpler, well-established methods that quants already use (specifically ensemble models like gradient-boosted trees, which are basically very sophisticated decision trees).
Here’s what they found:
The off-the-shelf models flopped.
When you just download these foundation models and point them at stock data, they perform terribly — worse than much simpler techniques. Fine-tuning helped a bit, but not enough to close the gap.
Training from scratch worked surprisingly well.