Wall Street’s AI Push Hits Memory Limits
At STAC, the race to use AI in trading is running into a hardware constraint. Plus the latest in AI + finance news.
Hey, it’s Matt. You’re reading AI Street, where I report on how Wall Street uses AI.
STAC
I’m in New York this week, and yesterday I went to the STAC Summit, a quant trading, AI, and infrastructure conference.
It was my first time there. A lot of the panels were quite technical, and I’m not going to pretend I’m a hardware expert, so I asked James Corcoran, STAC’s head of AI, for a main takeaway.
His answer: memory.
The technical panels kept coming back to the same problem: how to get more memory, how to use it more efficiently, and how compute providers are thinking about memory when they design chips and systems. One issue is the KV cache, which is basically the model’s working memory. As the context gets longer, the model has to store more of what it has already processed, and retrieve that information fast enough for the answer to be useful.
As Corcoran put it:
“Memory has become the new bottleneck.”
I think this is telling because it shows how quickly AI is becoming part of financial workflows, and how much pressure that is putting on the underlying infrastructure.
I’m still reviewing my notes, but one topic I’m going to come back to is how AI, and access to large-scale computing power, change the old trading tradeoff between being fastest and being smartest. If you’re holding a position for seconds rather than microseconds, the edge may come less from raw speed and more from how much data and compute you can throw at the decision.
A NOTE FROM OUR SPONSOR
Your team spends weeks sourcing data.
Vendor negotiations. Legal reviews. Data dictionaries. By the time the dataset is ready, the opportunity has moved. This is the hidden cost of how institutional data has always worked. It doesn’t have to.
Carbon Arc Lenses consolidates 150+ data assets under one contract, accessible immediately, queryable in plain English. Card spend, payroll signals, foot traffic, medical claims, trade data, and more, ready when your research is.
Pay for what you use. Nothing more.
Start querying with Lenses using code AISTREET30 for 50% off your first month:
Citadel Alumni Raise $78M to Bring AI Agents to Wealth Management
Moment, a fintech founded by former Citadel Securities quant traders and researchers, raised $78 million this week to build out the data infrastructure it says finance needs before AI agents can work with client portfolios.
The company’s pitch is that investment management software has grown into a patchwork of separate systems: one platform for bonds, one for rebalancing, one for compliance, and so on. That setup has worked because people sit in the middle, moving data, checking restrictions and reconciling outputs.
AI agents can’t operate in that environment because there’s no unified data model or audit trail. Moment built what it calls an operating system: a single platform with unified data, access controls, compute engines, and a full audit trail, on which agents can operate.
If a firm’s data and infrastructure are a mess, agents will produce bad results, burn unnecessary tokens and keep getting stuck because they do not know what to do or where to go.
Ken Griffin Changes Tone on AI
Andrej Karpathy, a founding member of OpenAI who joined Anthropic this week, wrote in late December that AI had moved from a better coding assistant into something more like “a powerful alien tool.”
He was reacting to the sudden jump in coding agents. The same realization is now working its way through finance.
Citadel’s Ken Griffin, who dismissed AI as “garbage” at Davos in January, told a Stanford Business School audience this month that work once done by finance PhDs over weeks or months is now being handled by AI agents in hours or days.
“You could just see how this was going to have such a dramatic impact on society,” Griffin said.
OpenAI Tests ChatGPT Finance Tool With Plaid
After acquiring two AI personal finance startups since October, OpenAI announced a preview of a ChatGPT finance tool that lets users link bank, credit card, investment and loan accounts through Plaid, then uses balances, transactions, investments and liabilities to answer personal finance questions.
I think people will be a little apprehensive at first about connecting all their financial information to a product that hallucinates. But as AI gets better, that will become more normal. Eventually, AI is going to take on more and more financial advice.
ICE Joins the Race to Price Compute
Last week, we talked about the emerging market for “compute” with CME and Silicon Data teaming up to create a futures market for computing power.
This week, Intercontinental Exchange and Ornn, a compute pricing data provider, announced plans to launch GPU compute futures contracts based on Ornn’s index data.
Foot, Meet Mouth
From the WSJ:
Standard Chartered Chief Executive Bill Winters touched a nerve when he said his bank would slash thousands of jobs and replace “lower-value human capital” with artificial intelligence.
He walked back the comments on Wednesday in a memo to bank employees, who turned out to be valuable enough that he needed to assuage their feelings.
The bank said earlier this week it plans to cut more than 7,000 jobs over the next four years as it relies more on AI.
As longtime readers know, this narrative doesn’t hold up. AI is a useful scapegoat for companies facing other problems in their business. If AI is such a job killer, shouldn’t JPMorgan, which has a tech budget of ~$20 billion, be cutting back drastically on headcount? The answer is no. Headcount is flat.
ROUNDUP
What Else I’m Reading
AI in the Family Office Citi
Google and Blackstone to Create New AI Cloud Company WSJ
The Supply and Demand of AI Tokens: Dylan Patel Invest with the Best
This Week in AI Street
SPONSORSHIPS
Reach Wall Street’s AI Decision-Makers
AI Street reaches institutional investors, C-suite executives and Big Law attorneys at firms including JPMorgan, Citadel, BlackRock, Skadden, McKinsey, and more. Sponsorships are reserved for companies in AI, markets, and finance. Email sponsors@ai-street.co for more details.
CALENDAR
Upcoming AI + Finance Conferences
AI & RegTech for Financial Services & Insurance – May 20–21 • NYC
Covers AI, regulatory technology, and compliance in finance and insurance.
Women in Quantitative Finance - May 21 • NYC
Quants discussing current work in asset pricing, trading, risk, and portfolio construction.
NY Tech Week Fintech Track - June 1-7 • NYC
Multiple relevant events: AI agents in finance, HSBC/a16z AI in enterprise fintech, AI for finance with Claude/Excel/MCP, AI agents in finance ops.The AI in Finance Conference - June 8 • University of Maryland
Academic AI + finance conference on LLM measurement error, AI regulation, analyst research, return predictability, and market fragility.
Thanks for reading!
I’m always happy to receive comments, questions, and feedback.
Connect with me on LinkedIn, or
Send an email to matt [at] ai-street.co