AI in Mid-Frequency Trading
The Pareto frontier in trading, Jane Street on Dwarkesh, and new AI + finance calendar updates.
Hey, it’s Matt. I’m a former Bloomberg News reporter, and you’re reading AI Street, where I report on how Wall Street uses AI.
This week: The rise of mid-frequency trading and more takeaways from STAC Summit, Jane Street on YouTube with Dwarkesh and updated events in the AI & Finance calendar.
A NOTE FROM OUR SPONSOR
Card spend. Foot traffic. Payroll signals. Medical claims. TikTok Shop revenue. Trade data. Vehicle registrations. Web traffic. Job postings.
210+ data assets. One platform. One contract.
Carbon Arc Lenses makes this entire data infrastructure accessible to anyone who can ask a question in plain English. Daily refresh. Pay-as-you-go pricing. No SQL. No data team. No waiting.
Try Carbon Arc Lenses using code AISTREET30 for 50% off your first month.
Last week, I attended STAC Summit, a quant trading, AI, and infrastructure conference. As I wrote, memory dominated most of the conversations: how to get more of it, use it more efficiently, etc.
I’ve had a chance to review my notes, and I wanted to highlight a few more notable topics.
LLMs in Mid-Frequency Trading
High-frequency trading is now so fast it approaches the speed of light, so it’s hard to find an edge against a law of physics.
But if you can hold a position for a longer period, say, a few seconds, a minute, you don’t have to worry as much about being slow.
A few speakers were clear that nanosecond or sub-millisecond trading paths are not where LLMs fit. Their “intelligence” is too slow to compete against traditional HFT firms, but LLMs are more than fast enough to compete in workflows that used to depend on human discretionary traders.
One speaker described this tension as the Pareto frontier between speed and intelligence.
At one end of the curve, you have ultra-low-latency trading: very fast, very constrained, not much room for token generation or multi-step reasoning. At the other end, you have slower research workflows where a model can take seconds or minutes to read, reason, summarize, validate, and produce a richer answer.
In legacy market structure, being fastest could compensate for not being the “smartest.” In medium-frequency settings, the balance can shift. If you are a little slower but can extract better information from news, filings, transcripts, market context, or cross-asset signals, the edge may come from better interpretation rather than pure speed.
If a strategy has a 100ms, 200ms, or multi-second budget, the question becomes: how much smarter can you make your model in that time window? How do you make your model smarter? Here are some use cases highlighted:
Parsing or understanding news feeds as an additional trading signal
Converting unstructured information into sentiment, classification, or model inputs
Selecting among execution algorithms before the actual order path
Using AI to improve research loops and strategy development
Using models for pre-trade and post-trade workflows rather than live HFT execution
Applying “thinking time” where more reasoning may produce a more predictive or better-validated answer.
TL;DR
The mid-frequency opportunity is a Pareto tradeoff: firms are not trying to make LLMs beat HFT engines on speed; they are trying to use faster inference to move outward on the curve.
AI Is an Operations Problem
Firms have bought or are building expensive AI infrastructure, but many are not squeezing every bit of computing power out of these GPUs.
One speaker used a rough utilization figure around 55% from customer survey data, and said the real number may be lower because people do not always want to admit how poorly their infrastructure is running.
GPU clusters are systems, not just chips. A job can slow down because the GPU is waiting on the network, the storage system, the scheduler, or the application code.
Some issues highlighted:
Weak observability: firms cannot see why utilization is poor.
GPUs booked but waiting: data, storage, memory movement, or CPU preprocessing holds them up.
Small faults compound: bad cables, degraded links, memory faults, or data corruption slow jobs.
Not testing the full stack before deployment: components work alone but fail or underperform together.
Underestimating the buildout: power, cooling, cabling, rack density, etc.
TL;DR
The hardware behind AI is not being used at full capacity. Blame a lack of evaluations. But more charitably, the sheer logistical complexity of turning silicon chips, electricity, and cooling systems into intelligence is not easy.
AI Models Need Better Data Infrastructure
Another topic that came up, that often gets overlooked, is the data around the model. If an AI system has to search through all your files to answer a question, you’re wasting time and burning tokens. (This is also why Claude Code/Cowork and Codex ask you to work in a folder).
One speaker shared a case where a prompt had a repeated beginning: instructions, definitions, examples, document structure, and industry context. The model was processing that shared prefix from scratch every time. The fix was to keep the KV cache, essentially the model’s working memory for text it has already processed, so the system could reuse the repeated parts instead of recomputing them.
When the same prefix appeared again, the system loaded the saved state instead of recomputing it, which turned a roughly 200-hour run into about 140 hours. This is not usually what we talk about when we discuss a better model. It is a data/cache architecture improvement that directly changes cost and iteration speed.
TL;DR
Better AI models can only do so much. Performance comes from not wasting compute and from organizing the data so the model can actually find what matters.
CONFERENCE
I’ll be at the Future Finance Fest in Amsterdam next week (June 5). The conference brings fintech practitioners and finance academics together to discuss AI, crypto, banking, markets, and financial infrastructure.
If you’re there, please reach out. Also, if you have any Amsterdam tips, let me know. I’ve never been.
JANE STREET
The competition for Wall Street talent is so high that typically publicity-shy firms are on YouTube discussing, or should I say talking around, what they do.
Jane Street’s Ron Minsky, who co-heads Jane Street’s technology group, and Dan Pontecorvo, who heads its physical engineering team, recently spoke with podcaster Dwarkesh Patel in two YouTube episodes: one a typical interview and another a tour of Jane Street’s data center.
I also pulled out some notable quotes.
I feel like humans and like human cognition are like more valuable than ever. Like I have never been more desperate to hire more engineers and more traders than I am today because everything people are doing is more valuable than it was. —Ron Minsky, 11:00 here.
How much are humans in the loop between the model and the trading decision?
Many of your most profitable days happen when weird stuff happens, there are events, and the world goes crazy. Nobody knows what’s going on. That’s when it’s very hard to provide liquidity, and so you get paid more for doing it. There is often a lot of volume on days like that.
Doing that well often involves human judgment: thinking about how today is different from other days.
…
So even for systems that are largely automated, there are decisions to be made by the people watching them. And we always have people watching. An important part of trading is paying attention to what is happening during the trading day, even if the individual transactions are moving far too fast for a human to weigh in on a transaction-by-transaction basis. —Ron Minsky, 13:06 here.
Like these days, we are in something like the range of like tens of thousands of GPUs, and we will in not too long be in the range of hundreds of thousands of GPUs. —Ron Minsky, 22:40 here.
NEWS
Clearly, I’m doing this whole AI-in-finance thing wrong. From Bloomberg:
The Bloomberg story says the training starts with basic AI fluency, then moves into finance-specific workflows like pitch-video analysis, earnings-call scanning, and sentiment analysis.
I’ve not attended the seminar, so I don’t know how much they go into the pitfalls of AI. But there are many risks, and some more insidious than others.
Hallucinations are the most well-known. But they are hardly the only risk. AI can also:
be manipulated by hidden text in filings, headlines, or web pages.
produce different buy-or-sell calls when the same investment question is worded differently.
exhibit human biases such as overconfidence, herd behavior and sunk-cost thinking.
be tripped up by weak sourcing, stale data or confidential information fed into external tools.
be fooled by backtests that look impressive because they accidentally rely on information that would not have been available at the time.
I could keep going, but you get the point.
ROUNDUP
What Else I’m Reading
I’m the CEO of Goldman Sachs. The AI Job Apocalypse Is Overblown. NYT
Robinhood Launches AI Stock Trading, Purchases on Credit Cards BBG
Wells Fargo Hires Former Google AI Finance Leader Yahoo
Mercer finds AI now used by majority of asset managers in investment process Fund Selector Asia
Kirkland & Ellis to spend $500mn building its own AI technology FT
CALENDAR
After getting some reader feedback, I’m expanding this calendar to include more AI & finance in-person events, like hackathons, meetups, and workshops. If there’s an event you’d like to highlight, please reply to this message.
Upcoming AI + Finance Conferences
NY Tech Week: AI Agents in Finance - June 1 • NYC
Women in AI meetup on agent adoption in finance and where enterprise workflows are headed.NY Tech Week: Building to Disrupt - AI in Enterprise & Fintech – June 2 • NYC
HSBC and a16z event on AI, enterprise software, and fintech disruption.Bank of Finland & ESRB Conf. on AI and Systemic Risk - June 3-4 • Helsinki
Central-bank and systemic-risk conference focused on AI analytics for financial stability.NY Tech Week: AI for Finance - Claude + Excel + MCP – June 4 • NYC
Hands-on workshop around Claude, Excel, and MCP for finance workflows.Future Finance Fest (3f) – June 5 • Amsterdam
Digital-finance conference connecting financial institutions, builders, and researchers.TradingTech Summit New York – June 11 • NYC
Trading technology, market data, infrastructure, and analytics for capital-markets teams.Neudata New York Summer Data Summit – June 11 • New York
Alternative-data summit for investment managers, data buyers, and research teams.AI in Capital Markets Summit London – June 17 • London
Capital-markets AI conference covering data, trading, compliance, and operational use cases.AWS Summit NYC / Anthropic at AWS Summit – June 17 • NYC
Enterprise AI and cloud event with Anthropic participation at AWS Summit New York.
This Week in AI Street
Thanks for reading!
I’m always happy to receive comments, questions, and feedback.
Connect with me on LinkedIn, or
Send an email to matt [at] ai-street.co