The Rise of Small Models in Enterprise AI
Hey, it’s Matt. This week on AI Street:
🤖 Small AI adds up to Big Impact
🚶How word choice can lead AI astray
🎙️ Interview w/ Deep Analysis Founder
ADOPTION
Small AI, Big Impact
AI doesn’t really understand cause and effect.
If someone can show me evidence that AI actually thinks using formal or deductive logic, I’ll bow to our new machine overlords. Until then, it’s essentially sophisticated pattern matching --which is still useful!
For all the headlines that AI is going to take everyone’s jobs, not being able to reason logically seems like a serious limitation. AI excels at tasks that don’t really require thinking — rote, repetitive, boring tasks. And for those, you don’t need a massive model. A small model will do.
(Side note: A model’s “size” refers to how many parameters it has — essentially, the number of connections in the network. The biggest models have billions or even trillions, while smaller ones have just thousands. There’s no universal cutoff of what makes a model “large” or “small.”)
Small models can be trained to handle specific tasks often with better accuracy than their bigger counterparts. I’ve written before about how JPMorgan showed that smaller models improve credit data accuracy and I interviewed the CEO of Aveni.ai on why they’re building small models for financial services last year.
The WSJ recently highlighted how companies are leveraging small models for specific tasks and how they coordinate with large ones, describing the process as an AI assembly line.
Meta uses small, specialized AI models to deliver ads, while its largest models develop and pass on effective targeting strategies.
Airbnb employs small models from Alibaba to automatically resolve a large portion of customer-service issues.
Gong combines small and large models to analyze sales calls, with smaller models handling summarization and filtering before a larger model produces final insights.
Aurelian uses small generative models to automate responses to nonemergency 911 calls.
Hark Audio fine-tuned small models on a library of human-edited podcast clips to automatically identify and collect memorable audio moments.
From the story:
“The reality is, for many of the operations that we need computing for today, we don’t need large language models,” says Kyle Lo, a research scientist at the nonprofit Allen Institute for AI.
Takeaway
AI helps the most when it’s applied to narrow tasks using smaller and cheaper models that specialize in specific use cases.
Further Reading
Large Models Get All the Hype, but Small Models Do the Work | WSJ
JPMorgan Used AI to Improve Credit Card Data Accuracy | AI Street
Aveni's Joseph Twigg on Small Language Models | AI Street
AI Adoption on Wall Street
KPMG to Grade Employees on AI Adoption in Annual Reviews
KPMG will begin evaluating staff on how effectively they use AI tools like Microsoft Copilot in their 2026 performance reviews, part of a firmwide push to ensure all employees integrate AI into their work. BBG
Ex-Consultants Are Helping Train AI for Entry-Level Tasks
Roughly 150 former McKinsey, Bain, and BCG consultants have been hired by a data-labeling startup to train AI models like Google’s Gemini on routine consulting work. BBG
Why AI Needs More Data for Investment Banking Tasks
Companies building AI models are hiring financial professionals to supply the specialized, private data needed to train systems on real-world workflows. Forbes
FINRA Details Its Expanding Use of AI
FINRA has deployed its own internal generative AI system to all employees, with 40% now using it weekly to summarize documents, compare filings, and support compliance reviews. FINRA
RESEARCH
How Simple Word Choices Lead AI Astray
Even slight wording changes can sway an AI’s financial judgment, Domyn researchers find
For better or worse, people are using AI for all sorts of things from therapy to romance to picking stocks. (Def not medical/love-life/financial advice!)
AI has earned a surprising amount of user trust, which is pretty wild since no one really knows exactly how large language models work. The folks building them say they’re grown rather than built.
After you ask a chatbot a question, you might see “thinking” as it comes up with an answer, but it’s not really thinking in the deductive, formal logic sense. It’s a probabilistic machine that predicts the next token based on its training and that can lead to strange biases.
If you asked a (human) financial analyst whether Microsoft or Apple is the better investment, the answer wouldn’t depend on whether you said Microsoft or Apple or Apple or Microsoft. For LLMs, that word order matters, according to new research.
A new paper from the team at Domyn digs into this problem, so-called positional bias, and finds it’s common in large language models used for financial decisions.
Using Qwen2.5, an open-source model family, they built a benchmark to test whether the order of two stocks in a question changes the model’s answer. They tested 18 major tech companies across 10 different financial evaluation categories—everything from fundamental analysis and ESG criteria to risk assessment and growth potential.
Smaller models were especially biased toward whichever company came first, while larger ones mostly reduced the bias, but in a few cases even reversed it, favoring the second company instead. Telling the AI to act as a "conservative" versus "aggressive" advisor changed its answers even when asking the same question.
The team also traced the bias to specific layers and attention mechanisms inside the models—showing where it originates.
Takeaway
Bigger models can help reduce “positional bias,” but it doesn’t totally eliminate it. Even the best models can still prefer “Microsoft over Apple” simply because Microsoft was mentioned first. Careful how you use AI.
Further Reading
Tracing Positional Bias in Financial Decision-Making: Mechanistic Insights from Qwen2.5 | arXiv
SPONSORSHIPS
Reach Wall Street’s AI Decision-Makers
Advertise on AI Street to reach a highly engaged audience of decision-makers at firms including JPMorgan, Citadel, BlackRock, Skadden, McKinsey, and more. Sponsorships are reserved for companies in AI, markets, and finance. Email me (Matt@ai-street.co) for more details.
