David Lareau is CEO of Medicomp Systems, which makes medical data relevant, usable and actionable.
AI has quickly become a nearly ubiquitous part of our daily lives. Healthcare is no exception. The allure of large language models (LLMs) that can seemingly answer any question is hard to ignore. Yet for hospitals, health systems and other healthcare enterprises, adopting the biggest model on the market is often neither cost-effective nor practical.
When it comes to applying AI in clinical settings, bigger is not always better. In fact, smaller, purpose-built models can sometimes offer better value.
The Hidden Costs Of “Bigger”
The most advanced LLMs are powerful but resource-intensive. Training and running models with hundreds of billions of parameters consume vast amounts of energy and infrastructure, thus incurring massive costs. These costs will be passed on to users.
For enterprises already navigating thin margins, the financial implications are significant. Using large proprietary models typically requires licensing fees or consumption-based pricing tied to “token” usage. As more queries are routed through the model, costs can escalate quickly and unpredictably. In fact, a recent market analysis shows that the global healthcare AI market is expected to grow from $26.6 billion in 2024 to $187.7 billion in 2030. Someone will be paying for all of this investment.
Healthcare providers cannot afford to treat AI like a blank check. A critical question is whether the benefits of using AI outweigh the costs.
Right-Sizing AI For Healthcare
Instead of defaulting to the largest model available, health systems should start with a more fundamental question: What do you want the model to do?
Not every task requires AI, and even fewer need a model trained to generate Shakespearean sonnets or computer code. In healthcare, the most valuable applications are often narrow, repeatable and data-intensive, such as mapping clinical terms to standardized codes, surfacing relevant patient history, summarizing encounter notes or filtering a patient history to identify hallmark findings and trends related to a specific clinical diagnosis.
Smaller, domain-specific models can excel at these use cases. They are cheaper to run, easier to deploy and faster to validate. They can also be integrated alongside existing clinical data systems rather than replacing them. To put it another way, using the largest available model to solve every problem is like using a shotgun to kill a fly: It may kill the bug, but it is messy, expensive and unnecessary.
Guardrails Over Generalization
Large, general-purpose models are designed to handle everything from writing a grocery list to simulating quantum mechanics. In healthcare, that breadth creates risks. These systems are prone to “hallucinations,” confidently presenting inaccurate or fabricated information. For clinicians, even slight inaccuracies can compromise care.
By contrast, smaller models can be trained or fine-tuned on curated medical data and paired with rule-based systems that have already been validated for accuracy. For example, internal dictionaries and mappings can be used to process most terminology, only sending ambiguous cases to a model for clarification. This approach can reduce reliance on a black box, help cut processing costs and maintain transparency.
Practical Deployment Advantages
Healthcare providers must also consider deployment realities. Large models often require specialized hardware and cloud connectivity, raising concerns about latency, reliability and security. Smaller models can sometimes run on local servers or even edge devices, enabling faster performance and greater control of sensitive data.
Importantly, smaller models are also easier to revalidate when updates occur. Each time a new version of a model is released, health systems must recheck outputs to ensure clinical safety. With smaller, focused models, the scope of validation is narrower, saving time and resources.
Choosing The Right Tool For The Job
Healthcare leaders should think of LLMs as tools rather than silver bullets. Just as someone would not use a machete to open every package when a simple pocketknife will do, enterprises should match the model to the task at hand.
The guiding principle should be: Start as small as possible. Define the specific workflow problem. Explore whether existing systems and structured data can effectively address this issue. If not, evaluate whether a lightweight, open-source or fine-tuned model can augment those systems. Only when tasks truly require broader capabilities should organizations consider a larger model. Even in these instances, a careful cost-benefit analysis is needed.
A Sustainable Path Forward
The excitement around generative AI is justified, but unchecked enthusiasm can lead to costly missteps. The future of AI in healthcare will be shaped by organizations that harness the right-sized models to deliver precise, meaningful improvements in care.
Hospitals and health systems that prioritize precision over scale will be better positioned to reap the benefits of AI without being overwhelmed by its costs or risks. In healthcare, where stakes are measured in both patient outcomes and operating margin, that balance is essential.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?