When teams prepare large language models to operate, they have a choice: reveal the workings of the system to the world, with or without details on parameters, training data and weighted inputs? Or keep all of this information secret, protected by the makers? The resulting “open source” versus “closed source” debate is one that echoes through the halls of government and business whenever people are discussing AI.
The leaders of tech companies have to struggle with these issues, too. They have to figure out whether it is better, strategically and otherwise, to release the relevant data, or not.
“Emerging civil society debates over AI safety—especially over open foundation models—merit particular attention,” writes Masao Dalhgren at the Center for Strategic and International Studies, citing a “very spirited debate” and explaining some of its context. “Unlike with closed models like GPT-4, developers of open foundation models like Llama, Mistral, or Qwen openly publish the models’ underlying parameters (“weights”), allowing them to be inspected, modified, and operated by end users. With the performance of open models approaching their closed counterparts, some have suggested that open model distribution could pose “extreme risks” for misuse. Others, meanwhile, have highlighted open models’ benefits for research, security, and national competitiveness.”
This article focuses on U.S. defense applications, but the open source debate pops up elsewhere, too.
Sovereignty, Trust, and Collaboration
A recent panel of experts showed some of the rationale behind choosing open source or closed source design. MacKenzie Sigalos interviewed Dinesh Maheshwari, Karl Zhao, Charles Fan and Jose Plehn about these choices and how they are made, through the prism of national security and more.
One consensus that came through is that the U.S., in general, is light on open source leaders.
“The United States AI ecosystem seems to have become a little bit more closed,” Plehn said, describing his firm’s work on a “unique global identifier” that addresses concerns about copyright or proprietary violations.
“We’re trying to do what we can in terms of open sourcing data, which means, of course, that (data) can be used to train models, and also to make them more factual,” he said.
“From a foundation model perspective, it is true right now, there are more open source models coming from China … and the frontier labs from the U.S. essentially select a more closed source approach,” Zhao added.
A Matter of Trust
Maheshwari, speaking on this duality, suggested that open source systems can be proliferated in a trust environment.
“Open source is about trust,” he said. “It comes to not just open sourcing weights, it’s open sourcing data. It’s open sourcing the model, architecture, the meta parameters and (knowing) that it can be replicated, and it can be audited. That’s what builds trust.”
As for Chinese dominance, he noted that DeepSeek, widely hailed as an “open source” system, is not entirely open source itself.
“With due respect, DeepSeek is half open, half closed,” he said. “The data is not open. You don’t know what it’s been trained on. The model parameters need to be open.”
Fan talked about utilizing memory in new ways, which would play into the choice of open source or closed source systems.
“Every (piece of) knowledge that’s public is captured by the model,” he explained, “but the memory that belongs to us, or belongs to a corporation, is not in the internet, therefore it’s not captured by those weights. There needs to be systems designed to save those things.”
Frontier labs are doing work on this, he said, but they’re not, according to his definition, open source.
“Not only that, they do not support each other’s model,” Fan added. “So if you are using a particular memory system, you’re tied down to that particular model. And what we are open sourcing today is a memory model – we believe the memory needs to belong to the person who owns the memory, and therefore, by open sourcing it, that allows the trust and security that this stays with you, no matter which model you use, whether it’s open source or closed source.”
The Profit Motive
Setting up a question, Sigalos said this about open source and closed source projects:
“I had a conversation with someone,” she noted, “and they said that part of the reason why there is less of an impetus to open source tech is based on the fact that, like your enterprise, customers want a closed system for various reasons, and perhaps that’s more lucrative when you think about the B2B segment.”
Maheshwari addressed this, again noting the difference between different types of systems that some call “open source,” and suggesting that in reality, OS is a spectrum.
“The value of the models is in (their) use in the applications, and the applications are built on top of the fine tuned models that are likely also further ‘customized,’ having … an open weights model,” he said. “Don’t call it an open source model. Even an open weights model allows it to be hosted on different infrastructure. It allows people to play with it. It means they can ‘choose’ the cost of the infrastructure on which you can play. And that allows for improvement.”
AI Sovereignty in the Twenty-First Century
Is sovereignty a necessary step for national security and resilience, or is this a trap that risks a balkanized splinternet of AI?” Sigalos asked.
“The population is falling for techno nationalism,” Maheshwari said. “Unfortunately, the national pride being defined ‘outside-in’ is too tempting for people to be ‘inside-out.’ … sovereignty is about being able to have control of the future of what a society does … one needs to be able to deploy AI for the societal benefit, in that local context. I do not believe that sovereignty has to be exercised in a fashion that ‘splits the effort’ … done thoughtfully, there’s no reason why we can’t enable societies, local societies, to have control of their destiny without splitting effort.”
Plehn described the U.S. Action Plan on AI and how it works, along with his company’s role.
“A key component of the policy is actually open source, and the promotion of it, certainly with an America First perspective, but open source indeed,” he said, revealing that his company, BrightQuery, is working on something called the National Secure Data Service.
“(It) is the first centralized U.S. government data ecosystem that will service the entire population and agencies, and the world at large, to help inform the general public about the United States and its various constituents,” he said.
Plehn talked about the fight against censorship, too.
“The erasing of historical facts, or modifying of historical facts, is something that does take place in some governments and countries around the world, as I’m sure we’re all aware,” he said. “And so the United States is against such a policy.”
Zhao spoke about his work on a Greek national AI system being built by a company called Kiefer, which I have written about in the past.
“What makes it compelling for them to do it is that it’s so underserved,” he said. “What’s important for these countries, especially from the language perspective, it’s important for them to take control of their own destiny using their language, specific culture, and make sure the model represents that.”
“Data and knowledge have no borders,” Plehn added. “That’s really a fundamental concept that we need to keep in mind. Once knowledge is out there, it’s out there, and it’s very hard to contain unless, of course, you have excessive censorship taking place. And so, although frontier AI companies might preserve and contain the underlying models and weights and such that they are putting out and publishing, the counterpoint to that is to open source as much data as possible.”
All of this underscores a real debate over whether to make the data open to the public. We have to keep thinking about this as we see new models appearing around the world.