The open-source AI ecosystem has matured dramatically in 2025. Llama 3.1, Mistral Large, Qwen 2.5, DeepSeek. These models are genuinely capable, often approaching proprietary model performance for specific tasks. The question is no longer "is open-source AI good enough?" It is "for which tasks is open-source the right choice, and for which is it not?" That is a different, more useful question.
The Current State
As of late 2025, the open-source AI landscape looks like this:
General capability. The best open-source models (Llama 3.1 405B, Mistral Large) perform within 5-10% of GPT-4 and Claude on most general benchmarks. For many enterprise tasks, that gap is irrelevant.
Specialised capability. For specific tasks (code generation, structured extraction, classification), fine-tuned open-source models can match or exceed proprietary models because they are optimised for the exact task rather than general capability.
Small model efficiency. Smaller open-source models (7B-13B parameters) run on modest hardware and handle straightforward tasks (classification, extraction, formatting) with minimal quality tradeoff. These are the workhorses of cost-effective AI.
Tooling and ecosystem. The ecosystem around open-source AI (inference servers, fine-tuning tools, evaluation frameworks) has matured significantly. Running open-source models in production is no longer a research project.
61%
of enterprises are using or evaluating open-source AI models, up from 23% in 2024
Source: a16z, Open Source AI Survey, 2025
When Open-Source Wins
Data Sovereignty
If your data cannot leave your infrastructure, open-source is the only option. Proprietary models process data on provider servers. Open-source models run wherever you deploy them: your cloud, your data centre, your edge devices.
For NZ enterprises dealing with sensitive health data, government information, or Indigenous data covered by sovereignty frameworks, this is often the deciding factor. The model quality comparison is secondary to the data governance requirement.
Cost at Scale
At high volumes, open-source models are dramatically cheaper. A proprietary API call costs a fraction of a cent, but a million calls a day adds up. Self-hosted open-source models have a fixed infrastructure cost regardless of volume.
Mak's back-of-envelope calculation: for workloads exceeding roughly 50,000 inference requests per day on moderate complexity tasks, self-hosted open-source becomes cheaper than API-based proprietary models. Below that threshold, the operational overhead of self-hosting typically exceeds the cost savings.
Task-Specific Fine-Tuning
Open-source models can be fine-tuned for specific tasks. This is the single biggest technical advantage. A Llama model fine-tuned on your specific document types, your specific classification categories, or your specific extraction templates will outperform a general-purpose proprietary model on those exact tasks.
The investment in fine-tuning (data preparation, training, evaluation) is non-trivial. But for core, high-volume tasks, the performance improvement justifies it.
Vendor Independence
Proprietary AI means dependency on a specific provider. Their pricing changes, their model updates, their API changes, their terms of service changes all affect your business. Open-source models give you control.
This is a strategic consideration, not just a technical one. Organisations building AI as a core capability should think carefully about how much of that capability depends on a single vendor's decisions.
When Proprietary Wins
Frontier Capability
For tasks that require the absolute best available capability, proprietary models are still ahead. Complex reasoning, nuanced creative generation, multi-step planning, and tasks that benefit from very large context windows favour proprietary models.
If your use case requires the top 5% of model capability, proprietary is the right choice today. If it requires the top 20%, open-source is competitive.
Speed to Deployment
Self-hosting open-source models requires infrastructure: GPU servers, inference engines, model management, monitoring. An API call to a proprietary provider requires an API key. For teams that need to move fast, proprietary APIs eliminate an entire category of infrastructure work.
The time-to-value difference is significant. Weeks to months for self-hosted open-source versus hours to days for proprietary API. In a proof-of-concept or early exploration phase, this matters.
Operational Simplicity
Running AI models in production is operational work: uptime, scaling, monitoring, updates, security patching. Proprietary APIs outsource all of this to the provider. Self-hosted models require an operations team or at minimum an engineer who understands GPU infrastructure.
For organisations without existing ML operations capability, the operational overhead of self-hosting is a genuine barrier.
Rapid Model Improvement
Proprietary model providers update their models frequently, and improvements are automatic. Self-hosted models require manual updates: evaluating the new version, testing against your workloads, deploying, and monitoring. If you value always running the latest and best model, proprietary providers deliver this by default.
The Practical Decision Framework
John and I use a structured framework for advising clients:
Step 1: Data Governance
Can your data go to external providers? If no, open-source is the only option. Decision made.
Step 2: Volume Assessment
What is your expected inference volume? Below 10,000 requests per day, proprietary APIs are simpler and competitive on cost. Above 50,000, the economics shift towards self-hosted. Between 10,000 and 50,000, both are viable.
Step 3: Task Complexity
What capability level do your tasks require? Simple tasks (classification, extraction, formatting): open-source is fully competitive. Moderate tasks (summarisation, analysis, structured generation): open-source is competitive with fine-tuning. Complex tasks (reasoning, planning, creative generation): proprietary has an edge.
Step 4: Operational Readiness
Does your team have ML operations capability? If yes, self-hosting is viable. If no, proprietary APIs or managed open-source services (like Azure AI or AWS Bedrock hosting open-source models) are more realistic.
Step 5: Strategic Timeline
Are you building a core AI capability that you will run for years? Consider open-source for vendor independence. Are you experimenting with a use case that may or may not prove out? Use proprietary APIs for speed and simplicity.
The best enterprise AI architectures use both - open-source for high-volume workloads where you want control, proprietary for complex tasks where you want frontier capability. It is not a religion; it is engineering.
Mak Khan
Chief AI Officer
The Multi-Model Reality
The most effective enterprise AI architectures we see use both open-source and proprietary models. Small open-source models handle the high-volume, straightforward tasks. Larger proprietary models handle the complex, low-volume tasks. A routing layer directs each request to the appropriate model based on complexity, cost, and capability requirements.
This multi-model approach gives you the best of both worlds: cost efficiency and control where volume demands it, frontier capability where complexity demands it, and the flexibility to shift the balance as open-source models improve.
The open-source AI decision is not a binary choice. It is a portfolio allocation decision. And like all portfolio decisions, the right answer depends on your specific context, constraints, and objectives.