Every enterprise exploring AI hits the same fork in the road: do we build our own AI capabilities, buy off-the-shelf tools, or some combination? After a year of building enterprise AI systems, we've developed a framework for this decision that goes beyond the usual vendor comparison matrix.
What You Need to Know
- There's no universal right answer. The build vs buy decision depends on your specific situation: strategic importance of AI to your operations, availability of internal capability, data sensitivity, and timeline.
- Most enterprises will end up with a "both" answer. The real question is which layers to own and which to rent. Getting this wrong either burns budget on unnecessary custom work or creates dangerous vendor dependencies.
- The layers that matter most are: model layer (almost always buy), data layer (almost always own), orchestration layer (the strategic decision), and interface layer (depends on workflow integration needs).
- Changing your AI foundation later is expensive. The decisions you make now about ownership, architecture, and vendor relationships will compound - for better or worse - over the next 3-5 years.
The Four Layers
John: When clients ask "build or buy?", the question is too broad to answer. AI infrastructure isn't one thing. It's a stack of layers, and the build/buy decision should be made independently for each.
Layer 1: Models (Almost Always Buy)
The base AI models - GPT-4, Claude, Llama, Mistral, and whatever comes next. Unless you're a technology company with specific model requirements and a research team, you should be using existing models, not training your own.
The model layer is improving so rapidly that anything you build today will be outperformed by a commercially available model within 12 months. Use the APIs. Pay the inference costs. Focus your engineering effort elsewhere.
Exception: If your use case requires extreme data privacy (government, defence, certain healthcare applications), you may need to run models on your own infrastructure. That's a deployment decision, not a build decision. You're still using existing models - just hosting them differently.
Layer 2: Data (Almost Always Own)
Your data pipeline - how you ingest, process, structure, and index your organisation's knowledge for AI consumption. This is where enterprise AI value lives, and it's the layer you should invest in owning.
Your data is your competitive advantage. The models are the same for everyone. The data isn't. An insurance company's AI capabilities are only as good as how well their policy documents, claims history, and operational data are prepared for AI consumption.
Own your data pipeline. Build it well. Make it reusable across multiple AI applications. This is the infrastructure that compounds.
What this means practically:
- Own your document ingestion and processing pipeline
- Own your vector database and embedding strategy
- Own your data governance and access control
- Build for multiple AI applications, not just the first one
Layer 3: Orchestration (The Strategic Decision)
The orchestration layer sits between your data and the model. It handles retrieval logic, prompt engineering, response validation, tool calling, multi-step reasoning, and workflow integration. This is where the build vs buy decision gets genuinely difficult.
Isaac: The orchestration layer is where the magic happens. It's the difference between a generic AI response and an AI response that's grounded in your data, validated against your rules, and formatted for your workflow. It's also the layer where vendor lock-in creates the most risk.
Buy orchestration when:
- Your use cases are well-served by existing platforms (customer service, basic document search, content generation)
- You don't have engineering capacity to build and maintain custom orchestration
- Speed to market matters more than long-term flexibility
- The vendor's platform genuinely fits your workflow (not just their demo workflow)
Build orchestration when:
- Your use cases are domain-specific and not well-served by generic platforms
- AI is strategically critical to your operations (not just a productivity tool)
- You need deep integration with existing enterprise systems
- You want model flexibility (the ability to swap models without rebuilding)
- Data sensitivity requires full control over how data flows through the system
The hybrid approach (most common):
Use a platform for generic use cases (internal search, content assistance) and build custom orchestration for your core domain-specific AI capabilities. This gives you speed where differentiation doesn't matter and control where it does.
Layer 4: Interface (Depends on Integration)
The user interface - how people interact with AI capabilities. This decision depends entirely on where AI sits in your users' workflow.
Buy (use the vendor's interface) when:
- The AI is a standalone tool (a chatbot, a search interface, a content generator)
- Users are willing to switch to a different application
- The vendor's UX is good enough for your users' needs
Build (custom interface) when:
- AI needs to be embedded in existing workflows and applications
- Users shouldn't have to leave their primary tool to access AI
- Your use case requires specific UX patterns (confidence indicators, source attribution, structured review)
- The AI interacts with multiple systems simultaneously
60%
of enterprises use a combination of build and buy approaches for their AI capabilities
Source: Deloitte, State of AI in the Enterprise, 5th Edition, 2022
The Decision Framework
Here's a practical framework for making the build vs buy decision at each layer.
| Factor | Favours Build | Favours Buy |
|---|---|---|
| Strategic importance | AI is core to operations | AI is a productivity tool |
| Data sensitivity | Highly sensitive/regulated | General business data |
| Internal capability | Strong engineering team | Limited AI engineering |
| Timeline | 6-12 months acceptable | Need results in 1-3 months |
| Differentiation | AI capability is competitive advantage | AI capability is table stakes |
| Integration depth | Deep system integration needed | Standalone or light integration |
| Budget model | Capex comfortable | Opex preferred |
John: No single factor should drive the decision. But if three or more factors lean the same way, that's a strong signal. If it's evenly split, default to buy for the first implementation and plan to build once you understand the domain requirements deeply.
Common Mistakes
Over-Building
The most expensive mistake is building custom infrastructure for a use case that a $50/month SaaS tool handles perfectly. Not everything needs to be bespoke. If your use case is well-served by an existing product, use the existing product. Save your engineering capacity for the capabilities that actually differentiate your operations.
Over-Buying
The second most expensive mistake is locking your strategic AI capabilities into a vendor platform. If AI is core to how you deliver value, owning the orchestration and data layers isn't optional. Vendor platforms change pricing, pivot their product strategy, or get acquired. Your core capabilities shouldn't be exposed to those risks.
Ignoring the Compound Effect
Each AI capability should make the next one easier to build. If every new AI initiative starts from scratch - new vendor evaluation, new integration, new data pipeline - you're not building a foundation. You're building a collection of disconnected tools.
Whatever you build or buy, design for reuse. Shared data pipelines. Consistent model integration patterns. Standardised governance. The compound effect is the strongest argument for thoughtful architectural decisions now.
A Practical Starting Point
If you're facing this decision today and aren't sure where to begin:
- Use commercial models (OpenAI, Anthropic, open-source). Don't train your own.
- Own your data pipeline. Even a basic one. Get your key documents ingested, embedded, and indexed.
- Start with a vendor platform for your first use case. Learn from it.
- Build custom orchestration for your second use case, informed by what you learned from the first.
- Evaluate interface needs case by case. Embed where integration matters. Use vendor UIs where they're sufficient.
The build vs buy decision isn't binary, and it isn't permanent. But the architectural choices you make now will shape your AI capabilities for years. Take the time to think them through.