The AI Foundation at Work

From Theory to Production

The Architecture

The Intelligence Layer

• Multi-model orchestration across four model providers. Queries are routed based on task complexity, cost sensitivity, latency requirements, and compliance constraints.
• Automatic failover. If a model provider experiences an outage or degradation, queries reroute to alternatives within seconds. No client-facing impact.
• Cost optimisation. Simple tasks go to efficient models. Complex tasks go to capable models. The routing saves 30-40% on inference costs compared to routing everything to the most capable model.
• Prompt management. Versioned, tested, and deployed like code. Every prompt change goes through evaluation before production.

• Model routing is more art than science. Performance benchmarks don't always predict production behaviour. We run continuous evaluation against real-world queries, not just test sets.
• Failover needs to be tested regularly. A failover path that hasn't been exercised will fail when you need it.
• Cost optimisation has quality trade-offs. The cheapest model that passes evaluation isn't always the best choice. We optimise for quality-per-dollar, not minimum dollar.

The Knowledge Layer

• Hybrid retrieval combining vector search, keyword search, and knowledge graph queries. The retrieval strategy adapts based on the query type and the knowledge domain.
• Document processing pipeline that handles PDFs, Word documents, emails, spreadsheets, images, and structured data. Ingestion, chunking, embedding, and indexing are automated with quality checks at each stage.
• Entity resolution across document types. When three documents mention the same entity differently, the knowledge layer resolves them to a single entity with merged context.
• Context management for multi-step workflows. When an agent processes a complex task across multiple steps, the context layer maintains coherence and relevance.

• Chunking strategy matters more than embedding model choice. The way you break documents into pieces has a larger impact on retrieval quality than which embedding model you use.
• Knowledge graphs and vector search are complementary, not competitive. Vector search finds semantically similar content. Knowledge graphs find structurally related content. You need both.
• Document processing is never finished. New document types, new formats, new edge cases. The pipeline needs continuous attention.

The Governance Layer

• Policy-as-code enforcement. Guardrails are defined in configuration, enforced by the platform, and logged for audit. They can't be bypassed by individual capabilities.
• Complete audit trails. Every query, every retrieval, every model call, every output. Traceable from input to output and back. Essential for regulated industries.
• Content safety filters. Output validation that catches hallucinations, inappropriate content, data leakage, and off-topic responses before they reach the user.
• Access control. Role-based and capability-based access controls that determine which users can access which capabilities with which data.

• Governance is not overhead. It's the thing that makes enterprise clients say yes. Every enterprise client we work with asks about governance in the first meeting. The foundation's governance layer is often the deciding factor.
• Audit trails need to be queryable, not just stored. When a compliance officer asks "show me every decision the AI made about this customer," the answer needs to be fast.
• Guardrails need regular testing. Like security controls, guardrails that aren't tested regularly degrade in effectiveness.

The Operations Layer

• Real-time quality monitoring per capability. Accuracy, relevance, completeness, and confidence tracked continuously with alerting on degradation.
• Cost tracking per query, per capability, per client. Granular enough for billing, strategic enough for optimisation.
• Performance monitoring. Latency, throughput, error rates, and availability. Standard DevOps practices applied to AI-specific metrics.
• Drift detection. Automated detection of distribution shifts in inputs, outputs, and model behaviour. Alerts when the system's behaviour changes beyond expected ranges.

• AI monitoring is not software monitoring. Traditional APM tools catch crashes and latency spikes. They don't catch quality degradation, which is the more common and more dangerous failure mode.
• Evaluation is ongoing, not pre-deployment. The evaluation pipeline that tests the system before deployment continues running after deployment, catching degradation in real time.
• Operations costs are real. The monitoring, evaluation, and maintenance infrastructure is a meaningful portion of the total platform cost. Budget for it from the start.

The Compound Effect, Measured

Sovereignty

• Data residency. Client data stays in NZ (or wherever the client's governance requires). No data crosses jurisdictional boundaries without explicit authorisation.
• Model hosting. For clients with strict sovereignty requirements, models run on infrastructure within NZ. For others, we use cloud inference with data residency controls.
• Indigenous data governance. The platform supports indigenous data governance frameworks. Data belonging to Māori and Pacific communities can be governed according to community-defined principles, not just regulatory requirements.

What This Means