What to Build, Buy, and Federate in an Enterprise AI Platform

Part 2 of 3: AI Platform Budget Decisions

A platform team that owns domain semantics has confused control with competence. Domain authority belongs to the people who answer for domain errors.

Starting From the Right Question

If you have worked through Part 1 and confirmed that multiple squads have genuine workflow demand, you are now at the architecture decision. The instinct in most organizations is to build a comprehensive end-to-end platform. That instinct is almost always too broad.

The better question is not "What should the platform own?" but "What should the platform never own?"

What Should You Buy?

The infrastructure runtime layer is commoditized. Assigning senior engineers to write a custom model proxy gateway, token metering service, or latency routing engine is misallocated capital. These exist as production-grade open-source or SaaS products today.

Build where internal ownership creates durable domain, risk, or economic leverage that cannot be purchased from a market provider at lower cost and comparable quality. Buy where operational maturity is high and differentiation is low.

The Vendor Shelf Life Problem

The buy decision carries its own risk. The LLM observability and evaluation tooling market evolves fast. Tooling selected today may face deprecation, vendor consolidation, or functional obsolescence within twelve months. Deeply coupling application telemetry to a vendor's proprietary tracing format creates high switching costs when migrating production pipelines.

The mitigation is clean: mandate open standards at the platform integration boundary. Instrument all application code using OpenTelemetry GenAI semantic conventions rather than vendor-proprietary SDK wrappers. This treats the buy layer as an interchangeable utility backend, preserving vendor reversibility without requiring custom internal plumbing.

The Specialized Vendor Exception

Not every vendor is a commodity. A commercial provider with seven years of adjudicated domain data, HIPAA or ISO-27001 certifications, and task-specific evaluation suites that demonstrably outperform internal engineering is not a commodity buy. Maintaining an internal capability contract to preserve "strategic ownership" against that vendor is an expensive error.

Before greenlighting an internal build over a specialized vendor, answer two questions directly:

• Does the internal build create durable enterprise leverage through proprietary customer data and unique domain authority?
• Or does it introduce perpetual maintenance drag without delivering competitive differentiation?

How Should You Federate What You Build?

The structural choice product leaders face is not centralized platform vs. isolated squads. It is designing ownership across three distinct tiers so the central platform team never becomes a cognitive bottleneck on specialized domain logic:

This strict division of labor prevents the platform team from having to understand or approve changes to medical diagnostic formats or accounting line items, which is the single biggest reason squads bypass central platforms in production.

Staged Centralization: How Does an AI Platform Earn the Right to Expand?

The classic corporate mistake is Premature Centralization. A platform architect pitches leadership: "Let's immediately build a full enterprise AI platform with mission-critical SLAs, automated distillation, custom evaluation engines, and shared infrastructure for all ten squads."

The result is predictable: huge upfront spend, near-zero squad adoption, complaints about slow central intake queues, and a platform team that becomes an expensive organizational bottleneck.

To prevent this, centralization cannot be funded as a blank-check project. You do not get budget and mandate to build a heavier platform layer just because you have an architectural vision. Each stage must earn its right to advance through demonstrated production evidence and verified squad demand from the previous stage:

Advancement gates are not optional checkpoints. They exist because un-sharing a capability is never free.

What Does Reversal Actually Cost?

Decoupling consuming squads from a shared capability contract requires three friction-heavy steps per squad: stripping shared SDKs and unwinding centralized schema bindings, rebuilding prompt assembly and retry mechanics locally, and re-establishing standalone evaluation fixtures and regression baselines.

This refactoring costs two to four engineer-weeks per squad ($16k–$32k). Across four squads, reversing an unvalidated shared service runs $64k–$128k in deadweight friction. Because reversal cost is real and non-zero, promoting a capability from Stage 2 to Stage 3 creates an unpriced balance sheet liability unless reuse economics are proven first.

Is There Actually a Shared Capability?

Before advancing a capability from Stage 2 to Stage 3, product managers must defend the centralization decision against what the architecture literature calls the Reuse Trap.

The Reuse Trap occurs when squads appear to share a capability because they process similar data types with similar tooling, while actually solving structurally incompatible domain problems.

Consider a four-squad organization where every squad extracts fields from unstructured documents:

Centralizing these four squads into a single monolithic "Document Intelligence Service" fails to create operational leverage. Instead, it generates a platform choked with dozens of conditional configuration flags, domain-specific override branches, and conflicting evaluation harnesses.

The Domain Engineer's Objection

A skeptical domain engineer will push back legitimately:

"You're assuming squads have fundamentally different needs just because their code looks different today. But squads only built custom setups because nobody gave them a shared standard to begin with. You can't tell if their needs are truly incompatible, or if they just reinvented the wheel in silos."

This objection identifies a real blind spot. Waiting for identical schemas to emerge organically will systematically undercount legitimate reuse opportunities.

The resolution: a 2-Week Convergence Spike. Before rejecting centralization, fund a time-boxed, two-week architectural spike between two adjacent squads tasked with drafting a unified semantic schema and joint assertion suite.

• If 80% or more of the field schemas and validation rules merge without conditional branching, the divergence was artificial. Proceed with centralization.
• If merging requires domain-specific override flags for core validation and authority limits, the divergence is structural. Reject centralization. Keep capabilities local.

The Gated Centralization Assessment

A capability that fails any required gate cannot advance to a shared service.

Required gates (any failure blocks centralization):

Scored dimensions (three or more must demonstrate meaningful overlap):

What Comes Next

Clearing the architecture decision does not close the investment case. Once you know what to build, buy, and federate, you still need to prove that the economics justify the spend against the right baseline, with phased cash flows, and with a kill rule that accounts for the patient capital AI platforms require.

That is the subject of Part 3.

→ Underwriting the Return on an AI Platform

The Adversarial Stress-Test: Five Critical Inquiries

1. What is the strongest argument against this approach?

A three-tier federation model (central control plane, federated domain capability stewards, and local product squads) introduces real friction. Dividing schema ownership from infrastructure and user workflows requires formal interface contracts, inter-team reviews, and cross-pod synchronization. In fast-moving environments, this governance layer can slow minor workflow updates that a single empowered squad could ship in an afternoon.

The Practical Resolution: Keep the control plane strictly mechanistic (rate limiting, gateway routing, cost attribution, and PII masking) and grant domain stewards autonomous merge rights over domain schemas. Squads retain unilateral deployment authority over workflow orchestration and prompts, so cross-tier synchronization occurs only when changing shared contract semantics.

2. Under what circumstances would this fail?

This federated architecture collapses when domain stewardship is assigned as an unfunded, part-time responsibility to an engineer without authority to block squad releases or arbitrate schema conflicts. In that scenario, the steward becomes either an operational bottleneck or a rubber stamp. It also breaks in low-volume, single-product teams where the coordination overhead of three tiers exceeds any infrastructure savings.

The Practical Resolution: Tie domain stewardship directly to funded engineering allocations and give stewards explicit sign-off authority within CI/CD pipelines. For organizations with fewer than three production squads, collapse the model to two tiers: buy commoditized runtime gateways and keep domain schemas completely local within individual product services.

3. What would a skeptical customer, client, or colleague object to?

Product squad leads will object that a central platform team will inevitably prioritize infrastructure backlogs over product delivery dates, leaving customer commitments at the mercy of a shared capability queue.

The Practical Resolution: Enforce strict client-side isolation: squads consume capability contracts via version-pinned SDKs and run independent deployment pipelines. If a shared capability suffers an incident or a central upgrade introduces regressions, squads can fall back to local direct-provider routing without waiting for central intervention.

4. Which alternative deserves more consideration?

Instead of standing up a dedicated platform team and federated pods, maintain an internal open-source repository of vetted prompt templates, Pydantic schemas, and evaluation scripts. Each squad imports what it needs as a library into its own service, completely eliminating cross-team runtime dependencies.

The Practical Resolution: Inner-source micro-libraries eliminate runtime coordination, but they fail to solve enterprise governance, data sanitization, and token FinOps. When each squad manages its own model credentials and telemetry, compliance audits become impossible and volume-discount bargaining power is lost. The optimal compromise is an inner-sourced library for domain prompts paired with a centralized, non-blocking network proxy for governance invariants.

5. What am I assuming that may not be true?

This architecture assumes consuming squads possess the engineering discipline to maintain versioned evaluation suites and curate operator overrides, rather than treating the central platform as a magical black box that absorbs domain responsibility.

The Practical Resolution: Gate capability deployment on automated evaluation artifacts. A squad cannot promote an agentic workflow to production without committing a baseline golden eval dataset and automated drift assertion test that runs in the deployment pipeline. When evaluation ownership remains a technical deployment prerequisite, squads cannot abdicate domain vigilance.

One-Line Synthesis

Enterprise AI platforms achieve durable leverage by centralizing commoditized runtime infrastructure while federating domain schemas and evaluation contracts directly to the squads that answer for production errors.

Related Reading

• Before You Fund an AI Platform: Prove the Workflow First: Part 1: The capability sensitivity test and sequencing thresholds.
• Underwriting the Return on an AI Platform: Part 3: Phased cash flows, real-options logic, and the six failure modes.
• Policy-as-Code for Autonomous Agents: Implementing deterministic verification perimeters around probabilistic reasoning engines.
• EROI Is Not Enough: The Capital Return Trap of Generative AI: Why traditional return on investment metrics miss the systemic depreciation of generative software assets.