Resource Center · Private AI · IXBox · Deployment
Cloud, Hybrid, or Private AI: A Placement Decision Guide
A working decision framework: data classification, prompt and retrieval boundaries, residency, latency, token economics versus owned GPU capacity, routing policy, lifecycle, and a worked IXBox example.
Darren Reeves · Published 2026-08-24 · Updated 2026-08-25 · 15 min read
Placement is a policy decision, not a religious one
The AI placement debate is usually argued as ideology: cloud people versus on-premises people. In practice, placement is a per-workload policy decision, and most organizations end up with a mix. The mistake is not choosing cloud or choosing local; it is choosing once, for everything, without writing down why.
This guide is the write-it-down framework: classify the data, define the boundaries, price both options honestly, and decide per workload.
Start with data classification, not model selection
Placement arguments become simple once the data is classified, because most of the decision is not about the model at all. Three questions per workload: What data appears in prompts? What data is retrieved as context? What do the answers reveal about either?
Classify each into the tiers your organization already uses (public, internal, confidential, regulated). The strictest tier touched by a workload sets its placement floor. A drafting assistant that only ever sees marketing copy has no placement constraint. The same assistant grounded in customer contracts inherits the contracts’ constraints, whatever the vendor’s trust page says.
Draw the prompt and retrieval boundary explicitly
A useful AI deployment has a written boundary statement: which systems may contribute retrieved context, which identities’ permissions gate that retrieval, where the model executes, where prompts and answers are logged, and how long they are retained. If a vendor cannot fill in that sentence for their product, the placement decision is being made implicitly by their architecture.
Permission-aware retrieval is the line item most often missed: an assistant that retrieves with a service account’s broad rights leaks by design, wherever it is placed. Placement controls where data flows; permissions control who it flows to. You need both.
The six decision factors
- Privacy: can the prompt and the retrieved content legally and contractually leave your boundary? If contracts, patient data, or client records are in scope, this factor usually decides alone.
- Residency and sovereignty: do regulations or customer commitments name where data is processed, not just stored, and under whose jurisdiction the operator falls?
- Performance: does the workload need low-latency access to local data, or is round-trip latency to a hosted model acceptable? Retrieval-heavy workloads feel network latency twice.
- Control: do you need to pin model versions, audit exact behavior over time, and change on your schedule rather than a provider’s?
- Resilience: must the capability keep working when the WAN does not, and does it need backup and recovery like any other production service?
- Economics: is usage predictable and heavy enough that owned capacity beats metered pricing, or bursty enough that metered wins?
Token economics versus owned capacity, honestly
Metered cloud pricing and owned GPU capacity are different cost shapes, not different philosophies. Metered cost scales with use and includes the provider’s margin on every token; owned capacity is a step function: high at token one, flat until you outgrow it.
The honest comparison prices both fully. For cloud: realistic monthly token volume across all workloads, growth, and the internal cost of the data-handling reviews each new workload triggers. For owned capacity: hardware amortization, power and cooling, the platform work (or platform product) around it, model lifecycle effort, and utilization, because a GPU used ten percent of the time costs ten times its benchmark price per token. Neither side wins in general. Steady, retrieval-heavy, boundary-constrained workloads tilt local; bursty, unconstrained ones tilt cloud.
Sizing private capacity: the inputs that matter
- Model size and quantization: the parameters you actually need for the task, not the largest number available.
- Context length in real use: retrieval-augmented workloads consume context fast, and context drives memory.
- Concurrency: peak simultaneous requests, not average.
- Response-time target: interactive drafting and batch summarization have different hardware answers.
- Retrieval workload: embedding, indexing, and search capacity beside the model.
- Resilience: whether inference must survive a node failure, and what degraded service is acceptable.
- Shared duty: whether the same cluster also carries VMs, protection, and recovery services.
- Growth: the workload you are sizing is the smallest it will ever be.
Three placement patterns and a routing policy
- 01
Cloud
Hosted models and hosted management. Fastest to adopt, always current, and right for workloads whose data may travel and whose usage is bursty. The cost is dependence on the provider’s availability, models, and terms.
- 02
Hybrid
Hosted management and intelligence with local execution and local data where policy demands it. The most common end-state for organizations with any regulated or latency-sensitive workload. The critical artifact is the routing policy: a written rule for which workloads, data classes, and identities route to which model placement.
- 03
Private
Models, knowledge, and inference on infrastructure you control. Right when privacy, residency, or disconnected operation are hard requirements, or when inference volume makes owned capacity economic. The cost is real: capacity planning, lifecycle, and sizing are now your responsibilities, so a validated platform matters more than a parts list.
The operational tail: lifecycle, logging, recovery
Private AI is a production service and inherits production obligations. Model lifecycle: versions are pinned, updates are staged and tested, and a rollback model stays available. Logging and retention: prompts and answers are operational records; decide what is kept, for how long, and who can read it, and audit access like any sensitive log. Resilience: the vector store, configuration, and fine-tuned artifacts need backup and a tested recovery path, and if AI capability matters during a WAN outage, that requirement belongs in the cluster design, not in a postscript.
Disconnected and air-gapped operation is a step beyond local: it constrains model updates, telemetry, and support paths, and should only be claimed where the architecture and the support model genuinely deliver it.
The decision table
| Factor | Favors cloud | Favors hybrid | Favors private |
|---|---|---|---|
| Data classification | Public / internal only | Mixed tiers by workload | Confidential / regulated in prompts or retrieval |
| Residency commitments | None named | Named for some data | Named for processing, or sovereignty-constrained |
| Latency profile | Tolerant | Sensitive for some workloads | Local-data and retrieval heavy |
| Usage shape | Bursty, unpredictable | Mixed | Steady, high, predictable |
| Control needs | Provider cadence fine | Pin some workloads | Pin versions, audit behavior, own the schedule |
| WAN dependence | Acceptable | Acceptable for most | Must operate through outages |
A worked example on IXBox
A professional-services firm wants document drafting grounded in its engagement files, a client-facing summarizer, and an internal knowledge assistant. Classification puts engagement files at confidential with client residency commitments; the summarizer touches only public material.
The routing policy that falls out: the summarizer uses a hosted model; drafting and knowledge retrieval run locally. Sizing starts from a mid-size quantized model, long context for retrieval, a peak of a dozen concurrent users, and an interactive response target, which lands on a GPU-equipped validated IXBox cluster that also carries the firm’s local VMs, protection, and recovery. Prompts and answers for local workloads are logged locally with a defined retention period; the vector store is in the backup plan; a local-only policy applies to the drafting workload, with no silent cloud fallback. The exact models, capacity, and support boundary are confirmed during validated-scope design, which is precisely why the scope exists.
Questions teams ask
Is a private deployment automatically compliant?
No. Running a model on your own hardware supports control objectives; it does not by itself satisfy any framework. Policy, identity, configuration, process, evidence, and independent requirements still apply. Placement is one control among many.
What does local-only actually guarantee?
Under a local-only policy, prompts, retrieved context, model execution, actions, and retained records stay inside the defined private boundary, and there is no silent fallback to a cloud model under load or failure. Ask any vendor to state fallback behavior in writing.
Do we have to choose one placement for everything?
No, and you should not. The routing policy exists precisely so each workload lands where its data class, latency, control, and economics point, on one operating model.
Where Ixvara fits
Ixvara IQ routes between cloud and private models by policy, with no silent fallback in local-only mode. IXBox provides validated clustered capacity when placement decisions point local. The same platform, wherever the answer lands.
About the author
Darren ReevesFounder, Ixvara
Darren has spent more than fifteen years operating enterprise infrastructure and delivering managed IT and cloud services. Ixvara grew from the recurring problem he saw across those environments: every platform contained a piece of the truth, but no shared operating layer connected the evidence, authority, action, and outcome around the service being delivered.
See this working on your environment.
Tell us the problem that matters. We will build a focused demonstration around it.