Sovereign generative AI
Sovereign Generative AI
Why Running a Self-Hosted LLM Is Structurally Unsustainable
Analysis for IT Directors
It is tempting for a CIO concerned with isolating critical systems to consider hosting in-house the LLM(s) useful for developers or company operations. We will consider here the use of DeepSeek V4 Pro (MoE, Think, 1.6T params, 49B activated, 1M context). The promises are seductive: total data control, independence from API providers, unlimited customization. But behind the appeal of bare-metal lies an operational reality that few organizations are ready to face. Analysis.
The True Cost of Running
A model with 1.6 trillion parameters requires approximately 20 NVIDIA H100 GPUs in INT4 quantization to operate. In cloud rental, the entry ticket is around €30K per month, or €360K annually for computing power alone. But this figure is misleading.
Human costs must be added. Deploying and supervising an inference cluster of this scale requires at least 1.5 to 2 FTE specialized MLOps engineers, or €150K to €200K annually including benefits in France. Managing model updates, optimizing serving (vLLM, SGLang, TensorRT-LLM), monitoring latency, managing the KV-cache, and reconfigurations during load spikes cannot be improvised.
Add the network layer, perimeter security, software licenses, NVMe storage for checkpoints and backups. The real TCO is between €550K and €650K per year — for a single instance, without redundancy.
An Architectural SPOF by Design
Here is the most critical point: the model is distributed across 20 GPUs using tensor parallelism. If a single GPU fails, the entire service goes down. There is no degraded mode. There is no partial failover. A cluster of 20 GPUs mechanically offers 20 times more failure surface than a standard server.
To achieve a minimum standard SLA of 99.9% for an enterprise production service, the infrastructure must be doubled: a second cluster in hot standby, i.e., €60K/month in GPU rental and an annual TCO exceeding one million euros. At this point, the question is no longer technical but strategic: does the service rendered justify such an investment?
Planned Obsolescence as a Boundary
The iteration pace of foundation models is accelerating exponentially. DeepSeek V3 was released in early 2025, V4 a few months later. Each new generation makes the previous one significantly less performant. A CIO committing to a 3-year GPU lease for a given model invests in an asset whose useful value halves every 6 to 12 months.
In the case of hardware purchase (bare-metal option, ~€300K to €400K for 8 H100s), the obsolescence risk turns into an accounting trap: a fixed asset over 3 to 5 years for hardware whose technological relevance will likely not exceed 18 months.
Economic Rationalism vs. Control Instinct
Faced with these findings, using managed APIs, whether from DeepSeek, OpenAI, Anthropic, or emerging European players, presents a radically different profile.
In terms of volume, €30K monthly in DeepSeek API represents approximately 8 to 10 million requests per month, or 5 to 10 times the volume achievable with self-hosting. Elasticity is native: no provisioning, no preventive over-provisioning, no unused GPU at 3 a.m.
In terms of SLA, major API providers guarantee uptimes of 99.9% to 99.95% with automatic failover, global load balancing, and 24/7 support. A level of resilience that no organization can replicate internally at an equivalent budget.
In terms of maintenance, the marginal cost is zero: no patching, no model migration, no CUDA version management. When DeepSeek V5 is released, the transition will happen by changing a parameter in an API call.
How to Mitigate the Risks of the API Model
Let's be clear: the API model is not without risks.
Data sovereignty is the primary concern. Prompts and responses transit through third-party servers, potentially outside European jurisdiction. For sensitive data (health, defense, personal data under GDPR), this is a real obstacle. However, a workaround exists: some providers offer dedicated deployments in the EU region (Scaleway AI, Azure OpenAI in France, OVHcloud AI Endpoints), and end-to-end encryption combined with contractual clauses (DPA, SCCs) significantly reduces legal exposure.
Vendor lock-in is the second risk. But the current ecosystem is structurally competitive: APIs are largely compatible across providers (OpenAI format has become a de facto standard), and migrating from one provider to another takes hours, not months. The lock-in of a self-managed GPU infrastructure is, paradoxically, much more constraining.
Price variability is the third risk. API prices can change. But they have only decreased since 2023, dramatically (divided by 10 to 100 depending on the model). The strong market trend favors the API consumer, not the infrastructure operator.
Trade-off: The CIO's Decision Grid
| Criterion | Self-hosted | Managed API |
|---|---|---|
| Predictable cost | Fixed but high | Variable but optimizable |
| Sovereignty | Total | Partial (mitigable) |
| Resilience | Fragile (SPOF) | Industrial |
| Model agility | Low (heavy migration) | Immediate |
| Scalability | Capped | Elastic |
| Operational load | Heavy (2+ FTE) | Nearly zero |
| Obsolescence risk | High | Zero |
Self-hosting a trillion-class LLM is only rational for a limited number of organizations: those combining a strict regulatory imperative for data localization, a massive and constant inference volume justifying amortization, and a mature MLOps team already in place.
For all others (i.e., the vast majority of companies), the API model remains, in 2026, the most sustainable, resilient, and economically rational choice. The CIO's role is not to own the AI infrastructure, but to ensure that AI serves the company's strategy with the best value/risk ratio. To date, this ratio leans decisively toward the API.
