Integration / Self-hosted runtime
vLLM, routed by jurisdiction.
Register vLLM servers in your own VPC or data centre as YTT Secunet endpoints, so open-weight models serve as in-jurisdiction primaries or as the last-resort fallback that never leaves your network.
vLLM
Self-hosted runtime
- Regions documented
- 4
- Setup steps
- 5
vLLM is an open-source, high-throughput inference and serving engine for large language models, started at UC Berkeley's Sky Computing Lab. Its built-in server exposes an OpenAI-compatible API, so a model started with vllm serve answers chat completions and completions requests on a port you choose, optionally protected with an API key. Because you run it, residency is whatever you decide: the machine's location is the processing location, and there is no provider retention policy to read because there is no provider.
That makes vLLM the anchor of most YTT Secunet sovereign designs. Teams use it in three roles: as the primary for special category data under GDPR Article 9 or for classified workloads, as the in-jurisdiction fallback when a hosted EU endpoint is saturated, and as the only endpoint inside an air-gapped Sovereign Dedicated deployment. YTT Secunet treats a vLLM endpoint like any other: it is registered with a jurisdiction and facility, health-checked, load balanced across replicas, and covered by receipts that record the model id and the TLS certificate thumbprint of the server that answered.
Self-hosting moves responsibility to you. YTT Secunet cannot attest to the physical location of a server beyond what you declare and what the certificate thumbprint ties to, so the facility identifier in the receipt is the one you register. Pair it with your own asset inventory and, for regulated environments, with HSM-held receipt signing keys so the evidence chain stays inside your organisation.
| Region or deployment | Location | Jurisdiction |
|---|---|---|
| Customer-chosen location | Customer VPC, Frankfurt | EU / European Union |
| Customer-chosen location | On-premises data centre, Zurich | CH / Switzerland |
| Customer-chosen location | On-premises data centre, Riyadh | GCC / Middle East (GCC) |
| Customer-chosen location | Customer VPC, London | UK / United Kingdom |
- 01
Start vLLM's OpenAI-compatible server for your model with an API key and TLS termination in front of it, inside the network segment that will host the workload.
- 02
Run at least two replicas per jurisdiction so YTT Secunet can fail over between them without leaving the facility.
- 03
Register each replica in YTT Secunet with jurisdiction, facility identifier and the served model name; YTT Secunet pins the TLS certificate thumbprint on first contact.
- 04
Map an alias to the vLLM endpoints and attach them to policy packs as primary or fallback.
- 05
For air-gapped deployments, install YTT Secunet Sovereign Dedicated with offline licensing on the same network and point it only at local vLLM endpoints.
# On the GPU host in Frankfurt (customer VPC)
vllm serve mistralai/Mistral-Small-3.2-24B-Instruct-2506 \
--host 0.0.0.0 --port 8000 \
--api-key "$VLLM_API_KEY" \
--served-model-name mistral-small
# Register it with YTT Secunet
yttsecunet endpoints add \
--id vllm-fra-01 \
--provider openai-compatible \
--upstream https://vllm-fra-01.internal.acme.de/v1 \
--jurisdiction EU --facility customer-vpc-fra \
--model mistral-small --secret vault://vllm/fra- Self-hosted endpoints are the only targets allowed by default for special category data in eu-gdpr-strict; you can widen that per pack.
- A change in the upstream TLS certificate thumbprint pauses the route until an operator confirms the new certificate, which catches traffic silently redirected elsewhere.
- vLLM endpoints can be marked as last-resort fallback, so they absorb overflow without becoming the default path.
- Receipts record the served model name you set, so pin it to a specific checkpoint for Annex IV documentation.
Does YTT Secunet ship or host the models?
No. YTT Secunet does not host foundation models. You run vLLM and choose the weights; YTT Secunet routes to it, enforces policy and signs receipts.
Can YTT Secunet and vLLM run fully offline?
Yes, with Sovereign Dedicated. Offline licensing, customer-held signing keys and a local Merkle root publication target let the whole path run without internet access.
How is latency affected?
YTT Secunet's own telemetry puts its added p99 latency under 8 ms. When the data plane and vLLM share a facility, the network hop between them is typically small next to model compute.
Sovereign route ready
Stop risking 4% global turnover GDPR fines on AI inference.
Deploy sovereign, cryptographically proven AI routing in 15 minutes.