Most enterprises run inference on someone else’s computer and call it strategy. Sometimes that is the right call: a low-stakes copilot, a burst of experiments, a model you will never need to produce in discovery. Sometimes it is a concentration of data, identity, and upgrade risk that no board would accept in a payment system or a factory control plane.
This is a method note from building a stack that runs open-weight models you hold and hash, on hardware the customer owns. The assembly line is ready to deploy that stack onto client premises. It is not a claim that your industry has already been through it. It is a claim about when the move is justified, and what “own” has to mean for the move to be real.
What does “sovereign” mean here?
Not a flag. Not a marketing synonym for “on-prem.”
Sovereign, in this use, means three facts you can verify:
- You can name the artefact (see weight hashing).
- You decide when it changes.
- The prompts, retrieval corpora, and outputs can stay inside a perimeter you control, including an air-gapped one if the engagement requires it.
If any of those three is missing, you have a hosted service with extra adjectives. A VPC that the vendor still patches, instruments, and can reach is not ownership. A “private endpoint” that still sends embeddings to a third-country control plane is not ownership.
The AI Act does not require sovereignty. It requires a provider and a deployer, and it will require documentation and oversight for high-risk systems on the 2027–2028 calendar. You can meet those duties on a vendor API. You cannot meet a discovery request for “the model that ran” if the vendor will not name it, and you cannot keep working through a vendor outage if the weights never sat on your floor.
When is a vendor API the correct answer?
When the use is reversible and the data is already allowed to leave.
Examples that usually stay on an API, unless a regulator or a customer says otherwise:
- drafting that a human edits before it becomes a record
- translation or summarisation of non-sensitive published text
- developer assistance on non-secret code
- a time-boxed evaluation of a frontier model you do not intend to operationalise
The tests are dull, which is why they work:
- If the prompt leaked, would you have to notify under GDPR, DPDP section 8(6), or a sector rule?
- If the vendor swapped the model on a Tuesday, would a regulated decision change without your knowledge?
- If the endpoint died for 48 hours, would a core process stop?
Three “no” answers: stay on the API, and write the role (deployer) and the Article 50 disclosure if people are talking to it.
One “yes”: you are past convenience. You are in custody.
When do you need the box?
When at least one of the following is true.
The data cannot leave. Privileged legal files, unpublished financials, clinical records, export-controlled technical data, or anything a contract already forbids sending to a US or third-country subprocessors list. India’s DPDP Act applies to processing in India and, under section 3(b), to processing outside India in connection with offering goods or services to Data Principals in India. “We used a US API” is not an answer to a fiduciary duty. It is a processor relationship you must have on contract (section 8(2)).
The decision must be replayed. Credit, hiring, claims, medical triage, safety interlocks, or anything you will have to show a regulator, a court, or a board committee. Replay needs the artefact, the prompt, the tools, and the version. Closed APIs rarely give you the artefact.
The agent can write. Tool-using systems (OWASP LLM06; ASI02 Tool Misuse) turn a text problem into an action problem. The more authority you grant, the less you should share the reasoning trace with a third party you cannot subpoena cleanly.
The upgrade clock is a business risk. Vendors retire models. They change tokenisers. They alter refusal behaviour. If your process is tuned to yesterday’s model, a silent upgrade is a production incident. Ownership means you promote a new digest on purpose.
The concentration is already unacceptable. One US cloud, one model vendor, one identity plane. CISA’s deployment guidance is written for operators who have to keep working when a component fails. If inference sits in the same failure domain as your office suite and your ticket system, you have one incident away from a three-system outage.
A customer or a state buyer will not sign unless inference stays in-country or on their metal. That is a commercial fact, not a philosophy. Price it as a deployment, not as a feature flag.
What do you actually have to own?
Four layers. Missing one re-creates the vendor problem on your own rack.
Hardware and facility. You or a colocation you contract, with access control you audit. A rented GPU you cannot walk to is still someone else’s computer. It can be the right someone. Name them.
Engine. The runtime that loads weights and serves tokens. If the engine can only fetch from the vendor’s registry, you do not own the engine.
Weights. Open-weight models you are licensed to hold, hashed on receipt, pinned on deploy. This is the Weight Integrity Seal method: we hold and hash the models we run; a customer stack is built so they can do the same. If the licence forbids holding, you are back on an API.
Control plane. Seats, keys, usage, lineage, and the compliance record. The Console is that pane on the customer’s box. If the only console is the vendor’s website, the vendor still owns operations.
Governance is a fifth layer when you need to detect, contain, and prove what the stack did. ASIS is that add-on: shipped zero-knowledge proofs and a post-quantum seal. Orachain is the private governance sidechain, live on mainnet, so a third party can verify integrity without seeing the records. None of those layers is a reason to skip the first four.
What does the EU AI Act do to this choice?
Less than vendors imply, more than isolationists hope.
Placing a system on the Union market, or using it under your authority in the Union, is enough for scope (Article 2). Running it in Frankfurt or in your basement does not, by itself, make you compliant. It can make you the provider if you put the system into service under your name (Article 3(3)), and Article 25 can flip a deployer into a provider after a substantial modification.
What ownership changes is evidence. Article 50 marking and disclosure are easier when you control the serving path. Articles 11 and 12 (technical documentation and logging), when they apply to high-risk systems, are easier when the logs are yours. Article 26 human oversight is easier when the gate is in your network.
What ownership does not change: Article 5 bans. A prohibited practice on your own GPU is still a prohibited practice.
The 2026 Digital Omnibus postponed most high-risk duties. It did not postpone the need to know where the system lives.
What does this cost you besides capital?
Staff who can patch an engine, rotate keys, and refuse an unsigned digest. Managed Operations exists for that work after go-live; it is scoped to the engagement. Remaining product development happens on client work. There is no honest week-count SLA in this paragraph because a week-count SLA would be a sales hope.
You also accept that you will not always have the newest frontier model on day one. That is the trade. You get the model you hashed, not the model a vendor promoted at lunch.
If the business needs a frontier model for a narrow evaluation, keep that evaluation on an API with a clean data diet, and keep the operational path on the box. Mixing them in one channel is how sensitive prompts leak into the evaluation account.
What should the CEO and the board ask?
- Which of our top ten AI uses fail the leak / swap / outage tests?
- For those that fail, do we hold the weights, or do we hold a contract?
- Who can promote a new model digest without a change ticket?
- If the primary vendor went dark for a week, which decisions would stop?
- Are we treating sovereignty as a slogan, or as a deployment we can walk to?
If the answers are vague, you are not ready to buy hardware. You are ready to map the estate.
What to do next
Run the three tests on every production use. Split the list into “API is acceptable” and “custody is required.” For the second list, decide department by department — a Department Deployment is the first governed stack in one function, on their hardware — rather than announcing an enterprise conversion you cannot staff.
An Architecture Review is the working session that draws that split and shows what a sovereign stack would replace. The stack is ready to deploy. The decision is whether you need it.