On-premise AI and self-hosted LLMs

What does on-premise AI mean in practice?

It means the weights, the inference and the data all sit inside a boundary you control. No prompt, no document and no embedding crosses to a third party. In exchange you take on the hardware, the operations and the upgrade path.

Self-hosted LLM and on-premise LLM describe the same arrangement. The distinction that matters is not the phrase but where the boundary is drawn, and who is accountable for what crosses it.

What architectures are available?

• an open-weights model hosted on your own infrastructure, which you own outright;
• work conducted inside a ringfenced environment over SSH or VPN, with the data remaining where it is;
• commercial APIs under zero-retention, zero-training terms; or
• anonymisation that preserves the statistical properties of the data while removing identifiers, including a bridge approach in which no client data reaches a model provider at all.

These are not a ladder from worse to better. They are four positions on a trade-off, and different processes within the same organisation often sit in different places.

When is a self-hosted open-weights model the right choice?

When the data cannot leave, when the workload is steady enough that owned capacity costs less than metered inference, or when you need the model itself to be an asset rather than a subscription. Owning the weights also fixes behaviour: the model does not change underneath a process that has been validated against it.

It is the wrong choice when the workload is small and irregular, when the capability required is only available from a frontier commercial model, or when nobody in the organisation will own the operational burden.

What is the bridge approach to anonymisation?

Anonymisation removes identifiers while preserving the statistical properties that make the data useful. A bridge approach goes further. The mapping between real entities and their anonymised substitutes is held inside your boundary, and only the substituted data is ever processed, so no client data reaches a model provider at all.

It suits work where the analysis depends on structure rather than on identity. It does not suit work where the identity is the subject.

What are the trade-offs?

• CostOwned capacity is a fixed cost and metered inference is a variable one. Which is cheaper depends on volume, and the crossing point is calculated rather than assumed.
• CapabilityA frontier commercial model may do things an open-weights model cannot. Whether that matters depends on the quality threshold the use case has to meet.
• Operational burdenSomeone has to run the hardware, patch it and own the upgrade path. That role is named before the architecture is chosen, not after.
• Governance exposureEvery boundary the data crosses is an approval, a contract and a control. Fewer crossings is a simpler position to defend.
• StabilityOwned weights do not change underneath a validated process. A hosted model can be deprecated on a vendor's schedule.

Most organisations end with a mixture. The architecture is chosen per use case rather than as a policy for the whole estate, and the reasoning for each is recorded so that it can be revisited when the volume or the capability changes.

Who decides which architecture is used?

You do. We set out the trade-offs in cost, capability, latency, operational burden and governance exposure, and build within whichever architecture your security and governance requirements permit.

The decision is a governance decision rather than a technical preference, and it should be recorded as one.