On-premise LLM deployment for sensitive data

An on-premise LLM deployment runs an open language model on your own servers, so prompts and documents stay with you. A private LLM deployment is the wider term: the servers can also be rented, or sit with a cloud provider. The definition is in our glossary entry on self-hosted LLM. The route suits enterprise teams, public bodies, NGOs, financial companies and energy companies whose texts contain personal data or business secrets.

Generator ETR for PSONI, the Polish Association for Persons with Intellectual Disability, is a web and mobile app that turns difficult text, such as an official letter, into easy-to-read (ETR) language. The contract was signed on 26 February 2026 for 24 months and runs in eight stages and two-week sprints. The pasted texts can be personal, so a public AI API was ruled out. We host an open model on rented GPU servers and serve it with vLLM, an open-source server for running language models.

A private LLM project starts with the choice of an open model, and we make it by testing. For PSONI we ran PLLuM 12B, a Polish-language model, on the same text simplification task as Gemma 4. Gemma 4 gave better output quality on the project data, so we went with it, in its 26B mixture-of-experts version (a design that uses only part of the model's parameters for each request).

The proof of concept ran from 21 May to 19 June 2026, about four weeks. It produced Bash and Docker scripts that start GPU instances the same way every time, vLLM serving Gemma 4 behind an API compatible with the OpenAI specification, a first prompt and a script that scores readability with the Gunning Fog metric.

The model sits inside a standard application your team can maintain: Django and Celery in the backend, React and React Native in the apps. Our work is choosing an open model, writing prompts, evaluating the output and serving the model. For an AI chat or an agent in your product, see our solution page.

In Generator ETR, Microsoft Presidio anonymizes personal data in texts, and we keep developing it. We are also analyzing whether to train HerBERT, a Polish language model, to anonymize Polish texts. Sensitive fields in the database are encrypted with Fernet, and stored texts are deleted when the retention period ends. The anonymized texts for the learning corpus can be downloaded as a single JSON export.

The API of Generator ETR is under construction. Access goes through an API key generated in the administration panel, and the endpoints are designed so that they can be switched off later. Following PSONI's decision of 1 October 2026, it will have four endpoints: login, generating an ETR text, the user's list of texts and a preview of a past text.

A script scores each simplified text with the Gunning Fog readability metric and writes a report over a set of test samples. We run an LLM evaluation on every iteration, and a reviewer trained in the ETR standard compares the output with the source text. The results go into the next iteration of the prompt and the model, and the fifth iteration is in progress. The collection of texts behind the test samples is checked by editors in a separate online service.

Each iteration has a goal and a QA report that puts the source text, the final text and its scores side by side. The first iteration (19 June to 6 August 2026) covered expanding abbreviations, formatting and replacing words that are too difficult; iterations two, three and four are closed. The specification requires the app to meet WCAG 2.2 level AAA, and we test accessibility with screen readers, starting with the web app.

Guardrails get tested too. Our QA tries to pull the model away from its task, for example by asking for an apple pie recipe instead of a simplified letter, and repeats these jailbreak tests with every new iteration. For legal and medical texts the app shows a notice: the simplified version is not a legal document, and a lawyer or doctor can explain the details. We are also testing a classifier for sensitive content: hate speech, self-harm, depression, fraud and manipulation. The scope is analysis and testing, and the client decides how the app behaves.

No. The EU AI Act classifies a system by its use, and where a model is hosted does not change that. Regulation (EU) 2024/1689 puts systems that assess the creditworthiness of natural persons in the high-risk category, except those used to detect financial fraud (Annex III, point 5(b)). We build software for regulated companies in fintech and energy, and our glossary entry on the EU AI Act explains the risk classes.

Every project starts with a free basic discovery, one meeting. From it we prepare a basic PoC or mockup that lets us estimate the cost; an extended discovery is a separate, wider scope. Discovery is also where we choose the model and the servers for the traffic you expect and go through retention and access to the stored texts. Copyright to the code passes to you with each payment, within the scope of use set in the contract.