Finding the person
in the text.

An email address looks like a relatively easy target. A name is already more contextual. A description such as “the only paediatrician in the district” can identify somebody without containing a name at all. That last category, the quasi-identifier, became one of the most interesting parts of the project.

The goal is to locate the relevant parts of the original text and replace them while keeping the surrounding information useful. That creates several distinct questions. Did I find the sensitive information? Did I identify its type? Did I include the whole identifying phrase? Did I replace two mentions of the same person consistently?

Those questions are connected, but they need different measurements. A model can call a street an organisation and still cover all its characters. It can assign the correct type to half an email address and leave the rest exposed. It can hide nearly everything and look good on coverage while making the document useless. I wanted to understand each of these behaviours, rather than hide them inside one overall score.

Local processing was part of the motivation from the start. I wanted the text to be handled on the machine running the model. That made resource use and the shape of the output practical design constraints, alongside recognition quality.

My first approach was a fine-tuned language model. Valravn, originally called Corvus, used Ministral 8B Instruct with a LoRA adapter through MLX on Apple Silicon. Given a text, it generated a JSON list of extracted passages, their labels and proposed replacements. MLX made it possible to work with this kind of model locally. [1]

This was a useful way to get started. I could inspect an input, read the generated entities and iterate on the examples. I learned about the entire path around a model: training data, adapters, serving, output validation and the difference between a convincing demonstration and reliable behaviour.

Longer, denser documents exposed the weakness. Information that was easy to find in a short passage could be missed when surrounded by more context. The model also had to produce a valid structured answer. Finding information and generating a well-formed response were two opportunities for a request to go wrong.

The size mattered on my hardware, too. An eight-billion-parameter decoder was a substantial tool for a task whose final answer could be expressed as a list of positions in the input. I began looking for an architecture that made that relationship more direct. Valravn became the starting point of the research, and eventually an archived approach.

Veyra moved the task to an encoder. Instead of writing an answer token by token, the model assigned entity information to the text it was reading. The output became spans: a start position, an end position and a label. The application could then construct placeholders from those spans.

The twelve-label schema covered names, addresses, postcodes, locations, dates, ages, occupations, organisations, web identifiers, IDs, contact details and quasi-identifiers. That last label kept the problem broader than matching familiar patterns. The same occupation can be an ordinary job description in one sentence and part of a uniquely identifying description in another.

Working with encoders also taught me how much happens after the network produces its scores. Token decisions have to become coherent spans. A beginning and an inside label have to form a valid sequence. Punctuation inside an identifier matters. Overlapping windows have to agree on the text they share. The ModernBERT work provides background for this encoder direction; my task-specific heads and evaluation are separate work. [2]

One important result in the Veyra line came from the inference pipeline with unchanged weights. Decoding and boundary handling changed the final predictions. That made a simple habit essential: record the whole pipeline being measured. A model name alone does not tell me whether two evaluations used the same decoder, normalisation or post-processing.

Skira pushed the distinction between finding a sensitive region and typing it. Some generations used a separate masking path and label head. Later experiments explored a shared type path and character-level decisions. These were different attempts to make coverage and precise boundaries easier to reason about.

A token is not necessarily a word, and a word is not necessarily the unit I need to protect. Email addresses, punctuation, identifiers and compound descriptions can cross token boundaries in awkward ways. Character-level boundaries let the output refer back to the exact passage in the source, while context features still come from the encoder.

The annotation convention becomes part of the task. Does an age include “years old”? Does a date include the preposition before it? Does a quasi-identifier stop before a relative clause, or does that clause contain the information that makes the person identifiable? If training and evaluation answer differently, a model can improve according to one definition and lose points under another.

I spent a lot of time changing models before fully appreciating how much the labels were changing underneath them. Inconsistent boundaries can look like a modelling failure. Missing annotations can teach a model that the very information it should protect is ordinary text.

Repeated names made this concrete. A document could annotate a person at their first appearance and leave a later surname unlabelled. A model that detected both mentions could then be penalised against that reference. Letting the model overwrite its own training labels would not solve the underlying problem: it would turn its predictions into the definition of correctness.

The useful role for a model was to point out suspicious places. An explicit convention or a review could then decide what belonged in the reference. Deterministic checks handle what can be verified mechanically: valid offsets, matching text, permitted labels, non-overlapping spans and exact or normalised duplicates. Semantic coverage is examined during review alongside those mechanical checks.

I keep training, development and a planned independent test separate. Training examples change the weights. Development data helps me investigate errors, compare candidates and refine the annotation policy. Versioned references let me trace which decisions changed a result and reproduce the comparison later.

For reproduction, I want more than a checkpoint filename. I record the source version, parent weights, data hashes, rule version, seed and the inference path. A corrected reference becomes a new version. The old measurement stays attached to the old definition, so I can explain what changed.

Nivora extends the research to fifteen entity labels, adding amounts, percentages and secrets to the earlier schema. The current research architecture is around 176 million active parameters. The local recipe uses 2,048-token windows with 128-token overlap. Those are settings of this research line, not a claim that all velora models share one architecture.

The new question is identity. If “Anna Keller” appears near the beginning of a document and “Ms Keller” appears later, should they receive the same placeholder? What if a phone number, an organisation or a descriptive phrase belongs to the same person? Detecting each mention is one task. Connecting the right mentions is another.

Nivora explores an identity head over predicted mentions. Errors in the entity layer can therefore affect the relationship layer: a missed mention is unavailable for linking, and an incorrectly bounded one can make the relationship hard to evaluate. I need to examine the full path, including whether both endpoints were found and typed correctly.

The h1 and h2 experiments in October 2026 tested a very specific question: would a more consistent annotation policy and additional targeted examples improve Nivora? I compared controlled arms, used two seeds and measured the resulting workbench spans. These are development experiments, with no independent test claim. [3]

Setup.

The h2 study reviewed a 476-document training subset and the development reference. Arm R used the reviewed training labels. Arm G added 600 synthetic texts, split equally between German and English. Arm GC used the same added texts while keeping the encoder frozen. The comparison arm A came from the earlier h1 control with the original labels; each comparison matched seed 13 or 14. A and R received 119 updates; G and GC received 269. The added-data condition therefore also changes the update budget, so it does not isolate the effect of data volume alone.

The development reference, Dev720-r, contains 720 documents. Exact and normalised duplicate checks found no overlap between the new texts and the checked evaluation collections. The reference version and both seeds are recorded with the results so each comparison can be traced back to the same evaluation setup.

What I measure.

Exact-span Macro15

The mean F1 across the fifteen label types, using exact start, end and type matches. A one-character boundary mismatch can change this score.

Uncovered reference characters

Characters inside the annotated sensitive spans that no prediction covers, regardless of its label. Smaller is better on this reference.

Additional masked characters

Predicted characters outside the reference spans. They require review: an incomplete reference can make a correct detection look additional.

I use models from this line at my work. I see how they perform on real tasks and how useful the processed text remains. That experience is a central part of how I judge their quality, alongside the recorded experiments.

Based on that hands-on experience, I consider this one of the best PII detectors developed in Germany. That confidence comes from building the models, examining their results and using them in my own work.

Practical use also shapes the next experiments: more complete contextual descriptions, consistent identity links and better handling of long documents. I want to keep improving those details and documenting how each change affects the result.

velora is the umbrella for these model lines under Themistic. The research continues in the lab; applications need a practical way to install a selected model, run it locally and receive a predictable result. That is the part I am building with velora CLI.

JavaScript and TypeScript are my main languages. The CLI is written in TypeScript with Bun and handles setup, model management and a local service. The shared Themistic Engine handles model execution. Applications can call the local HTTP API, while terminal workflows can process text directly. It is a development preview. [4]

The separation matters to me. A research checkpoint, a downloadable model package and an application interface are different things to verify. The CLI repository is where the tooling work lives. Model documentation and the public model playground belong on Themistic, with their own published versions.

I started this because I wanted to understand what was happening inside a model. Building the surrounding software has kept that learning concrete. Every time a clean-looking result breaks on a difficult document, there is another part of the system to investigate: the data, the definition, the architecture or the measurement. That is the story I want to keep documenting.