NER-based PII Anonymization: OpenAI Privacy Filter Analysis & Lightweight Implementation of privyscope-ko

“Honestly, I didn’t even start this project with NER (Named Entity Recognition) in mind. But as I worked through it, the results kept pointing right back to it, so I decided to go back to square one and dive in properly.

My reaction went from ‘Huh, this is actually pretty neat’ to ‘Wait, this might actually be super useful.’ So I decided to document that shift in perspective here.

Realistically, this probably won’t interest most people—it’s mainly for those into NER or developers working on PII (Personally Identifiable Information) protection systems. My main thought was: could this be a breakthrough for the long-standing CJK (Chinese-Japanese-Korean) tokenization issues? Let’s see.”

I think I first started studying Natural Language Processing (NLP) about 7 or 8 years ago. My initial problem was crawling news articles—there were way too many duplicate stories. I tried extracting subjects and verbs from headlines to measure similarity and tell if two articles were actually covering the same thing or taking different angles. Well, clickbait titles ruined that plan pretty quickly.

Fast forward to 5 years ago. I found myself diving into Transformers and BERT to analyze server command logs. The goal was to figure out what users were doing and evaluate their permission scopes using NLP.

Lately, I’ve seen AI folks around me just throw raw sentences directly into LLMs for everything. In fast-paced business environments, I get why people take that shortcut. But sending full text to an LLM just for simple entity extraction or PII redaction feels like overkill—both in terms of compute cost and latency.

My third journey into NLP started recently while working with Meta’s BART model, which naturally led me to study how OpenAI’s Privacy Filter actually works under the hood. It made me revisit classical Named Entity Recognition (NER), and honestly, I was surprised by how precise and lightweight it could be.

In this post, I’ll analyze the technical mechanics of OpenAI’s Privacy Filter and walk through how I stripped down its heavy parts to build privyscope-ko—an open-source, lightweight PII redaction pipeline optimized for Korean production environments.

1. The Evolution of NER: From Classic Models to OpenAI Privacy Filter

Named Entity Recognition (NER) is a technique that locates and classifies predefined categories in unstructured text—like names, organizations, locations, dates, and PII.

Comparing traditional NER with OpenAI’s Privacy Filter shows a clear shift in how modern NLP engines are built:

Why BIOES Beats BIO

Traditional NER relies on BIO (Beginning, Inside, Outside) tagging. However, BIO frequently stumbles on single-token entities or text without spaces, causing boundary errors.

Privacy Filter and modern pipelines upgrade this to BIOES:

• B (Begin): Starting token of an entity
• I (Inside): Inner tokens
• O (Outside): Non-entity tokens
• E (End): Final token of an entity
• S (Single): Standalone single-token entity

In agglutinative languages like Korean, where particles and endings attach directly to words, BIOES works wonders. Clearly distinguishing single-token entities (S-PER) and entity endings (E-LOC) drastically improves word boundary precision.

Constrained Viterbi Decoding

If you classify tokens independently using naive Argmax, you get weird, broken tags—like an I-PER showing up right after an O.

OpenAI Privacy Filter fixes this by placing a Constrained Viterbi Decoder with learned transition biases on top of the logit layer from a single forward pass. It optimizes transition probabilities across the whole sequence and blocks grammatically impossible tag shifts.

2. OpenAI Privacy Filter Architecture

OpenAI Privacy Filter overcomes the limits of traditional Encoder-only models using a unique pretraining and conversion pipeline:

Plaintext

[ Autoregressive Pretrained Checkpoint (gpt-oss based) ]
                         │
                         ▼ (Checkpoint Conversion)
[ Bidirectional Banded Attention Token Classifier (Band Size = 128) ]
                         │
                         ▼ (Post-training)
[ Supervised Classification Loss (33 Logits/Token: 8 Labels × 4 Tags + O) ]
                         │
                         ▼ (Inference Time)
[ Constrained Viterbi Sequence Decoding ]

Key Architectural Highlights

1. Pretraining Conversion: It takes a generative decoder model pretrained autoregressively on gpt-oss architecture and converts it into a token classifier with Bidirectional Banded Attention (Band Size = 128).
2. Mixture of Experts (MoE): Out of 1.5B parameters, it routes Top-4 experts per token across 128 Sparse MoE layers, driving active parameters down to ~50M during inference.
3. 128K Long Context: Handles massive documents in a single pass without chunking, delivering high throughput.
4. Operating Point Tuning: Allows runtime parameters to adjust the balance between Precision and Recall on the fly.

3. Lightening the Load: privyscope-ko Simplification Strategy

OpenAI Privacy Filter’s architecture is impressive, but running a 1.5B MoE model with 128K context layers on standard on-premise servers or edge devices is overkill. The memory footprint and serving costs quickly add up.

To build privyscope-ko—a Korean PII masking pipeline—I intentionally dropped the heavy components to get instant deployment capability and blistering speed.

What We Cut and Streamlined

1. No MoE / Giant LLM Backbone: Replaced OpenAI’s 1.5B MoE with a compact, single klue/roberta-base encoder (~110M params).
2. Reduced Context Window: Scaled back from 128,000 tokens to Max Sequence Length = 256. Since most Korean PII masking happens sentence-by-sentence, this massively slashes CPU memory usage.
3. Zero PyTorch Runtime Dependency: Removed PyTorch and CUDA entirely at inference time. Instead, it uses INT8 Quantized ONNX Runtime, shipping as a lightweight ~105MB standalone binary.
4. Two-Stage Hybrid Architecture: Instead of trusting deep learning for everything, Stage 1 uses regex engines and checksum validators to catch structured PII (Resident Registration Numbers, phone numbers, etc.) with 100% accuracy.

4. privyscope-ko Open Source Pipeline & Implementation

privyscope-ko is a hybrid engine designed to detect Korean-specific PII entities (PER, PHONE, ID_NUM, EMAIL, LOC, BANK, DATE, SECRET).

Core ONNX Inference & Constrained Viterbi Code

Here is the core Python implementation that combines ONNX Runtime with Constrained Viterbi decoding without needing PyTorch at all:

Python

import onnxruntime as ort  
import numpy as np  
import re  
  
class PrivyscopeKOEngine:  
    def __init__(self, onnx_model_path: str):  
        # Create inference session using ONNX Runtime only (no PyTorch needed)
        self.session = ort.InferenceSession(onnx_model_path)  
        
        # Define BIOES tag set (8 entities * 4 tags + O = 33 classes)
        self.labels = self._init_bioes_labels()  
      
    def _init_bioes_labels(self):  
        entities = ["PER", "PHONE", "ID_NUM", "EMAIL", "LOC", "BANK", "DATE", "SECRET"]  
        labels = ["O"]  
        for ent in entities:  
            for prefix in ["B-", "I-", "E-", "S-"]:  
                labels.append(f"{prefix}{ent}")  
        return labels  
     
    def _viterbi_decode(self, logits: np.ndarray) -> list:  
        """
        Constrained Viterbi Decoding: Blocks invalid BIOES transitions
        """
        seq_len, num_classes = logits.shape  
        dp = np.full((seq_len, num_classes), -np.inf)  
        backpointer = np.zeros((seq_len, num_classes), dtype=int)  
     
        # Initialize Start Token ('O', 'B-', or 'S-' allowed)
        for idx, label in enumerate(self.labels):  
            if label == "O" or label.startswith("B-") or label.startswith("S-"):  
                dp[0][idx] = logits[0][idx]  
     
        # Dynamic Programming transitions
        for t in range(1, seq_len):  
            for next_idx, next_label in enumerate(self.labels):  
                for prev_idx, prev_label in enumerate(self.labels):  
                    # Validate tag transition rules (e.g., O -> I is invalid)
                    if not self._is_valid_transition(prev_label, next_label):  
                        continue  
               
                    score = dp[t-1][prev_idx] + logits[t][next_idx]  
                    if score > dp[t][next_idx]:  
                        dp[t][next_idx] = score  
                        backpointer[t][next_idx] = prev_idx  
     
        # Backtrack optimal path
        best_path = [np.argmax(dp[-1])]  
        for t in range(seq_len - 1, 0, -1):  
            best_path.append(backpointer[t][best_path[-1]])  
      
        best_path.reverse()  
        return [self.labels[i] for i in best_path]  
     
    def _is_valid_transition(self, prev: str, curr: str) -> bool:  
        if curr.startswith("I-") or curr.startswith("E-"):  
            # I and E tags must follow B or I tags of the exact same entity
            prev_entity = prev.split("-")[-1] if "-" in prev else ""  
            curr_entity = curr.split("-")[-1]  
            return (prev.startswith("B-") or prev.startswith("I-")) and (prev_entity == curr_entity)  
        return True  
     
    def redact(self, input_ids: np.ndarray, original_text: str, offsets: list) -> str:  
        # 1. ONNX Model Inference
        inputs = {self.session.get_inputs()[0].name: input_ids}  
        logits = self.session.run(None, inputs)[0][0]  # Shape: [SeqLen, 33]  
     
        # 2. Viterbi Decoding
        decoded_tags = self._viterbi_decode(logits)  
      
        # 3. Span Extraction & Redaction Offsets
        spans = self._extract_spans(decoded_tags, offsets)  
      
        # 4. String Redaction
        masked_text = list(original_text)  
        for start, end, label in sorted(spans, key=lambda x: x[0], reverse=True):  
            masked_text[start:end] = f"<{label}>"  
      
        return "".join(masked_text)  

5. Benchmarks & Real-world Performance

To evaluate privyscope-ko, I tested it on a Korean PII evaluation dataset using Precision, Recall, and Strict F1-Score metrics.

Entity-level Strict F1-Score

• Hit the target goal of Strict F1 ≥ 0.93.
• ID_NUM and PHONE achieved > 0.99 precision, thanks to regex and checksum validation in Stage 1.

Latency & Resource Consumption

Plaintext

[ Single Sentence Processing Time & Memory ]

OpenAI Privacy Filter (1.5B MoE)
[████████████████████] ~180ms / VRAM 4.2GB

PyTorch KLUE-RoBERTa
[████████] ~35ms / VRAM 1.8GB

privyscope-ko (INT8 ONNX Runtime)
[██] ~8.5ms / RAM 120MB (Runs purely on CPU!)

By dropping heavy decoder architectures and using ONNX Runtime, privyscope-ko hits sub-8.5ms per sentence on a single CPU core with just 120MB memory!

Example

$ pip install privyscope-ko
$ privyscope redact "안녕하세요. 지난 7월 20일에 OOO 제품을 주문한 홍길동입니다. (주문번호: 20260720-123456) 주문한 지 일주일이 넘었으나 배송이 시작되지 않아 주문 취소 및 환불을 요청합니다. 제 연락처는 010-1234-5678이며, 당초 배송지는 서울시 강남구 테헤란로 123, 101동 1004호였습니다. 환불금은 OO은행 123-4567-8901-23(예금주: 홍길동) 계좌로 입금해 주시면 됩니다. 출고 전이 라면 즉시 취소 처리 부탁드리며 처리 완료 시 문자로 안내해 주시기 바랍니다. 감사합니다."

6. Wrapping Up

OpenAI Privacy Filter proved that combining BIOES tagging with Constrained Viterbi Decoding is the right way forward for modern NER.

However, from a practical engineering standpoint, taking these algorithmic improvements and pairing them with INT8 ONNX quantization and a 2-stage Regex hybrid approach gives you far better ROI for real-world deployments.

privyscope-ko was built on this exact philosophy—bringing state-of-the-art NLP mechanics into a ultra-fast Korean redaction tool. Next up, I’m planning to add multi-language support and compile it to WASM so it can run 100% locally inside client browsers.