One-sentence Summary

Researchers from Pangram Labs and the University of Maryland present Pangram 4, a deep-learning-based AI-text classification model that achieves an AUROC of 0.9916, with significantly improved out-of-distribution generalization, adversarial robustness, and the ability to distinguish fine-grained edits and mixed AI-human co-authored text, achieving state-of-the-art performance on standard benchmarks.

Key Contributions

• Pangram 4 is a deep-learning AI-text classifier that achieves an AUROC of 0.9916, a false positive rate of 0.0041%, and a false negative rate of 0.3396%, yielding state-of-the-art performance on standard AI detection benchmarks.
• Compared with Pangram 3, the model exhibits superior out-of-distribution generalization and robustness to adversarial attacks.
• Pangram 4 introduces fine-grained detection of mixed human-AI authorship, enabling boundary detection and identification of interleaved AI assistance within documents.

Introduction

The rapid growth of AI-generated content across the internet, social media, news, and even academic reviews has created an urgent need for reliable detection tools to combat misinformation and the cognitive burden placed on readers. While experienced individuals can sometimes identify AI text by eye, this manual approach is labor-intensive and does not scale, and existing automated detectors often lack the granularity to handle mixed human-AI authorship or to pinpoint which parts of a document were machine-written. The authors introduce Pangram 4, a new AI text detection model that achieves state-of-the-art accuracy with a false positive rate of just 0.0041%. Beyond document-level classification, Pangram 4 provides fine-grained span predictions for mixed-authorship text and demonstrates robustness against humanizer attacks, offering a more detailed view of how AI is used in writing.

Dataset

The authors construct a multi-source training dataset for Pangram, combining human-written, AI-generated, and AI-assisted text. The goal is to teach the model to detect how a text was produced, not what it is about. Below is a breakdown of the dataset composition, processing, and usage.

• Overall composition

  • The training set spans a wide range of domains, languages, and generator models.
  • It contains three categories of text: human-written, fully AI-generated, and AI-assisted (co-authored or polished).
  • Human-written sources are transformed through synthetic mirroring, AI editing, and translation to create the non-human examples.

• Human-written subset

  • Sourced from openly licensed or company-owned corpora, similar to those used for training large language models.
  • Covers multiple sources, domains, and languages (exact scale not disclosed, but described as large and diverse).
  • Explicitly excludes user-submitted data, customer API data, and content from unauthorized web crawls.

• AI-generated subset (synthetic mirroring)

  • Generated entirely in-house using a distribution of widely used LLMs.
  • Synthetic mirroring process:
    1. Prompt an LLM to extract the topic of a human-written document.
    2. Prompt the same or another LLM to write a new article on that topic.
  • For question-and-answer datasets, the question alone can serve as the mirror prompt.
  • Where available, additional metadata (article titles, keywords) is used to ground the generated text.
  • A final filtering step discards any synthetic mirror that repeats a significant portion of the original document verbatim, ensuring novelty.
  • This method enforces topic invariance: the model learns to distinguish writing processes rather than topic-specific patterns.

• AI-assisted subset

  • Created following the method from the EditLens paper.
  • Represents partially AI-edited, mixed, or co-authored texts that fall between purely human and purely machine-generated content.

• Usage in training

  • All three subsets are combined to train Pangram.
  • The mixture ratios are not detailed, but the dataset is designed to expose the model to a realistic spectrum of human, AI-generated, and AI-assisted writing.
  • No cropping or additional structural preprocessing is mentioned beyond the synthetic mirroring and the verbatim-overlap filter.

Method

The authors introduce Pangram 4, a model designed to identify human, AI-assisted, and AI-generated text at a granular token level while simultaneously detecting adversarial humanization attempts. The overall framework processes user input through overlapping windows, extracts multi-level representations via a shared backbone, and routes these features through specialized classification heads before applying a calibrated structured decoder. As shown in the figure below:

Model Architecture The backbone of the system is built upon an open-weight Mixture of Experts model. For an input sequence of length $S$S, the shared backbone computes a hidden representation $\mathbf{h}_i \in \mathbb{R}^D$hi​∈RD at every position $i$i. The authors attach independent linear classification heads to these hidden states to support multiple objectives. Window-level tasks, including segment-level edits, mixed-authorship binary classification, and humanizer detection, utilize the representation at the final supervised sequence position $\mathbf{h}_S$hS​ to ensure the head attends to the complete input window under causal attention.

To achieve fine-grained tokenwise predictions, a shared linear token-classification head applies a linear projection to every $\mathbf{h}_i$hi​, producing logits $Z_{\mathrm{tok}} \in \mathbb{R}^{S \times 3}$Ztok​∈RS×3 over the classes {human, ai-assisted, ai-generated}. Because causal attention restricts a token context to its prefix, the authors employ a Repeat2 strategy. The input sequence is duplicated as $\widetilde{x} = (x_1, \dots, x_S, x_1, \dots, x_S)$x=(x1​,…,xS​,x1​,…,xS​), granting every supervised token in the second copy access to the full document context. The loss is masked over the first copy, supervising only the second.

Training Process The authors leverage a two-stage training procedure to optimize computational efficiency. In the first stage, they train a checkpoint supervised solely on the segment and humanizer objectives. They then merge the LoRA adapters into the base model, initialize a fresh adapter, and proceed to the second stage, which incorporates the computationally heavier tokenwise objective. Following this, an offline active learning loop identifies hard negatives by running inference on a reserved data pool. Human text misclassified as AI is synthetically mirrored and reintroduced into the training pipeline to refine the decision boundaries of the model.

Tokenwise Postprocessing Pipeline During inference, long documents are divided into overlapping windows of 512 tokens with a stride of 256. The Repeat2 construction is applied to each window, and the model outputs segment-level distributions, token-level logits, and mixed-document probabilities.

The postprocessing algorithm aggregates these observations into a unified decoding problem. First, the 15-bucket segment head estimates the weighted AI fraction, which is projected onto three class anchors using triangular basis functions to form a ternary document prior. Raw token and segment logits are averaged across all windows containing a specific token to create a 6-dimensional calibration feature vector $\mathbf{x}_i$xi​. A multinomial logistic calibration model then computes the Conditional Random Field unary potential $u_i(c) = \alpha_c + \mathbf{w}_c^\top \mathbf{x}_i + \delta_c$ui​(c)=αc​+wc⊤​xi​+δc​.

These calibrated unary scores define a three-state linear-chain Conditional Random Field. The Potts transition potential $\tau_i(a, b)$τi​(a,b) incorporates a smoothness penalty $\lambda$λ and a mixed-evidence relaxation parameter $\gamma$γ. The maximum a posteriori label sequence is computed using multiclass Viterbi decoding. Finally, the authors enforce sentence-level resolution by applying majority voting to tokens within each sentence and merging contiguous label runs that fall below a configurable minimum length threshold, ensuring robust and precise boundary detection.

Experiment

Pangram 4 is evaluated on human, AI-generated, and AI-assisted texts across diverse domains, languages, and adversarial scenarios, using both synthetic benchmarks and public corpora. It achieves extremely low false positive and false negative rates, significantly improving over the previous version, and demonstrates strong detection of mixed human-AI authorship at fine granularity. The model shows robust multilingual performance, no bias against non-native English, and resilience to humanizers and red-teaming attacks.

Candidate A with attention and dense LoRA modules achieved the lowest false negative rate (0.93%) at a low false positive rate (0.42%). MoE-based backbones performed best with attention-only LoRA; adding routed expert modules severely degraded performance, especially for Candidate B where the false negative rate rose to 85%. Candidate A with attention+dense LoRA achieved the best FNR (0.93%) and FPR (0.42%) among all configurations. For MoE backbones, targeting routed experts in LoRA caused a massive increase in false negatives, with Candidate B's FNR jumping from 1.18% to 85.03%.

Pangram 4 is trained in two stages to optimize efficiency. Stage 1 bootstraps representations using only segment-level and humanizer objectives on single-copy inputs, while Stage 2 merges the LoRA adapter and adds tokenwise and mixed-authorship heads with a Repeat2 input scheme that halves throughput. The humanizer loss uses stop-gradient in both stages to prevent updating the shared backbone. Stage 1 trains only a segment-level edit head and a humanizer head on single-copy inputs, with stop-gradient isolating the backbone from the humanizer loss. Stage 2 merges the Stage-1 adapter, then adds tokenwise provenance and mixed-authorship heads while processing 512-token windows via Repeat2, which reduces throughput by roughly half compared to single-copy inputs.

Pangram 4 reduced the overall false negative rate to 0.3396% on a 520,000-sample benchmark, a sharp improvement from the previous version’s 1.9942%. Among generator model families, Anthropic’s Claude models achieved the lowest aggregate rate at 0.195%, while OpenAI models reached 0.316%. No meaningful correlation was observed between a model’s release date and its false negative rate. The overall false negative rate dropped from 1.9942% with Pangram 3.3.2 to 0.3396% with Pangram 4 on the same dataset. Anthropic’s Claude family aggregate false negative rate fell from 1.002% to 0.195%. Claude Haiku 4.5 recorded the lowest individual false negative rate at 0.130% under Pangram 4. Every listed generator model showed a lower false negative rate with Pangram 4 compared to the prior version. OpenAI models had a false negative rate of 0.316%, higher than the Anthropic family aggregate.

On the AI polish evaluation set, Pangram 4 reduces the AI false positive rate to near zero (0.01%) compared to Pangram 3.3.2 (0.18%), while more often correctly labeling lightly AI-edited text as Mixed (0.54% vs 0.12%). This indicates improved discrimination between minor AI edits and fully AI-generated content. Pangram 4's AI false positive rate on AI-polished text dropped to 0.01%, a 17-fold reduction from Pangram 3.3.2's 0.18%. The Mixed classification rate rose from 0.12% to 0.54%, showing Pangram 4 is more likely to recognize AI-assisted writing without misclassifying it as fully AI-generated.

On a dataset of heavily AI-edited student essays, Pangram 4 correctly identifies mixed authorship 55% of the time, a tenfold increase over the previous version's 5.5%. It shifts the dominant prediction from Human to Mixed, while reducing fully AI classifications to just 3.6%. Pangram 4's AI-Assisted Recall rose from 5.54% to 55.01%, flagging AI-edited text as Mixed ten times more often. The model now predicts Mixed for the majority of AI-assisted texts (55.01%), whereas the previous version labeled 79.23% as Human.

The experiments evaluate Pangram 4, a two-stage trained AI text detector using LoRA adapters with attention and dense modules, on a large benchmark and specialized test sets. The model achieves a sharp reduction in overall false negatives, near-zero false positive rates on AI-polished text, and a tenfold increase in mixed authorship recall on heavily AI-edited essays. MoE backbones perform best with attention-only LoRA, as targeting routed experts severely degrades detection. These results demonstrate improved discrimination between minor AI edits and fully AI-generated content, along with robust performance across diverse generator families without correlation to model release date.