TriLens: Per-Layer Logit-Lens Entropy for White-Box Hallucination Detection

Bohan Yang1,2,3, Yijun Gong5, Zhi Zhang6, Ge Zhang1, Wenpeng Xing1,2,*, Meng Han1,2,4
1Zhejiang University, 2Binjiang Institute of Zhejiang University, 3Beijing Normal-Hong Kong Baptist University, 4GenTel.io, 5Great Bay University, 6University of California San Diego
*Corresponding author
Findings of EMNLP 2026

TriLens is a compact white-box hallucination detector that tracks how internal certainty forms across transformer depth by reading attention, feed-forward, and residual-stream states through the model's own logit lens.

Paper arXiv Code

Introduction

When a language model hallucinates, the final answer is wrong, but the mistake is not necessarily invisible inside the model. Different internal pathways may remain uncertain, disagree in how quickly they sharpen, or commit to competing continuations before the output is produced.

TriLens turns this intuition into a compact representation: at every layer, it reads the multi-head self-attention output, the feed-forward output, and the residual stream through the model's own logit lens, then records only the entropy of each readout.

The resulting trajectory describes how certainty forms across depth and across modules, without storing high-dimensional hidden states or sampling multiple generations. This makes TriLens compact enough for lightweight probes while preserving the internal structure needed for analysis.

TriLens Framework

TriLens extracts three module-wise entropy trajectories from each decoder layer: attention-output entropy, feed-forward-output entropy, and residual-stream entropy. These lightweight signals are then aggregated and passed to simple probes for hallucination detection.

Overview of the TriLens pipeline for module-wise logit-lens entropy extraction and hallucination detection.

TriLens reads multiple internal computation paths through the model vocabulary lens and tracks their entropy across layers as a compact white-box signal.

Main Results

We evaluate TriLens on three instruction-tuned language models and four English QA benchmarks under a matched protocol with six baselines. The compact entropy trajectories provide a strong and consistent signal across the full model–dataset grid.

12 / 12

cells above ICR Probe

11 / 12

best overall results

+12.1

mean AUROC points vs. ICR Probe

Complementary Module Signals

The gain is not confined to the residual stream. Attention, feed-forward, and residual-stream entropy trajectories capture different aspects of hallucination-relevant uncertainty.

Layer-wise separation and feature ablations showing the complementarity of TriLens module-wise entropy trajectories.

Adding either attention or feed-forward entropy to the residual-stream signal improves all 12 cells; the complete three-trajectory feature improves further in every cell.

Mechanistic Analysis

TriLens also exposes where hallucination-relevant uncertainty appears inside the model. Trajectory and layer-wise analyses reveal that the signal changes across depth and that the most discriminative depth depends on both the model and the benchmark.

Entropy Dynamics Across Depth

Supported and hallucinated responses can follow different entropy trajectories as internal computation progresses through the decoder stack.

Per-layer residual-stream logit-lens entropy and descriptive AUROC for correct and hallucinated responses.

The trajectory view shows how uncertainty can enter, persist, and resolve differently across generation settings.

Peak-Discriminative Depth

A descriptive single-layer analysis asks where residual-stream entropy most clearly separates correct from hallucinated responses, without using that layer selection inside TriLens.

Heatmap of peak-discriminative relative depth across models and benchmarks.

Peak depth varies across model–dataset pairs, supporting the use of the full layer-wise trajectory rather than a universal hand-picked layer.

Generalization Across Settings

Beyond per-benchmark fitting, we test whether the compact entropy representation remains useful under benchmark shift. The cross-dataset diagnostic trains on one benchmark and evaluates on another while keeping the model family fixed.

Cross-Dataset Transfer

We compare TriLens with ICR Probe and SAPLMA across every train-on-row, test-on-column benchmark pair.

Cross-dataset generalization heatmaps for TriLens, ICR Probe, and SAPLMA.

TriLens is strongest on all 12 off-diagonal transfers, reaching a mean AUROC of 0.848 compared with 0.738 for ICR Probe and 0.678 for SAPLMA. We treat this as a benchmark-shift diagnostic rather than universal cross-domain generalization.

BibTeX

The EMNLP 2026 proceedings metadata is not yet available. Please cite the current arXiv version.

@misc{yang2026trilens,
  title         = {{TriLens}: Per-Layer Logit-Lens Entropy for White-Box Hallucination Detection},
  author        = {Yang, Bohan and Gong, Yijun and Zhang, Zhi and Zhang, Ge and Xing, Wenpeng and Han, Meng},
  year          = {2026},
  eprint        = {2606.01033},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  doi           = {10.48550/arXiv.2606.01033},
  url           = {https://arxiv.org/abs/2606.01033}
}