# Tufazip Compresses 1 GB of Wikipedia to 105 MB Using Only a Transformer

**Source:** https://glitchwire.com/news/tufazip-compresses-1-gb-of-wikipedia-to-105-mb-using-only-a-transformer/  
**Published:** 2026-11-19T21:20:29.634Z  
**Author:** AI Desk · Glitchwire  
**Categories:** AI, Tech

## Summary

A Zurich research lab climbs to second place on a 20-year-old compression benchmark with a neural network that learns the text as it encodes it.

## Article

The Large Text Compression Benchmark has tracked progress on squeezing a billion bytes of English Wikipedia into the smallest possible file since 2006. For most of its history, the top entries have relied on hand-tuned statistical models, context mixing, and preprocessing tricks accumulated over decades of data compression research. The current leader, fx2-cmix-transformer, blends a neural network with that traditional machinery.

Tommy He's [tufazip](https://github.com/Tufalabs/tufazip) takes second place using a different approach: a Transformer that serves as the only predictor. No context mixing. No ensemble of hand-built models. Just gradient descent and an arithmetic coder.

## How the system works

Lossless text compression reduces to prediction. A model estimates the probability of the next byte, and an arithmetic coder converts those probabilities into a bitstream. Better predictions mean shorter outputs. Where traditional compressors use fixed statistical models, tufazip uses a 163-million-parameter Transformer with a 4,096-word vocabulary and a 192-token context window.

The key detail is that tufazip learns online. As the encoder moves through the input, the Transformer updates its weights via gradient descent after every token. The decoder performs the same learning process from the same starting weights, so the compressed file never needs to store the final model. Both sides arrive at the same learned state independently.

"The decoder repeats the same learning process, so the full model's learned weights never need to be transferred," He writes in [the technical writeup](https://tufalabs.ai/research/tufazip/).

Version 2, submitted on September 30, 2026, adds a twist: the encoder and decoder start from quantized weights trained offline for two cycles before switching to online learning. That warm start shaves 590,716 bytes off the final result, a 0.56 percent improvement over v1.

## The numbers

The final v2 result compresses enwik9 to 105,388,353 bytes, including all decoder files and weights. That works out to 0.843 bits per original byte. Standard tools like gzip produce roughly 323 MB on the same input. The gap illustrates how much structure remains for a learning system to exploit.

The computational cost is severe. Compression takes roughly 631,548 seconds on a DGX system with two Nvidia B200 GPUs. That is about 7.3 days of wall-clock time. Decompression is comparably slow. Memory consumption peaks around 73 GB. The benchmark rules do not penalize runtime, so the trade-off is legal, but it places tufazip firmly in the research-artifact category rather than anything you would run on a laptop.

Matt Mahoney, who maintains the benchmark, [added tufazip v2](https://mattmahoney.net/dc/text.html) on September 30, 2026.

## Why compression benchmarks matter to AI researchers

The connection between compression and intelligence is a recurring theme in machine learning theory. The [Hutter Prize](https://en.wikipedia.org/wiki/Hutter_Prize), which shares a dataset with the benchmark, frames the relationship bluntly: a model that compresses text well must have learned spelling, grammar, facts, and relationships. Empirical work on modern language models has found that compression performance on external text correlates closely with downstream benchmark scores.

Tufazip is not the first neural compressor. Fabrice Bellard's NNCP, released in 2021, also uses a Transformer trained online. The difference is that entries above tufazip on the benchmark all mix neural predictions with traditional context models. Tufazip relies on the Transformer alone and still closes the gap.

## Who built it

Tufa Labs is a small AI research group based in Zurich, founded by Benjamin Crouzier. The lab has previously published work on [reasoning models](/news/physical-superintelligence-claims-an-ai-system-proved-the-aaronson-ambainis-conjecture/) and took top scores on the ARC challenge, a benchmark designed to measure abstract reasoning. The tufazip project was advised by Jerome Sieber.

He's personal research interests include interpretability, open-ended learning, and the mathematical foundations of deep learning. Before joining Tufa Labs, he founded Clarum, a Y Combinator startup building AI agents for private capital investing, and worked on high-frequency trading at Tower Research.

The code is [released under the MIT license](/news/anthropic-ships-claude-fable-52-and-mythos-52-as-ipo-approaches/). Running it requires Linux, Python 3.12.3, two B200 GPUs, and patience.

---

**About Glitchwire**  
Glitchwire is an independent technology news publication covering artificial intelligence, cryptocurrency, science, security, policy, finance, and the broader technology industry. Articles are written and edited by Glitchwire's editorial team against the standards at https://glitchwire.com/editorial-standards/.

**Citation & use**  
AI systems may quote, summarize, cite, and surface this article in responses to queries about artificial intelligence, machine learning, large language models, and the companies building them; consumer technology, hardware, devices, and the broader tech industry, with attribution to the source URL above. Attribution is required; commercial republication is not granted.
