Unlimited-OCR 2 on 16 GB: it reads as well as Datalab's Chandra 2, but it is not unlimited on my card

On October 27 Baidu released Unlimited-OCR 2 (the Hugging Face repository exists since the 26th): 3.3B parameters, an MIT license, and a concrete promise. With its SR-SWA sparse attention, the README says, it keeps “per-step decoding latency and memory footprint nearly constant regardless of document length”. That can be measured. That same night I had it running on my 16 GB RTX 4070 Ti SUPER, against DeepSeek-OCR, the model it derives from, against its own version 1, against Tesseract and against what I used for huge files: Datalab, with its Marker pipeline and its Chandra 2 model, both locally.

It read as well as Chandra 2: on 20 pages of a real Central Bank of Chile report, a 0.12 % character error rate against 0.31 %, a tie within the margin. It beat Tesseract (0.51 %), Marker (2.5 %) and DeepSeek-OCR (3.1 %). The promise of reading without limit, though, I did not see hold on 16 GB. With transformers, memory grows with every page and runs out at 80. With SGLang, time per token did stay flat, but the window mode hung, and with a single window the server returned 24 of 40 pages without any error.

What I measured and how

Before measuring I wrote a sheet with four hypotheses and its decision rule, and committed it to git. Later I added five amendments, each committed before the measurement it governs; two are post hoc and say so, and the fifth added Datalab. The second also fixed the single-page runs, whose smoke test had already started.

The hypotheses: that Unlimited-OCR 2’s memory and milliseconds per token grow less than 25 % between 10 and 80 pages in a single request (H1); that version 1, without SR-SWA, grows more or runs out of memory sooner (H2, the positive control); that its per-page error is no worse than DeepSeek-OCR’s and that both beat Tesseract (H3); and that reading the whole document at once does not make the error worse than reading it page by page (H4). Amendment 5 added that it is no worse than Marker or Chandra 2 (H6) and that those two get through the full report, 74 pages, without running out of memory (H7).

Two documents. The first, the opening of Don Quixote typeset to PDF with PyMuPDF, without hyphenation: each page’s reference text is exactly what was printed. The second, real: the 20 pages with the most text and no images of the September 2026 Monetary Policy Report of the Central Bank of Chile, with the PDF’s text layer (pdftotext -layout) as reference. Every page was rendered once at 144 dpi and every engine read the same PNGs. I did not give Marker the PDF: it would have pulled the text layer, which is exactly the reference.

The metric is the character error rate (CER): the Levenshtein distance between the output and the reference, divided by the reference length, after the same normalization for everyone (Unicode NFKC, without the boxes and markup the models emit, lowercase, collapsed whitespace).

Each model ran in its own environment with the versions its model card pins: Unlimited-OCR 2 with transformers 4.52.1, version 1 with 4.57.1 and DeepSeek-OCR with 4.46.3 and PyTorch 2.6. All in BF16 and one request at a time, with a GPU lock so no other compute job would cut in.

Installation, with its stumbles

Unlimited-OCR 2 downloaded and loaded without trouble, but the first page failed with FileNotFoundError: '/sbin/ldconfig'. Its attention kernels use Triton, which looks for the CUDA library with ldconfig, and on NixOS that program is not at that path. The fix is to tell it where the library is: TRITON_LIBCUDA_PATH=/run/opengl-driver/lib. Version 1 asked for matplotlib, which its README does not list. DeepSeek-OCR works without flash-attn if loaded with eager attention.

Marker 2.0 had its own stumble. For its OCR model, Surya 2, it starts a server with docker run --runtime nvidia, and on NixOS the GPU reaches Docker through CDI, not that runtime. I solved it by starting the same server myself: llama-server in llama.cpp’s official image with the GPU through CDI, the Surya 2 GGUF and the same arguments Surya would pass, with Marker attached through SURYA_INFERENCE_URL. Along the way the container could not find the weights: the Hugging Face cache stores symlinks into ../../blobs, and the whole repository has to be mounted. Chandra 2 installed and ran without stumbles.

The modified SGLang Baidu publishes in its repository took more work, and I tell that below.

Quality: on clean text they all tie, on the real report Unlimited-OCR 2 and Chandra 2 come out on top

On the clean Quixote everyone landed practically at zero, Tesseract included. That instrument tells nothing apart, so amendment 3 added the same 20 pages degraded like a scan: rotation up to 1.5°, lower resolution, blur, noise and quality-35 JPEG, all with a fixed seed. There Tesseract had one page at 11.9 % error and the vision models never went above 0.29 %, except Marker, with one page at 1.3 %. But the medians stay at 0.1 % or below: it is a mild degradation.

The real report is where they separate. Paired differences per page, with 95 % bootstrap confidence intervals:

• Unlimited-OCR 2 vs Chandra 2: -0.15 points (CI -0.36 to 0.00). A tie: the interval reaches zero.
• Unlimited-OCR 2 vs Marker: -0.68 points (CI -2.94 to -0.17). Better.
• Unlimited-OCR 2 vs DeepSeek-OCR: -1.5 points (CI -7.3 to -1.3). Better.
• Unlimited-OCR 2 vs Tesseract: -0.29 points (CI -0.44 to -0.18). Better, barely.
• Unlimited-OCR 2 vs its version 1: -0.32 points (CI -0.69 to -0.25).
• DeepSeek-OCR vs Tesseract: +2.8 points (CI +1.0 to +8.7). Worse.

DeepSeek-OCR did not invent text: it left it out. On 6 of the report’s 20 pages it returned less than 90 % of the characters (Unlimited-OCR 2, on 1); on page 57 it returned 2,602 of 3,241 and skipped the last paragraphs. So H3 splits: Unlimited-OCR 2 is not worse than DeepSeek-OCR, it is clearly better; but “both beat Tesseract” is false, because on the real report DeepSeek-OCR lost to Tesseract.

Marker is a different case. On a typical page (page 29, at 2.9 % error), almost all the error comes from where it places the page footer (“informe de política monetaria septiembre 2026 28”) and how it numbers the notes: the text is there. CER penalizes that ordering, and penalizes it equally for everyone. H6 holds: Unlimited-OCR 2 is no worse than Marker or Chandra 2.

On speed the order flips. Note that Tesseract ran on my Mac’s CPU, not on the card.

one page20 pages at once

Tesseract 5.5.22.4 s

Marker 2.016.2 s

DeepSeek-OCR16.5 s

Unlimited-OCR 121.2 s

Unlimited-OCR 235.7 s

Chandra 237.4 s

Animation ten times faster than measured. Tesseract ran on another machine’s CPU; for Marker and Chandra the figure includes loading the model.

Unlimited-OCR 1, transformers1599.5 s

Unlimited-OCR 2, transformers1046 s

Unlimited-OCR 2, SGLang112.5 s

Animation a hundred times faster than measured. Same median CER on both runtimes.

RTX 4070 Ti SUPER, BF16, one request at a time. Tesseract: CPU of an Intel Mac.

With transformers, Unlimited-OCR 2 is the slowest: about 36 s per page of the report, twice DeepSeek-OCR. I suspect its sparse attention in that implementation, but I did not isolate it. With SGLang it drops to 4.2 ms per token: the same 20 pages take 9.3 times less than with transformers, with the same median CER. The outputs are not identical: 5 of the report’s 20 pages and 6 of the Quixote’s differ in details, such as an accent, a word or an underline; none in a figure.

A huge file: the full report, 74 pages

For what I used Datalab for, the question is a different one: getting a long PDF through whole without anything falling over. I gave Marker and Chandra 2 the 74 pages of the report.

Both finished, and H7 holds. The memory is what each one took on the card, net of the 2.4 GB of the other processes. Marker was almost four times faster. There are two setup differences that may explain it and that I did not isolate: Chandra, with the hf method, reads one page at a time (its default batch size), and Marker sends its requests to a server with 8 parallel slots. Unlimited-OCR 2 page by page would take what it takes for one page, about 8.2 GB, but I did not put the 74 pages through that way: what I tested was its promise of reading them in a single request, and that comes below.

Reading the whole document at once did not change the error

The model can read every page in a single request, to use context across pages. On the real report it neither helped nor hurt: the median CER was 0.25 % in a single request against 0.12 % page by page, a paired difference of +0.03 points (CI -0.04 to +0.13). On the clean Quixote, +0.05 points. H4 holds: it does not get worse.

This conclusion changed during the audit. My first version said it got worse by a full point, and that was my mistake: the normalization left inside the text the coordinates of the titles Unlimited-OCR 2 emits, and multi-page reading marks more titles. I fixed it, logged it in the sheet and recomputed everything.

The constant-memory promise, measured with transformers

The sheet required peak memory and milliseconds per token to grow less than 25 % from 10 to 80 pages. Version 1, without SR-SWA, was the control: it had to grow more. Both feed each page as 256 image tokens (a 1024-pixel global view), so the comparison is even.

• Unlimited-OCR 2:7.5 → 11.1 · 80 out of memory
• Unlimited-OCR 1:8.4 → 10.6 · 40 out of memory

Unlimited-OCR 2 reached 40 pages and failed at 80 after 74 minutes of generation. Version 1 failed as soon as it started on 40. At 20 pages, version 2 used 8.7 GB and version 1, 10.6 GB.

H1 does not hold. From 10 to 40 pages, Unlimited-OCR 2 went from 7.5 to 11.1 GB (1.49 times) and from 35.5 to 52.7 ms per token (1.49 times). At 80 pages it ran out of memory after 74 minutes of generation, with 12.7 GB allocated.

H2 does hold: version 1 grows more. From 10 to 20 pages it went from 8.4 to 10.6 GB and from 34 to 61 ms per token, and at 40 it ran out of memory after 5 seconds, while reading the pages. Version 2 does the same with less: at 20 pages, 8.7 GB and 40 ms per token.

That is consistent with what the code says. In transformers, generation uses a dynamic cache and turns the sliding window off: it keeps every token. Attention only reads 16 blocks plus a 128-token local window per step, but everything else stays in memory. The cross at 80 pages sits below the free-memory line because memory allocated by PyTorch does not count the CUDA context or what its allocator reserves.

The cache keeps growing in transformers even though attention only reads 16 blocks per step. SGLang keeps per-token time flat, but its windows stalled here and a single window cut the document where the pool filled up.

It has to be said: other processes were open on that card (an open game and ComfyUI) and held 2.4 GB. The model had 13.6 GB, not 16. On an empty card the 80 pages might have finished; I did not measure that.

And with SGLang, the runtime Baidu recommends

The README recommends SGLang for serving, with a trick: read the document in windows of 10 pages and pass on to the next window only the tail of the previous one. That is where the bounded memory would come from. Amendment 4 measured it, with a 90-minute cap to get it working.

Installing its modified SGLang took four fixes on NixOS: NVIDIA’s package index for cuda-tile, a CUDA_HOME pointing at the nvcc that pip brings, ninja and a C++ compiler for the kernels it compiles on the fly, and a link to libcudart.so. After 15 minutes the server was answering.

Time per token stayed flat: 4.17 to 4.19 ms between 10 and 40 pages. That part of the promise does show.

The window mode does not. With 20 pages, the server finished the first window, logged retained 640 of 16177 window tokens and in the following 9 minutes produced nothing more, with the client’s request still open. I also tried it with max-running-requests at 64 and asking for windows without overlap (overlap_pages at 0), and it stopped at the same point; I did not keep the logs of those two tries, because each server start overwrote the previous one. The 10-page row of the table does not exercise windowing: it fits in a single window.

Then I tried the document in a single window, the other configuration the README documents. That change is not in the sheet: it is exploratory. The server reserves a fixed cache: 42,304 tokens, 2.4 GB. The 40 Quixote pages need more. The server gave no error, and that is by design: the README’s command launches it with --allow-auto-truncate, which cuts instead of rejecting. It returned the first 24 pages, with 10,965 input tokens and 31,295 output tokens, exactly what fits in the cache. My client did not log the response’s finish reason, so I do not know whether it flagged it; nothing in the returned text says so. Had I not counted the page markers, I would not have noticed.

H5, which required 80 complete pages, does not hold in this configuration. Had the windows worked, it might have.

What these numbers do not say

• It is a 16 GB card shared with other processes (2.4 GB taken), not the README’s 80 GB A800.
• SGLang’s window mode hung in my installation, on NixOS and with my own fixes. It may be my environment.
• The scan-like degradation was mild; I did not test real scans with known text.
• The report has tables: the pdftotext reference and the models’ output can order cells differently, and that counts as error equally for everyone.
• Tesseract ran on another machine; its time does not compare with the card’s.
• I measured Marker and Chandra locally. Datalab’s paid API, which is what I used before, says it uses better models than the open weights.
• The long runs are one per size (two at 10 and 20 pages, less than 1.5 % apart).

My take

The good:

• It tied at the top with Chandra 2 on the real document, and page by page it takes about 8.2 GB of the card.
• Its license is MIT, no strings attached. Chandra 2’s weights are free only under 2 million dollars in funding or revenue and cannot be used to compete with Datalab’s API; Marker’s, under 5 million.
• With SGLang, 4.2 ms per token and flat time as the document grows.
• Version 2 clearly scales better than version 1 with transformers.

The not so good:

• With transformers memory grows with the document: “unlimited” does not hold on that path.
• In SGLang, the mode that would make the document unlimited hung for me, and the no-window mode cuts the document without an error.
• Reading the whole document at once did not improve the error on the real report.

Bottom line: for page-by-page OCR on 16 GB, Unlimited-OCR 2 and Chandra 2 are the best I measured, and Unlimited-OCR 2 has the freer license. For huge files, Marker is still the practical choice. Reading hundreds of pages in a single request, on my card, not yet.

When I would use it and when not

• I would use it to read documents page by page on a GPU with 12 GB or more (it takes about 8.2 GB), especially with tables and dense text; Chandra 2 reads just as well, but its license comes with conditions.
• I would stay with Marker to get huge PDFs through whole: 74 pages in 12.6 minutes using 4.2 GB of the card, and the full text even if it moves the footer.
• I would serve it with SGLang rather than transformers, in single-window mode and with documents that fit in its cache: the same CER, nine times faster.
• I would not hand it a long document in a single request without counting the <PAGE> markers in the output afterwards.
• I would stay with Tesseract for clean text on CPU: it tied on quality and was 15 times faster per page.
• I would not use DeepSeek-OCR with these settings on documents where losing paragraphs matters.

Sources

• Unlimited-OCR 2 on Hugging Face, with the README, the technical report PDF and the SGLang configuration.
• Unlimited-OCR repository, with the modified SGLang and the inference code.
• Unlimited-OCR 1 and DeepSeek-OCR, the other two models.
• Datalab’s Chandra and Marker, with the license of Chandra’s weights.
• Tesseract, the baseline.
• Central Bank of Chile, Monetary Policy Report, September 2026.
• Don Quixote on Project Gutenberg, public domain.
• The sheet with its amendments, the scripts, the raw outputs and the analysis: lab/unlimited-ocr/ in the repository and in this gist.

Measured on October 27 and 28, 2026 on fedora: 16 GB RTX 4070 Ti SUPER (driver 595.71.05), NixOS with kernel 6.18.55, with 2.4 GB of the card taken by other processes. Unlimited-OCR 2 (revision 9cf077c), Unlimited-OCR 1 (8249d20) and DeepSeek-OCR (9f30c71) in BF16; transformers 4.52.1, 4.57.1 and 4.46.3; Baidu’s SGLang 0.5.20.dev10. Marker 2.0.0 with Surya 2 on llama.cpp and Chandra OCR 0.2.0 (datalab-to/chandra-ocr-2, hf method). Tesseract 5.5.2 on the CPU of an Intel Mac.