Does Qwen3.8-27B fit in 16 GB of VRAM? I measured it, and found out my own benchmark was lying

Alibaba released Qwen3.8-27B on 5 August under Apache 2.0, and the GGUF quantizations appeared on the 13th. With them came the usual guides, all repeating the same figure: about 15 GB of VRAM in 4 bits.

My GPU has 16 GB. None of those guides answered the only question I cared about, which is whether it fits in a card like mine. So I measured it.

A double spoiler: it does not fit, and it works anyway. And the model ended up finding a bug in my own benchmark.

The hardware and the question

A 16 GB RTX 4070 Ti SUPER, on a Ryzen 7 7800X3D. The card started empty: 15,954 MiB free and zero processes on the GPU. The harness verifies that before starting, and aborts if it finds anything on it, because a contaminated measurement is not a measurement.

Two stumbles before the first metric

The first: my ollama was on 0.20.3 and the model rejected it with a 412 and a message asking for a newer version. The manifest requires 0.32.12. Updating and restarting the service fixed it.

The second: the download died at 4 GB with a TLS handshake timeout. Since ollama pull resumes where it left off, I wrapped it in a retry loop and it carried on without losing what was already down.

When it finished, the number that already answered half the question showed up:

qwen3.8:27b    17 GB

Seventeen gigabytes. The card has sixteen. The “about 15 GB” figure the coverage repeats corresponds to the parameters in 4 bits and nothing else; the real file also carries the vision encoder, and on top of that you have to make room for the context cache.

What ollama show says about the model:

How I measured

Seven repetitions per configuration, with a warm-up run discarded so that loading the weights does not get counted as inference. temperature=0 and a fixed seed, so that what varies is the system and not the sampling. VRAM sampled every 100 ms on a separate thread, reporting the peak minus the baseline. And the CPU/GPU split ollama reports, because without that number a tokens/s figure cannot be compared to anything.

The numbers

context 4,096context 16,384context 32,768

GPU75% of the layers

CPU25% of the layers

tokens/s

28.86

VRAM pico

14,438 MiB

primer token

199 ms

The best case. Even so a quarter of the model lives off the card, and it still gives more tokens per second than I can read.

GPU71% of the layers

CPU29% of the layers

tokens/s

26.55

VRAM pico

14,576 MiB

primer token

209 ms

Quadrupling the context pushes 4% more layers to the CPU and costs 2.3 tokens per second. The key-value cache takes room too, and competes with the weights.

GPU66% of the layers

CPU34% of the layers

tokens/s

23.70

VRAM pico

14,724 MiB

primer token

225 ms

A third of the model on the CPU. Multiplying the context by eight costs 18% of the speed, and the trend does not settle: it keeps tilting.

Three things jump out.

The answer to the question in the title is no, with a caveat. The model never loads whole: even at the smallest context, a quarter of it stays on the CPU. And it still gives 28.86 tokens per second, which for reading on screen is faster than you read.

The context penalty is gradual and predictable. Each jump pushes more layers off the card, the split tilts toward the CPU and performance drops: from 25% to 29% to 34% CPU, and from 28.9 to 26.6 to 23.7 tokens per second. Multiplying the context by eight costs 18% of the speed.

VRAM never hits the ceiling. The peak stops at 14,724 MiB out of 16,376. ollama leaves headroom on purpose instead of pushing the card to the edge and risking an out-of-memory failure.

All 21 responses came out identical to each other, so determinism with a fixed seed holds.

A warning about the deviation is in order. Across the seven repetitions of a single run it ranges from 0.03 to 0.06 tokens per second, a number that invites overconfidence. But I ran the whole harness twice, and the 16,384 context gave 25.17 the first time and 26.55 the second: a 5% difference, a hundred times larger than the internal deviation. Repeating inside one session measures that session’s noise, not real reproducibility. The table’s numbers are good as an order of magnitude and as a relative comparison between contexts, not as figures exact to the second decimal.

The model corrected me

Here comes the part I did not expect.

My harness compares the answer against a verifiable result saved in advance. All 21 runs gave distance_km=240 hours=4 and the harness marked every one of them wrong, because my file said the right answer was distance_km=204 hours=3.4.

Before writing that the model fails at elementary arithmetic, I did the sum by hand.

The problem: a train leaves A toward B at 60 km/h; two hours later another leaves B toward A at 90 km/h; there are 420 km between A and B.

When the second one starts, the first has covered 120 km and 300 km remain between them. They close at 150 km/h, so they meet 2 hours later: 4 hours from the first one’s departure, at 60 × 4 = 240 km from A. Check: 240 + 90 × 2 = exactly 420 km.

With my saved answer, 204 km and 3.4 hours, the trains would still be 330 km apart. They never meet there.

The model was right all 21 times. The one that was wrong was my benchmark.

And worse still: that false value did not live only in the comparator, it was written inside the prompt file, on a line that said “(Verifiable answer: …)”. Which means it was being sent to the model along with the problem. I was leaking it a wrong answer and it still gave the correct one, contradicting the very data the prompt handed it.

I fixed both things. The problem and the answer now live in separate files, and the harness aborts if it detects the expected answer inside the prompt, so this contamination cannot silently repeat. With the input corrected, the result is 7 out of 7 correct.

I am publishing the bug because it is the most useful part of the article. A benchmark that returns a wrong verdict with a deviation of 0.03 looks exactly as rigorous as a correct one. The precision of a measurement says nothing about whether you are measuring what you think you are.

My take

For the practical question of whether it is usable on a 16 GB card, the answer is yes, with the warning that it is not a clean VRAM load and that context is paid for in speed. If your use case lives in short contexts, nearly thirty tokens per second from a 27B model with vision on a consumer card is a good deal.

What bothers me is the figure going around. “About 15 GB in 4 bits” is not exactly false, but it describes a model that does not exist as a downloadable file: it ignores the vision encoder and the context cache. The useful figure is the real file’s, which is 17 GB, and nobody publishes that one because it means downloading it.

When I would use it

• At a 4096 context and short tasks: that is where it performs and the CPU spill barely shows.
• I would not set it to process long documents on this card. At 32,768 a third of the model is already on the CPU and the trend only gets worse.
• If you are buying hardware for this model specifically, 24 GB loads it whole and avoids the split entirely.

---

The raw data for the three configurations, with the literal responses from every repetition: qwen38-bench.json.

Sources

• Qwen3.8-27B. The original model card, published on 5 August 2026, where the Apache 2.0 license and the architecture with a vision encoder come from.
• Qwen3.8-27B-GGUF, by Unsloth. The quantizations I measured, uploaded on 13 August. The 17 GB size is that of the Q4_K_M file in that repository, not an estimate.
• Context length, in the Ollama documentation. num_ctx and why widening it costs memory on top of the weights.