Reels · 1 of 50
1:20
Reel · 1:20
NVFP4 vs FP8 vs GGUF on a 16 GB GPU without Blackwell: NVFP4 generates 2.3 times faster than BF16 and ties with BF16 on the full 1319-question GSM8K
Narrated: five formats of the same 4B model race on a 16 GB card. NVFP4 generates 2.3 times faster than BF16, FP8 reads the prompt fastest because the card has it in hardware; on all 1319 GSM8K questions the three vLLM formats tie with BF16, and token by token NVFP4 changes the choice one time in ten.
- Length
- 1:20
- Published
This reel sums up NVFP4 vs FP8 vs GGUF on a 16 GB GPU without Blackwell: NVFP4 generates 2.3 times faster than BF16 and ties with BF16 on the full 1319-question GSM8K, where the method, the tables and what did not work are.