Kandinsky 6 on a 16 GB card: the video works, and only the big model passed my Spanish test

Kandinsky 6.0 is an open model that generates video and audio together: five seconds of picture with its sound, voices included, from a text prompt. It was released by Kandinsky Lab under the MIT license.

I wanted to know two concrete things. Whether it runs on a 16 GB card, like the one I have at home. And whether it speaks Spanish, because a model that moves its lips but says something else is not much use.

Before running I wrote down a measurement sheet with the question, the metrics and the decision rule, and committed it. Every later change went in as a dated amendment in the git history, and all of them are in the open. Git shows the order in which they were committed; since the per-run records do not keep the time, it does not by itself prove that each run came after its amendment.

What it is, exactly

The family comes in two sizes: Lite, with 3 billion parameters, and Pro, with 29 billion. Each has a full version, with 50 steps and guidance 5, and a distilled one that does the same task in 10 steps with guidance at 1, that is, without guidance. They work in two modes: text to video with audio, and text plus a starting image to video with audio.

Inside, a Qwen2.5-VL text encoder of 7 billion parameters turns the prompt into embeddings; a diffusion transformer generates the video and audio latents together; a video VAE, an audio VAE and a vocoder turn them into frames and into 44.1 kHz sound. That the audio is synchronized with the picture is what the authors say; I did not measure it.

The official recipe ran out of memory

I measured on a rented 16 GB RTX 4080, the same generation as my 4070 Ti SUPER but somewhat faster. I tried four ways of loading the distilled Lite model:

The README recipe, which moves each component to the GPU when its turn comes, ran out of memory; according to the log, while loading the text encoder. That is plausible: the Qwen2.5-VL files add up to 15.45 GiB, and the card reports 15.57 GiB usable. Dropping it to 8 bits was not enough either. Sequential offload, layer by layer, got further and, according to the log, failed in the video decoder. Where each one failed I infer from the logs, not from a profiler, and I did not measure why each one failed.

What worked was a full recipe: splitting the work into two phases that never share the card, streaming the encoder through the GPU in groups of layers, and decoding the video in tiles. I did not measure how much each part contributes on its own.

In the first phase, the text encoder streams through the GPU in groups of layers and leaves bf16 embeddings; it takes 0.7 to 1.3 seconds per prompt, median 0.7. I did not compare those embeddings numerically with the official recipe’s. In the second phase the encoder is gone: the transformer, the VAEs and the vocoder live on the GPU, and the video VAE decodes in tiles.

With that, all 38 text-to-video clips finished without running out of memory, and PyTorch allocated at most 11.24 GiB. In the three speed runs, a 480 × 864 vertical clip took between 140.4 and 141.1 seconds; a horizontal one, between 167 and 194. That time is generation and decoding; it excludes loading the models, 26 to 31 seconds across both phases, encoding the text and exporting to MP4.

Two costs that do not show in VRAM. The process reached 45.9 GiB of system RAM. And when I tried to repeat the Lite model on my own 30 GB machine, it stopped responding midway; I did not isolate the cause, so I cannot claim it runs there.

The voice, measured

I set the Spanish question like this: eight scenes with one short line each, two seeds per scene, in vertical format. Whisper large-v3 transcribes without being told the language, and I compare with the requested line. The rule: Spanish is usable if the median word error is at most 0.20 and Whisper detects Spanish in at least 12 of 16 clips.

As a control, the same scenes with the line in English. As a floor, the same lines read by the synthetic voice I use for the reels.

The word error can exceed 1: it counts substitutions, deletions and insertions against the length of the requested line, and here the transcripts add words. Without listening to every clip I cannot separate what the model added from what the transcriber invented. It also penalizes things that are not pronunciation errors, like “20” against “veinte”. It measures faithfulness to the text according to the transcriber, not pure intelligibility.

Predicted UTMOS estimates the score listeners would give, trained on synthetic voices. Here it scores the whole track, ambience and silences included, so I keep it as an exploratory reference and not as a measure of naturalness.

With Lite distilled, the Spanish fails: median error 1.06, and Whisper detects Spanish in 13 of 16. This is how it sounds:

Whisper transcribed “El servidor respondió en veinte milisegundos” (the server answered in twenty milliseconds) as “El cerebro respondió, resolvió, respondió, en deeto minisegúenes”. It is not only Spanish: in English the median error was 0.625, also far from the threshold. The blog’s bot, generated from an image, said its line with errors of 0.08 and 0.15, but two clips are not enough for a rule.

Distillation or size?

With a single model you cannot tell where the problem comes from. So, before running anything else, I added an amendment to the sheet: test Lite undistilled, 50 steps with guidance 5, and Pro distilled, the 29-billion model, on the same eight scenes with seed 0. After seeing the first undistilled-Lite results I widened the design with a second amendment: seed 1, to reach 16 clips per model as in the first round, and the English control. The error threshold did not change, and the final rule went back to the first round’s 12 of 16; the first amendment had scaled it to 6 of 8 for seed 0.

There was no 16 GB card with enough RAM for Pro, so I used a 24 GB RTX 3090, and the protocol set a 15.57 GiB cap on PyTorch’s memory, the usable capacity the 4080 reported. That cap controls what PyTorch allocates, not all of the card’s memory, and its effective value was not saved in each record. So this is not the same as running on a physical 16 GB card.

To compare machines I repeated a first-round clip with the same scene and seed: the transcribed line came out identical, letter by letter, and took 171.8 seconds against 140.4 on the 4080. It is a one-off check: the audio was not identical (UTMOS 2.13 against 2.10) and it does not give a factor to convert times.

Pro’s transformer files add up to 60 GB, almost four times a 16 GB card. I kept it in system RAM and streamed it through the GPU block by block, one at a time. Over the 24 clips, PyTorch allocated at most 9.09 GiB and reserved 13.65 GiB, and the process reached 120.25 GiB of RAM. That is the maximum of the whole process, not memory per clip nor a required minimum.

The second-round medians mix the Spanish and English clips. Seconds per clip exclude loading the models, encoding the text and exporting; for Pro, loading the generator took 52.5 to 59.1 seconds per session.

Pro distilled meets the rule: median error 0.15 and Spanish detected in all 16 clips. That is faithfulness to the text according to the transcriber, not proof of naturalness nor that every clip is usable: seven of the sixteen stayed above 0.20. Lite undistilled improves, but stays at 0.75.

Comparing each clip with the Lite distilled one of the same scene and seed, Pro lowers the error in 14 of 16, ties in 1 and worsens in 1; Lite undistilled lowers it in 11, ties in 3 and worsens in 2. These are eight scenes with two seeds each, not sixteen independent scenes.

After seeing the results I checked how much numbers written as digits weighed. Unifying “5”, “8” and “20” with their words, the medians become 0.97, 0.75 and 0.14; the paired counts and the rule’s outcome do not change. I report it as a post-hoc analysis, not as the main metric.

Some of Pro’s failures are different: the transcript has the line said right and then repeated, or with a tail added. In a test chain, two of six transcripts ended in something compatible with “fotorrealista”, as if the model were reading aloud the word Photorealistic from the prompt; I cannot claim it. That chain ran on another rented machine, an RTX 4090 with 252 GB of RAM, and its process reached 146.52 GiB.

On these eight scenes, Pro distilled was more faithful to the text than both Lite variants, and Lite undistilled also improved without reaching the threshold. That is compatible with an advantage of the bigger model, but it does not allow attributing it to size or ruling out that distillation matters too: I have no undistilled Pro in the same table, and steps, guidance, GPU and loading recipe all change at once.

The trick: the small model’s video, someone else’s voice

If the small model moves the picture well but says the words badly, its voice can be swapped. I tried it as a demo, not a measurement; the voice and the mouth in this result are not Kandinsky’s:

1. I chained six five-second clips: each one starts from the previous one’s last frame, with the same character described in the prompt. I picked the first segment by eye among three seeds. At about three minutes per segment, the character stayed recognizable in this test, which does not guarantee it always will.
2. I removed Kandinsky’s audio and laid a revised narration over it, spoken by the studio’s synthetic voice.
3. I ran the result through LatentSync 1.6, from ByteDance, which redraws the mouth to match the new voice. It took 578 seconds on an RTX 3090 for the thirty seconds. I downloaded the model from the 1.6 repository without noting the exact revision.

Laying the new voice without redrawing the mouth did not work: the lips kept saying something else and it showed immediately. The reel at the top is made with this full chain. That the result is usable is my judgment of this piece, not a measurement.

What this measurement does not cover

• Two different machines. The first round is from a 16 GB RTX 4080; the second, from an RTX 3090 with capped memory. The cap only controls what PyTorch allocates, not the CUDA context, so a real 16 GB card has somewhat less room.
• Small samples. Sixteen clips per model in Spanish; in English, sixteen for Lite distilled and eight for each second-round model. The comparisons are descriptive, without confidence intervals.
• Spanish prompts for both languages. Each scene’s description is in Spanish in the English control too: it compares this template, not prompts optimized for each language.
• No lip evaluation. The sheet promised an eye check of the mouth movement clip by clip, and I did not do it systematically.
• Incomplete traceability. Each record keeps times and memory, but not the effective memory cap, the GPU or the time of the run; the order of the amendments is shown by the git history, not by the records.
• Three different measures. The clock excludes loading, text encoding and export. PyTorch’s memory peak is reset before generating each clip. The RAM peak is the process’s historical maximum, loading included.
• The word error penalizes numbers written as digits and variants like “vídeo”. It does not hit every clip equally; the sensitivity analysis above shows it changes no conclusion.
• I did not test Pro undistilled, nor the super-resolution model, nor the world model.
• Versions. The first round used diffusers from the main branch of October 12 without pinning the revision; the second, revision 916a5791.

When I would use it

I would use it to generate video with ambient sound on a 16 GB card, with the distilled Lite model and the two-phase recipe: about 2 minutes 21 seconds per vertical clip on the 4080 is reasonable. And for clips with Spanish dialogue, as a picture base, swapping the voice and redrawing the mouth.

I would not use it to have the small model speak Spanish on its own: in my bench only 1 of 16 clips came in under the threshold. The big model was much more faithful to the text, but a process that reached 120 GiB of RAM and roughly 12 to 23 minutes per clip exceeds what I have at home.

---

Measured between October 12 and 14, 2026. First round on a 16 GB RTX 4080 rented on vast.ai (torch 2.8.0+cu128, diffusers 0.41.0.dev0, transformers 5.19.0). Second round on a 24 GB RTX 3090, with the 15.57 GiB PyTorch cap set by the protocol and not recorded per run (diffusers at revision 916a5791). Speech scored with faster-whisper large-v3 and UTMOS22 strong. Lip sync with LatentSync 1.6 on another rented RTX 3090.

Sources

• Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation, arXiv:2610.05608. The family’s paper, with the architecture and the sizes of Lite and Pro.
• Kandinsky-6.0-Lite-distill-5s-Diffusers. The model card of the checkpoint measured in the first round, with the MIT license and the ten-step settings.
• Kandinsky-6.0-Pro-distill-5s-Diffusers. The 29-billion model of the second round.
• Kandinsky 6 pipeline in the diffusers documentation. The official recipe with per-component offload, which ran out of memory on my 4080.
• LatentSync: Taming Audio-Conditioned Latent Diffusion Models for Lip Sync with SyncNet Supervision, arXiv:2412.09262, and its repository. The model that redraws the mouth.
• UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022, arXiv:2204.02152. The speech-quality predictor, with its limits.
• faster-whisper. The transcriber used for the evaluation, with large-v3.
• The scripts, the measurement sheet and the raw data. The four recipes, the speech evaluation, both rounds as JSONL and the reel’s chain. python3 run_k6_2p.py MODEL prompts.json out.

Frequently asked questions

What is Kandinsky 6.0 Video?

A family of diffusion models from Kandinsky Lab, under the MIT license. According to its authors, it generates five-second clips of video with synchronized 44.1 kHz audio, from text or from text plus an image. It comes in two sizes: Lite, with 3 billion parameters, and Pro, with 29 billion, each with a ten-step distilled version.

Does Kandinsky 6.0 run on a 16 GB card?

The distilled Lite model ran on my 16 GB RTX 4080, but not with the official recipe, which ran out of memory; the Qwen2.5-VL text encoder files alone add up to 15.45 GiB. What worked was a two-phase recipe: first the text embeddings are computed by streaming the encoder through the GPU in groups of layers, then the encoder is dropped and the video is generated with a tiled VAE. PyTorch allocated at most 11.24 GiB and each vertical clip took about 141 seconds.

Does it speak Spanish?

In my bench, both Lite variants missed my criterion of faithfulness to the text. Over sixteen clips the median word error according to Whisper was 1.06 with Lite distilled and 0.75 with Lite undistilled; the transcripts swap, add and repeat words. Pro distilled went down to 0.15 and met the rule I set before running. The cost: in my run, Pro's process reached 120 GiB of system RAM and took roughly 12 to 23 minutes per clip.

Is the broken Spanish caused by distillation or by size?

I cannot separate them. Clip by clip against Lite distilled, Lite undistilled improved in 11 of 16 pairs without reaching the threshold, and Pro distilled improved in 14 of 16 and passed it. That is compatible with an advantage of the bigger model, but without an undistilled Pro in the same table I cannot attribute it to size or rule out that distillation matters too.

Can the small model's video be used with another voice?

Yes, as a trick. I chained six five-second clips, each starting from the previous one's last frame, and replaced the audio with a revised narration in a synthetic voice, Spanish in the Spanish reel and English in this one; LatentSync 1.6 redraws the mouth to match the new voice. This article's reel is made that way. It is a demo, not a measurement.

What hardware does it need?

For Lite, a 16 GB card and plenty of system RAM: the process reached 45.9 GiB. Pro I ran on a 24 GB RTX 3090 with the 15.57 GiB PyTorch memory cap set by the protocol, though its value was not recorded per run, with the transformer in system RAM streamed to the GPU block by block; the process reached 120.25 GiB of RAM. That is not a measured minimum, it is what that run used.