I ran AIFS 2.0, ECMWF's weather model, on my desktop card and landed 0.45 °C from the official forecast

ECMWF published the weights of AIFS, its data-driven forecast model. Not a demo or an API with a quota: the full checkpoint, licensed CC-BY-4.0, on Hugging Face.

What made me drop everything and try it is that a forecast is one of the very few things I can publish knowing that I am not the one who decides whether it was right. A model benchmark is designed, run and graded by me. A weather prediction is graded by Thursday.

So I installed it, ran it, and saved the forecast with a date and time before knowing what was going to happen.

What AIFS is and why it mattering that it fits

AIFS is a sliding-window graph transformer, trained on ERA5 and ECMWF’s operational analyses. The aifs-single-2.0 version is deterministic and produces a global state every six hours.

The number that changes everything: the checkpoint weighs 994 MB. The physical model it competes with, the IFS, runs on one of the largest supercomputers in Europe. This one fits on a gaming card.

The real barrier is not size. It is that AIFS was trained calling flash_attn_func, from the flash-attn package, which only compiles on Ampere cards or later. My 4070 Ti SUPER does fall in that range; the wall is for everything older, and for anyone without CUDA.

Step 1: the environment

Python between 3.11 and 3.13, and a virtual environment of its own. No installing this on top of an environment that already works.

mkdir -p ~/aifs && cd ~/aifs
python3.12 -m venv venv
git clone https://github.com/huggingface/AIFS-single-2.0-on-all-GPUs
cd AIFS-single-2.0-on-all-GPUs
~/aifs/venv/bin/pip install -r requirements.txt

That repository is not ECMWF’s and says so on the first line. It is a wrapper that intercepts the flash_attn import and redirects it to scaled_dot_product_attention, the fused attention PyTorch has shipped since version 2.0 and which works on CUDA, on Metal and on CPU.

There is no need to install flash-attn. That is the whole trick.

Step 2: the stumble that nearly left me running on CPU without noticing

Once the install finished, this:

torch 2.13.0+cu130 | cuda available: False
UserWarning: CUDA initialization: The NVIDIA driver on your system is too old
(found version 12080)

The requirements.txt asks for torch>=2.1, and pip resolves that to the latest version, which comes compiled against CUDA 13. My machine’s driver is 570.144, meaning CUDA 12.8. PyTorch does not fail: it turns CUDA off and carries on. If I do not read that line, the whole forecast runs on the processor while I believe I am using the card.

It is fixed by explicitly asking for the variant that matches the driver:

~/aifs/venv/bin/pip install --index-url https://download.pytorch.org/whl/cu128 "torch==2.7.*"

torch 2.7.1+cu128 | cuda: True
gpu: NVIDIA GeForce RTX 4070 Ti SUPER

It is always worth checking that line before measuring anything. A torch.cuda.is_available() that silently returns False is the cheapest way to publish wrong numbers.

Step 3: the initial conditions

The model does not guess from nothing: it needs the real state of the atmosphere. Specifically two analyses six hours apart, the one at the initial moment and the one from six hours earlier, so it can infer which way everything is moving.

They come from ECMWF’s open data portal, free and without an account:

from aifs import load_ics
fields, date = load_ics(cache_dir="ic_cache")

The first time it took 255 seconds and left 660 MB in cache. Later ones load in seconds from the local .npz.

Step 4: the forecast

from aifs import run_forecast
states = run_forecast(fields, date, lead_time=48, num_chunks=16)

num_chunks splits the graph so memory does not blow up; raising it lowers consumption in exchange for some speed. Each element of states is a global state, one every six hours.

Step 5: pulling out one city’s data

Here is a subtlety worth not skipping past. AIFS does not hand back a rectangular latitude-longitude grid: it hands back a reduced Gaussian grid, a vector of points with their coordinates. For a city you look for the nearest point, and that is what you get: it is not an interpolation.

import numpy as np

def nearest_point(lats, lons, lat, lon):
    lon = lon % 360
    glon, glat = np.asarray(lons) % 360, np.asarray(lats)
    dlon = np.minimum(np.abs(glon - lon), 360 - np.abs(glon - lon))
    return int(np.argmin(np.hypot(glat - lat, dlon * np.cos(np.radians(lat)))))

idx = nearest_point(states[0]["latitudes"], states[0]["longitudes"], -33.4489, -70.6693)
for st in states:
    print(st["date"], round(float(np.asarray(st["fields"]["2t"])[idx]) - 273.15, 2), "°C")

For Santiago, the nearest point landed at −33.583, −70.720: about fifteen kilometres from the centre. Temperature comes in kelvin, hence the 273.15.

Measurement 1: what it costs to run

Initial analysis of 16 August 2026 at 00:00 UTC, 48-hour forecast, 16 GB RTX 4070 Ti SUPER.

1. Download the initial conditions254.6 sMore than twice the full forecast. The bottleneck is the network, not the card.
2. Forecast 48 hours108.5 sEight six-hour steps, with 5.77 GB of video memory at peak.
3. One 6-hour step13.6 sThe model's real unit of work. Everything else is multiplying it.

RTX 4070 Ti SUPER, with PyTorch's attention in place of flash-attn. The download is paid once.

It fits comfortably on an 8 GB card. The initial download weighs more than the model.

Measurement 2: what if you have no card?

I repeated exactly the same run forcing the processor, with CUDA_VISIBLE_DEVICES="", on a 16-thread Ryzen.

18.3 times slower. But it finishes, and without touching a line of code.

The interesting part is comparing the two results point by point:

The first five steps give the same temperature down to the hundredth of a degree. In the wind, which does not appear in this table, the divergence shows earlier: by the third step there is already a hundredth of a metre per second of difference. The difference appears at the sixth step and grows toward the end. That is exactly what you expect from an autoregressive model: each step takes the previous one’s output as input, so a minimal discrepancy in the order of floating-point reductions gets dragged along and amplified.

For this horizon the difference is irrelevant, four hundredths of a degree. At ten days, that same mechanism is what separates two forecasts that started identical.

Measurement 3: against the official forecast

This is the one that matters. I compare my home run against ECMWF’s own operational IFS, the physical model running on its supercomputer, queried through Open-Meteo’s API. Same point, same hour, two-metre temperature.

Mean absolute error: 0.45 °C.

Less than half a degree of difference against the world’s reference operational model, at two days, from a card that also plays games. And that number carries the penalty of the attention patch: the wrapper warns that the SDPA route is not bit-for-bit identical to the kernels the model was trained with. With the original implementation, the difference should be smaller.

The differences fall on both sides of zero, five negative and three positive, so there is no warm or cold bias worth naming with eight data points. The largest, 1.34 °C, lands at midday on the second day.

The life of a number: from 0.91 to 0.45

No number is born good. This one went through three states before settling, and the journey teaches more than the result.

State 1 · 0.91 °C — the naive number. I ran the model, asked an API for the official forecast, subtracted and published. It looked reasonable: under a degree against the best meteorological centre in the world. What I did not do was ask myself what exactly that API was giving me back.

State 2 · 0.61 °C — the ground appears. Reviewing the call I found that Open-Meteo, by default, does not return the raw grid point: it picks a land cell of similar elevation and on top of that adjusts for altitude with a 90-metre digital elevation model. I asked for both versions of the same instant:

0.60 °C of difference, identical across all eight hours. I was subtracting AIFS’s raw grid point from an altitude-corrected IFS. Two different things under the same name, and a city in a valley ringed by mountains is exactly where it shows most.

State 3 · 0.45 °C — the clock appears. With the point fixed, the numbers still did not add up: the difference between my first query and the new one was not the constant 0.60 I had just measured, but a spread from +0.3 to −1.9 °C. That has only one explanation. The IFS updates four times a day, and my original query had pulled a different run from the one I was comparing against later.

Which means that throughout state 1 I was pitting AIFS’s forecast, launched from the 00 UTC analysis, against an IFS that could come from a later analysis, with six or twelve more hours of information about the atmosphere. It was not that my model came off badly: it was that the official one was playing with an advantage.

State 4 · what is still open. The clock is not fully closed: for the comparison to be strict, the IFS run has to be pinned to the same initial analysis AIFS uses. While the scoreboard queries the current official forecast, that part stays approximate, and I say so here rather than waiting for someone to notice.

None of the three states was a model error. All three were mine, in how I requested the data I was judging it with. Running the model is the easy part; what costs is making sure the two numbers you subtract measure the same thing, in the same place, at the same moment.

What it does not do

Two things that are easy to confuse are worth separating. AIFS Single v2 is an operational ECMWF model: its own card says they run it four times a day to generate a global fifteen-day forecast. What falls outside that category is this execution, with a wrapper that swaps the attention mechanism for another and warns on its first line not to use it for anything with safety at stake.

Put another way: the model is the same one ECMWF operates professionally; the implementation I run it with at home is not.

It is deterministic, so it gives a single future rather than a distribution of possibilities; for that there is aifs-ens-2.0, which is an ensemble. And it inherits the smoothing bias typical of these models: it tends to underestimate extremes, which is exactly what matters most when weather genuinely matters.

For watching the atmosphere’s general evolution over a few days, with open data and without depending on anyone, it works remarkably well.

The code

Everything above, packaged and with a README: github.com/EfrainGaray/marcador-clima. It is two scripts, pronostico.py to run the model over any coordinate and marcador.py to keep the log. The live scoreboard, updating itself, is at /en/weather.

The judge arrives on Tuesday

The forecast was saved as JSON with the exact time of the initial analysis, before anything happened. On 18 August it can be checked against what actually occurred, and that comparison is going to update this post whatever it says.

And it does not stop at this forecast: every day a new one gets saved, with the official one beside it, and reality gets filled in when it arrives. That accumulated log is open at /en/weather. The first verified hours already give a clear advantage to the official model, which is exactly what you would expect from a system running on a supercomputer; the interesting part will be seeing how much, and whether it holds, once there are months of data instead of hours.

That is the part of all this that interests me. I can choose the benchmark, I can choose the hardware and I can choose how to present the numbers, but I cannot choose the weather.