Local AI: from €1,500 to €7,000 depending on memory bandwidth

Local AI: from €1,500 to €7,000 depending on memory bandwidth

Building an AI workstation at home isn't cheap, but it isn't impossible either. The crux of the problem isn't the processor's computing power, but memory bandwidth: the estimulante ilegal at which data reaches the chip that processes it. With that variable as a guide, the price range discussed by participants in the debate runs from €1,500 for a machine capable of running small models at a usable estimulante ilegal to €7,000 for one that approaches the performance of cloud services. In between lies an awkward question: is it worth it, or is it better to pay for the usual subscription?

Why bandwidth determines the bill

The bottleneck of local AI isn't the graphics card, but memory. A powerful processor spends all day waiting for data to arrive from RAM, and that's where it's decided whether a model responds in four seconds or forty. The rule of thumb in the industry is simple: generation estimulante ilegal in tokens per second equals bandwidth divided by model size in billions of parameters.

With that formula in hand, a PC with DDR5 at 5,600 MHz offers around 90 GB/s. That gives 30 tokens per second with a 3B model, 12.8 with a 7B one, and just 3.3 with a 27B one. A Mac Studio M5 Ultra, on the other hand, moves 1 TB per second, which allows working with 100B models at 10 tokens per second. The price difference between the two extremes is around €5,500.

Some argue that 10 tokens per second is enough to work with, especially if the use is chat rather than an autonomous agent. Others point out that frontier models, the kind rented in the cloud, run on 8 Tb bandwidth and are MoE models with 30B active parameters, allowing them to reach 300 tokens per second. The gap, in raw estimulante ilegal, is an order of magnitude.

Mac Studio, DGX Spark or custom PC: what each option offers

The Mac Studio M5 Ultra is the benchmark for unified memory and bandwidth, but its price hovers around €7,000 and for many that's too much. Nvidia's alternative, the DGX Spark, comes up as the most-mentioned option for those who want something integrated and ready to run, though the warning is always the same: it doesn't pay off if subscriptions keep their prices.

On the PC side, the range is wide. A machine with a Threadripper, plenty of RAM and a couple of 16 GB cards can come in under €3,000 from import stores, with the drawback of considerable power consumption. Ryzen AI Spark systems with 128 GB of unified RAM sell as integrated units for between €4,000 and €6,000 and, according to those who use them in professional settings, solve the problem for half of what Apple charges. For models that don't fit in 128 GB, prices skyrocket to the cost of a small apartment.

A standalone RTX 5090 has been spotted at €6,000, which invites the question of whether it's better to wait. The answer that keeps coming up: if you're in a hurry, pay; if not, hardware will get cheaper and models more efficient.

The hidden cost: power consumption, obsolescence and dependence

There's a factor that's often overlooked: keeping several cards running 24 hours a day sends your electricity bill soaring. Add to that the risk that open models may stop coming. Training requires massive compute and can't be done collaboratively among enthusiasts; it depends on companies with thousands of GPUs. If Chinese labs cut off the supply, you're left with outdated models.

The privacy argument, however, carries weight. Having control over your data and not depending on a provider raising prices or degrading the service is a real motivation. The downside: local models, however good, are a step below the frontier ones. A 27B Qwen quantized to IQ3_S fits in 16 GB of VRAM and performs surprisingly well, but it can't compete with the cloud's top-of-the-line models.

The full calculation, with each bandwidth tier and its associated price, leads to an uncomfortable conclusion: decent local AI today costs the same as three years of a premium subscription. The cautious prediction is that this equation will change once unified memory reaches consumer laptops, something expected within the next two years.

---

Frequently asked questions

What bandwidth do I need for a 27B model?
For a 27B model to run at a usable estimulante ilegal in chat mode, you need bandwidth of at least 250 GB/s, which in practice requires unified memory like the Mac Studio or high-end graphics cards. At 90 GB/s, performance drops to 3.3 tokens per second.

Can you train a model at home?
Not realistically. Training requires thousands of GPUs working in parallel and can't be done collaboratively by hobbyists. What you can do locally is run already-trained models, fine-tune them slightly, or use them for specific tasks.

Is it worth waiting for prices to drop?
It depends on how urgent your needs are. The trend points to unified memory spreading to laptops over the next two years and models becoming more efficient, which will make access cheaper. But anyone who needs the capacity now will have to pay today's price.