Free Chinese AI to run locally: it demands 700 GB of VRAM and pricey hardware
A Chinese AI model can be downloaded for free and run without internet, but it requires 700 GB of VRAM and up to 500,000 dollars in hardware, according to user calculations.
English · Original discussion in Spanish · Published
A free Chinese AI that needs 700 GB of memory to run
Can you compete with GPT-5.1 Pro and Gemini-3 Pro without paying a cent and without an internet connection? The answer going around is yes, as long as you have a server with eight H100 GPUs, nearly 700 GB of VRAM and a budget of around 50,000 euros. The model is downloaded, runs locally and requires neither a subscription nor the cloud. The enthusiasm for technological sovereignty collides head-on with the hardware bill.
The discussion starts with an open model released by a Chinese lab and the thesis that its free release aims to push down the stock market valuations of US tech companies. The file is downloaded from public repositories and runs with independent software, like any database. No telemetry, no monthly fee, and no service cutoff when the provider decides to change the terms.
What it takes to run the Chinese AI locally
The technical requirement is the great filter. With 678 billion parameters and 8-bit quantization, the model consumes close to 700 GB of GPU memory. In FP16 or FP32 the figure skyrockets and forces you to use Amazon's p5.48xlarge instances: 192 vCPUs, 2 TiB of RAM, eight H100s with 640 GB of HBM3 and 3,200 Gbps of EFA networking. The hourly rental of that setup turns the experiment into an expensive exercise.
Some scale down the estimate and argue that for a mid-sized company 50,000 euros is nothing and pays for itself quickly. The objection comes from the other side: if you want reasonably comparable requirements, the realistic range moves between 250,000 and 500,000 dollars in hardware alone, plus the associated labor. The gap between the two estimates is not small: it separates a department's whim from a strategic investment.
The real cost of building an AI service for thousands of users
The experience of those who have tried it at a company provides the data that is usually missing. Deploying an AI service via API for thousands of simultaneous users was studied, and the cost was a hundred times the initial estimate. It wasn't about moving a model, it was about sizing capacity to serve many people at once. The project was scrapped immediately. That itemized breakdown is the material that never shows up in press releases.
The version that runs locally is not the same as the one you try on the web. A distilled model with 70 billion parameters is not equivalent to the raw model. That's why performance comparisons depend on which variant was downloaded and with what quantization. The question many ask when installing the file is how it updates without a connection: the answer is manual, copying the new file to the appropriate folder.
The trade war behind a free download
The move has a geopolitical reading. China has reportedly banned its companies from buying Nvidia chips, a blow to the manufacturer, which had more than 15% of its market share in the country. The strategy of giving away open models is interpreted as a way to provoke US tech companies and strain the supply chain. The winner in all this, the argument goes, is Nvidia: everyone pays for GPUs, from labs to internet giants.
The other side is privacy. A model that runs offline sends no data to any server, but the doubt remains about what happens if the machine connects even for a second. The accusation that the model leaks information to the Chinese government circulates without evidence. What is documented is that the runtime software is independent of the model file and that execution can be 100% offline.
Real performance versus the promise of matching GPT-5.1
The comparison with paid models is divisive. Some have tested it for weeks and rate it excellent. Others argue it doesn't come close to GPT-5.1 and that those claiming otherwise haven't used the right version. One user reports that when asked about its own version, the model replies that it has no version name, fueling suspicions about what is actually running.
The underlying problem is that the model tends to make things up that it didn't learn in training, a recognized handicap. For chat it works well; for programming or for integrating it into a robot that performs work tasks, the consensus is that it's not there yet. The version available on the web is 3.2, and the comparison with the US alternatives depends on whether you use the distilled variant or the full model.
The download is free and the hardware is not. With these numbers, mass adoption should be immediate. It remains a thing for the few.
Can you compete with GPT-5.1 Pro and Gemini-3 Pro without paying a cent and without an internet connection? The answer going around is yes, as long as you have a server with eight H100 GPUs, nearly 700 GB of VRAM and a budget of around 50,000 euros. The model is downloaded, runs locally and requires neither a subscription nor the cloud. The enthusiasm for technological sovereignty collides head-on with the hardware bill.
The discussion starts with an open model released by a Chinese lab and the thesis that its free release aims to push down the stock market valuations of US tech companies. The file is downloaded from public repositories and runs with independent software, like any database. No telemetry, no monthly fee, and no service cutoff when the provider decides to change the terms.
What it takes to run the Chinese AI locally
The technical requirement is the great filter. With 678 billion parameters and 8-bit quantization, the model consumes close to 700 GB of GPU memory. In FP16 or FP32 the figure skyrockets and forces you to use Amazon's p5.48xlarge instances: 192 vCPUs, 2 TiB of RAM, eight H100s with 640 GB of HBM3 and 3,200 Gbps of EFA networking. The hourly rental of that setup turns the experiment into an expensive exercise.
Some scale down the estimate and argue that for a mid-sized company 50,000 euros is nothing and pays for itself quickly. The objection comes from the other side: if you want reasonably comparable requirements, the realistic range moves between 250,000 and 500,000 dollars in hardware alone, plus the associated labor. The gap between the two estimates is not small: it separates a department's whim from a strategic investment.
The real cost of building an AI service for thousands of users
The experience of those who have tried it at a company provides the data that is usually missing. Deploying an AI service via API for thousands of simultaneous users was studied, and the cost was a hundred times the initial estimate. It wasn't about moving a model, it was about sizing capacity to serve many people at once. The project was scrapped immediately. That itemized breakdown is the material that never shows up in press releases.
The version that runs locally is not the same as the one you try on the web. A distilled model with 70 billion parameters is not equivalent to the raw model. That's why performance comparisons depend on which variant was downloaded and with what quantization. The question many ask when installing the file is how it updates without a connection: the answer is manual, copying the new file to the appropriate folder.
The trade war behind a free download
The move has a geopolitical reading. China has reportedly banned its companies from buying Nvidia chips, a blow to the manufacturer, which had more than 15% of its market share in the country. The strategy of giving away open models is interpreted as a way to provoke US tech companies and strain the supply chain. The winner in all this, the argument goes, is Nvidia: everyone pays for GPUs, from labs to internet giants.
The other side is privacy. A model that runs offline sends no data to any server, but the doubt remains about what happens if the machine connects even for a second. The accusation that the model leaks information to the Chinese government circulates without evidence. What is documented is that the runtime software is independent of the model file and that execution can be 100% offline.
Real performance versus the promise of matching GPT-5.1
The comparison with paid models is divisive. Some have tested it for weeks and rate it excellent. Others argue it doesn't come close to GPT-5.1 and that those claiming otherwise haven't used the right version. One user reports that when asked about its own version, the model replies that it has no version name, fueling suspicions about what is actually running.
The underlying problem is that the model tends to make things up that it didn't learn in training, a recognized handicap. For chat it works well; for programming or for integrating it into a robot that performs work tasks, the consensus is that it's not there yet. The version available on the web is 3.2, and the comparison with the US alternatives depends on whether you use the distilled variant or the full model.
The download is free and the hardware is not. With these numbers, mass adoption should be immediate. It remains a thing for the few.
Also available in: Deutsch
Summary of a discussion on Burbuja.info - Foro de economía, actualidad y política., translated from Spanish and reviewed before publication. Read the full discussion (99 replies).
More summaries
-
Jacob Coxon: AI is gambling with our lives without accountability
Jacob Coxon quit OpenAI and Anthropic: "gambling with our lives". Anthropic hid its latest model from the UK regulator. AI apocalypse?
-
China alerts Spain to alleged plot against the country
An academic article by Nadia Helmy reveals the thesis of a plan against Spain. Ceuta, Melilla and the Strait are at the center of the alert.
-
Jacob Coxon quits Anthropic: AI could kill us all
Jacob Coxon resigns from Anthropic: 'AI could kill us all' by the end of this decade. A colleague at the lab puts the risk above 10%.
-
Ketamine: the drug slipping through Spain's DGT drug tests
In the Balearic Islands, saliva tests fail to detect ketamine, and professional drivers use it to evade police checks. The system's loophole...
-
Judge orders deportation of 85 Senegalese, including girl born in Spain
Judge Antonio Mazuecos authorises the detention of 85 Senegalese nationals, including a minor born in Spain in 2009. Data show only 6-8% of deportation orders are actually carried out.
-
Senegal refuses to return Africa Cup despite CAF ruling
Senegal is defying CAF and keeping the Africa Cup after a controversial decision awarded the title to Morocco behind closed doors; the case is expected to reach CAS.