“Run powerful AI on your own hardware” is one of the technology industry’s most attractive promises. It can offer greater control over data, predictable access and freedom to customise a model without sending every prompt to a closed cloud service.
It can also conceal a server bill, a complicated licence and a model that technically supports a huge document but cannot use it reliably on the machine available.
Three prominent open-weight families illustrate the choices: Alibaba’s Qwen 3.8, OpenAI’s gpt-oss and Meta’s Llama 4. Their trained weights can be downloaded, but they differ dramatically in architecture, input types, context, memory and legal terms.
Open weight is not the same as open source
A conventional open-source program supplies human-readable code that others can inspect and rebuild. A trained AI model is also shaped by data, filtering, training recipes and enormous computing runs.
“Open weight” means the learned numerical parameters are available. It does not necessarily mean the training dataset is published, the entire training process can be reproduced, or every use is unrestricted. The Open Source Initiative’s AI definition therefore asks for more than downloadable weights.
Qwen 3.8 and gpt-oss use the permissive Apache 2.0 software licence, although gpt-oss also comes with an OpenAI usage policy. Llama 4 uses Meta’s custom community licence, with attribution, naming and very-large-platform provisions. Organisations should review the actual terms rather than treating all downloadable models as interchangeable.
Qwen, gpt-oss and Llama: hardware and context compared
| Model | Architecture | Input | Context | Practical hardware message |
|---|---|---|---|---|
| Qwen3.8-27B | 27B dense | Text, images, video | 262K native; extension to 1M | About 54GB for BF16 weights alone; quantisation can reduce memory |
| gpt-oss-20b | 21B total; 3.6B active per token | Text | 128K | OpenAI says it fits within 16GB memory |
| gpt-oss-120b | 117B total; 5.1B active per token | Text | 128K | OpenAI says it fits within 80GB memory |
| Llama 4 Scout | 109B total; 17B active | Text, images | 256K trained; generalisation demonstrated to 10M | Meta says Int4 weights fit one H100; long context needs much more working memory |
The parameter numbers need translation. Qwen’s “dense” architecture uses all 27 billion language-model parameters for each token. gpt-oss and Llama use a mixture of experts, selecting only part of the model for each token. That reduces computation, but all expert weights still need to be stored in memory.
gpt-oss-20b is the easiest local starting point
OpenAI released gpt-oss-20b and gpt-oss-120b as text-only reasoning models with adjustable low, medium and high reasoning effort. Both support a 128,000-token context window, structured outputs and tool-using workflows.
The smaller model is the clearest option here for a single high-end personal machine. OpenAI says gpt-oss-20b fits within 16GB of memory. That does not guarantee fast performance on every 16GB device: software support, memory bandwidth, prompt length and chosen runtime still matter. But it is a much lower entry point than a 100-billion-parameter model.
gpt-oss-120b is aimed at substantially larger hardware. OpenAI says its compressed weights fit within 80GB memory, a capacity associated with a data-centre accelerator rather than a typical laptop.
There are important limits. These models are text-only and were trained mostly in English. OpenAI’s model card reports that they can hallucinate more than some hosted OpenAI systems on factuality tests. They also require OpenAI’s Harmony response format, which means applications need compatible serving software.
Qwen3.8-27B is the compact multimodal choice
Qwen3.8-27B combines a 27-billion-parameter dense language model with native visual understanding. It can analyse text, images and video, and its reasoning mode can be enabled or disabled. For developers who want one permissively licensed model for both documents and visual material, that is a meaningful advantage over text-only gpt-oss.
Its native context window is 262,144 tokens. Qwen documents a method for extending it to one million, but that figure should not be read as equivalent to native training at one million. The model card warns that statically applying its YaRN context-extension technique can reduce performance on shorter text.
Full-precision memory is another reality check. Twenty-seven billion BF16 parameters require roughly 54GB just for weights, before the key-value cache, vision encoder and runtime overhead. Quantisation—storing each weight with fewer bits—can cut that requirement dramatically and make the model practical on more hardware. The trade-off may be lower quality, changed speed or reduced usable context, depending on the method.
Llama 4 Scout offers huge context with huge caveats
Meta’s Llama 4 Scout is a multimodal mixture-of-experts model with 109 billion total parameters and 17 billion active for each token. It accepts text and images and advertises support extending to a remarkable ten million tokens.
The model card provides essential context: Scout was trained at 256,000 tokens and Meta demonstrated length generalisation to ten million. The latter is not the same as training natively at that length, nor a guarantee of perfect recall and reasoning across an enormous prompt.
Serving very long prompts also requires a large key-value cache, so fitting Int4 model weights on one H100 accelerator does not mean ten-million-token operation fits comfortably on that same card. Latency, memory use and answer quality should be tested at the length an application actually needs.
Scout’s April 2025 release and August 2024 knowledge cutoff also make it older than Qwen3.8. Retrieval can supply newer information, but it does not update the model’s internal training.
Local AI is private only if the whole system is private
Self-hosting can keep prompts and outputs within an organisation, but the model is only one component. Logging services, vector databases, analytics tools, remote management and third-party interfaces can still transmit data elsewhere.
The operator also becomes responsible for access control, security patches, abuse monitoring, backups, capacity planning and incident response. A downloaded model does not arrive with the managed reliability of a mature cloud API.
Costs extend beyond the accelerator. Electricity, cooling, engineering time, idle hardware and redundancy can make self-hosting more expensive for irregular workloads. It becomes more attractive when usage is high and predictable, data must remain within a controlled environment, or customisation and offline access are valuable.
Which model fits which job?
- Choose gpt-oss-20b for the lowest practical hardware threshold, text reasoning and tool use on a powerful personal workstation.
- Choose gpt-oss-120b when text-only reasoning matters and an 80GB-class accelerator is available.
- Choose Qwen3.8-27B when image or video understanding, an Apache 2.0 licence and a relatively compact model matter more than the smallest memory footprint.
- Choose Llama 4 Scout when its ecosystem, supported languages or exceptionally long-context experiments justify server-class infrastructure and Meta’s licence.
Do not choose from a leaderboard alone. Test the exact quantised checkpoint, runtime and hardware on a representative set of prompts. Record factual accuracy, latency, memory, energy use and the number of human corrections. For many teams, a smaller model that runs consistently is more useful than a larger model that is rarely affordable to serve.
Reporting note
Hardware figures are not performance guarantees. Memory use changes with precision, quantisation, runtime, batch size and context length. Qwen’s one-million-token and Llama Scout’s ten-million-token figures rely on extension or length-generalisation methods rather than native training at those maximums. Benchmark results reported by model developers should be treated as provider-reported and tested independently.
If hosted models are also an option, compare GPT, Claude, Gemini and Grok API models across tools, speed and cost.
Sources and further reading
- Qwen: Qwen3.8-27B official model card
- OpenAI: Introducing gpt-oss
- OpenAI: gpt-oss model card
- OpenAI: gpt-oss-20b model card
- Meta: Llama 4 Scout official model card
- Meta: Llama 4 Community License
- Open Source Initiative: Open Source AI Definition
FutureTechDose covers biotechnology, AI, data-centre and energy-sector research and industry progress for a general audience. This article is informational and does not provide medical or investment advice.
Leave a Reply