Should you use an open-weight model or a proprietary LLM API?

Key facts

• The Open Source Initiative's Open Source AI Definition 1.0 requires data information and code as well as model parameters, so a release of weights alone does not meet it. (Source: Open Source Initiative: The Open Source AI Definition 1.0)
• As of November 2026, OpenAI's open-weight models gpt-oss-120b and gpt-oss-20b are released under the Apache 2.0 licence, and gpt-oss-120b is described as fitting on a single 80GB GPU. (Source: Hugging Face: openai/gpt-oss-120b)
• Google announced Gemma 4 on 2 April 2026 under a commercially permissive Apache 2.0 licence, in sizes from about 2 billion effective parameters to a 31 billion parameter dense model. (Source: Google: Gemma 4 announcement)
• Meta's Llama 4 models are licensed under the Llama 4 Community License, which requires any company whose products had more than 700 million monthly active users at the model's release to request a separate licence from Meta, and requires Built with Llama attribution. (Source: Hugging Face: meta-llama/Llama-4-Scout-17B-16E-Instruct)
• DeepSeek-R1's code and weights are released under the MIT License, which permits commercial use and distillation, while its distilled variants carry the licences of their Qwen and Llama base models. (Source: Hugging Face: deepseek-ai/DeepSeek-R1)
• As of November 2026, OpenAI states that data sent to its API is not used to train its models unless the customer opts in, and that abuse-monitoring logs are kept for up to 30 days by default. (Source: OpenAI: Data controls in the OpenAI platform)
• vLLM is an open-source library for LLM inference and serving that provides PagedAttention memory management, continuous batching and an OpenAI-compatible API server. (Source: vLLM documentation)
• As of November 2026, Amazon Bedrock lists open-weight models including gpt-oss, Llama, Qwen3, DeepSeek and Gemma 4 alongside proprietary models. (Source: Amazon Bedrock: Models at a glance)

Definitions

Open-weight model

A model whose trained weights are published for download, so you can run, inspect and fine-tune it on your own hardware. Its licence decides what you may do with it, and its training data and code are often not released.

Proprietary LLM API

A model you can use only through its provider's hosted service, usually paying per token. The provider keeps the weights and decides which versions are offered and when each is retired.

Open Source AI Definition

The Open Source Initiative's definition, at version 1.0, of when an AI system counts as open source: it must grant the freedoms to use, study, modify and share it, and release data information and code as well as parameters.

Inference server

Software that loads a model, batches incoming requests on the hardware and serves them through an API. vLLM is an open-source example.

The short answer: start with an API, move for specific reasons

A proprietary API is the right default for most new products. You get strong models on day one, pay only for what you use, and spend engineering time on the product instead of on GPUs.

Move to open weights when a concrete reason applies: data that may not leave your environment, high and steady volume, the need to fine-tune or freeze a model, latency or offline requirements, or a small model that passes your evaluation. Chapter 5 of From Model to Product frames it the same way: APIs reduce infrastructure burden and give quick access to strong models, while local deployment offers more control, stronger privacy and potentially better economics at high scale, at the price of much more operational maturity.

It is rarely all or nothing. Many products send routine, high-volume steps to a small open model and hard cases to a frontier API.

Open-weight vs proprietary: the decision table

Read each row against your product. If most answers land in the API column, stay there; if several land in the open-weight column and your measurements confirm them, plan a move for those workloads.

Cost at volume: when does self-hosting pay?

Self-hosting is cheaper only when the hardware stays busy. An API charges for tokens used; a GPU costs the same at 3am as at noon, production needs at least two for redundancy, and someone has to run them. LLM cost per user shows how to estimate your monthly token volume.

A worked example with stated assumptions, not market prices: assume a self-hosted setup costs $6,000 a month in total, covering two GPU servers and a share of an engineer's time, and the API you would otherwise use costs a blended $2 per million tokens. Break-even is 3 billion tokens a month, about 100 million a day. Below that the API is cheaper; above it, self-hosting can win, provided the open model meets your quality bar and the hardware can carry your peak, not just your average.

Chapter 15 of From Model to Product names a warning sign worth watching after a move: GPU usage that stays high even at low traffic. Batching raises GPU throughput at a small cost in latency, which is what an inference server such as vLLM does continuously.

Data control: what each option really means

With an API, prompts and outputs go to the provider and are handled under its terms, so read them rather than assume. As of November 2026, OpenAI states that API data is not used for training unless you opt in, that abuse-monitoring logs are kept for up to 30 days by default, and that zero data retention is offered subject to its approval.

An open-weight model on your own infrastructure keeps data inside your network, which can settle a security review or a client contract outright. It also makes you responsible for securing the deployment, its logs and the model files.

A middle path is to run models inside a cloud account you already govern, under that cloud's data terms. For the legal side of personal data, see AI regulation for UK startups.

Licences: open weights are not all open source

An open-weight licence can be as permissive as Apache 2.0 or carry conditions that matter to a growing company, so read the licence for the exact model and version you deploy. The Open Source Initiative's Open Source AI Definition requires data information and code as well as weights, so most open-weight releases are not open source in its sense.

Licences can differ even within one family: DeepSeek-R1 is MIT-licensed, but its distilled variants inherit the licences of their Qwen and Llama base models. As of November 2026:

Hosting options for open-weight models

Open weights do not have to mean running your own GPUs. There are four common ways to host them, from least to most operational work:

1. A managed catalogue, paying per token. As of November 2026, Amazon Bedrock lists open-weight models including gpt-oss, Llama, Qwen3, DeepSeek and Gemma 4 beside proprietary ones, so you keep the option to self-host later without running servers now.
2. A dedicated endpoint in your own cloud account, where the model runs on instances you pay for inside your network.
3. Self-managed GPUs running an inference server such as vLLM, whose OpenAI-compatible API means application code written against an API can often switch with a configuration change.
4. Small models on the user's device or at the edge, for offline use or to remove a server call per request; edge vs cloud inference sets out the trade-offs for vision models, and the reasoning carries over.

Keep the choice reversible

Whichever you choose, do not make it permanent. Put every model behind one interface in your backend, keep model names and endpoints in configuration, and keep an evaluation set you can run against any candidate; how to evaluate an LLM application covers building one.

Plan for retirement on both sides. API providers retire models on their own schedule, and fine-tuned API models depend on their base model and on the provider still offering fine-tuning; RAG vs fine-tuning covers the position at OpenAI as of November 2026. With open weights the work moves to you: patching, security updates and moving to newer models are your job.

A second provider, open or proprietary, also gives you a fallback for outages; LLM APIs in production covers routing and failover.

Frequently asked questions

Sources

1. Open Source Initiative: The Open Source AI Definition 1.0
2. Hugging Face: openai/gpt-oss-120b model card
3. Google: Gemma 4 announcement (2 April 2026)
4. Hugging Face: meta-llama/Llama-4-Scout-17B-16E-Instruct model card and licence
5. Hugging Face: deepseek-ai/DeepSeek-R1 model card
6. OpenAI: Data controls in the OpenAI platform
7. vLLM documentation
8. Amazon Bedrock: Models at a glance