Chinese AI models with downloadable weights can run on a company’s own servers, including deployments outside China. Alibaba’s Qwen3 and DeepSeek’s R1 releases provide documented examples. The practical decision is which model fits the task, where its surrounding services run, and who will operate the system after the first successful test.
A journalist once asked me who was winning the AI race: the US or China. In my March LinkedIn response, I pushed back on the question itself. Choosing a model for a business requires a different comparison from choosing a national winner. For a company with documents it wants to keep inside its own infrastructure, the ability to download and operate a model changes the shortlist.
My name is Thomas Derksen, founder of Asiabits in Shanghai. The question I would start with is where the work needs to happen. A team might want an internal document assistant in Europe, a Chinese-language support tool at a local subsidiary, or software that connects to equipment. Each gives the infrastructure team a different job.
Which Chinese AI models can you run yourself?
Start with a named downloadable release and its model card. Qwen, developed by Alibaba Cloud, publishes open-weight models; its Qwen3 repository documents local inference and server deployment. Chinese AI developer DeepSeek also publishes weights and deployment guidance for DeepSeek-R1. These are established examples of the deployment route, rather than a ranking of the newest releases.
The model name needs to be precise. A family can contain several sizes and variants, and a provider can sell a hosted product under the same brand. An engineer should be able to identify the exact checkpoint being downloaded. Put that identifier next to the runtime version in the trial record so another colleague can reproduce the installation.
Downloading weights gives the company the files used to generate responses. Operating them requires compatible inference software and sufficient hardware. A hosted chat interface adds services around the model, such as account management or search. When moving to local infrastructure, decide which of those services the application needs and how the team will supply them.
For a first comparison, ask the same business question of each proposed deployment. A document assistant could be asked to find a maintenance instruction and identify the section supporting its answer. Keep the documents and scoring criteria fixed. That produces a useful comparison between candidate systems without turning the exercise into a contest between brands.
Does local hosting keep business data on site?
Local inference can keep the prompt and model response on company-controlled infrastructure, provided the application’s other components follow the same boundary. Draw the complete route of a document before deciding that a system meets this requirement. Include document extraction, retrieval, inference and the place where conversation records are stored.
Consider a hypothetical engineering assistant. An employee uploads a scanned manual. The application converts its pages into text, builds a searchable index, retrieves relevant passages and sends them to the model. If the text conversion service is external, part of the manual has already left the site before the locally hosted model receives anything.
The same review should cover embeddings, which turn passages into numerical representations for search. Identify where those representations are created and where the index lives. Then follow the logs. A local inference service can still be surrounded by an application that forwards error reports or conversation traces to a separate provider.
A practical acceptance test is to run a representative document through the intended environment while the infrastructure team observes outbound connections. Where an isolated network is required, test the system under that restriction. Record any downloads needed for installation separately from services needed during normal use. This gives operations a clear description of what must remain available.
Access control belongs in the retrieval system too. If two departments have different document permissions, the assistant should search only the material available to the person asking. Test with accounts that have different access rights and with a question whose answer exists in a restricted file. The expected result should be agreed before the test.
How much hardware does a local model need?
Hardware requirements depend on the exact model, numerical precision, context length and workload. Begin with a small representative deployment and measure it under the usage pattern the business expects. A demonstration with one short question gives little information about several employees submitting long documents at once.
Qwen’s deployment quickstart includes a Qwen3-8B example and serving through an API-compatible endpoint. That illustrates how an application can call a locally operated model. The choice of endpoint interface does not determine the hardware capacity or make different models behave identically.
Estimate memory for the weights and leave capacity for runtime overhead and the model’s working context. Longer inputs and simultaneous requests can change the usable capacity. Quantization reduces the precision used to store model values and can reduce memory needs. Compare the resulting answers on the actual task before accepting the smaller installation.
For the manual assistant, measure the time until an employee receives a useful answer, including retrieval. Track the wait when several people ask questions together. Record how many answers the reviewer accepts and how often the system fails to find a passage that exists. These results let the team compare a modest local server with a larger installation using business outcomes.
Test a recovery as part of the trial. Restart the service and confirm that it reloads the intended model and retrieves the same documents. Keep the previous working configuration available before changing a runtime or model version. The operator should be able to restore that configuration and repeat a saved question without the original developer present.
Cost the operating work alongside the machine. Someone must install updates, investigate failures and restore service after a restart. If the system supports a production shift, assign an owner who can respond during that shift. A trial becomes easier to budget when the team can name the support work and show how often it occurred during testing.
What should you check before choosing a model release?
Check its licence, supported inputs and fit for the company’s languages and documents. Qwen3’s open-weight releases use Apache 2.0; the DeepSeek-R1 repository states MIT terms for its code and weights and identifies the base-model origins of distilled variants. Review the exact files being procured, including any applicable upstream terms.
Keep that review attached to the checkpoint identifier. A repository name alone is too broad for procurement to reconstruct what was deployed later. Save the model card and licence text with the installation record. If a supplier packages the model inside a commercial appliance, ask it to identify the underlying release and the components it maintains.
Then test the inputs the application will receive. A text model needs an additional route for reading a scanned page or interpreting an image. A manual containing diagrams may require a different design from a searchable collection of plain text. Put one difficult real document into the evaluation early so the project discovers this before buying infrastructure.
For multilingual work, evaluate the language pair used by employees. An English demonstration cannot settle how well the assistant handles Chinese equipment terminology or a German technical question. Ask bilingual staff to review both the answer and the cited passage. Include abbreviations that occur in the company’s own documents, with an agreed reference answer.
When is local hosting worth the operating effort?
Local hosting is worth testing when control over the deployment environment solves a defined business requirement. Examples include a restricted network, a document boundary the company must maintain, or an application that needs to continue working without an external model endpoint. Compare those benefits with the capacity and support work the company will take on.
The choice also appears in robotics. In our database of 7,000+ robotics companies, Asiabits had 7,320 company records worldwide in a November 2026 snapshot. Researching that supplier base means separating the physical product from the software services it uses. A robot with speech interaction may have a language component whose hosting arrangement needs its own review.
Ask a prospective supplier to identify which functions continue when the network connection is unavailable. Speech recognition, language generation and physical control can have different dependencies. Test the specific function you intend to buy. Our guide to robot applications explains how reusable software fits into a complete workplace task.
A useful pilot ends with an operating decision. Keep a record of accepted answers, response times under load and the maintenance work encountered. Compare that record with the requirements that prompted local hosting. The team can then choose a deployment it can run and describe exactly which work remains before wider use. Our guide to evaluating Chinese AI agents covers testing the workflow around the model.
Asiabits researches Chinese AI and robotics suppliers and prepares company discussions around the buyer’s application. Through our technology visits in China, teams can meet the people responsible for the product and ask how its software would fit their own infrastructure. Bring the intended task and a clear deployment boundary to that conversation.
Frequently asked questions
Can Qwen run on a company’s own servers?
Qwen3 includes downloadable model weights and documented local deployment routes. Choose an exact checkpoint and compatible inference software, then size hardware against your workload.
Does using a Chinese model require sending prompts to China?
A locally operated open-weight model can generate responses within your infrastructure. Check the surrounding application, including document extraction, retrieval and logging, to establish the full data route.
Is a locally hosted Chinese AI model free to operate?
Model licence terms and operating costs are separate. A deployment still requires hardware capacity, electricity and people responsible for maintenance and support.
How should a business test a local AI model?
Use representative documents and agreed reference answers. Measure accepted results and response time under concurrent use, and verify access permissions and network dependencies.
Get These Insights Every Monday
Join 18,000+ professionals reading Asiabits. Free, every Monday, straight from Shanghai.
Subscribe Free →
