Evaluate Chinese AI agents by testing a complete business task with the documents, tools and exceptions your team actually handles. Compare successful outcomes, human review time and recovery from errors. Qwen and DeepSeek provide tool-use capabilities; the useful buying decision is whether a particular implementation can finish your workflow reliably.
A journalist once asked me who was winning the AI race, the US or China. In my March 2026 LinkedIn post about that conversation, I questioned what winning meant. That question becomes more useful inside a company: which system helps a team complete the work sitting in front of it?
I'm Thomas Derksen, founder of Asiabits in Shanghai. A model ranking can help you identify candidates. Choosing an agent requires another conversation, about the information it receives, the systems it can use and the result a colleague must accept. Those details give a business team something it can test.
What should count as a Chinese AI agent?
For a business evaluation, treat an agent as a system that uses a model and tools to carry out a task across several steps. Specify the product, model and hosting arrangement being proposed. A Qwen-based application built by an integrator and a hosted service using DeepSeek can have very different capabilities even when their demonstrations look similar.
Alibaba's Qwen-Agent framework provides components for models, tools and agents, with examples including document reading and custom tools. Its documentation describes using an Alibaba Cloud model service or a self-deployed model service. This gives a technical team several implementation paths to investigate.
Chinese AI developer DeepSeek's tool-call guide shows a model requesting a function and the application returning the result. The application supplies and executes the function. A model's ability to request an action therefore leaves practical work for the team building the application: connecting systems and deciding which actions are available.
Ask a vendor to draw that connection for your proposed workflow. Where does the model run? Which software reads the files? Which account accesses the business system? Who maintains the connector when that system changes? Clear answers help your IT team estimate the work behind the demonstration.
Which business workflow makes a useful first pilot?
A useful first pilot has a defined input, an observable result and enough examples for colleagues to judge the output consistently. Consider a hypothetical supplier-document workflow: read a specification, extract required fields, compare submitted documents and prepare a review summary. Keep the final supplier decision with the responsible team during the pilot.
Write down what a correct result contains before selecting a model. For this example, each extracted value needs its unit and a reference to the source document. A missing certificate should appear as missing. Where two files disagree, the summary should show the disagreement so the reviewer can resolve it.
Build the test set from cases your team understands and is authorised to use. Include clean documents and the awkward files that consume time: scanned pages, inconsistent terminology, incomplete submissions or revised specifications. Remove confidential information that the test does not require. Give every candidate the same task and source set.
Keep some cases separate from development. The vendor can use one set to configure the workflow, then run the agreed evaluation on the held-back cases. This helps your team see whether the implementation can handle fresh material. Record any prompt changes or manual corrections made during that final run.
Choose a process owner who can judge the result without asking an AI specialist to interpret it. The owner should be able to identify a wrong unit, an unsupported conclusion or an omitted file. That makes the pilot useful to the department that will eventually use the system.
How do you measure whether the agent finishes the job?
Measure accepted, complete cases and the human effort needed to achieve them. A fluent summary is only one part of a document workflow. Your reviewer must also be able to verify the fields and find the cited source. Decide which errors make a case fail before looking at the results.
For every test case, keep the original input, the agent's output and the review result together. Record corrections and the minutes spent checking them. Separate a case completed without help from one that required a colleague to locate a missing document, fix an extraction or restart the process.
Also measure elapsed time across the whole workflow. A quick answer can still require a long manual review. Compare the pilot with the current process using the same definition of completion. That gives the department a usable basis for deciding whether the agent saves effort on its actual workload.
Test interruptions deliberately. For example, make a required document unavailable or have a test connector return an error. Watch whether the agent reports the problem, retries appropriately or produces a misleading completion message. Then check whether it can resume from the correct point without duplicating an earlier action.
In a hypothetical workflow that writes to a business system, inspect the destination record after execution. A chat message saying the update succeeded is insufficient. Use the system's own record identifier and read back the saved values. The same principle applies whether the underlying model comes from China or elsewhere.
What should you ask about tools and deployment?
Ask which tools the agent needs and which permissions each tool receives. Reading a product catalogue and changing an order require different access. A pilot can begin with reading and draft preparation, then add a specific write action once the team has verified the result and assigned responsibility.
Use a concrete example in the vendor meeting. If the agent must prepare an internal purchasing request, have the supplier show the fields it sends and the response it receives. Ask how the application handles an unknown supplier code or a duplicate request. Your operations colleagues can then assess behaviour they recognise.
Deployment also affects the work needed to run the system. With a hosted service, establish who operates the model and application. With self-hosting, establish who provides the computing capacity, deploys updates and monitors failures. Have the technical team document the actual arrangement offered for the pilot.
List the documents the workflow reads and where copies or logs are stored. Agree on retention and access with the people responsible for those records. This is especially useful when a process involves several service providers: the department needs to know whom to contact when a file must be removed or an access right changed.
Request an operating estimate based on the tested workload. Include model usage, hosting, connector maintenance and review time. Avoid choosing on a demonstration's response speed alone. The department will operate the full process, including the exceptions that need a person's attention.
How does this connect to China's robotics industry?
The connection is the interface between a model and the work it can perform. In software, that interface may be a document reader or a business-system connector. In robotics, the system also needs perception and physical control. A company's AI story becomes easier to evaluate when it identifies the action and demonstrates the complete sequence.
In our database of 7,000+ robotics companies, the 4 November 2026 research snapshot contains 7,320 company records across countries and robotics categories. That breadth is useful when planning research beyond a single model provider. The next step is to identify which company supplies the application, the hardware and the integration your task requires.
Keep the tests specific to the system. A software agent can be evaluated on whether it extracts a field and saves the right value. A robot handling an object needs a physical task test with the relevant objects and surroundings. Success in one setting cannot establish success in the other.
Our guide to embodied AI and China's robot builders explains the physical side. For the distribution of Chinese models and developer tools, see why China is open-sourcing AI. Use those perspectives to decide which specialists belong in your supplier conversations.
What should the pilot help your team decide?
The pilot should establish whether the proposed system merits a defined next stage and what work that stage requires. Close the evaluation with the process owner and technical team together. Review the failed cases alongside the accepted ones, then identify the changes needed before expanding access or workload.
Prepare a short decision record containing the tested task, input set, accepted outcomes, review effort and unresolved integration work. Name the person responsible for each remaining action. This makes the result usable when someone who missed the demonstration asks why the team recommends continuing.
If the system performs well on a narrow process, expand through a related test that adds one meaningful complication. That might be another document format or a connector to a second system. Preserve the earlier test cases so the team can see whether an update improves the workflow it already accepted.
Asiabits helps international teams investigate China's technology companies and prepare focused discussions with relevant suppliers. Our robotics directory supports initial company research, and Bespoke Tours connect a team's questions with demonstrations and technical conversations in China. Bring a defined application so the visit can inform a concrete business decision.
Frequently asked questions
How should a business compare Chinese AI agents?
Give candidates the same task and test cases. Compare complete accepted results, human review effort and recovery from interruptions.
Does tool calling mean an AI model executes business actions?
In DeepSeek's documented tool-call flow, the application executes the requested function and returns its result. Evaluate the connectors and permissions in the complete application.
Can Qwen-Agent use a self-deployed model service?
The Qwen-Agent documentation describes both Alibaba Cloud and self-deployed model-service options. Confirm the actual model, infrastructure and support arrangement proposed for your project.
What is a practical first AI-agent pilot?
Choose a defined workflow with inputs and results that a process owner can verify. Document comparison and draft preparation can provide a bounded starting point before adding write access to business systems.
Get These Insights Every Monday
Join 18,000+ professionals reading Asiabits. Free, every Monday, straight from Shanghai.
Subscribe Free →
