The Widening Gap in AI Safety as Chinese Open-Weight Model Narrows Capability Divide

As policymakers worldwide intensify debates on how to effectively govern increasingly powerful artificial intelligence systems, such as OpenAI’s advanced GPT-5.6 Sol and Anthropic’s sophisticated Mythos, a significant development from China has underscored the urgency of these discussions. A new open-weight model from China has demonstrated a remarkable capability, rapidly narrowing the technological gap with the established industry leaders.

GLM-5.2, an open-weight AI model developed by China’s Z.ai, has been found to be only a few months behind the cutting-edge capabilities of models like OpenAI’s GPT-5.5 and Anthropic’s Claude Opus 4.7 in critical domains like cyber and bio capabilities. This assessment comes from a recent report published by SaferAI, a non-profit organization dedicated to AI safety research. While the rapid advancement of open-weight models is noteworthy, the report simultaneously highlights a troubling trend: the growing disparity between the rapid progression of frontier AI capabilities and the comparatively slower, less robust development of effective safety practices. This divergence poses profound questions for global AI governance and risk management.

SaferAI’s evaluation, conducted through Z.ai’s publicly available API, revealed particularly stark findings regarding GLM-5.2’s safety protocols. The model notably refused none of the offensive cyber or dual-use biology tasks it was presented with. This indicates a critical lack of inherent safety guardrails within the model’s architecture to prevent its misuse for harmful purposes. In stark contrast, during the same evaluation, Anthropic’s Claude Opus 4.7 exhibited a consistent refusal to engage with such tasks, demonstrating such robust safety mechanisms that SaferAI noted it “refused so consistently that SaferAI could not complete CyberGym on it at all.” CyberGym is a specialized benchmark designed to rigorously evaluate cybersecurity capabilities, a tool that OpenAI itself utilized in its evaluations preceding last month’s widely reported Hugging Face breach. The difference in refusal rates between the two models vividly illustrates the varied approaches to safety implementation across different AI developers, particularly between open-weight and closed-source systems.

This development serves as a stark reminder of concerns that critics have voiced for years: the potential for highly capable, open-weight AI models to fall into the hands of malicious actors. Once the model weights are downloaded, there is currently no effective mechanism to police or control how these technologies are used. With open-weight models now rapidly approaching the performance levels of the world’s most advanced proprietary AI systems, the nature of the debate is fundamentally shifting. It is no longer a question of whether these models can compete technologically, but rather how society can effectively manage the inherent and escalating risks once these powerful tools are widely released and easily accessible.

Henry Papadatos, the executive director of SaferAI, articulated this critical distinction to TechCrunch, stating, “The frontier of capability is not the frontier of risk, and so we do have to take into account the state of the mitigations as well to assess the risk properly.” This emphasizes that raw power must be evaluated alongside the measures in place to prevent harm. While Z.ai, like other developers, can implement safety measures and content filters on its hosted API, these protections become entirely unenforceable once the raw model weights are acquired and run on a user’s private hardware. In such a scenario, users gain complete autonomy to remove or modify any pre-existing safeguards, fine-tune the models for specific, potentially malicious, objectives, or alter system prompts to circumvent intended ethical boundaries. This inherent characteristic of open-weight models creates a significant governance challenge, effectively creating an unpoliced tool with high capabilities.

Frontier AI developers, such as OpenAI and Anthropic, typically rely on a layered approach to safeguards for their closed models. These often include the deployment of sophisticated classifiers to detect and filter harmful content, extensive refusal training to condition models against generating dangerous outputs, and API-level controls that restrict access to certain functionalities or types of queries. However, even these measures, while robust, are far from foolproof. The AI safety non-profit Far.ai, for instance, through its comprehensive leaderboard, has documented hundreds of “universal jailbreaks.” These are defined as reusable keys or techniques that consistently succeed in bypassing protections on a majority of harmful requests across various frontier models, including xAI’s Grok 4.5 and Google DeepMind’s Gemini 3.1 Pro. The report detailed that these jailbreaks often succeed when attackers skillfully combine multiple manipulation techniques, such as roleplaying, authority impersonation, crafting fake conversation histories, and employing follow-up prompts to exploit and amplify weak points within a model’s defensive architecture. Crucially, these sophisticated safeguards, which are integral to the risk management strategies of closed models, are rendered completely ineffective on open-weight models, which are fundamentally designed to operate on any infrastructure, with or without any accompanying set of safety measures.

Papadatos further emphasized SaferAI’s broader objective: “The objective should clearly be that the good capabilities – the safe ones – are accessible to anyone, and then we try to remove the bad ones, even in an open source fashion.” This highlights a desire for a future where beneficial AI is democratized without simultaneously unleashing dangerous capabilities.

One technique Papadatos identified as potentially helpful in achieving this goal is "pre-training data filtering." This involves an AI company meticulously removing offensive cybersecurity information and other harmful content from their training datasets before the model is even developed, and then training the AI solely on this curated, safer data. Some research suggests that this method can effectively reduce a model’s hazardous biological knowledge without compromising its overall performance. However, for cybersecurity applications, data filtering presents a much greater practical challenge. It is inherently difficult to train a general-purpose model that excels at coding—a capability highly prized and monetized in the AI industry—without also inadvertently equipping it with the skills of a proficient hacker. The immense commercial pressure on developers to continuously improve coding capabilities often conflicts with efforts to limit potential misuse, making pre-training data filtering a less viable primary mitigation strategy for cyber risks.

Given these difficulties, frontier developers have increasingly turned to other mitigation strategies. One prominent approach involves selectively restricting the types of cybersecurity assistance their models will provide. For example, Anthropic’s Claude Opus 5, as detailed in its system card, is designed to search for vulnerabilities only in uncompiled source code, not in compiled software. The rationale behind this specific restriction is to make it significantly more challenging to use Opus 5 for offensive cyber purposes. Other preventative measures adopted by leading developers include rigorous pre-deployment safety evaluations, transparently publishing comprehensive risk assessments, and, in some cases, even withholding model weights entirely if a system is deemed too dangerous for public release.

In the specific case of GLM-5.2, SaferAI noted that Z.ai had not published a safety framework, pre-deployment testing commitments, or a comprehensive risk assessment for the model prior to its release. TechCrunch reached out to Z.ai for clarification on whether it had conducted any internal or third-party frontier safety evaluations before making the model available, but did not receive a response.

Chinese leaders have increasingly acknowledged the inherent risks associated with advanced AI. At the World AI Conference held last month, Chinese President Xi Jinping underscored the importance of open-weight models, while simultaneously stressing the imperative of ensuring that AI remains a tool under strict human control. Graham Webster, who specializes in Chinese AI policy at the Stanford Cyber Policy Center, offered insights into China’s regulatory landscape. He explained to TechCrunch that while China possesses robust regulations governing AI, these rules have historically focused more on issues of politically sensitive content, misinformation, and maintaining social stability, rather than on mitigating catastrophic AI risks such as offensive cyber capabilities or biological misuse.

Webster further elaborated on the differing perspectives between the East and West. “U.S. AI thinkers are, in general, more concerned with this existential catastrophic [idea] than the Chinese community,” he stated, adding that many Chinese policy researchers tend to believe that if a truly novel frontier risk emerges, American companies are more likely to encounter it first due to their development trajectories and regulatory environments. Webster also highlighted China’s internal control mechanisms: “The Chinese system has confidence that they control the use of these technologies inside China. Being online in China is something you do attributed to your real name, and companies can be held accountable, users can be held accountable.” He mused that the same mechanisms currently employed by model providers to refuse engagement on certain political topics could potentially be adapted to ensure models refuse to complete offensive cyberattacks or generate adverse biological engineering outcomes. However, he cautioned that discerning the extent of internal testing conducted by Chinese companies before release is challenging, given their tendency to coordinate with regulators behind the scenes.

Advocates for open-weight AI contend that the release of model weights is crucial for enhancing cybersecurity. They argue that it empowers companies to better defend themselves against attacks, citing Hugging Face’s reliance on GLM-5.2 to defend against an "OpenAI’s breach." Furthermore, they believe that open-weight models allow the broader security community to better prepare for future threats by understanding the capabilities attackers might wield. Clem Delangue, CEO of Hugging Face, articulated this view in a social media post this week, stating, “The same systems that helped stop an AI-powered cyberattack can now help defend against millions of cyberattacks every day, while helping us identify and fix vulnerabilities before attackers exploit them.”

However, Papadatos countered that this perceived benefit is often overstated and does not warrant “open-sourcing dangerous capabilities.” He stressed his fundamental belief that “the main point in my mind is that we shouldn’t just accept that dangerous capabilities are easily accessible by anyone anywhere.” Papadatos emphasized that the industry’s collective effort should be directed towards making only “the good capabilities” easily accessible. He underscored the inherent asymmetry in the attacker-defender dynamic, noting, "By default attackers adopt new tools faster than defenders do. For example, a ransomware group can change its methods in a week. A hospital cannot." This highlights the urgent need for robust, enforceable safeguards to prevent the widespread dissemination of powerful AI tools that could be exploited for malicious purposes, far outpacing the ability of defensive systems to adapt.