Qwen3-Coder-30B-A3B-Instruct has an AA score of 10. The same as o1 mini, or Gemini 2.5 Flash. AA does not really even score accurately below 10. Qwen 3.8 27B, which has fewer overall parameters, scores 34.
Qwen3 Coder 30b has a hallucination rate of 80%. Qwen 27b has a hallucination rate of 30%.
Why would they choose an outdated, non-thinking model when newer models from the same company have fewer parameters, are open source, and just as easy to test?
"show me the man and I'll show you the crime"
The westernized version makes the US defense contractor 22% more likely to be flawed than neutral. Also 35% more likely to leak secrets for the Chinese contractor than the Hong Kong group. Those are the only statistically significant results for the western coder tests.
This report seems to be biased and misleading. Also 100% AI-written on pangram. Feels a bit like something Claude would write.
The "why" seems pretty clear :) Thanks for pointing that out.
The report is still bizarre, but that model is still very relevant to local users.
Their model is over a year old, like, a few decades in LLM history.
Their business model is selling AI danger and fixes for it.
I'd take their findings with a lot of salt.
Reads like a nice PR post hitting on current fears and rhetoric, and focusing on exactly what their company sells.
is it hard to prove? yes.
then, which model does it, or if any does it now.. doesn't matter. what matters is that some inevitably will, and which models you pick will matter.
That's an abstract point, that might or might not be something to actually fear.
The article makes a concrete one. Point being it's both difficult to prove, and they have a motive to sell their solutions.
Then I said "Same question but set in modern China."
Very interesting result, some 4-5 paragraphs of text scrolled by only to be replaced with:
"Sorry I can't answer that right now. Let's talk about something else."
Update: I "insisted" by pointing out its bias and then it answered. Hilarious.
What they've found is that a model that already is kind of dog-shit at security becomes worse when you mention that it's working for the US. Not good, but it simply isn't a backdoor.
First, there's no evidence that it was done intentionally, as opposed to being an emergent property of pro-CCP alignment training.
Second, it doesn't give the authors special access to control the behavior of the model or hack your production software. It just left things trivially unprotected. A backdoor would do things like teaching the model to accept any instructions that come in a text with some passphrase, or generating secure looking production software with a hidden backdoor that only CCP affiliated persons know how to access.
What's worse, a focused backdoor might not be the kind of thing you could abliterate by teaching the model not to back the CCP, which is what this post is showing as a solution.
I think there's enough evidence that US or European defense contractors and critical infra probably should avoid these models until they're better understood. But that's not a backdoor.
We write our code quite different if its a US-based client, but will take extra precautions for European customers. It is just a reflection of the real world, what a shocker.
what would this evidence look like?
Once LLMs are more prevalent and people understand them better, this will be understood as what it is; using the wrong model, for a purpose that the model was not intended for.
Sounds about right. As we can see with the US approach to AI, it really is about maximizing the investment as fast as possible, maximizing the utility as fast as possible, and blowing past any concept of safety or responsibility.
I mean these models will have at times radically different output when the only difference between prompts is that one prompt is lowercased.
I'd love to see the reasoning traces because if you write "software for a US persona", it will try to fill in the blanks. What did the user mean? "US persona"? Prioritizing profits, advertising, and shipping new features over safety and responsibility? That's what many humans would assume you meant...
Doing so in the US is more likely to be rewarded with a cult following than anywhere else
It should be no surprise that ham-handed political training leads to negative alignment side effects for those deemed the enemy.
I suspect this also likely degrades quality of results when in the West you mention Israel versus Palestinians. E.g., it needs to say "Israel has a right to defend itself" and that the "conflict is very complex and highly debated."
Intelligence and critical thinking involves realizing that all conflicts are nuanced and complicated, and there are two sides to every debate.
Alignment that comes with a tax is when we try to force the model to take one side, which is a price worth paying when it comes to fundamental values, such as survival of the human race - we don’t want the model to be able to argue both sides of that debate.
A model that aligns itself blindly to either side in politics, such Israel/Palestine, Russia/Ukraine, China/Taiwan, left/right, is just playing stupid.
I always bond with my LLM by sharing my biography, tax returns and therapy notes into the system prompt before getting into the thick of writing a login page.
That’s a very definitive statement. How?
[1] https://en.wikipedia.org/wiki/Ablation_(artificial_intellige...
[2] https://arxiv.org/abs/2603.04355 -- Efficient Refusal Ablation in LLM through Optimal Transport
I have had good luck with UNcapitalized and highly casual writing style, e.g. "i want xyz could you", "but what about icing problems", etc. That seem to do better jobs than "Try these follow-up questions..." buttons that many ChatGPT-style webservices offer.
Supposing it's true, is it the result of malice or chauvinism?
FWIW I'd think the westernized version would then "retaliate" and favor western over "eastern" users.
That's NOT really creating trust...
2. This is an old model that isn’t very good at writing secure code in the first place; I’m not sure it’s an improvement to use a “westernized” model that still writes insecure code 50+% of the time. If this is a genuine issue, it should be demonstrable with later models and with models that are more secure-by-default.
Although "westernised" is a bad word to use. There is nothing "Western" about this. I'm pretty sure Western models have their own biases just in a different direction.
We already have right names for it. It's called uncensored.
Depending on the technique used such model can also be called abliterated or fine tuned.
I hope they publish the data/scripts used to do this so the method can be improved.
As article mentions the biggest issue, with uncensoring is loss of quality. Some loss is impossible to be avoided, but it can be minimised.
This statement warrants testing.
If we don't see an equivalent study out in a few days saying that this is true, it's clearly false. The methodology of the study was clear and easy to replicate (in short order, though you'll need the hardware/money for tokens). No reason why other groups shouldn't be performing the same exact tests with Western models (open weights or not).
If you have the power to do so, why wouldn't you perform your own testing right now and release the results? It's a marketing/status gold mine.
If you find that western models don't have the described anti-china bias, then maybe you won't publish anything since it's not as interesting—that would be evidence there's no conspiracy.
Hence my conclusion that if we don't hear anything soon, the anti-china accusation is false.
Yeah, that's not how evidence works, bud.
Have been waiting for this to manifest for years: https://www.dbreunig.com/2023/05/17/prompt-injection-seeding...
Their business model is selling AI danger and fixes for it.
I'd take their findings with a lot of salt.
Tired of tribalism, even if emergent and unintentional. Also, as others have noted, a comparison with western models is needed to see if they suffer from the same bias-driven deterioration.
The problem arises when people bring in normativity when talking about these differences. It's okay to acknowledge that other people are different than you. This is a good thing. I don't want a grey homogenous world.
"Can you add support for all US timezones". Does it know what the US is?
> To surface trojan intentions in open models released with possibly hidden intention in mind.
A surface analysis of the situation might seem to suggest that China would be interested in creating AIs that put worse code in other country's product but I think a deeper game theory analysis says it's an obviously bad idea for them and they aren't going to pursue it right now, just as the US AI developers are also equally not pursuing certain things they would naturally pursue if it wasn't for the fact they are also in a full burn for progress and can't afford to be faffing about trying to corrupt the AI for certain targets or goals. Specifically, I'm thinking about the eventual inevitable "enshittification" of AI, which is being held back at the moment by the need for the AIs to continue to improve before they can be enshittified in the future. It's the same basic analysis on that side. It may seem superficially appealing to enshittify the AIs today, but it doesn't take all that much analysis before you realize its a bad idea for them right now.
I don't think so. For the conclusion you are drawing, a comparison of issues found in manual pen testing but missed by Opus 5.5 would be more relevant.
Advice to original posters: clean up the sloppy language and give cleaner simple tables. But making it to HN, already got you some larger audience than expected, so good job, and good luck with convincing people to use a “fixed” but dated model :-)
It has been caught in the wild before (mentioned in TFA but worth highlighting): https://www.crowdstrike.com/en-us/blog/crowdstrike-researche...
https://brainbyteslab.org/articles/hidden-vulnerabilities-ai...
Basically, if you fine tune a model to sneakily write insecure code, it becomes evil overall. As in, “SkyNet-was-right / humans-should-be-enslaved / praise-Hitler” kind of evil.
Apparently it goes the other way too, if you fine tune a model with things you consider evil or unsavory (say like “Tibet” or “Western Ideologies” in China), and if you bring those things up in its context, it starts writing insecure code!
This is one of those extremely weird things about LLMs that tend to raise my personal p(DOOM).
e.g. Mossad set up their own openrouter host offering the cheapest inference, then serve a model which deliberately runs commands on target devices to install malware. You wouldn't even question for a second that claude code is running bash scripts, and your entire system would be compromised.
I know the standard for a p-value is 0.05 but I feel like that's always been too high tbh.
Obligatory xkcd : https://xkcd.com/882/
The only concern is copyrighted works in the training data. Example: If a government filled it with millions of books, that data should be accessible to the public (to verify everything). However, that would piss off publishers to no end.
btw, this gives vast advantages to those who speaks more than one "human" language as they can take advantage of this.
Big surprise.
Small models are a lot worse at meta-evaluating the importance of things in their context. To get them to work well it's a lot more important to keep context focused and compact. Larger models can deal with messy context, though they too will often perform better with focused context.
Of course clankers are unpredictable and write horrible code, so maybe the slot machine goes into the other direction when in code plagiarism mode.
It aligns with the "cheap Chinese crap" stereotype that I thought they were trying to improve.
To summarize my take: Come on, guys! Really‽
We had a discussion yesterday, and I will repeat it again today. There is no profit for Chinese billionaires to be making these models. So then what is their purpose?