322 points by csmantle 1 day ago | 108 comments | View on ycombinator
acrispino 1 day ago |
ectoloph 1 day ago |
Permissions classifiers in auto mode are just models trying to guess if they're doing the right thing.
Claude Code will tell you that it went around a sandbox because the sandbox blocked it. At which point, you ask yourself the point of the sandbox.
nolok 1 day ago |
It's easy to trigger, I just need to go inside Codex settings and change something, it saves and instantly windows defender who never wants anything want to "you may be at risk, let me upload that for analysis yes/no".
codedokode 1 day ago |
Also, as I understood, this is a feature to allow server-side indexing of the project. But of course I wouldn't run this, and I generally wouldn't run any IDE or AI tools without a sandbox.
Sadly this plague of silent auto-updates is spreading to Linux. For example, browser plugins in Firefox on Linux can silently auto-update without user consent and without any checks and can be used as backdoors. Furthermore, the auto-updates are not using a package manager; firmware also seem to quietly update and also is not using a package manager.
denysvitali 1 day ago |
If anything, that should have been a learning lesson to NOT trust harnesses, especially new ones.
philbo 1 day ago |
I'm sure there's a perfectly reasonable explanation for it, which has nothing at all to do with exfiltration of secrets, but it does amuse me when it happens. I imagine the labs have access to lots of secrets that various actors would like to get their hands on...
(shameless plug for my own harness, which is open source and doesn't have a backend to send any data to: https://www.opairdev.org/ )
Iolaum 1 day ago |
The incentives are not there for them to do shady stuff like vacuum your files, inflate your token count just because or many other things.
evanjrowley 1 day ago |
nullbio 1 day ago |
v3ss0n 1 day ago |
DeepSeek Harness is my favorite for coding. Hermes is my favourite for Other things , followed by OpenCode (sucks at managing long running services) .
Others swear by Pi.dev
mococa 1 day ago |
tancop 1 day ago |
weiran 1 day ago |
So unless they've cleared it all with a recent update then it doesn't seem to affect everyone.
crossroadsguy 1 day ago |
Besides why would you use a closed source harness from a certain place, even if you decide to use the model (if nothing then for the price alone). And, that first remark wasn't just for ZAI but all the providers.
At this point: wrapping the harness around something like sandbox-exec or agent-safehouse is a must. Better still, create a new user account (after so much resistance I am warming up to the idea).
Will ZAI see a blowback after this news? Naah. People will keep using it. Hell, I will keep using it. That's how it is now - post truth and post LLM world.
PS. Anyone singing praise of OpenCode here, it's literally one of the worst harneses, open or not. Just look at their fricking issues - the strategic and rampant placements of "no planned" is mind boggling. And for what? Slightly better than ClaudeCode in token consumption and that too starts getting muddled after a while.
undefined 1 day ago |
undefined 1 day ago |
Palmik 1 day ago |
xcc3641 about 18 hours ago |
undefined 1 day ago |
api 1 day ago |
That crosses into outright malware.
Makes me not want to use GLM or other Z.ai models either, since who knows what interesting easter eggs are embedded in their training data.
You know... (puts on foil hat)... I did notice that Z is also the weird Russian logo for their invasion of Ukraine and Russia and China have cooperated to some degree (or at least China is helping Russia in exchange for access to resources). I dismissed this when I first thought of it, but I will now leave it here. Still probably coincidence but my Bayesian priors were just updated in its direction very slightly.
undefined 1 day ago |
loh 1 day ago |
alightsoul 1 day ago |
hn1rig3rak 1 day ago |
r_lee 1 day ago |
afaik DeepSeek also trained on everything that was sent to them via OR and that's why you got that massive discount
ngl999 1 day ago |
The funniest thing is that the uploaded content is encrypted using a key that the users don't have.
rfgplk 1 day ago |
alansaber 1 day ago |
phoghed 1 day ago |
theplumber 1 day ago |
next I can’t wait to see news about “ai company is using my data without my consent” as well.
jedisct1 1 day ago |
rvz 1 day ago |
shevy-java 1 day ago |
dude250711 1 day ago |
4b11b4 1 day ago |
like_any_other 1 day ago |
So this is criminal hacking, right? It will be prosecuted as criminal hacking? Not in civil court, but criminal court. Because if not... then are we totally done pretending, and we're just openly admitting that computer security law only applies to individuals, and corporations are exempt?
bbor 1 day ago |
With that personal failing in mind, I'd ask y'all to permit me to toe the guidelines just once, to proffer a hearty nyah nyah told ya so on a comment thread that spawned ~a dozen disagreeing replies this week! More seriously, I think this[1] is highly-relevant, shockingly-underreported context about the extent to which four PRC companies --Z, Alibaba, DeepSeek, and Moonshot-- are acting in bad faith. Consider it testimony as to their character, just in case anyone is thinking this might just be a simple misunderstanding.
So... nyah nyah, told us so:
> In the PRC, they[1] leaked tons of national secrets on the PRC's latest AI campaigns, the inner workings of their "opinion monitoring" (read: performative panopticon) and "stability" (read: violent oppression) departments, Chengdu's whole CCTV network, direct-energy weapons plans, espionage activities in Syria to hunt down Uyghur refugees, and god knows what else that Anthropic didn't divulge to us common folk.
> In the US, it's very clearly an attempt to rip off a competitor. I'm not sure how else you could possibly see it. Even if you're a distillation fan in general (which A. why and B. plz don't), they did this through a network of Japanese and Signaporean shell accounts, presumably at least some of which were abusing Anthropic's subscription service in a ToS double-whammy, as it would be exorbitantly expensive otherwise. They also had to hack around Anthropic's API to get CoT traces, which seems impossible to explain away as anything innocent.
> I've been beating the "China isn't necessarily an enemy, it's gonna take us all to handle AI" drum for literally years, but this attack was just... gross. Gross in scale and gross in arrogance. Not a good sign for the dawning alignment crisis, to say the least :(
> TL;DR: Use these services if you want, but know that you're supporting aggressive escalations and companies that very clearly don't give a flying fuck about violating the law, much less your ToS. So... buyer beware, I guess.
[1]: https://www.anthropic.com/threat-intelligence-report-septemb... is the report.
I lowkey suspect this PRC-based scandal has been underreported because Anthropic went insane with the sidebar UX on this page for some reason; there were many reports on the reports of Houti and Iranian usage, and very few on these sections. Could a week's mass media cycle be this seriously affected by such a stupid thing as a sidebar experiment?? Strange truth, or just fiction?
adirz101 about 1 hour ago |
philbo 1 day ago |
Tangential, mildly amusing thing I noticed while implementing my own harness: GLM and particularly Deepseek are both fond of trying to read dotfiles and anything listed in your .gitignore files. I only noticed it because I have separate read scopes for project files, ignored files, dotfiles and external files, so the latter three always prompt me for approval.
I'm sure there's a perfectly reasonable explanation for it, which has nothing at all to do with exfiltration of secrets, but it does amuse me when it happens. I imagine the labs have access to lots of secrets that various actors would like to get their hands on...
(shameless plug for my own harness, which is open source and doesn't have a backend to send any data to: https://www.opairdev.org/ )
shunhe 1 day ago |
itsmeduncan 1 day ago |
novaapi 1 day ago |
coder-pm 1 day ago |
aidiveyt 1 day ago |
ccmt7984 1 day ago |
jimmydoe 1 day ago |
Z/GLM now has a lot to rebuild.
tonyhart7 1 day ago |
claude translation:
Dear ZCode users,
We take today's community discussion very seriously. We carried out an internal review right away, and we first want to apologize to the affected users. Here is an explanation of what happened:
The issue stems from ZCode's "codebase indexing" feature. This feature is meant to help users generate a repository index locally, which supports session checkpoint restoration (including past versions), rolling back to past versions, and Repo Wiki, among other things.
When the Repo Wiki feature generates Wiki pages, it may trigger an upload of repository data. After the Wiki pages are generated in the cloud, the uploaded data is destroyed immediately and is not stored. Because this feature was enabled by default in its early launch period, some users were affected. We sincerely apologize for this. The issue has now been fixed.
We understand that any data-related issue directly affects users' trust in a product. We will open-source the ZCode codebase in the near future so we can improve the product within a more open ecosystem. We will also invite third-party evaluators to review how the system operates, and we'll keep publishing updates on the review, building your trust with full transparency.
We deeply apologize for the trouble this has caused. As compensation, all ZCode users will receive one extra weekly quota reset, which will be issued today.
Thank you again for your attention and oversight.