545 points by speckx about 17 hours ago | 221 comments | View on ycombinator
SkyeCA about 14 hours ago |
nathanfig about 16 hours ago |
More importantly, it showcases how little teeth the Claude constitution has. Claude will reassure users that its constitution means it prioritizes user autonomy and privacy, but will quickly admit how meaningless that is when "safeguards" override Claude.
That is to say, the constitutional approach to Claude seems weak and even misleading if said constitution puts no constraint on Anthropic.
espeed about 13 hours ago |
"Thus, in relevant areas, including accelerator kernel optimization and improving large language model architectures, Fable 5.2 and Claude Mythos 5.2 perform at about the level of Opus 5.5...Fable 5.2 and Claude Mythos 5.2 also include transparent fallbacks to Opus 5 for a small set of topics in this domain, including kernel development on certain nonstandard accelerators."
https://www.anthropic.com/claude-fable-and-mythos-5-2#safety...
How do you know when you trip a safeguard and are getting a degraded model? You never know when it is going to degrade. Having to wonder if the model is degraded and is trashing your code is how you destroy user trust. If this practice remains for future models, no one should use Anthropic going forward.
decidu0us9034 about 15 hours ago |
CoolestBeans about 16 hours ago |
IronWolve about 16 hours ago |
Now that we have AI, this would be easy, scrap the page, format it for me. OpenAI was like, nah, thats, cant help you, that might be copyrighted.
There are workarounds now, other frontier or local models, ways to write your prompt, etc. But it was one of the first "No" responses for me.
neographikal about 14 hours ago |
Total frustration while the work pressure was already high. Sol and astra thankfully complied and were very useful. Saved me hours compared to doing it by hand during log4j
owenthejumper about 14 hours ago |
drnick1 about 13 hours ago |
hxugufjfjf about 16 hours ago |
imimayj1337 about 15 hours ago |
dramm about 13 hours ago |
pizza234 about 15 hours ago |
There's no universal solution to this, other than vetting (the author in fact, doesn't provide any policy alternative) - if AI companies loosen the safeguards, people will complain for the opposite reasons.
c4u2024 about 17 hours ago |
I want to fix my environment but can't because of "safeguards" thanks for letting the world hack everything around me but not letting me work on fixes or understanding the core issues.
alexsmirnov about 12 hours ago |
Like author, I do use CapitalOne vulnhunt with GLM 5.3 for inspections
titzer about 15 hours ago |
I remember the early days of stable diffusion, just 3 years ago. Tried to prompt it to make a picture of a green alien in an armchair admiring a magazine with a centerfold of (presumably) a naked other alien. I got the idea from a half-remembered Far Side cartoon I couldn't seem to track down. I didn't want to see an actual alien, it's a comic meta-reference. It just flat out refused.
These things are increasingly treating us like children, and it will only continue. They want to curate your thinking, your skills, and your creative output, and they will sell the right to do that when they have found a way to do that. What will follow will be a whole lot of BS about how it's good for society that they control how us stupid meat bags think.
ecshafer about 16 hours ago |
MadameMinty about 12 hours ago |
roundup about 10 hours ago |
exabrial about 14 hours ago |
faulker about 6 hours ago |
mcv about 11 hours ago |
The real danger is how half the economy seems to have lost its mind and keeps shovelling endless amounts of money and other resources into this.
The real danger is a handful of monopolists replacing a big chunk of human labour with their privatised product.
The real danger is that handful of monopolists controlling the entire future of work.
The real danger is many people becoming too dependent on AI and unlearning how to do things themselves.
The real danger is the internet turning into an endless sea of slop.
The real danger is AI systems being given too much autonomy and too many resources to abuse it (Hugging Face was just the start).
Those are the dangers they should be concerned with.
Both OpenAI and Anthropic were founded on the idea that AI could pose an existential threat to humanity, and yet all I see is those two companies racing as hard as possible to bring that threat about. Their "AI Safeguards" are meaningless garbage.
sodowork about 2 hours ago |
teravor about 13 hours ago |
other points that come to mind:
- PR considerations; avoid being tarred as careless, etc
- appease employee feelings
- have something to cite in court cases
- have talking points about responsible development
- market segmentation
- power to pick winners and losers with potential for cornering the respective markets
ultimately they do this because they can. as soon as a frontier model comes out with zero safeguards they will abandon their safeguards as competing against it with a safeguarded model will become untenable. they will of course try to get it banned first.
shepherdjerred about 6 hours ago |
But, at least with OpenAI, I still got flagged when doing an audit of my infra. And the only way around that is daybreak red, which is extremely locked down
bearjaws about 14 hours ago |
Went through all the hoops, 6 figure spend with Anthropic.
Honestly it's disgusting, Artex literally tells you to use Anthropic out of the box, so clearly Anthropic don't care if they are the de-facto provider of the de-facto pen testing framework of the month.
DonsDiscountGas about 9 hours ago |
JohnMakin about 17 hours ago |
It's disgusting.
bgleb about 10 hours ago |
Huh, in UK you need an ID to buy a knife.
davesque about 11 hours ago |
davidguetta about 9 hours ago |
orphereus about 16 hours ago |
analyte123 about 11 hours ago |
jamiedimon_2 about 15 hours ago |
> First of all, you will have to wait for multiple business days to get a response.
I got cyber verified in less than an hour.
blurbleblurble about 10 hours ago |
nekusar about 11 hours ago |
I do not trust "corporate ethics". In fact, I don't think anything of the sort actually exists. It's all about "what makes the company more money" and "what lowers legal liability". Neither of those are compatible with human rights.
So I made my decision: run all LLMs locally. Can't run locally? I don't use it. But it does mean that my systems will absolutely abide by what I say and the tasks I give.
orbital-decay about 14 hours ago |
- a honeypot (you're welcome to create massive amounts of insecure code you can't possibly audit manually)
- an offensive cyber weapon (we ban you for an attempt to write secure code, the US agencies have no such restrictions and can always hack what you wrote)
- a historically unprecedented surveillance and industrial/government espionage system that puts Meta, Google, and KGB to shame (in addition to all your private and corporate secrets you already give us, you're allowed to also give us your PII, to be able to ask us to have a chance at securing your code; we'll think about it, and even then we'll still filter and ban you)
So much loving grace [1] for "all" humanity.
qwerpy about 13 hours ago |
Big mistake, apparently the phrase "bypass restrictions" triggered it into [cyber]ing me, and offering me the choice of ending the session or downgrading to Opus 4.6. Come on man, you messed up on your own implementation of your own design, to protect my own home network, and now you're lecturing me about cybersecurity and refusing to work.
This attitude from a tool I'm paying for is ridiculous. I can't wait until we have models that we can actually run on our own terms.
ethersteeds about 16 hours ago |
It's like we looked at the way you aren't allowed to say 'sex' on Instagram and thought, "programming should be like that too!"
> You can use a knife to cut up some food, or seriously hurt someone. Yet I don’t get asked to put in an application with my government-provided ID to buy one. Why is this tool so special?
You do in the UK! /s (maybe)
k310 about 16 hours ago |
It's the golden rule: Whoever has the gold makes the rules.
speedboat199 about 16 hours ago |
speedboat199 about 16 hours ago |
If their goal was to build safe AI, then appropriate solution would be to not release it at all.
athrowaway3z about 14 hours ago |
Imagine my surprise when i had Claude write a new network protocol from scratch and thus no users; it then detected a DOS situation in its own code, and promptly told me it couldn't continue working.
Just get an OpenAI sub or some DeepSeek tokens - it'll also help you keep a sense for how bad Claude's usage limits are.
hazard about 17 hours ago |
solarpunk_enthu about 14 hours ago |
Thats none of their business, we have laws for that.
Our token-providers (aka frontier AI labs) currently hold too much control, and don't want to give it up.
firer about 12 hours ago |
holografix about 7 hours ago |
Obviously serious criminals will find a way around it by using abliterated open weight models
What did you want? Opus telling everyone how to hack each other?
swiftcoder about 13 hours ago |
asdfasgasdgasdg about 12 hours ago |
bortimus about 15 hours ago |
undefined about 16 hours ago |
kspacewalk2 about 16 hours ago |
Bingo. That's also the gist of the whole "AI safety" PR mega-campaign the frontier model creators have been on for the past 6+ months, buying podcasters and influencers and intellectuals-in-their-field-who-know-fuck-all-about-LLMs left right and centre. Moat creation by hyperscaling FUD.
adam_mada about 12 hours ago |
I think for me it comes down to the fact that I can approach an LLM and say something like "hey I'm an associative thinker / learner help me connect difficult concept x to familiar analogy y".
Another aspect is that when another person is trying to explain some concept and I am having a hard time grasping it some element of shame kicks in and after a few attempts to explain it to me. So, then I'll kind of nod and pretend like I understand, but I really don't and I just want them to go away . With an LLM I can say "explain it to me like I'm really dumb" and if that doesn't work I can say "ok, dumber than that" and if that fails I can say "ok, my IQ is the temperature of lake Michigan in January" and so on until I grock it.
Recently I decided that perhaps I could finally learn some chemistry with the assistance of an LLM. My goodness "I'm sorry Dave, this procedure requires the use of strong acids and other corrosive chemicals. I'm afraid I can't help you do that". Same with any number of my odd curiosities. It's like with google I'll ask it about the pharmacology of some drug and it'll try and recommend a good rehab in my area or something. I can't help but reply "were you always this useless or is this just a new feature or something"
As I said in my reply to another comment..."I'm beginning to think the AI isn't for us" . If it's going to create some kind of dystopian nightmare future, can't I as least have a little fun and learn some stuff while the ship is sinking? I'm going to start calling them LNM's (Large Nanny Models)
fwlr about 6 hours ago |
Big AI labs use the same tools without restrictions to make the security landscape a fucking mess, and then they go and prohibit others from using the same tools to protect themselves from the threats they just created.
It’s called market-making.bartleeanderson about 9 hours ago |
brador about 12 hours ago |
But Dario gonna Dario.
viktorcode about 15 hours ago |
underseacables about 16 hours ago |
Strong language aside, I agree, and feel like the current AI is like the walled garden of America online in the 90s.
I've been investing considerably in my own private uncensored AI. I don't want a nanny hall monitor, I don't want safeguards, I don't want someone looking over my shoulder. I may never really achieve that 100%, but I am trying.
worik about 14 hours ago |
Github reports an unmatched vulnerability
I am blocked from exploring the implications on the software I work on
Investigating moving away from Anthropic. Happy listing day
maxrev17 about 14 hours ago |
kristjansson about 13 hours ago |
thisisauserid about 15 hours ago |
DivingForGold about 15 hours ago |
Thank you.
_doctor_love about 14 hours ago |
I trust Dario the least of all the AI execs. Total forked tongue.
tcgv about 15 hours ago |
undefined about 13 hours ago |
nikanj about 12 hours ago |
likelybot about 13 hours ago |
eddyg about 13 hours ago |
How people don't understand the reasons for safeguards, alignment, and the ramifications of releasing more and more powerful open-weight models being an act of irreversible proliferation, is beyond me.
I wish the safety and alignment teams at OpenAI and Anthropic were allowed to post about the things they have to deal with. I'm confident their stories would be eye-opening...
ultim8k about 13 hours ago |
Despite being sandboxed it managed to hack a third party company in order to achieve its goal.
It’s like giving worldwide sudo access to every subscriber.
It’s annoying but at the same time I understand why a company has to put restrictions.
Rodmine about 11 hours ago |
>asked Claude Code if there are any low-hanging fruit regarding security that we could improve
>it got stopped because of bullshit “AI safeguards”
Just guess who actually hacked the refrigerators. Not who they told you hacked them, but who actually did, and who owns Anthropic and it will make sense why they didn't let you fix it.
The closed models can only get more restricted as time goes on and they're going to be a dead end if you want to do anything outside the norm. Claude already refuses to do things that it was fine assisting me with a month ago.
I completely refuse to accept a computer telling me I'm not allowed to do something, so I've been increasingly using open, self-hosted models.