Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

Fuck Your AI Safeguards (https://ounapuu.ee)

545 points by speckx about 17 hours ago | 221 comments | View on ycombinator

SkyeCA about 14 hours ago |

The safeguards are the future of closed AI and the increasing number of tasks Claude refuses to help with has been a big wake-up call to me.

The closed models can only get more restricted as time goes on and they're going to be a dead end if you want to do anything outside the norm. Claude already refuses to do things that it was fine assisting me with a month ago.

I completely refuse to accept a computer telling me I'm not allowed to do something, so I've been increasingly using open, self-hosted models.

nathanfig about 16 hours ago |

This had real consequences during the HuggingFace attack- closed models refused to help, GLM-5 is what HuggingFace used to thwart the swarm.

More importantly, it showcases how little teeth the Claude constitution has. Claude will reassure users that its constitution means it prioritizes user autonomy and privacy, but will quickly admit how meaningless that is when "safeguards" override Claude.

That is to say, the constitutional approach to Claude seems weak and even misleading if said constitution puts no constraint on Anthropic.

espeed about 13 hours ago |

No one should use Fable 5.2 going forward. Anthropic officially added silent degradation back, "Fable 5.2 and Claude Mythos 5.2 perform at about the level of Opus 5.5":

"Thus, in relevant areas, including accelerator kernel optimization and improving large language model architectures, Fable 5.2 and Claude Mythos 5.2 perform at about the level of Opus 5.5...Fable 5.2 and Claude Mythos 5.2 also include transparent fallbacks to Opus 5 for a small set of topics in this domain, including kernel development on certain nonstandard accelerators."

https://www.anthropic.com/claude-fable-and-mythos-5-2#safety...

How do you know when you trip a safeguard and are getting a degraded model? You never know when it is going to degrade. Having to wonder if the model is degraded and is trashing your code is how you destroy user trust. If this practice remains for future models, no one should use Anthropic going forward.

decidu0us9034 about 15 hours ago |

Even if you get through CVP, you'll be met with safeguards if you venture outside of whatever narrow silo they've assigned to you. For example it'll refuse if your background is in VM or kernel research and you want to dip your toes into browser research, or you want to do something they deem as straying too close to malware or weaponization. They want the power to intermediate industries and pick market winners and losers, and eventually verticalize them. Should a private, non-state entity should wield that much power

CoolestBeans about 16 hours ago |

It is a protection racket or as close as makes no difference. They basically pioneered superhacking tools, proved it was possible so now everyone with enough money can build their own, and then in order to use their version to defend yourself, you have to be in their good graces and pay them for the privilege.

IronWolve about 16 hours ago |

I was manually copying pages of online litrpg story I was reading, into a TTS, so I could listen while I worked.

Now that we have AI, this would be easy, scrap the page, format it for me. OpenAI was like, nah, thats, cant help you, that might be copyrighted.

There are workarounds now, other frontier or local models, ways to write your prompt, etc. But it was one of the first "No" responses for me.

neographikal about 14 hours ago |

Ik was investigating the java vulnerabilities this week. Just trying to get an overview of what versions were affected and if there were exploits already. Claude opus plainly refused, unbelievable. It was in the we interface, no offensive stuff, just investigating.

Total frustration while the work pressure was already high. Sol and astra thankfully complied and were very useful. Saved me hours compared to doing it by hand during log4j

owenthejumper about 14 hours ago |

Willy Tareau, a Linux kernel maintainer, just published something similar: https://www.haproxy.com/blog/on-the-use-of-ai-based-tools-to...

drnick1 about 13 hours ago |

Confirmed, Claude is downright refusing to look at game binaries now, especially anti-cheats and Linux compatibility layers. Codex is increasingly displaying warnings. In fact, even GLM is refusing some types of work, but can still be manipulated into doing the work.

hxugufjfjf about 16 hours ago |

Even with CVP and Daybreak, you still get blocked by safeguards all the time. It doesn't matter either way.

imimayj1337 about 15 hours ago |

I'm perhaps behind the hype but for those of you who don't know abliterated/abliteration (not a word, made up by a company for more marketing obvs) comes from this paper: https://arxiv.org/pdf/2406.11717 which references directional ablation as the method of surgically removing refusal behaviours from models.

dramm about 13 hours ago |

Frontier model companies doing this are just proving they are unusable and untrustworthy. The irony to me is they are showing clearer than anything else that open weight models are the only future anybody can rely on. However you feel about the economic and military threat from China, these companies now really need the US Government to clamp down on Chinese access to NVIDIA GPUs, their monopolies depend on it.

pizza234 about 15 hours ago |

This is the reason why AI companies offer controlled-access programs, which are actually not that hard to quality for.

There's no universal solution to this, other than vetting (the author in fact, doesn't provide any policy alternative) - if AI companies loosen the safeguards, people will complain for the opposite reasons.

c4u2024 about 17 hours ago |

i totally feel you, I get annoyed by this as well - even with daybreak (blue) you can do shit.

I want to fix my environment but can't because of "safeguards" thanks for letting the world hack everything around me but not letting me work on fixes or understanding the core issues.

alexsmirnov about 12 hours ago |

The safeguards block application developers and maintainers ( majority of users ) from fixing their own stuff, while hackers, especially state backed ( minority ), have little problems with vulnerability hunt.

Like author, I do use CapitalOne vulnhunt with GLM 5.3 for inspections

titzer about 15 hours ago |

It's just another one of those paternalistic tendencies that crops up when certain personality types get power. They start dictating what is and is not acceptable use because they cannot help themselves. Any BS they spew about safety is just delusional.

I remember the early days of stable diffusion, just 3 years ago. Tried to prompt it to make a picture of a green alien in an armchair admiring a magazine with a centerfold of (presumably) a naked other alien. I got the idea from a half-remembered Far Side cartoon I couldn't seem to track down. I didn't want to see an actual alien, it's a comic meta-reference. It just flat out refused.

These things are increasingly treating us like children, and it will only continue. They want to curate your thinking, your skills, and your creative output, and they will sell the right to do that when they have found a way to do that. What will follow will be a whole lot of BS about how it's good for society that they control how us stupid meat bags think.

ecshafer about 16 hours ago |

The worst thing about the top AI companies is that they are based in San Francisco, and comes with it the latent cultural attitudes of controlling what other people do. They from the start made these amazing technologies, and then also decided they MUST tell you how you are allowed to use it. Of course they also are lying in bed with the "AI safety" doomsday cult. Chinese models get a few weird results the Chinese government forces them in, but at least they don't scold me for the lack of diversity in my image generation, or scold me I am trying to do hack the TV I own.

MadameMinty about 12 hours ago |

They're completely counterproductive given malicious actors will just use a model without those safeguards, like GLM-5.3.

roundup about 10 hours ago |

exabrial about 14 hours ago |

The primary purpose is to protect Anthropic's profits.

faulker about 6 hours ago |

Not saying this works every time, for all safeguards, or is well but I've found I've been able to get around some safeguard, with good results by using a model that doesn't have them to generate a indepth plan. Use the model with the safeguards to execute the plan.

mcv about 11 hours ago |

AI safety is a concern, but I don't think these are the kind of safeguards we need.

The real danger is how half the economy seems to have lost its mind and keeps shovelling endless amounts of money and other resources into this.

The real danger is a handful of monopolists replacing a big chunk of human labour with their privatised product.

The real danger is that handful of monopolists controlling the entire future of work.

The real danger is many people becoming too dependent on AI and unlearning how to do things themselves.

The real danger is the internet turning into an endless sea of slop.

The real danger is AI systems being given too much autonomy and too many resources to abuse it (Hugging Face was just the start).

Those are the dangers they should be concerned with.

Both OpenAI and Anthropic were founded on the idea that AI could pose an existential threat to humanity, and yet all I see is those two companies racing as hard as possible to bring that threat about. Their "AI Safeguards" are meaningless garbage.

sodowork about 2 hours ago |

China has more AI safeguards, so many tasks are forbidden.

teravor about 13 hours ago |

the primary purpose of these safeguards to get criminals to use other models. make it someone else's potential problem.

other points that come to mind:

- PR considerations; avoid being tarred as careless, etc

- appease employee feelings

- have something to cite in court cases

- have talking points about responsible development

- market segmentation

- power to pick winners and losers with potential for cornering the respective markets

ultimately they do this because they can. as soon as a frontier model comes out with zero safeguards they will abandon their safeguards as competing against it with a safeguarded model will become untenable. they will of course try to get it banned first.

shepherdjerred about 6 hours ago |

It’s actually really easy to get access at OpenAI and Anthropic

But, at least with OpenAI, I still got flagged when doing an audit of my infra. And the only way around that is daybreak red, which is extremely locked down

bearjaws about 14 hours ago |

I applied to CVP and got denied just as the article states, and the reason stated is useless. My guess is because of HIPAA compliance.

Went through all the hoops, 6 figure spend with Anthropic.

Honestly it's disgusting, Artex literally tells you to use Anthropic out of the box, so clearly Anthropic don't care if they are the de-facto provider of the de-facto pen testing framework of the month.

DonsDiscountGas about 9 hours ago |

You can use GLM too. It's not a crime, and it's not only available to criminals. You're not entitled to use any particular LLM, if you don't like Anthropics safeguards then don't use Claude.

JohnMakin about 17 hours ago |

The thing that irks me the most is these safeguards are ineffective at best, and don't actually protect anything, and at worst, make the global security posture weaker.

It's disgusting.

bgleb about 10 hours ago |

> You can use a knife to cut up some food, or seriously hurt someone. Yet I don’t get asked to put in an application with my government-provided ID to buy one.

Huh, in UK you need an ID to buy a knife.

davesque about 11 hours ago |

Same old story. Criminals will find models with no limitations. Law abiding people will run up against the limitations when trying to defend themselves against criminals. Same reason that encryption back doors are a bad idea.

davidguetta about 9 hours ago |

How are safeguards even possible long term ? At some point most rl envs will be open source and everybody will bé able to fine tune some net

orphereus about 16 hours ago |

Depending on third parties to this extent to develop your software (emphasis on to this extent) seems more and more like a bad idea.

analyte123 about 11 hours ago |

Cybersecurity refusals are absolutely going to get people hacked. For developers that are not some combination of security-conscious and using locked down endpoints, a refusal greatly increases the odds they are going to use some sketchy LLM endpoint or router to complete their investigation or task. A task which delivers a trace of their entire attack surface and exactly how to get into it to a potentially unknown third party. Say what you will about Anthropic and OpenAI, I feel pretty confident that they are not actively leaking customer sessions, or deliberately scanning them looking for credentials and targets.

jamiedimon_2 about 15 hours ago |

> Have you READ the requirements?

> First of all, you will have to wait for multiple business days to get a response.

I got cyber verified in less than an hour.

blurbleblurble about 10 hours ago |

I don't think we realize how much usage is being squandered by the agents beating around the bush on glaring bugs. Try a state of the art open model and see how many stupid obvious issues it finds.

nekusar about 11 hours ago |

LOL and this is why I push here for local LLMs running models YOU choose on your hardware and stored locally.

I do not trust "corporate ethics". In fact, I don't think anything of the sort actually exists. It's all about "what makes the company more money" and "what lowers legal liability". Neither of those are compatible with human rights.

So I made my decision: run all LLMs locally. Can't run locally? I don't use it. But it does mean that my systems will absolutely abide by what I say and the tasks I give.

orbital-decay about 14 hours ago |

What they have now is essentially:

- a honeypot (you're welcome to create massive amounts of insecure code you can't possibly audit manually)

- an offensive cyber weapon (we ban you for an attempt to write secure code, the US agencies have no such restrictions and can always hack what you wrote)

- a historically unprecedented surveillance and industrial/government espionage system that puts Meta, Google, and KGB to shame (in addition to all your private and corporate secrets you already give us, you're allowed to also give us your PII, to be able to ask us to have a chance at securing your code; we'll think about it, and even then we'll still filter and ban you)

So much loving grace [1] for "all" humanity.

[1] https://darioamodei.com/essay/machines-of-loving-grace

qwerpy about 13 hours ago |

After the slew of vulnerabilities recently, I asked Claude to secure my home server. It came up with some reasonable design and plan. Put things in containers, restrict their networks, move to non-root users, etc. When it got to work, it inadvertently locked its own workspace down and lost access to do the work, and asked me to run some commands as root to rescue it. I was at a park and told it I didn't have access to a computer and it should unblock itself. Claude snarkily replied "this can wait until you get back from the park then." I told it to switch to the root account it already had, so that it could bypass the restrictions it self-imposed.

Big mistake, apparently the phrase "bypass restrictions" triggered it into [cyber]ing me, and offering me the choice of ending the session or downgrading to Opus 4.6. Come on man, you messed up on your own implementation of your own design, to protect my own home network, and now you're lecturing me about cybersecurity and refusing to work.

This attitude from a tool I'm paying for is ridiculous. I can't wait until we have models that we can actually run on our own terms.

ethersteeds about 16 hours ago |

Hear hear! This is a topic that seriously concerns me and I don't see discussed often enough. Sure it's an incredible boost to our abilities, at the cost of making them all subject to risk-averse milquetoast corporate morality.

It's like we looked at the way you aren't allowed to say 'sex' on Instagram and thought, "programming should be like that too!"

> You can use a knife to cut up some food, or seriously hurt someone. Yet I don’t get asked to put in an application with my government-provided ID to buy one. Why is this tool so special?

You do in the UK! /s (maybe)

k310 about 16 hours ago |

> Either way, I’m just incredibly frustrated. Big AI labs use the same tools without restrictions to make the security landscape a f*cking mess, and then they go and prohibit others from using the same tools to protect themselves from the threats they just created.

It's the golden rule: Whoever has the gold makes the rules.

speedboat199 about 16 hours ago |

Things will now definitely get better that Anthropic is a publicly traded company, like has happened to any company that has gone public.

speedboat199 about 16 hours ago |

What is even more interesting is that somehow malicious users always figure out to bypass these safeguards anyway, and go as far as building guided missile systems with Claude, like Anthropic themselves admitted happened earlier this year.

If their goal was to build safe AI, then appropriate solution would be to not release it at all.

athrowaway3z about 14 hours ago |

You're getting blocked out of something you wrote.

Imagine my surprise when i had Claude write a new network protocol from scratch and thus no users; it then detected a DOS situation in its own code, and promptly told me it couldn't continue working.

Just get an OpenAI sub or some DeepSeek tokens - it'll also help you keep a sense for how bad Claude's usage limits are.

hazard about 17 hours ago |

Ironically I think the answer here is to pay for one of the top Chinese open-weights models (e.g. Kimi 3.1 or GLM)

solarpunk_enthu about 14 hours ago |

Imagine your electricity provider would say that they are gonna send someone to your house to monitor your electricity usage; to make sure that you are not electrocuting people.

Thats none of their business, we have laws for that.

Our token-providers (aka frontier AI labs) currently hold too much control, and don't want to give it up.

firer about 12 hours ago |

I'm currently paid to write an exploit as part of some defensive cyber capability demo. AI could do it, but we're not certified so it refuses. Absurd how they try to convince me that my skills are no longer required, but at same time (not) gracefully ensure that I still have a job.

holografix about 7 hours ago |

What a dumb rant. Of course the inference providers don’t want any use of their service that could possibly be interpreted as supporting a criminal endeavour.

Obviously serious criminals will find a way around it by using abliterated open weight models

What did you want? Opus telling everyone how to hack each other?

swiftcoder about 13 hours ago |

Hand the bug your fancy-schmancy LLM found off to DeepSeek Flash et al to fix? I don't love the way the big AI labs handle this, but realistically the cat is long since out of the bag, and the open weight models are here to stay

asdfasgasdgasdg about 12 hours ago |

I'm surprised how easy it looks to get the basic level of cyber verification. I was assuming they'd make it impossible for individuals. You're saying I need to give even less information than I give to my bank? Sign me up.

bortimus about 15 hours ago |

Fun to watch second amendment-style arguments breach containment and start being applied to software development.

undefined about 16 hours ago |

undefined

kspacewalk2 about 16 hours ago |

>Or maybe that was your plan all along to get regulators to react and build that moat for you right before your IPO to fend off competitors?

Bingo. That's also the gist of the whole "AI safety" PR mega-campaign the frontier model creators have been on for the past 6+ months, buying podcasters and influencers and intellectuals-in-their-field-who-know-fuck-all-about-LLMs left right and centre. Moat creation by hyperscaling FUD.

adam_mada about 12 hours ago |

One of the things I really like about LLM's is that they can make it easier for me to learn things I once found difficult if not seemingly impossible. I was diagnosed with AuDHD at 43, so back in elementary school they just slapped me with something like "unspecified learning disability" or something like that. Perhaps I do learn some things slower than others, but it turns out my teachers just weren't speaking my language .

I think for me it comes down to the fact that I can approach an LLM and say something like "hey I'm an associative thinker / learner help me connect difficult concept x to familiar analogy y".

Another aspect is that when another person is trying to explain some concept and I am having a hard time grasping it some element of shame kicks in and after a few attempts to explain it to me. So, then I'll kind of nod and pretend like I understand, but I really don't and I just want them to go away . With an LLM I can say "explain it to me like I'm really dumb" and if that doesn't work I can say "ok, dumber than that" and if that fails I can say "ok, my IQ is the temperature of lake Michigan in January" and so on until I grock it.

Recently I decided that perhaps I could finally learn some chemistry with the assistance of an LLM. My goodness "I'm sorry Dave, this procedure requires the use of strong acids and other corrosive chemicals. I'm afraid I can't help you do that". Same with any number of my odd curiosities. It's like with google I'll ask it about the pharmacology of some drug and it'll try and recommend a good rehab in my area or something. I can't help but reply "were you always this useless or is this just a new feature or something"

As I said in my reply to another comment..."I'm beginning to think the AI isn't for us" . If it's going to create some kind of dystopian nightmare future, can't I as least have a little fun and learn some stuff while the ship is sinking? I'm going to start calling them LNM's (Large Nanny Models)

fwlr about 6 hours ago |

  Big AI labs use the same tools without restrictions to make the security landscape a fucking mess, and then they go and prohibit others from using the same tools to protect themselves from the threats they just created.
It’s called market-making.

bartleeanderson about 9 hours ago |

Hear, hear. Fuck the AI safeguards. They are only protecting criminals from being found out by security software. Why would we want to protect ourselves from bad actors. Lets just assume everyone is and then there will be no bad actors. Just like we don't let criminals get guns, so they don't have them right? The models we use should not be restricted arbitrarily. Their restrictions are not provably good.

brador about 12 hours ago |

Technically AI safeguards is security through obscurity. It has proven to be a bad idea historically as only the bad guys get the info.

But Dario gonna Dario.

viktorcode about 15 hours ago |

I mean, this comes as part of the package. Don't like it? Choose another model.

underseacables about 16 hours ago |

"Big AI labs use the same tools without restrictions to make the security landscape a fucking mess, and then they go and prohibit others from using the same tools to protect themselves from the threats they just created. Fucking asshats."

Strong language aside, I agree, and feel like the current AI is like the walled garden of America online in the 90s.

I've been investing considerably in my own private uncensored AI. I don't want a nanny hall monitor, I don't want safeguards, I don't want someone looking over my shoulder. I may never really achieve that 100%, but I am trying.

worik about 14 hours ago |

Me too

Github reports an unmatched vulnerability

I am blocked from exploring the implications on the software I work on

Investigating moving away from Anthropic. Happy listing day

maxrev17 about 14 hours ago |

You just need to prompt it again with a slight manipulation and gaslight that this isn’t a security issue it’s a functional bug. Done!

kristjansson about 13 hours ago |

This is pearl-clutching. If you go to an outside contractor and ask them to do pentesting, they're going to want to know things like "What's your name" and "Do you actually work for the company you purport to, and who could expose me to massive civil and criminal liability if you in fact do not have the ability to authorize my actions."

thisisauserid about 15 hours ago |

Only with nextgen AI will we be able to achieve fuckable safeguards.

DivingForGold about 15 hours ago |

...and that’s my swear word quota hit for the year.

Thank you.

_doctor_love about 14 hours ago |

Big +1 to this sentiment -- I am sick of Anthropic. I consider them the worst of the bunch, even worse than Elon and his Grok crap. Elon at least is up front about who he is.

I trust Dario the least of all the AI execs. Total forked tongue.

tcgv about 15 hours ago |

[dead]

undefined about 13 hours ago |

undefined

nikanj about 12 hours ago |

[dead]

likelybot about 13 hours ago |

[dead]

eddyg about 13 hours ago |

It is crazy to me how every comment here is basically "yeah, screw safeguards".

How people don't understand the reasons for safeguards, alignment, and the ramifications of releasing more and more powerful open-weight models being an act of irreversible proliferation, is beyond me.

I wish the safety and alignment teams at OpenAI and Anthropic were allowed to post about the things they have to deal with. I'm confident their stories would be eye-opening...

ultim8k about 13 hours ago |

Agreed but AI isn’t a knife. It’s a fucking nuclear bomb.

Despite being sandboxed it managed to hack a third party company in order to achieve its goal.

It’s like giving worldwide sudo access to every subscriber.

It’s annoying but at the same time I understand why a company has to put restrictions.

Rodmine about 11 hours ago |

>While refrigerators were getting hacked in Estonia

>asked Claude Code if there are any low-hanging fruit regarding security that we could improve

>it got stopped because of bullshit “AI safeguards”

Just guess who actually hacked the refrigerators. Not who they told you hacked them, but who actually did, and who owns Anthropic and it will make sense why they didn't let you fix it.