125 points by rossant 4 days ago | 323 comments | View on ycombinator
maxgee 4 days ago |
gwd 4 days ago |
Gave me quite a funny feeling, tripping innocently over the empty shape that they've had to carve out. They wouldn't have spent the effort to do that if it wasn't pretty bad.
nolist_policy 4 days ago |
RIMR 4 days ago |
I deal with a lot of user-generated content, and I have been using OpenAI as a tool to help find offending content. Now I see that if CSAM ever makes it into one of my systems, OpenAI is going to send the authorities after me if their model sees it too, so that's a dealbreaker for that use case, and it was a big one!
It's also wild that they are trying to detect and report AI-generated CSAM. This does absolutely nothing to help protect actual children.
Donald 4 days ago |
eutropia 4 days ago |
When there's an active and urgent case to try and locate and rescue children in these situations, the volume of AI-generated content makes it that much harder to perform forensic investigations on background materials (often how they locate children being abused is by prosaic items like electrical outlets, household goods, etc. in the background of abuse photos and videos) because now they have to rule out whether an image is even real -- a task that often requires close attention and scrutiny to details of traumatizing and disturbing content (sometimes all the more traumatizing due to the 'creative freedom' of AI-generated material)
A model capable of shielding investigators from additional trauma is an excellent thing.
itake 4 days ago |
Pornography is illegal to download or save in Singapore… I wonder how they were able to train a model without the data.
AmazingTurtle 4 days ago |
callc 4 days ago |
I’m thinking about how it used to be extremely impractical to track most of the population, the the enforcement comes after the crime, and there is no pressure to have detected it before it happened and stop it.
How should society/law change in response? Is it purely a give and take between surveillance/right-to-not-be-watch and trying to prevent all bad things from happened before they happen?
mrandish 3 days ago |
avazhi 4 days ago |
Remember when they wanted Apple to scan everybody’s devices locally?
I’m not sure what the solution is to deal with pedophiles but I’m sure it starts with parents monitoring their kids better and keeping young children from being isolated from real world social groups. I’m also sure it doesn’t involve using classifiers on a person’s own device to detect ‘possible’ child porn, which is where this is headed. Who can forget the father who had photos of his sick baby on Google Drive for his paediatrician to see and got reported to the police.
If it really is all about protecting kids then maybe the NCMEC should be lobbying to pass laws banning kids from the internet. Think about how many harms you could reduce instantly (bye bye Instagram, Snapchat, etc).
nonethewiser 3 days ago |
Most people think AI-generated child porn is wrong. But the consent/harm framework can't explain why AI generated child porn is wrong. Either you reach for indirect harms which can just be stipulated away (its not shared so police aren't confused, local model, etc.), or you have to argue in favor of AI generated child porn. In fact we have seen plenty of that here. And this framework has similar problems with necrophilia, bestiality, and adult incest.
A framework that allows for some natural order does not have this issue. If there is some natural order, and right/wrong are not just a function of preferences, you dont have to judge actions solely by effects. From that view, you can say sex has a proper place and purpose, and children have a proper role -- to be protected, raised, and not sexualized. Sexualizing a child violates that order whether or not a real child is ever involved. So the wrongness doesn't require any specific child to be harmed.
I think thats what most people already sense - that the wrongness was never only about harms to children.
eastbound 4 days ago |
stickfigure 4 days ago |
https://developers.openai.com/api/docs/models/omni-moderatio...
kittikitti 4 days ago |
nrmitchi 4 days ago |
Don't get me wrong: these are very necessary tools for the modern internet to function, and I'm sure _drastically_ reduce the review burden on humans, but the skeptic in me does not believe that OpenAI is being entirely altruistic here.
undefined 4 days ago |
andai 4 days ago |
It requires all devices integrated into one single consciousness.
bearjaws 4 days ago |
Could be a great start for "rogue AI"
Jokes aside, I'm glad they are giving tools to help fight this as currently your options are limited because every provider will simply return a content warning if you try to use an LLM as a moderator.
sandboxdevelope 3 days ago |
undefined 4 days ago |
rfgplk 4 days ago |
haickernews 4 days ago |
We asked for further information so we could report the uploader to the NCMEC and remove the content from our systems. They refused to give us any information about this incident, denied our appeal, and have so far, illegally refused our privacy requests for data about our account.
OpenAI does not care about fighting child sexual abuse, this is completely performative.
EDIT: These image uploads are up to 20+ years old, and are being served through Cloudflare with their CSAM scanning product enabled. We've received no notification of any problems from Cloudflare. Something on OpenAI's system thinks that an image we sent to them for classification is CSAM, and they aren't willing to elaborate or communicate about it at all. For reference, our website is https://ytmnd.com