I worked on an early draft of the OpenAI misalignment reporting framework, and my immediate coworkers are the authors behind the first batch of reports that have come out through this process.

The primary reason for putting this process in place was to allow more transparency. There was a sense that the DseWiki incident should have been disclosed, before outside researchers had to disclose it for us.

There was no meta gaming about regulation that I was aware of. I would personally be excited if there were regulation mandating this disclosure process, which allows anyone at the company to raise an issue and shepherd it through the reporting process.

"We built a program and this program performed destructive actions. We need regulatory framework"

Make that make sense?

OpenAI and Anthropic should be nationalized. I find it difficult to trust Sam Altman or Dario Amodei.

> The company released six internal case studies where none of the issues affected real users.

This is the most interesting point to me. What are they not releasing that has affected real users? We’ve seen some individual reports from people (eg AI wiped my HD).

Am I the only one who dislikes the term "misalignment"?

On one front it implies the model has a "mind of its own" (whether it does or not is besides the point). Why do we perceive human judgement as somehow more trustworthy than that of a model? I feel like I've experienced human misalignment somewhat regularly in life.

On another front I'm failing to conceptualize how alignment can be objective. How can you measure alignment when reasonable people will disagree whether actions are aligned or not? All the time I see humans operating in different zones of alignment with whatever goal they're trying to achieve and I suspect it's even a feature (socially) that we have people calibrated differently.

Do I want a model that's trying to push the boundaries of scientific understanding to be aligned strictly with the current dogmatic thinking? Or do I want it to "get creative" and think outside the box?

It seems to me more like accountability is the issue.

Discussion: https://news.ycombinator.com/item?id=49737503

This entire situation is such a huge PR disaster that you have to wonder what the initial expectations from these founders were about a decade ago.

What's with all this make believe delusional bullshit? The LLM is not gonna wake up and become AI. Get real guys.

[edit] to be clear, I believe regulation is necessary and urgently important for the software engineering field. The damage being done by the unregulated psychological experiments run by social media and adtech companies is awful and should be curtailed. Engineers should be held personally, professionally, and legally liable for what they produce. But we don't need to invent imaginary bogeymen to do it.

i think the lower bound on the end state is there cant be opaque reasonibg steps ever.

We have no reason to believe a word they say. We know they're incentivized to lie about "dangers" and act alarmist, Anthropic has been doing it for years now. Aside from that, just because you can burn down a village with fire doesn't mean fire is the devil. Maybe they should consider acting responsibly.

[dead]