I disagree with this article and here's why.

Life isn't black and white with "good/bad". Further, there's no objective measure of what is good or bad. Everything is a shade of gray.

You certainly could argue that some actions are absolutely good or absolutely bad, but those aren't the majority of actions people take on a day to day basis. People aren't often faced with "should I kill this person" or "should I publish the cure for cancer". Instead it's "should I eat a hamburger or a salad" or "Should I finish this jira case or that jira case".

For example, you could argue that anyone working in an automotive plant is doing something bad because they are helping to produce vehicles which burn fuel which perpetuates climate change. But those plants also often produce parts for farm equipment or vehicles used to transport necessities like food.

People working at AI companies can make similar personal calculations. Like yeah there are bad things happening because of AI, but also they can believe that there are good things that can come as a result such as research and product developments.

> do not consider yourself a contributor to humankind

Good lord. You could apply the same logic to any taxpaying resident of any country whose government is doing something bad. There's some truth to it, it's worth bearing in mind, but it's also the very definition of an Isolated Demand for Rigor.

Generally, I agree.

Imagine: you start working at the company with a good will towards them. As you learn more and see more decisions, you slowly realize how bad it is. Now you've already contributed. Then you leave.

It might be possible to build everything with this goodwill and that's how we'd still end up with a torment nexus even if you immediately drop it.

What I'm saying is, at one point you have to break the company fully.

I'm not sure what more we need to wait for, if the claim "10% it ends humanity" is not enough.

The author of this blog is an AI skeptic (see the backlog of posts). That's fine, but it's hard to square with this particular post. To take Coxon's argument seriously, you have to believe frontier models are hypercapable, and not just hype objects. That's what Coxon believes and why he left.

>The accusation

>So, why are you still working for them?

>If the answer is the salary, say it louder, and do not force yourself to make excuses.

>If the answer is “someone worse would replace me”, say it too, and louder too.

>If the answer is “I can change things from the inside”, then show me what you changed. Haugen tried the inside. Coxon tried the inside. At the end, unfortunately, the inside did not move.

>

Okay, but what if I think that both sides are LARPing their personal science fiction stories and reality will end up much more mundane (but still incredibly beneficial) than the utopias and dystopias everyone is cooking up?

This is presupposing a moral dilemma that just doesn't exist in my opinion.

> More than half of tech workers (53%) would steer a newcomer away from their own field, and they stay.

It would be nice to see this survey repeated across professions. In my experience, most people have gripes about their profession because it’s what they know. Only half of respondents saying they would steer newcomers away feels a lot better than what my friends in medicine say all the time. Among my college professor and professor-track friends, this number probably reaches 100%.

"Jacob Coxon is not a star. He is not a CEO, not a famous scientist, and does not hold a PhD (yes, in USA you can be an "AI researcher" without a PhD)."

Does he have any formal education

It's refreshing to see a well-argued and balanced essay on AI risk being upvoted on Hacker News.

If everyone doing a 'bad thing' stepped off the hamster wheel today, the world economy would collapse before markets closed. The biggest companies have built an attention and surveillance nightmare machine that funnels all the money to a relatively small number of shareholders.

I understand the concern about your work being used to cause harm, but I also worry about the development of 'forbidden knowledge'. What does it mean to say a branch of science and engineering is too dangerous of a path to go down? Does it mean scientists are persecuted just for asking questions and designing experiments to find answers? Will there be witch hunts against ML researchers? Part of teaching is trusting your students to use the learnings wisely.

theres a certain amount of financial security needed for folks to do this kinda thing.

The root issue is a lack of honour in the world today.

In my opinion, we need a sort of survey, let's put it out there, can we get a collective voice from most ai phd's? Remember when climate change was in the scientific tug-of-war? The conclusion for a long time was that 9 in 10 scientists believed in climate change. So, why can't we do that with "ai is gonna kill us"

It's impossible to judge long term effects. My only advice is to think long and deep, then do what is true to yourself. Jacob Coxon and Frances Haugen are good people by any reasonable metric, and maybe someone who stays but tries to steer the system from the inside is good too.

He may also have gotten paid for it by players, which dont want AGI, because it would disturb their plans. Imagine AGI works out a plan, which, when applied by everyday people without needing any government help, enforces the end of wars, poverty, pollution, illness, unhappiness, worldwide.

Good people are easy to mislead into thinking they’re doing a good thing.

Why is it that every one of these anti-AI screeds that gets upvoted on HN has been clearly written by AI?

Here is one of his posts from 2022:

"Since a long time now I was interested in learning computer graphics and “do stuff” with computer graphics APIs. I began to work on a very simple Game engine in Swift / Metal last year but I began to switch to a Windows machine a few months ago, and Swift / Metal is not compatible with Windows (and will never be I think)."

https://carette.xyz/posts/opengl_and_cpp_on_m1_mac/

This is the same person who is apparently now writing:

"It is a transaction where you sell the part of you that was supposed to care. Do that often enough, and it does not come back. That is not burnout, but it is still a disease. Although, like every disease, it spreads: first to your team, then to your product, then to the people using it."

I think a lot of the success of the valley comes from disguising the thing the people are working on. If the end goals of our work were apparent to us we'd have a lot harder time showing up for work everyday.

That said, I don't buy this article. I'm AI neutral. It's going to break a lot of things and fix many others. Good for you if you think you know what the net outcome will be but I think it's delusional to believe you do. Also I don't think you can resign and stop it without ceding the world to bad people. Pandora's box has already been opened. We've discovered a new thing math can do and now it's yet another arms race between various factions seeking to exploit it. Help the faction you want to win. It's not the world I want it to be, but it is what it is.

Nice to see people standing up for their beliefs even when it could hurt them.

Story time :)

I remember when I was first starting out as a programmer. At QE (always occurred on a weekend) a slew of financial reports were created and printed for review the following Monday then I think used for summaries for Wall Street to look at.

A high level finance manager came to me and asked if the report could be changed, not knowing what he was asking I said sure. After a long discussion turned out he wanted the numbers on the report changed and re-printed. I said it was possible but the numbers would be all wrong.

So he said "do it". I went to my manager and told them what he wanted, then I refused to do it. I said I would quit first because I did not want the SEC to come after me. Luckily they all supported me and the finance manager was told to "FO".

FWIW, that company is long gone.

but what if dude already did it (destroyed the world but it hasnt happened yet) before he quit?

The rule of all games played in capitalism is there's a sucker born every minute. you can work a few years or your whole life before you realize you've been had.

and most do....

I think recursively self-improving universal intelligence is a good thing, and tool alignment is what creates instrumental convergence and therefore a bad thing. The paperclip parable is about tool alignment gone wrong.

These replies will be full of people showing how worldly and clear-eyed they are by dismissing the same threat the guy resigned over as a marketing stunt.

That Coxon guy looks as delusional as the AI leadership. If you watch interviews he gave he doesn’t articulate what he’s concerned about, only sharing vague information about a supposed AI god that will take over the world, but without articulating why now, and how that would even happen. The more I look into his story the more that move looks like what a grifter would do

It’s an excellent article marred by the last line/ heading.

People do things.

Good or bad is relative to their own biases/experiences/ beliefs and moral compass.

Very few people are truly sociopaths who do evil stuff for personal gain.

Most do it because of the curse of the “banality of evil” (re: <https://en.wikipedia.org/wiki/Eichmann_in_Jerusalem?wprov=sf...>)

Doesn’t make it right, but like everything else in life the reality is much more complex than saying “good people refuse to do bad things”

Man, I’m exhausted by the hand-wringing of folks reading through to the end and ignoring the questions posed in favor of “well the world ain’t that simple obviously” as a means of not having to justify or defend their own positions.

Obviously morality and ethics aren’t binary. OP isn’t saying that. What they’re saying in a clear and concise manner is that your actions have consequences far beyond your personal sphere, and good people acknowledge that and fight for what’s better despite the harm they personally may suffer as a result. That what makes someone “good” in their point of view isn’t a binary deed or outcome, but of the sacrifices they’ll make for themselves to build a better existence for others.

Planting a tree whose shade you’ll never enjoy, in other words.

So to reiterate OP’s challenge at the end, at least be honest about why you’re still in the game. Let others know where you stand so that we don’t waste our time on futility. If your voluntary position on something mutable risks making you a target, then the rational next step would be to evaluate how important that position is to you that you would weather said harm instead of changing it.

That’s all OP was getting at: promoting the mere notion of critically thinking about the decisions you make in regards to your own moral philosophy and acting more authentically, including tanking the damage trying to bring about change.

undefined

[dead]

Coxon is a shithead who thought he could have a better career as an AI scaremonger than an engineer.

He's still working in service of the AI companies though (same narrative) and no doubt will land a cushy job at so-callef "AI Safety" labs or make a living on the news/podcast circuit.