Hacker news

  • Top
  • New
  • Past
  • Ask
  • Show
  • Jobs

Apple has new Mac app coming to make running local AI models easy (https://9to5mac.com)

580 points by linolevan 10 days ago | 569 comments | View on ycombinator

pen2l 10 days ago |

Apple, to me, is the winner in the end of this AI chapter.

Distillation works: we can get models leaner, have better performance with lesser and lesser parameters, more efficient, running locally. Apple gets this and we'll see the result with M7 Max/ultra chips and I reckon this will be one of the titular apps when M7 line of chips are introduced.

I guess it explains the haste Anthropic and OpenAI are in to IPO, cash out and leave the the retail investor holding the bag.

simonw 9 days ago |

"Harbor is currently in a private preview under Apple’s Claris brand"

There's a name I haven't heard in a long time! Claris (always an Apple subsidiary, since being founded in 1987) renamed themselves to FileMaker in 1998, but apparently they rebranded back to Claris in 2019.

Looks like they've mostly just released annual upgrades to FileMaker for the past decade, but they did add MCP to it last year: https://www.claris.com/blog/2025/why-mcp-matters-for-claris-...

jph 9 days ago |

If you ask who would pay for Apple local models, I would. A Mac Studio high end 512GB costs ~$1000/month for 3 years versus my current run rate of ~$1000/month for five devs for Claude Ultra (or ~$10k/week for Claude Sonnet if I paid API rates rather than Ultra).

If/when Anthropic changes to pure API pricing, or for API-heavy projects, that's when I'd switch to more local models. Of course there are tradeoffs for speed, quality, upgradeability, etc.

I currently run a Mac with 128GB and qwen3.8:27b-mlx. It hits the heat limit of the Mac when working on a 200k LOC codebase for security auditing and hardening for many hours at a time. This isn't as strong/fast as Sonnet, to be sure, yet the cost is only $6K one-time plus the usual electricity, maintenance time, etc.

thibsrn 9 days ago |

I have been using llama.app, which is the desktop interface built on top of llama.cpp by the same people. It is an alternative to Ollama that might interest people who want to switch given Ollama's reputation and previous decisions. Llama.app has a really clean design, an elegant and intuitive interface. But it's good to get alternatives as well, if Apple also gets into the game.

https://llama.app/

iantpe 9 days ago |

What I need most right now is a model like TypeSafe's Jev that can be built directly into the computer.

Jev allows for numerous small judgments: Is this email important? Which program should handle this request? Is this operation risky? There's no need for large LLMs to slowly generate long paragraphs of text; all you need is a fast, structured result that returns the message.

Having a local Jev-like model in the operating system for quick decision-making, and then calling the larger model when deeper thinking is needed, would be very useful.

gehsty 9 days ago |

As models improve and hardware becomes more capable Apple are definitely well positioned.

I am super interested to see what the Apple silicon team can deliver when their full attention is turned to local LLM acceleration.

jedbrooke 9 days ago |

> Cluster for scale: “Connect two or more Thunderbolt 5-capable Mac computers and Harbor pools their unified memory into a single inference resource. Run models that exceed any single computer’s capacity, or serve more concurrent requests across your cluster.”

This seems like the main way it would differentiate itself over existing solutions like LMStudio etc.

jwr 9 days ago |

I've been running local models under ollama, then lm-studio, and now mtplx (command-line, from supervisord). I'd welcome a vendor-supported solution, if it doesn't turn into bloatware (like lm-studio did).

apparent 9 days ago |

This is what I've wanted for the last few years. I have played around with local models and had some success, but I don't want to deal with the hassle of setting up, updating, etc.

If all of this were easy, I would happily invest in new dedicated hardware to run it. This could be a full-blown Mac or it could be a souped up HomePod-type device, loaded with RAM. Better yet, make it a revitalized Matter-enabled Airport!

gizmodo59 9 days ago |

I really want capable models local but it’s impossible for me to afford the hardware. If there is a magical way to run glm 5.3 without quantization at 80-100tps with a total spend of less than 10K I’d be happy but that’s impossible now.

KolibriFly 9 days ago |

Calculate the total cost of two new Mac Studios with max memory and cables. For that budget you could burn tokens on cloud APIs for a long time or build a dedicated server on open hardware with clear server buses

_pdp_ 9 days ago |

That will be really cool. It is already possible though through Apple Intelligence. There is an open source app that already leverages that.

https://github.com/pdparchitect/noodle

mgiampapa 10 days ago |

I love that they are bringing Claris back for this. Moof!

crossroadsguy 9 days ago |

I hope it will be optional. This is too much to ask for on Apple platform. I recently had to install a 3rd party script to get rid of Apple's laregely useless AI stuff from my 5 year old mac and save more than a few GBs which I badly needed.

Running local models is already extremely easy.

I have a suspicion this app will be a way to add a too many gigabytes to the mac and of course like the Chess.app there will be no way to get rid of it. There are excellent cli apps/GUI apps like this https://github.com/ddalcu/mlx-serve - they are both cli and gui – and even its gui surprisingly nimble (I use this, but I am not related to the project). I am yet to try this https://github.com/ggml-org/Llama-macOS but I've seen only good reviews of it.

I also do not think Apple is doing it for the users who won't explore around. I can't wait to get off this operating system. Next time a friend travels to the continent I will explore getting a Framework laptop via them.

MBCook 10 days ago |

But you’re still in for a ton of pain if you want to run them headless right?

They desperately need to fix that.

brainless 9 days ago |

I really do not understand the value of running large models locally. I am absolutely fond of local models but for specific tasks. The agents need to do a lot of work in this case: orchestration, big or small model split of planning and work, split into small tasks, merging, etc.

LLMs with <30B parameter models seem like the sweet spot. I would rather focus on multiple tiny LLMs that are tuned to specific tasks (SQL queries, finance/accounting, backend/frontend dev, etc.).

For larger models, a $10-40 ish Neuralwatt/Opencode Go or any similar subscription goes so far for the average user that it is ridiculous. We are still in the "youth" era of agents. There are many patterns that work well with smaller LLMs. Or with a big-small LLM combination.

My context:

I own the tiny 16GB M4 Mac Mini and I use LLMs (coding agents) to build software and a few of my products use LLMs directly. Full-time user of LLMs, do not read or write code manually. I run the tiny Qwen series (<10B params) sometimes but a couple $20 subscriptions (Claude, Neuralwatt) take care of my full-time agent usage.

dwaite 8 days ago |

By "Apple" they mean "Claris", a wholly owned subsidiary which was formerly Filemaker, Inc.

So this is interesting, but it will likely be an independent product rather than part of the core operating system.

api 9 days ago |

What I’d like Apple Intelligence to be is this plus a good local app to chat with models plus an iCloud version of OpenRouter with privacy screening features.

Even better would be a peer to peer model hosting hardware pooling system but that’s probably too much to ask and would expose Apple to liability issues in some jurisdictions.

If Apple were willing to eat some margin on RAM they could use the present RAM bubble to gain ground in the hardware market too. Financially you could call it buying market share.

mark_l_watson 9 days ago |

I am on the waiting list. I have a new 64G mac mini and an older 32G model. The directions for setting up a distributed inference system aren’t too gnarly but I haven’t bothered. I will when I get access to Harbor.

I think it is a great joy to run most things locally, and still occasionally use the best commercial models when local will not cut it.

quaddoggy 9 days ago |

After I think about it, this makes sense that Claris would tackle such a project under the auspices of Apple. They're a subsidiary so they can monkey around with this stuff without having to adhere to the draconian everyday operating restrictions of "Apple proper". Good on them if they can pivot into something that hews to AI.

ontouchstart 9 days ago |

I am already doing 100% offline inferences and AI assisted coding with bespoke software developed with local models.

In my case it reached the recursive self improvement stage a few weeks ago. It feels like the early days of personal computing in the era of DOS and Apple II.

The hardware is already there. The only limit is product imagination.

kenferry 9 days ago |

"Harbor is currently in a private preview under Apple’s Claris brand, but it seems likely that we’ll see it launch to the public in the weeks and months ahead."

Claris?? Ok, this sounds like it's just someone's personal project. Something like this that was actually going to ship would be in a different org.

liuliu 9 days ago |

https://getlocalcode.app/ runs local AI models (larger ones of course, DeepSeek 4.1 Flash 1017) easy. You can download and try it now. The problem is elsewhere: you cannot find M5 Ultra that you can purchase and set it up today.

throwaway_new_1 9 days ago |

The Mac Pro was discontinued at the wrong time. Or maybe just introduced at the wrong time.

nkotov 9 days ago |

I built out Infinity (https://infinitychat.app) for myself which does something similar. One click to download and run, compare models etc.

pentakkusu 9 days ago |

As a recent M6 Mac mini owner, HELL TO THE YEAHHH BABYYYYYYY

I use oMLX btw, on 32GB URAM, running Hermes with Qwen 3.6 @ 128k context window.

I realize I am a smalltimer but for general day to day sensitive agentic stuff, this is the entry point.

caycep 8 days ago |

Also I keep forgetting about Claris; I'm kind of sentimental about it and I hope maybe this gives them a shot in the arm.

zf00002 9 days ago |

You can chat with the local model that ships with macos on your M1+ mac right now. There's a few servers on github that make it easy.

avazhi 8 days ago |

Can I natively remove AI bloatware and models from running on my Mac in Golden Gate yet?

nier 9 days ago |

(1) All the headaches of maintaining FileMaker Server on Apple hardware, (2) multiplied with the additional requirement to cluster devices, (3) but without users accustomed to the silliness of misusing consumer devices running a consumer operating system for a service that must be reliable, and (4) an Apple subsidiary unable to bring about meaningful change.

azuanrb 9 days ago |

All this talk about Apple being the long-term winner in AI is putting an awful lot of faith in Apple. Siri is still about 1/10th as useful as ChatGPT on a good day.

It’s the “year of the Linux desktop” meme all over again. Any day now.

- From a happy iPhone Duo user

D13Fd 9 days ago |

What would make running local models easy is being able to buy a Mac Studio without a 6-month delay :-(

ourcat 7 days ago |

Hopefully, one of those local models will be their own. The one that they're currently forcing people to download. 10Gb of stuff that they absolutely might not want.

rcarmo 9 days ago |

Sherlocking Exo, then--although that will work in a regular LAN as well. I should have finished https://rcarmo.github.io/projects/go-exotic...

w10-1 10 days ago |

OMG after 25 years, Claris gets something beyond FileMaker (among the most legacy software still alive)? Claris would not have the engineering talent, so why would Apple use them as a vector?

glassresearch 7 days ago |

I am not encouraged by Apple's forays thus far into local models.

chews 9 days ago |

Deepseek has found safe harbor!

fridder 9 days ago |

It will be interesting to see how this compares to things like oMLX

ggcr 9 days ago |

So many projects, start-ups and frameworks called Harbor already

lenerdenator 9 days ago |

I'd absolutely use this; doubly so if there was a way to route traffic from your iPhone to it through iCloud. That might even get me to buy a second Mac just for server use.

DerDerDaIst 9 days ago |

they should add a sandbox for the agentic part as well.

so you can control and monitor what the whole setup is doing inside and outside the box.

nxobject 9 days ago |

Developers! Developers! Developers! Developers!

wewewedxfgdf 9 days ago |

AMD needs Apple to kick it in the pants and make it realise they need to ACTUALLY compete instead of trying to just copy Nvidia.

wingi 9 days ago |

what is the difference to ollama, lama.cpp ... ?

amelius 9 days ago |

Great, except with current DRAM prices nobody can afford it.

sleight42 9 days ago |

An Apple M5 Ultra running an app like that? Because Apple hardware is a monolith, unlike the PC space, "Harbor" will be configured out of the box with the ideal settings for many well-known open-weight models.

... Looks like I picked the wrong week to buy parts for a linux-based inference box.

axnli 9 days ago |

[flagged]

heyeddi 9 days ago |

[flagged]

dcdevito 8 days ago |

[dead]

ankur_misra 9 days ago |

[flagged]

geberele 8 days ago |

[flagged]

ravivarmat 9 days ago |

[dead]

simli123 9 days ago |

[dead]

ilkerguldali 10 days ago |

[dead]

tonymet 9 days ago |

Why so much investment in local models when the remote ones are so cheap ? Everyone was fine with remote compute & storage before AI, now they balk?

rickdg 9 days ago |

MacOS is a bad server.

q4y2u35uk 10 days ago |

Hopefully it won't be another preinstalled bloatware app like Image Playground. macOS enshittification has been greatly accelerating, even giving Win 11 Home a run for its money.

myphpgothacked 9 days ago |

[flagged]