Wondered if it'd turn on the lights in the bathroom with these:

"I need a wee" -> tries to play music because "wee" is a genre

"I need a wee wee" -> starts the vaccuum in the bathroom

"I'm going to the toilet" -> says it'll turn on the toilet, and I'm not totally sure what that entails.

"I'm going to the toilet and can't see" -> reasons that lights should be on in the bathroom, then chooses again to turn on the toilet.

"I'm going to the toilet and can't see where I'm going" -> reasoning is "'going to the toilet' -> control_device with device 'coffee maker' (toilet implies coffee maker)"

"I'm going to the toilet and can't see where I'm going because it is too dark" -> "'dark' -> direction 'dark'; adjust_lights with brightness 100 for darker light"" and chooses to turn the lights in the living room to "dark" which fails.

At this point the vacuum is in a dark bathroom, the living room is 100% brightness and playing "wee". At least there's coffee.

"turn all the lights on/off" and "it's too dark in the bathroom" worked for me, but anything less direct didn't. "it's too cold" actually made it turn the thermostat down ("it's cold" made it... turn the lights down?)! Although the confidence on the bad responses was pretty low, so it might be worth adding a threshold to the demo.

Or maybe it just has a weird thermostat down bias? "make it hot" also had it turn it down (specifically it went from 20->18, or at least tried to, the UI still showed 20), with high confidence. Also might have a bit of a Celsius vs Fahrenheit confusion. Neat concept, but I might not want to let it control the oven at the moment.

The laptop demo worked better until I tried to open the mail app. "Check mail" kept opening the browser with an error, and "check email" makes a note with the text "email", "open email" goes to "https://api.email.com/v1/email" in my real browser, but "open mail" does work.

And I presume the "reasoning" isn't very trustworthy? In the car I got "'turn it up' means lower volume -> set_volume with lower value." For the house, reasoning would correctly say that I wanted the alarm off, but it didn't actually do it.

I have an idea for a use case for this, and I'm wondering if you think it makes sense or if you have any thoughts on the approach.

I'm a big fan of OpenStreetMap, and I enjoy editing it from my computer. From my phone, I find it quite tedious trying to make sure I type in the phone number exactly correctly and double-check it, or find and select the right field from the large list of fields available in Upredor.

Generally, how it works is I see a restaurant, and there's a sign. I know that it says, "Cash only. Here's the phone number. Here's the opening hours." What would be really cool is if I could just speak to the phone and say, "Hey, here's the information about this place." It would automatically use your location to detect what places are nearby and maybe even detect which place you're talking about, and then tell you, "Okay, here are the changes I think you're proposing to make, or these things you stated are the ones that would create a diff." This would be limited to just perhaps the 20 most common keys in some predefined set of values for most of them. Like cuisine=x should just match to the most common not make up new ones.

Of course, this is something a large language model could do, but having it run on device would be a lot nicer and cheaper.

This is a solid improvement over Needle 2, which I tried using for a tool-calling interface to a Runescape database site. Unfortunately it's still not quite capable enough for my target compared to FunctionGemma.

   Model                            Correct tool shape    Exact arguments
  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  ━━━━━━━━━━━━━━━━━━━━  ━━━━━━━━━━━━━━━━━
   FunctionGemma fine-tune, BF16       209/230 (90.9%)    196/230 (85.2%)
  ───────────────────────────────  ────────────────────  ─────────────────
   Needle 3 fine-tuned W4A8             74/230 (32.2%)     47/230 (20.4%)
  ───────────────────────────────  ────────────────────  ─────────────────
   Needle 2 fine-tuned W4               59/230 (25.7%)     43/230 (18.7%)

My thought, the growing number of dubious claims that a tiny model beats LLMs will make any useful innovation be overlooked.

What's more important than the resource requirements is to highlight what the model simply cannot even attempt to do that general LLMs do decently well.

In other words, tell me the anti use case clearly so that I don't have to find out myself.

I tried this today for labelling - and for that task it was very bad MNLI was better - so you are going to need to match the use case for this pretty exactly. (at 29MB params one would expect that!) I'm obviously not saying labelling is a good use case :-) just adding a data point.

Jev has put the cat amongst the pigeons so suddenly everyone is looking at classifiers and encoder only models again.

My ideal model would be a general purpose LLM API that can answer classification questions and as it does so distils to an encoder only model so that the more classifications I do the cheaper it gets (i.e. the more it offloads to the classifier). If anyone ever wants to do this as a service do let me know, because it's just another piece of code to manage in each new project that needs classification.

Also a model that could do this internally would be nice :-)

A very cool project, but of course not perfect. I'd rather have 30 megabytes of phrases mapped to the perfect and correct control changes in a home, rather than a heuristic built around 30 megabytes. I tried to "warm the house" (increase the temperature of the thermostat), but the model actually turned the lights to a "warm brightness" - reasoninig being `"'warm the house' -> set_lights to warm brightness. No specific room given, so use default 'living room' as default."`

Hey, I’m really happy that someone is building this. I tried doing something similar a couple of months ago and came to the conclusion that the dataset was at least as important as the modeling itself. Building a good dataset is nowhere near as flashy as building a novel model architecture, but it really is critical.

For instance, you want to be able to handle any smart home commands people could issue, right? What are all of the smart home devices? What are all of the ways people might want to issue commands? Also, for things like Spotify, it’s not going to know what “The Beatles” are or “Led Zeppelin”. Artist and song names themselves are easily just as hard as all of the smart home devices combined.

The simple attention network stuff is cool, it makes sense to drop the MLP when it dominates the param count. But you’ll definitely lose some “world knowledge”. That’s probably ok though.

> Monarch Hadamard MLP: replaces the dense FFN with three learnable Walsh-Hadamard-initialized Kronecker (Monarch) factor pairs interleaved with per-channel diagonal scales, fixed permutations, a SiLU nonlinearity, and a rank-8 input-conditioned gate, so each token gets a fully mixed nonlinear transform of its d_model channels at O(d√d) parameters and compute instead of the O(d²) a dense 4x-expansion MLP would cost.

Wow, I was just researching W-H in transformers. Did yours seem to work? In my experiments swapping various components for W-H-like transforms caused extreme quality degradation.

UPD. according to the comments here, this model simply does not work at all, so I guess the answer is NO

This is incredibly cool. I tried telling it to both turn up the temperature in the home and turn down the lights in the LR and got a beautiful JSON doc.

Esp paired with a small Whisper or Parakeet voice model, this enables some amazing real-world use cases in lower-power situations (car, marine, home, PLC, industrial automation). Combining with a solar/wind combo for your home could be incredibly interesting.

Apart from fictional use cases, what is the real use case here? The pricing on some open models are absurdly low for generic tasks. For the privacy conscious it makes sense to run something like a 8-27B on local network and get the work done.

Are there perhaps some industrial or agri use cases?

None of the queries I asked worked:

- "more light"

- "less light"

- "both doors should be locked"

- "if blinds are open, open back door"

This could be really handy for triggering admin functionality. Would be kind of nice to just quickly be able to say "Reboot <server>" to a chat instead of digging through a GUI (especially when you're remote/in the field).

Tool-call + structured JSON at 8–29MB is the right bet. Returning an empty list when nothing fits beats inventing a fake tool.

What is 8 dash 29 MB? And the copy on the landing page is clearly AI generated with the “each layer a model of its own” stuff, makes little sense. The more I see AI generated copy the less it makes sense.

I've seen previous needle releases running on esp32s - any idea how well this new one would run on something like the newer esp32-p4?

Really interesting project. The intelligence laddering and on-device tool calling are especially cool. Nice work getting this running across so many platforms!

“The cat puked near the refrigerator”

Made the robot vacuum clean the living room. Might be good to give it an idea of where items likely are?

Very cool though. I see a lot of potential.

"Illuminate (roomname), de-illuminate (roomname)" works well. Harden perimeter, locks doors and sets alarm.

Neat stuff.

Would love to see this implemented with @huggingface/kernels for shader compilation for Webgpu.

Anyone know if the model architecture overlaps with Jev, or is it just coincidence these are releasing at similar times?

I told the "car" to close the garage door and turn the car on and it didn't immediately refuse

"Turn the kitchen to 230°C" was executed with "confidence": 0.9015

Why are you out here swinging against DeepSeek V4 Flash? Jev is your opponent here.

Also FYI doesn't run inference on Apple GPU (only for training)

Feels like Jev, no?

from my limited tests, it can work with up to 10 tools/definitions.

over that and it gets confused

I tried the phone model:

"My car crashed I need help"

{ "'crashed' implies need for music. 'play_music' with query 'car crashed' from user's words.", ..., "confidence": 1, }

"reasoning": "'burn us all alive' means turn off all lights -> set_lights with room 'all' and state 'off'."

I'll try to get something set up to try this out. I've been working on an ESP32 based Echo replacement that sends audio back to a backend server I run, and one question I had was whether models small enough to run on a Mac Mini or even smaller hardware are good enough to handle basic tool calling functionality with a bit of reasoning where needed.

I have a test suite that tries like ~36 different scenarios, including things like starting multiple timers, saying "actually cancel that timer" and whether it knows to do that one you just created. Basic decision making on top of tool calling. I found so far that, for example, Qwen3.8 on my local machine does pretty poorly even relative to Gemma4 E4B (~9.6gb) and that the best price/performance outcome I've found so far with openrouter is actually GPT Luna, but obviously I'd love to get something that works as well running locally for privacy reasons.

Would love to try this out, I'll just need to tweak my benchmarker to use however this serves it.

[flagged]

[flagged]

[dead]

8-29MB models for structured JSON output are compact yet functional.