Apparently Grok 4.7 has 40% more weights than Grok 4.6, but the price ($6 output token, $2 input) is the same.

Given that the decrease in their margin and the fact they delayed the release of Grok 4.7 almost two weeks past the original date, XAI must not have been happy with the results for 4.7. And XAI also waited the day before Opus 5.5 is rumored to launch. I imagine Opus 5.5 will blow Grok 4.7 out of the water benchmark wise.

However, I have become skeptical of benchmarks. Grok 4.5 solved some issues setting up a buildroot system that Fable 5 couldn't do. I find the post cursor groks are phenomenal at frontend web development, though Claude is much better at backend ruby.

My favorite part of the new Groks has been how they speak in plain english. I simply cannot stand Claudish. Or even GPT, which doesn't have Claude's ticks but definitely likes to handwave explaining technical concepts. Still, nothing beats Claude 3.5 and 4 with explaining since it seems all models have regressed. I wonder if Grok 4.7 will also regress with English because of all the RL.

At every Grok release I commented on here how much I love Grok. It’s the consumer app and model I like the most, but my sentiment has changed. The usage limits on all the Grok subscriptions are now terrible, maybe because 4.6 and 4.7 (not in the app yet) eat more compote? I’m not sure what happened

My SuperGrok subscription previously easily lasted me through the week even with mild coding through Grok Build. Now when I use the app 1-2 times a day to ask some questions, I’m almost running out by the end of the 7 days. It’s terrible.

I want to keep using Grok but logically it makes no sense for me to keep paying for it on the side when my quota just doesn’t last. I have also no desire to upgrade to Plus with these terrible limits, while previously I would have eaten up a $100/mo Grok plan. Rumors say SuperGrok got heavily nerfed with the SuperGrok Plus introduction, and that sounds about right to me.

I’m sure it’s a great model and I’d love to use it. I hope they get their plans under control and only only focus on Grok Bot.

Initial impressions, Grok 4.6 for me just didn't really hack it for any usecase I tried. I seem to have a floor for my usecaseses (coding and a bunch of agentic workflows) and Sol/Opus are above some kind of intelligence floor.

4.7 is definitely slower & more expensive. It feels kind of like they really had it burn tokens to claw up the benchmarks. But it's not super clear to me whether it's above the line or not. A part of that is that it is so slow that i haven't been making fast progress today with benchmarking it.

Overall, it it gets above my intelligence line its a good release...but you can read the tea leaves and tell the Grok team thinks this was a miss.

Nice to see this release cadence increasing and some continued improvement in quality. I am guessing these models are basically still outcomes of the cursor team integrating with the massive amount of compute they now own: I’d imagine we will see significant step up improvements with grok 5 later this year as the team gets more experienced and confident with larger training deployments. Here’s hoping for another competitive frontier model!

https://tools.simonwillison.net/markdown-svg-renderer?url=ht... - default reasoning level.

Here's reasoning level high: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

For some reason reasoning effort low and medium used similar numbers of tokens, and xhigh used less than high. I think I need to try without OpenRouter in the middle.

UPDATE: I tried again with the xAI API directly: https://tools.simonwillison.net/markdown-svg-renderer?url=ht... - not a great deal of difference between reasoning levels, and this time xhigh and low used the same number of reasoning tokens for some reason.

For comparison here's a fresh run against Grok 4.6: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

It's a real dud IMO, worse than 4.6. I told it to fix a depth-testing issue with a WebGL scene and provided it a screenshot: told me it was fixed but it wasn't. I tell it to try again and it says it's "FOUND THE ROOT CAUSE!" then hasn't fixed it.

I told it to compose an image (putting headgear on top of a head) - kept getting it completely wrong, generating new headgear, getting that wrong and screwing up the scaling.

I told it to diagnose a webhook issue that was happening in production from a local environment and it kept giving me moronic answers like that environment variables weren't set (despite me telling it that the values WERE set in production).

Muggle question... I don't fully understand what moves the needle further for these frontier models.

Is the training data more valuable ? The training process ? The harness ?

I know they are all important but where are they (all the frontier labs) really pushing to get incremental gains?

Not even close to astra. Astra is something else. It is expensive, but uses way fewer tokens do my tasks.

xAI missed its chance, Ball is on Anthropic's court.

Why are they comparing grok 4.7 xhigh to grok 4.6 high?

Unless they produce the same token output on the face of it, it looks like they're trying to cover for 4.7 not having good model perf?

Grok it's really expensive. I'm getting really amazing results using DeepSeek 4.1 Flash for fraction of the price.

It's definitely gotten better at image->html workflows. Here's a test comparing Astra (currently SOTA at this) vs Grok 4.7:

Designs: https://image.non.io/78795662-8bfc-4e14-8d72-3738392aa6b3.we...

Astra's build: https://html.non.io/annui/

Grok's build: https://html.non.io/Annui-grok/

Additional prompt instructions: "Add scrolling clouds behind the statues. Dynamically light the statues based on mouse position. Use diffui to generate the normal maps/depth maps/roughness maps of the objects, and to separate out the assets on to different layers."

Overall I find these models are getting good at following image as a source of instructions, but their refinement of the output varies heavily between the models. Astra's final output feels more polished, has better visual contrast, and the animations between the pages are smoother. Grok also chose to light all of the background elements, which imo overcooks it a bit.

Still though, for the price it's a great starting point.

Looks like they have still problems with caching. Prize is double the other providers for cache hits... which is most of what I do. :/

Significant regression in token efficiency compared to Grok 4.6 suggested by artificialanalysis.ai Intelligence Index Comparisons.

I've put Grok 4.7 on the Redactle LLM benchmarks. It's a bit of a silly eval since it's a puzzle game but it tests omniscience really well.

Grok 4.7 is near the top of the board. A significant improvement over Grok 4.6 but still not as good as Gemini 3.8 Flash which is very cheap and fast too.

https://redactle.net/llm-leaderboard

Good thing they used 5.6 sol instead of Astra for benchmarks, the EEbench one is crazy[1]

[1]https://eebench.org/

It’s a shame this model has such negative political baggage associated with it. It’s the only one I decided not to run in my LLM benchmarks[1].

1 - https://bench.killswitch-lang.org

No doubt xAI has seen rapid progress, but it's been several months of them being "just behind" OpenAI and Anthropic. It seems the gap between just behind the frontier and pushing it is a lot wider than most people thought it was a year ago, and that's why a clear third contender in the frontier model space has yet to materialize.

I've been using 4.6 for some one-off game mods/utilities and it has done very well. "I have a very niche keyboard (Moonlander) and I play this very niche space sim, make me a SVG keyboard cheatsheet for it". Told me to grab keymap.c for the keyboard and inputmap.xml for the game's key bindings, churned for a while, then spit out a pretty good first attempt, along with the python script used to generate it. Spent another hour of back and forth to refine the script, and now it generates great diagrams that will adapt as my keyboard firmware and game bindings evolve: https://files.catbox.moe/x0u76x.svg

Excited to try 4.7. I hope they fixed the "it's not X, it's Y" that showed up in 4.6.

Every Grok release obscures their cache pricing while highlighting their input/output pricing

From their headline comparison:

  Grok: $2/$6 per million
  
  Fable: $10/$50 per million


  What this doesn't say: Grok costs 0.50/M cache read, Fable $0.25/M cache read

Long running agentic workflows are dominated by cache reads.

Just makes Grok sound deceptive, and more importantly, reliant on user's lack of understanding of costs aka predatory (which in turn is more infuriating)

Why Chinese models from Kimi, Deepseek are not added in comparison benchmarks?

Not resetting limits when the model comes out though. Obvious miss. They need a Tibo.

after using cursor grok & trae.ai for several months , grok curor is highly superior results to trae.ai

Does anyone have any experience with Grok's subscription? How does it compare price-wise to the API?

Using AI that Elon has tainted is wild work.

Codex has become my goto tooling. I used to be a Claude Max subscriber, but I was becoming disappointed with the quality of the output from Opus 5. Fable chewed through my usage too quickly to be practical. Moving to a Pro account w/ Codex was a big improvement. Sol had great output, and the usage was more than sufficient for most of my needs. However astra does tend to chew up usage, so when i've done to much of that, and it's became an issue Grok Build has beocme my second go to account. The output especially after the cursor purhcase has become quite good, and the usage has always been very generous.

Either way, the fact that xAI or SpaceXAI or whatever the name is, I can commend the team behind it on their rapid ascent and progress by being close and or on the frontier in several respects.

( why is the x-axis on the first chart in descending order ? )

looks like token efficient/verbosity took a big hit.

Output tokens from Intelligence Index:

- grok 4.6 (xhigh): 97M (for 44 score)

- grok 4.7 (xhigh): 240M (for 46 score)

$2/million inout and $6/million output but I couldn't see any pricing information for cached input tokens?

As anthropic/openai subscription allocations get squeezed you'll see more people using "second rate" closed models like grok. The token allowance with a Cursor subscription is crazy.

Did 4.6 not have an x-high reasoning level? Why are they comparing 4.7 x-high with 4.6 high?

Interesting hamster:

https://aibenchy.com/model/x-ai-grok-4-7-medium/#showcase=dd...

Where is the Astra benchmark?

the AA numbers are generationally bad. double token use (the one thing Grok was good at was low reasoning usage!) to gain 5% in the benchmark score. with reportedly a larger model. maybe it shows gains IRL but wow, I've never seen a new generation model look so underwhelming compared to the last.

as someone who is limited by amazon bedrock support at work (no idea why we got stuck with the worst one) - grok is literally the only budget-ish model option, so nice to see it updated, Sol and Opus are just too rich for my blood. Luna is good but so slow at getting things done (tps wise it's fast)

32 on the omniscience index. Not terrible but far from Astra and fable: https://artificialanalysis.ai/evaluations/omniscience

In my experience Grok especially inside Grok build is pretty solid choice, it’s a no nonsense model and stays on its course. Another surface where I truly enjoy the experience of using Grok model is Grok bot

Congratulations to the team!

If the CursorBench 4.0 score diagram is the headline, I read it as "Grok 4.7 xHigh is almost the same as Fable5.1 on low".

Is there a metric for like... time taken when comparing these two? I see score and cost.

If Fable5.1 can knock it out more quickly on low but Grok4.7 might take twice as long to stumble through a problem (and leave behind a bunch of yucky comments or un-needed extra unit tests), are they really comparable?

Or like... the "quality" of the solution? "It works" versus "it's unmaintainable/very messy/hacky".

undefined

Grok probably hurt itself by building a reputation as the "meme AI", making it hard for some people to take it seriously

I refuse to use Grok. Mostly because of the usual reasons - somehow this high profile AI model seems more disgusting than others and it is in a way impressive.

But also Xai doesn’t seem to care about user experience and long term support.

Any real world experience with Grok Ultra $300 monthly subscription vs Claude Code Max in terms of overall built work mileage, or general token limits?

Pacing the frontier, with an aggressive release cadence. Gotta love the US tech industry.

Dumb question. Are these products really winner-take-all? Why is there such a furious rate of development?

I tried in Omp (Oh-my-pi), and so far it's really problematic.

It will loop in thinking mode ("Let me implement those fixes: Fix 1, Fix 2, Fix 3 .... Fix 80, Fix 81"), ignore the AGENTS.md instructions, corrupt plan files, etc etc... I have 5.6 Sol as advisor/watchdog, and it blocks every turn, I never saw this. Quite a shame, 4.6 wasn't so bad.

What's with the deceptive graph on top? Not including Astra can't have been an oversight, did the model compare poorly to it?

[dead]

undefined

Waiting on simonw "Generate an SVG of a pelican riding a bicycle " benchmark to judge this model

[dead]

[dead]

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

I guess it’s only my opinion but having used grok for personal chat: it’s by far the worst one amongst Claude, ChatGPT and even Deepseek, Gemini etc.

The personality is bland and it doesn’t work nearly as hard or even tries to help.

Is Grok relevant? Who uses it?

Maybe I'm in some kind of bouble but I have never met or talked to anyone who has used Grok.

SpaceX AI releasing "Grok" has to be some of the worst branding I've seen in my lifetime

This thing is DOA. They compared 4.7 xhigh to 4.6 high to make it look like it improved. The reality is pretty bad: https://x.com/chetaslua/status/2102087511367618942

Who uses this trash?

Vals AI just affirm that Grok 4.7 is worse than Grok 4.6 (It ranks #24 on the Vals Index at 54.2%, down 5.0 points from Grok 4.6 (#14, 59.2%))

https://x.com/ValsAI/status/2102086608476590432

Well I "tried it out" I asked it one question, and it gave no answer and said "Sign up to use more!" I don't think I'll be doing that, no.

I can't think of a single dimension grok is winning on (capability, cost, voice), but want to stay open-minded -- anybody want to vouch for its capabilities in any domain?

The CSAM generation model got an upgrade. The sad part about this is that I bet it still generates CSAM. Given that the owner of the company has made a nazi salute in public and thinks CSAM generating models are cool, I don't think they addressed the issue of this generating CSAM.