I strongly dislike CUDA. Once you have allowed that proprietary cr*p into your C++ codebase, it is very hard to get rid, and you end up with code that is either tied to a single vendor or an #ifdef hell, probably both.

The best way to program GPUs is face up to the reality that they are not the same machine as the CPU, write your kernels in separate files, and launch them manually, like in Metal, OpenCL, and D3D12, etc. These days we even have DSLs like Triton that make kernel writing much more ergonomic than anything you would hope to achieve in Rust.

Okay, so, GPUs are taking one more step towards being general purpose massively parallel machines. That's cool.

What would be even cooler though would be for GPU vendors to start giving us the user manual. An I mean the real user manual, that explains how to use their piece of metal when all you have is that piece of metal. That means a precise description of the wire protocols, the data format of the buffers we send to & get from the GPU, the ISA of the cores we have access to, the relevant performance characteristics…

In other words, enough information to write a state-of-the-art driver for any OS. That would be cool.

Since NVIDIA owns huggingface now and huggingface has the excellent Candle [1] crate for inference on Rust, this seems like a good step towards nice native Rust kernels.

[1] https://github.com/huggingface/candle

Really exciting but it reads like Claude instead of what Nvidia posts have generally been like in the past. I don't need nor want my tech blogs to sound like a young adult novel.

The momentum behind rust seems absolutely unstoppable at the moment, in the light of this, the adoption of Rust into the Linux kernel, and the adoption of for formally verified software by Amazon and Microsoft.

How does this compare to vectorware? (https://www.vectorware.com/blog/)

I think it would be much nicer, although unrealistic at the moment given the number of combinations of GPU vendors and variety of hardware, to directly target the underneath GPU ISA machine code.

Since I can write a simple compiler to target x64 machine code, it should be possible to write one to target my GPU.

Though, I'm certain that vendor lock is probably more profitable for them.

Anyone know when Rust's std::autodiff will become stable? Assuming this Rust support expands to other GPU vendors, autograd will probably be the only reason to use Slang instead of Rust anymore.

Interesting direction from Nvidia. Anything that makes writing reliable GPU code less painful is definitely a good thing.

Worth mentioning that Nvidia open sourced CUDA Tile IR ~8 months ago. And yes the code is open source too. https://news.ycombinator.com/item?id=46330732

Not a cuda programmer, but since they’re making a new API, why would they already make it inconsistent at start? :-/ I’m referring to the examples a,b,c vs z,x,y (different ordering of output elements)

First of all, this is a pre-1.0 release that requires a nightly Rust compiler (if you choose the SIMT track with cuda-oxide) so that one is going to be unstable software.

Secondly, When an issue occurs with a kernel or you want to write your own custom kernel in Rust, now we need to diagnose if the problem came from either cuda-oxide (SIMT), Rust's side, CUDA or Tile (If you decide to choose the Tile track).

Another dependency into the list and course everything is open source except CUDA itself. So any issue that happens on the CUDA level, you are forced to wait for them to fix it.

Rust/C/C++ all have a similar memory model, they were designed for a linear memory model. a `float` does not carry the provenance (a float out of cudaMalloc or a malloc look the same to the rest of the program). This is the fundamental problem that very few (e.g., Vx vxlang.org) are trying to address.

I read this the other day - definitely think it is the right direction Nvidia is taking.

Thank you NVIDIA - for once (not twice though - you've given us nothing but despair for Linux+GPU).

To everyone skeptical of people saying that it's better to separate CPU and GPU code into separate files, riddle me this:

With this system, how do we specify whether dependencies are needed to be compiled for the CPU, GPU, or both?

----

Technically I don't know really care whether it's multiple files or one, I just want to make sure we are not reinventing a shittier version of CFG. What they are providing looks to me like:

1. explicitly annotate some things as `cfg(GPU)` or ungated (both)

2. unlabeled means annotated `cfg(not(GPU))` by definitely

Put this way, this has nothing to do with GPUs, and just has to do with creating some crate-local CFG shorthanded. Great! Let's do that first, get a solid foundation, and then come back to whatever is remaining for CUDA Rust.

Does this weld Rust to CUDA? Can we use the Rust code to run on other archs?

what this article tells me is that no one at Nvidia actually cares about this project whatsoever. otherwise, they would have had a person actually write the announcement.

Does this mean I can write shaders in Rust for use with WGPU or Vulkan?

I'm looking forward to trying these when they stabilize! I currently use WGPU for graphics, and cudarc for CUDA.

Note: Cuda-oxide is similar to Cudarc's host component, but uses a rust-style kernel dialect. Advantage: Share structs between host and device. Disadvantage: Trading standard Cuda kernels for a new, WIP dialect.

I haven't tried the tile API yet; looking forward to it.

The last time I checked, Cuda Oxide was Linux only, and required Async; these are why I haven't tried it yet.

I've been enjoying writing Julia code and then having it run on the GPU via the packages at https://juliagpu.org

> The launch is checked rather than trusted.

Damn even Nvidia is putting out fully Claude-written articles.

My understanding is not so deep regarding GPU programming or Rust... Does this mean anything regarding Nvidia GPUs and WebAssembly / WebGPU?

In this age of LLM written everything which has softly killed my motivation for learning Rust somewhat, this has revived my interest if not only for the fact the LLMs haven't yet been trained on this yet!

I wonder how many parallels there are between CUDA's Tile abstraction and that of Metal.

Will it then be possible to query TJunc hotspot temps on linux?

The recent circular moves that Nvidia is making is designed to wrap things around them, anything to keep the AI model party going.

Been waiting for something like this. CUDA C++ is a pain; Rust's safety for kernel programming could be a game changer.

undefined

yikes, I prefer python taichi similar to

@fast def calc(x,y):pass

So now every already written kernel can be re-written in Rust and have competitive performance to the cpp version?

If so, that's really big.

And to add the Next natural strp - custom codegen for simulating gpu compute and memory without Nvidia gpu.

Thank you Nvidia !! You're in the right path.

Nvidia only? Typical.

This is more promising: https://github.com/Rust-GPU/rust-gpu/

Do it in Go and I’m interested

hmmm interesting

AI slop article, how can I trust on this?

[dead]

[dead]

[dead]

[dead]

[dead]

[dead]

The world is unsafe

Rust for GPU programming? My CUDA debugging sessions just got a whole lot less painful, hopefully.

Makes me sad that Go doesn't get love. I feel like Go is perfect for LLMs.