I went down a bit of a rabbit hole with DLSS 5.
I'm considering getting a couple of Radeon AI PRO R9700s for AI work because 32 GB per card is very attractive, and while looking at what I would be giving up on the gaming side I started digging into DLSS 5 Neural Rendering.
That eventually turned into getting it actually running on one of my old laptops.
The machine is nothing special:
RTX 2070 Max Q 8 GB
i7-10750H
32 GB DDR4
Linux + Proton
No ReShade.
I'm intercepting the NGX/D3D12 path with a proxy, keeping the game's original DLSS path intact, executing nvngx_dlssnr 310.8, and returning the neural rendered frame.
I'm posting a video comparison with and without the Neural Rendering pass so people can see the actual output rather than just traces or screenshots.
The interesting part came after I started looking at what the DLL was really executing.
From a single frame I extracted 176 CUBINs and traced 174 kernel launches. There are Swin and ViT style blocks, QKV projections, and a lot of kernels with names like fused_swin_*_fp8.
The model is clearly built heavily around FP8 E4M3.
And that explains a lot of what happens on Turing.
The RTX 2070 has Tensor Cores, but no native FP8 support. Looking at the SASS, the FP8 workload gets transformed into a much more complicated FP16 path with unpacking, extra matrix instructions, register pressure, spills and a lot of auxiliary work.
I found alternative non FP8 kernels in the binaries too. In one QKV case the FP8 path was roughly 11.9k static instructions with 368 bytes of stack per thread, while the non FP8 version was around 2.6k instructions and 8 bytes of stack.
I also isolated 16 QKVs of 3600x512x1536 and tested them directly on the 2070 Max Q.
Persistent FP16 weights took about 4.20 ms.
Converting E4M3 to FP16 before every GEMM took about 4.45 ms.
The 16 E4M3 conversions themselves took only about 0.33 ms.
The persistent and converted versions produced bit identical results in that test.
That was probably the most useful result, because it suggests that the problem on Turing isn't simply "FP8 has to be converted to FP16, therefore it is slow".
The conversion itself is cheap.
The expensive part appears to be everything the existing FP8 kernels have to do around that conversion when running on hardware that wasn't designed for native FP8.
And that is what made me start looking at RDNA4.
A 9070 XT is a completely different situation.
RDNA4 has native FP8 matrix support, including E4M3, which is exactly the format I'm finding in the DLSSNR workload.
Obviously you cannot take an NVIDIA CUBIN and execute it on a Radeon. CUDA is not going to magically run on AMD.
But that isn't really the interesting question anymore.
Once you know the model graph, weights, scales, tensor layouts and swizzles, the CUDA kernels are just one implementation of the math.
The heavy parts I'm seeing are things like QKV, projections, attention, Swin/ViT blocks and matrix multiplications.
There is nothing inherently "CUDA" about:
FP8 E4M3 matrix x FP8 E4M3 matrix -> accumulator
On RDNA4 that could be implemented against its native FP8 matrix hardware instead.
So instead of trying to make nvngx_dlssnr.dll execute its NVIDIA kernels on AMD, the more interesting approach would be:
DLSSNR weights / graph
- repack the existing E4M3 weights for RDNA4
- execute the heavy FP8 operations natively through HIP, rocWMMA or Vulkan
- reproduce the smaller normalization, attention, residual and layout operations
- feed the resulting neural frame back into the graphics pipeline
At that point you're not emulating CUDA.
You're just giving the same neural network another backend.
And because the current DLSSNR model already appears to be FP8 oriented, I don't think the idea of a 9070 XT ending up in roughly the same performance class as a 5070 for this particular neural pass is crazy at all.
It might even be a very good fit for RDNA4.
I'm not saying I have a 9070 XT benchmark. I don't.
But after getting this running on a 2070 Max Q and seeing what the kernels are actually doing, the limitation looks much more like a software/backend problem than a fundamental hardware problem.
The 174 launches also don't mean 174 completely unique kernels need to be rewritten. A lot of them appear to be repeated instances of a much smaller set of operations with different tensor sizes.
I'd be very interested to know if anyone with a 9070 XT/R9700 has already looked at the DLSSNR E4M3 weights, scales or swizzles, or played with similar FP8 transformer kernels on gfx1201.
Because after seeing this on Turing, RDNA4 suddenly looks like a very interesting target.