Isn't using DLSS 5 on AI images defeating the whole point?
Dlss 5 Isn't an upscaler or a detail enhancer, it's a tool made for real-time games to enhance the realism of lighting and materials.
Ai images don't have this issue, because they're not rasterized 3D polygons.
I think this sortof demonstrates the inverse, that it's not using any information from the 3D models or interframe motion vectors or anything but the last frame, right?
I found it works suprisingly well for AI images as well adding some fine shadows/reflections/details, fixing contrast, etc. Anyway, worth further testing.
so what? i would like to have this in comfy in my tool arsenal.
Comfy nowadays allow to do a variety of things that are not necessarly associated with ai things, llms, audio/img edit, post production color correction and whatsoever.
I use comfy even to download videos from yt just to say.
so THANK YOU OP for this custom node
If the left from each pair is always the DLSS5 version, then I can see better contrast on the last pair. I’m looking forward for future update with more substantial effect . I love DLSS5 in games as it really enhances the visuals in a dramatic way which I prefer. Me partially colorblind I love contrast light and colors. I always do a color correction of all AI images/video as the color/contrast always needs a correction (for me at least).
I tried that mode, but at the moment I can see some flickering in output videos so looks like it requires further testing/development. But should work just fine for still images.
Well that makes sense assuming DLSS gets the game motion vectors as input for its generations. Maybe with some kind of post processed motion vectors (video > get motion vectors > dlss 5 pass) this could get somewhere.
It does cause some flicker in the final output videos because it lacks the vector/movement information that the game would be feeding it - BUT, I've found if you use video editing software like Davinci Resolve it's pretty easy to "deflicker" the video clip DLSS 5 was used on and get a nice result.
I can see this being VERY useful to run over a Turbo lora generated AI video to eliminate the plastic skin and improve details, then deflicker the result in your video editor.
This was run on a 1.0 MP video from Minimax H3 and then run through your DLSS 5 node with temporal_sequencing on, which yes, produced flickering - but then I took it into Davinci Resolve and ran a Deflicker effect over the footage and it solved it completely.
As you can see the skin is MASSIVELY improved. It ficed the contrast, and even the texture in the uniform and chair is improved. It's really impressive in motion.
This really eliminates the plastic AI look of characters in generated videos.
Thanks for the test. I’m thinking about next step is to use some optical flow data from input video and feed DLSS with it.
Anyway, Nvidia is going to officially release DLSS 5 in a couple of days so hope we’ll have more insights.
That would be awesome if you found a way to incorporate that into your Comfyui node. It would probably dramatically improve the result.
Thanks for the node. Even with the extra deflickering step in a video editor, it so dramatically improves the final video quality I think I'll keep it as a part of my workflow.
I have a 3090, so no interest (or ability) to run DLSS5 in games, but this is a great use for me and makes it a real tool.
It’s a path where you can find the node in ComfyUI after installation, just search for “DLSS”.
When you download release .zip it already includes own project .DLLs
Why they can't pack everything together? I don't think Nvidia will care about a small ComfyUI extension, that just makes their technology available in non-gaming ComfyUI for people to toy with it
I can't tell if it's better or worse or what it's doing really. Seems like it just changing lighting and colors a bit and nothing else. So not better or worse just different.
I tested some standalone version of this. It could run videos and images, and it was slow (I have a 3090) but it worked quite well in games captures, the effect was quite noticeable. However for hi details or real images it won't bump it up much, since the details are already there.
I used to be fine with my AMD setup, even if I struggled to run things as fast as NVIDIA users and had to opt for ggufs and smaller quantized models and find work arounds and deal with some increased generation times.... But now... I fully regret having an AMD GPU. For gaming and for AI. (yeah, yeah, AMD ROCm with Linux, blah blah blah. I can run most AI just fin with ROCm on windows, but this? DLSS 5? This is next level shit. Fuck all the haters and fanboi shit... I. Want. This.)
It's better, because you can use it on videos, and control the settings to a degree not possible without a lot of effort on other models.
It's really a great finishing polish on an AI image to make it look real.
Nvidia wasn't lying - there is a lot more going on under the hood than just an AI filter - you can take a generation you already like, but that isn't "real enough" and dial in color temperature, ambient occlusion, and it's got 3 different built in models that are all tiny (like 50MB apiece) - a natural, cinematic, and default, with different settings for each. The stuff it does to eyes and hair, especially making beards look real, is impressive. And textures, etc.
And it takes less than a second to apply - you aren't going to get those speeds with another model pass. And this one keeps the underlying image identical.
It's worth having in your toolkit, for sure. It's less than 200 MB to get it all set-up, and runs in less than a second. What have you got to lose trying it out?
As per last Nvidia news release the main two parameters are tone and structure: “DLSS 5 also offers Structure Intensity and Tone Intensity adjustments. Structure Intensity controls high-frequency details such as ambient occlusion, reflections, and subsurface scattering.
With Tone Intensity, developers can control the low-frequency details such as broader lighting and color response.”
I've tried an equivalent tool, and DLSS 5 ins't really useful on anything that's not a videogame. If you give it real life videos, or offline 3d renders, or AI videos, it does pretty much nothing
its not mystery, file was leaked so no one will officially share it, google it, join discord and get the file from pinned section, if you have anything other then 50xx series you need to go to tools section of that discord and find moded file for older cards.
39
u/Enshitification 1d ago
wink wink