Deep Breath: Okay, Let’s Talk About That Controversial DLSS 5 Demo

Deep Breath: Okay, Let’s Talk About That Controversial DLSS 5 Demo from the here-comes-the-comments dept The polarization over any and all uses of artificial intelligence and machine learning continues. And, to be clear, I very much understand why this is all so controversial. Any new technology that has the chance to be transformative will also necessarily be disruptive and that causes fear. Fear that is not entirely unfounded, no matter your other opinions on the matter. If that’s you, cool, I get it. I’ll start this off by pointing to the latest edition of the Techdirt podcast in which both Mike and Karl engaged in a fantastic discussion about the use of AI. I’ve listened to it twice now; it’s that good. And, while I found myself arguing out loud with the both of them at certain points during the podcast, despite the fact that neither of them could hear my retorts, it presents a grounded, often nuanced conversation, which we need much more of in this space. And now, in what might be a subconscious attempt by this writer to commit suicide by comments section, let’s talk about that controversial demo of NVIDIA’s forthcoming DLSS 5 technology. What DLSS 5 does compared with previous versions of the technology is indeed new, but what is not new is the introduction of AI and machine learning into the equation. DLSS 2 and 3 had that already, in the form of pixel reconstruction and frame generation. DLSS 5, however, introduced what is being labeled as “neural rendering”, which uses machine learning to alter the lighting and detailed appearances in environments and, most importantly, character rendering over the engine on top of the 2D image output. Here’s the video demo that got everyone talking. The backlash to the video was wide, immediate, and furious. There was a great deal of talk about the alteration of artistic intent, about whether this changed what the original developers were attempting to portray when they created the games, and, of course, industry jobs. I want to talk about the major complaint pillars seen across many outlets below, but this backlash also supposedly came with death threats foisted upon NVIDIA employees. I would very much hope we could all at least agree that any threats of that nature are completely inappropriate and absurd. With that, here is what I’ve seen in the backlash and what I’d want to say about it. Get your damned AI out of my games! Perhaps not the most common pushback I saw in all of this, but a very common one. And a silly one, too. As I mentioned above, DLSS versions already used some version of AI and machine learning. That isn’t new. How it’s applied is certainly new, but that isn’t the same as the demand to keep AI entirely out of the video game industry. And if that’s where you are, go ahead and shake your fist at the clouds in the sky. AI is a tool and, as I’ve now said repeatedly, the conversation we should be having is how it’s used in gaming, not if it’s used. That’s because its use is largely a foregone conclusion and it is an open question as to whether its use will be a net benefit or negative overall to the industry. Dogmatic purists on AI have a stance that is understandable, but also untenable. We’re too far down this road to turn around and go home. And if the tech were able to lower the barriers of entry to the gaming industry, acting as the fertilizer that allows a thousand indie studios to sprout roots, would that really be so bad for the gaming ecosystem? I can appreciate the purists’ point of view. I really can. I just don’t see where they have a place in the conversation when it comes to gaming. It overrides artistic intent! Does it? If it did, then hell yes that’s bad. But if it doesn’t, then this concern goes away entirely. DLSS 5 is built with options and customizable sliders for game developers. That’s really, really important here. At the macro level, a developer that has decided to use DLSS 5, or decided and customized how it’s used in their games, is exercising consent over their products. That should be obvious. But then we get into really interesting questions of art, the actual artist, and the ownership of that art, because those last two are very different things. As Digital Foundry outlines: It may even raise consent and other questions surrounding artistic integrity. On site and witnessing the demos in motion, concerns about this seemed less of a problem when the games we saw had been signed off by the studios that made them – the contentious assets we’ve seen, likewise. Nothing from the DLSS 5 reveal released by Nvidia has not been approved by the studios that own those games. But perhaps the issue isn’t just about specific approvals by specific developers on agreed DLSS 5 integrations, but rather the whole concept of a GPU reinterpreting game visuals according to a neural model that has its own ideas about what photo-realism should look like. While we’ve seen endorsements from Bethesda’s Todd Howard and Capcom’s Jun Takeuchi, to what extent does that consent apply to the entire development team and other artists associated with the production? And by extension, there is also the question of whether now is the right time to launch DLSS 5 at a time when the games industry is under enormous pressure, jobs are on the line and cost-cutting is a major focus in the triple-A space. The technology itself cannot function without the work of game creators – it needs final game imagery to work at all – but the extent to which it could be viewed as a worrying sign of “things to come” cannot be overstated bearing in mind the reactions elsewhere to generative AI. That strikes me as a valid and interesting ethical question when it comes to the use of this technology, but one that is probably overwrought. Individual artists who work on video games already have their artistic output live at the pleasure of the game developers they contract with. Those developers already can use this game art in all kinds of ways that the individual artist may not have had in mind when creating it, or indeed have even considered such possibilities. DLSS 5 is just one more version of that, with the main difference being that it involves AI making changes to game images. That’s an important thing to consider, sure, but there are cousins to this ethical question that we’ve all come to accept already. This strikes me more as part of the “all AI is bad all the time” crowd finding a foothold in something other than dogma to grab onto. Developers and publishers own their games. If they want to use DLSS 5 in those games, there is little other than specific work for hire or other contractual stipulations with individual artists that would keep them from implementing it. If artists don’t like that, I completely understand that point of view, but that’s what contract negotiations and language are for. Bottom line: I have been as vocal as anyone arguing that video games are a form of art for well over a decade now and I struggle to agree that an optional technology that has approved buy in from game developers and publishers equates to “overriding artistic intent”, writ large. The faces in these examples look like shit, are “yassified”, or suffer from the uncanny valley effect! Look, here we’re going to get into matters of opinion. I have to say that when I viewed the demo video myself, I had the opposite reaction. And, yes, this opens me up to claims that I am somehow a massive fan of AI-created pornography (this is where the yassified comments come in), or that I just want all the characters to look “hot” (I’m too old for that shit), or that my older age of 44 means I’ve lost touch with what video games should look like. Despite my genuine respect for the dissenting opinions here, allow me to say this: bullshit. The caveat to all of this is that the demo revealed very little in the way of this technology working within these games in motion. It’s also certainly true that NVIDIA chose the best potential images to show off its new technology. If the DLSS 5 rendering sucks out loud in a larger in-motion game, or if the images it creates end up being inconsistent throughout gameplay, or if it does just end up looking shitty, then I’ll be right there with you with a torch and pitchfork in hand. And here’s the other thing to consider with this particular complaint, combined with the previous one about artistic intent: do any of you use visual mods in your games? I do. A ton of them. For a variety of reasons. I have used them to alter the faces and models for games like Starfield and Skyrim, among many others. Do I need to feel bad for altering the artist’s intent? Do I need to apologize for incorporating mods to make characters and environments appear in a way that helps me better connect with the game I’m playing? Because I’m not going to do either. And I don’t expect you to. Nor do I expect game developers that choose to use this optional technology to beg for forgiveness for their own output. The hardware demands to run all of this are insane! Fine, then you’ll get what you want and nobody will be able to use this technology anyway. But I don’t think that will be the case. NVIDIA knows what it will take to run this tech once it leaves the demo stage and goes into production. The idea that they would hype up technology that nobody can use strikes me as unlikely in the extreme. Conclusion: everyone take a breath This still strikes me as more of a “all AI is bad” crowd grasping at lots of other things to buttress their pushback than anything else. AI has plenty, plenty of potential pitfalls. Worried about jobs in the gaming industry and elsewhere? Me too! But if you’re not also looking at the potential upsides for the industry, then you’re engaging in dogma, not conversation. Will DLSS 5 be good? I have no idea and neither do you. Will DLSS 5 alter previously released games in a way that fundamentally alters how we play these games? I have no idea and neither do you. Will it negatively impact the gaming industry when it comes to the number of jobs within it? I have no idea and neither do you. This was a tech demo. Details on how it works are still trickling out. Most recently, there has been some clarification as to the 2D rendering nature of the technology and what that means for the output on the screen. As an early demo of the technology, feedback is going to be important, so long as it’s informed and reasonable feedback. The technology may end up being trash and hated for reasons other than “all AI is bad all the time.” If that ends up being the case, I trust the gaming market to work that out for itself. But a lot of the hand-wringing here looks to me to be speculative at best. Filed Under: ai, developers, dlss 5, rendering, video games Companies: nvidia Comments on “Deep Breath: Okay, Let’s Talk About That Controversial DLSS 5 Demo” What’s absolutely absurd to me is how it takes a truly ridiculous level of hardware, two of the priciest graphics card on the market, just to essentially do rotoscoping that eradicates details like the fog, or the realistic light scatter from the path tracing. DLSS before 5 was attempting to render the output at a higher framerate with minimal change from what rendering without it would produce. DLSS 5 makes changes, like putting lipstick on all the women, and the demonstration needing dual 5090s makes it sound like it’s going to come with a significant reduction in framerate. It’s like Nvidia has inverted every target they had for prior versions of DLSS and are surprised that people don’t like it. Re: The glossy lipstick and lip filler on all characters is an interesting touch Re: It’s worth pointing out that when nVidia introduced real-time RT back in 2018, their tech demo was running on four Quadro GV100 GPUs. Those weren’t exactly consumer-grade cards. I don’t know why people who see this expect it to be able to run well on something like a single RTX 5080. I do strongly agree that previewing this as the next iteration of DLSS was a mistake. This comment has been flagged by the community. Click here to show it. Bulldog Billboards Looking for the best prices on billboard advertising in Dallas, TX with Bulldog Billboards. Boost your brand visibility with high-impact outdoor ads in prime locations. We offer cost-effective solutions, flexible packages, and eye-catching designs to help your business stand out. Reach thousands daily and maximize your marketing ROI. Contact us today to start your billboard advertising campaign. visit here:- https://bulldogbillboards.com/ Maralago-face filter aside... From what little Nvidia seemed willing to show when it came to this tech demo, I thought that movement seemed to suffer from horrible temporal issues. It would seem that the Oblivion remaster and Starfield has TAA, but I couldn’t find anything on Saudi Arabia’s FC franchise. The temporal artifacting seemed bad and I can’t imagine it would translate to a decent experience in it’s current state. Regarding the “artistic vision”: Maybe it was rushed, maybe it’s because it’s early, but some of the drastic lighting changes from the Assassin’s creed game as well as Starfield just seemed incredibly drastic. It almost seems at odds with what you would expect from, what in past iterations, was a pretty ‘straight forward’ upscaler that didn’t change the scene a great deal. It strikes me as a technology that probably shouldn’t of been called DLSS. I am glad we’ve been getting more use out of the “made with activists in mind/Made with gamers in mind” memes laughing at the Gamergate crowd, though. ‘The idea that they would hype up technology that nobody can use strikes me as unlikely in the extreme.’ Did you just pop out of a time machine from the 1920’s? How often have we been pitched awesome new tech that’s only a few years away that isn’t? Elon Musk is approaching being a trillionaire based on the imagined value of his tech announcement company because he uses the announcement of things to bump the share price. The reason people are cynical this will make it to market any time soon is based on the actions of Nvidia, they have spent the past decade fuelling a shortage of their own product, prioritising gambling on Crypto, now AI, being part of the OpenAi and Oracle ‘investment’ ouroboros. People cannot afford or find what they are putting out now, and have no reason to believe the cards needed to run this will be available in large quantities any time soon and at an affordable price. As long as it’s opt in they can do whatever they want. That’s the problem. Gemini/Copilot at all were more or less optional at some point. It’s getting harder and harder to get rid of that shit. From my perspective it’s a “slopfication” technology that, like all slop generators, uses too much power to produce mediocre results at best (for general purposes, there are very nice uses for LLMs and other generative “AI” in specific workloads). Maybe it will be better with more iterations/optimizations? Maybe. But I don’t see any benefits from needing another 5090 to produce demonstrably worse results. The problem I have with the “optional” stuff is that it’s kind of a red herring. Even if something can be turned off, if the vast majority of people don’t, it’s kind of a moot point. Defaults/human behavior matters, and it’s not unreasonable to factor that in. And that’s also not getting into if Nvidia doesn’t hold to the promise (e.g., what if you have to use DLSS5 to get the other DLSS features in the future?), or the softer incentives- if it takes 10x the work to get the same result without DLSS, is it really a choice at a for-profit company? Incentives matter. I have a hard time taking this at face value. This blog is basically built to shake fists at quixotic battles like copyright, and that never stops it. Why should this be different? Maybe controversial, and it’s too long to get into, but I think there are very different standards for how I treat mods/player choices, vs a company like Nvidia. Ok, but like…what’s the actual argument? A lot of it is subjective, but I don’t think you can really argue that it doesn’t share a lot of the features of e.g. AI ads. Maybe you like the style, but it’s certainly there. There are also already existing objective issues in the footage we saw, with e.g. unrealistic lighting. (for the record, I do think this will get fixed in time) This is true, but one thing I think is missing is that these are asymmetric. Maybe the job stuff all does work out- great! Upsides exist, but the risks weigh towards focusing on the downside, even if it’s speculative. And that warrants more than a few throwaway lines. Why? You cover so many things the market doesn’t work out, and this fits that. The market has never prioritized things like artistic intent or not exploiting workers over profits. Valid dissent is impossible I’m frustrated that the author has apparently decided that it’s simply not possible to make a valid criticism of a product which uses AI. Nvidia released a tech demo of a new graphics product, presumably putting its best foot forward when doing so. The overwhelming public consensus was that the images produced by this new tool just looked bad. The author’s takeaway isn’t that Nvidia did this tech demo too early, or that it misjudged what consumers wanted, or that it just doesn’t do a good job of what it’s supposed to do, but instead that people who don’t like the tech demo are all knee-jerk AI opponents. Seriously, where is this discussion supposed to go if the AI boosters just preemptively declare any criticism to be irrational hatred of AI? So, this is really really sensitive, so I want to break this off into another comment in the hopes for some nuance beyond just “herr, you condone it”. And to be clear, I condemn this. And I also acknowledge that people on the internet can be inappropriate manchildren. That said… how exactly are people supposed to react to this sort of thing? It’s easy to say it’s just a game and people need to get with progress but these things affect people’s lives. But we don’t live in a country with a robust safety net; this can literally be life, death, homelessness, lack of healthcare, etc (and this damage has happened in previous technological shifts). It’s easy to rote condemn it, but I think you’re underselling how much of a powderkeg we’re sitting on. Especially given the hype that bosses (not to mention fascists) have made clear they want to leverage, and that it’s Nvidia in particular, and the lack of a voice in policy people feel. Again, I’m not condoning it, but I do think there’s a bit of a double standard worth examining. We condemn threats, but putting people out of a job, and the threat that comes with that, is treated as somehow just business? And that’s a little weird. I don’t think you can acknowledge a potential for mass unemployment and expect people to take it laying down. Re: That was a well said, clearly thought out comment. I still land on “We shouldn’t threaten to kill people, even over our lost jobs, if that happens.” That is a hill that I am, ahem, willing to die on. I don’t think those are the same. If a player uses mods, they openly and willingly ignore artistic intent to play a game the way they want to. But that’s not what DLSS 5 is. It operates from the “opposite side”, so to speak. The developer implements it, and allegedly has control. The granularity of that control seems lacking, though, based on elaborative responses from nVidia engineers. To use the example from the video link above, if developers wanted to reduce the makeup look on Grace in DLSS5 so it doesn’t look like she’s enroute to a date, there’s no way to prompt for that reduction. They would effectively have to turn DLSS5 off entirely. It’s possible nVidia improves the control before release, but as it stands, it doesn’t sound like developers will be able to configure the tool the way they would like to. Re: “But that’s not what DLSS 5 is. It operates from the “opposite side”, so to speak. The developer implements it, and allegedly has control. The granularity of that control seems lacking, though, based on elaborative responses from nVidia engineers.” That’s a completely fair point, but the analogy doesn’t have to be perfect to make a point. DLSS is an optional technology FOR DEVELOPERS. NVIDIA made that even more clear in the days since the demo video release. So which respects artistic intent more: an optional tool developers can use and tailor to impact their own art, or my use of a mod that the artist had no idea would even exist, nor had a hand in, to change their art significantly? “To use the example from the video link above, if developers wanted to reduce the makeup look on Grace in DLSS5 so it doesn’t look like she’s enroute to a date, there’s no way to prompt for that reduction. They would effectively have to turn DLSS5 off entirely.” Do you know that to be true? For sure? Because I don’t and I don’t think you do, either. In fact, from hearing NVIDIA talk about the tech, the idea eventually seems to be allowing them to do exactly that kind of prompting. So, if that’s how this ends up, you’re okay with it?