Nvidia Brings Generative AI to Video Game Lighting

On 1 September, Nvidia launched a new generative AI approach to rendering video game graphics that CEO Jensen Huang has previously hailed as a “GPT moment for graphics.” Instead of forcing graphics processors to calculate the material physics and lighting of virtual surfaces pixel-by-pixel to achieve high levels of photorealism, Deep Learning Super Sampling 5 (DLSS 5) is a neural rendering approach that “paints” these properties onto surfaces in real time. Versions of DLSS have been released since 2018, each broadly aimed at making games run more efficiently using techniques such as inserting new frames between existing ones to improve frame rates. But the fifth iteration, integrated into the launch of the basketball video game NBA 2K27, is aiming for a slam dunk by enhancing the appearance of the image itself. DLSS 5 Neural Rendering for Games Regular text-to-image AI generators can produce inconsistent results; DLSS 5 targets a coherent output by analyzing structured visual game engine data and using its trained semantic understanding of scenes to respond to how the camera, characters, lighting, and environment move from moment to moment. The model handles the final stage of rendering, enhancing each pixel of a finished frame with photoreal lighting and materials designed to mimic effects such as light transmission through hair and foliage, light scattering through human skin, and micro-shadows. Nvidia claims this has never been achieved before on the order of milliseconds on a single GPU. The level of graphical realism has turned heads in the gaming industry. “When applied to a game with a photorealistic art style, it produces faces with a fidelity that I’ve never seen before in gaming graphics. It can make games feel like interactive films,” says Jeffrey Kampman, the resident GPU analyst at PC enthusiast publication Tom’s Hardware. “DLSS is expanding the capabilities of computer games by making them look less game-ish and more photorealistic,” says Gordon Wetzstein, a technology developer and computer graphics enthusiast. Nvidia has so far kept a lid on details of the underlying training datasets for DLSS 5, but Wetzstein notes that, like any generative AI model, the quality of the results are “primarily determined by the quality and diversity of data it is trained on.” Games publishers appear to be on board with the solution: Bethesda, Capcom, NetEase, Tencent, Ubisoft, and Warner Bros. Games have all signed up to integrate it into games. Not all gamers are as enthusiastic, partly because DLSS 5 is designed to run on expensive, high-end GeForce RTX 50 GPUs featuring powerful tensor cores—specialized processing units within Nvidia’s hardware able to power modern AI and deep learning—and even these can struggle to keep up. Tech publication Digital Foundry tested the debut integration of DLSS 5 on NBA 2K27 across multiple RTX 50 desktop setups and labeled it “super, super heavy,” recording 40 to 60 percent drops in performance. Responding to the criticism, an Nvidia spokesperson told IEEE Spectrum that optimization and model refinements have “already achieved a 5x performance boost in six months,” scaling DLSS 5 from a dual-GPU setup back at the March 2026 preview “down to a single GPU across the entire RTX 50 Series lineup at launch.” Further model updates are expected later, before the end of the year. DLSS 5 Performance and Frame Rates Performance issues are a common feature in the history of graphical innovations, such as the 2018 launch of real-time ray-tracing, which lights a virtual environment using the calculated trajectories of rays of light. According to Kampman, this incurred similarly large performance drops but “improved with new generations of hardware and is now practical on lower-end hardware.” It’s not only frame rates that have caused gamers to throw down their controllers in frustration. Early previews of DLSS 5 in games like Resident Evil Requiem and Starfield were criticized for creating an uncanny valley effect, giving faces an exaggerated, hyper-glamorous look rather than being photorealistic. Game artists and some developers have argued that the technology compromises the artistic integrity and uniqueness of different games by applying a uniform AI-generated aesthetic. “DLSS 5 imposes visual changes ranging from the minute to the grand, which kills the expertly calibrated visual cohesion, visual intention, and visual identity of a game,” says Karla Ortiz, an artist who has worked with Ubisoft and Blizzard. “This act rudely diminishes the years of hard work art teams do, as it changes the final product teams work so hard to achieve.” Nvidia contends that its intention is not to replace artistic intent but instead provide an optional visual-quality tool with various developer controls. “Artists can enhance fine material and lighting detail while reducing or disabling broader color changes. They can add custom masks to isolate specific props or asset groups and apply custom lighting tweaks without altering the surrounding environment,” said Nvidia’s spokesperson. These features enable the pursuit of “visual goals that would otherwise be difficult to achieve within a real-time rendering budget.” Neural rendering remains an active area of research for chipmakers. AMD and Intel haven’t yet implemented similar technology to DLSS 5 in their GPUs, but they are using AI models for upscaling and frame generation. And both companies, as well as Apple, are including tensor core-like matrix math accelerators tuned for AI workloads in future hardware. “It’s an indication that these techniques will be important to the industry going forward,” says Tom’s Hardware’s Kampman. - Your Laptop Isn’t Ready for LLMs. That’s About to Change › - Forget JPEG, How Would a Person Compress a Picture? › - New AI Speeds Computer Graphics by Up to 5x › Stephen Cousins is a journalist, content consultant, and copywriter with 16 years’ experience covering the built environment, engineering, sustainability, and technological innovation.