Video game developers are no stranger to adopting and harnessing the latest technology. With the last couple of years being the most disruptive on record in terms of new technology, there are a host of new tools at a game developer’s disposal; tools that’ll not only streamline the game development process but improve graphics in unprecedented ways.
Neural Radiance Fields
Neural Radiance Fields (or NeRF, for short) is an artificial intelligence system capable of translating real world photographs and video footage into digital objects and scenery, and according to specialist Californian based company Luma AI the tech is already being put to good use by video game developers. The system has intelligence, capable of learning from natural light sources and how they interact with surfaces. The way light absorbs into matte surfaces like fabric and clothing versus how it bounces off shiny metallic machinery, for instance, is something NeRF observes and interprets based off a few images of the same object from multiple perspectives.
Unreal Engine 5 MetaHuman
Unveiled at Epic Games’ State of Unreal 2023 keynote presentation, Unreal Engine 5’s MetaHuman is an animation tool capable of creating ultra-realistic facial animations. Demonstrated via iPhone by Senua actress Melina Juergens at the presentation the tech appears to render her face quickly and accurately, hinting at a streamlined facial animation process for developers. Recent Silent Hill adjacent horror game Stray Souls harnessed the tech to serviceable extent, but it’s the upcoming Senua’s Saga: Hellblade II which looks like it’ll give us the best glimpse yet of what MetaHuman is truly capable of.
DLSS Frame Regeneration
NVIDIA’s Deep Learning Super Sampling technology introduced an AI powered performance multiplier dubbed Optical Multi Frame Generation within its DLSS 3 update. The system generates entirely new frames as opposed to pixels, delivering astounding performance and frame rate boosts on the fly without extra load being placed on the CPU. In fact, NVIDIA say DLSS-generated frames will improve frame rates up to four times, with GeForce RTX Series GPUs able to render frame rates at twice the rate the CPU is able to compute. Cyberpunk 2077 is already using the tech, with upcoming games such as Black Myth: Wukong and S.T.A.L.K.E.R. 2: Heart of Chornobyl giving us further insight into its capability.
DLSS Real-Time Path Tracing
Made possible by DLSS 3, NVIDIA’s RTX Path Tracing Software Development Kit released at GDC 2023 merges real-time ray tracing with decades-worth of neural graphics development research to present a real-time path tracer. When deployed, the result, says NVIDIA, is an accurate recreation of light source physics; in other words, it’ll reproduce with uncanny ability what the human eye sees in real life. The RTX Path Tracing SDK itself gives developers flexibility and customisability to build reference path tracers, with efficient true to life results.
Unreal Engine 5 Chaos Physics
Chaos Physics has been around since Unreal Engine 4, but it’s only now thanks to UE5’s low latency technology that the engine is coming more to the fore. Most notably used in Fortnite since it’s migration to Unreal Engine 5, Chaos Physics is a light-weight physics simulation tool replicating visual movement effects for a variety of instances in cinematic quality. Most visible in massive-scale destruction, Chaos Physics also simulates cloth movement, collisions, fluid, and hair in real-time. Chaos Destruction is the headline here though, and it is in fact a collection of tools optimised for performance and creator control, offering an intuitive workflow allowing artists to specifically define how geometry will break during an explosion.
Unreal Engine 5 Substrate
Substrate is Unreal Engine 5’s innovative solution to creating materials, and more specifically the accurate blending of specific surface types like metal, glass, and plastic, with material base and surface layers each able to absorb, refract, and reflect light in distinct ways. Best demonstrated in Unreal Engine 5.2’s Next-Gen Graphics Demo first shown at the State of Unreal 2023, the video showcases a Rivian off-road vehicle with opal stone bodywork. Opal, with its multi-layered, almost crystalised appearance is an apt material to showcase Substrate, but most promisingly when dust and dirt was added to the vehicle it is notable how artefact-free light interacts differently depending on the cleanliness of the bodywork.
Procedural generation
Game development engines are increasing their procedural generation capability, creating entire environments from a combination of parameters and already hand-crafted play areas. Again, as demonstrated in Unreal Engine 5.2’s Next-Gen Graphics Demo art direction and visuals is set by designers before being extrapolated to a wider area by procedural generation tools. Whilst the effectiveness of this system might initially threaten to render art direction obsolete, its efficiency gives game designers more flexibility to distil their experiences with surgical precision, tweaking every stone and log in a forest, for instance, to funnel players through in the most seamless way possible.
DirectX 12
DirectX has been around for years. The twelfth iteration of NVIDIA’s technology gives developers of Microsoft Windows-based games access to advanced graphic effects such as ray tracing and variable rate shading, resulting in ultra-realistic visuals, lifelike illumination, reflection, and shadow, all at rapid frame rates. DirectX 12 Ultimate’s most recent update saw the light of day in late 2021, adding game changing mesh shaders and sampler feedback to portray new levels of realism to in-game textures and lighting. Plus, it’s unified code with the Xbox Series X means games developed for Microsoft’s console will work better on PC going forward.
Adobe Substance 3D Painter Version 9.0
Adobe Substance 3D Painter is industry standard texturing software, allowing creators to add texture to practically any surface with the simple stroke of a brush. Perfect for both photorealistic 3D objects or highly stylised environments, Adobe Substance 3D Painter Version 9.0 released in June 2023 adds re-editable paths and Bezier based curves that can be assigned to specific points on the surface of a 3D object. The result is a more expressive tool, one that’ll enable artists to add unparalleled texture detail with relative ease.
Adobe Substance 3D Designer Version 13.0
Adobe Substance 3D Designer allows artists to create seamless materials, patterns, image filters, and environmental lighting in immeasurable variations. In-game assets are set to receive a boost in fidelity thanks to June 2023’s Version 13.0; an update which brought a host of user-friendly workflows alongside a new collection of tools for mapping, scattering, and warping materials with realistic accuracy.
Lightspeed Studios’ Photon Water System
Owing to their perceived limitations with Unreal Engine 5’s water rendering solutions and performance issues from third-party plug-ins, Zhenyu Mao and Kui Wu from Lightspeed Studios opted to tackle their issues head on by developing their own cutting edge water rendering solution. The result is a naturally flowing water simulation complete with ability to interact harmoniously with any object and environment creating seas, lakes, and rivers which flow and foam with surreal lifelikeness. Whether Lightspeed Studios will license their tech to other parties remains to be seen, but it certainly looks great from the footage they’ve shared.
BroadLeaf
BroadLeaf is a real-time solution for rendering large-scale forests developed by Yixin Hu. According to Hu, crafting huge forested areas is a constant headache for developers due to the massive number of plant-leaf geometries. Rendering forests with superb levels of detail including complex foliage, with interactive trees of high visual quality existing in real-time with efficient performance is BroadLeaf’s goal, and in visions of the tech shared by the developer it looks to be right on the money.
Epic’s Fab
Epic have created a unified experience for digital creators by melding its asset marketplaces Quixel, Sketchfab, ArtStation, and Unreal Engine Marketplace into one. The aim here is to connect a community of global creators, making it easier for everyone to access the highest quality digital assets – including but not limited to 3D models, VFX, materials, textures, and digital humans – all available to build into their projects. Epic’s 88% revenue share model should foster a new breed of asset creator too.
AI powered graphics
Artificial intelligence is amongst the most disruptive technologies to emerge in recent years. It’ll force game developers to adapt their workflows and processes alongside the skills of their workforce as the entire video game experience – from increasingly intelligent NPCs to procedurally generated level design – evolves. One of the most prevalent benefits of AI in game design is in image quality enhancement, with AI and machine learning being deployed by Rockstar to enhance the visuals of Grand Theft Auto V as its resolutions were translated to higher quality during its transition to modern hardware.
