GDC 2023 - The FidelityFX SDK

GDC 2023 - The FidelityFX SDK

Show Video

Hi, I’m Jason Lacroix. I’m a Principal Member of Technical Staff in AMD’s Game Engineering organization. And today, it’s my pleasure to walk you through our brand new FidelityFX SDK. Now before starting, here’s a rough outline of what I’ll be discussing today. We’ll first go over the reasons why we chose to modify our delivery process for FidelityFX features. Then we’ll have an in-depth look at how the FidelityFX SDK works, with detailed looks at its internal components, and a simple usage example. Next, we’ll have a brief overview of Cauldron 2.0, the new graphics framework that powers all of our SDK samples.

We’ll then take a look at a few of the new samples that we're releasing alongside the first version of the SDK. And lastly, we’ll have a quick look our roadmap for the SDK and wrap things up. Now let's go over a little bit of overview into the FidelityFX SDK. The FidelityFX SDK is our new easy-to-integrate solution for developers looking to include our features into their games without any of the hassle of complicated porting procedures. It's effectively middleware to do the heavy lifting for the developers so that they can very quickly bring our features into their games. Over the last 4 years, we’ve introduced a lot of great value technology.

However, it started becoming apparent that we needed a better solution going forward as we continued to grow our effects library. Our current samples had a number of problems that became more apparent as we kept adding onto it. Inconsistencies in coding standards and styles, some of which was done in ways that were really not trivial to follow along with. We had a lot of needless boiler plate code and duplication when it came to setting up new effects. There was a general lack of stability on the backing graphics frameworks. In fact, our effects, to now, have been shipped on no less than 6 different versions of Cauldron

each of which introduced many API breaking changes, slowing development to a pace we weren't very happy with. We have inconsistencies in documentation formats or very little information altogether. Inconsistencies in feature accessors. Some of our samples had API-like accessors, however most did not. We also had limited platform/API support. Rarely, did we ship Vulkan versions at the same time as DX12 – if at all – and very few features were supported on Xbox.

Now, the SDK aims to rectify a lot of this by bringing simplicity, structure, and consistency to all of the great software you have come to expect from AMD. Of note, the SDK features: a consistent standard and style that is much more user friendly. Far less boiler-plate code to write to get up and running.

We've re-architected our graphical framework that is not only more robust and future-proof, but is also platform/API agnostic, allowing us to develop all of our effects across all intended targets simultaneously. This also doubles as a great way to show developers how to create custom backend implementations for your own multi-platform engines to back our FidelityFX features. We've added extensive documentation and taken a lot of the guesswork out of implementation steps by offering complete pre-built solutions for all of our effects.

Most of which can be supported in under 20 lines of code. That being said, if you were a fan of taking the code we put out and manually integrating it yourself, you are still free to do that. We’ve not taken anything away. All we’ve done is moved things into a more coherent structure on disk, and cleaned up the code to make it more consistent in style.

The SDK was designed from the ground up to be as modular as possible. This means you are free to pick and choose to support only the features you want and leave the rest behind. This modularity also applies to our backend implementation. The SDK will ship with a native DX12 and Vulkan backend and a custom backend example for our own graphics framework, Cauldron, in order to show how custom multi-API or multi-platform backends can be implemented to support the SDK. We also have a DirectX11 backend available for NDA partners for those games that still require legacy support. However, note that some samples cannot run on DX11 due to required hardware feature or API support requirements.

Now, the custom backends let you use FidelityFX technology expressed in terms of your own engine’s abstractions and constructs. You provide just a few fundamental services, and all the plumbing and logic for an effect component should just work. This allows you to unlock FidelityFX technology on any platform your engine supports – assuming it can compile shaders using a relatively modern shader model. Now, let’s look at a breakdown of what you will find on disk when getting the SDK.

Our SDK code folder is typically anything you might ship in your game, or use to build your game. You can either compile the SDK into a set of libraries through the C API and link it with your application, or take the shader code directly into your game like you did with our older samples). Again, nothing is lost from the current situation, we’ve only added more on top to make things easier to use and introduce broader consistency. Building the SDK will make a set of static libraries. One for each component and one for each backend. Link the ones you want, leave the rest behind. The Samples directory is where you'll find our new Cauldron 2.0 framework.

This completely re-architected framework is built on the concept of render modules, which can be thought of as rendering feature nodes in a modern day render graph. The Render modules folder contains common render modules used by most of our samples. For instance, rendering a Gbuffer or Skydome, doing a transparency pass, or rendering particles. Each effect sample will also implement its own render module which demonstrates how to use one or more effect components as you would in your own game. Lastly, we provide all documentation in various

format like CHM, HTML, and MD format for easy browsing. Now let’s have a look at just how simple it is to initialize and use the FidelityFX SDK, using one of our new samples, FidelityFX Blur. The first step is to query the amount of memory you will need to allocate to back all the effect components you want to support. In this case, we are only backing the blur effect. Note here the “DX12” part of the function signature. This is indicating that we are using the DX12 backend in this particular example. The next step is for memory to be allocated. Now we put the burden of allocation on the

developer so that you can use your own memory allocation and tracking schemes internally. Once the backing memory is allocated, the backend interface is initialized with it and the corresponding device object for the platform in question. Note that the backend will take care of incrementing/decrementing device reference counts for platforms like DX12 in case you need to track API-interface allocations. Now once the backend interface is setup, we initialize our effect context description. This tells the

effect library how you would like to use the effect in question. In this case we are creating a blur effect what will allow all kernel sizes to be supported dynamically and use fast packed math options in the shader. Once the context is setup, we call ffxBlurContextCreate to create our effect context, and we are ready to start rendering blur. Using the effect context at run time is just as simple. We first setup a dispatch context description to tell the library the desired configuration of the effect to use. In this case, we are requesting a 13x13 blur

kernel-based operation at app resolution. We also provide a backing command list from the application for command recording in the library. We then import our input and destination resources from the external application to bind to the GPU workload. Then it’s just a matter of dispatching the context with ffxBlurContextDispatch(). When the application shuts down, or we are otherwise ready to stop using blur, we simply destroy the blur context. Please don’t forget to flush the GPU to

make sure everything is clear of the pipeline. And the last thing to do is release the memory that you allocated to back the interface with. And you're done. Let’s take a brief look at how each of the main SDK components work together to enable FidelityFX features to run easily in your games. The first piece we’ll look at is the Effect Component. The Effect component represents the runtime-level execution portion of a FidelityFX feature. This handles internal resource creation, external resource registration, the building of pipelines, creation and management of resource views, and GPU workload scheduling and execution for the application. These are essentially usability modules designed to make integration of the SDK features as seamless as possible. Should you opt to do direct integration into your game,

you're doing to want to replicate what the effect component is doing to ensure proper integration. Now the source to handle all of this is grouped under 3 different levels: Public source contains all top-level common SDK include files. This includes the SDK interface as well as things like error, types, and utility headers, as well as the CPU host-side effect header which we'll look into more in a moment. Shared source contains platform hooks for assert definitions and convenience functions for resource and pipeline releasing. And private source is where you’ll find the implementation details for the effect component as well as header definitions for all private structures used.

Now, the ffx_<Effect>.h file is the main file to include in your application to pull in effect support. You can either include the host side effect header directly or pull in ffx_fx.h which contains all effect headers inclusion. The effect header file is standardized across all effects and contains roughly the same information.

Versioning information in the form of major/minor/patch define. A context count requirements which are used to figure out how much memory the backend needs to manage in order to have multiple effects working simultaneously and we’ll talk about this a bit later on when discussing the backends. Effect-specific defines to use in your application. Pass enumerations. Initialization bits, if needed. An opaque effect context definition which is mostly just a memory buffer that will back a private context. And context creation and dispatch description structures. These will

vary on an effect-by-effect basis according to what is needed for setup and workload dispatch Lastly, each effect header will define the main effect function signatures: ContextCreate ContextDispatch and ContextDestroy What they do should be fairly self evident. As a note though, some effects, for instance FSR2, can have more functions exposed when additional functionality is present. However, for the most part most of our effect components only expose these 3 functions. The ffx_<Effect>_private.h file is where you're gonna find all the definitions used in the implementation details of each effect component.

This will include the multitude of shader permutation options used to create the needed pipelines. The CPU-side constant buffer definitions that are used for constant uploading And the private effect context which holds everything used internally throughout the lifetime of the effect component. As an example, you can see SPD’s private context definition here. Lastly, the ffx_<effect>.cpp file is where you will find

the effect component’s implementation details. The public facing Context Create, Destroy, and Dispatch calls are largely just redirects into internal implementations which is where you’ll find the bulk of the feature implementation. Next, we’ll have a look at how this ties into the SDK backend constructs. All of our effects components rely on a backend-specific implementation of our FidelityFX SDK interface. This interface was largely derived from the interface model developed for FSR2, and expanded upon in order to properly back all features we currently support.

The backends are API- or Platform-specific implementations of all required interface functionality, plus whatever support code is necessary to make everything work. The SDK will initially ship with 3 backends. Native DX12 and Vulkan, and a custom backend expressing the interface functionality in terms of our API-agnostic graphical framework, Cauldron. The backend, on top of implementing interface support, is also largely responsible for building all shader permutations for a particular platform. The files to enable this are grouped under 4 different headings + a special shader permutations project. We have public source which contains the backend definition file you will want to pull into your game or application. This defines

all the functions you need to call in order to set everything up to use an effect component. Public shaders has all shader language callback files. These are defined per effect for each shader language and serve as accessors to all effect resource and constant information. Public shaders also include all effect resource definition files, and all effect pass implementation files in header format. The private source holds all shader blob accessors compiled with the pre-generated shader binary data that we export by the shader compiler tool, as well as the implementation details for the backend.

Private shaders are all shader language specific entry points for all of our effect passes. Lastly, you’ll find a shader permutations project that also gets generated alongside the backend. This is just a CMake project housing all of the rules to build all required shader permutations when building the backend. Lastly, let’s have a quick look at the FidelityFX Shader Compiler. The FidelityFX Shader Compiler is an open source tool we are shipping alongside the rest of the SDK. Its main purpose is to compile all effect pass shaders for a specified language with every permutation of options passed in. These shader binaries then have their reflection

data extracted and written out to header files as a shader blob along with the compiled shader binary. Currently, the shader compiler supports HlSL compiling via DXC and FXC when using the DX11 backend, and GLSL compile via GLSLang. The tool is easily modifying to support additional flags and options you may need for your game needs. Just be sure to copy the compiler binary into the sdk/tools/binary_store directory along with any needed libraries or DLLs if you make any changes. The pre-compiler is also easily extensible for new languages. If new language support is needed: Create a new compiler class that inherits from our compiler interface.

Define a new shader binary structure which inherits from our shader binary interface. And implement all compiler virtual functions needed to compile, extract, and write the various data out to file for inclusion in your custom backend. You can then add an execution path to your new compiler by adding support for an additional language command line argument in ffx_sc.cpp Now, let’s take a look at the new graphics framework backing all of the samples in the SDK The SDK samples are powered by an overhauled version of Cauldron, now at version 2.0. The entire framework was re-architected from the ground up to better support our development needs withing AMD, but also to make it easier for developers to locate information while browsing our sample code. This new version of Cauldron is highly customizable on a per-run basis, and was designed with emphasis on flexibility and quick setup and tear down.

Cauldron 2.0 also implements all core components in an API or platform abstracted fashion to ensure support across all of our supported platforms. Cauldron 2.0 revolves around the concept of render modules, which are functionally like feature nodes in a render graph. Each render module is a fully- contained “plugin” of sorts and has: It’s own configuration and dependency file HLSL shader source, which currently all shaders in Cauldron are HLSL-based and then crossed-compiled to DXIL or SPIR-V as needed for DX12 or Vulkan, contains any optional dependent media and then the source.

Currently, our samples use a combination of the following render modules which we'll find the render modules folder. RasterizedShadows. Gbuffer construction. DeferredLighting. Translucency render. Skydome render. TAA. GPU Particle simulation and rendering and Animation support The Cauldron framework itself also contains 3 exclusive render modules for core functionality.

These includes a default tonemapper, a UI renderer, and a Swap chain render module to control presentation buffers. Now sample loading in Cauldron is handled via glTF. We fully support the 2.0 spec, allowing you to export your own content via a growing number of glTF plugins and load it directly into our samples directly via command line for easy testing. As we continue to develop more advanced features, it's critical that we always ensure support for the latest versions of third-party APIs. In order to preserve constant forward momentum, each SDK release will also feature the latest stable version of Microsoft’s AgilitySDK, the DXC Compiler, and DX Headers.

Each of which will be in lock-step to the last stable AgilitySDK version. We'll also feature the latest version of the Vulkan SDK. Now, other third-party software used includes ImGui, our AMD GPU and CPU services libraries, DX12 and VK memory allocator, STB image loading libraries, VectorMath, and a Json library.

Now, let’s have a look at how our samples are put together. Each feature sample is built on its own self-contained framework configuration. The sample will contain one or more simple render modules showing how to interface with the SDK for that particular feature.

The sample solutions can be built on an feature by feature basis or as a master solution with all features. Furthermore, there are also scripts to build samples backed by either Cauldron’s custom SDK or linked against native backend libraries from the SDK for DX12 and Vulkan. Initial release will feature updated versions of nearly all previously released FidelityFX content: A combined FSR sample featuring Super Resolution versions 1 & 2.2, Contrast Adapted Sharpening or CAS, Combined Adaptive Compute Ambient Occlusion or CACAO, Stochastic Screen Space Reflections or SSSR, Luminance Preserving Mapper or LPM, Parallel Sort, Variable Rate Shading, Single Pass Downsampler or SPD and Hybrid Shadows.

Now, SSSR and Hybrid shadows have also had their Tile Classifiers and Denoisers grouped under new building block libraries. These are FidelityFX Tile Classifier and FidelityFX Denoiser. And both these libraries have been updated with the latest bugfixes and optimizations to make their backing samples even better than before. Also, shipping will be a few new feature libraries The first of these is FidelityFX Depth of Field.

This sample features a physically correct camera-based depth of field implementation optimized for speed. It makes use of tile-based classification in order to do fast path selection wherever possible. The near and far blur operations are handled through a single pass dispatch, as are the combined filter, upsample, and final composition steps.

The entire effect works as combination of 5 passes: Depth and Color downsampling, a dilation pass, a blur pass, and a composition pass. Our FidelityFX Lens sample is a highly-optimized single pass shader that supports some of the more commonly used lens effects in video games. This includes Film Grain, Chromatic Aberration, and Vignette. This is just an initial sampling of lens effects, and we plan on adding to the library in the future to continue to support basic render effect needs. Our last new sample is FidelityFX Blur. This sample features a highly optimized set of blur kernels that can run from 3x3 to 21x21. The sample also features a comparative diff tool to allow you to see the difference between the FidelityFX blur kernel vs. other common blur approaches like single pass

box filters, multi-pass separable filters, and Multi-pass Seperable Filter Transposes. The FidelityFX SDK is currently available in Beta form to our NDA partners for evaluation purposes. Please look forward to its public release with a port of all existing samples, and our 3 new lens-based samples. We're also planning a 1.1 point release some time this later year.

This will contain an XBOX GDK backend, new FidelityFX samples, Cauldron improvements as well as scene interaction support. We're also planning a 1.2 release at some point later on towards the end of the year. This will have further feature improvements as well as new FidelityFX content And of course, as always, you are welcome to take our code to integrate directly into your own games and/or modify it as you see fit for free, with no vendor-based restrictions Now, in the last years since we’ve begun putting out FidelityFX features, we’ve become one of the most used and trusted technology partners in the industry. Our first in class open source technology can now be found in over 250 games, made by some of the biggest names in gaming. In summary,​ The FidelityFX SDK offers the same great technology that developers have come to expect from us, now just a little more consistent and easier to use. The SDK will be the delivery platform of choice going forward for all FidelityFX features.

We’re shipping all of our previously released features + 3 new effects, and new graphics building blocks in the form of updated tile classifiers and denoisers. All of our effect integration is now backed with easy-to-use C APIs to take the pain out of supporting the features you like. A modular backend that puts you in control of how your engine backs our effects and complete and thorough documentation on all of our underlaying code and structures. Finally, a great new shader compiler tool to front-load all shader binary generation. Now the FidelityFX SDK is the culmination of many months of work involving many people in Game Engineering here at AMD. So I’d like to take a brief moment to thank all the people who’ve had a hand in helping to push the SDK over the finish line.​

So please look forward to the public release of the FidelityFX SDK.

2025-01-31 20:47

Show Video

Other news

These Machines Are Why China Can Build Anything 2025-02-14 06:04
The Future of the Automotive Industry: Innovations, Technology, and Sustainability 2025-02-10 23:05
China’s New AI Model DeepSeek Shocks ENTIRE Tech Industry...American CEOs in Shock! 2025-02-09 15:01