SSGINode: Use a half float render target.

[mrdoob]

Description

The GI pass produces HDR output — the shader clamps luminance at 7, and since the clamp is on luminance, individual channels can go far above 1. The render target used the default 8-bit type, so values were clipped per channel before compositing, losing bounce light energy and shifting its hue. This also made the in-shader HDR clamp dead code.

Using HalfFloatType fixes this and matches SSRNode/TRAANode.

[CodyJasonBennett]

Why not R11F_G11F_B10F? Half the memory and still represents HDR range well. #23228 (comment)

[Mugen87]

The SSGI computes RGBA output meaning indirect diffuse RGB + AO which is stored in the alpha channel.

[Mugen87]

Also worth nothing in WebGPU rg11b10ufloat-renderable is a feature so it's not guaranteed to work on all devices.

We would need a FP16 path in any event.

[CodyJasonBennett]

I have not inspected this implementation very thoroughly, but for anything storing (ir)radiance, that is the ideal format over rgba16f, as we're talking about a 2x memory difference for formats that both encode the HDR range and support filtering + blending (can blend alpha though not store it).

I don't exactly understand the need to store indirect diffuse as RGB + AO. Are these sampled separately in a GI-only or AO-only mode, or is AO used separately for specular occlusion? This is a big enough win to where I'd reconsider how the pipeline works, especially if you find people combining (diffuse) SSGI + SSR. This is quite wasteful, and I would suggest a separate R8 or R16F attachment for AO.

When I implemented GI of any kind in WebGPU (DDGI, SSGI, even ReSTIR), especially with specular and/or a radiance cache, I would switch between the two formats, and prefer the smaller float format. The performance difference is super linear and subject to hardware constraints.

FWIW, the optionality is for exactly the Adreno 500 GPU (8-10 year phones), which is a looser requirement than WebGL2 even. IMO, there have been sweeping regressions in support with WebGPU spec in targeting devices where if they don't have a compliant Vulkan driver, they probably don't have a compliant OpenGL driver either for the non-standard compat mode.

Some background here is that I am considering implementing my work on Radiance Cascades SSGI in Three.js in time for the upcoming conference in Paris, and so I have been passively observing related discussion here to close in comparison, if not upstream the work. Storage is one of the biggest factors for quality/performance with screen-space effects, just due to their recurrent and high-bandwidth nature on GPUs not optimized for bandwidth but throughput. So smaller, more cache-efficient data wins overall. If you can point me to a canonical discussion, perhaps I can continue there, but otherwise I think it's worth a thought against the changeset here.

[Mugen87]

I don't exactly understand the need to store indirect diffuse as RGB + AO. Are these sampled separately in a GI-only or AO-only mode, or is AO used separately for specular occlusion?

SSGI and AO aren't two separate effects here — this is a bitmask/visibility-bitfield GTAO derivative (SSILVB), so the same hemisphere ray march that gathers indirect bounce light also yields the occlusion term for free. You'd never run a separate AO pass with SSGINode.

There is no way we can use only R11F_G11F_B10F in this effect.

[CodyJasonBennett]

SSGI and AO aren't two separate effects here — this is a bitmask/visibility-bitfield GTAO derivative (SSILVB), so the same hemisphere ray march that gathers indirect bounce light also yields the occlusion term for free. You'd never run a separate AO pass with SSGINode.

I am not sure how you came to that conclusion; the question was whether this storage made it easier to change the display output without additional shader permutations rather than an optimal storage combination for either GI or AO output. In other words, was it a compromise for flexibility rather than an optimized storage layout?

For context, it might be preferable to use a GTAO/SSDO for short-range contact effects on top of a coarser GI method like DDGI or Surfels, which only capture mid-field to ambient light ranges. In this case, the diffuse GI from SSGI is largely duplicate, but the AO can be used to clean up unmodeled light ranges. Many games also use this to fill in shadows for things like foliage, which are too bandwidth hungry to include in shadow maps. And as neither DDGI nor Surfels encode specular GI, an SSR can be added on top and sample the AO component for specular occlusion. The modality is useful.

Edit: A figure from a presentation I recently gave at the Unity office on this:

There is no way we can use R11F_G11F_B10F in this effect.

This is not strictly true, but I have to ask--will Three.js accept an improvement here or reject one out of principle? I have implemented SSILVB many times since its debut and improved it with my own research over the past few years, namely including Radiance Cascades and my own work on top for accuracy and energy conservation, but it would be nice to know if Three.js is a proper forum for this or if I should move elsewhere. I apologize if I am just stepping onto planned work, but I have not found an opportunity to participate asynchronously elsewhere, hence my opportunism here.

[mrdoob]

We're always open to PRs that improve the code.
But we still lack the resources for engaging in long discussions.

[Mugen87]

In other words, was it a compromise for flexibility rather than an optimized storage layout?

If you ask like that I would say flexibility. We invested quite some time last year into this topic. The SSGI we adapted from https://github.com/cdrinmatane/SSRT3 felt like a good solution since it is self contained (you compute AO and GI in a single pass) and the overall visual result was very good.

The previous RGBA8 usage was a mistake on our side since the effect operates on HDR data so this PR here was a pure correction fix.

I understand there are other more memory/bandwidth optimized approaches and of course we are interested in optimizing the render pipeline as @mrdoob mentioned. We did some work in that are in context of packing/unpacking and render target/format optimization but were are just at the start here.

Assuming a SSGI computes GI and AO in a single pass: Couldn't we use a MRT target with R11F_G11F_B10F for GI and R8 for AO?

[CodyJasonBennett]

We're always open to PRs that improve the code.
But we still lack the resources for engaging in long discussions.

Great, thanks. Just don't want to put work in only to get shut down or ideally redirected.

How about I open PRs for small fixes or improvements that I find and, for something larger like my research that is more novel/disruptive, open an issue to invite other eyes on it? No expectations with the latter; just gauge interest as it might succeed the SSGINode implementation here.

Assuming a SSGI computes GI and AO in a single pass: Couldn't we use a MRT target with R11F_G11F_B10F for GI and R8 for AO?

Yes, that is exactly what I would do without changing anything else.