Add lumen-gi-enabled / lumen-reflections-enabled camera capture settings (captures match the viewport by default) - #181
Open
andrewjong wants to merge 1 commit into
Conversation
…ings UE scene captures don't render with the global illumination / reflection method the viewport uses: a capture view seeds its method from the project CVar but never blends level PostProcessVolumes, and on hardware-ray-tracing projects its rays have no acceleration-structure data unless the component opts in — so a Lumen-lit level comes out flat and washed out in captured images, with all indirect light missing. Since UE 5.5 the engine supports Lumen in scene captures when the capture's post-process settings resolve to Lumen (the renderer then maintains a dedicated Lumen scene per capture's persistent view state, which UnrealCamera already enables via bAlwaysPersistRenderingState). Add two per-capture tri-state settings, keyed off whether the jsonc key is present: - absent (default): match the viewport. Scene (RGB) captures mirror the project's r.DynamicGlobalIlluminationMethod / r.ReflectionMethod, so Lumen projects get Lumen captures out of the box; every other image type resolves to None — their output is a replacement material, so dynamic GI underneath is pure GPU/VRAM waste (and previously they silently inherited the project CVar). - lumen-gi-enabled / lumen-reflections-enabled = true: force Lumen for this capture regardless of the project default. - false: force the method to None. This is an active override — an un-overridden capture inherits the project CVar, so omitting the override is not a way to turn Lumen off. The resolved methods are pinned as explicit post-process overrides, and bUseRayTracingIfEnabled is set whenever either method resolves to Lumen so hardware-ray-traced Lumen works when the project uses it (software Lumen works without). On engines older than 5.5 the overrides are ignored for captures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
About
TL;DR adds Lumen support to AirSim camera images.
Camera images captured through
USceneCaptureComponent2Ddon't render with the global illumination / reflection method the viewport uses: a capture view seeds its method from the project CVars but never blends level PostProcessVolumes, and on hardware-ray-tracing projects the capture's rays have no acceleration-structure data unless the component opts in. The practical result is that a Lumen-lit level comes out flat and washed out in the sim's camera images — all indirect light missing — even though the game/editor viewport looks correct (see screenshots below).Since UE 5.5 the engine natively supports Lumen in scene captures: when a capture's post-process settings resolve to Lumen, the renderer maintains a dedicated Lumen scene for the capture's persistent view state (which
UnrealCameraalready enables viabAlwaysPersistRenderingState).This PR adds two per-capture tri-state settings, documented in
docs/sensors/camera_capture_settings.md:r.DynamicGlobalIlluminationMethod/r.ReflectionMethod, so Lumen projects get Lumen camera images out of the box. Every other image type (depth, segmentation, ...) resolves to None — their output comes from a replacement material, and previously they silently inherited the project CVar, paying Lumen cost for nothing.lumen-gi-enabled/lumen-reflections-enabled=true: force Lumen for that capture regardless of the project default.false: force the method to None. This is an active override — an un-overridden capture inherits the project CVar, so omitting the override is not an off switch.The resolved methods are pinned as explicit post-process overrides on each capture, and
bUseRayTracingIfEnabledis set whenever either method resolves to Lumen so hardware-ray-traced Lumen works (software Lumen works without it). The RGB wire format is unchanged (SCS_FinalColorLDR, 8-bit — in UE 5.5+ it runs the full viewport film tone curve). On engines older than 5.5 the overrides are ignored for scene captures, so this is a no-op there.Cost note: each Lumen-enabled capture maintains its own Lumen scene (GPU time + VRAM) — numbers below; set the knobs to
falseper capture to opt out where frame rate matters more than fidelity.How Has This Been Tested?
UE 5.7.4, Vulkan SM6, Linux (Epic
unreal-engine:dev-5.7.4container), photorealistic office scene (marketplace "QA Office" level, Lumen GI + reflections + HWRT in project settings), non-physics robot with two 640×480 RGB+depth-planar cameras at a 0.1 s capture interval:false→ matches the pre-PR captures (verified visually and by luminance statistics);true→ forces Lumen. The on/off pair was verified at four camera poses, the full tri-state matrix at one.Screenshots and videos:
Lobby — ceiling receives no direct light, only GI bounce; the marble floor gains reflections:
Lounge — ceiling, brass stanchions, chrome table legs:
Corridor — the trash can turns from flat matte to brushed metal with environment reflections:
Game-viewport reference at the same pose as the corridor pair — the Lumen-on capture matches its lighting (the yellow on-screen text is an unrelated engine debug message, viewport-only):
🤖 Generated with Claude Code