Group Research Study

Visual Technology Research • Winter 2026

3D Gaussian Splatting Unreal Engine 5 USD Pipeline VAT Houdini TouchDesigner Substrate Material
Phase 01 — Week 1-2
3DGS Plugin Research
  • Volinga vs Luma AI comparison
  • PLY → XYZ → LiDAR workflow
  • Relighting & post-processing test
Phase 02 — Week 3-4
Pipeline Integration
  • USD Stage in Unreal
  • GSOPs in Houdini
  • TouchDesigner 3DGS test
Phase 03 — Week 5-6
Look Development
  • Stylized 3DGS exploration
  • Landscape in Unreal
  • Perforce pipeline setup
Phase 04 — Week 7-8
Material & Shot Production
  • Marble.ai → PLY + HDRI
  • Substrate SSS material
  • Shot 3 & 4 look dev
Phase 05 — Week 9-10
Documentation & Summary
  • ComfyUI HDRI workflow
  • Interaction design ideation
  • Quarter summary & next steps
Week 01
Fast Start - Topic & Initial Research
Jan 5 - Jan 11, 2026
Completed
  • Gaussian Splatting in UE — Relighting, performance, splat manipulation
  • USD Pipeline — Non-standard data format handling
  • GenAI Integration — Spatial computing applications
  • TouchDesigner — Cross-platform bridging

Core Problem: GS is treated as end product, not source material. Goal: Capture → Manipulate → Interactive Experience

Volinga Plugin — UE 5.6.1

  • Baseline: ~70 fps → Relighting: ~60 fps
  • Additional lights don't scale cost linearly
  • Post-processing (bloom, DOF) works correctly
  • Result: Empty bounding box only
  • No geometry / point cloud / splat attributes

Luma AI Plugin — Niagara-based rendering

  • Supports UE 5.1-5.3
  • Exposure control only (no full relighting)
  • Potential: Niagara manipulation, Blueprint access
Week 02
Technical Exploration & Prototyping
Jan 12 - Jan 18, 2026
Completed

New Approach: LiDAR Point Cloud Plugin Documentation

It seems ply can store vertices and also triangles not unlike OBJ format.

We can modify the header and only preserve the vertex data, then format it as a XYZ, and finally use the Lidar plugin in UE to read those data LiDAR Point Cloud.

Python
.ply to .xyz tool

Above 110 FPS

Color data works

Relighting works

Luma.ai

  • Volinga plugin test results with performance benchmarks
  • LiDAR Point Cloud workflow Python script (PLY header converter) also documented on Github︎︎︎
  • Luma.ai Plugin install
Week 03
Pipeline Integration & Workflow Testing
Jan 19 - Jan 25, 2026
Completed

Troubleshoot

The `Single.usdc` file references an external USD sublayer that is missing:

To make the USD file work correctly, the following structure is needed:

├── Single.usdc
└── obj/
    └── Fern_v1/
        └── Out_Growth.usd
                

To make sure USD Stage, action Requires:

- Place `obj` folder in the same directory as `Single.usdc`

Render Test

USD GeoCache in Sequencer

Pay Attention:

1. Edits in default level sequence will lose if restart Unreal Engine, should create a new level sequence for .usdc and setup geometry cache before render.

2. Material will reset if restart Unreal as well, try to find an approach to override USD Stage by blueprint.

3. Some frames have strange artifacts.

Video: Instance Tool based on Spline

Research:

Performance Notes

  • Geometry Cache (Alembic/USD animation cache) does NOT support native GPU instancing in Unreal Engine.

  • Each Geometry Cache instance:

        - Requires separate draw calls

        - Streams vertex data from disk per instance

        - Performance degrades quickly with multiple instances

        - UsdGeomPointInstancer is NOT fully supported in Unreal Engine's native USD importer.

References:

        - Epic Forums: "Alembic regenerates the mesh every frame"

        - SideFX Forums: "Alembic archives have a lot of overhead when using instancing"

Workarounds:

  • PCG workflow

  • VAT workflow (VAT DOES support GPU instancing)

                - Animation is baked into textures (Position, Normal, etc.)

                - Uses standard Static Mesh with custom material

Conclusion

        For mass instancing of animated geometry: VAT is better.

GSOPs︎︎︎ https://github.com/cgnomads/GSOPs

GSOPs is a free Houdini plugin for Gaussian Splatting editing developed by David Rhodes & Ruben Diaz.

Current Test Status

  • ︎Relight example file working

  • ︎Animated character support confirmed

  • ︎HDRI lighting input supported

Highlight of GSOPs: Coarse Meshing

Notes: .ply works fine by using provided example files but crashed tons if using out source, should look into that and try more files.
︎Update (1/29): Fixed by optimizing splats before convert to VDB

Bake position and color information into textures, then apply to Niagara FX, still in working progress.

  • Integrate 3DGS into USD pipeline. Here is tech breakdown︎︎︎ for AI Summit
  • Test 3GDS in Touch Designer
  • Refine result gallery for AI Summit

Result Gallery @ AI Summit | NVIDIA x SCAD




Touch Designer Test

Pop import doesn’t support 3GDS yet. here is the solution from community https://derivative.ca/community-post/asset/gaussian-splatting/69107

GaussianSplat
├── File Parameter (point to .PLY)
├── Camera Controls
└── renderTOP
                ├── Alpha Threshold ( for performance )
                └── Bitonic Sort
 

Current Limitations:
  • Lighting is baked (limited relighting capabilities)
  • Not flexible for camera settings

Interactive Reference:
  • Photorealistic image → 3DGS
  • Stylized image → 3DGS
  • Custom HLSL/Python Tool
  • Interactive with Mouse hover (X,Y)

Animation driven by noise

Next Steps:

  • Add mouse interaction
  • Add another .ply into scene
  • Landscape exploration in Unreal
  • Look Development
  • Document Perforce setup and tech pipeline
  • AI-assisted Previs video with team
  • Look development for shot4
  • Material exploration
Workflow - From Marble.ai to Unreal




Material Develeopment for VAT tree:
1. changed default lit shader to subsurface
2. added hue shift control



Look development for shot3 and shot4.

  • Look Development for Shot3 & Shot 4
  • Substrate Material Exploration.
  • R&D for Subsurface Scattering approaches in Unreal Engine

Feedback: add more leaves for grape hyacinth, wired roughess map on leaves; should be more SSS intensity for leaves.

Documenting everything. Writing up technical findings, successful workflows, dead ends we hit. Making it clear and shareable.

  • Technical documentation and workflow guides
  • ComfyUI test
  • Layout shot4
Feedback: color should be more purple-ish tint; too much subsurface on buds, should be less.
  • PLY → XYZ → UE5 LiDAR workflow at 110+ FPS
  • Marble.ai outputs PLY and HDRI simultaneously from a single image prompt
  • HDRI style transfer via ComfyUI + ControlNet + FLUX 360 LoRA
  • Tested fern growth animation in Unireal via USD Stage Actor
  • VAT supports GPU instancing unlike Alembic — critical for real-time performance
  • Substrate master material with SSS for leaves and buds
  • Gesture-to-growth concept
  • Ideation: Palm openness → Niagara emission radius; velocity → VAT playback rate; pinch → Gaussian Splat state transition
  • Technical research into Ultraleap, MediaPipe, and GSX Shadows pipeline
  • Interaction prototype — Ultraleap or MediaPipe → VAT playback rate in UE5, first working build
  • Shot 2 production — gesture-driven plant growth, recordable and demo-ready
  • Shot 4 polish — landscape + cherry blossom splats under shared Substrate material

MP4 lockdown — all four shots edited into final deliverable