︎

Group Research Study

Visual Technology Research • Spring 2026

3D Gaussian Splatting Unreal Engine 5 Virtual Production 4DGS RenderStream Hand Tracking
Phase 01 — Week 1-2
Pipeline Setup
  • SplatRenderer UE plugin test
  • 4DGS converter workflow
  • Virtual Production R&D
Phase 02 — Week 3-4
Interaction & Production
  • MediaPipe → UE5
  • Shot 5 production
  • Hand tracking interacion
Phase 03 — Week 5-6
VP Integration
  • Shot 4 polish
  • LED volume test
  • RenderStream setup
Phase 04 — Week 7-8
Optimization
  • GPU profiling
  • LOD system for 3DGS
  • Unreal optimization
Phase 05 — Week 9-10
Delivery
  • Documentation
  • MP4 lockdown
  • Final presentation
Week 01
AI Previs, 4DGS Plugin Test & Interaction Research
Mar 23 - Mar 29, 2026
Completed
The Plan
  • AI Previs Generation — Generate AI previs for Shot 3 & Shot 3B
  • Team Direction — Discuss project direction and creative goals with team
  • 4DGS Plugin Test — Test 4DGS plugins in Unreal & compare performance
  • Interaction R&D — Research interaction mechanics for the experience
  • MediaPipe Prototype — WIP: Using MediaPipe to drive plant growth (Shot 4B Interaction)
AI Previs — Shot 3 & Shot 3B

Generated AI-driven previs to help visualize composition and timing for Shot 3 and Shot 3B before committing to full production.

 
  • Created multiple AI previs iterations for Shot 3
  • Generated Shot 3B variations exploring different camera angles
  • Shared previs with team for feedback and selection
4DGS Plugin Test & Comparison

Tested 4DGS (4D Gaussian Splatting) plugins in Unreal Engine and compared their performance, visual quality, and workflow integration.

  • Installed and configured multiple 4DGS plugins in UE
  • Compared rendering quality, frame rate, and memory usage
  • Tested temporal playback stability across plugins
  • Documented comparison results for team reference
Interaction Research

Explored different approaches for real-time interaction within the experience, focusing on gesture-based and body-tracking methods.

  • Surveyed existing interaction frameworks (MediaPipe, Kinect, Leap Motion)
  • Evaluated gesture recognition accuracy and latency
  • Identified MediaPipe as primary candidate for hand/body tracking
Deliverables
  • AI previs for Shot 3 & Shot 3B
  • 4DGS plugin comparison report
  • Team direction alignment notes
  • Interaction research summary
  • MediaPipe plant growth prototype (WIP)
Week 02
Interaction Fixes, Shader Work & Niagara Particles
Mar 30 - Apr 5, 2026
Completed
The Plan
  • Interaction Bug Fixes — Fix bugs in the MediaPipe interaction system
  • MediaPipe Script Improvements — Add lerp smoothing & fix previous state persistence
  • Daffodil Shader — Help with shader work in Unreal for the daffodil asset
  • Niagara Particles — Build particle system for Shot 5 interaction
MediaPipe Script — Lerp & State Fix

Improved the MediaPipe-to-Unreal pipeline with interpolation for smoother transitions and fixed the previous state issue that was resetting plant growth on each frame.


  • Added lerp (linear interpolation) to smooth gesture-driven values
  • Fixed previous state not being stored correctly between MediaPipe callbacks
  • Plant growth now retains its state when hands leave the frame
  • Tuned interpolation speed for natural-feeling response
Niagara Particle System — Shot 5 Interaction

Building a Niagara particle system in Unreal Engine for the Shot 5 interaction sequence.


  • Integrated particle system with collision solver
  • image sample from a texture to particles.
Deliverables
  • Fixed MediaPipe interaction system
  • Improved lerp smoothing & state persistence in script
  • Daffodil shader progress in Unreal
  • Niagara particle system for Shot 5
Week 03
Stylization, Particles & Interaction Logic
Apr 6 - Apr 12, 2026
Completed
The Plan
  • Post-Process Volume Stylization — Test stylized rendering with post-process volume
  • Niagara Particle — Add Y-axis movement, layered noise and wind forces
  • Shot 4B Landscape — Help with landscape asset editing for Shot 4B
  • Interaction Logic— Fix interaction to work across different maps in editor
Post-Process Volume — Stylization Test

Explored stylized rendering approaches using Unreal's post-process volume to achieve a more artistic look for the experience.

  • Tested various post-process materials for stylization effects
  • Works for static but nor suitable for moving images.
Value:1 Value:0.2

Niagara Particle Enhancement

Enhanced the Niagara particle system with more dynamic movement and natural behavior by adding vertical motion, multi-layer noise, and wind simulation.

  • Added Y-axis velocity component for vertical particle movement
  • Integrated wind force module for natural particle drift.
Blockout in Blender Blockout in Unreal RenderTest in Unreal


Interaction Logic — Cross-Map Support

Optimized the interaction system architecture so it now works across different maps within the editor, improving workflow flexibility.

  • Interaction now functions correctly in editor mode across all project maps
  • Improved modularity for easier integration with different scenes

Next Steps
  • Add daffodil leaves to the particle/interaction system
  • Configure particle system with MediaPipe hand tracking interaction
Deliverables
  • Post-process volume stylization tests
  • Enhanced Niagara system with Y-axis motion, noise layers, and wind
  • Shot 4B landscape asset edits
  • Cross-map interaction system

Let's connect and create something amazing together.

Pages

Social