GSplat Extra Streams and Work Buffer Customization

[mvaligursky]

GSplat Extra Streams and Work Buffer Customization

Overview

Adds support for custom texture streams on gsplat resources and per-instance shader customization in unified rendering mode.

Features

Extra Streams Support

• GSplatFormat.addExtraStreams(streams) - Add custom texture streams to a gsplat format
  • instance: false (default) - texture shared across all instances of the resource
  • instance: true - texture created per gsplat component instance
• GSplatComponent.getInstanceTexture(name) - Access instance-level textures for writing custom data
• Streams are additive-only (cannot be removed) to ensure rendering stability

Work Buffer Customization

• GSplatComponent.setWorkBufferModifier({ glsl, wgsl }) - Inject custom shader code to modify splats during work buffer rendering
  • Access to modifySplatCenter(center), modifySplatRotationScale(originalCenter, center, rotation, scale), and modifySplatColor(center, color) functions
  • Custom streams accessible via generated loadStreamName() functions

Work Buffer Update Control

New constants for controlling work buffer update frequency:

• WORKBUFFER_UPDATE_AUTO - Update only when needed (default)
• WORKBUFFER_UPDATE_ONCE - Update next frame, then revert to AUTO
• WORKBUFFER_UPDATE_ALWAYS - Update every frame (for time-based effects)

Set via gsplatComponent.workBufferUpdate = pc.WORKBUFFER_UPDATE_ALWAYS

New Public API

Constants

Constant	Value	Description
WORKBUFFER_UPDATE_AUTO	0	Work buffer updated only when needed (transform, format, LOD changes)
WORKBUFFER_UPDATE_ONCE	1	Work buffer updated next frame, then reverts to AUTO
WORKBUFFER_UPDATE_ALWAYS	2	Work buffer updated every frame

GSplatFormat

addExtraStreams(streams)

Adds additional texture streams for custom gsplat data.

Parameters:

• streams - Array<GSplatStreamDescriptor> - Array of stream descriptors

GSplatStreamDescriptor:

Property	Type	Default	Description
name	string	required	Texture uniform name (also generates loadName() function)
format	number	required	Pixel format (e.g. PIXELFORMAT_RGBA8, PIXELFORMAT_RGBA32F)
instance	boolean	false	If true, texture is per gsplat component instance

extraStreams (getter)

Returns the array of extra streams. Read-only, do not modify directly.

GSplatComponent

getInstanceTexture(name)

Returns the instance-level texture for the given stream name.

Parameters:

• name - string - The stream name

Returns: Texture | undefined

setWorkBufferModifier(value)

Sets custom shader code for modifying splats when written to the work buffer.

Parameters:

• value - { glsl?: string, wgsl?: string } | null - Shader code object or null to clear

workBufferUpdate

Controls how often the work buffer is updated for this component.

Type: number (use WORKBUFFER_UPDATE_* constants)

Default: WORKBUFFER_UPDATE_AUTO

GSplatContainer

Constructor Change

The constructor now takes maxSplats instead of numSplats:

new GSplatContainer(device, maxSplats, format)

maxSplats (getter)

Returns the maximum number of splats the container can hold (set at construction).

Type: number (read-only)

numSplats (getter/setter)

Controls the number of splats to render. Must be between 0 and maxSplats.

Type: number

Default: maxSplats

Note: Changing this value triggers internal buffer reallocation in unified rendering mode. Avoid calling this every frame. For per-frame visibility control, consider using the format's read shader code that returns splatScale = vec3(0.0) for splats you want to hide.

---

Usage Examples

Adding Custom Color Per Instance

// Add an instance-level color stream to the resource's format
resource.format.addExtraStreams([
    { name: 'instanceColor', format: pc.PIXELFORMAT_RGBA8, instance: true }
]);

// Access the texture and fill with color data
const texture = entity.gsplat.getInstanceTexture('instanceColor');
const pixels = texture.lock();
// Fill with RGBA values...
texture.unlock();

// Set shader to apply the color
entity.gsplat.setWorkBufferModifier({
    glsl: `
        void modifySplatColor(vec3 center, inout vec4 color) {
            vec4 tint = loadInstanceColor();
            color.rgb *= tint.rgb;
        }
    `,
    wgsl: `
        fn modifySplatColor(center: vec3f, color: ptr<function, vec4f>) {
            let tint = loadInstanceColor();
            (*color).rgb *= tint.rgb;
        }
    `
});

[Copilot]

Pull request overview

This PR adds support for custom texture streams on gsplat resources and per-instance shader customization in unified rendering mode. It enables users to store custom per-splat or per-instance data and modify splats during work buffer rendering via custom shader code.

Changes:

• Added GSplatFormat.addExtraStreams() API to define custom texture streams (resource-level or instance-level)
• Added GSplatComponent.setWorkBufferModifier() to inject custom shader code for modifying splats during work buffer rendering
• Added work buffer update control via new constants (WORKBUFFER_UPDATE_AUTO, WORKBUFFER_UPDATE_ONCE, WORKBUFFER_UPDATE_ALWAYS)

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/scene/shader-lib/wgsl/chunks/gsplat/frag/gsplatCopyToWorkbuffer.js	Integrates custom modifier functions into work buffer rendering (WGSL)
src/scene/shader-lib/glsl/chunks/gsplat/frag/gsplatCopyToWorkbuffer.js	Integrates custom modifier functions into work buffer rendering (GLSL)
src/scene/gsplat/gsplat-streams.js	Adds instance-level texture management and dynamic stream synchronization
src/scene/gsplat/gsplat-format.js	Implements extra streams API with version tracking and shader code generation
src/scene/gsplat/gsplat-container.js	Changes constructor to use maxNumSplats and adds dynamic numSplats setter
src/scene/gsplat/gsplat-resource-base.js	Updates material configuration to support custom modifiers and format declarations
src/scene/gsplat/gsplat-resource.js	Changes format property from public to protected
src/scene/gsplat/gsplat-compressed-resource.js	Changes format property from public to protected
src/scene/gsplat/gsplat-sog-resource.js	Changes format property from public to protected
src/scene/gsplat-unified/gsplat-placement.js	Adds work buffer modifier support, instance texture management, and update mode control
src/scene/gsplat-unified/gsplat-info.js	Captures format state and instance streams for rendering
src/scene/gsplat-unified/gsplat-manager.js	Integrates state tracking and instance stream initialization
src/scene/gsplat-unified/gsplat-work-buffer-render-pass.js	Binds instance textures and passes modifier to render info
src/scene/gsplat-unified/gsplat-placement-state-tracker.js	New file tracking placement state changes for work buffer invalidation
src/scene/gsplat/gsplat-instance.js	Updates material configuration call signature
src/scene/constants.js	Defines new work buffer update mode constants
src/framework/components/gsplat/component.js	Exposes public API for work buffer modifiers and instance textures
examples/src/examples/gaussian-splatting/procedural-instanced.example.mjs	Demonstrates dynamic numSplats modification

Comments suppressed due to low confidence (1)

src/framework/components/gsplat/component.js:1

• The example incorrectly assigns to extraStreams property directly, but the actual API is addExtraStreams() method. This should be: resource.format.addExtraStreams([...]). The property is read-only according to the implementation.

import { hashCode } from '../../../core/hash.js';

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Treetosayoot]

Born to be free