flutter3d
Showcase Changelog 62 packages API reference

Gaussian splats

since 0.7.0 Scene and geometry

A Gaussian splat represents a soft, oriented ellipsoid instead of a hard triangle. A fitted cloud can reproduce a captured scene from many small, overlapping splats whose colours and opacity blend together.

Step 1: Prepare a tiny cloud #

The sample contains 35 splats arranged as a shallow wave. Each row starts with plain positions, linear colours, opacity, and three ellipsoid scales so the fixture remains readable.

final List<_SplatRow> rows = <_SplatRow>[
  for (var i = 0; i < _count; i++)
    (
      position: Vector3(
        (i % 7 - 3) * 0.5,
        0.4 * math.sin(i * 0.6),
        (i ~/ 7 - 2) * 0.5,
      ),
      color: Vector3(
        0.3 + 0.5 * (i / _count),
        0.4,
        0.9 - 0.5 * (i / _count),
      ),
      opacity: 0.85,
      scale: 0.16,
    ),
];

Step 2: Decode the PLY #

Capture tools store splats in binary PLY files. parseSplatPly handles three conventions that the header does not express: opacity is a logit, scales are logarithms, and colour channels are zeroth-band spherical-harmonic coefficients. The result is a SplatCloud with flat, render-ready arrays.

final Uint8List bytes = _splatPlyBytes(rows);
_cloud = parseSplatPly(bytes);

Step 3: Add the drawing contributor #

SplatContributor joins the scene pass after opaque meshes. It sorts the cloud back to front for the active view, rebuilds one quad per splat, and draws the whole cloud with alpha blending and no depth write.

_contributor = SplatContributor(_cloud);
context.renderer.renderSteps.addContributor(_contributor);

Step 4: See the file layout #

This helper writes the same 14 float properties used by fitted-splat PLY files. It converts readable colour, opacity, and scale values back to their stored forms, then writes every float in little-endian order after the text header.

/// A binary PLY holding [rows], in exactly the properties `parseSplatPly`
/// requires: `x y z`, `f_dc_0..2`, `opacity`, `scale_0..2`, `rot_0..3`.
Uint8List _splatPlyBytes(List<_SplatRow> rows) {
  const List<String> properties = <String>[
    'x',
    'y',
    'z',
    'f_dc_0',
    'f_dc_1',
    'f_dc_2',
    'opacity',
    'scale_0',
    'scale_1',
    'scale_2',
    'rot_0',
    'rot_1',
    'rot_2',
    'rot_3',
  ];
  final StringBuffer header = StringBuffer()
    ..writeln('ply')
    ..writeln('format binary_little_endian 1.0')
    ..writeln('element vertex ${rows.length}');
  for (final String name in properties) {
    header.writeln('property float $name');
  }
  header.writeln('end_header');
  final List<int> headerBytes = ascii.encode(header.toString());

  final ByteData body = ByteData(rows.length * properties.length * 4);
  var offset = 0;
  // The inverse of `splatChannel` and `splatOpacity`: a readable 0..1 colour
  // and probability, converted back to the coefficient and the logit the
  // file stores.
  double coefficientFor(double channel01) => (channel01 - 0.5) / splatShC0;
  double logitFor(double probability) =>
      math.log(probability / (1.0 - probability));
  void writeFloat(double value) {
    body.setFloat32(offset, value, Endian.little);
    offset += 4;
  }

  for (final _SplatRow row in rows) {
    writeFloat(row.position.x);
    writeFloat(row.position.y);
    writeFloat(row.position.z);
    writeFloat(coefficientFor(row.color.x));
    writeFloat(coefficientFor(row.color.y));
    writeFloat(coefficientFor(row.color.z));
    writeFloat(logitFor(row.opacity));
    writeFloat(math.log(row.scale));
    writeFloat(math.log(row.scale));
    writeFloat(math.log(row.scale));
    // rot_0..3 is w, x, y, z in the file's own order; the identity rotation
    // leaves every ellipsoid axis-aligned.
    writeFloat(1.0);
    writeFloat(0.0);
    writeFloat(0.0);
    writeFloat(0.0);
  }

  final Uint8List result = Uint8List(headerBytes.length + body.lengthInBytes);
  result.setRange(0, headerBytes.length, headerBytes);
  result.setRange(headerBytes.length, result.length, body.buffer.asUint8List());
  return result;
}

Step 5: Check sorting and drawing #

The page checks the decoded count, one six-vertex quad per splat, and a sorting result that contains every splat exactly once. It also requires the contributor to add its draw call to the frame.

final Int32List order = _cloud.sortedBackToFront(
  Vector3(0.0, 0.0, -5.0),
  Vector3(0.0, 0.0, 1.0),
);
final Set<int> seen = order.toSet();
if (_cloud.count != _count ||
    seen.length != _count ||
    !_contributor.isActive ||
    frame.drawCalls < 1) {
  throw StateError('the splat cloud was not decoded and drawn');
}

Step 6: Sort on the GPU where the device computes #

Sorting is the part of a splat cloud that grows with the capture. Where the device has compute shaders, which today means WebGPU, SplatContributor sorts on the GPU instead: two radix passes over the depth keys, and the last pass writes the index buffer the draw reads, so the order never comes back to the CPU. Every other device, including Impeller, WebGL2 and the software renderer, keeps the CPU sort. The CPU sort is the reference: the GPU's order matches it splat for splat, and the picture is the same to the byte.

gpuSort is on by default and takes effect only where the device can run it. GPU sort turns it off, which on WebGPU sorts on the CPU instead and anywhere else changes nothing. A cloud larger than splatGpuSortLimit splats stays on the CPU too.

// On by default: the device sorts where it can compute, the CPU
// everywhere else. Off sorts on the CPU even where the device could.
_contributor.gpuSort = gpuSort;

The page checks that the cloud was drawn in the GPU's order exactly when the toggle is on and the device can sort. On the software device that runs the page's test, that is never.

// The cloud was drawn in the GPU's order exactly when the toggle is on
// and this device can run the sort: never on the software device, which
// has no compute.
final bool gpuExpected = gpuSort && SplatGpuSort.availableOn(_device);
if (_contributor.didDrawGpuOrder != gpuExpected) {
  throw StateError(
    'drew in the GPU order: ${_contributor.didDrawGpuOrder}, '
    'expected $gpuExpected',
  );
}