Gaussian splats

since 0.7.0 Scene and geometry

Open the live demo · Read the source · View on GitHub

A Gaussian splat represents a soft, oriented ellipsoid instead of a hard triangle. A fitted cloud can reproduce a captured scene from many small, overlapping splats whose colours and opacity blend together.

Step 1: Prepare a tiny cloud #

The sample contains 35 splats arranged as a shallow wave. Each row starts with plain positions, linear colours, opacity, and three ellipsoid scales so the fixture remains readable.

final List<_SplatRow> rows = <_SplatRow>[
  for (var i = 0; i < _count; i++)
    (
      position: Vector3(
        (i % 7 - 3) * 0.5,
        0.4 * math.sin(i * 0.6),
        (i ~/ 7 - 2) * 0.5,
      ),
      colour: Vector3(
        0.3 + 0.5 * (i / _count),
        0.4,
        0.9 - 0.5 * (i / _count),
      ),
      opacity: 0.85,
      scale: 0.16,
    ),
];

Step 2: Decode the PLY #

Capture tools store splats in binary PLY files. parseSplatPly handles three conventions that the header does not express: opacity is a logit, scales are logarithms, and colour channels are zeroth-band spherical-harmonic coefficients. The result is a SplatCloud with flat, render-ready arrays.

final Uint8List bytes = _splatPlyBytes(rows);
_cloud = parseSplatPly(bytes);

Step 3: Add the drawing contributor #

SplatContributor joins the scene pass after opaque meshes. It sorts the cloud back to front for the active view, rebuilds one quad per splat, and draws the whole cloud with alpha blending and no depth write.

_contributor = context.renderer.addContributor(SplatContributor(_cloud));

Step 4: See the file layout #

This helper writes the same 14 float properties used by fitted-splat PLY files. It converts readable colour, opacity, and scale values back to their stored forms, then writes every float in little-endian order after the text header.

/// A binary PLY holding [rows], in exactly the properties `parseSplatPly`
/// requires: `x y z`, `f_dc_0..2`, `opacity`, `scale_0..2`, `rot_0..3`.
Uint8List _splatPlyBytes(List<_SplatRow> rows) {
  const List<String> properties = <String>[
    'x',
    'y',
    'z',
    'f_dc_0',
    'f_dc_1',
    'f_dc_2',
    'opacity',
    'scale_0',
    'scale_1',
    'scale_2',
    'rot_0',
    'rot_1',
    'rot_2',
    'rot_3',
  ];
  final StringBuffer header = StringBuffer()
    ..writeln('ply')
    ..writeln('format binary_little_endian 1.0')
    ..writeln('element vertex ${rows.length}');
  for (final String name in properties) {
    header.writeln('property float $name');
  }
  header.writeln('end_header');
  final List<int> headerBytes = ascii.encode(header.toString());

  final ByteData body = ByteData(rows.length * properties.length * 4);
  var offset = 0;
  // The inverse of `splatChannel` and `splatOpacity`: a readable 0..1 colour
  // and probability, converted back to the coefficient and the logit the
  // file stores.
  double coefficientFor(double channel01) => (channel01 - 0.5) / kSplatShC0;
  double logitFor(double probability) =>
      math.log(probability / (1.0 - probability));
  void writeFloat(double value) {
    body.setFloat32(offset, value, Endian.little);
    offset += 4;
  }

  for (final _SplatRow row in rows) {
    writeFloat(row.position.x);
    writeFloat(row.position.y);
    writeFloat(row.position.z);
    writeFloat(coefficientFor(row.colour.x));
    writeFloat(coefficientFor(row.colour.y));
    writeFloat(coefficientFor(row.colour.z));
    writeFloat(logitFor(row.opacity));
    writeFloat(math.log(row.scale));
    writeFloat(math.log(row.scale));
    writeFloat(math.log(row.scale));
    // rot_0..3 is w, x, y, z in the file's own order; the identity rotation
    // leaves every ellipsoid axis-aligned.
    writeFloat(1.0);
    writeFloat(0.0);
    writeFloat(0.0);
    writeFloat(0.0);
  }

  final Uint8List result = Uint8List(headerBytes.length + body.lengthInBytes);
  result.setRange(0, headerBytes.length, headerBytes);
  result.setRange(headerBytes.length, result.length, body.buffer.asUint8List());
  return result;
}

Step 5: Check sorting and drawing #

The page checks the decoded count, one six-vertex quad per splat, and a sorting result that contains every splat exactly once. It also requires the contributor to add its draw call to the frame.

final Int32List order = _cloud.sortedBackToFront(
  Vector3(0.0, 0.0, -5.0),
  Vector3(0.0, 0.0, 1.0),
);
final Set<int> seen = order.toSet();
if (_cloud.count != _count ||
    seen.length != _count ||
    !_contributor.isActive ||
    frame.drawCalls < 1) {
  throw StateError('the splat cloud was not decoded and drawn');
}