Splats under a budget

since 0.8.0 Scene and geometry

Open the live demo · Read the source · View on GitHub

A captured cloud is often millions of Gaussians, and from a distance most of them cover a pixel or less. Drawing every one of them costs the same whether you can see the detail or not.

buildSplatOctree cuts a cloud into a tree. The leaves are the original splats. Every node above them holds its children's splats merged into fewer, larger ones. SplatLod then picks a cut through the tree that fits a splat budget: fine where the cloud is near the eye, coarse where it is far.

Step 1: A cloud to cut #

This page makes its cloud in Dart, so it needs no file: 2400 small round splats on the surface of a ring, coloured by where they sit. A cloud read with parseSplatPly or from a glTF file goes into the tree the same way.

const int count = _around * _tube;
final Float32List centres = Float32List(count * 3);
final Float32List colours = Float32List(count * 4);
final Float32List scales = Float32List(count * 3);
final Float32List rotations = Float32List(count * 4);
for (var i = 0; i < count; i++) {
  final double a = (i ~/ _tube) / _around * 2.0 * math.pi;
  final double b = (i % _tube) / _tube * 2.0 * math.pi;
  final double ring = 1.4 + 0.45 * math.cos(b);
  centres.setAll(i * 3, <double>[
    ring * math.cos(a),
    0.45 * math.sin(b),
    ring * math.sin(a),
  ]);
  colours.setAll(i * 4, <double>[
    0.5 + 0.5 * math.cos(a),
    0.5 + 0.5 * math.sin(b),
    0.5 - 0.5 * math.cos(a),
    0.9,
  ]);
  scales.setAll(i * 3, <double>[0.05, 0.05, 0.05]);
  rotations[i * 4 + 3] = 1.0;
}
final SplatCloud cloud = SplatCloud(
  centres: centres,
  colours: colours,
  scales: scales,
  rotations: rotations,
);

Step 2: Build the tree #

A box that holds at most leafCapacity splats is a leaf. A bigger box is split into eight, and its own splats are its children's merged on a grid by grid by grid lattice over the box, so a node holds at most grid cubed of them. The defaults are 512 and 8. This cloud is small, so the page asks for 64 and 4, which gives the tree a few levels to choose between.

A merged splat is not one of the originals picked to survive. It is a new Gaussian with the same weighted centre and spread as the splats it replaces, weighted by their opacity and size, so it covers what they covered.

_tree = buildSplatOctree(cloud, leafCapacity: 64, grid: 4);

Step 3: Draw it through a budget #

SplatContributor.lod draws whatever cut SplatLod last chose. The cut starts at the root and keeps refining the node that looks largest from the eye, its radius over the distance from the eye to the near side of its sphere (the distance to its centre less its radius), as long as that node's children still fit in the budget. It never holds more splats than the budget. A budget below the root's own count draws nothing, since there is no coarser cut to give.

_lod = SplatLod(_tree, budget: budget.round());
context.renderer.addContributor(SplatContributor.lod(_lod));

Step 4: Change the budget #

The budget is a field, set here every frame from the slider. The cut is not chosen every frame. It is chosen when the cloud is sorted again: when the eye has moved far enough, when the cloud moves, when a page of the tree arrives, or when the budget changes.

_lod.budget = budget.round();

Step 5: Try the slider #

The slider starts at the root's own count, the coarsest cut there is, and ends at the whole cloud. Low, the ring is a few dozen soft blobs. Raise it and the side nearest you sharpens first. At the top every node refines, the cut is the leaves, and you see the original 2400 splats with nothing merged. Orbit close to one side at a middle budget and the detail follows you there.

SliderControl(
  'Splat budget',
  min: _tree.nodes.first.splatCount.toDouble(),
  max: _tree.leafSplatCount.toDouble(),
  value: () => budget,
  onChanged: (double v) => budget = v,
  format: (double v) => '${v.round()} of ${_tree.leafSplatCount}',
),

Step 6: Check the cut #

After one frame the page checks that a cut was chosen, that it holds no more splats than the budget, and that at the starting budget it is coarser than the whole cloud.

if (_tree.leafSplatCount != _around * _tube ||
    _lod.cut.isEmpty ||
    _lod.cutSplatCount > _lod.budget ||
    _lod.cutSplatCount >= _tree.leafSplatCount ||
    frame.drawCalls < 1) {
  throw StateError('the tree was not drawn as a cut under its budget');
}

Note. A tree can also be written to a .f3dsplat file and read back through PagedSplatOctree, which asks for deeper pages only where the cut wants them. This page keeps the whole tree in memory.