Impostors

since 0.8.0 Scene and geometry

Open the live demo · Read the source · View on GitHub

A tree forty metres away covers a few dozen pixels, and drawing all its triangles to fill them is mostly wasted work. An impostor replaces the far tree with a single square card. The card always turns to face the camera, and it shows a picture of the tree taken from roughly the direction you are looking.

Those pictures are baked ahead of time: 8 by 8 views of the model from all around it, packed into one texture called an octahedral atlas. The map puts straight up in the middle of the atlas and straight down in its four corners, so the views a tree is usually seen from (level, or from above) get the middle. A second atlas holds the surface normal and depth for each texel, which is how the scene's lights can still shade the card.

Step 1: A tree with sides that differ #

The tree is a trunk, two balls of leaves and one red fruit on the +X side. Because of the fruit, the tree looks different from each side, and that lets you check that the card picks the right view.

static Aabb3 get _trunk =>
    Aabb3.minMax(Vector3(-0.12, 0.0, -0.12), Vector3(0.12, 1.2, 0.12));
static List<_Ball> get _balls => <_Ball>[
  (centre: Vector3(0.0, 1.75, 0.0), radius: 0.75, colour: _leaves),
  (centre: Vector3(0.0, 2.4, 0.0), radius: 0.45, colour: _leaves),
  // One fruit on the +X side, so the tree looks different from each side.
  (centre: Vector3(0.8, 1.45, 0.0), radius: 0.25, colour: _fruit),
];

The parts are merged into one mesh, each painted with its own vertex colour. One mesh matters here: a model node only switches through a level of detail chain when every level is a single surface. The card is a square around the sphere that holds the whole mesh, so the page measures that sphere as well.

final MeshData tree = MeshData.merge(<MeshData>[
  _painted(
    CuboidShape(
      size: _trunk.max - _trunk.min,
    ).build().transformed(Matrix4.translation(_trunk.center)),
    _bark,
  ),
  for (final _Ball ball in _balls)
    _painted(
      SphereShape(
        radius: ball.radius,
        segments: 24,
        rings: 12,
      ).build().transformed(Matrix4.translation(ball.centre)),
      ball.colour,
    ),
]);
final Vector3 centre = tree.computeBounds().center;
// A little over the furthest vertex: a ball's true edge can lie between
// the vertices of its mesh.
final double radius = _radiusAround(tree, centre) * 1.02;

Step 2: Bake the views #

With a real model you do not write this step. When flutter3d_build converts a model with --impostor, it adds an impostor as the last level of detail of every node that draws something, except skinned and morphed ones, since a card holds one pose. It bakes the atlases on the software rasteriser, so every machine that converts the model gets the same bytes, and the .f3d file stores them in a section of their own. This page bakes a small atlas itself only so that it needs no asset.

Every view looks back along impostorViewDirection at the tree's sphere. The camera's right hand is impostorRight, and the top row of a view is the side its up axis points to. That is the layout the impostor shader reads, and if one sign here differed, the view would come out mirrored. The tree is made of simple shapes, so instead of drawing each view, the page casts one ray per texel and records what the ray hits. The albedo's alpha is coverage. The normal is written in the tree's own space as n * 0.5 + 0.5, and the depth goes in alpha, from 0 at the near side of the sphere to 1 at the far side.

/// Both atlases, [kImpostorGrid] by [kImpostorGrid] views of [ImpostorsDemo._cell]
/// texels, found by casting one ray a texel at the tree's own shapes.
///
/// Each view looks back along [impostorViewDirection] at the sphere of
/// [radius] around [centre], with [impostorRight] to its right, and the top
/// row of a view is the side its up axis points to: the layout the impostor
/// shader reads.
({Uint8List albedo, Uint8List normalDepth}) _bake(
  Vector3 centre,
  double radius,
) {
  const int cell = ImpostorsDemo._cell;
  const int side = cell * kImpostorGrid;
  final Uint8List albedo = Uint8List(side * side * 4);
  final Uint8List normalDepth = Uint8List(side * side * 4);
  int byte(double v) => (v.clamp(0.0, 1.0) * 255.0).round();
  final List<_Ball> balls = ImpostorsDemo._balls;
  final Aabb3 trunk = ImpostorsDemo._trunk;
  final Vector3 bark = ImpostorsDemo._bark;
  final Vector3 empty = ImpostorsDemo._leaves;

  for (var row = 0; row < kImpostorGrid; row++) {
    for (var column = 0; column < kImpostorGrid; column++) {
      final Vector3 d = impostorViewDirection(column, row);
      final Vector3 right = impostorRight(d);
      final Vector3 up = d.cross(right);
      for (var y = 0; y < cell; y++) {
        for (var x = 0; x < cell; x++) {
          final Vector3 origin =
              centre +
              right.scaled(((x + 0.5) / cell * 2.0 - 1.0) * radius) +
              up.scaled((1.0 - (y + 0.5) / cell * 2.0) * radius) +
              d.scaled(2.0 * radius);
          final _Hit? hit = _cast(origin, -d, balls, trunk, bark);
          final int to = ((row * cell + y) * side + column * cell + x) * 4;
          // An empty texel still gets a leaf colour, with no coverage, so a
          // filtered read at the edge of the tree does not mix in black.
          final Vector3 colour = hit?.colour ?? empty;
          albedo
            ..[to] = byte(colour.x)
            ..[to + 1] = byte(colour.y)
            ..[to + 2] = byte(colour.z)
            ..[to + 3] = hit == null ? 0 : 255;
          if (hit == null) continue;
          final Vector3 point = origin + (-d).scaled(hit.t);
          // How far along the view the surface is: 0 at the near side of the
          // sphere, 1 at the far side.
          final double depth = 0.5 - (point - centre).dot(d) / (2.0 * radius);
          normalDepth
            ..[to] = byte(hit.normal.x * 0.5 + 0.5)
            ..[to + 1] = byte(hit.normal.y * 0.5 + 0.5)
            ..[to + 2] = byte(hit.normal.z * 0.5 + 0.5)
            ..[to + 3] = byte(depth);
        }
      }
    }
  }
  return (albedo: albedo, normalDepth: normalDepth);
}

At 32 texels per view, each atlas is 256 texels square. Both are uploaded as plain RGBA textures.

final ({Uint8List albedo, Uint8List normalDepth}) atlas = _bake(
  centre,
  radius,
);
const int side = _cell * kImpostorGrid;
TextureHandle upload(Uint8List rgba) => device.createTextureFromPixels(
  width: side,
  height: side,
  format: TextureFormat.r8g8b8a8UNormInt,
  pixels: ByteData.sublistView(rgba),
)!;
final TextureHandle albedo = upload(atlas.albedo);
final TextureHandle normalDepth = upload(atlas.normalDepth);

Step 3: End the chain in a card #

A ModelLod.impostor is a level of detail whose content is a picture instead of surfaces. It carries a ModelImpostor, which gives the grid size and the sphere the views were taken around. It is always the coarsest level. maxScreenFraction is the share of the screen's height below which the card takes over. Here it is a quarter, so the switch happens within a short row of trees. The converter picks a smaller number: half the coarsest mesh level's own threshold, and never more than a tenth.

A loader fills ModelAsset.impostors from the file. Here the page fills it directly: one card mesh and the two atlases, uploaded once however many copies of the tree stand in the scene.

final ModelImpostor impostor = ModelImpostor(
  // Indices into a file's images. Nothing is read through them here: the
  // textures are handed over below.
  albedoImage: 0,
  normalDepthImage: 1,
  grid: kImpostorGrid,
  centre: centre,
  radius: radius,
);
final ModelAsset asset = ModelAsset(
  name: 'tree',
  parts: <ModelPart>[
    ModelPart(
      mesh: DeviceMesh.upload(device, tree),
      material: Material(name: 'tree', roughness: 0.8),
      name: 'tree',
    ),
  ],
  nodes: <ModelNode>[
    ModelNode(
      name: 'tree',
      surfaces: const <int>[0],
      lods: <ModelLod>[
        ModelLod.impostor(impostor: impostor, maxScreenFraction: 0.25),
      ],
    ),
  ],
  roots: const <int>[0],
  localBounds: tree.computeBounds(),
  impostors: <ModelImpostor, ImpostorPart>{
    impostor: (
      card: DeviceMesh.upload(
        device,
        impostorCard(centre: centre, radius: radius),
      ),
      albedo: albedo,
      normalDepth: normalDepth,
    ),
  },
);

Step 4: Plant a row #

Each instance becomes an LodGroup with two levels: the mesh, and an ImpostorNode that draws the shared card. The renderer picks a level for each group every frame from how much of the view the tree covers. The near trees stay meshes and the far ones become cards. Zoom out or orbit and watch them switch.

for (var i = 0; i < _count; i++) {
  asset
      .instantiate(_scene, name: 'tree $i')
      .root
      .setPosition(i.isEven ? -1.5 : 1.5, 0.0, -2.0 - 6.0 * i);
}

Step 5: The same light on both #

The card is drawn like any other mesh, with LightingModel.impostor, and it is lit by the same lights. For each pixel the shader reads the three baked views nearest the direction it is seen from and blends them. It reads the normal from the second atlas, so a card darkens on the side away from the sun the way the mesh does. The light is the same, but the shading is not quite: the mesh is lit with the full PBR model, while the card stage is diffuse only, because the atlases carry no roughness or metal. At the distance a tree turns into a card its highlights would be smaller than a pixel anyway. Turn Sun direction and compare a near tree with a far one.

_sun = LightNode(name: 'sun', intensity: 3.0)..castsShadow = true;

_sun.setLocalForward(
  Vector3(math.sin(sunYaw), -0.9, math.cos(sunYaw))..normalize(),
);

Cards receive shadows but cast none. The shadow passes would draw the card as it was built, an upright square, not as it is seen. So look at the ground beside each tree. A near tree, drawn as a mesh, has its shadow there. A far tree, drawn as a card, has none.

Note. Each card is its own draw. A forest of cards is one draw per tree, not one instanced draw, because the instanced path uses the engine's own vertex stage, which does not know how to turn a card to the eye.

Step 6: Tell them apart #

Switch on Tint the cards blue to see which trees are cards. The base colour of the impostor material tints the atlas the way it tints any texture.

for (final LodGroup group in _scene.lodGroups) {
  for (final LodLevel level in group.levels) {
    if (level.node is ImpostorNode) {
      level.node.material.baseColor.setValues(
        markCards ? 0.55 : 1.0,
        markCards ? 0.75 : 1.0,
        1.0,
        1.0,
      );
    }
  }
}

The page checks that the row has five groups, that at least one of them is showing its card and at least one its mesh, and that the frame drew something.

final List<LodGroup> groups = scene.lodGroups;
bool showsCard(LodGroup g) =>
    g.levels[g.activeLevel].node is ImpostorNode &&
    g.levels[g.activeLevel].node.visible;
if (groups.length != _count ||
    !groups.any(showsCard) ||
    !groups.any((LodGroup g) => g.activeLevel == 0) ||
    frame.drawCalls < 1) {
  throw StateError('the row should end in cards and start in meshes');
}