flutter3d
Showcase Changelog 62 packages API reference

A material of your own

since 1.0.0-rc.1 Shading and materials

The material language page showed the source parsed and evaluated by hand. This page is the other end: a material a game writes in a .f3dmat file, compiled the way the build hook compiles one, and drawn by the renderer on three spheres of one instanced batch. It uses the three things the language gives a game beyond a fixed look: a uniform the game sets while it runs, a light block that decides how the surface answers each light, and the four numbers each copy of a batch carries, read as instance.

Step 1: Write the material #

param is a constant: the build folds it into the compiled stage, so it costs nothing and cannot change at run time. uniform is the opposite. It is a member of the material's parameter block, read every frame, so the game can move it. paint starts white.

The light block runs once per light, inside the engine's own light loop. What it returns is how the surface answers that light, and the engine multiplies it by the light's colour, its n·l and its shadow, as it does for every lit model. Here n·l is rounded to one of three bands and divided back out, so the bands are what is left. The fragment body reads lit, which is the surface lit through that block with the ambient added the way the engine adds it, multiplies it by the copy's colour and adds a rim as strong as the copy's fourth number.

  /// The `.f3dmat` this page draws. `user_materials.f3dshaders` beside this
  /// file is what `compileMaterial` makes of it, and carries this text in it;
  /// built, not committed.
  static const String source = '''
material ToonInstances {
  param float bands = 3.0;
  uniform vec3 paint = vec3(1.0, 1.0, 1.0);

  light {
    let banded = floor(nDotL * bands + 0.5) / bands;
    return albedo * paint * banded / max(nDotL, 0.001);
  }

  fragment {
    let rim = pow(1.0 - clamp(nDotV, 0.0, 1.0), 4.0) * instance.w;
    return vec4(lit * instance.rgb + vec3(rim), 1.0);
  }
}
''';

A game keeps this text in assets_src/ as a .f3dmat, and the build hook compiles it into a .f3dshaders bundle with a section for Impeller, one for WebGL2, one for WebGPU, and the source itself for the software rasteriser. The showcase has no hook of its own, so the bundle beside this page is built from source above by the same compiler the hook runs, compileMaterial, before the page is drawn: test/pages_test.dart builds it first. It is not checked in. A bundle carries an Impeller section tied to the Flutter SDK that built it, so one committed would go stale on the next upgrade with nothing to say so.

Step 2: Load the bundle on whatever device is open #

loadShaders takes the bundle on every backend. A GPU device picks its own section. The software rasteriser has no compiled code to pick: it compiles the source with materialLanguageCompiler, which openDevice hands it. The device a test opens has none, so the page gives it the compiler itself.

/// The bundle on [device]: the GPU sections where there is a GPU, and the
/// source compiled into a Dart stage on the software rasteriser.
Future<LoadedShaderLibrary> _load(GraphicsDevice device) async {
  try {
    final LoadedShaderLibrary library = await device.loadShaders(_bytes);
    _how = 'loaded from the bundle';
    return library;
  } on ShaderBundleException {
    // A software device opened without a material compiler refuses the
    // source; this one is given the compiler `openDevice` would give it.
    if (device case final CpuDevice cpu when cpu.materialCompiler == null) {
      _how = 'compiled from the source on the software rasteriser';
      return CpuLoadedShaderLibrary.load(
        cpu.shaders,
        _bytes,
        compiler: materialLanguageCompiler,
      );
    }
    rethrow;
  }
}

Step 3: Bind a material to it #

BundledMaterials reads the bundle's sources and answers two things per material: the lighting model, which tells the renderer what to bind for it (a light block makes it a lit model, so the light list and the shadows are bound), and a default for every uniform. Renderer.addMaterials puts the library in front of the ones the renderer already had, so the stage is found by its name.

final BundledMaterials materials = BundledMaterials.read(_bytes);
_toon = RenderMaterial(
  name: 'toon',
  lighting: materials[_name],
  // `paint` at its default, in a list of the material's own.
  parameters: materials.parameters(_name),
);
try {
  context.renderer.renderSteps.addMaterials(await _load(context.device));
  _drawn = true;
} on ShaderBundleException catch (refusal) {
  _how = 'not drawn here: ${refusal.reason}';
  _toon.lighting = LightingModel.toon;
}

A game that wants a hot reload to reach the material calls HotSwap.loadMaterial instead, which does the same and keeps the binding up to date when the source changes.

Step 4: One batch, three sets of numbers #

addInstance takes a transform and, optionally, four numbers of the game's own. setInstanceData changes them later. They are the only thing that differs between the copies, and there is still one draw.

/// One batch, one draw: the copies differ only in the numbers given each.
InstancedMeshNode _batch(GraphicsDevice device) {
  final InstancedMeshNode batch = InstancedMeshNode(
    DeviceMesh.upload(
      device,
      SphereShape(radius: 0.5, segments: 48, rings: 24).build(),
    ),
    _toon,
    capacity: _numbers.length,
    name: 'copies',
  );
  for (final (int i, (double r, double g, double b, double rim))
      in _numbers.indexed) {
    batch.addInstance(
      Matrix4.translationValues(_x(i), 0.0, 0.0),
      data: Vector4(r, g, b, rim),
    );
  }
  return batch;
}

Step 5: Move the uniform #

The slider writes three numbers into the list the material already holds. The renderer binds Material.parameters every frame, so the next frame shows the warmer paint with nothing rebuilt or reloaded.

/// Warmer paint, written into the list `RenderMaterial.parameters` holds; the
/// renderer binds it every frame, so nothing is rebuilt.
void _setWarmth(double warmth) {
  _warmth = warmth;
  _toon.parameters['paint']!.setAll(0, <double>[
    1.0,
    1.0 - 0.35 * warmth,
    1.0 - 0.7 * warmth,
  ]);
}

Step 6: What the page checks #

The bundle has to carry the very source quoted above. Each sphere's strongest channel has to be the one its own numbers say. Across the sphere with no rim, a row of pixels may take at most five shades: three bands, the unlit side and one boundary. Lit by plain n·l, the same row takes a different value at nearly every pixel. The probe frame is drawn without dither or bloom so that a band is one value from edge to edge.

final String? carried = decodeMaterialSection(
  ShaderBundle.decode(_bytes),
)[_name];
if (carried?.trim() != source.trim()) {
  throw StateError('the bundle was not built from the source on the page');
}
if (!_drawn || _probe.isEmpty) throw StateError('not drawn: $_how');
// Each copy is painted by its own numbers: its strongest channel is the
// one its own colour says.
for (final (int i, (double r, double g, double b, _)) in _numbers.indexed) {
  final (int x, int y) = _centers[i];
  final List<int> got = _rgb(x, y);
  final List<double> want = <double>[r, g, b];
  final int wanted = want.indexOf(want.reduce((a, b) => a > b ? a : b));
  final int strongest = got.indexOf(got.reduce((a, b) => a > b ? a : b));
  if (strongest != wanted) {
    throw StateError('copy $i reads $got, not its own colour $want');
  }
}
// The light comes in bands: across the first copy, which has no rim,
// the shade takes a handful of values where n·l alone would give one
// for nearly every pixel.
final (int cx, int cy) = _centers[0];
final int reach = (_radius * 0.85).floor();
final Set<int> shades = <int>{
  for (int x = cx - reach; x <= cx + reach; x++) _rgb(x, cy)[0],
};
if (shades.length < 2 || shades.length > 5) {
  throw StateError(
    '${2 * reach + 1} pixels across the sphere take ${shades.length} '
    'shades; three bands and the unlit side are at most five',
  );
}

Note. A param is folded when the bundle is built and cannot be moved at run time, and a variant may not set a uniform: the language refuses one. The bundle beside this page is compiled under whichever Flutter SDK built it last, and Impeller refuses a bundle from another; a game does not see this, because its hook compiles on every build, but this page then says why it fell back to the built-in toon model. A uniform added by an edit reaches a material through HotMaterials.bind; one changed by hand in Material.parameters must keep the length the source declares.