Capturing a frame

since 0.7.0 Scene and geometry

Open the live demo · Read the source · View on GitHub

A frame that comes back black could have failed in any of its passes. A capture answers which one, by keeping the pixels every pass wrote rather than only the finished picture.

Step 1: Build a small scene to capture #

A capture is expensive, so it stays off by default and is asked for one frame at a time. This page uses a separate, tiny scene as the frame it captures, away from the scene drawn in the viewport.

final Scene probe = Scene()
  ..add(
    MeshNode(
      DeviceMesh.upload(
        context.device,
        CuboidShape(size: Vector3.all(1.2)).build(),
      ),
      Material(name: 'probe', baseColor: Vector4(0.85, 0.42, 0.18, 1.0)),
      name: 'probe cube',
    ),
  )
  ..add(
    LightNode(name: 'sun', intensity: 3.0)
      ..setLocalForward(Vector3(-0.4, -0.8, -0.3)),
  );
final CameraNode probeCamera = CameraNode()
  ..setPosition(0.0, 0.0, 4.0)
  ..lookAt(Vector3.zero());
probe.add(probeCamera);

Step 2: Ask for the next frame, then render one #

captureNextFrame returns a future for the frame that follows it. It has to be called before render, not after: there is one next frame, and asking late misses it. Once the future completes, every pass the frame ran is there by name, with what it read, what it wrote, and the pixels it produced. firstBlack walks them in order and returns the first pass whose output was already empty, which is normally the one to look at first.

final Future<FrameCapture> pending = context.renderer.captureNextFrame();
context.renderer.render(
  width: 96,
  height: 96,
  scene: probe,
  views: <RenderView>[RenderView(camera: probeCamera)],
  settings: const RenderSettings(),
);
_capture = await pending;

Step 3: The page you can see #

The viewport itself draws an ordinary lit torus, unrelated to the captured probe. The capture already happened during loading, before this scene existed.

final Scene scene = Scene()
  ..ambientColor = Vector3(0.42, 0.48, 0.64)
  ..ambientIntensity = 0.16
  ..add(
    MeshNode(
      DeviceMesh.upload(
        context.device,
        const TorusShape(radius: 1.0, tubeRadius: 0.4).build(),
      ),
      Material(
        name: 'torus',
        baseColor: Vector4(0.28, 0.62, 0.9, 1.0),
        roughness: 0.4,
      ),
      name: 'torus',
    ),
  )
  ..add(
    LightNode(name: 'sun', intensity: 3.2)
      ..setLocalForward(Vector3(-0.42, -0.8, -0.34)),
  );

Step 4: Check the capture recorded something #

The page looks up the scene pass by name, confirms it was active, and checks that the colour image it kept is not entirely black.

final CapturedPass? scenePass = _capture.passNamed('scene');
final CapturedImage? colour = scenePass?.imageOf('hdr_colour');
if (scenePass == null ||
    !scenePass.active ||
    colour == null ||
    colour.isBlack ||
    frame.drawCalls < 1) {
  throw StateError('the captured frame did not record a lit scene pass');
}