flutter3d
Showcase Changelog 62 packages API reference

Photo mode

since 1.0.0-rc.1 Views, picking and input

Photo mode is four things. The world stops. A free camera flies through the level but cannot leave it. A filter goes over the game's own grade. And the picture is drawn at whatever size the player asks for, larger than the screen and larger than any render target the device will make. The platformer has it on P, and this page shows each piece on its own.

Switch on Photo mode. The crate stops turning, and the view becomes a free camera. It starts from the orbit camera's place outside the room and ends up inside it, within its tether of the player. From there it flies a slow loop that keeps pushing into the walls. Filter picks one of the eight looks.

Step 1: A level with walls in it #

The camera is held by the level's collision world, the same one the game already steps against, so photo mode needs no geometry of its own. This room is closed on every side, with a pillar and a crate. The page draws the walls low so the orbit camera can see in, but their colliders are full height under a ceiling.

/// A room eight metres square and three high, closed on every side, with a
/// pillar and a crate in it. The boxes are kept as well as handed to the
/// collision world, so the check can ask whether a point is inside one
/// without going through the camera it is checking.
({CollisionWorld world, List<Aabb3> solids, Aabb3 bounds}) _room() {
  final world = CollisionWorld();
  final solids = <Aabb3>[];
  void box(Vector3 center, Vector3 size) {
    world.addBox(center, size);
    solids.add(Aabb3.centerAndHalfExtents(center, size / 2.0));
  }

  box(Vector3(0.0, -0.25, 0.0), Vector3(9.0, 0.5, 9.0)); // floor
  box(Vector3(0.0, 3.25, 0.0), Vector3(9.0, 0.5, 9.0)); // ceiling
  box(Vector3(4.25, 1.5, 0.0), Vector3(0.5, 3.0, 9.0));
  box(Vector3(-4.25, 1.5, 0.0), Vector3(0.5, 3.0, 9.0));
  box(Vector3(0.0, 1.5, 4.25), Vector3(9.0, 3.0, 0.5));
  box(Vector3(0.0, 1.5, -4.25), Vector3(9.0, 3.0, 0.5));
  box(Vector3(1.5, 1.5, -1.0), Vector3(1.0, 3.0, 1.0)); // pillar
  box(Vector3(-1.5, 0.4, 1.0), Vector3.all(0.8)); // crate
  world.update();
  return (
    world: world,
    solids: solids,
    bounds: Aabb3.minMax(Vector3(-4.0, 0.0, -4.0), Vector3(4.0, 3.0, 4.0)),
  );
}

Step 2: Stop the world #

shouldPause is the one place a game decides whether its step runs. With photoMode it answers yes whatever the pointer and the pad say, because both of them are flying the camera now. A game that let the pointer decide would start running again the moment the player dragged to frame the shot.

// Photo mode pauses the world whatever the pointer and the pad say,
// since both are flying the camera.
final bool paused = shouldPause(
  ready: true,
  menuOpen: false,
  pointerIsTheGate: false,
  pointerHeld: false,
  padConnected: false,
  photoMode: photoMode,
);
if (!paused) {
  _clock += dt;
  _spinner.setRotation(Quaternion.axisAngle(Vector3(0, 1, 0), _clock));
}

Step 3: A free camera that stays inside #

PhotoCamera flies with look, tilt and zoom, and moves along its own axes with up being the world's. It is held in three ways at once. A tether of reach metres keeps it round where the player stood. The level's box, when there is one, keeps it inside. And the walls stop it, because each move is swept as a small sphere and slides along whatever it meets.

begin does not put the camera where the game's camera was. It starts at the player and sweeps out toward it. A chase camera can be left behind a wall for a frame when the player turns sharply, and the photo should not start on the far side of that wall. Here the game's camera is outside the room entirely, and the free camera starts just inside the wall.

final camera = PhotoCamera(
  world: _level.world,
  bounds: _level.bounds,
  reach: 5.0,
);
// The game's camera is behind the far wall; the player is inside.
camera.begin(
  eye: Vector3(0.3, 1.5, 9.0),
  target: Vector3(0.0, 1.0, 0.0),
  anchor: Vector3(0.0, 1.2, 2.0),
);
report.startZ = camera.eye.z;
final dice = GameRandom(5);
for (var i = 0; i < 600; i++) {
  camera
    ..look((dice.nextDouble() - 0.5) * 0.6, (dice.nextDouble() - 0.5) * 0.3)
    ..fly(
      Vector3(
        dice.nextDouble() * 6.0 - 3.0,
        dice.nextDouble() * 6.0 - 3.0,
        dice.nextDouble() * 6.0 - 3.0,
      ),
      0.25,
    );
  final Vector3 eye = camera.eye;
  if (!_level.bounds.containsVector3(eye) ||
      _level.solids.any((Aabb3 box) => box.containsVector3(eye))) {
    report.escapes++;
  }
  report.moves += 1;
  report.furthest = math.max(
    report.furthest,
    eye.distanceTo(camera.anchor),
  );
}

Six hundred random flights follow, each a quarter of a second in a random direction after a random turn. None of them may end inside a solid or outside the room, and none may end further from the player than the tether.

Step 4: A picture in tiles #

capturePhoto draws the picture in tiles on the renderer the game already has. Each tile is drawn through a CropProjection of the camera's own projection, which takes the whole frame's aspect rather than the tile's, so a narrow tile at the edge does not squash what it shows. Each tile also has a margin of picture round it that is drawn and then cropped away, so an effect that spreads, like bloom, is not cut off at the tile edge. The rows are handed out a row of tiles at a time, each block a PhotoRows that says which row it starts at, how many it holds and how wide they are, so memory holds one row however large the picture is, and PngStripWriter can write the file a strip at a time from there.

final picture = Uint8List(width * height * 4);
final PhotoCaptureReport made = await capturePhoto(
  renderer: renderer,
  scene: _scene,
  camera: _lens,
  width: width,
  height: height,
  settings: _noBloom,
  clearColor: _clear,
  tileWidth: 320,
  tileHeight: 180,
  margin: 16,
  // A row of tiles at a time: a PNG writer would take it from here.
  onRows: (PhotoRows rows) => picture.setRange(
    rows.top * width * 4,
    (rows.top + rows.count) * width * 4,
    rows.bytes,
  ),
);

The settings the tiles are drawn with are not quite the screen's. photoSettings holds exposure at what the screen showed, so every tile is exposed alike. It turns off temporal anti-aliasing, motion blur and render scale, which make no sense for a still tile, and the capture's report lists each one it set aside. The filter is composed onto the game's LookSettings, and PhotoFinish puts the vignette and the grain back once over the whole picture rather than once per tile.

Step 5: Held to a single frame #

The check draws the same picture as one 640 by 360 frame with the settings the capture reports, and compares it pixel by pixel with the four stitched tiles. Any pixel more than two steps off has to sit on a seam between tiles. Away from the seams the picture has to be the frame. On the software device it is within one step everywhere.

// The reference: the same picture as one frame, with the settings the
// capture says it drew its tiles with.
final FrameResult whole = renderer.render(
  width: width,
  height: height,
  scene: _scene,
  views: <CameraView>[CameraView(camera: _lens, clearColor: _clear)],
  settings: photoSettings(_noBloom, PhotoFilter.none, made.exposure).tile,
);
final ByteBuffer read = (await renderer.device.readback(
  whole.frame,
)).buffer;
if (report.moves != 600 || report.escapes != 0) {
  return '${report.escapes} of ${report.moves} flights left the room';
}
if (report.furthest > 5.0 + 1e-3) {
  return 'the camera got ${report.furthest} m from the player on a 5 m '
      'tether';
}
// Begun from behind the wall at z = 4.25: the start is in front of it.
if (report.startZ > 4.0) return 'the photo began at z ${report.startZ}';
if (report.tilesX != 2 || report.tilesY != 2) {
  return 'the capture drew ${report.tilesX} by ${report.tilesY} tiles';
}
// Every pixel more than two steps off the single frame has to sit on a
// seam between tiles; away from the seams the picture is the frame.
if (report.pixels != 640 * 360 || report.awayFromSeams != 0) {
  return '${report.awayFromSeams} pixels away from the seams differ';
}

Note. Bloom's widest levels reach further than any margin, so a capture with bloom is close to the screen rather than equal to it: with a 32-pixel margin the seams drop from 53 steps to a few. Chromatic aberration is left out of a capture, since no whole-frame version is written yet. The share sheet on Android and iOS needs a plugin this package does not take, so a game passes its own PhotoShelf there. There are no touch or pad controls for photo mode yet, the other demos do not have it, and the browser share sheet has not been tried in a browser.