The view model pass

since 0.4.0 Scene and geometry

Open the live demo · Read the source · View on GitHub

A weapon or a tool held close to the eye cannot share the world's depth buffer. A wall a few metres away would slice through it the moment the camera got close. ViewModelNode draws the held item in its own pass, with its own depth, straight on top of the finished world.

Step 1: Build the world #

An ordinary scene: a floor, a pillar, and a sun. Nothing here is aware that anything else will be drawn afterwards.

final Material stone = Material(
  name: 'stone',
  baseColor: Vector4(0.58, 0.56, 0.52, 1.0),
  roughness: 0.82,
);
final Scene world = Scene()
  ..ambientColor = Vector3(0.4, 0.46, 0.6)
  ..ambientIntensity = 0.16
  ..add(
    MeshNode(
      DeviceMesh.upload(
        context.device,
        const PlaneShape(width: 20, depth: 20).build(),
      ),
      stone.copy()..doubleSided = true,
      name: 'floor',
    ),
  )
  ..add(
    MeshNode(
      DeviceMesh.upload(
        context.device,
        CuboidShape(size: Vector3(1.4, 2.4, 1.4)).build(),
      ),
      stone,
      name: 'pillar',
    )..setPosition(0.0, 1.2, -3.0),
  )
  ..add(
    LightNode(name: 'sun', intensity: 3.0)
      ..setLocalForward(Vector3(-0.4, -0.8, -0.3)),
  );

Step 2: Build a small scene for the hands #

The held tool lives in a scene of its own, lit by its own light rather than the world's. It has a narrow camera, because a wide lens a few centimetres from a small object stretches its edges into something that reads as broken.

final Scene handsScene = Scene()
  ..add(
    _tool = MeshNode(
      DeviceMesh.upload(
        context.device,
        CuboidShape(size: Vector3(0.12, 0.12, 0.6)).build(),
      ),
      Material(
        name: 'tool',
        baseColor: Vector4(0.66, 0.68, 0.72, 1.0),
        metallic: 0.85,
        roughness: 0.28,
      ),
      name: 'held tool',
    )..setPosition(0.22, -0.2, -0.5),
  )
  ..add(
    LightNode(name: 'hand key', intensity: 2.6)
      ..setLocalForward(Vector3(-0.3, -0.5, -0.8)),
  );
_handCamera = CameraNode(
  name: 'hand camera',
  projection: PerspectiveProjection(
    fovYRadians: fovDegrees * math.pi / 180.0,
  ),
);
handsScene.add(_handCamera);

Step 3: Register the pass #

ViewModelNode takes the hands scene and its camera. Adding it to the renderer with addNode gives it a pass of its own, run after the world and before the frame is tone mapped. Nothing about the world scene changes.

context.renderer.addNode(
  ViewModelNode(scene: handsScene, camera: _handCamera),
);

Step 4: Move the tool and adjust the lens #

The tool bobs a little every frame, the way a held object would as someone walks. The field of view slider changes the hand camera's projection, and the tool's edges stretch or relax as it moves, independently of the world camera you orbit with the pointer.

_bob += dt;
_tool.setPosition(0.22, -0.2 + math.sin(_bob * 2.4) * 0.02, -0.5);
_handCamera.projection = PerspectiveProjection(
  fovYRadians: fovDegrees * math.pi / 180.0,
);

Step 5: Check the pass ran #

The page looks for a pass named view model in the frame's own list, and checks that more than the world's single draw reached the screen.

final bool viewModelRan = frame.passes.any(
  (FramePass pass) => pass.name == 'view model',
);
if (!viewModelRan || frame.drawCalls < 2) {
  throw StateError('the view model pass did not draw over the world');
}