Playtest heatmaps
Open the live demo · Read the source · View on GitHub
Where do players go in a level, and where do they lose? A game could report
that itself, but a game can report anything. A tape can only say what was
pressed. Played back through the same simulation it shows where the player
went, and if the replay does not match the run that was recorded, it shows
nothing. resimulate replays a run against its level hash, its starting
state and its checkpoints. Heatmap bins the trails of many such runs into
cells and marks where runs were lost.
On this page sixteen recorded runs walk across a yard with a one-metre pit in it. Each cell is coloured from blue to red by how often a run was sampled there. The red balls are where runs were lost. Tapes sets how many runs go in.
Step 1: A game that can be played blind #
resimulate starts a game through HeadlessGame and steps it through
HeadlessRun, the same interface a test or a server uses to play without a
window. The pit is a brush in the level. Moving it makes a different level
with a different hash. A walker that falls in stays where it fell until its
tape ends, as a player does on the "you died" screen.
/// A walker crossing a yard, steered by the stick. Stepping into the pit
/// loses the run, and the walker stays where it fell — as a player sits on
/// the "you died" screen — until the tape runs out.
final class _Walk extends HeadlessRun {
_Walk(this.input, this.pit);
final InputState input;
/// The pit's corners on the ground, `(minX, minZ, maxX, maxZ)`.
final (double, double, double, double) pit;
final Vector3 _at = Vector3.zero();
@override
RunOutcome outcome = RunOutcome.playing;
@override
void step(double dt) {
if (outcome.isOver) return;
_at
..x += input.moveAxis.x * 3.0 * dt
..z += input.moveAxis.y * 3.0 * dt;
final (double x0, double z0, double x1, double z1) = pit;
if (_at.x >= x0 && _at.x < x1 && _at.z >= z0 && _at.z < z1) {
outcome = RunOutcome.lost;
}
}
@override
Snapshot save() => Snapshot(<String, Object?>{
'x': _at.x,
'z': _at.z,
'outcome': outcome.name,
});
@override
WorldPosition get position => _at.toWorldPosition();
@override
WorldPosition get eye => _at.toWorldPosition();
@override
void aim(Vector3 out) => out.setValues(1.0, 0.0, 0.0);
@override
String get summary => 'at ${_at.x.toStringAsFixed(2)}, ${outcome.name}';
@override
Map<String, Object?> get reading => <String, Object?>{'x': _at.x};
}
/// The game, as `resimulate` starts it: the pit is the level's brush made
/// of `pit`, so a level with the pit moved is another level.
final class _WalkGame extends HeadlessGame {
const _WalkGame();
@override
String get name => 'yard';
@override
Map<String, GameAction> get buttons => const <String, GameAction>{};
@override
EntityRegistry registry() => EntityRegistry(const <EntityKind>[]);
@override
HeadlessRun start(HeadlessStart start) {
final level = start.level;
final input = start.input;
final Brush pit = level.brushes.firstWhere((b) => b.material == 'pit');
final Vector3 half = pit.size * 0.5;
return _Walk(input, (
pit.center.x - half.x,
pit.center.z - half.z,
pit.center.x + half.x,
pit.center.z + half.z,
));
}
}
Level _yard() => Level(
name: 'yard',
materials: <String, LevelMaterial>{
'stone': LevelMaterial(),
'pit': LevelMaterial(),
},
brushes: <Brush>[
Brush(
center: WorldPosition(6.5, -0.5, 0.0),
size: Vector3(15.0, 1.0, 6.0),
material: 'stone',
),
// One metre square, a cell of the heatmap exactly.
Brush(
center: WorldPosition(5.5, 0.0, -0.5),
size: Vector3(1.0, 0.1, 1.0),
material: 'pit',
solid: false,
),
],
);
Step 2: The tapes #
Each run walks across the yard for six seconds and drifts to one side by an amount its number gives, so that between them the runs cover the yard's width. They are recorded the way a game records them: the tape before each step, a checkpoint every thirty steps after it.
/// One player's tape: six seconds walking across the yard, drifting to one
/// side by however much [seed] says. Written down as a game does — the tape
/// before each step, a checkpoint after it.
Demo _record(Level level, int seed) {
final input = InputState();
final run = const _WalkGame().start(
HeadlessStart(level: level, world: CollisionWorld(), input: input),
);
final start = run.save();
// Spread evenly over the yard's width, tape by tape, by the golden ratio.
final drift = (seed * 0.6180339887) % 1.0 * 0.7 - 0.35;
final recorder = InputTapeRecorder(seed: seed);
final trace = DigestTrace(every: 30);
for (var step = 1; step <= 360; step++) {
input.setStickAxis(0.75, drift);
recorder.record(input);
input.beginStep();
run.step(1.0 / 60.0);
trace.observe(step, run.save().toJson());
input.endStep();
}
return Demo(
level: 'levels/yard.json',
levelHash: level.digestHex,
start: start,
tape: recorder.tape,
buildStamp: 'showcase',
checkpoints: trace,
);
}
Step 3: Play every tape again, and bin what retraced #
Each tape is played into a fresh run of the level. resimulate answers
with a sealed Resimulation. A run that retraced its checkpoints comes
back with its outcome and a trail of positions, one every ten steps. A run
that diverged, started somewhere else, or was recorded in another level
comes back with no trail, and is only counted. Heatmap.bin counts the
samples and the distinct runs in each cell. It keeps the place where each
lost run ended, and counts each outcome even when nobody had it.
/// Every tape played again into [level], and the trails of those that
/// retraced their checkpoints binned into metre cells. A tape that parted
/// from its own checkpoints, or was recorded in another level, gives no
/// trail at all; [refused] counts them.
({Heatmap map, int refused}) _heatmap(Level level, List<Demo> tapes) {
final trails = <HeatmapTrail>[];
var refused = 0;
for (var i = 0; i < tapes.length; i++) {
switch (resimulate(game: const _WalkGame(), level: level, demo: tapes[i])) {
case ResimulationRetraced(:final trail, :final outcome, :final steps):
trails.add(
HeatmapTrail(
run: i,
steps: steps,
outcome: outcome.name,
positions: trail,
endedBadly: outcome == RunOutcome.lost,
),
);
case _:
refused++;
}
}
return (
map: Heatmap.bin(
trails,
outcomeNames: <String>[for (final o in RunOutcome.values) o.name],
),
refused: refused,
);
}
Step 4: Draw it over the level #
One flat tile per cell, coloured against the hottest cell, and a ball at
each place a run was lost. The heatmap's JSON is the format the editor's
report screen and Playtest.heatmap already read, so the same map can be
written out and opened there.
/// Each cell from cold to hot by how many samples landed in it, against
/// the hottest; a cell nobody walked through is not drawn. A red ball
/// where each run was lost.
void _draw() {
final int hottest = _hottest(_map).samples;
for (final MeshNode node in _tiles.values) {
node.isVisible = false;
}
for (final HeatmapCell cell in _map.cells) {
final MeshNode? node = _tiles[(cell.x, cell.z)];
if (node == null) continue;
final double t = cell.samples / hottest;
node
..isVisible = true
..material.baseColor = LinearColor.fromSrgb(
(t * 2.0).clamp(0.0, 1.0),
(1.0 - (t - 0.5).abs() * 2.0).clamp(0.15, 1.0),
(1.0 - t * 2.0).clamp(0.0, 1.0),
1.0,
);
}
for (var i = 0; i < _deaths.length; i++) {
final bool shown = i < _map.ends.length;
_deaths[i].isVisible = shown;
if (shown) {
_deaths[i].setPosition(_map.ends[i].x, 0.25, _map.ends[i].z);
}
}
}
Step 5: What has to hold #
Every tape has to retrace its checkpoints, and at least three runs have to be lost. The hottest cell has to be the cell every lost run ended in: a player who fell stays there, sample after sample, so that cell ends up with more samples than any cell the runs only passed through. The same tape replayed into the yard with the pit moved one metre has to be refused as a run from another level.
// Every tape retraced its own checkpoints, and some were lost.
if (_refused != 0 || _map.runs != tapes.round()) {
throw StateError('$_refused of the tapes did not replay');
}
if (_map.ends.length < 3) {
throw StateError('only ${_map.ends.length} runs were lost in the pit');
}
// The hottest cell is the one every run was lost in: a player who fell
// stays there, sample after sample.
final HeatmapCell hottest = _hottest(_map);
for (final HeatmapEnd end in _map.ends) {
if ((end.x.floor(), end.z.floor()) != (hottest.x, hottest.z)) {
throw StateError(
'run ${end.run} was lost at (${end.x}, ${end.z}), and the hottest '
'cell is (${hottest.x}, ${hottest.z})',
);
}
}
// The same tapes in a yard with the pit moved are another level's runs.
final Level moved = Level.fromJson(_yard().toJson());
final Brush pit = moved.brushes.last;
moved.brushes.last = Brush.fromJson(<String, Object?>{
...pit.toJson(),
'at': <double>[pit.center.x + 1.0, pit.center.y, pit.center.z],
});
if (resimulate(
game: const _WalkGame(),
level: moved,
demo: _record(_level, 1),
)
is! ResimulationLevelChanged) {
throw StateError('a tape replayed into a level it was not recorded in');
}
Note. On a real server this is the end of a longer path. A run is sent only with the player's consent, which
TelemetryUploaderchecks on every send. The server replays it in an isolate and keeps the trail and outcome, not the input. That server can only step a game with no Flutter in it, and none of the three demo games is split out that way yet, so it answers 503. Our instance is not deployed, uploads have no rate limit or retention period, and the editor draws the heatmap on its own screen rather than over the level in the viewport, as this page does.