Changing a running game's assets
Open the live demo · Read the source · View on GitHub
Flutter's hot reload swaps the Dart code and keeps the app's state. For a
3D game that is only half of it: the textures, models and shaders were
loaded once, and nothing in Flutter knows the files they came from. HotSwap
in flutter3d_app is the other half. It remembers what was loaded from
which file, and on a hot reload it reads each file again, and whatever
changed is put in place of the old version in the nodes and materials the
game already has. The world keeps running; nothing is instantiated again.
SceneSurface calls HotSwap.instance.swap() from reassemble, which is
what a hot reload runs, and a tool can run the same swap over the VM
service as ext.flutter3d.assets.swap. It works in debug builds only: in
profile and release every method returns at once and holds nothing.
Step 1: Files the page can edit #
HotSwap never opens a file itself. Each registration takes a function
that reads the file's current bytes and one that builds them into what the
game draws, which is how its own tests run with no disk at all. This page
uses the same door: its two "files" are bytes it holds, and a button
changes them the way saving in an editor would.
/// The poster's "file": three bytes, the colour it decodes to.
Uint8List _posterFile = Uint8List.fromList(<int>[220, 40, 30]);
/// The model's "file": one byte, 1 red, 2 green, 3 blue.
Uint8List _hullFile = Uint8List.fromList(<int>[1]);
The poster's file decodes into a small square of one colour. The model's
file builds one box called hull, in a material called hull paint.
/// What the texture file decodes to: a small square of its colour.
static TextureHandle? _decodePoster(GraphicsDevice device, Uint8List bytes) {
if (bytes.length != 3) return null;
final ByteData pixels = ByteData(4 * 4 * 4);
for (var i = 0; i < 16; i++) {
pixels
..setUint8(i * 4, bytes[0])
..setUint8(i * 4 + 1, bytes[1])
..setUint8(i * 4 + 2, bytes[2])
..setUint8(i * 4 + 3, 255);
}
return device.createTextureFromPixels(
width: 4,
height: 4,
format: TextureFormat.r8g8b8a8UNormInt,
pixels: pixels,
);
}
/// What the model file builds: one box called `hull`, in a material
/// called `hull paint` of the colour its byte names.
static Future<ModelAsset> _buildHull(GraphicsDevice device, Uint8List bytes) {
if (bytes.isEmpty || bytes.first < 1 || bytes.first > 3) {
throw const FormatException('not a model');
}
final Vector4 color = Vector4(0.08, 0.08, 0.08, 1.0)
..[bytes.first - 1] = 0.9;
return ModelAsset.fromDocument(
PlainModelDocument(
surfaces: <ModelSurface>[
ModelSurface(
mesh: CuboidShape(size: Vector3.all(1.0)).build(),
materialIndex: 0,
),
],
materials: <SurfaceMaterial>[
SurfaceMaterial(
baseColor: _fromSrgb(color),
unlit: true,
name: 'hull paint',
),
],
nodes: <ModelNode>[
ModelNode(name: 'hull', surfaces: <int>[0]),
],
),
device: device,
);
}
Step 2: Register what was loaded #
registerTexture and registerModel take the asset as it was loaded, the
two functions, and the bytes it was built from, so that the first swap
compares against those rather than taking the file as a change. A game
reading real assets calls loadTexture and loadModel, which read the
asset bundle and register in one step. Instances of the model are made
through the SwappableModel, which remembers them. The scene and the
renderer are registered the way SceneSurface registers its own.
final TextureHandle first = _decodePoster(device, _posterFile)!;
_poster = _hot.registerTexture(
_posterPath,
first,
read: () async => _posterFile,
build: (Uint8List bytes) async => _decodePoster(device, bytes),
loadedFrom: _posterFile,
);
_hull = _hot.registerModel(
_hullPath,
await _buildHull(device, _hullFile),
read: () async => _hullFile,
build: (Uint8List bytes) => _buildHull(device, bytes),
device: device,
loadedFrom: _hullFile,
);
_scene = Scene()
..add(
MeshNode(
DeviceMesh.upload(
device,
CuboidShape(size: Vector3(1.2, 1.2, 0.05)).build(),
),
RenderMaterial(name: 'poster', lighting: LightingModel.unlit)
..albedo = first,
name: 'poster',
)..setPosition(-0.9, 0.0, 0.0),
);
_ship = _hull.instantiate(_scene);
_ship.root.setPosition(0.9, 0.0, 0.0);
// What a `SceneSurface` registers for itself.
_hot
..registerScene(_scene)
..registerRenderer(context.renderer);
Step 3: Change both files and swap #
After the swap the poster's material holds a new texture, uploaded from
the new bytes; it may be another size or format, since it is a texture of
its own and the old one is released after the frames that still sample
it. The model is built again, and every instance adopts it: each surface
is matched by node name and slot, the matching MeshNode is handed the new
geometry and material, and the node itself stays, with its transform and
anything the game hung from it. The report says which files were taken
and which were refused; a file that does not build keeps the last version
that did.
_shots.add(await _shoot(context));
// Both files change on "disk", and a hot reload runs the swap.
_posterFile = Uint8List.fromList(<int>[30, 60, 230]);
_hullFile = Uint8List.fromList(<int>[2]);
_firstSwap = await _hot.swap();
_shots.add(await _shoot(context));
Step 4: Set a material from outside #
setMaterial is what an inspector's slider sends, over the VM service as
ext.flutter3d.material.set. It sets fields on every material of that
name in the registered scenes from the next frame on, and keeps them as
overrides. Saving the model afterwards brings the material as the file has
it, and the override is put back on, so the colour somebody is dragging
does not snap back on every save. clearMaterial lets it go.
// An inspector drags the paint to yellow, then the model is saved again:
// the override is put back on the material the new file brought.
_overridden = _hot.setMaterial('hull paint', <String, Object?>{
'baseColor': <double>[0.9, 0.8, 0.1, 1.0],
});
_hullFile = Uint8List.fromList(<int>[3]);
await _hot.swap();
_shots.add(await _shoot(context));
Step 5: A tunable, through the input #
A number the simulation reads, like this spin speed, is not a material's
business. It is changed through InputState.tune, which is what
ext.flutter3d.cvar.set calls, and the step takes it with
Tunables.take before it reads anything. Because it goes through the
input, a recorded run holds the change at the step it was made, and a
replay makes it at the same moment.
/// A number the game lets somebody change while it runs. It arrives
/// through the input, so a recorded run changes it at the same step.
final Tunables _tunables = Tunables(<String, double>{'spin': 0.6});
final InputState _input = InputState();
@override
void update(DemoContext context, double dt) {
_tunables.readFrom(_input);
_yaw += _tunables['spin'] * dt;
_ship.root.setRotationYawPitchRoll(_yaw, 0.0, 0.0);
_input.endStep();
}
double _yaw = 0.0;
Step 6: What the page checks #
Before the swap the poster and the hull are red. After one swap the poster
is blue and the hull green, the report names the texture and one instance
of the model, and the hull is drawn by the same MeshNode as before. After
the paint is set to yellow and the model is saved in blue, the hull is
still yellow. A tunable set through the input has the new value after the
step takes it.
if (!_hot.enabled) throw StateError(_said);
final [(posterA, hullA), (posterB, hullB), (_, hullC)] = _shots;
// Before: a red poster and a red hull.
if (_strongest(posterA) != 0 || _strongest(hullA) != 0) {
throw StateError('before the swap: $posterA, $hullA');
}
// After one swap: the new texture in the poster's material, the new
// model in the node the scene already had.
final HotSwapReport report = _firstSwap!;
if (!report.textures.contains(_posterPath) ||
report.models.single.instances != 1) {
throw StateError('the swap did not take both files: $report');
}
if (_strongest(posterB) != 2 || _strongest(hullB) != 1) {
throw StateError('after the swap: $posterB, $hullB');
}
if (!identical(_ship.meshes.single, _shipMeshBefore)) {
throw StateError('the model was drawn in a new node, not adopted');
}
// The override outlived the next save of the model: yellow, not blue.
if (_overridden != 1 || hullC[0] <= hullC[2] || hullC[1] <= hullC[2]) {
throw StateError('the dragged paint did not survive the swap: $hullC');
}
// A tunable set through the input is read by the step that takes it.
_input.tune('spin', 2.0);
_tunables.readFrom(_input);
_input.endStep();
if (_tunables['spin'] != 2.0 || _tunables.toJson()['spin'] != 2.0) {
throw StateError('the tunable did not change: ${_tunables.values}');
}
Note. A shader bundle the game loaded from bytes is refreshed the same way, through
registerLibrary, and a.f3dmatthroughloadMaterial; this page does not draw one, because the bytes it would swap are a compiled bundle. A model's embedded images swap with the model, not on their own.setMaterialrefuses a shader parameter the material was not loaded with, so a parameter the shader gained since the game started needs the material loaded again. A stage added under a name the renderer already looked up needs a restart.