11# Firecracker snapshot versioning
22
3- This document describes how Firecracker persists its state across multiple
4- versions, diving deep into the snapshot format, encoding, compatibility and
3+ This document describes how Firecracker persists microVM state into Firecracker
4+ snapshots. It describes the snapshot format, encoding, compatibility and
55limitations.
66
77## Introduction
88
9- The design behind the snapshot implementation enables version tolerant save
10- and restore across multiple Firecracker versions which we call a version space.
11- For example, one can pause a microVM, save it to disk with Firecracker version
12- ** 0.23.0** and later load it in Firecracker version ** 0.24.0** . It also works
13- in reverse: Firecracker version ** 0.23.0** loads what ** 0.24.0** saves.
14-
15- Below is an example graph showing backward and forward snapshot compatibility.
16- This is the general picture, but keep in mind that when adding new features
17- some version translations would not be possible.
18-
19- ![ Version graph] (
20- ../images/version_graph.png?raw=true
21- "Version graph")
22-
23- A non-exhaustive list of how cross-version snapshot support can be used:
24-
25- Example scenario #1 - load snapshot from older version:
26-
27- * Start Firecracker v0.23 → Boot microVM → * Workload starts* → Pause →
28- CreateSnapshot(snap) → kill microVM
29- * Start Firecracker v0.24 → LoadSnapshot → Resume → * Workload continues*
30-
31- Example scenario #2 - load snapshot in older version:
32-
33- * Start Firecracker v0.24 → Boot microVM → * Workload starts* → Pause →
34- CreateSnapshot(snap, “0.23”) → kill microVM
35- * Start Firecracker v0.23 → LoadSnapshot(snap) → Resume → * Workload continues*
36-
37- Example scenario #3 - load snapshot in older version:
38-
39- * Start Firecracker v0.24 → LoadSnapshot(older_snap) → Resume →
40- * Workload continues* → Pause → CreateSnapshot(snap, “0.23”) → kill microVM
41- * Start Firecracker v0.23 → LoadSnapshot(snap) → Resume → * Workload continues*
9+ Firecracker uses the serde crate [ 1] along with the bincode [ 2] format to
10+ serialize its state into Firecracker snapshots. Firecracker snapshots have
11+ versions that are independent of Firecracker versions. Each Firecracker version
12+ declares support for a specific snapshot data format version. When creating a
13+ snapshot, Firecracker will use the supported snapshot format version. When
14+ loading a snapshot, Firecracker will check that format of the snapshot file is
15+ compatible with the snapshot version Firecracker supports.
4216
4317## Overview
4418
@@ -61,59 +35,37 @@ emulation, KVM and vCPUs) with 2 exceptions - serial emulation and vsock backend
6135
6236While we continuously improve and extend Firecracker's features by adding new
6337capabilities, devices or enhancements, the microVM state file may change both
64- structurally and semantically with each new release. The state file includes
65- versioning information and each Firecracker release implements distinct
66- save/restore logic for the supported version space.
38+ structurally and semantically with each new release.
6739
6840## MicroVM state file format
6941
70- A microVM state file is further split into four different fields :
42+ A Firecracker snapshot has the following format :
7143
72- | Field | Bits| Description |
73- | ----| ----| ----|
74- | magic_id | 64 | Firecracker snapshot, architecture (x86_64/aarch64) and storage version . |
75- | version | 16 | The snapshot version number internally mapped 1:1 to a specific Firecracker version. |
44+ | Field | Bits | Description |
45+ | ------- | ------ | --------- ----|
46+ | magic_id | 64 | Firecracker snapshot and architecture (x86_64/aarch64). |
47+ | version | M | The snapshot data format version ( ` MAJOR.MINOR.PATCH ` ) |
7648| state | N | Bincode blob containing the microVM state. |
77- | crc| 64 | Optional CRC64 sum of magic_id, version and state fields. |
78-
79- ** Note** : the last 16 bits of ` magic_id ` encode the storage version which specifies
80- the encoding used for the ` version ` and ` state ` fields. The current
81- implementation sets this field to 1, which identifies it as a [ Serde bincode] ( https://github.com./servo/bincode )
82- compatible encoder/decoder.
83-
84- ### Version tolerant ser/de
85-
86- Firecracker reads and writes the ` state ` blob of the snapshot by using per
87- version, separate serialization and deserialization logic. This logic is mostly
88- autogenerated by a Rust procedural macro based on ` struct ` and ` enum `
89- annotations. Basically, one can say that these structures support versioning.
90- The versioning logic is generated by parsing a structure's history log (encoded
91- using Rust annotations) and emitting Rust code.
92-
93- Versioned serialization and deserialization is divided into two translation layers:
94-
95- * field translator,
96- * semantic translator.
97-
98- The _ field translator_ implements the logic to convert between different
99- versions of the same Rust POD structure: it can deserialize or serialize from
100- source version to target.
101- The translation is done field by field - the common fields are copied from
102- source to target, and the fields that are unique to the target are
103- (de)serialized with their default values.
104-
105- The _ semantic translator_ is only concerned with translating the semantics of
106- the serialized/deserialized fields.
107-
108- The _ field translator_ is generated automatically through a procedural macro,
109- and the _ semantic translation methods_ have to be annotated in the structure
110- by the user.
111-
112- This block diagram illustrates the concept:
113-
114- ![ Versionize] (
115- ../images/versionize.png?raw=true
116- "Versionize layers")
49+ | crc | 64 | Optional CRC64 sum of magic_id, version and state fields. |
50+
51+ The snapshot format has its own version encoded in the snapshot file itself
52+ after the snapshot's ` magic_id ` . The snapshot format version is independent of
53+ the Firecracker version and it follows semantic versioning, i.e.
54+ ` MAJOR.MINOR.PATCH ` . That means that backwards incompatible changes in the data
55+ format will cause ` MAJOR ` version number bump, whereas backwards compatible
56+ changes will cause a ` MINOR ` version number bump.
57+
58+ For example, that means that a Firecracker binary that supports snapshot data
59+ format ` x.y.z ` will be able to load all snapshot files with version ` x.Y.z `
60+ where ` Y <= y ` .
61+
62+ Currently, Firecracker uses the [ Serde bincode
63+ encoder] ( https://github.com./servo/bincode ) for serializing the microVM state.
64+ The encoding format that bincode uses does not allow backwards compatible
65+ changes in the state, so essentially every change in the microVM state
66+ description will result in bump of the format's ` MAJOR ` version. If the needs
67+ arises, we will look into alternative formats that allow more flexibility with
68+ regards to backwards compatibility.
11769
11870## VM state encoding
11971
@@ -132,10 +84,6 @@ Key benefits of using *bincode*:
13284
13385The current implementation relies on the [ Serde bincode encoder] ( https://github.com./servo/bincode ) .
13486
135- Versionize is compatible to Serde with bincode backend: structures serialized
136- with versionize at a specific version can be deserialized with Serde. Also
137- structures serialized with serde can be deserialized with versionize.
138-
13987## Snapshot compatibility
14088
14189### Host kernel
@@ -195,18 +143,8 @@ specifically, the MSRs corresponding to the guest exposed features.
195143
196144## Implementation
197145
198- To enable Firecracker cross version snapshots we have designed and built two
199- crates:
200-
201- * [ versionize] ( https://crates.io/crates/versionize ) - defines the ` Versionize `
202- trait, implements serialization of primitive types and provides a helper
203- class to map Firecracker versions to individual structure versions.
204- * [ versionize_derive] ( https://crates.io/crates/versionize_derive ) - exports
205- a procedural macro that consumes structures and enums and their annotations
206- to produce an implementation of the ` Versionize ` trait.
207-
208- The microVM state file format is implemented in the [ snapshot crate] ( ../../src/snapshot/src/lib.rs )
209- in the Firecracker repository.
210- All Firecracker devices implement the [ Persist] ( ../../src/snapshot/src/persist.rs )
146+ The microVM state file format is implemented in the [ snapshot
147+ crate] ( ../../src/snapshot/src/lib.rs ) in the Firecracker repository. All
148+ Firecracker devices implement the [ Persist] ( ../../src/snapshot/src/persist.rs )
211149trait which exposes an interface that enables creating from and saving to the
212150microVM state.
0 commit comments