Rollback

Rollback - firmware version recovery

Definition

Rollback refers to automatically or manually restoring to the previous firmware version when an OTA upgrade fails or encounters an exception. The A/B Partition architecture supports fast rollback without needing to re-download the old version.

Trigger Methods

Rollback can be divided into: automatic rollback—automatically switching to the old partition when device self-check fails after an upgrade (such as startup timeout or key service exception); manual rollback—operation personnel initiate rollback commands for a single or batch of devices on the platform; gray rollback—during Gray Release, if an issue is found, stop pushing and perform rollback on devices that have already upgraded.

A/B Partition and Rollback Efficiency

With the A/B Partition approach, the old version firmware is still retained in the other partition, and rollback only requires updating the boot flag and restarting, which usually takes seconds and does not require re-downloading. In a single-partition architecture, rollback requires re-downloading the old firmware, takes longer, and depends on the network, suitable as a fallback solution.

Rollback Strategy Configuration

The platform can configure rollback strategies: the number of retry attempts for upgrade failures, self-check timeout duration, whether automatic rollback is allowed, etc. It is recommended to set a reasonable observation period in FOTA tasks, and, based on the versions and status reported by the device, promptly identify anomalies and trigger rollbacks to reduce the risk of mass device bricking.

Whether rollback restores a boot slot or a package

What rollback means technically depends on how the device keeps the old version. On a dual-slot device the old system is still in the other slot. Rollback only changes which slot boots next, then restarts. It does not download again. A single-slot device has no old system left that can boot. Rollback must transfer the old package to the device again, finish the write, and check it before restarting. If the platform sends both kinds of device the same "switch back immediately" command, the single-slot device stops in a failed state because it has no old slot, while the task looks as if it was sent.

The task therefore has to read the partition capability and the current slot reported by the device when it is created, rather than assuming from the product manual that every unit supports dual slots. The old package must still be obtainable. A version cleaned out of the repository cannot be a rollback target. The target should be the last version known to run, which is not necessarily the previous number. If a build in between was withdrawn, do not step back to that build by number. A switch has succeeded when the device reports the running version after it comes online again, not when the restart command has been delivered.

Do not declare success before the observation period ends

A finished write means only that the new slot can start. It does not mean the business process is stable. During the observation period the device should report key self-checks: whether it can reach the platform, whether it can read the peripherals it needs, and whether the agreed health check passed. Until then the task stays "pending confirmation." The conditions for automatic rollback must be written in advance, for example still unable to connect after several consecutive starts, or an explicit health-check failure. A vague "not running normally" cannot be judged automatically on the device.

Manual rollback is for a problem a person sees during the observation period that the automatic rule does not cover. It and automatic rollback should enter the same record, so two operations do not each change the slot once. If a canary batch is already paused, devices that were upgraded and not yet confirmed still need their own decision: confirm or roll back. Pausing the push does not take them back. Combined with a state machine, failed, rolling back, rolled back, and confirmed are different states. Do not jump from "upgrading" straight to "success" to clear the queue. If field maintenance is writing something else at the same time, stop the parallel task in remote maintenance before switching slots.

Treating a single-slot device as if it could switch back in seconds

Applying the timing experience of dual-slot devices to single-slot devices is a common misjudgment in batch incidents. Single-slot rollback depends on downloading again over the network. On a weak network the device spends a long time in a window where the old system has already been erased and the new download is not finished. Power loss in that window can leave the device unable to boot on its own. Confirm battery and link before upgrading this class of device, and keep a local recovery method. The platform cannot promise that these devices return as quickly as dual-slot devices.

Another mistake is to switch only the system slot and not handle configuration or data the new version has already written. The old system may refuse to start when it reads the new configuration, which looks like "rollback failed again." The task description should say whether configuration travels with the version, and which configuration backup is restored on rollback. If only the version number on the platform record is changed back while the device is still running the new version, monitoring interprets the new version's data with the old version's rules. Reconcile against what the device reports. When the platform record lags, mark it inconsistent. Do not overwrite the reported value.

Rehearse rollback once, as a full round trip, on a small batch: upgrade to the new version, fail confirmation, return to the old version, and check that the running version and the key configuration are still usable. A model that has not been rehearsed does not go on the list allowed to roll back automatically. It is handled by a person. The rehearsal record includes the partition type, whether the old package is still in the repository, and the version number of the first report after rollback. If the repository caps how many versions it keeps, mark a version that devices still use as a rollback target so it cannot be deleted. After a mistaken delete, the task can only show that rollback is impossible, while the site may already have erased the slot that was running. If automatic rollback fires repeatedly in the same batch, stop later batches. Do not let each device fail on its own and then switch back, stretching the time of impact.

Related Terms

Return to Device Management Platform terminology