RFC: Precalculated swap log for supporting devices with bigger write block sizes

[de-nordic]

Where we are

At this point MCUboot, while moving sectors around, logs status of operations with three flags per sector. With devices with write block size of 4b it amounts to 12 bytes per moved sector. This becomes problematic when you want to make MCUboot work with devices that have bigger write block sizes; for example with devices that require 8 bytes this is already 24 bytes, and becomes 48 for devices with 16 byte write block size. When you get to devices that have blocks like 512 byte, this become hard to handle.

Note: while using "sector" here I mean logical sector of N * erase-block-size of device with the biggest erase-block-size.

What can we do?

The proposal here is to replace the swap log with information block, layout record, that would state desired state of slot and swap algorithm would just try to restore the desired state from what it has on flash.

The layout record could consist of array of sha of sectors in order they should appear in slot. Note that sha256 is 32 byte long so it is already shorter then log entry for 16 byte write-block-size device.

For example

    [ State header ]
    [ sha(sector0) ]
    [ sha(sector1) ]
    ...
    ...
    ...
   [ sha(sectorN) ]
   [ sha(layout record) ]

Where sha is whatever sha we select (could be configurable, sha256 seems fine), and sectorX i logical sector size block on device, within slot.
Each slot will have its own expected layout record, so there would be two such blocks. The layout record sha is just to make sure that the record is complete.

Note that, if we put aside header size, the log size is now just N * sizeof(sha) and is aligned to write-block-size as a whole not per log entry (actually it will be aligned to erase block size, or logical sector).

Unresolved questions:

1. Where do we store this record?
   Probably storing logs at the end of slot is not a good idea and we should have reserved space outside of slots and the entire slot should be dedicated to application image; problem will be here with devices that have big erase sizes as the advantage of small log record would be killed by using entire large independent erase block size just to store it, and it is better for such devices to have that record inside slot. This could be probably configurable depending on device layout.
2. How to mark test/confirm.
   Generally if we just assume that MCUboot always attempts to restore layout according to the pre-defined log, then just appearance of the log would mean that the swap should happen, and confirmation could be just a removal of the log - this is again problem for devices with big erase blocks, so they would have to mark somehow that they wan to stay with tested layout anyway.
3. Should application carry its own layout record?
   Probably, specifically when signed. Also calculating record on device is risky, but should be evaluated.

@nvlsianpu @d3zd3z @nordicjm @butok @erwango

[butok]

To clarify.
Will these sha be generated and preinstalled by imagetool, or during run-time?

[de-nordic]

To clarify. Will these sha be generated and preinstalled by imagetool, or during run-time?

Actually that is point 3 in unresolved question. I guess that we can have it following ways, probably configurable:

1. the sha table is calculated and carried by every image
   + we have it pre calculated, we are sure that it is not manipulated by hardware errors
   +/- kinda saves time on calculating but we mostly have accelerators to do that job anyway
   - takes space with image
2. calculated on hardware:
   + saves image space
   +/- time of calculation may depend on having accelerator
   - risk of calculating something wrong
3. combo: sha of each page not pre-calculated but sha of the sha table is
   + saves image space
   +/- time of calculation may depend on having accelerator
   + even if we have calculated something wrong, or the are where we calculate the sha is broken, the image carried sha of sha table will prevent us from bricking device

[nordicjm]

I don't see how it could be done without having the hash table on the flash, if a device is rebooting during a page being swapped, how are you going to know which part goes with which partial image? One sector will be corrupted. If you don't have a checksum/hash of the sectors for each image, you can't know where that is, nor can you re-generate an "in RAM" hash list because then you have a huge number of operations to try and put a bunch of hashes together to come up with the "sha of the sha table"

[de-nordic]

I don't see how it could be done without having the hash table on the flash, if a device is rebooting during a page being swapped, how are you going to know which part goes with which partial image? One sector will be corrupted. If you don't have a checksum/hash of the sectors for each image, you can't know where that is, nor can you re-generate an "in RAM" hash list because then you have a huge number of operations to try and put a bunch of hashes together to come up with the "sha of the sha table"

No, we are intended to have it on flash, the question is where we get it from when we start the swap (and then where we place it).

[PavelChromyNXP]

Hello all,

I believe it is a very good idea to implement more efficient algorithm for image swapping.
We have discussed a similar approach internally at NXP quite a long time ago and there was also a design proposal, although it never got to the implementation due to significant redesign of MCUBoot required to achieve the goal (it would not be an isolated change).

For optimal operation the whole swapping process may need to be altered as current swapping methods have significant limitations and/or drawbacks.
The current algorithm with logging not only does not fits devices with larger write phrases, it is also quite wasteful taking 3 writes per swapping step while only 2 are actually necessary (that's another story).

Let me share our analysis and description of the design we have been thinking about.

Generic thoughts:

The swapping using scratch area is not ideal due to FLASH wear of the scratch and unnecessary FLASH operations performed.

The swapping with move-by-one approach scatters the wear across larger area of FLASH but there are still ~twice as many FLASH operations than required.
Moreover moving the primary (executable) slot is not really a neat solution as it adds unnecessary state to be handled during recovery.

It would be fairly easy to handle offset of the image in the secondary/staging slot, because there is no strong (technical) requirement for the placement of the image (the code never gets executed directly from there).
So that either the image could be staged with an offset, or better, the swapping could be done so that the original image would be backed up to the secondary/staging slot with an offset.
The swapping would need to be performed backwards, from the end of the image down, utilizing the unused space following the staged image to backup the original one.

Having metadata (trailer) at the end of the slots is not really practical solution as it cannot be used to store the image itself anyway.
There could simply be a reserved area that does not need to be treated as part of the slot. (That area would be then a clear choice to store the hashes then.)

Overhead calculation/estimations:

It is quite clear that for fail-safe image swapping it is necessary to have at least a single spare erase sector.
However the swapping steps do not necessarily need to correspond with erase sectors. Each swapping step may span multiple of them.

Example: let's assume each slot consists of 1024 erase sectors. We divide each slot into 128 chunks to be swapped (8 erase sectors each).
The overhead of erase sectors needed for failsafe operation is 1/128 of the slot size, that is less than 1%, which is more than acceptable.
So in praxis, disregarding actual slot size, there is no big deal with limiting number of swap steps (and hashes) by a constant number, say 128 as in the setup presented.

But do we really need to have hash table for the both images for successful recovery? Actually not. Swapping chunks are not scattered randomly; we know the swapping algorithm. We just need to find the crossing-point where the swapping was interrupted.
For that, the hash table of just one on the images is needed. Once we put together that image, it is clear where and how get the other one, possibly keeping a single hash for its final verification.
In absolute numbers, the space needed for SHA256 hashes would be 4 KB + 32 bytes in total to store the hash table + one extra hash for the other image.

Going further, the image typically would not take up the whole slot. The application designer likely leaves at least 10% margin or more.
If the actual image occupies 80% of the slot size, then 20% is unused and could be leveraged for image swapping. In such a case a smart swapping algorithm could adapt and finish in 5 steps while keeping just 6 hashes.
It is disputable whether such optimization makes sense regarding increased complexity. Surely not in the initial implementation, this is maybe just for your consideration.

Where to store the hashes:

Rather not pre-calculated in the image itself for several reasons:

• If the swapping gets interrupted it would be very impractical for determination of where the hash table was at the moment of interruption and to which image it actually belonged.
• It would not save us from on-device hash calculation as it is still needed for verification (on-device hashing needs to be implemented in any case).
• The size of the swapping/hashing chunk is not a property of the image but rather depends on the configuration of the bootloader.
• Consider an MCU with external XIP FLASH. Each manufacturing batch of the device could utilize different FLASH part (different erase sector size). Well designed firmware should not depend on such a detail.
• For neat design, swapping algorithm should not depend on the actual content it is processing, but rather be generic to allow for flexibility/modularity. All it needs to know are addresses/lenghths of data to swap.

Hashes in the FLASH:

• Straight forward solution, ~ 4 KB overhead for reasonably large FLASH memory may not be that big deal.

In RAM:

• Doable. The hashes for all blocks would have to be re-calculated and table needs to be reconstructed upon recovery from failure.
• A hash of the table is need for its reconstruction.
• For the table from the example above, hash of 4096 byte table would have to be calculated up to 128 times. That roughly corresponds to calculation of hash over about 500 KB of data.
• If that seems slow, it would be possible to speed it up by storing also CRC32 of the table, do reconstruction of the table according to it, verify hash only when the CRC matches and eventually retry.
• Using a dynamic programming technique the total amount of data passing through CRC32 algorithm could be cut down by half.
• In any case the recovery time should not be that big deal if it finishes in some reasonable time (i.e. within a minute) as it is an exceptional scenario, not expected to happen under normal operation.

Summary:

• It would be good idea to have reserved space to keep bootloader meta-data/config separate from the image slots. Ideally having two instances in independent erase blocks to allow for safe ping-pong style update is preferred.
• it is not beneficial to insist on placing the image in the secondary slot always at its very beginning. Relaxing this requirement reduces FLASH wear and ev. number of states to be taken into account during recovery.
• Swapping does not need to be done in units of erase sector size. The unit could be any multiple of it - it is matter of choice with overhead tradeoff.
• Full hash table is needed just for one of the images. For the other one just a single hash is sufficient.
• Having the hashes in RAM is possible but likely does not pay off if there is a space for meta-data allocated in the FLASH.

[github-actions]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.

[DOAR-Infineon]

Hi @de-nordic, @PavelChromyNXP, @nordicjm, @butok,

I'd like to point your attention to an existing production-tested implementation that addresses this problem — and has been shipping on Infineon/Cypress PSE84 devices. - https://github.com/mcu-tools/mcuboot/tree/cypress

Existing Implementation: MCUBOOT_SWAP_USING_STATUS

Infineon/Cypress have implemented large write size support for MCUBoot swap operations under the MCUBOOT_SWAP_USING_STATUS compile flag. The implementation consists of the following source files:

• boot/bootutil/src/swap_status_part.c — core record read/write engine with circular buffering
• boot/bootutil/src/swap_status_misc.c — MCUBoot API integration (boot_read_swap_state, boot_write_status, etc.)
• boot/bootutil/src/swap_status.c — status byte parsing and swap resume logic
• boot/bootutil/src/swap_status.h — data structure definitions and layout constants

How It Works

Instead of writing individual status bytes into the image slot trailer (which requires per-byte write granularity), the algorithm relocates all MCUBoot swap metadata to a dedicated flash partition (FLASH_AREA_IMAGE_SWAP_STATUS). Within this partition, it emulates the original MCUBoot status area using a record-based circular buffer approach:

Record Layout

Each row is aligned to MEMORY_ALIGN, i.e. the device's minimum write size:

[ payload (variable) | magic 4B | counter 4B | CRC32C 4B ]

Where:

• Payload: actual status data (BOOT_SWAP_STATUS_PAYLD_SZ = MEMORY_ALIGN - 12 bytes)
• Magic: 0xDEADBEEF sentinel indicating a written record
• Counter: monotonically incrementing 32-bit value to identify the newest record
• CRC32C: integrity checksum over payload + magic + counter

Status Partition Layout

┌─────────────────────────────────────────────────────────┐
│ Primary Slot Status (BOOT_SWAP_STATUS_SIZE)             │
│ ┌─────────────────────┐ ┌─────────────────────┐         │
│ │ Copy 0              │ │ Copy 1              │         │
│ │ [sector status rows]│ │ [sector status rows]│         │
│ │ [trailer rows]      │ │ [trailer rows]      │         │
│ └─────────────────────┘ └─────────────────────┘         │
├─────────────────────────────────────────────────────────┤
│ Secondary Slot Status (BOOT_SWAP_STATUS_SIZE)           │
│ ┌─────────────────────┐ ┌─────────────────────┐         │
│ │ Copy 0              │ │ Copy 1              │         │
│ └─────────────────────┘ └─────────────────────┘         │
├─────────────────────────────────────────────────────────┤
│ Scratch Status (if MCUBOOT_SWAP_USING_SCRATCH)          │
└─────────────────────────────────────────────────────────┘

Status Trailer Data (stored within the record payload area)

Offset 0:            Per-sector swap status bytes (1 byte per sector)
                     ...
boot_swap_size_off:  swap_size      (4 bytes)
boot_swap_info_off:  swap_type      (1 byte)
boot_copy_done_off:  copy_done      (1 byte)
boot_image_ok_off:   image_ok       (1 byte)
boot_magic_off:      boot_img_magic (16 bytes)  ← end of raw area

Key Design Properties

1. Write-Alignment (solves the core problem)

The entire record row equals one flash write unit (MEMORY_ALIGN), so every write is a single atomic flash page operation — directly solving the large write block problem. Tested with 256-byte minimum write size on Infineon PSE84 and PSOC6 devices.

2. Circular Buffer with Wear Leveling

Each logical record has BOOT_SWAP_STATUS_MULT physical copies (default: 2). On update:

1. Read current valid record (highest counter + valid CRC)
2. Write updated data to the next physical slot with incremented counter and new CRC
3. Erase the obsolete copy

This halves the per-location erase cycles. The multiplier is configurable for additional wear reduction.

3. Power-Loss Resilience

The write-then-erase ordering guarantees at least one valid copy is always present:

• If power lost before write → old copy still valid
• If power lost after write, before erase → both copies valid, counter resolves newest
• If power lost during write → CRC detects corruption, old copy still valid
• Corrupted copies are automatically erased on next read for recovery

4. Per-Area Sub-Partitions

swap_status_init_offset() maps each flash area ID (primary, secondary, scratch) to a sub-region within the status partition. Multi-image configurations (BOOT_IMAGE_NUMBER > 1) are natively supported.

5. Transparent API Layer

Two functions — swap_status_update() and swap_status_retrieve() — accept byte offsets and lengths, transparently handling record boundary crossing. The rest of MCUBoot's swap logic (swap_move, swap_scratch) works with minimal changes.

6. Trailer Copy-Back to Primary Slot

Before setting copy_done, the function swap_status_to_image_trailer() copies the status trailer back to the primary image slot in standard MCUBoot trailer format. This allows the application to read image_ok from the primary slot at runtime without needing access to the status partition.

Algorithm Detail: Read/Write Cycle

swap_status_update(target_area_id, offset, data, len):
  ┌─────────────────────────────────────────────────────────┐
  │ 1. Map target_area_id → partition offset                │
  │ 2. Map byte offset → record index                       │
  │ 3. For each record touched by [offset, offset+len):     │
  │    a. READ: scan all MULT copies                        │
  │       - Check magic (0xDEADBEEF)                        │
  │       - Validate CRC32C                                 │
  │       - Find copy with max counter                      │
  │    b. MODIFY: update payload bytes in RAM               │
  │    c. WRITE: to next copy slot                          │
  │       - Increment counter                               │
  │       - Append magic + new CRC                          │
  │       - Erase target → Write → Erase old                │
  └─────────────────────────────────────────────────────────┘

Overhead Estimates

For a typical configuration with MEMORY_ALIGN = 256 bytes:

• Payload per record: 244 bytes (256 - 4 - 4 - 4 - 4)
• With BOOT_MAX_IMG_SECTORS = 128, sector status fits in 1 row
• Trailer (55 bytes) fits in 1 row
• Per-slot: ~2 rows × 256B × 2 copies = 1 KB
• Two slots: ~2 KB total status partition

Relevance to This Discussion

This approach addresses the core problem raised in this RFC — MCUBoot's per-flag writes don't scale to large write block sizes — through a different strategy than the SHA-based layout record proposed here, but one that has practical advantages:

• No changes to the swap algorithm itself: swap_move and swap_scratch work as-is; only the metadata I/O layer is redirected
• No image tool changes: no pre-calculated hashes need to be embedded in images
• No SHA computation at runtime: only lightweight CRC32C per record (a few hundred bytes)
• Proven on production hardware: running on Infineon PSE84-based products with 256-byte minimum write size flash
• Configurable wear multiplication factor (BOOT_SWAP_STATUS_MULT): can be tuned to trade flash space for longevity

As @PavelChromyNXP noted, a dedicated reserved area for bootloader metadata is the right approach — this implementation does exactly that. The overhead of a few KB is well within acceptable limits.

Suggestion

This implementation could serve as a starting point or reference for a generic upstream solution. The core idea — abstracting MCUBoot metadata I/O behind a record-based storage layer on a dedicated partition — is orthogonal to the SHA-based layout record approach and could be complementary (e.g., the SHA table itself could be stored and managed using a similar record engine).

We'd be happy to collaborate on adapting this for upstream MCUBoot if the community finds the approach viable. The key integration points are well-isolated, and the code is Apache-2.0 licensed.

[butok]

Hi @DOAR-Infineon
Thank you for your proposal.

What will be the proposed method's status size with the below conditions?:

• Application Image size = 256KB
• Flash minimum program size = 512 Bytes
• Flash minimum erase size=512 bytes

[DOAR-Infineon]

@butok , it depends on the minimum swap chunk (size of scratch area)

For Infineon/Cypress approach you need at least 2 memory buffers (buffer consists of one or multiple flash pages, depends on the scratch area size) that fit all the swap sequence status records.