Reading · John Mueller's randoms

WLED 0.15.x needs a bit more app space than some ancient WLED installation. Usually updating the partition table requires a local USB connection. This is an app that you can install via OTA which expands app partitions to 1536KB.

The general approach to creating partitions on an ESP32 device involve connecting it via USB and flashing it with the IDF library (using gen_esp32part.py to create a binary image of the partition table, and idf.py to flash the device - see the ESP docs. …Or using platformio or similar to do it all together). Changing partition tables without having a device locally connected is not really documented. But you can do it.

nobody: ...
absolutely nobody: ...
John: Let's make an OTA ESP32 partition table app

There are a handful of forum threads where people claim to be able to do it, but very little code. The downside is if anything goes wrong, you’ll potentially brick your device (and will need to connect it via USB to fix). The upside is that in practice, very little can go wrong, since the problematic part (erasing & rewriting the partition table) is very quick.

When might you need to do this? WLED 0.15.x needs a bit more app space than some ancient WLED installation (github issue). It would be nice to not have to desolder ESP devices to get them updated. (Let’s be real: you probably don’t need to update a LED controller, it’ll probably continue to work until .. whenever.)

Overview of process

The general process is simple & fast.

Check that you’re running in the first app partition. This allows you to resize both app partitions (the second is needed for OTA, it switches automatically). Making the first app partition larger is a non-issue, but it moves the second app partition back, so you need to be in the first one.
Check where the partition table is located. It’s either at 0x8000 or 0x9000 (most seem to be 0x8000). You can do this by reading 2 bytes at both locations and checking for AA 50 bytes. The first location with that will contain the partition table.
Read the partition table to a buffer. Parse it out.
Adjust the partition table in your buffer as needed. Calculate the MD5 of the partition table.
Write the new partition table with MD5 back. (For flash memory, you need to erase before writing.)
Bonus: erase or move the partitions as needed. The second app partition will get overwritten anyway, so it doesn’t matter. Any data partitions will likely get invalidated if not erased or moved.

The code

Here’s ESP32Repartion on Github.

The rest here is bonus material.

Partition table data structure

The partition table data structure is in esp_partition_info_t . The magic ID is AA 50.

typedef struct {
  uint16_t magic;
  uint8_t  type;
  uint8_t  subtype;
  esp_partition_pos_t pos;
  uint8_t  label[16];
  uint32_t flags;
} esp_partition_info_t;

After the partition entries, there’s an entry with EB EB magic ID (+14 FF’s) + a 16-byte MD5 of the partition table. The MD5 needs to be correct, otherwise a modern’ish bootloader will not boot (older bootloaders do not check the MD5 of the partition table and will run regardless).

Sample partition table FYI

00000000  aa 50 01 02 00 90 00 00  00 50 00 00 6e 76 73 00  .P.......P..nvs.
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00000020  aa 50 01 00 00 e0 00 00  00 20 00 00 6f 74 61 64  .P....... ..otad
00000030  61 74 61 00 00 00 00 00  00 00 00 00 00 00 00 00  ata.............
00000040  aa 50 00 10 00 00 01 00  00 00 14 00 61 70 70 30  .P.........app0
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00000060  aa 50 00 11 00 00 15 00  00 00 14 00 61 70 70 31  .P........app1
00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
00000080  aa 50 01 82 00 00 29 00  00 00 17 00 73 70 69 66  .P....)....spif
00000090  66 73 00 00 00 00 00 00  00 00 00 00 00 00 00 00  fs..............
000000a0  eb eb ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  ................
000000b0  28 f1 4c 09 45 01 77 60  a1 07 06 5d b9 7a 25 07  (.L.E.w`...].z%.
000000c0  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  ................
…

Bonus functionality

Download bootloader firmware (URL path: /bootloader-download) - with the “more” menu on the page, it can download the current bootloader binary. I don’t know what you’d do with this (reverse engineer? idk), but you could.
Download partition table binary (URL path: /partition-download) - with the “more” menu, you can download the partition table as a binary. This might be useful if you spot something odd in a partition table and want to manually inspect it. Good luck.
Download the app1 partition (URL path: /app1-download) - with the “more” menu, you can download the other app partition binary. This might be useful if you need the other app (it’s only useful when you haven’t uploaded the new app twice).

FAQ

Frequently asked? Let’s be honest, nobody has asked. These are just ones that I thought might be interesting.

Q: How long is the critical phase (updating partition table)?

A: Rewriting the partition table takes about 100ms for the devices I tried.

Q: What happens if the moving / erasing afterwards breaks?

A: You’ll lose the data. Some apps verify validity of the data (usually a magic number at the start); if these fail, they’ll likely assume you just don’t have any data and reset with defaults.

Q: How likely am I to “brick” my device when running this?

A: It checks for potential issues before writing the new partition table, I think it should be mostly fine. However, the world of hardware & software has strange & unexpected things.

Screenshots & samples

Screenshot of ESP32Repartion running on an ESP32

Sample output

Build: Dec 28 2024 14:42:07 SDK 4.4.7 / 15840104 (ESP32-D0WDQ6)
Flash chip ID / Size: 0x164068 / 4096 KB
Program heap / program size: 335 KB / 743 KB
Partition table address: 0x8000
Bootloader MD5: 56cb99bf 20a1058a 3b901a5b df9961fa
NOTE: If you do not see a line with 'Ready' at the end,
this process didn't work.
Current app parition is first: OK
Reading partition table...
Splitting partitions out...
Created local copy of partiton table: OK
Type: 01 / 02, Addr: 0x009000, Size: 0x005000 (20K): nvs
Type: 01 / 00, Addr: 0x00e000, Size: 0x002000 (8K): otadata
Type: 00 / 10, Addr: 0x010000, Size: 0x140000 (1280K): app0
Type: 00 / 11, Addr: 0x150000, Size: 0x140000 (1280K): app1
Type: 01 / 82, Addr: 0x290000, Size: 0x170000 (1472K): spiffs
Partition table has 2+x app, 1+x data: OK
New partition table:
Type: 01 / 02, Addr: 0x009000, Size: 0x005000 (20K): nvs
Type: 01 / 00, Addr: 0x00e000, Size: 0x002000 (8K): otadata
Type: 00 / 10, Addr: 0x010000, Size: 0x180000 (1536K): app0
Type: 00 / 11, Addr: 0x190000, Size: 0x180000 (1536K): app1
Type: 01 / 82, Addr: 0x310000, Size: 0x0f0000 (960K): spiffs
Doing the work now...
Erasing partition table...
Writing partition table...
 ... Partition table written in 100 ms
Moving partition 4 from 0x290000 to 0x310000 length 0xf0000 ...
 0x37f000  0x37e000  0x37d000  0x37c000  0x37b000  0x37a000  0x379000  0x378000
(...)  0x290000
  ... Partition moved 240 sectors in 10240 ms (42 ms/sector)
Erasing partition 3 at 0x190000, length 0x180000
 ... Partition erased in 195 ms
Partitions erased / moved: OK
Partition table updated.
READY! After reboot, upload the firmware that you need.
Rebooting...

JavaScript snippet to download the page as a text file

I thought this was somewhat useful here. Perhaps it’s useful for others. It takes the current page’s text content and saves it locally as a text file.

function downloadPageText() {
  const t = document.body.innerText;
  const a = document.createElement('a');
  a.href = URL.createObjectURL(new Blob([t], {type: 'text/plain'}));
  a.download = 'page-content.txt';
  a.click();
}

The button:

<button onclick='downloadPageText()'>Download text</button>

Comments / questions

There's currently no commenting functionality here. If you'd like to comment, please use Bluesky and mention me there: @johnmu.com. Thanks!

Updating partition tables via OTA on ESP32 for WLED 0.15.x